Custom image message type

asked 2018-09-12 02:21:20 -0500

latida gravatar image

updated 2018-10-17 03:15:31 -0500

I'm writing to enquire about possibility of ROS supporting or creating a custom sensor message type for video stream (images).

Our product, based on the FPGA, generates the rosbag file and logs one video stream in one file. It uses the VideoLink protocol, which is packet based, so all events on the parallel interface need to be packetized. The ROS message data here holds a data block.

The internal structure of a data block consists of a series of data packets each reflecting one line of the video data, or other data arriving via video link. Short data packets (without actual image) are here to store specific events that are natural to the video link and need to be stored in order to allow all features to be present during playback in a later moment. Long data packets contain one line of a video frame, but may also contain a metadata info block coming over the video link, as sent from camera.

Our goal is to be able to access the video (or images) from the recorded bag files using CvBridge or similar library which supports OpenCV image file format. In a current configuration, ROS environment doesn't recognize this type of message. Is there a posibility to include some plug-in conversion function or a different ROS library which will convert our format to some useable ROS message type. Thank you for your consideration!

Edit: These sensors are commonly used in automotive, and they're able to send multiple expositions. We would like to propose to the comminity to create a format for these messages so they can be widely used. Here's a short presentation:

edit retag flag offensive close merge delete


Related / history: #q302472.

gvdhoorn gravatar image gvdhoorn  ( 2018-09-12 02:24:38 -0500 )edit

If the goal is to be able to pass this video to all types of ROS consumers then converting your datastream into sensor_msgs/Image would be the way to go. The "specific events" would probably need custom messages.

Conversion to sensor_msgs/Image would be up to you I believe, I don't know ..

gvdhoorn gravatar image gvdhoorn  ( 2018-09-12 02:26:18 -0500 )edit

.. of a library available that takes in videolink packets and converts it to camera frames. The msg structures for the events can be created using standard functionality.

As soon as you're using standard infrastructure, rosbag and other ROS nodes should be able to work with your data.

If your ..

gvdhoorn gravatar image gvdhoorn  ( 2018-09-12 02:27:39 -0500 )edit

.. goal is to use rosbags (ie: .bag files) to store your original data stream (so packet-per-packet) then please clarify. It could still be done (ie: create a custom msg for a videolink packet and store those), but there would be no compatibility with cv_bridge in that case.

gvdhoorn gravatar image gvdhoorn  ( 2018-09-12 02:29:00 -0500 )edit

Hi, thank you for your answer.To clarify, we're using Omnivision OV2775 and Sony IMX290 image sensors and images (video) is taken in HDR.Those sensors are commonly used in automotive. So, what we want is to store our original data in .bag and than be able to convert them into OpenCV compatible type

latida gravatar image latida  ( 2018-09-12 05:04:34 -0500 )edit

If you want to store data in its original format, then you'll have to create custom msgs that can encode that format.

Conversion to a ROS cv_bridge compatible format (I assume that is what you meant) would have to be done afterwards, by a tool you write yourself. Input would be ..

gvdhoorn gravatar image gvdhoorn  ( 2018-09-12 07:39:16 -0500 )edit

.. the .bag with the custom msgs, and output could/would be a .bag with sensor_msgs/Image (in any of the encodings supported).

Conversion could of course also happen live, and then a single back could contain both the original data as well as the converted data.

gvdhoorn gravatar image gvdhoorn  ( 2018-09-12 07:40:58 -0500 )edit

The problem is that we don't record our data using ROS, just store our data in .bag files using rosbag file format respectively. So, creating custom messages and converting them by writing our own tool would be inconvenient.

latida gravatar image latida  ( 2018-09-13 03:27:57 -0500 )edit