Streaming Video over mobile network

asked 2022-01-13 06:39:57 -0500

501 ●14 ●58 ●31

updated 2022-01-13 06:40:46 -0500

It seems to me baffling how this problem can be so hard to find a solution for. On my robot I produce 15 fps 720p footage which I sometimes need to check while running. This is proving increasingly difficult. Subscribing to any image output topic grinds the whole connection to a halt until I stop subscribing, since the bandwidth required to transport raw images at that rate greatly exceeds what I can expect to get out of a 4G based mobile connection. I can't possibly be alone in experiencing this problem. I'm running ROS2 Foxy on a Jetson Xavier.

Here are a few things I have tried:

Use Gstreamer to stream via RTSP. Works fine, but locks the camera for use with Gstreamer, meaning SLAM packages can't use it.
Use image_transport with compressed data. Does not work, seems to have issues with OpenCV [image_view_node]: OpenCV(4.5.4) /tmp/OpenCV/modules/core/src/matrix.cpp:250: error: (-215:Assertion failed) s >= 0 in function 'setSize'
Use image_transport with theora data. Does not work, same crash as above. Also seems like a bad match for ROS2 since there's no latching (right?), and the theora header needs to come before the rest of the data.

I'm at quite a loss. The data is transmitted fine, and with compressed (or theora) data I can receive it without choking my network. I can not, for the life of me, view the image, however. What other options are reasonable to try here? Am I misconfiguring something? Is this too much to ask?

edit retag flag offensive close merge delete

add a comment

answered 2022-01-14 19:24:17 -0500

osilva

1650 ●3 ●172 ●19 https://github.com/rob...

updated 2022-01-14 19:25:10 -0500

Hi @Per Edwardsson,

I must warn you, I haven't tried this so please take with a grain of salt. I recalled reading about Isaac Image Pipeline found here: https://github.com/NVIDIA-ISAAC-ROS/i...

Similar to image_pipleline for CPU but it leverages the capabilities Nvidia Jeston.

More info here: https://developer.nvidia.com/isaac-sdk.

Performance metrics are impressive, considering they test them with DNN-based pipelines, such as object detection, 6D pose. And with non-DNN based pipelines like 2 lidars, shows under 7 seconds for initialization & localization.

Comments

@Per Edwardsson I'm not aware of any standard solutions for getting a a high quality video stream off of a robot. I suspect this is because there is too much variation in the requirements (e.g. frame size, frame rate, desired compressed stream bitrate, computation resources on the robot, camera interface), plus the fact that to get high video quality the load on resources is heavy and the developers must hand optimize the system to get it to work well.