Ask Your Question
0

Intercepting images before they are published

asked 2021-02-25 13:27:16 -0500

Tom64 gravatar image

I can successfully publish video frames from my Raspberry Pi 4 (running ROS melodic on a Ubuntu 18.04 image) to a topic using the raspicam node (found here) and see those images on my computer. However, I would like to do some image processing on the raspberry pi before publishing the frames. Is there any way to do this without subscribing to the output of the node and then re-publishing the images to a separate namespace? Is it possible to edit any of the properties of the package to allow a python (or C++) script to process the frames before they are published?

I have been trying to figure out which part of the package is directly responsible for pulling the images from the camera as this seems like an ideal place to have the script run. I'm still fairly new to ROS so am not sure what I am looking for.

Any help for any of these questions would be greatly appreciated.

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted
0

answered 2021-02-28 14:45:30 -0500

tryan gravatar image

updated 2021-03-02 08:30:34 -0500

That functionality isn't built in. The raspicam_node reads the image data from the buffer, builds the image message, and publishes it in the section after line 266 of raspicam_node.cpp. You could insert some image processing there if you build the package from source, but I recommend writing your own node to process the output and publish the results to a new topic as you mentioned. raspicam_node has some configurable parameters (see Usage and reconfigure_callback() on line 569), but if you want to do computer vision-type tasks, that's generally outside the scope of a driver node. Let the driver node do its job and have a separate node do the image processing. That way, the node responsibilities stay modular, and the architecture stays flexible. If you decide to use different hardware later, you can just swap out the driver node and reuse the same image processing node. If the processing is too intensive (and your application allows it), you can just run that node on your computer instead of on the Pi. You may have valid reasons for doing it differently, though. Feel free to elaborate or ask questions, and we can discuss how best to achieve your goals.

edit flag offensive delete link more

Comments

Hi, thanks for your answer. I fully agree that it would be better to have a separate driver node, however would this require having 2 topics being published as this is how data is communicated between nodes, one by the driver node and then one by the computer vision node? I'm intending to build the system as a modular and possibly large scale deployment so want to get the data being sent by the system as low as possible to keep the network bandwidth down. This also means I will always be doing the image processing on the RaspberryPI's. I am doing additional computer vision on the host computer, but it required pre-processed information from the raspberry pi's. How would I go about creating a basic node that can read data from the cameras and push the data into a script (without using topics) which could be ...(more)

Tom64 gravatar image Tom64  ( 2021-03-02 23:35:30 -0500 )edit

I think I have thought of a workaround for now. As there is minimal data being added the the image by the raspberry pi, I'm thinking to just send the data calculated by the Pi as a topic and using that to draw onto the original image at the host computer. There may be some small snyc issues but I think this may be the simplest way forwards.

Tom64 gravatar image Tom64  ( 2021-03-02 23:45:43 -0500 )edit

Yes, there would be 2 topics, but if the first topic has only local subscribed nodes, it would not increase network traffic. You can have separate nodes on the Pi with no extra data transferred to your host computer. Since ROS is peer-to-peer, the Pi will only send required data to the host computer.

Your workaround is another option. Stamped messages are very useful for syncing, and there are tools like the message_filters::TimeSynchronizer (wiki) to help.

tryan gravatar image tryan  ( 2021-03-03 09:10:46 -0500 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

Stats

Asked: 2021-02-25 13:27:16 -0500

Seen: 28 times

Last updated: Mar 02