slow filling of uint8 data field from python

asked 2020-07-08 10:58:13 -0500

esterina
3 ●1 ●2 ●3

I have found this issue in both ros2 eloquent and foxy. When filling an Image message or a PointCloud2 message from a numpy.array, filling the data field is extremely slow.

For instance, with the point cloud message:

cloud_msg = PointCloud2()
cloud_msg.data = cloud_arr.tostring() # cloud_arr is my point cloud as numpy array

The tostring() itself is fast, but the assignment to cloud_msg.data is what takes time. I cannot get the publishing rate over 1 Hz because of how much time is spent in filling the message.

What should I do to publish point clouds in python at a decent rate?

EDIT: I managed to get it a bit faster by setting PYTHONOPTIMIZE to 1. I noticed that the point cloud filling was performing a lot of extra checks if __debug__ was set to True. This allows me to get the message filling down from 0.4 seconds to 0.002. Now the publish() part is the slowest, taking 0.04 seconds. Is there any chance of improving this as well, or is this as fast as it can get?

answered 2020-07-08 13:23:55 -0500

Dirk Thomas
16276 ●62 ●190 ●215

To fill the message field you shouldn't convert your numpy array to a string and then during the assignment convert the string back into an array.array (which is the type of the datafields, see http://design.ros2.org/articles/idl_i...).

Have you tried to populate the array.array directly from the (1 dimensional?) numpy array? (Without having run the following code) something like this:

cloud_msg.data.frombytes(cloud_arr[0])

edit flag offensive delete link

Comments

Thanks! That makes it faster, without having to set PYTHONOPTIMIZE to 1!

I still need to use the tostring() method because my array is not an array of bytes, it has N entries each with x, y, z and R,G,B fields, but once I have the bytes calling frombytes() instead of assigning it is much faster.

Since I did not find a built in python function to fill a PointCloud2 message, I adapted the code from this: link text. That's where I took the assignment of data instead of using frombytes.

esterina ( 2020-07-09 04:36:47 -0500 )edit

add a comment

slow filling of uint8 data field from python

1 Answer

Comments

Question Tools

Stats

Related questions

slow filling of uint8 data field from python edit

1 Answer

Comments

Question Tools

Stats

Related questions

slow filling of uint8 data field from python