ROS Resources: Documentation | Support | Discussion Forum | Index | Service Status | ros @ Robotics Stack Exchange
Ask Your Question

How to collect messages from all topics and put them into a csv file?

asked 2018-05-28 07:26:53 -0500

kolner gravatar image

updated 2018-05-29 11:36:36 -0500

Hello everyone!

I have a .bag file with recorded messages from 4 different sensors (2xIMU, 2xINEMO), which have been worked simultaneously. I need to collect all messages from all sensors' topics and put them into one .csv file to do further research. So I want to get a table like this:

(rosbagTimestamp, IMU_1_data, IMU_2_data, INEMO_1_data, INEMO_2_data)

So how can I get such table with synchronically collected messages (data) from all topics?

Edit: Would the solution provided by message_filters.ApproximateTimeSynchronizer solve the problem?

edit retag flag offensive close merge delete



There is no reason why the four sensors you're using would be synchronised like this, so you would not be able to represent them this way. If you could tell us what type of sensor you're using then we might be able to help you.

PeteBlackerThe3rd gravatar image PeteBlackerThe3rd  ( 2018-05-28 12:35:37 -0500 )edit

Sorry for that incompleteness. There are 2 IMUs and 2 INEMOs that are working simultaneously during a surgical operation. So the information from all those sensors for each time step is required for further analyse of the accomplished operation.

kolner gravatar image kolner  ( 2018-05-28 14:24:15 -0500 )edit

The problem is the 4 IMUs will not be sampled at exactly the same time intervals, so you'll be able to collect 4 sensor readings at almost the same time but not exactly the same time. So if you try and represent 4 readings with a single time stamp you'll be introducing temporal errors.

PeteBlackerThe3rd gravatar image PeteBlackerThe3rd  ( 2018-05-28 15:05:15 -0500 )edit

Okay, do any workarounds exist for that problem? E.g. timing approximation, i.e. to sample sensors' data not every millisecond, but every second or something like that.. I am pretty sure that someone has already struggled with such problems.

kolner gravatar image kolner  ( 2018-05-28 15:15:30 -0500 )edit

1 Answer

Sort by » oldest newest most voted

answered 2018-05-28 12:52:36 -0500

mntan gravatar image

updated 2018-05-29 07:29:18 -0500

A brute force approach could be to make a node that subscribes to the topics and prints exactly what you want in the csv file. Run this node and redirect the output into a text file, like this:

rosrun example_pkg example_node > csv.txt

Run the bag file and csv.txt should contain everything that would have been printed to the console

In order to synchronize the data, you can store the most updated value for each sensor. In each callback, update these values. At a fixed time interval, print these most updated values along with the time.

edit flag offensive delete link more


Thank you for the answer! When I do in this way, will the messages in the csv.txt be synchronized displayed? (i.e. whether the data e.g. from first sensor will match to the data from second sensor in the csv.txt for each time step)

kolner gravatar image kolner  ( 2018-05-28 14:37:33 -0500 )edit

Just to clarify, do you want to attempt to "sample" all sensors at fixed intervals? So basically, at a time step, it will return the most recent sensor reading from each sensor. The time reported won't be the rostime of when the data came in, but an approximation.

mntan gravatar image mntan  ( 2018-05-28 19:19:52 -0500 )edit

Exactly, I want to sample all sensors at fixed intervals. When it is not possible, the approximated time would be good for me as well, I do not think that there would be significant deviations.

kolner gravatar image kolner  ( 2018-05-29 01:12:57 -0500 )edit

I've found probably more elegant way to provide sychronized reading, which involves a class message_filters.ApproximateTimeSynchronizer in order to synchronize callbacks of IMUs' subscribers. Would it work appropriately?

kolner gravatar image kolner  ( 2018-05-29 11:35:25 -0500 )edit

The algorithm that the ApproximateTimeSynchronizer uses is explained in some detail here. If that type of synchronization is appropriate for your use case (which only you can answer), then yes, it could work.

jarvisschultz gravatar image jarvisschultz  ( 2018-05-29 14:07:47 -0500 )edit

Although, as @PeteBlackerThe3rd pointed out, this could be introducing temporal error to some of your data. If you really need exactly synchronized data then either your hardware needs to provide the capability to synchronize somehow (e.g. triggering, shared clocks, etc.) or you should be aware...

jarvisschultz gravatar image jarvisschultz  ( 2018-05-29 14:10:13 -0500 )edit

... of how the approximations used in synchronizing are affecting the data and how these errors might change results further down the computation pipeline. You could always do some custom online smoothing+splining+resampling to produce approximate data that is synchronized according to your needs.

jarvisschultz gravatar image jarvisschultz  ( 2018-05-29 14:12:53 -0500 )edit

Thanks so much for your answers! Due to them the situation bacame clear.

kolner gravatar image kolner  ( 2018-05-30 06:30:05 -0500 )edit

Question Tools

1 follower


Asked: 2018-05-28 07:26:53 -0500

Seen: 445 times

Last updated: May 29 '18