How to collect messages from all topics and put them into a csv file?

asked 2018-05-28 07:26:53 -0600

kolner
5 ●3 ●4 ●6

updated 2018-05-29 11:36:36 -0600

Hello everyone!

I have a .bag file with recorded messages from 4 different sensors (2xIMU, 2xINEMO), which have been worked simultaneously. I need to collect all messages from all sensors' topics and put them into one .csv file to do further research. So I want to get a table like this:

(rosbagTimestamp, IMU_1_data, IMU_2_data, INEMO_1_data, INEMO_2_data)

So how can I get such table with synchronically collected messages (data) from all topics?

Edit: Would the solution provided by message_filters.ApproximateTimeSynchronizer solve the problem?

Comments

There is no reason why the four sensors you're using would be synchronised like this, so you would not be able to represent them this way. If you could tell us what type of sensor you're using then we might be able to help you.

PeteBlackerThe3rd ( 2018-05-28 12:35:37 -0600 )edit

Sorry for that incompleteness. There are 2 IMUs and 2 INEMOs that are working simultaneously during a surgical operation. So the information from all those sensors for each time step is required for further analyse of the accomplished operation.

kolner ( 2018-05-28 14:24:15 -0600 )edit

The problem is the 4 IMUs will not be sampled at exactly the same time intervals, so you'll be able to collect 4 sensor readings at almost the same time but not exactly the same time. So if you try and represent 4 readings with a single time stamp you'll be introducing temporal errors.

PeteBlackerThe3rd ( 2018-05-28 15:05:15 -0600 )edit

Okay, do any workarounds exist for that problem? E.g. timing approximation, i.e. to sample sensors' data not every millisecond, but every second or something like that.. I am pretty sure that someone has already struggled with such problems.

kolner ( 2018-05-28 15:15:30 -0600 )edit

add a comment

answered 2018-05-28 12:52:36 -0600

mntan
51 ●1 ●4

updated 2018-05-29 07:29:18 -0600

A brute force approach could be to make a node that subscribes to the topics and prints exactly what you want in the csv file. Run this node and redirect the output into a text file, like this:

rosrun example_pkg example_node > csv.txt

Run the bag file and csv.txt should contain everything that would have been printed to the console

In order to synchronize the data, you can store the most updated value for each sensor. In each callback, update these values. At a fixed time interval, print these most updated values along with the time.

edit flag offensive delete link

Comments

Thank you for the answer! When I do in this way, will the messages in the csv.txt be synchronized displayed? (i.e. whether the data e.g. from first sensor will match to the data from second sensor in the csv.txt for each time step)

kolner ( 2018-05-28 14:37:33 -0600 )edit

Just to clarify, do you want to attempt to "sample" all sensors at fixed intervals? So basically, at a time step, it will return the most recent sensor reading from each sensor. The time reported won't be the rostime of when the data came in, but an approximation.

mntan ( 2018-05-28 19:19:52 -0600 )edit

Exactly, I want to sample all sensors at fixed intervals. When it is not possible, the approximated time would be good for me as well, I do not think that there would be significant deviations.

kolner ( 2018-05-29 01:12:57 -0600 )edit

I've found probably more elegant way to provide sychronized reading, which involves a class message_filters.ApproximateTimeSynchronizer in order to synchronize callbacks of IMUs' subscribers. Would it work appropriately?

kolner ( 2018-05-29 11:35:25 -0600 )edit

The algorithm that the ApproximateTimeSynchronizer uses is explained in some detail here. If that type of synchronization is appropriate for your use case (which only you can answer), then yes, it could work.

jarvisschultz ( 2018-05-29 14:07:47 -0600 )edit

Although, as @PeteBlackerThe3rd pointed out, this could be introducing temporal error to some of your data. If you really need exactly synchronized data then either your hardware needs to provide the capability to synchronize somehow (e.g. triggering, shared clocks, etc.) or you should be aware...

jarvisschultz ( 2018-05-29 14:10:13 -0600 )edit

... of how the approximations used in synchronizing are affecting the data and how these errors might change results further down the computation pipeline. You could always do some custom online smoothing+splining+resampling to produce approximate data that is synchronized according to your needs.

jarvisschultz ( 2018-05-29 14:12:53 -0600 )edit

Thanks so much for your answers! Due to them the situation bacame clear.

kolner ( 2018-05-30 06:30:05 -0600 )edit

add a comment

How to collect messages from all topics and put them into a csv file?

Comments

1 Answer

Comments

Question Tools

Stats

Related questions

How to collect messages from all topics and put them into a csv file? edit

Comments

1 Answer

Comments

Question Tools

Stats

Related questions

How to collect messages from all topics and put them into a csv file?