ROS Resources: Documentation | Support | Discussion Forum | Index | Service Status | ros @ Robotics Stack Exchange
Ask Your Question
23

How to extract data from *.bag?

asked 2011-02-16 18:38:45 -0500

Changhyun Jun gravatar image

updated 2014-01-28 17:09:10 -0500

ngrennan gravatar image

How to extract data from *.bag?

I recorded my odomerty and laser data using rosbag. I want to extract (specific) data stream(text format) from recorded bagfiles. (time, step, odometry[x,y,...], laser data, ...) Becuase my unfinshied project are programed by matlab.

Question. 1) How to extract data from *.bag? (using matlab, C/C++ or ROS API...)

edit retag flag offensive close merge delete

Comments

when you see an answer you like, mark it as an accepted answer
mmwise gravatar image mmwise  ( 2011-02-17 09:32:40 -0500 )edit

Ok, is it possible to extract the contents of the bag file time, step, odometry[x,y..] and write our own customize code in cpp or java to move the robot through particular location of the map. Does we need to create the package with our customize code? How to combine all in built packages of ros and our own package?

RB gravatar image RB  ( 2013-10-27 19:40:10 -0500 )edit

The script worked well for me, but I didn't test for a bunch of different bags. I'd imagine it wouldn't work if the bag contains data of a type that is not just a simple number

mariakatosvich gravatar image mariakatosvich  ( 2016-09-09 01:49:54 -0500 )edit

@changhyun, did you find the relevant solution? I am also trying to retrieve the odom data in terms of each step , data and all. The below "highlighted" answer is not giving me the required output. Did you find the solution? Kindly help

Ayush Sharma gravatar image Ayush Sharma  ( 2017-03-12 08:18:09 -0500 )edit

13 Answers

Sort by ยป oldest newest most voted
34

answered 2011-02-17 04:12:51 -0500

kwc gravatar image

One more tip:

For simple message formats, you can convert directly to CSV from the command line:

rostopic echo -b file.bag -p /topic

This won't work well if your bag file contains arrays/images/point clouds/etc... as it renders the entire message to text.

edit flag offensive delete link more

Comments

9

This is the best answer I've found. Add a carat to save the text from terminal to txt--

rostopic echo -b file.bag -p /topic > data.txt
AndyZe gravatar image AndyZe  ( 2015-08-22 12:57:21 -0500 )edit
1

How might this work if you're trying to extract both structured data into a CSV and images at the same time?

pkpp1233 gravatar image pkpp1233  ( 2016-10-16 12:18:17 -0500 )edit

I am getting a blank txt file after implementing the following command "rostopic echo -b file.bag -p /topic > data.txt" and i am not able to get any output with the command "rostopic echo -b file.bag -p /topic".

Ayush Sharma gravatar image Ayush Sharma  ( 2017-03-12 08:18:46 -0500 )edit

Neither I get anything from those commands

jescasany gravatar image jescasany  ( 2017-03-13 07:57:13 -0500 )edit

Right @jescasany. Though i can view the "bag" data on "rqt", still the above commands are showing no output. Kindly check my question related to this.

Ayush Sharma gravatar image Ayush Sharma  ( 2017-03-13 09:12:12 -0500 )edit

I tried the "rostopic echo -b file.bag -p /topic > data.txt" but I am getting a blank txt file " and i am not able to observe any output with the command "rostopic echo -b file.bag -p /topic.

mariakatosvich gravatar image mariakatosvich  ( 2017-03-14 08:50:29 -0500 )edit

If this solution has been highlighted as a correct answer, then there must a way to resolve this issue. Probably the solution provider can help u sin this regard. But i was able to view the bag data via the "rqt" command. Use the commands "rqt" and open the bag file, you will get the whole data.

Ayush Sharma gravatar image Ayush Sharma  ( 2017-03-14 09:46:50 -0500 )edit

Thanks, " rostopic echo -b file.bag -p /topic > data.txt " works for me. Do you know where i can find the command for more about the command. For example, could i add other suffix instead of -p or -b?

mike.min gravatar image mike.min  ( 2018-01-17 15:19:23 -0500 )edit
12

answered 2014-02-03 05:43:04 -0500

x75 gravatar image

updated 2014-04-20 14:17:49 -0500

Evgeny gravatar image

just for the fun of it, here is a bag2csv one-liner for bash

for topic in `rostopic list -b bagfile.bag` ; do rostopic echo -p -b bagfile.bag $topic >bagfile-${topic//\//_}.csv ; done
edit flag offensive delete link more

Comments

1

I must be doing something wrong - I cant find where the file is saved? or is it just an oversight of mine that its not saving?

kleinsplash gravatar image kleinsplash  ( 2014-02-09 21:17:41 -0500 )edit

Nice! That's a lot quicker than my implementation...

NickSpeal gravatar image NickSpeal  ( 2014-06-15 21:12:28 -0500 )edit
12

answered 2011-02-16 19:55:09 -0500

Lorenz gravatar image

You easily can use the rosbag C++ or python API to directly load data from the bag. You find a tiny example on the rosbag C++ API documentation:

http://www.ros.org/doc/api/rosbag/html/c++/

edit flag offensive delete link more
6

answered 2013-05-29 11:29:05 -0500

NickSpeal gravatar image

updated 2013-06-03 09:23:44 -0500

I could not find an easy way to save all topics to a csv file for post-processing, so I wrote a python script to do it. I hope others will find it useful:

http://www.speal.ca/bag2csv.py (bag2csv.py)

Usage1 (for one bag file):

python bag2csv.py filename.bag

Usage 2 (for all bag files in current directory):

python bag2csv.py

I wasn't sure if this would be the right place to share this script for others to use. Can anybody else please advise?

Please let me know if you find this useful, or if you have any suggestions for how to improve it.

edit flag offensive delete link more

Comments

Hi! I tried to use your script, but I got the following error: File "bag2csv.py", line 94, in <module> values.append(pair[1]) IndexError: list index out of range Did anyone faced this problem before?

atcortes gravatar image atcortes  ( 2014-04-04 04:54:08 -0500 )edit

Hi atcortes. I'm sorry but I'm not sure what the solution to your problem is. I suspect it could be a problem with the bag file. Somebody else emailed me with the same problem, but didn't get back to me with more details about their situation. Do you get this problem with all bag files?

NickSpeal gravatar image NickSpeal  ( 2014-04-06 12:57:39 -0500 )edit

Hi NickSpeal, sorry for the late answer. This problem happens with all my bag files. A folder is created with the bag file and a csv file, however this one is not completed and I think the bag file corrupted (I cannot open it)

atcortes gravatar image atcortes  ( 2014-06-09 10:08:15 -0500 )edit

I'm not sure the solution to your problem. The script worked well for me, but I didn't test for a bunch of different bags. I'd imagine it wouldn't work if the bag contains data of a type that is not just a simple number. Camera images or point clouds, etc, would not work properly in .csv format...

NickSpeal gravatar image NickSpeal  ( 2014-06-15 21:17:50 -0500 )edit

FYI, I just got to try your script since I was trying to make something similar, and I'm getting similar errors to the ones reported by atcortes. I found out that in my case these were due to '/rosout' messages containing the '\n' inside the message string, causing 'nameValuePair's with no ':'.

Miguel Prada gravatar image Miguel Prada  ( 2014-12-16 09:54:10 -0500 )edit

I used it and it was sweet

georgebrindeiro gravatar image georgebrindeiro  ( 2017-03-12 18:14:04 -0500 )edit

I found the mention of this script here : https://www.clearpathrobotics.com/ass...

and was able to use it perfectly using : python bag2csv.py filename.bag

k2108 gravatar image k2108  ( 2021-09-10 12:59:33 -0500 )edit
5

answered 2011-02-16 22:26:32 -0500

Ryohei Ueda gravatar image

updated 2011-02-16 22:27:38 -0500

I have written a really simple script to output ros topics to a text file. you can find the script here; https://gist.github.com/831611

for example, we need the timestamp, width and height of sensor_msgs::PointCloud2;

$ ./topics_to_ssv -d " " hoge.ssv /camera/depth/points2/header/stamp/secs /camera/depth/points2/header/stamp/nsecs /camera/depth/points2/width /camera/depth/points2/height
$ cat hoge.ssv
/camera/depth/points2/header/stamp/secs /camera/depth/points2/header/stamp/nsecs /camera/depth/points2/width /camera/depth/points2/height 
1297944909 267601871 640 480 
1297944909 310369177 640 480 
...

I hope it help you...

edit flag offensive delete link more
1

answered 2011-02-16 19:40:59 -0500

updated 2011-02-16 22:19:11 -0500

Edit, because the answer by Lorenz is better than my previous suggestion: If you want to parse a bagfile yourself this might be interesting: http://www.ros.org/wiki/Bags/Format/2.0

Btw: It would be really cool if there was some kind of converter from bag-files to yaml or xml files.

edit flag offensive delete link more
1

answered 2011-02-17 04:22:43 -0500

updated 2013-02-21 11:39:04 -0500

Not sure if this will be useful for your specific case, but in one of the packages in starmac-ros-pkg there is a tool called bag2mat that reads in bag files for use in Python and can also save to a binary .mat file:

http://ros.org/doc/fuerte/api/starmac_tools/html/bag2mat_8py_source.html

edit flag offensive delete link more
0

answered 2014-11-13 21:42:00 -0500

aktaylor08 gravatar image

Here is a small python framework that extracts bag information from a bag file and puts it in a pandas dataframe in python. It also has two small scripts to dump it into a csv file or place graph information from a topic. https://github.com/aktaylor08/RosbagP...

edit flag offensive delete link more

Comments

Hi I have also used the standard way to extract bag data:

rostopic echo -b bag_name.bag -p/topic_desired > name_of_topic.csv

B I used the option:

rosbag record -a

Thus I have recorded about 200 topics of bag data not just one. And it would be very exhaustive type the 1st command above one by one to convert all the 200 topic's data to csv. It there a command or script that I run only once and convert all my topics to different csvs? For example something like this:

rostopic echo -b bag_name.bag -p/ALLTOPICS desired > Separated_topics_filenames.csv

And have:

planning.csv odometry.csv pose.csv etc

I have tried this:

$ rostopic echo -b all_topics_bag.bag -p / current_pose > current_pose.csv

Ok It works and created the csv, then I tried this to get all data

$ rostopic echo -b all_topics_bag.bag -p / current_pose > current_pose.csv ...
(more)
Vini71 gravatar image Vini71  ( 2021-09-22 14:07:49 -0500 )edit
0

answered 2020-10-11 21:31:57 -0500

cybodroid gravatar image

Check a relevant answer on reading bag file in python here: https://answers.ros.org/question/2883...

edit flag offensive delete link more
0

answered 2019-12-20 06:00:13 -0500

convert a bag file to .csv format, use

$ rostopic echo /topicname -b bagFileName.bag -p > file.csv

Remember to replace topicname with your topic (/tactile). And bagFileName.bag need to be replace with your bag file that you need to convert. And finally file.csv will be your output file. Instead of having .csv you can try .txt also. But for the best practice .csv file is good.

edit flag offensive delete link more

Question Tools

7 followers

Stats

Asked: 2011-02-16 18:38:45 -0500

Seen: 94,238 times

Last updated: Oct 11 '20