Ask Your Question
5

Playing back large files with rosbag

asked 2011-08-08 07:38:41 -0500

Hordur gravatar image

Does rosbag read the whole bag file or a large portion of it before playing? Often when I play back a large bag file it can take a long time before the playback starts.

edit retag flag offensive close merge delete

3 Answers

Sort by ยป oldest newest most voted
9

answered 2011-08-09 02:07:10 -0500

Hordur gravatar image

I looked over the rosbag code. There is function called startReadingVersion200 in bag.cpp that reads in chunk headers and connection index records. The chunk headers are spaced at 1MB interval in the file. So for a 30GB file you have 30000 chunks. I'm guessing that reading each chunk is causing a seek, so with a 5ms seek time reading the file would take around 2min. Which matches what I have seeing. This would explain why it loads quickly when reloading. Because the header don't take up that much space and are probably being buffered by the OS.

The load time could be improved if the chunk headers could be processed as they are encountered in the bag file. I'm not familiar with the bag format so I don't know how easy or hard that would be.

edit flag offensive delete link more

Comments

3

Note that this is still an issue: https://github.com/ros/ros_comm/issues/117

jbohren gravatar imagejbohren ( 2013-03-11 16:54:20 -0500 )edit

I'm still having this issue. I have to replay a 270Gb bagfile and iotop shows painfully slow disk reads:

Actual DISK READ: 35.90 M/s | Actual DISK WRITE: 0.00 B/s

17726 be/4 root 17.11 M/s 0.00 B/s 0.00 % 3.69 % play --queue 100 --rate 1.0 --delay 2.0 --start 0.0 2016-02-12-11-15-41.bag

Avio gravatar imageAvio ( 2016-02-16 07:03:28 -0500 )edit
2

answered 2013-03-12 15:35:16 -0500

jbohren gravatar image

In ROS Groovy, rqt_bag seems pretty capable of dealing with large bagfiles. Even a 75GB file opened in in a little under a minute.

edit flag offensive delete link more
1

answered 2011-08-08 13:05:58 -0500

DiogoCorrea gravatar image

In my case the playback starts fast (less than a second). I think that rosbag don't read the whole bagfile because I'm using a bagfile of 7.5GB I think it may depend on the configuration of your computer. I have a core i7 with 4GB RAM and a NVIDIA GTS 450. Hugs.

edit flag offensive delete link more

Comments

1
I have a 50GB file that I'm loading and it takes few minutes before playing back. My machine is a core i7, 16GB RAM, running Ubuntu 11.04 64bit. When loading there isn't much CPU load. If I start the playback again then it starts quickly, only few seconds. Maybe the OS is doing some buffering.
Hordur gravatar imageHordur ( 2011-08-09 00:57:18 -0500 )edit

Your local hard disk would be SSD then, if it takes less than a second to play 7.5 GB file. For 123 GB bag file, it took 1 hour 30 minutes to play from HDD, while took less than 10 seconds to play from SSD.
I dont think there is GPU utilization in playing bag file, its not even multicored.

warriorUSP gravatar imagewarriorUSP ( 2019-05-03 12:55:34 -0500 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

2 followers

Stats

Asked: 2011-08-08 07:38:41 -0500

Seen: 2,159 times

Last updated: Mar 12 '13