Ask Your Question
1

reading large rosbag

asked 2016-12-09 11:03:08 -0500

dinesh gravatar image

updated 2016-12-09 11:12:39 -0500

Hello i have a rosbag of 300 mb which contains the pattern of an object for object recognition purpose. My problem is that when i read it, it takes too long time to read it. Is their any other methods which i can use for storing large collection of data or map and than read it very fast?

Is their any way i can access the rosbag datas, numbers with their index or something directly without having to read the hole data first? Cas the size of rosbag can also reach 1gb if object has large surface.

edit retag flag offensive close merge delete

2 Answers

Sort by ยป oldest newest most voted
1

answered 2016-12-10 00:14:20 -0500

ahendrix gravatar image

rosbag, the rosbag file format, and the rosbag python and C++ APIs are optimized to read data one message at a time, sequentially. In particular, the API only reads messages from disk as you request them (so you don't pay for the read until you do it) but the API for seeking isn't very efficient, so if you want to access your data in random order it may not be the best choice.

If you find yourself wishing for something better, consider:

  • Your disk is much slower than RAM; if you need the whole dataset, the fastest and most efficient thing to do is load the whole dataset into RAM and then do random accesses on it (if it fits in RAM).
  • Try to measure the read speed of your disk drive to get an idea of how fast the most efficient file format could possibly be, and compare that to your existing tools before you invent something new. (if there's not much to be gained, it's not worth spending time on it)
  • Disks (particularly magnetic drives, but SSDs too) are much better at sequential reads than they are at random access. If you can store your data in chunks that are processed together, this will make your disk accesses faster and more efficient. (ROS messages stored in a bag might make good chunks)
  • The most effective thing to do may be to invest in a fast SSD to make disk reads faster.
edit flag offensive delete link more
1

answered 2016-12-09 20:13:37 -0500

barefooter22 gravatar image

updated 2016-12-09 20:14:09 -0500

You can use rosbag play -r [rate] to play bag files faster. Like rosbag play -r 5 bag_name.bag would play back at 5 times speed. I used it in building maps so that I could watch the map build really fast

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

2 followers

Stats

Asked: 2016-12-09 11:03:08 -0500

Seen: 833 times

Last updated: Dec 10 '16