How do you split a rosbag into several files without calling rosbag filter multiple times?
I want to split a 100-GB rosbag into 100 1-GB bags. I tried using rosbag filter but it takes a long time as I have to run each filter manually and each time, it performs a scan of the full bag. Is there a better way to perform this split (either through command line or Python script)?
I've tried writing a Python script that reads through the rosbag message-by-message and writes each to a new bag (i.e. first GB of messages goes in bag 1, 2nd GB goes in bag 2, etc.). However, when I try to run the outputted bags, I get a "Expected CONNECTION op not found" error. What's the cause of this?
For reference, the python script is as follows:
import rosbag
num_msgs_per_bag = 1400
inputBag = 'Jan28.bag'
total_num_msgs_in_bag = rosbag.Bag(inputBag).get_message_count()
print (total_num_msgs_in_bag)
num_output_bags = total_num_msgs_in_bag / num_msgs_per_bag
bagCount = 1
msgCount = 0
outbag = rosbag.Bag('Jan28-'+str(bagCount)+'.bag', 'w')
for topic, msg, t in rosbag.Bag(inputBag).read_messages():
outbag.write(topic, msg, t)
print (msgCount)
msgCount = msgCount + 1
if msgCount % num_msgs_per_bag == 0:
bagCount = bagCount + 1
print ('******NEW BAG*******')
outbag = rosbag.Bag('Jan28-'+str(bagCount)+'.bag', 'w')
Well, I have no experience with the API, but you could also replay it and record everything with the
--split
command:rosbag record -a --split --size=1024
@mig I tried that and it hangs partway through (maybe some issue with the buffer?). The rosbag which I'm trying to split is 360GB and I don't want to slow down the playback rate as that'll slow down the recorded rosbag.
360GB is actually huge. This then also depends on how fast writing to the HDD is. There are Parameters
--buffsize
and--chunksize
with which you could Play around. But no idea, if this helps. The docu might help: http://wiki.ros.org/rosbag/Commandlin...