Recording Large Rosbags of Lidar & Camera Data

edit

I have been trying to find some documentation on this but was unable to really see anyhting helpful.

We have 5 cameras and 3 Lidars and need to record data from all sensors. For now we have been recording only two cameras (Allied Vision G-319C) and one Lidar (Velodyne VLP-32C) and are already running into problems.

The data is published at 20 Hz (lidar) and 30 Hz (cameras). Recording only one image for each camera in compressed format and the /velodyne_points results in the buffer being overflown and frames/points being dropped. The data is recorded to an internal SSD but I also tried an external SSD which works slightly worse.

I am wondering what the best practice would be in this case such as recording multiple rosbags at the same time, for example record one rosbag for each camera or something like that to distribute the load - or not even using rosbags but some other tool that i am unaware of.

(I have played around with different buffer sizes (setting it to 0 results in the rosbag automatically shutting down quickly) and splitting the rosbags at different sizes (500 MB, 5000 MB) without much improvement.)

Hardware: Astuff Spectra/Neuosys 6109GC with 32 GB of ram, two internal 265 GB and 1 TB SSDs, external 2 TB SSD, XEON E2278G 8th Gen

Thanks