Rosbag broken with lz4 compression
Hello ROS users:
We have made several robots with identical hardware settings recently, and the robots are constantly running while recording data as rosbags for almost a month. We discover that we can not open few of the bags with native rosbag tool. Both rosbag info and rosbag play give error messages. The messages are not identical however, some say the bag is unindexed, however the bag becomes empty after we run rosbag reindex. Others just show a bunch of python errors.
We dive into the bugs, and find that lz4 compression seems broken on these bags. While recording, we use --lz4 command to save disk space. In normal bags, we can clearly see the marks of encryption. Specifically, in the first few (about 100) lines, each contains both a few recognizable words, and a large portion of seemingly random ascii characters, a clear sign of encryption we think. However in those broken bags, although the header clearly indicates that this is a lz4-compressed rosbag, the first few lines are all human-readable, indicating there are no encryption at all.
We have two questions:
Q1: Is there any way to fix those bad bags, other than rosbag reindex or rosbag fix ? We have tried both, sadly none works.
Q2: What triggers this lz4 bug after all? Can it be fixed through code editing? We are using ROS kinetic on Ubuntu 16, we understand that this is a very old version of ROS, but we are unable to migrate our codes to other versions currently.
Thank you for your answes! And do forgive my poor language as a non-native speaker.
What was your exact record command line for a broken bag file? Are you playing the bag on the same type of cpu that created it (in particular, 32bit vs 64 bit, and big-endian vs. little-endian)? Do you notice anything interesting in the file size of the broken bags vs. not-broken bags?
the command was
rosbag record -b 2048 --lz4 /all/the/topics/needed, and our robots are equipped with 6th-gen i5 cpus, 8GB memories and 512GB Samsung SSDs. We tried to play the broken bags on the exact robots that recorded them, the outcomes were same. We didn't notice significant abnormalities with the sizes of broken bags. They were identical to those normal bags, maybe a little bit larger. Since all robots run different routes every time, it's quite hard to compare the sizes, nor have we done any specific experiments, because the problems are rare and hard to reproduce (It happened several times out of maybe several hundred times of recording)Are you aware of this wiki page: http://wiki.ros.org/Bags/Format
Are you able to share a broken bag file? For now, just the beginning of the file - use this command to copy the first 50MB, where "yourfile.bag" is the name of your file:
Also tell us the exact size of the complete file.