ROS Resources: Documentation | Support | Discussion Forum | Index | Service Status | ros @ Robotics Stack Exchange
Ask Your Question
1

How to detect broken rosbag files

asked 2017-02-21 10:58:34 -0600

Tobias Neumann gravatar image

updated 2017-02-21 11:16:01 -0600

Hello ROS users,

I've recorded plenty rosbag files during a testrun with a 1min split and lz4 compression. Afterwards some of the data got corrupted (by the harddrive, not an error with rosbag). When playing the rosbag files, it works and starts normally, but if rosbag hits a broken file, I get the following msg:

ROSLZ4_DATA_ERROR: malformed data to decompress

I now want to identify the broken rosbag-files and either try to fix them or if they are not fixable, delete them. But I don't find a way to identify these broken files, does anybody know a way?

My first thought was to use "rosbag check", but this seems not to be intended for this use, or is it and I am not able to see this?

Does anybody had a similar problem and does know a solution in the ROS eco-system?

kindly regards
Tobias

edit retag flag offensive close merge delete

Comments

So you tried all commandline options http://wiki.ros.org/rosbag/Commandline ?

130s gravatar image 130s  ( 2017-02-21 11:41:17 -0600 )edit

I did not try all options, but I read about all of them, and "rosbag check" was the only one that seemed it might be suitable, but it was not, therefore my question, did I miss something or is there no such command and I have to write a script to play them all and wait for this error?

Tobias Neumann gravatar image Tobias Neumann  ( 2017-02-22 04:23:09 -0600 )edit

If I were in your shoe I'd try all commands. Since you say in the title that you want to detect files with issues, I thought if a command that indicates "something is wrong with this file". E.g. if rosbag info returns something wrong or doesn't terminate correctly, then that could meet your needs.

130s gravatar image 130s  ( 2017-02-22 05:00:44 -0600 )edit

The files only create problems when they are played and even then only when there are subscriber on the topics :( I wrote a script to play them one after another and display the faulty bag files.

Tobias Neumann gravatar image Tobias Neumann  ( 2017-02-22 10:36:37 -0600 )edit
1

The script:

#!/bin/bash
for i in `seq 0 35`; do
  echo $i
  if [[ -n $( rosbag play -i [cds][aew]*_$i.bag --clock 2>&1 > /dev/null ) ]]; then
    echo [cds][aew]*_$i.bag
  fi  
done
Tobias Neumann gravatar image Tobias Neumann  ( 2017-02-22 10:37:26 -0600 )edit

1 Answer

Sort by ยป oldest newest most voted
3

answered 2017-02-23 06:41:36 -0600

Tobias Neumann gravatar image

Since there does not seem to be a solution within the ROS eco system, I wrote a script that uses rosbag decompress to go though all bag-files, detects which are broken and then reindexes them.

#!/bin/bash
decompressed_bags=/media/tneumann/fd783860-83b8-4729-8ea5-eee3e08bb9c5/trash/
reindexed_bags=/media/tneumann/fd783860-83b8-4729-8ea5-eee3e08bb9c5/reindexed/
broken_bags=""
for bag in *.bag; do
  if [[ -n $( rosbag decompress --output-dir=$decompressed_bags $bag 2>&1 > /dev/null ) ]]; then
    echo $bag
    broken_bags="$broken_bags $bag"
  fi  
  rm ${decompressed_bags}/${bag}
done

echo -e "Broken bag files:\n $broken_bags\nStart reindexing\n"
mkdir broken/

for bag in $broken_bags; do
  mv $bag broken/
  rosbag reindex --output-dir=$reindexed_bags broken/$bag
done

This is of course not my preferred solution but it works, if your bag-files are not compress this might not work, you can try to use "rosbag play -i" instead, but this just triggers an error if you also listen on the published topics.

regards
Tobias

edit flag offensive delete link more

Question Tools

3 followers

Stats

Asked: 2017-02-21 10:58:34 -0600

Seen: 1,855 times

Last updated: Feb 23 '17