ROS Resources: Documentation | Support | Discussion Forum | Index | Service Status | Q&A answers.ros.org
Ask Your Question
2

Using rosbag to get size of each topic

asked 2019-03-17 10:47:07 -0600

Escontrela gravatar image

I'm trying to remove unnecessary topics from a rosbag file to make it easier to distribute. In doing so, i'd like to know the size of each topic, not just the size of the bag file as a whole. However, running rosbag info example_bag.bag only provides me with the size of the bag file and not the size of individual topics.

Is there any way to use the rosbag command line tool to obtain the size of individual topics within the bag file?

edit retag flag offensive close merge delete

Comments

What do you mean by size of topics? You can get the number of messages in the particular topic and data type using rosbag info

simbha gravatar image simbha  ( 2019-03-19 16:10:05 -0600 )edit

2 Answers

Sort by » oldest newest most voted
11

answered 2019-05-20 05:11:38 -0600

MumboJumbo gravatar image

updated 2019-05-30 17:23:19 -0600

lucasw gravatar image

I had the same issue and wrote a python script to solve it.

#!/usr/bin/env python
# Print total cumulative serialized msg size per topic
import rosbag
import sys
topic_size_dict = {}
for topic, msg, time in rosbag.Bag(sys.argv[1], 'r').read_messages(raw=True):
  topic_size_dict[topic] = topic_size_dict.get(topic, 0) + len(msg[1])
topic_size = list(topic_size_dict.items())
topic_size.sort(key=lambda x: x[1])
for topic, size in topic_size:
  print(topic, size)

And to explain to simbha why this is useful and needed: I have 102 topics generated by 40+ nodes. Hundreds of thousands of message. I'm not familiar with any of these nodes. But one of them is causing excessive size of bags. As a system administrator/data warehouse guy I want to quickly find out which one exactly so I can contact the responsible developer to fix the issue.

edit flag offensive delete link more

Comments

And if you get "UnicodeEncodeError", add lines belows

import sys
sys.setdefaultencoding("utf-8")
raucha gravatar image raucha  ( 2019-06-19 08:14:55 -0600 )edit
0

answered 2022-11-14 12:50:03 -0600

Combinacijus gravatar image

I modified MumboJumbo code to give human readable output:

To run: python rosbag_size_by_topic.py BAG_FILE_PATH or

rosrun PACKAGE rosbag_size_by_topic.py BAG_FILE_PATH

rosbag_size_by_topic.py:

#!/usr/bin/env python
# Prints total cumulative serialized msg size in bytes per topic
# Calculation takes approximately 1sec/GB
# Usage:   python rosbag_size_by_topic.py BAG_FILE_PATH

import rosbag
import sys

topic_size_dict = {}
for topic, msg, time in rosbag.Bag(sys.argv[1], "r").read_messages(raw=True):
    topic_size_dict[topic] = topic_size_dict.get(topic, 0) + len(msg[1])

topic_size = list(topic_size_dict.items())
topic_size.sort(key=lambda x: x[1], reverse=False)

for topic, size in topic_size:
    if size < 1000:
        print("{:7d}B  {}".format(size, topic))
    elif size >= 1000000000000:
        print("{:7.2f}T  {}".format(size/1000000000000, topic))
    elif size >= 1000000000:
        print("{:7.2f}G  {}".format(size/1000000000, topic))
    elif size >= 1000000:
        print("{:7.2f}M  {}".format(size/1000000, topic))
    elif size >= 1000:
        print("{:7.2f}K  {}".format(size/1000, topic))

Example of output:

     86B  /tf_static
    775B  /move_base/global_costmap/inflater_layer/parameter_descriptions
    775B  /move_base/local_costmap/inflater_layer/parameter_descriptions
   1.01K  /mobile_base_controller/parameter_descriptions
   1.33K  /move_base/goal
   1.88K  /voxel_grid/parameter_descriptions
 866.76K  /move_base/GlobalPlanner/plan
 921.19K  /path_planner/visualization_marker
   1.10M  /move_base/TebLocalPlannerROS/local_plan
   1.18M  /imu/mag
   1.19M  /move_base/TebLocalPlannerROS/teb_markers
  19.89M  /tf
 284.01M  /move_base/GlobalPlanner/potential
  19.54G  /move_base/global_costmap/obstacles_layer/voxel_grid
edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

2 followers

Stats

Asked: 2019-03-17 10:47:07 -0600

Seen: 3,251 times

Last updated: Nov 14