Rosbag filter memory error

edit

Hi,

My hardware: Intel NUC8i7BEHI, with 16Gb or RAM and a SSD. My software: ROS1 Kinetic Desktop-Full, Ubuntu 16.04.6 Xenial, Python 2.7.12. I am getting the following error while I try to filter a bag (rosbag filter input.bag output.bag "topic in ['/topic1','/topic2','/topic3']"):

raceback (most recent call last):
File "/opt/ros/kinetic/bin/rosbag", line 35, in <module> rosbag.rosbagmain()
File "/opt/ros/kinetic/lib/python2.7/dist-packages/rosbag/rosbag_main.py", line 896, in rosbagmain cmdscmd
File "/opt/ros/kinetic/lib/python2.7/dist-packages/rosbag/rosbag_main.py", line 362, in filter_cmd msg.deserialize(serialized_bytes)
File "/tmp/genpy_hai_6g/tmpjn80x2.py", line 162, in deserialize end += struct.calcsize(pattern)
MemoryError

What I found is that I get the following error in random bags. The size or the duration of the bags do not matter. It also stops at random % of completion. Probably that random is something that I currently do not understand. What is strange is that the memory is not surpassing 4.5 GB. While doing the free -h command, I can see a minimum of 10GB available of memory right before the error stops the program.

Thanks for the help!

Update: while running with strace I get the following:

_llseek(4, 1771724800, [1771724800], SEEK_SET) = 0 _llseek(4, 1771724800, [1771724800], SEEK_SET) = 0 read(4, "9]X\276\360\17)\0\0\0\252\n\0\0\350S9]\237>\354\17\t\0\0\0base_l"..., 4096) = 4096 _llseek(4, 1771728896, [1771728896], SEEK_SET) = 0 _llseek(4, 1771728896, [1771728896], SEEK_SET) = 0 write(3, "&\0\0\0\r\0\0\0time=\350S9]\223\t\274\17\t\0\0\0conn=\2\0"..., 367) = 367 fstat64(3, {st_mode=S_IFREG|0664, st_size=806901049, ...}) = 0 _llseek(3, 806899712, [806899712], SEEK_SET) = 0 read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 1337 _llseek(4, 1771728896, [1771728896], SEEK_SET) = 0 read(4, "\276\320\37\231\317\276\0\37\231\317\276\0\37\231\317\276\0\37\231\317\276\2\37\231\317\276 \37\231\317\276\260"..., 126226432) = 126226432 read(4, "\37\231\317\2760\37\231\317\276\2\37\231\317\276\2\37\231\317\276\0\37\231\317\276\r\37\231\317\276\r\37\231"..., 4096) = 4096 mmap2(NULL, 1514790912, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x4ca80000 mmap2(NULL, 126234624, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7e5000 mmap2(NULL, 504930304, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory) brk(0x65b00000)
= 0x47956000 mmap2(NULL, 505061376, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory) mmap2(NULL, 2097152, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0xb2a1c000 munmap(0xb2a1c000, 933888) = 0 munmap(0xb2c00000, 114688)