rosbag exits recording abruptly after maximum file size is reached
I'm running octomap_mapping
on my robot, and also want to record data for post processing. The data mostly includes /tf
messages and /pointCloud2
data. I running my code on a real robot, so I do wireless ssh into the robot and then record the data on my remote machine. In the middle of recording, rosbag exits after recording 167Mb data and prints "Killed" and exit, leaving a .active file.
I have not had this issue before (when I run rosbag on the same computer as running ROS), I was wondering if this is happening because I'm recording data on a remote machine rather than the computer on the robot. Can someone confirm this theory and suggest a solution?
Here's a link to someone else having similar issues: http://ros-users.122217.n3.nabble.com/rosbag-exits-abnormally-td3637291.html
Edit: Here is the actual output from remote machine running rosbag command over ssh. This time it didn't crap out at 167Mb, 13Mb instead. I think 167Mb is not even a significant number as this may be just a chunk of data it was trying to store before it exited.
terminate called after throwing an instance of 'rosbag::BagIOException'
what(): Error writing to file: writing 13571512 bytes, wrote 7480895 bytes
Also, here is the kernel log:
[13694.190638] wlan0: deauthenticating from d8:67:d9:c2:d0:a0 by local choice (reason=3)
[13694.207447] cfg80211: Calling CRDA to update world regulatory domain
[13694.216192] cfg80211: World regulatory domain updated:
[13694.216198] cfg80211: (start_freq - end_freq @ bandwidth), (max_antenna_gain, max_eirp)
[13694.216202] cfg80211: (2402000 KHz - 2472000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
[13694.216205] cfg80211: (2457000 KHz - 2482000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
[13694.216208] cfg80211: (2474000 KHz - 2494000 KHz @ 20000 KHz), (300 mBi, 2000 mBm)
[13694.216211] cfg80211: (5170000 KHz - 5250000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
[13694.216213] cfg80211: (5735000 KHz - 5835000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
[13695.752099] PM: Syncing filesystems ... done.
[13695.982204] PM: Preparing system for mem sleep
[13697.548484] Freezing user space processes ... (elapsed 0.002 seconds) done.
[13697.550530] Freezing remaining freezable tasks ... (elapsed 0.001 seconds) done.
[13697.551847] PM: Entering mem sleep
[13697.552000] Suspending console(s) (use no_console_suspend to debug)
[13697.552241] sd 0:0:0:0: [sda] Synchronizing SCSI cache
[13697.552364] sd 0:0:0:0: [sda] Stopping disk
[13698.153598] [drm] Disabling audio 0 support
[13698.548633] PM: suspend of devices complete after 995.697 msecs
[13698.548815] PM: late suspend of devices complete after 0.179 msecs
[13698.564658] pcieport 0000:00:04.0: System wakeup enabled by ACPI
[13698.596559] ehci-pci 0000:00:12.2: System wakeup enabled by ACPI
[13698.612545] ohci-pci 0000:00:12.0: System wakeup enabled by ACPI
[13698.612632] PM: noirq suspend of devices complete after 63.761 msecs
[13698.612680] ACPI: Preparing to enter system sleep state S3
[13698.624727] PM: Saving platform NVS memory
[13698.628027] Disabling non-boot CPUs ...
[13698.628477] Broke affinity for irq 17
[13698.629498] kvm: disabling virtualization on CPU1
[13698.732585] smpboot: CPU 1 is now offline
[13698.734239] kvm: disabling virtualization on CPU2
[13698.836674] smpboot: CPU 2 is now offline
[13698.837263] Broke affinity for irq 43
[13698.838272] kvm: disabling virtualization on CPU3
[13698.940752] smpboot: CPU 3 is now offline
[13698.941416] ACPI: Low-level resume complete
[13698.941455] PM: Restoring platform NVS memory
[13698.943072] LVT offset 1 assigned for vector 0x400
[13698.943080] IBS: LVT offset 1 assigned
[13698.943155] Enabling non-boot CPUs ...
[13698.943190] x86: Booting SMP configuration:
[13698.943191] smpboot: Booting Node 0 Processor 1 APIC 0x1
[13698.953169] Initializing CPU#1
[13698.954160] kvm: enabling virtualization on CPU1
[13698.956337] process: Switch to broadcast mode on CPU1
[13698.956501] CPU1 is up
[13698.956515] smpboot: Booting Node 0 Processor 2 APIC 0x2
[13698.966489] Initializing CPU#2
[13698.967480] kvm: enabling virtualization on CPU2
[13698.969646] process: Switch to broadcast mode on CPU2
[13698.969768] CPU2 is up
[13698.969781] smpboot: Booting Node 0 Processor 3 APIC 0x3
[13698.979755] Initializing CPU#3
[13698.980746] kvm: enabling virtualization on CPU3
[13698.982946] process: Switch to broadcast mode on CPU3
[13698.983081] CPU3 is up
[13698.984375] ACPI: Waking up from system sleep state S3
[13698.984833] ohci-pci 0000:00:12.0: System wakeup disabled by ACPI
[13698.998808] ehci-pci 0000:00:12.2: System wakeup disabled by ACPI
[13699.094971] PM: noirq resume of devices complete after 110.227 msecs
[13699.095096] PM: early resume of devices complete after 0.100 msecs
[13699.096306] ath: phy0: ASPM enabled: 0x42
[13699.098431] [drm] enabling PCIE gen 2 link speeds, disable with radeon.pcie_gen2=0
[13699.098864] pcieport 0000:00:04.0: System wakeup disabled by ACPI
[13699.099724] snd_hda_intel 0000:01:00.1: irq 45 for MSI/MSI-X
[13699.143723] [drm] PCIE GART of 1024M enabled (table at 0x0000000000273000).
[13699.143816] radeon 0000:01:00.0: WB enabled
[13699.143819] radeon 0000:01:00.0: fence driver on ring 0 use gpu addr 0x0000000040000c00 and cpu addr 0xffd54c00
[13699.143821] radeon 0000:01:00.0: fence driver on ring 3 use gpu addr 0x0000000040000c0c and cpu addr 0xffd54c0c
[13699.145264] radeon 0000:01:00.0: fence driver on ring 5 use gpu addr 0x0000000000072118 and cpu addr 0xf90b2118
[13699.161560] [drm] ring test on 0 succeeded in 2 usecs
[13699.161626] [drm] ring test on 3 succeeded in 1 usecs
[13699.294993] usb 2-1: reset high-speed USB device number 2 using ehci-pci
[13699.348119] [drm] ring test on 5 succeeded in 2 usecs
[13699.348126] [drm] UVD initialized successfully.
[13699.348127] [drm] Enabling audio 0 support
[13699.348205] HDMI ATI/AMD: no speaker allocation for ELD
[13699.348217] [drm] ib test on ring 0 succeeded in 0 usecs
[13699.348296] [drm] ib test on ring 3 succeeded in 1 usecs
[13699.500173] [drm] ib test on ring 5 succeeded
[13699.591259] ata2: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
[13699.595455] ata2.00: configured for UDMA/100
[13699.647347] HDMI ATI/AMD: no speaker allocation for ELD
[13699.947597] HDMI ATI/AMD: no speaker allocation for ELD
[13700.247838] HDMI ATI/AMD: no speaker allocation for ELD
[13700.548079] HDMI ATI/AMD: no speaker allocation for ELD
[13700.720150] psmouse serio1: elantech: retrying ps2 command 0xe6 (2).
[13700.848329] HDMI ATI/AMD: no speaker allocation for ELD
[13701.148571] HDMI ATI/AMD: no speaker allocation for ELD
[13701.234167] psmouse serio1: elantech: retrying ps2 command 0xf8 (2).
[13701.272622] ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[13701.282584] ata1.00: configured for UDMA/133
[13701.296711] sd 0:0:0:0: [sda] Starting disk
[13701.336912] PM: resume of devices complete after 2239.987 msecs
[13701.337122] PM: Finishing wakeup.
[13701.337124] Restarting tasks ... done.
[13701.448818] HDMI ATI/AMD: no speaker allocation for ELD
[13701.749079] HDMI ATI/AMD: no speaker allocation for ELD
[13701.781237] video LNXVIDEO:01: Restoring backlight state
[13701.937270] psmouse serio1: elantech: retrying ps2 command 0xf8 (1).
[13702.118638] tg3 0000:02:00.0: irq 46 for MSI/MSI-X
[13702.129732] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[13702.174361] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[13703.100176] wlan0: authenticate with 80:1f:02:d4:b5:44
[13703.119419] wlan0: send auth to 80:1f:02:d4:b5:44 (try 1/3)
[13703.121364] wlan0: authenticated
[13703.122077] wlan0: associate with 80:1f:02:d4:b5:44 (try 1/3)
[13703.138624] wlan0: RX AssocResp from 80:1f:02:d4:b5:44 (capab=0x411 status=0 aid=4)
[13703.138683] wlan0: associated
[13703.138727] IPv6: ADDRCONF(NETDEV_CHANGE): wlan0: link becomes ready
[16080.871636] cfg80211: Calling CRDA to update world regulatory domain
[16080.881217] cfg80211: World regulatory domain updated:
[16080.881222] cfg80211: (start_freq - end_freq @ bandwidth), (max_antenna_gain, max_eirp)
[16080.881225] cfg80211: (2402000 KHz - 2472000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
[16080.881228] cfg80211: (2457000 KHz - 2482000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
[16080.881230] cfg80211: (2474000 KHz - 2494000 KHz @ 20000 KHz), (300 mBi, 2000 mBm)
[16080.881232] cfg80211: (5170000 KHz - 5250000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
[16080.881234] cfg80211: (5735000 KHz - 5835000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
[16081.799974] wlan0: authenticate with 80:1f:02:d4:b5:44
[16081.826605] wlan0: send auth to 80:1f:02:d4:b5:44 (try 1/3)
[16081.828658] wlan0: authenticated
[16081.830578] wlan0: associate with 80:1f:02:d4:b5:44 (try 1/3)
[16081.847145] wlan0: RX AssocResp from 80:1f:02:d4:b5:44 (capab=0x411 status=0 aid=4)
[16081.847234] wlan0: associated
[16810.385376] wlan0: deauthenticating from 80:1f:02:d4:b5:44 by local choice (reason=3)
[16810.398778] cfg80211: Calling CRDA to update world regulatory domain
[16810.405462] cfg80211: World regulatory domain updated:
[16810.405468] cfg80211: (start_freq - end_freq @ bandwidth), (max_antenna_gain, max_eirp)
[16810.405471] cfg80211: (2402000 KHz - 2472000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
[16810.405473] cfg80211: (2457000 KHz - 2482000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
[16810.405475] cfg80211: (2474000 KHz - 2494000 KHz @ 20000 KHz), (300 mBi, 2000 mBm)
[16810.405477] cfg80211: (5170000 KHz - 5250000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
[16810.405479] cfg80211: (5735000 KHz - 5835000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
[16813.332870] PM: Syncing filesystems ... done.
[16813.542431] PM: Preparing system for mem sleep
[16813.968246] Freezing user space processes ... (elapsed 0.002 seconds) done.
[16813.970383] Freezing remaining freezable tasks ... (elapsed 0.001 seconds) done.
[16813.971685] PM: Entering mem sleep
[16813.971838] Suspending console(s) (use no_console_suspend to debug)
[16813.972084] sd 0:0:0:0: [sda] Synchronizing SCSI cache
[16813.972207] sd 0:0:0:0: [sda] Stopping disk
[16814.534973] [drm] Disabling audio 0 support
[16814.878899] PM: suspend of devices complete after 906.198 msecs
[16814.879079] PM: late suspend of devices complete after 0.177 msecs
[16814.894785] pcieport 0000:00:04.0: System wakeup enabled by ACPI
[16814.926798] ehci-pci 0000:00:12.2: System wakeup enabled by ACPI
[16814.942812] ohci-pci 0000:00:12.0: System wakeup enabled by ACPI
[16814.942897] PM: noirq suspend of devices complete after 63.762 msecs
[16814.942943] ACPI: Preparing to enter system sleep state S3
[16814.962892] PM: Saving platform NVS memory
[16814.966252] Disabling non-boot CPUs ...
[16814.967586] kvm: disabling virtualization on CPU1
[16814.967612] smpboot: CPU 1 is now offline
[16814.968117] Broke affinity for irq 17
[16814.969139] kvm: disabling virtualization on CPU2
[16815.070826] smpboot: CPU 2 is now offline
[16815.072326] kvm: disabling virtualization on CPU3
[16815.174903] smpboot: CPU 3 is now offline
[16815.175502] ACPI: Low-level resume complete
[16815.175541] PM: Restoring platform NVS memory
[16815.177180] LVT offset 1 assigned for vector 0x400
[16815.177187] IBS: LVT offset 1 assigned
[16815.177243] Enabling non-boot CPUs ...
[16815.177278] x86: Booting SMP configuration:
[16815.177280] smpboot: Booting Node 0 Processor 1 APIC 0x1
[16815.187258] Initializing CPU#1
[16815.188249] kvm: enabling virtualization on CPU1
[16815.190435] process: Switch to broadcast mode on CPU1
[16815.190603] CPU1 is up
[16815.190617] smpboot: Booting Node 0 Processor 2 APIC 0x2
[16815.200591] Initializing CPU#2
[16815.201582] kvm: enabling virtualization on CPU2
[16815.203747] process: Switch to broadcast mode on CPU2
[16815.203866] CPU2 is up
[16815.203878] smpboot: Booting Node 0 Processor 3 APIC 0x3
[16815.213852] Initializing CPU#3
[16815.214844] kvm: enabling virtualization on CPU3
[16815.217014] process: Switch to broadcast mode on CPU3
[16815.217139] CPU3 is up
[16815.218442] ACPI: Waking up from system sleep state S3
[16815.218900] ohci-pci 0000:00:12.0: System wakeup disabled by ACPI
[16815.233003] ehci-pci 0000:00:12.2: System wakeup disabled by ACPI
[16815.329171] PM: noirq resume of devices complete after 110.359 msecs
[16815.329295] PM: early resume of devices complete after 0.099 msecs
[16815.332649] [drm] enabling PCIE gen 2 link speeds, disable with radeon.pcie_gen2=0
[16815.333029] pcieport 0000:00:04.0: System wakeup disabled by ACPI
[16815.333155] snd_hda_intel 0000:01:00.1: irq 45 for MSI/MSI-X
[16815.334093] ath: phy0: ASPM enabled: 0x42
[16815.338809] [drm] PCIE GART of 1024M enabled (table at 0x0000000000273000).
[16815.338902] radeon 0000:01:00.0: WB enabled
[16815.338905] radeon 0000:01:00.0: fence driver on ring 0 use gpu addr 0x0000000040000c00 and cpu addr 0xffd54c00
[16815.338907] radeon 0000:01:00.0: fence driver on ring 3 use gpu addr 0x0000000040000c0c and cpu addr 0xffd54c0c
[16815.340350] radeon 0000:01:00.0: fence driver on ring 5 use gpu addr 0x0000000000072118 and cpu addr 0xf90b2118
[16815.356647] [drm] ring test on 0 succeeded in 2 usecs
[16815.356713] [drm] ring test on 3 succeeded in 1 usecs
[16815.525208] usb 2-1: reset high-speed USB device number 2 using ehci-pci
[16815.543206] [drm] ring test on 5 succeeded in 2 usecs
[16815.543213] [drm] UVD initialized successfully.
[16815.543215] [drm] Enabling audio 0 support
[16815.543314] [drm] ib test on ring 0 succeeded in 0 usecs
[16815.543316] HDMI ATI/AMD: no speaker allocation for ELD
[16815.543392] [drm] ib test on ring 3 succeeded in 1 usecs
[16815.695326] [drm] ib test on ring 5 succeeded
[16815.825465] ata2: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
[16815.829661] ata2.00: configured for UDMA/100
[16815.841527] HDMI ATI/AMD: no speaker allocation for ELD
[16816.141787] HDMI ATI/AMD: no speaker allocation for ELD
[16816.442023] HDMI ATI/AMD: no speaker allocation for ELD
[16816.742260] HDMI ATI/AMD: no speaker allocation for ELD
[16816.954352] psmouse serio1: elantech: retrying ps2 command 0xe6 (2).
[16817.042507] HDMI ATI/AMD: no speaker allocation for ELD
[16817.342761] HDMI ATI/AMD: no speaker allocation for ELD
[16817.469261] psmouse serio1: elantech: retrying ps2 command 0xf8 (2).
[16817.506809] ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[16817.514809] ata1.00: configured for UDMA/133
[16817.530902] sd 0:0:0:0: [sda] Starting disk
[16817.579121] PM: resume of devices complete after 2247.991 msecs
[16817.579331] PM: Finishing wakeup.
[16817.579332] Restarting tasks ... done.
[16817.642887] HDMI ATI/AMD: no speaker allocation for ELD
[16817.943241] HDMI ATI/AMD: no speaker allocation for ELD
[16818.175355] psmouse serio1: elantech: retrying ps2 command 0xf8 (1).
[16818.455819] video LNXVIDEO:01: Restoring backlight state
[16818.805212] init: anacron main process (20415) killed by TERM signal
[16819.019324] tg3 0000:02:00.0: irq 46 for MSI/MSI-X
[16819.031026] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[16819.076297] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[16820.822092] wlan0: authenticate with d8:67:d9:c2:d0:a0
[16820.849096] wlan0: send auth to d8:67:d9:c2:d0:a0 (try 1/3)
[16820.853845] wlan0: authenticated
[16820.857558] wlan0: associate with d8:67:d9:c2:d0:a0 (try 1/3)
[16820.861319] wlan0: RX AssocResp from d8:67:d9:c2:d0:a0 (capab=0x411 status=0 aid=10)
[16820.861372] wlan0: associated
[16820.861384] IPv6: ADDRCONF(NETDEV_CHANGE): wlan0: link becomes ready
[16820.862499] cfg80211: Calling CRDA for country: US
[16820.866506] ath: EEPROM regdomain: 0x8348
[16820.866511] ath: EEPROM indicates we should expect a country code
[16820.866513] ath: doing EEPROM country->regdmn map search
[16820.866515] ath: country maps to regdmn code: 0x3a
[16820.866517] ath: Country alpha2 being used: US
[16820.866518] ath: Regpair used: 0x3a
[16820.866521] ath: regdomain 0x8348 dynamically updated by country IE
[16820.866546] cfg80211: Regulatory domain changed to country: US
[16820.866548] cfg80211: (start_freq - end_freq @ bandwidth), (max_antenna_gain, max_eirp)
[16820.866550] cfg80211: (2402000 KHz - 2472000 KHz @ 40000 KHz), (300 mBi, 2700 mBm)
[16820.866553] cfg80211: (5170000 KHz - 5250000 KHz @ 40000 KHz), (300 mBi, 1700 mBm)
[16820.866554] cfg80211: (5250000 KHz - 5330000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
[16820.866556] cfg80211: (5490000 KHz - 5600000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
[16820.866558] cfg80211: (5650000 KHz - 5710000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
[16820.866560] cfg80211: (5735000 KHz - 5835000 KHz @ 40000 KHz), (300 mBi, 3000 mBm)
[16820.866562] cfg80211: (57240000 KHz - 63720000 KHz @ 2160000 KHz), (N/A, 4000 mBm)
[16820.882686] wlan0: deauthenticating from d8:67:d9:c2:d0:a0 by local choice (reason=2)
[16820.895623] cfg80211: Calling CRDA to update world regulatory domain
[16820.896967] wlan0: authenticate with d8:67:d9:c2:d0:a0
[16820.916484] wlan0: send auth to d8:67:d9:c2:d0:a0 (try 1/3)
[16820.916908] cfg80211: World regulatory domain updated:
[16820.916920] cfg80211: (start_freq - end_freq @ bandwidth), (max_antenna_gain, max_eirp)
[16820.916928] cfg80211: (2402000 KHz - 2472000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
[16820.916935] cfg80211: (2457000 KHz - 2482000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
[16820.916940] cfg80211: (2474000 KHz - 2494000 KHz @ 20000 KHz), (300 mBi, 2000 mBm)
[16820.916945] cfg80211: (5170000 KHz - 5250000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
[16820.916950] cfg80211: (5735000 KHz - 5835000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
[16820.921333] wlan0: authenticated
[16820.925568] wlan0: associate with d8:67:d9:c2:d0:a0 (try 1/3)
[16820.929885] wlan0: RX AssocResp from d8:67:d9:c2:d0:a0 (capab=0x411 status=0 aid=10)
[16820.929944] wlan0: associated
[16820.930026] cfg80211: Calling CRDA for country: US
[16820.933162] ath: EEPROM regdomain: 0x8348
[16820.933166] ath: EEPROM indicates we should expect a country code
[16820.933169] ath: doing EEPROM country->regdmn map search
[16820.933171] ath: country maps to regdmn code: 0x3a
[16820.933173] ath: Country alpha2 being used: US
[16820.933174] ath: Regpair used: 0x3a
[16820.933176] ath: regdomain 0x8348 dynamically updated by country IE
[16820.933202] cfg80211: Regulatory domain changed to country: US
[16820.933204] cfg80211: (start_freq - end_freq @ bandwidth), (max_antenna_gain, max_eirp)
[16820.933206] cfg80211: (2402000 KHz - 2472000 KHz @ 40000 KHz), (300 mBi, 2700 mBm)
[16820.933208] cfg80211: (5170000 KHz - 5250000 KHz @ 40000 KHz), (300 mBi, 1700 mBm)
[16820.933210] cfg80211: (5250000 KHz - 5330000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
[16820.933212] cfg80211: (5490000 KHz - 5600000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
[16820.933214] cfg80211: (5650000 KHz - 5710000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
[16820.933216] cfg80211: (5735000 KHz - 5835000 KHz @ 40000 KHz), (300 mBi, 3000 mBm)
[16820.933217] cfg80211: (57240000 KHz - 63720000 KHz @ 2160000 KHz), (N/A, 4000 mBm)
[17495.198789] [sched_delayed] \x014CE: hpet increased min_delta_ns to 20115 nsec
[17515.853406] usb 3-3: new full-speed USB device number 2 using ohci-pci
[17516.030579] usb 3-3: New USB device found, idVendor=045e, idProduct=028e
[17516.030586] usb 3-3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[17516.030589] usb 3-3: Product: Controller
[17516.030592] usb 3-3: Manufacturer: \xffffffc2\xffffffa9\xffffffa9Microsoft Corporation
[17516.030594] usb 3-3: SerialNumber: 2162D9B
[17516.103789] input: Microsoft X-Box 360 pad as /devices/pci0000:00/0000:00:12.0/usb3/3-3/3-3:1.0/input/input14
[17516.106815] usbcore: registered new interface driver xpad
[17519.781091] usb 3-3: USB disconnect, device number 2
[17616.643903] wlan0: authenticate with d8:67:d9:c2:d3:18
[17616.665004] wlan0: send auth to d8:67:d9:c2:d3:18 (try 1/3)
[17616.665262] cfg80211: Calling CRDA to update world regulatory domain
[17616.668712] wlan0: authenticated
[17616.669781] wlan0: associate with d8:67:d9:c2:d3:18 (try 1/3)
[17616.679578] wlan0: RX AssocResp from d8:67:d9:c2:d3:18 (capab=0x411 status=0 aid=4)
[17616.679659] wlan0: associated
[17616.763350] cfg80211: World regulatory domain updated:
[17616.763355] cfg80211: (start_freq - end_freq @ bandwidth), (max_antenna_gain, max_eirp)
[17616.763359] cfg80211: (2402000 KHz - 2472000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
[17616.763361] cfg80211: (2457000 KHz - 2482000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
[17616.763363] cfg80211: (2474000 KHz - 2494000 KHz @ 20000 KHz), (300 mBi, 2000 mBm)
[17616.763365] cfg80211: (5170000 KHz - 5250000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
[17616.763367] cfg80211: (5735000 KHz - 5835000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
[17616.763736] cfg80211: Calling CRDA for country: US
[17616.766417] ath: EEPROM regdomain: 0x8348
[17616.766421] ath: EEPROM indicates we should expect a country code
[17616.766424] ath: doing EEPROM country->regdmn map search
[17616.766426] ath: country maps to regdmn code: 0x3a
[17616.766427] ath: Country alpha2 being used: US
[17616.766429] ath: Regpair used: 0x3a
[17616.766431] ath: regdomain 0x8348 dynamically updated by country IE
[17616.766457] cfg80211: Regulatory domain changed to country: US
[17616.766459] cfg80211: (start_freq - end_freq @ bandwidth), (max_antenna_gain, max_eirp)
[17616.766461] cfg80211: (2402000 KHz - 2472000 KHz @ 40000 KHz), (300 mBi, 2700 mBm)
[17616.766463] cfg80211: (5170000 KHz - 5250000 KHz @ 40000 KHz), (300 mBi, 1700 mBm)
[17616.766465] cfg80211: (5250000 KHz - 5330000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
[17616.766467] cfg80211: (5490000 KHz - 5600000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
[17616.766469] cfg80211: (5650000 KHz - 5710000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
[17616.766470] cfg80211: (5735000 KHz - 5835000 KHz @ 40000 KHz), (300 mBi, 3000 mBm)
[17616.766472] cfg80211: (57240000 KHz - 63720000 KHz @ 2160000 KHz), (N/A, 4000 mBm)
[18142.807543] show_signal_msg: 123 callbacks suppressed
[18142.807554] rviz[23963]: segfault at 3f800000 ip 3f800000 sp bfa76c6c error 14 in SYSV00000000 (deleted)[a281a000+fa000]
[18217.146489] wlan0: authenticate with d8:67:d9:c2:d0:a0
[18217.165644] wlan0: send auth to d8:67:d9:c2:d0:a0 (try 1/3)
[18217.165763] cfg80211: Calling CRDA to update world regulatory domain
[18217.169805] wlan0: authenticated
[18217.171360] wlan0: associate with d8:67:d9:c2:d0:a0 (try 1/3)
[18217.171787] cfg80211: World regulatory domain updated:
[18217.171792] cfg80211: (start_freq - end_freq @ bandwidth), (max_antenna_gain, max_eirp)
[18217.171794] cfg80211: (2402000 KHz - 2472000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
[18217.171797] cfg80211: (2457000 KHz - 2482000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
[18217.171799] cfg80211: (2474000 KHz - 2494000 KHz @ 20000 KHz), (300 mBi, 2000 mBm)
[18217.171801] cfg80211: (5170000 KHz - 5250000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
[18217.171802] cfg80211: (5735000 KHz - 5835000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
[18217.175336] wlan0: RX AssocResp from d8:67:d9:c2:d0:a0 (capab=0x411 status=0 aid=5)
[18217.175399] wlan0: associated
[18217.175708] cfg80211: Calling CRDA for country: US
[18217.179545] ath: EEPROM regdomain: 0x8348
[18217.179550] ath: EEPROM indicates we should expect a country code
[18217.179553] ath: doing EEPROM country->regdmn map search
[18217.179555] ath: country maps to regdmn code: 0x3a
[18217.179557] ath: Country alpha2 being used: US
[18217.179559] ath: Regpair used: 0x3a
[18217.179561] ath: regdomain 0x8348 dynamically updated by country IE
[18217.179587] cfg80211: Regulatory domain changed to country: US
[18217.179589] cfg80211: (start_freq - end_freq @ bandwidth), (max_antenna_gain, max_eirp)
[18217.179591] cfg80211: (2402000 KHz - 2472000 KHz @ 40000 KHz), (300 mBi, 2700 mBm)
[18217.179594] cfg80211: (5170000 KHz - 5250000 KHz @ 40000 KHz), (300 mBi, 1700 mBm)
[18217.179596] cfg80211: (5250000 KHz - 5330000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
[18217.179598] cfg80211: (5490000 KHz - 5600000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
[18217.179600] cfg80211: (5650000 KHz - 5710000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
[18217.179602] cfg80211: (5735000 KHz - 5835000 KHz @ 40000 KHz), (300 mBi, 3000 mBm)
[18217.179604] cfg80211: (57240000 KHz - 63720000 KHz @ 2160000 KHz), (N/A, 4000 mBm)
[18545.193209] type=1400 audit(1412184066.598:74): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="/usr/lib/cups/backend/cups-pdf" pid=26205 comm="apparmor_parser"
[18545.193225] type=1400 audit(1412184066.598:75): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="/usr/sbin/cupsd" pid=26205 comm="apparmor_parser"
[18545.194058] type=1400 audit(1412184066.602:76): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="/usr/sbin/cupsd" pid=26205 comm="apparmor_parser"
Asked by ajain on 2014-09-30 14:55:34 UTC
Answers
It doesn't look like rosbag is being killed because it's out of memory; rather it looks like it's dying because it doesn't write the expected number of bytes to disk.
The causes for this sort of failure are fairly obscure. The man page for write(2) suggests a few possible causes:
- The disk is full
- You've hit your maximum file size limit (RLIMIT_FSIZE)
- The call to write was interrupted
You've already indicated that that disk isn't full, which rules out #1 It fails at different sizes for different runs, which probably rules out #2
That leaves #3, which suggests that the call is being interrupted for some reason. I don't really have much insight into what might be interrupting it, and the error reporting within rosbag isn't reporting which errno was set after the failed write, so it's difficult to diagnose further.
The write man page also suggests that there are other, system-specific reasons that write could fail, but they're even more obscure and difficult to diagnose.
You may want to try some experiments with a simpler file creation tool to determine if the particular chunk size or file systems limits are getting in the way. I would try:
dd if=/dev/zero of=test1.data bs=13M count=1 # write a 13MB chunk to disk. If this fails, the chunk size may be the problem
dd if=/dev/zero of=test2.data bs=1M # write 1MB chunks to disk until it's full. This is similar to how rosbag writes, but much simpler. If this fails for any reason other than the disk being full, that is very interesting
dd if=/dev/zero of=test3.data bs=13M # write 13MB chunks to disk until it's full.
(you should probably remove the test files created between tests)
If you can provide the flags that you're passing to rosbag, those may also help debug the problem further.
Asked by ahendrix on 2014-10-01 13:27:47 UTC
Comments
Thanks for the explanation ahendrix. It makes sense. I haven't been using any flags other than -a for these experiments (rosbag record -a). Next thing worth trying would be to simply run rosbag on the robot computer itself rather than recording it on a remote machine over a network using ssh.
Asked by ajain on 2014-10-01 13:47:43 UTC
You definitely do not want to record everything remotely. You will saturate your network quickly and lose a lot of information.
Asked by tfoote on 2014-10-01 18:19:46 UTC
Comments
Can you explain what you mean by "maximum file size reached" in the question title? Is the robot's disk full?
Asked by ahendrix on 2014-09-30 15:02:43 UTC
When I start recording data on my remote machine, rosbag process exits after recording 167Mb of data. There is more than enough space on my remote machine to store up to GBs of data.
Asked by ajain on 2014-09-30 15:22:22 UTC
Which command-line options are you passing to rosbag? Is there anything in your kernel log (dmesg) indicating that rosbag was killed because the system ran out of memory?
Asked by ahendrix on 2014-09-30 16:21:51 UTC
Deafult max size of rosbag is 256MB
use -b option to speciy bag size and try recording
Asked by bvbdort on 2014-09-30 17:30:45 UTC
-b
specifies the buffer size, not the maximum bag file size. This is how much memory rosbag will use while recording messages. If the buffer fills up, additional messages will be dropped until some data can be written to disk.Asked by ahendrix on 2014-09-30 18:25:50 UTC
I was confused by the same thing. "internal buffer" refers to the internal memory that ROS uses to store data temporarily before dumping it on to the disk.
Asked by ajain on 2014-09-30 18:43:32 UTC
Sorry, got blocked on hardware, will post results as soon as robot is working again. Thanks
Asked by ajain on 2014-09-30 18:44:04 UTC
This pops up another question though, what does dropping additional data mean, will ROS not record more data? If so, how can I make sure data is continuously recording (and being stored on disk before filling up internal buffer) and I don't miss any pieces of it?
Asked by ajain on 2014-09-30 18:47:42 UTC
That is correct - any messages that are dropped are not recorded. The only way to guarantee that you don't miss any data is to make sure that your disk has sufficient write bandwidth for the amount of data that you're trying to record.
Asked by ahendrix on 2014-09-30 19:02:55 UTC
Can you share error log or screenshot the shot of exact error you are getting
Asked by bvbdort on 2014-10-01 02:40:26 UTC
Adding logs and other details to the edit above
Asked by ajain on 2014-10-01 12:42:32 UTC