Robotics StackExchange | Archived questions

roscore using up all available memory

Hello,

I am running ros noetic inside a docker container. For that I use the following image:

ros:noetic-ros-base-focal

I sat the memory limit of the container to 8gb. When I run roscore without any other node running, it uses all memory, hits the 8gb limit and restart the process with the following error:

started core service [/rosout]
[rosout-1] process has died [pid 33303, exit code -9, cmd /opt/ros/noetic/lib/rosout/rosout __name:=rosout __log:=/root/.ros/log/3332fba4-07df-11ee-aa40-a87eea88c6dd/rosout-1.log].
log file: /root/.ros/log/3332fba4-07df-11ee-aa40-a87eea88c6dd/rosout-1*.log
[rosout-1] restarting process

The log file mentioned in the error is empty, but this is the contents of the master.log file:

[rosmaster.main][INFO] 2023-06-11 00:36:11,859: initialization complete, waiting for shutdown
[rosmaster.main][INFO] 2023-06-11 00:36:11,860: Starting ROS Master Node
[xmlrpc][INFO] 2023-06-11 00:36:11,861: XML-RPC server binding to 0.0.0.0:11311
[xmlrpc][INFO] 2023-06-11 00:36:11,861: Started XML-RPC server [http://fedora:11311/]
[xmlrpc][INFO] 2023-06-11 00:36:11,861: xml rpc node: starting XML-RPC server
[rosmaster.master][INFO] 2023-06-11 00:36:11,861: Master initialized: port[11311], uri[http://fedora:11311/]
[rosmaster.master][INFO] 2023-06-11 00:36:11,898: +PARAM [/run_id] by /roslaunch
[rosmaster.master][INFO] 2023-06-11 00:36:11,899: +PARAM [/roslaunch/uris/host_fedora__39467] by /roslaunch
[rosmaster.master][INFO] 2023-06-11 00:36:11,964: +PARAM [/rosversion] by /roslaunch
[rosmaster.master][INFO] 2023-06-11 00:36:11,964: +PARAM [/rosdistro] by /roslaunch
[rosmaster.main][INFO] 2023-06-11 00:36:19,873: keyboard interrupt, will exit
[rosmaster.main][INFO] 2023-06-11 00:36:19,874: stopping master...
[rospy.core][INFO] 2023-06-11 00:36:19,874: signal_shutdown [atexit]

I already tried this (https://answers.ros.org/question/336963/rosout-high-memory-usage/) but it did not fix the error.

Thank you in advance and let me know if you need any more information.

Asked by md42 on 2023-06-10 17:57:30 UTC

Comments

Please edit your post to include the command you use to start docker container. It seems very likely that the link you included correctly identifies the cause of the high memory use.

Asked by Mike Scheutzow on 2023-06-11 05:54:32 UTC

Thank you very much for your comment. It made me look at the post again and you are correct. Looking through the answers I noticed the ulimit -Sn 524288 again. It turns out I need to set the soft limit first and then the hard limit: ulimit -Sn 524288 && ulimit -Hn 524288. I did it this way because I wanted to fix this issue on an existing container.

Asked by md42 on 2023-06-12 19:10:54 UTC

Answers

The solution is to run ulimit -Sn 524288 && ulimit -Hn 524288. Thank you to Mike Scheutzow and this post

Asked by md42 on 2023-06-12 19:16:35 UTC

Comments

The number refers to the number of files+connections open at once for a linux process. I would configure this for the specific docker container, not all processes on the machine. If only for the docker container, you can probably use a much smaller number, like 1024 or 4096.

Asked by Mike Scheutzow on 2023-06-13 08:08:08 UTC