Image related nodes eating up majority of cpu until system becomes unresponsive?

asked 2016-04-26 08:27:56 -0500

lucasw gravatar image

updated 2020-11-21 12:08:16 -0500

Has anyone seen issues with nodes consuming a great deal of cpu (e.g. 400% plus), making gnome terminal turn gray and unresponsive, and either requiring ctrl-c killing the roslaunch containing the offending node (which sometimes works, usually taking 10 seconds or more for the nodes to come down), or killing gnome-terminal which Ubuntu itself sometimes suggests but all the open terminals have to be quit, or ctrl-alt-fX or sshing in from another computer to run top and kill because the entire window manager is so unresponsive?

It has been hard to track this down because system and ros tools become unresponsive as well- I could have a second machine try to set the first as ros master, or run the ros master on the second to begin with so that it can more likely be reached.

What I have seen is that Image related nodes seem to be at the top of top- rqt_image_view more than a few times, image_proc some other times. I've deliberately throttled frame rates to 5 hz or so to avoid legitimate high cpu usage- but maybe if these nodes had large queue sizes and got behind, then suddenly wanted to process a bunch of old images they would slow way down?

Should they be spanning multiple cpu cores? My own nodes usually don't, and when I've done something wrong the damage is usually limited to a single core going to 100% which means tons of cpu left over for the OS to allow me to see that and stop them, and then fix them.

I'd like to limit all nodes to a certain percentage of cpu- cpulimit came up in searches, it looks like there would have to be a script that gets a list of processes then limits them after they have started.

I think this started happening with Ubuntu 14.04 and Jade. I believe I've seen other non-ros instances coincident with upgrading to 14.04 of something happening to cause gnome-terminal to turn gray which was initially very surprising, but was intermittent enough I didn't invest time in tracking it down. This is on a fast desktop with 8 real cores (which is seen as 16 in linux), and an i7 class laptop (both around a year old so not the latest intel architecture).

edit retag flag offensive close merge delete

Comments

I am having latest architecture, 8th gen and even I am having same problem. Suddenly while working my CPU usage on single core turns 100% and system crash. I am using Image_view and image_proc.

i5 8600k 6 Physical cores and GTX 1080.

I am only detecting different whycon markers and nothing else. Usually cpu usage remains in range 15-20 but after sometime it increases to 100% on single core and system crash.

I have already over-clocked my PC to 4.5 Ghz.

nalin1997 gravatar image nalin1997  ( 2018-02-15 14:46:41 -0500 )edit

Please repost this as a comment, as this is not an answer to @lucasw's question.

gvdhoorn gravatar image gvdhoorn  ( 2018-02-16 01:45:18 -0500 )edit