How do I limit the number of cores used in the build process by catkin?

asked 2013-03-24 03:17:35 -0500

FranciscoD
128 ●13 ●15 ●24

updated 2014-01-28 17:15:52 -0500

ngrennan
1 ●1 ●1

I've been building groovy using the instructions from the wiki. My processor is an i5, 2 cores, 4 threads. Building PCL is a huge CPU hog and my system takes a few hours when all 4 threads are being almost fully used throughout. The problem is, my laptop tends to overheat and shut down when it gets to critical temp because of all this work (yes, the room temperature isn't helping). I was wondering if someone could tell me where catkin picks the "-jX" value from? It uses -j4 at the moment. I was wondering if reducing it to a -j3 or a -j2 might help keep the temperature down. It might take longer, but I can leave it overnight knowing that it isn't going to shut down after overheating.

During normal usage, my system is at about 45*c, so building PCL does take quite some work.

answered 2013-03-24 12:18:25 -0500

jbohren

5799 ●55 ●87 ●102 http://dscl.lcsr.jhu.e...

updated 2013-03-25 07:18:07 -0500

If you don't specify the numbers of jobs explicitly with -jn for n jobs when calling make or catkin_make, Catkin gets the number of jobs from the ROS_PARALLEL_JOBS environment variable. From the catkin_make help:

  -j [JOBS], --jobs [JOBS]
                        Specifies the number of jobs (commands) to run
                        simultaneously. Defaults to the environment variable
                        ROS_PARALLEL_JOBS and falls back to the number of CPU
                        cores.

If you want to use n jobs by default, you can do the following before building a catkin project in catkin version 0.5.63:

export ROS_PARALLEL_JOBS=n

This is currently considered a compatibility bug, since for rosbuild, this should be:

export ROS_PARALLEL_JOBS=-jn

In the future versions of catkin, I believe the second option above will be the correct one. See this ticket here: https://github.com/ros/catkin/issues/335

edit flag offensive delete link

Comments

This is not a good variable to depend on probably since it is ROS_ prefixed, I would just advise using make's own arguments to achieve this: catkin_make -j1 ...

William ( 2013-03-24 21:31:44 -0500 )edit

Yeah, I think you're right, this environment variable is far too convenient and saves time for the user so it's probably going to be removed from catkin in the future...

jbohren ( 2013-03-25 02:08:13 -0500 )edit

@jbohren catkin strives to be ROS agnostic, it picks up the ROS_PARALLEL_JOBS environment variable as a convenience. I doubt it will ever be removed, but since many people will find this Q/A I think we should recommend that they use tools we don't maintain to solve the problem, aka make -jn

William ( 2013-03-25 06:49:59 -0500 )edit

I'd expect ROS_PARALLEL_JOBS to be removed from catkin, or at least renamed to CATKIN_PARALLEL_JOBS. I was surprised when catkin picked up my ROS_PARALLEL_JOBS environment variable. Partially because I know catkin intends to be unaware of ROS, but also because it interpreted it differently.

jbohren ( 2013-03-25 07:15:08 -0500 )edit

MAKEFLAGS is probably yet a better option, but catkin_make currently clobbers this option, so I opened a ticket about changing that: https://github.com/ros/catkin/issues/400

William ( 2013-03-25 13:55:00 -0500 )edit

add a comment

How do I limit the number of cores used in the build process by catkin?

1 Answer

Comments

Question Tools

Stats

Related questions

How do I limit the number of cores used in the build process by catkin? edit

1 Answer

Comments

Question Tools

Stats

Related questions

How do I limit the number of cores used in the build process by catkin?