Attention: Answers.ros.org is deprecated as of August the 11th, 2023. Please visit robotics.stackexchange.com to ask a new question. This site will remain online in read-only mode during the transition and into the foreseeable future. Selected questions and answers have been migrated, and redirects have been put in place to direct users to the corresponding questions on Robotics Stack Exchange. Additional details are available here.

Buildfarm remote execution scaling

asked 2022-06-22 12:03:19 -0500

durosu1
1 ●1

I'm using the latest Buildfarm version. I managed to setup remote execution with Bazel. I can see the build time improvement (~25%) when I add the second remote server/worker. Beyond that, adding worker/server 3 and 4 does not improve the build time almost at all.

So there is a bottleneck somewhere but I could not figure out where. My buildfarm server config and the worker config are slightly modified example configs. (Higher cache limits, timeouts, cores)

Note: For this comparison I was building with the caching disabled --noremote_accept_cached.

edit retag flag offensive close merge delete

add a comment

1 Answer

Sort by » oldest newest most voted

answered 2022-06-22 22:56:19 -0500

tfoote

58457 ●128 ●550 ●526 http://www.ros.org/

There's likely a bottleneck. The first place to look is likely in the ability of Bazel to parallelize the specific package that you're attempting to build. Not all build steps can be parallelized so adding more workers beyond the maximum level of parallelization won't continue to result in faster builds.

edit flag offensive delete link

add a comment

Question Tools

1 follower

Stats

Asked: 2022-06-22 12:03:19 -0500

Seen: 62 times

Last updated: Jun 22 '22

Buildfarm remote execution scaling edit

1 Answer

Question Tools

Stats

Related questions

Buildfarm remote execution scaling