What is the proper way to set up a build farm

asked 2019-08-07 19:10:28 -0500

max-krichenbauer
17 ●5 ●10 ●13

Hello!

Sorry to bother people with noob-level questions, but I’m currently learning how to set up a ROS build for work and am trying to compile the complete process into one compehensive step-by-step guide, but there are some points that are unclear and I’m not sure if I even understood the whole process correctly to begin with.

Could someone please help me out in pulling together the various information?

(Q1:) The documentation of buildfarm_deployment states that buildfarm is "based on Ubuntu 16.04 Xenial", but it is unclear if that is the OS that the machines (physical servers) need to have or if that is the Docker environment that will run on them after installation. Is there a required OS for the physical servers?

(Q2:) The ROS2 CI tutorial on index.ros.org describes a completely different process, but sometimes seems to use some of the same scripts and repos. Is this the "new way" for ROS2? Is the process the same for ROS1 and ROS2?

(Q3:) Why is it necessary to fork buildfarm_deployment_config and buildfarm_config? As far as I can see it will later just be cloned during deployment or accessed via Python urllib. Is this just for convenience when deploying many build farms on a reguar basis so we don’t have to edit the config on the machine itself? Shouldn't it be possible to just clone or download the *_config repos and edit the settings locally?

(Q4:) The whole process (including AWS servers, repo-forking, Docker, and Puppet) seems quite extensive for setting up one build farm. I seems like this is intended for people who intend to deploy build farms on a regular basis. Is it even recommended to follow this guide when you build just one build farm?

I’d be happy if someone with experience could help me out with these issues.

Thank you!

edit retag flag offensive close merge delete

add a comment

answered 2019-08-07 20:19:34 -0500

tfoote

58457 ●128 ●550 ●526 http://www.ros.org/

Q1) That's the host OS. The docker images vary based on the targets.

Q2) The ROS 2 CI setup is a separate system that we use for core development. You can see it at: https://ci.ros2.org/ We reuse as much of the existing tools as possible.

Q3) It is recommended to fork the config because it's expected that the buildfarm agents automatically update from the config regularly. If you modify the checkout locally it will be overwritten the next time that the machine automatically updates. It also means that when you setup a new agent your configuration is available and consistent across all the machines.

Q4) The documentation is how we setup and run the production build farm at http://build.ros.org/ They are optimized to be able to be reproducible at the scale of whole or partial rosdistros across multiple target platforms and be maintained over time. There is a good amount of setup required. There is definitely more opportunity to make it easier to stand up a buildfarm but there are diminishing returns as we try to automate more and more of the steps. And there's many areas that need to be customized for individual hosting configurations etc. If you're looking to just build a few packages it's likely that lighter weight solutions are better for your needs.

Comments

Thank you so much! That made things much clearer for me.

max-krichenbauer ( 2019-08-07 21:30:21 -0500 )edit

add a comment

What is the proper way to set up a build farm

1 Answer

Comments

Question Tools

Stats

Related questions

What is the proper way to set up a build farm edit

1 Answer

Comments

Question Tools

Stats

Related questions

What is the proper way to set up a build farm