How to add a new distro to a ROS build farm?
I've got an existing local build farm that has been building Indigo / Kinetic packages for a while now, and I want to add Melodic to it. Is the upgrade process documented anywhere?
I've tried to figure it out myself but am running into some problems. So far I have:
- Edited the repo.yaml file in my buildfarm_deployment_config repository to add "bionic" to the list of distros in my building/testing/main repositories and the ros_bootstraml.yaml file
- Added doc/release/source yaml files into a "melodic" directory in my ros_buildfarm_config repository, then referenced them in the index.yaml file
- Run buildfarm_config/reconfigure.bash on my repository machine
- Run the ros_buildfarm generate_all_jobs.py script on my new ros_buildfarm_config
That was enough to create all of the M* jobs in the Manage tab in Jenkins, as well as the M* tabs.
I would expect the next step I need to run is to use the import_upstream job with the ros_bootstramp.yaml config file, but it fails like so:
# BEGIN SUBSECTION: import debian packages for ubuntu_building
11:19:04 + python -u /home/jenkins-agent/workspace/import_upstream/reprepro-updater/scripts/import_upstream.py ubuntu_building /home/jenkins-agent/reprepro_config/ros_bootstrap.yaml --commit
11:19:04 loading config file: /home/jenkins-agent/reprepro_config/ros_bootstrap.yaml
11:19:04 adding arches set(['amd64', 'source', 'i386', 'armel', 'armhf', 'arm64']) and suites: set(['trusty', 'bionic', 'xenial'])
11:19:04 target_distros ['bionic', 'trusty', 'xenial']
11:19:04 target_arches ['bionic', 'trusty', 'xenial']
11:19:04 Updating for bionic amd64 to update into repo /var/repos/ubuntu/building
11:19:04 I have a lock on /var/repos/ubuntu/building/lock
11:19:04 Creating updates file /var/repos/ubuntu/building/conf/updates
11:19:04 Name: ros_bootstrap
11:19:04 Method: http://repos.ros.org/repos/ros_bootstrap
11:19:04 Suite: bionic
11:19:04 Components: main
11:19:04 Architectures: amd64
11:19:04 VerifyRelease: blindtrust
11:19:04
11:19:04 Creating distributions file /var/repos/ubuntu/building/conf/distributions
11:19:04 running command ['reprepro', '-v', '-b', '/var/repos/ubuntu/building', '--noskipold', 'update', 'bionic']
11:19:05 No distribution definition of 'bionic' found in '/var/repos/ubuntu/building/conf/distributions'!
11:19:05 There have been errors!
11:19:06 Execution of [['reprepro', '-v', '-b', '/var/repos/ubuntu/building', '--noskipold', 'update', 'bionic']] Failed: Command '['reprepro', '-v', '-b', '/var/repos/ubuntu/building', '--noskipold', 'update', 'bionic']' returned non-zero exit status 249
11:19:06 Traceback (most recent call last):
11:19:06 File "/home/jenkins-agent/workspace/import_upstream/reprepro-updater/scripts/import_upstream.py", line 68, in <module>
11:19:06 updates_generator, distro, arch, options.commit)
11:19:06 File "/home/jenkins-agent/workspace/import_upstream/reprepro-updater/src/reprepro_updater/helpers.py", line 185, in run_update
11:19:06 raise RuntimeError('update command failed')
11:19:06 RuntimeError: update command failed
11:19:06 Build step 'Execute shell' marked build as failure
11:19:06 Finished: FAILURE
After running it, I can verify that the file at /var/repos/ubuntu/building/conf/distributions only has entries for trusty and ...
I figured out that the configuration for that script is in ~jenkins-agent/.buildfarm/reprepro_updater.ini . I manually updated it and it then successfully ran and created the repository. I would have expected that file to be updated by running reconfigure.bash, but maybe I'm still missing something.
Re-running
reconfigure.bash
is actually not recommended, as is also mentioned in the docs ofbuildfarm_deployment_config
and in other places.I don't necessarily know how to do what you wanted to do though. I assume @nuclearsandwich will respond with some suggestions at a later time.
I don't see anything in the documentation for
buildfarm_deployment_config
orbuildfarm_deployment
that advises against re-runningreconfigure.bash
; in fact, that seems at odds with remarks inside the script itself that refer to re-running it. What's the right way to reconfigure a machine?Can't find the reference in the docs right now, but see ros-infrastructure/buildfarm_deployment#160 fi.
The repository and agent roles are safe to reconfigure. #160 only applies to master. build.ros.org and build.ros2.org have autoreconfigure set for both. Although usually when I make a change to repo hosts I'm too impatient to wait for it to auto run and do a git pull and ./reconfigure myself.