ros buildfarm for internal projects only

buildfarm

asked 2016-03-02 11:58:18 -0500

8729 ●137 ●239 ●256 https://github.com/lucasw

updated 2016-03-02 17:57:43 -0500

I've created three virtual machines for master/repo/slave following http://wiki.ros.org/buildfarm and https://github.com/ros-infrastructure... . My goal is to build a set of packages from my intranet gitlab server. I don't want to build any official ros packages, just use the release versions of already built ones. Is that possible?

http://roscon.ros.org/2015/presentati... indicates it is or intended to be possible soon.

Do I need to fork rosdistro? Yes. And follow https://groups.google.com/forum/#!top... if so?

Do I run a customized bloom internally to release packages? No, it looks like I set ROSDISTRO_INDEX_URL to my internal rosdistro location.

It would be fantastic to have a set of github customized buildfarm config and rosdistro repos that are a complete working example of building 2 or 3 example 'private' packages that are also in github, and are dependent on public ros packages. It then could easily be replicated locally (add plenty of warnings in yaml comments to change the keys and passwords, but even with the default ones it would be safe to set up an experimental system behind a firewall). Maybe the minimum thing that would have to be changed is notification emails, and the user would be responsible for making sure the default server names resolve to their local master/repo/slave machine IPs.

(I can work on this but need to be able to get to that working system first)

Some additional more minor questions and comments

I'll probably answer for myself soon (or that https://github.com/ros-infrastructure... files already answer), but any help is welcome:

I've now forked ros_buildfarm_config onto my gitlab server, and I'm editing index.yaml. It seems obvious to edit notification_emails.

Is the PGP key the same as the one created for the buildfarm deployment provisioning stage? Yes, the key is the same in the ros_buildfarm_config/index.yaml and the buildfarm_deployment_config/repo/common.yaml.

Is upload_credential_id the same as credentials::jenkins-slave::id?

Should the status_page_repositories map to my repo machine from the buildfarm deployment provisioning stage? Yes 54.183.54.232 appears in nearly every ros_buildfarm_config yaml, so replace it with my repo ip/name.

Change jenkins_url from 54.183.26.131 to my jenkins master ip/name?

Comment out arm: if I don't want to build for the arm architecture? No

https://github.com/yujinrobot/rosdistro could be valuable resource, but they are using buildbot and not the jenkins buildfarm system?

How about generating all these config files from a script, and only provide emails and keys and so forth once? (though it might sacrifice flexibility, maybe there could be a single best practice configuration that this assumes) It looks like the debian_repository_key is duplicated 10 times!

edit retag flag offensive close merge delete

add a comment

answered 2016-03-02 18:09:58 -0500

tfoote

58457 ●128 ●550 ●526 http://www.ros.org/

For how to setup the custom rosdistro that's the right discussion, there's a little bit of an exercise left to the user.

Yes the upload_credential_id needs to be the same as credentials::jenkins-slave::id though it's fine to use the default.

You should change the jenkins_url should be set to a value that resolves to your master instance.

If you don't want to build for arm you can comment out or delete the entry in the index.yaml for the arm buildfile. We used it as an example of building for two different sets of architectures with different settings.

Automating generating these config files is definitely doable. A wizard/prompt to ask all the appropriate questions would probably be the best. However, the number of times you're expected to do this is relatively low which makes the value of automating it much lower. The repeated values for things like the repository key are there because the system is designed such that you do not need to upload all packages to the same repository for different build files. We could add complexity to abstract away the configuration but since the configuration is quite static managing the parallel config does not cost much in the long run and keeping the files self contained and easy to read has value.

Comments

Thanks. I've gotten to the point where I have a generate_all_job.py creating a jenkins page full of jobs, but they mostly aren't working. I'll probably open a new question focused on rosdistro configuration.