rosdistro add-on fork

asked 2016-03-03 17:00:28 -0500

lucasw

8729 ●137 ●239 ●256 https://github.com/lucasw

updated 2016-03-03 21:57:46 -0500

I'm following custom_rosdistro.rst#add-on-fork and this ros-sig-buildfarm thread about using the custom rosdistros

list that additional distribution file after the "normal" one in the index.yaml

in rosdistro/index.yaml is this adding a second jade: entry like:

...
 indigo:
    distribution: [indigo/distribution.yaml]
    distribution_cache: http://repositories.ros.org/rosdistro_cache/indigo-cache.yaml.gz
  jade:
    distribution: [jade/distribution.yaml]
    distribution_cache: http://repositories.ros.org/rosdistro_cache/jade-cache.yaml.gz
  jade:
    distribution: [jade/addon_distribution.yaml]
    distribution_cache: http://ros-build-farm-repo/rosdistro_cache/jade-cache.yaml.gz
type: index
version: 3

Or add duplicate entries for all of the distributions? Or don't have the extra 'jade:', just add another distribution and distribution_cache directly under the other two?

To my addon_distribution.yaml I added:

tags: 
- custom

Do I need to refer to this anywhere else? Will the tag become prominent in jenkins?

I created an import_jade.yaml like suggested and put it into my forked buildfarm_deployment_config:

jenkins-slave::reprepro_config:
    '/home/jenkins-slave/reprepro_config/ros_bootstrap.yaml':
        ensure: 'present'
        content: |
            name: ros_bootstrap
            ...
            verify_release: blindtrust
    '/home/jenkins-slave/reprepro_config/import_jade.yaml':
        ensure: 'present'
        content: |
            name: backfill-ros
            method: http://packages.ros.org/ros/ubuntu/
            suites: [trusty]
            component: main
            architectures: [amd64, source]
            filter_formula: Package (% ros-jade-* )

Some sources suggested it should go into /home/jenkins-slave/workspace, but I think reprepro_config is easier to get to jenkins when setting the job parameter.

If you want to maintain and use your custom version of the rosdep database included in the forked rosdistro repository you have to perform the following changes:

I don't want more maintenance, but do I have to in order to do the add-on fork? How do my custom packages get into the rosdep database if I don't modify it? Otherwise what is the basis for making this decision beyond the additional work/further divergence from upstream files?

After forking the repository you must update the URLs to your custom locations for each distribution:

Is this just for my one addon_distribution entry like shown above, or every distribution_cache in that list needs an updated url to point at my local repo machine?

Every user must set an environment variable on the machine to use the custom rosdistro database:

Does this matter within the virtual environment on my repo machine, the master or the slave? Or just users elsewhere?

Note if you're overlaying you need to add a rodep file for the underlayed rosdistro if using one: https://github.com/mikepurvis/rosdep-...

Is overlaying different than add-on forking? Does it interact with the choice above to have a custom rosdep database?

So far after going through the above steps with the above assumptions I can generate_all_jobs.py on my custom ros_buildfarm_config index.yaml, and it completes without error. The next thing to do is import_upstream, which is now in jenkins, so I went there, clicked 'Build with Parameters' and edit the config_file text entry box to use import_jade.xml (Which does exist). I then see a message pending--There are no nodes with the label 'building_repository'

Many of the other jenkins jobs are getting triggered ... (more)

edit retag flag offensive close merge delete

add a comment

2 Answers

Sort by » oldest newest most voted

answered 2016-03-04 14:31:10 -0500

lucasw

8729 ●137 ●239 ●256 https://github.com/lucasw

updated 2016-03-07 16:32:53 -0500

pending--There are no nodes with the label 'building_repository'

This was the result of a misconfigured repo/common.yaml (one of the ssh keys wasn't consistent with the master/common.yaml jenkins::private_ssh_key).

Upon trying to run reconfigure.bash repo a couple of times it looked like the change wasn't working. I ended up doing this on the repo machine:

rm -rf /var/log/puppet.log  # Just to get a clean log, not necessary
sudo deluser jenkins-slave
sudo rm -rf /home/jenkins-slave
sudo su
./install_prerequisites.bash
./reconfigure.bash repo

And now I have a 'building_repository' build executor on my master jenkins server. I also reran reconfigure on the master and slave several times, so I'm not 100% sure cleaning out jenkins-slave on the repo was the entire solution.

Update

I encountered some issues where simply deleting jenkins-slave did not do enough- trying to run reconfigure did not recreate the jenkins-slave home directory. I ended up reinstalling ubuntu 14.04 server on the same repo vm instance and starting from scratch (and cloned the vm prior to making changes to save a little time if this step needed again).

edit flag offensive delete link

add a comment

answered 2016-03-04 13:28:24 -0500

tfoote

58457 ●128 ●550 ●526 http://www.ros.org/

updated 2016-03-04 13:45:09 -0500

For the first quiestion about index.yaml, you cannot have duplicate entries in the dict. You would want to have myproject-jade or something like that to differentiate your packages from the mainline ones. Note that to extend jade you should add to the list of distribution files not add a new entry.

For the reprepro_config's your location is fine. They just need to be consistent enough for you

You don't have to use a custom rosdep database. That's only necessary if you want to be able to support injecting your own non-ros packages as dependencies.

The custom rosdistro paths should be dealt with on the buildfarm from the config files. That should only require customization for the end users/ non-buildfarm machines.

The repo machine should be running and connected to the jenkins instance as a slave named building_repository so that it can execute all the jobs. You should review the deployment docs to make sure you have all 3 types of machines deployed correctly: https://github.com/ros-infrastructure...

We haven't fully defined the overlay term. docs for addon and rosdep rules can be found here: https://github.com/ros-infrastructure...

Note creating a separable questions is usually more effective on this forum. (If there are related questions cross linking can help give context, but try to keep them self contained) I've tried to answer as much as I can but iterating it back and forth between a list of questions and a list of answers can be hard to follow when reading later.

edit flag offensive delete link

Comments

I've been going through the ros-sig-buildfarm messages and it looks like that is better suited to multiple question dumps because it supports inline answering (and is more likely to reach a focused audience of already knowledgeable people instead of the subset also monitoring questions here).

lucasw ( 2016-03-04 14:22:16 -0500 )edit

distribution: [jade/distribution.yaml, jade/addon_distribution.yaml]distribution_cache: http://ros-build-farm-repo/rosdistro_cache/jade-cache.yaml.gz (overwrite the previous distribution_cache entirely since it is not a list)

lucasw ( 2016-03-04 14:25:19 -0500 )edit

add a comment

rosdistro add-on fork

2 Answers

Update

Comments

Question Tools

Stats

Related questions

rosdistro add-on fork edit

2 Answers

Update

Comments

Question Tools

Stats

Related questions

rosdistro add-on fork