Failed to fetch from local buildfarm repo

asked 2016-03-07 11:48:32 -0500

8729 ●137 ●239 ●256 https://github.com/lucasw

updated 2016-03-07 12:23:06 -0500

I'm at the rosdistro cache step of deploy_configuration.rst

import_upstream was successful, but other jenkins jobs try to interact with my local buildfarm repo (ros-build-farm-repo) and fail:

 ...
13:46:06 Ign http://ros-build-farm-repo trusty InRelease
13:46:06 Ign http://ros-build-farm-repo trusty Release.gpg
13:46:06 Ign http://ros-build-farm-repo trusty Release
13:46:06 Err http://ros-build-farm-repo trusty/main amd64 Packages
13:46:06   
13:46:06 Err http://ros-build-farm-repo trusty/main amd64 Packages
13:46:06   
13:46:06 Err http://ros-build-farm-repo trusty/main amd64 Packages
13:46:06   
13:46:06 Err http://ros-build-farm-repo trusty/main amd64 Packages
13:46:06   
13:46:06 Err http://ros-build-farm-repo trusty/main amd64 Packages
13:46:06   404  Not Found
...
13:46:19 W: Failed to fetch http://ros-build-farm-repo/ubuntu/building/dists/trusty/main/binary-amd64/Packages  404  Not Found

http://ros-build-farm-repo/ubuntu/bui... exists but nothing is inside it both as viewed through a browser and sshed in and looking in ros-build-farm-repo:/var/repos/ubuntu/building/dists/trusty/main/binary-amd64. building/pool/main/r has 900 GB of ros-jade deb/dsc/tar.gz files but the adjacent directories are empty.

import_upstream import_jade.yaml console output ends like this:

10:21:02 Updating for trusty source to update into repo /var/repos/ubuntu/main
10:21:02 I have a lock on /var/repos/ubuntu/main/lock
10:21:02 Creating updates file /var/repos/ubuntu/main/conf/updates
10:21:02   Name: backfill-ros
10:21:02   Method: http://packages.ros.org/ros/ubuntu/
10:21:02   Suite: trusty
10:21:02   Components: main
10:21:02   Architectures: source
10:21:02   FilterFormula: Package (% ros-jade-* )
10:21:02   
10:21:02 Creating distributions file /var/repos/ubuntu/main/conf/distributions
10:21:02 running command ['reprepro', '-v', '-b', '/var/repos/ubuntu/main', '--noskipold', 'update', 'trusty']
10:21:02 Warning: No VerifyRelease line in 'backfill-ros' or any rule it includes via 'From:'.
10:21:02 Release.gpg cannot be checked unless you tell which key to check with.
10:21:02 (To avoid this warning and not check signatures add 'VerifyRelease: blindtrust').
10:21:02 aptmethod got 'http://packages.ros.org/ros/ubuntu/dists/trusty/InRelease'
10:21:02 Calculating packages to get...
10:21:02 + echo # END SUBSECTION
10:21:02 # END SUBSECTION
10:21:02 + echo # END SECTION
10:21:02 # END SECTION
10:21:02 Finished: SUCCESS

It didn't update much on this run because it already got all the files on a previous execution.

edit retag flag offensive close merge delete

add a comment

1

answered 2016-03-07 17:21:52 -0500

lucasw

8729 ●137 ●239 ●256 https://github.com/lucasw

updated 2016-03-07 18:21:29 -0500

The answer turned out to be similar to http://answers.ros.org/question/22816... , though it wasn't enough to simply clean out jenkins-slave on the repo machine, I ended up starting over with a clean Ubuntu 14.04 server.

Now I have a Packages file available on my (intranet) http://ros-build-farm-repo/ubuntu/bui...

The gpg/pgp key I generated earlier gave me a problem:

gpgme gave error GPGME:11:  Bad passphrase
14:36:44 ERROR: Could not finish exporting 'trusty'!
14:36:44 This means that from outside your repository will still look...

I went back to the default key pair from the upstream repo for repo/common.yaml, which is not viable for external use but is fine for now. (Was I supposed to generate a key pair with no pass phrase? Answer is yes as stated below. 'You can extract the repository PGP key from the buildfarm_deployment configuration. ' https://github.com/ros-infrastructure... )

I initially forgot to also update the corresponding keys in ros_buildfarm_config, which resulted in the following error from the rosdistro cache job:

15:51:26 W: GPG error: http://ros-build-farm-repo trusty InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY ...

Making the keys consistent clears that error up. (What about rosdistro/repos.key?)

edit flag offensive delete link

Comments

1

Yes, the key needs to be useable without a pass phrase. Otherwise the automated scripts cannot sign with it. (You might be able to setup a keyring to unlock the key in some way, but either way it's useable w/o a pass phrase.)

tfoote ( 2016-03-07 17:42:29 -0500 )edit

1

The keys in the rosdistro need to match the signing key for the repo, otherwise apt-get will complain about unsigned keys when downloading on the buildfarm. And users need to have that key in their apt keyring to download as well.

tfoote ( 2016-03-07 18:57:59 -0500 )edit

add a comment

0

answered 2016-03-07 12:11:09 -0500

tfoote

58457 ●128 ●550 ●526 http://www.ros.org/

I suspect that your building_repository slave is still offline.

From this question: http://answers.ros.org/question/22816...

I then see a message pending--There are no nodes with the label 'building_repository'

Those jobs are what setup the repository. If no job has ever been run on that machine the repositories will not be properly configured. (Some minimal level of directory structure will exist but the jobs create the indexes like the one you're getting a 404 for. )

edit flag offensive delete link

Comments

I did get build_repository online, and some directories exist (just updated above), and import_upstream reported success- but maybe didn't actually fully finish?

lucasw ( 2016-03-07 12:17:05 -0500 )edit

I didn't run ros_bootstrap.yaml, just import_jade.yaml- it looks like it is going to take a while but maybe that was my issue.

lucasw ( 2016-03-07 12:29:49 -0500 )edit

The updated console output looks right for a job that's already run. Any time reprepro is run it should generate the Packages file for the repo as it's index. Can you fetch the Packages file from the url listed as 404?

tfoote ( 2016-03-07 12:46:48 -0500 )edit

No the ubuntu/building/dists/trusty/main/binary-amd64/ folder is empty. I finished running ros_bootstrap.yaml (edited to only do trusty amd64), and ran import_jade.yaml again- both showed success (just the 'Release.gpg cannot be checked unless you tell which key to check with' warning)

lucasw ( 2016-03-07 14:17:44 -0500 )edit

I'm going to clean out the repo jenkins-slave user again and start again from a couple steps back.

lucasw ( 2016-03-07 14:18:49 -0500 )edit

1

I'd suggest trying to use reprepro directly in the folder and try running export on the repo. That's what generates the Package file and see if you can get any more specific debugging.

tfoote ( 2016-03-07 14:41:22 -0500 )edit

add a comment

0

answered 2017-06-13 18:20:26 -0500

jgoppert
41 ●2 ●5 ●4

I setup a straight clone of the buildfarm and just changed the servers to my ec2 instances. This almost works, but I'm running into this 404 issue. I have verified that it isn't an issue with the ssh keys by looking at the auth logs. Also syslog/puppet.log don't show anything useful.

Am I missing a step? Is it necessary that I do something the gpg or ssh keys? Do I need to run prereleash manually to populate the repos before jenkins will work properly?

edit flag offensive delete link

add a comment

Failed to fetch from local buildfarm repo

3 Answers

Comments

Comments

Question Tools

Stats

Related questions

Failed to fetch from local buildfarm repo edit

3 Answers

Comments

Comments

Question Tools

Stats

Related questions

Failed to fetch from local buildfarm repo