snap build failures on riscv64

Asked by Stéphane Graber

We've recently enabled riscv64 building for the latest/edge snap on LXD but so far we're yet to see a single build manage to pull all our build artifacts from Github.

snap is at https://code.launchpad.net/~ubuntu-lxc/+snap/lxd-latest-edge

Most recent failures are:
https://launchpadlibrarian.net/569777968/buildlog_snap_ubuntu_focal_riscv64_lxd-latest-edge_BUILDING.txt.gz
https://launchpadlibrarian.net/569637349/buildlog_snap_ubuntu_focal_riscv64_lxd-latest-edge_BUILDING.txt.gz

Both are failing to clone https://github.com/tianocore/edk2 from
Github. I believe that's the same we've seen in every failure so far
(maybe 10 or so builds were attempted).

Question information

Language:
English Edit question
Status:
Open
For:
Launchpad itself Edit question
Assignee:
Colin Watson Edit question
Last query:
Last reply:
Revision history for this message
Stéphane Graber (stgraber) said :
#2

This is completely unrelated, this issue is about a build infrastructure issue and has nothing to do with code hosting.

The ticket was open at the direct request of the Launchpad team so they can track this internally.

Revision history for this message
M.D. Walters (mdwalters124) said :
#3

Can you add some more information on this? Also, did you make the version control system from Bazzar to Git?

Revision history for this message
Heinrich Schuchardt (xypron) said :
#4

@mdwalters124:

The appended complete buildlog indicates that the build machine was riscv64-qemu-lcy01-014.
A command to clone an external git repository failed because the connection was interrupted.

     git clone https://github.com/tianocore/edk2 . -b edk2-stable202108

As the git server is easily reachable on a developer machine: Why is this happening repeatedly on our build farm?

Revision history for this message
Heinrich Schuchardt (xypron) said (last edit ):
#5

To debug the GNUTLS activity during the clone operation the following environment variables can be used:

export GIT_TRACE_PACKET=1
export GIT_TRACE=1
export GIT_CURL_VERBOSE=1

Reportedly the same problem was observed outside Canonical with inappropriate settings of http.postBuffer which defaults to 1 MiB:

git config --global http.postBuffer 524288000

To reduce traffic --depth=1 could be added to the git clone statement.

Revision history for this message
M.D. Walters (mdwalters124) said (last edit ):
#6

Hi, I think I have the answer:
1. You may have used the wrong URL. What you where trying to direct to Git, was the repository website, which is the link to direct to web browsers. On your GitHub repository you may have not looked at the greenn code button. It will show you the correct link, which should end with the file extension *.git, which, Git can read.
2. Maybe your internet connection dropped... Try cloning another repo, if that works, try adding in the *.git extension.
And besides, the url you entered, says "Not found". I think it wasn't your internet connection, but rather the file you tried to direct to Git. I'd suggest you use this URL: https://github.com/tianocore/edk2.git.

Revision history for this message
M.D. Walters (mdwalters124) said :
#7

Also, I looked up what riscv64 is, and it appears to be a port of OpenBSD for RISC-V systems. What version of riscv64 are you using?

Revision history for this message
Dimitri John Ledkov (xnox) said :
#8

@mdwalters124 https://launchpad.net/ubuntu/focal/riscv64 the Ubuntu port obviously.

Revision history for this message
Dimitri John Ledkov (xnox) said :
#9

@mdwalters127 this request is for Launchpad Admins to process. Are you part of the Launchpad Admin team?

Revision history for this message
Dimitri John Ledkov (xnox) said (last edit ):
#10

I have attempted to skip cloning edk2 to get the build further

https://launchpad.net/~xnox/+snap/any-riscv64/+build/1591895

and I managed to get it to effectively time out after 3.25 hours of building:

[23/Nov/2021:21:43:10 +0000] "GET http://ftpmaster.internal/ubuntu/pool/main/n/nano/nano_4.8-1ubuntu1_riscv64.deb HTTP/1.1" 407 2146 "-" "Debian APT-HTTP/1.3 (2.0.6)"

Err nano_4.8-1ubuntu1_riscv64.deb

  407 Proxy Authentication Required [IP: 10.10.10.1 8222]

Given how long build times are on riscv64, is it possible to increase the timeout on the proxy authentication tokens for riscv64 builders?

Revision history for this message
Stéphane Graber (stgraber) said :
#11

Ah yeah, the proxy auth token timing out would make sense. I guess if it was to hit halfway through downloading git artifacts from Github it may cause what we're seeing too. It would indicate pretty damn near identical timing though for it to hit always at the same spot ;)

Can you help with this problem?

Provide an answer of your own, or ask Stéphane Graber for more information if necessary.

To post a message you must log in.