Launchpad itself

Repeated Chromium build cancellations

Asked by Rob Savoury

Hi, for almost two days now I have been seeing the latest Chromium builds I have uploaded (to ppa:savoury1/savos) being repeatedly cancelled. There are no build logs, the builds are simply suddenly ended and "Failed to build" is stated on the build page, but with no further information available.

This issue with the Chromium builds is something I have repeatedly seen in recent months, however this time round is by far the worst example of it so far. At this stage I have probably restarted the builds about 20 times already, yet not once have they been allowed to complete.

While I understand that there are no service guarantees with Launchpad, in terms of no guarantee given for when any uploaded build might be completed, this is becoming very time wasting and frustrating for me. It would be helpful if someone on the Launchpad team could please explain the algorithm or process that is being used to cancel these Chromium builds of mine. As it is certainly not at all clear to me why the Chromium builds that I upload seem to be getting cancelled so often.

Any information about this issue would be appreciated.

Thank you in advance for your assistance.

Question information

Language:
English Edit question
Status:
Solved
For:
Launchpad itself Edit question
Assignee:
No assignee Edit question
Solved by:
Colin Watson
Solved:
Last query:
Last reply:
Revision history for this message
Manfred Hampl (m-hampl) said : 		#1

Maybe the same problem as https://answers.launchpad.net/launchpad/+question/708252

Revision history for this message
Best Colin Watson (cjwatson) said : 		#2

Thanks for the report. It looks like we missed an `ANALYZE` step when creating a couple of test rebuilds of Ubuntu, which meant that all PostgreSQL's statistics were way off and so lots of buildd-manager's performance-critical queries were running much more slowly than normal. We've rectified this and things seem happier now; I've retried your builds and they're building.

Revision history for this message
Colin Watson (cjwatson) said : 		#3

(A build failing without a build log always means that something went wrong at an infrastructure level, by the way: either the builder crashing so hard that buildd-manager was unable to retrieve logs for it, or some kind of timeout in buildd-manager that was bad enough that it had to give up on the build. Cancellation is a separate kind of thing and not what was happening here.)

Revision history for this message
Rob Savoury (savoury1) said : 		#4

Hi Colin, thanks for the background information about the issue. When I looked at the build farm status page (https://launchpad.net/builders) over the weekend I did see what looked like large test rebuilds of Ubuntu taking place, and I wondered if that extra load on the build infrastructure might be some part of the issue with the Chromium builds I uploaded Saturday. Thanks for restarting the Chromium builds, one seemed to have failed again but I've restarted it just now and it's proceeding. Also, I understand from your additional comment that when there is no build log it was an infrastructure issue behind the build failure, rather than any specific cancellation as such. Appreciate the assistance and extra details about the situation. Thanks, Rob

Revision history for this message
Rico Tzschichholz (ricotz) said : 		#5

Hello,

I am currently seeing this problem with numerous builds of firefox, thunderbird and libreoffice.

Especially the missing riscv64 build of libreoffice in jammy-backport is a problem.
https://launchpad.net/ubuntu/+source/libreoffice/4:7.5.7-0ubuntu0.23.04.1~bpo22.04.1/+build/26821168

Regards,
Rico