Launchpad itself

PPS kmod signing key generation fails (and fallback also fails)

Asked by Andy Whitcroft

I am trying to publish a test signing package (which utilises a number of the keys). This is failing with a Gateway Time-out trying to generate the keys. It attempts to fall back to the local key material which also fails as this does not have any.

2020-10-20 10:20:18 DEBUG Processing queue for ~apw/ubuntu/signing5 groovy
2020-10-20 10:20:18 DEBUG Processing queue item 24560936
2020-10-20 10:20:18 DEBUG Publishing build to ubuntu/groovy/amd64
2020-10-20 10:20:18 DEBUG ... signing-plain/0.28.1groovy1 (Arch Specific)
2020-10-20 10:20:18 DEBUG Publishing custom signing-plain, signing-plain_0.28.1groovy1_amd64.tar.gz to ubuntu/groovy
2020-10-20 10:20:19 DEBUG feature match flag='archivepublisher.signing_service.enabled' value=u'on' scope='default'
2020-10-20 10:20:19 DEBUG Starting new HTTP connection (1): signing.lp.internal:8000
2020-10-20 10:20:19 DEBUG http://signing.lp.internal:8000 "POST /nonce HTTP/1.1" 201 45
2020-10-20 10:20:19 DEBUG Starting new HTTP connection (1): signing.lp.internal:8000
2020-10-20 10:20:19 DEBUG http://signing.lp.internal:8000 "GET /service-key HTTP/1.1" 200 63
2020-10-20 10:20:19 DEBUG Starting new HTTP connection (1): signing.lp.internal:8000
2020-10-20 10:20:24 DEBUG http://signing.lp.internal:8000 "POST /generate HTTP/1.1" 201 2544
2020-10-20 10:20:24 DEBUG Starting new HTTP connection (1): signing.lp.internal:8000
2020-10-20 10:20:24 DEBUG http://signing.lp.internal:8000 "POST /nonce HTTP/1.1" 201 45
2020-10-20 10:20:25 DEBUG Starting new HTTP connection (1): signing.lp.internal:8000
2020-10-20 10:20:27 DEBUG http://signing.lp.internal:8000 "POST /sign HTTP/1.1" 200 3784
2020-10-20 10:20:27 DEBUG Starting new HTTP connection (1): signing.lp.internal:8000
2020-10-20 10:20:27 DEBUG http://signing.lp.internal:8000 "POST /nonce HTTP/1.1" 201 45
2020-10-20 10:20:28 DEBUG Starting new HTTP connection (1): signing.lp.internal:8000
2020-10-20 10:20:28 DEBUG http://signing.lp.internal:8000 "POST /generate HTTP/1.1" 201 2236
2020-10-20 10:20:28 DEBUG Starting new HTTP connection (1): signing.lp.internal:8000
2020-10-20 10:20:28 DEBUG http://signing.lp.internal:8000 "POST /nonce HTTP/1.1" 201 45
2020-10-20 10:20:28 DEBUG Starting new HTTP connection (1): signing.lp.internal:8000
2020-10-20 10:20:28 DEBUG http://signing.lp.internal:8000 "POST /sign HTTP/1.1" 200 10580
2020-10-20 10:20:28 DEBUG Starting new HTTP connection (1): signing.lp.internal:8000
2020-10-20 10:20:28 DEBUG http://signing.lp.internal:8000 "POST /nonce HTTP/1.1" 201 45
2020-10-20 10:20:28 DEBUG Starting new HTTP connection (1): signing.lp.internal:8000
2020-10-20 10:20:28 DEBUG http://signing.lp.internal:8000 "POST /generate HTTP/1.1" 201 2244
2020-10-20 10:20:28 DEBUG Starting new HTTP connection (1): signing.lp.internal:8000
2020-10-20 10:20:28 DEBUG http://signing.lp.internal:8000 "POST /nonce HTTP/1.1" 201 45
2020-10-20 10:20:28 DEBUG Starting new HTTP connection (1): signing.lp.internal:8000
2020-10-20 10:20:28 DEBUG http://signing.lp.internal:8000 "POST /sign HTTP/1.1" 200 229000
2020-10-20 10:20:28 DEBUG Starting new HTTP connection (1): signing.lp.internal:8000
2020-10-20 10:20:28 DEBUG http://signing.lp.internal:8000 "POST /nonce HTTP/1.1" 201 45
2020-10-20 10:20:28 DEBUG Starting new HTTP connection (1): signing.lp.internal:8000
2020-10-20 10:20:29 DEBUG http://signing.lp.internal:8000 "POST /generate HTTP/1.1" 201 2544
2020-10-20 10:20:29 DEBUG Starting new HTTP connection (1): signing.lp.internal:8000
2020-10-20 10:20:29 DEBUG http://signing.lp.internal:8000 "POST /nonce HTTP/1.1" 201 45
2020-10-20 10:20:29 DEBUG Starting new HTTP connection (1): signing.lp.internal:8000
2020-10-20 10:20:30 DEBUG http://signing.lp.internal:8000 "POST /sign HTTP/1.1" 200 3784
2020-10-20 10:20:30 DEBUG Starting new HTTP connection (1): signing.lp.internal:8000
2020-10-20 10:20:30 DEBUG http://signing.lp.internal:8000 "POST /nonce HTTP/1.1" 201 45
2020-10-20 10:20:30 DEBUG Starting new HTTP connection (1): signing.lp.internal:8000
2020-10-20 10:20:35 DEBUG http://signing.lp.internal:8000 "POST /generate HTTP/1.1" 504 None
2020-10-20 10:20:35 ERROR Error generating signing key for ~apw/ubuntu/signing5: HTTPError 504 Server Error: Gateway Time-out
Traceback (most recent call last):
  File "/srv/launchpad.net/production/launchpad-rev-58bb2f3096f16f0e0acc917602669aecb4ffaf54/lib/lp/archivepublisher/signing.py", line 373, in signUsingSigningService
    key_type, description, self.archive).signing_key
  File "/srv/launchpad.net/production/launchpad-rev-58bb2f3096f16f0e0acc917602669aecb4ffaf54/lib/lp/services/signing/model/signingkey.py", line 230, in generate
    signing_key = SigningKey.generate(key_type, description)
  File "/srv/launchpad.net/production/launchpad-rev-58bb2f3096f16f0e0acc917602669aecb4ffaf54/lib/lp/services/signing/model/signingkey.py", line 101, in generate
    openpgp_key_algorithm=openpgp_key_algorithm, length=length)
  File "/srv/launchpad.net/production/launchpad-rev-58bb2f3096f16f0e0acc917602669aecb4ffaf54/lib/lp/services/signing/proxy.py", line 173, in generate
    data=self._encryptPayload(nonce, payload))
  File "/srv/launchpad.net/production/launchpad-rev-58bb2f3096f16f0e0acc917602669aecb4ffaf54/lib/lp/services/signing/proxy.py", line 96, in _requestJson
    response = urlfetch(url, method=method.lower(), **kwargs)
  File "/srv/launchpad.net/production/launchpad-rev-58bb2f3096f16f0e0acc917602669aecb4ffaf54/lib/lp/services/timeout.py", line 408, in urlfetch
    return URLFetcher().fetch(url, **request_kwargs)
  File "/srv/launchpad.net/production/launchpad-rev-58bb2f3096f16f0e0acc917602669aecb4ffaf54/lib/lp/services/timeout.py", line 246, in call_with_timeout
    reraise(exc_info[0], exc_info[1], tb=exc_info[2])
  File "/srv/launchpad.net/production/launchpad-rev-58bb2f3096f16f0e0acc917602669aecb4ffaf54/lib/lp/services/timeout.py", line 158, in run
    self.result = self.target(*self.args, **self.kwargs)
  File "/srv/launchpad.net/production/launchpad-rev-58bb2f3096f16f0e0acc917602669aecb4ffaf54/lib/lp/services/timeout.py", line 383, in fetch
    raise_for_status_redacted(response)
  File "/srv/launchpad.net/production/launchpad-rev-58bb2f3096f16f0e0acc917602669aecb4ffaf54/lib/lp/services/timeout.py", line 333, in raise_for_status_redacted
    re.sub(r" for url: .*", "", e.args[0]), response=e.response)
HTTPError: 504 Server Error: Gateway Time-out
2020-10-20 10:20:35 INFO OOPS-9f7c4cb4c6e2bea2b2f4936b34d36911
2020-10-20 10:20:35 WARNING Signing service will try to fallback to local key. Reason: SigningServiceError (Could not generate key Kmod: 504 Server Error: Gateway Time-out)
2020-10-20 10:20:35 INFO OOPS-44aed2255fc0bff41e29d91f1a82b0e5
2020-10-20 10:20:35 ERROR Failure processing queue_item 24560936 (OOPS-d96bde739df7b6c63e366cce9b706208)
2020-10-20 10:20:35 INFO OOPS-b7f9d67a126486364088c755cb9a0188

Question information

Language:
English Edit question
Status:
Solved
For:
Launchpad itself Edit question
Assignee:
No assignee Edit question
Solved by:
Andy Whitcroft
Solved:
Last query:
Last reply:
Revision history for this message
Andy Whitcroft (apw) said : 		#1

Ok it appears that it managed to make some of the keys in each publisher run and eventually managed to make them all.