API issue: API sends back empty data

Asked by Jeroen Bobbeldijk

Sometimes the API just returns an empty body it seems:

URL: https://launchpad.net/api/devel/~strukturag/+archive/ubuntu/libde265
Status Code: 200
Body: b''

Request Headers:
{'Accept': 'application/json'}

Response Headers:
[('Date', 'Fri, 13 Oct 2023 11:39:11 GMT'), ('Server', 'gunicorn'), ('Content-Type', 'application/json'), ('x-powered-by', 'Zope (www.zope.org), Python (www.python.org)'), ('Content-Length', '0'), ('content-security-policy', "frame-ancestors 'self';"), ('ETag', '"8d97a612cce8d1f79bfeb337dcbc9df891a362e9-d436ad11fd487e0f0b3cf2d1063add1f6491f60c"'), ('strict-transport-security', 'max-age=15552000'), ('Vary', 'Accept'), ('x-content-type-options', 'nosniff'), ('x-frame-options', 'SAMEORIGIN'), ('x-launchpad-revision', 'bd6cfd0cfc024dbe1dcd7d5d91165fb4f6a6c596'), ('x-lazr-notifications', '[]'), ('x-xss-protection', '1; mode=block'), ('x-vcs-revision', 'bd6cfd0cfc024dbe1dcd7d5d91165fb4f6a6c596'), ('x-request-id', '8b5e07ab-2f2e-4631-bbd6-4609aa239602'), ('Age', '280'), ('X-Cache', 'HIT from juju-98d295-prod-launchpad-29'), ('X-Cache-Lookup', 'HIT from juju-98d295-prod-launchpad-29:3128'), ('Via', '1.1 juju-98d295-prod-launchpad-29 (squid/4.10)'), ('Connection', 'close')]

===========

OLD description:
When we run the "add-apt-repository -y ppa:strukturag/libheif" command, it often fails for the project it often fails, when looking into it, the API seems to return an odd response quite often on the URL https://launchpad.net/api/1.0/strukturag

"Object: <lp.systemhomes.WebServiceApplication object at 0x7fb159678340>, name: 'strukturag'"

Any idea what's going on?

Question information

Language:
English Edit question
Status:
Answered
For:
Launchpad itself Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
Guruprasad (lgp171188) said :
#1

Hi Jeroen, while we have received reports of issues with add-apt-repository in the past 24 hours or so, the API URL for the team is https://api.launchpad.net/1.0/~strukturag (notice the ~ character before the team name). Can you elaborate where you saw this incorrect URL?

Revision history for this message
Jeroen Bobbeldijk (jerbob92) said :
#2

Hi! I did not see the incorrect URL in the logs since it was not there, the error that I did have was:

```
Traceback (most recent call last):
  File "/usr/bin/add-apt-repository", line 137, in <module>
    shortcut = shortcut_handler(line)
  File "/usr/lib/python3/dist-packages/softwareproperties/SoftwareProperties.py", line 885, in shortcut_handler
    ret = factory(shortcut)
  File "/usr/lib/python3/dist-packages/softwareproperties/ppa.py", line 469, in shortcut_handler
    return PPAShortcutHandler(shortcut)
  File "/usr/lib/python3/dist-packages/softwareproperties/ppa.py", line 426, in __init__
    info = get_ppa_info(self.shortcut)
  File "/usr/lib/python3/dist-packages/softwareproperties/ppa.py", line 392, in get_ppa_info
    _get_suggested_ppa_message(user, ppa))
  File "/usr/lib/python3/dist-packages/softwareproperties/ppa.py", line 349, in _get_suggested_ppa_message
    lp_user = get_info_from_lp(LAUNCHPAD_USER_API % user)
  File "/usr/lib/python3/dist-packages/softwareproperties/ppa.py", line 104, in get_info_from_lp
    return get_info_from_https(lp_url, True)
  File "/usr/lib/python3/dist-packages/softwareproperties/ppa.py", line 98, in get_info_from_https
    return json.loads(data)
  File "/usr/lib/python3.8/json/__init__.py", line 357, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.8/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.8/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
```

So I looked up the line, checked what it was doing, reconstructed the URL and then tried to go there with the browser, that's how I ended up with that URL, when I visited the URL I saw a response that would go with the JSON error, so I assumed I had the URL correct.

I will do some more tests to see what the URL is that it actually makes.

Revision history for this message
Jeroen Bobbeldijk (jerbob92) said :
#3

Yep, you're right, it looks like this was just a red herring, it does indeed correctly build that URL (https://launchpad.net/api/1.0/~strukturag), it looks like this exception swallows the actual error from "get_ppa_info".

I can only reproduce this in our CI environment, so I'll try to figure out how to debug that.

Revision history for this message
Jeroen Bobbeldijk (jerbob92) said :
#4

Ok so sometimes the API just returns an empty body it seems:

URL: https://launchpad.net/api/devel/~strukturag/+archive/ubuntu/libde265
Status Code: 200
Body: b''

Request Headers:
{'Accept': 'application/json'}

Response Headers:
[('Date', 'Fri, 13 Oct 2023 11:39:11 GMT'), ('Server', 'gunicorn'), ('Content-Type', 'application/json'), ('x-powered-by', 'Zope (www.zope.org), Python (www.python.org)'), ('Content-Length', '0'), ('content-security-policy', "frame-ancestors 'self';"), ('ETag', '"8d97a612cce8d1f79bfeb337dcbc9df891a362e9-d436ad11fd487e0f0b3cf2d1063add1f6491f60c"'), ('strict-transport-security', 'max-age=15552000'), ('Vary', 'Accept'), ('x-content-type-options', 'nosniff'), ('x-frame-options', 'SAMEORIGIN'), ('x-launchpad-revision', 'bd6cfd0cfc024dbe1dcd7d5d91165fb4f6a6c596'), ('x-lazr-notifications', '[]'), ('x-xss-protection', '1; mode=block'), ('x-vcs-revision', 'bd6cfd0cfc024dbe1dcd7d5d91165fb4f6a6c596'), ('x-request-id', '8b5e07ab-2f2e-4631-bbd6-4609aa239602'), ('Age', '280'), ('X-Cache', 'HIT from juju-98d295-prod-launchpad-29'), ('X-Cache-Lookup', 'HIT from juju-98d295-prod-launchpad-29:3128'), ('Via', '1.1 juju-98d295-prod-launchpad-29 (squid/4.10)'), ('Connection', 'close')]

Revision history for this message
Ross Vandegrift (ross-kallisti) said :
#5

I've also seen errors from apt-add-repository, starting about a week ago. It's particularly bad in Github Actions. Last Thursday it was reproducible from my laptop, but I didn't think to collect details. I will try to do so today.

Revision history for this message
Ross Vandegrift (ross-kallisti) said :
#6

So far, I cannot reproduce this locally - only from Github Actions. Maybe a new source-based rate limit has been deployed?

For the past few hours, my laptop has been running a loop that does `apt-add-repository` in a fresh ubuntu 22.04 container. So far, I've seen zero failures.

But during that same time, team members have seen multiple failures of the exact same thing in Github Actions.

Revision history for this message
Colin Watson (cjwatson) said :
#7

See bug 2039251, where I've been debugging this.

Revision history for this message
Colin Watson (cjwatson) said :
#8

This should be fixed now.

Can you help with this problem?

Provide an answer of your own, or ask Jeroen Bobbeldijk for more information if necessary.

To post a message you must log in.