Fetching binary builds of a specific package using the HTTP API

Asked by David Lindsay

Hi!

The page https://launchpad.net/ubuntu/+source/chromium-browser contains a list of dropdowns/expando sections (which are also links that lead to the same information on separate pages) that in turn provide lists of downloadable files for the Chromium package.

What's the most efficient sequence of HTTP requests I can send to the Launchpad API to obtain an equivalent file list? More concretely, given a package name, how can I get the list(s) of downloadable files for that package, for all distributions?

I presume I'll need to send discrete requests to compute what distributions are available and probably execute several other intermediate requests too.

I don't know (or like) python (it reads like line noise to me - no idea why...!) so would have no idea where to start with launchpadlib. If someone can give an idea what sorts of launchpadlib API calls I'd need to execute I could figure out the HTTP requests from there.

---

Explanation/rationale for the curious with ~3 minutes:

It seems that Ubuntu maintains the only 32-bit builds of Chromium with debugging symbols, so I periodically fetch the latest version from Launchpad and fiddle with it to get it running on my Slackware installation. I've been doing this since Google stopped 32-bit builds, and can report that everything actually works with zero shared library/etc hitches! (*Fingers crossed*... I can always fallback to Slackware's debug-less 32-bit Chromium build in an emergency, and I'm seriously looking at Firefox nowadays too, but I digress.)

But. Navigating Launchpad, getting the files downloaded, and then getting the debug symbols into the right place so gcc could find them all took several days to get working last time I updated, so I've been putting updating off for months.

Yesterday I (finally) decided to make the effort to move off Chromium 57, and I figured the best way would be to figure out the Launchpad API so I could script/automate the process and make it easier to update more frequently in future.

Unfortunately, I'm struggling (a LOT) to understand the Launchpad HTTP API. I understand very few people use it directly, but trying to use launchpadlib would just make everything exponentially trickier as I find python's syntax and grammar distracting/surprising/confusing.

My biggest problem is not knowing where to find relevant documentation. I find myself repeatedly landing at https://api.launchpad.net/1.0/ and there is interesting information here, but for example *nothing* in that page explains the relationship between "Custom GET methods" and "ws.op=", for example.

I previously used the https://launchpad.net/~saiarcot895/+archive/ubuntu/chromium-dev PPA but have subsequently realized that https://launchpad.net/ubuntu/+source/chromium-browser also provides debug symbols too so I'm aiming for that this time. Back when I was using the PPA I wrote an updater (as a commandline PHP script) and IIRC I was only able to figure out the calls I needed (including ws.op) by running some sample/unrelated launchpadlib code in a python REPL with Wireshark (!!) running in the background. I was simply not able to find these details documented anywhere.

It seems the Launchpad API doesn't just confound me:

https://answers.launchpad.net/launchpad/+question/272474 was suggested as a related question before I went ahead and wrote this, and one of the answers there references https://help.launchpad.net/API/Hacking, but even that page does not reference ws.op and similar sorts of things either!

Also, a preliminary web search to see if there were any breadcrumbs I could follow led me to https://gist.github.com/NickGeek/8129223, which (ab)uses the +listing-archive-extra XHR call to fetch exactly the sort of information I'm looking for. I find it amusing that script was written in Python but doesn't use launchpadlib.

I wanted the advantage of being able to use the API and have the script not randomly break. I'm very seriously considering just giving up and doing HTML scraping like the script linked immediately above.

Question information

Language:
English Edit question
Status:
Solved
For:
Launchpad itself Edit question
Assignee:
No assignee Edit question
Solved by:
David Lindsay
Solved:
Last query:
Last reply:
Revision history for this message
David Lindsay (asmqb7) said :
#1

Maybe someone with the same problem as me will find this in the future.

For reference:

> https://api.launchpad.net/1.0/ubuntu/+archive/primary?ws.op=getPublishedSources&pocket=Release&source_name=chromium-browser&order_by_date=true&exact_match=true&status=Published

This returns a series of |source_package_publishing_history| items. Grab the number from the end of such an item's |self_link| and then (for example)

> https://api.launchpad.net/1.0/ubuntu/+archive/primary/+sourcepub/8421250?ws.op=binaryFileUrls

Et voilà!

I'm still working on my updater (which I think might turn out to be a shell script this time). I might post it here when it's finished, if I can figure out a way to do so.

So long, and thanks for the textbox! Apologies about the rant.