Need to acquire a list (or json) of all packages for a particular Ubuntu distro

Asked by Christina Haig on 2019-02-05

I am looking for some help (an example would be excellent) figuring out how to just acquire a list of package names for a certain version of Ubuntu. For example, using the web interface I could simply search in the Ubuntu section of Launchpad for a package, and it will give me all of the sub-packages (components?), e.g.: https://launchpad.net/ubuntu/+source/linux-meta

What I am looking for is a list of upstream packages + all of their sub-packages and all of their dependencies. I'm currently just interested in Ubuntu-17.10-desktop, but I do have an eye toward automation in the future.

Unfortunately the list of provided examples is pretty sparse, so I'm having trouble understanding which function to use.

Please launchpad, help me with my data science project. :-)

Question information

Language:
English Edit question
Status:
Expired
For:
Ubuntu python-launchpadlib Edit question
Assignee:
No assignee Edit question
Last query:
2019-02-06
Last reply:
2019-02-22

you can see the installed packages and versions using:

dpkg -l

This can obviously be grepped to be useful

You can see available versions using:

apt-cache search packagename

This also shows the installed version and the available version in the package sources

Christina Haig (chaig) said : #2

I'm afraid you may have misunderstood the question. This was posted to the Ubuntu python-launchpadlib forum because I'm looking for an example of Python code calling launchpadlib that yields either a Python list of package names, or just a json that I can access the package names out of. I don't actually have the Ubuntu image installed, I simply need a full list of packages for Artful (or any generic installation really) including source packages. This is for a data science project I am working on.

Any launchpadlib users out there?

Manfred Hampl (m-hampl) said : #3

Warning: Ubuntu 17.10 (Artful) is already end of life, and most of the contents related to that release have already been removed from Launchpad.

Christina Haig (chaig) said : #4

Here is some sample code that I hope is moving me in the right direction:

import pandas as pd
from launchpadlib.launchpad import Launchpad
import launchpadlib as lp

launchpad = Launchpad.login_anonymously('just testing', 'production', cachedir, version='devel')
ubuntu = launchpad.distributions['ubuntu']
series = ubuntu.getSeries(name_or_version='17.10')
archive = ubuntu.main_archive
arch_series = series.getDistroArchSeries(archtag='amd64')

manifest = pd.DataFrame(columns=['asset','pkg_set'])
pkgs = launchpad.packagesets

for i in range(34):
    name = pkgs.getBySeries(distroseries=series)[i].name
    sources_incl = pkgs.getBySeries(distroseries=series)[i].getSourcesIncluded()
    new_man=pd.DataFrame({'asset':sources_incl,'pkg_set':[name]*len(sources_incl)})
    manifest = manifest.append(new_man,ignore_index=True)
manifest=manifest.sort_values(by=['asset'])

It took me awhile to get here, so hopefully I'm on the right track. Ideally I should be able to modify this script slightly to change to other Ubuntu series, particularly newer versions. However, I am a novice in the field of API JSON fetching, so I could use some help.

For instance, knowing that there are 34 package sets in Artful I came to through experimentation. It would be nice if there were some property I could fetch to know that answer for a given series.

In addition, I would like to be able to fetch version numbers for each source, specific to Artful, but I can't seem to figure out how to do that except through .getPublishedSources, i.e.:

name = 'acpid'
comp = archive.getPublishedSources(source_name=name,distro_series=series)[0].component_name
source = archive.getPublishedSources(source_name=name,distro_series=series)[0].source_package_name
version = archive.getPublishedSources(source_name=name,distro_series=series)[0].source_package_version

It does not seem to be allowed to fetch the entirety of the database, instead you must specify a source_name.

Anyone out there, or am I throwing my questions into the void?

Launchpad Janitor (janitor) said : #5

This question was expired because it remained in the 'Open' state without activity for the last 15 days.