Upgrade from 5.4.0 to 5.8.0-36 or -38 failed to install linux-modules-extra-$(uname -r) and resulted in loss of sound + internet connectivity after restart.

Asked by scitoast on 2021-01-15

Hi there,

I'm not sure how to report this bug as it is an upgrade issue and it's not clear to me what behavior is attributable to what package!

Upgrade from focal 5.4.0 to 5.8.0-36 and from 5.4.0 to 5.8.0-38 are affected. I've verified it (and had to fix the problem) on two independent and very different desktop systems, one running Kubuntu focal and the other running Lubuntu focal.

- After upgrading, AFAICT, any hardware that requires modules in the linux-modules-extra package will be nonfunctional (because that package wasn't installed.) This particularly includes, as far as I can tell, many realtek peripherals including wifi, ethernet and sound. This results in loss of internet connectivity after restarting subsequent to upgrade, making the trouble hard to fix! A nasty little problem.

Affected devices:

- In my experience, devices requiring kernel modules included in package "linux-modules-extra-$(uname -r)". This package simply won't be installed during the upgrade process.

Symptoms:

- Devices simply don't have kernel modules available and won't work.

- Affects all my realtek devices and any other hardware that uses linux-modules-extra

- lshw will of course show the devices as "UNCLAIMED".

Cause:

- As indicated already -- AFAICT the Ubuntu upgrade tool does not automatically detect and select linux-modules-extra-$(uname -r)-generic for installation during the upgradeprocess. This is true even if the system has hardware requiring these modules and even if this package is installed / used in the system prior to upgrade.

Temporary Fix:

- Prior to upgrade, determine which kernel you're upgrading to, and manually download the .deb files manually prior to the upgrade so you can add them manually after rebooting.

e.g.

sudo apt-get clean

sudo apt-get --download-only install linux-modules-extra-5.8.0-38-
generic

Then copy the .deb files from /var/cache/apt/archives just in case
they might get deleted during the upgrade process.

After restart, realtek wifi, ethernet and sound etc. won't work. Manually install the linux-modules-extra .deb file; the affected hardware will be functional after another restart (simply because the kernel modules will now be present on the system!)

Question information

Language:
English Edit question
Status:
Solved
For:
Ubuntu Edit question
Assignee:
No assignee Edit question
Solved by:
Manfred Hampl
Solved:
2021-01-19
Last query:
2021-01-19
Last reply:
2021-01-19
Manfred Hampl (m-hampl) said : #1

I assume that such problem is caused by errors in the meta-packages for the kernel.

Questions to be answered:

What action has caused the upgrade the kernel packages from 5.4.0-* to 5.8.0-*
Which kernel meta-packages were installed before the upgrade (and which of them are installed now)?

What is the output of the command

cat /var/log/apt/history.log | grep -E 'Command|hwe|5.8.0'
zcat /var/log/apt/history.log.1.gz | grep -E 'Command|hwe|5.8.0'

scitoast (scitoast) said : #2

Hi Manfred,

- The action which caused the upgrade was the auto-update feature in each case; e.g. the "Updates available" notifier.

- Kernel meta-packages that were installed are the same as the ones installed now; I see the following listed by apt in both cases:

linux-generic
linux-headers-generic
linux-image-generic
linux-firmware

linux-headers-5.x.0-y-generic
linux-image-5.x.0-y-generic
linux-modules.5.x.0-y-generic
linux-modules-extra-5.x.0-y-generic [Note: present upgrade. Installed manually after upgrade failed to install it.]
nvidia-kernel-common-450
linux-modules-nvidia-450-5.x.0-y-generic

[Hmm, I do not see a generic extra modules metapackage listed or available in the repo (e.g. like "linux-modules-extra-generic" ); I see only specific linux-modules-extra-5.x.0-y-generic versioned packages. ]

etc.

The apt log on my main machine is very mucked up, since I installed and uninstalled tons of things attempting to figure out the recent issue after the upgrade, including trying to build from source etc. Please allow me a day to get a clean apt log from the other machine, which will tell a more coherent story.

Thanks

Manfred Hampl (m-hampl) said : #3

Do/did you also have some of the linux-*-hwe* meta packages installed

The linux-moduels-extra-(versionnumber) package should be a dependency of the linux-image meta package.

scitoast (scitoast) said : #4

Hello, forgive my delayed response. Ridiculously enough, I allowed an unattended upgrade to happen (5.8.0-36 to -38) which made the same issue happen (no linux-modules-extra was installed during upgrade), so I was offline for awhile.

Output from cat /var/log/apt/history.log | grep -E 'Command|hwe|5.8.0':

Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Upgrade: linux-modules-nvidia-450-generic-hwe-20.04:amd64 (5.4.0-54.60, 5.4.0-56.62)
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Upgrade: linux-modules-nvidia-450-generic-hwe-20.04:amd64 (5.4.0-56.62, 5.4.0-58.64)
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/sbin/synaptic
Commandline: aptdaemon role='role-upgrade-system' sender=':1.120'

scitoast (scitoast) said : #5

Output from zcat /var/log/apt/history.log.1.gz | grep -E 'Command|hwe|5.8.0':

Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Upgrade: linux-modules-nvidia-450-generic-hwe-20.04:amd64 (5.4.0-58.64, 5.4.0-59.65)
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Install: linux-modules-5.8.0-34-generic:amd64 (5.8.0-34.37~20.04.2, automatic), linux-modules-nvidia-450-5.8.0-34-generic:amd64 (5.8.0-34.37~20.04.2, automatic), linux-image-5.8.0-34-generic:amd64 (5.8.0-34.37~20.04.2, automatic)
Upgrade: linux-modules-nvidia-450-generic-hwe-20.04:amd64 (5.4.0-59.65, 5.8.0-34.37~20.04.2)
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Install: linux-modules-5.8.0-36-generic:amd64 (5.8.0-36.40~20.04.1, automatic), linux-modules-nvidia-450-5.8.0-36-generic:amd64 (5.8.0-36.40~20.04.1, automatic), dkms:amd64 (2.8.1-5ubuntu1, automatic), nvidia-dkms-450:amd64 (450.102.04-0ubuntu0.20.04.1, automatic), linux-image-5.8.0-36-generic:amd64 (5.8.0-36.40~20.04.1, automatic)
Upgrade: linux-modules-nvidia-450-generic-hwe-20.04:amd64 (5.8.0-34.37~20.04.2, 5.8.0-36.40~20.04.1), libnvidia-compute-450:amd64 (450.80.02-0ubuntu0.20.04.2, 450.102.04-0ubuntu0.20.04.1), libnvidia-compute-450:i386 (450.80.02-0ubuntu0.20.04.2, 450.102.04-0ubuntu0.20.04.1), libnvidia-encode-450:amd64 (450.80.02-0ubuntu0.20.04.2, 450.102.04-0ubuntu0.20.04.1), libnvidia-encode-450:i386 (450.80.02-0ubuntu0.20.04.2, 450.102.04-0ubuntu0.20.04.1), nvidia-kernel-common-450:amd64 (450.80.02-0ubuntu0.20.04.2, 450.102.04-0ubuntu0.20.04.1), xserver-xorg-video-nvidia-450:amd64 (450.80.02-0ubuntu0.20.04.2, 450.102.04-0ubuntu0.20.04.1), libnvidia-gl-450:amd64 (450.80.02-0ubuntu0.20.04.2, 450.102.04-0ubuntu0.20.04.1), libnvidia-gl-450:i386 (450.80.02-0ubuntu0.20.04.2, 450.102.04-0ubuntu0.20.04.1), libnvidia-fbc1-450:amd64 (450.80.02-0ubuntu0.20.04.2, 450.102.04-0ubuntu0.20.04.1), libnvidia-fbc1-450:i386 (450.80.02-0ubuntu0.20.04.2, 450.102.04-0ubuntu0.20.04.1), libnvidia-decode-450:amd64 (450.80.02-0ubuntu0.20.04.2, 450.102.04-0ubuntu0.20.04.1), libnvidia-decode-450:i386 (450.80.02-0ubuntu0.20.04.2, 450.102.04-0ubuntu0.20.04.1), libnvidia-cfg1-450:amd64 (450.80.02-0ubuntu0.20.04.2, 450.102.04-0ubuntu0.20.04.1), nvidia-utils-450:amd64 (450.80.02-0ubuntu0.20.04.2, 450.102.04-0ubuntu0.20.04.1), nvidia-compute-utils-450:amd64 (450.80.02-0ubuntu0.20.04.2, 450.102.04-0ubuntu0.20.04.1), libnvidia-ifr1-450:amd64 (450.80.02-0ubuntu0.20.04.2, 450.102.04-0ubuntu0.20.04.1), libnvidia-ifr1-450:i386 (450.80.02-0ubuntu0.20.04.2, 450.102.04-0ubuntu0.20.04.1), nvidia-driver-450:amd64 (450.80.02-0ubuntu0.20.04.2, 450.102.04-0ubuntu0.20.04.1), libnvidia-extra-450:amd64 (450.80.02-0ubuntu0.20.04.2, 450.102.04-0ubuntu0.20.04.1), nvidia-kernel-source-450:amd64 (450.80.02-0ubuntu0.20.04.2, 450.102.04-0ubuntu0.20.04.1)
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: aptdaemon role='role-commit-packages' sender=':1.783'
Commandline: aptdaemon role='role-upgrade-system' sender=':1.79'
Install: linux-modules-extra-5.4.0-62-generic:amd64 (5.4.0-62.70, automatic), linux-headers-5.4.0-62-generic:amd64 (5.4.0-62.70, automatic), linux-modules-5.8.0-38-generic:amd64 (5.8.0-38.43~20.04.1, automatic), linux-modules-5.4.0-62-generic:amd64 (5.4.0-62.70, automatic), linux-modules-nvidia-450-5.8.0-38-generic:amd64 (5.8.0-38.43~20.04.1, automatic), linux-headers-5.4.0-62:amd64 (5.4.0-62.70, automatic), linux-image-5.8.0-38-generic:amd64 (5.8.0-38.43~20.04.1, automatic), linux-image-5.4.0-62-generic:amd64 (5.4.0-62.70, automatic)
Upgrade: linux-modules-nvidia-450-generic-hwe-20.04:amd64 (5.8.0-36.40~20.04.1, 5.8.0-38.43~20.04.1), linux-headers-generic:amd64 (5.4.0.60.63, 5.4.0.62.65), linux-libc-dev:amd64 (5.4.0-60.67, 5.4.0-62.70), linux-image-generic:amd64 (5.4.0.60.63, 5.4.0.62.65), lsof:amd64 (4.93.2+dfsg-1, 4.93.2+dfsg-1ubuntu0.20.04.1), linux-generic:amd64 (5.4.0.60.63, 5.4.0.62.65)
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Remove: linux-modules-nvidia-450-5.8.0-34-generic:amd64 (5.8.0-34.37~20.04.2), linux-image-5.8.0-34-generic:amd64 (5.8.0-34.37~20.04.2)
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Remove: linux-modules-5.8.0-34-generic:amd64 (5.8.0-34.37~20.04.2)
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade
Commandline: /usr/bin/unattended-upgrade

scitoast (scitoast) said : #6

AFAICT, the only "hwe" package that is installed is the nvidia-450-generic-hwe-20.04. I have no experience with hwe packages and in general have never had them except when e.g. the nvidia driver install process has automatically put it on.

Thank you again. I suspect there must be other people out there with this problem -- there's nothing particularly special or strange about my machines. Both were just a generic install. Previously, kernel upgrades were happening without any incidents. Only when 5.8.0 arrived on the scene did the problem occur.

Best,

Best Manfred Hampl (m-hampl) said : #7

It seems to me that the following happened:

You had linux-image-generic installed and (for whatever reason) nvidia-450-generic-hwe-20.04.

linux-image-generic has as dependencies linux-image-(version)-generic, linux-modules-... and linux-modules-extra-...
nvidia-450-generic-hwe-20.04 has as dependencies only linux-image-(version)-generic and linux-modules-...

In the past linux-image-generic and nvidia-450-generic-hwe-20.04 had the same version (5.4.0.*).
Recently nvidia-450-generic-hwe-20.04 was upgraded to version 5.8.0.*, but linux-image-generic stayed on 5.4.0.*

This lead to the situation that you have upgraded versions of linux-image-(version)-generic and linux-modules-..., but linux-modules-extra-... still only in the 4.5.0.* version.

This seems to be a bug in the dependencies of the packages.

Proposal for solution:
sudo apt install linux-generic-hwe-20.04

This should care for installing also the modules-extra package with the next upgrade of the kernel packages.

Manfred Hampl (m-hampl) said : #8

(Typo, should read 5.4.0.* instead of 4.5.0.*)

scitoast (scitoast) said : #9

Ah ok, so I will follow your recommendation and switch over to the generic-hwe stack / rolling releases. That makes sense. Probably for many home desktop systems the hwe stack is the preferable route anyway these days. Thank you, I will try it, and report back with the result.

Best,

scitoast (scitoast) said : #10

Yes that worked; I let everything upgrade and restarted the system just to be on the safe side. ✓ Everything is functional and now based on the hwe release. I am very grateful for your help! (And, hopefully this thread will be useful for anyone else who encounters the trouble.)

Cheers,

scitoast (scitoast) said : #11

Thanks Manfred Hampl, that solved my question.