Cannot boot customized Server ISO with updated kernel

Asked by Jean Marrara

Hi guys!

I'm using Cubic version 2021.06-52-release~202106292153~ubuntu20.04.1 on an Ubuntu 20.04.1 machine.
I'm trying to customize a ubuntu-20.04.2-live-server-amd64 and trying to upgrade the kernel to 5.10 in order to use it to install on some machines that have the i225 NIC (only supported from kernel 5.8).
It turns out that I cannot boot from the resulting iso, it opens Grub and when I select the Install Ubuntu it freezes.
Can you help?

Thank you in advance!
Jean Marrara

Question information

Language:
English Edit question
Status:
Solved
For:
Cubic Edit question
Assignee:
No assignee Edit question
Solved by:
Cubic PPA
Solved:
Last query:
Last reply:
Revision history for this message
Cubic PPA (cubic-wizard) said :
#1

How did you upgrade the kernel?

Does it boot/install if you do not upgrade the kernel?

Revision history for this message
Jean Marrara (jmarrara) said :
#2
Revision history for this message
Cubic PPA (cubic-wizard) said :
#3

Did dpkg automatically update-initramfs ?
If not you may need to do that.

On the Options page, Kernel tab, did you select your newly installed kernel, or you used the default kernel from the original ISO?

This kernel is used for bootstrapping the ISO. You may want to continue to select the original ISO kernel on this tab.
The drawback is that once your ISO boots, it will be running the old kernel in the Live Environment.
However, once you install the OS, the most recently added kernel is used in the newly installed system.

Revision history for this message
Jean Marrara (jmarrara) said :
#4

Yes, it updated the initramfs and Linuz and I selected the new kernel on Kernel Tab.
If I select the default kernel from original ISO it works fine.
However, I have this new i225 network interface that I need to enable upon installation/live CD and need new kernel for that.
It seems that when initramfs is updated the live iso init script does not work properly.

Revision history for this message
Cubic PPA (cubic-wizard) said (last edit ):
#5

I tested this using "xubuntu-20.04.1-desktop-amd64.iso"...

------------------------------------------------------------------------
( 1 ) DOWNLOAD KERNELS
------------------------------------------------------------------------

In Cubic, I executed the following...

root@cubic:~# mkdir kernels
root@cubic:~# cd kernels

root@cubic:~/kernels# wget https://kernel.ubuntu.com/~kernel-ppa/mainline/v5.10/amd64/linux-headers-5.10.0-051000-generic_5.10.0-051000.202012132330_amd64.deb
root@cubic:~/kernels# wget https://kernel.ubuntu.com/~kernel-ppa/mainline/v5.10/amd64/linux-headers-5.10.0-051000_5.10.0-051000.202012132330_all.deb
root@cubic:~/kernels# wget https://kernel.ubuntu.com/~kernel-ppa/mainline/v5.10/amd64/linux-image-unsigned-5.10.0-051000-generic_5.10.0-051000.202012132330_amd64.deb
root@cubic:~/kernels# wget https://kernel.ubuntu.com/~kernel-ppa/mainline/v5.10/amd64/linux-modules-5.10.0-051000-generic_5.10.0-051000.202012132330_amd64.deb

------------------------------------------------------------------------
( 2 ) LIST CURRENT KERNELS
------------------------------------------------------------------------

root@cubic:~/kernels# ls -l /boot

-rw------- 1 root root 4738430 Jul 9 2020 System.map-5.4.0-42-generic
-rw-r--r-- 1 root root 237773 Jul 9 2020 config-5.4.0-42-generic
drwxr-xr-x 2 root root 80 Jul 31 2020 grub
lrwxrwxrwx 1 root root 27 Jul 31 2020 initrd.img -> initrd.img-5.4.0-42-generic [BROKEN LINK]
lrwxrwxrwx 1 root root 27 Jul 31 2020 initrd.img.old -> initrd.img-5.4.0-42-generic [BROKEN LINK]
-rw-r--r-- 1 root root 182704 Feb 13 2020 memtest86+.bin
-rw-r--r-- 1 root root 184380 Feb 13 2020 memtest86+.elf
-rw-r--r-- 1 root root 184884 Feb 13 2020 memtest86+_multiboot.bin
lrwxrwxrwx 1 root root 24 Jul 31 2020 vmlinuz -> vmlinuz-5.4.0-42-generic [BROKEN LINK]
lrwxrwxrwx 1 root root 24 Jul 31 2020 vmlinuz.old -> vmlinuz-5.4.0-42-generic [BROKEN LINK]

------------------------------------------------------------------------
( 3 ) INSTALL NEW KERNEL
------------------------------------------------------------------------

root@cubic:~/kernels# dpkg -i *.deb

Selecting previously unselected package linux-headers-5.10.0-051000-generic.
(Reading database ... 143400 files and directories currently installed.)
Preparing to unpack linux-headers-5.10.0-051000-generic_5.10.0-051000.202012132330_amd64.deb ...
Unpacking linux-headers-5.10.0-051000-generic (5.10.0-051000.202012132330) ...
Selecting previously unselected package linux-headers-5.10.0-051000.
Preparing to unpack linux-headers-5.10.0-051000_5.10.0-051000.202012132330_all.deb ...
Unpacking linux-headers-5.10.0-051000 (5.10.0-051000.202012132330) ...
Selecting previously unselected package linux-image-unsigned-5.10.0-051000-generic.
Preparing to unpack linux-image-unsigned-5.10.0-051000-generic_5.10.0-051000.202012132330_amd64.deb ...
Unpacking linux-image-unsigned-5.10.0-051000-generic (5.10.0-051000.202012132330) ...
Selecting previously unselected package linux-modules-5.10.0-051000-generic.
Preparing to unpack linux-modules-5.10.0-051000-generic_5.10.0-051000.202012132330_amd64.deb ...
Unpacking linux-modules-5.10.0-051000-generic (5.10.0-051000.202012132330) ...
Setting up linux-headers-5.10.0-051000 (5.10.0-051000.202012132330) ...
Setting up linux-headers-5.10.0-051000-generic (5.10.0-051000.202012132330) ...
Setting up linux-image-unsigned-5.10.0-051000-generic (5.10.0-051000.202012132330) ...
I: /boot/vmlinuz.old is now a symlink to vmlinuz-5.10.0-051000-generic
I: /boot/initrd.img.old is now a symlink to initrd.img-5.10.0-051000-generic
I: /boot/vmlinuz is now a symlink to vmlinuz-5.10.0-051000-generic
I: /boot/initrd.img is now a symlink to initrd.img-5.10.0-051000-generic
Setting up linux-modules-5.10.0-051000-generic (5.10.0-051000.202012132330) ...
Processing triggers for linux-image-unsigned-5.10.0-051000-generic (5.10.0-051000.202012132330) ...
/etc/kernel/postinst.d/initramfs-tools:
update-initramfs: Generating /boot/initrd.img-5.10.0-051000-generic
cryptsetup: ERROR: Couldn't resolve device tmpfs
cryptsetup: WARNING: Couldn't determine root device
W: Possible missing firmware /lib/firmware/i915/tgl_huc_7.5.0.bin for module i915
W: Possible missing firmware /lib/firmware/i915/tgl_huc_7.5.0.bin for module i915
W: Possible missing firmware /lib/firmware/i915/tgl_dmc_ver2_08.bin for module i915
W: Possible missing firmware /lib/firmware/i915/rkl_dmc_ver2_02.bin for module i915

------------------------------------------------------------------------
( 4 ) LIST NEW INSTALLED KERNELS
------------------------------------------------------------------------

root@cubic:~/kernels# ls -l /boot

-rw------- 1 root root 5894469 Dec 13 2020 System.map-5.10.0-051000-generic
-rw------- 1 root root 4738430 Jul 9 2020 System.map-5.4.0-42-generic
-rw-r--r-- 1 root root 250506 Dec 13 2020 config-5.10.0-051000-generic
-rw-r--r-- 1 root root 237773 Jul 9 2020 config-5.4.0-42-generic
drwxr-xr-x 2 root root 80 Jul 31 2020 grub
lrwxrwxrwx 1 root root 32 Aug 30 16:16 initrd.img -> initrd.img-5.10.0-051000-generic
-rw-r--r-- 1 root root 90568833 Aug 30 16:16 initrd.img-5.10.0-051000-generic
lrwxrwxrwx 1 root root 32 Aug 30 16:16 initrd.img.old -> initrd.img-5.10.0-051000-generic
-rw-r--r-- 1 root root 182704 Feb 13 2020 memtest86+.bin
-rw-r--r-- 1 root root 184380 Feb 13 2020 memtest86+.elf
-rw-r--r-- 1 root root 184884 Feb 13 2020 memtest86+_multiboot.bin
lrwxrwxrwx 1 root root 29 Aug 30 16:16 vmlinuz -> vmlinuz-5.10.0-051000-generic
-rw------- 1 root root 14143520 Dec 13 2020 vmlinuz-5.10.0-051000-generic
lrwxrwxrwx 1 root root 29 Aug 30 16:16 vmlinuz.old -> vmlinuz-5.10.0-051000-generic

------------------------------------------------------------------------
( 5 ) GENERATE CUSTOM ISO
------------------------------------------------------------------------

On Cubic's Option page, Kernel tab, I ensured that Kernel Version 5.10.0-051000 was selected.

Then I proceeded to generate the ISO.

------------------------------------------------------------------------
( 6 ) TEST ISO
------------------------------------------------------------------------

I was able to successfully boot into the generated ISO in VirtualBox.

------------------------------------------------------------------------
( 7 ) VERIFY KERNEL
------------------------------------------------------------------------

Here is the output after logging into the customized Live ISO...

xubuntu@xubuntu:~$ uname -r
5.10.0-051000-generic

xubuntu@xubuntu:~$ lsb_release --description
Description: Xubuntu 20.04.1 2021.08.30 LTS customized using Cubic on 2021-08-30 16:04

Revision history for this message
Jean Marrara (jmarrara) said :
#6

Hi Cubic PPA,

Tried exact same steps with ubuntu-18.04.5-desktop-amd64 and it worked.
However, when I tried the same with ubuntu-18.04.5-live-server-amd64 it did not work. It freezes just after selecting the "Install Ubuntu Server" option on Grub.

Revision history for this message
Cubic PPA (cubic-wizard) said :
#7

I'll try with the 20.04 server ISO.
May be there is a difference.

Revision history for this message
Jean Marrara (jmarrara) said :
#8

Hi all,

I believe I fixed the issue, on ubuntu-18.04.5-live-server-amd64, for some reason, once kernel is upgraded I had to install initramfs-tools and casper. Then upon system start through Live CD it will launch the subiquity installer and everything will work.

Thank you!

Revision history for this message
Jean Marrara (jmarrara) said :
#9

My solution only works for the Ubuntu Server Live Install which comes with the new subiquity installer. It does not work for the ubuntu-18.04.5-server-amd64 that comes with the debian installer.
I need to user debian installer because I`m using a preseed file ad subiquty does not accept this type of files.
Can you help?

Revision history for this message
Cubic PPA (cubic-wizard) said :
#10

I thought you were working with Ubuntu 20.04 based on your original question?
Are you trying to customize "ubuntu-18.04.5-server-amd64" or "ubuntu-20.04.2-live-server-amd64"?

Just so I am clear,
- Ubuntu 18.04 Server uses a debian installer ?
- Ubuntu 20.04 Server uses Sububiquity ?

Revision history for this message
Cubic PPA (cubic-wizard) said :
#11

My understanding is that you want to use ubuntu-18.04.5-live-server-amd64.iso because it supports preseed files.
You also want to be able to install kernel 5.10.0-051000-generic because this kernel supports your hardware (i225 NIC).

Let me know if I understand your requirements correctly?

Revision history for this message
Cubic PPA (cubic-wizard) said (last edit ):
#12

I just downloaded ubuntu-18.04.5-live-server-amd64.iso, and I was able to update the kernel and boot into the installer successfully.

You were right that casper needs to be installed.
But initramfs-tools does not need to be installed.

I also added the missing i915 firmware modules, although it is not necessary.

I think you should try customizing 18.04.5 server again; it worked for me.

Here are the steps I followed...

========================================================================

apt update
apt install --no-install-recommends casper

mkdir -p /lib/firmware/i915
cd /lib/firmware/i915

wget https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/i915/skl_huc_2.0.0.bin
wget https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/i915/skl_guc_33.0.0.bin
wget https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/i915/bxt_huc_2.0.0.bin
wget https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/i915/bxt_guc_33.0.0.bin
wget https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/i915/kbl_huc_4.0.0.bin
wget https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/i915/kbl_guc_33.0.0.bin
wget https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/i915/glk_huc_4.0.0.bin
wget https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/i915/glk_guc_33.0.0.bin
wget https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/i915/kbl_huc_4.0.0.bin
wget https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/i915/kbl_guc_33.0.0.bin
wget https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/i915/cml_huc_4.0.0.bin
wget https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/i915/cml_guc_33.0.0.bin
wget https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/i915/icl_huc_9.0.0.bin
wget https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/i915/icl_guc_33.0.0.bin
wget https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/i915/ehl_huc_9.0.0.bin
wget https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/i915/ehl_guc_33.0.4.bin
wget https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/i915/tgl_huc_7.5.0.bin
wget https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/i915/tgl_guc_35.2.0.bin
wget https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/i915/tgl_huc_7.5.0.bin
wget https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/i915/tgl_guc_35.2.0.bin
wget https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/i915/bxt_dmc_ver1_07.bin
wget https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/i915/skl_dmc_ver1_27.bin
wget https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/i915/kbl_dmc_ver1_04.bin
wget https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/i915/glk_dmc_ver1_04.bin
wget https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/i915/cnl_dmc_ver1_07.bin
wget https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/i915/icl_dmc_ver1_09.bin
wget https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/i915/tgl_dmc_ver2_08.bin
wget https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/i915/rkl_dmc_ver2_02.bin

mkdir ~/kernels
cd ~/kernels

wget https://kernel.ubuntu.com/~kernel-ppa/mainline/v5.10/amd64/linux-headers-5.10.0-051000-generic_5.10.0-051000.202012132330_amd64.deb
wget https://kernel.ubuntu.com/~kernel-ppa/mainline/v5.10/amd64/linux-headers-5.10.0-051000_5.10.0-051000.202012132330_all.deb
wget https://kernel.ubuntu.com/~kernel-ppa/mainline/v5.10/amd64/linux-image-unsigned-5.10.0-051000-generic_5.10.0-051000.202012132330_amd64.deb
wget https://kernel.ubuntu.com/~kernel-ppa/mainline/v5.10/amd64/linux-modules-5.10.0-051000-generic_5.10.0-051000.202012132330_amd64.deb

dpkg -i *.deb

After generating the customized ISO, I installed the OS, and I was able to login and verify the release and kernel version:

$ uname -r
5.10.0-051000-generic

$ lsb_release --description
Description: Ubuntu-Server 18.04.5 2021.08.31 customized using Cubic on 2021-08-31 22:35

Revision history for this message
Jean Marrara (jmarrara) said :
#13

Thank you for the testing it.
I`m trying to customize Ubuntu 18.04 Server with kernel 5.10.0-051000-generic
The ubuntu-18.04.5-live-server-amd64 comes with the new subiquity installer and I was able to follow your steps and it works, however it does not support preseed files anymore.
The ubuntu-18.04.5-server-amd64 (not live) still comes with the debian-installer which works with preseed files but I was not able to make it work when I upgrade the Kernel.

Revision history for this message
Cubic PPA (cubic-wizard) said :
#14

Can you share the download links for both ISOs?

If you share the links, I will download these and see what can be done.

Revision history for this message
Jean Marrara (jmarrara) said :
#15
Revision history for this message
Cubic PPA (cubic-wizard) said :
#16

Can you explain what you mean when you say, "ubuntu-18.04.5-live-server-amd64 comes with the new subiquity installer ... however it does not support preseed files anymore" ?

When I look in the /preseed directory of ubuntu-18.04.5-live-server-amd64.iso, I see lots of preseed files:

    cli.seed
    cloud.seed
    hwe-cli.seed
    hwe-cloud.seed
    hwe-ubuntu-server-minimal.seed
    hwe-ubuntu-server-minimalvm.seed
    hwe-ubuntu-server.seed
    ubuntu-server-minimal.seed
    ubuntu-server-minimalvm.seed
    ubuntu-server.seed

Isn't it fair to assume that these files must be used by the installer, if they are present on the ISO?

How does one determine if preseed is supported or not supported?

Revision history for this message
Jean Marrara (jmarrara) said :
#17

You're right, the preseed files are still there (on 18.04.5-live-server), however I cannot use them anymore.
Tried the following on a non touched ubuntu-18.04.5-live-server-amd64.iso:

#kvm -no-reboot -m 1024 -drive file=image.img,format=raw,cache=none,if=virtio -cdrom ~/Downloads/ubuntu-18.04.5-live-server-amd64.iso

On Grub, added: file=/cdrom/preseed/ubuntu-server.seed auto=true

It starts the installer, then in the installer checked the Installer version and it says: Ubuntu Server Installer as you can check in the link below

https://drive.google.com/file/d/1iOeKfmGRXNIPaI38616kK7LWEAvQMN9N/view?usp=sharing

Also I can check subiquity is running instead of debian-intall (screenshot below):

https://drive.google.com/file/d/1SCEv_Vd72MCbVHZJ60oBGUSEDWdErxtO/view?usp=sharing

Also, preseed was not used during installation at all.

Thank you for all your help!!!!

Revision history for this message
Cubic PPA (cubic-wizard) said :
#18

I understand that you want to customize Ubuntu server with a specific kernel.
You also want a version of Ubuntu that supports preseed, so you can do an automated install.
Unfortunately, newer versions of the Ubuntu server ISO do not allow presseeding.

Would you be willing to consider using cloud-init?
It seems to allow automated installs and customization, just like preseed.

If you could get cloud-init to work like preseed, would that suit your needs?

Revision history for this message
Jean Marrara (jmarrara) said :
#19

HI guys, yes cloud-init would work. Sorry for the delay in the response.

Revision history for this message
Best Cubic PPA (cubic-wizard) said :
#20

Although I am able to install the 5.10 kernel in Cubic, cloud-init keeps overwriting the installed kernel with 5.11.
Cloud-init seems to do an update on its own, and I haven't been able to figure out how to stop that.

Nevertheless, here's some info. May be it can help you or give you some insight...

Download the Ubuntu 21.04 Live Server ISO:
http://releases.ubuntu.com/hirsute/ubuntu-21.04-live-server-amd64.iso

------------------------------------------------------------------------
Cubic Options page, Kernel tab
------------------------------------------------------------------------

On the Options page, Kernel tab, select

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
5.10.62-051062 Kernel Version
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

------------------------------------------------------------------------
Cubic Options page, Preseed tab
------------------------------------------------------------------------

On the Options page, Preseed tab, add two new cloud-init configuration files.

(1)
Click on the "preseed" folder, and then click the "Create new file" button in the headerbar.
Create a new file named "meta-data".

Leave this file blank.

(2)
Click on the "preseed" folder, and then click the "Create new file" button in the headerbar.
Create a new file named "user-data".

Paste the following into the new "user-data" file:

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
#cloud-config
autoinstall:
  version: 1
  # Update the installer?
  refresh-installer:
  update: no
  # interactive-sections:
  # - storage
  locale: en_US.UTF-8
  keyboard:
    layout: us
  identity:
    hostname: ubuntu-server
    # The encrypted password is ubuntu
    password: "$6$exDY1mhS4KUYCE/2$zmn9ToZwTKLhCw.b4/b.ZRTIZM30JZ4QrOQ2aOXJ8yk96xpcCof0kxKwuX1kqLG/ygbJ1f8wxED22bTL4F46P0"
    username: ubuntu
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

------------------------------------------------------------------------
Cubic Options page, Boot tab
------------------------------------------------------------------------

On the Options page, Boot tab, add the following menuentry above the "Ubuntu Server" menuentry line.

(Notice that we added: autoinstall "ds=nocloud;s=/cdrom/preseed/")

Note this is for UEFI. You'll have to make similar changes in other boot config files for booting legacy BIOS.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
menuentry "Install Ubuntu Server (Automated)" {
 set gfxpayload=keep
 linux /casper/vmlinuz boot=casper autoinstall "ds=nocloud;s=/cdrom/preseed/" ---
 initrd /casper/initrd.lz
}
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Revision history for this message
Jean Marrara (jmarrara) said :
#21

Thank you!

Revision history for this message
Jean Marrara (jmarrara) said :
#22

Thanks Cubic PPA, that solved my question.