NBDE with multiple disks does not work with clevis-initramfs

Asked by Jaimes Joschko

Hi,

I'm unable to get automatic Network-Bound Disk Encryption (NBDE) with multiple disks to work when using clevis-initramfs.
I see Bug "#1948598 Can't unlock multiple devices in initramfs" is the same symptom but I'm not certain it is the same cause and it says this has been fixed in jammy.

Q1: Am I missing a step for clevis-initramfs? or is this a bug with clevis-initramfs?
Q2: If this is a bug should I open a new bug report or add this to bug #1948598?
Q3: Is this the correct platform to open this bug report on, or should I go upstream to https://github.com/latchset/clevis and open an issue there?

I've done the following testing:

Note: all machines (tang server, dracut client, initramfs client) are on Ubuntu 22.04 (Jammy Jellyfish).

# Set up tang server

```
sudo apt update && sudo apt upgrade -y
sudo apt install tang
sudo systemctl enable tangd.socket --now
systemctl show tangd.socket -p Listen
reboot
```

Note: The tang server is at 192.168.122.40.

# Set up clevis-dracut client (works as expected)

start with ubuntu-22.04.2-desktop-amd64.iso and install it in a VM with Virtual Machine Manager (i.e. libvirt)
    - Configure to use LVM with disk encryption
    - Otherwise just the defaults

```
sudo apt update && sudo apt upgrade -y
sudo apt install clevis clevis-luks clevis-udisks2 clevis-systemd clevis-dracut
sudo clevis luks bind -d /dev/vda4 tang '{"url": "http://192.168.122.40:80"}'
sudo dracut -fv --regenerate-all --hostonly-cmdline
sudo reboot
```

The disk is automatically unlocked as expected.
Powering off, and adding a second virtual disk. Then rebooting.

```
ubuntu@ubuntu-01:~$ lsblk -e 7
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
sr0 11:0 1 1024M 0 rom
vda 252:0 0 25G 0 disk
\u251c\u2500vda1 252:1 0 1M 0 part
\u251c\u2500vda2 252:2 0 513M 0 part /boot/efi
\u251c\u2500vda3 252:3 0 1.7G 0 part /boot
\u2514\u2500vda4 252:4 0 22.8G 0 part
  \u2514\u2500vda4_crypt 253:0 0 22.8G 0 crypt
    \u251c\u2500vgubuntu-swap_1 253:1 0 2.5G 0 lvm [SWAP]
    \u2514\u2500vgubuntu-root 253:2 0 20.3G 0 lvm /var/snap/firefox/common/host-hunspell
                                                /
vdb 252:16 0 20G 0 disk
```

Setting up the second disk:

```
sudo cryptsetup luksFormat /dev/vdb
sudo cryptsetup luksOpen /dev/vdb disk2
sudo mkfs.ext4 /dev/mapper/disk2
sudo mkdir /disk2
```

Added `/dev/mapper/disk2 /disk2 ext4 defaults 0 0` to `/etc/fstab`
Added `disk2 /dev/vdb none` to `/etc/crypttab`

```
sudo reboot
```

The first disk vda4 decrypts automatically and the second disk prompts for the password, as expected.

Setting up clevis for the second disk:

```
sudo clevis luks bind -d /dev/vdb tang '{"url": "http://192.168.122.40:80"}'
sudo dracut -fv --regenerate-all --hostonly-cmdline
```

```
sudo reboot
```

Both disks decrypt automatically (starting with vda4, then vdb), as expected.
Note: You need to wait about 20 seconds

So with clevis-dracut this all seem to work fine.

# Set up clevis-initramfs client (does not work as expected)

start with ubuntu-22.04.2-desktop-amd64.iso and install it in a VM with Virtual Machine Manager (i.e. libvirt)
    - Configure to use LVM with disk encryption
    - Otherwise just the defaults

```
sudo apt update && sudo apt upgrade -y
sudo apt install clevis clevis-luks clevis-udisks2 clevis-systemd clevis-initramfs
sudo clevis luks bind -d /dev/vda4 tang '{"url": "http://192.168.122.40:80"}'
sudo update-initramfs -u -k 'all'
sudo reboot
```

The disk is automatically unlocked as expected.
Powering off, and adding a second virtual disk. Then rebooting.

```
ubuntu@ubuntu-01:~$ lsblk -e 7
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
sr0 11:0 1 1024M 0 rom
vda 252:0 0 25G 0 disk
\u251c\u2500vda1 252:1 0 1M 0 part
\u251c\u2500vda2 252:2 0 513M 0 part /boot/efi
\u251c\u2500vda3 252:3 0 1.7G 0 part /boot
\u2514\u2500vda4 252:4 0 22.8G 0 part
  \u2514\u2500vda4_crypt 253:0 0 22.8G 0 crypt
    \u251c\u2500vgubuntu-swap_1 253:1 0 2.5G 0 lvm [SWAP]
    \u2514\u2500vgubuntu-root 253:2 0 20.3G 0 lvm /var/snap/firefox/common/host-hunspell
                                                /
vdb 252:16 0 20G 0 disk
```

Setting up the second disk:

```
sudo cryptsetup luksFormat /dev/vdb
sudo cryptsetup luksOpen /dev/vdb disk2
sudo mkfs.ext4 /dev/mapper/disk2
sudo mkdir /disk2
```

Added `/dev/mapper/disk2 /disk2 ext4 defaults 0 0` to `/etc/fstab`
Added `disk2 /dev/vdb none` to `/etc/crypttab`

```
sudo reboot
```

Neither disk decrypts automatically and you must enter the passphrase for both disks. Not expected.

Setting up clevis for the second disk:

```
sudo clevis luks bind -d /dev/vdb tang '{"url": "http://192.168.122.40:80"}'
sudo update-initramfs -u -k 'all'
```

```
sudo reboot
```

Neither disk decrypts automatically and you must enter the passphrase for both disks. Not expected.

So with clevis-initramfs, multi disk decryption does not seem to work as expected.
I dug through the logs but I don't really see much of a difference between dracut and initramfs across the different boots.

I'm happy to help test a fix but I don't have the knowledge/skills yet to dig much further into this on my own.

Thanks for spending the time reading through all this. It is very much appreciated.

Cheers,
Jaimes Joschko

Question information

Language:
English Edit question
Status:
Open
For:
Ubuntu clevis Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
actionparsnip (andrew-woodhead666) said :
#1

Did you grab the tang keys from /var/db/tang on your tang server? If the folder is empty then you can generate then with:

sudo /usr/libexec/tangd-keygen /var/db/tang

Revision history for this message
Jaimes Joschko (jjoschko) said :
#2

No, I just used the default key that was generated. I don't think that is the cause though, since it works with dracut (single and multiple disk cases), and the single disk case for initramfs.

When I do `tang-show-keys` on the tang server, the same key shows up that is presented when doing

```
sudo clevis luks bind -d /dev/vdb tang '{"url": "http://192.168.122.40:80"}'
```

The directories /var/db or /var/db/tang do not exist.

I did try your suggestion just to make sure. Thus, I created the directories, generated new keys using `sudo /usr/libexec/tangd-keygen /var/db/tang` and rebound the luks disks partitions but no luk (pun intended). Does Ubuntu use this directory for the keys even?

While going through the process again I did notice that, for the two disk case with initramfs, shutdown/reboot would hang with:

systemd-shutdown[1]: Waiting for process: clevis-luks-ask

Thanks for your suggestion @actionparsnip

Please let me know if I'm mistaken or you have another idea.

Revision history for this message
actionparsnip (andrew-woodhead666) said :
#3

Is that service set to "enabled" so it starts at boot?

Revision history for this message
Jaimes Joschko (jjoschko) said :
#4

# On the initramfs client I get:

```
ubuntu@ubuntu-01:~$ systemctl status clevis-luks-askpass.service
\u25cb clevis-luks-askpass.service - Forward Password Requests to Clevis
     Loaded: loaded (/lib/systemd/system/clevis-luks-askpass.service; static)
     Active: inactive (dead) since Tue 2023-07-25 09:19:43 PDT; 5min ago
TriggeredBy: \u25cf clevis-luks-askpass.path
       Docs: man:clevis-luks-unlockers(7)
    Process: 843 ExecStart=/usr/libexec/clevis-luks-askpass -l (code=exited, status=0/SUCCESS)
   Main PID: 843 (code=exited, status=0/SUCCESS)
        CPU: 3.521s

Jul 25 09:19:36 ubuntu-01 clevis-luks-askpass[1449]: Error communicating with the server!
Jul 25 09:19:36 ubuntu-01 clevis-luks-askpass[1492]: Error communicating with the server!
Jul 25 09:19:37 ubuntu-01 clevis-luks-askpass[1691]: Error communicating with the server!
Jul 25 09:19:37 ubuntu-01 clevis-luks-askpass[1734]: Error communicating with the server!
Jul 25 09:19:38 ubuntu-01 clevis-luks-askpass[1933]: Error communicating with the server!
Jul 25 09:19:38 ubuntu-01 clevis-luks-askpass[1976]: Error communicating with the server!
Jul 25 09:19:39 ubuntu-01 clevis-luks-askpass[2182]: Error communicating with the server!
Jul 25 09:19:39 ubuntu-01 clevis-luks-askpass[2225]: Error communicating with the server!
Jul 25 09:19:43 ubuntu-01 systemd[1]: clevis-luks-askpass.service: Deactivated successfully.
Jul 25 09:19:43 ubuntu-01 systemd[1]: clevis-luks-askpass.service: Consumed 3.521s CPU time.
```

# On the dracut client I get:

```
clevis-luks-askpass.service - Forward Password Requests to Clevis
     Loaded: loaded (/lib/systemd/system/clevis-luks-askpass.service; static)
     Active: inactive (dead) since Tue 2023-07-25 09:24:03 PDT; 52s ago
TriggeredBy: \u25cf clevis-luks-askpass.path
       Docs: man:clevis-luks-unlockers(7)
    Process: 3229 ExecStart=/usr/libexec/clevis-luks-askpass -l (code=exited, status=0/SUCCESS)
   Main PID: 3229 (code=exited, status=0/SUCCESS)
        CPU: 8.369s

Jul 25 09:23:53 ubuntu-01 systemd[1]: Started Forward Password Requests to Clevis.
Jul 25 09:23:57 ubuntu-01 clevis-luks-askpass[3229]: Unlocked /dev/vdb (UUID=b49b8452-0eb5-462a-a218-237f8aed9e70) successfully
Jul 25 09:24:03 ubuntu-01 systemd[1]: clevis-luks-askpass.service: Deactivated successfully.
Jul 25 09:24:03 ubuntu-01 systemd[1]: clevis-luks-askpass.service: Consumed 8.369s CPU time.
```

Can you help with this problem?

Provide an answer of your own, or ask Jaimes Joschko for more information if necessary.

To post a message you must log in.