Ubuntu 20.04 arm64 server-cloud image hits kernel panic but boots successfully on retry

This is the image we are using:
https://cloud-images.ubuntu.com/releases/20.04/release/ubuntu-20.04-server-cloudimg-arm64.img

When the image boots, it first hits a kernel panic. Then the Grub menu appears, counts down and
successfully boots.

From looking a bit at the grub config file: /boot/grub/grub.cfg, it seems to attempt the first boot without any initrd.
if [ "${initrdfail}" = 1 ]; then
    linux /boot/vmlinuz-5.4.0-28-generic root=PARTUUID=a1dc7417-50de-4301-b9b4-ce567c4e4f27 ro quiet splash $vt_handoff
    initrd /boot/initrd.img-5.4.0-28-generic
else
    linux /boot/vmlinuz-5.4.0-28-generic root=PARTUUID=a1dc7417-50de-4301-b9b4-ce567c4e4f27 ro quiet splash $vt_handoff panic=-1
fi

The boot without an initrd seems to hit a panic. Then grub retries the boot with the initrd and the boot succeeds.

It goes through this sequence every time we boot the image.

The question here is if there is any way to avoid this initial panic?
We understand that we could modify the grub.cfg to always use the intird. That clearly eliminates the issue.
However, we were hoping for another option that avoids modifying a generated file.

It is also worth mentioning that we tried the amd64 version of this image, and it does not have this issue (ubuntu-20.04-server-cloudimg-amd64.img).

Here is the console output from the panic:

 [ 2.973416] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)
[ 2.974472] CPU: 1 PID: 1 Comm: swapper/0 Not tainted 5.4.0-28-generic #32-Ubuntu
[ 2.975376] Hardware name: QEMU KVM Virtual Machine, BIOS 0.0.0 02/06/2015
[ 2.976305] Call trace:
[ 2.976625] dump_backtrace+0x0/0x188
[ 2.977094] show_stack+0x24/0x30
[ 2.977524] dump_stack+0xb4/0xf8
[ 2.978260] panic+0x14c/0x358
[ 2.978916] mount_block_root+0x244/0x2fc
[ 2.979776] mount_root+0x44/0x4c
[ 2.980488] prepare_namespace+0x138/0x1b8
[ 2.981367] kernel_init_freeable+0x26c/0x2b0
[ 2.982315] kernel_init+0x18/0x108
[ 2.983068] ret_from_fork+0x10/0x18
[ 2.983889] SMP: stopping secondary CPUs
[ 2.985013] Kernel Offset: disabled
[ 2.985772] CPU features: 0x0002,22208238
[ 2.986313] Memory Limit: none

This is our QEMU command line to launch the image:
qemu-system-aarch64 -nographic -machine virt,gic-version=max -m 4G -cpu max -smp 8 -drive file=ubuntu.aarch64.img,if=none,id=drive0,cache=writeback -device virtio-blk,drive=drive0,bootindex=0 -drive file=flash0.img,format=raw,if=pflash -drive file=flash1.img,format=raw,if=pflash