bootloader only see first disk

bootloader only see first disk

[Aurélien]

Hi,

I don't know if it is the right place to report a bug, sorry if not.

I'm a Promox user and the last version (5.2) has a package versionned
20180612-1 actually  (
https://git.proxmox.com/?p=pve-edk2-firmware.git;a=summary ) which is
based on 5a56c0493955cf55e7eef96dbba815cfbb067d7d  commit of edk2. A
build from the current master a few days ago has the same problem.
The previous Promox version (5.1 whith this pve-qemu-kvm package) use a
edk2 version dating from 2017 or before (it could be from before
September 2016 but I'm not sure), which has no problem. I tested on my
workstation running gentoo which has a version based on UDK2017 branch
with source code from February 11, 2018 and it's working too.
So I think the problem is located in  the OVMF_CODE.fd code from 2018
(but without any certainty).


Actual behaviour (with all the various Linux distribution: Debian 9,
Ubuntu 18.04, Centos 7.5 ...) with a VM which has EFI enabled:
- grub is loaded and only see the first disk (where it is installed),
the "ls" command in the grub shell return: "(hd0) (hd0,gpt1) (cd0)"  
and  there is also hd1 and hd2 and in my case this is problematic as
/boot is on hd1.
- when pressing "escape" on the boot to go the the setup screen and just
chose "continue" it's ok (grub sees all the disks), my current
workaround to boot the installed OS.
- when installing an OS like Ubuntu, CentOS ... on the first reboot it's
ok (I thinks something in the installation, like seting a new boot entry
with grub-install, efibootmgr)

Expected behaviour:
- grub see all the disk (the ls command return hd0, hd1 hd2 ....) and
not the first one (where the EFI boot partition is located).
(sorry, I haven't tried any other boot loader)

- Step to reproduce:
 start a VM with 3 disks and EFI enabled (OVMF_CODE), install a Linux on
the 2nd disk (/ & swap) and the EFI boot partition on the first disk
(/boot/efi on "sda1"), cold start the VM.

I can help by testing with specific version (or another boot loader),
tell me.

Regards

Aurélien

ps: typical command line to start the VM:
/usr/bin/kvm \
  -id 20 \
  -name testvm.mydomain \
  -chardev 'socket,id=qmp,path=/var/run/qemu-server/20.qmp,server,nowait' \
  -mon 'chardev=qmp,mode=control' \
  -pidfile /var/run/qemu-server/20.pid \
  -daemonize \
  -smbios 'type=1,uuid=88dc4e6b-34b8-46a7-a573-26596c34855d' \
  -drive
'if=pflash,unit=0,format=raw,readonly,file=/usr/share/pve-edk2-firmware//OVMF_CODE.fd'
\
  -drive
'if=pflash,unit=1,format=raw,id=drive-efidisk0,file=/mnt/pve/pve-nfs-sas/images/20/vm-20-disk-4.raw'
\
  -smp '8,sockets=1,cores=8,maxcpus=8' \
  -nodefaults \
  -boot
'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg'
\
  -vga qxl \
  -vnc unix:/var/run/qemu-server/20.vnc,x509,password \
  -cpu host,+kvm_pv_unhalt,+kvm_pv_eoi \
  -m 512 \
  -object 'memory-backend-ram,id=ram-node0,size=512M' \
  -numa 'node,nodeid=0,cpus=0-7,memdev=ram-node0' \
  -k fr \
  -object 'iothread,id=iothread-virtioscsi1' \
  -object 'iothread,id=iothread-virtioscsi2' \
  -machine 'type=pc-i440fx-2.9' \
  -device 'pci-bridge,id=pci.3,chassis_nr=3,bus=pci.0,addr=0x5' \
  -device 'pci-bridge,id=pci.1,chassis_nr=1,bus=pci.0,addr=0x1e' \
  -device 'pci-bridge,id=pci.2,chassis_nr=2,bus=pci.0,addr=0x1f' \
  -device 'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2' \
  -chardev 'socket,path=/var/run/qemu-server/20.qga,server,nowait,id=qga0' \
  -device 'virtio-serial,id=qga0,bus=pci.0,addr=0x8' \
  -device 'virtserialport,chardev=qga0,name=org.qemu.guest_agent.0' \
  -spice
'tls-port=61004,addr=127.0.0.1,tls-ciphers=HIGH,seamless-migration=on' \
  -device 'virtio-serial,id=spice,bus=pci.0,addr=0x9' \
  -chardev 'spicevmc,id=vdagent,name=vdagent' \
  -device 'virtserialport,chardev=vdagent,name=com.redhat.spice.0' \
  -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3' \
  -iscsi 'initiator-name=iqn.1993-08.org.debian:01:f9e4631950da' \
  -drive 'if=none,id=drive-ide2,media=cdrom,aio=threads' \
  -device 'ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2,bootindex=100' \
  -device 'virtio-scsi-pci,id=virtioscsi0,bus=pci.3,addr=0x1' \
  -drive
'file=/mnt/pve/pve-nfs-sas/images/20/vm-20-disk-1.raw,if=none,id=drive-scsi0,format=raw,cache=none,aio=native,detect-zeroes=on'
\
  -device
'scsi-hd,bus=virtioscsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0,id=scsi0,bootindex=200'
\
  -device
'virtio-scsi-pci,id=virtioscsi1,bus=pci.3,addr=0x2,iothread=iothread-virtioscsi1'
\
  -drive
'file=/mnt/pve/pve-nfs-sas/images/20/vm-20-disk-2.raw,if=none,id=drive-scsi1,format=raw,cache=none,aio=native,detect-zeroes=on'
\
  -device
'scsi-hd,bus=virtioscsi1.0,channel=0,scsi-id=0,lun=1,drive=drive-scsi1,id=scsi1'
\
  -device
'virtio-scsi-pci,id=virtioscsi2,bus=pci.3,addr=0x3,iothread=iothread-virtioscsi2'
\
  -drive
'file=/mnt/pve/pve-nfs-sas/images/20/vm-20-disk-3.raw,if=none,id=drive-scsi2,format=raw,cache=none,aio=native,detect-zeroes=on'
\
  -device
'scsi-hd,bus=virtioscsi2.0,channel=0,scsi-id=0,lun=2,drive=drive-scsi2,id=scsi2'
\
  -netdev
'type=tap,id=net0,ifname=tap20i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on'
\
  -device
'virtio-net-pci,mac=4A:63:6D:11:11:11,netdev=net0,bus=pci.0,addr=0x12,id=net0'

Re: bootloader only see first disk

[Laszlo Ersek]

Hello Aurélien,

adding Thomas to the CC list, and commenting at the bottom:

On 09/15/18 18:36, Aurélien wrote:
> Hi,
>
> I don't know if it is the right place to report a bug, sorry if not.
>
> I'm a Promox user and the last version (5.2) has a package versionned
> 20180612-1 actually  (
> https://git.proxmox.com/?p=pve-edk2-firmware.git;a=summary ) which is
> based on 5a56c0493955cf55e7eef96dbba815cfbb067d7d  commit of edk2. A
> build from the current master a few days ago has the same problem. The
> previous Promox version (5.1 whith this pve-qemu-kvm package) use a
> edk2 version dating from 2017 or before (it could be from before
> September 2016 but I'm not sure), which has no problem. I tested on my
> workstation running gentoo which has a version based on UDK2017 branch
> with source code from February 11, 2018 and it's working too. So I
> think the problem is located in  the OVMF_CODE.fd code from 2018 (but
> without any certainty).
>
> Actual behaviour (with all the various Linux distribution: Debian 9,
> Ubuntu 18.04, Centos 7.5 ...) with a VM which has EFI enabled:
> - grub is loaded and only see the first disk (where it is installed),
>   the "ls" command in the grub shell return: "(hd0) (hd0,gpt1)
>   (cd0)"   and  there is also hd1 and hd2 and in my case this is
>   problematic as /boot is on hd1.
> - when pressing "escape" on the boot to go the the setup screen and
>   just chose "continue" it's ok (grub sees all the disks), my current
>   workaround to boot the installed OS.
> - when installing an OS like Ubuntu, CentOS ... on the first reboot
>   it's ok (I thinks something in the installation, like seting a new
>   boot entry with grub-install, efibootmgr)
>
> Expected behaviour:
> - grub see all the disk (the ls command return hd0, hd1 hd2 ....) and
>   not the first one (where the EFI boot partition is located).
>   (sorry, I haven't tried any other boot loader)
>
> - Step to reproduce:
>   start a VM with 3 disks and EFI enabled (OVMF_CODE), install a Linux
>   on the 2nd disk (/ & swap) and the EFI boot partition on the first
>   disk (/boot/efi on "sda1"), cold start the VM.
>
> I can help by testing with specific version (or another boot loader),
> tell me.
>
> Regards
>
> Aurélien
>
> ps: typical command line to start the VM:
> /usr/bin/kvm \
>   -id 20 \
>   -name testvm.mydomain \
>   -chardev 'socket,id=qmp,path=/var/run/qemu-server/20.qmp,server,nowait' \
>   -mon 'chardev=qmp,mode=control' \
>   -pidfile /var/run/qemu-server/20.pid \
>   -daemonize \
>   -smbios 'type=1,uuid=88dc4e6b-34b8-46a7-a573-26596c34855d' \
>   -drive 'if=pflash,unit=0,format=raw,readonly,file=/usr/share/pve-edk2-firmware//OVMF_CODE.fd' \
>   -drive 'if=pflash,unit=1,format=raw,id=drive-efidisk0,file=/mnt/pve/pve-nfs-sas/images/20/vm-20-disk-4.raw' \
>   -smp '8,sockets=1,cores=8,maxcpus=8' \
>   -nodefaults \
>   -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' \
>   -vga qxl \
>   -vnc unix:/var/run/qemu-server/20.vnc,x509,password \
>   -cpu host,+kvm_pv_unhalt,+kvm_pv_eoi \
>   -m 512 \
>   -object 'memory-backend-ram,id=ram-node0,size=512M' \
>   -numa 'node,nodeid=0,cpus=0-7,memdev=ram-node0' \
>   -k fr \
>   -object 'iothread,id=iothread-virtioscsi1' \
>   -object 'iothread,id=iothread-virtioscsi2' \
>   -machine 'type=pc-i440fx-2.9' \
>   -device 'pci-bridge,id=pci.3,chassis_nr=3,bus=pci.0,addr=0x5' \
>   -device 'pci-bridge,id=pci.1,chassis_nr=1,bus=pci.0,addr=0x1e' \
>   -device 'pci-bridge,id=pci.2,chassis_nr=2,bus=pci.0,addr=0x1f' \
>   -device 'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2' \
>   -chardev 'socket,path=/var/run/qemu-server/20.qga,server,nowait,id=qga0' \
>   -device 'virtio-serial,id=qga0,bus=pci.0,addr=0x8' \
>   -device 'virtserialport,chardev=qga0,name=org.qemu.guest_agent.0' \
>   -spice 'tls-port=61004,addr=127.0.0.1,tls-ciphers=HIGH,seamless-migration=on' \
>   -device 'virtio-serial,id=spice,bus=pci.0,addr=0x9' \
>   -chardev 'spicevmc,id=vdagent,name=vdagent' \
>   -device 'virtserialport,chardev=vdagent,name=com.redhat.spice.0' \
>   -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3' \
>   -iscsi 'initiator-name=iqn.1993-08.org.debian:01:f9e4631950da' \
>   -drive 'if=none,id=drive-ide2,media=cdrom,aio=threads' \
>   -device 'ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2,bootindex=100' \
>   -device 'virtio-scsi-pci,id=virtioscsi0,bus=pci.3,addr=0x1' \
>   -drive 'file=/mnt/pve/pve-nfs-sas/images/20/vm-20-disk-1.raw,if=none,id=drive-scsi0,format=raw,cache=none,aio=native,detect-zeroes=on' \
>   -device 'scsi-hd,bus=virtioscsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0,id=scsi0,bootindex=200' \
>   -device 'virtio-scsi-pci,id=virtioscsi1,bus=pci.3,addr=0x2,iothread=iothread-virtioscsi1' \
>   -drive 'file=/mnt/pve/pve-nfs-sas/images/20/vm-20-disk-2.raw,if=none,id=drive-scsi1,format=raw,cache=none,aio=native,detect-zeroes=on' \
>   -device 'scsi-hd,bus=virtioscsi1.0,channel=0,scsi-id=0,lun=1,drive=drive-scsi1,id=scsi1' \
>   -device 'virtio-scsi-pci,id=virtioscsi2,bus=pci.3,addr=0x3,iothread=iothread-virtioscsi2' \
>   -drive 'file=/mnt/pve/pve-nfs-sas/images/20/vm-20-disk-3.raw,if=none,id=drive-scsi2,format=raw,cache=none,aio=native,detect-zeroes=on' \
>   -device 'scsi-hd,bus=virtioscsi2.0,channel=0,scsi-id=0,lun=2,drive=drive-scsi2,id=scsi2' \
>   -netdev 'type=tap,id=net0,ifname=tap20i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on' \
>   -device 'virtio-net-pci,mac=4A:63:6D:11:11:11,netdev=net0,bus=pci.0,addr=0x12,id=net0'

This is a very good issue report, thank you for it. The amount of detail
you provided allows me to explain the status & behavior to you.

(1) Originally, OVMF used to connect all drivers to all devices in the
system. In a sense this was convenient, however it resulted in a lot of
wasted work too -- if you had tens or hundreds of devices, the boot
could take extremely long. And again, most of that work was wasted,
since you almost never want to actually attempt booting off of tens or
hundreds of devices. You only really want to connect those devices that
you reasonably want to attempt booting off of as well.

(In the above, "connect" is used in the sense "bind" or "probe". Esp.
with a Linux background, "probe" is the most fitting word, probably.)


(2) Because the set of devices connected at boot is platform policy
(i.e., UEFI platforms are at liberty to decide what devices they
connect), we changed the above behavior to a minimalistic approach, in
the following patch series:

  [edk2] [PATCH 0/6] OvmfPkg, ArmVirtQemu: leaner platform BDS policy for connecting devices
  http://mid.mail-archive.com/20180313212233.19215-1-lersek@redhat.com

This was then pushed as commit range 12957e56d26d..ff1d0fbfbaec, in
March 2018.

With this set in place, if you provided *some* devices with the
'bootindex=N' property, then OVMF would no longer connect all devices in
the system, it would connect only those that you marked with the
bootindex property. (And then OVMF would auto-generate UEFI boot options
for those only, as well.)

In other words, the bootindex property was promoted from its previous
"boot option filtering/reordering only" role to a "steering of device
probing" role as well.


(3) The EFI build of grub most likely doesn't find your disks -- after
(2) -- because it doesn't connect devices itself, it relies on the
platform firmware to connect disks.


(4) If you interrupt the boot process and enter the setup utility, the
latter will connect all devices, regardless of (2). This is your current
workaround, IIUC.


(5) There are many classes of devices, and some of those should be
connected even if they are not "bootable" at all. The series in (2)
caused a regression for them, and we had to fix them up gradually. See
the following commits:

- OvmfPkg/PlatformBootManagerLib: add USB keyboard to ConIn
- OvmfPkg/PlatformBootManagerLib: connect consoles unconditionally
- OvmfPkg/PlatformBootManagerLib: connect Virtio RNG devices again

However, the disks that you mention are *not* like this; they should
indeed *not* be connected unless you mark them for booting.


(6) The solution is either to instruct grub (I don't know how) to
connect the disks in question itself, *or* to add the "bootindex=N"
properties (with some suitable N values) to the disk devices that you
want OVMF to connect for you (even if they contain no EFI system
partition and/or a UEFI boot loader application). Given your current
command line, the latter option translates to the "scsi-hd" devices with
identifiers "scsi1" and "scsi2". For example:

  -device scsi-hd,bus=virtioscsi1.0,channel=0,scsi-id=0,lun=1,drive=drive-scsi1,id=scsi1,bootindex=201 \
  -device scsi-hd,bus=virtioscsi2.0,channel=0,scsi-id=0,lun=2,drive=drive-scsi2,id=scsi2,bootindex=202 \

Hope this helps,
Laszlo

Re: bootloader only see first disk

[Thomas Lamprecht]

Hi Laszlo,

On 9/16/18 12:04 PM, Laszlo Ersek wrote:
> Hello Aurélien,
> 
> adding Thomas to the CC list, and commenting at the bottom:

thanks for CC'ing and thanks to you Aurélien for writing your report.

> 
> On 09/15/18 18:36, Aurélien wrote:
>> Hi,
>>
>> I don't know if it is the right place to report a bug, sorry if not.
>>
>> I'm a Promox user and the last version (5.2) has a package versionned
>> 20180612-1 actually  (
>> https://git.proxmox.com/?p=pve-edk2-firmware.git;a=summary ) which is
>> based on 5a56c0493955cf55e7eef96dbba815cfbb067d7d  commit of edk2. A
>> build from the current master a few days ago has the same problem. The
>> previous Promox version (5.1 whith this pve-qemu-kvm package) use a
>> edk2 version dating from 2017 or before (it could be from before
>> September 2016 but I'm not sure), which has no problem. I tested on my
>> workstation running gentoo which has a version based on UDK2017 branch
>> with source code from February 11, 2018 and it's working too. So I
>> think the problem is located in  the OVMF_CODE.fd code from 2018 (but
>> without any certainty).
>>
>> Actual behaviour (with all the various Linux distribution: Debian 9,
>> Ubuntu 18.04, Centos 7.5 ...) with a VM which has EFI enabled:
>> - grub is loaded and only see the first disk (where it is installed),
>>   the "ls" command in the grub shell return: "(hd0) (hd0,gpt1)
>>   (cd0)"   and  there is also hd1 and hd2 and in my case this is
>>   problematic as /boot is on hd1.
>> - when pressing "escape" on the boot to go the the setup screen and
>>   just chose "continue" it's ok (grub sees all the disks), my current
>>   workaround to boot the installed OS.
>> - when installing an OS like Ubuntu, CentOS ... on the first reboot
>>   it's ok (I thinks something in the installation, like seting a new
>>   boot entry with grub-install, efibootmgr)
>>
>> Expected behaviour:
>> - grub see all the disk (the ls command return hd0, hd1 hd2 ....) and
>>   not the first one (where the EFI boot partition is located).
>>   (sorry, I haven't tried any other boot loader)
>>
>> - Step to reproduce:
>>   start a VM with 3 disks and EFI enabled (OVMF_CODE), install a Linux
>>   on the 2nd disk (/ & swap) and the EFI boot partition on the first
>>   disk (/boot/efi on "sda1"), cold start the VM.
>>
>> I can help by testing with specific version (or another boot loader),
>> tell me.
>>
>> Regards
>>
>> Aurélien
>>
>> ps: typical command line to start the VM:
>> /usr/bin/kvm \
>>   -id 20 \
>>   -name testvm.mydomain \
>>   -chardev 'socket,id=qmp,path=/var/run/qemu-server/20.qmp,server,nowait' \
>>   -mon 'chardev=qmp,mode=control' \
>>   -pidfile /var/run/qemu-server/20.pid \
>>   -daemonize \
>>   -smbios 'type=1,uuid=88dc4e6b-34b8-46a7-a573-26596c34855d' \
>>   -drive 'if=pflash,unit=0,format=raw,readonly,file=/usr/share/pve-edk2-firmware//OVMF_CODE.fd' \
>>   -drive 'if=pflash,unit=1,format=raw,id=drive-efidisk0,file=/mnt/pve/pve-nfs-sas/images/20/vm-20-disk-4.raw' \
>>   -smp '8,sockets=1,cores=8,maxcpus=8' \
>>   -nodefaults \
>>   -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' \
>>   -vga qxl \
>>   -vnc unix:/var/run/qemu-server/20.vnc,x509,password \
>>   -cpu host,+kvm_pv_unhalt,+kvm_pv_eoi \
>>   -m 512 \
>>   -object 'memory-backend-ram,id=ram-node0,size=512M' \
>>   -numa 'node,nodeid=0,cpus=0-7,memdev=ram-node0' \
>>   -k fr \
>>   -object 'iothread,id=iothread-virtioscsi1' \
>>   -object 'iothread,id=iothread-virtioscsi2' \
>>   -machine 'type=pc-i440fx-2.9' \
>>   -device 'pci-bridge,id=pci.3,chassis_nr=3,bus=pci.0,addr=0x5' \
>>   -device 'pci-bridge,id=pci.1,chassis_nr=1,bus=pci.0,addr=0x1e' \
>>   -device 'pci-bridge,id=pci.2,chassis_nr=2,bus=pci.0,addr=0x1f' \
>>   -device 'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2' \
>>   -chardev 'socket,path=/var/run/qemu-server/20.qga,server,nowait,id=qga0' \
>>   -device 'virtio-serial,id=qga0,bus=pci.0,addr=0x8' \
>>   -device 'virtserialport,chardev=qga0,name=org.qemu.guest_agent.0' \
>>   -spice 'tls-port=61004,addr=127.0.0.1,tls-ciphers=HIGH,seamless-migration=on' \
>>   -device 'virtio-serial,id=spice,bus=pci.0,addr=0x9' \
>>   -chardev 'spicevmc,id=vdagent,name=vdagent' \
>>   -device 'virtserialport,chardev=vdagent,name=com.redhat.spice.0' \
>>   -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3' \
>>   -iscsi 'initiator-name=iqn.1993-08.org.debian:01:f9e4631950da' \
>>   -drive 'if=none,id=drive-ide2,media=cdrom,aio=threads' \
>>   -device 'ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2,bootindex=100' \
>>   -device 'virtio-scsi-pci,id=virtioscsi0,bus=pci.3,addr=0x1' \
>>   -drive 'file=/mnt/pve/pve-nfs-sas/images/20/vm-20-disk-1.raw,if=none,id=drive-scsi0,format=raw,cache=none,aio=native,detect-zeroes=on' \
>>   -device 'scsi-hd,bus=virtioscsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0,id=scsi0,bootindex=200' \
>>   -device 'virtio-scsi-pci,id=virtioscsi1,bus=pci.3,addr=0x2,iothread=iothread-virtioscsi1' \
>>   -drive 'file=/mnt/pve/pve-nfs-sas/images/20/vm-20-disk-2.raw,if=none,id=drive-scsi1,format=raw,cache=none,aio=native,detect-zeroes=on' \
>>   -device 'scsi-hd,bus=virtioscsi1.0,channel=0,scsi-id=0,lun=1,drive=drive-scsi1,id=scsi1' \
>>   -device 'virtio-scsi-pci,id=virtioscsi2,bus=pci.3,addr=0x3,iothread=iothread-virtioscsi2' \
>>   -drive 'file=/mnt/pve/pve-nfs-sas/images/20/vm-20-disk-3.raw,if=none,id=drive-scsi2,format=raw,cache=none,aio=native,detect-zeroes=on' \
>>   -device 'scsi-hd,bus=virtioscsi2.0,channel=0,scsi-id=0,lun=2,drive=drive-scsi2,id=scsi2' \
>>   -netdev 'type=tap,id=net0,ifname=tap20i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on' \
>>   -device 'virtio-net-pci,mac=4A:63:6D:11:11:11,netdev=net0,bus=pci.0,addr=0x12,id=net0'
> 
> This is a very good issue report, thank you for it. The amount of detail
> you provided allows me to explain the status & behavior to you.
> 
> (1) Originally, OVMF used to connect all drivers to all devices in the
> system. In a sense this was convenient, however it resulted in a lot of
> wasted work too -- if you had tens or hundreds of devices, the boot
> could take extremely long. And again, most of that work was wasted,
> since you almost never want to actually attempt booting off of tens or
> hundreds of devices. You only really want to connect those devices that
> you reasonably want to attempt booting off of as well.
> 
> (In the above, "connect" is used in the sense "bind" or "probe". Esp.
> with a Linux background, "probe" is the most fitting word, probably.)
> 
> 
> (2) Because the set of devices connected at boot is platform policy
> (i.e., UEFI platforms are at liberty to decide what devices they
> connect), we changed the above behavior to a minimalistic approach, in
> the following patch series:
> 
>   [edk2] [PATCH 0/6] OvmfPkg, ArmVirtQemu: leaner platform BDS policy for connecting devices
>   http://mid.mail-archive.com/20180313212233.19215-1-lersek@redhat.com
> 
> This was then pushed as commit range 12957e56d26d..ff1d0fbfbaec, in
> March 2018.
> 
> With this set in place, if you provided *some* devices with the
> 'bootindex=N' property, then OVMF would no longer connect all devices in
> the system, it would connect only those that you marked with the
> bootindex property. (And then OVMF would auto-generate UEFI boot options
> for those only, as well.)
> 
> In other words, the bootindex property was promoted from its previous
> "boot option filtering/reordering only" role to a "steering of device
> probing" role as well.
> 

Ah yes, I should have figured that out, I even mentioned the faster boot
on multiple devices in the package changelog but never wondered about what
this implies for setups like Aurélien has, argh!

> 
> (3) The EFI build of grub most likely doesn't find your disks -- after
> (2) -- because it doesn't connect devices itself, it relies on the
> platform firmware to connect disks.
> 
> 
> (4) If you interrupt the boot process and enter the setup utility, the
> latter will connect all devices, regardless of (2). This is your current
> workaround, IIUC.
> 
> 
> (5) There are many classes of devices, and some of those should be
> connected even if they are not "bootable" at all. The series in (2)
> caused a regression for them, and we had to fix them up gradually. See
> the following commits:
> 
> - OvmfPkg/PlatformBootManagerLib: add USB keyboard to ConIn
> - OvmfPkg/PlatformBootManagerLib: connect consoles unconditionally
> - OvmfPkg/PlatformBootManagerLib: connect Virtio RNG devices again
> 
> However, the disks that you mention are *not* like this; they should
> indeed *not* be connected unless you mark them for booting.
> 
> 
> (6) The solution is either to instruct grub (I don't know how) to
> connect the disks in question itself, *or* to add the "bootindex=N"
> properties (with some suitable N values) to the disk devices that you
> want OVMF to connect for you (even if they contain no EFI system
> partition and/or a UEFI boot loader application). Given your current
> command line, the latter option translates to the "scsi-hd" devices with
> identifiers "scsi1" and "scsi2". For example:
> 
>   -device scsi-hd,bus=virtioscsi1.0,channel=0,scsi-id=0,lun=1,drive=drive-scsi1,id=scsi1,bootindex=201 \
>   -device scsi-hd,bus=virtioscsi2.0,channel=0,scsi-id=0,lun=2,drive=drive-scsi2,id=scsi2,bootindex=202 \

So we probably want to add a per-disk 'bootable' flag in our frontend,
which adds a higher bootindex (= lower priority) to all marked disks.
This would allow to still boot faster with lots of disks but allows
setups like yours, Aurélien.

> 
> Hope this helps,

Yes, it helps a lot! Much thanks for your explanation, appreciated!

cheers,
Thomas

bootloader only see first disk

[Aurelien]

Hi Laszlo,

On 16/09/2018 12:04, Laszlo Ersek wrote:
> Hello Aurélien,
> 
> adding Thomas to the CC list, and commenting at the bottom:
> 
> On 09/15/18 18:36, Aurélien wrote:
>> Hi,
>>
>> I don't know if it is the right place to report a bug, sorry if not.
>>
>> I'm a Promox user and the last version (5.2) has a package versionned
>> 20180612-1 actually  (
>> https://git.proxmox.com/?p=pve-edk2-firmware.git;a=summary ) which is
>> based on 5a56c0493955cf55e7eef96dbba815cfbb067d7d  commit of edk2. A
>> build from the current master a few days ago has the same problem. The
>> previous Promox version (5.1 whith this pve-qemu-kvm package) use a
>> edk2 version dating from 2017 or before (it could be from before
>> September 2016 but I'm not sure), which has no problem. I tested on my
>> workstation running gentoo which has a version based on UDK2017 branch
>> with source code from February 11, 2018 and it's working too. So I
>> think the problem is located in  the OVMF_CODE.fd code from 2018 (but
>> without any certainty).
>>
>> Actual behaviour (with all the various Linux distribution: Debian 9,
>> Ubuntu 18.04, Centos 7.5 ...) with a VM which has EFI enabled:
>> - grub is loaded and only see the first disk (where it is installed),
>>   the "ls" command in the grub shell return: "(hd0) (hd0,gpt1)
>>   (cd0)"   and  there is also hd1 and hd2 and in my case this is
>>   problematic as /boot is on hd1.
>> - when pressing "escape" on the boot to go the the setup screen and
>>   just chose "continue" it's ok (grub sees all the disks), my current
>>   workaround to boot the installed OS.
>> - when installing an OS like Ubuntu, CentOS ... on the first reboot
>>   it's ok (I thinks something in the installation, like seting a new
>>   boot entry with grub-install, efibootmgr)
>>
>> Expected behaviour:
>> - grub see all the disk (the ls command return hd0, hd1 hd2 ....) and
>>   not the first one (where the EFI boot partition is located).
>>   (sorry, I haven't tried any other boot loader)
>>
>> - Step to reproduce:
>>   start a VM with 3 disks and EFI enabled (OVMF_CODE), install a Linux
>>   on the 2nd disk (/ & swap) and the EFI boot partition on the first
>>   disk (/boot/efi on "sda1"), cold start the VM.
>>
>> I can help by testing with specific version (or another boot loader),
>> tell me.
>>
>> Regards
>>
>> Aurélien
>>
>> ps: typical command line to start the VM:
>> /usr/bin/kvm \
>>   -id 20 \
>>   -name testvm.mydomain \
>>   -chardev 'socket,id=qmp,path=/var/run/qemu-server/20.qmp,server,nowait' \
>>   -mon 'chardev=qmp,mode=control' \
>>   -pidfile /var/run/qemu-server/20.pid \
>>   -daemonize \
>>   -smbios 'type=1,uuid=88dc4e6b-34b8-46a7-a573-26596c34855d' \
>>   -drive 'if=pflash,unit=0,format=raw,readonly,file=/usr/share/pve-edk2-firmware//OVMF_CODE.fd' \
>>   -drive 'if=pflash,unit=1,format=raw,id=drive-efidisk0,file=/mnt/pve/pve-nfs-sas/images/20/vm-20-disk-4.raw' \
>>   -smp '8,sockets=1,cores=8,maxcpus=8' \
>>   -nodefaults \
>>   -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' \
>>   -vga qxl \
>>   -vnc unix:/var/run/qemu-server/20.vnc,x509,password \
>>   -cpu host,+kvm_pv_unhalt,+kvm_pv_eoi \
>>   -m 512 \
>>   -object 'memory-backend-ram,id=ram-node0,size=512M' \
>>   -numa 'node,nodeid=0,cpus=0-7,memdev=ram-node0' \
>>   -k fr \
>>   -object 'iothread,id=iothread-virtioscsi1' \
>>   -object 'iothread,id=iothread-virtioscsi2' \
>>   -machine 'type=pc-i440fx-2.9' \
>>   -device 'pci-bridge,id=pci.3,chassis_nr=3,bus=pci.0,addr=0x5' \
>>   -device 'pci-bridge,id=pci.1,chassis_nr=1,bus=pci.0,addr=0x1e' \
>>   -device 'pci-bridge,id=pci.2,chassis_nr=2,bus=pci.0,addr=0x1f' \
>>   -device 'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2' \
>>   -chardev 'socket,path=/var/run/qemu-server/20.qga,server,nowait,id=qga0' \
>>   -device 'virtio-serial,id=qga0,bus=pci.0,addr=0x8' \
>>   -device 'virtserialport,chardev=qga0,name=org.qemu.guest_agent.0' \
>>   -spice 'tls-port=61004,addr=127.0.0.1,tls-ciphers=HIGH,seamless-migration=on' \
>>   -device 'virtio-serial,id=spice,bus=pci.0,addr=0x9' \
>>   -chardev 'spicevmc,id=vdagent,name=vdagent' \
>>   -device 'virtserialport,chardev=vdagent,name=com.redhat.spice.0' \
>>   -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3' \
>>   -iscsi 'initiator-name=iqn.1993-08.org.debian:01:f9e4631950da' \
>>   -drive 'if=none,id=drive-ide2,media=cdrom,aio=threads' \
>>   -device 'ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2,bootindex=100' \
>>   -device 'virtio-scsi-pci,id=virtioscsi0,bus=pci.3,addr=0x1' \
>>   -drive 'file=/mnt/pve/pve-nfs-sas/images/20/vm-20-disk-1.raw,if=none,id=drive-scsi0,format=raw,cache=none,aio=native,detect-zeroes=on' \
>>   -device 'scsi-hd,bus=virtioscsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0,id=scsi0,bootindex=200' \
>>   -device 'virtio-scsi-pci,id=virtioscsi1,bus=pci.3,addr=0x2,iothread=iothread-virtioscsi1' \
>>   -drive 'file=/mnt/pve/pve-nfs-sas/images/20/vm-20-disk-2.raw,if=none,id=drive-scsi1,format=raw,cache=none,aio=native,detect-zeroes=on' \
>>   -device 'scsi-hd,bus=virtioscsi1.0,channel=0,scsi-id=0,lun=1,drive=drive-scsi1,id=scsi1' \
>>   -device 'virtio-scsi-pci,id=virtioscsi2,bus=pci.3,addr=0x3,iothread=iothread-virtioscsi2' \
>>   -drive 'file=/mnt/pve/pve-nfs-sas/images/20/vm-20-disk-3.raw,if=none,id=drive-scsi2,format=raw,cache=none,aio=native,detect-zeroes=on' \
>>   -device 'scsi-hd,bus=virtioscsi2.0,channel=0,scsi-id=0,lun=2,drive=drive-scsi2,id=scsi2' \
>>   -netdev 'type=tap,id=net0,ifname=tap20i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on' \
>>   -device 'virtio-net-pci,mac=4A:63:6D:11:11:11,netdev=net0,bus=pci.0,addr=0x12,id=net0'
> 
> This is a very good issue report, thank you for it. The amount of detail
> you provided allows me to explain the status & behavior to you.
> 
> (1) Originally, OVMF used to connect all drivers to all devices in the
> system. In a sense this was convenient, however it resulted in a lot of
> wasted work too -- if you had tens or hundreds of devices, the boot
> could take extremely long. And again, most of that work was wasted,
> since you almost never want to actually attempt booting off of tens or
> hundreds of devices. You only really want to connect those devices that
> you reasonably want to attempt booting off of as well.
> 
> (In the above, "connect" is used in the sense "bind" or "probe". Esp.
> with a Linux background, "probe" is the most fitting word, probably.)
> 
> 
> (2) Because the set of devices connected at boot is platform policy
> (i.e., UEFI platforms are at liberty to decide what devices they
> connect), we changed the above behavior to a minimalistic approach, in
> the following patch series:
> 
>   [edk2] [PATCH 0/6] OvmfPkg, ArmVirtQemu: leaner platform BDS policy for connecting devices
>   http://mid.mail-archive.com/20180313212233.19215-1-lersek@redhat.com
> 
> This was then pushed as commit range 12957e56d26d..ff1d0fbfbaec, in
> March 2018.
> 
> With this set in place, if you provided *some* devices with the
> 'bootindex=N' property, then OVMF would no longer connect all devices in
> the system, it would connect only those that you marked with the
> bootindex property. (And then OVMF would auto-generate UEFI boot options
> for those only, as well.)
> 
> In other words, the bootindex property was promoted from its previous
> "boot option filtering/reordering only" role to a "steering of device
> probing" role as well.
> 
> 
> (3) The EFI build of grub most likely doesn't find your disks -- after
> (2) -- because it doesn't connect devices itself, it relies on the
> platform firmware to connect disks.
> 
> 
> (4) If you interrupt the boot process and enter the setup utility, the
> latter will connect all devices, regardless of (2). This is your current
> workaround, IIUC.
> 
> 
> (5) There are many classes of devices, and some of those should be
> connected even if they are not "bootable" at all. The series in (2)
> caused a regression for them, and we had to fix them up gradually. See
> the following commits:
> 
> - OvmfPkg/PlatformBootManagerLib: add USB keyboard to ConIn
> - OvmfPkg/PlatformBootManagerLib: connect consoles unconditionally
> - OvmfPkg/PlatformBootManagerLib: connect Virtio RNG devices again
> 
> However, the disks that you mention are *not* like this; they should
> indeed *not* be connected unless you mark them for booting.
> 
> 
> (6) The solution is either to instruct grub (I don't know how) to
> connect the disks in question itself, *or* to add the "bootindex=N"
> properties (with some suitable N values) to the disk devices that you
> want OVMF to connect for you (even if they contain no EFI system
> partition and/or a UEFI boot loader application). Given your current
> command line, the latter option translates to the "scsi-hd" devices with
> identifiers "scsi1" and "scsi2". For example:
> 
>   -device scsi-hd,bus=virtioscsi1.0,channel=0,scsi-id=0,lun=1,drive=drive-scsi1,id=scsi1,bootindex=201 \
>   -device scsi-hd,bus=virtioscsi2.0,channel=0,scsi-id=0,lun=2,drive=drive-scsi2,id=scsi2,bootindex=202 \
> 
> Hope this helps,
> Laszlo
> 


Thank you very much for you detailed answer, it helps a lot.
Now I fully understand what's going on and the new behaviour:
"minimalistic approach", introduced during march.
Effectively no need to have all devices connected by the platform
firmware but just a few of them (in my case the first 2 disks).
I thought that bootindex was a bit legacy while using EFI, and
was limited to choose boot order between CD, Hard drives or PXE.
I tested the solution you point in [6] with adding bootindex=202 to the
second disk and it works perfectly.


Best Regards

Aurélien

Re: bootloader only see first disk

[Aurelien Minet]

Hi Thomas,

Le 17/09/2018 à 10:34, Thomas Lamprecht a écrit :
> Hi Laszlo,
>
> On 9/16/18 12:04 PM, Laszlo Ersek wrote:
>> Hello Aurélien,
>>
>> adding Thomas to the CC list, and commenting at the bottom:
> thanks for CC'ing and thanks to you Aurélien for writing your report.
>
>> On 09/15/18 18:36, Aurélien wrote:
>>> Hi,
>>>
>>> I don't know if it is the right place to report a bug, sorry if not.
>>>
>>> I'm a Promox user and the last version (5.2) has a package versionned
>>> 20180612-1 actually  (
>>> https://git.proxmox.com/?p=pve-edk2-firmware.git;a=summary ) which is
>>> based on 5a56c0493955cf55e7eef96dbba815cfbb067d7d  commit of edk2. A
>>> build from the current master a few days ago has the same problem. The
>>> previous Promox version (5.1 whith this pve-qemu-kvm package) use a
>>> edk2 version dating from 2017 or before (it could be from before
>>> September 2016 but I'm not sure), which has no problem. I tested on my
>>> workstation running gentoo which has a version based on UDK2017 branch
>>> with source code from February 11, 2018 and it's working too. So I
>>> think the problem is located in  the OVMF_CODE.fd code from 2018 (but
>>> without any certainty).
>>>
>>> Actual behaviour (with all the various Linux distribution: Debian 9,
>>> Ubuntu 18.04, Centos 7.5 ...) with a VM which has EFI enabled:
>>> - grub is loaded and only see the first disk (where it is installed),
>>>   the "ls" command in the grub shell return: "(hd0) (hd0,gpt1)
>>>   (cd0)"   and  there is also hd1 and hd2 and in my case this is
>>>   problematic as /boot is on hd1.
>>> - when pressing "escape" on the boot to go the the setup screen and
>>>   just chose "continue" it's ok (grub sees all the disks), my current
>>>   workaround to boot the installed OS.
>>> - when installing an OS like Ubuntu, CentOS ... on the first reboot
>>>   it's ok (I thinks something in the installation, like seting a new
>>>   boot entry with grub-install, efibootmgr)
>>>
>>> Expected behaviour:
>>> - grub see all the disk (the ls command return hd0, hd1 hd2 ....) and
>>>   not the first one (where the EFI boot partition is located).
>>>   (sorry, I haven't tried any other boot loader)
>>>
>>> - Step to reproduce:
>>>   start a VM with 3 disks and EFI enabled (OVMF_CODE), install a Linux
>>>   on the 2nd disk (/ & swap) and the EFI boot partition on the first
>>>   disk (/boot/efi on "sda1"), cold start the VM.
>>>
>>> I can help by testing with specific version (or another boot loader),
>>> tell me.
>>>
>>> Regards
>>>
>>> Aurélien
>>>
>>> ps: typical command line to start the VM:
>>> /usr/bin/kvm \
>>>   -id 20 \
>>>   -name testvm.mydomain \
>>>   -chardev 'socket,id=qmp,path=/var/run/qemu-server/20.qmp,server,nowait' \
>>>   -mon 'chardev=qmp,mode=control' \
>>>   -pidfile /var/run/qemu-server/20.pid \
>>>   -daemonize \
>>>   -smbios 'type=1,uuid=88dc4e6b-34b8-46a7-a573-26596c34855d' \
>>>   -drive 'if=pflash,unit=0,format=raw,readonly,file=/usr/share/pve-edk2-firmware//OVMF_CODE.fd' \
>>>   -drive 'if=pflash,unit=1,format=raw,id=drive-efidisk0,file=/mnt/pve/pve-nfs-sas/images/20/vm-20-disk-4.raw' \
>>>   -smp '8,sockets=1,cores=8,maxcpus=8' \
>>>   -nodefaults \
>>>   -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' \
>>>   -vga qxl \
>>>   -vnc unix:/var/run/qemu-server/20.vnc,x509,password \
>>>   -cpu host,+kvm_pv_unhalt,+kvm_pv_eoi \
>>>   -m 512 \
>>>   -object 'memory-backend-ram,id=ram-node0,size=512M' \
>>>   -numa 'node,nodeid=0,cpus=0-7,memdev=ram-node0' \
>>>   -k fr \
>>>   -object 'iothread,id=iothread-virtioscsi1' \
>>>   -object 'iothread,id=iothread-virtioscsi2' \
>>>   -machine 'type=pc-i440fx-2.9' \
>>>   -device 'pci-bridge,id=pci.3,chassis_nr=3,bus=pci.0,addr=0x5' \
>>>   -device 'pci-bridge,id=pci.1,chassis_nr=1,bus=pci.0,addr=0x1e' \
>>>   -device 'pci-bridge,id=pci.2,chassis_nr=2,bus=pci.0,addr=0x1f' \
>>>   -device 'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2' \
>>>   -chardev 'socket,path=/var/run/qemu-server/20.qga,server,nowait,id=qga0' \
>>>   -device 'virtio-serial,id=qga0,bus=pci.0,addr=0x8' \
>>>   -device 'virtserialport,chardev=qga0,name=org.qemu.guest_agent.0' \
>>>   -spice 'tls-port=61004,addr=127.0.0.1,tls-ciphers=HIGH,seamless-migration=on' \
>>>   -device 'virtio-serial,id=spice,bus=pci.0,addr=0x9' \
>>>   -chardev 'spicevmc,id=vdagent,name=vdagent' \
>>>   -device 'virtserialport,chardev=vdagent,name=com.redhat.spice.0' \
>>>   -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3' \
>>>   -iscsi 'initiator-name=iqn.1993-08.org.debian:01:f9e4631950da' \
>>>   -drive 'if=none,id=drive-ide2,media=cdrom,aio=threads' \
>>>   -device 'ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2,bootindex=100' \
>>>   -device 'virtio-scsi-pci,id=virtioscsi0,bus=pci.3,addr=0x1' \
>>>   -drive 'file=/mnt/pve/pve-nfs-sas/images/20/vm-20-disk-1.raw,if=none,id=drive-scsi0,format=raw,cache=none,aio=native,detect-zeroes=on' \
>>>   -device 'scsi-hd,bus=virtioscsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0,id=scsi0,bootindex=200' \
>>>   -device 'virtio-scsi-pci,id=virtioscsi1,bus=pci.3,addr=0x2,iothread=iothread-virtioscsi1' \
>>>   -drive 'file=/mnt/pve/pve-nfs-sas/images/20/vm-20-disk-2.raw,if=none,id=drive-scsi1,format=raw,cache=none,aio=native,detect-zeroes=on' \
>>>   -device 'scsi-hd,bus=virtioscsi1.0,channel=0,scsi-id=0,lun=1,drive=drive-scsi1,id=scsi1' \
>>>   -device 'virtio-scsi-pci,id=virtioscsi2,bus=pci.3,addr=0x3,iothread=iothread-virtioscsi2' \
>>>   -drive 'file=/mnt/pve/pve-nfs-sas/images/20/vm-20-disk-3.raw,if=none,id=drive-scsi2,format=raw,cache=none,aio=native,detect-zeroes=on' \
>>>   -device 'scsi-hd,bus=virtioscsi2.0,channel=0,scsi-id=0,lun=2,drive=drive-scsi2,id=scsi2' \
>>>   -netdev 'type=tap,id=net0,ifname=tap20i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on' \
>>>   -device 'virtio-net-pci,mac=4A:63:6D:11:11:11,netdev=net0,bus=pci.0,addr=0x12,id=net0'
>> This is a very good issue report, thank you for it. The amount of detail
>> you provided allows me to explain the status & behavior to you.
>>
>> (1) Originally, OVMF used to connect all drivers to all devices in the
>> system. In a sense this was convenient, however it resulted in a lot of
>> wasted work too -- if you had tens or hundreds of devices, the boot
>> could take extremely long. And again, most of that work was wasted,
>> since you almost never want to actually attempt booting off of tens or
>> hundreds of devices. You only really want to connect those devices that
>> you reasonably want to attempt booting off of as well.
>>
>> (In the above, "connect" is used in the sense "bind" or "probe". Esp.
>> with a Linux background, "probe" is the most fitting word, probably.)
>>
>>
>> (2) Because the set of devices connected at boot is platform policy
>> (i.e., UEFI platforms are at liberty to decide what devices they
>> connect), we changed the above behavior to a minimalistic approach, in
>> the following patch series:
>>
>>   [edk2] [PATCH 0/6] OvmfPkg, ArmVirtQemu: leaner platform BDS policy for connecting devices
>>   http://mid.mail-archive.com/20180313212233.19215-1-lersek@redhat.com
>>
>> This was then pushed as commit range 12957e56d26d..ff1d0fbfbaec, in
>> March 2018.
>>
>> With this set in place, if you provided *some* devices with the
>> 'bootindex=N' property, then OVMF would no longer connect all devices in
>> the system, it would connect only those that you marked with the
>> bootindex property. (And then OVMF would auto-generate UEFI boot options
>> for those only, as well.)
>>
>> In other words, the bootindex property was promoted from its previous
>> "boot option filtering/reordering only" role to a "steering of device
>> probing" role as well.
>>
> Ah yes, I should have figured that out, I even mentioned the faster boot
> on multiple devices in the package changelog but never wondered about what
> this implies for setups like Aurélien has, argh!
>
>> (3) The EFI build of grub most likely doesn't find your disks -- after
>> (2) -- because it doesn't connect devices itself, it relies on the
>> platform firmware to connect disks.
>>
>>
>> (4) If you interrupt the boot process and enter the setup utility, the
>> latter will connect all devices, regardless of (2). This is your current
>> workaround, IIUC.
>>
>>
>> (5) There are many classes of devices, and some of those should be
>> connected even if they are not "bootable" at all. The series in (2)
>> caused a regression for them, and we had to fix them up gradually. See
>> the following commits:
>>
>> - OvmfPkg/PlatformBootManagerLib: add USB keyboard to ConIn
>> - OvmfPkg/PlatformBootManagerLib: connect consoles unconditionally
>> - OvmfPkg/PlatformBootManagerLib: connect Virtio RNG devices again
>>
>> However, the disks that you mention are *not* like this; they should
>> indeed *not* be connected unless you mark them for booting.
>>
>>
>> (6) The solution is either to instruct grub (I don't know how) to
>> connect the disks in question itself, *or* to add the "bootindex=N"
>> properties (with some suitable N values) to the disk devices that you
>> want OVMF to connect for you (even if they contain no EFI system
>> partition and/or a UEFI boot loader application). Given your current
>> command line, the latter option translates to the "scsi-hd" devices with
>> identifiers "scsi1" and "scsi2". For example:
>>
>>   -device scsi-hd,bus=virtioscsi1.0,channel=0,scsi-id=0,lun=1,drive=drive-scsi1,id=scsi1,bootindex=201 \
>>   -device scsi-hd,bus=virtioscsi2.0,channel=0,scsi-id=0,lun=2,drive=drive-scsi2,id=scsi2,bootindex=202 \
> So we probably want to add a per-disk 'bootable' flag in our frontend,
> which adds a higher bootindex (= lower priority) to all marked disks.
> This would allow to still boot faster with lots of disks but allows
> setups like yours, Aurélien.

As actually it is only possible to chose one drive (which add "d"  to
boot property and drive id to bootdisk property).
I was thinking of just being able to chose another drive in the GUI (for
"boot device 3") which would result by adding this drive after the first
one (and a coma) to the bootdisk property.
So your proposition sounds good to me :)  and would be wonderful as it
will support  more possibilities.

Best regards

Aurélien



>
>> Hope this helps,
> Yes, it helps a lot! Much thanks for your explanation, appreciated!
>
> cheers,
> Thomas
>
>