From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=212.186.127.180; helo=proxmox-new.maurer-it.com; envelope-from=t.lamprecht@proxmox.com; receiver=edk2-devel@lists.01.org Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [212.186.127.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id DDA3D21136003 for ; Mon, 17 Sep 2018 01:34:36 -0700 (PDT) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 78DCA43C41; Mon, 17 Sep 2018 10:34:34 +0200 (CEST) To: Laszlo Ersek , =?UTF-8?B?QXVyw6lsaWVu?= Cc: edk2-devel@lists.01.org References: <0784c256-7009-94d4-ce3a-c020e0317e43@free.fr> <3614532a-fd89-0e9d-2435-530e343024a6@redhat.com> From: Thomas Lamprecht Message-ID: <0fbf7745-3899-f2e6-fe9c-e8935cdc1aed@proxmox.com> Date: Mon, 17 Sep 2018 10:34:32 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.0 MIME-Version: 1.0 In-Reply-To: <3614532a-fd89-0e9d-2435-530e343024a6@redhat.com> Subject: Re: bootloader only see first disk X-BeenThere: edk2-devel@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: EDK II Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Sep 2018 08:34:37 -0000 Content-Type: text/plain; charset=iso-8859-15 Content-Language: en-US Content-Transfer-Encoding: quoted-printable Hi Laszlo, On 9/16/18 12:04 PM, Laszlo Ersek wrote: > Hello Aur=E9lien, >=20 > adding Thomas to the CC list, and commenting at the bottom: thanks for CC'ing and thanks to you Aur=E9lien for writing your report. >=20 > On 09/15/18 18:36, Aur=E9lien wrote: >> Hi, >> >> I don't know if it is the right place to report a bug, sorry if not. >> >> I'm a Promox user and the last version (5.2) has a package versionned >> 20180612-1 actually ( >> https://git.proxmox.com/?p=3Dpve-edk2-firmware.git;a=3Dsummary ) which= is >> based on 5a56c0493955cf55e7eef96dbba815cfbb067d7d commit of edk2. A >> build from the current master a few days ago has the same problem. The= >> previous Promox version (5.1 whith this pve-qemu-kvm package) use a >> edk2 version dating from 2017 or before (it could be from before >> September 2016 but I'm not sure), which has no problem. I tested on my= >> workstation running gentoo which has a version based on UDK2017 branch= >> with source code from February 11, 2018 and it's working too. So I >> think the problem is located in the OVMF_CODE.fd code from 2018 (but >> without any certainty). >> >> Actual behaviour (with all the various Linux distribution: Debian 9, >> Ubuntu 18.04, Centos 7.5 ...) with a VM which has EFI enabled: >> - grub is loaded and only see the first disk (where it is installed), >> the "ls" command in the grub shell return: "(hd0) (hd0,gpt1) >> (cd0)" and there is also hd1 and hd2 and in my case this is >> problematic as /boot is on hd1. >> - when pressing "escape" on the boot to go the the setup screen and >> just chose "continue" it's ok (grub sees all the disks), my current >> workaround to boot the installed OS. >> - when installing an OS like Ubuntu, CentOS ... on the first reboot >> it's ok (I thinks something in the installation, like seting a new >> boot entry with grub-install, efibootmgr) >> >> Expected behaviour: >> - grub see all the disk (the ls command return hd0, hd1 hd2 ....) and >> not the first one (where the EFI boot partition is located). >> (sorry, I haven't tried any other boot loader) >> >> - Step to reproduce: >> start a VM with 3 disks and EFI enabled (OVMF_CODE), install a Linux= >> on the 2nd disk (/ & swap) and the EFI boot partition on the first >> disk (/boot/efi on "sda1"), cold start the VM. >> >> I can help by testing with specific version (or another boot loader), >> tell me. >> >> Regards >> >> Aur=E9lien >> >> ps: typical command line to start the VM: >> /usr/bin/kvm \ >> -id 20 \ >> -name testvm.mydomain \ >> -chardev 'socket,id=3Dqmp,path=3D/var/run/qemu-server/20.qmp,server,= nowait' \ >> -mon 'chardev=3Dqmp,mode=3Dcontrol' \ >> -pidfile /var/run/qemu-server/20.pid \ >> -daemonize \ >> -smbios 'type=3D1,uuid=3D88dc4e6b-34b8-46a7-a573-26596c34855d' \ >> -drive 'if=3Dpflash,unit=3D0,format=3Draw,readonly,file=3D/usr/share= /pve-edk2-firmware//OVMF_CODE.fd' \ >> -drive 'if=3Dpflash,unit=3D1,format=3Draw,id=3Ddrive-efidisk0,file=3D= /mnt/pve/pve-nfs-sas/images/20/vm-20-disk-4.raw' \ >> -smp '8,sockets=3D1,cores=3D8,maxcpus=3D8' \ >> -nodefaults \ >> -boot 'menu=3Don,strict=3Don,reboot-timeout=3D1000,splash=3D/usr/sha= re/qemu-server/bootsplash.jpg' \ >> -vga qxl \ >> -vnc unix:/var/run/qemu-server/20.vnc,x509,password \ >> -cpu host,+kvm_pv_unhalt,+kvm_pv_eoi \ >> -m 512 \ >> -object 'memory-backend-ram,id=3Dram-node0,size=3D512M' \ >> -numa 'node,nodeid=3D0,cpus=3D0-7,memdev=3Dram-node0' \ >> -k fr \ >> -object 'iothread,id=3Diothread-virtioscsi1' \ >> -object 'iothread,id=3Diothread-virtioscsi2' \ >> -machine 'type=3Dpc-i440fx-2.9' \ >> -device 'pci-bridge,id=3Dpci.3,chassis_nr=3D3,bus=3Dpci.0,addr=3D0x5= ' \ >> -device 'pci-bridge,id=3Dpci.1,chassis_nr=3D1,bus=3Dpci.0,addr=3D0x1= e' \ >> -device 'pci-bridge,id=3Dpci.2,chassis_nr=3D2,bus=3Dpci.0,addr=3D0x1= f' \ >> -device 'piix3-usb-uhci,id=3Duhci,bus=3Dpci.0,addr=3D0x1.0x2' \ >> -chardev 'socket,path=3D/var/run/qemu-server/20.qga,server,nowait,id= =3Dqga0' \ >> -device 'virtio-serial,id=3Dqga0,bus=3Dpci.0,addr=3D0x8' \ >> -device 'virtserialport,chardev=3Dqga0,name=3Dorg.qemu.guest_agent.0= ' \ >> -spice 'tls-port=3D61004,addr=3D127.0.0.1,tls-ciphers=3DHIGH,seamles= s-migration=3Don' \ >> -device 'virtio-serial,id=3Dspice,bus=3Dpci.0,addr=3D0x9' \ >> -chardev 'spicevmc,id=3Dvdagent,name=3Dvdagent' \ >> -device 'virtserialport,chardev=3Dvdagent,name=3Dcom.redhat.spice.0'= \ >> -device 'virtio-balloon-pci,id=3Dballoon0,bus=3Dpci.0,addr=3D0x3' \ >> -iscsi 'initiator-name=3Diqn.1993-08.org.debian:01:f9e4631950da' \ >> -drive 'if=3Dnone,id=3Ddrive-ide2,media=3Dcdrom,aio=3Dthreads' \ >> -device 'ide-cd,bus=3Dide.1,unit=3D0,drive=3Ddrive-ide2,id=3Dide2,bo= otindex=3D100' \ >> -device 'virtio-scsi-pci,id=3Dvirtioscsi0,bus=3Dpci.3,addr=3D0x1' \ >> -drive 'file=3D/mnt/pve/pve-nfs-sas/images/20/vm-20-disk-1.raw,if=3D= none,id=3Ddrive-scsi0,format=3Draw,cache=3Dnone,aio=3Dnative,detect-zeroe= s=3Don' \ >> -device 'scsi-hd,bus=3Dvirtioscsi0.0,channel=3D0,scsi-id=3D0,lun=3D0= ,drive=3Ddrive-scsi0,id=3Dscsi0,bootindex=3D200' \ >> -device 'virtio-scsi-pci,id=3Dvirtioscsi1,bus=3Dpci.3,addr=3D0x2,iot= hread=3Diothread-virtioscsi1' \ >> -drive 'file=3D/mnt/pve/pve-nfs-sas/images/20/vm-20-disk-2.raw,if=3D= none,id=3Ddrive-scsi1,format=3Draw,cache=3Dnone,aio=3Dnative,detect-zeroe= s=3Don' \ >> -device 'scsi-hd,bus=3Dvirtioscsi1.0,channel=3D0,scsi-id=3D0,lun=3D1= ,drive=3Ddrive-scsi1,id=3Dscsi1' \ >> -device 'virtio-scsi-pci,id=3Dvirtioscsi2,bus=3Dpci.3,addr=3D0x3,iot= hread=3Diothread-virtioscsi2' \ >> -drive 'file=3D/mnt/pve/pve-nfs-sas/images/20/vm-20-disk-3.raw,if=3D= none,id=3Ddrive-scsi2,format=3Draw,cache=3Dnone,aio=3Dnative,detect-zeroe= s=3Don' \ >> -device 'scsi-hd,bus=3Dvirtioscsi2.0,channel=3D0,scsi-id=3D0,lun=3D2= ,drive=3Ddrive-scsi2,id=3Dscsi2' \ >> -netdev 'type=3Dtap,id=3Dnet0,ifname=3Dtap20i0,script=3D/var/lib/qem= u-server/pve-bridge,downscript=3D/var/lib/qemu-server/pve-bridgedown,vhos= t=3Don' \ >> -device 'virtio-net-pci,mac=3D4A:63:6D:11:11:11,netdev=3Dnet0,bus=3D= pci.0,addr=3D0x12,id=3Dnet0' >=20 > This is a very good issue report, thank you for it. The amount of detai= l > you provided allows me to explain the status & behavior to you. >=20 > (1) Originally, OVMF used to connect all drivers to all devices in the > system. In a sense this was convenient, however it resulted in a lot of= > wasted work too -- if you had tens or hundreds of devices, the boot > could take extremely long. And again, most of that work was wasted, > since you almost never want to actually attempt booting off of tens or > hundreds of devices. You only really want to connect those devices that= > you reasonably want to attempt booting off of as well. >=20 > (In the above, "connect" is used in the sense "bind" or "probe". Esp. > with a Linux background, "probe" is the most fitting word, probably.) >=20 >=20 > (2) Because the set of devices connected at boot is platform policy > (i.e., UEFI platforms are at liberty to decide what devices they > connect), we changed the above behavior to a minimalistic approach, in > the following patch series: >=20 > [edk2] [PATCH 0/6] OvmfPkg, ArmVirtQemu: leaner platform BDS policy f= or connecting devices > http://mid.mail-archive.com/20180313212233.19215-1-lersek@redhat.com >=20 > This was then pushed as commit range 12957e56d26d..ff1d0fbfbaec, in > March 2018. >=20 > With this set in place, if you provided *some* devices with the > 'bootindex=3DN' property, then OVMF would no longer connect all devices= in > the system, it would connect only those that you marked with the > bootindex property. (And then OVMF would auto-generate UEFI boot option= s > for those only, as well.) >=20 > In other words, the bootindex property was promoted from its previous > "boot option filtering/reordering only" role to a "steering of device > probing" role as well. >=20 Ah yes, I should have figured that out, I even mentioned the faster boot on multiple devices in the package changelog but never wondered about wha= t this implies for setups like Aur=E9lien has, argh! >=20 > (3) The EFI build of grub most likely doesn't find your disks -- after > (2) -- because it doesn't connect devices itself, it relies on the > platform firmware to connect disks. >=20 >=20 > (4) If you interrupt the boot process and enter the setup utility, the > latter will connect all devices, regardless of (2). This is your curren= t > workaround, IIUC. >=20 >=20 > (5) There are many classes of devices, and some of those should be > connected even if they are not "bootable" at all. The series in (2) > caused a regression for them, and we had to fix them up gradually. See > the following commits: >=20 > - OvmfPkg/PlatformBootManagerLib: add USB keyboard to ConIn > - OvmfPkg/PlatformBootManagerLib: connect consoles unconditionally > - OvmfPkg/PlatformBootManagerLib: connect Virtio RNG devices again >=20 > However, the disks that you mention are *not* like this; they should > indeed *not* be connected unless you mark them for booting. >=20 >=20 > (6) The solution is either to instruct grub (I don't know how) to > connect the disks in question itself, *or* to add the "bootindex=3DN" > properties (with some suitable N values) to the disk devices that you > want OVMF to connect for you (even if they contain no EFI system > partition and/or a UEFI boot loader application). Given your current > command line, the latter option translates to the "scsi-hd" devices wit= h > identifiers "scsi1" and "scsi2". For example: >=20 > -device scsi-hd,bus=3Dvirtioscsi1.0,channel=3D0,scsi-id=3D0,lun=3D1,d= rive=3Ddrive-scsi1,id=3Dscsi1,bootindex=3D201 \ > -device scsi-hd,bus=3Dvirtioscsi2.0,channel=3D0,scsi-id=3D0,lun=3D2,d= rive=3Ddrive-scsi2,id=3Dscsi2,bootindex=3D202 \ So we probably want to add a per-disk 'bootable' flag in our frontend, which adds a higher bootindex (=3D lower priority) to all marked disks. This would allow to still boot faster with lots of disks but allows setups like yours, Aur=E9lien. >=20 > Hope this helps, Yes, it helps a lot! Much thanks for your explanation, appreciated! cheers, Thomas