From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=66.187.233.73; helo=mx1.redhat.com; envelope-from=dgilbert@redhat.com; receiver=edk2-devel@lists.01.org Received: from mx1.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 22E2C20955F3E for ; Fri, 2 Mar 2018 04:47:46 -0800 (PST) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id AFDAE4036114; Fri, 2 Mar 2018 12:53:53 +0000 (UTC) Received: from work-vm (ovpn-117-242.ams2.redhat.com [10.36.117.242]) by smtp.corp.redhat.com (Postfix) with ESMTPS id AA13F1C735; Fri, 2 Mar 2018 12:53:48 +0000 (UTC) Date: Fri, 2 Mar 2018 12:53:46 +0000 From: "Dr. David Alan Gilbert" To: Laszlo Ersek Cc: Thomas Lamprecht , qemu-discuss@nongnu.org, edk2-devel-01 Message-ID: <20180302125345.GA8948@work-vm> References: <96f46413-6eb6-21f4-d07c-89aebea6f233@proxmox.com> <03691be2-0fcd-ace6-2ec9-d005f85782c3@redhat.com> MIME-Version: 1.0 In-Reply-To: <03691be2-0fcd-ace6-2ec9-d005f85782c3@redhat.com> User-Agent: Mutt/1.9.2 (2017-12-15) X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Fri, 02 Mar 2018 12:53:54 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Fri, 02 Mar 2018 12:53:54 +0000 (UTC) for IP:'10.11.54.5' DOMAIN:'int-mx05.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'dgilbert@redhat.com' RCPT:'' X-Mailman-Approved-At: Fri, 02 Mar 2018 08:44:56 -0800 Subject: Re: How to handle pflash backed OVMF FW upgrade and live migration best? X-BeenThere: edk2-devel@lists.01.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: EDK II Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Mar 2018 12:47:46 -0000 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline * Laszlo Ersek (lersek@redhat.com) wrote: > CC Dave > > On 03/01/18 12:21, Thomas Lamprecht wrote: > > Hi, > > > > I'm currently evaluating how to update the firmware (OVMF) code image > > without impacting a KVM/QEMU VM on live migration. I.e., the FW code lives > > under /usr/share/OVMF/OVMF_CODE.fd and gets passed to the QEMU command with: > > > > qemu-binary [...] -drive "if=pflash,unit=0,format=raw,readonly,file=/usr/share/OVMF/OVMF_CODE.fd" > > > > Now if the target node has an updated version of OVMF the VM does not really > > likes that, as from its POV it gets effectively another code image loaded > > from one moment at the other without any notice. > > This should not cause any issues. On the destination host, the > destination QEMU instance should load the (different) OVMF_CODE.fd image > into the pflash chip, at startup. However, the incoming migration stream > contains, as a RAMBlock, the original OVMF_CODE.fd image. In other > words, the original firmware image is migrated (in memory, as part of > the migration stream) too. Yep. > ( > > BTW, there is very little firmware code in OVMF that actually *executes* > from pflash -- that's just the SEC module. SEC decompresses the PEI and > DXE firmware volumes from pflash to RAM, and the rest of the firmware > runs from normal RAM. This applies to runtime firmware services as well. > So about the only times when OVMF_CODE.fd (in the pflash chip) and > migration intersect are: > - if you migrate while SEC is running from pflash (i.e. the earliest > part of the boot), > - if you warm-reboot on the destination host after migration -- in this > case, the OVMF_CODE.fd binary (that got migrated in the pflash RAMBlock > from the source host) will again boot from pflash. > > ) > > > So my questions is if it would make sense to see this read-only pflash > > content as "VM state" and send it over during live migration? > > That's what already happens. > > Now, if you have a differently *sized* OVMF_CODE.fd image on the > destination host, that could be a problem. Avoiding such problems is an > IT / distro job. Yeh; padding the binaries to a power-of-2 with a bit of space left is a good rule of thumb. > There are some "build aspects" of OVMF that can make two OVMF binaries > "incompatible" in this sense. Using *some* different build flags it's > also possible to make (a) an OVMF binary and (b) a varstore file > originally created for another OVMF binary, incompatible. Fun. > > This would > > make migration way easier. Else we need to save all FW files and track which > > one the VM is using, so that when starting the migration target VM we pass > > along the correct pflash drive file. Sending over a pflash drive could maybe > > only get done when a special flag is set for the pflash drive? > > > > As said I can work around in our management stack, but saving the FW image > > and tracking which VM uses what version, and that cluster wide, may get > > quite a headache and we would need to keep all older OVMF binaries around... > > When you deploy new OVMF binaries (packages) to a subset of your > virtualization hosts, you are responsible for keeping those compatible. > (They *can* contain code updates, but those updates have to be > compatible.) If a new OVMF binary is built that is known to be > incompatible, then it has to be installed under a different pathname > (either via a separate package; or in the same package, but still under > a different pathname). > > To give you the simplest example, binaries (and varstores) corresponding > to FD_SIZE_2MB and FD_SIZE_4MB are incompatible. If a domain is > originally defined on top of an FD_SIZE_2MB OVMF, then it likely cannot > be migrated to a host where the same OVMF pathname refers to an > FD_SIZE_4MB binary. If you have a mixed environment, then you need to > carry both binaries to all hosts (and if you backport fixes from > upstream edk2, you need to backport those to both binaries). > > In addition, assuming the domain is powered down for good (the QEMU > process terminates), and you update the domain XML from the FD_SIZE_2MB > OVMF binary to the FD_SIZE_4MB binary, you *also* have to > delete/recreate the domain's variable store file (losing all UEFI > variables the domain has accumulated until then). This is because the > FD_SIZE_4MB binary is incompatible with the varstore that was originally > created for the FD_SIZE_2MB binary (and vice versa). I assume that gives a clear and obvious error message - right? Dave > Thanks > Laszlo > > > If I'm missing something and there's already an easy way for this I'd be > > very happy to hear from it. > > > > Besides qemu-discuss I posted it to edk2-devel as there maybe more people > > are in the QEMU and OVMF user intersection. :) -- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK