Re: How to handle pflash backed OVMF FW upgrade and live migration best?

["Dr. David Alan Gilbert" <dgilbert@redhat.com>]

* Laszlo Ersek (lersek@redhat.com) wrote:
> CC Dave
> 
> On 03/01/18 12:21, Thomas Lamprecht wrote:
> > Hi,
> > 
> > I'm currently evaluating how to update the firmware (OVMF) code image
> > without impacting a KVM/QEMU VM on live migration. I.e., the FW code lives
> > under /usr/share/OVMF/OVMF_CODE.fd and gets passed to the QEMU command with:
> > 
> > qemu-binary [...] -drive "if=pflash,unit=0,format=raw,readonly,file=/usr/share/OVMF/OVMF_CODE.fd"
> > 
> > Now if the target node has an updated version of OVMF the VM does not really
> > likes that, as from its POV it gets effectively another code image loaded
> > from one moment at the other without any notice.
> 
> This should not cause any issues. On the destination host, the
> destination QEMU instance should load the (different) OVMF_CODE.fd image
> into the pflash chip, at startup. However, the incoming migration stream
> contains, as a RAMBlock, the original OVMF_CODE.fd image. In other
> words, the original firmware image is migrated (in memory, as part of
> the migration stream) too.

Yep.

> (
> 
> BTW, there is very little firmware code in OVMF that actually *executes*
> from pflash -- that's just the SEC module. SEC decompresses the PEI and
> DXE firmware volumes from pflash to RAM, and the rest of the firmware
> runs from normal RAM. This applies to runtime firmware services as well.
> So about the only times when OVMF_CODE.fd (in the pflash chip) and
> migration intersect are:
> - if you migrate while SEC is running from pflash (i.e. the earliest
> part of the boot),
> - if you warm-reboot on the destination host after migration -- in this
> case, the OVMF_CODE.fd binary (that got migrated in the pflash RAMBlock
> from the source host) will again boot from pflash.
> 
> )
> 
> > So my questions is if it would make sense to see this read-only pflash
> > content as "VM state" and send it over during live migration?
> 
> That's what already happens.
> 
> Now, if you have a differently *sized* OVMF_CODE.fd image on the
> destination host, that could be a problem. Avoiding such problems is an
> IT / distro job.

Yeh; padding the binaries to a power-of-2 with a bit of space left is
a good rule of thumb.

> There are some "build aspects" of OVMF that can make two OVMF binaries
> "incompatible" in this sense. Using *some* different build flags it's
> also possible to make (a) an OVMF binary and (b) a varstore file
> originally created for another OVMF binary, incompatible.

Fun.

> > This would
> > make migration way easier. Else we need to save all FW files and track which
> > one the VM is using, so that when starting the migration target VM we pass
> > along the correct pflash drive file. Sending over a pflash drive could maybe
> > only get done when a special flag is set for the pflash drive?
> > 
> > As said I can work around in our management stack, but saving the FW image
> > and tracking which VM uses what version, and that cluster wide, may get
> > quite a headache and we would need to keep all older OVMF binaries around...
> 
> When you deploy new OVMF binaries (packages) to a subset of your
> virtualization hosts, you are responsible for keeping those compatible.
> (They *can* contain code updates, but those updates have to be
> compatible.) If a new OVMF binary is built that is known to be
> incompatible, then it has to be installed under a different pathname
> (either via a separate package; or in the same package, but still under
> a different pathname).
> 
> To give you the simplest example, binaries (and varstores) corresponding
> to FD_SIZE_2MB and FD_SIZE_4MB are incompatible. If a domain is
> originally defined on top of an FD_SIZE_2MB OVMF, then it likely cannot
> be migrated to a host where the same OVMF pathname refers to an
> FD_SIZE_4MB binary. If you have a mixed environment, then you need to
> carry both binaries to all hosts (and if you backport fixes from
> upstream edk2, you need to backport those to both binaries).
> 
> In addition, assuming the domain is powered down for good (the QEMU
> process terminates), and you update the domain XML from the FD_SIZE_2MB
> OVMF binary to the FD_SIZE_4MB binary, you *also* have to
> delete/recreate the domain's variable store file (losing all UEFI
> variables the domain has accumulated until then). This is because the
> FD_SIZE_4MB binary is incompatible with the varstore that was originally
> created for the FD_SIZE_2MB binary (and vice versa).

I assume that gives a clear and obvious error message - right?

Dave

> Thanks
> Laszlo
> 
> > If I'm missing something and there's already an easy way for this I'd be
> > very happy to hear from it.
> > 
> > Besides qemu-discuss I posted it to edk2-devel as there maybe more people
> > are in the QEMU and OVMF user intersection. :)
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK