From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from us-smtp-1.mimecast.com (us-smtp-1.mimecast.com [207.211.31.120]) by mx.groups.io with SMTP id smtpd.web11.1236.1582663218968398507 for ; Tue, 25 Feb 2020 12:40:19 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=Ic2bgUsX; spf=pass (domain: redhat.com, ip: 207.211.31.120, mailfrom: lersek@redhat.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1582663218; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2WCWKphiBS0xj7Sl+tfjRPksqThPH1Gt4fZQI3V0OgU=; b=Ic2bgUsXnZMXNDU80+0n0ZJHWhOKPIV9Q3CftwsdcPtUdyn48KAg5+1+ED/sRAEk91FmY3 YL5Isd4vJM7Iw5phYm1/6nBeFEOHbl8luV2j0/5i7jiSO5INeXc7PNoUG7VsdlJ5II5sff Q574nnHOAaHk0I9xe5DlQgVjd3UC2Ts= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-149-LGZ7GUFMMUqsum40kIDNYA-1; Tue, 25 Feb 2020 15:40:08 -0500 X-MC-Unique: LGZ7GUFMMUqsum40kIDNYA-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 84FBDDB21; Tue, 25 Feb 2020 20:40:06 +0000 (UTC) Received: from lacos-laptop-7.usersys.redhat.com (ovpn-117-104.ams2.redhat.com [10.36.117.104]) by smtp.corp.redhat.com (Postfix) with ESMTP id B9A255C13D; Tue, 25 Feb 2020 20:40:01 +0000 (UTC) Subject: Re: [edk2-devel] A problem with live migration of UEFI virtual machines To: Andrew Fish , devel@edk2.groups.io Cc: wuchenye1995 , zhoujianjay , =?UTF-8?Q?Alex_Benn=c3=a9e?= , berrange@redhat.com, "Dr. David Alan Gilbert" , qemu-devel@nongnu.org, discuss References: <87sgjhxbtc.fsf@zen.linaroharston> <20200224152810.GX635661@redhat.com> <8b0ec286-9322-ee00-3729-6ec7ee8260a6@redhat.com> <3E8BB07B-8730-4AB8-BCB6-EA183FB589C5@apple.com> From: "Laszlo Ersek" Message-ID: <465a5a84-cac4-de39-8956-e38771807450@redhat.com> Date: Tue, 25 Feb 2020 21:40:00 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: <3E8BB07B-8730-4AB8-BCB6-EA183FB589C5@apple.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi Andrew, On 02/25/20 19:56, Andrew Fish wrote: > Laszlo, >=20 > If I understand this correctly is it not more complicated than just size= . It also assumes the memory layout is the same? Yes. > The legacy BIOS used fixed magic address ranges, but UEFI uses dynamical= ly allocated memory so addresses are not fixed. While the UEFI firmware doe= s try to keep S3 and S4 layouts consistent between boots, I'm not aware of = any mechanism to keep the memory map address the same between versions of t= he firmware? It's not about RAM, but platform MMIO. The core of the issue here is that the -D FD_SIZE_4MB and -D FD_SIZE_2MB build options (or more directly, the different FD_SIZE_IN_KB macro settings) set a bunch of flash-related build-time constant macros, and PCDs, differently, in the following files: - OvmfPkg/OvmfPkg.fdf.inc - OvmfPkg/VarStore.fdf.inc - OvmfPkg/OvmfPkg*.dsc As a result, the OVMF_CODE.fd firmware binary will have different hard-coded references to the variable store pflash addresses. (Guest-physical MMIO addresses that point into the pflash range.) If someone tries to combine an OVMF_CODE.fd firmware binary from e.g. the 4MB build, with a variable store file that was originally instantiated from an OVMF_VARS.fd varstore template from the 2MB build, then the firmware binary's physical address references and various size references will not match the contents / layout of the varstore pflash chip, which maps an incompatibly structured varstore file. For example, "OvmfPkg/VarStore.fdf.inc" describes two incompatible EFI_FIRMWARE_VOLUME_HEADER structures (which "build" generates for the OVMF_VARS.fd template) between the 4MB (total size) build, and the 1MB/2MB (total size) build. The commit message below summarizes the internal layout differences, from 1MB/2MB -> 4MB: https://github.com/tianocore/edk2/commit/b24fca05751f Excerpt (relevant for OVMF_VARS.fd): Description Compression type Size [KB] ------------------------- ----------------- ---------------------- Non-volatile data storage open-coded binary 128 -> 528 ( +400) data Variable store 56 -> 256 ( +200) Event log 4 -> 4 ( +0) Working block 4 -> 4 ( +0) Spare area 64 -> 264 ( +200) Thanks Laszlo >> On Feb 25, 2020, at 9:53 AM, Laszlo Ersek wrote: >> >> On 02/24/20 16:28, Daniel P. Berrang=C3=A9 wrote: >>> On Tue, Feb 11, 2020 at 05:39:59PM +0000, Alex Benn=C3=A9e wrote: >>>> >>>> wuchenye1995 writes: >>>> >>>>> Hi all, >>>>> We found a problem with live migration of UEFI virtual machines >>>>> due to size of OVMF.fd changes. >>>>> Specifically, the size of OVMF.fd in edk with low version such as >>>>> edk-2.0-25 is 2MB while the size of it in higher version such as >>>>> edk-2.0-30 is 4MB. >>>>> When we migrate a UEFI virtual machine from the host with low >>>>> version of edk2 to the host with higher one, qemu component will >>>>> report an error in function qemu_ram_resize while >>>>> checking size of ovmf_pcbios: Length mismatch: pc.bios: 0x200000 in >>>>> !=3D 0x400000: Invalid argument. >>>>> We want to know how to solve this problem after updating the >>>>> version of edk2. >>>> >>>> You can only migrate a machine that is identical - so instantiating a >>>> empty machine with a different EDK image is bound to cause a problem >>>> because the machines don't match. >>> >>> I don't believe we are that strict for firmware in general. The >>> firmware is loaded when QEMU starts, but that only matters for the >>> original source host QEMU. During migration, the memory content of the >>> original firmware will be copied during live migration, overwriting >>> whatever the target QEMU loaded off disk. This works....provided the >>> memory region is the same size on source & target host, which is where >>> the problem arises in this case. >>> >>> If there's a risk that newer firmware will be larger than old firmware >>> there's only really two options: >>> >>> - Keep all firmware images forever, each with a unique versioned >>> filename. This ensures target QEMU will always load the original >>> smaller firmware >>> >>> - Add padding to the firmware images. IOW, if the firmware is 2 MB, >>> add zero-padding to the end of the image to round it upto 4 MB >>> (whatever you anticipate the largest size wil be in future). >>> >>> Distros have often taken the latter approach for QEMU firmware in the >>> past. The main issue is that you have to plan ahead of time and get >>> this padding right from the very start. You can't add the padding >>> after the fact on an existing VM. >> >> Following up here *too*, just for completeness. >> >> The query in this thread has been posted three times now (and I have >> zero idea why). Each time it generated a different set of responses. Fo= r >> completes, I'm now going to link the other two threads here (because th= e >> present thread seems to have gotten the most feedback). >> >> To the OP: >> >> - please do *NOT* repost the same question once you get an answer. It >> only fragments the discussion and creates confusion. It also doesn't >> hurt if you *confirm* that you understood the answer. >> >> - Yet further, if your email address has @gmail.com for domain, but you= r >> msgids contain "tencent", that raises some eyebrows (mine for sure). >> You say "we" in the query, but never identify the organization behind >> the plural pronoun. >> >> (I've been fuming about the triple-posting of the question for a while >> now, but it's only now that, upon seeing how much work Dan has put into >> his answer, I've decided that dishing out a bit of netiquette would be >> in order.) >> >> * First posting: >> - msgid: > >> - edk2-devel: https://edk2.groups.io/g/devel/message/54146 >> - qemu-devel: https://lists.gnu.org/archive/html/qemu-devel/2020-02/msg= 02419.html >> >> * my response: >> - msgid: <12553.1581366059422195003@groups.io > >> - edk2-devel: https://edk2.groups.io/g/devel/message/54161 >> - qemu-devel: none, because (as an exception) I used the stupid >> groups.io web interface to respond= , and so my response >> never reached qemu-devel >> >> * Second posting (~4 hours after the first) >> - msgid: > >> - edk2-devel: https://edk2.groups.io/g/devel/message/54147 >> - qemu-devel: https://lists.gnu.org/archive/html/qemu-devel/2020-02/msg= 02415.html >> >> * Dave's response: >> - msgid: <20200220154742.GC2882@work-vm> >> - edk2-devel: https://edk2.groups.io/g/devel/message/54681 >> - qemu-devel: https://lists.gnu.org/archive/html/qemu-devel/2020-02/= msg05632.html >> >> * Third posting (next day, present thread) -- cross posted to yet >> another list (!), because apparently Dave's feedback and mine had not >> been enough: >> - msgid: > >> - edk2-devel: https://edk2.groups.io/g/devel/message/54220 >> - edk2-discuss: https://edk2.groups.io/g/discuss/message/135 >> - qemu-devel: https://lists.gnu.org/archive/html/qemu-devel/2020-02/m= sg02735.html >> >> Back on topic: see my response again. The answer is, you can't solve th= e >> problem (specifically with OVMF), and QEMU in fact does you service by >> preventing the migration. >> >> Laszlo >> >> >>=20 >=20 >=20