From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from ma1-aaemail-dr-lapp03.apple.com (ma1-aaemail-dr-lapp03.apple.com [17.171.2.72]) by mx.groups.io with SMTP id smtpd.web12.664.1582656997191249683 for ; Tue, 25 Feb 2020 10:56:38 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@apple.com header.s=20180706 header.b=I24odrpw; spf=pass (domain: apple.com, ip: 17.171.2.72, mailfrom: afish@apple.com) Received: from pps.filterd (ma1-aaemail-dr-lapp03.apple.com [127.0.0.1]) by ma1-aaemail-dr-lapp03.apple.com (8.16.0.27/8.16.0.27) with SMTP id 01PIqMku047386; Tue, 25 Feb 2020 10:56:35 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=apple.com; h=sender : from : message-id : content-type : mime-version : subject : date : in-reply-to : cc : to : references; s=20180706; bh=zLkbU4y8JpOgYkoY+ETmf3UoUeTmxcL7qfoDHOvHCAg=; b=I24odrpwNIEoBLFotlz3BcGvaATlpKwHOLqJh6fkcjxRfgOHBtxGgFlZqRMkLpbuz4pm B041NkrZfzzRta7wmJnb6US4OnU36fuxInrTGP1cqFAGaC0jZTRjuj5KW/aXWhadi2b0 1e2uairlKdwojA7jdEFBas0hrnAXqJyKgOhJZ88dQBZZ6kYbb4vnesofH7dGxpdRVyqm apr+cOkdMnNrKv1k8f+B8XiSwCkfQrjm8gey/PP3UO6YyCBZdo4AGKtMfpcGtUiex/Pp O7/MW3ruGqi+vl3NqRgdYndR31ZZwa7vdFIX+9RsLD6aQWPXe6Eb34VkeRLNEQWf3PjY Qw== Received: from rn-mailsvcp-mta-lapp04.rno.apple.com (rn-mailsvcp-mta-lapp04.rno.apple.com [10.225.203.152]) by ma1-aaemail-dr-lapp03.apple.com with ESMTP id 2yb462v4m8-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Tue, 25 Feb 2020 10:56:35 -0800 Received: from rn-mailsvcp-mmp-lapp02.rno.apple.com (rn-mailsvcp-mmp-lapp02.rno.apple.com [17.179.253.15]) by rn-mailsvcp-mta-lapp04.rno.apple.com (Oracle Communications Messaging Server 8.1.0.1.20190704 64bit (built Jul 4 2019)) with ESMTPS id <0Q6900ZBFTYAGD10@rn-mailsvcp-mta-lapp04.rno.apple.com>; Tue, 25 Feb 2020 10:56:34 -0800 (PST) Received: from process_milters-daemon.rn-mailsvcp-mmp-lapp02.rno.apple.com by rn-mailsvcp-mmp-lapp02.rno.apple.com (Oracle Communications Messaging Server 8.1.0.1.20190704 64bit (built Jul 4 2019)) id <0Q6901000TS2QC00@rn-mailsvcp-mmp-lapp02.rno.apple.com>; Tue, 25 Feb 2020 10:56:34 -0800 (PST) X-Va-A: X-Va-T-CD: 08777febe38bb384cc57fda39d0586b7 X-Va-E-CD: 74fbc9fcbd3d4b0e941105e5641a1eeb X-Va-R-CD: 7f28ace2b24f1e656a1dc26e6a401e24 X-Va-CD: 0 X-Va-ID: 5ce502b8-9844-4c46-ab4a-f161e4e7df8a X-V-A: X-V-T-CD: 08777febe38bb384cc57fda39d0586b7 X-V-E-CD: 74fbc9fcbd3d4b0e941105e5641a1eeb X-V-R-CD: 7f28ace2b24f1e656a1dc26e6a401e24 X-V-CD: 0 X-V-ID: 7556cd4c-9a7f-409a-8087-f733f2c67c5d X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138,18.0.572 definitions=2020-02-25_07:2020-02-25,2020-02-25 signatures=0 Received: from [17.235.17.79] by rn-mailsvcp-mmp-lapp02.rno.apple.com (Oracle Communications Messaging Server 8.1.0.1.20190704 64bit (built Jul 4 2019)) with ESMTPSA id <0Q69002JUTY63R20@rn-mailsvcp-mmp-lapp02.rno.apple.com>; Tue, 25 Feb 2020 10:56:34 -0800 (PST) Sender: afish@apple.com From: "Andrew Fish" Message-id: <3E8BB07B-8730-4AB8-BCB6-EA183FB589C5@apple.com> MIME-version: 1.0 (Mac OS X Mail 13.0 \(3594.4.17\)) Subject: Re: [edk2-devel] A problem with live migration of UEFI virtual machines Date: Tue, 25 Feb 2020 10:56:30 -0800 In-reply-to: <8b0ec286-9322-ee00-3729-6ec7ee8260a6@redhat.com> Cc: wuchenye1995 , zhoujianjay , =?utf-8?Q?Alex_Benn=C3=A9e?= , berrange@redhat.com, "Dr. David Alan Gilbert" , qemu-devel@nongnu.org, discuss To: devel@edk2.groups.io, lersek@redhat.com References: <87sgjhxbtc.fsf@zen.linaroharston> <20200224152810.GX635661@redhat.com> <8b0ec286-9322-ee00-3729-6ec7ee8260a6@redhat.com> X-Mailer: Apple Mail (2.3594.4.17) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2020-02-25_07:,, signatures=0 Content-type: multipart/alternative; boundary="Apple-Mail=_6814FC48-F933-49FD-A21B-5F491265B587" --Apple-Mail=_6814FC48-F933-49FD-A21B-5F491265B587 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Laszlo, If I understand this correctly is it not more complicated than just size. = It also assumes the memory layout is the same? The legacy BIOS used fixed m= agic address ranges, but UEFI uses dynamically allocated memory so addresse= s are not fixed. While the UEFI firmware does try to keep S3 and S4 layouts= consistent between boots, I'm not aware of any mechanism to keep the memor= y map address the same between versions of the firmware?=20 Thanks, Andrew Fish > On Feb 25, 2020, at 9:53 AM, Laszlo Ersek wrote: >=20 > On 02/24/20 16:28, Daniel P. Berrang=C3=A9 wrote: >> On Tue, Feb 11, 2020 at 05:39:59PM +0000, Alex Benn=C3=A9e wrote: >>>=20 >>> wuchenye1995 writes: >>>=20 >>>> Hi all, >>>> We found a problem with live migration of UEFI virtual machines >>>> due to size of OVMF.fd changes. >>>> Specifically, the size of OVMF.fd in edk with low version such as >>>> edk-2.0-25 is 2MB while the size of it in higher version such as >>>> edk-2.0-30 is 4MB. >>>> When we migrate a UEFI virtual machine from the host with low >>>> version of edk2 to the host with higher one, qemu component will >>>> report an error in function qemu_ram_resize while >>>> checking size of ovmf_pcbios: Length mismatch: pc.bios: 0x200000 in >>>> !=3D 0x400000: Invalid argument. >>>> We want to know how to solve this problem after updating the >>>> version of edk2. >>>=20 >>> You can only migrate a machine that is identical - so instantiating a >>> empty machine with a different EDK image is bound to cause a problem >>> because the machines don't match. >>=20 >> I don't believe we are that strict for firmware in general. The >> firmware is loaded when QEMU starts, but that only matters for the >> original source host QEMU. During migration, the memory content of the >> original firmware will be copied during live migration, overwriting >> whatever the target QEMU loaded off disk. This works....provided the >> memory region is the same size on source & target host, which is where >> the problem arises in this case. >>=20 >> If there's a risk that newer firmware will be larger than old firmware >> there's only really two options: >>=20 >> - Keep all firmware images forever, each with a unique versioned >> filename. This ensures target QEMU will always load the original >> smaller firmware >>=20 >> - Add padding to the firmware images. IOW, if the firmware is 2 MB, >> add zero-padding to the end of the image to round it upto 4 MB >> (whatever you anticipate the largest size wil be in future). >>=20 >> Distros have often taken the latter approach for QEMU firmware in the >> past. The main issue is that you have to plan ahead of time and get >> this padding right from the very start. You can't add the padding >> after the fact on an existing VM. >=20 > Following up here *too*, just for completeness. >=20 > The query in this thread has been posted three times now (and I have > zero idea why). Each time it generated a different set of responses. For > completes, I'm now going to link the other two threads here (because the > present thread seems to have gotten the most feedback). >=20 > To the OP: >=20 > - please do *NOT* repost the same question once you get an answer. It > only fragments the discussion and creates confusion. It also doesn't > hurt if you *confirm* that you understood the answer. >=20 > - Yet further, if your email address has @gmail.com for domain, but your > msgids contain "tencent", that raises some eyebrows (mine for sure). > You say "we" in the query, but never identify the organization behind > the plural pronoun. >=20 > (I've been fuming about the triple-posting of the question for a while > now, but it's only now that, upon seeing how much work Dan has put into > his answer, I've decided that dishing out a bit of netiquette would be > in order.) >=20 > * First posting: > - msgid: > > - edk2-devel: https://edk2.groups.io/g/devel/message/54146 > - qemu-devel: https://lists.gnu.org/archive/html/qemu-devel/2020-02/msg0= 2419.html >=20 > * my response: > - msgid: <12553.1581366059422195003@groups.io > > - edk2-devel: https://edk2.groups.io/g/devel/message/54161 > - qemu-devel: none, because (as an exception) I used the stupid > groups.io web interface to respond,= and so my response > never reached qemu-devel >=20 > * Second posting (~4 hours after the first) > - msgid: > > - edk2-devel: https://edk2.groups.io/g/devel/message/54147 > - qemu-devel: https://lists.gnu.org/archive/html/qemu-devel/2020-02/msg0= 2415.html >=20 > * Dave's response: > - msgid: <20200220154742.GC2882@work-vm> > - edk2-devel: https://edk2.groups.io/g/devel/message/54681 > - qemu-devel: https://lists.gnu.org/archive/html/qemu-devel/2020-02/m= sg05632.html >=20 > * Third posting (next day, present thread) -- cross posted to yet > another list (!), because apparently Dave's feedback and mine had not > been enough: > - msgid: > > - edk2-devel: https://edk2.groups.io/g/devel/message/54220 > - edk2-discuss: https://edk2.groups.io/g/discuss/message/135 > - qemu-devel: https://lists.gnu.org/archive/html/qemu-devel/2020-02/ms= g02735.html >=20 > Back on topic: see my response again. The answer is, you can't solve the > problem (specifically with OVMF), and QEMU in fact does you service by > preventing the migration. >=20 > Laszlo >=20 >=20 >=20 --Apple-Mail=_6814FC48-F933-49FD-A21B-5F491265B587 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 Laszlo,
If I understand this correctly is it not= more complicated than just size. It also assumes the memory layout is the = same? The legacy BIOS used fixed magic address ranges, but UEFI uses dynami= cally allocated memory so addresses are not fixed. While the UEFI firmware = does try to keep S3 and S4 layouts consistent between boots, I'm not aware = of any mechanism to keep the memory map address the same between versions o= f the firmware? 

Thanks,

An= drew Fish

On Feb 25, 2020, at 9:53 AM, Laszlo Ersek <lersek@redhat.com> wrote:<= /div>
On 02/2= 4/20 16:28, Daniel P. Berrang=C3=A9 wrote:
On Tue, Feb 11, 2020 at 05:39:59PM +0000, = Alex Benn=C3=A9e wrote:
=
wuchenye1995 <wuchenye1995@gmail.com> writes:

Hi all,
  W= e found a problem with live migration of UEFI virtual machines
  due to size of OVMF.fd changes.
  Spe= cifically, the size of OVMF.fd in edk with low version such as
  edk-2.0-25 is 2MB while the size of it in higher version such= as
  edk-2.0-30 is 4MB.
  = When we migrate a UEFI virtual machine from the host with low
  version of edk2 to the host with higher one, qemu component wi= ll
  report an error in function qemu_ram_resize wh= ile
checking size of ovmf_pcbios: Length mismatch: pc.bios: 0= x200000 in
!=3D 0x400000: Invalid argument.
&nb= sp; We want to know how to solve this problem after updating the
  version of edk2.

You can only migrate a machine that is identical - so instantiating = a
empty machine with a different EDK image is bound to cause = a problem
because the machines don't match.

I don't believe we are that strict for firmware in= general. The
firmware is loaded when QEMU starts, but that o= nly matters for the
original source host QEMU. During migrati= on, the memory content of the
original firmware will be copie= d during live migration, overwriting
whatever the target QEMU= loaded off disk. This works....provided the
memory region is= the same size on source & target host, which is where
th= e problem arises in this case.

If there's a ri= sk that newer firmware will be larger than old firmware
there= 's only really two options:

 - Keep all f= irmware images forever, each with a unique versioned
 &n= bsp; filename. This ensures target QEMU will always load the original<= br class=3D"">   smaller firmware

 - Add padding to the firmware images. IOW, if the firmware is 2 MB= ,
   add zero-padding to the end of the image = to round it upto 4 MB
   (whatever you anticip= ate the largest size wil be in future).

Distro= s have often taken the latter approach for QEMU firmware in the
past. The main issue is that you have to plan ahead of time and get
this padding right from the very start. You can't add the padding=
after the fact on an existing VM.

Following up here *too*, jus= t for completeness.

The query in this thread has been pos= ted three times now (and I have
zero idea why). Each time it generated a different set of response= s. For
completes, I'm n= ow going to link the other two threads here (because the
present thread seems to have gotten the m= ost feedback).

To the OP:

- please do *NOT*= repost the same question once you get an answer. It
 only fragments the discussion and creat= es confusion. It also doesn't
=  hurt if you *confirm* that you understood the answer.
- Yet further, if your email address has @gmail.com for domain, but= your
 msgids cont= ain "tencent", that raises some eyebrows (mine for sure).
 You say "we" in the query, but = never identify the organization behind
 the plural pronoun.

(I've been fuming= about the triple-posting of the question for a while
now, but it's only now that, upon seeing how= much work Dan has put into
his answer, I've decided that dishing out a bit of netiquette would = be
in order.)
= * First posting:
- msgid:      <tencent_F1295F826E46EDFF3D77812B@qq.com>
- edk2-devel: https://edk2= .groups.io/g/devel/message/54146
= - qemu-devel: https://lists.gnu.org= /archive/html/qemu-devel/2020-02/msg02419.html

 * my resp= onse:
   = ;- msgid:      <12553.1581366059422195003@groups.io>
   - edk2-devel: htt= ps://edk2.groups.io/g/devel/message/54161
   - qemu-devel: none, because (as an except= ion) I used the stupid
=             &nb= sp;    groups.io web interface to respond, and so my response=
   &nbs= p;            &= nbsp;never reached qemu-devel
=
* Second posting (~4 hours af= ter the first)
- msgid:=      <ten= cent_3CD8845EC159F0161725898B@qq.com>
- edk2-devel: https://edk2.groups.io= /g/devel/message/54147
- q= emu-devel: https://lists.gnu.org/archive/= html/qemu-devel/2020-02/msg02415.html

 * Dave's respons= e:
   - = msgid:      <20200220154742.GC2882@work-vm><= /span>
   - edk= 2-devel: https://edk2.groups.io/g/devel/message/54681
   - qemu-devel: https://lists.gnu.org/archive/html/qemu-deve= l/2020-02/msg05632.html

* Third posting (next day, present th= read) -- cross posted to yet
<= span style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size:= 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; = letter-spacing: normal; text-align: start; text-indent: 0px; text-transform= : none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: = 0px; text-decoration: none; float: none; display: inline !important;" class= = =3D""> another list (!), because apparently Dave's feedback and mine = had not
 been enou= gh:
- msgid:  &nbs= p;     <te= ncent_BC7FD00363690990994E90F8@qq.com>
- edk2-devel:   https://edk2.groups.io/g/devel/message/54220
- edk2-discuss: https://edk= 2.groups.io/g/discuss/message/135
- qemu-devel:   https://lists.gnu.org/archive/html/qemu-devel/2020-02/msg0= 2735.html

Back on topic: see my response again. The answer is,= you can't solve the
pr= oblem (specifically with OVMF), and QEMU in fact does you service by=
preventing the migration.

Laszlo



--Apple-Mail=_6814FC48-F933-49FD-A21B-5F491265B587--