[edk2-devel] PR#5939 broke the legacy VM bootup

[edk2-devel] PR#5939 broke the legacy VM bootup

[Min Xu via groups.io]

[-- Attachment #1: Type: text/plain, Size: 2033 bytes --]

Hi, Crystal
PR#5939 (https://github.com/tianocore/edk2/pull/5939) was merged yesterday but it seems it broke the legacy VM bootup.

Below is the crash log of the failure.

Booting `CentOS Stream (6.2.0-6.2.19.2.52.x86_64) 9'

ConvertPages: range 1000000 - 5A27FFF covers multiple entries
!!!! X64 Exception Type - 0E(#PF - Page-Fault)  CPU Apic ID - 00000000 !!!!
ExceptionData - 0000000000000003  I:0 R:0 U:0 W:1 P:1 PK:0 SS:0 SGX:0
RIP  - 0000000074BAA1EE, CS  - 0000000000000038, RFLAGS - 0000000000210002
RAX  - 00000000739B4000, RCX - 000000007F9EC018, RDX - 000000007E0BE000
RBX  - 000000007E0BE000, RSP - 000000007FE6D0B0, RBP - 000000007FE6D200
RSI  - 000000007F9EC018, RDI - 000000007EC7D618
R8   - 00000000739B4390, R9  - 000000007BCEB6FD, R10 - 000000007DA55DB9
R11  - 0000000000000077, R12 - 000000007E0C3318, R13 - 000000000026A628
R14  - 000000007EC7D618, R15 - 000000007DFC3518
DS   - 0000000000000030, ES  - 0000000000000030, FS  - 0000000000000030
GS   - 0000000000000030, SS  - 0000000000000030
CR0  - 0000000080010033, CR2 - 0000000074BB1BE8, CR3 - 000000007FC01000
CR4  - 0000000000000668, CR8 - 0000000000000000
DR0  - 0000000000000000, DR1 - 0000000000000000, DR2 - 0000000000000000
DR3  - 0000000000000000, DR6 - 00000000FFFF0FF0, DR7 - 0000000000000400
GDTR - 000000007F9DC000 0000000000000047, LDTR - 0000000000000000
IDTR - 000000007F515018 0000000000000FFF,   TR - 0000000000000000
FXSAVE_STATE - 000000007FE6CD10
!!!! Find image based on IP(0x74BAA1EE)  Build/OvmfX64/DEBUG_GCC5/X64/OvmfPkg/VirtioRngDxe/VirtioRng/DEBUG/VirtioRngDxe.dll (ImageBase=0000000000D8EF34, EntryPoint=0000000000D90DF5) !

Thanks
Min


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#120986): https://edk2.groups.io/g/devel/message/120986
Mute This Topic: https://groups.io/mt/110601533/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-



[-- Attachment #2: Type: text/html, Size: 4741 bytes --]

Re: [edk2-devel] PR#5939 broke the legacy VM bootup

[Ard Biesheuvel via groups.io]

On Tue, 14 Jan 2025 at 03:54, Xu, Min M <min.m.xu@intel.com> wrote:
>
> Hi, Crystal
>
> PR#5939 (https://github.com/tianocore/edk2/pull/5939) was merged yesterday but it seems it broke the legacy VM bootup.
>
>
>
> Below is the crash log of the failure.
>

I take it this crash occurs after loading shim?


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#120988): https://edk2.groups.io/g/devel/message/120988
Mute This Topic: https://groups.io/mt/110601533/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] PR#5939 broke the legacy VM bootup

[Min Xu via groups.io]

On Tuesday, January 14, 2025 4:22 PM, Ard Biesheuvel wrote:
> 
> On Tue, 14 Jan 2025 at 03:54, Xu, Min M <min.m.xu@intel.com> wrote:
> >
> > Hi, Crystal
> >
> > PR#5939 (https://github.com/tianocore/edk2/pull/5939) was merged yesterday
> but it seems it broke the legacy VM bootup.
> >
> >
> >
> > Below is the crash log of the failure.
> >
> 
> I take it this crash occurs after loading shim?

That's right. It seems in the EDK2 CI, there are only cases of "run to shell". So it is not triggered in CI test.

Thanks
Min


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#120989): https://edk2.groups.io/g/devel/message/120989
Mute This Topic: https://groups.io/mt/110601533/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] PR#5939 broke the legacy VM bootup

[Ard Biesheuvel via groups.io]

On Tue, 14 Jan 2025 at 09:31, Xu, Min M <min.m.xu@intel.com> wrote:
>
> On Tuesday, January 14, 2025 4:22 PM, Ard Biesheuvel wrote:
> >
> > On Tue, 14 Jan 2025 at 03:54, Xu, Min M <min.m.xu@intel.com> wrote:
> > >
> > > Hi, Crystal
> > >
> > > PR#5939 (https://github.com/tianocore/edk2/pull/5939) was merged yesterday
> > but it seems it broke the legacy VM bootup.
> > >
> > >
> > >
> > > Below is the crash log of the failure.
> > >
> >
> > I take it this crash occurs after loading shim?
>
> That's right. It seems in the EDK2 CI, there are only cases of "run to shell". So it is not triggered in CI test.
>

This is a long standing bug in shim, that has been fixed 2 years ago
[0] but the distros keep shipping outdated versions, due to the
reluctance of MS to sign new shim builds.

The solutions are:
- disable the EFI memory attributes protocol altogether.
- ignore the distros so they are forced to clean up their mess.



[0] https://github.com/rhboot/shim/commit/c7b305152802c8db688605654f75e1195def9fd6


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#120990): https://edk2.groups.io/g/devel/message/120990
Mute This Topic: https://groups.io/mt/110601533/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] PR#5939 broke the legacy VM bootup

[Gerd Hoffmann via groups.io]

> This is a long standing bug in shim, that has been fixed 2 years ago
> [0] but the distros keep shipping outdated versions, due to the
> reluctance of MS to sign new shim builds.

Yep, the process is slow and tedious.  latest centos stream shim review
is here: https://github.com/rhboot/shim-review/issues/454

To be fair fedora and rhel managed to roll out the updates quite a while
ago, so it is probably not the signing process alone.

Workaround for centos stream 9:
 (1) turn off secure boot (if enabled)
 (2) install latest shim (unsigned):
     sudo dnf install shim-unsigned-x64
 (3) replace shim on the ESP:
     sudo cp /usr/share/shim/15.8-2.el9/x64/shimx64.efi /boot/efi/EFI/centos/shimx64.efi

HTH & take care,
  Gerd



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#120991): https://edk2.groups.io/g/devel/message/120991
Mute This Topic: https://groups.io/mt/110601533/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] PR#5939 broke the legacy VM bootup

[Crystal Lee via groups.io]

Hi Min,

We have the same issue with Ubuntu 22.04.04 after loading shim, Ubuntu 22.04.5(using shim 15.8) has fixed the issue.

Thanks,
Crystal
-----Original Message-----
From: Ard Biesheuvel <ardb@kernel.org>
Sent: Tuesday, January 14, 2025 4:35 PM
To: Xu, Min M <min.m.xu@intel.com>
Cc: devel@edk2.groups.io; Crystal Lee (李怡萱) <CrystalLee@ami.com>; Felix Polyudov <Felixp@ami.com>; David Hsieh (謝坤智) <DavidHsieh@ami.com>; James Wang (王家明) <JamesWang@ami.com>; Ni, Ray <ray.ni@intel.com>; Sun, CepingX <cepingx.sun@intel.com>; Yao, Jiewen <jiewen.yao@intel.com>; Gerd Hoffmann <kraxel@redhat.com>; Tom Lendacky <thomas.lendacky@amd.com>; Ard Biesheuvel <ardb+tianocore@kernel.org>.
Subject: [EXTERNAL] Re: PR#5939 broke the legacy VM bootup


**CAUTION: The e-mail below is from an external source. Please exercise caution before opening attachments, clicking links, or following guidance.**

On Tue, 14 Jan 2025 at 09:31, Xu, Min M <min.m.xu@intel.com> wrote:
>
> On Tuesday, January 14, 2025 4:22 PM, Ard Biesheuvel wrote:
> >
> > On Tue, 14 Jan 2025 at 03:54, Xu, Min M <min.m.xu@intel.com> wrote:
> > >
> > > Hi, Crystal
> > >
> > > PR#5939
> > > (https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%252
> > > Fgithub.com%2Ftianocore%2Fedk2%2Fpull%2F5939&data=05%7C02%7Ccrysta
> > > llee%40ami.com%7C953a53f4b7ed44f018c008dd34766aab%7C27e97857e15f48
> > > 6cb58e86c2b3040f93%7C1%7C0%7C638724405414190413%7CUnknown%7CTWFpbG
> > > Zsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIs
> > > IkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=FQomYihkLvcY9d
> > > jL%2BQmI0by2vmbQkDfodjny3MOexVY%3D&reserved=0) was merged
> > > yesterday
> > but it seems it broke the legacy VM bootup.
> > >
> > >
> > >
> > > Below is the crash log of the failure.
> > >
> >
> > I take it this crash occurs after loading shim?
>
> That's right. It seems in the EDK2 CI, there are only cases of "run to shell". So it is not triggered in CI test.
>

This is a long standing bug in shim, that has been fixed 2 years ago [0] but the distros keep shipping outdated versions, due to the reluctance of MS to sign new shim builds.

The solutions are:
- disable the EFI memory attributes protocol altogether.
- ignore the distros so they are forced to clean up their mess.



[0] https://github.com/rhboot/shim/commit/c7b305152802c8db688605654f75e1195def9fd6
-The information contained in this message may be confidential and proprietary to American Megatrends (AMI). This communication is intended to be read only by the individual or entity to whom it is addressed or by their designee. If the reader of this message is not the intended recipient, you are on notice that any distribution of this message, in any form, is strictly prohibited. Please promptly notify the sender by reply e-mail or by telephone at 770-246-8600, and then delete or destroy all copies of the transmission.


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#121025): https://edk2.groups.io/g/devel/message/121025
Mute This Topic: https://groups.io/mt/110601533/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] PR#5939 broke the legacy VM bootup

[Chenyi Qiang via groups.io]

[-- Attachment #1: Type: text/plain, Size: 476 bytes --]

I tried this workaround in CentOS 9, but it still doesn't work. Is it enough to only replace this shim file?


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#121202): https://edk2.groups.io/g/devel/message/121202
Mute This Topic: https://groups.io/mt/110601533/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-



[-- Attachment #2: Type: text/html, Size: 905 bytes --]

Re: [edk2-devel] PR#5939 broke the legacy VM bootup

[Fiona Ebner]

Hi,

Am 14.01.25 um 09:35 schrieb Ard Biesheuvel via groups.io:
> On Tue, 14 Jan 2025 at 09:31, Xu, Min M <min.m.xu@intel.com> wrote:
>>
>> On Tuesday, January 14, 2025 4:22 PM, Ard Biesheuvel wrote:
>>>
>>> On Tue, 14 Jan 2025 at 03:54, Xu, Min M <min.m.xu@intel.com> wrote:
>>>>
>>>> Hi, Crystal
>>>>
>>>> PR#5939 (https://github.com/tianocore/edk2/pull/5939) was merged yesterday
>>> but it seems it broke the legacy VM bootup.
>>>>
>>>>
>>>>
>>>> Below is the crash log of the failure.
>>>>
>>>
>>> I take it this crash occurs after loading shim?
>>
>> That's right. It seems in the EDK2 CI, there are only cases of "run to shell". So it is not triggered in CI test.
>>
> 
> This is a long standing bug in shim, that has been fixed 2 years ago
> [0] but the distros keep shipping outdated versions, due to the
> reluctance of MS to sign new shim builds.
> 
> The solutions are:
> - disable the EFI memory attributes protocol altogether.

Is this possible via a build macro/flag? Or alternatively via the QEMU
commandline/fw_cfg?

> - ignore the distros so they are forced to clean up their mess.

Is this the solution the EDK2 project opted for/status quo?

Best Regards,
Fiona



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#121217): https://edk2.groups.io/g/devel/message/121217
Mute This Topic: https://groups.io/mt/110601533/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] PR#5939 broke the legacy VM bootup

[Gerd Hoffmann via groups.io]

  Hi,

> > The solutions are:
> > - disable the EFI memory attributes protocol altogether.
> 
> Is this possible via a build macro/flag? Or alternatively via the QEMU
> commandline/fw_cfg?

fedora builds are using this downstream:
https://github.com/tianocore/edk2/pull/10667

> > - ignore the distros so they are forced to clean up their mess.
> 
> Is this the solution the EDK2 project opted for/status quo?

It's the status quo.

take care,
  Gerd



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#121218): https://edk2.groups.io/g/devel/message/121218
Mute This Topic: https://groups.io/mt/110601533/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] PR#5939 broke the legacy VM bootup

[Ard Biesheuvel via groups.io]

On Mon, 24 Mar 2025 at 18:27, Gerd Hoffmann <kraxel@redhat.com> wrote:
>
>   Hi,
>
> > > The solutions are:
> > > - disable the EFI memory attributes protocol altogether.
> >
> > Is this possible via a build macro/flag? Or alternatively via the QEMU
> > commandline/fw_cfg?
>
> fedora builds are using this downstream:
> https://github.com/tianocore/edk2/pull/10667
>
> > > - ignore the distros so they are forced to clean up their mess.
> >
> > Is this the solution the EDK2 project opted for/status quo?
>
> It's the status quo.
>

Note that EDK2 is the reference implementation of UEFI. OVMF is an
outlier because it is used in production almost unmodified, but in the
general case, the reference implementation is used as the basis for
production platforms shipping in the future. Therefore, it is not the
place to put hacks to work around nasty issues that the downstream
users cannot be bothered to address.


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#121219): https://edk2.groups.io/g/devel/message/121219
Mute This Topic: https://groups.io/mt/110601533/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] PR#5939 broke the legacy VM bootup

[Min Xu via groups.io]

Hi, Gerd
We tried to fix this issue based on your suggested workaround (at Jan 14) but it still hang.
Did we miss something? 
@Qiang, Chenyi Can provide more information.

>Workaround for centos stream 9:
> (1) turn off secure boot (if enabled)
> (2) install latest shim (unsigned):
>     sudo dnf install shim-unsigned-x64
> (3) replace shim on the ESP:
>     sudo cp /usr/share/shim/15.8-2.el9/x64/shimx64.efi /boot/efi/EFI/centos/shimx64.efi


> -----Original Message-----
> From: Ard Biesheuvel <ardb@kernel.org>
> Sent: Tuesday, March 25, 2025 3:48 PM
> To: Gerd Hoffmann <kraxel@redhat.com>
> Cc: Fiona Ebner <f.ebner@proxmox.com>; devel@edk2.groups.io; Xu, Min M
> <min.m.xu@intel.com>; crystallee@ami.com; POLUDOV, FELIX
> <felixp@ami.com>; davidhsieh@ami.com; jameswang@ami.com; Ni, Ray
> <ray.ni@intel.com>; Sun, CepingX <cepingx.sun@intel.com>; Yao, Jiewen
> <jiewen.yao@intel.com>; Tom Lendacky <thomas.lendacky@amd.com>; Ard
> Biesheuvel <ardb+tianocore@kernel.org>
> Subject: Re: [edk2-devel] PR#5939 broke the legacy VM bootup
> 
> On Mon, 24 Mar 2025 at 18:27, Gerd Hoffmann <kraxel@redhat.com> wrote:
> >
> >   Hi,
> >
> > > > The solutions are:
> > > > - disable the EFI memory attributes protocol altogether.
> > >
> > > Is this possible via a build macro/flag? Or alternatively via the
> > > QEMU commandline/fw_cfg?
> >
> > fedora builds are using this downstream:
> > https://github.com/tianocore/edk2/pull/10667
> >
> > > > - ignore the distros so they are forced to clean up their mess.
> > >
> > > Is this the solution the EDK2 project opted for/status quo?
> >
> > It's the status quo.
> >
> 
> Note that EDK2 is the reference implementation of UEFI. OVMF is an outlier
> because it is used in production almost unmodified, but in the general case,
> the reference implementation is used as the basis for production platforms
> shipping in the future. Therefore, it is not the place to put hacks to work
> around nasty issues that the downstream users cannot be bothered to
> address.


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#121220): https://edk2.groups.io/g/devel/message/121220
Mute This Topic: https://groups.io/mt/110601533/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-