public inbox for devel@edk2.groups.io
 help / color / mirror / Atom feed
* ArmVirtPkg: non-executable EFI_LOADER_DATA breaks GRUB on Ubuntu 22.04
@ 2023-07-08 21:16 osy
  2023-07-10 15:58 ` [edk2-devel] " Pedro Falcato
  0 siblings, 1 reply; 10+ messages in thread
From: osy @ 2023-07-08 21:16 UTC (permalink / raw)
  To: devel

[-- Attachment #1: Type: text/plain, Size: 792 bytes --]

I have an existing install of Ubuntu 22.04 on a QEMU virtual machine which I've decided to update the UEFI firmware. After doing so, GRUB no longer boots ("Synchronous Exception" message seen). After a git bisect session, I found the problematic 2997ae38739756ecba9b0de19e86032ebc689ef9. The comment says GRUB should have been fixed in 2017, but for one reason or another, my VM which was built in 2022 still had the issue. Regardless, I don't think it's a good idea to break GRUB, even if it's fixed in 2017. In the very least, a better error message would be preferable to crashing with an "Synchronous Exception." Googling this error message shows that other people may be hitting this issue as well but the vague error symptom means its impossible to know if it's the same issue or not.

[-- Attachment #2: Type: text/html, Size: 792 bytes --]

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [edk2-devel] ArmVirtPkg: non-executable EFI_LOADER_DATA breaks GRUB on Ubuntu 22.04
  2023-07-08 21:16 ArmVirtPkg: non-executable EFI_LOADER_DATA breaks GRUB on Ubuntu 22.04 osy
@ 2023-07-10 15:58 ` Pedro Falcato
  2023-07-10 17:09   ` osy
  2023-07-11  6:58   ` Gerd Hoffmann
  0 siblings, 2 replies; 10+ messages in thread
From: Pedro Falcato @ 2023-07-10 15:58 UTC (permalink / raw)
  To: devel, osy; +Cc: Ard Biesheuvel, Gerd Hoffmann, Leif Lindholm, dann frazier

On Mon, Jul 10, 2023 at 2:28 PM <osy@turing.llc> wrote:
>
> I have an existing install of Ubuntu 22.04 on a QEMU virtual machine which I've decided to update the UEFI firmware. After doing so, GRUB no longer boots ("Synchronous Exception" message seen). After a git bisect session, I found the problematic 2997ae38739756ecba9b0de19e86032ebc689ef9. The comment says GRUB should have been fixed in 2017, but for one reason or another, my VM which was built in 2022 still had the issue. Regardless, I don't think it's a good idea to break GRUB, even if it's fixed in 2017. In the very least, a better error message would be preferable to crashing with an "Synchronous Exception." Googling this error message shows that other people may be hitting this issue as well but the vague error symptom means its impossible to know if it's the same issue or not.

+CC Some of the folks involved in the original discussion

In the original thread, people discussed some alternative behavior to
just crashing on a NX fault. Is this still an alternative?
I'm kind of thinking this should be addressed by distros anyway....
How is $CURRENT_YEAR Ubuntu still shipping bad GRUBs? I know the
situation around GRUB and distro patching is complicated but...
Do we have any idea of how many distros/GRUBs are affected by this?

Personally, I would like to avoid loosening up memory permissions.

--
Pedro

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [edk2-devel] ArmVirtPkg: non-executable EFI_LOADER_DATA breaks GRUB on Ubuntu 22.04
  2023-07-10 15:58 ` [edk2-devel] " Pedro Falcato
@ 2023-07-10 17:09   ` osy
  2023-07-11  6:58   ` Gerd Hoffmann
  1 sibling, 0 replies; 10+ messages in thread
From: osy @ 2023-07-10 17:09 UTC (permalink / raw)
  To: Pedro Falcato
  Cc: devel, Ard Biesheuvel, Gerd Hoffmann, Leif Lindholm, dann frazier

On Mon, Jul 10, 2023 at 8:58 AM Pedro Falcato <pedro.falcato@gmail.com> wrote:
>
> On Mon, Jul 10, 2023 at 2:28 PM <osy@turing.llc> wrote:
> >
> > I have an existing install of Ubuntu 22.04 on a QEMU virtual machine which I've decided to update the UEFI firmware. After doing so, GRUB no longer boots ("Synchronous Exception" message seen). After a git bisect session, I found the problematic 2997ae38739756ecba9b0de19e86032ebc689ef9. The comment says GRUB should have been fixed in 2017, but for one reason or another, my VM which was built in 2022 still had the issue. Regardless, I don't think it's a good idea to break GRUB, even if it's fixed in 2017. In the very least, a better error message would be preferable to crashing with an "Synchronous Exception." Googling this error message shows that other people may be hitting this issue as well but the vague error symptom means its impossible to know if it's the same issue or not.
>
> +CC Some of the folks involved in the original discussion
>
> In the original thread, people discussed some alternative behavior to
> just crashing on a NX fault. Is this still an alternative?
> I'm kind of thinking this should be addressed by distros anyway....
> How is $CURRENT_YEAR Ubuntu still shipping bad GRUBs? I know the
> situation around GRUB and distro patching is complicated but...
> Do we have any idea of how many distros/GRUBs are affected by this?
>
> Personally, I would like to avoid loosening up memory permissions.
>
> --
> Pedro

I agree with the desire to be more secure, but the way I see it is
that the firmware's first priority is to load the bootloader and
gracefully hand off control. The fact that a bootloader is "too old"
shouldn't be a reason to just crash and die. Once downstream projects
like QEMU start picking up this change, this could potentially become
an issue for a lot of users. If the EDK2 project decides that there is
no technical way to both have secure memory permissions and support
older GRUB installs, I think this breakage must be communicated
effectively and be made explicit. I agree that distros probably
shouldn't ship bad GRUBs (here's a bug report in Fedora addressing the
same issue https://bugzilla.redhat.com/show_bug.cgi?id=2149020) and
perhaps they have already fixed it but at the end of the day, the UEFI
firmware likely isn't maintained by the distros. When a user updates
their firmware (from QEMU or from a board vendor), they don't expect
their OS to stop booting. If this happened on real hardware, it could
actually be a real pain to downgrade the firmware or recover and
update GRUB.

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [edk2-devel] ArmVirtPkg: non-executable EFI_LOADER_DATA breaks GRUB on Ubuntu 22.04
  2023-07-10 15:58 ` [edk2-devel] " Pedro Falcato
  2023-07-10 17:09   ` osy
@ 2023-07-11  6:58   ` Gerd Hoffmann
  2023-07-13 16:57     ` Pedro Falcato
  1 sibling, 1 reply; 10+ messages in thread
From: Gerd Hoffmann @ 2023-07-11  6:58 UTC (permalink / raw)
  To: Pedro Falcato; +Cc: devel, osy, Ard Biesheuvel, Leif Lindholm, dann frazier

On Mon, Jul 10, 2023 at 04:58:15PM +0100, Pedro Falcato wrote:
> On Mon, Jul 10, 2023 at 2:28 PM <osy@turing.llc> wrote:
> >
> > I have an existing install of Ubuntu 22.04 on a QEMU virtual machine which I've decided to update the UEFI firmware. After doing so, GRUB no longer boots ("Synchronous Exception" message seen). After a git bisect session, I found the problematic 2997ae38739756ecba9b0de19e86032ebc689ef9. The comment says GRUB should have been fixed in 2017, but for one reason or another, my VM which was built in 2022 still had the issue. Regardless, I don't think it's a good idea to break GRUB, even if it's fixed in 2017. In the very least, a better error message would be preferable to crashing with an "Synchronous Exception." Googling this error message shows that other people may be hitting this issue as well but the vague error symptom means its impossible to know if it's the same issue or not.
> 
> +CC Some of the folks involved in the original discussion
> 
> In the original thread, people discussed some alternative behavior to
> just crashing on a NX fault. Is this still an alternative?

The idea is: Improve page fault handler to (a) print a big'n'fat
warning, and (b) loosening up memory permissions for the faulting
page address.

No patch for that emerged (yet?).

> I'm kind of thinking this should be addressed by distros anyway....
> How is $CURRENT_YEAR Ubuntu still shipping bad GRUBs? I know the
> situation around GRUB and distro patching is complicated but...
> Do we have any idea of how many distros/GRUBs are affected by this?

Too many :(

> Personally, I would like to avoid loosening up memory permissions.

Well, you can't have both.  You have to pick between strict nx handling
and grub bug compatibility ...

take care,
  Gerd


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [edk2-devel] ArmVirtPkg: non-executable EFI_LOADER_DATA breaks GRUB on Ubuntu 22.04
  2023-07-11  6:58   ` Gerd Hoffmann
@ 2023-07-13 16:57     ` Pedro Falcato
  2023-07-13 17:20       ` Ard Biesheuvel
  2023-07-17  9:30       ` Gerd Hoffmann
  0 siblings, 2 replies; 10+ messages in thread
From: Pedro Falcato @ 2023-07-13 16:57 UTC (permalink / raw)
  To: Gerd Hoffmann; +Cc: devel, osy, Ard Biesheuvel, Leif Lindholm, dann frazier

On Tue, Jul 11, 2023 at 7:58 AM Gerd Hoffmann <kraxel@redhat.com> wrote:
>
> On Mon, Jul 10, 2023 at 04:58:15PM +0100, Pedro Falcato wrote:
> > On Mon, Jul 10, 2023 at 2:28 PM <osy@turing.llc> wrote:
> > >
> > > I have an existing install of Ubuntu 22.04 on a QEMU virtual machine which I've decided to update the UEFI firmware. After doing so, GRUB no longer boots ("Synchronous Exception" message seen). After a git bisect session, I found the problematic 2997ae38739756ecba9b0de19e86032ebc689ef9. The comment says GRUB should have been fixed in 2017, but for one reason or another, my VM which was built in 2022 still had the issue. Regardless, I don't think it's a good idea to break GRUB, even if it's fixed in 2017. In the very least, a better error message would be preferable to crashing with an "Synchronous Exception." Googling this error message shows that other people may be hitting this issue as well but the vague error symptom means its impossible to know if it's the same issue or not.
> >
> > +CC Some of the folks involved in the original discussion
> >
> > In the original thread, people discussed some alternative behavior to
> > just crashing on a NX fault. Is this still an alternative?
>
> The idea is: Improve page fault handler to (a) print a big'n'fat
> warning, and (b) loosening up memory permissions for the faulting
> page address.
>
> No patch for that emerged (yet?).

Ack. I can work on that.

> > I'm kind of thinking this should be addressed by distros anyway....
> > How is $CURRENT_YEAR Ubuntu still shipping bad GRUBs? I know the
> > situation around GRUB and distro patching is complicated but...
> > Do we have any idea of how many distros/GRUBs are affected by this?
>
> Too many :(

Ugh, even the latest releases?

> > Personally, I would like to avoid loosening up memory permissions.
>
> Well, you can't have both.  You have to pick between strict nx handling
> and grub bug compatibility ...

Yes. IMO it should be ok to add a hack around NX handling if there's a
solid plan for dealing with this from the distros' side (and phasing
this out). And I'm assuming upstream GRUB has this fixed.
This whole situation is kind of messy as firmware people add new
restrictions that weren't really there in the first place.

Also, what's the situation on this for x86? I assume it's a lot worse there?

-- 
Pedro

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [edk2-devel] ArmVirtPkg: non-executable EFI_LOADER_DATA breaks GRUB on Ubuntu 22.04
  2023-07-13 16:57     ` Pedro Falcato
@ 2023-07-13 17:20       ` Ard Biesheuvel
  2023-07-13 17:41         ` Pedro Falcato
  2023-07-17  9:30       ` Gerd Hoffmann
  1 sibling, 1 reply; 10+ messages in thread
From: Ard Biesheuvel @ 2023-07-13 17:20 UTC (permalink / raw)
  To: Pedro Falcato; +Cc: Gerd Hoffmann, devel, osy, Leif Lindholm, dann frazier

On Thu, 13 Jul 2023 at 18:57, Pedro Falcato <pedro.falcato@gmail.com> wrote:
>
> On Tue, Jul 11, 2023 at 7:58 AM Gerd Hoffmann <kraxel@redhat.com> wrote:
> >
> > On Mon, Jul 10, 2023 at 04:58:15PM +0100, Pedro Falcato wrote:
> > > On Mon, Jul 10, 2023 at 2:28 PM <osy@turing.llc> wrote:
> > > >
> > > > I have an existing install of Ubuntu 22.04 on a QEMU virtual machine which I've decided to update the UEFI firmware. After doing so, GRUB no longer boots ("Synchronous Exception" message seen). After a git bisect session, I found the problematic 2997ae38739756ecba9b0de19e86032ebc689ef9. The comment says GRUB should have been fixed in 2017, but for one reason or another, my VM which was built in 2022 still had the issue. Regardless, I don't think it's a good idea to break GRUB, even if it's fixed in 2017. In the very least, a better error message would be preferable to crashing with an "Synchronous Exception." Googling this error message shows that other people may be hitting this issue as well but the vague error symptom means its impossible to know if it's the same issue or not.
> > >
> > > +CC Some of the folks involved in the original discussion
> > >
> > > In the original thread, people discussed some alternative behavior to
> > > just crashing on a NX fault. Is this still an alternative?
> >
> > The idea is: Improve page fault handler to (a) print a big'n'fat
> > warning, and (b) loosening up memory permissions for the faulting
> > page address.
> >
> > No patch for that emerged (yet?).
>
> Ack. I can work on that.
>
> > > I'm kind of thinking this should be addressed by distros anyway....
> > > How is $CURRENT_YEAR Ubuntu still shipping bad GRUBs? I know the
> > > situation around GRUB and distro patching is complicated but...
> > > Do we have any idea of how many distros/GRUBs are affected by this?
> >
> > Too many :(
>
> Ugh, even the latest releases?
>
> > > Personally, I would like to avoid loosening up memory permissions.
> >
> > Well, you can't have both.  You have to pick between strict nx handling
> > and grub bug compatibility ...
>
> Yes. IMO it should be ok to add a hack around NX handling if there's a
> solid plan for dealing with this from the distros' side (and phasing
> this out). And I'm assuming upstream GRUB has this fixed.
> This whole situation is kind of messy as firmware people add new
> restrictions that weren't really there in the first place.
>
> Also, what's the situation on this for x86? I assume it's a lot worse there?
>

To be honest, I have little sympathy for the gigantic mess that the
distros have created for themselves with GRUB, shim, etc. Mainline
GRUB works fine with mainline EDK2, and secure boot in a arm64 VM is
rather pointless, given that the [emulated] NOR flash is writable by
the guest OS. The breakage is in the downstream GRUB changes that make
it interoperate with shim, and its hacked up PE loader.

If we are going to accommodate every broken GRUB build that the
distros ever released, we won't be able to make any progress on this
front. I understand that the distros need to support their existing
user bases, so I am willing to consider facilities that make it easier
to create builds that work around such issues.

However, just turning off NX support is not one of the options.
Upstream is not what the distros are shipping, this applies to GRUB
and shim as well as EDK2: so if their downstream GRUB breaks EDK2,
they can fix it in their EDK2 builds, either by carrying a code
change, or by enabling an upstream build flag that is off by default.

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [edk2-devel] ArmVirtPkg: non-executable EFI_LOADER_DATA breaks GRUB on Ubuntu 22.04
  2023-07-13 17:20       ` Ard Biesheuvel
@ 2023-07-13 17:41         ` Pedro Falcato
  2023-07-13 18:22           ` Oliver Smith-Denny
  0 siblings, 1 reply; 10+ messages in thread
From: Pedro Falcato @ 2023-07-13 17:41 UTC (permalink / raw)
  To: Ard Biesheuvel; +Cc: Gerd Hoffmann, devel, osy, Leif Lindholm, dann frazier

On Thu, Jul 13, 2023 at 6:20 PM Ard Biesheuvel <ardb@kernel.org> wrote:
>
> On Thu, 13 Jul 2023 at 18:57, Pedro Falcato <pedro.falcato@gmail.com> wrote:
> >
> > On Tue, Jul 11, 2023 at 7:58 AM Gerd Hoffmann <kraxel@redhat.com> wrote:
> > >
> > > On Mon, Jul 10, 2023 at 04:58:15PM +0100, Pedro Falcato wrote:
> > > > On Mon, Jul 10, 2023 at 2:28 PM <osy@turing.llc> wrote:
> > > > >
> > > > > I have an existing install of Ubuntu 22.04 on a QEMU virtual machine which I've decided to update the UEFI firmware. After doing so, GRUB no longer boots ("Synchronous Exception" message seen). After a git bisect session, I found the problematic 2997ae38739756ecba9b0de19e86032ebc689ef9. The comment says GRUB should have been fixed in 2017, but for one reason or another, my VM which was built in 2022 still had the issue. Regardless, I don't think it's a good idea to break GRUB, even if it's fixed in 2017. In the very least, a better error message would be preferable to crashing with an "Synchronous Exception." Googling this error message shows that other people may be hitting this issue as well but the vague error symptom means its impossible to know if it's the same issue or not.
> > > >
> > > > +CC Some of the folks involved in the original discussion
> > > >
> > > > In the original thread, people discussed some alternative behavior to
> > > > just crashing on a NX fault. Is this still an alternative?
> > >
> > > The idea is: Improve page fault handler to (a) print a big'n'fat
> > > warning, and (b) loosening up memory permissions for the faulting
> > > page address.
> > >
> > > No patch for that emerged (yet?).
> >
> > Ack. I can work on that.
> >
> > > > I'm kind of thinking this should be addressed by distros anyway....
> > > > How is $CURRENT_YEAR Ubuntu still shipping bad GRUBs? I know the
> > > > situation around GRUB and distro patching is complicated but...
> > > > Do we have any idea of how many distros/GRUBs are affected by this?
> > >
> > > Too many :(
> >
> > Ugh, even the latest releases?
> >
> > > > Personally, I would like to avoid loosening up memory permissions.
> > >
> > > Well, you can't have both.  You have to pick between strict nx handling
> > > and grub bug compatibility ...
> >
> > Yes. IMO it should be ok to add a hack around NX handling if there's a
> > solid plan for dealing with this from the distros' side (and phasing
> > this out). And I'm assuming upstream GRUB has this fixed.
> > This whole situation is kind of messy as firmware people add new
> > restrictions that weren't really there in the first place.
> >
> > Also, what's the situation on this for x86? I assume it's a lot worse there?
> >
>
> To be honest, I have little sympathy for the gigantic mess that the
> distros have created for themselves with GRUB, shim, etc. Mainline
> GRUB works fine with mainline EDK2, and secure boot in a arm64 VM is
> rather pointless, given that the [emulated] NOR flash is writable by
> the guest OS. The breakage is in the downstream GRUB changes that make
> it interoperate with shim, and its hacked up PE loader.
>
> If we are going to accommodate every broken GRUB build that the
> distros ever released, we won't be able to make any progress on this
> front. I understand that the distros need to support their existing
> user bases, so I am willing to consider facilities that make it easier
> to create builds that work around such issues.
>
> However, just turning off NX support is not one of the options.
> Upstream is not what the distros are shipping, this applies to GRUB
> and shim as well as EDK2: so if their downstream GRUB breaks EDK2,
> they can fix it in their EDK2 builds, either by carrying a code
> change, or by enabling an upstream build flag that is off by default.

I understand your sentiment, but I don't see how this can work. Let's
say Fedora has a fixed GRUB (I have no idea if this is the case), so
they have a fully-NX'd edk2. Then someone tries to boot Ubuntu 22.04 -
it crashes because Ubuntu's GRUB is borked; what now? I don't know if
this case is super prevalent in virtualization, but it's definitely a
problem (and it seems to have happened to osy here?).

I agree that turning off NX sucks, but so does not being able to boot
into distros as recent as Ubuntu 22.04. It might be that a single
Sleep(10); +  a nice loud error message gets the idea across? maybe
over a stable tag or so, then we remove the hack and add full-NX. What
do you think?

-- 
Pedro

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [edk2-devel] ArmVirtPkg: non-executable EFI_LOADER_DATA breaks GRUB on Ubuntu 22.04
  2023-07-13 17:41         ` Pedro Falcato
@ 2023-07-13 18:22           ` Oliver Smith-Denny
  2023-07-13 23:03             ` Michael D Kinney
  0 siblings, 1 reply; 10+ messages in thread
From: Oliver Smith-Denny @ 2023-07-13 18:22 UTC (permalink / raw)
  To: devel, pedro.falcato, Ard Biesheuvel
  Cc: Gerd Hoffmann, osy, Leif Lindholm, dann frazier

On 7/13/2023 10:41 AM, Pedro Falcato wrote:
> On Thu, Jul 13, 2023 at 6:20 PM Ard Biesheuvel <ardb@kernel.org> wrote:
>>
>> On Thu, 13 Jul 2023 at 18:57, Pedro Falcato <pedro.falcato@gmail.com> wrote:
>>>
>>> On Tue, Jul 11, 2023 at 7:58 AM Gerd Hoffmann <kraxel@redhat.com> wrote:
>>>>
>>>> On Mon, Jul 10, 2023 at 04:58:15PM +0100, Pedro Falcato wrote:
>>>>> On Mon, Jul 10, 2023 at 2:28 PM <osy@turing.llc> wrote:
>>>>>>
>>>>>> I have an existing install of Ubuntu 22.04 on a QEMU virtual machine which I've decided to update the UEFI firmware. After doing so, GRUB no longer boots ("Synchronous Exception" message seen). After a git bisect session, I found the problematic 2997ae38739756ecba9b0de19e86032ebc689ef9. The comment says GRUB should have been fixed in 2017, but for one reason or another, my VM which was built in 2022 still had the issue. Regardless, I don't think it's a good idea to break GRUB, even if it's fixed in 2017. In the very least, a better error message would be preferable to crashing with an "Synchronous Exception." Googling this error message shows that other people may be hitting this issue as well but the vague error symptom means its impossible to know if it's the same issue or not.
>>>>>
>>>>> +CC Some of the folks involved in the original discussion
>>>>>
>>>>> In the original thread, people discussed some alternative behavior to
>>>>> just crashing on a NX fault. Is this still an alternative?
>>>>
>>>> The idea is: Improve page fault handler to (a) print a big'n'fat
>>>> warning, and (b) loosening up memory permissions for the faulting
>>>> page address.
>>>>
>>>> No patch for that emerged (yet?).
>>>
>>> Ack. I can work on that.
>>>
>>>>> I'm kind of thinking this should be addressed by distros anyway....
>>>>> How is $CURRENT_YEAR Ubuntu still shipping bad GRUBs? I know the
>>>>> situation around GRUB and distro patching is complicated but...
>>>>> Do we have any idea of how many distros/GRUBs are affected by this?
>>>>
>>>> Too many :(
>>>
>>> Ugh, even the latest releases?
>>>
>>>>> Personally, I would like to avoid loosening up memory permissions.
>>>>
>>>> Well, you can't have both.  You have to pick between strict nx handling
>>>> and grub bug compatibility ...
>>>
>>> Yes. IMO it should be ok to add a hack around NX handling if there's a
>>> solid plan for dealing with this from the distros' side (and phasing
>>> this out). And I'm assuming upstream GRUB has this fixed.
>>> This whole situation is kind of messy as firmware people add new
>>> restrictions that weren't really there in the first place.
>>>
>>> Also, what's the situation on this for x86? I assume it's a lot worse there?
>>>
>>
>> To be honest, I have little sympathy for the gigantic mess that the
>> distros have created for themselves with GRUB, shim, etc. Mainline
>> GRUB works fine with mainline EDK2, and secure boot in a arm64 VM is
>> rather pointless, given that the [emulated] NOR flash is writable by
>> the guest OS. The breakage is in the downstream GRUB changes that make
>> it interoperate with shim, and its hacked up PE loader.
>>
>> If we are going to accommodate every broken GRUB build that the
>> distros ever released, we won't be able to make any progress on this
>> front. I understand that the distros need to support their existing
>> user bases, so I am willing to consider facilities that make it easier
>> to create builds that work around such issues.
>>
>> However, just turning off NX support is not one of the options.
>> Upstream is not what the distros are shipping, this applies to GRUB
>> and shim as well as EDK2: so if their downstream GRUB breaks EDK2,
>> they can fix it in their EDK2 builds, either by carrying a code
>> change, or by enabling an upstream build flag that is off by default.
> 
> I understand your sentiment, but I don't see how this can work. Let's
> say Fedora has a fixed GRUB (I have no idea if this is the case), so
> they have a fully-NX'd edk2. Then someone tries to boot Ubuntu 22.04 -
> it crashes because Ubuntu's GRUB is borked; what now? I don't know if
> this case is super prevalent in virtualization, but it's definitely a
> problem (and it seems to have happened to osy here?).
> 
> I agree that turning off NX sucks, but so does not being able to boot
> into distros as recent as Ubuntu 22.04. It might be that a single
> Sleep(10); +  a nice loud error message gets the idea across? maybe
> over a stable tag or so, then we remove the hack and add full-NX. What
> do you think?
> 

I agree with Ard here. If other software is buggy and outdated, our
general approach should be fix your outdated and buggy software,
especially when there are security and safety implications to bending
backwards for broken code.

A downstream platform may well need to support buggy OSes, etc. for the
lifespan of the device, but upstream edk2 is not the right place to add
hacks for buggy external software. edk2 is our opportunity to do the
right thing and help encourage the community to the right place.

When distros have the maintenance burden of working with downstream
platforms to support their lack of updating grub etc., they may decide
it is worthwhile to actually update grub...

Thanks,
Oliver

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [edk2-devel] ArmVirtPkg: non-executable EFI_LOADER_DATA breaks GRUB on Ubuntu 22.04
  2023-07-13 18:22           ` Oliver Smith-Denny
@ 2023-07-13 23:03             ` Michael D Kinney
  0 siblings, 0 replies; 10+ messages in thread
From: Michael D Kinney @ 2023-07-13 23:03 UTC (permalink / raw)
  To: devel@edk2.groups.io, osde@linux.microsoft.com,
	pedro.falcato@gmail.com, Ard Biesheuvel
  Cc: Gerd Hoffmann, osy@turing.llc, Leif Lindholm, dann frazier,
	Kinney, Michael D

A general approach to this type of issues is to enable the new feature by default
and optionally provide a setup option to disable the new feature.

If there is a way to detect the failure case and fail gracefully back to the 
boot manager with an error message that indicates the type of issue and also
indicates if there is a setup option to disable a feature to allow that OS
to boot, then the user can choose to opt-in to disabling the feature.

Can also consider a warning message when a system is in a defeatured state
so the user knows that every time they boot so they can re-enable if they
only needed to disable for a short period of time.

Mike

> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Oliver
> Smith-Denny
> Sent: Thursday, July 13, 2023 11:23 AM
> To: devel@edk2.groups.io; pedro.falcato@gmail.com; Ard Biesheuvel
> <ardb@kernel.org>
> Cc: Gerd Hoffmann <kraxel@redhat.com>; osy@turing.llc; Leif Lindholm
> <quic_llindhol@quicinc.com>; dann frazier <dann.frazier@canonical.com>
> Subject: Re: [edk2-devel] ArmVirtPkg: non-executable EFI_LOADER_DATA
> breaks GRUB on Ubuntu 22.04
> 
> On 7/13/2023 10:41 AM, Pedro Falcato wrote:
> > On Thu, Jul 13, 2023 at 6:20 PM Ard Biesheuvel <ardb@kernel.org> wrote:
> >>
> >> On Thu, 13 Jul 2023 at 18:57, Pedro Falcato <pedro.falcato@gmail.com>
> wrote:
> >>>
> >>> On Tue, Jul 11, 2023 at 7:58 AM Gerd Hoffmann <kraxel@redhat.com>
> wrote:
> >>>>
> >>>> On Mon, Jul 10, 2023 at 04:58:15PM +0100, Pedro Falcato wrote:
> >>>>> On Mon, Jul 10, 2023 at 2:28 PM <osy@turing.llc> wrote:
> >>>>>>
> >>>>>> I have an existing install of Ubuntu 22.04 on a QEMU virtual
> machine which I've decided to update the UEFI firmware. After doing so,
> GRUB no longer boots ("Synchronous Exception" message seen). After a git
> bisect session, I found the problematic
> 2997ae38739756ecba9b0de19e86032ebc689ef9. The comment says GRUB should
> have been fixed in 2017, but for one reason or another, my VM which was
> built in 2022 still had the issue. Regardless, I don't think it's a good
> idea to break GRUB, even if it's fixed in 2017. In the very least, a
> better error message would be preferable to crashing with an
> "Synchronous Exception." Googling this error message shows that other
> people may be hitting this issue as well but the vague error symptom
> means its impossible to know if it's the same issue or not.
> >>>>>
> >>>>> +CC Some of the folks involved in the original discussion
> >>>>>
> >>>>> In the original thread, people discussed some alternative behavior
> to
> >>>>> just crashing on a NX fault. Is this still an alternative?
> >>>>
> >>>> The idea is: Improve page fault handler to (a) print a big'n'fat
> >>>> warning, and (b) loosening up memory permissions for the faulting
> >>>> page address.
> >>>>
> >>>> No patch for that emerged (yet?).
> >>>
> >>> Ack. I can work on that.
> >>>
> >>>>> I'm kind of thinking this should be addressed by distros
> anyway....
> >>>>> How is $CURRENT_YEAR Ubuntu still shipping bad GRUBs? I know the
> >>>>> situation around GRUB and distro patching is complicated but...
> >>>>> Do we have any idea of how many distros/GRUBs are affected by
> this?
> >>>>
> >>>> Too many :(
> >>>
> >>> Ugh, even the latest releases?
> >>>
> >>>>> Personally, I would like to avoid loosening up memory permissions.
> >>>>
> >>>> Well, you can't have both.  You have to pick between strict nx
> handling
> >>>> and grub bug compatibility ...
> >>>
> >>> Yes. IMO it should be ok to add a hack around NX handling if there's
> a
> >>> solid plan for dealing with this from the distros' side (and phasing
> >>> this out). And I'm assuming upstream GRUB has this fixed.
> >>> This whole situation is kind of messy as firmware people add new
> >>> restrictions that weren't really there in the first place.
> >>>
> >>> Also, what's the situation on this for x86? I assume it's a lot
> worse there?
> >>>
> >>
> >> To be honest, I have little sympathy for the gigantic mess that the
> >> distros have created for themselves with GRUB, shim, etc. Mainline
> >> GRUB works fine with mainline EDK2, and secure boot in a arm64 VM is
> >> rather pointless, given that the [emulated] NOR flash is writable by
> >> the guest OS. The breakage is in the downstream GRUB changes that
> make
> >> it interoperate with shim, and its hacked up PE loader.
> >>
> >> If we are going to accommodate every broken GRUB build that the
> >> distros ever released, we won't be able to make any progress on this
> >> front. I understand that the distros need to support their existing
> >> user bases, so I am willing to consider facilities that make it
> easier
> >> to create builds that work around such issues.
> >>
> >> However, just turning off NX support is not one of the options.
> >> Upstream is not what the distros are shipping, this applies to GRUB
> >> and shim as well as EDK2: so if their downstream GRUB breaks EDK2,
> >> they can fix it in their EDK2 builds, either by carrying a code
> >> change, or by enabling an upstream build flag that is off by default.
> >
> > I understand your sentiment, but I don't see how this can work. Let's
> > say Fedora has a fixed GRUB (I have no idea if this is the case), so
> > they have a fully-NX'd edk2. Then someone tries to boot Ubuntu 22.04 -
> > it crashes because Ubuntu's GRUB is borked; what now? I don't know if
> > this case is super prevalent in virtualization, but it's definitely a
> > problem (and it seems to have happened to osy here?).
> >
> > I agree that turning off NX sucks, but so does not being able to boot
> > into distros as recent as Ubuntu 22.04. It might be that a single
> > Sleep(10); +  a nice loud error message gets the idea across? maybe
> > over a stable tag or so, then we remove the hack and add full-NX. What
> > do you think?
> >
> 
> I agree with Ard here. If other software is buggy and outdated, our
> general approach should be fix your outdated and buggy software,
> especially when there are security and safety implications to bending
> backwards for broken code.
> 
> A downstream platform may well need to support buggy OSes, etc. for the
> lifespan of the device, but upstream edk2 is not the right place to add
> hacks for buggy external software. edk2 is our opportunity to do the
> right thing and help encourage the community to the right place.
> 
> When distros have the maintenance burden of working with downstream
> platforms to support their lack of updating grub etc., they may decide
> it is worthwhile to actually update grub...
> 
> Thanks,
> Oliver
> 
> 
> 
> 


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [edk2-devel] ArmVirtPkg: non-executable EFI_LOADER_DATA breaks GRUB on Ubuntu 22.04
  2023-07-13 16:57     ` Pedro Falcato
  2023-07-13 17:20       ` Ard Biesheuvel
@ 2023-07-17  9:30       ` Gerd Hoffmann
  1 sibling, 0 replies; 10+ messages in thread
From: Gerd Hoffmann @ 2023-07-17  9:30 UTC (permalink / raw)
  To: Pedro Falcato; +Cc: devel, osy, Ard Biesheuvel, Leif Lindholm, dann frazier

  Hi,

> > The idea is: Improve page fault handler to (a) print a big'n'fat
> > warning, and (b) loosening up memory permissions for the faulting
> > page address.
> >
> > No patch for that emerged (yet?).
> 
> Ack. I can work on that.

FYI: There was a patch series on the list last week to move various
paging / security related options from compile time (PCD) to runtime
(config struct in HOB).  All NX settings are in there, also page guard
and heap guard.  Also some (very basic) support for config profiles.

With that in place it would be possible to make this configurable in
uefi firmware settings (or via fw_cfg, or both).

> Also, what's the situation on this for x86? I assume it's a lot worse there?

Currently x86 is less problematic in practice, but only because many of
the security features are not (yet) enabled.

Note it's not only grub+shim, the linux kernel stub is affected too.

The new, uefi-stub-only archs (armv7, armv8,riscv) are fixed meanwhile,
and they all use the common zboot code.  x86 is wip still, ard has a
patch series in flight, it's more tricky there due to hybrid bios/uefi
kernels and other legacy cruft ...

take care,
  Gerd



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#106960): https://edk2.groups.io/g/devel/message/106960
Mute This Topic: https://groups.io/mt/100057351/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-



^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2023-07-17  9:31 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-07-08 21:16 ArmVirtPkg: non-executable EFI_LOADER_DATA breaks GRUB on Ubuntu 22.04 osy
2023-07-10 15:58 ` [edk2-devel] " Pedro Falcato
2023-07-10 17:09   ` osy
2023-07-11  6:58   ` Gerd Hoffmann
2023-07-13 16:57     ` Pedro Falcato
2023-07-13 17:20       ` Ard Biesheuvel
2023-07-13 17:41         ` Pedro Falcato
2023-07-13 18:22           ` Oliver Smith-Denny
2023-07-13 23:03             ` Michael D Kinney
2023-07-17  9:30       ` Gerd Hoffmann

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox