Re: [edk2-devel] [PATCH v5 0/7] MdeModulePkg: add support for dispatching foreign arch PE/COFF images

Re: [edk2-devel] [PATCH v5 0/7] MdeModulePkg: add support for dispatching foreign arch PE/COFF images

[Ard Biesheuvel]

On Thu, 11 Apr 2019 at 11:58, Ard Biesheuvel via Groups.Io
<ard.biesheuvel=linaro.org@groups.io> wrote:
>
> Add the basic plumbing to DXE core, the PCI bus driver and the boot manager
> to allow PE/COFF images to be dispatched that target an architecture that is
> not native for the platform, but which is supported by one of potentially
> several available emulators.
>
> One implementation of such an emulator can be found here:
> https://github.com/ardbiesheuvel/X86EmulatorPkg/tree/upstream-v4
>
> This also allows us to get rid of the special treatment of EBC images in
> core code. Instead, the EbcDxe driver is augmented with an implementation
> of the EDK2 PE/COFF image emulator protocol so that internal knowledge of
> how EBC is implemented (I-cache flushing, thunks) is removed from the DXE
> core.
>
> Changes since v4:
> - Fix an issue in the protocol notify handler pointed out by Mike Kinney (#2)
>
> Changes since v3:
> - Simplify the handling of option ROMs and Driver#### images, by simply
>   deferring to the LoadImage() boot service to decide whether an image
>   can be supported or not - this removes some redundant checks from the
>   BDS layer and the PCI bus driver.
> - Move the machine type supported by the emulator into the protocol struct,
>   so we can optimize away calls into the emulator for each image loaded.
>   Instead, the LoadImage() code will only invoke the IsSupported() method for
>   images that are known to have a matching machine type.
>
> Note that I have considered, but ultimately dismissed the suggestion to
> register and unregister emulators via a new protocol. The main issue is
> that registering and unregistering structs containing sets of function
> pointers is awfully similar to managing a protocol database, and we already
> have the code to do that in EDK2.
>
> So instead, I have removed all the code that iterates over a handle buffer
> of emu protocols and invokes each one to see if it will support the image.
> Instead, this is all done by CoreLoadImage().
>
> Changes since v2:
> - incorporate feedback from Andrew Fish (delivered in person):
>   * pass a device path into the IsImageSupported() protocol method so that an
>     implementation can blacklist or whitelist certain devices, or implement
>     other policies that depend on the device where the driver originated
>   * allow the emulator to supersede the native loading of the image - this
>     permits things like X86 on X86 emulators for security sandboxing or debug
>
> Changes since v1:
> - subsume the EBC handling into the EDK2 emulator protocol and abstract
>   away from EBC specifics in core code.
> - allow multiple emulator implementations to co-exist
> - incorporate Star's review feedback
>


In order not to spam everyone with another revision within a couple of
hours, I have applied the feedback from Mike for the VS build errors
and pushed the result here:

https://github.com/ardbiesheuvel/edk2/pull/new/pecoff-emu-v5+

in case the MdeModulePkg maintainers want to pull the code and test it
during review.

Re: [edk2-devel] [PATCH v5 0/7] MdeModulePkg: add support for dispatching foreign arch PE/COFF images

[Michael D Kinney]

Ard,

We have verified that the build failures are resolved.

Series Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com>

Mike

> -----Original Message-----
> From: Ard Biesheuvel [mailto:ard.biesheuvel@linaro.org]
> Sent: Thursday, April 11, 2019 1:58 PM
> To: edk2-devel-groups-io <devel@edk2.groups.io>; Ard
> Biesheuvel <ard.biesheuvel@linaro.org>
> Cc: Zimmer, Vincent <vincent.zimmer@intel.com>;
> Richardson, Brian <brian.richardson@intel.com>; Kinney,
> Michael D <michael.d.kinney@intel.com>; Andrew Fish
> <afish@apple.com>; Leif Lindholm
> <leif.lindholm@linaro.org>; Zeng, Star
> <star.zeng@intel.com>; Dong, Eric
> <eric.dong@intel.com>; Ni, Ray <ray.ni@intel.com>; Gao,
> Liming <liming.gao@intel.com>; Carsey, Jaben
> <jaben.carsey@intel.com>; Shi, Steven
> <steven.shi@intel.com>
> Subject: Re: [edk2-devel] [PATCH v5 0/7] MdeModulePkg:
> add support for dispatching foreign arch PE/COFF images
> 
> On Thu, 11 Apr 2019 at 11:58, Ard Biesheuvel via
> Groups.Io
> <ard.biesheuvel=linaro.org@groups.io> wrote:
> >
> > Add the basic plumbing to DXE core, the PCI bus
> driver and the boot manager
> > to allow PE/COFF images to be dispatched that target
> an architecture that is
> > not native for the platform, but which is supported
> by one of potentially
> > several available emulators.
> >
> > One implementation of such an emulator can be found
> here:
> >
> https://github.com/ardbiesheuvel/X86EmulatorPkg/tree/up
> stream-v4
> >
> > This also allows us to get rid of the special
> treatment of EBC images in
> > core code. Instead, the EbcDxe driver is augmented
> with an implementation
> > of the EDK2 PE/COFF image emulator protocol so that
> internal knowledge of
> > how EBC is implemented (I-cache flushing, thunks) is
> removed from the DXE
> > core.
> >
> > Changes since v4:
> > - Fix an issue in the protocol notify handler pointed
> out by Mike Kinney (#2)
> >
> > Changes since v3:
> > - Simplify the handling of option ROMs and Driver####
> images, by simply
> >   deferring to the LoadImage() boot service to decide
> whether an image
> >   can be supported or not - this removes some
> redundant checks from the
> >   BDS layer and the PCI bus driver.
> > - Move the machine type supported by the emulator
> into the protocol struct,
> >   so we can optimize away calls into the emulator for
> each image loaded.
> >   Instead, the LoadImage() code will only invoke the
> IsSupported() method for
> >   images that are known to have a matching machine
> type.
> >
> > Note that I have considered, but ultimately dismissed
> the suggestion to
> > register and unregister emulators via a new protocol.
> The main issue is
> > that registering and unregistering structs containing
> sets of function
> > pointers is awfully similar to managing a protocol
> database, and we already
> > have the code to do that in EDK2.
> >
> > So instead, I have removed all the code that iterates
> over a handle buffer
> > of emu protocols and invokes each one to see if it
> will support the image.
> > Instead, this is all done by CoreLoadImage().
> >
> > Changes since v2:
> > - incorporate feedback from Andrew Fish (delivered in
> person):
> >   * pass a device path into the IsImageSupported()
> protocol method so that an
> >     implementation can blacklist or whitelist certain
> devices, or implement
> >     other policies that depend on the device where
> the driver originated
> >   * allow the emulator to supersede the native
> loading of the image - this
> >     permits things like X86 on X86 emulators for
> security sandboxing or debug
> >
> > Changes since v1:
> > - subsume the EBC handling into the EDK2 emulator
> protocol and abstract
> >   away from EBC specifics in core code.
> > - allow multiple emulator implementations to co-exist
> > - incorporate Star's review feedback
> >
> 
> 
> In order not to spam everyone with another revision
> within a couple of
> hours, I have applied the feedback from Mike for the VS
> build errors
> and pushed the result here:
> 
> https://github.com/ardbiesheuvel/edk2/pull/new/pecoff-
> emu-v5+
> 
> in case the MdeModulePkg maintainers want to pull the
> code and test it
> during review.

Re: [edk2-devel] [PATCH v5 0/7] MdeModulePkg: add support for dispatching foreign arch PE/COFF images

[Ard Biesheuvel]

On Fri, 12 Apr 2019 at 13:58, Kinney, Michael D
<michael.d.kinney@intel.com> wrote:
>
> Ard,
>
> We have verified that the build failures are resolved.
>
> Series Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com>
>


Mike,

Thanks a lot for the effort in testing and reviewing.

I'll wait for the MdeModulePkg maintainers' feedback before spinning a
final version and pushing it.

Thanks,
Ard.