Self-replicating image

Self-replicating image

[Tomas Pilar (tpilar)]

Hi,

I am currently pondering the most elegant way to implement capsule update for our devices that would work in the presence of multiple devices in the host.

Capsule allows embedding a driver that is executed prior to the update, which is very handy. Crypto library is quite large and would not fit into an OptionROM, so being able to supply FMP driver in the capsule is great.

However, if only one instance of the driver loads, the FMP upstream is currently written to support only one device per instance. So I wonder if there is a easy, neat way for my image to replicate on DriverBinding so that I end up with one instance per device.

It looks like I should be able to do it with gBS->LoadImage() and passing information about currently loaded image though I might have to CopyMem() the image itself to new location.

Thoughts?

Cheers,
Tom

Re: Self-replicating image

[Andrew Fish]

> On Feb 8, 2019, at 6:42 AM, Tomas Pilar (tpilar) <tpilar@solarflare.com> wrote:
> 
> Hi,
> 
> I am currently pondering the most elegant way to implement capsule update for our devices that would work in the presence of multiple devices in the host.
> 
> Capsule allows embedding a driver that is executed prior to the update, which is very handy. Crypto library is quite large and would not fit into an OptionROM, so being able to supply FMP driver in the capsule is great.
> 
> However, if only one instance of the driver loads, the FMP upstream is currently written to support only one device per instance. So I wonder if there is a easy, neat way for my image to replicate on DriverBinding so that I end up with one instance per device.
> 


Tom,

The usually model in EFI is to have one driver handle multiple things. 

> It looks like I should be able to do it with gBS->LoadImage() and passing information about currently loaded image though I might have to CopyMem() the image itself to new location.
> 

gBS->LoadImage() will load and relocate the image to a malloced address of the correct memory type for the image. The inputs are just the source of the image, so no need to CopyMem() anything. gBS->StartImage() calls the entry point.

Thanks,

Andrew Fish

> Thoughts?
> 
> Cheers,
> Tom
> _______________________________________________
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel

Re: Self-replicating image

[Kinney, Michael D]

Hi Thomas,

I have been looking into this topic of multiple controllers this
week.  I have some prototype code that I think resolves the issues
but needs a bit more work on some corner cases.

I am using the PCI Option ROM use case to evaluate the changes.
PCI Option ROM can use Bus Specific Driver Override Protocol so
the driver from the option ROM manages the PCI controller the option
ROM is attached.  PCI Option ROMs can also use the Driver Family
Override Protocol so one of the PCI Option ROMs can manage a group
of PCI controllers.

It is also possible for an FMP driver for integrated devices to
manage multiple integrated devices if there is more than one of
the same device with FW storage.  The multiple controller use case
is not limited to busses like PCI.

The current version of the FmpDeviceLib is optimized for an FMP
instance that manages a single FW storage device.  If an FmpDeviceLib
is intended to manage multiple FW storage devices, then a few
extra services in the FmpDeviceLib are required.

The concept is to extend the FmpDeviceLib with a couple extra
APIs that add support for Stop()/Unload() operations and to
to set the context for the current FmpDeviceLib actions.

Have you entered a BZ for this issue yet?  I can start adding
details in the BZ and post some proposed changes soon.

Best regards,

Mike

> -----Original Message-----
> From: edk2-devel [mailto:edk2-devel-
> bounces@lists.01.org] On Behalf Of Andrew Fish via
> edk2-devel
> Sent: Friday, February 8, 2019 9:16 AM
> To: Tomas Pilar (tpilar) <tpilar@solarflare.com>
> Cc: edk2-devel@lists.01.org
> Subject: Re: [edk2] Self-replicating image
> 
> 
> 
> > On Feb 8, 2019, at 6:42 AM, Tomas Pilar (tpilar)
> <tpilar@solarflare.com> wrote:
> >
> > Hi,
> >
> > I am currently pondering the most elegant way to
> implement capsule update for our devices that would
> work in the presence of multiple devices in the host.
> >
> > Capsule allows embedding a driver that is executed
> prior to the update, which is very handy. Crypto
> library is quite large and would not fit into an
> OptionROM, so being able to supply FMP driver in the
> capsule is great.
> >
> > However, if only one instance of the driver loads,
> the FMP upstream is currently written to support only
> one device per instance. So I wonder if there is a
> easy, neat way for my image to replicate on
> DriverBinding so that I end up with one instance per
> device.
> >
> 
> 
> Tom,
> 
> The usually model in EFI is to have one driver handle
> multiple things.
> 
> > It looks like I should be able to do it with gBS-
> >LoadImage() and passing information about currently
> loaded image though I might have to CopyMem() the image
> itself to new location.
> >
> 
> gBS->LoadImage() will load and relocate the image to a
> malloced address of the correct memory type for the
> image. The inputs are just the source of the image, so
> no need to CopyMem() anything. gBS->StartImage() calls
> the entry point.
> 
> Thanks,
> 
> Andrew Fish
> 
> > Thoughts?
> >
> > Cheers,
> > Tom
> > _______________________________________________
> > edk2-devel mailing list
> > edk2-devel@lists.01.org
> > https://lists.01.org/mailman/listinfo/edk2-devel
> 
> _______________________________________________
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel

Re: Self-replicating image

[Tomas Pilar (tpilar)]

Hi Mike,

Sorry to say I have not yet filed a BZ. Would you like me to, or are you happy doing it?

Cheers,
Tom

On 08/02/2019 17:59, Kinney, Michael D wrote:
> Hi Thomas,
>
> I have been looking into this topic of multiple controllers this
> week.  I have some prototype code that I think resolves the issues
> but needs a bit more work on some corner cases.
>
> I am using the PCI Option ROM use case to evaluate the changes.
> PCI Option ROM can use Bus Specific Driver Override Protocol so
> the driver from the option ROM manages the PCI controller the option
> ROM is attached.  PCI Option ROMs can also use the Driver Family
> Override Protocol so one of the PCI Option ROMs can manage a group
> of PCI controllers.
>
> It is also possible for an FMP driver for integrated devices to
> manage multiple integrated devices if there is more than one of
> the same device with FW storage.  The multiple controller use case
> is not limited to busses like PCI.
>
> The current version of the FmpDeviceLib is optimized for an FMP
> instance that manages a single FW storage device.  If an FmpDeviceLib
> is intended to manage multiple FW storage devices, then a few
> extra services in the FmpDeviceLib are required.
>
> The concept is to extend the FmpDeviceLib with a couple extra
> APIs that add support for Stop()/Unload() operations and to
> to set the context for the current FmpDeviceLib actions.
>
> Have you entered a BZ for this issue yet?  I can start adding
> details in the BZ and post some proposed changes soon.
>
> Best regards,
>
> Mike
>
>> -----Original Message-----
>> From: edk2-devel [mailto:edk2-devel-
>> bounces@lists.01.org] On Behalf Of Andrew Fish via
>> edk2-devel
>> Sent: Friday, February 8, 2019 9:16 AM
>> To: Tomas Pilar (tpilar) <tpilar@solarflare.com>
>> Cc: edk2-devel@lists.01.org
>> Subject: Re: [edk2] Self-replicating image
>>
>>
>>
>>> On Feb 8, 2019, at 6:42 AM, Tomas Pilar (tpilar)
>> <tpilar@solarflare.com> wrote:
>>> Hi,
>>>
>>> I am currently pondering the most elegant way to
>> implement capsule update for our devices that would
>> work in the presence of multiple devices in the host.
>>> Capsule allows embedding a driver that is executed
>> prior to the update, which is very handy. Crypto
>> library is quite large and would not fit into an
>> OptionROM, so being able to supply FMP driver in the
>> capsule is great.
>>> However, if only one instance of the driver loads,
>> the FMP upstream is currently written to support only
>> one device per instance. So I wonder if there is a
>> easy, neat way for my image to replicate on
>> DriverBinding so that I end up with one instance per
>> device.
>>
>> Tom,
>>
>> The usually model in EFI is to have one driver handle
>> multiple things.
>>
>>> It looks like I should be able to do it with gBS-
>>> LoadImage() and passing information about currently
>> loaded image though I might have to CopyMem() the image
>> itself to new location.
>> gBS->LoadImage() will load and relocate the image to a
>> malloced address of the correct memory type for the
>> image. The inputs are just the source of the image, so
>> no need to CopyMem() anything. gBS->StartImage() calls
>> the entry point.
>>
>> Thanks,
>>
>> Andrew Fish
>>
>>> Thoughts?
>>>
>>> Cheers,
>>> Tom
>>> _______________________________________________
>>> edk2-devel mailing list
>>> edk2-devel@lists.01.org
>>> https://lists.01.org/mailman/listinfo/edk2-devel
>> _______________________________________________
>> edk2-devel mailing list
>> edk2-devel@lists.01.org
>> https://lists.01.org/mailman/listinfo/edk2-devel

Re: Self-replicating image

[Kinney, Michael D]

Hi Tomas,

I have posted a branch in edk2-staging with the proposed
design changes to support multiple controller and a functional
example in OVMF using multiple E1000 adapters.

https://github.com/tianocore/edk2-staging/tree/Bug_1525_FmpDevicePkg_MultipleControllers

QEMU has a few limitations, so some aspects are simulated,
but you should be able to use the example as a template to
try your use case.

The E1000 UEFI PCI device driver that uses FmpDxe as a
Library is located at the following location.

	OvmfPkg/E1000Fmp

It uses the E1000 specific FmpDeviceLib instance at:

    OvmfPkg/Feature/Capsule/Library/FmpDeviceLibE1000

I have added device specific DSC files for each FMP driver
that are included from the OVMF DSC file.  The one for
E1000 is at:

    OvmfPkg/Feature/Capsule/FmpE1000.dsc

QEMU does not support update of the PCI Option ROM 
mapped to a PCI device.  So I simulate the update of a
FW storage device using a 32-byte UEFI Variable that
is unique for each adapter.

Best regards,

Mike

> -----Original Message-----
> From: Tomas Pilar (tpilar)
> [mailto:tpilar@solarflare.com]
> Sent: Monday, February 11, 2019 2:37 AM
> To: Kinney, Michael D <michael.d.kinney@intel.com>;
> Andrew Fish <afish@apple.com>; edk2-devel@lists.01.org
> Subject: Re: [edk2] Self-replicating image
> 
> Hi Mike,
> 
> Sorry to say I have not yet filed a BZ. Would you like
> me to, or are you happy doing it?
> 
> Cheers,
> Tom
> 
> On 08/02/2019 17:59, Kinney, Michael D wrote:
> > Hi Thomas,
> >
> > I have been looking into this topic of multiple
> controllers this
> > week.  I have some prototype code that I think
> resolves the issues
> > but needs a bit more work on some corner cases.
> >
> > I am using the PCI Option ROM use case to evaluate
> the changes.
> > PCI Option ROM can use Bus Specific Driver Override
> Protocol so
> > the driver from the option ROM manages the PCI
> controller the option
> > ROM is attached.  PCI Option ROMs can also use the
> Driver Family
> > Override Protocol so one of the PCI Option ROMs can
> manage a group
> > of PCI controllers.
> >
> > It is also possible for an FMP driver for integrated
> devices to
> > manage multiple integrated devices if there is more
> than one of
> > the same device with FW storage.  The multiple
> controller use case
> > is not limited to busses like PCI.
> >
> > The current version of the FmpDeviceLib is optimized
> for an FMP
> > instance that manages a single FW storage device.  If
> an FmpDeviceLib
> > is intended to manage multiple FW storage devices,
> then a few
> > extra services in the FmpDeviceLib are required.
> >
> > The concept is to extend the FmpDeviceLib with a
> couple extra
> > APIs that add support for Stop()/Unload() operations
> and to
> > to set the context for the current FmpDeviceLib
> actions.
> >
> > Have you entered a BZ for this issue yet?  I can
> start adding
> > details in the BZ and post some proposed changes
> soon.
> >
> > Best regards,
> >
> > Mike
> >
> >> -----Original Message-----
> >> From: edk2-devel [mailto:edk2-devel-
> >> bounces@lists.01.org] On Behalf Of Andrew Fish via
> >> edk2-devel
> >> Sent: Friday, February 8, 2019 9:16 AM
> >> To: Tomas Pilar (tpilar) <tpilar@solarflare.com>
> >> Cc: edk2-devel@lists.01.org
> >> Subject: Re: [edk2] Self-replicating image
> >>
> >>
> >>
> >>> On Feb 8, 2019, at 6:42 AM, Tomas Pilar (tpilar)
> >> <tpilar@solarflare.com> wrote:
> >>> Hi,
> >>>
> >>> I am currently pondering the most elegant way to
> >> implement capsule update for our devices that would
> >> work in the presence of multiple devices in the
> host.
> >>> Capsule allows embedding a driver that is executed
> >> prior to the update, which is very handy. Crypto
> >> library is quite large and would not fit into an
> >> OptionROM, so being able to supply FMP driver in the
> >> capsule is great.
> >>> However, if only one instance of the driver loads,
> >> the FMP upstream is currently written to support
> only
> >> one device per instance. So I wonder if there is a
> >> easy, neat way for my image to replicate on
> >> DriverBinding so that I end up with one instance per
> >> device.
> >>
> >> Tom,
> >>
> >> The usually model in EFI is to have one driver
> handle
> >> multiple things.
> >>
> >>> It looks like I should be able to do it with gBS-
> >>> LoadImage() and passing information about currently
> >> loaded image though I might have to CopyMem() the
> image
> >> itself to new location.
> >> gBS->LoadImage() will load and relocate the image to
> a
> >> malloced address of the correct memory type for the
> >> image. The inputs are just the source of the image,
> so
> >> no need to CopyMem() anything. gBS->StartImage()
> calls
> >> the entry point.
> >>
> >> Thanks,
> >>
> >> Andrew Fish
> >>
> >>> Thoughts?
> >>>
> >>> Cheers,
> >>> Tom
> >>> _______________________________________________
> >>> edk2-devel mailing list
> >>> edk2-devel@lists.01.org
> >>> https://lists.01.org/mailman/listinfo/edk2-devel
> >> _______________________________________________
> >> edk2-devel mailing list
> >> edk2-devel@lists.01.org
> >> https://lists.01.org/mailman/listinfo/edk2-devel

Re: Self-replicating image

[Tomas Pilar (tpilar)]

Hi Mike,

This all looks really good, I didn't envisage this way to design it (swapping contexts on demand) but it's much simpler than what I had in mind. I should be able to integrate this easily with my current FMP driver for our device and I should be able to get you some good testing.

Cheers,
Tom

On 28/02/2019 03:33, Kinney, Michael D wrote:
> Hi Tomas,
>
> I have posted a branch in edk2-staging with the proposed
> design changes to support multiple controller and a functional
> example in OVMF using multiple E1000 adapters.
>
> https://github.com/tianocore/edk2-staging/tree/Bug_1525_FmpDevicePkg_MultipleControllers
>
> QEMU has a few limitations, so some aspects are simulated,
> but you should be able to use the example as a template to
> try your use case.
>
> The E1000 UEFI PCI device driver that uses FmpDxe as a
> Library is located at the following location.
>
> 	OvmfPkg/E1000Fmp
>
> It uses the E1000 specific FmpDeviceLib instance at:
>
>     OvmfPkg/Feature/Capsule/Library/FmpDeviceLibE1000
>
> I have added device specific DSC files for each FMP driver
> that are included from the OVMF DSC file.  The one for
> E1000 is at:
>
>     OvmfPkg/Feature/Capsule/FmpE1000.dsc
>
> QEMU does not support update of the PCI Option ROM 
> mapped to a PCI device.  So I simulate the update of a
> FW storage device using a 32-byte UEFI Variable that
> is unique for each adapter.
>
> Best regards,
>
> Mike
>
>> -----Original Message-----
>> From: Tomas Pilar (tpilar)
>> [mailto:tpilar@solarflare.com]
>> Sent: Monday, February 11, 2019 2:37 AM
>> To: Kinney, Michael D <michael.d.kinney@intel.com>;
>> Andrew Fish <afish@apple.com>; edk2-devel@lists.01.org
>> Subject: Re: [edk2] Self-replicating image
>>
>> Hi Mike,
>>
>> Sorry to say I have not yet filed a BZ. Would you like
>> me to, or are you happy doing it?
>>
>> Cheers,
>> Tom
>>
>> On 08/02/2019 17:59, Kinney, Michael D wrote:
>>> Hi Thomas,
>>>
>>> I have been looking into this topic of multiple
>> controllers this
>>> week.  I have some prototype code that I think
>> resolves the issues
>>> but needs a bit more work on some corner cases.
>>>
>>> I am using the PCI Option ROM use case to evaluate
>> the changes.
>>> PCI Option ROM can use Bus Specific Driver Override
>> Protocol so
>>> the driver from the option ROM manages the PCI
>> controller the option
>>> ROM is attached.  PCI Option ROMs can also use the
>> Driver Family
>>> Override Protocol so one of the PCI Option ROMs can
>> manage a group
>>> of PCI controllers.
>>>
>>> It is also possible for an FMP driver for integrated
>> devices to
>>> manage multiple integrated devices if there is more
>> than one of
>>> the same device with FW storage.  The multiple
>> controller use case
>>> is not limited to busses like PCI.
>>>
>>> The current version of the FmpDeviceLib is optimized
>> for an FMP
>>> instance that manages a single FW storage device.  If
>> an FmpDeviceLib
>>> is intended to manage multiple FW storage devices,
>> then a few
>>> extra services in the FmpDeviceLib are required.
>>>
>>> The concept is to extend the FmpDeviceLib with a
>> couple extra
>>> APIs that add support for Stop()/Unload() operations
>> and to
>>> to set the context for the current FmpDeviceLib
>> actions.
>>> Have you entered a BZ for this issue yet?  I can
>> start adding
>>> details in the BZ and post some proposed changes
>> soon.
>>> Best regards,
>>>
>>> Mike
>>>
>>>> -----Original Message-----
>>>> From: edk2-devel [mailto:edk2-devel-
>>>> bounces@lists.01.org] On Behalf Of Andrew Fish via
>>>> edk2-devel
>>>> Sent: Friday, February 8, 2019 9:16 AM
>>>> To: Tomas Pilar (tpilar) <tpilar@solarflare.com>
>>>> Cc: edk2-devel@lists.01.org
>>>> Subject: Re: [edk2] Self-replicating image
>>>>
>>>>
>>>>
>>>>> On Feb 8, 2019, at 6:42 AM, Tomas Pilar (tpilar)
>>>> <tpilar@solarflare.com> wrote:
>>>>> Hi,
>>>>>
>>>>> I am currently pondering the most elegant way to
>>>> implement capsule update for our devices that would
>>>> work in the presence of multiple devices in the
>> host.
>>>>> Capsule allows embedding a driver that is executed
>>>> prior to the update, which is very handy. Crypto
>>>> library is quite large and would not fit into an
>>>> OptionROM, so being able to supply FMP driver in the
>>>> capsule is great.
>>>>> However, if only one instance of the driver loads,
>>>> the FMP upstream is currently written to support
>> only
>>>> one device per instance. So I wonder if there is a
>>>> easy, neat way for my image to replicate on
>>>> DriverBinding so that I end up with one instance per
>>>> device.
>>>>
>>>> Tom,
>>>>
>>>> The usually model in EFI is to have one driver
>> handle
>>>> multiple things.
>>>>
>>>>> It looks like I should be able to do it with gBS-
>>>>> LoadImage() and passing information about currently
>>>> loaded image though I might have to CopyMem() the
>> image
>>>> itself to new location.
>>>> gBS->LoadImage() will load and relocate the image to
>> a
>>>> malloced address of the correct memory type for the
>>>> image. The inputs are just the source of the image,
>> so
>>>> no need to CopyMem() anything. gBS->StartImage()
>> calls
>>>> the entry point.
>>>>
>>>> Thanks,
>>>>
>>>> Andrew Fish
>>>>
>>>>> Thoughts?
>>>>>
>>>>> Cheers,
>>>>> Tom
>>>>> _______________________________________________
>>>>> edk2-devel mailing list
>>>>> edk2-devel@lists.01.org
>>>>> https://lists.01.org/mailman/listinfo/edk2-devel
>>>> _______________________________________________
>>>> edk2-devel mailing list
>>>> edk2-devel@lists.01.org
>>>> https://lists.01.org/mailman/listinfo/edk2-devel