Storing Non volatile variables on SD/NAND

Storing Non volatile variables on SD/NAND

[Udit Kumar]

Hi EDK-2 Experts, 
I am looking to store NV variables on SD/NAND device. 

While browsing, I came across some old post at link, 
http://feishare.com/efimail/messages/20130319-1700-Re__edk2__Regarding_storing_Boot_Device_Config_in_persistent_memory-Olivier_Martin.html 

Looks like, this is possible easily. 
>> What you need to support Non-Volatile UEFI variables is a Non-Volatile Memory. And also a driver that implements the EFI Firmware Volume Block protocol for this NVM device.

But MdeModulePkg does Copymem from NV variable start memory to some allocated buffers.  With SD/NAND Copymem is not possible, Is this something changes since 2013 or there are some other way to use SD/NAND 

Thanks
Udit 

Re: Storing Non volatile variables on SD/NAND

[Ard Biesheuvel]

On 18 September 2017 at 06:52, Udit Kumar <udit.kumar@nxp.com> wrote:
> Hi EDK-2 Experts,
> I am looking to store NV variables on SD/NAND device.
>
> While browsing, I came across some old post at link,
> http://feishare.com/efimail/messages/20130319-1700-Re__edk2__Regarding_storing_Boot_Device_Config_in_persistent_memory-Olivier_Martin.html
>
> Looks like, this is possible easily.

That's a bold statement dude :-)

>>> What you need to support Non-Volatile UEFI variables is a Non-Volatile Memory. And also a driver that implements the EFI Firmware Volume Block protocol for this NVM device.
>
> But MdeModulePkg does Copymem from NV variable start memory to some allocated buffers.  With SD/NAND Copymem is not possible, Is this something changes since 2013 or there are some other way to use SD/NAND
>

No, SD/MMC cannot currently be used as the backing store for the EFI
variable store. The problem is that the variable protocols are
architectural protocols in PI that need to be present before any
driver model drivers are dispatched, and so putting the variable store
on block devices is not something that the PI software architecture
currently supports (unless you reimplement the whole driver stack as
DXE drivers).

On top of that, it is almost impossible to share a block device that
sits behind a controller between the firmware and the OS at runtime
(i.e., for SetVariable() calls made by efibootmgr under Linux),
because only a single agent can take ownership of the controller at
any given time. (You /could/ dedicate the SD/MMC to the firmware
entirely, and boot from SATA or USB, but this is out of the question
on most platforms that need to use SD/MMC for that variable backing
store, i.e., mobile platforms)

The best thing would be for you to convince the hardware architects in
your company to design and implement dual-ported SD/MMC controllers
that allow a single SD/MMC to have two logical views that are
independent (although I'm unsure if that is even possible in the
context of the SD/MMC specifications)

Thanks,
Ard.

Re: Storing Non volatile variables on SD/NAND

[Vladimir Olovyannikov]

> -----Original Message-----
> From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of
> Ard Biesheuvel
> Sent: Monday, September 18, 2017 8:43 AM
> To: Udit Kumar
> Cc: grant.likely@linaro.org.; edk2-devel@lists.01.org;
> Olivier.Martin@arm.com
> Subject: Re: [edk2] Storing Non volatile variables on SD/NAND
>
> On 18 September 2017 at 06:52, Udit Kumar <udit.kumar@nxp.com> wrote:
> > Hi EDK-2 Experts,
> > I am looking to store NV variables on SD/NAND device.
> >
> > While browsing, I came across some old post at link,
> > http://feishare.com/efimail/messages/20130319-1700-
> Re__edk2__Regarding
> > _storing_Boot_Device_Config_in_persistent_memory-Olivier_Martin.html
> >
> > Looks like, this is possible easily.
>
> That's a bold statement dude :-)
>
> >>> What you need to support Non-Volatile UEFI variables is a
Non-Volatile
> Memory. And also a driver that implements the EFI Firmware Volume Block
> protocol for this NVM device.
> >
> > But MdeModulePkg does Copymem from NV variable start memory to
> some
> > allocated buffers.  With SD/NAND Copymem is not possible, Is this
> > something changes since 2013 or there are some other way to use
> > SD/NAND
> >
>
> No, SD/MMC cannot currently be used as the backing store for the EFI
> variable store. The problem is that the variable protocols are
architectural
> protocols in PI that need to be present before any driver model drivers
are
> dispatched, and so putting the variable store on block devices is not
> something that the PI software architecture currently supports (unless
you
> reimplement the whole driver stack as DXE drivers).
>
> On top of that, it is almost impossible to share a block device that
sits behind
> a controller between the firmware and the OS at runtime (i.e., for
> SetVariable() calls made by efibootmgr under Linux), because only a
single
> agent can take ownership of the controller at any given time. (You
/could/
> dedicate the SD/MMC to the firmware entirely, and boot from SATA or USB,
> but this is out of the question on most platforms that need to use
SD/MMC
> for that variable backing store, i.e., mobile platforms)
>
> The best thing would be for you to convince the hardware architects in
your
> company to design and implement dual-ported SD/MMC controllers that
> allow a single SD/MMC to have two logical views that are independent
> (although I'm unsure if that is even possible in the context of the
SD/MMC
> specifications)
>
> Thanks,
> Ard.
Udit,
Ard is absolutely right.

Following design I had to implement variable store on the EMMC boot
partition 1 (not exactly SD card, but close; the same is applied to NAND I
guess).
To do that I forked VariableRuntime driver and changed it to do cache
writes into the store (just modifying store memory) at runtime.
This is because you cannot share eMMC store between OS (Linux)  and the
UEFI service at the same time, and you cannot predict when OS writes/reads
to/from the EMMC device.
So I do cache writes at runtime (just modifying store in memory), and
flush the store on reboot/shutdown.
For this purpose my ResetSystemLib calls into the Variable service and
forces flush. Variable service reinitiates eMMC controller
and really writes into the store on the eMMC. Real flush is also performed
on ExitBootServices().

As a big drawback an OS should either reset or shutdown (get into UEFI
ResetSystemLib) in order for variable store to be updated physically;
Physical flush is also forced on ExitBootServices - be it a reset command,
or booting an OS.
Just pressing a reset button (example: Linux Kernel panic with no reset)
would cause variable changes for this session to get lost.

Thank you,
Vladimir
> _______________________________________________
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel

Re: Storing Non volatile variables on SD/NAND

[Udit Kumar]

Thanks Vladimir, 
With your design, you did delayed write to eMMC due to sharing with OS.  But it works for you:)
Say if eMMC controllers offers you a status bit, if eMMC storage is being used for not. Then this 
could be possible to update at run time, both OS/UEFI needs to check and wait if controller is being used.
For sure,  some synchronization issues need to be ironed out (or maybe I am just dreaming here). 

On part 2) where you forked VariableRuntime driver , could we think of updating VariableRuntime driver, 
to support non-XIP or memory mapped devices. 

Thanks
Udit 

> -----Original Message-----
> From: Vladimir Olovyannikov [mailto:vladimir.olovyannikov@broadcom.com]
> Sent: Monday, September 18, 2017 10:22 PM
> To: Ard Biesheuvel <ard.biesheuvel@linaro.org>; Udit Kumar
> <udit.kumar@nxp.com>
> Cc: grant.likely@linaro.org; edk2-devel@lists.01.org; Olivier.Martin@arm.com
> Subject: RE: [edk2] Storing Non volatile variables on SD/NAND
> 
> > -----Original Message-----
> > From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of
> > Ard Biesheuvel
> > Sent: Monday, September 18, 2017 8:43 AM
> > To: Udit Kumar
> > Cc: grant.likely@linaro.org.; edk2-devel@lists.01.org;
> > Olivier.Martin@arm.com
> > Subject: Re: [edk2] Storing Non volatile variables on SD/NAND
> >
> > On 18 September 2017 at 06:52, Udit Kumar <udit.kumar@nxp.com> wrote:
> > > Hi EDK-2 Experts,
> > > I am looking to store NV variables on SD/NAND device.
> > >
> > > While browsing, I came across some old post at link,
> > > http://feishare.com/efimail/messages/20130319-1700-
> > Re__edk2__Regarding
> > > _storing_Boot_Device_Config_in_persistent_memory-Olivier_Martin.html
> > >
> > > Looks like, this is possible easily.
> >
> > That's a bold statement dude :-)
> >
> > >>> What you need to support Non-Volatile UEFI variables is a
> Non-Volatile
> > Memory. And also a driver that implements the EFI Firmware Volume
> > Block protocol for this NVM device.
> > >
> > > But MdeModulePkg does Copymem from NV variable start memory to
> > some
> > > allocated buffers.  With SD/NAND Copymem is not possible, Is this
> > > something changes since 2013 or there are some other way to use
> > > SD/NAND
> > >
> >
> > No, SD/MMC cannot currently be used as the backing store for the EFI
> > variable store. The problem is that the variable protocols are
> architectural
> > protocols in PI that need to be present before any driver model
> > drivers
> are
> > dispatched, and so putting the variable store on block devices is not
> > something that the PI software architecture currently supports (unless
> you
> > reimplement the whole driver stack as DXE drivers).
> >
> > On top of that, it is almost impossible to share a block device that
> sits behind
> > a controller between the firmware and the OS at runtime (i.e., for
> > SetVariable() calls made by efibootmgr under Linux), because only a
> single
> > agent can take ownership of the controller at any given time. (You
> /could/
> > dedicate the SD/MMC to the firmware entirely, and boot from SATA or
> > USB, but this is out of the question on most platforms that need to
> > use
> SD/MMC
> > for that variable backing store, i.e., mobile platforms)
> >
> > The best thing would be for you to convince the hardware architects in
> your
> > company to design and implement dual-ported SD/MMC controllers that
> > allow a single SD/MMC to have two logical views that are independent
> > (although I'm unsure if that is even possible in the context of the
> SD/MMC
> > specifications)
> >
> > Thanks,
> > Ard.
> Udit,
> Ard is absolutely right.
> 
> Following design I had to implement variable store on the EMMC boot partition
> 1 (not exactly SD card, but close; the same is applied to NAND I guess).
> To do that I forked VariableRuntime driver and changed it to do cache writes
> into the store (just modifying store memory) at runtime.
> This is because you cannot share eMMC store between OS (Linux)  and the UEFI
> service at the same time, and you cannot predict when OS writes/reads to/from
> the EMMC device.
> So I do cache writes at runtime (just modifying store in memory), and flush the
> store on reboot/shutdown.
> For this purpose my ResetSystemLib calls into the Variable service and forces
> flush. Variable service reinitiates eMMC controller and really writes into the
> store on the eMMC. Real flush is also performed on ExitBootServices().
> 
> As a big drawback an OS should either reset or shutdown (get into UEFI
> ResetSystemLib) in order for variable store to be updated physically; Physical
> flush is also forced on ExitBootServices - be it a reset command, or booting an
> OS.
> Just pressing a reset button (example: Linux Kernel panic with no reset) would
> cause variable changes for this session to get lost.
> 
> Thank you,
> Vladimir
> > _______________________________________________
> > edk2-devel mailing list
> > edk2-devel@lists.01.org
> > https://lists.01.org/mailman/listinfo/edk2-devel

Re: Storing Non volatile variables on SD/NAND

[Ard Biesheuvel]

On 18 September 2017 at 22:28, Udit Kumar <udit.kumar@nxp.com> wrote:
> Thanks Vladimir,
> With your design, you did delayed write to eMMC due to sharing with OS.  But it works for you:)
> Say if eMMC controllers offers you a status bit, if eMMC storage is being used for not. Then this
> could be possible to update at run time, both OS/UEFI needs to check and wait if controller is being used.

That is the problem right there. The nice thing about a firmware spec
is that you don't have to care about how it was implemented if you
adhere to the API rules. Imposing additional restrictions (such as
requiring the OS to be careful about not using the eMMC when it may be
in use by the firmware) defeats the purpose of using UEFI, since you
won't be able to use a generic OS anyway.


> For sure,  some synchronization issues need to be ironed out (or maybe I am just dreaming here).
>
> On part 2) where you forked VariableRuntime driver , could we think of updating VariableRuntime driver,
> to support non-XIP or memory mapped devices.
>

I think being able to support non-memorymapped FV volumes for the
variable store would be a big improvement. This does require changes
to both the FaultTolerantWrite drivers and the VariableRuntime
drivers, which both appear in PEI, DXE and SMM flavors, and require
thorough review due to the security impact bugs have in this layer, so
this is a rather large chunk of work to take on.

Re: Storing Non volatile variables on SD/NAND

[Udit Kumar]

> On 18 September 2017 at 22:28, Udit Kumar <udit.kumar@nxp.com> wrote:
> > Thanks Vladimir,
> > With your design, you did delayed write to eMMC due to sharing with
> > OS.  But it works for you:) Say if eMMC controllers offers you a
> > status bit, if eMMC storage is being used for not. Then this could be possible to
> update at run time, both OS/UEFI needs to check and wait if controller is being
> used.
> 
> That is the problem right there. The nice thing about a firmware spec is that you
> don't have to care about how it was implemented if you adhere to the API rules.

Yup, we are fine as long as long UEFI firmware is stored on dedicated media. 

> Imposing additional restrictions (such as requiring the OS to be careful about not
> using the eMMC when it may be in use by the firmware) defeats the purpose of
> using UEFI, since you won't be able to use a generic OS anyway.
> 

Hmm,  so far, I haven't come across where UEFI specs says, we need a separate
Storage for firmware. (May be I missed some part of specs)
Irrespective of storage media, we have this problem if OS and UEFI shares same  
storage. 

> > For sure,  some synchronization issues need to be ironed out (or maybe I am
> just dreaming here).
> >
> > On part 2) where you forked VariableRuntime driver , could we think of
> > updating VariableRuntime driver, to support non-XIP or memory mapped
> devices.
> >
> 
> I think being able to support non-memorymapped FV volumes for the variable
> store would be a big improvement. This does require changes to both the
> FaultTolerantWrite drivers and the VariableRuntime drivers, which both appear
> in PEI, DXE and SMM flavors, and require thorough review due to the security
> impact bugs have in this layer, so this is a rather large chunk of work to take on.

Thanks,  your list is longer than what I was thinking :-)
I think, for embedded world with UEFI, later or sooner, this will be required.

Thanks
Udit

Re: Storing Non volatile variables on SD/NAND

[Andrew Fish]

> On Sep 19, 2017, at 9:27 PM, Udit Kumar <udit.kumar@nxp.com> wrote:
> 
> 
>> On 18 September 2017 at 22:28, Udit Kumar <udit.kumar@nxp.com> wrote:
>>> Thanks Vladimir,
>>> With your design, you did delayed write to eMMC due to sharing with
>>> OS.  But it works for you:) Say if eMMC controllers offers you a
>>> status bit, if eMMC storage is being used for not. Then this could be possible to
>> update at run time, both OS/UEFI needs to check and wait if controller is being
>> used.
>> 
>> That is the problem right there. The nice thing about a firmware spec is that you
>> don't have to care about how it was implemented if you adhere to the API rules.
> 
> Yup, we are fine as long as long UEFI firmware is stored on dedicated media. 
> 
>> Imposing additional restrictions (such as requiring the OS to be careful about not
>> using the eMMC when it may be in use by the firmware) defeats the purpose of
>> using UEFI, since you won't be able to use a generic OS anyway.
>> 
> 
> Hmm,  so far, I haven't come across where UEFI specs says, we need a separate
> Storage for firmware. (May be I missed some part of specs)
> Irrespective of storage media, we have this problem if OS and UEFI shares same  
> storage. 
> 

Udit,

Can you point out the spec that states you can't boot Linux and Windows at the same time on a PC? :)

When you write a spec it is not practical do document what is not possible, you can only document the API the rest is implied by the implementation. So for example the UEFI spec does not document why the firmware and OS can't share a hardware device, just like you can't have 2 operating systems running on bare metal at the same time. It is a little like Occam's Razor the reason that the firmware and the OS can not share a hardware device is the mechanics of how to share a hardware device is not defined in the spec, thus it is not part of the API and not possible. 

Thanks,

Andrew Fish

>>> For sure,  some synchronization issues need to be ironed out (or maybe I am
>> just dreaming here).
>>> 
>>> On part 2) where you forked VariableRuntime driver , could we think of
>>> updating VariableRuntime driver, to support non-XIP or memory mapped
>> devices.
>>> 
>> 
>> I think being able to support non-memorymapped FV volumes for the variable
>> store would be a big improvement. This does require changes to both the
>> FaultTolerantWrite drivers and the VariableRuntime drivers, which both appear
>> in PEI, DXE and SMM flavors, and require thorough review due to the security
>> impact bugs have in this layer, so this is a rather large chunk of work to take on.
> 
> Thanks,  your list is longer than what I was thinking :-)
> I think, for embedded world with UEFI, later or sooner, this will be required.
> 
> Thanks
> Udit
> _______________________________________________
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel

Re: Storing Non volatile variables on SD/NAND

[Udit Kumar]

> > On Sep 19, 2017, at 9:27 PM, Udit Kumar <udit.kumar@nxp.com> wrote:
> >
> >
> >> On 18 September 2017 at 22:28, Udit Kumar <udit.kumar@nxp.com> wrote:
> >>> Thanks Vladimir,
> >>> With your design, you did delayed write to eMMC due to sharing with
> >>> OS.  But it works for you:) Say if eMMC controllers offers you a
> >>> status bit, if eMMC storage is being used for not. Then this could
> >>> be possible to
> >> update at run time, both OS/UEFI needs to check and wait if
> >> controller is being used.
> >>
> >> That is the problem right there. The nice thing about a firmware spec
> >> is that you don't have to care about how it was implemented if you adhere to
> the API rules.
> >
> > Yup, we are fine as long as long UEFI firmware is stored on dedicated media.
> >
> >> Imposing additional restrictions (such as requiring the OS to be
> >> careful about not using the eMMC when it may be in use by the
> >> firmware) defeats the purpose of using UEFI, since you won't be able to use a
> generic OS anyway.
> >>
> >
> > Hmm,  so far, I haven't come across where UEFI specs says, we need a
> > separate Storage for firmware. (May be I missed some part of specs)
> > Irrespective of storage media, we have this problem if OS and UEFI
> > shares same storage.
> >
> 
> Udit,
> 
> Can you point out the spec that states you can't boot Linux and Windows at the
> same time on a PC? :)
> 
> When you write a spec it is not practical do document what is not possible, you
> can only document the API the rest is implied by the implementation. So for
> example the UEFI spec does not document why the firmware and OS can't share
> a hardware device, just like you can't have 2 operating systems running on bare
> metal at the same time. It is a little like Occam's Razor the reason that the
> firmware and the OS can not share a hardware device is the mechanics of how
> to share a hardware device is not defined in the spec, thus it is not part of the
> API and not possible.

Right,  This is left on implementation how to put firmware and OS.
Ideally, keeping both storage separate  is best case, no need to sync between two.

My reply to Ard, was to highlight that in any case (NOR or eMMC /NAND)
if we are keeping OS and firmware on same storage, we will have same 
issue not limited to  eMMC.

For some requirement, if we need to keep firmware and OS on same media, 
Then implementation should make sure there is exclusive access (be it
NOR controller, SD controller etc). 

Thanks
Udit

> Thanks,
> 
> Andrew Fish
> 
> >>> For sure,  some synchronization issues need to be ironed out (or
> >>> maybe I am
> >> just dreaming here).
> >>>
> >>> On part 2) where you forked VariableRuntime driver , could we think
> >>> of updating VariableRuntime driver, to support non-XIP or memory
> >>> mapped
> >> devices.
> >>>
> >>
> >> I think being able to support non-memorymapped FV volumes for the
> >> variable store would be a big improvement. This does require changes
> >> to both the FaultTolerantWrite drivers and the VariableRuntime
> >> drivers, which both appear in PEI, DXE and SMM flavors, and require
> >> thorough review due to the security impact bugs have in this layer, so this is a
> rather large chunk of work to take on.
> >
> > Thanks,  your list is longer than what I was thinking :-) I think, for
> > embedded world with UEFI, later or sooner, this will be required.
> >
> > Thanks
> > Udit
> > _______________________________________________
> > edk2-devel mailing list
> > edk2-devel@lists.01.org
> > https://lists.01.org/mailman/listinfo/edk2-devel

Re: Storing Non volatile variables on SD/NAND

[Andrew Fish]

> On Sep 19, 2017, at 10:09 PM, Udit Kumar <udit.kumar@nxp.com> wrote:
> 
>>> On Sep 19, 2017, at 9:27 PM, Udit Kumar <udit.kumar@nxp.com> wrote:
>>> 
>>> 
>>>> On 18 September 2017 at 22:28, Udit Kumar <udit.kumar@nxp.com> wrote:
>>>>> Thanks Vladimir,
>>>>> With your design, you did delayed write to eMMC due to sharing with
>>>>> OS.  But it works for you:) Say if eMMC controllers offers you a
>>>>> status bit, if eMMC storage is being used for not. Then this could
>>>>> be possible to
>>>> update at run time, both OS/UEFI needs to check and wait if
>>>> controller is being used.
>>>> 
>>>> That is the problem right there. The nice thing about a firmware spec
>>>> is that you don't have to care about how it was implemented if you adhere to
>> the API rules.
>>> 
>>> Yup, we are fine as long as long UEFI firmware is stored on dedicated media.
>>> 
>>>> Imposing additional restrictions (such as requiring the OS to be
>>>> careful about not using the eMMC when it may be in use by the
>>>> firmware) defeats the purpose of using UEFI, since you won't be able to use a
>> generic OS anyway.
>>>> 
>>> 
>>> Hmm,  so far, I haven't come across where UEFI specs says, we need a
>>> separate Storage for firmware. (May be I missed some part of specs)
>>> Irrespective of storage media, we have this problem if OS and UEFI
>>> shares same storage.
>>> 
>> 
>> Udit,
>> 
>> Can you point out the spec that states you can't boot Linux and Windows at the
>> same time on a PC? :)
>> 
>> When you write a spec it is not practical do document what is not possible, you
>> can only document the API the rest is implied by the implementation. So for
>> example the UEFI spec does not document why the firmware and OS can't share
>> a hardware device, just like you can't have 2 operating systems running on bare
>> metal at the same time. It is a little like Occam's Razor the reason that the
>> firmware and the OS can not share a hardware device is the mechanics of how
>> to share a hardware device is not defined in the spec, thus it is not part of the
>> API and not possible.
> 
> Right,  This is left on implementation how to put firmware and OS.
> Ideally, keeping both storage separate  is best case, no need to sync between two.
> 
> My reply to Ard, was to highlight that in any case (NOR or eMMC /NAND)
> if we are keeping OS and firmware on same storage, we will have same 
> issue not limited to  eMMC.
> 
> For some requirement, if we need to keep firmware and OS on same media, 
> Then implementation should make sure there is exclusive access (be it
> NOR controller, SD controller etc). 
> 

Udit,

Sorry I'm a little swamped on my email right now and might be a little behind on the thread....

Yea the only way to realistically Implement an EFI runtime service in UEFI is to have UEFI own the hardware device. There is no architecture for sharing the device, and the type of device is not really relevant. 

Thanks,

Andrew Fish

> Thanks
> Udit
> 
>> Thanks,
>> 
>> Andrew Fish
>> 
>>>>> For sure,  some synchronization issues need to be ironed out (or
>>>>> maybe I am
>>>> just dreaming here).
>>>>> 
>>>>> On part 2) where you forked VariableRuntime driver , could we think
>>>>> of updating VariableRuntime driver, to support non-XIP or memory
>>>>> mapped
>>>> devices.
>>>>> 
>>>> 
>>>> I think being able to support non-memorymapped FV volumes for the
>>>> variable store would be a big improvement. This does require changes
>>>> to both the FaultTolerantWrite drivers and the VariableRuntime
>>>> drivers, which both appear in PEI, DXE and SMM flavors, and require
>>>> thorough review due to the security impact bugs have in this layer, so this is a
>> rather large chunk of work to take on.
>>> 
>>> Thanks,  your list is longer than what I was thinking :-) I think, for
>>> embedded world with UEFI, later or sooner, this will be required.
>>> 
>>> Thanks
>>> Udit
>>> _______________________________________________
>>> edk2-devel mailing list
>>> edk2-devel@lists.01.org
>>> https://lists.01.org/mailman/listinfo/edk2-devel

Re: Storing Non volatile variables on SD/NAND

[Pankaj Bansal]

This use case also arises for single-board systems like raspberry-pi, which do not have an onboard flash.
The boot firmware/bootloader as well as operating system are loaded from SD card.
https://www.raspberrypi.org/documentation/configuration/config-txt/

Thanks & Regards,
Pankaj Bansal

-----Original Message-----
From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of Andrew Fish
Sent: Wednesday, September 20, 2017 10:48 AM
To: Udit Kumar <udit.kumar@nxp.com>
Cc: edk2-devel@lists.01.org; Vladimir Olovyannikov <vladimir.olovyannikov@broadcom.com>; Olivier.Martin@arm.com; Ard Biesheuvel <ard.biesheuvel@linaro.org>
Subject: Re: [edk2] Storing Non volatile variables on SD/NAND


> On Sep 19, 2017, at 10:09 PM, Udit Kumar <udit.kumar@nxp.com> wrote:
> 
>>> On Sep 19, 2017, at 9:27 PM, Udit Kumar <udit.kumar@nxp.com> wrote:
>>> 
>>> 
>>>> On 18 September 2017 at 22:28, Udit Kumar <udit.kumar@nxp.com> wrote:
>>>>> Thanks Vladimir,
>>>>> With your design, you did delayed write to eMMC due to sharing 
>>>>> with OS.  But it works for you:) Say if eMMC controllers offers 
>>>>> you a status bit, if eMMC storage is being used for not. Then this 
>>>>> could be possible to
>>>> update at run time, both OS/UEFI needs to check and wait if 
>>>> controller is being used.
>>>> 
>>>> That is the problem right there. The nice thing about a firmware 
>>>> spec is that you don't have to care about how it was implemented if 
>>>> you adhere to
>> the API rules.
>>> 
>>> Yup, we are fine as long as long UEFI firmware is stored on dedicated media.
>>> 
>>>> Imposing additional restrictions (such as requiring the OS to be 
>>>> careful about not using the eMMC when it may be in use by the
>>>> firmware) defeats the purpose of using UEFI, since you won't be 
>>>> able to use a
>> generic OS anyway.
>>>> 
>>> 
>>> Hmm,  so far, I haven't come across where UEFI specs says, we need a 
>>> separate Storage for firmware. (May be I missed some part of specs) 
>>> Irrespective of storage media, we have this problem if OS and UEFI 
>>> shares same storage.
>>> 
>> 
>> Udit,
>> 
>> Can you point out the spec that states you can't boot Linux and 
>> Windows at the same time on a PC? :)
>> 
>> When you write a spec it is not practical do document what is not 
>> possible, you can only document the API the rest is implied by the 
>> implementation. So for example the UEFI spec does not document why 
>> the firmware and OS can't share a hardware device, just like you 
>> can't have 2 operating systems running on bare metal at the same 
>> time. It is a little like Occam's Razor the reason that the firmware 
>> and the OS can not share a hardware device is the mechanics of how to 
>> share a hardware device is not defined in the spec, thus it is not part of the API and not possible.
> 
> Right,  This is left on implementation how to put firmware and OS.
> Ideally, keeping both storage separate  is best case, no need to sync between two.
> 
> My reply to Ard, was to highlight that in any case (NOR or eMMC /NAND) 
> if we are keeping OS and firmware on same storage, we will have same 
> issue not limited to  eMMC.
> 
> For some requirement, if we need to keep firmware and OS on same 
> media, Then implementation should make sure there is exclusive access 
> (be it NOR controller, SD controller etc).
> 

Udit,

Sorry I'm a little swamped on my email right now and might be a little behind on the thread....

Yea the only way to realistically Implement an EFI runtime service in UEFI is to have UEFI own the hardware device. There is no architecture for sharing the device, and the type of device is not really relevant. 

Thanks,

Andrew Fish

> Thanks
> Udit
> 
>> Thanks,
>> 
>> Andrew Fish
>> 
>>>>> For sure,  some synchronization issues need to be ironed out (or 
>>>>> maybe I am
>>>> just dreaming here).
>>>>> 
>>>>> On part 2) where you forked VariableRuntime driver , could we 
>>>>> think of updating VariableRuntime driver, to support non-XIP or 
>>>>> memory mapped
>>>> devices.
>>>>> 
>>>> 
>>>> I think being able to support non-memorymapped FV volumes for the 
>>>> variable store would be a big improvement. This does require 
>>>> changes to both the FaultTolerantWrite drivers and the 
>>>> VariableRuntime drivers, which both appear in PEI, DXE and SMM 
>>>> flavors, and require thorough review due to the security impact 
>>>> bugs have in this layer, so this is a
>> rather large chunk of work to take on.
>>> 
>>> Thanks,  your list is longer than what I was thinking :-) I think, 
>>> for embedded world with UEFI, later or sooner, this will be required.
>>> 
>>> Thanks
>>> Udit
>>> _______________________________________________
>>> edk2-devel mailing list
>>> edk2-devel@lists.01.org
>>> https://lists.01.org/mailman/listinfo/edk2-devel

_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: Storing Non volatile variables on SD/NAND

[Udit Kumar]

When we want to have UEFI and OS accessing same media , 
Possibilities I see 

1- Patch OS For status check of media (diversion from generic OS), Good case will be modify low level driver. 
But we may end up some surprises on synchronization.
 
2- no runtime service for OS . I guess this will not be possible

3- Way the  Vladimir implemented for eMMC, This has risk of losing data in case of AC power off. 

4- update hardware with dual view (Ard suggestion) 

Thanks to add, if I missed some option here. 

> This use case also arises for single-board systems like raspberry-pi, which do not
> have an onboard flash.
> The boot firmware/bootloader as well as operating system are loaded from SD
> card.
> https://www.raspberrypi.org/documentation/configuration/config-txt/


Raspberry follow the different boot scheme.  What I think, Linux running on Raspberry/ARM is unware of UEFI boot. 

Copied text 
Stage 1 boot is in the on-chip ROM. Loads Stage 2 in the L2 cache
Stage 2 is bootcode.bin. Enables SDRAM and loads Stage 3
Stage 3 is loader.bin. It knows about the .elf format and loads start.elf
Stage 4: start.elf loads kernel.img. It then also reads config.txt, cmdline.txt 
and bcm2835.dtb If the dtb file exists, it is loaded at 0×100 & kernel @ 0×8000 
If disable_commandline_tags is set it loads kernel @ 0×0 Otherwise it loads kernel @ 0×8000 and put ATAGS at 0×100
Stage 5: kernel.img is then run on the ARM

I think up to stage 4, we have GPU . 
Here SD is exclusivity used by UEFI or OS. Any corrections ?  
 

Thanks
Udit

> -----Original Message-----
> From: Pankaj Bansal
> Sent: Wednesday, September 20, 2017 8:22 PM
> To: Andrew Fish <afish@apple.com>; Udit Kumar <udit.kumar@nxp.com>; edk2-
> devel@lists.01.org
> Cc: Vladimir Olovyannikov <vladimir.olovyannikov@broadcom.com>;
> Olivier.Martin@arm.com; Ard Biesheuvel <ard.biesheuvel@linaro.org>
> Subject: RE: [edk2] Storing Non volatile variables on SD/NAND
> 
> This use case also arises for single-board systems like raspberry-pi, which do not
> have an onboard flash.
> The boot firmware/bootloader as well as operating system are loaded from SD
> card.
> https://www.raspberrypi.org/documentation/configuration/config-txt/
> 
> Thanks & Regards,
> Pankaj Bansal
> 
> -----Original Message-----
> From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of
> Andrew Fish
> Sent: Wednesday, September 20, 2017 10:48 AM
> To: Udit Kumar <udit.kumar@nxp.com>
> Cc: edk2-devel@lists.01.org; Vladimir Olovyannikov
> <vladimir.olovyannikov@broadcom.com>; Olivier.Martin@arm.com; Ard
> Biesheuvel <ard.biesheuvel@linaro.org>
> Subject: Re: [edk2] Storing Non volatile variables on SD/NAND
> 
> 
> > On Sep 19, 2017, at 10:09 PM, Udit Kumar <udit.kumar@nxp.com> wrote:
> >
> >>> On Sep 19, 2017, at 9:27 PM, Udit Kumar <udit.kumar@nxp.com> wrote:
> >>>
> >>>
> >>>> On 18 September 2017 at 22:28, Udit Kumar <udit.kumar@nxp.com>
> wrote:
> >>>>> Thanks Vladimir,
> >>>>> With your design, you did delayed write to eMMC due to sharing
> >>>>> with OS.  But it works for you:) Say if eMMC controllers offers
> >>>>> you a status bit, if eMMC storage is being used for not. Then this
> >>>>> could be possible to
> >>>> update at run time, both OS/UEFI needs to check and wait if
> >>>> controller is being used.
> >>>>
> >>>> That is the problem right there. The nice thing about a firmware
> >>>> spec is that you don't have to care about how it was implemented if
> >>>> you adhere to
> >> the API rules.
> >>>
> >>> Yup, we are fine as long as long UEFI firmware is stored on dedicated
> media.
> >>>
> >>>> Imposing additional restrictions (such as requiring the OS to be
> >>>> careful about not using the eMMC when it may be in use by the
> >>>> firmware) defeats the purpose of using UEFI, since you won't be
> >>>> able to use a
> >> generic OS anyway.
> >>>>
> >>>
> >>> Hmm,  so far, I haven't come across where UEFI specs says, we need a
> >>> separate Storage for firmware. (May be I missed some part of specs)
> >>> Irrespective of storage media, we have this problem if OS and UEFI
> >>> shares same storage.
> >>>
> >>
> >> Udit,
> >>
> >> Can you point out the spec that states you can't boot Linux and
> >> Windows at the same time on a PC? :)
> >>
> >> When you write a spec it is not practical do document what is not
> >> possible, you can only document the API the rest is implied by the
> >> implementation. So for example the UEFI spec does not document why
> >> the firmware and OS can't share a hardware device, just like you
> >> can't have 2 operating systems running on bare metal at the same
> >> time. It is a little like Occam's Razor the reason that the firmware
> >> and the OS can not share a hardware device is the mechanics of how to
> >> share a hardware device is not defined in the spec, thus it is not part of the
> API and not possible.
> >
> > Right,  This is left on implementation how to put firmware and OS.
> > Ideally, keeping both storage separate  is best case, no need to sync between
> two.
> >
> > My reply to Ard, was to highlight that in any case (NOR or eMMC /NAND)
> > if we are keeping OS and firmware on same storage, we will have same
> > issue not limited to  eMMC.
> >
> > For some requirement, if we need to keep firmware and OS on same
> > media, Then implementation should make sure there is exclusive access
> > (be it NOR controller, SD controller etc).
> >
> 
> Udit,
> 
> Sorry I'm a little swamped on my email right now and might be a little behind on
> the thread....
> 
> Yea the only way to realistically Implement an EFI runtime service in UEFI is to
> have UEFI own the hardware device. There is no architecture for sharing the
> device, and the type of device is not really relevant.
> 
> Thanks,
> 
> Andrew Fish
> 
> > Thanks
> > Udit
> >
> >> Thanks,
> >>
> >> Andrew Fish
> >>
> >>>>> For sure,  some synchronization issues need to be ironed out (or
> >>>>> maybe I am
> >>>> just dreaming here).
> >>>>>
> >>>>> On part 2) where you forked VariableRuntime driver , could we
> >>>>> think of updating VariableRuntime driver, to support non-XIP or
> >>>>> memory mapped
> >>>> devices.
> >>>>>
> >>>>
> >>>> I think being able to support non-memorymapped FV volumes for the
> >>>> variable store would be a big improvement. This does require
> >>>> changes to both the FaultTolerantWrite drivers and the
> >>>> VariableRuntime drivers, which both appear in PEI, DXE and SMM
> >>>> flavors, and require thorough review due to the security impact
> >>>> bugs have in this layer, so this is a
> >> rather large chunk of work to take on.
> >>>
> >>> Thanks,  your list is longer than what I was thinking :-) I think,
> >>> for embedded world with UEFI, later or sooner, this will be required.
> >>>
> >>> Thanks
> >>> Udit
> >>> _______________________________________________
> >>> edk2-devel mailing list
> >>> edk2-devel@lists.01.org
> >>> https://lists.01.org/mailman/listinfo/edk2-devel
> 
> _______________________________________________
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel

Re: Storing Non volatile variables on SD/NAND

[Ard Biesheuvel]

On 20 September 2017 at 10:34, Udit Kumar <udit.kumar@nxp.com> wrote:
>
> When we want to have UEFI and OS accessing same media ,
> Possibilities I see
>
> 1- Patch OS For status check of media (diversion from generic OS), Good case will be modify low level driver.
> But we may end up some surprises on synchronization.
>
> 2- no runtime service for OS . I guess this will not be possible
>
> 3- Way the  Vladimir implemented for eMMC, This has risk of losing data in case of AC power off.
>
> 4- update hardware with dual view (Ard suggestion)
>

5 - abstract direct block device access into a firmware service that
is exposed via a DXE_RUNTIME_DRIVER.

The UEFI spec allows you to expose entry points into a
DXE_RUNTIME_DRIVER module via a UEFI configuration table, and the OS
can use a driver that uses the abstracted device rather than the real
device. Performance is going to be terrible, probably, and lots of
things that are specific to SD/MMC will no longer work, but it is a
possibility nonetheless.

Re: Storing Non volatile variables on SD/NAND

[Ard Biesheuvel]

On 20 September 2017 at 10:39, Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
> On 20 September 2017 at 10:34, Udit Kumar <udit.kumar@nxp.com> wrote:
>>
>> When we want to have UEFI and OS accessing same media ,
>> Possibilities I see
>>
>> 1- Patch OS For status check of media (diversion from generic OS), Good case will be modify low level driver.
>> But we may end up some surprises on synchronization.
>>
>> 2- no runtime service for OS . I guess this will not be possible
>>
>> 3- Way the  Vladimir implemented for eMMC, This has risk of losing data in case of AC power off.
>>
>> 4- update hardware with dual view (Ard suggestion)
>>
>
> 5 - abstract direct block device access into a firmware service that
> is exposed via a DXE_RUNTIME_DRIVER.
>
> The UEFI spec allows you to expose entry points into a
> DXE_RUNTIME_DRIVER module via a UEFI configuration table, and the OS
> can use a driver that uses the abstracted device rather than the real
> device. Performance is going to be terrible, probably, and lots of
> things that are specific to SD/MMC will no longer work, but it is a
> possibility nonetheless.

BTW this would go beyond the UEFI spec, and would effectively be a
PI/UEFI dependent feature.

Re: Storing Non volatile variables on SD/NAND

[Andrew Fish]

> On Sep 20, 2017, at 10:39 AM, Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
> 
> On 20 September 2017 at 10:39, Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
>> On 20 September 2017 at 10:34, Udit Kumar <udit.kumar@nxp.com> wrote:
>>> 
>>> When we want to have UEFI and OS accessing same media ,
>>> Possibilities I see
>>> 
>>> 1- Patch OS For status check of media (diversion from generic OS), Good case will be modify low level driver.
>>> But we may end up some surprises on synchronization.
>>> 
>>> 2- no runtime service for OS . I guess this will not be possible
>>> 
>>> 3- Way the  Vladimir implemented for eMMC, This has risk of losing data in case of AC power off.
>>> 
>>> 4- update hardware with dual view (Ard suggestion)
>>> 
>> 
>> 5 - abstract direct block device access into a firmware service that
>> is exposed via a DXE_RUNTIME_DRIVER.
>> 
>> The UEFI spec allows you to expose entry points into a
>> DXE_RUNTIME_DRIVER module via a UEFI configuration table, and the OS
>> can use a driver that uses the abstracted device rather than the real
>> device. Performance is going to be terrible, probably, and lots of
>> things that are specific to SD/MMC will no longer work, but it is a
>> possibility nonetheless.
> 
> BTW this would go beyond the UEFI spec, and would effectively be a
> PI/UEFI dependent feature.

The concept of a runtime EFI driver is in the UEFI spec, but the issue is there is no way to tell the OS to not bind it's driver for that device that is universal. If you boot an unmodified OS it is going to conflict with the EFI runtime service. 

My gut feel is if the OS has a driver for the device it may be better to make the media format the architectural point. That way the OS can read/write it via a driver/app. There could be an UEFI Services Table entry that implies what scheme is being used. That way there is only every one owner of the hardware device. I assume #3 is like this?

Thanks,

Andrew Fish

> _______________________________________________
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel

Re: Storing Non volatile variables on SD/NAND

[Udit Kumar]

Hi Andrew, 

> The concept of a runtime EFI driver is in the UEFI spec, but the issue is there is no
> way to tell the OS to not bind it's driver for that device that is universal. If you
> boot an unmodified OS it is going to conflict with the EFI runtime service.

In order to bind driver with universal device, OS driver could be modified
and device tree could indicate if this storage is shared with BIOS runtime. (Assuming booting Linux)
 
But I am struggling here, how OS will get services of this driver. 
I assume in Linux, struct efi_runtime_services is fixed  as per specs. 

Will this modified driver, needs to access unsigned long runtime; of struct efi to get 
UEFI run time services ? 


> My gut feel is if the OS has a driver for the device it may be better to make the
> media format the architectural point. That way the OS can read/write it via a
> driver/app. There could be an UEFI Services Table entry that implies what
> scheme is being used. That way there is only every one owner of the hardware
> device. I assume #3 is like this?

Yeah , probably can think of this,  if OS update the runtime variables without
help of UEFI runtime services using some sort of file system.  

Thanks
Udit

> -----Original Message-----
> From: afish@apple.com [mailto:afish@apple.com]
> Sent: Wednesday, September 20, 2017 11:16 PM
> To: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> Cc: Udit Kumar <udit.kumar@nxp.com>; edk2-devel@lists.01.org;
> Olivier.Martin@arm.com; Vladimir Olovyannikov
> <vladimir.olovyannikov@broadcom.com>
> Subject: Re: [edk2] Storing Non volatile variables on SD/NAND
> 
> 
> > On Sep 20, 2017, at 10:39 AM, Ard Biesheuvel <ard.biesheuvel@linaro.org>
> wrote:
> >
> > On 20 September 2017 at 10:39, Ard Biesheuvel <ard.biesheuvel@linaro.org>
> wrote:
> >> On 20 September 2017 at 10:34, Udit Kumar <udit.kumar@nxp.com> wrote:
> >>>
> >>> When we want to have UEFI and OS accessing same media ,
> >>> Possibilities I see
> >>>
> >>> 1- Patch OS For status check of media (diversion from generic OS), Good
> case will be modify low level driver.
> >>> But we may end up some surprises on synchronization.
> >>>
> >>> 2- no runtime service for OS . I guess this will not be possible
> >>>
> >>> 3- Way the  Vladimir implemented for eMMC, This has risk of losing data in
> case of AC power off.
> >>>
> >>> 4- update hardware with dual view (Ard suggestion)
> >>>
> >>
> >> 5 - abstract direct block device access into a firmware service that
> >> is exposed via a DXE_RUNTIME_DRIVER.
> >>
> >> The UEFI spec allows you to expose entry points into a
> >> DXE_RUNTIME_DRIVER module via a UEFI configuration table, and the OS
> >> can use a driver that uses the abstracted device rather than the real
> >> device. Performance is going to be terrible, probably, and lots of
> >> things that are specific to SD/MMC will no longer work, but it is a
> >> possibility nonetheless.
> >
> > BTW this would go beyond the UEFI spec, and would effectively be a
> > PI/UEFI dependent feature.
> 
> The concept of a runtime EFI driver is in the UEFI spec, but the issue is there is no
> way to tell the OS to not bind it's driver for that device that is universal. If you
> boot an unmodified OS it is going to conflict with the EFI runtime service.
> 
> My gut feel is if the OS has a driver for the device it may be better to make the
> media format the architectural point. That way the OS can read/write it via a
> driver/app. There could be an UEFI Services Table entry that implies what
> scheme is being used. That way there is only every one owner of the hardware
> device. I assume #3 is like this?
> 
> Thanks,
> 
> Andrew Fish
> 
> > _______________________________________________
> > edk2-devel mailing list
> > edk2-devel@lists.01.org
> > https://lists.01.org/mailman/listinfo/edk2-devel

Re: Storing Non volatile variables on SD/NAND

[Udit Kumar]

Thanks Ard. 

> The UEFI spec allows you to expose entry points into a DXE_RUNTIME_DRIVER
> module via a UEFI configuration table, and the OS can use a driver that uses the
> abstracted device rather than the real device. Performance is going to be

Could you point me to some sample driver using this scheme, 
Mainly around OS implementation 

Regards 
Udit

> -----Original Message-----
> From: Ard Biesheuvel [mailto:ard.biesheuvel@linaro.org]
> Sent: Wednesday, September 20, 2017 11:09 PM
> To: Udit Kumar <udit.kumar@nxp.com>
> Cc: Pankaj Bansal <pankaj.bansal@nxp.com>; Andrew Fish <afish@apple.com>;
> Olivier.Martin@arm.com; Vladimir Olovyannikov
> <vladimir.olovyannikov@broadcom.com>; edk2-devel@lists.01.org
> Subject: Re: [edk2] Storing Non volatile variables on SD/NAND
> 
> On 20 September 2017 at 10:34, Udit Kumar <udit.kumar@nxp.com> wrote:
> >
> > When we want to have UEFI and OS accessing same media , Possibilities
> > I see
> >
> > 1- Patch OS For status check of media (diversion from generic OS), Good case
> will be modify low level driver.
> > But we may end up some surprises on synchronization.
> >
> > 2- no runtime service for OS . I guess this will not be possible
> >
> > 3- Way the  Vladimir implemented for eMMC, This has risk of losing data in
> case of AC power off.
> >
> > 4- update hardware with dual view (Ard suggestion)
> >
> 
> 5 - abstract direct block device access into a firmware service that is exposed via
> a DXE_RUNTIME_DRIVER.
> 
> The UEFI spec allows you to expose entry points into a DXE_RUNTIME_DRIVER
> module via a UEFI configuration table, and the OS can use a driver that uses the
> abstracted device rather than the real device. Performance is going to be
> terrible, probably, and lots of things that are specific to SD/MMC will no longer
> work, but it is a possibility nonetheless.

Re: Storing Non volatile variables on SD/NAND

[Ard Biesheuvel]

On 27 October 2017 at 10:37, Udit Kumar <udit.kumar@nxp.com> wrote:
> Thanks Ard.
>
>> The UEFI spec allows you to expose entry points into a DXE_RUNTIME_DRIVER
>> module via a UEFI configuration table, and the OS can use a driver that uses the
>> abstracted device rather than the real device. Performance is going to be
>
> Could you point me to some sample driver using this scheme,
> Mainly around OS implementation
>

I don't know of any examples, unfortunately. This is uncharted territory afaik.

Re: Storing Non volatile variables on SD/NAND

[Jeremy Linton]

On 09/20/2017 12:39 PM, Ard Biesheuvel wrote:
> On 20 September 2017 at 10:34, Udit Kumar <udit.kumar@nxp.com> wrote:
>>
>> When we want to have UEFI and OS accessing same media ,
>> Possibilities I see
>>
>> 1- Patch OS For status check of media (diversion from generic OS), Good case will be modify low level driver.
>> But we may end up some surprises on synchronization.
>>
>> 2- no runtime service for OS . I guess this will not be possible
>>
>> 3- Way the  Vladimir implemented for eMMC, This has risk of losing data in case of AC power off.
>>
>> 4- update hardware with dual view (Ard suggestion)
>>
> 
> 5 - abstract direct block device access into a firmware service that
> is exposed via a DXE_RUNTIME_DRIVER.

(along those lines)

6 - Build an emulated disk controller as well as NV region in el3 (or 
el2) and export them to UEFI & the OS as real devices. Then trap/forward 
requests to the actual storage device, which is "hidden". This AFAIK was 
the basic idea behind the PS/2 emulation in x86/SMM. Again, probably not 
a high performance option.


> 
> The UEFI spec allows you to expose entry points into a
> DXE_RUNTIME_DRIVER module via a UEFI configuration table, and the OS
> can use a driver that uses the abstracted device rather than the real
> device. Performance is going to be terrible, probably, and lots of
> things that are specific to SD/MMC will no longer work, but it is a
> possibility nonetheless.
> _______________________________________________
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel
> 

Re: Storing Non volatile variables on SD/NAND

[Udit Kumar]

> (along those lines)
> 
> 6 - Build an emulated disk controller as well as NV region in el3 (or
> el2) and export them to UEFI & the OS as real devices. Then trap/forward
> requests to the actual storage device, which is "hidden". This AFAIK was the
> basic idea behind the PS/2 emulation in x86/SMM. Again, probably not a high
> performance option.

You mean, have a driver in el3 or el2 and UEFI or OS is doing smc call to get things done. 
On this line,  some sort of permission manager could reside in el3 or el2. 
Either UEFI or OS driver needs to make a call, if they are allowed to access this 
specific controller or other driver is accessing it. 
With this  performance issue could be ironed out . 

Regards
Udit

> -----Original Message-----
> From: Jeremy Linton [mailto:jeremy.linton@arm.com]
> Sent: Friday, October 27, 2017 11:16 PM
> To: Ard Biesheuvel <ard.biesheuvel@linaro.org>; Udit Kumar
> <udit.kumar@nxp.com>
> Cc: edk2-devel@lists.01.org; Andrew Fish <afish@apple.com>;
> Olivier.Martin@arm.com; Vladimir Olovyannikov
> <vladimir.olovyannikov@broadcom.com>
> Subject: Re: [edk2] Storing Non volatile variables on SD/NAND
> 
> On 09/20/2017 12:39 PM, Ard Biesheuvel wrote:
> > On 20 September 2017 at 10:34, Udit Kumar <udit.kumar@nxp.com> wrote:
> >>
> >> When we want to have UEFI and OS accessing same media , Possibilities
> >> I see
> >>
> >> 1- Patch OS For status check of media (diversion from generic OS), Good case
> will be modify low level driver.
> >> But we may end up some surprises on synchronization.
> >>
> >> 2- no runtime service for OS . I guess this will not be possible
> >>
> >> 3- Way the  Vladimir implemented for eMMC, This has risk of losing data in
> case of AC power off.
> >>
> >> 4- update hardware with dual view (Ard suggestion)
> >>
> >
> > 5 - abstract direct block device access into a firmware service that
> > is exposed via a DXE_RUNTIME_DRIVER.
> 
> (along those lines)
> 
> 6 - Build an emulated disk controller as well as NV region in el3 (or
> el2) and export them to UEFI & the OS as real devices. Then trap/forward
> requests to the actual storage device, which is "hidden". This AFAIK was the
> basic idea behind the PS/2 emulation in x86/SMM. Again, probably not a high
> performance option.
> 
> 
> >
> > The UEFI spec allows you to expose entry points into a
> > DXE_RUNTIME_DRIVER module via a UEFI configuration table, and the OS
> > can use a driver that uses the abstracted device rather than the real
> > device. Performance is going to be terrible, probably, and lots of
> > things that are specific to SD/MMC will no longer work, but it is a
> > possibility nonetheless.
> > _______________________________________________
> > edk2-devel mailing list
> > edk2-devel@lists.01.org
> >
> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.01
> .org%2Fmailman%2Flistinfo%2Fedk2-
> devel&data=02%7C01%7Cudit.kumar%40nxp.com%7Cfe11f07ea67a4efa7d1b08
> d51d629deb%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C63644723
> 1755528994&sdata=FYnH3ItGhXmqxNr%2BnaJBFMcKKduf%2FcS06JEA6dT6ZQA
> %3D&reserved=0
> >

Re: Storing Non volatile variables on SD/NAND

[Jeremy Linton]

Hi,

On 10/27/2017 10:09 PM, Udit Kumar wrote:
>> (along those lines)
>>
>> 6 - Build an emulated disk controller as well as NV region in el3 (or
>> el2) and export them to UEFI & the OS as real devices. Then trap/forward
>> requests to the actual storage device, which is "hidden". This AFAIK was the
>> basic idea behind the PS/2 emulation in x86/SMM. Again, probably not a high
>> performance option.
> 
> You mean, have a driver in el3 or el2 and UEFI or OS is doing smc call to get things done.
> On this line,  some sort of permission manager could reside in el3 or el2.
> Either UEFI or OS driver needs to make a call, if they are allowed to access this
> specific controller or other driver is accessing it.
> With this  performance issue could be ironed out .

That isn't really what I meant, what I was thinking about was creating 
an emulated (AHCI for example) controller where accesses to the 
"register" space would trap to synchronous data/external aborts. That 
way the firmware and OS could use existing drivers without knowledge 
that the device was in any way special. If you create a driver and do 
SMC/whatever calls, then your basically doing #5..




> 
> Regards
> Udit
> 
>> -----Original Message-----
>> From: Jeremy Linton [mailto:jeremy.linton@arm.com]
>> Sent: Friday, October 27, 2017 11:16 PM
>> To: Ard Biesheuvel <ard.biesheuvel@linaro.org>; Udit Kumar
>> <udit.kumar@nxp.com>
>> Cc: edk2-devel@lists.01.org; Andrew Fish <afish@apple.com>;
>> Olivier.Martin@arm.com; Vladimir Olovyannikov
>> <vladimir.olovyannikov@broadcom.com>
>> Subject: Re: [edk2] Storing Non volatile variables on SD/NAND
>>
>> On 09/20/2017 12:39 PM, Ard Biesheuvel wrote:
>>> On 20 September 2017 at 10:34, Udit Kumar <udit.kumar@nxp.com> wrote:
>>>>
>>>> When we want to have UEFI and OS accessing same media , Possibilities
>>>> I see
>>>>
>>>> 1- Patch OS For status check of media (diversion from generic OS), Good case
>> will be modify low level driver.
>>>> But we may end up some surprises on synchronization.
>>>>
>>>> 2- no runtime service for OS . I guess this will not be possible
>>>>
>>>> 3- Way the  Vladimir implemented for eMMC, This has risk of losing data in
>> case of AC power off.
>>>>
>>>> 4- update hardware with dual view (Ard suggestion)
>>>>
>>>
>>> 5 - abstract direct block device access into a firmware service that
>>> is exposed via a DXE_RUNTIME_DRIVER.
>>
>> (along those lines)
>>
>> 6 - Build an emulated disk controller as well as NV region in el3 (or
>> el2) and export them to UEFI & the OS as real devices. Then trap/forward
>> requests to the actual storage device, which is "hidden". This AFAIK was the
>> basic idea behind the PS/2 emulation in x86/SMM. Again, probably not a high
>> performance option.
>>
>>
>>>
>>> The UEFI spec allows you to expose entry points into a
>>> DXE_RUNTIME_DRIVER module via a UEFI configuration table, and the OS
>>> can use a driver that uses the abstracted device rather than the real
>>> device. Performance is going to be terrible, probably, and lots of
>>> things that are specific to SD/MMC will no longer work, but it is a
>>> possibility nonetheless.
>>> _______________________________________________
>>> edk2-devel mailing list
>>> edk2-devel@lists.01.org
>>>
>> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.01
>> .org%2Fmailman%2Flistinfo%2Fedk2-
>> devel&data=02%7C01%7Cudit.kumar%40nxp.com%7Cfe11f07ea67a4efa7d1b08
>> d51d629deb%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C63644723
>> 1755528994&sdata=FYnH3ItGhXmqxNr%2BnaJBFMcKKduf%2FcS06JEA6dT6ZQA
>> %3D&reserved=0
>>>
> 

Re: Storing Non volatile variables on SD/NAND

[Udit Kumar]

Thanks Jeremy 

> Hi,
> 
> On 10/27/2017 10:09 PM, Udit Kumar wrote:
> >> (along those lines)
> >>
> >> 6 - Build an emulated disk controller as well as NV region in el3 (or
> >> el2) and export them to UEFI & the OS as real devices. Then
> >> trap/forward requests to the actual storage device, which is
> >> "hidden". This AFAIK was the basic idea behind the PS/2 emulation in
> >> x86/SMM. Again, probably not a high performance option.
> >
> > You mean, have a driver in el3 or el2 and UEFI or OS is doing smc call to get
> things done.
> > On this line,  some sort of permission manager could reside in el3 or el2.
> > Either UEFI or OS driver needs to make a call, if they are allowed to
> > access this specific controller or other driver is accessing it.
> > With this  performance issue could be ironed out .
> 
> That isn't really what I meant, what I was thinking about was creating an
> emulated (AHCI for example) controller where accesses to the "register" space
> would trap to synchronous data/external aborts. That way the firmware and OS
> could use existing drivers without knowledge that the device was in any way
> special. If you create a driver and do SMC/whatever calls, then your basically
> doing #5..

This may work if both OS and UEFI are residing is same EL level.  
Do you mean, to get access to controller use this trap or entire AHCI space in
unmapped area, which will leads to synchronous aborts.

> 
> >
> > Regards
> > Udit
> >
> >> -----Original Message-----
> >> From: Jeremy Linton [mailto:jeremy.linton@arm.com]
> >> Sent: Friday, October 27, 2017 11:16 PM
> >> To: Ard Biesheuvel <ard.biesheuvel@linaro.org>; Udit Kumar
> >> <udit.kumar@nxp.com>
> >> Cc: edk2-devel@lists.01.org; Andrew Fish <afish@apple.com>;
> >> Olivier.Martin@arm.com; Vladimir Olovyannikov
> >> <vladimir.olovyannikov@broadcom.com>
> >> Subject: Re: [edk2] Storing Non volatile variables on SD/NAND
> >>
> >> On 09/20/2017 12:39 PM, Ard Biesheuvel wrote:
> >>> On 20 September 2017 at 10:34, Udit Kumar <udit.kumar@nxp.com> wrote:
> >>>>
> >>>> When we want to have UEFI and OS accessing same media ,
> >>>> Possibilities I see
> >>>>
> >>>> 1- Patch OS For status check of media (diversion from generic OS),
> >>>> Good case
> >> will be modify low level driver.
> >>>> But we may end up some surprises on synchronization.
> >>>>
> >>>> 2- no runtime service for OS . I guess this will not be possible
> >>>>
> >>>> 3- Way the  Vladimir implemented for eMMC, This has risk of losing
> >>>> data in
> >> case of AC power off.
> >>>>
> >>>> 4- update hardware with dual view (Ard suggestion)
> >>>>
> >>>
> >>> 5 - abstract direct block device access into a firmware service that
> >>> is exposed via a DXE_RUNTIME_DRIVER.
> >>
> >> (along those lines)
> >>
> >> 6 - Build an emulated disk controller as well as NV region in el3 (or
> >> el2) and export them to UEFI & the OS as real devices. Then
> >> trap/forward requests to the actual storage device, which is
> >> "hidden". This AFAIK was the basic idea behind the PS/2 emulation in
> >> x86/SMM. Again, probably not a high performance option.
> >>
> >>
> >>>
> >>> The UEFI spec allows you to expose entry points into a
> >>> DXE_RUNTIME_DRIVER module via a UEFI configuration table, and the OS
> >>> can use a driver that uses the abstracted device rather than the
> >>> real device. Performance is going to be terrible, probably, and lots
> >>> of things that are specific to SD/MMC will no longer work, but it is
> >>> a possibility nonetheless.
> >>> _______________________________________________
> >>> edk2-devel mailing list
> >>> edk2-devel@lists.01.org
> >>>
> >> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fli
> >> sts.01
> >> .org%2Fmailman%2Flistinfo%2Fedk2-
> >>
> devel&data=02%7C01%7Cudit.kumar%40nxp.com%7Cfe11f07ea67a4efa7d1b08
> >>
> d51d629deb%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C63644723
> >>
> 1755528994&sdata=FYnH3ItGhXmqxNr%2BnaJBFMcKKduf%2FcS06JEA6dT6ZQA
> >> %3D&reserved=0
> >>>
> >

Re: Storing Non volatile variables on SD/NAND

[Andrew Fish]

Pankaj,

We understand the use cases. We just don't have a good solution for the spec that is OS, Firmware, and hardware agnostic. 

Thanks,

Andrew Fish

> On Sep 20, 2017, at 7:51 AM, Pankaj Bansal <pankaj.bansal@nxp.com> wrote:
> 
> This use case also arises for single-board systems like raspberry-pi, which do not have an onboard flash.
> The boot firmware/bootloader as well as operating system are loaded from SD card.
> https://www.raspberrypi.org/documentation/configuration/config-txt/
> 
> Thanks & Regards,
> Pankaj Bansal
> 
> -----Original Message-----
> From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of Andrew Fish
> Sent: Wednesday, September 20, 2017 10:48 AM
> To: Udit Kumar <udit.kumar@nxp.com>
> Cc: edk2-devel@lists.01.org; Vladimir Olovyannikov <vladimir.olovyannikov@broadcom.com>; Olivier.Martin@arm.com; Ard Biesheuvel <ard.biesheuvel@linaro.org>
> Subject: Re: [edk2] Storing Non volatile variables on SD/NAND
> 
> 
>> On Sep 19, 2017, at 10:09 PM, Udit Kumar <udit.kumar@nxp.com> wrote:
>> 
>>>> On Sep 19, 2017, at 9:27 PM, Udit Kumar <udit.kumar@nxp.com> wrote:
>>>> 
>>>> 
>>>>> On 18 September 2017 at 22:28, Udit Kumar <udit.kumar@nxp.com> wrote:
>>>>>> Thanks Vladimir,
>>>>>> With your design, you did delayed write to eMMC due to sharing 
>>>>>> with OS.  But it works for you:) Say if eMMC controllers offers 
>>>>>> you a status bit, if eMMC storage is being used for not. Then this 
>>>>>> could be possible to
>>>>> update at run time, both OS/UEFI needs to check and wait if 
>>>>> controller is being used.
>>>>> 
>>>>> That is the problem right there. The nice thing about a firmware 
>>>>> spec is that you don't have to care about how it was implemented if 
>>>>> you adhere to
>>> the API rules.
>>>> 
>>>> Yup, we are fine as long as long UEFI firmware is stored on dedicated media.
>>>> 
>>>>> Imposing additional restrictions (such as requiring the OS to be 
>>>>> careful about not using the eMMC when it may be in use by the
>>>>> firmware) defeats the purpose of using UEFI, since you won't be 
>>>>> able to use a
>>> generic OS anyway.
>>>>> 
>>>> 
>>>> Hmm,  so far, I haven't come across where UEFI specs says, we need a 
>>>> separate Storage for firmware. (May be I missed some part of specs) 
>>>> Irrespective of storage media, we have this problem if OS and UEFI 
>>>> shares same storage.
>>>> 
>>> 
>>> Udit,
>>> 
>>> Can you point out the spec that states you can't boot Linux and 
>>> Windows at the same time on a PC? :)
>>> 
>>> When you write a spec it is not practical do document what is not 
>>> possible, you can only document the API the rest is implied by the 
>>> implementation. So for example the UEFI spec does not document why 
>>> the firmware and OS can't share a hardware device, just like you 
>>> can't have 2 operating systems running on bare metal at the same 
>>> time. It is a little like Occam's Razor the reason that the firmware 
>>> and the OS can not share a hardware device is the mechanics of how to 
>>> share a hardware device is not defined in the spec, thus it is not part of the API and not possible.
>> 
>> Right,  This is left on implementation how to put firmware and OS.
>> Ideally, keeping both storage separate  is best case, no need to sync between two.
>> 
>> My reply to Ard, was to highlight that in any case (NOR or eMMC /NAND) 
>> if we are keeping OS and firmware on same storage, we will have same 
>> issue not limited to  eMMC.
>> 
>> For some requirement, if we need to keep firmware and OS on same 
>> media, Then implementation should make sure there is exclusive access 
>> (be it NOR controller, SD controller etc).
>> 
> 
> Udit,
> 
> Sorry I'm a little swamped on my email right now and might be a little behind on the thread....
> 
> Yea the only way to realistically Implement an EFI runtime service in UEFI is to have UEFI own the hardware device. There is no architecture for sharing the device, and the type of device is not really relevant. 
> 
> Thanks,
> 
> Andrew Fish
> 
>> Thanks
>> Udit
>> 
>>> Thanks,
>>> 
>>> Andrew Fish
>>> 
>>>>>> For sure,  some synchronization issues need to be ironed out (or 
>>>>>> maybe I am
>>>>> just dreaming here).
>>>>>> 
>>>>>> On part 2) where you forked VariableRuntime driver , could we 
>>>>>> think of updating VariableRuntime driver, to support non-XIP or 
>>>>>> memory mapped
>>>>> devices.
>>>>>> 
>>>>> 
>>>>> I think being able to support non-memorymapped FV volumes for the 
>>>>> variable store would be a big improvement. This does require 
>>>>> changes to both the FaultTolerantWrite drivers and the 
>>>>> VariableRuntime drivers, which both appear in PEI, DXE and SMM 
>>>>> flavors, and require thorough review due to the security impact 
>>>>> bugs have in this layer, so this is a
>>> rather large chunk of work to take on.
>>>> 
>>>> Thanks,  your list is longer than what I was thinking :-) I think, 
>>>> for embedded world with UEFI, later or sooner, this will be required.
>>>> 
>>>> Thanks
>>>> Udit
>>>> _______________________________________________
>>>> edk2-devel mailing list
>>>> edk2-devel@lists.01.org
>>>> https://lists.01.org/mailman/listinfo/edk2-devel
> 
> _______________________________________________
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel
> _______________________________________________
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel

Re: Storing Non volatile variables on SD/NAND

[Jeremy Linton]

On 09/18/2017 10:43 AM, Ard Biesheuvel wrote:
> On 18 September 2017 at 06:52, Udit Kumar <udit.kumar@nxp.com> wrote:
>> Hi EDK-2 Experts,
>> I am looking to store NV variables on SD/NAND device.
>>
>> While browsing, I came across some old post at link,
>> http://feishare.com/efimail/messages/20130319-1700-Re__edk2__Regarding_storing_Boot_Device_Config_in_persistent_memory-Olivier_Martin.html
>>
>> Looks like, this is possible easily.
> 
> That's a bold statement dude :-)
> 
>>>> What you need to support Non-Volatile UEFI variables is a Non-Volatile Memory. And also a driver that implements the EFI Firmware Volume Block protocol for this NVM device.
>>
>> But MdeModulePkg does Copymem from NV variable start memory to some allocated buffers.  With SD/NAND Copymem is not possible, Is this something changes since 2013 or there are some other way to use SD/NAND
>>
> 
> No, SD/MMC cannot currently be used as the backing store for the EFI
> variable store. The problem is that the variable protocols are
> architectural protocols in PI that need to be present before any
> driver model drivers are dispatched, and so putting the variable store
> on block devices is not something that the PI software architecture
> currently supports (unless you reimplement the whole driver stack as
> DXE drivers).
> 
> On top of that, it is almost impossible to share a block device that
> sits behind a controller between the firmware and the OS at runtime
> (i.e., for SetVariable() calls made by efibootmgr under Linux),
> because only a single agent can take ownership of the controller at
> any given time. (You /could/ dedicate the SD/MMC to the firmware
> entirely, and boot from SATA or USB, but this is out of the question
> on most platforms that need to use SD/MMC for that variable backing
> store, i.e., mobile platforms)
> 
> The best thing would be for you to convince the hardware architects in
> your company to design and implement dual-ported SD/MMC controllers
> that allow a single SD/MMC to have two logical views that are
> independent (although I'm unsure if that is even possible in the
> context of the SD/MMC specifications)

Which still has the problems of selecting "use whole disk" during an OS 
install bricking the machine. Or similarly if the emmc layout isn't just 
right having gparted automatically "fix" the partition table (as it does 
with many of the hikey images) again bricking the machine.

Having the firmware/variable store and OS root/boot on the same device 
is fundamentally flawed. I've went down the path of simply disabling the 
hikey/emmc for use beyond the firmware/variable storage on the hikey. Of 
course I ran smack into the problem of making the block device DXE's 
run-time safe which I've about concluded is far harder than simply 
writing a monolithic variablestore->emmc driver.

The ideal situation for the Hikey, is probably to solder a SPI flash to 
the SPI controller and put the firmware/variable store on that, and 
leave the eMMC entirely for linux's use.

Re: Storing Non volatile variables on SD/NAND

[Ard Biesheuvel]

On 18 September 2017 at 13:47, Jeremy Linton <jeremy.linton@arm.com> wrote:
> On 09/18/2017 10:43 AM, Ard Biesheuvel wrote:
>>
>> On 18 September 2017 at 06:52, Udit Kumar <udit.kumar@nxp.com> wrote:
>>>
>>> Hi EDK-2 Experts,
>>> I am looking to store NV variables on SD/NAND device.
>>>
>>> While browsing, I came across some old post at link,
>>>
>>> http://feishare.com/efimail/messages/20130319-1700-Re__edk2__Regarding_storing_Boot_Device_Config_in_persistent_memory-Olivier_Martin.html
>>>
>>> Looks like, this is possible easily.
>>
>>
>> That's a bold statement dude :-)
>>
>>>>> What you need to support Non-Volatile UEFI variables is a Non-Volatile
>>>>> Memory. And also a driver that implements the EFI Firmware Volume Block
>>>>> protocol for this NVM device.
>>>
>>>
>>> But MdeModulePkg does Copymem from NV variable start memory to some
>>> allocated buffers.  With SD/NAND Copymem is not possible, Is this something
>>> changes since 2013 or there are some other way to use SD/NAND
>>>
>>
>> No, SD/MMC cannot currently be used as the backing store for the EFI
>> variable store. The problem is that the variable protocols are
>> architectural protocols in PI that need to be present before any
>> driver model drivers are dispatched, and so putting the variable store
>> on block devices is not something that the PI software architecture
>> currently supports (unless you reimplement the whole driver stack as
>> DXE drivers).
>>
>> On top of that, it is almost impossible to share a block device that
>> sits behind a controller between the firmware and the OS at runtime
>> (i.e., for SetVariable() calls made by efibootmgr under Linux),
>> because only a single agent can take ownership of the controller at
>> any given time. (You /could/ dedicate the SD/MMC to the firmware
>> entirely, and boot from SATA or USB, but this is out of the question
>> on most platforms that need to use SD/MMC for that variable backing
>> store, i.e., mobile platforms)
>>
>> The best thing would be for you to convince the hardware architects in
>> your company to design and implement dual-ported SD/MMC controllers
>> that allow a single SD/MMC to have two logical views that are
>> independent (although I'm unsure if that is even possible in the
>> context of the SD/MMC specifications)
>
>
> Which still has the problems of selecting "use whole disk" during an OS
> install bricking the machine. Or similarly if the emmc layout isn't just
> right having gparted automatically "fix" the partition table (as it does
> with many of the hikey images) again bricking the machine.
>
> Having the firmware/variable store and OS root/boot on the same device is
> fundamentally flawed. I've went down the path of simply disabling the
> hikey/emmc for use beyond the firmware/variable storage on the hikey. Of
> course I ran smack into the problem of making the block device DXE's
> run-time safe which I've about concluded is far harder than simply writing a
> monolithic variablestore->emmc driver.
>
> The ideal situation for the Hikey, is probably to solder a SPI flash to the
> SPI controller and put the firmware/variable store on that, and leave the
> eMMC entirely for linux's use.
>

Oh, I completely agree that HiKey is a trainwreck in this regard. I
asked for a 96boards mezzanine board with a SPI NOR more than 2 years
ago so we could prototype this stuff properly, but it never
materialized.

But eMMC *can* potentially be used in a meaningful way, if we use a
MMC boot partition for the UEFI image and the RPMB partition for the
variable store (which would arguably be the only way to get a truly
tamper proof *and* rollback protected varstore, which is what you
minimally need to implement UEFI secure boot)

Re: Storing Non volatile variables on SD/NAND

[Udit Kumar]

Thanks Ard / Jeremy. 

Say SD/NAND is just used for UEFI firmware then this is something doable. 

> > Having the firmware/variable store and OS root/boot on the same device
> > is fundamentally flawed.

I agree in case of SD/NAND, partitioning and synchronization with OS will be tough task. 
Even in case of NOR flash, if this is exposed as mtd device as well, then accessing from 
OS and UEFI poses same challenge .

Thanks
Udit

> -----Original Message-----
> From: Ard Biesheuvel [mailto:ard.biesheuvel@linaro.org]
> Sent: Tuesday, September 19, 2017 2:23 AM
> To: Jeremy Linton <jeremy.linton@arm.com>
> Cc: Udit Kumar <udit.kumar@nxp.com>; grant.likely@linaro.org.
> <grant.likely@linaro.org>; edk2-devel@lists.01.org; Olivier.Martin@arm.com
> Subject: Re: [edk2] Storing Non volatile variables on SD/NAND
> 
> On 18 September 2017 at 13:47, Jeremy Linton <jeremy.linton@arm.com>
> wrote:
> > On 09/18/2017 10:43 AM, Ard Biesheuvel wrote:
> >>
> >> On 18 September 2017 at 06:52, Udit Kumar <udit.kumar@nxp.com> wrote:
> >>>
> >>> Hi EDK-2 Experts,
> >>> I am looking to store NV variables on SD/NAND device.
> >>>
> >>> While browsing, I came across some old post at link,
> >>>
> >>> http://feishare.com/efimail/messages/20130319-1700-Re__edk2__Regardi
> >>> ng_storing_Boot_Device_Config_in_persistent_memory-Olivier_Martin.ht
> >>> ml
> >>>
> >>> Looks like, this is possible easily.
> >>
> >>
> >> That's a bold statement dude :-)
> >>
> >>>>> What you need to support Non-Volatile UEFI variables is a
> >>>>> Non-Volatile Memory. And also a driver that implements the EFI
> >>>>> Firmware Volume Block protocol for this NVM device.
> >>>
> >>>
> >>> But MdeModulePkg does Copymem from NV variable start memory to some
> >>> allocated buffers.  With SD/NAND Copymem is not possible, Is this
> >>> something changes since 2013 or there are some other way to use
> >>> SD/NAND
> >>>
> >>
> >> No, SD/MMC cannot currently be used as the backing store for the EFI
> >> variable store. The problem is that the variable protocols are
> >> architectural protocols in PI that need to be present before any
> >> driver model drivers are dispatched, and so putting the variable
> >> store on block devices is not something that the PI software
> >> architecture currently supports (unless you reimplement the whole
> >> driver stack as DXE drivers).
> >>
> >> On top of that, it is almost impossible to share a block device that
> >> sits behind a controller between the firmware and the OS at runtime
> >> (i.e., for SetVariable() calls made by efibootmgr under Linux),
> >> because only a single agent can take ownership of the controller at
> >> any given time. (You /could/ dedicate the SD/MMC to the firmware
> >> entirely, and boot from SATA or USB, but this is out of the question
> >> on most platforms that need to use SD/MMC for that variable backing
> >> store, i.e., mobile platforms)
> >>
> >> The best thing would be for you to convince the hardware architects
> >> in your company to design and implement dual-ported SD/MMC
> >> controllers that allow a single SD/MMC to have two logical views that
> >> are independent (although I'm unsure if that is even possible in the
> >> context of the SD/MMC specifications)
> >
> >
> > Which still has the problems of selecting "use whole disk" during an
> > OS install bricking the machine. Or similarly if the emmc layout isn't
> > just right having gparted automatically "fix" the partition table (as
> > it does with many of the hikey images) again bricking the machine.
> >
> > Having the firmware/variable store and OS root/boot on the same device
> > is fundamentally flawed. I've went down the path of simply disabling
> > the hikey/emmc for use beyond the firmware/variable storage on the
> > hikey. Of course I ran smack into the problem of making the block
> > device DXE's run-time safe which I've about concluded is far harder
> > than simply writing a monolithic variablestore->emmc driver.
> >
> > The ideal situation for the Hikey, is probably to solder a SPI flash
> > to the SPI controller and put the firmware/variable store on that, and
> > leave the eMMC entirely for linux's use.
> >
> 
> Oh, I completely agree that HiKey is a trainwreck in this regard. I asked for a
> 96boards mezzanine board with a SPI NOR more than 2 years ago so we could
> prototype this stuff properly, but it never materialized.
> 
> But eMMC *can* potentially be used in a meaningful way, if we use a MMC boot
> partition for the UEFI image and the RPMB partition for the variable store (which
> would arguably be the only way to get a truly tamper proof *and* rollback
> protected varstore, which is what you minimally need to implement UEFI secure
> boot)