Re: how to load drivers from additional FV's?

[Michael Zimmermann <sigmaepsilon92@gmail.com>]

Laszlo, I don't know if I was clear enough but I've been able to load
drivers from a nested FV without any manual loading or decompressing
and without any changes to my code.

All I needed to do was putting this into FVMAIN:
  FILE FV_IMAGE = $(GUID) {
    SECTION FV_IMAGE = FVMAINPLATFORM
  }

Thanks
Michael

On Thu, Mar 16, 2017 at 1:27 PM, Laszlo Ersek <lersek@redhat.com> wrote:
> On 03/16/17 13:20, Michael Zimmermann wrote:
>> So if I understand this correctly there's no bug to fix because
>> nested, uncompressed FV's are already supported
>
> I'm not sure how to answer this; I guess the answer depends on what
> meaning you attribute to "supported". In OVMF it works, but it depends
> on two things: OVMF's SEC driver having access to writeable RAM, and
> OVMF manually messing with the firmware volumes (locating them,
> decompressing the outer file, moving stuff around, and so on).
>
> On a physical platform, I guess you can do the same, if you are willing
> to mess with the FVs manually, and you delay that code until at least
> the PEI phase sees the permanent RAM installed.
>
> Thanks
> Laszlo
>
>> and using compressed
>> ones is bad practice anyway?
>>
>> Thanks
>> Michael
>>
>> On Thu, Mar 16, 2017 at 12:20 PM, Laszlo Ersek <lersek@redhat.com> wrote:
>>> On 03/15/17 16:54, Andrew Fish wrote:
>>>>
>>>>> On Mar 15, 2017, at 8:38 AM, Laszlo Ersek <lersek@redhat.com
>>>>> <mailto:lersek@redhat.com>> wrote:
>>>>>
>>>>> On 03/15/17 16:13, Andrew Fish wrote:
>>>>>>
>>>>>>> On Mar 15, 2017, at 8:07 AM, Laszlo Ersek <lersek@redhat.com
>>>>>>> <mailto:lersek@redhat.com>> wrote:
>>>>>>>
>>>>>>> On 03/15/17 13:23, Michael Zimmermann wrote:
>>>>>>>> I'm trying to add another FV section FVMAIN_COMPACT so I can keep
>>>>>>>> Platform specific drivers in a separate, included fdf.
>>>>>>>>
>>>>>>>> I did this:
>>>>>>>> FILE FV_IMAGE = 9E21FD93-9C72-4c15-8C4B-E77F1DB2D792 {
>>>>>>>>   SECTION GUIDED EE4E5898-3914-4259-9D6E-DC7BD79403CF
>>>>>>>> PROCESSING_REQUIRED = TRUE {
>>>>>>>>     SECTION FV_IMAGE = FVMAIN
>>>>>>>>     SECTION FV_IMAGE = FVMAINPLATFORM
>>>>>>>>   }
>>>>>>>> }
>>>>>>>>
>>>>>>>> The image builds file and using uefitool I can verify that the new FV
>>>>>>>> is inside the compressed section.
>>>>>>>> But none of the drivers gets discovered/loaded and I get 'Protocol not
>>>>>>>> present!!' errors.
>>>>>>>
>>>>>>> The FVs need to be exposed to the DXE core via FV HOBs. See
>>>>>>> - 9.8.5 "Firmware Volume HOBs" in Volume 2 of the Platform Init 1.5
>>>>>>> spec,
>>>>>>> - and more importantly, 5.7 "Firmware Volume HOB" in Volume 3 of the
>>>>>>> same.
>>>>>>>
>>>>>>> You can use the BuildFvHob() function for this.
>>>>>>>
>>>>>>> If the firmware volume contains PEIMs (... as well), then it has to be
>>>>>>> exposed to the PEI core too, I think. I think the
>>>>>>> PeiServicesInstallFvInfoPpi() function can be used for that. (See 3.3
>>>>>>> "PEI" in Volume 3 of the PI spec.)
>>>>>>>
>>>>>>> ... I used the PeiFvInitialization() function in
>>>>>>> OvmfPkg/PlatformPei/Fv.c as a "cheat sheet" for the above.
>>>>>>>
>>>>>>
>>>>>> Laszlo,
>>>>>>
>>>>>> I think this case is an FV that is compressed and nested in another
>>>>>> FV that is discovered. I think the issues is multiple FV Sections in
>>>>>> an FV file are not currently supported.
>>>>>
>>>>> In OVMF we do the same:
>>>>>
>>>>> FILE FV_IMAGE = 9E21FD93-9C72-4c15-8C4B-E77F1DB2D792 {
>>>>>   SECTION GUIDED EE4E5898-3914-4259-9D6E-DC7BD79403CF
>>>>> PROCESSING_REQUIRED = TRUE {
>>>>>     #
>>>>>     # These firmware volumes will have files placed in them uncompressed,
>>>>>     # and then both firmware volumes will be compressed in a single
>>>>>     # compression operation in order to achieve better overall
>>>>> compression.
>>>>>     #
>>>>>     SECTION FV_IMAGE = PEIFV
>>>>>     SECTION FV_IMAGE = DXEFV
>>>>>   }
>>>>> }
>>>>>
>>>>> "OvmfPkg/Sec/SecMain.c" decompresses the compressed FFS file into
>>>>> memory -- which is special to OVMF, i.e. the fact that there is
>>>>> writeable memory in SEC --, then jumps to the PEI entry point.
>>>>> PlatformPei then exposes DXEFV to the PEI core (for loading the DXE
>>>>> core AIUI) and to the DXE core as well (for loading DXE modules).
>>>>>
>>>>> The multiple FV sections in the FFS file are not discovered
>>>>> automatically. First, the OVMF build process saves the final (=
>>>>> decompressed) in-memory locations of the nested FVs into PCDs. Second,
>>>>> in DecompressMemFvs(), the SEC module decompresses the outer,
>>>>> compressed, FFS file, then locates the embedded FVs manually with
>>>>> FindFfsSectionInstance(), then moves the decompressed FVs individually
>>>>> to their pre-recorded locations.
>>>>>
>>>>> In Michael's case, I think the same should be possible -- since FVMAIN
>>>>> is running for him, the outer, compressed file must already be
>>>>> decompressed (in full) to some temporary buffer in memory. If he can
>>>>> find, manually, the second section (with the embedded FVMAINPLATFORM
>>>>> firmware volume in it) in that area, then he can also expose it to the
>>>>> DXE core, I believe.
>>>>>
>>>>
>>>> Laszlo,
>>>>
>>>> The DXE Core will dispatch FVs with Depex "automagically" but only one
>>>> FV_IMAGE Section per FV_IMAGE file.
>>>>
>>>> So one workaround would be to add a 2nd FV_IMAGE file that contains the
>>>> 2nd FV in the 1st FV_IMAGE Section.
>>>>
>>>> Also I'm not sure if the outer FV is compressed, but compressing
>>>> compressed data usually does not end well as the algorithms leverage
>>>> entropy.
>>>
>>> I apologize if my previous emails were confusing; I didn't mean to imply
>>> nested compression. OVMF doesn't do that.
>>>
>>>> It is better to just compress the outer FV and have everything
>>>> else uncompressed and let it all get compressed together.
>>>
>>> Yes, that's what OVMF does.
>>>
>>> Did you perhaps misread the comment above that I quoted from the FDF?
>>>
>>> "These firmware volumes will have files placed in them *uncompressed*,
>>> and then both firmware volumes will be compressed in a single
>>> compression operation in order to achieve better overall compression."
>>>
>>> (Emphasis mine.)
>>>
>>> Thanks!
>>> Laszlo
>>>
>>>> This also
>>>> helps as there will be common dictionary entries so compressing as much
>>>> data as possible in one shot usually works better. So for example if the
>>>> outer FV is compressed it may have a dictionary entry that represents
>>>> CopyMem, and it can represent that pattern with a small number of bits.
>>>> Doing compression multiple times causes duplication as each compressed
>>>> image has one copy of CopyMem (massive over simplification). Also if you
>>>> compress a nested FV then the CopyMem pattern does not exist in the
>>>> compressed data, and since the compressed data is more random it is hard
>>>> to find patterns to compress, so it hurts the compression of the outer FV.
>>>>
>>>> Thanks,
>>>>
>>>> Andrew Fish
>>>>
>>>>> Thanks
>>>>> Laszlo
>>>>> _______________________________________________
>>>>> edk2-devel mailing list
>>>>> edk2-devel@lists.01.org <mailto:edk2-devel@lists.01.org>
>>>>> https://lists.01.org/mailman/listinfo/edk2-devel
>>>>
>>>
>