Re: [EXTERNAL] [edk2-devel] APRIORI in RISC-V or Where did OVMF APRIORIs come from?

["Michael D Kinney" <michael.d.kinney@intel.com>]

[-- Attachment #1: Type: text/plain, Size: 7646 bytes --]

Reversing the order of files in each FV of an FDF is a very good test case.

Mike

From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Andrew Fish via groups.io
Sent: Thursday, May 7, 2020 9:55 AM
To: devel@edk2.groups.io; bret.barkelew@microsoft.com
Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>; Daniel Schaefer <daniel.schaefer@hpe.com>; Chang, Abner (HPS SW/FW Technologist) <abner.chang@hpe.com>; Atish Patra <atishp@atishpatra.org>; Heinrich Schuchardt <xypron.glpk@gmx.de>; Atish Patra <atish.patra@wdc.com>; Alexander Graf <agraf@csgraf.de>; Anup Patel <anup.patel@wdc.com>; leif@nuviainc.com; Justen, Jordan L <jordan.l.justen@intel.com>; Laszlo Ersek <lersek@redhat.com>
Subject: Re: [EXTERNAL] [edk2-devel] APRIORI in RISC-V or Where did OVMF APRIORIs come from?

Bret,

How does that test work? Does it make a custom FDF file?

Thanks,

Andrew Fish


On May 7, 2020, at 9:45 AM, Bret Barkelew via groups.io<http://groups.io> <bret.barkelew=microsoft.com@groups.io<mailto:bret.barkelew=microsoft.com@groups.io>> wrote:

I know I’ve also seen tests that randomize the driver dispatch order to try to catch these “implementation-specific” edge cases. Perhaps we could instrument something similar with a weekly OVMF CI test?

- Bret

From: Andrew Fish via groups.io<mailto:afish=apple.com@groups.io>
Sent: Thursday, May 7, 2020 9:43 AM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Ard Biesheuvel<mailto:ard.biesheuvel@arm.com>
Cc: Daniel Schaefer<mailto:daniel.schaefer@hpe.com>; Chang, Abner (HPS SW/FW Technologist)<mailto:abner.chang@hpe.com>; Atish Patra<mailto:atishp@atishpatra.org>; Heinrich Schuchardt<mailto:xypron.glpk@gmx.de>; Atish Patra<mailto:atish.patra@wdc.com>; Alexander Graf<mailto:agraf@csgraf.de>; Anup Patel<mailto:anup.patel@wdc.com>; leif@nuviainc.com<mailto:leif@nuviainc.com>; Jordan Justen<mailto:jordan.l.justen@intel.com>; Laszlo Ersek<mailto:lersek@redhat.com>
Subject: [EXTERNAL] Re: [edk2-devel] APRIORI in RISC-V or Where did OVMF APRIORIs come from?





On May 7, 2020, at 6:53 AM, Ard Biesheuvel <Ard.Biesheuvel@arm.com<mailto:Ard.Biesheuvel@arm.com>> wrote:

(+ Laszlo)

On 5/7/20 3:43 PM, Daniel Schaefer wrote:


On 5/7/20 3:24 PM, Ard Biesheuvel wrote:


On 5/7/20 3:18 PM, Daniel Schaefer via groups.io<https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgroups.io%2F&data=02%7C01%7Cbret.barkelew%40microsoft.com%7C55cf729cb7f2493df70508d7f2a5b761%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637244665879150410&sdata=3m1H%2BjDxdT258g641hJMpIXoSR3C7EmkGo3fpkZhVgw%3D&reserved=0> wrote:


Hi Ard and others,

TLDR; We have APRIORI definitions from other places in EDK2 but there's no explanation as to why they are there.

I'm taking this to the EDK2 list, since it doesn't concern U-Boot.
I kept some other people related to UEFI, maybe you're interested ;)

On 2/25/20 10:07 AM, Ard Biesheuvel wrote:
 > What I did notice is the use of APRIORI PEI and APRIORI DXE sections
 > in your platform descriptions. I recommend you try to avoid that, as
 > it is a maintenance burden going forward: instead, please use dummy
 > protocols and NULL library class resolutions if you need to make
 > generic components depend on platform specific protocols. Also, please
 > document this - the APRIORI section does not explain *why* you have to
 > circumvent the ordinary dependency tree based module dispatch.

I'm taking a look at this right now.
You're absolutely right - we should reduce or document APRIORIs.

However, Abner told me that he had only copied most of the FDF from other
places in EDK2.This is what we currently have:

APRIORI PEI {
   INF MdeModulePkg/Universal/ReportStatusCodeRouter/Pei/ReportStatusCodeRouterPei.inf
   INF MdeModulePkg/Universal/StatusCodeHandler/Pei/StatusCodeHandlerPei.inf
   INF  MdeModulePkg/Universal/PCD/Pei/Pcd.inf
}
APRIORI DXE {
   INF MdeModulePkg/Universal/DevicePathDxe/DevicePathDxe.inf
   INF  MdeModulePkg/Universal/PCD/Dxe/Pcd.inf
   INF Platform/SiFive/U5SeriesPkg/Universal/Dxe/RamFvbServicesRuntimeDxe/FvbServicesRuntimeDxe.inf
}

I can remove all of APRIORI PEI and it boots properly. Of the DXEs I can only
remove FvbServicesRuntimeDxe, otherwise some DXEs are dispatched in the wrong
order and boot fails.

This means some modules have an undeclared dependency on one of the remaining modules. Can you elaborate on how the boot fails in this case?
The error is
  ASSERT [FvbServicesRuntimeDxe] /edk2/MdePkg/Library/DxePcdLib/DxePcdLib.c(72): !EFI_ERROR (Status)
In this line, DxePcdLib tries to consume gPcdProtocolGuid. Therefor if I add
the following to FvbServicesRuntimeDxe.inf:
[Depex]
  gEfiPcdProtocolGuid
[Protocols]
  gPcdProtocolGuid                              ## SOMETIMES_CONSUMES
  gEfiPcdProtocolGuid                           ## CONSUMES
  gGetPcdInfoProtocolGuid                       ## SOMETIMES_CONSUMES
  gEfiGetPcdInfoProtocolGuid                    ## SOMETIMES_CONSUMES
I can boot without error.
Looking at MdePkg/Library/DxePcdLib/DxePcdLib.inf I see that the library has
exactly the same Depex and Protocols specified. Do DXEs have re-specify them?
If yes, of what use is it to declare them for the library? Documentation only?

No this is unexpected. If the PcdLib dependency of FvbServicesRuntimeDxe.inf resolves to MdePkg/Library/DxePcdLib/DxePcdLib.inf, it should inherit the depex and the protocol dependencies.



Could be the dreaded my INF is wrong but it compiles issue. What happens is dependencies in the INF are missing but get resolved by the library instances or even worse libraries pulled in by the libraries. So it looks to me like it is a missing Depex in the driver. Not all instances of the library, that are valid for the module, imply the dependency.

/Volumes/Case/edk2(master)>git grep PcdLib -- *.inf | grep LIBRARY_CLASS
MdePkg/Library/BasePcdLibNull/BasePcdLibNull.inf:21:  LIBRARY_CLASS                  = PcdLib
MdePkg/Library/DxePcdLib/DxePcdLib.inf:35:  LIBRARY_CLASS                  = PcdLib|DXE_CORE DXE_DRIVER DXE_RUNTIME_DRIVER DXE_SMM_DRIVER SMM_CORE UEFI_APPLICATION UEFI_DRIVER
MdePkg/Library/PeiPcdLib/PeiPcdLib.inf:32:  LIBRARY_CLASS                  = PcdLib|PEIM PEI_CORE SEC



The easiest way to debug these kind of problems is to generate a report file by passing `-y REPORTFILE` to build. For my projects I usually always build a report file and dump it into the Build results directory. This will show what libraries got linked in, and what the actual dependency expression resolved to.




Should I/we try to remove the APRIORI entries from OVMF in a similar way?

Let's get to the bottom of this first. Laszlo may remember why exactly those entries are there in the first place, and I suspect it is a different issue.



It is good to get the history. In general the APRIORI files are used to force a dispatch order for debugging, like getting status codes or serial output quicker.

From an architectural point of view the dispatch order is only defined as "if your Depex is TRUE you can be dispatched". So they APRORI file lets you define the undefined behavior. The implementation happens to walk the FV in order  and will dispatch and drivers with a Depex of TRUE and keep iterating until all the Depex'es are TRUE or all the drivers dispatch. In the "good old days" we would sort the PEIMs in dispatch order so that everything would dispatch on the 1st pass to reduce the number of slow FLASH reads required to boot.

Thanks,

Andrew Fish





<3C121AAE0A3549E984277926BF20E545.png>



[-- Attachment #2: Type: text/html, Size: 60946 bytes --]