From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from ma1-aaemail-dr-lapp03.apple.com (ma1-aaemail-dr-lapp03.apple.com [17.171.2.72]) by mx.groups.io with SMTP id smtpd.web11.10629.1588870484189105663 for ; Thu, 07 May 2020 09:54:44 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@apple.com header.s=20180706 header.b=laGqEDs3; spf=pass (domain: apple.com, ip: 17.171.2.72, mailfrom: afish@apple.com) Received: from pps.filterd (ma1-aaemail-dr-lapp03.apple.com [127.0.0.1]) by ma1-aaemail-dr-lapp03.apple.com (8.16.0.42/8.16.0.42) with SMTP id 047Gq4ZY005879; Thu, 7 May 2020 09:54:38 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=apple.com; h=from : message-id : content-type : mime-version : subject : date : in-reply-to : cc : to : references; s=20180706; bh=0MlTLH/EN7DdVO9suo2VCup5IK0v+sUgN7eC3WGUDEQ=; b=laGqEDs3Yog1GdGrm2MrgBMKmWXIHgw9TgpaRbfE52vLazqeirV3vAES/TPXVIV2TsQ0 7x5hhuB2Ga6swPgURnsSjsFHySbswEmaW/unx/ZqscgXq72ckE2HHN1Sed+Ptr1MeyD8 YSntjvsK71yW8eGx+CcUiSPOoNXOIccTwkCm2EIJPdPuF1pHhhjPJDbqHuUAq8rRMD3u FOkznxw0Q72yaBjmU+8oMY0sLSwuHeTyEp05fgyV2FzTtNo+wobNkiNCgrcKhJG8WX/p fo3dA+vZfxyEvQT55/SMFXffDmZiF90GF+H8gbVVlhBJ44gSGWrmh6EbUIcV76xddD0j oA== Received: from rn-mailsvcp-mta-lapp01.rno.apple.com (rn-mailsvcp-mta-lapp01.rno.apple.com [10.225.203.149]) by ma1-aaemail-dr-lapp03.apple.com with ESMTP id 30s7uw10nh-28 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Thu, 07 May 2020 09:54:38 -0700 Received: from rn-mailsvcp-mmp-lapp02.rno.apple.com (rn-mailsvcp-mmp-lapp02.rno.apple.com [17.179.253.15]) by rn-mailsvcp-mta-lapp01.rno.apple.com (Oracle Communications Messaging Server 8.1.0.5.20200312 64bit (built Mar 12 2020)) with ESMTPS id <0Q9Z00WDH0B0ID30@rn-mailsvcp-mta-lapp01.rno.apple.com>; Thu, 07 May 2020 09:54:36 -0700 (PDT) Received: from process_milters-daemon.rn-mailsvcp-mmp-lapp02.rno.apple.com by rn-mailsvcp-mmp-lapp02.rno.apple.com (Oracle Communications Messaging Server 8.1.0.5.20200312 64bit (built Mar 12 2020)) id <0Q9Y00T00ZOWZ600@rn-mailsvcp-mmp-lapp02.rno.apple.com>; Thu, 07 May 2020 09:54:36 -0700 (PDT) X-Va-A: X-Va-T-CD: 678bf7de5df0d9ff994f556fd1b44182 X-Va-E-CD: 7902584702c768eb62e408b06827ab8f X-Va-R-CD: 3840e259befd0e6bbcb2b736ea75c1b4 X-Va-CD: 0 X-Va-ID: 734c88f2-3faa-452a-bd79-eaf9ca850bf3 X-V-A: X-V-T-CD: 678bf7de5df0d9ff994f556fd1b44182 X-V-E-CD: 7902584702c768eb62e408b06827ab8f X-V-R-CD: 3840e259befd0e6bbcb2b736ea75c1b4 X-V-CD: 0 X-V-ID: 294ba3e9-4755-4e16-8d51-8c77bcc51e85 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.216,18.0.676 definitions=2020-05-07_10:2020-05-07,2020-05-07 signatures=0 Received: from [17.235.26.108] (unknown [17.235.26.108]) by rn-mailsvcp-mmp-lapp02.rno.apple.com (Oracle Communications Messaging Server 8.1.0.5.20200312 64bit (built Mar 12 2020)) with ESMTPSA id <0Q9Z00N660AY7B00@rn-mailsvcp-mmp-lapp02.rno.apple.com>; Thu, 07 May 2020 09:54:35 -0700 (PDT) From: "Andrew Fish" Message-id: <3851BBC9-86F5-4A8B-BC24-E18960144192@apple.com> MIME-version: 1.0 (Mac OS X Mail 13.0 \(3594.4.17\)) Subject: Re: [EXTERNAL] [edk2-devel] APRIORI in RISC-V or Where did OVMF APRIORIs come from? Date: Thu, 07 May 2020 09:54:33 -0700 In-reply-to: Cc: Ard Biesheuvel , Daniel Schaefer , "Chang, Abner (HPS SW/FW Technologist)" , Atish Patra , Heinrich Schuchardt , Atish Patra , Alexander Graf , Anup Patel , "leif@nuviainc.com" , Jordan Justen , Laszlo Ersek To: devel@edk2.groups.io, bret.barkelew@microsoft.com References: <20200224221949.28826-1-atish.patra@wdc.com> <4de513ea-26fb-c830-2348-c0f57946fd3d@hpe.com> <3cb1d47e-983f-dd93-33d2-5a7896791dde@arm.com> <9B467C1E-7EAC-49D9-B646-D91E60FD7539@apple.com> X-Mailer: Apple Mail (2.3594.4.17) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.216,18.0.676 definitions=2020-05-07_10:2020-05-07,2020-05-07 signatures=0 Content-type: multipart/alternative; boundary="Apple-Mail=_06AB3140-7847-40C4-B6C0-5ABF10D5D8A5" --Apple-Mail=_06AB3140-7847-40C4-B6C0-5ABF10D5D8A5 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Bret, How does that test work? Does it make a custom FDF file?=20 Thanks, Andrew Fish > On May 7, 2020, at 9:45 AM, Bret Barkelew via groups.io wrote: >=20 > I know I=E2=80=99ve also seen tests that randomize the driver dispatch o= rder to try to catch these =E2=80=9Cimplementation-specific=E2=80=9D edge c= ases. Perhaps we could instrument something similar with a weekly OVMF CI t= est? > > - Bret > > From: Andrew Fish via groups.io > Sent: Thursday, May 7, 2020 9:43 AM > To: devel@edk2.groups.io ; Ard Biesheuvel <= mailto:ard.biesheuvel@arm.com> > Cc: Daniel Schaefer ; Chang, Abner (HPS = SW/FW Technologist) ; Atish Patra ; Heinrich Schuchardt ; Atish P= atra ; Alexander Graf ;= Anup Patel ; leif@nuviainc.com ; Jordan Justen ; Laszlo Ersek = > Subject: [EXTERNAL] Re: [edk2-devel] APRIORI in RISC-V or Where did OVMF= APRIORIs come from? > > >=20 >=20 > On May 7, 2020, at 6:53 AM, Ard Biesheuvel > wrote: > > (+ Laszlo) >=20 > On 5/7/20 3:43 PM, Daniel Schaefer wrote: >=20 > On 5/7/20 3:24 PM, Ard Biesheuvel wrote: >=20 > On 5/7/20 3:18 PM, Daniel Schaefer via groups.io wrote: >=20 > Hi Ard and others, >=20 > TLDR; We have APRIORI definitions from other places in EDK2 but there's = no explanation as to why they are there. >=20 > I'm taking this to the EDK2 list, since it doesn't concern U-Boot. > I kept some other people related to UEFI, maybe you're interested ;) >=20 > On 2/25/20 10:07 AM, Ard Biesheuvel wrote: > > What I did notice is the use of APRIORI PEI and APRIORI DXE sections > > in your platform descriptions. I recommend you try to avoid that, as > > it is a maintenance burden going forward: instead, please use dummy > > protocols and NULL library class resolutions if you need to make > > generic components depend on platform specific protocols. Also, pleas= e > > document this - the APRIORI section does not explain *why* you have t= o > > circumvent the ordinary dependency tree based module dispatch. >=20 > I'm taking a look at this right now. > You're absolutely right - we should reduce or document APRIORIs. >=20 > However, Abner told me that he had only copied most of the FDF from othe= r > places in EDK2.This is what we currently have: >=20 > APRIORI PEI { > INF MdeModulePkg/Universal/ReportStatusCodeRouter/Pei/ReportStatusCod= eRouterPei.inf=20 > INF MdeModulePkg/Universal/StatusCodeHandler/Pei/StatusCodeHandlerPei= .inf > INF MdeModulePkg/Universal/PCD/Pei/Pcd.inf > } > APRIORI DXE { > INF MdeModulePkg/Universal/DevicePathDxe/DevicePathDxe.inf > INF MdeModulePkg/Universal/PCD/Dxe/Pcd.inf > INF Platform/SiFive/U5SeriesPkg/Universal/Dxe/RamFvbServicesRuntimeDx= e/FvbServicesRuntimeDxe.inf=20 > } >=20 > I can remove all of APRIORI PEI and it boots properly. Of the DXEs I can= only > remove FvbServicesRuntimeDxe, otherwise some DXEs are dispatched in the = wrong > order and boot fails. >=20 > This means some modules have an undeclared dependency on one of the rema= ining modules. Can you elaborate on how the boot fails in this case? >=20 > The error is > ASSERT [FvbServicesRuntimeDxe] /edk2/MdePkg/Library/DxePcdLib/DxePcdLi= b.c(72): !EFI_ERROR (Status) > In this line, DxePcdLib tries to consume gPcdProtocolGuid. Therefor if I= add > the following to FvbServicesRuntimeDxe.inf: > [Depex] > gEfiPcdProtocolGuid > [Protocols] > gPcdProtocolGuid ## SOMETIMES_CONSUMES > gEfiPcdProtocolGuid ## CONSUMES > gGetPcdInfoProtocolGuid ## SOMETIMES_CONSUMES > gEfiGetPcdInfoProtocolGuid ## SOMETIMES_CONSUMES > I can boot without error. > Looking at MdePkg/Library/DxePcdLib/DxePcdLib.inf I see that the library= has > exactly the same Depex and Protocols specified. Do DXEs have re-specify = them? > If yes, of what use is it to declare them for the library? Documentation= only? >=20 > No this is unexpected. If the PcdLib dependency of FvbServicesRuntimeDxe= .inf resolves to MdePkg/Library/DxePcdLib/DxePcdLib.inf, it should inherit = the depex and the protocol dependencies. >=20 > > Could be the dreaded my INF is wrong but it compiles issue. What happens= is dependencies in the INF are missing but get resolved by the library ins= tances or even worse libraries pulled in by the libraries. So it looks to m= e like it is a missing Depex in the driver. Not all instances of the librar= y, that are valid for the module, imply the dependency.=20 > > /Volumes/Case/edk2(master)>git grep PcdLib -- *.inf | grep LIBRARY_CLASS > MdePkg/Library/BasePcdLibNull/BasePcdLibNull.inf:21: LIBRARY_CLASS = =3D PcdLib > MdePkg/Library/DxePcdLib/DxePcdLib.inf:35: LIBRARY_CLASS = =3D PcdLib|DXE_CORE DXE_DRIVER DXE_RUNTIME_DRIVER DXE_SMM_DRIVER SMM_COR= E UEFI_APPLICATION UEFI_DRIVER > MdePkg/Library/PeiPcdLib/PeiPcdLib.inf:32: LIBRARY_CLASS = =3D PcdLib|PEIM PEI_CORE SEC >=20 >=20 > The easiest way to debug these kind of problems is to generate a report = file by passing `-y REPORTFILE` to build. For my projects I usually always = build a report file and dump it into the Build results directory. This will= show what libraries got linked in, and what the actual dependency expressi= on resolved to.=20 > >=20 >=20 > Should I/we try to remove the APRIORI entries from OVMF in a similar way= ? >=20 > Let's get to the bottom of this first. Laszlo may remember why exactly t= hose entries are there in the first place, and I suspect it is a different = issue. >=20 > > It is good to get the history. In general the APRIORI files are used to = force a dispatch order for debugging, like getting status codes or serial o= utput quicker.=20 > > From an architectural point of view the dispatch order is only defined a= s "if your Depex is TRUE you can be dispatched". So they APRORI file lets y= ou define the undefined behavior. The implementation happens to walk the FV= in order and will dispatch and drivers with a Depex of TRUE and keep iter= ating until all the Depex'es are TRUE or all the drivers dispatch. In the "= good old days" we would sort the PEIMs in dispatch order so that everything= would dispatch on the 1st pass to reduce the number of slow FLASH reads re= quired to boot.=20 > > Thanks, > > Andrew Fish >=20 >=20 > > >=20 > <3C121AAE0A3549E984277926BF20E545.png> --Apple-Mail=_06AB3140-7847-40C4-B6C0-5ABF10D5D8A5 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 Bret,

How does that test work? Does it make a cu= stom FDF file? 

Thanks,

An= drew Fish

On May 7, 2020, at 9:45 AM, Bret Barkelew via groups.io <bret.barkelew=3Dmicrosoft.co= m@groups.io> wrote:

I know I=E2=80=99ve also seen tests t= hat randomize the driver dispatch order to try to catch these =E2=80=9Cimpl= ementation-specific=E2=80=9D edge cases. Perhaps we could instrument someth= ing similar with a weekly OVMF CI test?
 
- Bret
 
From: Andrew Fish = via groups.io
Sent: Thursday, May 7, 2020 9:43 AM
= To: devel@edk2.groups.io; Ard BiesheuvelCc: Daniel Schaefer; Chang, Abner (= HPS SW/FW Technologist); Atish Patra; Heinrich Schuchardt; = Atish Patra; Alexander Graf;&n= bsp;Anup Patel; leif@nuviainc.com; Jordan Justen; Laszlo Ersek
Subje= ct: [EXTERNAL] Re: [e= dk2-devel] APRIORI in RISC-V or Where did OVMF APRIORIs come from?
 
 


<= /div>
=
On May 7, 2020, at 6:53 AM, Ard= Biesheuvel <Ard.Biesheuvel@arm.com>= wrote:
 
(+ Laszlo)

On 5/7/20 3:43 PM, Daniel Schaefer= wrote:

On 5/7/20 3:24 PM, Ard Biesheuvel wrote:

On 5/7= /20 3:18 PM, Daniel Schaefer via = ;groups.io wrote:

=
Hi Ard and others,

= TLDR; We have APRIORI definitions from other places in EDK2 but there's no = explanation as to why they are there.

I'm taki= ng this to the EDK2 list, since it doesn't concern U-Boot.
I = kept some other people related to UEFI, maybe you're interested ;)

On 2/25/20 10:07 AM, Ard Biesheuvel wrote:
 > What I did notice is the use of APRIORI PEI and APRIORI D= XE sections
 > in your platform descriptions. I recom= mend you try to avoid that, as
 > it is a maintenance= burden going forward: instead, please use dummy
 > p= rotocols and NULL library class resolutions if you need to make
 > generic components depend on platform specific protocols. Als= o, please
 > document this - the APRIORI section does= not explain *why* you have to
 > circumvent the ordi= nary dependency tree based module dispatch.

I'= m taking a look at this right now.
You're absolutely right - = we should reduce or document APRIORIs.

However= , Abner told me that he had only copied most of the FDF from other
places in EDK2.This is what we currently have:

APRIORI PEI {
   INF MdeModulePkg/Universal/ReportStatusCodeRouter/P= ei/ReportStatusCodeRouterPei.inf = ;
  &n= bsp;INF MdeModulePkg/Universal/StatusCodeHandler/Pei/StatusCodeHandl= erPei.inf
  &= nbsp;INF  MdeModulePkg/Universal/PCD/Pei/Pcd.inf
= }
APRIORI DXE {
   INF MdeModulePkg/Universal/DevicePathDxe/De= vicePathDxe.inf
   INF  MdeModulePkg/Universal/PCD/Dxe/Pcd.inf
   INF P= latform/SiFive/U5SeriesPkg/Universal/Dxe/RamFvbServicesRuntimeDxe/FvbServic= esRuntimeDxe.inf 
}

I can remove all of APRIORI PEI and i= t boots properly. Of the DXEs I can only
remove FvbServicesRu= ntimeDxe, otherwise some DXEs are dispatched in the wrong
ord= er and boot fails.


This means some modules have an = undeclared dependency on one of the remaining modules. Can you elaborate on= how the boot fails in this case?

The error is
  ASSERT [FvbServicesRuntimeDxe]= /edk2/MdePkg/Library/DxePcdLib/DxePcdLib.c(72): !EFI_ERROR (Status)
In this line, DxePcdLib tries to consume gPcdProtocolGuid. Therefor= if I add
the following to FvbServicesRuntimeDxe.inf:
[Depex]
 &n= bsp;gEfiPcdProtocolGuid
[Protocols]
&nbs= p; gPcdProtocolGuid =             &nb= sp;            =     ## SOMETIMES_CONSUMES
  gEfiPcdProtocolGuid   &= nbsp;           &nbs= p;           ## CONSUMES<= br class=3D"">  gGet= PcdInfoProtocolGuid         &n= bsp;            = ; ## SOMETIMES_CONSUMES
  gEfiGetPcdInfoProtocolGuid     = ;            &n= bsp;  ## SOMETIMES_CONSUMES
I can boot without error.Looking at MdePkg/Library/DxePcdLib/DxePcdLib.inf I see that th= e library has
exactly the same Depex and Protocols specified.= Do DXEs have re-specify them?
If yes, of what use is it to d= eclare them for the library? Documentation only?

No this= is unexpected. If the PcdLib dependency of FvbServicesRuntimeDxe.inf resol= ves to MdePkg/Library/DxePcdLib/DxePcdLib.inf, it should inherit the depex = and the protocol dependencies.

 
Could be the dreaded my INF is wrong but it compiles issue. What happens i= s dependencies in the INF are missing but get resolved by the library insta= nces or even worse libraries pulled in by the libraries. So it looks to me = like it is a missing Depex in the driver. Not all instances of the library,= that are valid for the module, imply the dependency. =
 
/Volumes/Case/edk2(master)>git grep PcdLib -- *.i= nf | grep LIBRARY_CLASS
MdePkg/Library/BasePcdLibNull/BasePcdLibNul= l.inf:21:  LIBRARY_CLASS            &nbs= p;     =3D PcdLib
MdePkg/Library/DxePcdLib/DxePcdLib.inf:35= :  LIBRARY_CLASS               = ;   =3D PcdLib|DXE_CORE DXE_DRIVER DXE_RUNTIME_DRIVER DXE_SMM_DRIVER S= MM_CORE UEFI_APPLICATION UEFI_DRIVER
MdePkg/Library/PeiPcdLib/PeiPcdL= ib.inf:32:  LIBRARY_CLASS            &nb= sp;     =3D PcdLib|PEIM PEI_CORE SEC=


The easiest way to debug these kind of problems is to g= enerate a report file by passing `-y REPORTFILE` to build. For my proj= ects I usually always build a report file and dump it into the Build result= s directory. This will show what libraries got linked in, and what the actu= al dependency expression resolved to. 
 <= /div>


Should I/we try to remove the APRIORI entries from OVMF in a similar= way?

Let's get to the bottom of this first. Laszlo may= remember why exactly those entries are there in the first place, and I sus= pect it is a different issue.

 
= It is good to get the history. In general the APRIORI files are used to for= ce a dispatch order for debugging, like getting status codes or serial outp= ut quicker. 
 
From an architectural point of view the dispat= ch order is only defined as "if your Depex is TRUE you can be dispatched". = So they APRORI file lets you define the undefined behavior. The implementat= ion happens to walk the FV in order  and will dispatch and drivers wit= h a Depex of TRUE and keep iterating until all the Depex'es are TRUE or all= the drivers dispatch. In the "good old days" we would sort the PEIMs in di= spatch order so that everything would dispatch on the 1st pass to reduce th= e number of slow FLASH reads required to boot. <= /div>
&nbs= p;
Thanks,
<= o:p class=3D""> 
Andrew Fish


 
<= p class=3D"MsoNormal" style=3D"margin: 0in 0in 0.0001pt; font-size: 11pt; f= ont-family: Calibri, sans-serif;">

 
<3C121AAE0A3549E984277926BF20E545.png>

--Apple-Mail=_06AB3140-7847-40C4-B6C0-5ABF10D5D8A5--