From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from ma1-aaemail-dr-lapp02.apple.com (ma1-aaemail-dr-lapp02.apple.com [17.171.2.68]) by mx.groups.io with SMTP id smtpd.web10.10225.1588869783984089961 for ; Thu, 07 May 2020 09:43:04 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@apple.com header.s=20180706 header.b=MwnLyzg0; spf=pass (domain: apple.com, ip: 17.171.2.68, mailfrom: afish@apple.com) Received: from pps.filterd (ma1-aaemail-dr-lapp02.apple.com [127.0.0.1]) by ma1-aaemail-dr-lapp02.apple.com (8.16.0.42/8.16.0.42) with SMTP id 047Gd6OL061857; Thu, 7 May 2020 09:42:56 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=apple.com; h=from : message-id : content-type : mime-version : subject : date : in-reply-to : cc : to : references; s=20180706; bh=agImU2MClzt0PdGHSMwLCPxOHd2/ngBVuZgGAR9rEBU=; b=MwnLyzg0vjj/pKtyHV73dhA13AGyaA33Fu7qbSLEBxANxAqS/sODK3vDvirleb9dNFQN yZI+7Wg2VAP/3vwJOzblF1Q2q8MSjEacLgQ4b+sFipcvDfJk/4yd+YyCo4FjOE20xqrR 5IS/27SBAYHGNBQT1Q6EkkijOXDHAZZvnMB8bPIacFdZ2RE2nv8b+PXSIHgoJCpoJ5/v IxCt4vc/5vL+I6ldW3DkwCkW3BTmILb2V5gXGFtKG73Z51XfLzVBqfXG5H8MjdZOVzmY BT31CAjhLN30kl8wv+dIh/AR2fbhatZIMbKP34zPItLlkxb/3RHh7F1scy89MMALQBWt Eg== Received: from rn-mailsvcp-mta-lapp02.rno.apple.com (rn-mailsvcp-mta-lapp02.rno.apple.com [10.225.203.150]) by ma1-aaemail-dr-lapp02.apple.com with ESMTP id 30s5huk352-4 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Thu, 07 May 2020 09:42:56 -0700 Received: from rn-mailsvcp-mmp-lapp02.rno.apple.com (rn-mailsvcp-mmp-lapp02.rno.apple.com [17.179.253.15]) by rn-mailsvcp-mta-lapp02.rno.apple.com (Oracle Communications Messaging Server 8.1.0.5.20200312 64bit (built Mar 12 2020)) with ESMTPS id <0Q9Y0089MZRJB730@rn-mailsvcp-mta-lapp02.rno.apple.com>; Thu, 07 May 2020 09:42:56 -0700 (PDT) Received: from process_milters-daemon.rn-mailsvcp-mmp-lapp02.rno.apple.com by rn-mailsvcp-mmp-lapp02.rno.apple.com (Oracle Communications Messaging Server 8.1.0.5.20200312 64bit (built Mar 12 2020)) id <0Q9Y00U00ZP47A00@rn-mailsvcp-mmp-lapp02.rno.apple.com>; Thu, 07 May 2020 09:42:55 -0700 (PDT) X-Va-A: X-Va-T-CD: 678bf7de5df0d9ff994f556fd1b44182 X-Va-E-CD: 65487e2d067d889d0e2ef78838d2b299 X-Va-R-CD: aacc6f0df4934f088d8b0d7238b9f38b X-Va-CD: 0 X-Va-ID: d65c2776-6266-4ed1-8d9a-f911e2305714 X-V-A: X-V-T-CD: 678bf7de5df0d9ff994f556fd1b44182 X-V-E-CD: 65487e2d067d889d0e2ef78838d2b299 X-V-R-CD: aacc6f0df4934f088d8b0d7238b9f38b X-V-CD: 0 X-V-ID: 73bde5b3-659d-481a-ae03-51c2588a4d82 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.216,18.0.676 definitions=2020-05-07_10:2020-05-07,2020-05-07 signatures=0 Received: from [17.235.26.108] (unknown [17.235.26.108]) by rn-mailsvcp-mmp-lapp02.rno.apple.com (Oracle Communications Messaging Server 8.1.0.5.20200312 64bit (built Mar 12 2020)) with ESMTPSA id <0Q9Y00CC7ZRIWP00@rn-mailsvcp-mmp-lapp02.rno.apple.com>; Thu, 07 May 2020 09:42:55 -0700 (PDT) From: "Andrew Fish" Message-id: <9B467C1E-7EAC-49D9-B646-D91E60FD7539@apple.com> MIME-version: 1.0 (Mac OS X Mail 13.0 \(3594.4.17\)) Subject: Re: [edk2-devel] APRIORI in RISC-V or Where did OVMF APRIORIs come from? Date: Thu, 07 May 2020 09:42:53 -0700 In-reply-to: Cc: Daniel Schaefer , "Chang, Abner (HPS SW/FW Technologist)" , Atish Patra , Heinrich Schuchardt , Atish Patra , Alexander Graf , Anup Patel , "leif@nuviainc.com" , Jordan Justen , Laszlo Ersek To: devel@edk2.groups.io, Ard Biesheuvel References: <20200224221949.28826-1-atish.patra@wdc.com> <4de513ea-26fb-c830-2348-c0f57946fd3d@hpe.com> <3cb1d47e-983f-dd93-33d2-5a7896791dde@arm.com> X-Mailer: Apple Mail (2.3594.4.17) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.216,18.0.676 definitions=2020-05-07_10:2020-05-07,2020-05-07 signatures=0 Content-type: multipart/alternative; boundary="Apple-Mail=_DC6E3FA3-C044-495D-A896-0904E296165E" --Apple-Mail=_DC6E3FA3-C044-495D-A896-0904E296165E Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii > On May 7, 2020, at 6:53 AM, Ard Biesheuvel wrot= e: >=20 > (+ Laszlo) >=20 > On 5/7/20 3:43 PM, Daniel Schaefer wrote: >> On 5/7/20 3:24 PM, Ard Biesheuvel wrote: >>> On 5/7/20 3:18 PM, Daniel Schaefer via groups.io wrote: >>>> Hi Ard and others, >>>>=20 >>>> TLDR; We have APRIORI definitions from other places in EDK2 but there= 's no explanation as to why they are there. >>>>=20 >>>> I'm taking this to the EDK2 list, since it doesn't concern U-Boot. >>>> I kept some other people related to UEFI, maybe you're interested ;) >>>>=20 >>>> On 2/25/20 10:07 AM, Ard Biesheuvel wrote: >>>> > What I did notice is the use of APRIORI PEI and APRIORI DXE sectio= ns >>>> > in your platform descriptions. I recommend you try to avoid that, = as >>>> > it is a maintenance burden going forward: instead, please use dumm= y >>>> > protocols and NULL library class resolutions if you need to make >>>> > generic components depend on platform specific protocols. Also, pl= ease >>>> > document this - the APRIORI section does not explain *why* you hav= e to >>>> > circumvent the ordinary dependency tree based module dispatch. >>>>=20 >>>> I'm taking a look at this right now. >>>> You're absolutely right - we should reduce or document APRIORIs. >>>>=20 >>>> However, Abner told me that he had only copied most of the FDF from o= ther >>>> places in EDK2.This is what we currently have: >>>>=20 >>>> APRIORI PEI { >>>> INF MdeModulePkg/Universal/ReportStatusCodeRouter/Pei/ReportStatus= CodeRouterPei.inf=20 >>>> INF MdeModulePkg/Universal/StatusCodeHandler/Pei/StatusCodeHandler= Pei.inf >>>> INF MdeModulePkg/Universal/PCD/Pei/Pcd.inf >>>> } >>>> APRIORI DXE { >>>> INF MdeModulePkg/Universal/DevicePathDxe/DevicePathDxe.inf >>>> INF MdeModulePkg/Universal/PCD/Dxe/Pcd.inf >>>> INF Platform/SiFive/U5SeriesPkg/Universal/Dxe/RamFvbServicesRuntim= eDxe/FvbServicesRuntimeDxe.inf=20 >>>> } >>>>=20 >>>> I can remove all of APRIORI PEI and it boots properly. Of the DXEs I = can only >>>> remove FvbServicesRuntimeDxe, otherwise some DXEs are dispatched in t= he wrong >>>> order and boot fails. >>>=20 >>> This means some modules have an undeclared dependency on one of the re= maining modules. Can you elaborate on how the boot fails in this case? >>>=20 >> The error is >> ASSERT [FvbServicesRuntimeDxe] /edk2/MdePkg/Library/DxePcdLib/DxePcdL= ib.c(72): !EFI_ERROR (Status) >> In this line, DxePcdLib tries to consume gPcdProtocolGuid. Therefor if = I add >> the following to FvbServicesRuntimeDxe.inf: >> [Depex] >> gEfiPcdProtocolGuid >> [Protocols] >> gPcdProtocolGuid ## SOMETIMES_CONSUMES >> gEfiPcdProtocolGuid ## CONSUMES >> gGetPcdInfoProtocolGuid ## SOMETIMES_CONSUMES >> gEfiGetPcdInfoProtocolGuid ## SOMETIMES_CONSUMES >> I can boot without error. >> Looking at MdePkg/Library/DxePcdLib/DxePcdLib.inf I see that the librar= y has >> exactly the same Depex and Protocols specified. Do DXEs have re-specify= them? >> If yes, of what use is it to declare them for the library? Documentatio= n only? >=20 > No this is unexpected. If the PcdLib dependency of FvbServicesRuntimeDxe= .inf resolves to MdePkg/Library/DxePcdLib/DxePcdLib.inf, it should inherit = the depex and the protocol dependencies. >=20 Could be the dreaded my INF is wrong but it compiles issue. What happens i= s dependencies in the INF are missing but get resolved by the library insta= nces or even worse libraries pulled in by the libraries. So it looks to me = like it is a missing Depex in the driver. Not all instances of the library,= that are valid for the module, imply the dependency.=20 /Volumes/Case/edk2(master)>git grep PcdLib -- *.inf | grep LIBRARY_CLASS MdePkg/Library/BasePcdLibNull/BasePcdLibNull.inf:21: LIBRARY_CLASS = =3D PcdLib MdePkg/Library/DxePcdLib/DxePcdLib.inf:35: LIBRARY_CLASS = =3D PcdLib|DXE_CORE DXE_DRIVER DXE_RUNTIME_DRIVER DXE_SMM_DRIVER SMM_CORE = UEFI_APPLICATION UEFI_DRIVER MdePkg/Library/PeiPcdLib/PeiPcdLib.inf:32: LIBRARY_CLASS = =3D PcdLib|PEIM PEI_CORE SEC The easiest way to debug these kind of problems is to generate a report fi= le by passing `-y REPORTFILE` to build. For my projects I usually always bu= ild a report file and dump it into the Build results directory. This will s= how what libraries got linked in, and what the actual dependency expression= resolved to.=20 >> Should I/we try to remove the APRIORI entries from OVMF in a similar wa= y? >=20 > Let's get to the bottom of this first. Laszlo may remember why exactly t= hose entries are there in the first place, and I suspect it is a different = issue. >=20 It is good to get the history. In general the APRIORI files are used to fo= rce a dispatch order for debugging, like getting status codes or serial out= put quicker.=20 >>From an architectural point of view the dispatch order is only defined as = "if your Depex is TRUE you can be dispatched". So they APRORI file lets you= define the undefined behavior. The implementation happens to walk the FV i= n order and will dispatch and drivers with a Depex of TRUE and keep iterat= ing until all the Depex'es are TRUE or all the drivers dispatch. In the "go= od old days" we would sort the PEIMs in dispatch order so that everything w= ould dispatch on the 1st pass to reduce the number of slow FLASH reads requ= ired to boot.=20 Thanks, Andrew Fish >=20 --Apple-Mail=_DC6E3FA3-C044-495D-A896-0904E296165E Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii
On May 7,= 2020, at 6:53 AM, Ard Biesheuvel <Ard.Biesheuvel@arm.com> wrote:

(+ Laszlo)

On 5/7/20 3:43 PM, Daniel Schaefer wrote:
On 5/7/20 3:24 PM, Ard Biesheuvel wrote:<= br class=3D"">
On 5/7/20 3:18 PM, Danie= l Schaefer via groups.io wrote:=
Hi Ard and others,

TLDR; We have APRIORI definitions from other place= s in EDK2 but there's no explanation as to why they are there.

I'm taking this to the EDK2 list, since it doesn't concern= U-Boot.
I kept some other people related to UEFI, maybe you'= re interested ;)

On 2/25/20 10:07 AM, Ard Bies= heuvel wrote:
 > What I did notice is the use of APRI= ORI PEI and APRIORI DXE sections
 > in your platform = descriptions. I recommend you try to avoid that, as
 >= ; it is a maintenance burden going forward: instead, please use dummy
 > protocols and NULL library class resolutions if you nee= d to make
 > generic components depend on platform sp= ecific protocols. Also, please
 > document this - the= APRIORI section does not explain *why* you have to
 >= ; circumvent the ordinary dependency tree based module dispatch.

I'm taking a look at this right now.
Yo= u're absolutely right - we should reduce or document APRIORIs.

However, Abner told me that he had only copied most of the= FDF from other
places in EDK2.This is what we currently have= :

APRIORI PEI {
   INF MdeModulePkg/Universal/R= eportStatusCodeRouter/Pei/ReportStatusCodeRouterPei.inf 
   INF MdeModulePkg/Universal/StatusCodeHand= ler/Pei/StatusCodeHandlerPei.inf
   INF  MdeModulePkg/Universal/PCD/Pei= /Pcd.inf
}
APRIORI DXE {
 &n= bsp; INF MdeModulePkg/Uni= versal/DevicePathDxe/DevicePathDxe.inf
   INF  MdeModulePkg/Universal/P= CD/Dxe/Pcd.inf
   INF Platform/SiFive/U5SeriesPkg/Universal/Dxe/RamFvbServic= esRuntimeDxe/FvbServicesRuntimeDxe.inf 
}

I can remove al= l of APRIORI PEI and it boots properly. Of the DXEs I can only
remove FvbServicesRuntimeDxe, otherwise some DXEs are dispatched in the w= rong
order and boot fails.

This means some modules have an undeclared dependency on one of the= remaining modules. Can you elaborate on how the boot fails in this case?
The error is
 <= span class=3D"Apple-converted-space"> ASSERT [FvbServicesRuntim= eDxe] /edk2/MdePkg/Library/DxePcdLib/DxePcdLib.c(72): !EFI_ERROR (Status)In this line, DxePcdLib tries to consume gPcdProtocolGuid. The= refor if I add
the following to FvbServicesRuntimeDxe.inf:[Depex]
  gEfiPcdProtocolGuid
[Protocols]
  gPcdProtocolGuid&= nbsp;           &nbs= p;            &= nbsp;    ## SOMETIMES_CONSUMES
  gEfiPcdProtocolGuid  &n= bsp;            = ;            ## CONS= UMES
  gGetPcdInfoProtocolGuid        &nb= sp;            =   ## SOMETIMES_CONSUMES
  gEfiGetPcdInfoProtocolGuid    =             &nb= sp;   ## SOMETIMES_CONSUMES
I can boot without erro= r.
Looking at MdePkg/Library/DxePcdLib/DxePcdLib.inf I see th= at the library has
exactly the same Depex and Protocols speci= fied. Do DXEs have re-specify them?
If yes, of what use is it= to declare them for the library? Documentation only?

No this is unexpected. = If the PcdLib dependency of FvbServicesRuntimeDxe.inf resolves to MdePkg/Li= brary/DxePcdLib/DxePcdLib.inf, it should inherit the depex and the protocol= dependencies.


C= ould be the dreaded my INF is wrong but it compiles issue. What happens is = dependencies in the INF are missing but get resolved by the library instanc= es or even worse libraries pulled in by the libraries. So it looks to me li= ke it is a missing Depex in the driver. Not all instances of the library, t= hat are valid for the module, imply the dependency. 

/Volumes/Case/edk2(master)>git grep PcdLib -- *.inf | grep LIBRARY_CLASS
=
MdePkg/Library= /BasePcdLibNull/BasePcdLibNull.inf:21:  LIBRARY_CLASS    &nb= sp;             =3D PcdLib
MdePkg/Library/DxePc= dLib/DxePcdLib.inf:35:  LIBRARY_CLASS         = ;         =3D PcdLib|DXE_CORE DXE_DRIVER DXE_RUNTIME_DR= IVER DXE_SMM_DRIVER SMM_CORE UEFI_APPLICATION UEFI_DRIVER
MdePkg/Library/PeiP= cdLib/PeiPcdLib.inf:32:  LIBRARY_CLASS        &nbs= p;         =3D PcdLib|PEIM PEI_CORE SEC

The easiest way to debug th= ese kind of problems is to generate a report file by passing `-y REPOR= TFILE` to build. For my projects I usually always build a report file and d= ump it into the Build results directory. This will show what libraries got = linked in, and what the actual dependency expression resolved to. 


Should I/we try to remove the AP= RIORI entries from OVMF in a similar way?

Let's get to the bottom of this fir= st. Laszlo may remember why exactly those entries are there in the first pl= ace, and I suspect it is a different issue.


It is good to get the history. In general the A= PRIORI files are used to force a dispatch order for debugging, like getting= status codes or serial output quicker. 

From an architectural point of view the dispatch order is only defin= ed as "if your Depex is TRUE you can be dispatched". So they APRORI file le= ts you define the undefined behavior. The implementation happens to walk th= e FV in order  and will dispatch and drivers with a Depex of TRUE and = keep iterating until all the Depex'es are TRUE or all the drivers dispatch.= In the "good old days" we would sort the PEIMs in dispatch order so that e= verything would dispatch on the 1st pass to reduce the number of slow FLASH= reads required to boot. 

Thanks,<= /div>

Andrew Fish


--Apple-Mail=_DC6E3FA3-C044-495D-A896-0904E296165E--