Re: [PATCH 00/14] Introduce Lazy-accept for Tdx guest

["Min Xu" <min.m.xu@intel.com>]

Hi, All
Since the lazy-accept patch-set was sent for review, there are a lot of discussions in multiple OpenSource-Communities (EDK2/QEMU/LinuxKernel). 

Below are the summaries of the discussions/proposals
1. Accept memory under 4G
  - Tom Lendacky's suggestion for SEV-SNP is to pre-accept all memory under 4GB to make all complexity go way. 
     https://edk2.groups.io/g/devel/message/90539 
  - Gerd comments that before the automatic negotiation is available, we use some config option and then accept all memory below 4G or accept all memory.
     https://www.mail-archive.com/qemu-devel@nongnu.org/msg894393.html

2. GetMemoryMapEx / OsIndications
  - https://bugzilla.tianocore.org/show_bug.cgi?id=3987
  - GetMemoryMapEx: cannot resolve the boot-loader because it doesn't know about the kernel's capability.
  - OsIndications & self-report kernel header bit could be a full solution.
 
3.  fw_cfg
  - Add new fw_cfg item (opt/ovmf/AcceptAllMemory) to indicate how to handle the unaccepted memory.
      > True - accept all the memory
      > False - don't accept the memory
      > Default - It allows the firmware to choose depending on various factors.
  - Glaze has submit the patch https://lore.kernel.org/all/20220620223300.1555849-1-dionnaglaze@google.com/
 
4.Put some information in kernel image.  (Peter Gonda)
  - Put information into UTS_VERSION which can be read from the buld bzImage. https://www.spinics.net/lists/linux-mm/msg302506.html
  - Glaze denied it. https://www.spinics.net/lists/linux-mm/msg304973.html
 
5. A protocol kernel call from EFI stub before EBS().(Ard)
  - It doesn't work if boot loader is in the boot flow. 
  - https://www.spinics.net/lists/linux-mm/msg304896.html

Among the above 5 proposals, Proposal 1) is different from the other 4 proposals.
Proposal 1) uses a config option and hence there are 2 kinds of OVMF: OVMF-with-LazyAccept and OVMF-with-FullMemory. End user should be aware of it and choose the correct OVMF for the linux kernel (LazyAccept enlightened or not).
Proposal 3) only works for QEMU because of fw_cfg.

So essentially there are 2 questions about the lazy-accept.
1. If Lazy-accept capability should be automatically negotiated?
2. How to implement the Lazy-Accept in OVMF? 

I wonder if lazy-accept feature can be split into 2 stages. 
1. In first stage there is a config option to indicate if lazy-accept is enabled or not.
2. In the second stage the automatic negotiation is introduced so that lazy-accept is enabled or not by the negotiation result.

Thanks
Min

> -----Original Message-----
> From: Min Xu <min.m.xu@intel.com>
> Sent: Monday, June 6, 2022 11:00 AM
> To: devel@edk2.groups.io
> Cc: Xu, Min M <min.m.xu@intel.com>; Gao, Zhichao
> <zhichao.gao@intel.com>; Kinney, Michael D <michael.d.kinney@intel.com>;
> Liu, Zhiguang <zhiguang.liu@intel.com>; Wang, Jian J
> <jian.j.wang@intel.com>; Gao, Liming <gaoliming@byosoft.com.cn>; Ni, Ray
> <ray.ni@intel.com>; Aktas, Erdem <erdemaktas@google.com>; Gerd
> Hoffmann <kraxel@redhat.com>; James Bottomley <jejb@linux.ibm.com>;
> Yao, Jiewen <jiewen.yao@intel.com>; Tom Lendacky
> <thomas.lendacky@amd.com>; Gao, Jiaqi <jiaqi.gao@intel.com>
> Subject: [PATCH 00/14] Introduce Lazy-accept for Tdx guest
> 
> RFC: https://bugzilla.tianocore.org/show_bug.cgi?id=3937
> 
> UnacceptedMemory is one of the four defined types of TD memory in Intel
> TDX guest. TDVF must invoke TDCALL [TDG.MEM.PAGE.ACCEPT] the
> unaccepted memory before use it. See [TDVF] Section 7.1.
> TDVF: https://www.intel.com/content/dam/develop/external/us/en/
> documents/tdx-virtual-firmware-design-guide-rev-1.01.pdf
> 
> It is a time-consuming task which impacts the boot performance badly.
> One of the mitigation is the lazy-accept mechanism. That the whole system
> memory is divided into 2 parts, one is accepted in bios phase, the other is
> tagged as EfiGcdMemoryTypeUnaccepted and OS will handle these
> "unaccepted" memories.
> See "UEFI Spec v2.9 Table 7-5 Memory Type Usage before
> ExitBootServices()"
> 
> Patch 1-4:
>   Introduce lazy-accept related definitions.
> 
> Patch 5-6:
>   Update Dxe and shell for unaccepted memory.
> 
> Patch 7 - 11:
>   Update OvmfPkg for unaccepted memory.
> 
> Patch 12 - 13:
>   Introduce EfiMemoryAcceptProtocol and realize it in TdxDxe.
> 
> Patch 14:
>   Update Pool and Page functions to accept memory when OOM occurs.
> 
> Code: https://github.com/mxu9/edk2/tree/lazyaccept.v1
> 
> Cc: Zhichao Gao <zhichao.gao@intel.com>
> Cc: Michael D Kinney <michael.d.kinney@intel.com>
> Cc: Zhiguang Liu <zhiguang.liu@intel.com>
> Cc: Jian J Wang <jian.j.wang@intel.com>
> Cc: Liming Gao <gaoliming@byosoft.com.cn>
> Cc: Ray Ni <ray.ni@intel.com>
> Cc: Erdem Aktas <erdemaktas@google.com>
> Cc: Gerd Hoffmann <kraxel@redhat.com>
> Cc: James Bottomley <jejb@linux.ibm.com>
> Cc: Jiewen Yao <jiewen.yao@intel.com>
> Cc: Tom Lendacky <thomas.lendacky@amd.com>
> Signed-off-by: Jiaqi Gao <jiaqi.gao@intel.com>
> Signed-off-by: Min Xu <min.m.xu@intel.com>
> 
> Jiaqi Gao (2):
>   MdePkg: The prototype definition of EfiMemoryAcceptProtocol
>   MdeModulePkg: Pool and page functions accept memory when OOM
> occurs
> 
> Min M Xu (12):
>   MdeModulePkg: Add PrePiHob.h
>   MdePkg: Increase EFI_RESOURCE_MAX_MEMORY_TYPE
>   OvmfPkg: Use EFI_RESOURCE_MEMORY_UNACCEPTED which defined in
>     MdeModulePkg
>   MdePkg: Add UEFI Unaccepted memory definition
>   MdeModulePkg: Update Dxe to handle unaccepted memory type
>   ShellPkg: Update shell command memmap to show unaccepted memory
>   OvmfPkg: Add PCD and DEFINEs for Lazy Accept page.
>   OvmfPkg: Use PcdOvmfWorkAreaBase instead of PcdSevEsWorkAreaBase
>   OvmfPkg: Add MaxAcceptedMemoryAddress in TDX work area
>   OvmfPkg: Introduce lazy accept in PlatformInitLib and PlatformPei
>   OvmfPkg: Update ConstructFwHobList for lazy accept
>   OvmfPkg: Realize EfiMemoryAcceptProtocol in TdxDxe
> 
>  MdeModulePkg/Core/Dxe/DxeMain.inf             |   1 +
>  MdeModulePkg/Core/Dxe/Gcd/Gcd.c               |   5 +
>  MdeModulePkg/Core/Dxe/Mem/Imem.h              |  16 ++
>  MdeModulePkg/Core/Dxe/Mem/Page.c              | 218 ++++++++++++++++++
>  MdeModulePkg/Core/Dxe/Mem/Pool.c              |  14 ++
>  MdeModulePkg/Include/Pi/PrePiHob.h            |  20 ++
>  MdePkg/Include/Pi/PiDxeCis.h                  |   5 +
>  MdePkg/Include/Pi/PiHob.h                     |  11 +-
>  MdePkg/Include/Protocol/MemoryAccept.h        |  37 +++
>  MdePkg/Include/Uefi/UefiMultiPhase.h          |   5 +
>  MdePkg/MdePkg.dec                             |   3 +
>  OvmfPkg/Include/Library/PlatformInitLib.h     |   6 +
>  OvmfPkg/Include/WorkArea.h                    |   1 +
>  OvmfPkg/IntelTdx/IntelTdxX64.dsc              |   8 +
>  .../PrePiHobListPointer.c                     |   4 +-
>  .../PrePiHobListPointerLibTdx.inf             |   2 +-
>  OvmfPkg/Library/PeilessStartupLib/Hob.c       |  26 ++-
>  .../PeilessStartupLib/PeilessStartupLib.inf   |   1 +
>  OvmfPkg/Library/PlatformInitLib/IntelTdx.c    | 145 +++++++++++-
>  OvmfPkg/Library/PlatformInitLib/MemDetect.c   |  27 +++
>  .../PlatformInitLib/PlatformInitLib.inf       |   1 +
>  OvmfPkg/OvmfPkg.dec                           |   4 +
>  OvmfPkg/OvmfPkgX64.dsc                        |   9 +
>  OvmfPkg/PlatformPei/MemDetect.c               |   5 +
>  OvmfPkg/TdxDxe/TdxDxe.c                       | 103 +++++++++
>  OvmfPkg/TdxDxe/TdxDxe.inf                     |   2 +
>  .../UefiShellDebug1CommandsLib/MemMap.c       |   6 +
>  27 files changed, 666 insertions(+), 19 deletions(-)  create mode 100644
> MdeModulePkg/Include/Pi/PrePiHob.h
>  create mode 100644 MdePkg/Include/Protocol/MemoryAccept.h
> 
> --
> 2.29.2.windows.2