[PATCH V2 00/14] Introduce Lazy-accept for Tdx guest

[PATCH V2 00/14] Introduce Lazy-accept for Tdx guest

[Min Xu]

RFC: https://bugzilla.tianocore.org/show_bug.cgi?id=3937

UnacceptedMemory is one of the four defined types of TD memory in Intel
TDX guest. TDVF must invoke TDCALL [TDG.MEM.PAGE.ACCEPT] the unaccepted
memory before use it. See [TDVF] Section 7.1.
TDVF: https://www.intel.com/content/dam/develop/external/us/en/
documents/tdx-virtual-firmware-design-guide-rev-1.01.pdf 

It is a time-consuming task which impacts the boot performance badly.
One of the mitigation is the lazy-accept mechanism. That the whole system
memory is divided into 2 parts, one is accepted in bios phase, the other
is tagged as EfiGcdMemoryTypeUnaccepted and OS will handle these
"unaccepted" memories.
See "UEFI Spec v2.9 Table 7-5 Memory Type Usage before ExitBootServices()"

In current implementation, we configure the lazy accept memory size with
PcdLazyAcceptPartialMemorySize in build time. If the
PcdLazyAcceptPartialMemorySize is 0, it means to accept all the memory
under 4G. This is to optimize the performance.

Patch 1-4: 
  Introduce lazy-accept related definitions.

Patch 5-6:
  Update Dxe and shell for unaccepted memory.

Patch 7 - 11:
  Update OvmfPkg for unaccepted memory.

Patch 12 - 13:
  Introduce EfiMemoryAcceptProtocol and realize it in TdxDxe.

Patch 14:
  Update Pool and Page functions to accept memory when OOM occurs.

Code: https://github.com/mxu9/edk2/tree/lazyaccept.v2

v2 changes:
 - Fix a typo that change EfiUnacceptedMemory to EfiUnacceptedMemoryType.
 - Define EFI_GCD_MEMORY_TYPE_UNACCEPTED in PrePiDxeCis.h because it has
   not been defined in PI spec.
 - AllocatePages should return EFI_INVALID_PARAMETERS if input MemoryType
   is EfiUnacceptedMemoryType. 
 - Use EDKII_ prefix instead of EFI_ prefix in the protocol name of
   EDKII_MEMORY_ACCEPT_PROTOCOL_GUID. Because this protocol is not EFI
   defined.
 - Accept memory under 4G even if the PcdLazyAcceptPartialMemorySize is
   bigger than 4G. So with this setting, even if the
   PcdLazyAcceptPartialMemorySize is 0 (which means to accept all
   memories), only the memory under 4G will be accepted. This is to
   optimize the performance.

Cc: Zhichao Gao <zhichao.gao@intel.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Zhiguang Liu <zhiguang.liu@intel.com>
Cc: Jian J Wang <jian.j.wang@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Erdem Aktas <erdemaktas@google.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Jiaqi Gao <jiaqi.gao@intel.com>
Signed-off-by: Min Xu <min.m.xu@intel.com>

Jiaqi Gao (2):
  MdePkg: The prototype definition of EdkiiMemoryAcceptProtocol
  MdeModulePkg: Pool and page functions accept memory when OOM occurs

Min M Xu (12):
  MdeModulePkg: Add PrePiHob.h
  MdePkg: Increase EFI_RESOURCE_MAX_MEMORY_TYPE
  OvmfPkg: Use EFI_RESOURCE_MEMORY_UNACCEPTED which defined in
    MdeModulePkg
  MdePkg: Add UEFI Unaccepted memory definition
  MdeModulePkg: Update Dxe to handle unaccepted memory type
  ShellPkg: Update shell command memmap to show unaccepted memory
  OvmfPkg: Add PCD and DEFINEs for Lazy Accept page.
  OvmfPkg: Add MaxAcceptedMemoryAddress in TDX work area
  OvmfPkg: Introduce lazy accept in PlatformInitLib and PlatformPei
  OvmfPkg: Update ConstructFwHobList for lazy accept
  OvmfPkg: Realize EdkiiMemoryAcceptProtocol in TdxDxe
  OvmfPkg: Call gEdkiiMemoryAcceptProtocolGuid to accept pages

 MdeModulePkg/Core/Dxe/DxeMain.inf             |   1 +
 MdeModulePkg/Core/Dxe/Gcd/Gcd.c               |   6 +
 MdeModulePkg/Core/Dxe/Mem/Imem.h              |  16 ++
 MdeModulePkg/Core/Dxe/Mem/Page.c              | 253 ++++++++++++++++--
 MdeModulePkg/Core/Dxe/Mem/Pool.c              |  14 +
 MdeModulePkg/Include/Pi/PrePiDxeCis.h         |  25 ++
 MdeModulePkg/Include/Pi/PrePiHob.h            |  20 ++
 MdePkg/Include/Pi/PiDxeCis.h                  |  10 +-
 MdePkg/Include/Pi/PiHob.h                     |  11 +-
 MdePkg/Include/Protocol/MemoryAccept.h        |  37 +++
 MdePkg/Include/Uefi/UefiMultiPhase.h          |   5 +
 MdePkg/MdePkg.dec                             |   3 +
 OvmfPkg/Include/Library/PlatformInitLib.h     |   6 +
 OvmfPkg/Include/WorkArea.h                    |   1 +
 OvmfPkg/IntelTdx/IntelTdxX64.dsc              |   8 +
 .../BaseMemEncryptTdxLib.inf                  |   3 +
 .../BaseMemEncryptTdxLib/MemoryEncryption.c   |  12 +-
 OvmfPkg/Library/PeilessStartupLib/Hob.c       |  26 +-
 .../PeilessStartupLib/PeilessStartupLib.inf   |   1 +
 OvmfPkg/Library/PlatformInitLib/IntelTdx.c    | 154 ++++++++++-
 OvmfPkg/Library/PlatformInitLib/MemDetect.c   |  27 ++
 .../PlatformInitLib/PlatformInitLib.inf       |   1 +
 OvmfPkg/OvmfPkg.dec                           |   4 +
 OvmfPkg/OvmfPkgX64.dsc                        |   9 +
 OvmfPkg/PlatformPei/MemDetect.c               |   5 +
 OvmfPkg/TdxDxe/TdxDxe.c                       | 103 +++++++
 OvmfPkg/TdxDxe/TdxDxe.inf                     |   2 +
 .../UefiShellDebug1CommandsLib/MemMap.c       |  13 +
 .../UefiShellDebug1CommandsLib.uni            |   3 +-
 29 files changed, 741 insertions(+), 38 deletions(-)
 create mode 100644 MdeModulePkg/Include/Pi/PrePiDxeCis.h
 create mode 100644 MdeModulePkg/Include/Pi/PrePiHob.h
 create mode 100644 MdePkg/Include/Protocol/MemoryAccept.h

-- 
2.29.2.windows.2

[PATCH V2 01/14] MdeModulePkg: Add PrePiHob.h

[Min Xu]

From: Min M Xu <min.m.xu@intel.com>

RFC: https://bugzilla.tianocore.org/show_bug.cgi?id=3937

EFI_RESOURCE_MEMORY_UNACCEPTED is defined for unaccepted memory.
But this defitinion has not been officially in the PI spec. Base
on the code-first we define EFI_RESOURCE_MEMORY_UNACCEPTED at
MdeModulePkg/Include/Pi/PrePiHob.h.

Cc: Jian J Wang <jian.j.wang@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Erdem Aktas <erdemaktas@google.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Min Xu <min.m.xu@intel.com>
---
 MdeModulePkg/Include/Pi/PrePiHob.h | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)
 create mode 100644 MdeModulePkg/Include/Pi/PrePiHob.h

diff --git a/MdeModulePkg/Include/Pi/PrePiHob.h b/MdeModulePkg/Include/Pi/PrePiHob.h
new file mode 100644
index 000000000000..38a8f1be149d
--- /dev/null
+++ b/MdeModulePkg/Include/Pi/PrePiHob.h
@@ -0,0 +1,20 @@
+/** @file
+  HOB related definitions which has not been officially published in PI.
+
+Copyright (c) 2022, Intel Corporation. All rights reserved.<BR>
+SPDX-License-Identifier: BSD-2-Clause-Patent
+
+**/
+
+#ifndef MDE_MODULEPKG_PRE_PI_HOB_H_
+#define MDE_MODULEPKG_PRE_PI_HOB_H_
+
+//
+// EFI_RESOURCE_MEMORY_UNACCEPTED is defined for unaccepted memory.
+// But this defitinion has not been officially in the PI spec. Base
+// on the code-first we define EFI_RESOURCE_MEMORY_UNACCEPTED at
+// MdeModulePkg/Include/Pi/PrePiHob.h.
+//
+#define EFI_RESOURCE_MEMORY_UNACCEPTED  0x00000007
+
+#endif
-- 
2.29.2.windows.2

Re: [PATCH V2 01/14] MdeModulePkg: Add PrePiHob.h

[Gerd Hoffmann]

  Hi,

> EFI_RESOURCE_MEMORY_UNACCEPTED is defined for unaccepted memory.
> But this defitinion has not been officially in the PI spec.

I think this has just changed with uefi spec 2.10 released
in August.

take care,
  Gerd

Re: [PATCH V2 01/14] MdeModulePkg: Add PrePiHob.h

[Min Xu]

On September 1, 2022 11:48 PM, Gerd Hoffmann wrote:
> 
> > EFI_RESOURCE_MEMORY_UNACCEPTED is defined for unaccepted memory.
> > But this defitinion has not been officially in the PI spec.
> 
> I think this has just changed with uefi spec 2.10 released in August.
> 
Thanks for reminder. I will check the spec.

Thanks
Min

Re: [PATCH V2 01/14] MdeModulePkg: Add PrePiHob.h

[Min Xu]

On September 1, 2022 11:48 PM, Gerd Hoffmann wrote:
>   Hi,
> 
> > EFI_RESOURCE_MEMORY_UNACCEPTED is defined for unaccepted memory.
> > But this defitinion has not been officially in the PI spec.
> 
> I think this has just changed with uefi spec 2.10 released in August.
> 
I carefully checked the UEFI spec 2.10 but didn't find the unaccepted related changes in version 2.10. The latest one is in version 2.9. " 2134 Introduce unaccepted memory type "

Thanks
Min

Re: [PATCH V2 01/14] MdeModulePkg: Add PrePiHob.h

[Gerd Hoffmann]

On Sun, Sep 04, 2022 at 11:34:14AM +0000, Xu, Min M wrote:
> On September 1, 2022 11:48 PM, Gerd Hoffmann wrote:
> >   Hi,
> > 
> > > EFI_RESOURCE_MEMORY_UNACCEPTED is defined for unaccepted memory.
> > > But this defitinion has not been officially in the PI spec.
> > 
> > I think this has just changed with uefi spec 2.10 released in August.
> > 
> I carefully checked the UEFI spec 2.10 but didn't find the unaccepted related changes in version 2.10. The latest one is in version 2.9. " 2134 Introduce unaccepted memory type "

Hmm, seems PI spec has its own memory types instead if just using the
UEFI spec ones so unaccepted memory being in UEFI doesn't help here as
long as we don't have a new PI spec yet.  Is there any ETA for the next
PI spec update?

take care,
  Gerd

Re: [edk2-devel] [PATCH V2 01/14] MdeModulePkg: Add PrePiHob.h

[Min Xu]

On September 7, 2022 1:31 PM, Gerd Hoffmann wrote:
> On Sun, Sep 04, 2022 at 11:34:14AM +0000, Xu, Min M wrote:
> > On September 1, 2022 11:48 PM, Gerd Hoffmann wrote:
> > >   Hi,
> > >
> > > > EFI_RESOURCE_MEMORY_UNACCEPTED is defined for unaccepted
> memory.
> > > > But this defitinion has not been officially in the PI spec.
> > >
> > > I think this has just changed with uefi spec 2.10 released in August.
> > >
> > I carefully checked the UEFI spec 2.10 but didn't find the unaccepted
> related changes in version 2.10. The latest one is in version 2.9. " 2134
> Introduce unaccepted memory type "
> 
> Hmm, seems PI spec has its own memory types instead if just using the UEFI
> spec ones so unaccepted memory being in UEFI doesn't help here as long as
> we don't have a new PI spec yet.  Is there any ETA for the next PI spec update?
> 
I have no idea of the ETA.

Thanks
Min

[PATCH V2 02/14] MdePkg: Increase EFI_RESOURCE_MAX_MEMORY_TYPE

[Min Xu]

From: Min M Xu <min.m.xu@intel.com>

RFC: https://bugzilla.tianocore.org/show_bug.cgi?id=3937

EFI_RESOURCE_MEMORY_UNACCEPTED is defined for unaccepted memory.
But this defitinion has not been officially in the PI spec. Base
on the code-first we define EFI_RESOURCE_MEMORY_UNACCEPTED at
MdeModulePkg/Include/Pi/PrePiHob.h and update EFI_RESOURCE_MAX_MEMORY_TYPE
to 8. After EFI_RESOURCE_MEMORY_UNACCEPTED is officially published
in PI spec, we will re-visit here.

Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Zhiguang Liu <zhiguang.liu@intel.com>
Cc: Erdem Aktas <erdemaktas@google.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Reviewed-by: Jiewen Yao <jiewen.yao@intel.com>
Signed-off-by: Min Xu <min.m.xu@intel.com>
---
 MdePkg/Include/Pi/PiHob.h | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/MdePkg/Include/Pi/PiHob.h b/MdePkg/Include/Pi/PiHob.h
index e9f0ab4309d1..9af2e957fee5 100644
--- a/MdePkg/Include/Pi/PiHob.h
+++ b/MdePkg/Include/Pi/PiHob.h
@@ -232,7 +232,16 @@ typedef UINT32 EFI_RESOURCE_TYPE;
 #define EFI_RESOURCE_MEMORY_MAPPED_IO_PORT  0x00000004
 #define EFI_RESOURCE_MEMORY_RESERVED        0x00000005
 #define EFI_RESOURCE_IO_RESERVED            0x00000006
-#define EFI_RESOURCE_MAX_MEMORY_TYPE        0x00000007
+//
+// EFI_RESOURCE_MEMORY_UNACCEPTED is defined for unaccepted memory.
+// But this defitinion has not been officially in the PI spec. Base
+// on the code-first we define EFI_RESOURCE_MEMORY_UNACCEPTED at
+// MdeModulePkg/Include/Pi/PrePiHob.h and update EFI_RESOURCE_MAX_MEMORY_TYPE
+// to 8. After EFI_RESOURCE_MEMORY_UNACCEPTED is officially published
+// in PI spec, we will re-visit here.
+//
+// #define EFI_RESOURCE_MEMORY_UNACCEPTED      0x00000007
+#define EFI_RESOURCE_MAX_MEMORY_TYPE  0x00000008
 
 ///
 /// A type of recount attribute type.
-- 
2.29.2.windows.2

[PATCH V2 03/14] OvmfPkg: Use EFI_RESOURCE_MEMORY_UNACCEPTED which defined in MdeModulePkg

[Min Xu]

From: Min M Xu <min.m.xu@intel.com>

RFC: https://bugzilla.tianocore.org/show_bug.cgi?id=3937

EFI_RESOURCE_MEMORY_UNACCEPTED is defined in MdeModulePkg. The files
which use the definition are updated as well.

Cc: Erdem Aktas <erdemaktas@google.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Min Xu <min.m.xu@intel.com>
---
 OvmfPkg/Library/PeilessStartupLib/Hob.c    | 3 +--
 OvmfPkg/Library/PlatformInitLib/IntelTdx.c | 2 +-
 2 files changed, 2 insertions(+), 3 deletions(-)

diff --git a/OvmfPkg/Library/PeilessStartupLib/Hob.c b/OvmfPkg/Library/PeilessStartupLib/Hob.c
index 5fc84a809025..a9b92b5fbaba 100644
--- a/OvmfPkg/Library/PeilessStartupLib/Hob.c
+++ b/OvmfPkg/Library/PeilessStartupLib/Hob.c
@@ -20,10 +20,9 @@
 #include <IndustryStandard/UefiTcgPlatform.h>
 #include <Library/PlatformInitLib.h>
 #include <OvmfPlatforms.h>
+#include <Pi/PrePiHob.h>
 #include "PeilessStartupInternal.h"
 
-#define EFI_RESOURCE_MEMORY_UNACCEPTED  7
-
 /**
  * Construct the HobList in SEC phase.
  *
diff --git a/OvmfPkg/Library/PlatformInitLib/IntelTdx.c b/OvmfPkg/Library/PlatformInitLib/IntelTdx.c
index c6d7c8bb6e0e..396b14d919d2 100644
--- a/OvmfPkg/Library/PlatformInitLib/IntelTdx.c
+++ b/OvmfPkg/Library/PlatformInitLib/IntelTdx.c
@@ -20,11 +20,11 @@
 #include <Library/PeiServicesLib.h>
 #include <Library/TdxLib.h>
 #include <Library/SynchronizationLib.h>
+#include <Pi/PrePiHob.h>
 #include <WorkArea.h>
 #include <ConfidentialComputingGuestAttr.h>
 
 #define ALIGNED_2MB_MASK                0x1fffff
-#define EFI_RESOURCE_MEMORY_UNACCEPTED  7
 
 /**
   This function will be called to accept pages. Only BSP accepts pages.
-- 
2.29.2.windows.2

[PATCH V2 04/14] MdePkg: Add UEFI Unaccepted memory definition

[Min Xu]

From: Min M Xu <min.m.xu@intel.com>

RFC: https://bugzilla.tianocore.org/show_bug.cgi?id=3937

Plase refer to:
  UEFI Spec v2.9 Table 7-5 Memory Type Usage before ExitBootServices()

Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Zhiguang Liu <zhiguang.liu@intel.com>
Cc: Erdem Aktas <erdemaktas@google.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Reviewed-by: Jiewen Yao <jiewen.yao@intel.com>
Signed-off-by: Min Xu <min.m.xu@intel.com>
---
 MdeModulePkg/Include/Pi/PrePiDxeCis.h | 25 +++++++++++++++++++++++++
 MdePkg/Include/Pi/PiDxeCis.h          | 10 +++++++++-
 MdePkg/Include/Uefi/UefiMultiPhase.h  |  5 +++++
 3 files changed, 39 insertions(+), 1 deletion(-)
 create mode 100644 MdeModulePkg/Include/Pi/PrePiDxeCis.h

diff --git a/MdeModulePkg/Include/Pi/PrePiDxeCis.h b/MdeModulePkg/Include/Pi/PrePiDxeCis.h
new file mode 100644
index 000000000000..113ac37924cc
--- /dev/null
+++ b/MdeModulePkg/Include/Pi/PrePiDxeCis.h
@@ -0,0 +1,25 @@
+/** @file
+  Include file matches things in PI.
+
+Copyright (c) 2022, Intel Corporation. All rights reserved.<BR>
+SPDX-License-Identifier: BSD-2-Clause-Patent
+
+**/
+
+#ifndef MDE_MODULEPKG_PRE_PI_DXE_CIS_H_
+#define MDE_MODULEPKG_PRE_PI_DXE_CIS_H_
+
+///
+/// A memory region that describes system memory that has not been accepted
+/// by a corresponding call to the underlying isolation architecture.
+///
+/// This memory region has not been defined in PI spec, so it is defined in
+/// PrePiDxeCis.h. And it is defined in the format of captial letters
+/// because only capital letters are allowed to be used for #define declarations.
+///
+/// After this memory region is defined in PI spec, it should be a value in
+/// EFI_GCD_MEMORY_TYPE in PiDxeCis.h.
+///
+#define  EFI_GCD_MEMORY_TYPE_UNACCEPTED  7
+
+#endif
diff --git a/MdePkg/Include/Pi/PiDxeCis.h b/MdePkg/Include/Pi/PiDxeCis.h
index d0f2ed0e58df..27b219aa3ffa 100644
--- a/MdePkg/Include/Pi/PiDxeCis.h
+++ b/MdePkg/Include/Pi/PiDxeCis.h
@@ -56,7 +56,15 @@ typedef enum {
   /// system. If all memory has the same reliability, then this bit is not used.
   ///
   EfiGcdMemoryTypeMoreReliable,
-  EfiGcdMemoryTypeMaximum
+  // ///
+  // /// A memory region that describes system memory that has not been accepted
+  // /// by a corresponding call to the underlying isolation architecture.
+  // ///
+  // /// Please be noted:
+  // /// EfiGcdMemoryTypeUnaccepted is defined in PrePiDxeCis.h because it has not been
+  // /// defined in PI spec.
+  // EfiGcdMemoryTypeUnaccepted,
+  EfiGcdMemoryTypeMaximum = 8
 } EFI_GCD_MEMORY_TYPE;
 
 ///
diff --git a/MdePkg/Include/Uefi/UefiMultiPhase.h b/MdePkg/Include/Uefi/UefiMultiPhase.h
index 22bae43e36e8..7884913371a9 100644
--- a/MdePkg/Include/Uefi/UefiMultiPhase.h
+++ b/MdePkg/Include/Uefi/UefiMultiPhase.h
@@ -103,6 +103,11 @@ typedef enum {
   /// however it happens to also support byte-addressable non-volatility.
   ///
   EfiPersistentMemory,
+  ///
+  /// A memory region that describes system memory that has not been accepted
+  /// by a corresponding call to the underlying isolation architecture.
+  ///
+  EfiUnacceptedMemoryType,
   EfiMaxMemoryType
 } EFI_MEMORY_TYPE;
 
-- 
2.29.2.windows.2

[PATCH V2 05/14] MdeModulePkg: Update Dxe to handle unaccepted memory type

[Min Xu]

From: Min M Xu <min.m.xu@intel.com>

RFC: https://bugzilla.tianocore.org/show_bug.cgi?id=3937

Unaccepted memory is a kind of new memory type,
CoreInitializeGcdServices() and CoreGetMemoryMap() are updated to handle
the unaccepted memory type.

Ref: microsoft/mu_basecore@97e9c31

Cc: Jian J Wang <jian.j.wang@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Erdem Aktas <erdemaktas@google.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Min Xu <min.m.xu@intel.com>
---
 MdeModulePkg/Core/Dxe/Gcd/Gcd.c  |  6 +++
 MdeModulePkg/Core/Dxe/Mem/Page.c | 63 +++++++++++++++++++++++---------
 2 files changed, 52 insertions(+), 17 deletions(-)

diff --git a/MdeModulePkg/Core/Dxe/Gcd/Gcd.c b/MdeModulePkg/Core/Dxe/Gcd/Gcd.c
index 3763467bdbc0..9ba73d148af3 100644
--- a/MdeModulePkg/Core/Dxe/Gcd/Gcd.c
+++ b/MdeModulePkg/Core/Dxe/Gcd/Gcd.c
@@ -8,6 +8,8 @@ SPDX-License-Identifier: BSD-2-Clause-Patent
 
 **/
 
+#include <Pi/PrePiDxeCis.h>
+#include <Pi/PrePiHob.h>
 #include "DxeMain.h"
 #include "Gcd.h"
 #include "Mem/HeapGuard.h"
@@ -103,6 +105,7 @@ GLOBAL_REMOVE_IF_UNREFERENCED CONST CHAR8  *mGcdMemoryTypeNames[] = {
   "MMIO     ",  // EfiGcdMemoryTypeMemoryMappedIo
   "PersisMem",  // EfiGcdMemoryTypePersistent
   "MoreRelia",  // EfiGcdMemoryTypeMoreReliable
+  "Unaccepte",  // EFI_GCD_MEMORY_TYPE_UNACCEPTED
   "Unknown  "   // EfiGcdMemoryTypeMaximum
 };
 
@@ -2600,6 +2603,9 @@ CoreInitializeGcdServices (
         case EFI_RESOURCE_MEMORY_RESERVED:
           GcdMemoryType = EfiGcdMemoryTypeReserved;
           break;
+        case EFI_RESOURCE_MEMORY_UNACCEPTED:
+          GcdMemoryType = EFI_GCD_MEMORY_TYPE_UNACCEPTED;
+          break;
         case EFI_RESOURCE_IO:
           GcdIoType = EfiGcdIoTypeIo;
           break;
diff --git a/MdeModulePkg/Core/Dxe/Mem/Page.c b/MdeModulePkg/Core/Dxe/Mem/Page.c
index cc0b90ac0df5..160289c1f9ec 100644
--- a/MdeModulePkg/Core/Dxe/Mem/Page.c
+++ b/MdeModulePkg/Core/Dxe/Mem/Page.c
@@ -9,6 +9,7 @@ SPDX-License-Identifier: BSD-2-Clause-Patent
 #include "DxeMain.h"
 #include "Imem.h"
 #include "HeapGuard.h"
+#include <Pi/PrePiDxeCis.h>
 
 //
 // Entry for tracking the memory regions for each memory type to coalesce similar memory types
@@ -61,6 +62,7 @@ EFI_MEMORY_TYPE_STATISTICS  mMemoryTypeStatistics[EfiMaxMemoryType + 1] = {
   { 0, MAX_ALLOC_ADDRESS, 0, 0, EfiMaxMemoryType, FALSE, FALSE },  // EfiMemoryMappedIOPortSpace
   { 0, MAX_ALLOC_ADDRESS, 0, 0, EfiMaxMemoryType, TRUE,  TRUE  },  // EfiPalCode
   { 0, MAX_ALLOC_ADDRESS, 0, 0, EfiMaxMemoryType, FALSE, FALSE },  // EfiPersistentMemory
+  { 0, MAX_ALLOC_ADDRESS, 0, 0, EfiMaxMemoryType, TRUE,  FALSE },  // EfiUnacceptedMemoryType
   { 0, MAX_ALLOC_ADDRESS, 0, 0, EfiMaxMemoryType, FALSE, FALSE }   // EfiMaxMemoryType
 };
 
@@ -68,22 +70,23 @@ EFI_PHYSICAL_ADDRESS  mDefaultMaximumAddress = MAX_ALLOC_ADDRESS;
 EFI_PHYSICAL_ADDRESS  mDefaultBaseAddress    = MAX_ALLOC_ADDRESS;
 
 EFI_MEMORY_TYPE_INFORMATION  gMemoryTypeInformation[EfiMaxMemoryType + 1] = {
-  { EfiReservedMemoryType,      0 },
-  { EfiLoaderCode,              0 },
-  { EfiLoaderData,              0 },
-  { EfiBootServicesCode,        0 },
-  { EfiBootServicesData,        0 },
-  { EfiRuntimeServicesCode,     0 },
-  { EfiRuntimeServicesData,     0 },
-  { EfiConventionalMemory,      0 },
-  { EfiUnusableMemory,          0 },
-  { EfiACPIReclaimMemory,       0 },
-  { EfiACPIMemoryNVS,           0 },
-  { EfiMemoryMappedIO,          0 },
-  { EfiMemoryMappedIOPortSpace, 0 },
-  { EfiPalCode,                 0 },
-  { EfiPersistentMemory,        0 },
-  { EfiMaxMemoryType,           0 }
+  { EfiReservedMemoryType,          0 },
+  { EfiLoaderCode,                  0 },
+  { EfiLoaderData,                  0 },
+  { EfiBootServicesCode,            0 },
+  { EfiBootServicesData,            0 },
+  { EfiRuntimeServicesCode,         0 },
+  { EfiRuntimeServicesData,         0 },
+  { EfiConventionalMemory,          0 },
+  { EfiUnusableMemory,              0 },
+  { EfiACPIReclaimMemory,           0 },
+  { EfiACPIMemoryNVS,               0 },
+  { EfiMemoryMappedIO,              0 },
+  { EfiMemoryMappedIOPortSpace,     0 },
+  { EfiPalCode,                     0 },
+  { EfiPersistentMemory,            0 },
+  { EFI_GCD_MEMORY_TYPE_UNACCEPTED, 0 },
+  { EfiMaxMemoryType,               0 }
 };
 //
 // Only used when load module at fixed address feature is enabled. True means the memory is alreay successfully allocated
@@ -1286,7 +1289,7 @@ CoreInternalAllocatePages (
   }
 
   if (((MemoryType >= EfiMaxMemoryType) && (MemoryType < MEMORY_TYPE_OEM_RESERVED_MIN)) ||
-      (MemoryType == EfiConventionalMemory) || (MemoryType == EfiPersistentMemory))
+      (MemoryType == EfiConventionalMemory) || (MemoryType == EfiPersistentMemory) || (MemoryType == EfiUnacceptedMemoryType))
   {
     return EFI_INVALID_PARAMETER;
   }
@@ -1961,6 +1964,32 @@ CoreGetMemoryMap (
       MemoryMap = MergeMemoryMapDescriptor (MemoryMapStart, MemoryMap, Size);
     }
 
+    if (MergeGcdMapEntry.GcdMemoryType == EFI_GCD_MEMORY_TYPE_UNACCEPTED) {
+      //
+      // Page Align GCD range is required. When it is converted to EFI_MEMORY_DESCRIPTOR,
+      // it will be recorded as page PhysicalStart and NumberOfPages.
+      //
+      ASSERT ((MergeGcdMapEntry.BaseAddress & EFI_PAGE_MASK) == 0);
+      ASSERT (((MergeGcdMapEntry.EndAddress - MergeGcdMapEntry.BaseAddress + 1) & EFI_PAGE_MASK) == 0);
+
+      //
+      // Create EFI_MEMORY_DESCRIPTOR for every Unaccepted GCD entries
+      //
+      MemoryMap->PhysicalStart = MergeGcdMapEntry.BaseAddress;
+      MemoryMap->VirtualStart  = 0;
+      MemoryMap->NumberOfPages = RShiftU64 ((MergeGcdMapEntry.EndAddress - MergeGcdMapEntry.BaseAddress + 1), EFI_PAGE_SHIFT);
+      MemoryMap->Attribute     = MergeGcdMapEntry.Attributes |
+                                 (MergeGcdMapEntry.Capabilities & (EFI_MEMORY_RP | EFI_MEMORY_WP | EFI_MEMORY_XP | EFI_MEMORY_RO |
+                                                                   EFI_MEMORY_UC | EFI_MEMORY_UCE | EFI_MEMORY_WC | EFI_MEMORY_WT | EFI_MEMORY_WB));
+      MemoryMap->Type = EfiUnacceptedMemoryType;
+
+      //
+      // Check to see if the new Memory Map Descriptor can be merged with an
+      // existing descriptor if they are adjacent and have the same attributes
+      //
+      MemoryMap = MergeMemoryMapDescriptor (MemoryMapStart, MemoryMap, Size);
+    }
+
     if (Link == &mGcdMemorySpaceMap) {
       //
       // break loop when arrive at head.
-- 
2.29.2.windows.2

[PATCH V2 06/14] ShellPkg: Update shell command memmap to show unaccepted memory

[Min Xu]

From: Min M Xu <min.m.xu@intel.com>

RFC: https://bugzilla.tianocore.org/show_bug.cgi?id=3937

ShellCommandRunMemMap() is updated to handle the case of unaccepted
memory type.

Cc: Ray Ni <ray.ni@intel.com>
Cc: Zhichao Gao <zhichao.gao@intel.com>
Cc: Erdem Aktas <erdemaktas@google.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Min Xu <min.m.xu@intel.com>
---
 .../Library/UefiShellDebug1CommandsLib/MemMap.c     | 13 +++++++++++++
 .../UefiShellDebug1CommandsLib.uni                  |  3 ++-
 2 files changed, 15 insertions(+), 1 deletion(-)

diff --git a/ShellPkg/Library/UefiShellDebug1CommandsLib/MemMap.c b/ShellPkg/Library/UefiShellDebug1CommandsLib/MemMap.c
index 72f3c58b0970..a089953b286f 100644
--- a/ShellPkg/Library/UefiShellDebug1CommandsLib/MemMap.c
+++ b/ShellPkg/Library/UefiShellDebug1CommandsLib/MemMap.c
@@ -26,6 +26,7 @@ STATIC CONST CHAR16  NameEfiACPIMemoryNVS[]           = L"ACPIMemoryNVS";
 STATIC CONST CHAR16  NameEfiMemoryMappedIO[]          = L"MemoryMappedIO";
 STATIC CONST CHAR16  NameEfiMemoryMappedIOPortSpace[] = L"MemoryMappedIOPortSpace";
 STATIC CONST CHAR16  NameEfiPalCode[]                 = L"PalCode";
+STATIC CONST CHAR16  NameEfiUnacceptedMemoryType[]    = L"Unaccepted";
 
 //
 // Need short names for some memory types
@@ -151,6 +152,8 @@ ShellCommandRunMemMap (
   UINT64                 UnusableMemoryPagesSize;
   UINT64                 PalCodePages;
   UINT64                 PalCodePagesSize;
+  UINT64                 UnacceptedPages;
+  UINT64                 UnacceptedPagesSize;
   UINT64                 PersistentPages;
   UINT64                 PersistentPagesSize;
   BOOLEAN                Sfo;
@@ -175,6 +178,7 @@ ShellCommandRunMemMap (
   PalCodePages        = 0;
   PersistentPages     = 0;
   Size                = 0;
+  UnacceptedPages     = 0;
   Descriptors         = NULL;
   ShellStatus         = SHELL_SUCCESS;
   Status              = EFI_SUCCESS;
@@ -303,6 +307,11 @@ ShellCommandRunMemMap (
               TotalPages   += Walker->NumberOfPages;
               PalCodePages += Walker->NumberOfPages;
               break;
+            case EfiUnacceptedMemoryType:
+              ShellPrintHiiEx (-1, -1, NULL, (EFI_STRING_ID)(!Sfo ? STRING_TOKEN (STR_MEMMAP_LIST_ITEM) : STRING_TOKEN (STR_MEMMAP_LIST_ITEM_SFO)), gShellDebug1HiiHandle, NameEfiUnacceptedMemoryType, Walker->PhysicalStart, Walker->PhysicalStart+MultU64x64 (SIZE_4KB, Walker->NumberOfPages)-1, Walker->NumberOfPages, Walker->Attribute);
+              TotalPages      += Walker->NumberOfPages;
+              UnacceptedPages += Walker->NumberOfPages;
+              break;
             default:
               //
               // Shell Spec defines the SFO format.
@@ -335,6 +344,7 @@ ShellCommandRunMemMap (
         MmioSpacePagesSize      = MultU64x64 (SIZE_4KB, MmioSpacePages);
         MmioPortPagesSize       = MultU64x64 (SIZE_4KB, MmioPortPages);
         PalCodePagesSize        = MultU64x64 (SIZE_4KB, PalCodePages);
+        UnacceptedPagesSize     = MultU64x64 (SIZE_4KB, UnacceptedPages);
         PersistentPagesSize     = MultU64x64 (SIZE_4KB, PersistentPages);
         UnusableMemoryPagesSize = MultU64x64 (SIZE_4KB, UnusableMemoryPages);
         if (!Sfo) {
@@ -368,6 +378,8 @@ ShellCommandRunMemMap (
             MmioPortPagesSize,
             PalCodePages,
             PalCodePagesSize,
+            UnacceptedPages,
+            UnacceptedPagesSize,
             AvailPages,
             AvailPagesSize,
             PersistentPages,
@@ -422,6 +434,7 @@ ShellCommandRunMemMap (
             AcpiReclaimPagesSize,
             AcpiNvsPagesSize,
             PalCodePagesSize,
+            UnacceptedPagesSize,
             PersistentPagesSize
             );
         }
diff --git a/ShellPkg/Library/UefiShellDebug1CommandsLib/UefiShellDebug1CommandsLib.uni b/ShellPkg/Library/UefiShellDebug1CommandsLib/UefiShellDebug1CommandsLib.uni
index 6693be26e699..b1d239ed37ea 100644
--- a/ShellPkg/Library/UefiShellDebug1CommandsLib/UefiShellDebug1CommandsLib.uni
+++ b/ShellPkg/Library/UefiShellDebug1CommandsLib/UefiShellDebug1CommandsLib.uni
@@ -88,13 +88,14 @@
                                                   "  MMIO      : %,14ld Pages (%,ld Bytes)\r\n"
                                                   "  MMIO_Port : %,14ld Pages (%,ld Bytes)\r\n"
                                                   "  PalCode   : %,14ld Pages (%,ld Bytes)\r\n"
+                                                  "  Unaccepted: %,14ld Pages (%,ld Bytes)\r\n"
                                                   "  Available : %,14ld Pages (%,ld Bytes)\r\n"
                                                   "  Persistent: %,14ld Pages (%,ld Bytes)\r\n"
 #string STR_MEMMAP_LIST_SUMM_OTHER #language en-US    "  %08x  : %,14ld Pages (%,ld Bytes)\r\n"
 #string STR_MEMMAP_LIST_SUMM2     #language en-US "              -------------- \r\n"
                                                   "Total Memory: %,14ld MB (%,ld Bytes)\r\n"
 #string STR_MEMMAP_LIST_ITEM_SFO  #language en-US "MemoryMap,"%s","%LX","%LX","%LX","%LX"\r\n"
-#string STR_MEMMAP_LIST_SUMM_SFO  #language en-US "MemoryMapSummary,"%Ld","%Ld","%Ld","%Ld","%Ld","%Ld","%Ld","%Ld","%Ld","%Ld","%Ld","%Ld","%Ld","%Ld","%Ld","%Ld"\r\n"
+#string STR_MEMMAP_LIST_SUMM_SFO  #language en-US "MemoryMapSummary,"%Ld","%Ld","%Ld","%Ld","%Ld","%Ld","%Ld","%Ld","%Ld","%Ld","%Ld","%Ld","%Ld","%Ld","%Ld","%Ld", "%Ld"\r\n"
 
 #string STR_EFI_COMPRESS_FAIL     #language en-US "Unable to compress: %r.\r\n"
 #string STR_EFI_DECOMPRESS_FAIL   #language en-US "Unable to decompress: %r.\r\n"
-- 
2.29.2.windows.2

Re: [PATCH V2 06/14] ShellPkg: Update shell command memmap to show unaccepted memory

[Gerd Hoffmann]

On Sat, Aug 27, 2022 at 02:21:13PM +0800, Min Xu wrote:
> From: Min M Xu <min.m.xu@intel.com>
> 
> RFC: https://bugzilla.tianocore.org/show_bug.cgi?id=3937
> 
> ShellCommandRunMemMap() is updated to handle the case of unaccepted
> memory type.

Acked-by: Gerd Hoffmann <kraxel@redhat.com>

[PATCH V2 07/14] OvmfPkg: Add PCD and DEFINEs for Lazy Accept page.

[Min Xu]

From: Min M Xu <min.m.xu@intel.com>

RFC: https://bugzilla.tianocore.org/show_bug.cgi?id=3937

Lazy accept page can be controlled in build time like below:
  -D LAZY_ACCEPT_PARTIAL_MEM=512

The unit is MB. If it is 0 then it means Lazy-accept is turned off.

Lazy-accept is turned off by default in OvmfPkgX64.
Lazy-accept is turned on with 512MB by default in IntelTdxX64.

Cc: Erdem Aktas <erdemaktas@google.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Min Xu <min.m.xu@intel.com>
---
 OvmfPkg/IntelTdx/IntelTdxX64.dsc | 8 ++++++++
 OvmfPkg/OvmfPkg.dec              | 4 ++++
 OvmfPkg/OvmfPkgX64.dsc           | 9 +++++++++
 3 files changed, 21 insertions(+)

diff --git a/OvmfPkg/IntelTdx/IntelTdxX64.dsc b/OvmfPkg/IntelTdx/IntelTdxX64.dsc
index 71b1cf8e7090..7ab2dc6fffe4 100644
--- a/OvmfPkg/IntelTdx/IntelTdxX64.dsc
+++ b/OvmfPkg/IntelTdx/IntelTdxX64.dsc
@@ -62,6 +62,11 @@
   #
   DEFINE UP_CPU_DXE_GUID  = 6490f1c5-ebcc-4665-8892-0075b9bb49b7
 
+  #
+  # Define the size of lazy accepted memory. The unit is MB.
+  #
+  DEFINE LAZY_ACCEPT_PARTIAL_MEM = 512
+
 [BuildOptions]
   GCC:RELEASE_*_*_CC_FLAGS             = -DMDEPKG_NDEBUG
   INTEL:RELEASE_*_*_CC_FLAGS           = /D MDEPKG_NDEBUG
@@ -453,6 +458,9 @@
   # Point to the MdeModulePkg/Application/UiApp/UiApp.inf
   gEfiMdeModulePkgTokenSpaceGuid.PcdBootManagerMenuFile|{ 0x21, 0xaa, 0x2c, 0x46, 0x14, 0x76, 0x03, 0x45, 0x83, 0x6e, 0x8a, 0xb6, 0xf4, 0x66, 0x23, 0x31 }
 
+  # The partial memory size in Lazy accept
+  gUefiOvmfPkgTokenSpaceGuid.PcdLazyAcceptPartialMemorySize|$(LAZY_ACCEPT_PARTIAL_MEM)
+
 ################################################################################
 #
 # Pcd Dynamic Section - list of all EDK II PCD Entries defined by this Platform
diff --git a/OvmfPkg/OvmfPkg.dec b/OvmfPkg/OvmfPkg.dec
index 5af76a540529..49fc20831ad0 100644
--- a/OvmfPkg/OvmfPkg.dec
+++ b/OvmfPkg/OvmfPkg.dec
@@ -399,6 +399,10 @@
   ## The Tdx accept page size. 0x1000(4k),0x200000(2M)
   gUefiOvmfPkgTokenSpaceGuid.PcdTdxAcceptPageSize|0x200000|UINT32|0x65
 
+  ## The partial memory size in Lazy accept. Its unit is MB.
+  ## The default value is 0 which means lazy accept is turned off.
+  gUefiOvmfPkgTokenSpaceGuid.PcdLazyAcceptPartialMemorySize|0|UINT64|0x68
+
 [PcdsDynamic, PcdsDynamicEx]
   gUefiOvmfPkgTokenSpaceGuid.PcdEmuVariableEvent|0|UINT64|2
   gUefiOvmfPkgTokenSpaceGuid.PcdOvmfFlashVariablesEnable|FALSE|BOOLEAN|0x10
diff --git a/OvmfPkg/OvmfPkgX64.dsc b/OvmfPkg/OvmfPkgX64.dsc
index 6e68f60dc90f..026251ae7e69 100644
--- a/OvmfPkg/OvmfPkgX64.dsc
+++ b/OvmfPkg/OvmfPkgX64.dsc
@@ -78,6 +78,12 @@
   DEFINE UP_CPU_PEI_GUID  = 280251c4-1d09-4035-9062-839acb5f18c1
   DEFINE UP_CPU_DXE_GUID  = 6490f1c5-ebcc-4665-8892-0075b9bb49b7
 
+  #
+  # Define the size of lazy accepted memory. The unit is MB.
+  # In OvmfPkgX64, the lazy accept page is disabled by default.
+  #
+  DEFINE LAZY_ACCEPT_PARTIAL_MEM = 0
+
 [BuildOptions]
   GCC:RELEASE_*_*_CC_FLAGS             = -DMDEPKG_NDEBUG
   INTEL:RELEASE_*_*_CC_FLAGS           = /D MDEPKG_NDEBUG
@@ -600,6 +606,9 @@
   # Point to the MdeModulePkg/Application/UiApp/UiApp.inf
   gEfiMdeModulePkgTokenSpaceGuid.PcdBootManagerMenuFile|{ 0x21, 0xaa, 0x2c, 0x46, 0x14, 0x76, 0x03, 0x45, 0x83, 0x6e, 0x8a, 0xb6, 0xf4, 0x66, 0x23, 0x31 }
 
+  # The partial memory size in Lazy accept
+  gUefiOvmfPkgTokenSpaceGuid.PcdLazyAcceptPartialMemorySize|$(LAZY_ACCEPT_PARTIAL_MEM)
+
 ################################################################################
 #
 # Pcd Dynamic Section - list of all EDK II PCD Entries defined by this Platform
-- 
2.29.2.windows.2

Re: [PATCH V2 07/14] OvmfPkg: Add PCD and DEFINEs for Lazy Accept page.

[Gerd Hoffmann]

> Lazy accept page can be controlled in build time like below:
>   -D LAZY_ACCEPT_PARTIAL_MEM=512
> 
> The unit is MB. If it is 0 then it means Lazy-accept is turned off.
> 
> Lazy-accept is turned off by default in OvmfPkgX64.
> Lazy-accept is turned on with 512MB by default in IntelTdxX64.

> +  gUefiOvmfPkgTokenSpaceGuid.PcdLazyAcceptPartialMemorySize|$(LAZY_ACCEPT_PARTIAL_MEM)

Hmm, cover letter says 0 means accept all memory below 4G

Care to clarify?  Is the commit message outdated?

Also I think the discussion of the last revision ended with the
agreement that accepting all memory below 4G is a good idea because it
simplifies the workflow (no need to split low memory into accepted and
unaccepted ranges for example).

So why we are again talking about accepting 512 MB ?

take care,
  Gerd

Re: [edk2-devel] [PATCH V2 07/14] OvmfPkg: Add PCD and DEFINEs for Lazy Accept page.

[Min Xu]

On September 1, 2022 11:56 PM, Gerd Hoffmann wrote:
> 
> > Lazy accept page can be controlled in build time like below:
> >   -D LAZY_ACCEPT_PARTIAL_MEM=512
> >
> > The unit is MB. If it is 0 then it means Lazy-accept is turned off.
> >
> > Lazy-accept is turned off by default in OvmfPkgX64.
> > Lazy-accept is turned on with 512MB by default in IntelTdxX64.
> 
> > +
> > +
> gUefiOvmfPkgTokenSpaceGuid.PcdLazyAcceptPartialMemorySize|$(LAZY_AC
> C
> > + EPT_PARTIAL_MEM)
> 
> Hmm, cover letter says 0 means accept all memory below 4G
> 
> Care to clarify?  Is the commit message outdated?
> 
> Also I think the discussion of the last revision ended with the agreement that
> accepting all memory below 4G is a good idea because it simplifies the
> workflow (no need to split low memory into accepted and unaccepted ranges
> for example).
> 
> So why we are again talking about accepting 512 MB ?
> 
I may mis-understand the discussion of the last revision.
So the agreement in first stage is that:
1. Accept all memory below 4G.
2. The PcdLazyAcceptPartialMemorySize is not needed in current stage.
3. The patch of "Pool and page functions accept memory when OOM occurs" is not needed. Because OOM will not occur in the situation of accepting all memory below 4G.

If that is correct, I will update the patches based on above agreements.

Thanks
Min

Re: [edk2-devel] [PATCH V2 07/14] OvmfPkg: Add PCD and DEFINEs for Lazy Accept page.

[Gerd Hoffmann]

  Hi,

> So the agreement in first stage is that:
> 1. Accept all memory below 4G.

Yes.

> 2. The PcdLazyAcceptPartialMemorySize is not needed in current stage.

Question is whenever we ever need PcdLazyAcceptPartialMemorySize.  When
accepting all memory below 4G works fast enough (which I think is the
case when using 2M large pages) I don't see a reason to ever change
that.

> 3. The patch of "Pool and page functions accept memory when OOM
> occurs" is not needed. Because OOM will not occur in the situation of
> accepting all memory below 4G.

Probably not in the vast majority of use cases, but there are
exceptions.  When unlocking luks-encrypted partitions the grub boot
loader might need alot of memory for decryption (depends on encryption
parameters).  So being able to accept memory above 4G if needed would be
good.  Skipping that for the first stage and do it later is fine I
think.

take care,
  Gerd

[PATCH V2 08/14] OvmfPkg: Add MaxAcceptedMemoryAddress in TDX work area

[Min Xu]

From: Min M Xu <min.m.xu@intel.com>

RFC: https://bugzilla.tianocore.org/show_bug.cgi?id=3937

In lazy-accept the MaxAcceptedMemoryAddress is needed in
TransferTdxHobList and ContructFwHobList(at PeilessStartupLib). But in
SEC phase we cannot use a global variable to pass this value. So we add
a new field (MaxAcceptedMemoryAddress) in Tdx work area.

Cc: Erdem Aktas <erdemaktas@google.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Min Xu <min.m.xu@intel.com>
---
 OvmfPkg/Include/WorkArea.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/OvmfPkg/Include/WorkArea.h b/OvmfPkg/Include/WorkArea.h
index bf56fc4a6f65..2c2a5816b0dc 100644
--- a/OvmfPkg/Include/WorkArea.h
+++ b/OvmfPkg/Include/WorkArea.h
@@ -71,6 +71,7 @@ typedef struct _SEC_TDX_WORK_AREA {
   UINT32    PageTableReady;
   UINT32    Gpaw;
   UINT64    HobList;
+  UINT64    MaxAcceptedMemoryAddress;
 } SEC_TDX_WORK_AREA;
 
 typedef struct _TDX_WORK_AREA {
-- 
2.29.2.windows.2

[PATCH V2 09/14] OvmfPkg: Introduce lazy accept in PlatformInitLib and PlatformPei

[Min Xu]

From: Min M Xu <min.m.xu@intel.com>

RFC: https://bugzilla.tianocore.org/show_bug.cgi?id=3937

There are below major changes in PlatformInitLib/PlatformPei
1. ProcessHobList
  The unaccepted memory is accepted if the accumulated accepted memory is
  smaller than the LazyAcceptMemSize. If a EFI_RESOURCE_MEMORY_UNACCEPTED
  hob is cross the LazyAcceptMemSize, it will be split into 2 parts and
  only the left one is accepted. The max accepted memory address is
  stored in Tdx workarea which will be used in TransferTdxHobList.

  Please be noted: in current stage, we only accept the memory under 4G.

2. TransferTdxHobList
  Transfer the unaccepted memory hob to EFI_RESOURCE_SYSTEM_MEMORY hob
  if it is accepted. As it is mentioned in 1), there may be a
  EFI_RESOURCE_MEMORY_UNACCEPTED hob which only part of the memory
  describes in the hob is accepted. We also handles this situation
  in TransferTdxHobList.

3. PlatformAdjustSystemMemorySizeBelow4gbForLazyAccep
  The system memory size below 4GB may be larger than the accepted
  memory. This function is used to handle this situation.

Cc: Erdem Aktas <erdemaktas@google.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Min Xu <min.m.xu@intel.com>
---
 OvmfPkg/Include/Library/PlatformInitLib.h     |   6 +
 OvmfPkg/Library/PlatformInitLib/IntelTdx.c    | 152 ++++++++++++++++--
 OvmfPkg/Library/PlatformInitLib/MemDetect.c   |  27 ++++
 .../PlatformInitLib/PlatformInitLib.inf       |   1 +
 OvmfPkg/PlatformPei/MemDetect.c               |   5 +
 5 files changed, 180 insertions(+), 11 deletions(-)

diff --git a/OvmfPkg/Include/Library/PlatformInitLib.h b/OvmfPkg/Include/Library/PlatformInitLib.h
index 2987a367cc9c..187efcf34e14 100644
--- a/OvmfPkg/Include/Library/PlatformInitLib.h
+++ b/OvmfPkg/Include/Library/PlatformInitLib.h
@@ -144,6 +144,12 @@ PlatformGetSystemMemorySizeBelow4gb (
   IN EFI_HOB_PLATFORM_INFO  *PlatformInfoHob
   );
 
+UINT32
+EFIAPI
+PlatformAdjustSystemMemorySizeBelow4gbForLazyAccept (
+  IN UINT32  LowerMemorySize
+  );
+
 /**
   Initialize the PhysMemAddressWidth field in PlatformInfoHob based on guest RAM size.
 **/
diff --git a/OvmfPkg/Library/PlatformInitLib/IntelTdx.c b/OvmfPkg/Library/PlatformInitLib/IntelTdx.c
index 396b14d919d2..5c408758756e 100644
--- a/OvmfPkg/Library/PlatformInitLib/IntelTdx.c
+++ b/OvmfPkg/Library/PlatformInitLib/IntelTdx.c
@@ -7,6 +7,7 @@
 
 **/
 
+#include <Base.h>
 #include <PiPei.h>
 #include <Library/BaseLib.h>
 #include <Library/DebugLib.h>
@@ -24,7 +25,8 @@
 #include <WorkArea.h>
 #include <ConfidentialComputingGuestAttr.h>
 
-#define ALIGNED_2MB_MASK                0x1fffff
+#define ALIGNED_2MB_MASK  0x1fffff
+#define MEGABYTE_SHIFT    20
 
 /**
   This function will be called to accept pages. Only BSP accepts pages.
@@ -375,15 +377,33 @@ ProcessHobList (
   EFI_STATUS            Status;
   EFI_PEI_HOB_POINTERS  Hob;
   EFI_PHYSICAL_ADDRESS  PhysicalEnd;
+  TDX_WORK_AREA         *WorkArea;
+  UINT64                ResourceLength;
+  UINT64                AccumulateAcceptedMemory;
+  UINT64                LazyAcceptMemSize;
+  UINT64                MaxAcceptedMemoryAddress;
 
   Status = EFI_SUCCESS;
   ASSERT (VmmHobList != NULL);
   Hob.Raw = (UINT8 *)VmmHobList;
 
+  AccumulateAcceptedMemory = 0;
+  MaxAcceptedMemoryAddress = 0;
+  LazyAcceptMemSize        = FixedPcdGet64 (PcdLazyAcceptPartialMemorySize);
+  //
+  // If specified accept size is zero, accept all of the memory.
+  // Else transfer the size in megabyte to the number in byte.
+  //
+  if (LazyAcceptMemSize == 0) {
+    LazyAcceptMemSize = MAX_UINT64;
+  } else {
+    LazyAcceptMemSize <<= MEGABYTE_SHIFT;
+  }
+
   //
   // Parse the HOB list until end of list or matching type is found.
   //
-  while (!END_OF_HOB_LIST (Hob)) {
+  while (!END_OF_HOB_LIST (Hob) && AccumulateAcceptedMemory < LazyAcceptMemSize) {
     if (Hob.Header->HobType == EFI_HOB_TYPE_RESOURCE_DESCRIPTOR) {
       DEBUG ((DEBUG_INFO, "\nResourceType: 0x%x\n", Hob.ResourceDescriptor->ResourceType));
 
@@ -393,7 +413,26 @@ ProcessHobList (
         DEBUG ((DEBUG_INFO, "ResourceLength: 0x%llx\n", Hob.ResourceDescriptor->ResourceLength));
         DEBUG ((DEBUG_INFO, "Owner: %g\n\n", &Hob.ResourceDescriptor->Owner));
 
-        PhysicalEnd = Hob.ResourceDescriptor->PhysicalStart + Hob.ResourceDescriptor->ResourceLength;
+        PhysicalEnd    = Hob.ResourceDescriptor->PhysicalStart + Hob.ResourceDescriptor->ResourceLength;
+        ResourceLength = Hob.ResourceDescriptor->ResourceLength;
+
+        if (AccumulateAcceptedMemory + ResourceLength > LazyAcceptMemSize) {
+          //
+          // If the memory can't be accepted completely, accept the part of it to meet the
+          // PcdLazyAcceptPartialMemorySize.
+          //
+          ResourceLength = LazyAcceptMemSize - AccumulateAcceptedMemory;
+          PhysicalEnd    = Hob.ResourceDescriptor->PhysicalStart + ResourceLength;
+        }
+
+        if (PhysicalEnd > SIZE_4GB) {
+          //
+          // In current stage, we only accept the memory under 4G
+          //
+          ResourceLength    -= (PhysicalEnd - SIZE_4GB);
+          LazyAcceptMemSize -= (PhysicalEnd - SIZE_4GB);
+          PhysicalEnd        = SIZE_4GB;
+        }
 
         Status = BspAcceptMemoryResourceRange (
                    Hob.ResourceDescriptor->PhysicalStart,
@@ -402,12 +441,25 @@ ProcessHobList (
         if (EFI_ERROR (Status)) {
           break;
         }
+
+        AccumulateAcceptedMemory += ResourceLength;
+        MaxAcceptedMemoryAddress  = PhysicalEnd;
       }
     }
 
     Hob.Raw = GET_NEXT_HOB (Hob);
   }
 
+  //
+  // Record MaxAcceptedMemoryAddress in OvmfWorkArea.
+  // This information is useful later but in SEC phase we cannot use a global
+  // variable to pass this value. So it is stored in OvmfWorkarea.
+  //
+  WorkArea = (TDX_WORK_AREA *)FixedPcdGet32 (PcdOvmfWorkAreaBase);
+  ASSERT (WorkArea != NULL);
+  ASSERT (WorkArea->Header.GuestType == CcGuestTypeIntelTdx);
+  WorkArea->SecTdxWorkArea.MaxAcceptedMemoryAddress = MaxAcceptedMemoryAddress;
+
   return Status;
 }
 
@@ -460,6 +512,74 @@ ProcessTdxHobList (
   return Status;
 }
 
+/**
+ * Build ResourceDescriptorHob for the unaccepted memory region.
+ * This memory region may be splitted into 2 parts because of lazy accept.
+ *
+ * @param Hob     Point to the EFI_HOB_RESOURCE_DESCRIPTOR
+ * @param MaxAcceptedMemoryAddress The max accepted memory address
+ * @return VOID
+ */
+VOID
+BuildResourceDescriptorHobForUnacceptedMemory (
+  IN EFI_HOB_RESOURCE_DESCRIPTOR  *Hob,
+  IN UINT64                       MaxAcceptedMemoryAddress
+  )
+{
+  EFI_PHYSICAL_ADDRESS         PhysicalStart;
+  EFI_PHYSICAL_ADDRESS         PhysicalEnd;
+  UINT64                       ResourceLength;
+  EFI_RESOURCE_TYPE            ResourceType;
+  EFI_RESOURCE_ATTRIBUTE_TYPE  ResourceAttribute;
+  UINT64                       AcceptedResourceLength;
+
+  ASSERT (Hob->ResourceType == EFI_RESOURCE_MEMORY_UNACCEPTED);
+
+  ResourceType      = EFI_RESOURCE_MEMORY_UNACCEPTED;
+  ResourceAttribute = Hob->ResourceAttribute;
+  PhysicalStart     = Hob->PhysicalStart;
+  ResourceLength    = Hob->ResourceLength;
+  PhysicalEnd       = PhysicalStart + ResourceLength;
+
+  if (PhysicalEnd <= MaxAcceptedMemoryAddress) {
+    //
+    // This memory region has been accepted.
+    //
+    ResourceType       = EFI_RESOURCE_SYSTEM_MEMORY;
+    ResourceAttribute |= (EFI_RESOURCE_ATTRIBUTE_PRESENT | EFI_RESOURCE_ATTRIBUTE_INITIALIZED | EFI_RESOURCE_ATTRIBUTE_TESTED);
+  } else if (PhysicalStart >= MaxAcceptedMemoryAddress) {
+    //
+    // This memory region hasn't been accepted.
+    // So keep the ResourceType and ResourceAttribute unchange.
+    //
+  } else {
+    //
+    // This memory region is splitted into 2 parts:
+    // the accepted and unaccepted.
+    //
+    AcceptedResourceLength = MaxAcceptedMemoryAddress - Hob->PhysicalStart;
+
+    // We build the ResourceDescriptorHob for the accepted part.
+    // The unaccepted part will be build out side the if-else block.
+    BuildResourceDescriptorHob (
+      EFI_RESOURCE_SYSTEM_MEMORY,
+      ResourceAttribute | (EFI_RESOURCE_ATTRIBUTE_PRESENT | EFI_RESOURCE_ATTRIBUTE_INITIALIZED | EFI_RESOURCE_ATTRIBUTE_TESTED),
+      Hob->PhysicalStart,
+      AcceptedResourceLength
+      );
+
+    PhysicalStart   = Hob->PhysicalStart + AcceptedResourceLength;
+    ResourceLength -= AcceptedResourceLength;
+  }
+
+  BuildResourceDescriptorHob (
+    ResourceType,
+    ResourceAttribute,
+    PhysicalStart,
+    ResourceLength
+    );
+}
+
 /**
   Transfer the incoming HobList for the TD to the final HobList for Dxe.
   The Hobs transferred in this function are ResourceDescriptor hob and
@@ -477,6 +597,16 @@ TransferTdxHobList (
   EFI_PEI_HOB_POINTERS         Hob;
   EFI_RESOURCE_TYPE            ResourceType;
   EFI_RESOURCE_ATTRIBUTE_TYPE  ResourceAttribute;
+  UINT64                       MaxAcceptedMemoryAddress;
+  TDX_WORK_AREA                *WorkArea;
+
+  WorkArea = (TDX_WORK_AREA *)FixedPcdGet32 (PcdOvmfWorkAreaBase);
+  ASSERT (WorkArea != NULL);
+  ASSERT (WorkArea->Header.GuestType == CcGuestTypeIntelTdx);
+  MaxAcceptedMemoryAddress = WorkArea->SecTdxWorkArea.MaxAcceptedMemoryAddress;
+  if (MaxAcceptedMemoryAddress == 0) {
+    MaxAcceptedMemoryAddress = MAX_UINT64;
+  }
 
   //
   // PcdOvmfSecGhcbBase is used as the TD_HOB in Tdx guest.
@@ -489,16 +619,16 @@ TransferTdxHobList (
         ResourceAttribute = Hob.ResourceDescriptor->ResourceAttribute;
 
         if (ResourceType == EFI_RESOURCE_MEMORY_UNACCEPTED) {
-          ResourceType       = EFI_RESOURCE_SYSTEM_MEMORY;
-          ResourceAttribute |= (EFI_RESOURCE_ATTRIBUTE_PRESENT | EFI_RESOURCE_ATTRIBUTE_INITIALIZED | EFI_RESOURCE_ATTRIBUTE_TESTED);
+          BuildResourceDescriptorHobForUnacceptedMemory (Hob.ResourceDescriptor, MaxAcceptedMemoryAddress);
+        } else {
+          BuildResourceDescriptorHob (
+            ResourceType,
+            ResourceAttribute,
+            Hob.ResourceDescriptor->PhysicalStart,
+            Hob.ResourceDescriptor->ResourceLength
+            );
         }
 
-        BuildResourceDescriptorHob (
-          ResourceType,
-          ResourceAttribute,
-          Hob.ResourceDescriptor->PhysicalStart,
-          Hob.ResourceDescriptor->ResourceLength
-          );
         break;
       case EFI_HOB_TYPE_MEMORY_ALLOCATION:
         BuildMemoryAllocationHob (
diff --git a/OvmfPkg/Library/PlatformInitLib/MemDetect.c b/OvmfPkg/Library/PlatformInitLib/MemDetect.c
index 942eaf89cfcf..d7c8b938f263 100644
--- a/OvmfPkg/Library/PlatformInitLib/MemDetect.c
+++ b/OvmfPkg/Library/PlatformInitLib/MemDetect.c
@@ -42,6 +42,8 @@ Module Name:
 
 #include <Library/PlatformInitLib.h>
 
+#define MEGABYTE_SHIFT  20
+
 VOID
 EFIAPI
 PlatformQemuUc32BaseInitialization (
@@ -289,6 +291,31 @@ GetHighestSystemMemoryAddressFromPvhMemmap (
   return HighestAddress;
 }
 
+UINT32
+EFIAPI
+PlatformAdjustSystemMemorySizeBelow4gbForLazyAccept (
+  IN UINT32  LowerMemorySize
+  )
+{
+ #ifdef MDE_CPU_X64
+  UINT64  LazyAcceptMemSize;
+
+  LazyAcceptMemSize = FixedPcdGet64 (PcdLazyAcceptPartialMemorySize);
+  //
+  // If specified accept size is not zero,
+  // transfer the size in megabyte to the number in byte.
+  //
+  if (LazyAcceptMemSize != 0) {
+    LazyAcceptMemSize <<= MEGABYTE_SHIFT;
+    if (LazyAcceptMemSize < LowerMemorySize) {
+      LowerMemorySize = (UINT32)(UINTN)LazyAcceptMemSize;
+    }
+  }
+
+ #endif
+  return LowerMemorySize;
+}
+
 UINT32
 EFIAPI
 PlatformGetSystemMemorySizeBelow4gb (
diff --git a/OvmfPkg/Library/PlatformInitLib/PlatformInitLib.inf b/OvmfPkg/Library/PlatformInitLib/PlatformInitLib.inf
index d2fa2d998df8..1c5ed1067ad4 100644
--- a/OvmfPkg/Library/PlatformInitLib/PlatformInitLib.inf
+++ b/OvmfPkg/Library/PlatformInitLib/PlatformInitLib.inf
@@ -96,6 +96,7 @@
   gUefiOvmfPkgTokenSpaceGuid.PcdGuidedExtractHandlerTableSize
 
   gUefiOvmfPkgTokenSpaceGuid.PcdTdxAcceptPageSize
+  gUefiOvmfPkgTokenSpaceGuid.PcdLazyAcceptPartialMemorySize
 
 [FeaturePcd]
   gEfiMdeModulePkgTokenSpaceGuid.PcdDxeIplSwitchToLongMode
diff --git a/OvmfPkg/PlatformPei/MemDetect.c b/OvmfPkg/PlatformPei/MemDetect.c
index 2e47b1322990..acc1d7f63ee8 100644
--- a/OvmfPkg/PlatformPei/MemDetect.c
+++ b/OvmfPkg/PlatformPei/MemDetect.c
@@ -279,6 +279,11 @@ PublishPeiMemory (
     LowerMemorySize -= mPlatformInfoHob.Q35TsegMbytes * SIZE_1MB;
   }
 
+  //
+  // Adjustment for Lazy accept because it may accept part of the memory.
+  //
+  LowerMemorySize = PlatformAdjustSystemMemorySizeBelow4gbForLazyAccept (LowerMemorySize);
+
   S3AcpiReservedMemoryBase = 0;
   S3AcpiReservedMemorySize = 0;
 
-- 
2.29.2.windows.2

[PATCH V2 10/14] OvmfPkg: Update ConstructFwHobList for lazy accept

[Min Xu]

From: Min M Xu <min.m.xu@intel.com>

RFC: https://bugzilla.tianocore.org/show_bug.cgi?id=3937

In TDVF the hob list is constructed at the memory region which is the
largest one below 4GB. After lazy accept is introduced, the
MaxAcceptedMemoryAddress (which is tha max accpeted memory address in
lazy accept) should be considered.

Cc: Erdem Aktas <erdemaktas@google.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Min Xu <min.m.xu@intel.com>
---
 OvmfPkg/Library/PeilessStartupLib/Hob.c       | 23 ++++++++++++++++++-
 .../PeilessStartupLib/PeilessStartupLib.inf   |  1 +
 2 files changed, 23 insertions(+), 1 deletion(-)

diff --git a/OvmfPkg/Library/PeilessStartupLib/Hob.c b/OvmfPkg/Library/PeilessStartupLib/Hob.c
index a9b92b5fbaba..884490af68fd 100644
--- a/OvmfPkg/Library/PeilessStartupLib/Hob.c
+++ b/OvmfPkg/Library/PeilessStartupLib/Hob.c
@@ -21,6 +21,7 @@
 #include <Library/PlatformInitLib.h>
 #include <OvmfPlatforms.h>
 #include <Pi/PrePiHob.h>
+#include <WorkArea.h>
 #include "PeilessStartupInternal.h"
 
 /**
@@ -74,10 +75,13 @@ ConstructFwHobList (
   )
 {
   EFI_PEI_HOB_POINTERS  Hob;
+  EFI_PHYSICAL_ADDRESS  PhysicalStart;
   EFI_PHYSICAL_ADDRESS  PhysicalEnd;
   UINT64                ResourceLength;
   EFI_PHYSICAL_ADDRESS  LowMemoryStart;
   UINT64                LowMemoryLength;
+  UINT64                MaxAcceptedMemoryAddress;
+  TDX_WORK_AREA         *WorkArea;
 
   ASSERT (VmmHobList != NULL);
 
@@ -86,14 +90,31 @@ ConstructFwHobList (
   LowMemoryLength = 0;
   LowMemoryStart  = 0;
 
+  WorkArea = (TDX_WORK_AREA *)FixedPcdGet32 (PcdOvmfWorkAreaBase);
+  ASSERT (WorkArea != NULL);
+  ASSERT (WorkArea->Header.GuestType == CcGuestTypeIntelTdx);
+  MaxAcceptedMemoryAddress = WorkArea->SecTdxWorkArea.MaxAcceptedMemoryAddress;
+  if (MaxAcceptedMemoryAddress == 0) {
+    MaxAcceptedMemoryAddress = MAX_UINT64;
+  }
+
   //
   // Parse the HOB list until end of list or matching type is found.
   //
   while (!END_OF_HOB_LIST (Hob)) {
     if (Hob.Header->HobType == EFI_HOB_TYPE_RESOURCE_DESCRIPTOR) {
-      if (Hob.ResourceDescriptor->ResourceType == EFI_RESOURCE_MEMORY_UNACCEPTED) {
+      if ((Hob.ResourceDescriptor->ResourceType == EFI_RESOURCE_MEMORY_UNACCEPTED) && (Hob.ResourceDescriptor->PhysicalStart < MaxAcceptedMemoryAddress)) {
         PhysicalEnd    = Hob.ResourceDescriptor->PhysicalStart + Hob.ResourceDescriptor->ResourceLength;
         ResourceLength = Hob.ResourceDescriptor->ResourceLength;
+        PhysicalStart  = Hob.ResourceDescriptor->PhysicalStart;
+
+        if ((PhysicalEnd >= MaxAcceptedMemoryAddress) && (PhysicalStart < MaxAcceptedMemoryAddress)) {
+          //
+          // This memory region is split into 2 parts. The left part is accepted.
+          //
+          PhysicalEnd    = MaxAcceptedMemoryAddress;
+          ResourceLength = PhysicalEnd - PhysicalStart;
+        }
 
         if (PhysicalEnd <= BASE_4GB) {
           if (ResourceLength > LowMemoryLength) {
diff --git a/OvmfPkg/Library/PeilessStartupLib/PeilessStartupLib.inf b/OvmfPkg/Library/PeilessStartupLib/PeilessStartupLib.inf
index def50b4b019e..eed9f27d3d01 100644
--- a/OvmfPkg/Library/PeilessStartupLib/PeilessStartupLib.inf
+++ b/OvmfPkg/Library/PeilessStartupLib/PeilessStartupLib.inf
@@ -88,3 +88,4 @@
   gEfiMdeModulePkgTokenSpaceGuid.PcdNullPointerDetectionPropertyMask    ## CONSUMES
   gUefiOvmfPkgTokenSpaceGuid.PcdOvmfDxeMemFvBase
   gUefiOvmfPkgTokenSpaceGuid.PcdOvmfDxeMemFvSize
+  gUefiOvmfPkgTokenSpaceGuid.PcdOvmfWorkAreaBase
-- 
2.29.2.windows.2

[PATCH V2 11/14] MdePkg: The prototype definition of EdkiiMemoryAcceptProtocol

[Min Xu]

From: Jiaqi Gao <jiaqi.gao@intel.com>

RFC: https://bugzilla.tianocore.org/show_bug.cgi?id=3937

EdkiiMemoryAcceptProtocol is defined in MdePkg, the method AcceptMemory()
can be called when memory needs to be accepted.

EdkiiMemoryAcceptProtocol can be installed by architecture-specific
drivers such as TdxDxe. This allows different isolation architectures
to realize their own low-level methods to accept memory.

Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Zhiguang Liu <zhiguang.liu@intel.com>
Cc: Erdem Aktas <erdemaktas@google.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Jiaqi Gao <jiaqi.gao@intel.com>
Signed-off-by: Min Xu <min.m.xu@intel.com>
---
 MdePkg/Include/Protocol/MemoryAccept.h | 37 ++++++++++++++++++++++++++
 MdePkg/MdePkg.dec                      |  3 +++
 2 files changed, 40 insertions(+)
 create mode 100644 MdePkg/Include/Protocol/MemoryAccept.h

diff --git a/MdePkg/Include/Protocol/MemoryAccept.h b/MdePkg/Include/Protocol/MemoryAccept.h
new file mode 100644
index 000000000000..f7646e04d8a1
--- /dev/null
+++ b/MdePkg/Include/Protocol/MemoryAccept.h
@@ -0,0 +1,37 @@
+/** @file
+  The file provides the protocol to provide interface to accept memory.
+
+  Copyright (c) 2021 - 2022, Intel Corporation. All rights reserved.<BR>
+  SPDX-License-Identifier: BSD-2-Clause-Patent
+**/
+
+#ifndef MEMORY_ACCEPT_H_
+#define MEMORY_ACCEPT_H_
+
+#define EDKII_MEMORY_ACCEPT_PROTOCOL_GUID \
+  { 0x38c74800, 0x5590, 0x4db4, { 0xa0, 0xf3, 0x67, 0x5d, 0x9b, 0x8e, 0x80, 0x26 } };
+
+typedef struct _EDKII_MEMORY_ACCEPT_PROTOCOL EDKII_MEMORY_ACCEPT_PROTOCOL;
+
+/**
+  @param This                   A pointer to a EDKII_MEMORY_ACCEPT_PROTOCOL.
+**/
+typedef
+EFI_STATUS
+(EFIAPI *EDKII_ACCEPT_MEMORY)(
+  IN  EDKII_MEMORY_ACCEPT_PROTOCOL  *This,
+  IN  EFI_PHYSICAL_ADDRESS          StartAddress,
+  IN  UINTN                         Size
+  );
+
+///
+/// The EDKII_MEMORY_ACCEPT_PROTOCOL provides the ability for memory services
+/// to accept memory.
+///
+struct _EDKII_MEMORY_ACCEPT_PROTOCOL {
+  EDKII_ACCEPT_MEMORY    AcceptMemory;
+};
+
+extern EFI_GUID  gEdkiiMemoryAcceptProtocolGuid;
+
+#endif
diff --git a/MdePkg/MdePkg.dec b/MdePkg/MdePkg.dec
index f1ebf9e251c1..6b6bfbec29b3 100644
--- a/MdePkg/MdePkg.dec
+++ b/MdePkg/MdePkg.dec
@@ -1019,6 +1019,9 @@
   gEfiPeiDelayedDispatchPpiGuid  = { 0x869c711d, 0x649c, 0x44fe, { 0x8b, 0x9e, 0x2c, 0xbb, 0x29, 0x11, 0xc3, 0xe6 }}
 
 [Protocols]
+  ## Include/Protocol/MemoryAccept.h
+  gEdkiiMemoryAcceptProtocolGuid = { 0x38c74800, 0x5590, 0x4db4, { 0xa0, 0xf3, 0x67, 0x5d, 0x9b, 0x8e, 0x80, 0x26 }}
+
   ## Include/Protocol/Pcd.h
   gPcdProtocolGuid               = { 0x11B34006, 0xD85B, 0x4D0A, { 0xA2, 0x90, 0xD5, 0xA5, 0x71, 0x31, 0x0E, 0xF7 }}
 
-- 
2.29.2.windows.2

Re: [PATCH V2 11/14] MdePkg: The prototype definition of EdkiiMemoryAcceptProtocol

[Gerd Hoffmann]

On Sat, Aug 27, 2022 at 02:21:18PM +0800, Min Xu wrote:
> From: Jiaqi Gao <jiaqi.gao@intel.com>
> 
> RFC: https://bugzilla.tianocore.org/show_bug.cgi?id=3937
> 
> EdkiiMemoryAcceptProtocol is defined in MdePkg, the method AcceptMemory()
> can be called when memory needs to be accepted.
> 
> EdkiiMemoryAcceptProtocol can be installed by architecture-specific
> drivers such as TdxDxe. This allows different isolation architectures
> to realize their own low-level methods to accept memory.
> 
> Cc: Michael D Kinney <michael.d.kinney@intel.com>
> Cc: Liming Gao <gaoliming@byosoft.com.cn>
> Cc: Zhiguang Liu <zhiguang.liu@intel.com>
> Cc: Erdem Aktas <erdemaktas@google.com>
> Cc: Gerd Hoffmann <kraxel@redhat.com>
> Cc: James Bottomley <jejb@linux.ibm.com>
> Cc: Jiewen Yao <jiewen.yao@intel.com>
> Cc: Tom Lendacky <thomas.lendacky@amd.com>
> Signed-off-by: Jiaqi Gao <jiaqi.gao@intel.com>
> Signed-off-by: Min Xu <min.m.xu@intel.com>

Acked-by: Gerd Hoffmann <kraxel@redhat.com>

[PATCH V2 12/14] OvmfPkg: Realize EdkiiMemoryAcceptProtocol in TdxDxe

[Min Xu]

From: Min M Xu <min.m.xu@intel.com>

RFC: https://bugzilla.tianocore.org/show_bug.cgi?id=3937

Memory usage may exceed the amount accepted at the begining (SEC), TDVF
needs to accept memory dynamically when OUT_OF_RESOURCE occurs.

EdkiiMemoryAcceptProtocol is defined in MdePkg and is implementated /
installed in TdxDxe for Intel TDX memory acceptance.

Cc: Erdem Aktas <erdemaktas@google.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Min Xu <min.m.xu@intel.com>
---
 OvmfPkg/TdxDxe/TdxDxe.c   | 103 ++++++++++++++++++++++++++++++++++++++
 OvmfPkg/TdxDxe/TdxDxe.inf |   2 +
 2 files changed, 105 insertions(+)

diff --git a/OvmfPkg/TdxDxe/TdxDxe.c b/OvmfPkg/TdxDxe/TdxDxe.c
index 2318db989792..ca948522a42c 100644
--- a/OvmfPkg/TdxDxe/TdxDxe.c
+++ b/OvmfPkg/TdxDxe/TdxDxe.c
@@ -24,6 +24,7 @@
 #include <Library/HobLib.h>
 #include <Protocol/Cpu.h>
 #include <Protocol/MpInitLibDepProtocols.h>
+#include <Protocol/MemoryAccept.h>
 #include <Library/UefiBootServicesTableLib.h>
 #include <ConfidentialComputingGuestAttr.h>
 #include <IndustryStandard/Tdx.h>
@@ -32,6 +33,95 @@
 #include <TdxAcpiTable.h>
 #include <Library/MemEncryptTdxLib.h>
 
+#define ALIGNED_2MB_MASK  0x1fffff
+EFI_HANDLE  mTdxDxeHandle = NULL;
+
+EFI_STATUS
+EFIAPI
+TdxMemoryAccept (
+  IN EDKII_MEMORY_ACCEPT_PROTOCOL  *This,
+  IN EFI_PHYSICAL_ADDRESS          StartAddress,
+  IN UINTN                         Size
+  )
+{
+  EFI_STATUS  Status;
+  UINT32      AcceptPageSize;
+  UINT64      StartAddress1;
+  UINT64      StartAddress2;
+  UINT64      StartAddress3;
+  UINT64      Length1;
+  UINT64      Length2;
+  UINT64      Length3;
+  UINT64      Pages;
+
+  AcceptPageSize = FixedPcdGet32 (PcdTdxAcceptPageSize);
+  StartAddress1  = 0;
+  StartAddress2  = 0;
+  StartAddress3  = 0;
+  Length1        = 0;
+  Length2        = 0;
+  Length3        = 0;
+
+  if (Size == 0) {
+    return EFI_SUCCESS;
+  }
+
+  if (ALIGN_VALUE (StartAddress, SIZE_2MB) != StartAddress) {
+    StartAddress1 = StartAddress;
+    Length1       = ALIGN_VALUE (StartAddress, SIZE_2MB) - StartAddress;
+    if (Length1 >= Size) {
+      Length1 = Size;
+    }
+
+    StartAddress += Length1;
+    Size         -= Length1;
+  }
+
+  if (Size > SIZE_2MB) {
+    StartAddress2 = StartAddress;
+    Length2       = Size & ~(UINT64)ALIGNED_2MB_MASK;
+    StartAddress += Length2;
+    Size         -= Length2;
+  }
+
+  if (Size) {
+    StartAddress3 = StartAddress;
+    Length3       = Size;
+  }
+
+  Status = EFI_SUCCESS;
+  if (Length1 > 0) {
+    Pages  = Length1 / SIZE_4KB;
+    Status = TdAcceptPages (StartAddress1, Pages, SIZE_4KB);
+    if (EFI_ERROR (Status)) {
+      return Status;
+    }
+  }
+
+  if (Length2 > 0) {
+    Pages  = Length2 / AcceptPageSize;
+    Status = TdAcceptPages (StartAddress2, Pages, AcceptPageSize);
+    if (EFI_ERROR (Status)) {
+      return Status;
+    }
+  }
+
+  if (Length3 > 0) {
+    Pages  = Length3 / SIZE_4KB;
+    Status = TdAcceptPages (StartAddress3, Pages, SIZE_4KB);
+    ASSERT (!EFI_ERROR (Status));
+    if (EFI_ERROR (Status)) {
+      return Status;
+    }
+  }
+
+  return Status;
+}
+
+EDKII_MEMORY_ACCEPT_PROTOCOL  mMemoryAcceptProtocol = {
+  TdxMemoryAccept
+};
+
 VOID
 SetPcdSettings (
   EFI_HOB_PLATFORM_INFO  *PlatformInfoHob
@@ -277,6 +367,19 @@ TdxDxeEntryPoint (
          NULL
          );
 
+  //
+  // Install MemoryAccept protocol for TDX
+  //
+  Status = gBS->InstallProtocolInterface (
+                  &mTdxDxeHandle,
+                  &gEdkiiMemoryAcceptProtocolGuid,
+                  EFI_NATIVE_INTERFACE,
+                  &mMemoryAcceptProtocol
+                  );
+  if (EFI_ERROR (Status)) {
+    DEBUG ((DEBUG_ERROR, "Install EdkiiMemoryAcceptProtocol failed.\n"));
+  }
+
   //
   // Call TDINFO to get actual number of cpus in domain
   //
diff --git a/OvmfPkg/TdxDxe/TdxDxe.inf b/OvmfPkg/TdxDxe/TdxDxe.inf
index a7e0abda1522..9be021f28648 100644
--- a/OvmfPkg/TdxDxe/TdxDxe.inf
+++ b/OvmfPkg/TdxDxe/TdxDxe.inf
@@ -52,6 +52,7 @@
   gEfiAcpiTableProtocolGuid                        ## CONSUMES
   gEfiMpInitLibMpDepProtocolGuid
   gEfiMpInitLibUpDepProtocolGuid
+  gEdkiiMemoryAcceptProtocolGuid
 
 [Pcd]
   gUefiOvmfPkgTokenSpaceGuid.PcdPciIoBase
@@ -68,3 +69,4 @@
   gEfiMdePkgTokenSpaceGuid.PcdConfidentialComputingGuestAttr
   gEfiMdeModulePkgTokenSpaceGuid.PcdTdxSharedBitMask
   gEfiMdeModulePkgTokenSpaceGuid.PcdSetNxForStack
+  gUefiOvmfPkgTokenSpaceGuid.PcdTdxAcceptPageSize
-- 
2.29.2.windows.2

Re: [PATCH V2 12/14] OvmfPkg: Realize EdkiiMemoryAcceptProtocol in TdxDxe

[Gerd Hoffmann]

On Sat, Aug 27, 2022 at 02:21:19PM +0800, Min Xu wrote:
> From: Min M Xu <min.m.xu@intel.com>
> 
> RFC: https://bugzilla.tianocore.org/show_bug.cgi?id=3937
> 
> Memory usage may exceed the amount accepted at the begining (SEC), TDVF
> needs to accept memory dynamically when OUT_OF_RESOURCE occurs.
> 
> EdkiiMemoryAcceptProtocol is defined in MdePkg and is implementated /
> installed in TdxDxe for Intel TDX memory acceptance.
> 
> Cc: Erdem Aktas <erdemaktas@google.com>
> Cc: Gerd Hoffmann <kraxel@redhat.com>
> Cc: James Bottomley <jejb@linux.ibm.com>
> Cc: Jiewen Yao <jiewen.yao@intel.com>
> Cc: Tom Lendacky <thomas.lendacky@amd.com>
> Signed-off-by: Min Xu <min.m.xu@intel.com>

Acked-by: Gerd Hoffmann <kraxel@redhat.com>

[PATCH V2 13/14] OvmfPkg: Call gEdkiiMemoryAcceptProtocolGuid to accept pages

[Min Xu]

From: Min M Xu <min.m.xu@intel.com>

RFC: https://bugzilla.tianocore.org/show_bug.cgi?id=3937

After EdkiiMemoryAcceptProtocol is implemented in TdxDxe driver, we can
call it to accept pages in DXE phase.

Cc: Erdem Aktas <erdemaktas@google.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Min Xu <min.m.xu@intel.com>
---
 .../BaseMemEncryptTdxLib/BaseMemEncryptTdxLib.inf    |  3 +++
 .../Library/BaseMemEncryptTdxLib/MemoryEncryption.c  | 12 +++++++++---
 2 files changed, 12 insertions(+), 3 deletions(-)

diff --git a/OvmfPkg/Library/BaseMemEncryptTdxLib/BaseMemEncryptTdxLib.inf b/OvmfPkg/Library/BaseMemEncryptTdxLib/BaseMemEncryptTdxLib.inf
index a8abfec12fa3..11768825f8ca 100644
--- a/OvmfPkg/Library/BaseMemEncryptTdxLib/BaseMemEncryptTdxLib.inf
+++ b/OvmfPkg/Library/BaseMemEncryptTdxLib/BaseMemEncryptTdxLib.inf
@@ -42,3 +42,6 @@
 
 [Pcd]
   gEfiMdePkgTokenSpaceGuid.PcdConfidentialComputingGuestAttr
+
+[Protocols]
+  gEdkiiMemoryAcceptProtocolGuid
diff --git a/OvmfPkg/Library/BaseMemEncryptTdxLib/MemoryEncryption.c b/OvmfPkg/Library/BaseMemEncryptTdxLib/MemoryEncryption.c
index 9d11c6e4df72..503f626d75c6 100644
--- a/OvmfPkg/Library/BaseMemEncryptTdxLib/MemoryEncryption.c
+++ b/OvmfPkg/Library/BaseMemEncryptTdxLib/MemoryEncryption.c
@@ -27,6 +27,8 @@
 #include "VirtualMemory.h"
 #include <IndustryStandard/Tdx.h>
 #include <Library/TdxLib.h>
+#include <Library/UefiBootServicesTableLib.h>
+#include <Protocol/MemoryAccept.h>
 #include <ConfidentialComputingGuestAttr.h>
 
 typedef enum {
@@ -517,8 +519,9 @@ SetOrClearSharedBit (
   IN           UINT64              Length
   )
 {
-  UINT64  AddressEncMask;
-  UINT64  Status;
+  UINT64                        AddressEncMask;
+  UINT64                        Status;
+  EDKII_MEMORY_ACCEPT_PROTOCOL  *MemoryAcceptProtocol;
 
   AddressEncMask = GetMemEncryptionAddressMask ();
 
@@ -539,7 +542,10 @@ SetOrClearSharedBit (
   // If changing shared to private, must accept-page again
   //
   if (Mode == ClearSharedBit) {
-    TdAcceptPages (PhysicalAddress, Length / EFI_PAGE_SIZE, EFI_PAGE_SIZE);
+    Status = gBS->LocateProtocol (&gEdkiiMemoryAcceptProtocolGuid, NULL, (VOID **)&MemoryAcceptProtocol);
+    ASSERT (!EFI_ERROR (Status));
+    Status = MemoryAcceptProtocol->AcceptMemory (MemoryAcceptProtocol, PhysicalAddress, Length);
+    ASSERT (!EFI_ERROR (Status));
   }
 
   DEBUG ((
-- 
2.29.2.windows.2

Re: [PATCH V2 13/14] OvmfPkg: Call gEdkiiMemoryAcceptProtocolGuid to accept pages

[Gerd Hoffmann]

On Sat, Aug 27, 2022 at 02:21:20PM +0800, Min Xu wrote:
> From: Min M Xu <min.m.xu@intel.com>
> 
> RFC: https://bugzilla.tianocore.org/show_bug.cgi?id=3937
> 
> After EdkiiMemoryAcceptProtocol is implemented in TdxDxe driver, we can
> call it to accept pages in DXE phase.
> 
> Cc: Erdem Aktas <erdemaktas@google.com>
> Cc: James Bottomley <jejb@linux.ibm.com>
> Cc: Jiewen Yao <jiewen.yao@intel.com>
> Cc: Gerd Hoffmann <kraxel@redhat.com>
> Cc: Tom Lendacky <thomas.lendacky@amd.com>
> Signed-off-by: Min Xu <min.m.xu@intel.com>

Acked-by: Gerd Hoffmann <kraxel@redhat.com>

[PATCH V2 14/14] MdeModulePkg: Pool and page functions accept memory when OOM occurs

[Min Xu]

From: Jiaqi Gao <jiaqi.gao@intel.com>

RFC: https://bugzilla.tianocore.org/show_bug.cgi?id=3937

When CoreAllocatePages() / CoreAllocatePool() meets error of
EFI_OUT_OF_RESOURCES, locate the EdkiiMemoryAcceptProtocol and accept extra
memory dynamically.

Firstly, find the unaccpeted memory region with enough size in GCD
entries. Then locate the EdkiiMemoryAcceptProtocol and accept the memory.
Finally, update the GCD memory and gMemoryMap entries.

After updating the memory infomation, CoreInternalAllocatePages() /
CoreInternalAllocatePool() will be recalled to allocate pages / pool.

Cc: Jian J Wang <jian.j.wang@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Dandan Bi <dandan.bi@intel.com>
Cc: Erdem Aktas <erdemaktas@google.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Signed-off-by: Jiaqi Gao <jiaqi.gao@intel.com>
Signed-off-by: Min Xu <min.m.xu@intel.com>
---
 MdeModulePkg/Core/Dxe/DxeMain.inf |   1 +
 MdeModulePkg/Core/Dxe/Mem/Imem.h  |  16 +++
 MdeModulePkg/Core/Dxe/Mem/Page.c  | 190 ++++++++++++++++++++++++++++++
 MdeModulePkg/Core/Dxe/Mem/Pool.c  |  14 +++
 4 files changed, 221 insertions(+)

diff --git a/MdeModulePkg/Core/Dxe/DxeMain.inf b/MdeModulePkg/Core/Dxe/DxeMain.inf
index e4bca895773d..371ba45357be 100644
--- a/MdeModulePkg/Core/Dxe/DxeMain.inf
+++ b/MdeModulePkg/Core/Dxe/DxeMain.inf
@@ -169,6 +169,7 @@
   gEfiVariableArchProtocolGuid                  ## CONSUMES
   gEfiCapsuleArchProtocolGuid                   ## CONSUMES
   gEfiWatchdogTimerArchProtocolGuid             ## CONSUMES
+  gEdkiiMemoryAcceptProtocolGuid                ## CONSUMES
 
 [Pcd]
   gEfiMdeModulePkgTokenSpaceGuid.PcdLoadFixAddressBootTimeCodePageNumber    ## SOMETIMES_CONSUMES
diff --git a/MdeModulePkg/Core/Dxe/Mem/Imem.h b/MdeModulePkg/Core/Dxe/Mem/Imem.h
index 2f0bf2bf631f..22e0d0e44030 100644
--- a/MdeModulePkg/Core/Dxe/Mem/Imem.h
+++ b/MdeModulePkg/Core/Dxe/Mem/Imem.h
@@ -47,6 +47,22 @@ typedef struct {
 // Internal prototypes
 //
 
+/**
+  Internal function.  Used by the pool and page functions to accept memory
+  when OOM occurs.
+
+  @param  Type                   The type of allocation to perform.
+  @param  AcceptSize             Size of memory to be accepted.
+  @param  Memory                 Accept memory address
+
+**/
+EFI_STATUS
+AcceptMemoryResource (
+  IN EFI_ALLOCATE_TYPE         Type,
+  IN UINTN                     AcceptSize,
+  IN OUT EFI_PHYSICAL_ADDRESS  *Memory
+  );
+
 /**
   Internal function.  Used by the pool functions to allocate pages
   to back pool allocation requests.
diff --git a/MdeModulePkg/Core/Dxe/Mem/Page.c b/MdeModulePkg/Core/Dxe/Mem/Page.c
index 160289c1f9ec..513792a7fe04 100644
--- a/MdeModulePkg/Core/Dxe/Mem/Page.c
+++ b/MdeModulePkg/Core/Dxe/Mem/Page.c
@@ -10,6 +10,7 @@ SPDX-License-Identifier: BSD-2-Clause-Patent
 #include "Imem.h"
 #include "HeapGuard.h"
 #include <Pi/PrePiDxeCis.h>
+#include <Protocol/MemoryAccept.h>
 
 //
 // Entry for tracking the memory regions for each memory type to coalesce similar memory types
@@ -379,6 +380,176 @@ CoreFreeMemoryMapStack (
   mFreeMapStack -= 1;
 }
 
+/**
+  Used to accept memory when OOM occurs.
+
+  @param  Type                   The type of allocation to perform.
+  @param  AcceptSize             Size of memory to be accepted.
+  @param  Memory                 Accept memory address
+
+**/
+EFI_STATUS
+AcceptMemoryResource (
+  IN EFI_ALLOCATE_TYPE         Type,
+  IN UINTN                     AcceptSize,
+  IN OUT EFI_PHYSICAL_ADDRESS  *Memory
+  )
+{
+ #ifdef MDE_CPU_X64
+
+  LIST_ENTRY                    *Link;
+  EFI_GCD_MAP_ENTRY             *GcdEntry;
+  EFI_GCD_MAP_ENTRY             UnacceptedEntry;
+  EDKII_MEMORY_ACCEPT_PROTOCOL  *MemoryAcceptProtocol;
+  UINTN                         Start;
+  UINTN                         End;
+  EFI_STATUS                    Status;
+
+  //
+  // We accept n*32MB at one time to improve the efficiency.
+  //
+  AcceptSize = (AcceptSize + SIZE_32MB - 1) & ~(SIZE_32MB - 1);
+
+  if (AcceptSize == 0) {
+    return EFI_INVALID_PARAMETER;
+  }
+
+  Status = gBS->LocateProtocol (&gEdkiiMemoryAcceptProtocolGuid, NULL, (VOID **)&MemoryAcceptProtocol);
+  if (EFI_ERROR (Status)) {
+    return EFI_UNSUPPORTED;
+  }
+
+  if (Type == AllocateAddress) {
+    Start = *Memory;
+    End   = *Memory + AcceptSize;
+  }
+
+  if (Type == AllocateMaxAddress) {
+    if (*Memory < EFI_PAGE_MASK) {
+      return EFI_INVALID_PARAMETER;
+    }
+
+    if ((*Memory & EFI_PAGE_MASK) != EFI_PAGE_MASK) {
+      //
+      // Change MaxAddress to be 1 page lower
+      //
+      *Memory -= EFI_PAGE_SIZE;
+
+      //
+      // Set MaxAddress to a page boundary
+      //
+      *Memory &= ~(UINT64)EFI_PAGE_MASK;
+
+      //
+      // Set MaxAddress to end of the page
+      //
+      *Memory |= EFI_PAGE_MASK;
+    }
+  }
+
+  //
+  // Traverse the mGcdMemorySpaceMap to find out the unaccepted
+  // memory entry with big enough size.
+  //
+  Link = mGcdMemorySpaceMap.ForwardLink;
+  while (Link != &mGcdMemorySpaceMap) {
+    GcdEntry = CR (Link, EFI_GCD_MAP_ENTRY, Link, EFI_GCD_MAP_SIGNATURE);
+    if (GcdEntry->GcdMemoryType == EFI_GCD_MEMORY_TYPE_UNACCEPTED) {
+      if (Type == AllocateMaxAddress) {
+        if (GcdEntry->BaseAddress + AcceptSize - 1 > *Memory) {
+          Link = Link->ForwardLink;
+          continue;
+        }
+      } else if (Type == AllocateAddress) {
+        if ((GcdEntry->BaseAddress > *Memory) || (GcdEntry->EndAddress < *Memory + AcceptSize - 1)) {
+          Link = Link->ForwardLink;
+          continue;
+        }
+      }
+
+      //
+      // Is the entry big enough?
+      //
+      if (AcceptSize <= GcdEntry->EndAddress - GcdEntry->BaseAddress + 1) {
+        UnacceptedEntry = *GcdEntry;
+        if (Type != AllocateAddress) {
+          Start = UnacceptedEntry.BaseAddress;
+          End   = UnacceptedEntry.BaseAddress + AcceptSize - 1;
+        }
+
+        break;
+      }
+    }
+
+    Link = Link->ForwardLink;
+  }
+
+  if (Link == &mGcdMemorySpaceMap) {
+    return EFI_OUT_OF_RESOURCES;
+  }
+
+  //
+  // Accept memory using the interface provide by the protocol.
+  //
+  Status = MemoryAcceptProtocol->AcceptMemory (MemoryAcceptProtocol, Start, AcceptSize);
+  if (EFI_ERROR (Status)) {
+    return EFI_OUT_OF_RESOURCES;
+  }
+
+  //
+  // If memory is accepted successfully, remove the target memory space from GCD.
+  //
+  CoreRemoveMemorySpace (UnacceptedEntry.BaseAddress, UnacceptedEntry.EndAddress - UnacceptedEntry.BaseAddress + 1);
+
+  //
+  // Add the remain lower part of unaccepted memory to the
+  // Gcd memory space and memory map.
+  //
+  if (Start > UnacceptedEntry.BaseAddress) {
+    CoreAddMemorySpace (
+      EFI_GCD_MEMORY_TYPE_UNACCEPTED,
+      UnacceptedEntry.BaseAddress,
+      Start - UnacceptedEntry.BaseAddress,
+      UnacceptedEntry.Capabilities
+      );
+  }
+
+  //
+  // Update accepted part of the memory entry to type of EfiGcdMemoryTypeSystemMemory
+  // and add the range to the memory map.
+  //
+  CoreAddMemorySpace (
+    EfiGcdMemoryTypeSystemMemory,
+    Start,
+    AcceptSize,
+    //
+    // Hardcode memory space attributes.
+    //
+    EFI_MEMORY_CPU_CRYPTO | EFI_MEMORY_XP | EFI_MEMORY_RO | EFI_MEMORY_RP
+    );
+
+  //
+  // Add the remain higher part of unaccepted memory to the
+  // Gcd memory space and memory map.
+  //
+  if (UnacceptedEntry.EndAddress > End) {
+    CoreAddMemorySpace (
+      EFI_GCD_MEMORY_TYPE_UNACCEPTED,
+      End + 1,
+      UnacceptedEntry.EndAddress - End,
+      UnacceptedEntry.Capabilities
+      );
+  }
+
+  return EFI_SUCCESS;
+
+ #else
+
+  return EFI_UNSUPPORTED;
+
+ #endif
+}
+
 /**
   Find untested but initialized memory regions in GCD map and convert them to be DXE allocatable.
 
@@ -1486,6 +1657,25 @@ CoreAllocatePages (
                 Memory,
                 NeedGuard
                 );
+ #ifdef MDE_CPU_X64
+
+  if (Status == EFI_OUT_OF_RESOURCES) {
+    Status = AcceptMemoryResource (Type, NumberOfPages << EFI_PAGE_SHIFT, Memory);
+    if (!EFI_ERROR (Status)) {
+      Status = CoreInternalAllocatePages (
+                 Type,
+                 MemoryType,
+                 NumberOfPages,
+                 Memory,
+                 NeedGuard
+                 );
+    } else {
+      Status = EFI_OUT_OF_RESOURCES;
+    }
+  }
+
+ #endif
+
   if (!EFI_ERROR (Status)) {
     CoreUpdateProfile (
       (EFI_PHYSICAL_ADDRESS)(UINTN)RETURN_ADDRESS (0),
diff --git a/MdeModulePkg/Core/Dxe/Mem/Pool.c b/MdeModulePkg/Core/Dxe/Mem/Pool.c
index 7aaf501600cf..9e8c8611c1ef 100644
--- a/MdeModulePkg/Core/Dxe/Mem/Pool.c
+++ b/MdeModulePkg/Core/Dxe/Mem/Pool.c
@@ -273,6 +273,20 @@ CoreAllocatePool (
   EFI_STATUS  Status;
 
   Status = CoreInternalAllocatePool (PoolType, Size, Buffer);
+
+ #ifdef MDE_CPU_X64
+
+  if (Status == EFI_OUT_OF_RESOURCES) {
+    Status = AcceptMemoryResource (AllocateAnyPages, Size, NULL);
+    if (!EFI_ERROR (Status)) {
+      Status = CoreInternalAllocatePool (PoolType, Size, Buffer);
+    } else {
+      Status = EFI_OUT_OF_RESOURCES;
+    }
+  }
+
+ #endif
+
   if (!EFI_ERROR (Status)) {
     CoreUpdateProfile (
       (EFI_PHYSICAL_ADDRESS)(UINTN)RETURN_ADDRESS (0),
-- 
2.29.2.windows.2

Re: [PATCH V2 14/14] MdeModulePkg: Pool and page functions accept memory when OOM occurs

[Lendacky, Thomas]

On 8/27/22 01:21, Min Xu wrote:
> From: Jiaqi Gao <jiaqi.gao@intel.com>
> 
> RFC: https://bugzilla.tianocore.org/show_bug.cgi?id=3937
> 
> When CoreAllocatePages() / CoreAllocatePool() meets error of
> EFI_OUT_OF_RESOURCES, locate the EdkiiMemoryAcceptProtocol and accept extra
> memory dynamically.
> 
> Firstly, find the unaccpeted memory region with enough size in GCD

s/unaccpeted/unaccepted/

> entries. Then locate the EdkiiMemoryAcceptProtocol and accept the memory.
> Finally, update the GCD memory and gMemoryMap entries.
> 
> After updating the memory infomation, CoreInternalAllocatePages() /
> CoreInternalAllocatePool() will be recalled to allocate pages / pool.

What path does allocation take when called through boot services? If I set 
a 256MB accepted memory size, I can get to the bootloader and select my 
kernel. But then the kernel dies in efi_relocate_kernel() with:

EFI stub: ERROR: Failed to allocate usable memory for kernel.
EFI stub: ERROR: efi_relocate_kernel() failed!
EFI stub: ERROR: efi_main() failed!

because both efi_bs_call(allocate_pages, ...) and efi_low_alloc_above() fail.

Similar to DXE, should OVMF accept more memory through this path to let 
the kernel boot?

Thanks,
Tom

> 
> Cc: Jian J Wang <jian.j.wang@intel.com>
> Cc: Liming Gao <gaoliming@byosoft.com.cn>
> Cc: Dandan Bi <dandan.bi@intel.com>
> Cc: Erdem Aktas <erdemaktas@google.com>
> Cc: James Bottomley <jejb@linux.ibm.com>
> Cc: Jiewen Yao <jiewen.yao@intel.com>
> Cc: Tom Lendacky <thomas.lendacky@amd.com>
> Cc: Gerd Hoffmann <kraxel@redhat.com>
> Signed-off-by: Jiaqi Gao <jiaqi.gao@intel.com>
> Signed-off-by: Min Xu <min.m.xu@intel.com>
> ---
>   MdeModulePkg/Core/Dxe/DxeMain.inf |   1 +
>   MdeModulePkg/Core/Dxe/Mem/Imem.h  |  16 +++
>   MdeModulePkg/Core/Dxe/Mem/Page.c  | 190 ++++++++++++++++++++++++++++++
>   MdeModulePkg/Core/Dxe/Mem/Pool.c  |  14 +++
>   4 files changed, 221 insertions(+)
> 
> diff --git a/MdeModulePkg/Core/Dxe/DxeMain.inf b/MdeModulePkg/Core/Dxe/DxeMain.inf
> index e4bca895773d..371ba45357be 100644
> --- a/MdeModulePkg/Core/Dxe/DxeMain.inf
> +++ b/MdeModulePkg/Core/Dxe/DxeMain.inf
> @@ -169,6 +169,7 @@
>     gEfiVariableArchProtocolGuid                  ## CONSUMES
>     gEfiCapsuleArchProtocolGuid                   ## CONSUMES
>     gEfiWatchdogTimerArchProtocolGuid             ## CONSUMES
> +  gEdkiiMemoryAcceptProtocolGuid                ## CONSUMES
>   
>   [Pcd]
>     gEfiMdeModulePkgTokenSpaceGuid.PcdLoadFixAddressBootTimeCodePageNumber    ## SOMETIMES_CONSUMES
> diff --git a/MdeModulePkg/Core/Dxe/Mem/Imem.h b/MdeModulePkg/Core/Dxe/Mem/Imem.h
> index 2f0bf2bf631f..22e0d0e44030 100644
> --- a/MdeModulePkg/Core/Dxe/Mem/Imem.h
> +++ b/MdeModulePkg/Core/Dxe/Mem/Imem.h
> @@ -47,6 +47,22 @@ typedef struct {
>   // Internal prototypes
>   //
>   
> +/**
> +  Internal function.  Used by the pool and page functions to accept memory
> +  when OOM occurs.
> +
> +  @param  Type                   The type of allocation to perform.
> +  @param  AcceptSize             Size of memory to be accepted.
> +  @param  Memory                 Accept memory address
> +
> +**/
> +EFI_STATUS
> +AcceptMemoryResource (
> +  IN EFI_ALLOCATE_TYPE         Type,
> +  IN UINTN                     AcceptSize,
> +  IN OUT EFI_PHYSICAL_ADDRESS  *Memory
> +  );
> +
>   /**
>     Internal function.  Used by the pool functions to allocate pages
>     to back pool allocation requests.
> diff --git a/MdeModulePkg/Core/Dxe/Mem/Page.c b/MdeModulePkg/Core/Dxe/Mem/Page.c
> index 160289c1f9ec..513792a7fe04 100644
> --- a/MdeModulePkg/Core/Dxe/Mem/Page.c
> +++ b/MdeModulePkg/Core/Dxe/Mem/Page.c
> @@ -10,6 +10,7 @@ SPDX-License-Identifier: BSD-2-Clause-Patent
>   #include "Imem.h"
>   #include "HeapGuard.h"
>   #include <Pi/PrePiDxeCis.h>
> +#include <Protocol/MemoryAccept.h>
>   
>   //
>   // Entry for tracking the memory regions for each memory type to coalesce similar memory types
> @@ -379,6 +380,176 @@ CoreFreeMemoryMapStack (
>     mFreeMapStack -= 1;
>   }
>   
> +/**
> +  Used to accept memory when OOM occurs.
> +
> +  @param  Type                   The type of allocation to perform.
> +  @param  AcceptSize             Size of memory to be accepted.
> +  @param  Memory                 Accept memory address
> +
> +**/
> +EFI_STATUS
> +AcceptMemoryResource (
> +  IN EFI_ALLOCATE_TYPE         Type,
> +  IN UINTN                     AcceptSize,
> +  IN OUT EFI_PHYSICAL_ADDRESS  *Memory
> +  )
> +{
> + #ifdef MDE_CPU_X64
> +
> +  LIST_ENTRY                    *Link;
> +  EFI_GCD_MAP_ENTRY             *GcdEntry;
> +  EFI_GCD_MAP_ENTRY             UnacceptedEntry;
> +  EDKII_MEMORY_ACCEPT_PROTOCOL  *MemoryAcceptProtocol;
> +  UINTN                         Start;
> +  UINTN                         End;
> +  EFI_STATUS                    Status;
> +
> +  //
> +  // We accept n*32MB at one time to improve the efficiency.
> +  //
> +  AcceptSize = (AcceptSize + SIZE_32MB - 1) & ~(SIZE_32MB - 1);
> +
> +  if (AcceptSize == 0) {
> +    return EFI_INVALID_PARAMETER;
> +  }
> +
> +  Status = gBS->LocateProtocol (&gEdkiiMemoryAcceptProtocolGuid, NULL, (VOID **)&MemoryAcceptProtocol);
> +  if (EFI_ERROR (Status)) {
> +    return EFI_UNSUPPORTED;
> +  }
> +
> +  if (Type == AllocateAddress) {
> +    Start = *Memory;
> +    End   = *Memory + AcceptSize;
> +  }
> +
> +  if (Type == AllocateMaxAddress) {
> +    if (*Memory < EFI_PAGE_MASK) {
> +      return EFI_INVALID_PARAMETER;
> +    }
> +
> +    if ((*Memory & EFI_PAGE_MASK) != EFI_PAGE_MASK) {
> +      //
> +      // Change MaxAddress to be 1 page lower
> +      //
> +      *Memory -= EFI_PAGE_SIZE;
> +
> +      //
> +      // Set MaxAddress to a page boundary
> +      //
> +      *Memory &= ~(UINT64)EFI_PAGE_MASK;
> +
> +      //
> +      // Set MaxAddress to end of the page
> +      //
> +      *Memory |= EFI_PAGE_MASK;
> +    }
> +  }
> +
> +  //
> +  // Traverse the mGcdMemorySpaceMap to find out the unaccepted
> +  // memory entry with big enough size.
> +  //
> +  Link = mGcdMemorySpaceMap.ForwardLink;
> +  while (Link != &mGcdMemorySpaceMap) {
> +    GcdEntry = CR (Link, EFI_GCD_MAP_ENTRY, Link, EFI_GCD_MAP_SIGNATURE);
> +    if (GcdEntry->GcdMemoryType == EFI_GCD_MEMORY_TYPE_UNACCEPTED) {
> +      if (Type == AllocateMaxAddress) {
> +        if (GcdEntry->BaseAddress + AcceptSize - 1 > *Memory) {
> +          Link = Link->ForwardLink;
> +          continue;
> +        }
> +      } else if (Type == AllocateAddress) {
> +        if ((GcdEntry->BaseAddress > *Memory) || (GcdEntry->EndAddress < *Memory + AcceptSize - 1)) {
> +          Link = Link->ForwardLink;
> +          continue;
> +        }
> +      }
> +
> +      //
> +      // Is the entry big enough?
> +      //
> +      if (AcceptSize <= GcdEntry->EndAddress - GcdEntry->BaseAddress + 1) {
> +        UnacceptedEntry = *GcdEntry;
> +        if (Type != AllocateAddress) {
> +          Start = UnacceptedEntry.BaseAddress;
> +          End   = UnacceptedEntry.BaseAddress + AcceptSize - 1;
> +        }
> +
> +        break;
> +      }
> +    }
> +
> +    Link = Link->ForwardLink;
> +  }
> +
> +  if (Link == &mGcdMemorySpaceMap) {
> +    return EFI_OUT_OF_RESOURCES;
> +  }
> +
> +  //
> +  // Accept memory using the interface provide by the protocol.
> +  //
> +  Status = MemoryAcceptProtocol->AcceptMemory (MemoryAcceptProtocol, Start, AcceptSize);
> +  if (EFI_ERROR (Status)) {
> +    return EFI_OUT_OF_RESOURCES;
> +  }
> +
> +  //
> +  // If memory is accepted successfully, remove the target memory space from GCD.
> +  //
> +  CoreRemoveMemorySpace (UnacceptedEntry.BaseAddress, UnacceptedEntry.EndAddress - UnacceptedEntry.BaseAddress + 1);
> +
> +  //
> +  // Add the remain lower part of unaccepted memory to the
> +  // Gcd memory space and memory map.
> +  //
> +  if (Start > UnacceptedEntry.BaseAddress) {
> +    CoreAddMemorySpace (
> +      EFI_GCD_MEMORY_TYPE_UNACCEPTED,
> +      UnacceptedEntry.BaseAddress,
> +      Start - UnacceptedEntry.BaseAddress,
> +      UnacceptedEntry.Capabilities
> +      );
> +  }
> +
> +  //
> +  // Update accepted part of the memory entry to type of EfiGcdMemoryTypeSystemMemory
> +  // and add the range to the memory map.
> +  //
> +  CoreAddMemorySpace (
> +    EfiGcdMemoryTypeSystemMemory,
> +    Start,
> +    AcceptSize,
> +    //
> +    // Hardcode memory space attributes.
> +    //
> +    EFI_MEMORY_CPU_CRYPTO | EFI_MEMORY_XP | EFI_MEMORY_RO | EFI_MEMORY_RP
> +    );
> +
> +  //
> +  // Add the remain higher part of unaccepted memory to the
> +  // Gcd memory space and memory map.
> +  //
> +  if (UnacceptedEntry.EndAddress > End) {
> +    CoreAddMemorySpace (
> +      EFI_GCD_MEMORY_TYPE_UNACCEPTED,
> +      End + 1,
> +      UnacceptedEntry.EndAddress - End,
> +      UnacceptedEntry.Capabilities
> +      );
> +  }
> +
> +  return EFI_SUCCESS;
> +
> + #else
> +
> +  return EFI_UNSUPPORTED;
> +
> + #endif
> +}
> +
>   /**
>     Find untested but initialized memory regions in GCD map and convert them to be DXE allocatable.
>   
> @@ -1486,6 +1657,25 @@ CoreAllocatePages (
>                   Memory,
>                   NeedGuard
>                   );
> + #ifdef MDE_CPU_X64
> +
> +  if (Status == EFI_OUT_OF_RESOURCES) {
> +    Status = AcceptMemoryResource (Type, NumberOfPages << EFI_PAGE_SHIFT, Memory);
> +    if (!EFI_ERROR (Status)) {
> +      Status = CoreInternalAllocatePages (
> +                 Type,
> +                 MemoryType,
> +                 NumberOfPages,
> +                 Memory,
> +                 NeedGuard
> +                 );
> +    } else {
> +      Status = EFI_OUT_OF_RESOURCES;
> +    }
> +  }
> +
> + #endif
> +
>     if (!EFI_ERROR (Status)) {
>       CoreUpdateProfile (
>         (EFI_PHYSICAL_ADDRESS)(UINTN)RETURN_ADDRESS (0),
> diff --git a/MdeModulePkg/Core/Dxe/Mem/Pool.c b/MdeModulePkg/Core/Dxe/Mem/Pool.c
> index 7aaf501600cf..9e8c8611c1ef 100644
> --- a/MdeModulePkg/Core/Dxe/Mem/Pool.c
> +++ b/MdeModulePkg/Core/Dxe/Mem/Pool.c
> @@ -273,6 +273,20 @@ CoreAllocatePool (
>     EFI_STATUS  Status;
>   
>     Status = CoreInternalAllocatePool (PoolType, Size, Buffer);
> +
> + #ifdef MDE_CPU_X64
> +
> +  if (Status == EFI_OUT_OF_RESOURCES) {
> +    Status = AcceptMemoryResource (AllocateAnyPages, Size, NULL);
> +    if (!EFI_ERROR (Status)) {
> +      Status = CoreInternalAllocatePool (PoolType, Size, Buffer);
> +    } else {
> +      Status = EFI_OUT_OF_RESOURCES;
> +    }
> +  }
> +
> + #endif
> +
>     if (!EFI_ERROR (Status)) {
>       CoreUpdateProfile (
>         (EFI_PHYSICAL_ADDRESS)(UINTN)RETURN_ADDRESS (0),

Re: [edk2-devel] [PATCH V2 14/14] MdeModulePkg: Pool and page functions accept memory when OOM occurs

[Ni, Ray]

Can you please explain more about this patch?
I am a little nervous when seeing patches that change the fundamental memory services.

> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Lendacky, Thomas via groups.io
> Sent: Tuesday, August 30, 2022 4:47 AM
> To: Xu, Min M <min.m.xu@intel.com>; devel@edk2.groups.io
> Cc: Gao, Jiaqi <jiaqi.gao@intel.com>; Wang, Jian J <jian.j.wang@intel.com>; Gao, Liming <gaoliming@byosoft.com.cn>; Bi,
> Dandan <dandan.bi@intel.com>; Aktas, Erdem <erdemaktas@google.com>; James Bottomley <jejb@linux.ibm.com>; Yao,
> Jiewen <jiewen.yao@intel.com>; Gerd Hoffmann <kraxel@redhat.com>
> Subject: Re: [edk2-devel] [PATCH V2 14/14] MdeModulePkg: Pool and page functions accept memory when OOM occurs
> 
> On 8/27/22 01:21, Min Xu wrote:
> > From: Jiaqi Gao <jiaqi.gao@intel.com>
> >
> > RFC: https://bugzilla.tianocore.org/show_bug.cgi?id=3937
> >
> > When CoreAllocatePages() / CoreAllocatePool() meets error of
> > EFI_OUT_OF_RESOURCES, locate the EdkiiMemoryAcceptProtocol and accept extra
> > memory dynamically.
> >
> > Firstly, find the unaccpeted memory region with enough size in GCD
> 
> s/unaccpeted/unaccepted/
> 
> > entries. Then locate the EdkiiMemoryAcceptProtocol and accept the memory.
> > Finally, update the GCD memory and gMemoryMap entries.
> >
> > After updating the memory infomation, CoreInternalAllocatePages() /
> > CoreInternalAllocatePool() will be recalled to allocate pages / pool.
> 
> What path does allocation take when called through boot services? If I set
> a 256MB accepted memory size, I can get to the bootloader and select my
> kernel. But then the kernel dies in efi_relocate_kernel() with:
> 
> EFI stub: ERROR: Failed to allocate usable memory for kernel.
> EFI stub: ERROR: efi_relocate_kernel() failed!
> EFI stub: ERROR: efi_main() failed!
> 
> because both efi_bs_call(allocate_pages, ...) and efi_low_alloc_above() fail.
> 
> Similar to DXE, should OVMF accept more memory through this path to let
> the kernel boot?
> 
> Thanks,
> Tom
> 
> >
> > Cc: Jian J Wang <jian.j.wang@intel.com>
> > Cc: Liming Gao <gaoliming@byosoft.com.cn>
> > Cc: Dandan Bi <dandan.bi@intel.com>
> > Cc: Erdem Aktas <erdemaktas@google.com>
> > Cc: James Bottomley <jejb@linux.ibm.com>
> > Cc: Jiewen Yao <jiewen.yao@intel.com>
> > Cc: Tom Lendacky <thomas.lendacky@amd.com>
> > Cc: Gerd Hoffmann <kraxel@redhat.com>
> > Signed-off-by: Jiaqi Gao <jiaqi.gao@intel.com>
> > Signed-off-by: Min Xu <min.m.xu@intel.com>
> > ---
> >   MdeModulePkg/Core/Dxe/DxeMain.inf |   1 +
> >   MdeModulePkg/Core/Dxe/Mem/Imem.h  |  16 +++
> >   MdeModulePkg/Core/Dxe/Mem/Page.c  | 190 ++++++++++++++++++++++++++++++
> >   MdeModulePkg/Core/Dxe/Mem/Pool.c  |  14 +++
> >   4 files changed, 221 insertions(+)
> >
> > diff --git a/MdeModulePkg/Core/Dxe/DxeMain.inf b/MdeModulePkg/Core/Dxe/DxeMain.inf
> > index e4bca895773d..371ba45357be 100644
> > --- a/MdeModulePkg/Core/Dxe/DxeMain.inf
> > +++ b/MdeModulePkg/Core/Dxe/DxeMain.inf
> > @@ -169,6 +169,7 @@
> >     gEfiVariableArchProtocolGuid                  ## CONSUMES
> >     gEfiCapsuleArchProtocolGuid                   ## CONSUMES
> >     gEfiWatchdogTimerArchProtocolGuid             ## CONSUMES
> > +  gEdkiiMemoryAcceptProtocolGuid                ## CONSUMES
> >
> >   [Pcd]
> >     gEfiMdeModulePkgTokenSpaceGuid.PcdLoadFixAddressBootTimeCodePageNumber    ## SOMETIMES_CONSUMES
> > diff --git a/MdeModulePkg/Core/Dxe/Mem/Imem.h b/MdeModulePkg/Core/Dxe/Mem/Imem.h
> > index 2f0bf2bf631f..22e0d0e44030 100644
> > --- a/MdeModulePkg/Core/Dxe/Mem/Imem.h
> > +++ b/MdeModulePkg/Core/Dxe/Mem/Imem.h
> > @@ -47,6 +47,22 @@ typedef struct {
> >   // Internal prototypes
> >   //
> >
> > +/**
> > +  Internal function.  Used by the pool and page functions to accept memory
> > +  when OOM occurs.
> > +
> > +  @param  Type                   The type of allocation to perform.
> > +  @param  AcceptSize             Size of memory to be accepted.
> > +  @param  Memory                 Accept memory address
> > +
> > +**/
> > +EFI_STATUS
> > +AcceptMemoryResource (
> > +  IN EFI_ALLOCATE_TYPE         Type,
> > +  IN UINTN                     AcceptSize,
> > +  IN OUT EFI_PHYSICAL_ADDRESS  *Memory
> > +  );
> > +
> >   /**
> >     Internal function.  Used by the pool functions to allocate pages
> >     to back pool allocation requests.
> > diff --git a/MdeModulePkg/Core/Dxe/Mem/Page.c b/MdeModulePkg/Core/Dxe/Mem/Page.c
> > index 160289c1f9ec..513792a7fe04 100644
> > --- a/MdeModulePkg/Core/Dxe/Mem/Page.c
> > +++ b/MdeModulePkg/Core/Dxe/Mem/Page.c
> > @@ -10,6 +10,7 @@ SPDX-License-Identifier: BSD-2-Clause-Patent
> >   #include "Imem.h"
> >   #include "HeapGuard.h"
> >   #include <Pi/PrePiDxeCis.h>
> > +#include <Protocol/MemoryAccept.h>
> >
> >   //
> >   // Entry for tracking the memory regions for each memory type to coalesce similar memory types
> > @@ -379,6 +380,176 @@ CoreFreeMemoryMapStack (
> >     mFreeMapStack -= 1;
> >   }
> >
> > +/**
> > +  Used to accept memory when OOM occurs.
> > +
> > +  @param  Type                   The type of allocation to perform.
> > +  @param  AcceptSize             Size of memory to be accepted.
> > +  @param  Memory                 Accept memory address
> > +
> > +**/
> > +EFI_STATUS
> > +AcceptMemoryResource (
> > +  IN EFI_ALLOCATE_TYPE         Type,
> > +  IN UINTN                     AcceptSize,
> > +  IN OUT EFI_PHYSICAL_ADDRESS  *Memory
> > +  )
> > +{
> > + #ifdef MDE_CPU_X64
> > +
> > +  LIST_ENTRY                    *Link;
> > +  EFI_GCD_MAP_ENTRY             *GcdEntry;
> > +  EFI_GCD_MAP_ENTRY             UnacceptedEntry;
> > +  EDKII_MEMORY_ACCEPT_PROTOCOL  *MemoryAcceptProtocol;
> > +  UINTN                         Start;
> > +  UINTN                         End;
> > +  EFI_STATUS                    Status;
> > +
> > +  //
> > +  // We accept n*32MB at one time to improve the efficiency.
> > +  //
> > +  AcceptSize = (AcceptSize + SIZE_32MB - 1) & ~(SIZE_32MB - 1);
> > +
> > +  if (AcceptSize == 0) {
> > +    return EFI_INVALID_PARAMETER;
> > +  }
> > +
> > +  Status = gBS->LocateProtocol (&gEdkiiMemoryAcceptProtocolGuid, NULL, (VOID **)&MemoryAcceptProtocol);
> > +  if (EFI_ERROR (Status)) {
> > +    return EFI_UNSUPPORTED;
> > +  }
> > +
> > +  if (Type == AllocateAddress) {
> > +    Start = *Memory;
> > +    End   = *Memory + AcceptSize;
> > +  }
> > +
> > +  if (Type == AllocateMaxAddress) {
> > +    if (*Memory < EFI_PAGE_MASK) {
> > +      return EFI_INVALID_PARAMETER;
> > +    }
> > +
> > +    if ((*Memory & EFI_PAGE_MASK) != EFI_PAGE_MASK) {
> > +      //
> > +      // Change MaxAddress to be 1 page lower
> > +      //
> > +      *Memory -= EFI_PAGE_SIZE;
> > +
> > +      //
> > +      // Set MaxAddress to a page boundary
> > +      //
> > +      *Memory &= ~(UINT64)EFI_PAGE_MASK;
> > +
> > +      //
> > +      // Set MaxAddress to end of the page
> > +      //
> > +      *Memory |= EFI_PAGE_MASK;
> > +    }
> > +  }
> > +
> > +  //
> > +  // Traverse the mGcdMemorySpaceMap to find out the unaccepted
> > +  // memory entry with big enough size.
> > +  //
> > +  Link = mGcdMemorySpaceMap.ForwardLink;
> > +  while (Link != &mGcdMemorySpaceMap) {
> > +    GcdEntry = CR (Link, EFI_GCD_MAP_ENTRY, Link, EFI_GCD_MAP_SIGNATURE);
> > +    if (GcdEntry->GcdMemoryType == EFI_GCD_MEMORY_TYPE_UNACCEPTED) {
> > +      if (Type == AllocateMaxAddress) {
> > +        if (GcdEntry->BaseAddress + AcceptSize - 1 > *Memory) {
> > +          Link = Link->ForwardLink;
> > +          continue;
> > +        }
> > +      } else if (Type == AllocateAddress) {
> > +        if ((GcdEntry->BaseAddress > *Memory) || (GcdEntry->EndAddress < *Memory + AcceptSize - 1)) {
> > +          Link = Link->ForwardLink;
> > +          continue;
> > +        }
> > +      }
> > +
> > +      //
> > +      // Is the entry big enough?
> > +      //
> > +      if (AcceptSize <= GcdEntry->EndAddress - GcdEntry->BaseAddress + 1) {
> > +        UnacceptedEntry = *GcdEntry;
> > +        if (Type != AllocateAddress) {
> > +          Start = UnacceptedEntry.BaseAddress;
> > +          End   = UnacceptedEntry.BaseAddress + AcceptSize - 1;
> > +        }
> > +
> > +        break;
> > +      }
> > +    }
> > +
> > +    Link = Link->ForwardLink;
> > +  }
> > +
> > +  if (Link == &mGcdMemorySpaceMap) {
> > +    return EFI_OUT_OF_RESOURCES;
> > +  }
> > +
> > +  //
> > +  // Accept memory using the interface provide by the protocol.
> > +  //
> > +  Status = MemoryAcceptProtocol->AcceptMemory (MemoryAcceptProtocol, Start, AcceptSize);
> > +  if (EFI_ERROR (Status)) {
> > +    return EFI_OUT_OF_RESOURCES;
> > +  }
> > +
> > +  //
> > +  // If memory is accepted successfully, remove the target memory space from GCD.
> > +  //
> > +  CoreRemoveMemorySpace (UnacceptedEntry.BaseAddress, UnacceptedEntry.EndAddress -
> UnacceptedEntry.BaseAddress + 1);
> > +
> > +  //
> > +  // Add the remain lower part of unaccepted memory to the
> > +  // Gcd memory space and memory map.
> > +  //
> > +  if (Start > UnacceptedEntry.BaseAddress) {
> > +    CoreAddMemorySpace (
> > +      EFI_GCD_MEMORY_TYPE_UNACCEPTED,
> > +      UnacceptedEntry.BaseAddress,
> > +      Start - UnacceptedEntry.BaseAddress,
> > +      UnacceptedEntry.Capabilities
> > +      );
> > +  }
> > +
> > +  //
> > +  // Update accepted part of the memory entry to type of EfiGcdMemoryTypeSystemMemory
> > +  // and add the range to the memory map.
> > +  //
> > +  CoreAddMemorySpace (
> > +    EfiGcdMemoryTypeSystemMemory,
> > +    Start,
> > +    AcceptSize,
> > +    //
> > +    // Hardcode memory space attributes.
> > +    //
> > +    EFI_MEMORY_CPU_CRYPTO | EFI_MEMORY_XP | EFI_MEMORY_RO | EFI_MEMORY_RP
> > +    );
> > +
> > +  //
> > +  // Add the remain higher part of unaccepted memory to the
> > +  // Gcd memory space and memory map.
> > +  //
> > +  if (UnacceptedEntry.EndAddress > End) {
> > +    CoreAddMemorySpace (
> > +      EFI_GCD_MEMORY_TYPE_UNACCEPTED,
> > +      End + 1,
> > +      UnacceptedEntry.EndAddress - End,
> > +      UnacceptedEntry.Capabilities
> > +      );
> > +  }
> > +
> > +  return EFI_SUCCESS;
> > +
> > + #else
> > +
> > +  return EFI_UNSUPPORTED;
> > +
> > + #endif
> > +}
> > +
> >   /**
> >     Find untested but initialized memory regions in GCD map and convert them to be DXE allocatable.
> >
> > @@ -1486,6 +1657,25 @@ CoreAllocatePages (
> >                   Memory,
> >                   NeedGuard
> >                   );
> > + #ifdef MDE_CPU_X64
> > +
> > +  if (Status == EFI_OUT_OF_RESOURCES) {
> > +    Status = AcceptMemoryResource (Type, NumberOfPages << EFI_PAGE_SHIFT, Memory);
> > +    if (!EFI_ERROR (Status)) {
> > +      Status = CoreInternalAllocatePages (
> > +                 Type,
> > +                 MemoryType,
> > +                 NumberOfPages,
> > +                 Memory,
> > +                 NeedGuard
> > +                 );
> > +    } else {
> > +      Status = EFI_OUT_OF_RESOURCES;
> > +    }
> > +  }
> > +
> > + #endif
> > +
> >     if (!EFI_ERROR (Status)) {
> >       CoreUpdateProfile (
> >         (EFI_PHYSICAL_ADDRESS)(UINTN)RETURN_ADDRESS (0),
> > diff --git a/MdeModulePkg/Core/Dxe/Mem/Pool.c b/MdeModulePkg/Core/Dxe/Mem/Pool.c
> > index 7aaf501600cf..9e8c8611c1ef 100644
> > --- a/MdeModulePkg/Core/Dxe/Mem/Pool.c
> > +++ b/MdeModulePkg/Core/Dxe/Mem/Pool.c
> > @@ -273,6 +273,20 @@ CoreAllocatePool (
> >     EFI_STATUS  Status;
> >
> >     Status = CoreInternalAllocatePool (PoolType, Size, Buffer);
> > +
> > + #ifdef MDE_CPU_X64
> > +
> > +  if (Status == EFI_OUT_OF_RESOURCES) {
> > +    Status = AcceptMemoryResource (AllocateAnyPages, Size, NULL);
> > +    if (!EFI_ERROR (Status)) {
> > +      Status = CoreInternalAllocatePool (PoolType, Size, Buffer);
> > +    } else {
> > +      Status = EFI_OUT_OF_RESOURCES;
> > +    }
> > +  }
> > +
> > + #endif
> > +
> >     if (!EFI_ERROR (Status)) {
> >       CoreUpdateProfile (
> >         (EFI_PHYSICAL_ADDRESS)(UINTN)RETURN_ADDRESS (0),
> 
> 
> 
> 

Re: [edk2-devel] [PATCH V2 14/14] MdeModulePkg: Pool and page functions accept memory when OOM occurs

[Min Xu]

On August 30, 2022 8:29 AM, Ni, Ray wrote:
> Can you please explain more about this patch?
> I am a little nervous when seeing patches that change the fundamental
> memory services.
> 
With the introduction of lazy-page-accept, the OUT_OF_RESOURCE may occur in Memory Allocation logic (MdeModulePkg/Core/Dxe/Mem). To address this issue, this patch update the CoreAllocatePages()/CoreAllocatePool() so that when OUT_OF_RESOURCE is triggered, we can accept more unaccepted-memory with the help of EdkiiMemoryAcceptProtocol. 

Please refer to slide7-10 in https://edk2.groups.io/g/devel/files/Designs/2021/0830/ . It describes how the AcceptMemoryResource works.

You can also refer to the discussion in https://edk2.groups.io/g/devel/message/79971.

Thanks
Min

Re: [edk2-devel] [PATCH V2 14/14] MdeModulePkg: Pool and page functions accept memory when OOM occurs

[Ni, Ray]

Min,
My understanding is there is a TDX driver in DXE phase that accepts memory using MP protocol.
We only need to make sure there is sufficient memory for code running before that point.
Is my understanding correct?

Thanks,
Ray

> -----Original Message-----
> From: Xu, Min M <min.m.xu@intel.com>
> Sent: Tuesday, August 30, 2022 2:01 PM
> To: Ni, Ray <ray.ni@intel.com>; devel@edk2.groups.io;
> thomas.lendacky@amd.com
> Cc: Gao, Jiaqi <jiaqi.gao@intel.com>; Wang, Jian J <jian.j.wang@intel.com>;
> Gao, Liming <gaoliming@byosoft.com.cn>; Bi, Dandan
> <dandan.bi@intel.com>; Aktas, Erdem <erdemaktas@google.com>; James
> Bottomley <jejb@linux.ibm.com>; Yao, Jiewen <jiewen.yao@intel.com>;
> Gerd Hoffmann <kraxel@redhat.com>; Xu, Min M <min.m.xu@intel.com>
> Subject: RE: [edk2-devel] [PATCH V2 14/14] MdeModulePkg: Pool and page
> functions accept memory when OOM occurs
> 
> On August 30, 2022 8:29 AM, Ni, Ray wrote:
> > Can you please explain more about this patch?
> > I am a little nervous when seeing patches that change the fundamental
> > memory services.
> >
> With the introduction of lazy-page-accept, the OUT_OF_RESOURCE may
> occur in Memory Allocation logic (MdeModulePkg/Core/Dxe/Mem). To
> address this issue, this patch update the
> CoreAllocatePages()/CoreAllocatePool() so that when OUT_OF_RESOURCE is
> triggered, we can accept more unaccepted-memory with the help of
> EdkiiMemoryAcceptProtocol.
> 
> Please refer to slide7-10 in
> https://edk2.groups.io/g/devel/files/Designs/2021/0830/ . It describes how
> the AcceptMemoryResource works.
> 
> You can also refer to the discussion in
> https://edk2.groups.io/g/devel/message/79971.
> 
> Thanks
> Min

Re: [edk2-devel] [PATCH V2 14/14] MdeModulePkg: Pool and page functions accept memory when OOM occurs

[Min Xu]

On August 30, 2022 2:15 PM, Ni Ray wrote:
> Min,
> My understanding is there is a TDX driver in DXE phase that accepts memory
> using MP protocol.
EdkiiMemoryAcceptProtocol is a protocol which accept memories. It is implemented/installed by the platform drivers, such as TdxDxe driver, or SEV driver.
It is up to the platform driver (Tdx or SEV driver) if MP protocol is used.
> We only need to make sure there is sufficient memory for code running
> before that point.
> Is my understanding correct?
That's right. But we'd better accept sufficient memory for the whole DXE phase. If OOM is triggered it's better accept more memory than the size needed. In current implementation we accept the memory in units of 32MB.

Thanks
Min

Re: [edk2-devel] [PATCH V2 14/14] MdeModulePkg: Pool and page functions accept memory when OOM occurs

[Ni, Ray]

I saw the justification of this is to support different size of Linux kernel images.
But that's after TD DXE driver.
For code running before TD DXE driver, the memory needs is predicable, right?

> -----Original Message-----
> From: Xu, Min M <min.m.xu@intel.com>
> Sent: Tuesday, August 30, 2022 2:35 PM
> To: Ni, Ray <ray.ni@intel.com>; devel@edk2.groups.io;
> thomas.lendacky@amd.com
> Cc: Gao, Jiaqi <jiaqi.gao@intel.com>; Wang, Jian J <jian.j.wang@intel.com>;
> Gao, Liming <gaoliming@byosoft.com.cn>; Bi, Dandan
> <dandan.bi@intel.com>; Aktas, Erdem <erdemaktas@google.com>; James
> Bottomley <jejb@linux.ibm.com>; Yao, Jiewen <jiewen.yao@intel.com>;
> Gerd Hoffmann <kraxel@redhat.com>
> Subject: RE: [edk2-devel] [PATCH V2 14/14] MdeModulePkg: Pool and page
> functions accept memory when OOM occurs
> 
> On August 30, 2022 2:15 PM, Ni Ray wrote:
> > Min,
> > My understanding is there is a TDX driver in DXE phase that accepts
> memory
> > using MP protocol.
> EdkiiMemoryAcceptProtocol is a protocol which accept memories. It is
> implemented/installed by the platform drivers, such as TdxDxe driver, or SEV
> driver.
> It is up to the platform driver (Tdx or SEV driver) if MP protocol is used.
> > We only need to make sure there is sufficient memory for code running
> > before that point.
> > Is my understanding correct?
> That's right. But we'd better accept sufficient memory for the whole DXE
> phase. If OOM is triggered it's better accept more memory than the size
> needed. In current implementation we accept the memory in units of 32MB.
> 
> Thanks
> Min

Re: [edk2-devel] [PATCH V2 14/14] MdeModulePkg: Pool and page functions accept memory when OOM occurs

[Min Xu]

On August 30, 2022 3:12 PM, Ni Ray wrote:
> 
> I saw the justification of this is to support different size of Linux kernel
> images.
> But that's after TD DXE driver.
> For code running before TD DXE driver, the memory needs is predicable,
> right?
That's right. The memory before loading linux kernel is predicable.
To support different size of linux kernel, there are hot discussions in EDK2/QEMU/LinuxKernel communities. There is a summary of the discussion in this link https://edk2.groups.io/g/devel/message/92332

I am also nervous to change in the fundamental memory allocation service (MdeModulePkg/Core/Dxe/Mem).
Another option is that in current stage we accept memories under 4G and so that we can drop this patch (Pool and page functions accept memory when OOM occurs).

What's your thoughts?

Thanks
Min

Re: [edk2-devel] [PATCH V2 14/14] MdeModulePkg: Pool and page functions accept memory when OOM occurs

[Ni, Ray]

> On August 30, 2022 3:12 PM, Ni Ray wrote:
> >
> > I saw the justification of this is to support different size of Linux kernel
> > images.
> > But that's after TD DXE driver.
> > For code running before TD DXE driver, the memory needs is predicable,
> > right?
> That's right. The memory before loading linux kernel is predicable.
> To support different size of linux kernel, there are hot discussions in
> EDK2/QEMU/LinuxKernel communities. There is a summary of the discussion
> in this link https://edk2.groups.io/g/devel/message/92332
> 
> I am also nervous to change in the fundamental memory allocation service
> (MdeModulePkg/Core/Dxe/Mem).
> Another option is that in current stage we accept memories under 4G and so
> that we can drop this patch (Pool and page functions accept memory when
> OOM occurs).
> 
> What's your thoughts?

Do you mean:
Accepting 4G memory in ResetVector and accepting more memory in TD DXE driver through MP protocol?

Does that need more time in ResetVector?
If memory range to be accepted by ResetVector in a different meta data record than the memory ranges
to be accepted by TD DXE driver?

Thanks,
Ray

Re: [edk2-devel] [PATCH V2 14/14] MdeModulePkg: Pool and page functions accept memory when OOM occurs

[Min Xu]

On August 30, 2022 3:44 PM, Ni Ray wrote:
> > On August 30, 2022 3:12 PM, Ni Ray wrote:
> > >
> > > I saw the justification of this is to support different size of
> > > Linux kernel images.
> > > But that's after TD DXE driver.
> > > For code running before TD DXE driver, the memory needs is
> > > predicable, right?
> > That's right. The memory before loading linux kernel is predicable.
> > To support different size of linux kernel, there are hot discussions
> > in EDK2/QEMU/LinuxKernel communities. There is a summary of the
> > discussion in this link https://edk2.groups.io/g/devel/message/92332
> >
> > I am also nervous to change in the fundamental memory allocation
> > service (MdeModulePkg/Core/Dxe/Mem).
> > Another option is that in current stage we accept memories under 4G
> > and so that we can drop this patch (Pool and page functions accept
> > memory when OOM occurs).
> >
> > What's your thoughts?
> 
> Do you mean:
> Accepting 4G memory in ResetVector and accepting more memory in TD DXE
> driver through MP protocol?
The unaccepted-memory-region is described in TdHob which is passed by QEMU. It includes the start address and size of the unaccepted-memory-region. Accepting memories under 4G is to accept the memories which start-address and size is under 4G. If an unaccept-memory-region is cross 4G, then it is split into 2 parts. We accept the lower part and leave the higher part as unaccepted.

> 
> Does that need more time in ResetVector?
TDVF accepts memories in SEC phase to make sure there are enough memories for PEI/DXE. Of course it costs more time to accept more memories.

> Is memory range to be accepted by ResetVector in a different meta data
> record than the memory ranges to be accepted by TD DXE driver?
The unaccept-memory-region information is recorded first in TdHob (which is passed from QEMU). 
Before jumping to DXE phase the accepted and unaccepted memory region information are recorded in the HOBs which are transferred to DXE. 
In DXE these information will be recorded in mGcdMemorySpaceMap which will be used by the EdkiiMemoryAcceptProtocol.

Thanks
Min

Re: [edk2-devel] [PATCH V2 14/14] MdeModulePkg: Pool and page functions accept memory when OOM occurs

[Ni, Ray]

> > Do you mean:
> > Accepting 4G memory in ResetVector and accepting more memory in TD
> DXE
> > driver through MP protocol?
> The unaccepted-memory-region is described in TdHob which is passed by
> QEMU. It includes the start address and size of the unaccepted-memory-
> region. Accepting memories under 4G is to accept the memories which start-
> address and size is under 4G. If an unaccept-memory-region is cross 4G, then
> it is split into 2 parts. We accept the lower part and leave the higher part as
> unaccepted.

Then, how about you do a test boot to UEFI shell to collect the memory consumption
then only accept that amount of memory in SEC? The amount of memory might
cause SEC to accept memory above 4GB if the memory base is 3.99G already.

Re: [edk2-devel] [PATCH V2 14/14] MdeModulePkg: Pool and page functions accept memory when OOM occurs

[Ni, Ray]

> >
> > I am also nervous to change in the fundamental memory allocation service
> > (MdeModulePkg/Core/Dxe/Mem).
> > Another option is that in current stage we accept memories under 4G and
> so
> > that we can drop this patch (Pool and page functions accept memory when
> > OOM occurs).
> >
> > What's your thoughts?
> 
> Do you mean:
> Accepting 4G memory in ResetVector and accepting more memory in TD DXE
> driver through MP protocol?
> 
> Does that need more time in ResetVector?
> If memory range to be accepted by ResetVector in a different meta data

Type: If -> Is

> record than the memory ranges
> to be accepted by TD DXE driver?


> 
> Thanks,
> Ray
> 
> 
> 
> 

Re: [edk2-devel] [PATCH V2 14/14] MdeModulePkg: Pool and page functions accept memory when OOM occurs

[Min Xu]

On August 30, 2022 4:47 AM, Lendacky, Thomas wrote:
> On 8/27/22 01:21, Min Xu wrote:
> > From: Jiaqi Gao <jiaqi.gao@intel.com>
> >
> > RFC: https://bugzilla.tianocore.org/show_bug.cgi?id=3937
> >
> > When CoreAllocatePages() / CoreAllocatePool() meets error of
> > EFI_OUT_OF_RESOURCES, locate the EdkiiMemoryAcceptProtocol and
> accept
> > extra memory dynamically.
> >
> > Firstly, find the unaccpeted memory region with enough size in GCD
> 
> s/unaccpeted/unaccepted/
Thanks for reminder. It will be fixed in the next version.
> 
> > entries. Then locate the EdkiiMemoryAcceptProtocol and accept the memory.
> > Finally, update the GCD memory and gMemoryMap entries.
> >
> > After updating the memory infomation, CoreInternalAllocatePages() /
> > CoreInternalAllocatePool() will be recalled to allocate pages / pool.
> 
> What path does allocation take when called through boot services? If I set a
> 256MB accepted memory size, I can get to the bootloader and select my kernel.
> But then the kernel dies in efi_relocate_kernel() with:
> 
> EFI stub: ERROR: Failed to allocate usable memory for kernel.
> EFI stub: ERROR: efi_relocate_kernel() failed!
> EFI stub: ERROR: efi_main() failed!
> 
> because both efi_bs_call(allocate_pages, ...) and efi_low_alloc_above() fail.
> 
> Similar to DXE, should OVMF accept more memory through this path to let the
> kernel boot?
> 
Tom is using the grub boot, right? In this case, 256MB accepted memory size is too small. 
Because in grub boot, AllocatePages is called to allocate memory in grub boot. The allocation type is AllocateMaxAddress and the memory address is 100000. It means the memory should be allocated below 0x100000. 
If only 256MB is accepted, then probably this memory region (which is under 0x100000) has been used. And there is no unaccepted memory region which is available for this memory region (which is under 0x100000).
I tried 256MB and it works for direct boot, but not for grub boot.
I tried 384MB and it works for both grub boot and direct boot.
So my suggestion is that the minimal accept memory size is 384MB or more.

Thanks
Min

Re: [edk2-devel] [PATCH V2 14/14] MdeModulePkg: Pool and page functions accept memory when OOM occurs

[Lendacky, Thomas]

On 8/29/22 22:20, Xu, Min M wrote:
> On August 30, 2022 4:47 AM, Lendacky, Thomas wrote:
>> On 8/27/22 01:21, Min Xu wrote:
>>> From: Jiaqi Gao <jiaqi.gao@intel.com>
>>>
>>> RFC: https://bugzilla.tianocore.org/show_bug.cgi?id=3937
>>>
>>> When CoreAllocatePages() / CoreAllocatePool() meets error of
>>> EFI_OUT_OF_RESOURCES, locate the EdkiiMemoryAcceptProtocol and
>> accept
>>> extra memory dynamically.
>>>
>>> Firstly, find the unaccpeted memory region with enough size in GCD
>>
>> s/unaccpeted/unaccepted/
> Thanks for reminder. It will be fixed in the next version.
>>
>>> entries. Then locate the EdkiiMemoryAcceptProtocol and accept the memory.
>>> Finally, update the GCD memory and gMemoryMap entries.
>>>
>>> After updating the memory infomation, CoreInternalAllocatePages() /
>>> CoreInternalAllocatePool() will be recalled to allocate pages / pool.
>>
>> What path does allocation take when called through boot services? If I set a
>> 256MB accepted memory size, I can get to the bootloader and select my kernel.
>> But then the kernel dies in efi_relocate_kernel() with:
>>
>> EFI stub: ERROR: Failed to allocate usable memory for kernel.
>> EFI stub: ERROR: efi_relocate_kernel() failed!
>> EFI stub: ERROR: efi_main() failed!
>>
>> because both efi_bs_call(allocate_pages, ...) and efi_low_alloc_above() fail.
>>
>> Similar to DXE, should OVMF accept more memory through this path to let the
>> kernel boot?
>>
> Tom is using the grub boot, right? In this case, 256MB accepted memory size is too small.
> Because in grub boot, AllocatePages is called to allocate memory in grub boot. The allocation type is AllocateMaxAddress and the memory address is 100000. It means the memory should be allocated below 0x100000.
> If only 256MB is accepted, then probably this memory region (which is under 0x100000) has been used. And there is no unaccepted memory region which is available for this memory region (which is under 0x100000).
> I tried 256MB and it works for direct boot, but not for grub boot.
> I tried 384MB and it works for both grub boot and direct boot.
> So my suggestion is that the minimal accept memory size is 384MB or more.

Right, I've been able to boot with 512MB without issues. The address is 
0x1000000 (16MB) and so that makes sense, nothing that can be done for 
that situation.

Thanks,
Tom

> 
> Thanks
> Min