Re: [edk2-devel] [PATCH 1/1] OvmfPkg: Store PageTablePool in TdxWorkArea

Re: [edk2-devel] [PATCH 1/1] OvmfPkg: Store PageTablePool in TdxWorkArea

[Min Xu]

Hi, Gerd
Do you have any comments on this patch?

> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Min Xu
> Sent: Wednesday, September 7, 2022 10:18 AM
> To: devel@edk2.groups.io
> Cc: Xu, Min M <min.m.xu@intel.com>; Aktas, Erdem
> <erdemaktas@google.com>; Gerd Hoffmann <kraxel@redhat.com>; James
> Bottomley <jejb@linux.ibm.com>; Yao, Jiewen <jiewen.yao@intel.com>; Tom
> Lendacky <thomas.lendacky@amd.com>
> Subject: [edk2-devel] [PATCH 1/1] OvmfPkg: Store PageTablePool in
> TdxWorkArea
> 
> From: Min M Xu <min.m.xu@intel.com>
> 
> PeilessStartupLib is running in SEC phase. In that phase global variable is not
> allowed. So the mPageTablePool cannot be used. In stead the PageTablePool
> is stored in TdxWorkArea.
> 
> Cc: Erdem Aktas <erdemaktas@google.com>
> Cc: Gerd Hoffmann <kraxel@redhat.com>
> Cc: James Bottomley <jejb@linux.ibm.com>
> Cc: Jiewen Yao <jiewen.yao@intel.com>
> Cc: Tom Lendacky <thomas.lendacky@amd.com>
> Signed-off-by: Min M Xu <min.m.xu@intel.com>
> ---
>  OvmfPkg/Include/WorkArea.h                    |  1 +
>  .../PeilessStartupLib/PeilessStartupLib.inf   |  1 +
>  .../PeilessStartupLib/X64/VirtualMemory.c     | 82 +++++++++++++------
>  3 files changed, 60 insertions(+), 24 deletions(-)
> 
> diff --git a/OvmfPkg/Include/WorkArea.h b/OvmfPkg/Include/WorkArea.h
> index bf56fc4a6f65..0b2fb2f8c573 100644
> --- a/OvmfPkg/Include/WorkArea.h
> +++ b/OvmfPkg/Include/WorkArea.h
> @@ -71,6 +71,7 @@ typedef struct _SEC_TDX_WORK_AREA {
>    UINT32    PageTableReady;
>    UINT32    Gpaw;
>    UINT64    HobList;
> +  UINT64    PageTablePool;
>  } SEC_TDX_WORK_AREA;
> 
>  typedef struct _TDX_WORK_AREA {
> diff --git a/OvmfPkg/Library/PeilessStartupLib/PeilessStartupLib.inf
> b/OvmfPkg/Library/PeilessStartupLib/PeilessStartupLib.inf
> index def50b4b019e..eed9f27d3d01 100644
> --- a/OvmfPkg/Library/PeilessStartupLib/PeilessStartupLib.inf
> +++ b/OvmfPkg/Library/PeilessStartupLib/PeilessStartupLib.inf
> @@ -88,3 +88,4 @@
> 
> gEfiMdeModulePkgTokenSpaceGuid.PcdNullPointerDetectionPropertyMask
> ## CONSUMES
>    gUefiOvmfPkgTokenSpaceGuid.PcdOvmfDxeMemFvBase
>    gUefiOvmfPkgTokenSpaceGuid.PcdOvmfDxeMemFvSize
> +  gUefiOvmfPkgTokenSpaceGuid.PcdOvmfWorkAreaBase
> diff --git a/OvmfPkg/Library/PeilessStartupLib/X64/VirtualMemory.c
> b/OvmfPkg/Library/PeilessStartupLib/X64/VirtualMemory.c
> index 6877e521e485..7e33f044993a 100644
> --- a/OvmfPkg/Library/PeilessStartupLib/X64/VirtualMemory.c
> +++ b/OvmfPkg/Library/PeilessStartupLib/X64/VirtualMemory.c
> @@ -19,13 +19,9 @@ SPDX-License-Identifier: BSD-2-Clause-Patent
> #include <Guid/MemoryAllocationHob.h>  #include <Register/Intel/Cpuid.h>
> #include <Library/PlatformInitLib.h>
> +#include <WorkArea.h>
>  #include "PageTables.h"
> 
> -//
> -// Global variable to keep track current available memory used as page table.
> -//
> -PAGE_TABLE_POOL  *mPageTablePool = NULL;
> -
>  UINTN  mLevelShift[5] = {
>    0,
>    PAGING_L1_ADDRESS_SHIFT,
> @@ -50,6 +46,32 @@ UINT64  mLevelSize[5] = {
>    SIZE_512GB
>  };
> 
> +PAGE_TABLE_POOL *
> +GetPageTablePool (
> +  VOID
> +  )
> +{
> +  TDX_WORK_AREA  *TdxWorkArea;
> +
> +  TdxWorkArea = (TDX_WORK_AREA *)(UINTN)FixedPcdGet32
> + (PcdOvmfWorkAreaBase);  ASSERT (TdxWorkArea != NULL);
> +
> +  return (PAGE_TABLE_POOL *)TdxWorkArea-
> >SecTdxWorkArea.PageTablePool;
> +}
> +
> +VOID
> +SetPageTablePool (
> +  VOID  *PageTablePool
> +  )
> +{
> +  TDX_WORK_AREA  *TdxWorkArea;
> +
> +  TdxWorkArea = (TDX_WORK_AREA *)(UINTN)FixedPcdGet32
> + (PcdOvmfWorkAreaBase);  ASSERT (TdxWorkArea != NULL);
> +
> +  TdxWorkArea->SecTdxWorkArea.PageTablePool =
> +(UINT64)(UINTN)PageTablePool; }
> +
>  BOOLEAN
>  IsSetNxForStack (
>    VOID
> @@ -283,7 +305,10 @@ InitializePageTablePool (
>    IN UINTN  PoolPages
>    )
>  {
> -  VOID  *Buffer;
> +  VOID             *Buffer;
> +  PAGE_TABLE_POOL  *PageTablePool;
> +
> +  PageTablePool = GetPageTablePool ();
> 
>    DEBUG ((DEBUG_INFO, "InitializePageTablePool PoolPages=%d\n",
> PoolPages));
> 
> @@ -303,20 +328,21 @@ InitializePageTablePool (
>    //
>    // Link all pools into a list for easier track later.
>    //
> -  if (mPageTablePool == NULL) {
> -    mPageTablePool           = Buffer;
> -    mPageTablePool->NextPool = mPageTablePool;
> +  if (PageTablePool == NULL) {
> +    PageTablePool           = Buffer;
> +    PageTablePool->NextPool = PageTablePool;
> +    SetPageTablePool (PageTablePool);
>    } else {
> -    ((PAGE_TABLE_POOL *)Buffer)->NextPool = mPageTablePool->NextPool;
> -    mPageTablePool->NextPool              = Buffer;
> -    mPageTablePool                        = Buffer;
> +    ((PAGE_TABLE_POOL *)Buffer)->NextPool = PageTablePool->NextPool;
> +    PageTablePool->NextPool               = Buffer;
> +    PageTablePool                         = Buffer;
>    }
> 
>    //
>    // Reserve one page for pool header.
>    //
> -  mPageTablePool->FreePages = PoolPages - 1;
> -  mPageTablePool->Offset    = EFI_PAGES_TO_SIZE (1);
> +  PageTablePool->FreePages = PoolPages - 1;
> +  PageTablePool->Offset    = EFI_PAGES_TO_SIZE (1);
> 
>    return TRUE;
>  }
> @@ -343,28 +369,33 @@ AllocatePageTableMemory (
>    IN UINTN  Pages
>    )
>  {
> -  VOID  *Buffer;
> +  VOID             *Buffer;
> +  PAGE_TABLE_POOL  *PageTablePool;
> 
>    if (Pages == 0) {
>      return NULL;
>    }
> 
> -  DEBUG ((DEBUG_INFO, "AllocatePageTableMemory. mPageTablePool=%p,
> Pages=%d\n", mPageTablePool, Pages));
> +  PageTablePool = GetPageTablePool ();
> +
> +  DEBUG ((DEBUG_INFO, "AllocatePageTableMemory. PageTablePool=%p,
> + Pages=%d\n", PageTablePool, Pages));
>    //
>    // Renew the pool if necessary.
>    //
> -  if ((mPageTablePool == NULL) ||
> -      (Pages > mPageTablePool->FreePages))
> +  if ((PageTablePool == NULL) ||
> +      (Pages > PageTablePool->FreePages))
>    {
>      if (!InitializePageTablePool (Pages)) {
>        return NULL;
> +    } else {
> +      PageTablePool = GetPageTablePool ();
>      }
>    }
> 
> -  Buffer = (UINT8 *)mPageTablePool + mPageTablePool->Offset;
> +  Buffer = (UINT8 *)PageTablePool + PageTablePool->Offset;
> 
> -  mPageTablePool->Offset    += EFI_PAGES_TO_SIZE (Pages);
> -  mPageTablePool->FreePages -= Pages;
> +  PageTablePool->Offset    += EFI_PAGES_TO_SIZE (Pages);
> +  PageTablePool->FreePages -= Pages;
> 
>    DEBUG ((
>      DEBUG_INFO,
> @@ -618,10 +649,13 @@ EnablePageTableProtection (
>    PAGE_TABLE_POOL       *Pool;
>    UINT64                PoolSize;
>    EFI_PHYSICAL_ADDRESS  Address;
> +  PAGE_TABLE_POOL       *PageTablePool;
> 
>    DEBUG ((DEBUG_INFO, "EnablePageTableProtection\n"));
> 
> -  if (mPageTablePool == NULL) {
> +  PageTablePool = GetPageTablePool ();
> +
> +  if (PageTablePool == NULL) {
>      return;
>    }
> 
> @@ -632,10 +666,10 @@ EnablePageTableProtection (
>    AsmWriteCr0 (AsmReadCr0 () & ~CR0_WP);
> 
>    //
> -  // SetPageTablePoolReadOnly might update mPageTablePool. It's safer to
> +  // SetPageTablePoolReadOnly might update PageTablePool. It's safer to
>    // remember original one in advance.
>    //
> -  HeadPool = mPageTablePool;
> +  HeadPool = PageTablePool;
>    Pool     = HeadPool;
>    do {
>      Address  = (EFI_PHYSICAL_ADDRESS)(UINTN)Pool;
> --
> 2.29.2.windows.2
> 
> 
> 
> 
> 

Re: [edk2-devel] [PATCH 1/1] OvmfPkg: Store PageTablePool in TdxWorkArea

[Gerd Hoffmann]

On Thu, Sep 22, 2022 at 04:43:04AM +0000, Xu, Min M wrote:
> Hi, Gerd
> Do you have any comments on this patch?

Looks ok to me.

But I'm increasingly wondering whenever it actually was that a smart
move to ditch the PEI phase for the IntelTdx builds.  Anything which
can't be handled in DXE must be done in SEC, and the SEC restrictions
seem to be a constant struggle.

We had that when adding tdx measurement support which required sha*
hash support for SEC.  And this looks like another case ...

take care,
  Gerd

Re: [edk2-devel] [PATCH 1/1] OvmfPkg: Store PageTablePool in TdxWorkArea

[Yao, Jiewen]

My feeling is that this mPageTablePool is an driver *internal* data.
It is not the best idea to expose an internal data structure to *public* space.

Can we have a way to move mPageTablePool to stack instead of global data area? such as input as function parameter?

Thank you
Yao Jiewen


> -----Original Message-----
> From: Gerd Hoffmann <kraxel@redhat.com>
> Sent: Friday, September 23, 2022 1:44 PM
> To: Xu, Min M <min.m.xu@intel.com>
> Cc: devel@edk2.groups.io; Aktas, Erdem <erdemaktas@google.com>;
> James Bottomley <jejb@linux.ibm.com>; Yao, Jiewen
> <jiewen.yao@intel.com>; Tom Lendacky <thomas.lendacky@amd.com>
> Subject: Re: [edk2-devel] [PATCH 1/1] OvmfPkg: Store PageTablePool in
> TdxWorkArea
> 
> On Thu, Sep 22, 2022 at 04:43:04AM +0000, Xu, Min M wrote:
> > Hi, Gerd
> > Do you have any comments on this patch?
> 
> Looks ok to me.
> 
> But I'm increasingly wondering whenever it actually was that a smart
> move to ditch the PEI phase for the IntelTdx builds.  Anything which
> can't be handled in DXE must be done in SEC, and the SEC restrictions
> seem to be a constant struggle.
> 
> We had that when adding tdx measurement support which required sha*
> hash support for SEC.  And this looks like another case ...
> 
> take care,
>   Gerd

Re: [edk2-devel] [PATCH 1/1] OvmfPkg: Store PageTablePool in TdxWorkArea

[Min Xu]

On September 27, 2022 11:01 AM, Jiewen Yao wrote:
> 
> My feeling is that this mPageTablePool is an driver *internal* data.
> It is not the best idea to expose an internal data structure to *public* space.
> 
> Can we have a way to move mPageTablePool to stack instead of global data
> area? such as input as function parameter?
> 
Yes, we can do in that way. I will submit a new version of the patch so that the community can have a review.

Thanks
Min