[PATCH v2] OvmfPkg: Create additional PML1 entries for large SEV-SNP VMs

[PATCH v2] OvmfPkg: Create additional PML1 entries for large SEV-SNP VMs

[Mikolaj Lisik]

Edk2 was failing, rather than creating more PML4 entries, when they
weren't present in the initial memory acceptance flow. Because of that
VMs with more than 512G memory were crashing. This code fixes that.

This change affects only SEV-SNP VMs.

The code was tested by successfully booting a 512G SEV-SNP VM.

Signed-off-by: Mikolaj Lisik <lisik@google.com>
---
 .../X64/PeiDxeVirtualMemory.c                 | 26 ++++++++++++-------
 1 file changed, 17 insertions(+), 9 deletions(-)

diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiDxeVirtualMemory.c b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiDxeVirtualMemory.c
index b9c0a5b25a..75c2c36bb4 100644
--- a/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiDxeVirtualMemory.c
+++ b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiDxeVirtualMemory.c
@@ -548,6 +548,7 @@ InternalMemEncryptSevCreateIdentityMap1G (
   PAGE_MAP_AND_DIRECTORY_POINTER  *PageMapLevel4Entry;
   PAGE_TABLE_1G_ENTRY             *PageDirectory1GEntry;
   UINT64                          PgTableMask;
+  UINT64                          *NewPageTable;
   UINT64                          AddressEncMask;
   BOOLEAN                         IsWpEnabled;
   RETURN_STATUS                   Status;
@@ -602,15 +603,22 @@ InternalMemEncryptSevCreateIdentityMap1G (
     PageMapLevel4Entry  = (VOID *)(Cr3BaseAddress & ~PgTableMask);
     PageMapLevel4Entry += PML4_OFFSET (PhysicalAddress);
     if (!PageMapLevel4Entry->Bits.Present) {
-      DEBUG ((
-        DEBUG_ERROR,
-        "%a:%a: bad PML4 for Physical=0x%Lx\n",
-        gEfiCallerBaseName,
-        __FUNCTION__,
-        PhysicalAddress
-        ));
-      Status = RETURN_NO_MAPPING;
-      goto Done;
+      NewPageTable = AllocatePageTableMemory (1);
+      if (NewPageTable == NULL) {
+        DEBUG ((
+          DEBUG_ERROR,
+          "%a:%a: failed to allocate a new PML4 entry\n",
+          gEfiCallerBaseName,
+          __FUNCTION__
+          ));
+        Status = RETURN_NO_MAPPING;
+        goto Done;
+      }
+      SetMem (NewPageTable, EFI_PAGE_SIZE, 0);
+      PageMapLevel4Entry->Uint64          = (UINT64)(UINTN)NewPageTable | AddressEncMask;
+      PageMapLevel4Entry->Bits.MustBeZero = 0;
+      PageMapLevel4Entry->Bits.ReadWrite  = 1;
+      PageMapLevel4Entry->Bits.Present    = 1;
     }
 
     PageDirectory1GEntry = (VOID *)(
-- 
2.39.1.456.gfc5497dd1b-goog

Re: [edk2-devel] [PATCH v2] OvmfPkg: Create additional PML1 entries for large SEV-SNP VMs

[Gerd Hoffmann]

On Thu, Jan 26, 2023 at 08:26:40PM +0000, Mikolaj Lisik via groups.io wrote:
> Edk2 was failing, rather than creating more PML4 entries, when they
> weren't present in the initial memory acceptance flow. Because of that
> VMs with more than 512G memory were crashing. This code fixes that.
> 
> This change affects only SEV-SNP VMs.
> 
> The code was tested by successfully booting a 512G SEV-SNP VM.

Acked-by: Gerd Hoffmann <kraxel@redhat.com>

Re: [PATCH v2] OvmfPkg: Create additional PML1 entries for large SEV-SNP VMs

[Lendacky, Thomas]

On 1/26/23 14:26, Mikolaj Lisik wrote:
> Edk2 was failing, rather than creating more PML4 entries, when they
> weren't present in the initial memory acceptance flow. Because of that
> VMs with more than 512G memory were crashing. This code fixes that.
> 
> This change affects only SEV-SNP VMs.
> 
> The code was tested by successfully booting a 512G SEV-SNP VM.
> 
> Signed-off-by: Mikolaj Lisik <lisik@google.com>

Acked-by: Tom Lendacky <thomas.lendacky@amd.com>

> ---
>   .../X64/PeiDxeVirtualMemory.c                 | 26 ++++++++++++-------
>   1 file changed, 17 insertions(+), 9 deletions(-)
> 
> diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiDxeVirtualMemory.c b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiDxeVirtualMemory.c
> index b9c0a5b25a..75c2c36bb4 100644
> --- a/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiDxeVirtualMemory.c
> +++ b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiDxeVirtualMemory.c
> @@ -548,6 +548,7 @@ InternalMemEncryptSevCreateIdentityMap1G (
>     PAGE_MAP_AND_DIRECTORY_POINTER  *PageMapLevel4Entry;
>     PAGE_TABLE_1G_ENTRY             *PageDirectory1GEntry;
>     UINT64                          PgTableMask;
> +  UINT64                          *NewPageTable;
>     UINT64                          AddressEncMask;
>     BOOLEAN                         IsWpEnabled;
>     RETURN_STATUS                   Status;
> @@ -602,15 +603,22 @@ InternalMemEncryptSevCreateIdentityMap1G (
>       PageMapLevel4Entry  = (VOID *)(Cr3BaseAddress & ~PgTableMask);
>       PageMapLevel4Entry += PML4_OFFSET (PhysicalAddress);
>       if (!PageMapLevel4Entry->Bits.Present) {
> -      DEBUG ((
> -        DEBUG_ERROR,
> -        "%a:%a: bad PML4 for Physical=0x%Lx\n",
> -        gEfiCallerBaseName,
> -        __FUNCTION__,
> -        PhysicalAddress
> -        ));
> -      Status = RETURN_NO_MAPPING;
> -      goto Done;
> +      NewPageTable = AllocatePageTableMemory (1);
> +      if (NewPageTable == NULL) {
> +        DEBUG ((
> +          DEBUG_ERROR,
> +          "%a:%a: failed to allocate a new PML4 entry\n",
> +          gEfiCallerBaseName,
> +          __FUNCTION__
> +          ));
> +        Status = RETURN_NO_MAPPING;
> +        goto Done;
> +      }
> +      SetMem (NewPageTable, EFI_PAGE_SIZE, 0);
> +      PageMapLevel4Entry->Uint64          = (UINT64)(UINTN)NewPageTable | AddressEncMask;
> +      PageMapLevel4Entry->Bits.MustBeZero = 0;
> +      PageMapLevel4Entry->Bits.ReadWrite  = 1;
> +      PageMapLevel4Entry->Bits.Present    = 1;
>       }
>   
>       PageDirectory1GEntry = (VOID *)(

Re: [PATCH v2] OvmfPkg: Create additional PML1 entries for large SEV-SNP VMs

[Lendacky, Thomas]

On 1/26/23 14:26, Mikolaj Lisik wrote:
> Edk2 was failing, rather than creating more PML4 entries, when they
> weren't present in the initial memory acceptance flow. Because of that
> VMs with more than 512G memory were crashing. This code fixes that.
> 
> This change affects only SEV-SNP VMs.
> 
> The code was tested by successfully booting a 512G SEV-SNP VM.
> 
> Signed-off-by: Mikolaj Lisik <lisik@google.com>

I don't see where this was merged. Both Gerd and I acked it back in 
January, was there a problem with it?

Thanks,
Tom

> ---
>   .../X64/PeiDxeVirtualMemory.c                 | 26 ++++++++++++-------
>   1 file changed, 17 insertions(+), 9 deletions(-)
> 
> diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiDxeVirtualMemory.c b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiDxeVirtualMemory.c
> index b9c0a5b25a..75c2c36bb4 100644
> --- a/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiDxeVirtualMemory.c
> +++ b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiDxeVirtualMemory.c
> @@ -548,6 +548,7 @@ InternalMemEncryptSevCreateIdentityMap1G (
>     PAGE_MAP_AND_DIRECTORY_POINTER  *PageMapLevel4Entry;
>     PAGE_TABLE_1G_ENTRY             *PageDirectory1GEntry;
>     UINT64                          PgTableMask;
> +  UINT64                          *NewPageTable;
>     UINT64                          AddressEncMask;
>     BOOLEAN                         IsWpEnabled;
>     RETURN_STATUS                   Status;
> @@ -602,15 +603,22 @@ InternalMemEncryptSevCreateIdentityMap1G (
>       PageMapLevel4Entry  = (VOID *)(Cr3BaseAddress & ~PgTableMask);
>       PageMapLevel4Entry += PML4_OFFSET (PhysicalAddress);
>       if (!PageMapLevel4Entry->Bits.Present) {
> -      DEBUG ((
> -        DEBUG_ERROR,
> -        "%a:%a: bad PML4 for Physical=0x%Lx\n",
> -        gEfiCallerBaseName,
> -        __FUNCTION__,
> -        PhysicalAddress
> -        ));
> -      Status = RETURN_NO_MAPPING;
> -      goto Done;
> +      NewPageTable = AllocatePageTableMemory (1);
> +      if (NewPageTable == NULL) {
> +        DEBUG ((
> +          DEBUG_ERROR,
> +          "%a:%a: failed to allocate a new PML4 entry\n",
> +          gEfiCallerBaseName,
> +          __FUNCTION__
> +          ));
> +        Status = RETURN_NO_MAPPING;
> +        goto Done;
> +      }
> +      SetMem (NewPageTable, EFI_PAGE_SIZE, 0);
> +      PageMapLevel4Entry->Uint64          = (UINT64)(UINTN)NewPageTable | AddressEncMask;
> +      PageMapLevel4Entry->Bits.MustBeZero = 0;
> +      PageMapLevel4Entry->Bits.ReadWrite  = 1;
> +      PageMapLevel4Entry->Bits.Present    = 1;
>       }
>   
>       PageDirectory1GEntry = (VOID *)(

Re: [edk2-devel] [PATCH v2] OvmfPkg: Create additional PML1 entries for large SEV-SNP VMs

[Ard Biesheuvel]

On Fri, 2 Jun 2023 at 23:20, Lendacky, Thomas via groups.io
<thomas.lendacky=amd.com@groups.io> wrote:
>
> On 1/26/23 14:26, Mikolaj Lisik wrote:
> > Edk2 was failing, rather than creating more PML4 entries, when they
> > weren't present in the initial memory acceptance flow. Because of that
> > VMs with more than 512G memory were crashing. This code fixes that.
> >
> > This change affects only SEV-SNP VMs.
> >
> > The code was tested by successfully booting a 512G SEV-SNP VM.
> >
> > Signed-off-by: Mikolaj Lisik <lisik@google.com>
>
> I don't see where this was merged. Both Gerd and I acked it back in
> January, was there a problem with it?
>

I wasn't cc'ed.

I've queued this up now.



>
> > ---
> >   .../X64/PeiDxeVirtualMemory.c                 | 26 ++++++++++++-------
> >   1 file changed, 17 insertions(+), 9 deletions(-)
> >
> > diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiDxeVirtualMemory.c b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiDxeVirtualMemory.c
> > index b9c0a5b25a..75c2c36bb4 100644
> > --- a/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiDxeVirtualMemory.c
> > +++ b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiDxeVirtualMemory.c
> > @@ -548,6 +548,7 @@ InternalMemEncryptSevCreateIdentityMap1G (
> >     PAGE_MAP_AND_DIRECTORY_POINTER  *PageMapLevel4Entry;
> >     PAGE_TABLE_1G_ENTRY             *PageDirectory1GEntry;
> >     UINT64                          PgTableMask;
> > +  UINT64                          *NewPageTable;
> >     UINT64                          AddressEncMask;
> >     BOOLEAN                         IsWpEnabled;
> >     RETURN_STATUS                   Status;
> > @@ -602,15 +603,22 @@ InternalMemEncryptSevCreateIdentityMap1G (
> >       PageMapLevel4Entry  = (VOID *)(Cr3BaseAddress & ~PgTableMask);
> >       PageMapLevel4Entry += PML4_OFFSET (PhysicalAddress);
> >       if (!PageMapLevel4Entry->Bits.Present) {
> > -      DEBUG ((
> > -        DEBUG_ERROR,
> > -        "%a:%a: bad PML4 for Physical=0x%Lx\n",
> > -        gEfiCallerBaseName,
> > -        __FUNCTION__,
> > -        PhysicalAddress
> > -        ));
> > -      Status = RETURN_NO_MAPPING;
> > -      goto Done;
> > +      NewPageTable = AllocatePageTableMemory (1);
> > +      if (NewPageTable == NULL) {
> > +        DEBUG ((
> > +          DEBUG_ERROR,
> > +          "%a:%a: failed to allocate a new PML4 entry\n",
> > +          gEfiCallerBaseName,
> > +          __FUNCTION__
> > +          ));
> > +        Status = RETURN_NO_MAPPING;
> > +        goto Done;
> > +      }
> > +      SetMem (NewPageTable, EFI_PAGE_SIZE, 0);
> > +      PageMapLevel4Entry->Uint64          = (UINT64)(UINTN)NewPageTable | AddressEncMask;
> > +      PageMapLevel4Entry->Bits.MustBeZero = 0;
> > +      PageMapLevel4Entry->Bits.ReadWrite  = 1;
> > +      PageMapLevel4Entry->Bits.Present    = 1;
> >       }
> >
> >       PageDirectory1GEntry = (VOID *)(
>
>
> 
>
>

Re: [edk2-devel] [PATCH v2] OvmfPkg: Create additional PML1 entries for large SEV-SNP VMs

[Ard Biesheuvel]

On Sat, 3 Jun 2023 at 00:49, Ard Biesheuvel <ardb@kernel.org> wrote:
>
> On Fri, 2 Jun 2023 at 23:20, Lendacky, Thomas via groups.io
> <thomas.lendacky=amd.com@groups.io> wrote:
> >
> > On 1/26/23 14:26, Mikolaj Lisik wrote:
> > > Edk2 was failing, rather than creating more PML4 entries, when they
> > > weren't present in the initial memory acceptance flow. Because of that
> > > VMs with more than 512G memory were crashing. This code fixes that.
> > >
> > > This change affects only SEV-SNP VMs.
> > >
> > > The code was tested by successfully booting a 512G SEV-SNP VM.
> > >
> > > Signed-off-by: Mikolaj Lisik <lisik@google.com>
> >
> > I don't see where this was merged. Both Gerd and I acked it back in
> > January, was there a problem with it?
> >
>
> I wasn't cc'ed.
>
> I've queued this up now.
>

PR here
https://github.com/tianocore/edk2/pull/4473

CI is spuriously failing again.

(cc Mike + Mike + Sean)

Could someone please push the merge button on this? Thanks.

Re: [edk2-devel] [PATCH v2] OvmfPkg: Create additional PML1 entries for large SEV-SNP VMs

[Lendacky, Thomas]

On 6/2/23 17:49, Ard Biesheuvel wrote:
> On Fri, 2 Jun 2023 at 23:20, Lendacky, Thomas via groups.io
> <thomas.lendacky=amd.com@groups.io> wrote:
>>
>> On 1/26/23 14:26, Mikolaj Lisik wrote:
>>> Edk2 was failing, rather than creating more PML4 entries, when they
>>> weren't present in the initial memory acceptance flow. Because of that
>>> VMs with more than 512G memory were crashing. This code fixes that.
>>>
>>> This change affects only SEV-SNP VMs.
>>>
>>> The code was tested by successfully booting a 512G SEV-SNP VM.
>>>
>>> Signed-off-by: Mikolaj Lisik <lisik@google.com>
>>
>> I don't see where this was merged. Both Gerd and I acked it back in
>> January, was there a problem with it?
>>
> 
> I wasn't cc'ed.

Ah, I hadn't noticed that you weren't cc'd...

> 
> I've queued this up now.

Thanks, Ard!

Tom

> 
> 
> 
>>
>>> ---
>>>    .../X64/PeiDxeVirtualMemory.c                 | 26 ++++++++++++-------
>>>    1 file changed, 17 insertions(+), 9 deletions(-)
>>>
>>> diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiDxeVirtualMemory.c b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiDxeVirtualMemory.c
>>> index b9c0a5b25a..75c2c36bb4 100644
>>> --- a/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiDxeVirtualMemory.c
>>> +++ b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiDxeVirtualMemory.c
>>> @@ -548,6 +548,7 @@ InternalMemEncryptSevCreateIdentityMap1G (
>>>      PAGE_MAP_AND_DIRECTORY_POINTER  *PageMapLevel4Entry;
>>>      PAGE_TABLE_1G_ENTRY             *PageDirectory1GEntry;
>>>      UINT64                          PgTableMask;
>>> +  UINT64                          *NewPageTable;
>>>      UINT64                          AddressEncMask;
>>>      BOOLEAN                         IsWpEnabled;
>>>      RETURN_STATUS                   Status;
>>> @@ -602,15 +603,22 @@ InternalMemEncryptSevCreateIdentityMap1G (
>>>        PageMapLevel4Entry  = (VOID *)(Cr3BaseAddress & ~PgTableMask);
>>>        PageMapLevel4Entry += PML4_OFFSET (PhysicalAddress);
>>>        if (!PageMapLevel4Entry->Bits.Present) {
>>> -      DEBUG ((
>>> -        DEBUG_ERROR,
>>> -        "%a:%a: bad PML4 for Physical=0x%Lx\n",
>>> -        gEfiCallerBaseName,
>>> -        __FUNCTION__,
>>> -        PhysicalAddress
>>> -        ));
>>> -      Status = RETURN_NO_MAPPING;
>>> -      goto Done;
>>> +      NewPageTable = AllocatePageTableMemory (1);
>>> +      if (NewPageTable == NULL) {
>>> +        DEBUG ((
>>> +          DEBUG_ERROR,
>>> +          "%a:%a: failed to allocate a new PML4 entry\n",
>>> +          gEfiCallerBaseName,
>>> +          __FUNCTION__
>>> +          ));
>>> +        Status = RETURN_NO_MAPPING;
>>> +        goto Done;
>>> +      }
>>> +      SetMem (NewPageTable, EFI_PAGE_SIZE, 0);
>>> +      PageMapLevel4Entry->Uint64          = (UINT64)(UINTN)NewPageTable | AddressEncMask;
>>> +      PageMapLevel4Entry->Bits.MustBeZero = 0;
>>> +      PageMapLevel4Entry->Bits.ReadWrite  = 1;
>>> +      PageMapLevel4Entry->Bits.Present    = 1;
>>>        }
>>>
>>>        PageDirectory1GEntry = (VOID *)(
>>
>>
>> 
>>
>>