Re: [PATCH v8 42/46] UefiCpuPkg: Allow AP booting under SEV-ES

["Lendacky, Thomas" <thomas.lendacky@amd.com>]

On 6/1/20 2:28 AM, Dong, Eric wrote:
> Hi Tom,

Hi Eric,

> 
>> -----Original Message-----
>> From: Tom Lendacky <thomas.lendacky@amd.com>
>> Sent: Wednesday, May 20, 2020 5:51 AM
>> To: devel@edk2.groups.io
>> Cc: Justen, Jordan L <jordan.l.justen@intel.com>; Laszlo Ersek
>> <lersek@redhat.com>; Ard Biesheuvel <ard.biesheuvel@linaro.org>; Kinney,
>> Michael D <michael.d.kinney@intel.com>; Gao, Liming
>> <liming.gao@intel.com>; Dong, Eric <eric.dong@intel.com>; Ni, Ray
>> <ray.ni@intel.com>; Brijesh Singh <brijesh.singh@amd.com>
>> Subject: [PATCH v8 42/46] UefiCpuPkg: Allow AP booting under SEV-ES
>>
>> BZ: https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.tianocore.org%2Fshow_bug.cgi%3Fid%3D2198&amp;data=02%7C01%7Cthomas.lendacky%40amd.com%7Ccb3255e534ac46be29e108d805fd6f22%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637265933337671439&amp;sdata=Rc%2BTzZ4%2FcJnmX%2FMOEnY4ixy45vPKVo4DCBVjsmdGogk%3D&amp;reserved=0
>>
>> Typically, an AP is booted using the INIT-SIPI-SIPI sequence. This sequence is
>> intercepted by the hypervisor, which sets the AP's registers to the values
>> requested by the sequence. At that point, the hypervisor can start the AP,
>> which will then begin execution at the appropriate location.
>>
>> Under SEV-ES, AP booting presents some challenges since the hypervisor is
>> not allowed to alter the AP's register state. In this situation, we have to
>> distinguish between the AP's first boot and AP's subsequent boots.
>>
>> First boot:
>>  Once the AP's register state has been defined (which is before the guest  is
>> first booted) it cannot be altered. Should the hypervisor attempt to  alter the
>> register state, the change would be detected by the hardware  and the
>> VMRUN instruction would fail. Given this, the first boot for the  AP is
>> required to begin execution with this initial register state, which  is typically
>> the reset vector. This prevents the BSP from directing the  AP startup
>> location through the INIT-SIPI-SIPI sequence.
>>
>>  To work around this, the firmware will provide a build time reserved area
>> that can be used as the initial IP value. The hypervisor can extract this
>> location value by checking for the SEV-ES reset block GUID that must be
>> located 48-bytes from the end of the firmware. The format of the SEV-ES
>> reset block area is:
>>
>>    0x00 - 0x01 - SEV-ES Reset IP
>>    0x02 - 0x03 - SEV-ES Reset CS Segment Base[31:16]
>>    0x04 - 0x05 - Size of the SEV-ES reset block
>>    0x06 - 0x15 - SEV-ES Reset Block GUID
>>                    (00f771de-1a7e-4fcb-890e-68c77e2fb44e)
>>
>>    The total size is 22 bytes. Any expansion to this block must be done
>>    by adding new values before existing values.
>>
>>  The hypervisor will use the IP and CS values obtained from the SEV-ES  reset
>> block to set as the AP's initial values. The CS Segment Base  represents the
>> upper 16 bits of the CS segment base and must be left  shifted by 16 bits to
>> form the complete CS segment base value.
>>
>>  Before booting the AP for the first time, the BSP must initialize the  SEV-ES
>> reset area. This consists of programming a FAR JMP instruction  to the
>> contents of a memory location that is also located in the SEV-ES  reset area.
>> The BSP must program the IP and CS values for the FAR JMP  based on values
>> drived from the INIT-SIPI-SIPI sequence.
>>
>> Subsequent boots:
>>  Again, the hypervisor cannot alter the AP register state, so a method is
>> required to take the AP out of halt state and redirect it to the desired  IP
>> location. If it is determined that the AP is running in an SEV-ES  guest, then
>> instead of calling CpuSleep(), a VMGEXIT is issued with the  AP Reset Hold
>> exit code (0x80000004). The hypervisor will put the AP in  a halt state, waiting
>> for an INIT-SIPI-SIPI sequence. Once the sequence  is recognized, the
>> hypervisor will resume the AP. At this point the AP  must transition from the
>> current 64-bit long mode down to 16-bit real  mode and begin executing at
>> the derived location from the INIT-SIPI-SIPI  sequence.
>>
>>  Another change is around the area of obtaining the (x2)APIC ID during AP
>> startup. During AP startup, the AP can't take a #VC exception before the  AP
>> has established a stack. However, the AP stack is set by using the  (x2)APIC ID,
>> which is obtained through CPUID instructions. A CPUID  instruction will cause
>> a #VC, so a different method must be used. The  GHCB protocol supports a
>> method to obtain CPUID information from the  hypervisor through the GHCB
>> MSR. This method does not require a stack,  so it is used to obtain the
>> necessary CPUID information to determine the  (x2)APIC ID.
>>
>> The new 16-bit protected mode GDT entry is used in order to transition from
>> 64-bit long mode down to 16-bit real mode.
>>
>> A new assembler routine is created that takes the AP from 64-bit long mode
>> to 16-bit real mode.  This is located under 1MB in memory and transitions
>> from 64-bit long mode to 32-bit compatibility mode to 16-bit protected mode
>> and finally 16-bit real mode.
>>
>> Cc: Eric Dong <eric.dong@intel.com>
>> Cc: Ray Ni <ray.ni@intel.com>
>> Cc: Laszlo Ersek <lersek@redhat.com>
>> Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
>> ---
>>  UefiCpuPkg/Library/MpInitLib/DxeMpInitLib.inf |   3 +
>>  UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf |   3 +
>>  UefiCpuPkg/Library/MpInitLib/MpLib.h          |  60 ++++
>>  UefiCpuPkg/Library/MpInitLib/DxeMpLib.c       |  70 +++-
>>  UefiCpuPkg/Library/MpInitLib/MpLib.c          | 312 +++++++++++++++++-
>>  UefiCpuPkg/Library/MpInitLib/PeiMpLib.c       |  19 ++
>>  UefiCpuPkg/PiSmmCpuDxeSmm/X64/SmmFuncsArch.c  |   2 +-
>>  UefiCpuPkg/Library/MpInitLib/Ia32/MpEqu.inc   |   2 +-
>>  .../Library/MpInitLib/Ia32/MpFuncs.nasm       |  15 +
>>  UefiCpuPkg/Library/MpInitLib/X64/MpEqu.inc    |   4 +-
>>  UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm | 239 ++++++++++++++
>>  11 files changed, 714 insertions(+), 15 deletions(-)
>>
>> diff --git a/UefiCpuPkg/Library/MpInitLib/DxeMpInitLib.inf
>> b/UefiCpuPkg/Library/MpInitLib/DxeMpInitLib.inf
>> index 583276595619..1771575c69c1 100644
>> --- a/UefiCpuPkg/Library/MpInitLib/DxeMpInitLib.inf
>> +++ b/UefiCpuPkg/Library/MpInitLib/DxeMpInitLib.inf
>> @@ -52,6 +52,7 @@ [LibraryClasses]
>>    DebugAgentLib
>>    SynchronizationLib
>>    PcdLib
>> +  VmgExitLib
>>
>>  [Protocols]
>>    gEfiTimerArchProtocolGuid                     ## SOMETIMES_CONSUMES
>> @@ -72,4 +73,6 @@ [Pcd]
>>    gUefiCpuPkgTokenSpaceGuid.PcdCpuApTargetCstate                       ##
>> SOMETIMES_CONSUMES
>>
>> gUefiCpuPkgTokenSpaceGuid.PcdCpuApStatusCheckIntervalInMicroSeconds
>> ## CONSUMES
>>    gUefiCpuPkgTokenSpaceGuid.PcdSevEsIsEnabled                          ##
>> CONSUMES
>> +  gUefiCpuPkgTokenSpaceGuid.PcdSevEsWorkAreaBase                       ##
>> SOMETIMES_CONSUMES
>>    gEfiMdeModulePkgTokenSpaceGuid.PcdCpuStackGuard                      ##
>> CONSUMES
>> +  gEfiMdeModulePkgTokenSpaceGuid.PcdGhcbBase                           ##
>> CONSUMES
>> diff --git a/UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf
>> b/UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf
>> index 4b3d39fbf36c..34abf25d43cd 100644
>> --- a/UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf
>> +++ b/UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf
>> @@ -51,6 +51,7 @@ [LibraryClasses]
>>    SynchronizationLib
>>    PeiServicesLib
>>    PcdLib
>> +  VmgExitLib
>>
>>  [Pcd]
>>    gUefiCpuPkgTokenSpaceGuid.PcdCpuMaxLogicalProcessorNumber        ##
>> CONSUMES
>> @@ -62,6 +63,8 @@ [Pcd]
>>    gUefiCpuPkgTokenSpaceGuid.PcdCpuApLoopMode                       ##
>> CONSUMES
>>    gUefiCpuPkgTokenSpaceGuid.PcdCpuApTargetCstate                   ##
>> SOMETIMES_CONSUMES
>>    gUefiCpuPkgTokenSpaceGuid.PcdSevEsIsEnabled                      ## CONSUMES
>> +  gUefiCpuPkgTokenSpaceGuid.PcdSevEsWorkAreaBase                   ##
>> SOMETIMES_CONSUMES
>> +  gEfiMdeModulePkgTokenSpaceGuid.PcdGhcbBase                       ##
>> CONSUMES
>>
>>  [Ppis]
>>    gEdkiiPeiShadowMicrocodePpiGuid        ## SOMETIMES_CONSUMES
>> diff --git a/UefiCpuPkg/Library/MpInitLib/MpLib.h
>> b/UefiCpuPkg/Library/MpInitLib/MpLib.h
>> index 5b46c295b6b2..f0cbb3763b5d 100644
>> --- a/UefiCpuPkg/Library/MpInitLib/MpLib.h
>> +++ b/UefiCpuPkg/Library/MpInitLib/MpLib.h
>> @@ -173,6 +173,11 @@ typedef struct {
>>    UINT8             *RelocateApLoopFuncAddress;
>>    UINTN             RelocateApLoopFuncSize;
>>    UINTN             ModeTransitionOffset;
>> +  UINTN             SwitchToRealSize;
>> +  UINTN             SwitchToRealOffset;
>> +  UINTN             SwitchToRealNoNxOffset;
>> +  UINTN             SwitchToRealPM16ModeOffset;
>> +  UINTN             SwitchToRealPM16ModeSize;
>>  } MP_ASSEMBLY_ADDRESS_MAP;
>>
>>  typedef struct _CPU_MP_DATA  CPU_MP_DATA; @@ -211,6 +216,8 @@
>> typedef struct {
>>    // Enable5LevelPaging indicates whether 5-level paging is enabled in long
>> mode.
>>    //
>>    BOOLEAN               Enable5LevelPaging;
>> +  BOOLEAN               SevEsIsEnabled;
>> +  UINTN                 GhcbBase;
>>  } MP_CPU_EXCHANGE_INFO;
>>
>>  #pragma pack()
>> @@ -257,6 +264,7 @@ struct _CPU_MP_DATA {
>>    UINT8                          ApLoopMode;
>>    UINT8                          ApTargetCState;
>>    UINT16                         PmCodeSegment;
>> +  UINT16                         Pm16CodeSegment;
>>    CPU_AP_DATA                    *CpuData;
>>    volatile MP_CPU_EXCHANGE_INFO  *MpCpuExchangeInfo;
>>
>> @@ -278,8 +286,47 @@ struct _CPU_MP_DATA {
>>    BOOLEAN                        WakeUpByInitSipiSipi;
>>
>>    BOOLEAN                        SevEsIsEnabled;
>> +  UINTN                          SevEsAPBuffer;
>> +  UINTN                          SevEsAPResetStackStart;
>> +  CPU_MP_DATA                    *NewCpuMpData;
>> +
>> +  UINT64                         GhcbBase;
>>  };
>>
>> +#define AP_RESET_STACK_SIZE 64
>> +
>> +#pragma pack(1)
>> +
>> +typedef struct {
>> +  UINT8   InsnBuffer[8];
>> +  UINT16  Rip;
>> +  UINT16  Segment;
>> +} SEV_ES_AP_JMP_FAR;
>> +
>> +#pragma pack()
>> +
>> +/**
>> +  Assembly code to move an AP from long mode to real mode.
>> +
>> +  Move an AP from long mode to real mode in preparation to invoking
>> + the reset vector.  This is used for SEV-ES guests where a hypervisor
>> + is not allowed to set the CS and RIP to point to the reset vector.
>> +
>> +  @param[in]  BufferStart  The reset vector target.
>> +  @param[in]  Code16       16-bit protected mode code segment value.
>> +  @param[in]  Code32       32-bit protected mode code segment value.
>> +  @param[in]  StackStart   The start of a stack to be used for transitioning
>> +                           from long mode to real mode.
>> +**/
>> +typedef
>> +VOID
>> +(EFIAPI AP_RESET) (
>> +  IN UINTN    BufferStart,
>> +  IN UINT16   Code16,
>> +  IN UINT16   Code32,
>> +  IN UINTN    StackStart
>> +  );
>> +
>>  extern EFI_GUID mCpuInitMpLibHobGuid;
>>
>>  /**
>> @@ -385,6 +432,19 @@ GetModeTransitionBuffer (
>>    IN UINTN                BufferSize
>>    );
>>
>> +/**
>> +  Return the address of the SEV-ES AP jump table.
>> +
>> +  This buffer is required in order for an SEV-ES guest to transition
>> + from  UEFI into an OS.
>> +
>> +  @retval other   Return SEV-ES AP jump table buffer
>> +**/
>> +UINTN
>> +GetSevEsAPMemory (
>> +  VOID
>> +  );
>> +
>>  /**
>>    This function will be called by BSP to wakeup AP.
>>
>> diff --git a/UefiCpuPkg/Library/MpInitLib/DxeMpLib.c
>> b/UefiCpuPkg/Library/MpInitLib/DxeMpLib.c
>> index 8ccddf8e9f9c..19527300ff3a 100644
>> --- a/UefiCpuPkg/Library/MpInitLib/DxeMpLib.c
>> +++ b/UefiCpuPkg/Library/MpInitLib/DxeMpLib.c
>> @@ -12,6 +12,8 @@
>>  #include <Library/UefiBootServicesTableLib.h>
>>  #include <Library/DebugAgentLib.h>
>>  #include <Library/DxeServicesTableLib.h>
>> +#include <Register/Amd/Fam17Msr.h>
>> +#include <Register/Amd/Ghcb.h>
>>
>>  #include <Protocol/Timer.h>
>>
>> @@ -144,6 +146,39 @@ GetModeTransitionBuffer (
>>    return (UINTN)StartAddress;
>>  }
>>
>> +/**
>> +  Return the address of the SEV-ES AP jump table.
>> +
>> +  This buffer is required in order for an SEV-ES guest to transition
>> + from  UEFI into an OS.
>> +
>> +  @retval other   Return SEV-ES AP jump table buffer
>> +**/
>> +UINTN
>> +GetSevEsAPMemory (
>> +  VOID
>> +  )
>> +{
>> +  EFI_STATUS            Status;
>> +  EFI_PHYSICAL_ADDRESS  StartAddress;
>> +
>> +  //
>> +  // Allocate 1 page for AP jump table page  //  StartAddress =
>> + BASE_4GB - 1;  Status = gBS->AllocatePages (
>> +                  AllocateMaxAddress,
>> +                  EfiReservedMemoryType,
>> +                  1,
>> +                  &StartAddress
>> +                  );
>> +  ASSERT_EFI_ERROR (Status);
>> +
>> +  DEBUG ((DEBUG_INFO, "Dxe: SevEsAPMemory = %lx\n", (UINTN)
>> + StartAddress));
>> +
>> +  return (UINTN) StartAddress;
>> +}
>> +
>>  /**
>>    Checks APs status and updates APs status if needed.
>>
>> @@ -218,6 +253,38 @@ CheckApsStatus (
>>    }
>>  }
>>
>> +/**
>> +  Get Protected mode code segment with 16-bit default addressing
>> +  from current GDT table.
>> +
>> +  @return  Protected mode 16-bit code segment value.
>> +**/
>> +UINT16
>> +GetProtectedMode16CS (
>> +  VOID
>> +  )
>> +{
>> +  IA32_DESCRIPTOR          GdtrDesc;
>> +  IA32_SEGMENT_DESCRIPTOR  *GdtEntry;
>> +  UINTN                    GdtEntryCount;
>> +  UINT16                   Index;
>> +
>> +  Index = (UINT16) -1;
>> +  AsmReadGdtr (&GdtrDesc);
>> +  GdtEntryCount = (GdtrDesc.Limit + 1) / sizeof
>> +(IA32_SEGMENT_DESCRIPTOR);
>> +  GdtEntry = (IA32_SEGMENT_DESCRIPTOR *) GdtrDesc.Base;
>> +  for (Index = 0; Index < GdtEntryCount; Index++) {
>> +    if (GdtEntry->Bits.L == 0) {
>> +      if (GdtEntry->Bits.Type > 8 && GdtEntry->Bits.DB == 0) {
>> +        break;
>> +      }
>> +    }
>> +    GdtEntry++;
>> +  }
>> +  ASSERT (Index != GdtEntryCount);
>> +  return Index * 8;
>> +}
>> +
>>  /**
>>    Get Protected mode code segment from current GDT table.
>>
>> @@ -238,7 +305,7 @@ GetProtectedModeCS (
>>    GdtEntry = (IA32_SEGMENT_DESCRIPTOR *) GdtrDesc.Base;
>>    for (Index = 0; Index < GdtEntryCount; Index++) {
>>      if (GdtEntry->Bits.L == 0) {
>> -      if (GdtEntry->Bits.Type > 8 && GdtEntry->Bits.L == 0) {
>> +      if (GdtEntry->Bits.Type > 8 && GdtEntry->Bits.DB == 1) {
>>          break;
>>        }
>>      }
>> @@ -300,6 +367,7 @@ MpInitChangeApLoopCallback (
>>
>>    CpuMpData = GetCpuMpData ();
>>    CpuMpData->PmCodeSegment = GetProtectedModeCS ();
>> +  CpuMpData->Pm16CodeSegment = GetProtectedMode16CS ();
>>    CpuMpData->ApLoopMode = PcdGet8 (PcdCpuApLoopMode);
>>    mNumberToFinish = CpuMpData->CpuCount - 1;
>>    WakeUpAP (CpuMpData, TRUE, 0, RelocateApLoop, NULL, TRUE); diff --git
>> a/UefiCpuPkg/Library/MpInitLib/MpLib.c
>> b/UefiCpuPkg/Library/MpInitLib/MpLib.c
>> index a8b605f569bf..aeab575bb525 100644
>> --- a/UefiCpuPkg/Library/MpInitLib/MpLib.c
>> +++ b/UefiCpuPkg/Library/MpInitLib/MpLib.c
>> @@ -9,6 +9,9 @@
>>  **/
>>
>>  #include "MpLib.h"
>> +#include <Library/VmgExitLib.h>
>> +#include <Register/Amd/Fam17Msr.h>
>> +#include <Register/Amd/Ghcb.h>
>>
>>  EFI_GUID mCpuInitMpLibHobGuid = CPU_INIT_MP_LIB_HOB_GUID;
>>
>> @@ -314,6 +317,14 @@ GetApLoopMode (
>>        //
>>        ApLoopMode = ApInHltLoop;
>>      }
>> +
>> +    if (PcdGetBool (PcdSevEsIsEnabled)) {
>> +      //
>> +      // For SEV-ES, force AP in Hlt-loop mode in order to use the GHCB
>> +      // protocol for starting APs
>> +      //
>> +      ApLoopMode = ApInHltLoop;
>> +    }
>>    }
>>
>>    if (ApLoopMode != ApInMwaitLoop) {
>> @@ -610,6 +621,112 @@ InitializeApData (
>>    SetApState (&CpuMpData->CpuData[ProcessorNumber], CpuStateIdle);  }
>>
>> +/**
>> +  Get Protected mode code segment with 16-bit default addressing
>> +  from current GDT table.
>> +
>> +  @return  Protected mode 16-bit code segment value.
>> +**/
>> +STATIC
>> +UINT16
>> +GetProtectedMode16CS (
>> +  VOID
>> +  )
>> +{
>> +  IA32_DESCRIPTOR          GdtrDesc;
>> +  IA32_SEGMENT_DESCRIPTOR  *GdtEntry;
>> +  UINTN                    GdtEntryCount;
>> +  UINT16                   Index;
>> +
>> +  Index = (UINT16) -1;
>> +  AsmReadGdtr (&GdtrDesc);
>> +  GdtEntryCount = (GdtrDesc.Limit + 1) / sizeof
>> +(IA32_SEGMENT_DESCRIPTOR);
>> +  GdtEntry = (IA32_SEGMENT_DESCRIPTOR *) GdtrDesc.Base;
>> +  for (Index = 0; Index < GdtEntryCount; Index++) {
>> +    if (GdtEntry->Bits.L == 0 &&
>> +        GdtEntry->Bits.DB == 0 &&
>> +        GdtEntry->Bits.Type > 8) {
>> +      break;
>> +    }
>> +    GdtEntry++;
>> +  }
>> +  ASSERT (Index != GdtEntryCount);
>> +  return Index * 8;
>> +}
>> +
>> +/**
>> +  Get Protected mode code segment with 32-bit default addressing
>> +  from current GDT table.
>> +
>> +  @return  Protected mode 32-bit code segment value.
>> +**/
>> +STATIC
>> +UINT16
>> +GetProtectedMode32CS (
>> +  VOID
>> +  )
>> +{
>> +  IA32_DESCRIPTOR          GdtrDesc;
>> +  IA32_SEGMENT_DESCRIPTOR  *GdtEntry;
>> +  UINTN                    GdtEntryCount;
>> +  UINT16                   Index;
>> +
>> +  Index = (UINT16) -1;
>> +  AsmReadGdtr (&GdtrDesc);
>> +  GdtEntryCount = (GdtrDesc.Limit + 1) / sizeof
>> +(IA32_SEGMENT_DESCRIPTOR);
>> +  GdtEntry = (IA32_SEGMENT_DESCRIPTOR *) GdtrDesc.Base;
>> +  for (Index = 0; Index < GdtEntryCount; Index++) {
>> +    if (GdtEntry->Bits.L == 0 &&
>> +        GdtEntry->Bits.DB == 1 &&
>> +        GdtEntry->Bits.Type > 8) {
>> +      break;
>> +    }
>> +    GdtEntry++;
>> +  }
>> +  ASSERT (Index != GdtEntryCount);
>> +  return Index * 8;
>> +}
>> +
>> +/**
>> +  Reset an AP when in SEV-ES mode.
>> +
>> +  If successful, this function never returns.
>> +
>> +  @param[in] Ghcb                 Pointer to the GHCB
>> +  @param[in] CpuMpData            Pointer to CPU MP Data
>> +
>> +**/
>> +STATIC
>> +VOID
>> +MpInitLibSevEsAPReset (
>> +  IN GHCB                         *Ghcb,
>> +  IN CPU_MP_DATA                  *CpuMpData
>> +  )
>> +{
>> +  UINT16           Code16, Code32;
>> +  AP_RESET         *APResetFn;
>> +  UINTN            BufferStart;
>> +  UINTN            StackStart;
>> +
>> +  Code16 = GetProtectedMode16CS ();
>> +  Code32 = GetProtectedMode32CS ();
>> +
>> +  if (CpuMpData->WakeupBufferHigh != 0) {
>> +    APResetFn = (AP_RESET *) (CpuMpData->WakeupBufferHigh +
>> + CpuMpData->AddressMap.SwitchToRealNoNxOffset);
>> +  } else {
>> +    APResetFn = (AP_RESET *) (CpuMpData->MpCpuExchangeInfo-
>>> BufferStart
>> + + CpuMpData->AddressMap.SwitchToRealOffset);
>> +  }
>> +
>> +  BufferStart = CpuMpData->MpCpuExchangeInfo->BufferStart;
>> +  StackStart = CpuMpData->SevEsAPResetStackStart -
>> +                 (AP_RESET_STACK_SIZE * GetApicId ());
>> +
>> +  //
>> +  // This call never returns.
>> +  //
>> +  APResetFn (BufferStart, Code16, Code32, StackStart); }
>> +
>>  /**
>>    This function will be called from AP reset code if BSP uses WakeUpAP.
>>
>> @@ -671,7 +788,14 @@ ApWakeupFunction (
>>        InitializeApData (CpuMpData, ProcessorNumber, BistData,
>> ApTopOfStack);
>>        ApStartupSignalBuffer = CpuMpData-
>>> CpuData[ProcessorNumber].StartupApSignal;
>>
>> -      InterlockedDecrement ((UINT32 *) &CpuMpData->MpCpuExchangeInfo-
>>> NumApsExecuting);
>> +      //
>> +      // Delay decrementing the APs executing count when SEV-ES is enabled
>> +      // to allow the APs to issue an AP_RESET_HOLD before the BSP possibly
>> +      // performs another INIT-SIPI-SIPI sequence.
>> +      //
>> +      if (!CpuMpData->SevEsIsEnabled) {
>> +        InterlockedDecrement ((UINT32 *) &CpuMpData-
>>> MpCpuExchangeInfo->NumApsExecuting);
>> +      }
>>      } else {
>>        //
>>        // Execute AP function if AP is ready @@ -778,7 +902,52 @@
>> ApWakeupFunction (
>>        //
>>        while (TRUE) {
>>          DisableInterrupts ();
>> -        CpuSleep ();
>> +        if (CpuMpData->SevEsIsEnabled) {
>> +          MSR_SEV_ES_GHCB_REGISTER  Msr;
>> +          GHCB                      *Ghcb;
>> +          UINT64                    Status;
>> +          BOOLEAN                   DoDecrement;
>> +
>> +          if (CpuMpData->InitFlag == ApInitConfig) {
>> +            DoDecrement = TRUE;
>> +          }
>> +
>> +          while (TRUE) {
>> +            Msr.GhcbPhysicalAddress = AsmReadMsr64 (MSR_SEV_ES_GHCB);
>> +            Ghcb = Msr.Ghcb;
>> +
>> +            VmgInit (Ghcb);
>> +
>> +            if (DoDecrement) {
>> +              DoDecrement = FALSE;
>> +
>> +              //
>> +              // Perform the delayed decrement just before issuing the first
>> +              // VMGEXIT with AP_RESET_HOLD.
>> +              //
>> +              InterlockedDecrement ((UINT32 *) &CpuMpData-
>>> MpCpuExchangeInfo->NumApsExecuting);
> 
> 1. Why needs to postpose the CpuMpData->MpCpuExchangeInfo->NumApsExecuting update here?

This was done as a result of reports from backports of these patches. The
reading of the MSR and clearing of the GHCB took enough time that the BSP
was able to send another SIPI before the AP reached the point where the
VmgExit was issued. As a result, the AP would enter the AP reset hold and
not return because the SIPI will have already been issued.

> 
>> +            }
>> +
>> +            Status = VmgExit (Ghcb, SVM_EXIT_AP_RESET_HOLD, 0, 0);
> 
> 2. The AP will enter Halt mode after this call? Not continue the execute of the below code? I'm not clear how the AP works for this case.

The AP will be put in a pseudo-HLT mode by the VMM. A subsequent SIPI
request will result in the VMM making the vCPU runnable again, causing
this VMGEXIT to return and continue processing the code below.

> 
>> +            if ((Status == 0) && (Ghcb->SaveArea.SwExitInfo2 != 0)) {
>> +              VmgDone (Ghcb);
>> +              break;
>> +            }
>> +
>> +            VmgDone (Ghcb);
>> +          }
>> +
>> +          //
>> +          // Awakened in a new phase? Use the new CpuMpData
>> +          //
>> +          if (CpuMpData->NewCpuMpData) {
>> +            CpuMpData = CpuMpData->NewCpuMpData;
>> +          }
>> +
>> +          MpInitLibSevEsAPReset (Ghcb, CpuMpData);
> 
> 3. With this function, this AP will be wake up and execute the new procedure? 

Yes, the MpInitLibSevEsAPReset () function will cause the AP to transition
back to 16-bit mode and execute starting at the address from the SIPI
request that awoke it - imitating the INIT-SIPI-SIPI sequence.

I can add some more comments to the MpInitLibSevEsAPReset () function to
better describe this.

Thanks,
Tom

> 
> Thanks,
> Eric
> 
>> +        } else {
>> +          CpuSleep ();
>> +        }
>>          CpuPause ();
>>        }
>>      }
>> @@ -891,6 +1060,9 @@ FillExchangeInfoData (
>>    ExchangeInfo->Enable5LevelPaging = (BOOLEAN) (Cr4.Bits.LA57 == 1);
>>    DEBUG ((DEBUG_INFO, "%a: 5-Level Paging = %d\n", gEfiCallerBaseName,
>> ExchangeInfo->Enable5LevelPaging));
>>
>> +  ExchangeInfo->SevEsIsEnabled  = CpuMpData->SevEsIsEnabled;
>> +  ExchangeInfo->GhcbBase        = (UINTN) CpuMpData->GhcbBase;
>> +
>>    //
>>    // Get the BSP's data of GDT and IDT
>>    //
>> @@ -917,8 +1089,9 @@ FillExchangeInfoData (
>>    // EfiBootServicesCode to avoid page fault if NX memory protection is
>> enabled.
>>    //
>>    if (CpuMpData->WakeupBufferHigh != 0) {
>> -    Size = CpuMpData->AddressMap.RendezvousFunnelSize -
>> -           CpuMpData->AddressMap.ModeTransitionOffset;
>> +    Size = CpuMpData->AddressMap.RendezvousFunnelSize +
>> +             CpuMpData->AddressMap.SwitchToRealSize -
>> +             CpuMpData->AddressMap.ModeTransitionOffset;
>>      CopyMem (
>>        (VOID *)CpuMpData->WakeupBufferHigh,
>>        CpuMpData->AddressMap.RendezvousFunnelAddress + @@ -971,7
>> +1144,8 @@ BackupAndPrepareWakeupBuffer(
>>    CopyMem (
>>      (VOID *) CpuMpData->WakeupBuffer,
>>      (VOID *) CpuMpData->AddressMap.RendezvousFunnelAddress,
>> -    CpuMpData->AddressMap.RendezvousFunnelSize
>> +    CpuMpData->AddressMap.RendezvousFunnelSize +
>> +      CpuMpData->AddressMap.SwitchToRealSize
>>      );
>>  }
>>
>> @@ -992,6 +1166,44 @@ RestoreWakeupBuffer(
>>      );
>>  }
>>
>> +/**
>> +  Calculate the size of the reset stack.
>> +
>> +  @retval                 Total amount of memory required for stacks
>> +**/
>> +STATIC
>> +UINTN
>> +GetApResetStackSize (
>> +  VOID
>> +  )
>> +{
>> +  return AP_RESET_STACK_SIZE *
>> +PcdGet32(PcdCpuMaxLogicalProcessorNumber);
>> +}
>> +
>> +/**
>> +  Calculate the size of the reset vector.
>> +
>> +  @param[in]  AddressMap  The pointer to Address Map structure.
>> +
>> +  @retval                 Total amount of memory required for the AP reset area
>> +**/
>> +STATIC
>> +UINTN
>> +GetApResetVectorSize (
>> +  IN MP_ASSEMBLY_ADDRESS_MAP  *AddressMap
>> +  )
>> +{
>> +  UINTN  Size;
>> +
>> +  Size = ALIGN_VALUE (AddressMap->RendezvousFunnelSize +
>> +                        AddressMap->SwitchToRealSize +
>> +                        sizeof (MP_CPU_EXCHANGE_INFO),
>> +                      CPU_STACK_ALIGNMENT);  Size +=
>> + GetApResetStackSize ();
>> +
>> +  return Size;
>> +}
>> +
>>  /**
>>    Allocate reset vector buffer.
>>
>> @@ -1005,16 +1217,22 @@ AllocateResetVector (
>>    UINTN           ApResetVectorSize;
>>
>>    if (CpuMpData->WakeupBuffer == (UINTN) -1) {
>> -    ApResetVectorSize = CpuMpData->AddressMap.RendezvousFunnelSize +
>> -                          sizeof (MP_CPU_EXCHANGE_INFO);
>> +    ApResetVectorSize = GetApResetVectorSize (&CpuMpData-
>>> AddressMap);
>>
>>      CpuMpData->WakeupBuffer      = GetWakeupBuffer (ApResetVectorSize);
>>      CpuMpData->MpCpuExchangeInfo = (MP_CPU_EXCHANGE_INFO *)
>> (UINTN)
>> -                    (CpuMpData->WakeupBuffer + CpuMpData-
>>> AddressMap.RendezvousFunnelSize);
>> +                    (CpuMpData->WakeupBuffer +
>> +                       CpuMpData->AddressMap.RendezvousFunnelSize +
>> +                       CpuMpData->AddressMap.SwitchToRealSize);
>>      CpuMpData->WakeupBufferHigh  = GetModeTransitionBuffer (
>> -                                    CpuMpData->AddressMap.RendezvousFunnelSize -
>> +                                    CpuMpData->AddressMap.RendezvousFunnelSize +
>> +
>> + CpuMpData->AddressMap.SwitchToRealSize -
>>                                      CpuMpData->AddressMap.ModeTransitionOffset
>>                                      );
>> +    //
>> +    // The reset stack starts at the end of the buffer.
>> +    //
>> +    CpuMpData->SevEsAPResetStackStart = CpuMpData->WakeupBuffer +
>> + ApResetVectorSize;
>>    }
>>    BackupAndPrepareWakeupBuffer (CpuMpData);  } @@ -1029,7 +1247,31
>> @@ FreeResetVector (
>>    IN CPU_MP_DATA              *CpuMpData
>>    )
>>  {
>> -  RestoreWakeupBuffer (CpuMpData);
>> +  //
>> +  // If SEV-ES is enabled, the reset area is needed for AP parking and
>> +  // and AP startup in the OS, so the reset area is reserved. Do not
>> +  // perform the restore as this will overwrite memory which has data
>> +  // needed by SEV-ES.
>> +  //
>> +  if (!CpuMpData->SevEsIsEnabled) {
>> +    RestoreWakeupBuffer (CpuMpData);
>> +  }
>> +}
>> +
>> +/**
>> +  Allocate the SEV-ES AP jump table buffer.
>> +
>> +  @param[in, out]  CpuMpData  The pointer to CPU MP Data structure.
>> +**/
>> +VOID
>> +AllocateSevEsAPMemory (
>> +  IN OUT CPU_MP_DATA          *CpuMpData
>> +  )
>> +{
>> +  if (CpuMpData->SevEsAPBuffer == (UINTN) -1) {
>> +    CpuMpData->SevEsAPBuffer =
>> +      CpuMpData->SevEsIsEnabled ? GetSevEsAPMemory () : 0;
>> +  }
>>  }
>>
>>  /**
>> @@ -1066,6 +1308,7 @@ WakeUpAP (
>>        CpuMpData->InitFlag   != ApInitDone) {
>>      ResetVectorRequired = TRUE;
>>      AllocateResetVector (CpuMpData);
>> +    AllocateSevEsAPMemory (CpuMpData);
>>      FillExchangeInfoData (CpuMpData);
>>      SaveLocalApicTimerSetting (CpuMpData);
>>    }
>> @@ -1102,6 +1345,50 @@ WakeUpAP (
>>        }
>>      }
>>      if (ResetVectorRequired) {
>> +      //
>> +      // For SEV-ES, the initial AP boot address will be defined by
>> +      // PcdSevEsWorkAreaBase. The Segment/Rip must be the jump address
>> +      // from the original INIT-SIPI-SIPI.
>> +      //
>> +      if (CpuMpData->SevEsIsEnabled) {
>> +        SEV_ES_AP_JMP_FAR *JmpFar;
>> +        UINT32            Offset, InsnByte;
>> +        UINT8             LoNib, HiNib;
>> +
>> +        JmpFar = (SEV_ES_AP_JMP_FAR *) FixedPcdGet32
>> (PcdSevEsWorkAreaBase);
>> +        ASSERT (JmpFar != NULL);
>> +
>> +        //
>> +        // Obtain the address of the Segment/Rip location in the workarea.
>> +        // This will be set to a value derived from the SIPI vector and will
>> +        // be the memory address used for the far jump below.
>> +        //
>> +        Offset = FixedPcdGet32 (PcdSevEsWorkAreaBase);
>> +        Offset += sizeof (JmpFar->InsnBuffer);
>> +        LoNib = (UINT8) Offset;
>> +        HiNib = (UINT8) (Offset >> 8);
>> +
>> +        //
>> +        // Program the workarea (which is the initial AP boot address) with
>> +        // far jump to the SIPI vector (where XX and YY represent the
>> +        // address of where the SIPI vector is stored.
>> +        //
>> +        //   JMP FAR [CS:XXYY] => 2E FF 2E YY XX
>> +        //
>> +        InsnByte = 0;
>> +        JmpFar->InsnBuffer[InsnByte++] = 0x2E;  // CS override prefix
>> +        JmpFar->InsnBuffer[InsnByte++] = 0xFF;  // JMP (FAR)
>> +        JmpFar->InsnBuffer[InsnByte++] = 0x2E;  // ModRM (JMP memory
>> location)
>> +        JmpFar->InsnBuffer[InsnByte++] = LoNib; // YY offset ...
>> +        JmpFar->InsnBuffer[InsnByte++] = HiNib; // XX offset ...
>> +
>> +        //
>> +        // Program the Segment/Rip based on the SIPI vector (always at least
>> +        // 16-byte aligned, so Rip is set to 0).
>> +        //
>> +        JmpFar->Rip = 0;
>> +        JmpFar->Segment = (UINT16) (ExchangeInfo->BufferStart >> 4);
>> +      }
>>        //
>>        // Wakeup all APs
>>        //
>> @@ -1669,7 +1956,7 @@ MpInitLibInitialize (
>>    ASSERT (MaxLogicalProcessorNumber != 0);
>>
>>    AsmGetAddressMap (&AddressMap);
>> -  ApResetVectorSize = AddressMap.RendezvousFunnelSize + sizeof
>> (MP_CPU_EXCHANGE_INFO);
>> +  ApResetVectorSize = GetApResetVectorSize (&AddressMap);
>>    ApStackSize = PcdGet32(PcdCpuApStackSize);
>>    ApLoopMode  = GetApLoopMode (&MonitorFilterSize);
>>
>> @@ -1728,6 +2015,8 @@ MpInitLibInitialize (
>>    CpuMpData->CpuInfoInHob     = (UINT64) (UINTN) (CpuMpData->CpuData
>> + MaxLogicalProcessorNumber);
>>    InitializeSpinLock(&CpuMpData->MpLock);
>>    CpuMpData->SevEsIsEnabled = PcdGetBool (PcdSevEsIsEnabled);
>> +  CpuMpData->SevEsAPBuffer  = (UINTN) -1;
>> +  CpuMpData->GhcbBase       = PcdGet64 (PcdGhcbBase);
>>
>>    //
>>    // Make sure no memory usage outside of the allocated buffer.
>> @@ -1786,6 +2075,7 @@ MpInitLibInitialize (
>>      // APs have been wakeup before, just get the CPU Information
>>      // from HOB
>>      //
>> +    OldCpuMpData->NewCpuMpData = CpuMpData;
>>      CpuMpData->CpuCount  = OldCpuMpData->CpuCount;
>>      CpuMpData->BspNumber = OldCpuMpData->BspNumber;
>>      CpuMpData->CpuInfoInHob = OldCpuMpData->CpuInfoInHob; diff --git
>> a/UefiCpuPkg/Library/MpInitLib/PeiMpLib.c
>> b/UefiCpuPkg/Library/MpInitLib/PeiMpLib.c
>> index a548fed23fa7..e17a351e5cfd 100644
>> --- a/UefiCpuPkg/Library/MpInitLib/PeiMpLib.c
>> +++ b/UefiCpuPkg/Library/MpInitLib/PeiMpLib.c
>> @@ -280,6 +280,25 @@ GetModeTransitionBuffer (
>>    return 0;
>>  }
>>
>> +/**
>> +  Return the address of the SEV-ES AP jump table.
>> +
>> +  This buffer is required in order for an SEV-ES guest to transition
>> + from  UEFI into an OS.
>> +
>> +  @retval other   Return SEV-ES AP jump table buffer
>> +**/
>> +UINTN
>> +GetSevEsAPMemory (
>> +  VOID
>> +  )
>> +{
>> +  //
>> +  // PEI phase doesn't need to do such transition. So simply return 0.
>> +  //
>> +  return 0;
>> +}
>> +
>>  /**
>>    Checks APs status and updates APs status if needed.
>>
>> diff --git a/UefiCpuPkg/PiSmmCpuDxeSmm/X64/SmmFuncsArch.c
>> b/UefiCpuPkg/PiSmmCpuDxeSmm/X64/SmmFuncsArch.c
>> index 6298571e29b2..28f8e8e133e5 100644
>> --- a/UefiCpuPkg/PiSmmCpuDxeSmm/X64/SmmFuncsArch.c
>> +++ b/UefiCpuPkg/PiSmmCpuDxeSmm/X64/SmmFuncsArch.c
>> @@ -121,7 +121,7 @@ GetProtectedModeCS (
>>    GdtEntry = (IA32_SEGMENT_DESCRIPTOR *) GdtrDesc.Base;
>>    for (Index = 0; Index < GdtEntryCount; Index++) {
>>      if (GdtEntry->Bits.L == 0) {
>> -      if (GdtEntry->Bits.Type > 8 && GdtEntry->Bits.L == 0) {
>> +      if (GdtEntry->Bits.Type > 8 && GdtEntry->Bits.DB == 1) {
>>          break;
>>        }
>>      }
>> diff --git a/UefiCpuPkg/Library/MpInitLib/Ia32/MpEqu.inc
>> b/UefiCpuPkg/Library/MpInitLib/Ia32/MpEqu.inc
>> index efb1bc2bf7cb..4f5a7c859a56 100644
>> --- a/UefiCpuPkg/Library/MpInitLib/Ia32/MpEqu.inc
>> +++ b/UefiCpuPkg/Library/MpInitLib/Ia32/MpEqu.inc
>> @@ -19,7 +19,7 @@ CPU_SWITCH_STATE_IDLE         equ        0
>>  CPU_SWITCH_STATE_STORED       equ        1
>>  CPU_SWITCH_STATE_LOADED       equ        2
>>
>> -LockLocation                  equ        (RendezvousFunnelProcEnd -
>> RendezvousFunnelProcStart)
>> +LockLocation                  equ        (SwitchToRealProcEnd -
>> RendezvousFunnelProcStart)
>>  StackStartAddressLocation     equ        LockLocation + 04h
>>  StackSizeLocation             equ        LockLocation + 08h
>>  ApProcedureLocation           equ        LockLocation + 0Ch
>> diff --git a/UefiCpuPkg/Library/MpInitLib/Ia32/MpFuncs.nasm
>> b/UefiCpuPkg/Library/MpInitLib/Ia32/MpFuncs.nasm
>> index b74046b76af3..309d53bf3b37 100644
>> --- a/UefiCpuPkg/Library/MpInitLib/Ia32/MpFuncs.nasm
>> +++ b/UefiCpuPkg/Library/MpInitLib/Ia32/MpFuncs.nasm
>> @@ -215,6 +215,16 @@ CProcedureInvoke:
>>      jmp        $                 ; Never reach here
>>  RendezvousFunnelProcEnd:
>>
>> +;----------------------------------------------------------------------
>> +---------------
>> +;SwitchToRealProc procedure follows.
>> +;NOT USED IN 32 BIT MODE.
>> +;----------------------------------------------------------------------
>> +---------------
>> +global ASM_PFX(SwitchToRealProc)
>> +ASM_PFX(SwitchToRealProc):
>> +SwitchToRealProcStart:
>> +    jmp        $                 ; Never reach here
>> +SwitchToRealProcEnd:
>> +
>>  ;-------------------------------------------------------------------------------------
>>  ;  AsmRelocateApLoop (MwaitSupport, ApTargetCState, PmCodeSegment,
>> TopOfApStack, CountTofinish);
>>  ;-------------------------------------------------------------------------------------
>> @@ -263,6 +273,11 @@ ASM_PFX(AsmGetAddressMap):
>>      mov        dword [ebx + 0Ch], AsmRelocateApLoopStart
>>      mov        dword [ebx + 10h], AsmRelocateApLoopEnd -
>> AsmRelocateApLoopStart
>>      mov        dword [ebx + 14h], Flat32Start - RendezvousFunnelProcStart
>> +    mov        dword [ebx + 18h], SwitchToRealProcEnd -
>> SwitchToRealProcStart       ; SwitchToRealSize
>> +    mov        dword [ebx + 1Ch], SwitchToRealProcStart -
>> RendezvousFunnelProcStart ; SwitchToRealOffset
>> +    mov        dword [ebx + 20h], SwitchToRealProcStart - Flat32Start               ;
>> SwitchToRealNoNxOffset
>> +    mov        dword [ebx + 24h], 0                                                 ;
>> SwitchToRealPM16ModeOffset
>> +    mov        dword [ebx + 28h], 0                                                 ;
>> SwitchToRealPM16ModeSize
>>
>>      popad
>>      ret
>> diff --git a/UefiCpuPkg/Library/MpInitLib/X64/MpEqu.inc
>> b/UefiCpuPkg/Library/MpInitLib/X64/MpEqu.inc
>> index 58ef369342a7..c92daaaffd6b 100644
>> --- a/UefiCpuPkg/Library/MpInitLib/X64/MpEqu.inc
>> +++ b/UefiCpuPkg/Library/MpInitLib/X64/MpEqu.inc
>> @@ -19,7 +19,7 @@ CPU_SWITCH_STATE_IDLE         equ        0
>>  CPU_SWITCH_STATE_STORED       equ        1
>>  CPU_SWITCH_STATE_LOADED       equ        2
>>
>> -LockLocation                  equ        (RendezvousFunnelProcEnd -
>> RendezvousFunnelProcStart)
>> +LockLocation                  equ        (SwitchToRealProcEnd -
>> RendezvousFunnelProcStart)
>>  StackStartAddressLocation     equ        LockLocation + 08h
>>  StackSizeLocation             equ        LockLocation + 10h
>>  ApProcedureLocation           equ        LockLocation + 18h
>> @@ -41,3 +41,5 @@ ModeTransitionSegmentLocation       equ  LockLocation +
>> 98h
>>  ModeHighMemoryLocation              equ  LockLocation + 9Ah
>>  ModeHighSegmentLocation             equ  LockLocation + 9Eh
>>  Enable5LevelPagingLocation          equ  LockLocation + 0A0h
>> +SevEsIsEnabledLocation              equ  LockLocation + 0A1h
>> +GhcbBaseLocation                    equ  LockLocation + 0A2h
>> diff --git a/UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm
>> b/UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm
>> index 87f2523e856f..6956b408d004 100644
>> --- a/UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm
>> +++ b/UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm
>> @@ -184,9 +184,97 @@ Releaselock:
>>      add        edi, StackStartAddressLocation
>>      add        rax, qword [edi]
>>      mov        rsp, rax
>> +
>> +    lea        edi, [esi + SevEsIsEnabledLocation]
>> +    cmp        byte [edi], 1        ; SevEsIsEnabled
>> +    jne        CProcedureInvoke
>> +
>> +    ;
>> +    ; program GHCB
>> +    ;   Each page after the GHCB is a per-CPU page, so the calculation
>> programs
>> +    ;   a GHCB to be every 8KB.
>> +    ;
>> +    mov        eax, SIZE_4KB
>> +    shl        eax, 1                            ; EAX = SIZE_4K * 2
>> +    mov        ecx, ebx
>> +    mul        ecx                               ; EAX = SIZE_4K * 2 * CpuNumber
>> +    mov        edi, esi
>> +    add        edi, GhcbBaseLocation
>> +    add        rax, qword [edi]
>> +    mov        rdx, rax
>> +    shr        rdx, 32
>> +    mov        rcx, 0xc0010130
>> +    wrmsr
>>      jmp        CProcedureInvoke
>>
>>  GetApicId:
>> +    lea        edi, [esi + SevEsIsEnabledLocation]
>> +    cmp        byte [edi], 1        ; SevEsIsEnabled
>> +    jne        DoCpuid
>> +
>> +    ;
>> +    ; Since we don't have a stack yet, we can't take a #VC
>> +    ; exception. Use the GHCB protocol to perform the CPUID
>> +    ; calls.
>> +    ;
>> +    mov        rcx, 0xc0010130
>> +    rdmsr
>> +    shl        rdx, 32
>> +    or         rax, rdx
>> +    mov        rdi, rax             ; RDI now holds the original GHCB GPA
>> +
>> +    mov        rdx, 0               ; CPUID function 0
>> +    mov        rax, 0               ; RAX register requested
>> +    or         rax, 4
>> +    wrmsr
>> +    rep vmmcall
>> +    rdmsr
>> +    cmp        edx, 0bh
>> +    jb         NoX2ApicSevEs        ; CPUID level below
>> CPUID_EXTENDED_TOPOLOGY
>> +
>> +    mov        rdx, 0bh             ; CPUID function 0x0b
>> +    mov        rax, 040000000h      ; RBX register requested
>> +    or         rax, 4
>> +    wrmsr
>> +    rep vmmcall
>> +    rdmsr
>> +    test       edx, 0ffffh
>> +    jz         NoX2ApicSevEs        ; CPUID.0BH:EBX[15:0] is zero
>> +
>> +    mov        rdx, 0bh             ; CPUID function 0x0b
>> +    mov        rax, 0c0000000h      ; RDX register requested
>> +    or         rax, 4
>> +    wrmsr
>> +    rep vmmcall
>> +    rdmsr
>> +
>> +    ; Processor is x2APIC capable; 32-bit x2APIC ID is now in EDX
>> +    jmp        RestoreGhcb
>> +
>> +NoX2ApicSevEs:
>> +    ; Processor is not x2APIC capable, so get 8-bit APIC ID
>> +    mov        rdx, 1               ; CPUID function 1
>> +    mov        rax, 040000000h      ; RBX register requested
>> +    or         rax, 4
>> +    wrmsr
>> +    rep vmmcall
>> +    rdmsr
>> +    shr        edx, 24
>> +
>> +RestoreGhcb:
>> +    mov        rbx, rdx             ; Save x2APIC/APIC ID
>> +
>> +    mov        rdx, rdi             ; RDI holds the saved GHCB GPA
>> +    shr        rdx, 32
>> +    mov        eax, edi
>> +    wrmsr
>> +
>> +    mov        rdx, rbx
>> +
>> +    ; x2APIC ID or APIC ID is in EDX
>> +    jmp        GetProcessorNumber
>> +
>> +DoCpuid:
>>      mov        eax, 0
>>      cpuid
>>      cmp        eax, 0bh
>> @@ -253,12 +341,158 @@ CProcedureInvoke:
>>
>>  RendezvousFunnelProcEnd:
>>
>> +;----------------------------------------------------------------------
>> +---------------
>> +;SwitchToRealProc procedure follows.
>> +;ALSO THIS PROCEDURE IS EXECUTED BY APs TRANSITIONING TO 16 BIT
>> MODE.
>> +HENCE THIS PROC ;IS IN MACHINE CODE.
>> +;  SwitchToRealProc (UINTN BufferStart, UINT16 Code16, UINT16 Code32,
>> +UINTN StackStart) ;  rcx - Buffer Start ;  rdx - Code16 Selector Offset
>> +;  r8  - Code32 Selector Offset ;  r9  - Stack Start
>> +;----------------------------------------------------------------------
>> +---------------
>> +global ASM_PFX(SwitchToRealProc)
>> +ASM_PFX(SwitchToRealProc):
>> +SwitchToRealProcStart:
>> +BITS 64
>> +    cli
>> +
>> +    ;
>> +    ; Get RDX reset value before changing stacks since the
>> +    ; new stack won't be able to accomodate a #VC exception.
>> +    ;
>> +    push       rax
>> +    push       rbx
>> +    push       rcx
>> +    push       rdx
>> +
>> +    mov        rax, 1
>> +    cpuid
>> +    mov        rsi, rax                    ; Save off the reset value for RDX
>> +
>> +    pop        rdx
>> +    pop        rcx
>> +    pop        rbx
>> +    pop        rax
>> +
>> +    ;
>> +    ; Establish stack below 1MB
>> +    ;
>> +    mov        rsp, r9
>> +
>> +    ;
>> +    ; Push ultimate Reset Vector onto the stack
>> +    ;
>> +    mov        rax, rcx
>> +    shr        rax, 4
>> +    push       word 0x0002                 ; RFLAGS
>> +    push       ax                          ; CS
>> +    push       word 0x0000                 ; RIP
>> +    push       word 0x0000                 ; For alignment, will be discarded
>> +
>> +    ;
>> +    ; Get address of "16-bit operand size" label
>> +    ;
>> +    lea        rbx, [PM16Mode]
>> +
>> +    ;
>> +    ; Push addresses used to change to compatibility mode
>> +    ;
>> +    lea        rax, [CompatMode]
>> +    push       r8
>> +    push       rax
>> +
>> +    ;
>> +    ; Clear R8 - R15, for reset, before going into 32-bit mode
>> +    ;
>> +    xor        r8, r8
>> +    xor        r9, r9
>> +    xor        r10, r10
>> +    xor        r11, r11
>> +    xor        r12, r12
>> +    xor        r13, r13
>> +    xor        r14, r14
>> +    xor        r15, r15
>> +
>> +    ;
>> +    ; Far return into 32-bit mode
>> +    ;
>> +o64 retf
>> +
>> +BITS 32
>> +CompatMode:
>> +    ;
>> +    ; Set up stack to prepare for exiting protected mode
>> +    ;
>> +    push       edx                         ; Code16 CS
>> +    push       ebx                         ; PM16Mode label address
>> +
>> +    ;
>> +    ; Disable paging
>> +    ;
>> +    mov        eax, cr0                    ; Read CR0
>> +    btr        eax, 31                     ; Set PG=0
>> +    mov        cr0, eax                    ; Write CR0
>> +
>> +    ;
>> +    ; Disable long mode
>> +    ;
>> +    mov        ecx, 0c0000080h             ; EFER MSR number
>> +    rdmsr                                  ; Read EFER
>> +    btr        eax, 8                      ; Set LME=0
>> +    wrmsr                                  ; Write EFER
>> +
>> +    ;
>> +    ; Disable PAE
>> +    ;
>> +    mov        eax, cr4                    ; Read CR4
>> +    btr        eax, 5                      ; Set PAE=0
>> +    mov        cr4, eax                    ; Write CR4
>> +
>> +    mov        edx, esi                    ; Restore RDX reset value
>> +
>> +    ;
>> +    ; Switch to 16-bit operand size
>> +    ;
>> +    retf
>> +
>> +BITS 16
>> +    ;
>> +    ; At entry to this label
>> +    ;   - RDX will have its reset value
>> +    ;   - On the top of the stack
>> +    ;     - Alignment data (two bytes) to be discarded
>> +    ;     - IP for Real Mode (two bytes)
>> +    ;     - CS for Real Mode (two bytes)
>> +    ;
>> +PM16Mode:
>> +    mov        eax, cr0                    ; Read CR0
>> +    btr        eax, 0                      ; Set PE=0
>> +    mov        cr0, eax                    ; Write CR0
>> +
>> +    pop        ax                          ; Discard alignment data
>> +
>> +    ;
>> +    ; Clear registers (except RDX and RSP) before going into 16-bit mode
>> +    ;
>> +    xor        eax, eax
>> +    xor        ebx, ebx
>> +    xor        ecx, ecx
>> +    xor        esi, esi
>> +    xor        edi, edi
>> +    xor        ebp, ebp
>> +
>> +    iret
>> +
>> +SwitchToRealProcEnd:
>> +
>>  ;-------------------------------------------------------------------------------------
>>  ;  AsmRelocateApLoop (MwaitSupport, ApTargetCState, PmCodeSegment,
>> TopOfApStack, CountTofinish);
>>  ;-------------------------------------------------------------------------------------
>>  global ASM_PFX(AsmRelocateApLoop)
>>  ASM_PFX(AsmRelocateApLoop):
>>  AsmRelocateApLoopStart:
>> +BITS 64
>>      cli                          ; Disable interrupt before switching to 32-bit mode
>>      mov        rax, [rsp + 40]   ; CountTofinish
>>      lock dec   dword [rax]       ; (*CountTofinish)--
>> @@ -324,6 +558,11 @@ ASM_PFX(AsmGetAddressMap):
>>      mov        qword [rcx + 18h], rax
>>      mov        qword [rcx + 20h], AsmRelocateApLoopEnd -
>> AsmRelocateApLoopStart
>>      mov        qword [rcx + 28h], Flat32Start - RendezvousFunnelProcStart
>> +    mov        qword [rcx + 30h], SwitchToRealProcEnd -
>> SwitchToRealProcStart          ; SwitchToRealSize
>> +    mov        qword [rcx + 38h], SwitchToRealProcStart -
>> RendezvousFunnelProcStart    ; SwitchToRealOffset
>> +    mov        qword [rcx + 40h], SwitchToRealProcStart - Flat32Start                  ;
>> SwitchToRealNoNxOffset
>> +    mov        qword [rcx + 48h], PM16Mode -
>> RendezvousFunnelProcStart                 ; SwitchToRealPM16ModeOffset
>> +    mov        qword [rcx + 50h], SwitchToRealProcEnd - PM16Mode                       ;
>> SwitchToRealPM16ModeSize
>>      ret
>>
>>  ;-------------------------------------------------------------------------------------
>> --
>> 2.17.1
>