From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from us-smtp-1.mimecast.com (us-smtp-1.mimecast.com [205.139.110.120]) by mx.groups.io with SMTP id smtpd.web10.9669.1580981715328027682 for ; Thu, 06 Feb 2020 01:35:15 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=O2SdcGDa; spf=pass (domain: redhat.com, ip: 205.139.110.120, mailfrom: lersek@redhat.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1580981714; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LorZmYCLhv0eZjfzOf4lDRRf1YUGdjihv1SL3zujDMA=; b=O2SdcGDagxtMD5Nnod6UBdJspZLdsQASHSihXNsFBOQz7tSdSjXem72WJMf8nQismiMil3 jeSEaVHipnO8EXPu5S1lsWfLw7VrDo+ILS9DOCqdNf3ibPtv65eGooUuZ2Cd8K+9e/AHsy jcu9Gwoz23/90aMCyKAhL+OdBOgaHPM= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-280-MtvFjaNnNW6S2Z-Y7EwveA-1; Thu, 06 Feb 2020 04:35:10 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id F02471081FA1; Thu, 6 Feb 2020 09:35:08 +0000 (UTC) Received: from lacos-laptop-7.usersys.redhat.com (ovpn-117-20.ams2.redhat.com [10.36.117.20]) by smtp.corp.redhat.com (Postfix) with ESMTP id 48B681001B07; Thu, 6 Feb 2020 09:35:06 +0000 (UTC) Subject: Re: [edk2-devel] [PATCH v4 37/40] UefiCpuPkg: Allow AP booting under SEV-ES To: devel@edk2.groups.io, thomas.lendacky@amd.com Cc: Jordan Justen , Ard Biesheuvel , Michael D Kinney , Liming Gao , Eric Dong , Ray Ni , Brijesh Singh References: From: "Laszlo Ersek" Message-ID: <0aa6e1d9-338c-bc5f-8400-46389c1697df@redhat.com> Date: Thu, 6 Feb 2020 10:35:05 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: MtvFjaNnNW6S2Z-Y7EwveA-1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit On 02/05/20 00:01, Lendacky, Thomas wrote: > BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2198 > > Typically, an AP is booted using the INIT-SIPI-SIPI sequence. This > sequence is intercepted by the hypervisor, which sets the AP's registers > to the values requested by the sequence. At that point, the hypervisor can > start the AP, which will then begin execution at the appropriate location. > > Under SEV-ES, AP booting presents some challenges since the hypervisor is > not allowed to alter the AP's register state. In this situation, we have > to distinguish between the AP's first boot and AP's subsequent boots. > > First boot: > Once the AP's register state has been defined (which is before the guest > is first booted) it cannot be altered. Should the hypervisor attempt to > alter the register state, the change would be detected by the hardware > and the VMRUN instruction would fail. Given this, the first boot for the > AP is required to begin execution with this initial register state, which > is typically the reset vector. This prevents the BSP from directing the > AP startup location through the INIT-SIPI-SIPI sequence. > > To work around this, the firmware will provide a build time reserved area > that can be used as the initial IP value. The hypervisor can extract this > location value by checking for the SEV-ES reset block GUID that must be > located 48-bytes from the end of the firmware. The format of the SEV-ES > reset block area is: > > 0x00 - 0x01 - SEV-ES Reset IP > 0x02 - 0x03 - SEV-ES Reset CS Segment Base[32:16] Slight typo here I think: in the next patch, you correctly write [31:16]. Thanks Laszlo > 0x04 - 0x05 - Size of the SEV-ES reset block > 0x06 - 0x15 - SEV-ES Reset Block GUID > (00f771de-1a7e-4fcb-890e-68c77e2fb44e) > > The total size is 22 bytes. Any expansion to this block must be done > by adding new values before existing values. > > The hypervisor will use the IP and CS values obtained from the SEV-ES > reset block to set as the AP's initial values. The CS Segment Base > represents the upper 16 bits of the CS segment base and must be left > shifted by 16 bits to form the complete CS segment base value. > > Before booting the AP for the first time, the BSP must initialize the > SEV-ES reset area. This consists of programming a FAR JMP instruction > to the contents of a memory location that is also located in the SEV-ES > reset area. The BSP must program the IP and CS values for the FAR JMP > based on values drived from the INIT-SIPI-SIPI sequence. > > Subsequent boots: > Again, the hypervisor cannot alter the AP register state, so a method is > required to take the AP out of halt state and redirect it to the desired > IP location. If it is determined that the AP is running in an SEV-ES > guest, then instead of calling CpuSleep(), a VMGEXIT is issued with the > AP Reset Hold exit code (0x80000004). The hypervisor will put the AP in > a halt state, waiting for an INIT-SIPI-SIPI sequence. Once the sequence > is recognized, the hypervisor will resume the AP. At this point the AP > must transition from the current 64-bit long mode down to 16-bit real > mode and begin executing at the derived location from the INIT-SIPI-SIPI > sequence. > > Another change is around the area of obtaining the (x2)APIC ID during AP > startup. During AP startup, the AP can't take a #VC exception before the > AP has established a stack. However, the AP stack is set by using the > (x2)APIC ID, which is obtained through CPUID instructions. A CPUID > instruction will cause a #VC, so a different method must be used. The > GHCB protocol supports a method to obtain CPUID information from the > hypervisor through the GHCB MSR. This method does not require a stack, > so it is used to obtain the necessary CPUID information to determine the > (x2)APIC ID. > > The new 16-bit protected mode GDT entry is used in order to transition > from 64-bit long mode down to 16-bit real mode. > > A new assembler routine is created that takes the AP from 64-bit long mode > to 16-bit real mode. This is located under 1MB in memory and transitions > from 64-bit long mode to 32-bit compatibility mode to 16-bit protected > mode and finally 16-bit real mode. > > Cc: Eric Dong > Cc: Ray Ni > Cc: Laszlo Ersek > Signed-off-by: Tom Lendacky > --- > UefiCpuPkg/Library/MpInitLib/DxeMpInitLib.inf | 3 + > UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf | 3 + > UefiCpuPkg/Library/MpInitLib/MpLib.h | 60 ++++ > UefiCpuPkg/Library/MpInitLib/DxeMpLib.c | 70 ++++- > UefiCpuPkg/Library/MpInitLib/MpLib.c | 256 +++++++++++++++++- > UefiCpuPkg/Library/MpInitLib/PeiMpLib.c | 19 ++ > UefiCpuPkg/PiSmmCpuDxeSmm/X64/SmmFuncsArch.c | 2 +- > UefiCpuPkg/Library/MpInitLib/Ia32/MpEqu.inc | 2 +- > .../Library/MpInitLib/Ia32/MpFuncs.nasm | 15 + > UefiCpuPkg/Library/MpInitLib/X64/MpEqu.inc | 4 +- > UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm | 239 ++++++++++++++++ > 11 files changed, 659 insertions(+), 14 deletions(-) > > diff --git a/UefiCpuPkg/Library/MpInitLib/DxeMpInitLib.inf b/UefiCpuPkg/Library/MpInitLib/DxeMpInitLib.inf > index 2c26f20c1972..c52951651851 100644 > --- a/UefiCpuPkg/Library/MpInitLib/DxeMpInitLib.inf > +++ b/UefiCpuPkg/Library/MpInitLib/DxeMpInitLib.inf > @@ -51,6 +51,7 @@ [LibraryClasses] > UefiBootServicesTableLib > DebugAgentLib > SynchronizationLib > + VmgExitLib > > [Protocols] > gEfiTimerArchProtocolGuid ## SOMETIMES_CONSUMES > @@ -70,5 +71,7 @@ [Pcd] > gUefiCpuPkgTokenSpaceGuid.PcdCpuApTargetCstate ## SOMETIMES_CONSUMES > gUefiCpuPkgTokenSpaceGuid.PcdCpuShadowMicrocodeByFit ## CONSUMES > gUefiCpuPkgTokenSpaceGuid.PcdSevEsIsEnabled ## CONSUMES > + gUefiCpuPkgTokenSpaceGuid.PcdSevEsWorkAreaBase ## SOMETIMES_CONSUMES > gEfiMdeModulePkgTokenSpaceGuid.PcdCpuStackGuard ## CONSUMES > + gEfiMdeModulePkgTokenSpaceGuid.PcdGhcbBase ## CONSUMES > > diff --git a/UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf b/UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf > index 66b2acfe98e7..ff392aeec763 100644 > --- a/UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf > +++ b/UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf > @@ -50,6 +50,7 @@ [LibraryClasses] > UefiCpuLib > SynchronizationLib > PeiServicesLib > + VmgExitLib > > [Pcd] > gUefiCpuPkgTokenSpaceGuid.PcdCpuMaxLogicalProcessorNumber ## CONSUMES > @@ -62,6 +63,8 @@ [Pcd] > gUefiCpuPkgTokenSpaceGuid.PcdCpuApTargetCstate ## SOMETIMES_CONSUMES > gUefiCpuPkgTokenSpaceGuid.PcdCpuShadowMicrocodeByFit ## CONSUMES > gUefiCpuPkgTokenSpaceGuid.PcdSevEsIsEnabled ## CONSUMES > + gUefiCpuPkgTokenSpaceGuid.PcdSevEsWorkAreaBase ## SOMETIMES_CONSUMES > + gEfiMdeModulePkgTokenSpaceGuid.PcdGhcbBase ## CONSUMES > > [Guids] > gEdkiiS3SmmInitDoneGuid > diff --git a/UefiCpuPkg/Library/MpInitLib/MpLib.h b/UefiCpuPkg/Library/MpInitLib/MpLib.h > index 864b16872010..63d81ac3e42e 100644 > --- a/UefiCpuPkg/Library/MpInitLib/MpLib.h > +++ b/UefiCpuPkg/Library/MpInitLib/MpLib.h > @@ -170,6 +170,11 @@ typedef struct { > UINT8 *RelocateApLoopFuncAddress; > UINTN RelocateApLoopFuncSize; > UINTN ModeTransitionOffset; > + UINTN SwitchToRealSize; > + UINTN SwitchToRealOffset; > + UINTN SwitchToRealNoNxOffset; > + UINTN SwitchToRealPM16ModeOffset; > + UINTN SwitchToRealPM16ModeSize; > } MP_ASSEMBLY_ADDRESS_MAP; > > typedef struct _CPU_MP_DATA CPU_MP_DATA; > @@ -208,6 +213,8 @@ typedef struct { > // Enable5LevelPaging indicates whether 5-level paging is enabled in long mode. > // > BOOLEAN Enable5LevelPaging; > + BOOLEAN SevEsIsEnabled; > + UINTN GhcbBase; > } MP_CPU_EXCHANGE_INFO; > > #pragma pack() > @@ -256,6 +263,7 @@ struct _CPU_MP_DATA { > UINT8 ApLoopMode; > UINT8 ApTargetCState; > UINT16 PmCodeSegment; > + UINT16 Pm16CodeSegment; > CPU_AP_DATA *CpuData; > volatile MP_CPU_EXCHANGE_INFO *MpCpuExchangeInfo; > > @@ -275,8 +283,47 @@ struct _CPU_MP_DATA { > BOOLEAN WakeUpByInitSipiSipi; > > BOOLEAN SevEsIsEnabled; > + UINTN SevEsAPBuffer; > + UINTN SevEsAPResetStackStart; > + CPU_MP_DATA *NewCpuMpData; > + > + UINT64 GhcbBase; > }; > > +#define AP_RESET_STACK_SIZE 64 > + > +#pragma pack(1) > + > +typedef struct { > + UINT8 InsnBuffer[8]; > + UINT16 Rip; > + UINT16 Segment; > +} SEV_ES_AP_JMP_FAR; > + > +#pragma pack() > + > +/** > + Assembly code to move an AP from long mode to real mode. > + > + Move an AP from long mode to real mode in preparation to invoking > + the reset vector. This is used for SEV-ES guests where a hypervisor > + is not allowed to set the CS and RIP to point to the reset vector. > + > + @param[in] BufferStart The reset vector target. > + @param[in] Code16 16-bit protected mode code segment value. > + @param[in] Code32 32-bit protected mode code segment value. > + @param[in] StackStart The start of a stack to be used for transitioning > + from long mode to real mode. > +**/ > +typedef > +VOID > +(EFIAPI AP_RESET) ( > + IN UINTN BufferStart, > + IN UINT16 Code16, > + IN UINT16 Code32, > + IN UINTN StackStart > + ); > + > extern EFI_GUID mCpuInitMpLibHobGuid; > > /** > @@ -382,6 +429,19 @@ GetModeTransitionBuffer ( > IN UINTN BufferSize > ); > > +/** > + Return the address of the SEV-ES AP jump table. > + > + This buffer is required in order for an SEV-ES guest to transition from > + UEFI into an OS. > + > + @retval other Return SEV-ES AP jump table buffer > +**/ > +UINTN > +GetSevEsAPMemory ( > + VOID > + ); > + > /** > This function will be called by BSP to wakeup AP. > > diff --git a/UefiCpuPkg/Library/MpInitLib/DxeMpLib.c b/UefiCpuPkg/Library/MpInitLib/DxeMpLib.c > index b17e287bbf49..8df5b6d919e6 100644 > --- a/UefiCpuPkg/Library/MpInitLib/DxeMpLib.c > +++ b/UefiCpuPkg/Library/MpInitLib/DxeMpLib.c > @@ -12,6 +12,8 @@ > #include > #include > #include > +#include > +#include > > #include > > @@ -145,6 +147,39 @@ GetModeTransitionBuffer ( > return (UINTN)StartAddress; > } > > +/** > + Return the address of the SEV-ES AP jump table. > + > + This buffer is required in order for an SEV-ES guest to transition from > + UEFI into an OS. > + > + @retval other Return SEV-ES AP jump table buffer > +**/ > +UINTN > +GetSevEsAPMemory ( > + VOID > + ) > +{ > + EFI_STATUS Status; > + EFI_PHYSICAL_ADDRESS StartAddress; > + > + // > + // Allocate 1 page for AP jump table page > + // > + StartAddress = BASE_4GB - 1; > + Status = gBS->AllocatePages ( > + AllocateMaxAddress, > + EfiReservedMemoryType, > + 1, > + &StartAddress > + ); > + ASSERT_EFI_ERROR (Status); > + > + DEBUG ((DEBUG_INFO, "Dxe: SevEsAPMemory = %lx\n", (UINTN) StartAddress)); > + > + return (UINTN) StartAddress; > +} > + > /** > Checks APs status and updates APs status if needed. > > @@ -219,6 +254,38 @@ CheckApsStatus ( > } > } > > +/** > + Get Protected mode code segment with 16-bit default addressing > + from current GDT table. > + > + @return Protected mode 16-bit code segment value. > +**/ > +UINT16 > +GetProtectedMode16CS ( > + VOID > + ) > +{ > + IA32_DESCRIPTOR GdtrDesc; > + IA32_SEGMENT_DESCRIPTOR *GdtEntry; > + UINTN GdtEntryCount; > + UINT16 Index; > + > + Index = (UINT16) -1; > + AsmReadGdtr (&GdtrDesc); > + GdtEntryCount = (GdtrDesc.Limit + 1) / sizeof (IA32_SEGMENT_DESCRIPTOR); > + GdtEntry = (IA32_SEGMENT_DESCRIPTOR *) GdtrDesc.Base; > + for (Index = 0; Index < GdtEntryCount; Index++) { > + if (GdtEntry->Bits.L == 0) { > + if (GdtEntry->Bits.Type > 8 && GdtEntry->Bits.DB == 0) { > + break; > + } > + } > + GdtEntry++; > + } > + ASSERT (Index != GdtEntryCount); > + return Index * 8; > +} > + > /** > Get Protected mode code segment from current GDT table. > > @@ -239,7 +306,7 @@ GetProtectedModeCS ( > GdtEntry = (IA32_SEGMENT_DESCRIPTOR *) GdtrDesc.Base; > for (Index = 0; Index < GdtEntryCount; Index++) { > if (GdtEntry->Bits.L == 0) { > - if (GdtEntry->Bits.Type > 8 && GdtEntry->Bits.L == 0) { > + if (GdtEntry->Bits.Type > 8 && GdtEntry->Bits.DB == 1) { > break; > } > } > @@ -301,6 +368,7 @@ MpInitChangeApLoopCallback ( > > CpuMpData = GetCpuMpData (); > CpuMpData->PmCodeSegment = GetProtectedModeCS (); > + CpuMpData->Pm16CodeSegment = GetProtectedMode16CS (); > CpuMpData->ApLoopMode = PcdGet8 (PcdCpuApLoopMode); > mNumberToFinish = CpuMpData->CpuCount - 1; > WakeUpAP (CpuMpData, TRUE, 0, RelocateApLoop, NULL, TRUE); > diff --git a/UefiCpuPkg/Library/MpInitLib/MpLib.c b/UefiCpuPkg/Library/MpInitLib/MpLib.c > index 5e3183c2493b..ca8a3a3a7be9 100644 > --- a/UefiCpuPkg/Library/MpInitLib/MpLib.c > +++ b/UefiCpuPkg/Library/MpInitLib/MpLib.c > @@ -7,6 +7,9 @@ > **/ > > #include "MpLib.h" > +#include > +#include > +#include > > EFI_GUID mCpuInitMpLibHobGuid = CPU_INIT_MP_LIB_HOB_GUID; > > @@ -288,6 +291,14 @@ GetApLoopMode ( > // > ApLoopMode = ApInHltLoop; > } > + > + if (PcdGetBool (PcdSevEsIsEnabled)) { > + // > + // For SEV-ES, force AP in Hlt-loop mode in order to use the GHCB > + // protocol for starting APs > + // > + ApLoopMode = ApInHltLoop; > + } > } > > if (ApLoopMode != ApInMwaitLoop) { > @@ -579,6 +590,108 @@ InitializeApData ( > SetApState (&CpuMpData->CpuData[ProcessorNumber], CpuStateIdle); > } > > +/** > + Get Protected mode code segment with 16-bit default addressing > + from current GDT table. > + > + @return Protected mode 16-bit code segment value. > +**/ > +STATIC > +UINT16 > +GetProtectedMode16CS ( > + VOID > + ) > +{ > + IA32_DESCRIPTOR GdtrDesc; > + IA32_SEGMENT_DESCRIPTOR *GdtEntry; > + UINTN GdtEntryCount; > + UINT16 Index; > + > + Index = (UINT16) -1; > + AsmReadGdtr (&GdtrDesc); > + GdtEntryCount = (GdtrDesc.Limit + 1) / sizeof (IA32_SEGMENT_DESCRIPTOR); > + GdtEntry = (IA32_SEGMENT_DESCRIPTOR *) GdtrDesc.Base; > + for (Index = 0; Index < GdtEntryCount; Index++) { > + if (GdtEntry->Bits.L == 0 && > + GdtEntry->Bits.DB == 0 && > + GdtEntry->Bits.Type > 8) { > + break; > + } > + GdtEntry++; > + } > + ASSERT (Index != GdtEntryCount); > + return Index * 8; > +} > + > +/** > + Get Protected mode code segment with 32-bit default addressing > + from current GDT table. > + > + @return Protected mode 32-bit code segment value. > +**/ > +STATIC > +UINT16 > +GetProtectedMode32CS ( > + VOID > + ) > +{ > + IA32_DESCRIPTOR GdtrDesc; > + IA32_SEGMENT_DESCRIPTOR *GdtEntry; > + UINTN GdtEntryCount; > + UINT16 Index; > + > + Index = (UINT16) -1; > + AsmReadGdtr (&GdtrDesc); > + GdtEntryCount = (GdtrDesc.Limit + 1) / sizeof (IA32_SEGMENT_DESCRIPTOR); > + GdtEntry = (IA32_SEGMENT_DESCRIPTOR *) GdtrDesc.Base; > + for (Index = 0; Index < GdtEntryCount; Index++) { > + if (GdtEntry->Bits.L == 0 && > + GdtEntry->Bits.DB == 1 && > + GdtEntry->Bits.Type > 8) { > + break; > + } > + GdtEntry++; > + } > + ASSERT (Index != GdtEntryCount); > + return Index * 8; > +} > + > +/** > + Reset an AP when in SEV-ES mode. > + > + @retval EFI_DEVICE_ERROR Reset of AP failed. > +**/ > +STATIC > +VOID > +MpInitLibSevEsAPReset ( > + GHCB *Ghcb, > + CPU_MP_DATA *CpuMpData > + ) > +{ > + UINT16 Code16, Code32; > + AP_RESET *APResetFn; > + UINTN BufferStart; > + UINTN StackStart; > + > + Code16 = GetProtectedMode16CS (); > + Code32 = GetProtectedMode32CS (); > + > + if (CpuMpData->WakeupBufferHigh != 0) { > + APResetFn = (AP_RESET *) (CpuMpData->WakeupBufferHigh + CpuMpData->AddressMap.SwitchToRealNoNxOffset); > + } else { > + APResetFn = (AP_RESET *) (CpuMpData->MpCpuExchangeInfo->BufferStart + CpuMpData->AddressMap.SwitchToRealOffset); > + } > + > + BufferStart = CpuMpData->MpCpuExchangeInfo->BufferStart; > + StackStart = CpuMpData->SevEsAPResetStackStart - > + (AP_RESET_STACK_SIZE * GetApicId ()); > + > + // > + // This call never returns. > + // > + APResetFn (BufferStart, Code16, Code32, StackStart); > +} > + > /** > This function will be called from AP reset code if BSP uses WakeUpAP. > > @@ -734,7 +847,28 @@ ApWakeupFunction ( > // > while (TRUE) { > DisableInterrupts (); > - CpuSleep (); > + if (CpuMpData->SevEsIsEnabled) { > + MSR_SEV_ES_GHCB_REGISTER Msr; > + GHCB *Ghcb; > + > + Msr.GhcbPhysicalAddress = AsmReadMsr64 (MSR_SEV_ES_GHCB); > + Ghcb = Msr.Ghcb; > + > + VmgInit (Ghcb); > + VmgExit (Ghcb, SvmExitApResetHold, 0, 0); > + /*TODO: Check return value to verify SIPI issued */ > + > + // > + // Awakened in a new phase? Use the new CpuMpData > + // > + if (CpuMpData->NewCpuMpData) { > + CpuMpData = CpuMpData->NewCpuMpData; > + } > + > + MpInitLibSevEsAPReset (Ghcb, CpuMpData); > + } else { > + CpuSleep (); > + } > CpuPause (); > } > } > @@ -847,6 +981,9 @@ FillExchangeInfoData ( > ExchangeInfo->Enable5LevelPaging = (BOOLEAN) (Cr4.Bits.LA57 == 1); > DEBUG ((DEBUG_INFO, "%a: 5-Level Paging = %d\n", gEfiCallerBaseName, ExchangeInfo->Enable5LevelPaging)); > > + ExchangeInfo->SevEsIsEnabled = CpuMpData->SevEsIsEnabled; > + ExchangeInfo->GhcbBase = CpuMpData->GhcbBase; > + > // > // Get the BSP's data of GDT and IDT > // > @@ -873,8 +1010,9 @@ FillExchangeInfoData ( > // EfiBootServicesCode to avoid page fault if NX memory protection is enabled. > // > if (CpuMpData->WakeupBufferHigh != 0) { > - Size = CpuMpData->AddressMap.RendezvousFunnelSize - > - CpuMpData->AddressMap.ModeTransitionOffset; > + Size = CpuMpData->AddressMap.RendezvousFunnelSize + > + CpuMpData->AddressMap.SwitchToRealSize - > + CpuMpData->AddressMap.ModeTransitionOffset; > CopyMem ( > (VOID *)CpuMpData->WakeupBufferHigh, > CpuMpData->AddressMap.RendezvousFunnelAddress + > @@ -927,7 +1065,8 @@ BackupAndPrepareWakeupBuffer( > CopyMem ( > (VOID *) CpuMpData->WakeupBuffer, > (VOID *) CpuMpData->AddressMap.RendezvousFunnelAddress, > - CpuMpData->AddressMap.RendezvousFunnelSize > + CpuMpData->AddressMap.RendezvousFunnelSize + > + CpuMpData->AddressMap.SwitchToRealSize > ); > } > > @@ -948,6 +1087,40 @@ RestoreWakeupBuffer( > ); > } > > +/** > + Calculate the size of the reset stack. > +**/ > +STATIC > +UINTN > +GetApResetStackSize( > + VOID > + ) > +{ > + return AP_RESET_STACK_SIZE * PcdGet32(PcdCpuMaxLogicalProcessorNumber); > +} > + > +/** > + Calculate the size of the reset vector. > + > + @param[in] AddressMap The pointer to Address Map structure. > +**/ > +STATIC > +UINTN > +GetApResetVectorSize( > + IN MP_ASSEMBLY_ADDRESS_MAP *AddressMap > + ) > +{ > + UINTN Size; > + > + Size = ALIGN_VALUE (AddressMap->RendezvousFunnelSize + > + AddressMap->SwitchToRealSize + > + sizeof (MP_CPU_EXCHANGE_INFO), > + CPU_STACK_ALIGNMENT); > + Size += GetApResetStackSize (); > + > + return Size; > +} > + > /** > Allocate reset vector buffer. > > @@ -961,16 +1134,22 @@ AllocateResetVector ( > UINTN ApResetVectorSize; > > if (CpuMpData->WakeupBuffer == (UINTN) -1) { > - ApResetVectorSize = CpuMpData->AddressMap.RendezvousFunnelSize + > - sizeof (MP_CPU_EXCHANGE_INFO); > + ApResetVectorSize = GetApResetVectorSize (&CpuMpData->AddressMap); > > CpuMpData->WakeupBuffer = GetWakeupBuffer (ApResetVectorSize); > CpuMpData->MpCpuExchangeInfo = (MP_CPU_EXCHANGE_INFO *) (UINTN) > - (CpuMpData->WakeupBuffer + CpuMpData->AddressMap.RendezvousFunnelSize); > + (CpuMpData->WakeupBuffer + > + CpuMpData->AddressMap.RendezvousFunnelSize + > + CpuMpData->AddressMap.SwitchToRealSize); > CpuMpData->WakeupBufferHigh = GetModeTransitionBuffer ( > - CpuMpData->AddressMap.RendezvousFunnelSize - > + CpuMpData->AddressMap.RendezvousFunnelSize + > + CpuMpData->AddressMap.SwitchToRealSize - > CpuMpData->AddressMap.ModeTransitionOffset > ); > + // > + // The reset stack starts at the end of the buffer. > + // > + CpuMpData->SevEsAPResetStackStart = CpuMpData->WakeupBuffer + ApResetVectorSize; > } > BackupAndPrepareWakeupBuffer (CpuMpData); > } > @@ -985,7 +1164,31 @@ FreeResetVector ( > IN CPU_MP_DATA *CpuMpData > ) > { > - RestoreWakeupBuffer (CpuMpData); > + // > + // If SEV-ES is enabled, the reset area is needed for AP parking and > + // and AP startup in the OS, so the reset area is reserved. Do not > + // perform the restore as this will overwrite memory which has data > + // needed by SEV-ES. > + // > + if (!CpuMpData->SevEsIsEnabled) { > + RestoreWakeupBuffer (CpuMpData); > + } > +} > + > +/** > + Allocate the SEV-ES AP jump table buffer. > + > + @param[in, out] CpuMpData The pointer to CPU MP Data structure. > +**/ > +VOID > +AllocateSevEsAPMemory ( > + IN OUT CPU_MP_DATA *CpuMpData > + ) > +{ > + if (CpuMpData->SevEsAPBuffer == (UINTN) -1) { > + CpuMpData->SevEsAPBuffer = > + CpuMpData->SevEsIsEnabled ? GetSevEsAPMemory () : 0; > + } > } > > /** > @@ -1022,6 +1225,7 @@ WakeUpAP ( > CpuMpData->InitFlag != ApInitDone) { > ResetVectorRequired = TRUE; > AllocateResetVector (CpuMpData); > + AllocateSevEsAPMemory (CpuMpData); > FillExchangeInfoData (CpuMpData); > SaveLocalApicTimerSetting (CpuMpData); > } > @@ -1058,6 +1262,35 @@ WakeUpAP ( > } > } > if (ResetVectorRequired) { > + // > + // For SEV-ES, the initial AP boot address will be defined by > + // PcdSevEsWorkAreaBase. The Segment/Rip must be the jump address > + // from the original INIT-SIPI-SIPI. > + // > + if (CpuMpData->SevEsIsEnabled) { > + SEV_ES_AP_JMP_FAR *JmpFar; > + UINT32 Offset, InsnByte; > + UINT8 LoNib, HiNib; > + > + JmpFar = (SEV_ES_AP_JMP_FAR *) FixedPcdGet32 (PcdSevEsWorkAreaBase); > + ASSERT (JmpFar != NULL); > + > + Offset = FixedPcdGet32 (PcdSevEsWorkAreaBase); > + Offset += sizeof(JmpFar->InsnBuffer); > + LoNib = (UINT8) Offset; > + HiNib = (UINT8) (Offset >> 8); > + > + // JMP FAR [CS:XXYY] => 2E FF 2E YY XX > + InsnByte = 0; > + JmpFar->InsnBuffer[InsnByte++] = 0x2E; // CS override prefix > + JmpFar->InsnBuffer[InsnByte++] = 0xFF; // JMP (FAR) > + JmpFar->InsnBuffer[InsnByte++] = 0x2E; // ModRM (JMP memory location) > + JmpFar->InsnBuffer[InsnByte++] = LoNib; // YY offset ... > + JmpFar->InsnBuffer[InsnByte++] = HiNib; // XX offset ... > + > + JmpFar->Rip = 0; > + JmpFar->Segment = (UINT16) (ExchangeInfo->BufferStart >> 4); > + } > // > // Wakeup all APs > // > @@ -1625,7 +1858,7 @@ MpInitLibInitialize ( > ASSERT (MaxLogicalProcessorNumber != 0); > > AsmGetAddressMap (&AddressMap); > - ApResetVectorSize = AddressMap.RendezvousFunnelSize + sizeof (MP_CPU_EXCHANGE_INFO); > + ApResetVectorSize = GetApResetVectorSize (&AddressMap); > ApStackSize = PcdGet32(PcdCpuApStackSize); > ApLoopMode = GetApLoopMode (&MonitorFilterSize); > > @@ -1688,6 +1921,8 @@ MpInitLibInitialize ( > } > InitializeSpinLock(&CpuMpData->MpLock); > CpuMpData->SevEsIsEnabled = PcdGetBool (PcdSevEsIsEnabled); > + CpuMpData->SevEsAPBuffer = (UINTN) -1; > + CpuMpData->GhcbBase = PcdGet64 (PcdGhcbBase); > > // > // Make sure no memory usage outside of the allocated buffer. > @@ -1751,6 +1986,7 @@ MpInitLibInitialize ( > // APs have been wakeup before, just get the CPU Information > // from HOB > // > + OldCpuMpData->NewCpuMpData = CpuMpData; > CpuMpData->CpuCount = OldCpuMpData->CpuCount; > CpuMpData->BspNumber = OldCpuMpData->BspNumber; > CpuMpData->CpuInfoInHob = OldCpuMpData->CpuInfoInHob; > diff --git a/UefiCpuPkg/Library/MpInitLib/PeiMpLib.c b/UefiCpuPkg/Library/MpInitLib/PeiMpLib.c > index 06e3f5d0d3da..e8103a9ce094 100644 > --- a/UefiCpuPkg/Library/MpInitLib/PeiMpLib.c > +++ b/UefiCpuPkg/Library/MpInitLib/PeiMpLib.c > @@ -280,6 +280,25 @@ GetModeTransitionBuffer ( > return 0; > } > > +/** > + Return the address of the SEV-ES AP jump table. > + > + This buffer is required in order for an SEV-ES guest to transition from > + UEFI into an OS. > + > + @retval other Return SEV-ES AP jump table buffer > +**/ > +UINTN > +GetSevEsAPMemory ( > + VOID > + ) > +{ > + // > + // PEI phase doesn't need to do such transition. So simply return 0. > + // > + return 0; > +} > + > /** > Checks APs status and updates APs status if needed. > > diff --git a/UefiCpuPkg/PiSmmCpuDxeSmm/X64/SmmFuncsArch.c b/UefiCpuPkg/PiSmmCpuDxeSmm/X64/SmmFuncsArch.c > index 6298571e29b2..28f8e8e133e5 100644 > --- a/UefiCpuPkg/PiSmmCpuDxeSmm/X64/SmmFuncsArch.c > +++ b/UefiCpuPkg/PiSmmCpuDxeSmm/X64/SmmFuncsArch.c > @@ -121,7 +121,7 @@ GetProtectedModeCS ( > GdtEntry = (IA32_SEGMENT_DESCRIPTOR *) GdtrDesc.Base; > for (Index = 0; Index < GdtEntryCount; Index++) { > if (GdtEntry->Bits.L == 0) { > - if (GdtEntry->Bits.Type > 8 && GdtEntry->Bits.L == 0) { > + if (GdtEntry->Bits.Type > 8 && GdtEntry->Bits.DB == 1) { > break; > } > } > diff --git a/UefiCpuPkg/Library/MpInitLib/Ia32/MpEqu.inc b/UefiCpuPkg/Library/MpInitLib/Ia32/MpEqu.inc > index efb1bc2bf7cb..4f5a7c859a56 100644 > --- a/UefiCpuPkg/Library/MpInitLib/Ia32/MpEqu.inc > +++ b/UefiCpuPkg/Library/MpInitLib/Ia32/MpEqu.inc > @@ -19,7 +19,7 @@ CPU_SWITCH_STATE_IDLE equ 0 > CPU_SWITCH_STATE_STORED equ 1 > CPU_SWITCH_STATE_LOADED equ 2 > > -LockLocation equ (RendezvousFunnelProcEnd - RendezvousFunnelProcStart) > +LockLocation equ (SwitchToRealProcEnd - RendezvousFunnelProcStart) > StackStartAddressLocation equ LockLocation + 04h > StackSizeLocation equ LockLocation + 08h > ApProcedureLocation equ LockLocation + 0Ch > diff --git a/UefiCpuPkg/Library/MpInitLib/Ia32/MpFuncs.nasm b/UefiCpuPkg/Library/MpInitLib/Ia32/MpFuncs.nasm > index b74046b76af3..309d53bf3b37 100644 > --- a/UefiCpuPkg/Library/MpInitLib/Ia32/MpFuncs.nasm > +++ b/UefiCpuPkg/Library/MpInitLib/Ia32/MpFuncs.nasm > @@ -215,6 +215,16 @@ CProcedureInvoke: > jmp $ ; Never reach here > RendezvousFunnelProcEnd: > > +;------------------------------------------------------------------------------------- > +;SwitchToRealProc procedure follows. > +;NOT USED IN 32 BIT MODE. > +;------------------------------------------------------------------------------------- > +global ASM_PFX(SwitchToRealProc) > +ASM_PFX(SwitchToRealProc): > +SwitchToRealProcStart: > + jmp $ ; Never reach here > +SwitchToRealProcEnd: > + > ;------------------------------------------------------------------------------------- > ; AsmRelocateApLoop (MwaitSupport, ApTargetCState, PmCodeSegment, TopOfApStack, CountTofinish); > ;------------------------------------------------------------------------------------- > @@ -263,6 +273,11 @@ ASM_PFX(AsmGetAddressMap): > mov dword [ebx + 0Ch], AsmRelocateApLoopStart > mov dword [ebx + 10h], AsmRelocateApLoopEnd - AsmRelocateApLoopStart > mov dword [ebx + 14h], Flat32Start - RendezvousFunnelProcStart > + mov dword [ebx + 18h], SwitchToRealProcEnd - SwitchToRealProcStart ; SwitchToRealSize > + mov dword [ebx + 1Ch], SwitchToRealProcStart - RendezvousFunnelProcStart ; SwitchToRealOffset > + mov dword [ebx + 20h], SwitchToRealProcStart - Flat32Start ; SwitchToRealNoNxOffset > + mov dword [ebx + 24h], 0 ; SwitchToRealPM16ModeOffset > + mov dword [ebx + 28h], 0 ; SwitchToRealPM16ModeSize > > popad > ret > diff --git a/UefiCpuPkg/Library/MpInitLib/X64/MpEqu.inc b/UefiCpuPkg/Library/MpInitLib/X64/MpEqu.inc > index 58ef369342a7..c92daaaffd6b 100644 > --- a/UefiCpuPkg/Library/MpInitLib/X64/MpEqu.inc > +++ b/UefiCpuPkg/Library/MpInitLib/X64/MpEqu.inc > @@ -19,7 +19,7 @@ CPU_SWITCH_STATE_IDLE equ 0 > CPU_SWITCH_STATE_STORED equ 1 > CPU_SWITCH_STATE_LOADED equ 2 > > -LockLocation equ (RendezvousFunnelProcEnd - RendezvousFunnelProcStart) > +LockLocation equ (SwitchToRealProcEnd - RendezvousFunnelProcStart) > StackStartAddressLocation equ LockLocation + 08h > StackSizeLocation equ LockLocation + 10h > ApProcedureLocation equ LockLocation + 18h > @@ -41,3 +41,5 @@ ModeTransitionSegmentLocation equ LockLocation + 98h > ModeHighMemoryLocation equ LockLocation + 9Ah > ModeHighSegmentLocation equ LockLocation + 9Eh > Enable5LevelPagingLocation equ LockLocation + 0A0h > +SevEsIsEnabledLocation equ LockLocation + 0A1h > +GhcbBaseLocation equ LockLocation + 0A2h > diff --git a/UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm b/UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm > index 87f2523e856f..6956b408d004 100644 > --- a/UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm > +++ b/UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm > @@ -184,9 +184,97 @@ Releaselock: > add edi, StackStartAddressLocation > add rax, qword [edi] > mov rsp, rax > + > + lea edi, [esi + SevEsIsEnabledLocation] > + cmp byte [edi], 1 ; SevEsIsEnabled > + jne CProcedureInvoke > + > + ; > + ; program GHCB > + ; Each page after the GHCB is a per-CPU page, so the calculation programs > + ; a GHCB to be every 8KB. > + ; > + mov eax, SIZE_4KB > + shl eax, 1 ; EAX = SIZE_4K * 2 > + mov ecx, ebx > + mul ecx ; EAX = SIZE_4K * 2 * CpuNumber > + mov edi, esi > + add edi, GhcbBaseLocation > + add rax, qword [edi] > + mov rdx, rax > + shr rdx, 32 > + mov rcx, 0xc0010130 > + wrmsr > jmp CProcedureInvoke > > GetApicId: > + lea edi, [esi + SevEsIsEnabledLocation] > + cmp byte [edi], 1 ; SevEsIsEnabled > + jne DoCpuid > + > + ; > + ; Since we don't have a stack yet, we can't take a #VC > + ; exception. Use the GHCB protocol to perform the CPUID > + ; calls. > + ; > + mov rcx, 0xc0010130 > + rdmsr > + shl rdx, 32 > + or rax, rdx > + mov rdi, rax ; RDI now holds the original GHCB GPA > + > + mov rdx, 0 ; CPUID function 0 > + mov rax, 0 ; RAX register requested > + or rax, 4 > + wrmsr > + rep vmmcall > + rdmsr > + cmp edx, 0bh > + jb NoX2ApicSevEs ; CPUID level below CPUID_EXTENDED_TOPOLOGY > + > + mov rdx, 0bh ; CPUID function 0x0b > + mov rax, 040000000h ; RBX register requested > + or rax, 4 > + wrmsr > + rep vmmcall > + rdmsr > + test edx, 0ffffh > + jz NoX2ApicSevEs ; CPUID.0BH:EBX[15:0] is zero > + > + mov rdx, 0bh ; CPUID function 0x0b > + mov rax, 0c0000000h ; RDX register requested > + or rax, 4 > + wrmsr > + rep vmmcall > + rdmsr > + > + ; Processor is x2APIC capable; 32-bit x2APIC ID is now in EDX > + jmp RestoreGhcb > + > +NoX2ApicSevEs: > + ; Processor is not x2APIC capable, so get 8-bit APIC ID > + mov rdx, 1 ; CPUID function 1 > + mov rax, 040000000h ; RBX register requested > + or rax, 4 > + wrmsr > + rep vmmcall > + rdmsr > + shr edx, 24 > + > +RestoreGhcb: > + mov rbx, rdx ; Save x2APIC/APIC ID > + > + mov rdx, rdi ; RDI holds the saved GHCB GPA > + shr rdx, 32 > + mov eax, edi > + wrmsr > + > + mov rdx, rbx > + > + ; x2APIC ID or APIC ID is in EDX > + jmp GetProcessorNumber > + > +DoCpuid: > mov eax, 0 > cpuid > cmp eax, 0bh > @@ -253,12 +341,158 @@ CProcedureInvoke: > > RendezvousFunnelProcEnd: > > +;------------------------------------------------------------------------------------- > +;SwitchToRealProc procedure follows. > +;ALSO THIS PROCEDURE IS EXECUTED BY APs TRANSITIONING TO 16 BIT MODE. HENCE THIS PROC > +;IS IN MACHINE CODE. > +; SwitchToRealProc (UINTN BufferStart, UINT16 Code16, UINT16 Code32, UINTN StackStart) > +; rcx - Buffer Start > +; rdx - Code16 Selector Offset > +; r8 - Code32 Selector Offset > +; r9 - Stack Start > +;------------------------------------------------------------------------------------- > +global ASM_PFX(SwitchToRealProc) > +ASM_PFX(SwitchToRealProc): > +SwitchToRealProcStart: > +BITS 64 > + cli > + > + ; > + ; Get RDX reset value before changing stacks since the > + ; new stack won't be able to accomodate a #VC exception. > + ; > + push rax > + push rbx > + push rcx > + push rdx > + > + mov rax, 1 > + cpuid > + mov rsi, rax ; Save off the reset value for RDX > + > + pop rdx > + pop rcx > + pop rbx > + pop rax > + > + ; > + ; Establish stack below 1MB > + ; > + mov rsp, r9 > + > + ; > + ; Push ultimate Reset Vector onto the stack > + ; > + mov rax, rcx > + shr rax, 4 > + push word 0x0002 ; RFLAGS > + push ax ; CS > + push word 0x0000 ; RIP > + push word 0x0000 ; For alignment, will be discarded > + > + ; > + ; Get address of "16-bit operand size" label > + ; > + lea rbx, [PM16Mode] > + > + ; > + ; Push addresses used to change to compatibility mode > + ; > + lea rax, [CompatMode] > + push r8 > + push rax > + > + ; > + ; Clear R8 - R15, for reset, before going into 32-bit mode > + ; > + xor r8, r8 > + xor r9, r9 > + xor r10, r10 > + xor r11, r11 > + xor r12, r12 > + xor r13, r13 > + xor r14, r14 > + xor r15, r15 > + > + ; > + ; Far return into 32-bit mode > + ; > +o64 retf > + > +BITS 32 > +CompatMode: > + ; > + ; Set up stack to prepare for exiting protected mode > + ; > + push edx ; Code16 CS > + push ebx ; PM16Mode label address > + > + ; > + ; Disable paging > + ; > + mov eax, cr0 ; Read CR0 > + btr eax, 31 ; Set PG=0 > + mov cr0, eax ; Write CR0 > + > + ; > + ; Disable long mode > + ; > + mov ecx, 0c0000080h ; EFER MSR number > + rdmsr ; Read EFER > + btr eax, 8 ; Set LME=0 > + wrmsr ; Write EFER > + > + ; > + ; Disable PAE > + ; > + mov eax, cr4 ; Read CR4 > + btr eax, 5 ; Set PAE=0 > + mov cr4, eax ; Write CR4 > + > + mov edx, esi ; Restore RDX reset value > + > + ; > + ; Switch to 16-bit operand size > + ; > + retf > + > +BITS 16 > + ; > + ; At entry to this label > + ; - RDX will have its reset value > + ; - On the top of the stack > + ; - Alignment data (two bytes) to be discarded > + ; - IP for Real Mode (two bytes) > + ; - CS for Real Mode (two bytes) > + ; > +PM16Mode: > + mov eax, cr0 ; Read CR0 > + btr eax, 0 ; Set PE=0 > + mov cr0, eax ; Write CR0 > + > + pop ax ; Discard alignment data > + > + ; > + ; Clear registers (except RDX and RSP) before going into 16-bit mode > + ; > + xor eax, eax > + xor ebx, ebx > + xor ecx, ecx > + xor esi, esi > + xor edi, edi > + xor ebp, ebp > + > + iret > + > +SwitchToRealProcEnd: > + > ;------------------------------------------------------------------------------------- > ; AsmRelocateApLoop (MwaitSupport, ApTargetCState, PmCodeSegment, TopOfApStack, CountTofinish); > ;------------------------------------------------------------------------------------- > global ASM_PFX(AsmRelocateApLoop) > ASM_PFX(AsmRelocateApLoop): > AsmRelocateApLoopStart: > +BITS 64 > cli ; Disable interrupt before switching to 32-bit mode > mov rax, [rsp + 40] ; CountTofinish > lock dec dword [rax] ; (*CountTofinish)-- > @@ -324,6 +558,11 @@ ASM_PFX(AsmGetAddressMap): > mov qword [rcx + 18h], rax > mov qword [rcx + 20h], AsmRelocateApLoopEnd - AsmRelocateApLoopStart > mov qword [rcx + 28h], Flat32Start - RendezvousFunnelProcStart > + mov qword [rcx + 30h], SwitchToRealProcEnd - SwitchToRealProcStart ; SwitchToRealSize > + mov qword [rcx + 38h], SwitchToRealProcStart - RendezvousFunnelProcStart ; SwitchToRealOffset > + mov qword [rcx + 40h], SwitchToRealProcStart - Flat32Start ; SwitchToRealNoNxOffset > + mov qword [rcx + 48h], PM16Mode - RendezvousFunnelProcStart ; SwitchToRealPM16ModeOffset > + mov qword [rcx + 50h], SwitchToRealProcEnd - PM16Mode ; SwitchToRealPM16ModeSize > ret > > ;------------------------------------------------------------------------------------- >