From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from NAM02-BL2-obe.outbound.protection.outlook.com (NAM02-BL2-obe.outbound.protection.outlook.com [40.107.75.78]) by mx.groups.io with SMTP id smtpd.web10.6701.1581090715353199161 for ; Fri, 07 Feb 2020 07:51:55 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@amdcloud.onmicrosoft.com header.s=selector2-amdcloud-onmicrosoft-com header.b=T5kqOCR/; spf=none, err=SPF record not found (domain: amd.com, ip: 40.107.75.78, mailfrom: thomas.lendacky@amd.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=OIj9DwSO4empy1wgCWcyJ7JBD/b9GZyWbaHZ+mxjCyp8xilLDQLpC5yrGzdUG1+bbpUOIoPDGJK7/+x+w3/h2DHAd3iJUCKh460zHZ9ojZ+3yXFdZLJhbko+zclv46urhRd2WHtAvX3kF8X3uNAg/MP+qZKRNB/mw1R8FLju5iyrzjNWtVFEZMzN/wShTQa0WQRSl6ROyRQ8w/1nWw4GmzljqSCvej2GLrrYpwmUBZCwkReFFkyzyVzfLNSOznvD+5Diu5Vdlma+P6cvCh5eQFAv2ImNbSEzAxQBC1Rta+dWotnu4iupFJyeg305tvVr9pTcZMLcpOml+Ek0frDa6w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=egUp87+fTyWUPvdHA6FbtEbRQ2KOc/CNx2VJ7l7gt+0=; b=OwmrdR8QSIzbDMd/3D5sq7RJ5wOEBcJqL97BHRoI4GyjmcfpXiNEu3DknixKB4VfOEwWDgQq+p/5PXvq7CsPfBx9pCwSkb0YF+EXw4kjru3RGp3l1skhmXryEwj1Iig88PSopeogsFsACzFoiq+X0/IrVn/JguWKvT/ognVWR4GcmXwNkBABS4Hme8lmSXK+mLkC8dl7L7KJaHfK8tDRLDnqXlQxRGAAJbktifGzx8HjOSjfTqk2dJGdU4pfIUdKC1avqR9gHUW/flBHpaXAIGhPUXmT7ZQ/IiS3EphSqbGqRgsqMtuDWlgTagk0TTDyS0I1kF/4aESyX484BuaJ3A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amdcloud.onmicrosoft.com; s=selector2-amdcloud-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=egUp87+fTyWUPvdHA6FbtEbRQ2KOc/CNx2VJ7l7gt+0=; b=T5kqOCR/0Y/QYHUQx0+c4ZckXgfSalZpbsTgBf/qi7HynHUNXlngruBu+TJ0kPlmzIkKX9INtzM2HQoYasZZMsdSbpM85wGH341syKTDRrUFff9egToh8B9Wkl08IQpev/8zC5F58SOxkiGOsgEC0p4K6CbfdZGwZHGGXKoQXzo= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Thomas.Lendacky@amd.com; Received: from DM6PR12MB3163.namprd12.prod.outlook.com (20.179.71.154) by DM6PR12MB3419.namprd12.prod.outlook.com (20.178.199.32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2707.25; Fri, 7 Feb 2020 15:51:53 +0000 Received: from DM6PR12MB3163.namprd12.prod.outlook.com ([fe80::a0cd:463:f444:c270]) by DM6PR12MB3163.namprd12.prod.outlook.com ([fe80::a0cd:463:f444:c270%7]) with mapi id 15.20.2707.024; Fri, 7 Feb 2020 15:51:53 +0000 Subject: Re: [edk2-devel] [PATCH v4 37/40] UefiCpuPkg: Allow AP booting under SEV-ES To: Laszlo Ersek , devel@edk2.groups.io Cc: Jordan Justen , Ard Biesheuvel , Michael D Kinney , Liming Gao , Eric Dong , Ray Ni , Brijesh Singh References: <0aa6e1d9-338c-bc5f-8400-46389c1697df@redhat.com> From: "Lendacky, Thomas" Message-ID: <9bdc877f-9d9c-acb1-65fa-79b571692dce@amd.com> Date: Fri, 7 Feb 2020 09:51:51 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.4.1 In-Reply-To: <0aa6e1d9-338c-bc5f-8400-46389c1697df@redhat.com> X-ClientProxiedBy: DM5PR13CA0051.namprd13.prod.outlook.com (2603:10b6:3:117::13) To DM6PR12MB3163.namprd12.prod.outlook.com (2603:10b6:5:15e::26) Return-Path: thomas.lendacky@amd.com MIME-Version: 1.0 Received: from [10.236.30.74] (165.204.77.1) by DM5PR13CA0051.namprd13.prod.outlook.com (2603:10b6:3:117::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2729.9 via Frontend Transport; Fri, 7 Feb 2020 15:51:52 +0000 X-Originating-IP: [165.204.77.1] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 84a0eaa4-ff56-4ac4-8f0c-08d7abe5a6b6 X-MS-TrafficTypeDiagnostic: DM6PR12MB3419:|DM6PR12MB3419: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:10000; X-Forefront-PRVS: 0306EE2ED4 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(4636009)(39860400002)(376002)(366004)(396003)(136003)(346002)(199004)(189003)(66556008)(66946007)(66476007)(19627235002)(30864003)(86362001)(5660300002)(31696002)(2906002)(966005)(36756003)(81156014)(81166006)(478600001)(8676002)(8936002)(31686004)(4326008)(45080400002)(186003)(52116002)(2616005)(956004)(26005)(53546011)(316002)(54906003)(16526019)(16576012)(6486002)(579004);DIR:OUT;SFP:1101;SCL:1;SRVR:DM6PR12MB3419;H:DM6PR12MB3163.namprd12.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; Received-SPF: None (protection.outlook.com: amd.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: iZSEn8ccodyb57rTHTK1HbmqbXRUCdNaHLvVJGd9iAtV1EN1eGwJR0VTS14vPr55KPyce2Cg5vYOLPCky0wjEy/foEE6QxUu7YqSgTUV+AWWRHFGNMmpdhQjvSoA9wqjCKPJC05R23HrHrdCXvannSaXNkja1HlXBVpNu20SeCgfmwxkaE1nUEWZr9iNFY6dWaEA4RRepxKUnKX6p0aWjKi9dQ/4nZ9ny+h54+VcLlI4xqla8FipwmFSmcL+C+WTh3C6PVpGgF3pEuYtNL7pyCuErpls9H0fVl0xeRkkAFEKx017sfOT5/Cn1583N7e1S57Vv8ISF/K+gfy7ERleecDqWb3uvP9l8MLTbkbW6MPzZCm5C8PiyfFlIwmY+LBxBaFIcojIS8/n5j2f+W8Gj58N6XWtZvH+yJiWS8TLEg4fcQsaeMY26URD8JvSPfOu2Nr/FjarWpYTmEFK23xs9DIsv4raAY1u+xIQpFgW2uzq1qipwU8s0cWbs7/N05k+7Om6U4j6hiPjtPXMYEyknQ== X-MS-Exchange-AntiSpam-MessageData: Ka8lbGVtTqlRJ6bwsGK8ZcJh+yY1LNpmZEDXq6ElDkIQTcFnRMHr1y4rNzLYvVQv8LCSNiO+MYMVpumYBC4TE8cFdZqMS9jlohETpDDP6dJV/MvhiO759RcUWUC+ffCnrRUw6Ck0hc5K139QtVDpjg== X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 84a0eaa4-ff56-4ac4-8f0c-08d7abe5a6b6 X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Feb 2020 15:51:53.0955 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: g52PA6F5MoCj0Z7bprnsPHKUubAdHf6znv17xWBRKmGk3rM1GVSAPHlvd1JHJHvBtf+x2pP3afkN88oo194+Ug== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB3419 Content-Type: text/plain; charset=windows-1252 Content-Language: en-US Content-Transfer-Encoding: 7bit On 2/6/20 3:35 AM, Laszlo Ersek wrote: > On 02/05/20 00:01, Lendacky, Thomas wrote: >> BZ: https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.tianocore.org%2Fshow_bug.cgi%3Fid%3D2198&data=02%7C01%7Cthomas.lendacky%40amd.com%7Cf4521a4f24d54b8f122108d7aae7de74%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637165785175188824&sdata=l7ff9Q7OFJ5JYGHt0meU04nBuM9aP1o5RT0fl8s9PoA%3D&reserved=0 >> >> Typically, an AP is booted using the INIT-SIPI-SIPI sequence. This >> sequence is intercepted by the hypervisor, which sets the AP's registers >> to the values requested by the sequence. At that point, the hypervisor can >> start the AP, which will then begin execution at the appropriate location. >> >> Under SEV-ES, AP booting presents some challenges since the hypervisor is >> not allowed to alter the AP's register state. In this situation, we have >> to distinguish between the AP's first boot and AP's subsequent boots. >> >> First boot: >> Once the AP's register state has been defined (which is before the guest >> is first booted) it cannot be altered. Should the hypervisor attempt to >> alter the register state, the change would be detected by the hardware >> and the VMRUN instruction would fail. Given this, the first boot for the >> AP is required to begin execution with this initial register state, which >> is typically the reset vector. This prevents the BSP from directing the >> AP startup location through the INIT-SIPI-SIPI sequence. >> >> To work around this, the firmware will provide a build time reserved area >> that can be used as the initial IP value. The hypervisor can extract this >> location value by checking for the SEV-ES reset block GUID that must be >> located 48-bytes from the end of the firmware. The format of the SEV-ES >> reset block area is: >> >> 0x00 - 0x01 - SEV-ES Reset IP >> 0x02 - 0x03 - SEV-ES Reset CS Segment Base[32:16] > > Slight typo here I think: in the next patch, you correctly write [31:16]. Yup, good catch, should be 31. Thanks, Tom > > Thanks > Laszlo > >> 0x04 - 0x05 - Size of the SEV-ES reset block >> 0x06 - 0x15 - SEV-ES Reset Block GUID >> (00f771de-1a7e-4fcb-890e-68c77e2fb44e) >> >> The total size is 22 bytes. Any expansion to this block must be done >> by adding new values before existing values. >> >> The hypervisor will use the IP and CS values obtained from the SEV-ES >> reset block to set as the AP's initial values. The CS Segment Base >> represents the upper 16 bits of the CS segment base and must be left >> shifted by 16 bits to form the complete CS segment base value. >> >> Before booting the AP for the first time, the BSP must initialize the >> SEV-ES reset area. This consists of programming a FAR JMP instruction >> to the contents of a memory location that is also located in the SEV-ES >> reset area. The BSP must program the IP and CS values for the FAR JMP >> based on values drived from the INIT-SIPI-SIPI sequence. >> >> Subsequent boots: >> Again, the hypervisor cannot alter the AP register state, so a method is >> required to take the AP out of halt state and redirect it to the desired >> IP location. If it is determined that the AP is running in an SEV-ES >> guest, then instead of calling CpuSleep(), a VMGEXIT is issued with the >> AP Reset Hold exit code (0x80000004). The hypervisor will put the AP in >> a halt state, waiting for an INIT-SIPI-SIPI sequence. Once the sequence >> is recognized, the hypervisor will resume the AP. At this point the AP >> must transition from the current 64-bit long mode down to 16-bit real >> mode and begin executing at the derived location from the INIT-SIPI-SIPI >> sequence. >> >> Another change is around the area of obtaining the (x2)APIC ID during AP >> startup. During AP startup, the AP can't take a #VC exception before the >> AP has established a stack. However, the AP stack is set by using the >> (x2)APIC ID, which is obtained through CPUID instructions. A CPUID >> instruction will cause a #VC, so a different method must be used. The >> GHCB protocol supports a method to obtain CPUID information from the >> hypervisor through the GHCB MSR. This method does not require a stack, >> so it is used to obtain the necessary CPUID information to determine the >> (x2)APIC ID. >> >> The new 16-bit protected mode GDT entry is used in order to transition >> from 64-bit long mode down to 16-bit real mode. >> >> A new assembler routine is created that takes the AP from 64-bit long mode >> to 16-bit real mode. This is located under 1MB in memory and transitions >> from 64-bit long mode to 32-bit compatibility mode to 16-bit protected >> mode and finally 16-bit real mode. >> >> Cc: Eric Dong >> Cc: Ray Ni >> Cc: Laszlo Ersek >> Signed-off-by: Tom Lendacky >> --- >> UefiCpuPkg/Library/MpInitLib/DxeMpInitLib.inf | 3 + >> UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf | 3 + >> UefiCpuPkg/Library/MpInitLib/MpLib.h | 60 ++++ >> UefiCpuPkg/Library/MpInitLib/DxeMpLib.c | 70 ++++- >> UefiCpuPkg/Library/MpInitLib/MpLib.c | 256 +++++++++++++++++- >> UefiCpuPkg/Library/MpInitLib/PeiMpLib.c | 19 ++ >> UefiCpuPkg/PiSmmCpuDxeSmm/X64/SmmFuncsArch.c | 2 +- >> UefiCpuPkg/Library/MpInitLib/Ia32/MpEqu.inc | 2 +- >> .../Library/MpInitLib/Ia32/MpFuncs.nasm | 15 + >> UefiCpuPkg/Library/MpInitLib/X64/MpEqu.inc | 4 +- >> UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm | 239 ++++++++++++++++ >> 11 files changed, 659 insertions(+), 14 deletions(-) >> >> diff --git a/UefiCpuPkg/Library/MpInitLib/DxeMpInitLib.inf b/UefiCpuPkg/Library/MpInitLib/DxeMpInitLib.inf >> index 2c26f20c1972..c52951651851 100644 >> --- a/UefiCpuPkg/Library/MpInitLib/DxeMpInitLib.inf >> +++ b/UefiCpuPkg/Library/MpInitLib/DxeMpInitLib.inf >> @@ -51,6 +51,7 @@ [LibraryClasses] >> UefiBootServicesTableLib >> DebugAgentLib >> SynchronizationLib >> + VmgExitLib >> >> [Protocols] >> gEfiTimerArchProtocolGuid ## SOMETIMES_CONSUMES >> @@ -70,5 +71,7 @@ [Pcd] >> gUefiCpuPkgTokenSpaceGuid.PcdCpuApTargetCstate ## SOMETIMES_CONSUMES >> gUefiCpuPkgTokenSpaceGuid.PcdCpuShadowMicrocodeByFit ## CONSUMES >> gUefiCpuPkgTokenSpaceGuid.PcdSevEsIsEnabled ## CONSUMES >> + gUefiCpuPkgTokenSpaceGuid.PcdSevEsWorkAreaBase ## SOMETIMES_CONSUMES >> gEfiMdeModulePkgTokenSpaceGuid.PcdCpuStackGuard ## CONSUMES >> + gEfiMdeModulePkgTokenSpaceGuid.PcdGhcbBase ## CONSUMES >> >> diff --git a/UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf b/UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf >> index 66b2acfe98e7..ff392aeec763 100644 >> --- a/UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf >> +++ b/UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf >> @@ -50,6 +50,7 @@ [LibraryClasses] >> UefiCpuLib >> SynchronizationLib >> PeiServicesLib >> + VmgExitLib >> >> [Pcd] >> gUefiCpuPkgTokenSpaceGuid.PcdCpuMaxLogicalProcessorNumber ## CONSUMES >> @@ -62,6 +63,8 @@ [Pcd] >> gUefiCpuPkgTokenSpaceGuid.PcdCpuApTargetCstate ## SOMETIMES_CONSUMES >> gUefiCpuPkgTokenSpaceGuid.PcdCpuShadowMicrocodeByFit ## CONSUMES >> gUefiCpuPkgTokenSpaceGuid.PcdSevEsIsEnabled ## CONSUMES >> + gUefiCpuPkgTokenSpaceGuid.PcdSevEsWorkAreaBase ## SOMETIMES_CONSUMES >> + gEfiMdeModulePkgTokenSpaceGuid.PcdGhcbBase ## CONSUMES >> >> [Guids] >> gEdkiiS3SmmInitDoneGuid >> diff --git a/UefiCpuPkg/Library/MpInitLib/MpLib.h b/UefiCpuPkg/Library/MpInitLib/MpLib.h >> index 864b16872010..63d81ac3e42e 100644 >> --- a/UefiCpuPkg/Library/MpInitLib/MpLib.h >> +++ b/UefiCpuPkg/Library/MpInitLib/MpLib.h >> @@ -170,6 +170,11 @@ typedef struct { >> UINT8 *RelocateApLoopFuncAddress; >> UINTN RelocateApLoopFuncSize; >> UINTN ModeTransitionOffset; >> + UINTN SwitchToRealSize; >> + UINTN SwitchToRealOffset; >> + UINTN SwitchToRealNoNxOffset; >> + UINTN SwitchToRealPM16ModeOffset; >> + UINTN SwitchToRealPM16ModeSize; >> } MP_ASSEMBLY_ADDRESS_MAP; >> >> typedef struct _CPU_MP_DATA CPU_MP_DATA; >> @@ -208,6 +213,8 @@ typedef struct { >> // Enable5LevelPaging indicates whether 5-level paging is enabled in long mode. >> // >> BOOLEAN Enable5LevelPaging; >> + BOOLEAN SevEsIsEnabled; >> + UINTN GhcbBase; >> } MP_CPU_EXCHANGE_INFO; >> >> #pragma pack() >> @@ -256,6 +263,7 @@ struct _CPU_MP_DATA { >> UINT8 ApLoopMode; >> UINT8 ApTargetCState; >> UINT16 PmCodeSegment; >> + UINT16 Pm16CodeSegment; >> CPU_AP_DATA *CpuData; >> volatile MP_CPU_EXCHANGE_INFO *MpCpuExchangeInfo; >> >> @@ -275,8 +283,47 @@ struct _CPU_MP_DATA { >> BOOLEAN WakeUpByInitSipiSipi; >> >> BOOLEAN SevEsIsEnabled; >> + UINTN SevEsAPBuffer; >> + UINTN SevEsAPResetStackStart; >> + CPU_MP_DATA *NewCpuMpData; >> + >> + UINT64 GhcbBase; >> }; >> >> +#define AP_RESET_STACK_SIZE 64 >> + >> +#pragma pack(1) >> + >> +typedef struct { >> + UINT8 InsnBuffer[8]; >> + UINT16 Rip; >> + UINT16 Segment; >> +} SEV_ES_AP_JMP_FAR; >> + >> +#pragma pack() >> + >> +/** >> + Assembly code to move an AP from long mode to real mode. >> + >> + Move an AP from long mode to real mode in preparation to invoking >> + the reset vector. This is used for SEV-ES guests where a hypervisor >> + is not allowed to set the CS and RIP to point to the reset vector. >> + >> + @param[in] BufferStart The reset vector target. >> + @param[in] Code16 16-bit protected mode code segment value. >> + @param[in] Code32 32-bit protected mode code segment value. >> + @param[in] StackStart The start of a stack to be used for transitioning >> + from long mode to real mode. >> +**/ >> +typedef >> +VOID >> +(EFIAPI AP_RESET) ( >> + IN UINTN BufferStart, >> + IN UINT16 Code16, >> + IN UINT16 Code32, >> + IN UINTN StackStart >> + ); >> + >> extern EFI_GUID mCpuInitMpLibHobGuid; >> >> /** >> @@ -382,6 +429,19 @@ GetModeTransitionBuffer ( >> IN UINTN BufferSize >> ); >> >> +/** >> + Return the address of the SEV-ES AP jump table. >> + >> + This buffer is required in order for an SEV-ES guest to transition from >> + UEFI into an OS. >> + >> + @retval other Return SEV-ES AP jump table buffer >> +**/ >> +UINTN >> +GetSevEsAPMemory ( >> + VOID >> + ); >> + >> /** >> This function will be called by BSP to wakeup AP. >> >> diff --git a/UefiCpuPkg/Library/MpInitLib/DxeMpLib.c b/UefiCpuPkg/Library/MpInitLib/DxeMpLib.c >> index b17e287bbf49..8df5b6d919e6 100644 >> --- a/UefiCpuPkg/Library/MpInitLib/DxeMpLib.c >> +++ b/UefiCpuPkg/Library/MpInitLib/DxeMpLib.c >> @@ -12,6 +12,8 @@ >> #include >> #include >> #include >> +#include >> +#include >> >> #include >> >> @@ -145,6 +147,39 @@ GetModeTransitionBuffer ( >> return (UINTN)StartAddress; >> } >> >> +/** >> + Return the address of the SEV-ES AP jump table. >> + >> + This buffer is required in order for an SEV-ES guest to transition from >> + UEFI into an OS. >> + >> + @retval other Return SEV-ES AP jump table buffer >> +**/ >> +UINTN >> +GetSevEsAPMemory ( >> + VOID >> + ) >> +{ >> + EFI_STATUS Status; >> + EFI_PHYSICAL_ADDRESS StartAddress; >> + >> + // >> + // Allocate 1 page for AP jump table page >> + // >> + StartAddress = BASE_4GB - 1; >> + Status = gBS->AllocatePages ( >> + AllocateMaxAddress, >> + EfiReservedMemoryType, >> + 1, >> + &StartAddress >> + ); >> + ASSERT_EFI_ERROR (Status); >> + >> + DEBUG ((DEBUG_INFO, "Dxe: SevEsAPMemory = %lx\n", (UINTN) StartAddress)); >> + >> + return (UINTN) StartAddress; >> +} >> + >> /** >> Checks APs status and updates APs status if needed. >> >> @@ -219,6 +254,38 @@ CheckApsStatus ( >> } >> } >> >> +/** >> + Get Protected mode code segment with 16-bit default addressing >> + from current GDT table. >> + >> + @return Protected mode 16-bit code segment value. >> +**/ >> +UINT16 >> +GetProtectedMode16CS ( >> + VOID >> + ) >> +{ >> + IA32_DESCRIPTOR GdtrDesc; >> + IA32_SEGMENT_DESCRIPTOR *GdtEntry; >> + UINTN GdtEntryCount; >> + UINT16 Index; >> + >> + Index = (UINT16) -1; >> + AsmReadGdtr (&GdtrDesc); >> + GdtEntryCount = (GdtrDesc.Limit + 1) / sizeof (IA32_SEGMENT_DESCRIPTOR); >> + GdtEntry = (IA32_SEGMENT_DESCRIPTOR *) GdtrDesc.Base; >> + for (Index = 0; Index < GdtEntryCount; Index++) { >> + if (GdtEntry->Bits.L == 0) { >> + if (GdtEntry->Bits.Type > 8 && GdtEntry->Bits.DB == 0) { >> + break; >> + } >> + } >> + GdtEntry++; >> + } >> + ASSERT (Index != GdtEntryCount); >> + return Index * 8; >> +} >> + >> /** >> Get Protected mode code segment from current GDT table. >> >> @@ -239,7 +306,7 @@ GetProtectedModeCS ( >> GdtEntry = (IA32_SEGMENT_DESCRIPTOR *) GdtrDesc.Base; >> for (Index = 0; Index < GdtEntryCount; Index++) { >> if (GdtEntry->Bits.L == 0) { >> - if (GdtEntry->Bits.Type > 8 && GdtEntry->Bits.L == 0) { >> + if (GdtEntry->Bits.Type > 8 && GdtEntry->Bits.DB == 1) { >> break; >> } >> } >> @@ -301,6 +368,7 @@ MpInitChangeApLoopCallback ( >> >> CpuMpData = GetCpuMpData (); >> CpuMpData->PmCodeSegment = GetProtectedModeCS (); >> + CpuMpData->Pm16CodeSegment = GetProtectedMode16CS (); >> CpuMpData->ApLoopMode = PcdGet8 (PcdCpuApLoopMode); >> mNumberToFinish = CpuMpData->CpuCount - 1; >> WakeUpAP (CpuMpData, TRUE, 0, RelocateApLoop, NULL, TRUE); >> diff --git a/UefiCpuPkg/Library/MpInitLib/MpLib.c b/UefiCpuPkg/Library/MpInitLib/MpLib.c >> index 5e3183c2493b..ca8a3a3a7be9 100644 >> --- a/UefiCpuPkg/Library/MpInitLib/MpLib.c >> +++ b/UefiCpuPkg/Library/MpInitLib/MpLib.c >> @@ -7,6 +7,9 @@ >> **/ >> >> #include "MpLib.h" >> +#include >> +#include >> +#include >> >> EFI_GUID mCpuInitMpLibHobGuid = CPU_INIT_MP_LIB_HOB_GUID; >> >> @@ -288,6 +291,14 @@ GetApLoopMode ( >> // >> ApLoopMode = ApInHltLoop; >> } >> + >> + if (PcdGetBool (PcdSevEsIsEnabled)) { >> + // >> + // For SEV-ES, force AP in Hlt-loop mode in order to use the GHCB >> + // protocol for starting APs >> + // >> + ApLoopMode = ApInHltLoop; >> + } >> } >> >> if (ApLoopMode != ApInMwaitLoop) { >> @@ -579,6 +590,108 @@ InitializeApData ( >> SetApState (&CpuMpData->CpuData[ProcessorNumber], CpuStateIdle); >> } >> >> +/** >> + Get Protected mode code segment with 16-bit default addressing >> + from current GDT table. >> + >> + @return Protected mode 16-bit code segment value. >> +**/ >> +STATIC >> +UINT16 >> +GetProtectedMode16CS ( >> + VOID >> + ) >> +{ >> + IA32_DESCRIPTOR GdtrDesc; >> + IA32_SEGMENT_DESCRIPTOR *GdtEntry; >> + UINTN GdtEntryCount; >> + UINT16 Index; >> + >> + Index = (UINT16) -1; >> + AsmReadGdtr (&GdtrDesc); >> + GdtEntryCount = (GdtrDesc.Limit + 1) / sizeof (IA32_SEGMENT_DESCRIPTOR); >> + GdtEntry = (IA32_SEGMENT_DESCRIPTOR *) GdtrDesc.Base; >> + for (Index = 0; Index < GdtEntryCount; Index++) { >> + if (GdtEntry->Bits.L == 0 && >> + GdtEntry->Bits.DB == 0 && >> + GdtEntry->Bits.Type > 8) { >> + break; >> + } >> + GdtEntry++; >> + } >> + ASSERT (Index != GdtEntryCount); >> + return Index * 8; >> +} >> + >> +/** >> + Get Protected mode code segment with 32-bit default addressing >> + from current GDT table. >> + >> + @return Protected mode 32-bit code segment value. >> +**/ >> +STATIC >> +UINT16 >> +GetProtectedMode32CS ( >> + VOID >> + ) >> +{ >> + IA32_DESCRIPTOR GdtrDesc; >> + IA32_SEGMENT_DESCRIPTOR *GdtEntry; >> + UINTN GdtEntryCount; >> + UINT16 Index; >> + >> + Index = (UINT16) -1; >> + AsmReadGdtr (&GdtrDesc); >> + GdtEntryCount = (GdtrDesc.Limit + 1) / sizeof (IA32_SEGMENT_DESCRIPTOR); >> + GdtEntry = (IA32_SEGMENT_DESCRIPTOR *) GdtrDesc.Base; >> + for (Index = 0; Index < GdtEntryCount; Index++) { >> + if (GdtEntry->Bits.L == 0 && >> + GdtEntry->Bits.DB == 1 && >> + GdtEntry->Bits.Type > 8) { >> + break; >> + } >> + GdtEntry++; >> + } >> + ASSERT (Index != GdtEntryCount); >> + return Index * 8; >> +} >> + >> +/** >> + Reset an AP when in SEV-ES mode. >> + >> + @retval EFI_DEVICE_ERROR Reset of AP failed. >> +**/ >> +STATIC >> +VOID >> +MpInitLibSevEsAPReset ( >> + GHCB *Ghcb, >> + CPU_MP_DATA *CpuMpData >> + ) >> +{ >> + UINT16 Code16, Code32; >> + AP_RESET *APResetFn; >> + UINTN BufferStart; >> + UINTN StackStart; >> + >> + Code16 = GetProtectedMode16CS (); >> + Code32 = GetProtectedMode32CS (); >> + >> + if (CpuMpData->WakeupBufferHigh != 0) { >> + APResetFn = (AP_RESET *) (CpuMpData->WakeupBufferHigh + CpuMpData->AddressMap.SwitchToRealNoNxOffset); >> + } else { >> + APResetFn = (AP_RESET *) (CpuMpData->MpCpuExchangeInfo->BufferStart + CpuMpData->AddressMap.SwitchToRealOffset); >> + } >> + >> + BufferStart = CpuMpData->MpCpuExchangeInfo->BufferStart; >> + StackStart = CpuMpData->SevEsAPResetStackStart - >> + (AP_RESET_STACK_SIZE * GetApicId ()); >> + >> + // >> + // This call never returns. >> + // >> + APResetFn (BufferStart, Code16, Code32, StackStart); >> +} >> + >> /** >> This function will be called from AP reset code if BSP uses WakeUpAP. >> >> @@ -734,7 +847,28 @@ ApWakeupFunction ( >> // >> while (TRUE) { >> DisableInterrupts (); >> - CpuSleep (); >> + if (CpuMpData->SevEsIsEnabled) { >> + MSR_SEV_ES_GHCB_REGISTER Msr; >> + GHCB *Ghcb; >> + >> + Msr.GhcbPhysicalAddress = AsmReadMsr64 (MSR_SEV_ES_GHCB); >> + Ghcb = Msr.Ghcb; >> + >> + VmgInit (Ghcb); >> + VmgExit (Ghcb, SvmExitApResetHold, 0, 0); >> + /*TODO: Check return value to verify SIPI issued */ >> + >> + // >> + // Awakened in a new phase? Use the new CpuMpData >> + // >> + if (CpuMpData->NewCpuMpData) { >> + CpuMpData = CpuMpData->NewCpuMpData; >> + } >> + >> + MpInitLibSevEsAPReset (Ghcb, CpuMpData); >> + } else { >> + CpuSleep (); >> + } >> CpuPause (); >> } >> } >> @@ -847,6 +981,9 @@ FillExchangeInfoData ( >> ExchangeInfo->Enable5LevelPaging = (BOOLEAN) (Cr4.Bits.LA57 == 1); >> DEBUG ((DEBUG_INFO, "%a: 5-Level Paging = %d\n", gEfiCallerBaseName, ExchangeInfo->Enable5LevelPaging)); >> >> + ExchangeInfo->SevEsIsEnabled = CpuMpData->SevEsIsEnabled; >> + ExchangeInfo->GhcbBase = CpuMpData->GhcbBase; >> + >> // >> // Get the BSP's data of GDT and IDT >> // >> @@ -873,8 +1010,9 @@ FillExchangeInfoData ( >> // EfiBootServicesCode to avoid page fault if NX memory protection is enabled. >> // >> if (CpuMpData->WakeupBufferHigh != 0) { >> - Size = CpuMpData->AddressMap.RendezvousFunnelSize - >> - CpuMpData->AddressMap.ModeTransitionOffset; >> + Size = CpuMpData->AddressMap.RendezvousFunnelSize + >> + CpuMpData->AddressMap.SwitchToRealSize - >> + CpuMpData->AddressMap.ModeTransitionOffset; >> CopyMem ( >> (VOID *)CpuMpData->WakeupBufferHigh, >> CpuMpData->AddressMap.RendezvousFunnelAddress + >> @@ -927,7 +1065,8 @@ BackupAndPrepareWakeupBuffer( >> CopyMem ( >> (VOID *) CpuMpData->WakeupBuffer, >> (VOID *) CpuMpData->AddressMap.RendezvousFunnelAddress, >> - CpuMpData->AddressMap.RendezvousFunnelSize >> + CpuMpData->AddressMap.RendezvousFunnelSize + >> + CpuMpData->AddressMap.SwitchToRealSize >> ); >> } >> >> @@ -948,6 +1087,40 @@ RestoreWakeupBuffer( >> ); >> } >> >> +/** >> + Calculate the size of the reset stack. >> +**/ >> +STATIC >> +UINTN >> +GetApResetStackSize( >> + VOID >> + ) >> +{ >> + return AP_RESET_STACK_SIZE * PcdGet32(PcdCpuMaxLogicalProcessorNumber); >> +} >> + >> +/** >> + Calculate the size of the reset vector. >> + >> + @param[in] AddressMap The pointer to Address Map structure. >> +**/ >> +STATIC >> +UINTN >> +GetApResetVectorSize( >> + IN MP_ASSEMBLY_ADDRESS_MAP *AddressMap >> + ) >> +{ >> + UINTN Size; >> + >> + Size = ALIGN_VALUE (AddressMap->RendezvousFunnelSize + >> + AddressMap->SwitchToRealSize + >> + sizeof (MP_CPU_EXCHANGE_INFO), >> + CPU_STACK_ALIGNMENT); >> + Size += GetApResetStackSize (); >> + >> + return Size; >> +} >> + >> /** >> Allocate reset vector buffer. >> >> @@ -961,16 +1134,22 @@ AllocateResetVector ( >> UINTN ApResetVectorSize; >> >> if (CpuMpData->WakeupBuffer == (UINTN) -1) { >> - ApResetVectorSize = CpuMpData->AddressMap.RendezvousFunnelSize + >> - sizeof (MP_CPU_EXCHANGE_INFO); >> + ApResetVectorSize = GetApResetVectorSize (&CpuMpData->AddressMap); >> >> CpuMpData->WakeupBuffer = GetWakeupBuffer (ApResetVectorSize); >> CpuMpData->MpCpuExchangeInfo = (MP_CPU_EXCHANGE_INFO *) (UINTN) >> - (CpuMpData->WakeupBuffer + CpuMpData->AddressMap.RendezvousFunnelSize); >> + (CpuMpData->WakeupBuffer + >> + CpuMpData->AddressMap.RendezvousFunnelSize + >> + CpuMpData->AddressMap.SwitchToRealSize); >> CpuMpData->WakeupBufferHigh = GetModeTransitionBuffer ( >> - CpuMpData->AddressMap.RendezvousFunnelSize - >> + CpuMpData->AddressMap.RendezvousFunnelSize + >> + CpuMpData->AddressMap.SwitchToRealSize - >> CpuMpData->AddressMap.ModeTransitionOffset >> ); >> + // >> + // The reset stack starts at the end of the buffer. >> + // >> + CpuMpData->SevEsAPResetStackStart = CpuMpData->WakeupBuffer + ApResetVectorSize; >> } >> BackupAndPrepareWakeupBuffer (CpuMpData); >> } >> @@ -985,7 +1164,31 @@ FreeResetVector ( >> IN CPU_MP_DATA *CpuMpData >> ) >> { >> - RestoreWakeupBuffer (CpuMpData); >> + // >> + // If SEV-ES is enabled, the reset area is needed for AP parking and >> + // and AP startup in the OS, so the reset area is reserved. Do not >> + // perform the restore as this will overwrite memory which has data >> + // needed by SEV-ES. >> + // >> + if (!CpuMpData->SevEsIsEnabled) { >> + RestoreWakeupBuffer (CpuMpData); >> + } >> +} >> + >> +/** >> + Allocate the SEV-ES AP jump table buffer. >> + >> + @param[in, out] CpuMpData The pointer to CPU MP Data structure. >> +**/ >> +VOID >> +AllocateSevEsAPMemory ( >> + IN OUT CPU_MP_DATA *CpuMpData >> + ) >> +{ >> + if (CpuMpData->SevEsAPBuffer == (UINTN) -1) { >> + CpuMpData->SevEsAPBuffer = >> + CpuMpData->SevEsIsEnabled ? GetSevEsAPMemory () : 0; >> + } >> } >> >> /** >> @@ -1022,6 +1225,7 @@ WakeUpAP ( >> CpuMpData->InitFlag != ApInitDone) { >> ResetVectorRequired = TRUE; >> AllocateResetVector (CpuMpData); >> + AllocateSevEsAPMemory (CpuMpData); >> FillExchangeInfoData (CpuMpData); >> SaveLocalApicTimerSetting (CpuMpData); >> } >> @@ -1058,6 +1262,35 @@ WakeUpAP ( >> } >> } >> if (ResetVectorRequired) { >> + // >> + // For SEV-ES, the initial AP boot address will be defined by >> + // PcdSevEsWorkAreaBase. The Segment/Rip must be the jump address >> + // from the original INIT-SIPI-SIPI. >> + // >> + if (CpuMpData->SevEsIsEnabled) { >> + SEV_ES_AP_JMP_FAR *JmpFar; >> + UINT32 Offset, InsnByte; >> + UINT8 LoNib, HiNib; >> + >> + JmpFar = (SEV_ES_AP_JMP_FAR *) FixedPcdGet32 (PcdSevEsWorkAreaBase); >> + ASSERT (JmpFar != NULL); >> + >> + Offset = FixedPcdGet32 (PcdSevEsWorkAreaBase); >> + Offset += sizeof(JmpFar->InsnBuffer); >> + LoNib = (UINT8) Offset; >> + HiNib = (UINT8) (Offset >> 8); >> + >> + // JMP FAR [CS:XXYY] => 2E FF 2E YY XX >> + InsnByte = 0; >> + JmpFar->InsnBuffer[InsnByte++] = 0x2E; // CS override prefix >> + JmpFar->InsnBuffer[InsnByte++] = 0xFF; // JMP (FAR) >> + JmpFar->InsnBuffer[InsnByte++] = 0x2E; // ModRM (JMP memory location) >> + JmpFar->InsnBuffer[InsnByte++] = LoNib; // YY offset ... >> + JmpFar->InsnBuffer[InsnByte++] = HiNib; // XX offset ... >> + >> + JmpFar->Rip = 0; >> + JmpFar->Segment = (UINT16) (ExchangeInfo->BufferStart >> 4); >> + } >> // >> // Wakeup all APs >> // >> @@ -1625,7 +1858,7 @@ MpInitLibInitialize ( >> ASSERT (MaxLogicalProcessorNumber != 0); >> >> AsmGetAddressMap (&AddressMap); >> - ApResetVectorSize = AddressMap.RendezvousFunnelSize + sizeof (MP_CPU_EXCHANGE_INFO); >> + ApResetVectorSize = GetApResetVectorSize (&AddressMap); >> ApStackSize = PcdGet32(PcdCpuApStackSize); >> ApLoopMode = GetApLoopMode (&MonitorFilterSize); >> >> @@ -1688,6 +1921,8 @@ MpInitLibInitialize ( >> } >> InitializeSpinLock(&CpuMpData->MpLock); >> CpuMpData->SevEsIsEnabled = PcdGetBool (PcdSevEsIsEnabled); >> + CpuMpData->SevEsAPBuffer = (UINTN) -1; >> + CpuMpData->GhcbBase = PcdGet64 (PcdGhcbBase); >> >> // >> // Make sure no memory usage outside of the allocated buffer. >> @@ -1751,6 +1986,7 @@ MpInitLibInitialize ( >> // APs have been wakeup before, just get the CPU Information >> // from HOB >> // >> + OldCpuMpData->NewCpuMpData = CpuMpData; >> CpuMpData->CpuCount = OldCpuMpData->CpuCount; >> CpuMpData->BspNumber = OldCpuMpData->BspNumber; >> CpuMpData->CpuInfoInHob = OldCpuMpData->CpuInfoInHob; >> diff --git a/UefiCpuPkg/Library/MpInitLib/PeiMpLib.c b/UefiCpuPkg/Library/MpInitLib/PeiMpLib.c >> index 06e3f5d0d3da..e8103a9ce094 100644 >> --- a/UefiCpuPkg/Library/MpInitLib/PeiMpLib.c >> +++ b/UefiCpuPkg/Library/MpInitLib/PeiMpLib.c >> @@ -280,6 +280,25 @@ GetModeTransitionBuffer ( >> return 0; >> } >> >> +/** >> + Return the address of the SEV-ES AP jump table. >> + >> + This buffer is required in order for an SEV-ES guest to transition from >> + UEFI into an OS. >> + >> + @retval other Return SEV-ES AP jump table buffer >> +**/ >> +UINTN >> +GetSevEsAPMemory ( >> + VOID >> + ) >> +{ >> + // >> + // PEI phase doesn't need to do such transition. So simply return 0. >> + // >> + return 0; >> +} >> + >> /** >> Checks APs status and updates APs status if needed. >> >> diff --git a/UefiCpuPkg/PiSmmCpuDxeSmm/X64/SmmFuncsArch.c b/UefiCpuPkg/PiSmmCpuDxeSmm/X64/SmmFuncsArch.c >> index 6298571e29b2..28f8e8e133e5 100644 >> --- a/UefiCpuPkg/PiSmmCpuDxeSmm/X64/SmmFuncsArch.c >> +++ b/UefiCpuPkg/PiSmmCpuDxeSmm/X64/SmmFuncsArch.c >> @@ -121,7 +121,7 @@ GetProtectedModeCS ( >> GdtEntry = (IA32_SEGMENT_DESCRIPTOR *) GdtrDesc.Base; >> for (Index = 0; Index < GdtEntryCount; Index++) { >> if (GdtEntry->Bits.L == 0) { >> - if (GdtEntry->Bits.Type > 8 && GdtEntry->Bits.L == 0) { >> + if (GdtEntry->Bits.Type > 8 && GdtEntry->Bits.DB == 1) { >> break; >> } >> } >> diff --git a/UefiCpuPkg/Library/MpInitLib/Ia32/MpEqu.inc b/UefiCpuPkg/Library/MpInitLib/Ia32/MpEqu.inc >> index efb1bc2bf7cb..4f5a7c859a56 100644 >> --- a/UefiCpuPkg/Library/MpInitLib/Ia32/MpEqu.inc >> +++ b/UefiCpuPkg/Library/MpInitLib/Ia32/MpEqu.inc >> @@ -19,7 +19,7 @@ CPU_SWITCH_STATE_IDLE equ 0 >> CPU_SWITCH_STATE_STORED equ 1 >> CPU_SWITCH_STATE_LOADED equ 2 >> >> -LockLocation equ (RendezvousFunnelProcEnd - RendezvousFunnelProcStart) >> +LockLocation equ (SwitchToRealProcEnd - RendezvousFunnelProcStart) >> StackStartAddressLocation equ LockLocation + 04h >> StackSizeLocation equ LockLocation + 08h >> ApProcedureLocation equ LockLocation + 0Ch >> diff --git a/UefiCpuPkg/Library/MpInitLib/Ia32/MpFuncs.nasm b/UefiCpuPkg/Library/MpInitLib/Ia32/MpFuncs.nasm >> index b74046b76af3..309d53bf3b37 100644 >> --- a/UefiCpuPkg/Library/MpInitLib/Ia32/MpFuncs.nasm >> +++ b/UefiCpuPkg/Library/MpInitLib/Ia32/MpFuncs.nasm >> @@ -215,6 +215,16 @@ CProcedureInvoke: >> jmp $ ; Never reach here >> RendezvousFunnelProcEnd: >> >> +;------------------------------------------------------------------------------------- >> +;SwitchToRealProc procedure follows. >> +;NOT USED IN 32 BIT MODE. >> +;------------------------------------------------------------------------------------- >> +global ASM_PFX(SwitchToRealProc) >> +ASM_PFX(SwitchToRealProc): >> +SwitchToRealProcStart: >> + jmp $ ; Never reach here >> +SwitchToRealProcEnd: >> + >> ;------------------------------------------------------------------------------------- >> ; AsmRelocateApLoop (MwaitSupport, ApTargetCState, PmCodeSegment, TopOfApStack, CountTofinish); >> ;------------------------------------------------------------------------------------- >> @@ -263,6 +273,11 @@ ASM_PFX(AsmGetAddressMap): >> mov dword [ebx + 0Ch], AsmRelocateApLoopStart >> mov dword [ebx + 10h], AsmRelocateApLoopEnd - AsmRelocateApLoopStart >> mov dword [ebx + 14h], Flat32Start - RendezvousFunnelProcStart >> + mov dword [ebx + 18h], SwitchToRealProcEnd - SwitchToRealProcStart ; SwitchToRealSize >> + mov dword [ebx + 1Ch], SwitchToRealProcStart - RendezvousFunnelProcStart ; SwitchToRealOffset >> + mov dword [ebx + 20h], SwitchToRealProcStart - Flat32Start ; SwitchToRealNoNxOffset >> + mov dword [ebx + 24h], 0 ; SwitchToRealPM16ModeOffset >> + mov dword [ebx + 28h], 0 ; SwitchToRealPM16ModeSize >> >> popad >> ret >> diff --git a/UefiCpuPkg/Library/MpInitLib/X64/MpEqu.inc b/UefiCpuPkg/Library/MpInitLib/X64/MpEqu.inc >> index 58ef369342a7..c92daaaffd6b 100644 >> --- a/UefiCpuPkg/Library/MpInitLib/X64/MpEqu.inc >> +++ b/UefiCpuPkg/Library/MpInitLib/X64/MpEqu.inc >> @@ -19,7 +19,7 @@ CPU_SWITCH_STATE_IDLE equ 0 >> CPU_SWITCH_STATE_STORED equ 1 >> CPU_SWITCH_STATE_LOADED equ 2 >> >> -LockLocation equ (RendezvousFunnelProcEnd - RendezvousFunnelProcStart) >> +LockLocation equ (SwitchToRealProcEnd - RendezvousFunnelProcStart) >> StackStartAddressLocation equ LockLocation + 08h >> StackSizeLocation equ LockLocation + 10h >> ApProcedureLocation equ LockLocation + 18h >> @@ -41,3 +41,5 @@ ModeTransitionSegmentLocation equ LockLocation + 98h >> ModeHighMemoryLocation equ LockLocation + 9Ah >> ModeHighSegmentLocation equ LockLocation + 9Eh >> Enable5LevelPagingLocation equ LockLocation + 0A0h >> +SevEsIsEnabledLocation equ LockLocation + 0A1h >> +GhcbBaseLocation equ LockLocation + 0A2h >> diff --git a/UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm b/UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm >> index 87f2523e856f..6956b408d004 100644 >> --- a/UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm >> +++ b/UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm >> @@ -184,9 +184,97 @@ Releaselock: >> add edi, StackStartAddressLocation >> add rax, qword [edi] >> mov rsp, rax >> + >> + lea edi, [esi + SevEsIsEnabledLocation] >> + cmp byte [edi], 1 ; SevEsIsEnabled >> + jne CProcedureInvoke >> + >> + ; >> + ; program GHCB >> + ; Each page after the GHCB is a per-CPU page, so the calculation programs >> + ; a GHCB to be every 8KB. >> + ; >> + mov eax, SIZE_4KB >> + shl eax, 1 ; EAX = SIZE_4K * 2 >> + mov ecx, ebx >> + mul ecx ; EAX = SIZE_4K * 2 * CpuNumber >> + mov edi, esi >> + add edi, GhcbBaseLocation >> + add rax, qword [edi] >> + mov rdx, rax >> + shr rdx, 32 >> + mov rcx, 0xc0010130 >> + wrmsr >> jmp CProcedureInvoke >> >> GetApicId: >> + lea edi, [esi + SevEsIsEnabledLocation] >> + cmp byte [edi], 1 ; SevEsIsEnabled >> + jne DoCpuid >> + >> + ; >> + ; Since we don't have a stack yet, we can't take a #VC >> + ; exception. Use the GHCB protocol to perform the CPUID >> + ; calls. >> + ; >> + mov rcx, 0xc0010130 >> + rdmsr >> + shl rdx, 32 >> + or rax, rdx >> + mov rdi, rax ; RDI now holds the original GHCB GPA >> + >> + mov rdx, 0 ; CPUID function 0 >> + mov rax, 0 ; RAX register requested >> + or rax, 4 >> + wrmsr >> + rep vmmcall >> + rdmsr >> + cmp edx, 0bh >> + jb NoX2ApicSevEs ; CPUID level below CPUID_EXTENDED_TOPOLOGY >> + >> + mov rdx, 0bh ; CPUID function 0x0b >> + mov rax, 040000000h ; RBX register requested >> + or rax, 4 >> + wrmsr >> + rep vmmcall >> + rdmsr >> + test edx, 0ffffh >> + jz NoX2ApicSevEs ; CPUID.0BH:EBX[15:0] is zero >> + >> + mov rdx, 0bh ; CPUID function 0x0b >> + mov rax, 0c0000000h ; RDX register requested >> + or rax, 4 >> + wrmsr >> + rep vmmcall >> + rdmsr >> + >> + ; Processor is x2APIC capable; 32-bit x2APIC ID is now in EDX >> + jmp RestoreGhcb >> + >> +NoX2ApicSevEs: >> + ; Processor is not x2APIC capable, so get 8-bit APIC ID >> + mov rdx, 1 ; CPUID function 1 >> + mov rax, 040000000h ; RBX register requested >> + or rax, 4 >> + wrmsr >> + rep vmmcall >> + rdmsr >> + shr edx, 24 >> + >> +RestoreGhcb: >> + mov rbx, rdx ; Save x2APIC/APIC ID >> + >> + mov rdx, rdi ; RDI holds the saved GHCB GPA >> + shr rdx, 32 >> + mov eax, edi >> + wrmsr >> + >> + mov rdx, rbx >> + >> + ; x2APIC ID or APIC ID is in EDX >> + jmp GetProcessorNumber >> + >> +DoCpuid: >> mov eax, 0 >> cpuid >> cmp eax, 0bh >> @@ -253,12 +341,158 @@ CProcedureInvoke: >> >> RendezvousFunnelProcEnd: >> >> +;------------------------------------------------------------------------------------- >> +;SwitchToRealProc procedure follows. >> +;ALSO THIS PROCEDURE IS EXECUTED BY APs TRANSITIONING TO 16 BIT MODE. HENCE THIS PROC >> +;IS IN MACHINE CODE. >> +; SwitchToRealProc (UINTN BufferStart, UINT16 Code16, UINT16 Code32, UINTN StackStart) >> +; rcx - Buffer Start >> +; rdx - Code16 Selector Offset >> +; r8 - Code32 Selector Offset >> +; r9 - Stack Start >> +;------------------------------------------------------------------------------------- >> +global ASM_PFX(SwitchToRealProc) >> +ASM_PFX(SwitchToRealProc): >> +SwitchToRealProcStart: >> +BITS 64 >> + cli >> + >> + ; >> + ; Get RDX reset value before changing stacks since the >> + ; new stack won't be able to accomodate a #VC exception. >> + ; >> + push rax >> + push rbx >> + push rcx >> + push rdx >> + >> + mov rax, 1 >> + cpuid >> + mov rsi, rax ; Save off the reset value for RDX >> + >> + pop rdx >> + pop rcx >> + pop rbx >> + pop rax >> + >> + ; >> + ; Establish stack below 1MB >> + ; >> + mov rsp, r9 >> + >> + ; >> + ; Push ultimate Reset Vector onto the stack >> + ; >> + mov rax, rcx >> + shr rax, 4 >> + push word 0x0002 ; RFLAGS >> + push ax ; CS >> + push word 0x0000 ; RIP >> + push word 0x0000 ; For alignment, will be discarded >> + >> + ; >> + ; Get address of "16-bit operand size" label >> + ; >> + lea rbx, [PM16Mode] >> + >> + ; >> + ; Push addresses used to change to compatibility mode >> + ; >> + lea rax, [CompatMode] >> + push r8 >> + push rax >> + >> + ; >> + ; Clear R8 - R15, for reset, before going into 32-bit mode >> + ; >> + xor r8, r8 >> + xor r9, r9 >> + xor r10, r10 >> + xor r11, r11 >> + xor r12, r12 >> + xor r13, r13 >> + xor r14, r14 >> + xor r15, r15 >> + >> + ; >> + ; Far return into 32-bit mode >> + ; >> +o64 retf >> + >> +BITS 32 >> +CompatMode: >> + ; >> + ; Set up stack to prepare for exiting protected mode >> + ; >> + push edx ; Code16 CS >> + push ebx ; PM16Mode label address >> + >> + ; >> + ; Disable paging >> + ; >> + mov eax, cr0 ; Read CR0 >> + btr eax, 31 ; Set PG=0 >> + mov cr0, eax ; Write CR0 >> + >> + ; >> + ; Disable long mode >> + ; >> + mov ecx, 0c0000080h ; EFER MSR number >> + rdmsr ; Read EFER >> + btr eax, 8 ; Set LME=0 >> + wrmsr ; Write EFER >> + >> + ; >> + ; Disable PAE >> + ; >> + mov eax, cr4 ; Read CR4 >> + btr eax, 5 ; Set PAE=0 >> + mov cr4, eax ; Write CR4 >> + >> + mov edx, esi ; Restore RDX reset value >> + >> + ; >> + ; Switch to 16-bit operand size >> + ; >> + retf >> + >> +BITS 16 >> + ; >> + ; At entry to this label >> + ; - RDX will have its reset value >> + ; - On the top of the stack >> + ; - Alignment data (two bytes) to be discarded >> + ; - IP for Real Mode (two bytes) >> + ; - CS for Real Mode (two bytes) >> + ; >> +PM16Mode: >> + mov eax, cr0 ; Read CR0 >> + btr eax, 0 ; Set PE=0 >> + mov cr0, eax ; Write CR0 >> + >> + pop ax ; Discard alignment data >> + >> + ; >> + ; Clear registers (except RDX and RSP) before going into 16-bit mode >> + ; >> + xor eax, eax >> + xor ebx, ebx >> + xor ecx, ecx >> + xor esi, esi >> + xor edi, edi >> + xor ebp, ebp >> + >> + iret >> + >> +SwitchToRealProcEnd: >> + >> ;------------------------------------------------------------------------------------- >> ; AsmRelocateApLoop (MwaitSupport, ApTargetCState, PmCodeSegment, TopOfApStack, CountTofinish); >> ;------------------------------------------------------------------------------------- >> global ASM_PFX(AsmRelocateApLoop) >> ASM_PFX(AsmRelocateApLoop): >> AsmRelocateApLoopStart: >> +BITS 64 >> cli ; Disable interrupt before switching to 32-bit mode >> mov rax, [rsp + 40] ; CountTofinish >> lock dec dword [rax] ; (*CountTofinish)-- >> @@ -324,6 +558,11 @@ ASM_PFX(AsmGetAddressMap): >> mov qword [rcx + 18h], rax >> mov qword [rcx + 20h], AsmRelocateApLoopEnd - AsmRelocateApLoopStart >> mov qword [rcx + 28h], Flat32Start - RendezvousFunnelProcStart >> + mov qword [rcx + 30h], SwitchToRealProcEnd - SwitchToRealProcStart ; SwitchToRealSize >> + mov qword [rcx + 38h], SwitchToRealProcStart - RendezvousFunnelProcStart ; SwitchToRealOffset >> + mov qword [rcx + 40h], SwitchToRealProcStart - Flat32Start ; SwitchToRealNoNxOffset >> + mov qword [rcx + 48h], PM16Mode - RendezvousFunnelProcStart ; SwitchToRealPM16ModeOffset >> + mov qword [rcx + 50h], SwitchToRealProcEnd - PM16Mode ; SwitchToRealPM16ModeSize >> ret >> >> ;------------------------------------------------------------------------------------- >> >