From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by mx.groups.io with SMTP id smtpd.web11.1179.1594792557416989933 for ; Tue, 14 Jul 2020 22:55:57 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@intel.onmicrosoft.com header.s=selector2-intel-onmicrosoft-com header.b=ibPXTZrf; spf=pass (domain: intel.com, ip: 134.134.136.126, mailfrom: eric.dong@intel.com) IronPort-SDR: GR8NRymQBzA5YLbeLTtHxIO2L2QN29918Pe2kPSXguHHe17nPVt8AFL4dOF9FhpsziS4OMkTYR yGuzFBi9DCLQ== X-IronPort-AV: E=McAfee;i="6000,8403,9682"; a="136545309" X-IronPort-AV: E=Sophos;i="5.75,354,1589266800"; d="scan'208";a="136545309" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2020 22:55:56 -0700 IronPort-SDR: mZup3/VarkIu/mrlgh2fFrWnca5yeIza6y1xeojUMNq7fSx8bRUaLC2TRZlib42x3rXubQOsS5 /dZmaZMtbmcQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.75,354,1589266800"; d="scan'208";a="270195155" Received: from fmsmsx105.amr.corp.intel.com ([10.18.124.203]) by fmsmga008.fm.intel.com with ESMTP; 14 Jul 2020 22:55:56 -0700 Received: from fmsmsx156.amr.corp.intel.com (10.18.116.74) by FMSMSX105.amr.corp.intel.com (10.18.124.203) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 14 Jul 2020 22:55:56 -0700 Received: from FMSEDG001.ED.cps.intel.com (10.1.192.133) by fmsmsx156.amr.corp.intel.com (10.18.116.74) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 14 Jul 2020 22:55:55 -0700 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (104.47.66.47) by edgegateway.intel.com (192.55.55.68) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 14 Jul 2020 22:55:56 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kCt1V8jS6/WLiSRXQVpQVK1AVf4tQzrQqagE8XJuIDoeS9dTo2ApW53j/NN2KkjAIoqMyNN8liQvPgLS0aNRmgjgsck2Bux5VFv3z4okZ+jfsbNOxKFNcYku33JWKAMDKgNMy+owzKFiy2iBqiVeZJHgP6Sd1e0ZbngBOYLkynE8lB+NvEfdoBI19o5zppOBQ3vNe6ygSboLgNdLHgX7UX/8b0HpJx3TIIvgm1iZqSitaDVvcegx5/BLE0xTNOrWbTTY4FPv3ieE8DIuftJLbvAiI2tGW0sI/HUEo4M1YA+flay+hw3CPn+P18szA3jVAinFQAOr4lbaN0YzHjoDiA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=sPu5PtGjPS+OuOvee5E0U6+ift4hA3YmcU7OssanVE4=; b=mS1n9bI4uaEcmEboO9Lmb1ukf3Dw5UNtAeZGzY+14AWZkVhc6dnaHVMh36uTOShlq5h9v8A1hovmarg2crCrifRb18gHKMD0C23QPTqED0WQbrzUT1gthUQIwRmtXDZBdNWrMX8yMwqHWFztdIx+WLWD98f32X6gJLWY0Ypih1k59OEmv8oOjC8EmE5DlPuVrvKa9oSRlLNL2KMkK9Teia6XVxutOyeywIZZE0xs6aoR9Z4BmJXs5m0L3Yvoc4fpVOLnGchkj0+SeUdh0UvoJ5CduDzZElvWT1Zi7d3DEtyn5fYhbZAs6seDBLKAZjTDu5kQRKV6HCYFwUiXk7sD5w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=sPu5PtGjPS+OuOvee5E0U6+ift4hA3YmcU7OssanVE4=; b=ibPXTZrfUdy1M7F/N42s93zMbjqRw8Ta0ileL9LULyOidkn+usigoIu5QXpsElCVbcZ6Dmn0z8HIr9Pz/l1b3bhnVfsur2Dr1JYf6h1jVydt7R7CwmX/keYIZyUiMKsR/kYmNze3Qd1Haqc7jF6pfWOW0kurtxSyIJdSpS0oUlU= Received: from DM6PR11MB3274.namprd11.prod.outlook.com (2603:10b6:5:b::26) by DM5PR11MB0011.namprd11.prod.outlook.com (2603:10b6:4:6b::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3174.25; Wed, 15 Jul 2020 05:55:54 +0000 Received: from DM6PR11MB3274.namprd11.prod.outlook.com ([fe80::cc01:6f05:1402:e7d7]) by DM6PR11MB3274.namprd11.prod.outlook.com ([fe80::cc01:6f05:1402:e7d7%6]) with mapi id 15.20.3174.025; Wed, 15 Jul 2020 05:55:54 +0000 From: "Dong, Eric" To: Tom Lendacky , "devel@edk2.groups.io" CC: Brijesh Singh , Ard Biesheuvel , "Justen, Jordan L" , Laszlo Ersek , "Gao, Liming" , "Kinney, Michael D" , "Ni, Ray" Subject: Re: [PATCH v10 45/46] UefiCpuPkg/MpInitLib: Prepare SEV-ES guest APs for OS use Thread-Topic: [PATCH v10 45/46] UefiCpuPkg/MpInitLib: Prepare SEV-ES guest APs for OS use Thread-Index: AQHWWeyMPia3grl+K0KCIKRLpVy9tqkIJUHw Date: Wed, 15 Jul 2020 05:55:54 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: amd.com; dkim=none (message not signed) header.d=none;amd.com; dmarc=none action=none header.from=intel.com; x-originating-ip: [192.102.204.38] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: bbb1523d-f6cd-4dfa-97cd-08d82883bcab x-ms-traffictypediagnostic: DM5PR11MB0011: x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:8273; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: XA7IzpFETEQXgANV6kcfkEXNaYOOZpxlkXrN0P/hrqhUBkAYC/elPXmplWIxrKpRYjfZ2iN8sPgWAZjT73lQOGyjoNGt351DRfY7fR7PwLmXn9lOR2KW5YmpHqceMZf8rQ7c9YpUbzS20KbfD3g6r7yS/SoAaV/MAG6GNSlVPP20JnNMqTzBnX03gUreP7/a+7yB6uHBHElykclAB3hZ5LsXaXWWy3hZmGMYIOJ4T6+WY0m+8ioii8yzn18BATvQp58ZYa3k/luEI+zAyIMW9BBTW/OJISPqlJfo0SyN5Y1bfp+TuDpfcVRx8mlSft2gPkTI3ENehU0wnC8dIhX/VP4/sQ4Fo9nnMf50ahlsUybJfbfTmA6RIHS92UtXwNM9ng9UWQIlMSFgU1gXL1G+QA== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR11MB3274.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFTY:;SFS:(6029001)(4636009)(136003)(39860400002)(396003)(366004)(376002)(346002)(186003)(8676002)(30864003)(966005)(478600001)(52536014)(71200400001)(8936002)(5660300002)(26005)(7696005)(53546011)(6506007)(83380400001)(86362001)(316002)(107886003)(54906003)(110136005)(4326008)(2906002)(19627235002)(66946007)(66476007)(33656002)(66446008)(55016002)(76116006)(9686003)(64756008)(66556008);DIR:OUT;SFP:1102; x-ms-exchange-antispam-messagedata: VYDnrU0vshDu0SG7Dfr3pEUDfP6BLbxSlRpbiLxIGsiS8dcx2R/U9DPNDppyPBjnVW1fE7uupAcyalAimaWWx4vWbxkDOuKQXqOB4btlrLUUnRDp3I0XbLvoyK+nWg3RI5OMJSoPHSk4EtoDXJqvCpWeJSQ/iHSRmSA+bODOnx0KUFrgI6gk3rLxn11ATE2Y0QHVbRk3fJ8pWA05kZo3SuQ6kvbV+rIx/y7KYVe8g5/S2Kk54EMCKqVfyE92Cm54BsuwjKqh53EvLNeqqW0iICahpwpEDvP67iDrgeRg4pHMQoRQMgaCkldwp/xAggK7Q5jVSqfOIsZXBzqHK1yJahVpWFZNzwhLjtyoH/pa8nlQ1CeUZjLWIXAOvtavWg88tGyj5WshfAcp3qcD/X9fhHlwllZ1e2Zr7jT3MfGp3qdIq+niYKGym76H8G/oLwU/8+gH3JoqivA43ONWUP68XplnEzlIWxy947XH4TbSap0zNQrlxhEfeFmwbFSSRTRX MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM6PR11MB3274.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: bbb1523d-f6cd-4dfa-97cd-08d82883bcab X-MS-Exchange-CrossTenant-originalarrivaltime: 15 Jul 2020 05:55:54.2804 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: oHMNV8RJSE+1c/yRLj2QGUHsPRVMMIqQHrI5x68n12DBpacXboz+cbxCNpt+RMJI+wwgKAqdT14bShixF2IrIg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR11MB0011 Return-Path: eric.dong@intel.com X-OriginatorOrg: intel.com Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Reviewed-by: Eric Dong > -----Original Message----- > From: Tom Lendacky > Sent: Tuesday, July 14, 2020 10:38 PM > To: devel@edk2.groups.io > Cc: Brijesh Singh ; Ard Biesheuvel > ; Dong, Eric ; Justen, > Jordan L ; Laszlo Ersek ; > Gao, Liming ; Kinney, Michael D > ; Ni, Ray > Subject: [PATCH v10 45/46] UefiCpuPkg/MpInitLib: Prepare SEV-ES guest APs > for OS use >=20 > From: Tom Lendacky >=20 > BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3D2198 >=20 > Before UEFI transfers control to the OS, it must park the AP. This is don= e > using the AsmRelocateApLoop function to transition into 32-bit non-paging > mode. For an SEV-ES guest, a few additional things must be > done: > - AsmRelocateApLoop must be updated to support SEV-ES. This means > performing a VMGEXIT AP Reset Hold instead of an MWAIT or HLT loop. > - Since the AP must transition to real mode, a small routine is copied > to the WakeupBuffer area. Since the WakeupBuffer will be used by > the AP during OS booting, it must be placed in reserved memory. > Additionally, the AP stack must be located where it can be accessed > in real mode. > - Once the AP is in real mode it will transfer control to the > destination specified by the OS in the SEV-ES AP Jump Table. The > SEV-ES AP Jump Table address is saved by the hypervisor for the OS > using the GHCB VMGEXIT AP Jump Table exit code. >=20 > Cc: Eric Dong > Cc: Ray Ni > Cc: Laszlo Ersek > Signed-off-by: Tom Lendacky > --- > UefiCpuPkg/Library/MpInitLib/MpLib.h | 8 +- > UefiCpuPkg/Library/MpInitLib/DxeMpLib.c | 54 +++++++- > UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm | 131 > ++++++++++++++++-- > 3 files changed, 175 insertions(+), 18 deletions(-) >=20 > diff --git a/UefiCpuPkg/Library/MpInitLib/MpLib.h > b/UefiCpuPkg/Library/MpInitLib/MpLib.h > index b1a9d99cb3eb..267aa5201c50 100644 > --- a/UefiCpuPkg/Library/MpInitLib/MpLib.h > +++ b/UefiCpuPkg/Library/MpInitLib/MpLib.h > @@ -293,7 +293,8 @@ struct _CPU_MP_DATA { > UINT64 GhcbBase; > }; >=20 > -#define AP_RESET_STACK_SIZE 64 > +#define AP_SAFE_STACK_SIZE 128 > +#define AP_RESET_STACK_SIZE AP_SAFE_STACK_SIZE >=20 > #pragma pack(1) >=20 > @@ -349,8 +350,11 @@ VOID > IN BOOLEAN MwaitSupport, > IN UINTN ApTargetCState, > IN UINTN PmCodeSegment, > + IN UINTN Pm16CodeSegment, > IN UINTN TopOfApStack, > - IN UINTN NumberToFinish > + IN UINTN NumberToFinish, > + IN UINTN SevEsAPJumpTable, > + IN UINTN WakeupBuffer > ); >=20 > /** > diff --git a/UefiCpuPkg/Library/MpInitLib/DxeMpLib.c > b/UefiCpuPkg/Library/MpInitLib/DxeMpLib.c > index 9115ff9e3e30..7165bcf3124a 100644 > --- a/UefiCpuPkg/Library/MpInitLib/DxeMpLib.c > +++ b/UefiCpuPkg/Library/MpInitLib/DxeMpLib.c > @@ -12,6 +12,7 @@ > #include > #include > #include > +#include > #include > #include >=20 > @@ -85,6 +86,13 @@ GetWakeupBuffer ( > { > EFI_STATUS Status; > EFI_PHYSICAL_ADDRESS StartAddress; > + EFI_MEMORY_TYPE MemoryType; > + > + if (PcdGetBool (PcdSevEsIsEnabled)) { > + MemoryType =3D EfiReservedMemoryType; } else { > + MemoryType =3D EfiBootServicesData; > + } >=20 > // > // Try to allocate buffer below 1M for waking vector. > @@ -97,7 +105,7 @@ GetWakeupBuffer ( > StartAddress =3D 0x88000; > Status =3D gBS->AllocatePages ( > AllocateMaxAddress, > - EfiBootServicesData, > + MemoryType, > EFI_SIZE_TO_PAGES (WakeupBufferSize), > &StartAddress > ); > @@ -159,8 +167,10 @@ GetSevEsAPMemory ( > VOID > ) > { > - EFI_STATUS Status; > - EFI_PHYSICAL_ADDRESS StartAddress; > + EFI_STATUS Status; > + EFI_PHYSICAL_ADDRESS StartAddress; > + MSR_SEV_ES_GHCB_REGISTER Msr; > + GHCB *Ghcb; >=20 > // > // Allocate 1 page for AP jump table page @@ -176,6 +186,16 @@ > GetSevEsAPMemory ( >=20 > DEBUG ((DEBUG_INFO, "Dxe: SevEsAPMemory =3D %lx\n", (UINTN) > StartAddress)); >=20 > + // > + // Save the SevEsAPMemory as the AP jump table. > + // > + Msr.GhcbPhysicalAddress =3D AsmReadMsr64 (MSR_SEV_ES_GHCB); Ghcb > =3D > + Msr.Ghcb; > + > + VmgInit (Ghcb); > + VmgExit (Ghcb, SVM_EXIT_AP_JUMP_TABLE, 0, (UINT64) (UINTN) > + StartAddress); VmgDone (Ghcb); > + > return (UINTN) StartAddress; > } >=20 > @@ -330,17 +350,26 @@ RelocateApLoop ( > BOOLEAN MwaitSupport; > ASM_RELOCATE_AP_LOOP AsmRelocateApLoopFunc; > UINTN ProcessorNumber; > + UINTN StackStart; >=20 > MpInitLibWhoAmI (&ProcessorNumber); > CpuMpData =3D GetCpuMpData (); > MwaitSupport =3D IsMwaitSupport (); > + if (CpuMpData->SevEsIsEnabled) { > + StackStart =3D CpuMpData->SevEsAPResetStackStart; > + } else { > + StackStart =3D mReservedTopOfApStack; } > AsmRelocateApLoopFunc =3D (ASM_RELOCATE_AP_LOOP) (UINTN) > mReservedApLoopFunc; > AsmRelocateApLoopFunc ( > MwaitSupport, > CpuMpData->ApTargetCState, > CpuMpData->PmCodeSegment, > - mReservedTopOfApStack - ProcessorNumber * AP_SAFE_STACK_SIZE, > - (UINTN) &mNumberToFinish > + CpuMpData->Pm16CodeSegment, > + StackStart - ProcessorNumber * AP_SAFE_STACK_SIZE, > + (UINTN) &mNumberToFinish, > + CpuMpData->SevEsAPBuffer, > + CpuMpData->WakeupBuffer > ); > // > // It should never reach here > @@ -374,6 +403,21 @@ MpInitChangeApLoopCallback ( > while (mNumberToFinish > 0) { > CpuPause (); > } > + > + if (CpuMpData->SevEsIsEnabled && (CpuMpData->WakeupBuffer !=3D > (UINTN) -1)) { > + // > + // There are APs present. Re-use reserved memory area below 1MB from > + // WakeupBuffer as the area to be used for transitioning to 16-bit m= ode > + // in support of booting of the AP by an OS. > + // > + CopyMem ( > + (VOID *) CpuMpData->WakeupBuffer, > + (VOID *) (CpuMpData->AddressMap.RendezvousFunnelAddress + > + CpuMpData->AddressMap.SwitchToRealPM16ModeOffset), > + CpuMpData->AddressMap.SwitchToRealPM16ModeSize > + ); > + } > + > DEBUG ((DEBUG_INFO, "%a() done!\n", __FUNCTION__)); } >=20 > diff --git a/UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm > b/UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm > index 6956b408d004..3b8ec477b8b3 100644 > --- a/UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm > +++ b/UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm > @@ -465,6 +465,10 @@ BITS 16 > ; - IP for Real Mode (two bytes) > ; - CS for Real Mode (two bytes) > ; > + ; This label is also used with AsmRelocateApLoop. During MP finaliza= tion, > + ; the code from PM16Mode to SwitchToRealProcEnd is copied to the sta= rt > of > + ; the WakeupBuffer, allowing a parked AP to be booted by an OS. > + ; > PM16Mode: > mov eax, cr0 ; Read CR0 > btr eax, 0 ; Set PE=3D0 > @@ -487,32 +491,95 @@ PM16Mode: > SwitchToRealProcEnd: >=20 > ;-----------------------------------------------------------------------= -------------- > -; AsmRelocateApLoop (MwaitSupport, ApTargetCState, PmCodeSegment, > TopOfApStack, CountTofinish); > +; AsmRelocateApLoop (MwaitSupport, ApTargetCState, PmCodeSegment, > +Pm16CodeSegment, TopOfApStack, CountTofinish, SevEsAPJumpTable, > +WakeupBuffer); > ;-----------------------------------------------------------------------= -------------- > global ASM_PFX(AsmRelocateApLoop) > ASM_PFX(AsmRelocateApLoop): > AsmRelocateApLoopStart: > BITS 64 > + cmp qword [rsp + 56], 0 > + je NoSevEs > + > + ; > + ; Perform some SEV-ES related setup before leaving 64-bit mode > + ; > + push rcx > + push rdx > + > + ; > + ; Get the RDX reset value using CPUID > + ; > + mov rax, 1 > + cpuid > + mov rsi, rax ; Save off the reset value for RDX > + > + ; > + ; Prepare the GHCB for the AP_HLT_LOOP VMGEXIT call > + ; - Must be done while in 64-bit long mode so that writes to > + ; the GHCB memory will be unencrypted. > + ; - No NAE events can be generated once this is set otherwise > + ; the AP_RESET_HOLD SW_EXITCODE will be overwritten. > + ; > + mov rcx, 0xc0010130 > + rdmsr ; Retrieve current GHCB address > + shl rdx, 32 > + or rdx, rax > + > + mov rdi, rdx > + xor rax, rax > + mov rcx, 0x800 > + shr rcx, 3 > + rep stosq ; Clear the GHCB > + > + mov rax, 0x80000004 ; VMGEXIT AP_RESET_HOLD > + mov [rdx + 0x390], rax > + > + pop rdx > + pop rcx > + > +NoSevEs: > cli ; Disable interrupt before switching to= 32-bit mode > - mov rax, [rsp + 40] ; CountTofinish > + mov rax, [rsp + 48] ; CountTofinish > lock dec dword [rax] ; (*CountTofinish)-- > - mov rsp, r9 > - push rcx > - push rdx >=20 > - lea rsi, [PmEntry] ; rsi <- The start address of transitio= n code > + mov rax, [rsp + 56] ; SevEsAPJumpTable > + mov rbx, [rsp + 64] ; WakeupBuffer > + mov rsp, [rsp + 40] ; TopOfApStack > + > + push rax ; Save SevEsAPJumpTable > + push rbx ; Save WakeupBuffer > + push r9 ; Save Pm16CodeSegment > + push rcx ; Save MwaitSupport > + push rdx ; Save ApTargetCState > + > + lea rax, [PmEntry] ; rax <- The start address of transitio= n code >=20 > push r8 > - push rsi > - DB 0x48 > - retf > + push rax > + > + ; > + ; Clear R8 - R15, for reset, before going into 32-bit mode > + ; > + xor r8, r8 > + xor r9, r9 > + xor r10, r10 > + xor r11, r11 > + xor r12, r12 > + xor r13, r13 > + xor r14, r14 > + xor r15, r15 > + > + ; > + ; Far return into 32-bit mode > + ; > +o64 retf > + > BITS 32 > PmEntry: > mov eax, cr0 > btr eax, 31 ; Clear CR0.PG > mov cr0, eax ; Disable paging and caches >=20 > - mov ebx, edx ; Save EntryPoint to rbx, for rdmsr wil= l overwrite rdx > mov ecx, 0xc0000080 > rdmsr > and ah, ~ 1 ; Clear LME > @@ -525,6 +592,8 @@ PmEntry: > add esp, 4 > pop ecx, > add esp, 4 > + > +MwaitCheck: > cmp cl, 1 ; Check mwait-monitor support > jnz HltLoop > mov ebx, edx ; Save C-State to ebx > @@ -538,10 +607,50 @@ MwaitLoop: > shl eax, 4 > mwait > jmp MwaitLoop > + > HltLoop: > + pop edx ; PM16CodeSegment > + add esp, 4 > + pop ebx ; WakeupBuffer > + add esp, 4 > + pop eax ; SevEsAPJumpTable > + add esp, 4 > + cmp eax, 0 ; Check for SEV-ES > + je DoHlt > + > + cli > + ; > + ; SEV-ES is enabled, use VMGEXIT (GHCB information already > + ; set by caller) > + ; > +BITS 64 > + rep vmmcall > +BITS 32 > + > + ; > + ; Back from VMGEXIT AP_HLT_LOOP > + ; Push the FLAGS/CS/IP values to use > + ; > + push word 0x0002 ; EFLAGS > + xor ecx, ecx > + mov cx, [eax + 2] ; CS > + push cx > + mov cx, [eax] ; IP > + push cx > + push word 0x0000 ; For alignment, will be discarded > + > + push edx > + push ebx > + > + mov edx, esi ; Restore RDX reset value > + > + retf > + > +DoHlt: > cli > hlt > - jmp HltLoop > + jmp DoHlt > + > BITS 64 > AsmRelocateApLoopEnd: >=20 > -- > 2.27.0