From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (NAM10-BN7-obe.outbound.protection.outlook.com [40.107.92.82]) by mx.groups.io with SMTP id smtpd.web11.13365.1576254955816149973 for ; Fri, 13 Dec 2019 08:35:56 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@amdcloud.onmicrosoft.com header.s=selector2-amdcloud-onmicrosoft-com header.b=2g9XG1yl; spf=none, err=SPF record not found (domain: amd.com, ip: 40.107.92.82, mailfrom: thomas.lendacky@amd.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=JHqh3KMf7KVqpM23xvUEyTb47u33Hw0yRHEGXEqxiLKSqfZ8tKQpGKa+opFWrnq5BsvyhNyLKnBvZ2EqRai3eghAue56Cs2Ehzqd8H3cSbZB823vAsQqLkYjASC1wXeNE+neyQOmIczoaTK3zIXJcjr5+jxMACJMu8LqOIYmx3KPb1Oxf4g+5uJXyMPPcDu5u9SPsV5Tyz+o2PBQxl1ZJba+96GZn4qayWemz289xZMPRwFKzTaIjdkZSW96eBRbIdo1Xl3K+jduMygbTaPJym9w8GZs7GGs93uqs71zlZwzRdsCFLMXNLPsgYJx71RXtUXyFGDXS6WtUyv68S1KvQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=1M8wWbyy4Qhv3xvclAO0VTip8PK0vCtSltRlnpYvUx0=; b=LtZYOaCHlrD9zXy0I0aIC0rW/T+x74gpQdH5uNQAyyY43BflERSBhNdVl4UtGVs8jRkKzpqxRRqV1kdTaAx4kTHv9yh1zrKTvEuYvv5o2seBsTm1I1mb0OCLE9Kf701vuzMyiW6hqz4JCJj1EtdQBfnAQAY6XRTdU6o42MJKJOZBuJUFFWYzsqGr7dyJaCid4Do+6H31DZdQTkEPNLpcMXSi1qJVJoIJM8TiU3up4cUraRCPKQaDcJYokRm6krEs/WlGHVe0x3w/tA65z2i/a6awPTuo0gKGljmmyqdAMvHL5qu4pWhZKDUYpQ0mgtYrh3t3uiGBJRo9uUOvZFp0hQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amdcloud.onmicrosoft.com; s=selector2-amdcloud-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=1M8wWbyy4Qhv3xvclAO0VTip8PK0vCtSltRlnpYvUx0=; b=2g9XG1yloGvaKTayMCMwNdZIpsBNFjkk201OR3pwp0YqNoGbMBNb3wvS0L5phnVQvAstN9MsTg1YdLGJukNfWPR3ya/BqPlhwr4FhCmuh2oh6mFgSc/2Ptn/g6kb8NHpWYTVyAFYj+D4JajOBIb2/QCDYkfb6mwpyOlvFuVYldA= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Thomas.Lendacky@amd.com; Received: from BN8PR12MB3154.namprd12.prod.outlook.com (20.179.67.74) by BN8PR12MB2930.namprd12.prod.outlook.com (20.179.66.77) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2538.15; Fri, 13 Dec 2019 16:35:53 +0000 Received: from BN8PR12MB3154.namprd12.prod.outlook.com ([fe80::49ec:cc61:db3b:f884]) by BN8PR12MB3154.namprd12.prod.outlook.com ([fe80::49ec:cc61:db3b:f884%4]) with mapi id 15.20.2538.016; Fri, 13 Dec 2019 16:35:53 +0000 Subject: Re: [RFC PATCH v2 42/44] UefiCpuPkg/MpInitLib: Prepare SEV-ES guest APs for OS use To: "Ni, Ray" , "devel@edk2.groups.io" Cc: "Justen, Jordan L" , Laszlo Ersek , Ard Biesheuvel , "Kinney, Michael D" , "Gao, Liming" , "Dong, Eric" , "Singh, Brijesh" References: <00047308110ff3380000f6eb140e815c01499e3a.1568922729.git.thomas.lendacky@amd.com> <734D49CCEBEEF84792F5B80ED585239D5C399C13@SHSMSX104.ccr.corp.intel.com> From: "Lendacky, Thomas" Message-ID: Date: Fri, 13 Dec 2019 10:35:50 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.2.2 In-Reply-To: <734D49CCEBEEF84792F5B80ED585239D5C399C13@SHSMSX104.ccr.corp.intel.com> X-ClientProxiedBy: DM5PR04CA0052.namprd04.prod.outlook.com (2603:10b6:3:ef::14) To BN8PR12MB3154.namprd12.prod.outlook.com (2603:10b6:408:6d::10) Return-Path: thomas.lendacky@amd.com MIME-Version: 1.0 X-Originating-IP: [165.204.77.1] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: aa1d9afd-d7a1-465c-2128-08d77fea84f8 X-MS-TrafficTypeDiagnostic: BN8PR12MB2930:|BN8PR12MB2930: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7691; X-Forefront-PRVS: 0250B840C1 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(6029001)(4636009)(39860400002)(396003)(136003)(366004)(376002)(346002)(199004)(189003)(13464003)(52116002)(36756003)(186003)(6486002)(66556008)(31686004)(2906002)(4326008)(8936002)(81156014)(26005)(81166006)(30864003)(6506007)(53546011)(86362001)(66476007)(966005)(8676002)(66946007)(478600001)(110136005)(6512007)(5660300002)(2616005)(19627235002)(45080400002)(54906003)(316002)(31696002);DIR:OUT;SFP:1101;SCL:1;SRVR:BN8PR12MB2930;H:BN8PR12MB3154.namprd12.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; Received-SPF: None (protection.outlook.com: amd.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: wRivVxcOeMk0M2gmPpF/XXJO7oYX/4ULXEkFGR3dLTk3bTYfWu5pX129s/+b57mDmq9J4V7fcYCGEayl2uwUIIOcmaq5U0JEY+mmVu9b+nvfbxn3ljmpJUHgL59Db8Q6V8W/QDk1TOkKPnOUyIdsX8dKMyXPp6i+gMNFkApAmWxLrS0edn+l1WAFEFJKgqW8uUzUXS7wZFNamVvaAfU9MR+x9WXVhjV4N9dE5dsH1iQ//WMYuTd9iYP9lIDDcuoN1QVLAl8BUF9hpiS1611KzsrCCctUcLzqZ1wkQOm0Yshy3fzsc4dxbpB+9+uLhLOzGfLdvEvHv1S/mSPZUs15ksQ0k3n2/+04uw0c40/NV8vgZgYMovi0MGWormemBGvM6ytewMDWYI9V92SjCQCirTIEtBVgk2uri4EmvcYOuRfmYIMs0+gbuZwKoSHi/0/+8jZA1Kf9mGBqJttXT2j/11Dw3TPJ0bC+NC2m29hJXNN4aiNVy837HduoJIznNv60Zc5k/6FVoStbRm6plFl4NvMO3A27z5XEu9s3PY2GB8M= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: aa1d9afd-d7a1-465c-2128-08d77fea84f8 X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Dec 2019 16:35:52.8371 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: iPkwvy8GEpviSAsQyW3il46+sipTakj5PIPVFvMmWI3LN3IuFlzdGR1iFbrPkTsl/HFiA01mPwTaoPCpNvQ58Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN8PR12MB2930 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit On 12/12/19 2:24 AM, Ni, Ray wrote: > Tom, > why this cannot be done as today's code in ExitBS callback? > Because it should be done after interrupt is disabled? The problem comes from the calls during and after the ExitBS callbacks are performed. Part of the finalization support is to set the AP jump table in the GHCB MSR and to copy/move AP reset code for use when the OS starts the APs. The GHCB MSR will now not point to an actual GHCB, so we can't take any more #VCs at that point. When I look at the #VC count when running through CoreExitBootServices() in MdeModulePkg/Core/Dxe/DxeMain/DxeMain.c (with debug statements enabled), I see the following #VC counts: CoreNotifySignalList() = 12 MemoryProtectionExitBootServicesCallback() = 1908 If I disable serial debug (don't build with -DDEBUG_ON_SERIAL_PORT=TRUE), I see the following #VC counts: CoreNotifySignalList() = 2 MemoryProtectionExitBootServicesCallback() = 234 So even if I could ensure that an SEV-ES callback was the last callback in CoreNotifySignalList(), MemoryProtectionExitBootServicesCallback() will generate #VCs so we need to perform the finalization (or at least the GHCB MSR update) after that. I'm looking at whether I can eliminate the need for the finalization support. I might be able to perform the copy/move of the AP reset code in MpInitChangeApLoopCallback() - I just need to be careful that all APs are out of the area of code that will be updated. And if I implement a new GHCB protocol for registering the jump page, I can leave the GHCB MSR in tact. Just a concept at the moment, but it may be possible. Thanks, Tom > >> -----Original Message----- >> From: Lendacky, Thomas >> Sent: Friday, September 20, 2019 3:53 AM >> To: devel@edk2.groups.io >> Cc: Justen, Jordan L ; Laszlo Ersek ; Ard Biesheuvel >> ; Kinney, Michael D ; Gao, Liming ; Dong, >> Eric ; Ni, Ray ; Singh, Brijesh >> Subject: [RFC PATCH v2 42/44] UefiCpuPkg/MpInitLib: Prepare SEV-ES guest APs for OS use >> >> From: Tom Lendacky >> >> BZ: https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.tianocore.org%2Fshow_bug.cgi%3Fid%3D2198&data=02%7C01%7CThomas.Lendacky%40amd.com%7Ce9dfe136268e4b3b8bd308d77edca950%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637117358507708564&sdata=o7EsL8JXEpjg8sixUFshm3yIoCHi8T7dyCLEPNT1bqQ%3D&reserved=0 >> >> Before UEFI transfers control to the OS, it must park the AP. This is >> done using the AsmRelocateApLoop function to transition into 32-bit >> non-paging mode. For an SEV-ES guest, a few additional things must be >> done: >> - AsmRelocateApLoop must be updated to support SEV-ES. This means >> performing a VMGEXIT AP Reset Hold instead of an MWAIT or HLT loop. >> - Since the AP must transition to real mode, a small routine is copied >> to the WakeupBuffer area. Since the WakeupBuffer will be used by >> the AP during OS booting, it must be placed in reserved memory. >> Additionally, the AP stack must be located where it can be accessed >> in real mode. >> - Once the AP is in real mode it will transfer control to the >> destination specified by the OS in the SEV-ES AP Jump Table. The >> SEV-ES AP Jump Table address is communicated to the OS using the BSP >> GHCB MSR. >> >> Cc: Eric Dong >> Cc: Ray Ni >> Cc: Laszlo Ersek >> Signed-off-by: Tom Lendacky >> --- >> UefiCpuPkg/Library/MpInitLib/MpLib.h | 8 +- >> UefiCpuPkg/Library/MpInitLib/DxeMpLib.c | 54 ++++++- >> UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm | 134 ++++++++++++++++-- >> 3 files changed, 176 insertions(+), 20 deletions(-) >> >> diff --git a/UefiCpuPkg/Library/MpInitLib/MpLib.h b/UefiCpuPkg/Library/MpInitLib/MpLib.h >> index 5966510d4a1b..2d38a0e85a40 100644 >> --- a/UefiCpuPkg/Library/MpInitLib/MpLib.h >> +++ b/UefiCpuPkg/Library/MpInitLib/MpLib.h >> @@ -277,7 +277,8 @@ struct _CPU_MP_DATA { >> UINT64 GhcbBase; >> }; >> >> -#define AP_RESET_STACK_SIZE 64 >> +#define AP_SAFE_STACK_SIZE 128 >> +#define AP_RESET_STACK_SIZE AP_SAFE_STACK_SIZE >> >> typedef union { >> struct { >> @@ -331,8 +332,11 @@ VOID >> IN BOOLEAN MwaitSupport, >> IN UINTN ApTargetCState, >> IN UINTN PmCodeSegment, >> + IN UINTN Pm16CodeSegment, >> IN UINTN TopOfApStack, >> - IN UINTN NumberToFinish >> + IN UINTN NumberToFinish, >> + IN UINTN SevEsAPJumpTable, >> + IN UINTN WakeupBuffer >> ); >> >> /** >> diff --git a/UefiCpuPkg/Library/MpInitLib/DxeMpLib.c b/UefiCpuPkg/Library/MpInitLib/DxeMpLib.c >> index 16603ef3f20e..cf53b5026aa4 100644 >> --- a/UefiCpuPkg/Library/MpInitLib/DxeMpLib.c >> +++ b/UefiCpuPkg/Library/MpInitLib/DxeMpLib.c >> @@ -18,7 +18,6 @@ >> #include >> >> #define AP_CHECK_INTERVAL (EFI_TIMER_PERIOD_MILLISECONDS (100)) >> -#define AP_SAFE_STACK_SIZE 128 >> >> CPU_MP_DATA *mCpuMpData = NULL; >> EFI_EVENT mCheckAllApsEvent = NULL; >> @@ -86,6 +85,13 @@ GetWakeupBuffer ( >> { >> EFI_STATUS Status; >> EFI_PHYSICAL_ADDRESS StartAddress; >> + EFI_MEMORY_TYPE MemoryType; >> + >> + if (PcdGetBool (PcdSevEsActive)) { >> + MemoryType = EfiReservedMemoryType; >> + } else { >> + MemoryType = EfiBootServicesData; >> + } >> >> // >> // Try to allocate buffer below 1M for waking vector. >> @@ -98,7 +104,7 @@ GetWakeupBuffer ( >> StartAddress = 0x88000; >> Status = gBS->AllocatePages ( >> AllocateMaxAddress, >> - EfiBootServicesData, >> + MemoryType, >> EFI_SIZE_TO_PAGES (WakeupBufferSize), >> &StartAddress >> ); >> @@ -331,17 +337,26 @@ RelocateApLoop ( >> BOOLEAN MwaitSupport; >> ASM_RELOCATE_AP_LOOP AsmRelocateApLoopFunc; >> UINTN ProcessorNumber; >> + UINTN StackStart; >> >> MpInitLibWhoAmI (&ProcessorNumber); >> CpuMpData = GetCpuMpData (); >> MwaitSupport = IsMwaitSupport (); >> + if (CpuMpData->SevEsActive) { >> + StackStart = CpuMpData->SevEsAPResetStackStart; >> + } else { >> + StackStart = mReservedTopOfApStack; >> + } >> AsmRelocateApLoopFunc = (ASM_RELOCATE_AP_LOOP) (UINTN) mReservedApLoopFunc; >> AsmRelocateApLoopFunc ( >> MwaitSupport, >> CpuMpData->ApTargetCState, >> CpuMpData->PmCodeSegment, >> - mReservedTopOfApStack - ProcessorNumber * AP_SAFE_STACK_SIZE, >> - (UINTN) &mNumberToFinish >> + CpuMpData->Pm16CodeSegment, >> + StackStart - ProcessorNumber * AP_SAFE_STACK_SIZE, >> + (UINTN) &mNumberToFinish, >> + CpuMpData->SevEsAPBuffer, >> + CpuMpData->WakeupBuffer >> ); >> // >> // It should never reach here >> @@ -895,9 +910,34 @@ MpFinalize ( >> IN CPU_MP_DATA *CpuMpData >> ) >> { >> - // >> - // DXE phase will do this transition, but just return EFI_SUCCESS for now. >> - // >> + if (CpuMpData->SevEsActive) { >> + // >> + // Perform SEV-ES specific finalization >> + // >> + if (CpuMpData->WakeupBuffer == (UINTN) -1) { >> + // >> + // No APs parked in UEFI, clear the GHCB >> + // >> + AsmWriteMsr64 (MSR_SEV_ES_GHCB, 0); >> + } else { >> + // >> + // Re-use reserved memory area below 1MB from WakeupBuffer >> + // >> + CopyMem ( >> + (VOID *) CpuMpData->WakeupBuffer, >> + (VOID *) CpuMpData->AddressMap.RendezvousFunnelAddress + >> + CpuMpData->AddressMap.SwitchToRealPM16ModeOffset, >> + CpuMpData->AddressMap.SwitchToRealPM16ModeSize >> + ); >> + >> + // >> + // Point the GHCB at the AP jump table to communicate the address to >> + // the booting system. >> + // >> + AsmWriteMsr64 (MSR_SEV_ES_GHCB, (CpuMpData->SevEsAPBuffer) | 0x03); >> + } >> + } >> + >> return EFI_SUCCESS; >> } >> >> diff --git a/UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm b/UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm >> index bbc7432740ff..3cb0cd5bb306 100644 >> --- a/UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm >> +++ b/UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm >> @@ -465,6 +465,10 @@ BITS 16 >> ; - IP for Real Mode (two bytes) >> ; - CS for Real Mode (two bytes) >> ; >> + ; This label is also used with AsmRelocateApLoop. During MP finalization, >> + ; the code from PM16Mode to SwitchToRealProcEnd is copied to the start of >> + ; the WakeupBuffer, allowing a parked AP to be booted by an OS. >> + ; >> PM16Mode: >> mov eax, cr0 ; Read CR0 >> btr eax, 0 ; Set PE=0 >> @@ -487,32 +491,95 @@ PM16Mode: >> SwitchToRealProcEnd: >> >> ;------------------------------------------------------------------------------------- >> -; AsmRelocateApLoop (MwaitSupport, ApTargetCState, PmCodeSegment, TopOfApStack, CountTofinish); >> +; AsmRelocateApLoop (MwaitSupport, ApTargetCState, PmCodeSegment, Pm16CodeSegment, TopOfApStack, >> CountTofinish, SevEsAPJumpTable, WakeupBuffer); >> ;------------------------------------------------------------------------------------- >> global ASM_PFX(AsmRelocateApLoop) >> ASM_PFX(AsmRelocateApLoop): >> AsmRelocateApLoopStart: >> BITS 64 >> + cmp qword [rsp + 56], 0 >> + je NoSevEs >> + >> + ; >> + ; Perform some SEV-ES related setup before leaving 64-bit mode >> + ; >> + push rcx >> + push rdx >> + >> + ; >> + ; Get the RDX reset value using CPUID >> + ; >> + mov rax, 1 >> + cpuid >> + mov rsi, rax ; Save off the reset value for RDX >> + >> + ; >> + ; Prepare the GHCB for the AP_HLT_LOOP VMGEXIT call >> + ; - Must be done while in 64-bit long mode so that writes to >> + ; the GHCB memory will be unencrypted. >> + ; - No NAE events can be generated once this is set otherwise >> + ; the AP_RESET_HOLD SW_EXITCODE will be overwritten. >> + ; >> + mov rcx, 0xc0010130 >> + rdmsr ; Retrieve current GHCB address >> + shl rdx, 32 >> + or rdx, rax >> + >> + mov rdi, rdx >> + xor rax, rax >> + mov rcx, 0x800 >> + shr rcx, 3 >> + rep stosq ; Clear the GHCB >> + >> + mov rax, 0x80000004 ; VMGEXIT AP_RESET_HOLD >> + mov [rdx + 0x390], rax >> + >> + pop rdx >> + pop rcx >> + >> +NoSevEs: >> cli ; Disable interrupt before switching to 32-bit mode >> - mov rax, [rsp + 40] ; CountTofinish >> + mov rax, [rsp + 48] ; CountTofinish >> lock dec dword [rax] ; (*CountTofinish)-- >> - mov rsp, r9 >> - push rcx >> - push rdx >> >> - lea rsi, [PmEntry] ; rsi <- The start address of transition code >> + mov rax, [rsp + 56] ; SevEsAPJumpTable >> + mov rbx, [rsp + 64] ; WakeupBuffer >> + mov rsp, [rsp + 40] ; TopOfApStack >> + >> + push rax ; Save SevEsAPJumpTable >> + push rbx ; Save WakeupBuffer >> + push r9 ; Save Pm16CodeSegment >> + push rcx ; Save MwaitSupport >> + push rdx ; Save ApTargetCState >> + >> + lea rax, [PmEntry] ; rax <- The start address of transition code >> >> push r8 >> - push rsi >> - DB 0x48 >> - retf >> + push rax >> + >> + ; >> + ; Clear R8 - R15, for reset, before going into 32-bit mode >> + ; >> + xor r8, r8 >> + xor r9, r9 >> + xor r10, r10 >> + xor r11, r11 >> + xor r12, r12 >> + xor r13, r13 >> + xor r14, r14 >> + xor r15, r15 >> + >> + ; >> + ; Far return into 32-bit mode >> + ; >> +o64 retf >> + >> BITS 32 >> PmEntry: >> mov eax, cr0 >> btr eax, 31 ; Clear CR0.PG >> mov cr0, eax ; Disable paging and caches >> >> - mov ebx, edx ; Save EntryPoint to rbx, for rdmsr will overwrite rdx >> mov ecx, 0xc0000080 >> rdmsr >> and ah, ~ 1 ; Clear LME >> @@ -525,6 +592,8 @@ PmEntry: >> add esp, 4 >> pop ecx, >> add esp, 4 >> + >> +MwaitCheck: >> cmp cl, 1 ; Check mwait-monitor support >> jnz HltLoop >> mov ebx, edx ; Save C-State to ebx >> @@ -538,10 +607,53 @@ MwaitLoop: >> shl eax, 4 >> mwait >> jmp MwaitLoop >> + >> HltLoop: >> + pop edx ; PM16CodeSegment >> + add esp, 4 >> + pop ebx ; WakeupBuffer >> + add esp, 4 >> + pop eax ; SevEsAPJumpTable >> + add esp, 4 >> + cmp eax, 0 ; Check for SEV-ES >> + je DoHlt >> + >> + cli >> + ; >> + ; SEV-ES is active, use VMGEXIT (GHCB information already >> + ; set by caller) >> + ; >> + ; VMGEXIT is rep vmmcall >> + ; >> + db 0xf3 >> + db 0x0f >> + db 0x01 >> + db 0xd9 >> + >> + ; >> + ; Back from VMGEXIT AP_HLT_LOOP >> + ; Push the FLAGS/CS/IP values to use >> + ; >> + push word 0x0002 ; EFLAGS >> + xor ecx, ecx >> + mov cx, [eax + 2] ; CS >> + push cx >> + mov cx, [eax] ; IP >> + push cx >> + push word 0x0000 ; For alignment, will be discarded >> + >> + push edx >> + push ebx >> + >> + mov edx, esi ; Restore RDX reset value >> + >> + retf >> + >> +DoHlt: >> cli >> hlt >> - jmp HltLoop >> + jmp DoHlt >> + >> BITS 64 >> AsmRelocateApLoopEnd: >> >> -- >> 2.17.1 >