From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (NAM12-BN8-obe.outbound.protection.outlook.com [40.107.237.51]) by mx.groups.io with SMTP id smtpd.web09.9429.1603981180095068951 for ; Thu, 29 Oct 2020 07:19:41 -0700 Authentication-Results: mx.groups.io; dkim=fail reason="body hash did not verify" header.i=@amdcloud.onmicrosoft.com header.s=selector2-amdcloud-onmicrosoft-com header.b=uMELA6jr; spf=none, err=SPF record not found (domain: amd.com, ip: 40.107.237.51, mailfrom: thomas.lendacky@amd.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=MWrmJlIsy8H/Z4yLnFQ80cxdiqaZr+PvScG0qIEWLJuGgQ55N9bMMK1Yjc4vIz8K3NdtWXEL1C462DJD1UgRahdsdb+sjdslia/4u+NMpVL9MCftEaT5xcKbn3tWCX19E3346dN62TiDyD6ACNYLcaF6CD2pTcODcITzK7r5wRfLLHWb2QnMe9SyTABPebrthLIR3J5Du58czEyAlU0rM2pfYwny9qlRFtc+EwbH0PUTX/tkpQau1RZzOA97rLroBz+r6Eblob+US107EaRA/N23FCXkDw7AWrh9DrYVvIJaJphyxRak7AY9/eqjo+yYd0VGsfTCKkmjnPp89Vv6dQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=jQ7SI0mvfufgAjIlIXnOXi+VpDt9/K8/vYZNLZ1SsLg=; b=LrdCwt+WCFOlzRjXfEddhshADzgKOcIzBI24sf4uvDxhJbVt2RTKm21EK8nhX+ljB+lExYarvAA2S4YnqXqpLDOrVu0WF8VFwlJUYQGc3SGwQyL2SbujLTQszza9AQzxdfHIdiNmgrGa6VR/J+8DEBNDMtUFLrzVjzsf+9Xgi+uZoNnYzEU2WIb38a98DSMF7GnTjqZAJdtQ42XNCAv8U3RozFBV/W1mcCwsZeMpraapkU38BwHqZIdcHXNxrbfp2Golp93R4tTNTMSdvoIOVRp1zkTydpLyXRU+h7lTx63sNcPvVnQ0clkNsr7i/W4t6DKU6VeFdCPcKyIi/ItLow== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amdcloud.onmicrosoft.com; s=selector2-amdcloud-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=jQ7SI0mvfufgAjIlIXnOXi+VpDt9/K8/vYZNLZ1SsLg=; b=uMELA6jrI+puT1b5bjlxjy0ilmgPWhOiLTmfEoyJ506E0q57EfdbPgxKucMQKN/eowj0lQ/yabzAag+oStgUbTRSx0m91NxGbVfgwbfvaUhuXSVdcZlNSKI4LRvjT80W2WEOD1QdiD/yeUo50SV94enXXP7ad7X5PUE7PDiaC7I= Authentication-Results: edk2.groups.io; dkim=none (message not signed) header.d=none;edk2.groups.io; dmarc=none action=none header.from=amd.com; Received: from DM5PR12MB1355.namprd12.prod.outlook.com (2603:10b6:3:6e::7) by DM6PR12MB3276.namprd12.prod.outlook.com (2603:10b6:5:15e::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3477.24; Thu, 29 Oct 2020 14:19:35 +0000 Received: from DM5PR12MB1355.namprd12.prod.outlook.com ([fe80::e442:c052:8a2c:5fba]) by DM5PR12MB1355.namprd12.prod.outlook.com ([fe80::e442:c052:8a2c:5fba%6]) with mapi id 15.20.3499.027; Thu, 29 Oct 2020 14:19:35 +0000 From: "Lendacky, Thomas" To: devel@edk2.groups.io CC: Brijesh Singh , Eric Dong , Ray Ni , Laszlo Ersek , Rahul Kumar , Jordan Justen , Ard Biesheuvel Subject: [PATCH v3 10/11] UefiCpuPkg, OvmfPkg: Disable interrupts when using the GHCB Date: Thu, 29 Oct 2020 09:18:01 -0500 Message-ID: <0f3f11bec870bc4f10843beb53c6a919ac549ef4.1603981082.git.thomas.lendacky@amd.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: References: X-Originating-IP: [165.204.77.1] X-ClientProxiedBy: DM6PR18CA0012.namprd18.prod.outlook.com (2603:10b6:5:15b::25) To DM5PR12MB1355.namprd12.prod.outlook.com (2603:10b6:3:6e::7) Return-Path: thomas.lendacky@amd.com MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from tlendack-t1.amd.com (165.204.77.1) by DM6PR18CA0012.namprd18.prod.outlook.com (2603:10b6:5:15b::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3499.18 via Frontend Transport; Thu, 29 Oct 2020 14:19:35 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: ae0967f1-c50f-4a5f-e49e-08d87c15a9b5 X-MS-TrafficTypeDiagnostic: DM6PR12MB3276: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:9508; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: fStmiBoLTAmXNLf5FiFts+fV8hGM8xoqQSvAiJPJbiSxnULcoMQwPWN4OWoMCeg02ESie9J+LHlY8Jzbg1BgfamlfEyUC26MjAsHQg7klDrk6Ln+v+MxNXd3bvDGs6r2WJws0ij3VhZl5hB+hUVJMhyLvEpavTx+ahbHnQpmBgoirzQjiXQulXwBH4T4YEg3r7z+rwttunf1ACI87nsTlzyWdeZtGuAos2EJz89CDxqLLueaQzBkAQvmFveSLvzii+lprNLvUvSFi9x4mGisH0mjX2rFEXJcuNDbVoK3SKzUDWFoJxGsNwJawyc9XVQM5xxzd0oXk4fdbbh2Xd1Qucg8n5l5utsyViY+CahMwp0A2c5XXbsGZQFCt5Q1Q+uLTht49DT26WiEqwsjUbc//w== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM5PR12MB1355.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(39860400002)(396003)(376002)(136003)(346002)(366004)(26005)(8676002)(186003)(16526019)(4326008)(86362001)(6486002)(83380400001)(7696005)(52116002)(54906003)(66556008)(5660300002)(2906002)(6666004)(966005)(36756003)(2616005)(478600001)(316002)(6916009)(956004)(8936002)(66476007)(19627235002)(66946007);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: 1P4S1IB4JZF8cFGMLLnnQrnTETOzm4OwRIen9RJn1sPE55auG/ptd5g2yC+cwAkV4LSpNGrZ1ZCV8zDUX5zkEu6UXUoxkQOtUIrrfod/m8qHLzhybSb/0V10+gtC2J/28qS9w6eRYYu80PRZ+CQZTJSGbQCIFoU4H4/3GtESzud9LEg/+ipLcXNBAmFnmKYbhHZcpAHbxf1WV9CH1QABJRas/1iOmpQiOndhRFSFAjH6xFt49RaOqkER3NZP3Ty2e9GhvpOpb3ZqM2acYjPXZNC/Li7drghkY2GKHWx030Gz0Tik/lxNNa9fr6GCE723BQW7Woh2fzxIwAZwL0LPqly7Wdo1yEI3wkvrVCm/RkyAaXR2FtR3GRO5L5AM7W6rzWtCnohD3CSTOCHFGI+qjDoscaVHLPQgGVjn9hC1l21B7pVBoHFF/CIxCTJQp4yxDSfwVHZzAZV3KbQXw0QAmenxlnC4+CInaIqWOAIr3gTS3CpiuCibeTpCL1vlgpB4GSC148Vi9qrwUg6kx4dL2vxueNvR6I/HP5YRLPEMko3cqIuklJYgMgZvAge4sbrG6CqnmHX0PuRhRSziHdwkqG+IjSVkkUChu4P7SUJu8GO4BfuaQj22WD/KSrUg8qAKStf44+WiNrJShHc+IZx2Yg== X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: ae0967f1-c50f-4a5f-e49e-08d87c15a9b5 X-MS-Exchange-CrossTenant-AuthSource: DM5PR12MB1355.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Oct 2020 14:19:35.7971 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Njb+VkVQZrY8ZlsYqlmx01hOnyutjEEAMXtXv6EUtg4P9dlPC0DsPRVPH9P4AERj4gsxZSMLSkRwEY+b2CB/tQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB3276 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable From: Tom Lendacky BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3D3008 The QemuFlashPtrWrite() flash services runtime uses the GHCB and VmgExit() directly to perform the flash write when running as an SEV-ES guest. If an interrupt arrives between VmgInit() and VmgExit(), the Dr7 read in the interrupt handler will generate a #VC, which can overwrite information in the GHCB that QemuFlashPtrWrite() has set. This has been seen with the timer interrupt firing and the CpuExceptionHandlerLib library code, UefiCpuPkg/Library/CpuExceptionHandlerLib/X64/ Xcode5ExceptionHandlerAsm.nasm and ExceptionHandlerAsm.nasm reading the Dr7 register while QemuFlashPtrWrite() is using the GHCB. In general, it is necessary to protect the GHCB whenever it is used, not just in QemuFlashPtrWrite(). Disable interrupts around the usage of the GHCB by modifying the VmgInit() and VmgDone() interfaces: - VmgInit() will take an extra parameter that is a pointer to a BOOLEAN that will hold the interrupt state at the time of invocation. VmgInit() will get and save this interrupt state before updating the GHCB. - VmgDone() will take an extra parameter that is used to indicate whether interrupts are to be (re)enabled. Before exiting, VmgDone() will enable interrupts if that is requested. Fixes: 437eb3f7a8db7681afe0e6064d3a8edb12abb766 Cc: Eric Dong Cc: Ray Ni Cc: Laszlo Ersek Cc: Rahul Kumar Cc: Jordan Justen Cc: Ard Biesheuvel Cc: Tom Lendacky Cc: Brijesh Singh Reviewed-by: Laszlo Ersek Signed-off-by: Tom Lendacky --- UefiCpuPkg/Include/Library/VmgExitLib.h | 14 ++++++++--- OvmfPkg/Library/VmgExitLib/VmgExitLib.c | 26 +++++++++++++++= ++--- OvmfPkg/Library/VmgExitLib/VmgExitVcHandler.c | 5 ++-- OvmfPkg/QemuFlashFvbServicesRuntimeDxe/QemuFlashDxe.c | 5 ++-- UefiCpuPkg/Library/MpInitLib/DxeMpLib.c | 5 ++-- UefiCpuPkg/Library/MpInitLib/MpLib.c | 7 +++--- UefiCpuPkg/Library/VmgExitLibNull/VmgExitLibNull.c | 18 ++++++++------ 7 files changed, 55 insertions(+), 25 deletions(-) diff --git a/UefiCpuPkg/Include/Library/VmgExitLib.h b/UefiCpuPkg/Include/L= ibrary/VmgExitLib.h index 07e8af6450b9..061948cf840d 100644 --- a/UefiCpuPkg/Include/Library/VmgExitLib.h +++ b/UefiCpuPkg/Include/Library/VmgExitLib.h @@ -50,13 +50,16 @@ VmgExit ( Performs the necessary steps in preparation for invoking VMGEXIT. Must b= e called before setting any fields within the GHCB. =20 - @param[in, out] Ghcb A pointer to the GHCB + @param[in, out] Ghcb A pointer to the GHCB + @param[in, out] InterruptState A pointer to hold the current interrupt + state, used for restoring in VmgDone () =20 **/ VOID EFIAPI VmgInit ( - IN OUT GHCB *Ghcb + IN OUT GHCB *Ghcb, + IN OUT BOOLEAN *InterruptState ); =20 /** @@ -65,13 +68,16 @@ VmgInit ( Performs the necessary steps to cleanup after invoking VMGEXIT. Must be called after obtaining needed fields within the GHCB. =20 - @param[in, out] Ghcb A pointer to the GHCB + @param[in, out] Ghcb A pointer to the GHCB + @param[in] InterruptState An indicator to conditionally (re)enabl= e + interrupts =20 **/ VOID EFIAPI VmgDone ( - IN OUT GHCB *Ghcb + IN OUT GHCB *Ghcb, + IN BOOLEAN InterruptState ); =20 /** diff --git a/OvmfPkg/Library/VmgExitLib/VmgExitLib.c b/OvmfPkg/Library/VmgE= xitLib/VmgExitLib.c index 0540df8a04d4..bc5cd61d751f 100644 --- a/OvmfPkg/Library/VmgExitLib/VmgExitLib.c +++ b/OvmfPkg/Library/VmgExitLib/VmgExitLib.c @@ -132,15 +132,27 @@ VmgExit ( Performs the necessary steps in preparation for invoking VMGEXIT. Must b= e called before setting any fields within the GHCB. =20 - @param[in, out] Ghcb A pointer to the GHCB + @param[in, out] Ghcb A pointer to the GHCB + @param[in, out] InterruptState A pointer to hold the current interrupt + state, used for restoring in VmgDone () =20 **/ VOID EFIAPI VmgInit ( - IN OUT GHCB *Ghcb + IN OUT GHCB *Ghcb, + IN OUT BOOLEAN *InterruptState ) { + // + // Be sure that an interrupt can't cause a #VC while the GHCB is + // being used. + // + *InterruptState =3D GetInterruptState (); + if (*InterruptState) { + DisableInterrupts (); + } + SetMem (&Ghcb->SaveArea, sizeof (Ghcb->SaveArea), 0); } =20 @@ -150,15 +162,21 @@ VmgInit ( Performs the necessary steps to cleanup after invoking VMGEXIT. Must be called after obtaining needed fields within the GHCB. =20 - @param[in, out] Ghcb A pointer to the GHCB + @param[in, out] Ghcb A pointer to the GHCB + @param[in] InterruptState An indicator to conditionally (re)enabl= e + interrupts =20 **/ VOID EFIAPI VmgDone ( - IN OUT GHCB *Ghcb + IN OUT GHCB *Ghcb, + IN BOOLEAN InterruptState ) { + if (InterruptState) { + EnableInterrupts (); + } } =20 /** diff --git a/OvmfPkg/Library/VmgExitLib/VmgExitVcHandler.c b/OvmfPkg/Librar= y/VmgExitLib/VmgExitVcHandler.c index 9bf9d160179c..1671db3a01b1 100644 --- a/OvmfPkg/Library/VmgExitLib/VmgExitVcHandler.c +++ b/OvmfPkg/Library/VmgExitLib/VmgExitVcHandler.c @@ -1568,6 +1568,7 @@ VmgExitHandleVc ( SEV_ES_INSTRUCTION_DATA InstructionData; UINT64 ExitCode, Status; EFI_STATUS VcRet; + BOOLEAN InterruptState; =20 VcRet =3D EFI_SUCCESS; =20 @@ -1578,7 +1579,7 @@ VmgExitHandleVc ( Regs =3D SystemContext.SystemContextX64; Ghcb =3D Msr.Ghcb; =20 - VmgInit (Ghcb); + VmgInit (Ghcb, &InterruptState); =20 ExitCode =3D Regs->ExceptionData; switch (ExitCode) { @@ -1662,7 +1663,7 @@ VmgExitHandleVc ( VcRet =3D EFI_PROTOCOL_ERROR; } =20 - VmgDone (Ghcb); + VmgDone (Ghcb, InterruptState); =20 return VcRet; } diff --git a/OvmfPkg/QemuFlashFvbServicesRuntimeDxe/QemuFlashDxe.c b/OvmfPk= g/QemuFlashFvbServicesRuntimeDxe/QemuFlashDxe.c index f9b21b54137d..1b0742967f71 100644 --- a/OvmfPkg/QemuFlashFvbServicesRuntimeDxe/QemuFlashDxe.c +++ b/OvmfPkg/QemuFlashFvbServicesRuntimeDxe/QemuFlashDxe.c @@ -52,6 +52,7 @@ QemuFlashPtrWrite ( if (MemEncryptSevEsIsEnabled ()) { MSR_SEV_ES_GHCB_REGISTER Msr; GHCB *Ghcb; + BOOLEAN InterruptState; =20 Msr.GhcbPhysicalAddress =3D AsmReadMsr64 (MSR_SEV_ES_GHCB); Ghcb =3D Msr.Ghcb; @@ -63,12 +64,12 @@ QemuFlashPtrWrite ( // #VC exception. Instead, use the the VMGEXIT MMIO write support dire= ctly // to perform the update. // - VmgInit (Ghcb); + VmgInit (Ghcb, &InterruptState); Ghcb->SharedBuffer[0] =3D Value; Ghcb->SaveArea.SwScratch =3D (UINT64) (UINTN) Ghcb->SharedBuffer; VmgSetOffsetValid (Ghcb, GhcbSwScratch); VmgExit (Ghcb, SVM_EXIT_MMIO_WRITE, (UINT64) (UINTN) Ptr, 1); - VmgDone (Ghcb); + VmgDone (Ghcb, InterruptState); } else { *Ptr =3D Value; } diff --git a/UefiCpuPkg/Library/MpInitLib/DxeMpLib.c b/UefiCpuPkg/Library/M= pInitLib/DxeMpLib.c index 2c00d72ddefe..7839c249760e 100644 --- a/UefiCpuPkg/Library/MpInitLib/DxeMpLib.c +++ b/UefiCpuPkg/Library/MpInitLib/DxeMpLib.c @@ -171,6 +171,7 @@ GetSevEsAPMemory ( EFI_PHYSICAL_ADDRESS StartAddress; MSR_SEV_ES_GHCB_REGISTER Msr; GHCB *Ghcb; + BOOLEAN InterruptState; =20 // // Allocate 1 page for AP jump table page @@ -192,9 +193,9 @@ GetSevEsAPMemory ( Msr.GhcbPhysicalAddress =3D AsmReadMsr64 (MSR_SEV_ES_GHCB); Ghcb =3D Msr.Ghcb; =20 - VmgInit (Ghcb); + VmgInit (Ghcb, &InterruptState); VmgExit (Ghcb, SVM_EXIT_AP_JUMP_TABLE, 0, (UINT64) (UINTN) StartAddress)= ; - VmgDone (Ghcb); + VmgDone (Ghcb, InterruptState); =20 return (UINTN) StartAddress; } diff --git a/UefiCpuPkg/Library/MpInitLib/MpLib.c b/UefiCpuPkg/Library/MpIn= itLib/MpLib.c index 6d977d45bcdd..1f47ff3f73b5 100644 --- a/UefiCpuPkg/Library/MpInitLib/MpLib.c +++ b/UefiCpuPkg/Library/MpInitLib/MpLib.c @@ -884,6 +884,7 @@ ApWakeupFunction ( GHCB *Ghcb; UINT64 Status; BOOLEAN DoDecrement; + BOOLEAN InterruptState; =20 DoDecrement =3D (BOOLEAN) (CpuMpData->InitFlag =3D=3D ApInitConf= ig); =20 @@ -891,7 +892,7 @@ ApWakeupFunction ( Msr.GhcbPhysicalAddress =3D AsmReadMsr64 (MSR_SEV_ES_GHCB); Ghcb =3D Msr.Ghcb; =20 - VmgInit (Ghcb); + VmgInit (Ghcb, &InterruptState); =20 if (DoDecrement) { DoDecrement =3D FALSE; @@ -905,11 +906,11 @@ ApWakeupFunction ( =20 Status =3D VmgExit (Ghcb, SVM_EXIT_AP_RESET_HOLD, 0, 0); if ((Status =3D=3D 0) && (Ghcb->SaveArea.SwExitInfo2 !=3D 0)) = { - VmgDone (Ghcb); + VmgDone (Ghcb, InterruptState); break; } =20 - VmgDone (Ghcb); + VmgDone (Ghcb, InterruptState); } =20 // diff --git a/UefiCpuPkg/Library/VmgExitLibNull/VmgExitLibNull.c b/UefiCpuPk= g/Library/VmgExitLibNull/VmgExitLibNull.c index b47e282aff82..89b065cb3ff3 100644 --- a/UefiCpuPkg/Library/VmgExitLibNull/VmgExitLibNull.c +++ b/UefiCpuPkg/Library/VmgExitLibNull/VmgExitLibNull.c @@ -57,15 +57,16 @@ VmgExit ( Performs the necessary steps in preparation for invoking VMGEXIT. Must b= e called before setting any fields within the GHCB. =20 - The base library function does nothing. - - @param[in, out] Ghcb A pointer to the GHCB + @param[in, out] Ghcb A pointer to the GHCB + @param[in, out] InterruptState A pointer to hold the current interrupt + state, used for restoring in VmgDone () =20 **/ VOID EFIAPI VmgInit ( - IN OUT GHCB *Ghcb + IN OUT GHCB *Ghcb, + IN OUT BOOLEAN *InterruptState ) { } @@ -76,15 +77,16 @@ VmgInit ( Performs the necessary steps to cleanup after invoking VMGEXIT. Must be called after obtaining needed fields within the GHCB. =20 - The base library function does nothing. - - @param[in, out] Ghcb A pointer to the GHCB + @param[in, out] Ghcb A pointer to the GHCB + @param[in] InterruptState An indicator to conditionally (re)enabl= e + interrupts =20 **/ VOID EFIAPI VmgDone ( - IN OUT GHCB *Ghcb + IN OUT GHCB *Ghcb, + IN BOOLEAN InterruptState ) { } --=20 2.28.0