From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from us-smtp-1.mimecast.com (us-smtp-1.mimecast.com [205.139.110.120]) by mx.groups.io with SMTP id smtpd.web11.52726.1597088188632222033 for ; Mon, 10 Aug 2020 12:36:28 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=NtXxsfkp; spf=pass (domain: redhat.com, ip: 205.139.110.120, mailfrom: lersek@redhat.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1597088187; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZEW2nPkXMGDYBB2vUXsm/d87/OKNd+gkFf0drWxLa7E=; b=NtXxsfkpgNMkHlH85F+0VSfsMIvDxaemRBuK+IAWOOPfrWDWm0vabShM+8ou+RQHricpSu uK4TF928ye9HvMDPjR9dURto2mUI33cYCSx7K2Z0uE3Ira+sX5BHWoD01aRLR5R6MVjtOh jewQCBsdvAtRUWLWkkQO14RK/8YmjPE= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-342-NuiU0MaMPsiGTOzu9G96KQ-1; Mon, 10 Aug 2020 15:36:12 -0400 X-MC-Unique: NuiU0MaMPsiGTOzu9G96KQ-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 3B7F691271; Mon, 10 Aug 2020 19:36:10 +0000 (UTC) Received: from lacos-laptop-7.usersys.redhat.com (ovpn-112-53.ams2.redhat.com [10.36.112.53]) by smtp.corp.redhat.com (Postfix) with ESMTP id 77FA07C0F5; Mon, 10 Aug 2020 19:36:05 +0000 (UTC) Subject: Re: [edk2-devel] [PATCH v14 00/46] SEV-ES guest support To: devel@edk2.groups.io, thomas.lendacky@amd.com Cc: Brijesh Singh , Ard Biesheuvel , Eric Dong , Jordan Justen , Liming Gao , Michael D Kinney , Ray Ni , Andrew Fish , Anthony Perard , Benjamin You , Dandan Bi , Guo Dong , Hao A Wu , Jian J Wang , Julien Grall , Leif Lindholm , Maurice Ma References: From: "Laszlo Ersek" Message-ID: <4fa19934-008b-f8e5-6db8-0c39fe9875de@redhat.com> Date: Mon, 10 Aug 2020 21:36:04 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Firefox/52.0 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=lersek@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit On 08/07/20 21:38, Lendacky, Thomas wrote: > From: Tom Lendacky > > This patch series provides support for running EDK2/OVMF under SEV-ES. > > Secure Encrypted Virtualization - Encrypted State (SEV-ES) expands on the > SEV support to protect the guest register state from the hypervisor. See > "AMD64 Architecture Programmer's Manual Volume 2: System Programming", > section "15.35 Encrypted State (SEV-ES)" [1]. > > In order to allow a hypervisor to perform functions on behalf of a guest, > there is architectural support for notifying a guest's operating system > when certain types of VMEXITs are about to occur. This allows the guest to > selectively share information with the hypervisor to satisfy the requested > function. The notification is performed using a new exception, the VMM > Communication exception (#VC). The information is shared through the > Guest-Hypervisor Communication Block (GHCB) using the VMGEXIT instruction. > The GHCB format and the protocol for using it is documented in "SEV-ES > Guest-Hypervisor Communication Block Standardization" [2]. > > The main areas of the EDK2 code that are updated to support SEV-ES are > around the exception handling support and the AP boot support. > > Exception support is required starting in Sec, continuing through Pei > and into Dxe in order to handle #VC exceptions that are generated. Each > AP requires it's own GHCB page as well as a page to hold values specific > to that AP. > > AP booting poses some interesting challenges. The INIT-SIPI-SIPI sequence > is typically used to boot the APs. However, the hypervisor is not allowed > to update the guest registers. The GHCB document [2] talks about how SMP > booting under SEV-ES is performed. > > Since the GHCB page must be a shared (unencrypted) page, the processor > must be running in long mode in order for the guest and hypervisor to > communicate with each other. As a result, SEV-ES is only supported under > the X64 architecture. > > This series adds a new library requirement for the VmgExitLib library > against the UefiCpuPkg CpuExceptionHandlerLib library and the UefiCpuPkg > MpInitLib library. The edk2-platforms repo requires updates/patches to > add the new library requirement. To accomodate that, this series could be > split between: > > patch number 10: > UefiPayloadPkg: Prepare UefiPayloadPkg to use the VmgExitLib library > > and patch number 11: > UefiCpuPkg/CpuExceptionHandler: Add base support for the #VC exception > > The updates to edk2-platforms can be applied at the split. > > [1] https://www.amd.com/system/files/TechDocs/24593.pdf > [2] https://developer.amd.com/wp-content/resources/56421.pdf > > --- > > These patches are based on commit: > 9565ab67c209 ("ShellPkg: smbiosview - Change some type 17 field values format") > > A version of the tree can be found at: > https://github.com/AMDESE/ovmf/tree/sev-es-v22 > > Cc: Andrew Fish > Cc: Anthony Perard > Cc: Ard Biesheuvel > Cc: Benjamin You > Cc: Dandan Bi > Cc: Eric Dong > Cc: Guo Dong > Cc: Hao A Wu > Cc: Jian J Wang > Cc: Jordan Justen > Cc: Julien Grall > Cc: Laszlo Ersek > Cc: Leif Lindholm > Cc: Liming Gao > Cc: Maurice Ma > Cc: Michael D Kinney > Cc: Ray Ni > > Changes since v13: > - Fixup the AsmRelocateApLoop() call site so IA32 successfully boots APs. > Do this by appending the three new parameters without altering the > original parameter passing order. > - Minor updates to description text and help text to expand the GHCB > acronym. > > Changes since v12: > - Change IA32 VMGEXIT .nasm file to issue an int 3. Depending on the > version of NASM, the "BITS 64" trick to get NASM to recognize the > VMMCALL instruction (VMGEXIT is a REP VMMCALL) caused an error. Since > SEV-ES is X64 only, VMGEXIT should never be called in IA32. > > Changes since v11: > - Make the XGETBV and VMGEXIT .nasm files buildable for all environments > and remove the updates that add these instructions to GccInline.c > > Changes since v10: > - Fix conflicts around GccInline.c file after moving to latest commit > - Fix conflicts with OVMF PCD values after moving to latest commit > > Changes since v9: > - Fixed bit field declarations in the GHCB structure to use UINT32 > and not UINT64. > - Fixed a warning produced by VS2019 in the instruction parsing code > by expliciting casting a bit shift to an INT64. > - Sorted section entries in the OVMF VmgExitLib INF file. > - Moved the new Maintainers.txt entry so entries remain sorted. > - Documentation style fixes for return values. > - Miscellaneous code style fixes. > > Changes since v8: > - Move IOIO exit info definitions into Ghcb.h file > - Add a macro for calculating IO instruction bytes (IOIO_DATA_BYTES) > - Exception handler support for debug registers > - Moved the DRx register saving changes into the UefiCpuPkg patch for > base #VC support in CpuExceptionHandlerLib. > - OvmfPkg VmgExitLib > - Remove the .uni file > - Update .inf file: > - New file location for VmgExitVcHandler.c > - Add additional Packages and LibraryClasses > - Introduce a header file to hold the #VC instruction parsing related > definitions > - Include additional #defines for instruction decoding to replace > hard coded values for things like instruction prefixes and escapes. > - Replace hardcoded CPUID values with values from existing header files > and use existing CR4 definition for accessing CR4 data. > - Change the type used for obtaining data addresses in the instruction > parsing > - Switch from INTN to UINT64 and use compiler conversions and casting > to perform the correct address calculation > - ResetVector code: > - Revert some inadvertant changes introduced in v7 for reserving the > SEV-ES work area memory and for checking the status of SEV-ES. > - AP Booting > - Provide support for non-broadcast INIT-SIPI-SIPI AP boot (minimize > code duplication by creating a function to set the AP jump table > vector address). > - Fix file/directory entry in maintainer changes. > - Various coding style fixes > - Commenting, if statements, etc. > - Various documentation style fixes > > Changes since v7: > - Reserve the SEV-ES workarea when S3 is enabled > - Fix warnings issued by the Visual Studio compiler > - Create a NULL VmgExitLib instance that is used for VMGEXIT > related operations as well as #VC handling. Then create the full > VmgExitLib support only in OvmfPkg - where it will be used. This > removes a bunch of implementation code from platforms that will > not be using the functionality. > - Remove single use interfaces from the VmgExitLib (VmgMmioWrite > and VmgSetApJumpTable) > > Changes since v6: > - Add function comments to all functions, including local functions > - Add function parameter direction to all functions (in/out) > - Add support for MMIO MOVZX/MOVSX instructions > - Ensure the per-CPU variable page remains encrypted > - Coding-style fixes as identified by Ecc > > Changes since v5: > - Remove extraneous VmgExitLib usage > - Miscellaneous changes to address feedback (coding style, etc.) > > Changes since v4: > - Move the SEV-ES protocol negotiation out of the SEC exception handler > and into the SecMain.c file. As a result: > - Move the SecGhcb related PCDs out of UefiCpuPkg and into OvmfPkg > - Combine SecAMDSevVcHandler.c and PeiDxeAMDSevVcHandler.c into a > single AMDSevVcHandler.c > - Consolidate VmgExitLib usage into common LibraryClasses sections > - Add documentation comments to the VmgExitLib functions > > Changes since v3: > - Remove the need for the MP library finalization routine. The AP > jump table address will be held by the hypervisor rather than > communicated via the GHCB MSR. This removes some fragility around > the UEFI to OS transition. > - Rename the SEV-ES RIP reset area to SEV-ES workarea and use it to > communicate the SEV-ES status, so that SEC CPU exception handling is > only established for an SEV-ES guest. > - Fix SMM build breakageAdd around QemuFlashPtrWrite(). > - Fix SMM build breakage by adding VC exception support the SMM CPU > exception handling. > - Add memory fencing around the invocation of AsmVmgExit(). > - Clarify comments around the SEV-ES AP reset RIP values and usage. > - Move some PCD definitions from MdeModulePkg to UefiCpuPkg. > - Remove the 16-bit code selector definition from MdeModulePkg > > Changes since v2: > - Added a way to locate the SEV-ES fixed AP RIP address for starting > AP's to avoid updating the actual flash image (build time location > that is identified with a GUID value). > - Create a VmgExit library to replace static inline functions. > - Move some PCDs to the appropriate packages > - Add support for writing to QEMU flash under SEV-ES > - Add additional MMIO opcode support > - Cleaned up the GHCB MSR CPUID protocol support > > Changes since v1: > - Patches reworked to be more specific to the component/area being updated > and order of definition/usage > - Created a library for VMGEXIT-related functions to replace use of inline > functions > - Allocation method for GDT changed from AllocatePool to AllocatePages > - Early caching only enabled for SEV-ES guests > - Ensure AP loop mode set to halt loop mode for SEV-ES guests > - Reserved SEC GHCB-related memory areas when S3 is enabled > > Tom Lendacky (46): > MdeModulePkg: Create PCDs to be used in support of SEV-ES > UefiCpuPkg: Create PCD to be used in support of SEV-ES > MdePkg: Add the MSR definition for the GHCB register > MdePkg: Add a structure definition for the GHCB > MdeModulePkg/DxeIplPeim: Support GHCB pages when creating page tables > MdePkg/BaseLib: Add support for the XGETBV instruction > MdePkg/BaseLib: Add support for the VMGEXIT instruction > UefiCpuPkg: Implement library support for VMGEXIT > OvmfPkg: Prepare OvmfPkg to use the VmgExitLib library > UefiPayloadPkg: Prepare UefiPayloadPkg to use the VmgExitLib library > UefiCpuPkg/CpuExceptionHandler: Add base support for the #VC exception > OvmfPkg/VmgExitLib: Implement library support for VmgExitLib in OVMF > OvmfPkg/VmgExitLib: Add support for IOIO_PROT NAE events > OvmfPkg/VmgExitLib: Support string IO for IOIO_PROT NAE events > OvmfPkg/VmgExitLib: Add support for CPUID NAE events > OvmfPkg/VmgExitLib: Add support for MSR_PROT NAE events > OvmfPkg/VmgExitLib: Add support for NPF NAE events (MMIO) > OvmfPkg/VmgExitLib: Add support for WBINVD NAE events > OvmfPkg/VmgExitLib: Add support for RDTSC NAE events > OvmfPkg/VmgExitLib: Add support for RDPMC NAE events > OvmfPkg/VmgExitLib: Add support for INVD NAE events > OvmfPkg/VmgExitLib: Add support for VMMCALL NAE events > OvmfPkg/VmgExitLib: Add support for RDTSCP NAE events > OvmfPkg/VmgExitLib: Add support for MONITOR/MONITORX NAE events > OvmfPkg/VmgExitLib: Add support for MWAIT/MWAITX NAE events > OvmfPkg/VmgExitLib: Add support for DR7 Read/Write NAE events > OvmfPkg/MemEncryptSevLib: Add an SEV-ES guest indicator function > OvmfPkg: Add support to perform SEV-ES initialization > OvmfPkg: Create a GHCB page for use during Sec phase > OvmfPkg/PlatformPei: Reserve GHCB-related areas if S3 is supported > OvmfPkg: Create GHCB pages for use during Pei and Dxe phase > OvmfPkg/PlatformPei: Move early GDT into ram when SEV-ES is enabled > UefiCpuPkg: Create an SEV-ES workarea PCD > OvmfPkg: Reserve a page in memory for the SEV-ES usage > OvmfPkg/PlatformPei: Reserve SEV-ES work area if S3 is supported > OvmfPkg/ResetVector: Add support for a 32-bit SEV check > OvmfPkg/Sec: Add #VC exception handling for Sec phase > OvmfPkg/Sec: Enable cache early to speed up booting > OvmfPkg/QemuFlashFvbServicesRuntimeDxe: Bypass flash detection with > SEV-ES > UefiCpuPkg: Add a 16-bit protected mode code segment descriptor > UefiCpuPkg/MpInitLib: Add CPU MP data flag to indicate if SEV-ES is > enabled > UefiCpuPkg: Allow AP booting under SEV-ES > OvmfPkg: Use the SEV-ES work area for the SEV-ES AP reset vector > OvmfPkg: Move the GHCB allocations into reserved memory > UefiCpuPkg/MpInitLib: Prepare SEV-ES guest APs for OS use > Maintainers.txt: Add reviewers for the OvmfPkg SEV-related files > > MdeModulePkg/MdeModulePkg.dec | 9 + > OvmfPkg/OvmfPkg.dec | 9 + > UefiCpuPkg/UefiCpuPkg.dec | 17 + > OvmfPkg/OvmfPkgIa32.dsc | 6 + > OvmfPkg/OvmfPkgIa32X64.dsc | 6 + > OvmfPkg/OvmfPkgX64.dsc | 6 + > OvmfPkg/OvmfXen.dsc | 1 + > UefiCpuPkg/UefiCpuPkg.dsc | 2 + > UefiPayloadPkg/UefiPayloadPkgIa32.dsc | 2 + > UefiPayloadPkg/UefiPayloadPkgIa32X64.dsc | 2 + > OvmfPkg/OvmfPkgX64.fdf | 9 + > MdeModulePkg/Core/DxeIplPeim/DxeIpl.inf | 2 + > MdePkg/Library/BaseLib/BaseLib.inf | 4 + > OvmfPkg/Library/VmgExitLib/VmgExitLib.inf | 36 + > OvmfPkg/PlatformPei/PlatformPei.inf | 9 + > .../FvbServicesRuntimeDxe.inf | 2 + > OvmfPkg/ResetVector/ResetVector.inf | 8 + > OvmfPkg/Sec/SecMain.inf | 4 + > .../DxeCpuExceptionHandlerLib.inf | 1 + > .../PeiCpuExceptionHandlerLib.inf | 1 + > .../SecPeiCpuExceptionHandlerLib.inf | 1 + > .../SmmCpuExceptionHandlerLib.inf | 1 + > .../Xcode5SecPeiCpuExceptionHandlerLib.inf | 1 + > UefiCpuPkg/Library/MpInitLib/DxeMpInitLib.inf | 4 + > UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf | 4 + > .../Library/VmgExitLibNull/VmgExitLibNull.inf | 27 + > .../Core/DxeIplPeim/X64/VirtualMemory.h | 12 +- > MdePkg/Include/Library/BaseLib.h | 31 + > MdePkg/Include/Register/Amd/Fam17Msr.h | 46 + > MdePkg/Include/Register/Amd/Ghcb.h | 166 ++ > .../IndustryStandard/InstructionParsing.h | 83 + > OvmfPkg/Include/Library/MemEncryptSevLib.h | 12 + > .../QemuFlash.h | 13 + > UefiCpuPkg/CpuDxe/CpuGdt.h | 4 +- > UefiCpuPkg/Include/Library/VmgExitLib.h | 103 + > UefiCpuPkg/Library/MpInitLib/MpLib.h | 68 +- > .../Core/DxeIplPeim/Ia32/DxeLoadFunc.c | 4 +- > .../Core/DxeIplPeim/X64/DxeLoadFunc.c | 11 +- > .../Core/DxeIplPeim/X64/VirtualMemory.c | 57 +- > .../MemEncryptSevLibInternal.c | 75 +- > OvmfPkg/Library/VmgExitLib/VmgExitLib.c | 159 ++ > OvmfPkg/Library/VmgExitLib/VmgExitVcHandler.c | 1716 +++++++++++++++++ > OvmfPkg/PlatformPei/AmdSev.c | 89 + > OvmfPkg/PlatformPei/MemDetect.c | 43 + > .../QemuFlash.c | 23 +- > .../QemuFlashDxe.c | 40 + > .../QemuFlashSmm.c | 16 + > OvmfPkg/Sec/SecMain.c | 188 +- > UefiCpuPkg/CpuDxe/CpuGdt.c | 8 +- > .../CpuExceptionCommon.c | 10 +- > .../PeiDxeSmmCpuException.c | 20 +- > .../SecPeiCpuException.c | 19 + > UefiCpuPkg/Library/MpInitLib/DxeMpLib.c | 120 +- > UefiCpuPkg/Library/MpInitLib/MpLib.c | 337 +++- > UefiCpuPkg/Library/MpInitLib/PeiMpLib.c | 19 + > .../Library/VmgExitLibNull/VmgExitLibNull.c | 121 ++ > UefiCpuPkg/PiSmmCpuDxeSmm/X64/SmmFuncsArch.c | 2 +- > Maintainers.txt | 10 + > MdeModulePkg/MdeModulePkg.uni | 8 + > MdePkg/Library/BaseLib/Ia32/VmgExit.nasm | 38 + > MdePkg/Library/BaseLib/Ia32/XGetBv.nasm | 31 + > MdePkg/Library/BaseLib/X64/VmgExit.nasm | 32 + > MdePkg/Library/BaseLib/X64/XGetBv.nasm | 34 + > OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm | 100 + > OvmfPkg/ResetVector/Ia32/PageTables64.asm | 351 +++- > OvmfPkg/ResetVector/ResetVector.nasmb | 20 + > .../X64/ExceptionHandlerAsm.nasm | 17 + > .../X64/Xcode5ExceptionHandlerAsm.nasm | 17 + > UefiCpuPkg/Library/MpInitLib/Ia32/MpEqu.inc | 2 +- > .../Library/MpInitLib/Ia32/MpFuncs.nasm | 20 +- > UefiCpuPkg/Library/MpInitLib/X64/MpEqu.inc | 4 +- > UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm | 369 +++- > .../Library/VmgExitLibNull/VmgExitLibNull.uni | 15 + > .../ResetVector/Vtf0/Ia16/Real16ToFlat32.asm | 9 + > UefiCpuPkg/UefiCpuPkg.uni | 11 + > 75 files changed, 4777 insertions(+), 100 deletions(-) > create mode 100644 OvmfPkg/Library/VmgExitLib/VmgExitLib.inf > create mode 100644 UefiCpuPkg/Library/VmgExitLibNull/VmgExitLibNull.inf > create mode 100644 MdePkg/Include/Register/Amd/Ghcb.h > create mode 100644 OvmfPkg/Include/IndustryStandard/InstructionParsing.h > create mode 100644 UefiCpuPkg/Include/Library/VmgExitLib.h > create mode 100644 OvmfPkg/Library/VmgExitLib/VmgExitLib.c > create mode 100644 OvmfPkg/Library/VmgExitLib/VmgExitVcHandler.c > create mode 100644 UefiCpuPkg/Library/VmgExitLibNull/VmgExitLibNull.c > create mode 100644 MdePkg/Library/BaseLib/Ia32/VmgExit.nasm > create mode 100644 MdePkg/Library/BaseLib/Ia32/XGetBv.nasm > create mode 100644 MdePkg/Library/BaseLib/X64/VmgExit.nasm > create mode 100644 MdePkg/Library/BaseLib/X64/XGetBv.nasm > create mode 100644 OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm > create mode 100644 UefiCpuPkg/Library/VmgExitLibNull/VmgExitLibNull.uni > For all patches except #10 ("UefiPayloadPkg: Prepare UefiPayloadPkg to use the VmgExitLib library") and #46 ("Maintainers.txt: Add reviewers for the OvmfPkg SEV-related files"): Regression-tested-by: Laszlo Ersek Thanks Laszlo