> On May 9, 2020, at 7:34 AM, Lendacky, Thomas wrote: > > On 5/9/20 1:44 AM, Ni, Ray wrote: >> Tom, > > Hi Ray, > >> I have a bit concern on your change that directly modifies CpuExceptionHandlerLib to handle >> exception #29. Today's CpuExceptionHandlerLib simplify dumps the exception context for >> every exception. Any component which wants to do specific handling of certain exceptions >> should call RegisterCpuInterruptHandler(). Such as code in CpuDxe driver: >> if (HEAP_GUARD_NONSTOP_MODE || NULL_DETECTION_NONSTOP_MODE) { >> RegisterCpuInterruptHandler (EXCEPT_IA32_DEBUG, DebugExceptionHandler); >> RegisterCpuInterruptHandler (EXCEPT_IA32_PAGE_FAULT, PageFaultExceptionHandler); >> } >> Is it possible for your feature to follow the same pattern? > > There are two problems: > > The first is that RegisterCpuInterruptHandler() is not implemented for both the SEC and PEI phases, so it is not currently possible to register a handler that early. > > The second is that I need to be able to propagate an exception request from the hypervisor. With the current implementation there doesn't appear to be an easy way to perform this propagation. > > If there's a way to accomplish both of the above I wouldn't be opposed to using RegisterCpuInterruptHandler() as long as there are no #VCs that can occur between initializing exception handling and and registering the #VC handler. > Thomas, As you point out it is tricky dealing with XIP code. You can't have globals that you can write and generally you use a PEI service to look tings up, the most common thing being using a HOB. But SEC has no services and I'm not sure you really want to be calling into the PEI Core on a random exception. Here are the best options that popped into my head after reading your email 1) IDT in RAM If your code populates the IDT the IDTR gives you access to the address of the IDTR via an instruction. The PI Spec reserves IDT - sizeof (UNITN) for a cached copy of the PEI Services Table, but otther than that you are good to go. It should be possible to have a global so you can have the table required to implement RegisterCpuInterruptHandler(). There might be some usage of IDT - ( 2* sizeof(UINTN)), I know I'm guilty, so storing data after the IDT would be a good option. In general if your code allocates the memory for the IDT then you can treat the IDT as part of your private context data structure and that gives you access 2) IDT in ROM. For this it seems like you need a library to link in to the CpuExceptionHandlerLib that allows you to override the handler. If CpuInterruptHandlerOverride() returns NULL you do the current behavior if not NULL then you call the returned handler. EFI_CPU_INTERRUPT_HANDLER EFIAPI OverrideCpuInterruptHandler ( IN EFI_EXCEPTION_TYPE InterruptType ); Thanks, Andrew Fish PS Off topic, but it would also be useful to have a library that overrides the state dump display. For example using Xcode you can always display a stack frame from the exception handler. > Thanks, > Tom > >> Thanks, >> Ray >>> -----Original Message----- >>> From: Tom Lendacky >>> Sent: Saturday, May 9, 2020 3:16 AM >>> To: devel@edk2.groups.io >>> Cc: Justen, Jordan L ; Laszlo Ersek ; Ard Biesheuvel >>> ; Kinney, Michael D ; Gao, Liming ; Dong, >>> Eric ; Ni, Ray ; Brijesh Singh ; You, Benjamin >>> ; Bi, Dandan ; Dong, Guo ; Wu, Hao A >>> ; Wang, Jian J ; Ma, Maurice >>> Subject: Re: [PATCH v7 00/43] SEV-ES guest support >>> >>> I was able to use the pull request method that Laszlo documented and fixed >>> up all of the issues identified by the VS compiler. >>> >>> An additional change I'm planning to make for the next version (v8) of the >>> patches is to create a NULL library instance of the VmgExitLib that will >>> also include the #VC handler function. This will reduce the amount of code >>> associated with this feature for platforms that don't use/support SEV-ES. >>> >>> Laszlo, this will mean that I will introduce a version of the VmgExitLib >>> under OvmfPkg that will provide the majority of the functionality that is >>> present today in UefiCpuPkg. In essence, the functionality in v7 patches 8 >>> and 11 - 25 will now live under OvmfPkg instead of UefiCpuPkg. I think >>> this is the better way to do this. Let me know if you have any concerns. >>> >>> Thanks, >>> Tom >>> >>> On 4/22/20 12:41 PM, Tom Lendacky wrote: >>>> This patch series provides support for running EDK2/OVMF under SEV-ES. >>>> >>>> Secure Encrypted Virtualization - Encrypted State (SEV-ES) expands on the >>>> SEV support to protect the guest register state from the hypervisor. See >>>> "AMD64 Architecture Programmer's Manual Volume 2: System Programming", >>>> section "15.35 Encrypted State (SEV-ES)" [1]. >>>> >>>> In order to allow a hypervisor to perform functions on behalf of a guest, >>>> there is architectural support for notifying a guest's operating system >>>> when certain types of VMEXITs are about to occur. This allows the guest to >>>> selectively share information with the hypervisor to satisfy the requested >>>> function. The notification is performed using a new exception, the VMM >>>> Communication exception (#VC). The information is shared through the >>>> Guest-Hypervisor Communication Block (GHCB) using the VMGEXIT instruction. >>>> The GHCB format and the protocol for using it is documented in "SEV-ES >>>> Guest-Hypervisor Communication Block Standardization" [2]. >>>> >>>> The main areas of the EDK2 code that are updated to support SEV-ES are >>>> around the exception handling support and the AP boot support. >>>> >>>> Exception support is required starting in Sec, continuing through Pei >>>> and into Dxe in order to handle #VC exceptions that are generated. Each >>>> AP requires it's own GHCB page as well as a page to hold values specific >>>> to that AP. >>>> >>>> AP booting poses some interesting challenges. The INIT-SIPI-SIPI sequence >>>> is typically used to boot the APs. However, the hypervisor is not allowed >>>> to update the guest registers. The GHCB document [2] talks about how SMP >>>> booting under SEV-ES is performed. >>>> >>>> Since the GHCB page must be a shared (unencrypted) page, the processor >>>> must be running in long mode in order for the guest and hypervisor to >>>> communicate with each other. As a result, SEV-ES is only supported under >>>> the X64 architecture. >>>> >>>> [1] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.amd.com%2Fsystem%2Ffiles%2FTechDocs%2F24593.pdf&data=02%7C01%7Cthomas.lendacky%40amd.com%7Cf5d7875dfcf54e45c42208d7f3e4676b%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637246036118033165&sdata=H74fQl1n2sXzCMSoGm1tGOKc5epMtVkGJFCidwLMl5c%3D&reserved=0 >>>> [2] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdeveloper.amd.com%2Fwp-content%2Fresources%2F56421.pdf&data=02%7C01%7Cthomas.lendacky%40amd.com%7Cf5d7875dfcf54e45c42208d7f3e4676b%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637246036118033165&sdata=EwW9575nJMaWxizo2XrLHjrbUMJIB0WFTDLjwy%2BM%2F4k%3D&reserved=0 >>>> >>>> --- >>>> >>>> These patches are based on commit: >>>> be7295b36405 (".python/SpellCheck: Increase SpellCheck plugin max failures") >>>> >>>> Proper execution of SEV-ES relies on Bugzilla 2340 being fixed. >>>> >>>> A version of the tree (with an extra patch to workaround Bugzilla 2340) can >>>> be found at: >>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAMDESE%2Fovmf%2Ftree%2Fsev-es-v14&data=02%7C01%7Cthomas.lendacky%40amd.com%7Cf5d7875dfcf54e45c42208d7f3e4676b%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637246036118033165&sdata=U8fIzb%2F4A8WBaiVbScxUuGDw22kyxxnRP5olSyTedvE%3D&reserved=0 >>>> >>>> Cc: Ard Biesheuvel > >>>> Cc: Benjamin You > >>>> Cc: Dandan Bi > >>>> Cc: Eric Dong > >>>> Cc: Guo Dong > >>>> Cc: Hao A Wu > >>>> Cc: Jian J Wang > >>>> Cc: Jordan Justen > >>>> Cc: Laszlo Ersek > >>>> Cc: Liming Gao > >>>> Cc: Maurice Ma > >>>> Cc: Michael D Kinney > >>>> Cc: Ray Ni > >>>> >>>> Changes since v6: >>>> - Add function comments to all functions, including local functions >>>> - Add function parameter direction to all functions (in/out) >>>> - Add support for MMIO MOVZX/MOVSX instructions >>>> - Ensure the per-CPU variable page remains encrypted >>>> - Coding-style fixes as identified by Ecc >>>> >>>> Changes since v5: >>>> - Remove extraneous VmgExitLib usage >>>> - Miscellaneous changes to address feedback (coding style, etc.) >>>> >>>> Changes since v4: >>>> - Move the SEV-ES protocol negotiation out of the SEC exception handler >>>> and into the SecMain.c file. As a result: >>>> - Move the SecGhcb related PCDs out of UefiCpuPkg and into OvmfPkg >>>> - Combine SecAMDSevVcHandler.c and PeiDxeAMDSevVcHandler.c into a >>>> single AMDSevVcHandler.c >>>> - Consolidate VmgExitLib usage into common LibraryClasses sections >>>> - Add documentation comments to the VmgExitLib functions >>>> >>>> Changes since v3: >>>> - Remove the need for the MP library finalization routine. The AP >>>> jump table address will be held by the hypervisor rather than >>>> communicated via the GHCB MSR. This removes some fragility around >>>> the UEFI to OS transition. >>>> - Rename the SEV-ES RIP reset area to SEV-ES workarea and use it to >>>> communicate the SEV-ES status, so that SEC CPU exception handling is >>>> only established for an SEV-ES guest. >>>> - Fix SMM build breakageAdd around QemuFlashPtrWrite(). >>>> - Fix SMM build breakage by adding VC exception support the SMM CPU >>>> exception handling. >>>> - Add memory fencing around the invocation of AsmVmgExit(). >>>> - Clarify comments around the SEV-ES AP reset RIP values and usage. >>>> - Move some PCD definitions from MdeModulePkg to UefiCpuPkg. >>>> - Remove the 16-bit code selector definition from MdeModulePkg >>>> >>>> Changes since v2: >>>> - Added a way to locate the SEV-ES fixed AP RIP address for starting >>>> AP's to avoid updating the actual flash image (build time location >>>> that is identified with a GUID value). >>>> - Create a VmgExit library to replace static inline functions. >>>> - Move some PCDs to the appropriate packages >>>> - Add support for writing to QEMU flash under SEV-ES >>>> - Add additional MMIO opcode support >>>> - Cleaned up the GHCB MSR CPUID protocol support >>>> >>>> Changes since v1: >>>> - Patches reworked to be more specific to the component/area being updated >>>> and order of definition/usage >>>> - Created a library for VMGEXIT-related functions to replace use of inline >>>> functions >>>> - Allocation method for GDT changed from AllocatePool to AllocatePages >>>> - Early caching only enabled for SEV-ES guests >>>> - Ensure AP loop mode set to halt loop mode for SEV-ES guests >>>> - Reserved SEC GHCB-related memory areas when S3 is enabled >>>> >>>> Tom Lendacky (43): >>>> MdeModulePkg: Create PCDs to be used in support of SEV-ES >>>> UefiCpuPkg: Create PCD to be used in support of SEV-ES >>>> MdePkg: Add the MSR definition for the GHCB register >>>> MdePkg: Add a structure definition for the GHCB >>>> MdeModulePkg/DxeIplPeim: Support GHCB pages when creating page tables >>>> MdePkg/BaseLib: Add support for the XGETBV instruction >>>> MdePkg/BaseLib: Add support for the VMGEXIT instruction >>>> UefiCpuPkg: Implement library support for VMGEXIT >>>> OvmfPkg: Prepare OvmfPkg to use the VmgExitLib library >>>> UefiPayloadPkg: Prepare UefiPayloadPkg to use the VmgExitLib library >>>> UefiCpuPkg/CpuExceptionHandler: Add base support for the #VC exception >>>> UefiCpuPkg/CpuExceptionHandler: Add support for IOIO_PROT NAE events >>>> UefiCpuPkg/CpuExceptionHandler: Support string IO for IOIO_PROT NAE >>>> events >>>> UefiCpuPkg/CpuExceptionHandler: Add support for CPUID NAE events >>>> UefiCpuPkg/CpuExceptionHandler: Add support for MSR_PROT NAE events >>>> UefiCpuPkg/CpuExceptionHandler: Add support for NPF NAE events (MMIO) >>>> UefiCpuPkg/CpuExceptionHandler: Add support for WBINVD NAE events >>>> UefiCpuPkg/CpuExceptionHandler: Add support for RDTSC NAE events >>>> UefiCpuPkg/CpuExceptionHandler: Add support for RDPMC NAE events >>>> UefiCpuPkg/CpuExceptionHandler: Add support for INVD NAE events >>>> UefiCpuPkg/CpuExceptionHandler: Add support for VMMCALL NAE events >>>> UefiCpuPkg/CpuExceptionHandler: Add support for RDTSCP NAE events >>>> UefiCpuPkg/CpuExceptionHandler: Add support for MONITOR/MONITORX NAE >>>> events >>>> UefiCpuPkg/CpuExceptionHandler: Add support for MWAIT/MWAITX NAE >>>> events >>>> UefiCpuPkg/CpuExceptionHandler: Add support for DR7 Read/Write NAE >>>> events >>>> OvmfPkg/MemEncryptSevLib: Add an SEV-ES guest indicator function >>>> OvmfPkg: Add support to perform SEV-ES initialization >>>> OvmfPkg: Create a GHCB page for use during Sec phase >>>> OvmfPkg/PlatformPei: Reserve GHCB-related areas if S3 is supported >>>> OvmfPkg: Create GHCB pages for use during Pei and Dxe phase >>>> OvmfPkg/PlatformPei: Move early GDT into ram when SEV-ES is enabled >>>> UefiCpuPkg: Create an SEV-ES workarea PCD >>>> OvmfPkg: Reserve a page in memory for the SEV-ES usage >>>> OvmfPkg/ResetVector: Add support for a 32-bit SEV check >>>> OvmfPkg/Sec: Add #VC exception handling for Sec phase >>>> OvmfPkg/Sec: Enable cache early to speed up booting >>>> OvmfPkg/QemuFlashFvbServicesRuntimeDxe: Bypass flash detection with >>>> SEV-ES is enabled >>>> UefiCpuPkg: Add a 16-bit protected mode code segment descriptor >>>> UefiCpuPkg/MpInitLib: Add CPU MP data flag to indicate if SEV-ES is >>>> enabled >>>> UefiCpuPkg: Allow AP booting under SEV-ES >>>> OvmfPkg: Use the SEV-ES work area for the SEV-ES AP reset vector >>>> OvmfPkg: Move the GHCB allocations into reserved memory >>>> UefiCpuPkg/MpInitLib: Prepare SEV-ES guest APs for OS use >>>> >>>> MdeModulePkg/MdeModulePkg.dec | 9 + >>>> OvmfPkg/OvmfPkg.dec | 9 + >>>> UefiCpuPkg/UefiCpuPkg.dec | 17 + >>>> OvmfPkg/OvmfPkgIa32.dsc | 6 + >>>> OvmfPkg/OvmfPkgIa32X64.dsc | 6 + >>>> OvmfPkg/OvmfPkgX64.dsc | 6 + >>>> OvmfPkg/OvmfXen.dsc | 1 + >>>> UefiCpuPkg/UefiCpuPkg.dsc | 2 + >>>> UefiPayloadPkg/UefiPayloadPkgIa32.dsc | 2 + >>>> UefiPayloadPkg/UefiPayloadPkgIa32X64.dsc | 2 + >>>> OvmfPkg/OvmfPkgX64.fdf | 9 + >>>> MdeModulePkg/Core/DxeIplPeim/DxeIpl.inf | 2 + >>>> MdePkg/Library/BaseLib/BaseLib.inf | 4 + >>>> OvmfPkg/PlatformPei/PlatformPei.inf | 7 + >>>> .../FvbServicesRuntimeDxe.inf | 2 + >>>> OvmfPkg/ResetVector/ResetVector.inf | 8 + >>>> OvmfPkg/Sec/SecMain.inf | 4 + >>>> .../DxeCpuExceptionHandlerLib.inf | 5 + >>>> .../PeiCpuExceptionHandlerLib.inf | 5 + >>>> .../SecPeiCpuExceptionHandlerLib.inf | 5 + >>>> .../SmmCpuExceptionHandlerLib.inf | 5 + >>>> UefiCpuPkg/Library/MpInitLib/DxeMpInitLib.inf | 4 + >>>> UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf | 4 + >>>> UefiCpuPkg/Library/VmgExitLib/VmgExitLib.inf | 33 + >>>> .../Core/DxeIplPeim/X64/VirtualMemory.h | 12 +- >>>> MdePkg/Include/Library/BaseLib.h | 31 + >>>> MdePkg/Include/Register/Amd/Fam17Msr.h | 42 + >>>> MdePkg/Include/Register/Amd/Ghcb.h | 136 ++ >>>> OvmfPkg/Include/Library/MemEncryptSevLib.h | 12 + >>>> .../QemuFlash.h | 13 + >>>> UefiCpuPkg/CpuDxe/CpuGdt.h | 4 +- >>>> UefiCpuPkg/Include/Library/VmgExitLib.h | 117 ++ >>>> .../CpuExceptionHandlerLib/AMDSevVcCommon.h | 49 + >>>> .../CpuExceptionCommon.h | 2 + >>>> UefiCpuPkg/Library/MpInitLib/MpLib.h | 68 +- >>>> .../Core/DxeIplPeim/Ia32/DxeLoadFunc.c | 4 +- >>>> .../Core/DxeIplPeim/X64/DxeLoadFunc.c | 11 +- >>>> .../Core/DxeIplPeim/X64/VirtualMemory.c | 57 +- >>>> MdePkg/Library/BaseLib/Ia32/GccInline.c | 45 + >>>> MdePkg/Library/BaseLib/X64/GccInline.c | 47 + >>>> .../MemEncryptSevLibInternal.c | 75 +- >>>> OvmfPkg/PlatformPei/AmdSev.c | 89 + >>>> OvmfPkg/PlatformPei/MemDetect.c | 23 + >>>> .../QemuFlash.c | 23 +- >>>> .../QemuFlashDxe.c | 22 + >>>> .../QemuFlashSmm.c | 16 + >>>> OvmfPkg/Sec/SecMain.c | 188 +- >>>> UefiCpuPkg/CpuDxe/CpuGdt.c | 8 +- >>>> .../CpuExceptionHandlerLib/AMDSevVcHandler.c | 40 + >>>> .../CpuExceptionCommon.c | 2 +- >>>> .../Ia32/ArchAMDSevVcHandler.c | 38 + >>>> .../PeiDxeSmmCpuException.c | 16 + >>>> .../SecPeiCpuException.c | 16 + >>>> .../X64/ArchAMDSevVcHandler.c | 1699 +++++++++++++++++ >>>> UefiCpuPkg/Library/MpInitLib/DxeMpLib.c | 113 +- >>>> UefiCpuPkg/Library/MpInitLib/MpLib.c | 265 ++- >>>> UefiCpuPkg/Library/MpInitLib/PeiMpLib.c | 19 + >>>> UefiCpuPkg/Library/VmgExitLib/VmgExitLib.c | 293 +++ >>>> UefiCpuPkg/PiSmmCpuDxeSmm/X64/SmmFuncsArch.c | 2 +- >>>> MdeModulePkg/MdeModulePkg.uni | 8 + >>>> MdePkg/Library/BaseLib/Ia32/VmgExit.nasm | 37 + >>>> MdePkg/Library/BaseLib/Ia32/XGetBv.nasm | 31 + >>>> MdePkg/Library/BaseLib/X64/VmgExit.nasm | 32 + >>>> MdePkg/Library/BaseLib/X64/XGetBv.nasm | 34 + >>>> OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm | 100 + >>>> OvmfPkg/ResetVector/Ia32/PageTables64.asm | 350 +++- >>>> OvmfPkg/ResetVector/ResetVector.nasmb | 20 + >>>> .../X64/ExceptionHandlerAsm.nasm | 17 + >>>> UefiCpuPkg/Library/MpInitLib/Ia32/MpEqu.inc | 2 +- >>>> .../Library/MpInitLib/Ia32/MpFuncs.nasm | 15 + >>>> UefiCpuPkg/Library/MpInitLib/X64/MpEqu.inc | 4 +- >>>> UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm | 370 +++- >>>> UefiCpuPkg/Library/VmgExitLib/VmgExitLib.uni | 15 + >>>> .../ResetVector/Vtf0/Ia16/Real16ToFlat32.asm | 9 + >>>> UefiCpuPkg/UefiCpuPkg.uni | 11 + >>>> 75 files changed, 4707 insertions(+), 102 deletions(-) >>>> create mode 100644 UefiCpuPkg/Library/VmgExitLib/VmgExitLib.inf >>>> create mode 100644 MdePkg/Include/Register/Amd/Ghcb.h >>>> create mode 100644 UefiCpuPkg/Include/Library/VmgExitLib.h >>>> create mode 100644 UefiCpuPkg/Library/CpuExceptionHandlerLib/AMDSevVcCommon.h >>>> create mode 100644 UefiCpuPkg/Library/CpuExceptionHandlerLib/AMDSevVcHandler.c >>>> create mode 100644 UefiCpuPkg/Library/CpuExceptionHandlerLib/Ia32/ArchAMDSevVcHandler.c >>>> create mode 100644 UefiCpuPkg/Library/CpuExceptionHandlerLib/X64/ArchAMDSevVcHandler.c >>>> create mode 100644 UefiCpuPkg/Library/VmgExitLib/VmgExitLib.c >>>> create mode 100644 MdePkg/Library/BaseLib/Ia32/VmgExit.nasm >>>> create mode 100644 MdePkg/Library/BaseLib/Ia32/XGetBv.nasm >>>> create mode 100644 MdePkg/Library/BaseLib/X64/VmgExit.nasm >>>> create mode 100644 MdePkg/Library/BaseLib/X64/XGetBv.nasm >>>> create mode 100644 OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm >>>> create mode 100644 UefiCpuPkg/Library/VmgExitLib/VmgExitLib.uni >>>> > >