Re: [PATCH v7 00/43] SEV-ES guest support

["Lendacky, Thomas" <thomas.lendacky@amd.com>]

On 5/9/20 1:44 AM, Ni, Ray wrote:
> Tom,

Hi Ray,

> I have a bit concern on your change that directly modifies CpuExceptionHandlerLib to handle
> exception #29. Today's CpuExceptionHandlerLib simplify dumps the exception context for
> every exception. Any component which wants to do specific handling of certain exceptions
> should call RegisterCpuInterruptHandler(). Such as code in CpuDxe driver:
> 
>    if (HEAP_GUARD_NONSTOP_MODE || NULL_DETECTION_NONSTOP_MODE) {
>      RegisterCpuInterruptHandler (EXCEPT_IA32_DEBUG, DebugExceptionHandler);
>      RegisterCpuInterruptHandler (EXCEPT_IA32_PAGE_FAULT, PageFaultExceptionHandler);
>    }
> 
> Is it possible for your feature to follow the same pattern?

There are two problems:

The first is that RegisterCpuInterruptHandler() is not implemented for 
both the SEC and PEI phases, so it is not currently possible to register a 
handler that early.

The second is that I need to be able to propagate an exception request 
from the hypervisor. With the current implementation there doesn't appear 
to be an easy way to perform this propagation.

If there's a way to accomplish both of the above I wouldn't be opposed to 
using RegisterCpuInterruptHandler() as long as there are no #VCs that can 
occur between initializing exception handling and and registering the #VC 
handler.

Thanks,
Tom

> 
> Thanks,
> Ray
> 
>> -----Original Message-----
>> From: Tom Lendacky <thomas.lendacky@amd.com>
>> Sent: Saturday, May 9, 2020 3:16 AM
>> To: devel@edk2.groups.io
>> Cc: Justen, Jordan L <jordan.l.justen@intel.com>; Laszlo Ersek <lersek@redhat.com>; Ard Biesheuvel
>> <ard.biesheuvel@linaro.org>; Kinney, Michael D <michael.d.kinney@intel.com>; Gao, Liming <liming.gao@intel.com>; Dong,
>> Eric <eric.dong@intel.com>; Ni, Ray <ray.ni@intel.com>; Brijesh Singh <brijesh.singh@amd.com>; You, Benjamin
>> <benjamin.you@intel.com>; Bi, Dandan <dandan.bi@intel.com>; Dong, Guo <guo.dong@intel.com>; Wu, Hao A
>> <hao.a.wu@intel.com>; Wang, Jian J <jian.j.wang@intel.com>; Ma, Maurice <maurice.ma@intel.com>
>> Subject: Re: [PATCH v7 00/43] SEV-ES guest support
>>
>> I was able to use the pull request method that Laszlo documented and fixed
>> up all of the issues identified by the VS compiler.
>>
>> An additional change I'm planning to make for the next version (v8) of the
>> patches is to create a NULL library instance of the VmgExitLib that will
>> also include the #VC handler function. This will reduce the amount of code
>> associated with this feature for platforms that don't use/support SEV-ES.
>>
>> Laszlo, this will mean that I will introduce a version of the VmgExitLib
>> under OvmfPkg that will provide the majority of the functionality that is
>> present today in UefiCpuPkg. In essence, the functionality in v7 patches 8
>> and 11 - 25 will now live under OvmfPkg instead of UefiCpuPkg. I think
>> this is the better way to do this. Let me know if you have any concerns.
>>
>> Thanks,
>> Tom
>>
>> On 4/22/20 12:41 PM, Tom Lendacky wrote:
>>> This patch series provides support for running EDK2/OVMF under SEV-ES.
>>>
>>> Secure Encrypted Virtualization - Encrypted State (SEV-ES) expands on the
>>> SEV support to protect the guest register state from the hypervisor. See
>>> "AMD64 Architecture Programmer's Manual Volume 2: System Programming",
>>> section "15.35 Encrypted State (SEV-ES)" [1].
>>>
>>> In order to allow a hypervisor to perform functions on behalf of a guest,
>>> there is architectural support for notifying a guest's operating system
>>> when certain types of VMEXITs are about to occur. This allows the guest to
>>> selectively share information with the hypervisor to satisfy the requested
>>> function. The notification is performed using a new exception, the VMM
>>> Communication exception (#VC). The information is shared through the
>>> Guest-Hypervisor Communication Block (GHCB) using the VMGEXIT instruction.
>>> The GHCB format and the protocol for using it is documented in "SEV-ES
>>> Guest-Hypervisor Communication Block Standardization" [2].
>>>
>>> The main areas of the EDK2 code that are updated to support SEV-ES are
>>> around the exception handling support and the AP boot support.
>>>
>>> Exception support is required starting in Sec, continuing through Pei
>>> and into Dxe in order to handle #VC exceptions that are generated.  Each
>>> AP requires it's own GHCB page as well as a page to hold values specific
>>> to that AP.
>>>
>>> AP booting poses some interesting challenges. The INIT-SIPI-SIPI sequence
>>> is typically used to boot the APs. However, the hypervisor is not allowed
>>> to update the guest registers. The GHCB document [2] talks about how SMP
>>> booting under SEV-ES is performed.
>>>
>>> Since the GHCB page must be a shared (unencrypted) page, the processor
>>> must be running in long mode in order for the guest and hypervisor to
>>> communicate with each other. As a result, SEV-ES is only supported under
>>> the X64 architecture.
>>>
>>> [1] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.amd.com%2Fsystem%2Ffiles%2FTechDocs%2F24593.pdf&amp;data=02%7C01%7Cthomas.lendacky%40amd.com%7Cf5d7875dfcf54e45c42208d7f3e4676b%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637246036118033165&amp;sdata=H74fQl1n2sXzCMSoGm1tGOKc5epMtVkGJFCidwLMl5c%3D&amp;reserved=0
>>> [2] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdeveloper.amd.com%2Fwp-content%2Fresources%2F56421.pdf&amp;data=02%7C01%7Cthomas.lendacky%40amd.com%7Cf5d7875dfcf54e45c42208d7f3e4676b%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637246036118033165&amp;sdata=EwW9575nJMaWxizo2XrLHjrbUMJIB0WFTDLjwy%2BM%2F4k%3D&amp;reserved=0
>>>
>>> ---
>>>
>>> These patches are based on commit:
>>> be7295b36405 (".python/SpellCheck: Increase SpellCheck plugin max failures")
>>>
>>> Proper execution of SEV-ES relies on Bugzilla 2340 being fixed.
>>>
>>> A version of the tree (with an extra patch to workaround Bugzilla 2340) can
>>> be found at:
>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAMDESE%2Fovmf%2Ftree%2Fsev-es-v14&amp;data=02%7C01%7Cthomas.lendacky%40amd.com%7Cf5d7875dfcf54e45c42208d7f3e4676b%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637246036118033165&amp;sdata=U8fIzb%2F4A8WBaiVbScxUuGDw22kyxxnRP5olSyTedvE%3D&amp;reserved=0
>>>
>>> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
>>> Cc: Benjamin You <benjamin.you@intel.com>
>>> Cc: Dandan Bi <dandan.bi@intel.com>
>>> Cc: Eric Dong <eric.dong@intel.com>
>>> Cc: Guo Dong <guo.dong@intel.com>
>>> Cc: Hao A Wu <hao.a.wu@intel.com>
>>> Cc: Jian J Wang <jian.j.wang@intel.com>
>>> Cc: Jordan Justen <jordan.l.justen@intel.com>
>>> Cc: Laszlo Ersek <lersek@redhat.com>
>>> Cc: Liming Gao <liming.gao@intel.com>
>>> Cc: Maurice Ma <maurice.ma@intel.com>
>>> Cc: Michael D Kinney <michael.d.kinney@intel.com>
>>> Cc: Ray Ni <ray.ni@intel.com>
>>>
>>> Changes since v6:
>>> - Add function comments to all functions, including local functions
>>> - Add function parameter direction to all functions (in/out)
>>> - Add support for MMIO MOVZX/MOVSX instructions
>>> - Ensure the per-CPU variable page remains encrypted
>>> - Coding-style fixes as identified by Ecc
>>>
>>> Changes since v5:
>>> - Remove extraneous VmgExitLib usage
>>> - Miscellaneous changes to address feedback (coding style, etc.)
>>>
>>> Changes since v4:
>>> - Move the SEV-ES protocol negotiation out of the SEC exception handler
>>>     and into the SecMain.c file. As a result:
>>>     - Move the SecGhcb related PCDs out of UefiCpuPkg and into OvmfPkg
>>>     - Combine SecAMDSevVcHandler.c and PeiDxeAMDSevVcHandler.c into a
>>>       single AMDSevVcHandler.c
>>> - Consolidate VmgExitLib usage into common LibraryClasses sections
>>> - Add documentation comments to the VmgExitLib functions
>>>
>>> Changes since v3:
>>> - Remove the need for the MP library finalization routine. The AP
>>>     jump table address will be held by the hypervisor rather than
>>>     communicated via the GHCB MSR. This removes some fragility around
>>>     the UEFI to OS transition.
>>> - Rename the SEV-ES RIP reset area to SEV-ES workarea and use it to
>>>     communicate the SEV-ES status, so that SEC CPU exception handling is
>>>     only established for an SEV-ES guest.
>>> - Fix SMM build breakageAdd around QemuFlashPtrWrite().
>>> - Fix SMM build breakage by adding VC exception support the SMM CPU
>>>     exception handling.
>>> - Add memory fencing around the invocation of AsmVmgExit().
>>> - Clarify comments around the SEV-ES AP reset RIP values and usage.
>>> - Move some PCD definitions from MdeModulePkg to UefiCpuPkg.
>>> - Remove the 16-bit code selector definition from MdeModulePkg
>>>
>>> Changes since v2:
>>> - Added a way to locate the SEV-ES fixed AP RIP address for starting
>>>     AP's to avoid updating the actual flash image (build time location
>>>     that is identified with a GUID value).
>>> - Create a VmgExit library to replace static inline functions.
>>> - Move some PCDs to the appropriate packages
>>> - Add support for writing to QEMU flash under SEV-ES
>>> - Add additional MMIO opcode support
>>> - Cleaned up the GHCB MSR CPUID protocol support
>>>
>>> Changes since v1:
>>> - Patches reworked to be more specific to the component/area being updated
>>>     and order of definition/usage
>>> - Created a library for VMGEXIT-related functions to replace use of inline
>>>     functions
>>> - Allocation method for GDT changed from AllocatePool to AllocatePages
>>> - Early caching only enabled for SEV-ES guests
>>> - Ensure AP loop mode set to halt loop mode for SEV-ES guests
>>> - Reserved SEC GHCB-related memory areas when S3 is enabled
>>>
>>> Tom Lendacky (43):
>>>     MdeModulePkg: Create PCDs to be used in support of SEV-ES
>>>     UefiCpuPkg: Create PCD to be used in support of SEV-ES
>>>     MdePkg: Add the MSR definition for the GHCB register
>>>     MdePkg: Add a structure definition for the GHCB
>>>     MdeModulePkg/DxeIplPeim: Support GHCB pages when creating page tables
>>>     MdePkg/BaseLib: Add support for the XGETBV instruction
>>>     MdePkg/BaseLib: Add support for the VMGEXIT instruction
>>>     UefiCpuPkg: Implement library support for VMGEXIT
>>>     OvmfPkg: Prepare OvmfPkg to use the VmgExitLib library
>>>     UefiPayloadPkg: Prepare UefiPayloadPkg to use the VmgExitLib library
>>>     UefiCpuPkg/CpuExceptionHandler: Add base support for the #VC exception
>>>     UefiCpuPkg/CpuExceptionHandler: Add support for IOIO_PROT NAE events
>>>     UefiCpuPkg/CpuExceptionHandler: Support string IO for IOIO_PROT NAE
>>>       events
>>>     UefiCpuPkg/CpuExceptionHandler: Add support for CPUID NAE events
>>>     UefiCpuPkg/CpuExceptionHandler: Add support for MSR_PROT NAE events
>>>     UefiCpuPkg/CpuExceptionHandler: Add support for NPF NAE events (MMIO)
>>>     UefiCpuPkg/CpuExceptionHandler: Add support for WBINVD NAE events
>>>     UefiCpuPkg/CpuExceptionHandler: Add support for RDTSC NAE events
>>>     UefiCpuPkg/CpuExceptionHandler: Add support for RDPMC NAE events
>>>     UefiCpuPkg/CpuExceptionHandler: Add support for INVD NAE events
>>>     UefiCpuPkg/CpuExceptionHandler: Add support for VMMCALL NAE events
>>>     UefiCpuPkg/CpuExceptionHandler: Add support for RDTSCP NAE events
>>>     UefiCpuPkg/CpuExceptionHandler: Add support for MONITOR/MONITORX NAE
>>>       events
>>>     UefiCpuPkg/CpuExceptionHandler: Add support for MWAIT/MWAITX NAE
>>>       events
>>>     UefiCpuPkg/CpuExceptionHandler: Add support for DR7 Read/Write NAE
>>>       events
>>>     OvmfPkg/MemEncryptSevLib: Add an SEV-ES guest indicator function
>>>     OvmfPkg: Add support to perform SEV-ES initialization
>>>     OvmfPkg: Create a GHCB page for use during Sec phase
>>>     OvmfPkg/PlatformPei: Reserve GHCB-related areas if S3 is supported
>>>     OvmfPkg: Create GHCB pages for use during Pei and Dxe phase
>>>     OvmfPkg/PlatformPei: Move early GDT into ram when SEV-ES is enabled
>>>     UefiCpuPkg: Create an SEV-ES workarea PCD
>>>     OvmfPkg: Reserve a page in memory for the SEV-ES usage
>>>     OvmfPkg/ResetVector: Add support for a 32-bit SEV check
>>>     OvmfPkg/Sec: Add #VC exception handling for Sec phase
>>>     OvmfPkg/Sec: Enable cache early to speed up booting
>>>     OvmfPkg/QemuFlashFvbServicesRuntimeDxe: Bypass flash detection with
>>>       SEV-ES is enabled
>>>     UefiCpuPkg: Add a 16-bit protected mode code segment descriptor
>>>     UefiCpuPkg/MpInitLib: Add CPU MP data flag to indicate if SEV-ES is
>>>       enabled
>>>     UefiCpuPkg: Allow AP booting under SEV-ES
>>>     OvmfPkg: Use the SEV-ES work area for the SEV-ES AP reset vector
>>>     OvmfPkg: Move the GHCB allocations into reserved memory
>>>     UefiCpuPkg/MpInitLib: Prepare SEV-ES guest APs for OS use
>>>
>>>    MdeModulePkg/MdeModulePkg.dec                 |    9 +
>>>    OvmfPkg/OvmfPkg.dec                           |    9 +
>>>    UefiCpuPkg/UefiCpuPkg.dec                     |   17 +
>>>    OvmfPkg/OvmfPkgIa32.dsc                       |    6 +
>>>    OvmfPkg/OvmfPkgIa32X64.dsc                    |    6 +
>>>    OvmfPkg/OvmfPkgX64.dsc                        |    6 +
>>>    OvmfPkg/OvmfXen.dsc                           |    1 +
>>>    UefiCpuPkg/UefiCpuPkg.dsc                     |    2 +
>>>    UefiPayloadPkg/UefiPayloadPkgIa32.dsc         |    2 +
>>>    UefiPayloadPkg/UefiPayloadPkgIa32X64.dsc      |    2 +
>>>    OvmfPkg/OvmfPkgX64.fdf                        |    9 +
>>>    MdeModulePkg/Core/DxeIplPeim/DxeIpl.inf       |    2 +
>>>    MdePkg/Library/BaseLib/BaseLib.inf            |    4 +
>>>    OvmfPkg/PlatformPei/PlatformPei.inf           |    7 +
>>>    .../FvbServicesRuntimeDxe.inf                 |    2 +
>>>    OvmfPkg/ResetVector/ResetVector.inf           |    8 +
>>>    OvmfPkg/Sec/SecMain.inf                       |    4 +
>>>    .../DxeCpuExceptionHandlerLib.inf             |    5 +
>>>    .../PeiCpuExceptionHandlerLib.inf             |    5 +
>>>    .../SecPeiCpuExceptionHandlerLib.inf          |    5 +
>>>    .../SmmCpuExceptionHandlerLib.inf             |    5 +
>>>    UefiCpuPkg/Library/MpInitLib/DxeMpInitLib.inf |    4 +
>>>    UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf |    4 +
>>>    UefiCpuPkg/Library/VmgExitLib/VmgExitLib.inf  |   33 +
>>>    .../Core/DxeIplPeim/X64/VirtualMemory.h       |   12 +-
>>>    MdePkg/Include/Library/BaseLib.h              |   31 +
>>>    MdePkg/Include/Register/Amd/Fam17Msr.h        |   42 +
>>>    MdePkg/Include/Register/Amd/Ghcb.h            |  136 ++
>>>    OvmfPkg/Include/Library/MemEncryptSevLib.h    |   12 +
>>>    .../QemuFlash.h                               |   13 +
>>>    UefiCpuPkg/CpuDxe/CpuGdt.h                    |    4 +-
>>>    UefiCpuPkg/Include/Library/VmgExitLib.h       |  117 ++
>>>    .../CpuExceptionHandlerLib/AMDSevVcCommon.h   |   49 +
>>>    .../CpuExceptionCommon.h                      |    2 +
>>>    UefiCpuPkg/Library/MpInitLib/MpLib.h          |   68 +-
>>>    .../Core/DxeIplPeim/Ia32/DxeLoadFunc.c        |    4 +-
>>>    .../Core/DxeIplPeim/X64/DxeLoadFunc.c         |   11 +-
>>>    .../Core/DxeIplPeim/X64/VirtualMemory.c       |   57 +-
>>>    MdePkg/Library/BaseLib/Ia32/GccInline.c       |   45 +
>>>    MdePkg/Library/BaseLib/X64/GccInline.c        |   47 +
>>>    .../MemEncryptSevLibInternal.c                |   75 +-
>>>    OvmfPkg/PlatformPei/AmdSev.c                  |   89 +
>>>    OvmfPkg/PlatformPei/MemDetect.c               |   23 +
>>>    .../QemuFlash.c                               |   23 +-
>>>    .../QemuFlashDxe.c                            |   22 +
>>>    .../QemuFlashSmm.c                            |   16 +
>>>    OvmfPkg/Sec/SecMain.c                         |  188 +-
>>>    UefiCpuPkg/CpuDxe/CpuGdt.c                    |    8 +-
>>>    .../CpuExceptionHandlerLib/AMDSevVcHandler.c  |   40 +
>>>    .../CpuExceptionCommon.c                      |    2 +-
>>>    .../Ia32/ArchAMDSevVcHandler.c                |   38 +
>>>    .../PeiDxeSmmCpuException.c                   |   16 +
>>>    .../SecPeiCpuException.c                      |   16 +
>>>    .../X64/ArchAMDSevVcHandler.c                 | 1699 +++++++++++++++++
>>>    UefiCpuPkg/Library/MpInitLib/DxeMpLib.c       |  113 +-
>>>    UefiCpuPkg/Library/MpInitLib/MpLib.c          |  265 ++-
>>>    UefiCpuPkg/Library/MpInitLib/PeiMpLib.c       |   19 +
>>>    UefiCpuPkg/Library/VmgExitLib/VmgExitLib.c    |  293 +++
>>>    UefiCpuPkg/PiSmmCpuDxeSmm/X64/SmmFuncsArch.c  |    2 +-
>>>    MdeModulePkg/MdeModulePkg.uni                 |    8 +
>>>    MdePkg/Library/BaseLib/Ia32/VmgExit.nasm      |   37 +
>>>    MdePkg/Library/BaseLib/Ia32/XGetBv.nasm       |   31 +
>>>    MdePkg/Library/BaseLib/X64/VmgExit.nasm       |   32 +
>>>    MdePkg/Library/BaseLib/X64/XGetBv.nasm        |   34 +
>>>    OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm  |  100 +
>>>    OvmfPkg/ResetVector/Ia32/PageTables64.asm     |  350 +++-
>>>    OvmfPkg/ResetVector/ResetVector.nasmb         |   20 +
>>>    .../X64/ExceptionHandlerAsm.nasm              |   17 +
>>>    UefiCpuPkg/Library/MpInitLib/Ia32/MpEqu.inc   |    2 +-
>>>    .../Library/MpInitLib/Ia32/MpFuncs.nasm       |   15 +
>>>    UefiCpuPkg/Library/MpInitLib/X64/MpEqu.inc    |    4 +-
>>>    UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm |  370 +++-
>>>    UefiCpuPkg/Library/VmgExitLib/VmgExitLib.uni  |   15 +
>>>    .../ResetVector/Vtf0/Ia16/Real16ToFlat32.asm  |    9 +
>>>    UefiCpuPkg/UefiCpuPkg.uni                     |   11 +
>>>    75 files changed, 4707 insertions(+), 102 deletions(-)
>>>    create mode 100644 UefiCpuPkg/Library/VmgExitLib/VmgExitLib.inf
>>>    create mode 100644 MdePkg/Include/Register/Amd/Ghcb.h
>>>    create mode 100644 UefiCpuPkg/Include/Library/VmgExitLib.h
>>>    create mode 100644 UefiCpuPkg/Library/CpuExceptionHandlerLib/AMDSevVcCommon.h
>>>    create mode 100644 UefiCpuPkg/Library/CpuExceptionHandlerLib/AMDSevVcHandler.c
>>>    create mode 100644 UefiCpuPkg/Library/CpuExceptionHandlerLib/Ia32/ArchAMDSevVcHandler.c
>>>    create mode 100644 UefiCpuPkg/Library/CpuExceptionHandlerLib/X64/ArchAMDSevVcHandler.c
>>>    create mode 100644 UefiCpuPkg/Library/VmgExitLib/VmgExitLib.c
>>>    create mode 100644 MdePkg/Library/BaseLib/Ia32/VmgExit.nasm
>>>    create mode 100644 MdePkg/Library/BaseLib/Ia32/XGetBv.nasm
>>>    create mode 100644 MdePkg/Library/BaseLib/X64/VmgExit.nasm
>>>    create mode 100644 MdePkg/Library/BaseLib/X64/XGetBv.nasm
>>>    create mode 100644 OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm
>>>    create mode 100644 UefiCpuPkg/Library/VmgExitLib/VmgExitLib.uni
>>>