Re: [RFC PATCH v2 00/44] SEV-ES guest support

["Lendacky, Thomas" <thomas.lendacky@amd.com>]

On 9/19/19 2:52 PM, Lendacky, Thomas wrote:
> From: Tom Lendacky <thomas.lendacky@amd.com>
> 
> This patch series provides support for running EDK2/OVMF under SEV-ES.

I wanted to provide the full series so that everyone can see what is
involved with the support.

Providing multiple, smaller series of patches might be easier to review.
If that is desired, any suggestions/recommendations on how to best
approach that would be great.

Thanks,
Tom

> 
> Secure Encrypted Virtualization - Encrypted State (SEV-ES) expands on the
> SEV support to protect the guest register state from the hypervisor. See
> "AMD64 Architecture Programmer's Manual Volume 2: System Programming",
> section "15.35 Encrypted State (SEV-ES)" [1].
> 
> In order to allow a hypervisor to perform functions on behalf of a guest,
> there is architectural support for notifying a guest's operating system
> when certain types of VMEXITs are about to occur. This allows the guest to
> selectively share information with the hypervisor to satisfy the requested
> function. The notification is performed using a new exception, the VMM
> Communication exception (#VC). The information is shared through the
> Guest-Hypervisor Communication Block (GHCB) using the VMGEXIT instruction.
> The GHCB format and the protocol for using it is documented in "SEV-ES
> Guest-Hypervisor Communication Block Standardization" [2].
> 
> The main areas of the EDK2 code that are updated to support SEV-ES are
> around the exception handling support and the AP boot support.
> 
> Exception support is required starting in Sec, continuing through Pei
> and into Dxe in order to handle #VC exceptions that are generated.  Each
> AP requires it's own GHCB page as well as a page to hold values specific
> to that AP.
> 
> AP booting poses some interesting challenges. The INIT-SIPI-SIPI sequence
> is typically used to boot the APs. However, the hypervisor is not allowed
> to update the guest registers. The GHCB document [2] talks about how SMP
> booting under SEV-ES is performed.
> 
> Since the GHCB page must be a shared (unencrypted) page, the processor
> must be running in long mode in order for the guest and hypervisor to
> communicate with each other. As a result, SEV-ES is only supported under
> the X64 architecture.
> 
> [1] https://www.amd.com/system/files/TechDocs/24593.pdf
> [2] https://developer.amd.com/wp-content/resources/56421.pdf
> 
> ---
> 
> These patches are based on commit:
> ada905ab5c0e ("MdeModulePkg DxeCore: Fix for missing Memory Attributes Table (MAT) update")
> 
> A version of the tree can be found at:
> https://github.com/AMDESE/ovmf/tree/sev-es-v7
> 
> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> Cc: Benjamin You <benjamin.you@intel.com>
> Cc: Dandan Bi <dandan.bi@intel.com>
> Cc: Eric Dong <eric.dong@intel.com>
> Cc: Guo Dong <guo.dong@intel.com>
> Cc: Hao A Wu <hao.a.wu@intel.com>
> Cc: Jian J Wang <jian.j.wang@intel.com>
> Cc: Jordan Justen <jordan.l.justen@intel.com>
> Cc: Laszlo Ersek <lersek@redhat.com>
> Cc: Liming Gao <liming.gao@intel.com>
> Cc: Maurice Ma <maurice.ma@intel.com>
> Cc: Michael D Kinney <michael.d.kinney@intel.com>
> Cc: Ray Ni <ray.ni@intel.com>
> 
> Changes since v1:
> - Patches reworked to be more specific to the component/area being updated
>    and order of definition/usage
> - Created a library for VMGEXIT-related functions to replace use of inline
>    functions
> - Allocation method for GDT changed from AllocatePool to AllocatePages
> - Early caching only enabled for SEV-ES guests
> - Ensure AP loop mode set to halt loop mode for SEV-ES guests
> - Reserved SEC GHCB-related memory areas when S3 is enabled
> 
> Tom Lendacky (44):
>    MdePkg: Create PCDs to be used in support of SEV-ES
>    OvmfPkg/MemEncryptSevLib: Add an SEV-ES guest indicator function
>    OvmfPkg: Add support to perform SEV-ES initialization
>    OvmfPkg/ResetVector: Add support for a 32-bit SEV check
>    MdePkg: Add the MSR definition for the GHCB register
>    OvmfPkg: Create a GHCB page for use during Sec phase
>    OvmfPkg/PlatformPei: Reserve GHCB-related areas if S3 is supported
>    OvmfPkg: Create GHCB pages for use during Pei and Dxe phase
>    MdeModulePkg/DxeIplPeim: Support GHCB pages when creating page tables
>    OvmfPkg: A per-CPU variable area for #VC usage
>    OvmfPkg/PlatformPei: Move early GDT into ram when SEV-ES is enabled
>    MdePkg: Add a structure definition for the GHCB
>    MdePkg/BaseLib: Add support for the VMGEXIT instruction
>    UefiCpuPkg: Implement library support for VMGEXIT
>    UefiCpuPkg/CpuExceptionHandler: Add base support for the #VC exception
>    OvmfPkg/MemEncryptSevLib: Make MemEncryptSevLib available during SEC
>    UefiCpuPkg/CpuExceptionHandler: Add #VC exception handling for Sec
>      phase
>    OvmfPkg/Sec: Enable cache early to speed up booting
>    UefiCpuPkg/CpuExceptionHandler: Add support for IOIO_PROT NAE events
>    UefiCpuPkg/CpuExceptionHandler: Support string IO for IOIO_PROT NAE
>      events
>    MdePkg: Add support for the XGETBV instruction
>    UefiCpuPkg/CpuExceptionHandler: Add support for CPUID NAE events
>    UefiCpuPkg/CpuExceptionHandler: Add support for MSR_PROT NAE events
>    UefiCpuPkg/CpuExceptionHandler: Add support for NPF NAE events (MMIO)
>    UefiCpuPkg/CpuExceptionHandler: Add support for WBINVD NAE events
>    UefiCpuPkg/CpuExceptionHandler: Add support for RDTSC NAE events
>    UefiCpuPkg/CpuExceptionHandler: Add support for RDPMC NAE events
>    UefiCpuPkg/CpuExceptionHandler: Add support for INVD NAE events
>    UefiCpuPkg/CpuExceptionHandler: Add support for VMMCALL NAE events
>    UefiCpuPkg/CpuExceptionHandler: Add support for RDTSCP NAE events
>    UefiCpuPkg/CpuExceptionHandler: Add support for MONITOR/MONITORX NAE
>      events
>    UefiCpuPkg/CpuExceptionHandler: Add support for MWAIT/MWAITX NAE
>      events
>    UefiCpuPkg/CpuExceptionHandler: Add support for DR7 Read/Write NAE
>      events
>    UefiCpuPkg/MpInitLib: Update CPU MP data with a flag to indicate if
>      SEV-ES is active
>    MdeModulePkg: Reserve a 16-bit protected mode code segment descriptor
>    UefiCpuPkg: Add a 16-bit protected mode code segment descriptor
>    OvmfPkg: Add support for SEV-ES AP reset vector re-directing
>    UefiCpuPkg: Allow AP booting under SEV-ES
>    OvmfPkg: Move the GHCB allocations into reserved memory
>    MdePkg: Add a finalization function to the CPU protocol
>    UefiCpuPkg/MpInitLib: Add MP finalization interface to MpInitLib
>    UefiCpuPkg/MpInitLib: Prepare SEV-ES guest APs for OS use
>    UefiCpuPkg/CpuDxe: Provide an DXE MP finalization routine to support
>      SEV-ES
>    MdeModulePkg/DxeCore: Perform the CPU protocol finalization support
> 
>   MdeModulePkg/MdeModulePkg.dec                 |   15 +
>   OvmfPkg/OvmfPkg.dec                           |   10 +
>   UefiCpuPkg/UefiCpuPkg.dec                     |    3 +
>   OvmfPkg/OvmfPkgIa32.dsc                       |   10 +
>   OvmfPkg/OvmfPkgIa32X64.dsc                    |   10 +
>   OvmfPkg/OvmfPkgX64.dsc                        |   10 +
>   UefiCpuPkg/UefiCpuPkg.dsc                     |    5 +
>   UefiPayloadPkg/UefiPayloadPkgIa32.dsc         |    2 +
>   UefiPayloadPkg/UefiPayloadPkgIa32X64.dsc      |    2 +
>   OvmfPkg/OvmfPkgX64.fdf                        |    6 +
>   MdeModulePkg/Core/DxeIplPeim/DxeIpl.inf       |    2 +
>   MdePkg/Library/BaseLib/BaseLib.inf            |    4 +
>   .../BaseMemEncryptSevLib.inf                  |    2 +-
>   OvmfPkg/PlatformPei/PlatformPei.inf           |    7 +
>   OvmfPkg/ResetVector/ResetVector.inf           |    6 +
>   OvmfPkg/Sec/SecMain.inf                       |    2 +
>   .../DxeCpuExceptionHandlerLib.inf             |    5 +
>   .../PeiCpuExceptionHandlerLib.inf             |    5 +
>   .../SecPeiCpuExceptionHandlerLib.inf          |    5 +
>   UefiCpuPkg/Library/MpInitLib/DxeMpInitLib.inf |    3 +
>   UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf |    3 +
>   UefiCpuPkg/Library/VmgExitLib/VmgExitLib.inf  |   33 +
>   .../Core/DxeIplPeim/X64/VirtualMemory.h       |   12 +-
>   MdePkg/Include/Library/BaseLib.h              |   30 +
>   MdePkg/Include/Protocol/Cpu.h                 |   18 +
>   MdePkg/Include/Register/Amd/Fam17Msr.h        |   28 +
>   MdePkg/Include/Register/Amd/Ghcb.h            |  105 ++
>   OvmfPkg/Include/Library/MemEncryptSevLib.h    |   12 +
>   UefiCpuPkg/CpuDxe/CpuDxe.h                    |   12 +
>   UefiCpuPkg/CpuDxe/CpuGdt.h                    |    4 +-
>   UefiCpuPkg/Include/Library/MpInitLib.h        |   14 +
>   UefiCpuPkg/Include/Library/VmgExitLib.h       |   74 +
>   .../CpuExceptionHandlerLib/AMDSevVcCommon.h   |   19 +
>   .../CpuExceptionCommon.h                      |    2 +
>   UefiCpuPkg/Library/MpInitLib/MpLib.h          |   77 +-
>   MdeModulePkg/Core/Dxe/DxeMain/DxeMain.c       |    5 +
>   .../Core/DxeIplPeim/Ia32/DxeLoadFunc.c        |    6 +-
>   .../Core/DxeIplPeim/X64/DxeLoadFunc.c         |   11 +-
>   .../Core/DxeIplPeim/X64/VirtualMemory.c       |   49 +-
>   MdePkg/Library/BaseLib/Ia32/GccInline.c       |   45 +
>   MdePkg/Library/BaseLib/X64/GccInline.c        |   47 +
>   .../MemEncryptSevLibInternal.c                |   77 +-
>   OvmfPkg/PlatformPei/AmdSev.c                  |   78 ++
>   OvmfPkg/PlatformPei/MemDetect.c               |   23 +
>   OvmfPkg/Sec/SecMain.c                         |   39 +-
>   UefiCpuPkg/CpuDxe/CpuDxe.c                    |   21 +-
>   UefiCpuPkg/CpuDxe/CpuGdt.c                    |    8 +-
>   .../CpuExceptionCommon.c                      |    2 +-
>   .../Ia32/AMDSevVcCommon.c                     |   13 +
>   .../PeiDxeAMDSevVcHandler.c                   |   22 +
>   .../PeiDxeSmmCpuException.c                   |   16 +
>   .../SecAMDSevVcHandler.c                      |   45 +
>   .../SecPeiCpuException.c                      |   16 +
>   .../X64/AMDSevVcCommon.c                      | 1200 +++++++++++++++++
>   UefiCpuPkg/Library/MpInitLib/DxeMpLib.c       |  136 +-
>   UefiCpuPkg/Library/MpInitLib/MpLib.c          |  259 +++-
>   UefiCpuPkg/Library/MpInitLib/PeiMpLib.c       |   35 +
>   UefiCpuPkg/Library/VmgExitLib/VmgExitLib.c    |   75 ++
>   UefiCpuPkg/PiSmmCpuDxeSmm/X64/SmmFuncsArch.c  |    2 +-
>   MdePkg/Library/BaseLib/Ia32/VmgExit.nasm      |   31 +
>   MdePkg/Library/BaseLib/Ia32/XGetBv.nasm       |   31 +
>   MdePkg/Library/BaseLib/X64/VmgExit.nasm       |   32 +
>   MdePkg/Library/BaseLib/X64/XGetBv.nasm        |   34 +
>   OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm  |   80 ++
>   OvmfPkg/ResetVector/Ia32/PageTables64.asm     |  256 +++-
>   OvmfPkg/ResetVector/ResetVector.nasmb         |   13 +
>   .../X64/ExceptionHandlerAsm.nasm              |   15 +
>   UefiCpuPkg/Library/MpInitLib/Ia32/MpEqu.inc   |    2 +-
>   .../Library/MpInitLib/Ia32/MpFuncs.nasm       |   15 +
>   UefiCpuPkg/Library/MpInitLib/X64/MpEqu.inc    |    4 +-
>   UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm |  373 ++++-
>   UefiCpuPkg/Library/VmgExitLib/VmgExitLib.uni  |   15 +
>   .../ResetVector/Vtf0/Ia16/Real16ToFlat32.asm  |    9 +
>   73 files changed, 3600 insertions(+), 97 deletions(-)
>   create mode 100644 UefiCpuPkg/Library/VmgExitLib/VmgExitLib.inf
>   create mode 100644 MdePkg/Include/Register/Amd/Ghcb.h
>   create mode 100644 UefiCpuPkg/Include/Library/VmgExitLib.h
>   create mode 100644 UefiCpuPkg/Library/CpuExceptionHandlerLib/AMDSevVcCommon.h
>   create mode 100644 UefiCpuPkg/Library/CpuExceptionHandlerLib/Ia32/AMDSevVcCommon.c
>   create mode 100644 UefiCpuPkg/Library/CpuExceptionHandlerLib/PeiDxeAMDSevVcHandler.c
>   create mode 100644 UefiCpuPkg/Library/CpuExceptionHandlerLib/SecAMDSevVcHandler.c
>   create mode 100644 UefiCpuPkg/Library/CpuExceptionHandlerLib/X64/AMDSevVcCommon.c
>   create mode 100644 UefiCpuPkg/Library/VmgExitLib/VmgExitLib.c
>   create mode 100644 MdePkg/Library/BaseLib/Ia32/VmgExit.nasm
>   create mode 100644 MdePkg/Library/BaseLib/Ia32/XGetBv.nasm
>   create mode 100644 MdePkg/Library/BaseLib/X64/VmgExit.nasm
>   create mode 100644 MdePkg/Library/BaseLib/X64/XGetBv.nasm
>   create mode 100644 OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm
>   create mode 100644 UefiCpuPkg/Library/VmgExitLib/VmgExitLib.uni
>