Re: [edk2-devel] [RFC PATCH v2 00/44] SEV-ES guest support

["Lendacky, Thomas" <thomas.lendacky@amd.com>]

On 9/23/19 8:55 PM, Dong, Eric wrote:
> Hi Tom,

Hi Eric,

> 
> Thanks for you to contribute such a big changes. Seems like this is a big changes for current code, can you help to do a design review in TianoCore Design Meeting? It will be helpful for us to understand the code change and review it.
> 
> Ray is the owner of that meeting, you can contact him for the arrangement of your topic.

Thanks for the info, Eric.

Ray, I see on the TianoCore wiki that I should contact Stephano to be
added to the meeting agenda. Should I follow those directions?

Thanks,
Tom

> 
> Thanks,
> Eric
> 
>> -----Original Message-----
>> From: devel@edk2.groups.io [mailto:devel@edk2.groups.io] On Behalf Of
>> Lendacky, Thomas
>> Sent: Saturday, September 21, 2019 3:25 AM
>> To: devel@edk2.groups.io
>> Cc: Justen, Jordan L <jordan.l.justen@intel.com>; Laszlo Ersek
>> <lersek@redhat.com>; Ard Biesheuvel <ard.biesheuvel@linaro.org>; Kinney,
>> Michael D <michael.d.kinney@intel.com>; Gao, Liming
>> <liming.gao@intel.com>; Dong, Eric <eric.dong@intel.com>; Ni, Ray
>> <ray.ni@intel.com>; Singh, Brijesh <brijesh.singh@amd.com>; You, Benjamin
>> <benjamin.you@intel.com>; Bi, Dandan <dandan.bi@intel.com>; Dong, Guo
>> <guo.dong@intel.com>; Wu, Hao A <hao.a.wu@intel.com>; Wang, Jian J
>> <jian.j.wang@intel.com>; Ma, Maurice <maurice.ma@intel.com>
>> Subject: Re: [edk2-devel] [RFC PATCH v2 00/44] SEV-ES guest support
>>
>> On 9/19/19 2:52 PM, Lendacky, Thomas wrote:
>>> From: Tom Lendacky <thomas.lendacky@amd.com>
>>>
>>> This patch series provides support for running EDK2/OVMF under SEV-ES.
>>
>> I wanted to provide the full series so that everyone can see what is involved
>> with the support.
>>
>> Providing multiple, smaller series of patches might be easier to review.
>> If that is desired, any suggestions/recommendations on how to best approach
>> that would be great.
>>
>> Thanks,
>> Tom
>>
>>>
>>> Secure Encrypted Virtualization - Encrypted State (SEV-ES) expands on
>>> the SEV support to protect the guest register state from the
>>> hypervisor. See
>>> "AMD64 Architecture Programmer's Manual Volume 2: System
>> Programming",
>>> section "15.35 Encrypted State (SEV-ES)" [1].
>>>
>>> In order to allow a hypervisor to perform functions on behalf of a
>>> guest, there is architectural support for notifying a guest's
>>> operating system when certain types of VMEXITs are about to occur.
>>> This allows the guest to selectively share information with the
>>> hypervisor to satisfy the requested function. The notification is
>>> performed using a new exception, the VMM Communication exception
>>> (#VC). The information is shared through the Guest-Hypervisor
>> Communication Block (GHCB) using the VMGEXIT instruction.
>>> The GHCB format and the protocol for using it is documented in "SEV-ES
>>> Guest-Hypervisor Communication Block Standardization" [2].
>>>
>>> The main areas of the EDK2 code that are updated to support SEV-ES are
>>> around the exception handling support and the AP boot support.
>>>
>>> Exception support is required starting in Sec, continuing through Pei
>>> and into Dxe in order to handle #VC exceptions that are generated.
>>> Each AP requires it's own GHCB page as well as a page to hold values
>>> specific to that AP.
>>>
>>> AP booting poses some interesting challenges. The INIT-SIPI-SIPI
>>> sequence is typically used to boot the APs. However, the hypervisor is
>>> not allowed to update the guest registers. The GHCB document [2] talks
>>> about how SMP booting under SEV-ES is performed.
>>>
>>> Since the GHCB page must be a shared (unencrypted) page, the processor
>>> must be running in long mode in order for the guest and hypervisor to
>>> communicate with each other. As a result, SEV-ES is only supported
>>> under the X64 architecture.
>>>
>>> [1] https://www.amd.com/system/files/TechDocs/24593.pdf
>>> [2] https://developer.amd.com/wp-content/resources/56421.pdf
>>>
>>> ---
>>>
>>> These patches are based on commit:
>>> ada905ab5c0e ("MdeModulePkg DxeCore: Fix for missing Memory Attributes
>>> Table (MAT) update")
>>>
>>> A version of the tree can be found at:
>>> https://github.com/AMDESE/ovmf/tree/sev-es-v7
>>>
>>> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
>>> Cc: Benjamin You <benjamin.you@intel.com>
>>> Cc: Dandan Bi <dandan.bi@intel.com>
>>> Cc: Eric Dong <eric.dong@intel.com>
>>> Cc: Guo Dong <guo.dong@intel.com>
>>> Cc: Hao A Wu <hao.a.wu@intel.com>
>>> Cc: Jian J Wang <jian.j.wang@intel.com>
>>> Cc: Jordan Justen <jordan.l.justen@intel.com>
>>> Cc: Laszlo Ersek <lersek@redhat.com>
>>> Cc: Liming Gao <liming.gao@intel.com>
>>> Cc: Maurice Ma <maurice.ma@intel.com>
>>> Cc: Michael D Kinney <michael.d.kinney@intel.com>
>>> Cc: Ray Ni <ray.ni@intel.com>
>>>
>>> Changes since v1:
>>> - Patches reworked to be more specific to the component/area being
>> updated
>>>    and order of definition/usage
>>> - Created a library for VMGEXIT-related functions to replace use of inline
>>>    functions
>>> - Allocation method for GDT changed from AllocatePool to AllocatePages
>>> - Early caching only enabled for SEV-ES guests
>>> - Ensure AP loop mode set to halt loop mode for SEV-ES guests
>>> - Reserved SEC GHCB-related memory areas when S3 is enabled
>>>
>>> Tom Lendacky (44):
>>>    MdePkg: Create PCDs to be used in support of SEV-ES
>>>    OvmfPkg/MemEncryptSevLib: Add an SEV-ES guest indicator function
>>>    OvmfPkg: Add support to perform SEV-ES initialization
>>>    OvmfPkg/ResetVector: Add support for a 32-bit SEV check
>>>    MdePkg: Add the MSR definition for the GHCB register
>>>    OvmfPkg: Create a GHCB page for use during Sec phase
>>>    OvmfPkg/PlatformPei: Reserve GHCB-related areas if S3 is supported
>>>    OvmfPkg: Create GHCB pages for use during Pei and Dxe phase
>>>    MdeModulePkg/DxeIplPeim: Support GHCB pages when creating page
>> tables
>>>    OvmfPkg: A per-CPU variable area for #VC usage
>>>    OvmfPkg/PlatformPei: Move early GDT into ram when SEV-ES is enabled
>>>    MdePkg: Add a structure definition for the GHCB
>>>    MdePkg/BaseLib: Add support for the VMGEXIT instruction
>>>    UefiCpuPkg: Implement library support for VMGEXIT
>>>    UefiCpuPkg/CpuExceptionHandler: Add base support for the #VC exception
>>>    OvmfPkg/MemEncryptSevLib: Make MemEncryptSevLib available during
>> SEC
>>>    UefiCpuPkg/CpuExceptionHandler: Add #VC exception handling for Sec
>>>      phase
>>>    OvmfPkg/Sec: Enable cache early to speed up booting
>>>    UefiCpuPkg/CpuExceptionHandler: Add support for IOIO_PROT NAE events
>>>    UefiCpuPkg/CpuExceptionHandler: Support string IO for IOIO_PROT NAE
>>>      events
>>>    MdePkg: Add support for the XGETBV instruction
>>>    UefiCpuPkg/CpuExceptionHandler: Add support for CPUID NAE events
>>>    UefiCpuPkg/CpuExceptionHandler: Add support for MSR_PROT NAE events
>>>    UefiCpuPkg/CpuExceptionHandler: Add support for NPF NAE events (MMIO)
>>>    UefiCpuPkg/CpuExceptionHandler: Add support for WBINVD NAE events
>>>    UefiCpuPkg/CpuExceptionHandler: Add support for RDTSC NAE events
>>>    UefiCpuPkg/CpuExceptionHandler: Add support for RDPMC NAE events
>>>    UefiCpuPkg/CpuExceptionHandler: Add support for INVD NAE events
>>>    UefiCpuPkg/CpuExceptionHandler: Add support for VMMCALL NAE events
>>>    UefiCpuPkg/CpuExceptionHandler: Add support for RDTSCP NAE events
>>>    UefiCpuPkg/CpuExceptionHandler: Add support for MONITOR/MONITORX
>> NAE
>>>      events
>>>    UefiCpuPkg/CpuExceptionHandler: Add support for MWAIT/MWAITX NAE
>>>      events
>>>    UefiCpuPkg/CpuExceptionHandler: Add support for DR7 Read/Write NAE
>>>      events
>>>    UefiCpuPkg/MpInitLib: Update CPU MP data with a flag to indicate if
>>>      SEV-ES is active
>>>    MdeModulePkg: Reserve a 16-bit protected mode code segment descriptor
>>>    UefiCpuPkg: Add a 16-bit protected mode code segment descriptor
>>>    OvmfPkg: Add support for SEV-ES AP reset vector re-directing
>>>    UefiCpuPkg: Allow AP booting under SEV-ES
>>>    OvmfPkg: Move the GHCB allocations into reserved memory
>>>    MdePkg: Add a finalization function to the CPU protocol
>>>    UefiCpuPkg/MpInitLib: Add MP finalization interface to MpInitLib
>>>    UefiCpuPkg/MpInitLib: Prepare SEV-ES guest APs for OS use
>>>    UefiCpuPkg/CpuDxe: Provide an DXE MP finalization routine to support
>>>      SEV-ES
>>>    MdeModulePkg/DxeCore: Perform the CPU protocol finalization support
>>>
>>>   MdeModulePkg/MdeModulePkg.dec                 |   15 +
>>>   OvmfPkg/OvmfPkg.dec                           |   10 +
>>>   UefiCpuPkg/UefiCpuPkg.dec                     |    3 +
>>>   OvmfPkg/OvmfPkgIa32.dsc                       |   10 +
>>>   OvmfPkg/OvmfPkgIa32X64.dsc                    |   10 +
>>>   OvmfPkg/OvmfPkgX64.dsc                        |   10 +
>>>   UefiCpuPkg/UefiCpuPkg.dsc                     |    5 +
>>>   UefiPayloadPkg/UefiPayloadPkgIa32.dsc         |    2 +
>>>   UefiPayloadPkg/UefiPayloadPkgIa32X64.dsc      |    2 +
>>>   OvmfPkg/OvmfPkgX64.fdf                        |    6 +
>>>   MdeModulePkg/Core/DxeIplPeim/DxeIpl.inf       |    2 +
>>>   MdePkg/Library/BaseLib/BaseLib.inf            |    4 +
>>>   .../BaseMemEncryptSevLib.inf                  |    2 +-
>>>   OvmfPkg/PlatformPei/PlatformPei.inf           |    7 +
>>>   OvmfPkg/ResetVector/ResetVector.inf           |    6 +
>>>   OvmfPkg/Sec/SecMain.inf                       |    2 +
>>>   .../DxeCpuExceptionHandlerLib.inf             |    5 +
>>>   .../PeiCpuExceptionHandlerLib.inf             |    5 +
>>>   .../SecPeiCpuExceptionHandlerLib.inf          |    5 +
>>>   UefiCpuPkg/Library/MpInitLib/DxeMpInitLib.inf |    3 +
>>>   UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf |    3 +
>>>   UefiCpuPkg/Library/VmgExitLib/VmgExitLib.inf  |   33 +
>>>   .../Core/DxeIplPeim/X64/VirtualMemory.h       |   12 +-
>>>   MdePkg/Include/Library/BaseLib.h              |   30 +
>>>   MdePkg/Include/Protocol/Cpu.h                 |   18 +
>>>   MdePkg/Include/Register/Amd/Fam17Msr.h        |   28 +
>>>   MdePkg/Include/Register/Amd/Ghcb.h            |  105 ++
>>>   OvmfPkg/Include/Library/MemEncryptSevLib.h    |   12 +
>>>   UefiCpuPkg/CpuDxe/CpuDxe.h                    |   12 +
>>>   UefiCpuPkg/CpuDxe/CpuGdt.h                    |    4 +-
>>>   UefiCpuPkg/Include/Library/MpInitLib.h        |   14 +
>>>   UefiCpuPkg/Include/Library/VmgExitLib.h       |   74 +
>>>   .../CpuExceptionHandlerLib/AMDSevVcCommon.h   |   19 +
>>>   .../CpuExceptionCommon.h                      |    2 +
>>>   UefiCpuPkg/Library/MpInitLib/MpLib.h          |   77 +-
>>>   MdeModulePkg/Core/Dxe/DxeMain/DxeMain.c       |    5 +
>>>   .../Core/DxeIplPeim/Ia32/DxeLoadFunc.c        |    6 +-
>>>   .../Core/DxeIplPeim/X64/DxeLoadFunc.c         |   11 +-
>>>   .../Core/DxeIplPeim/X64/VirtualMemory.c       |   49 +-
>>>   MdePkg/Library/BaseLib/Ia32/GccInline.c       |   45 +
>>>   MdePkg/Library/BaseLib/X64/GccInline.c        |   47 +
>>>   .../MemEncryptSevLibInternal.c                |   77 +-
>>>   OvmfPkg/PlatformPei/AmdSev.c                  |   78 ++
>>>   OvmfPkg/PlatformPei/MemDetect.c               |   23 +
>>>   OvmfPkg/Sec/SecMain.c                         |   39 +-
>>>   UefiCpuPkg/CpuDxe/CpuDxe.c                    |   21 +-
>>>   UefiCpuPkg/CpuDxe/CpuGdt.c                    |    8 +-
>>>   .../CpuExceptionCommon.c                      |    2 +-
>>>   .../Ia32/AMDSevVcCommon.c                     |   13 +
>>>   .../PeiDxeAMDSevVcHandler.c                   |   22 +
>>>   .../PeiDxeSmmCpuException.c                   |   16 +
>>>   .../SecAMDSevVcHandler.c                      |   45 +
>>>   .../SecPeiCpuException.c                      |   16 +
>>>   .../X64/AMDSevVcCommon.c                      | 1200 +++++++++++++++++
>>>   UefiCpuPkg/Library/MpInitLib/DxeMpLib.c       |  136 +-
>>>   UefiCpuPkg/Library/MpInitLib/MpLib.c          |  259 +++-
>>>   UefiCpuPkg/Library/MpInitLib/PeiMpLib.c       |   35 +
>>>   UefiCpuPkg/Library/VmgExitLib/VmgExitLib.c    |   75 ++
>>>   UefiCpuPkg/PiSmmCpuDxeSmm/X64/SmmFuncsArch.c  |    2 +-
>>>   MdePkg/Library/BaseLib/Ia32/VmgExit.nasm      |   31 +
>>>   MdePkg/Library/BaseLib/Ia32/XGetBv.nasm       |   31 +
>>>   MdePkg/Library/BaseLib/X64/VmgExit.nasm       |   32 +
>>>   MdePkg/Library/BaseLib/X64/XGetBv.nasm        |   34 +
>>>   OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm  |   80 ++
>>>   OvmfPkg/ResetVector/Ia32/PageTables64.asm     |  256 +++-
>>>   OvmfPkg/ResetVector/ResetVector.nasmb         |   13 +
>>>   .../X64/ExceptionHandlerAsm.nasm              |   15 +
>>>   UefiCpuPkg/Library/MpInitLib/Ia32/MpEqu.inc   |    2 +-
>>>   .../Library/MpInitLib/Ia32/MpFuncs.nasm       |   15 +
>>>   UefiCpuPkg/Library/MpInitLib/X64/MpEqu.inc    |    4 +-
>>>   UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm |  373 ++++-
>>>   UefiCpuPkg/Library/VmgExitLib/VmgExitLib.uni  |   15 +
>>>   .../ResetVector/Vtf0/Ia16/Real16ToFlat32.asm  |    9 +
>>>   73 files changed, 3600 insertions(+), 97 deletions(-)
>>>   create mode 100644 UefiCpuPkg/Library/VmgExitLib/VmgExitLib.inf
>>>   create mode 100644 MdePkg/Include/Register/Amd/Ghcb.h
>>>   create mode 100644 UefiCpuPkg/Include/Library/VmgExitLib.h
>>>   create mode 100644
>> UefiCpuPkg/Library/CpuExceptionHandlerLib/AMDSevVcCommon.h
>>>   create mode 100644
>> UefiCpuPkg/Library/CpuExceptionHandlerLib/Ia32/AMDSevVcCommon.c
>>>   create mode 100644
>> UefiCpuPkg/Library/CpuExceptionHandlerLib/PeiDxeAMDSevVcHandler.c
>>>   create mode 100644
>> UefiCpuPkg/Library/CpuExceptionHandlerLib/SecAMDSevVcHandler.c
>>>   create mode 100644
>> UefiCpuPkg/Library/CpuExceptionHandlerLib/X64/AMDSevVcCommon.c
>>>   create mode 100644 UefiCpuPkg/Library/VmgExitLib/VmgExitLib.c
>>>   create mode 100644 MdePkg/Library/BaseLib/Ia32/VmgExit.nasm
>>>   create mode 100644 MdePkg/Library/BaseLib/Ia32/XGetBv.nasm
>>>   create mode 100644 MdePkg/Library/BaseLib/X64/VmgExit.nasm
>>>   create mode 100644 MdePkg/Library/BaseLib/X64/XGetBv.nasm
>>>   create mode 100644 OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm
>>>   create mode 100644 UefiCpuPkg/Library/VmgExitLib/VmgExitLib.uni
>>>
>>
>> 
>