Re: [edk2-devel] [RFC PATCH v3 37/43] OvmfPkg: Reserve a page in memory for the SEV-ES AP reset vector

["Lendacky, Thomas" <thomas.lendacky@amd.com>]

On 11/21/19 1:27 PM, Laszlo Ersek wrote:
> On 11/20/19 21:06, Lendacky, Thomas wrote:
>> BZ: https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.tianocore.org%2Fshow_bug.cgi%3Fid%3D2198&data=02%7C01%7Cthomas.lendacky%40amd.com%7C0b8d41fe61b5434f088908d76eb8ee62%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637099612867835131&sdata=6A9VajzefK968qLjGCnZ4cAoL4AGGyRbtl%2FLYW2RC6Y%3D&reserved=0
>>
>> Reserve a fixed area of memory for the SEV-ES AP reset vector and set
>> the fixed PCD, PcdSevEsResetRipBase, to this value.
>>
>> A hypervisor is not allowed to update an SEV-ES guest's register state,
>> so when booting an SEV-ES guest AP, the hypervisor is not allowed to
>> set the RIP to the guest requested value. Instead an SEV-ES AP must be
>> re-directed from within the guest to the actual requested staring location
>> as specified in the INIT-SIPI-SIPI sequence.
>>
>> Provide reset vector code that contains support to jump to the desired
>> RIP location after having been started. This is required for only the
>> very first AP reset.
> 
> (1) I suggest replacing "Provide reset vector code" with "Provide
> *memory for* reset vector code"

Will do.

> 
>>
>> This new reset vector code is used in place of the original code because
> 
> (2) I suggest replacing "reset vector code" with "source file".

Yup, makes sense.

> 
>> of the include path order set in OvmfPkg/ResetVector/ResetVector.inf
>> under "[BuildOptions]".
>>
>> Cc: Jordan Justen <jordan.l.justen@intel.com>
>> Cc: Laszlo Ersek <lersek@redhat.com>
>> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
>> Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
>> ---
>>  OvmfPkg/OvmfPkgX64.fdf                       |  3 +
>>  OvmfPkg/ResetVector/ResetVector.inf          |  2 +
>>  OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm | 94 ++++++++++++++++++++
>>  OvmfPkg/ResetVector/ResetVector.nasmb        |  1 +
>>  4 files changed, 100 insertions(+)
>>  create mode 100644 OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm
>>
>> diff --git a/OvmfPkg/OvmfPkgX64.fdf b/OvmfPkg/OvmfPkgX64.fdf
>> index 973b19fdbf19..b7618b376430 100644
>> --- a/OvmfPkg/OvmfPkgX64.fdf
>> +++ b/OvmfPkg/OvmfPkgX64.fdf
>> @@ -82,6 +82,9 @@ [FD.MEMFD]
>>  0x009000|0x002000
>>  gEfiMdeModulePkgTokenSpaceGuid.PcdSecGhcbBase|gEfiMdeModulePkgTokenSpaceGuid.PcdSecGhcbSize
>>  
>> +0x00B000|0x001000
>> +gUefiCpuPkgTokenSpaceGuid.PcdSevEsResetRipBase|gUefiCpuPkgTokenSpaceGuid.PcdSevEsResetRipSize
>> +
>>  0x010000|0x010000
>>  gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSecPeiTempRamBase|gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSecPeiTempRamSize
>>  
>> diff --git a/OvmfPkg/ResetVector/ResetVector.inf b/OvmfPkg/ResetVector/ResetVector.inf
>> index 266c5fc5c8b3..284f03c4fb1e 100644
>> --- a/OvmfPkg/ResetVector/ResetVector.inf
>> +++ b/OvmfPkg/ResetVector/ResetVector.inf
>> @@ -36,6 +36,8 @@ [BuildOptions]
>>  [Pcd]
>>    gEfiMdeModulePkgTokenSpaceGuid.PcdSecGhcbBase
>>    gEfiMdeModulePkgTokenSpaceGuid.PcdSecGhcbSize
>> +  gUefiCpuPkgTokenSpaceGuid.PcdSevEsResetRipBase
>> +  gUefiCpuPkgTokenSpaceGuid.PcdSevEsResetRipSize
> 
> (3) "RipSize" looks quite strange; how about:
> 
> - PcdSevEsResetVectorBase
> - PcdSevEsResetVectorSize
> 
> (i.e. s/Rip/Vector/)

Yup, I'll change that.

> 
> 
> (4) I think the module will not actually use the "size" PCD, so we can
> remove it from the INF file.

Done.

> 
>>    gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSecPageTablesBase
>>    gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSecPageTablesSize
>>    gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSecPeiTempRamBase
>> diff --git a/OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm b/OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm
>> new file mode 100644
>> index 000000000000..2a3c9bafb451
>> --- /dev/null
>> +++ b/OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm
>> @@ -0,0 +1,94 @@
>> +;------------------------------------------------------------------------------
>> +; @file
>> +; First code executed by processor after resetting.
>> +; Derived from UefiCpuPkg/ResetVector/Vtf0/Ia16/ResetVectorVtf0.asm
>> +;
>> +; Copyright (c) 2008 - 2014, Intel Corporation. All rights reserved.<BR>
>> +; SPDX-License-Identifier: BSD-2-Clause-Patent
>> +;
>> +;------------------------------------------------------------------------------
>> +
>> +BITS    16
>> +
>> +ALIGN   16
>> +
>> +;
>> +; Pad the image size to 4k when page tables are in VTF0
>> +;
>> +; If the VTF0 image has page tables built in, then we need to make
>> +; sure the end of VTF0 is 4k above where the page tables end.
>> +;
>> +; This is required so the page tables will be 4k aligned when VTF0 is
>> +; located just below 0x100000000 (4GB) in the firmware device.
>> +;
>> +%ifdef ALIGN_TOP_TO_4K_FOR_PAGING
>> +    TIMES (0x1000 - ($ - EndOfPageTables) - 0x20) DB 0
>> +%endif
>> +
>> +;
>> +; SEV-ES Processor Reset support
>> +;
>> +; sevEsResetBlock:
>> +;   For the initial boot of an AP under SEV-ES, the "reset" RIP must be
>> +;   programmed to the RAM area defined by SEV_ES_RESET_IP. A known offset
>> +;   and GUID will be used to locate this block in the firmware and extract
>> +;   the build time RIP value. The GUID must always be 48 bytes from the
>> +;   end of the firmware.
>> +;
>> +;   0xffffffca (-0x36) - IP value
>> +;   0xffffffcc (-0x34) - CS selector value
> 
> (5) I think the documentation of these four bytes is incorrect.
> (Similarly in the previous patch, in the commit message.) We populate
> these bytes with the *linear address* of the "reset trampoline" where
> the host will have to boot the AP for the first time. It's not expressed
> in real mode CS:IP notation / meaning.

This is correct. The code will be in "big real mode" as you note below
(the reset vector is actually 0xfffffff0, just below 4GB and running in
16-bit mode).

If you look in Qemu, target/i386/cpu.c, x86_cpu_reset(), you'll see where
the CS segment is being loaded with a base of 0xffff0000 and the eip gets
loaded with 0xfff0.

So the change to Qemu takes the "CS selector value" and effectively left
shifts it 16-bits to set the base and sets the eip to the "IP value" (in
actuality, Qemu reads this as a 32-bit value starting at the IP value and
ANDs it with 0xffff0000 to get the base and 0x0000ffff to get the eip.

> 
> 
> (6) Which brings me to my main point... Value 0x00B000 (for the "base"
> PCD) is relative to the base address of FD.MEMFD, namely 8MB (see
> MEMFD_BASE_ADDRESS).
> 
> IOW, the ultimate value of SEV_ES_RESET_IP will be 0x0x0080_B000. That
> address is larger than 1MB.

Which is ok since Qemu will set up CS appropriately.

> 
> Therefore, the host will have to launch the AP (for the first time)
> above 1MB (in guest-phys address space).

Which is also OK, just like the BSP was launched at 0xFFFF_FFF0.

> 
> How can that work, if the AP is supposed to start in real mode? The
> linear address 0x0x0080_B000 cannot be expressed in 16-bit real mode
> CS:IP notation at all.
> 
> Does the hypervisor start the AP in "big real mode" (16-bit mode with
> 4GB segment limits, and CS containing a segment selector, not the actual
> segment base)?
> 
> ... I guess that would answer my question (6) -- and the last few
> patches in this series, before this one, *do* suggest "big real mode" to
> me --, but the documentation around (5), and in the commit message of
> the previous patch, still doesn't match that.

I'll expand on the commit message to indicate that Qemu, or others, must
do something similar relative to the CS register setup.

Thanks,
Tom

> 
> ... AFAICS I'm only requesting small cleanups for this patch (naming,
> documentation, PCD listing), thus, conditional on everything addressed,
> I feel comfortable giving
> 
> Reviewed-by: Laszlo Ersek <lersek@redhat.com>
> 
> in advance.
> 
> Thanks!
> Laszlo
> 
>> +;   0xffffffce (-0x32) - Size of SEV-ES reset block
>> +;   0xffffffd0 (-0x30) - SEV-ES reset block GUID
>> +;                        (00f771de-1a7e-4fcb-890e-68c77e2fb44e)
>> +;
>> +
>> +TIMES (32 - (sevEsResetBlockEnd - sevEsResetBlockStart)) DB 0
>> +
>> +sevEsResetBlockStart:
>> +    DD      SEV_ES_RESET_IP
>> +    DW      sevEsResetBlockEnd - sevEsResetBlockStart
>> +    DB      0xDE, 0x71, 0xF7, 0x00, 0x7E, 0x1A, 0xCB, 0x4F
>> +    DB      0x89, 0x0E, 0x68, 0xC7, 0x7E, 0x2F, 0xB4, 0x4E
>> +sevEsResetBlockEnd:
>> +
>> +ALIGN   16
>> +
>> +applicationProcessorEntryPoint:
>> +;
>> +; Application Processors entry point
>> +;
>> +; GenFv generates code aligned on a 4k boundary which will jump to this
>> +; location.  (0xffffffe0)  This allows the Local APIC Startup IPI to be
>> +; used to wake up the application processors.
>> +;
>> +    jmp     EarlyApInitReal16
>> +
>> +ALIGN   8
>> +
>> +    DD      0
>> +
>> +;
>> +; The VTF signature
>> +;
>> +; VTF-0 means that the VTF (Volume Top File) code does not require
>> +; any fixups.
>> +;
>> +vtfSignature:
>> +    DB      'V', 'T', 'F', 0
>> +
>> +ALIGN   16
>> +
>> +resetVector:
>> +;
>> +; Reset Vector
>> +;
>> +; This is where the processor will begin execution
>> +;
>> +    nop
>> +    nop
>> +    jmp     EarlyBspInitReal16
>> +
>> +ALIGN   16
>> +
>> +fourGigabytes:
>> +
>> diff --git a/OvmfPkg/ResetVector/ResetVector.nasmb b/OvmfPkg/ResetVector/ResetVector.nasmb
>> index 708bbda6208f..2bf14681099e 100644
>> --- a/OvmfPkg/ResetVector/ResetVector.nasmb
>> +++ b/OvmfPkg/ResetVector/ResetVector.nasmb
>> @@ -81,5 +81,6 @@
>>  
>>  %include "Main.asm"
>>  
>> +  %define SEV_ES_RESET_IP  FixedPcdGet32 (PcdSevEsResetRipBase)
>>  %include "Ia16/ResetVectorVtf0.asm"
>>  
>>
>