Re: [edk2-devel] [PATCH V6 1/1] OvmfPkg: Enable TDX in ResetVector

["Brijesh Singh" <brijesh.singh@amd.com>]

Hi Min,

On 9/18/21 12:16 AM, Xu, Min M wrote:
> Hi, Brijesh
>
> On September 17, 2021 11:52 PM, Brijesh Singh wrote:
>> Hi Min,
>>
>> On 9/17/21 7:55 AM, Xu, Min M wrote:
>> ...
>>
>>> As I mentioned in my last mail, in the beginning I missed the limitation of
>> smsw.
>>> So I update the code (ResetVectorVtf0.asm) as below using mov CRx.
>>>                                                            <1> BITS 16
>>>     176 00000800 0F20C0              <1>     mov   eax, cr0    <-- previously it
>> was smsw
>>>     177 00000803 A801                  <1>     test    al, 1
>>>     178 00000805 7405                  <1>     jz       .Real
>>>     179                                               <1> BITS 32
>>>     180 00000807 E951FFFFFF      <1>     jmp   Main32
>>>     181                                               <1> BITS 16
>>>     182                                               <1> .Real:
>>>     183 0000080C E939FF              <1>     jmp   EarlyBspInitReal16
>>>
>>> I test the code in a AMD SEV server and try to launch a SEV guest. This time
>> it stuck at the *mov eax, cr0*.
>>> I am curious if *mov eax, cr0* works in real mode in a SEV guest?
>>> I also test the code in a legacy vm guest and td guest, all passed.
>>> Did I miss something?
>>>
>> Hmm, I am not aware of any limitation w.r.t encrypted VMs. I just added the
>> below code in my branch and I do not see any issues, my SEV, SEV-ES and
>> SEV-SNP all are able to boot fine. And KVM trace confirms that code it read
>>
>> diff --git a/OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm
>> b/OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm
>> index f0e509d0672e..98e34332b04c 100644
>> --- a/OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm
>> +++ b/OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm
>> @@ -175,9 +175,21 @@ resetVector:
>>   ;
>>   ; This is where the processor will begin execution
>>   ;
>> +%ifdef ARCH_IA32
>>       nop
>>       nop
>>       jmp     EarlyBspInitReal16
>> +%else
>> +    mov     eax, cr0
>> +    test    al, 1
>> +    jz      .Real
>> +BITS 32
>> +    hlt
>> +    ;jmp     Main32
>> +BITS 16
>> +.Real:
>> +    jmp     EarlyBspInitReal16
>> +%endif
>>
>>   ALIGN   16
>>
>>
>> And KVM trace:
>>
>> kvm_exit: vcpu 0 reason npf rip 0xfff0 info1 0x0000000500000014 info2
>> 0x00000000fffff000 intr_info 0x00000000 error_code 0x00000000
>> kvm_page_fault: address fffff000 error_code 500000014
>> kvm_entry: vcpu 0, rip 0xfff0
>> kvm_exit: vcpu 0 reason read_cr0 rip 0xfff0 info1 0x8000000000000000
>> info2 0x0000000000000000 intr_info 0x00000000 error_code 0x00000000
>> kvm_cr: cr_read 0 = 0x60000010
>> kvm_entry: vcpu 0, rip 0xfff3
>>
>> As we can see from the kvm trace, the first instruction here is the Cr0 read
>> and it was successfully intercepted and rip moved to next instruction.
>>
>> Can you please provide me KVM trace for your failure case ? Also, provide me
>> the output of "lscpu" and "dmesg" from the host.
> The OVMF image you tested is built with GCC tool chain, right?

Yes, we have been using the GCC tool chain only.


> I usually do the development in windows and build the OVMF image with VS2019.
> If the new feature works, then I cherry-pick the patch-sets to code base in ubuntu
> 18.04 and build/test the new feature.
>
> The weird thing is that, with VS2019, even the OVMF image is built from edk2-master, 
> such image doesn't work on AMD SEV server either. But if the image is built by Ubuntu 18.04,
> it does work on AMD SEV server.

This seems very strange that we are failing to execute the hand written
assembly code. I am wondering if somehow the VS compiler is generating a
wrong byte code and thus causing a trap on KVM that requires emulation.
Since the guest memory is encrypted, so KVM emulation code will not be
able to decode the instruction bytes and thus leading in repetitive
nested fault. Only way I could verify my theory is if I can get a KVM
trace or an OVMF binary. If you have have KVM trace or OVMF_CODE.fd
handy then please share.


> I applied my TDX patch-sets to the code base on my Ubuntu 18.04, and build the image.
> This image does work in both AMD SEV server and Intel TDX server.
>
> Thanks!
> Min