Re: [edk2-devel] [PATCH RFC v2 05/28] MdePkg: Add AsmPvalidate() support

["Brijesh Singh" <brijesh.singh@amd.com>]

On 5/4/21 8:58 AM, Laszlo Ersek wrote:
> On 04/30/21 13:51, Brijesh Singh wrote:
>> BZ: https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.tianocore.org%2Fshow_bug.cgi%3Fid%3D3275&data=04%7C01%7Cbrijesh.singh%40amd.com%7C6ceeec6c984d468bb87908d90f04b789%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637557335220626621%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=jF44vk%2FBUVukpuNg4vrcRqHnEa7nO%2FGPEe9ti720cnE%3D&reserved=0
>>
>> The PVALIDATE instruction validates or rescinds validation of a guest
>> page RMP entry. Upon completion, a return code is stored in EAX, rFLAGS
>> bits OF, ZF, AF, PF and SF are set based on this return code. If the
>> instruction completed succesfully, the rFLAGS bit CF indicates if the
>> contents of the RMP entry were changed or not.
>>
>> For more information about the instruction see AMD APM volume 3.
>>
>> Cc: James Bottomley <jejb@linux.ibm.com>
>> Cc: Min Xu <min.m.xu@intel.com>
>> Cc: Jiewen Yao <jiewen.yao@intel.com>
>> Cc: Tom Lendacky <thomas.lendacky@amd.com>
>> Cc: Jordan Justen <jordan.l.justen@intel.com>
>> Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
>> Cc: Laszlo Ersek <lersek@redhat.com>
>> Cc: Erdem Aktas <erdemaktas@google.com>
>> Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
>> ---
>>  MdePkg/Include/Library/BaseLib.h          | 37 +++++++++++++++++
>>  MdePkg/Library/BaseLib/BaseLib.inf        |  1 +
>>  MdePkg/Library/BaseLib/X64/Pvalidate.nasm | 43 ++++++++++++++++++++
>>  3 files changed, 81 insertions(+)
>>
>> diff --git a/MdePkg/Include/Library/BaseLib.h b/MdePkg/Include/Library/BaseLib.h
>> index 7253997a6f..92ce695e93 100644
>> --- a/MdePkg/Include/Library/BaseLib.h
>> +++ b/MdePkg/Include/Library/BaseLib.h
>> @@ -7518,5 +7518,42 @@ PatchInstructionX86 (
>>    IN  UINTN                    ValueSize
>>    );
>>
>> +/**
>> + Execute a PVALIDATE instruction to validate or rescnids validation of a guest
> (1) typo: "rescnids"

Noted.


>
>
>> + page's RMP entry.
>> +
>> + Upon completion, in addition to the return value the instruction also updates
>> + the eFlags. A caller must check both the return code as well as eFlags to
>> + determine if the RMP entry has been updated.
>> +
>> + The function is available on x64.
> (2) Please write "X64"; that's how the architecture is usually mentioned
> in both the UEFI spec and in edk2.

Noted.


>
>
>> +
>> + @param[in]    Address        The guest virtual address to validate.
>> + @param[in]    PageSize       The page size to use.
>> + @param[i]     Validate       Validate or rescinds.
>> + @param[out]   Eflags         The value of Eflags after PVALIDATE completion.
> (3) Typo: "[i]" should be "[in]".

Noted.


>
>
> (4) The order of parameters listed in this comment block differs from
> the actual parameter list.
>
> The ECC plugin of the edk2 CI will catch this issue anyway. So, before
> submitting the patch set to the list, please submit a personal PR on
> github.com against the main repo, just to run CI on your patches.
>
Interestingly I did ran the series with edk2 CI and everything passed
before I submitted for the review. But, I will go ahead and fix the order.



>> +
>> + @retval       PvalidateRetValue  The return value from the PVALIDATE instruction.
> More on the return value / type later, below.
>
>> +**/
>> +typedef enum {
>> +  PvalidatePageSize4K = 0,
>> +  PvalidatePageSize2MB,
>> +} PVALIDATE_PAGE_SIZE;
>> +
>> +typedef enum {
>> +  PvalidateRetSuccess = 0,
>> +  PvalidateRetFailInput = 1,
>> +  PvalidateRetFailSizemismatch = 6,
>> +} PVALIDATE_RET_VALUE;
>> +
> (5) These typedefs do not belong between the function leading comment
> and the function declaration. Please hoist the typedefs just above the
> leading comment, and add a separate comment for the typedefs -- using
> the proper comment style for typedefs, too.
>
>
>> +PVALIDATE_RET_VALUE
> (6) In my opinion, using an enum for an EFIAPI function's return type is
> problematic. According to the UEFI spec (v2.9), "Table 2-3 Common UEFI
> Data Types", <Enumerated Type> may correspond to INT32 or UINT32. I
> don't like that ambiguity here. The spec also says that such types
> should never be used at least as structure fields.
>
> I'm perfectly fine with functions in standard (ISO) C programs returning
> enums, but I think the situation is less clear in UEFI. I don't recall
> standard interfaces (spec-level, or even edk2 / MdePkg interfaces) that
> return enums.
>
> I suggest the following instead. Drop the PVALIDATE_RET_VALUE enum
> altogether. Specify EFI_STATUS as the return type, in the declaration of
> the function.

Works with me.


>
> In the UEFI spec, Appendix D specifies the numeric values of the status
> codes. Furthermore, there are examples for NASM sources using EFI_*
> status codes *numerically* in edk2; minimally:
>
> - IntelFsp2Pkg/FspSecCore/Ia32/FspApiEntryT.nasm
> - IntelFsp2WrapperPkg/Library/SecFspWrapperPlatformSecLibSample/Ia32/SecEntry.nasm
>
> Thus, please modify the assembly source code in this patch to return
> EFI_SUCCESS (already value 0, conveniently) if the instruction succeeds.

Works with me.


> Return EFI_INVALID_PARAMETER (0x8000_0002) in case the instruction fails
> with error code 1 (FAIL_INPUT).

Works with me.


>
> Return EFI_UNSUPPORTED (0x8000_0003), or even EFI_NO_MAPPING
> (0x8000_0017), for value 6 (FAIL_SIZEMISMATCH).

I am not sure if we really want to do this. You will see later in the
patches that in some cases the PVALIDATE will return a failure and we
will need to know the failure code to determine the next steps.
Especially this particular error code is used later. This error happens
when the page size of the backing pages does not match with the
pvalidated size. In those cases we need to retry the PVALIDATE with
lower page size so that a validation succeed. One such a example is:

- Guest ask hypervisor to add the page as 2M in RMP table.

- Hypervisor added the page as 512 4K pages - because it was not able to
find a large backing pages.

- Guest attempts to pvalidate the page as a 2M. The pvalidate will
return a failure saying its a size mismatch between the requested
pvalidated and RMP table. The recommendation is that guest should try
with a smaller page size.

I would prefer to pass the pvalidate error as-is to caller so that it
can make the correct decision.


>
> The leading comment block of the function is supposed to explain these
> associations:
>
>   @retval EFI_SUCCESS        Successful completion (regardless of
>                              whether the Validated bit changed state).
>   @retval INVALID_PARAMETER  Invalid input parameters (FAIL_INPUT).
>   @retval EFI_UNSUPPORTED    Page size mismatch between guest (2M) and
>                              RMP entry (4K) (FAIL_SIZEMISMATCH).
>
> (Passing in the PVALIDATE_PAGE_SIZE enum, as a parameter, should be
> fine, BTW)
>
>
> (7) According to the AMD APM, "Support for this instruction is indicated
> by CPUID Fn8000_001F_EAX[SNP]=1".
>
> Presumably, if the (physical, or emulated) hardware does not support
> PVALIDATE, an #UD is raised. That condition should be explained in the
> function's leading comment. (Mention the CPUID and the #UD, I guess.)
Noted.
>
>> +EFIAPI
>> +AsmPvalidate (
>> +  IN   PVALIDATE_PAGE_SIZE     PageSize,
>> +  IN   BOOLEAN                 Validate,
>> +  IN   UINTN                   Address,
> (8) This should be EFI_PHYSICAL_ADDRESS, not UINTN.

Noted.


>
>> +  OUT  IA32_EFLAGS32           *Eflags
>> +  );
>> +
>>  #endif // defined (MDE_CPU_IA32) || defined (MDE_CPU_X64)
>>  #endif // !defined (__BASE_LIB__)
> (9) Unless you foresee particular uses for eflags *other than* CF, I
> would suggest replacing the Eflags output parameter with
>
>   OUT BOOLEAN   *RmpEntryUpdated
>
> The function would still only have 4 parameters, which shouldn't be
> difficult to handle in the assembly implementation (i.e. write to the
> UINT8 (= BOOLEAN) object referenced by "RmpEntryUpdated"). EFIAPI means
> that the first four params are passed in RCX, RDX, R8, R9.
Works with me, thats what I ended up doing for the guest kernel patches
and will do the same for OVMF as well.
>
> Thus far, I can see only one AsmPvalidate() call: in IssuePvalidate(),
> from patch #21 ("OvmfPkg/MemEncryptSevLib: Add support to validate
> system RAM"). And there, CF looks sufficient.
>
>
> (10) The instruction is X64 only, but you are providing the declaration
> even if MDE_CPU_IA32 is #defined. That seems wrong; even the declaration
> should be invisible in that case. Please declare the function for
> MDE_CPU_X64 only.
>
Noted.


>> diff --git a/MdePkg/Library/BaseLib/BaseLib.inf b/MdePkg/Library/BaseLib/BaseLib.inf
>> index b76f3af380..d33b4a8f7d 100644
>> --- a/MdePkg/Library/BaseLib/BaseLib.inf
>> +++ b/MdePkg/Library/BaseLib/BaseLib.inf
>> @@ -321,6 +321,7 @@
>>    X64/XGetBv.nasm
>>    X64/XSetBv.nasm
>>    X64/VmgExit.nasm
>> +  X64/Pvalidate.nasm
>>    ChkStkGcc.c  | GCC
>>
>>  [Sources.EBC]
> (11) This list of source files is already not sorted alphabetically,
> unfortunately. But we can still do better than this: I suggest inserting
> "X64/Pvalidate.nasm" just before "X64/RdRand.nasm".

Noted.


>
> (12) Your git setup seems less than ideal for formatting edk2 patches.
> The @@ hunk header above does not show the INF file section being
> modified. It should look something like this:
>
>   @@ -317,6 +317,7 @@ [Sources.X64]
>                       ^^^^^^^^^^^^^
>
> Please run the "BaseTools/Scripts/SetupGit.py" script in your working
> tree.

Will do.


>
> Alternatively, please see "xfuncname" at
> <https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FLaszlo%2527s-unkempt-git-guide-for-edk2-contributors-and-maintainers%23contrib-05&amp;data=04%7C01%7Cbrijesh.singh%40amd.com%7C6ceeec6c984d468bb87908d90f04b789%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637557335220626621%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=W16GsCzqxFZlH7DBOIFY0EJy%2FS6DTSCt0qEVOMLjjRs%3D&amp;reserved=0>.
>
> This is of course not a bug in the patch, but fixing your setup will
> help with the next round of review.
>
>
>> diff --git a/MdePkg/Library/BaseLib/X64/Pvalidate.nasm b/MdePkg/Library/BaseLib/X64/Pvalidate.nasm
>> new file mode 100644
>> index 0000000000..f2aba114ac
>> --- /dev/null
>> +++ b/MdePkg/Library/BaseLib/X64/Pvalidate.nasm
>> @@ -0,0 +1,43 @@
>> +;-----------------------------------------------------------------------------
>> +;
>> +; Copyright (c) 2020-2021, AMD. All rights reserved.<BR>
> (13) I believe we don't introduce new files with copyright notices
> referring to the past. IOW, I think you should only say "2021" here.
>
Noted.


>> +; SPDX-License-Identifier: BSD-2-Clause-Patent
>> +;
>> +; Module Name:
>> +;
>> +;   Pvalidate.Asm
>> +;
>> +; Abstract:
>> +;
>> +;   AsmPvalidate function
>> +;
>> +; Notes:
>> +;
> (14) I defer to the MdePkg maintainers on this, but "Module Name" is
> plain wrong, and the Abstract is useless. Either fix those up please
> ("Abstract" could be a copy of the corrected leading comment block), or
> just drop them both.
>
>
>> +;-----------------------------------------------------------------------------
>> +
>> +    SECTION .text
>> +
>> +;-----------------------------------------------------------------------------
>> +;  PvalidateRetValue
>> +;  EFIAPI
>> +;  AsmPvalidate (
>> +;    IN   UINT32  RmpPageSize
>> +;    IN   UINT32  Validate,
>> +;    IN   UINTN   Address,
>> +;    OUT  UINTN  *Eflags,
>> +;    )
>> +;-----------------------------------------------------------------------------
> (15) Please update this accordingly to the corrected function
> specification.

Noted.


>
>> +global ASM_PFX(AsmPvalidate)
>> +ASM_PFX(AsmPvalidate):
>> +  mov     rax, r8
>> +
>> +  ; PVALIDATE instruction opcode
>> +  DB      0xF2, 0x0F, 0x01, 0xFF
> This is bad practice; we make every effort to avoid DB-encoded
> instructions.
>
> We have two PVALIDATE instances in the patch set (... that I can see
> immediateyl); the first here, and the other in
> "OvmfPkg/ResetVector/Ia32/PageTables64.asm" (from patch #17,
> "OvmfPkg/ResetVector: Invalidate the GHCB page"). Therefore, hiding the
> encoding of PVALIDATE behind a NASM macro definitely makes sense.
>
> (16a) Please file a NASM feature request for PVALIDATE at
> <https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.nasm.us%2F&amp;data=04%7C01%7Cbrijesh.singh%40amd.com%7C6ceeec6c984d468bb87908d90f04b789%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637557335220636620%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=VtpkUnBunOdo3n86u3dBhoJZPxLN8M95ClyYUtC7v9o%3D&amp;reserved=0>.

Will do.


>
> (16b) In the present MdePkg patch, please extend the file
>
>   MdePkg/Include/X64/Nasm.inc
>
> as follows:
>
>> diff --git a/MdePkg/Include/X64/Nasm.inc b/MdePkg/Include/X64/Nasm.inc
>> index 527f71e9eb4d..ff37f1e35707 100644
>> --- a/MdePkg/Include/X64/Nasm.inc
>> +++ b/MdePkg/Include/X64/Nasm.inc
>> @@ -33,6 +33,15 @@
>>      DB 0xF3, 0x48, 0x0F, 0xAE, 0xE8
>>  %endmacro
>>
>> +;
>> +; Macro for the PVALIDATE instruction, defined in AMD publication #24594
>> +; revision 3.32. NASM feature request URL:
>> +; <https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.nasm.us%2Fshow_bug.cgi%3Fid%3DFIXME&amp;data=04%7C01%7Cbrijesh.singh%40amd.com%7C6ceeec6c984d468bb87908d90f04b789%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637557335220636620%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=WeN3YmLN6OEpY8azwBt0ixogffkCCSk70P2Y%2Bff0Hlo%3D&amp;reserved=0>.
>> +;
>> +%macro PVALIDATE       0
>> +    DB 0xF2, 0x0F, 0x01, 0xFF
>> +%endmacro
>> +
>>  ; NASM provides built-in macros STRUC and ENDSTRUC for structure definition.
>>  ; For example, to define a structure called mytype containing a longword,
>>  ; a word, a byte and a string of bytes, you might code

Sure, I will add the new macro and use it.


> (16c) Please replace the FIXME placeholder above with the actual NASM BZ
> number (from (16a)).
>
> (16d) In the "MdePkg/Library/BaseLib/X64/Pvalidate.nasm" source file,
> and also (later) in the "OvmfPkg/ResetVector/Ia32/PageTables64.asm"
> source file, please use the PVALIDATE macro, in place of the naked DBs.
>
>
> Back to your patch:
>
> On 04/30/21 13:51, Brijesh Singh wrote:
>> +
>> +  ; Read the Eflags
>> +  pushfq
>> +  pop     r8
>> +  mov     [r9], r8
>> +
>> +  ; The PVALIDATE instruction returns the status in rax register.
>> +  ret
>>
> (17) The assembly code should be updated to match the new interface
> contract (parameter order, parameter types, return values).

Sure, I will update based on your feedback.


>
> I'll continue reviewing the series later this week (hopefully tomorrow).


thanks for all your review feedback.