Re: [edk2-devel] [PATCH v3 04/10] UefiCpuPkg: add CPU ejection support

["Ankur Arora" <ankur.a.arora@oracle.com>]

On 2021-01-15 12:17 a.m., Laszlo Ersek wrote:
> Hi Ankur,
> 
> On 01/15/21 08:45, Ankur Arora wrote:
>> Define CPU_HOT_EJECT_DATA and add PCD PcdCpuHotEjectDataAddress,
>> which would be used to share CPU ejection state between
>> PiSmmCpuDxeSmm and OvmfPkg/CpuHotPlugSmm.
>>
>> Cc: Eric Dong <eric.dong@intel.com>
>> Cc: Ray Ni <ray.ni@intel.com>
>> Cc: Laszlo Ersek <lersek@redhat.com>
>> Cc: Rahul Kumar <rahul1.kumar@intel.com>
>> Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
>> Cc: Aaron Young <aaron.young@oracle.com>
>> Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3132
>> Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com>
>> ---
>>   UefiCpuPkg/Include/CpuHotPlugData.h          | 21 +++++++++++++++++++++
>>   UefiCpuPkg/PiSmmCpuDxeSmm/PiSmmCpuDxeSmm.inf |  1 +
>>   UefiCpuPkg/UefiCpuPkg.dec                    |  5 +++++
>>   UefiCpuPkg/UefiCpuPkg.uni                    |  4 ++++
>>   4 files changed, 31 insertions(+)
>>
>> diff --git a/UefiCpuPkg/Include/CpuHotPlugData.h b/UefiCpuPkg/Include/CpuHotPlugData.h
>> index 6321a149fa52..86f964550655 100644
>> --- a/UefiCpuPkg/Include/CpuHotPlugData.h
>> +++ b/UefiCpuPkg/Include/CpuHotPlugData.h
>> @@ -2,6 +2,7 @@
>>   Definition for a structure sharing information for CPU hot plug.
>>   
>>   Copyright (c) 2013 - 2015, Intel Corporation. All rights reserved.<BR>
>> +Copyright (c) 2021, Oracle Corporation.<BR>
>>   SPDX-License-Identifier: BSD-2-Clause-Patent
>>   
>>   **/
>> @@ -24,4 +25,24 @@ typedef struct {
>>     UINT32    SmrrSize;
>>   } CPU_HOT_PLUG_DATA;
>>   
>> +typedef
>> +VOID
>> +(EFIAPI *CPU_HOT_EJECT_FN)(
>> +  IN UINTN  ProcessorNum
>> +  );
>> +
>> +#define CPU_EJECT_INVALID	(MAX_UINT64)
>> +#define CPU_EJECT_WORKER	(MAX_UINT64-1)
>> +
>> +#define  CPU_HOT_EJECT_DATA_REVISION_1         0x00000001
>> +
>> +typedef struct {
>> +  UINT32           Revision;          // Used for version identification for this structure
>> +  UINT32           ArrayLength;       // The entries number of the following ApicId array and SmBase array
>> +
>> +  UINT64           *ApicIdMap;        // Pointer to CpuIndex->ApicId map. Holds APIC IDs for pending ejects
>> +  CPU_HOT_EJECT_FN Handler;           // Handler for CPU ejection
>> +  UINT64           Reserved;
>> +} CPU_HOT_EJECT_DATA;
>> +
>>   #endif
> 
> I'm sorry, I still don't understand -- why is it necessary to modify
> UefiCpuPkg *at all*, for this feature?
> 
> (1) The structure type for the data exchange should be in an OvmfPkg
> header file.
> 
> (2) The dynamic PCD for transferring the address of the structure should
> be declared in the "OvmfPkg.dec" file.
> 
> (3) The "handler" call is made
> - from SmmCpuFeaturesRendezvousExit() in OvmfPkg/Library/SmmCpuFeaturesLib,
> - to CpuEject() in OvmfPkg/CpuHotplugSmm.
> 
> First, this call should not need a function pointer at all -- the
> CpuEject() logic should be flattened into
> OvmfPkg/Library/SmmCpuFeaturesLib).

Sure, it can be flattened -- but it's out of place in SmmCpuFeaturesLib.
All of the logic pertaining to the unplug is in CpuHotPlugSmm, so it seems to
make sense to locate the related ejection code along with it.

If you are concerned about paying the additional cost of an indirect call
then I think it should be possible to install the handler only when there
is an actual unplug to be done and remove it after.

> 
> Second, if that's not possible -- please explain why --, then a function
> pointer might be justified after all, but the information channel still
> seems to have zero relevance for UefiCpuPkg.

The reason for keeping the PCD in UefiCpuPkg was that its declaration and
init are in SmmCpuFeaturesLib which gets folded into the UefiCpuPkg/PiSmmCpuDxe
execution context and so the export happening via OvmfPkg.dec seemed off.

That said, I guess your underlying point is that this adds unnecessary state to
non OVMF builds (?), which it does, so I can move the PCD to OvmfPkg.dec.


Thanks
Ankur

> 
> It's possible I'm not seeing something; please explain.
> 
> Thanks
> Laszlo
> 
>> diff --git a/UefiCpuPkg/PiSmmCpuDxeSmm/PiSmmCpuDxeSmm.inf b/UefiCpuPkg/PiSmmCpuDxeSmm/PiSmmCpuDxeSmm.inf
>> index 76b146299679..f79c874d74f1 100644
>> --- a/UefiCpuPkg/PiSmmCpuDxeSmm/PiSmmCpuDxeSmm.inf
>> +++ b/UefiCpuPkg/PiSmmCpuDxeSmm/PiSmmCpuDxeSmm.inf
>> @@ -131,6 +131,7 @@
>>     gUefiCpuPkgTokenSpaceGuid.PcdCpuSmmApSyncTimeout                 ## CONSUMES
>>     gUefiCpuPkgTokenSpaceGuid.PcdCpuS3DataAddress                    ## SOMETIMES_CONSUMES
>>     gUefiCpuPkgTokenSpaceGuid.PcdCpuHotPlugDataAddress               ## SOMETIMES_PRODUCES
>> +  gUefiCpuPkgTokenSpaceGuid.PcdCpuHotEjectDataAddress              ## SOMETIMES_PRODUCES
>>     gUefiCpuPkgTokenSpaceGuid.PcdCpuSmmCodeAccessCheckEnable         ## CONSUMES
>>     gUefiCpuPkgTokenSpaceGuid.PcdCpuSmmSyncMode                      ## CONSUMES
>>     gUefiCpuPkgTokenSpaceGuid.PcdCpuSmmShadowStackSize               ## SOMETIMES_CONSUMES
>> diff --git a/UefiCpuPkg/UefiCpuPkg.dec b/UefiCpuPkg/UefiCpuPkg.dec
>> index d83c084467b3..704ccc05f662 100644
>> --- a/UefiCpuPkg/UefiCpuPkg.dec
>> +++ b/UefiCpuPkg/UefiCpuPkg.dec
>> @@ -339,6 +339,11 @@
>>     # @ValidList   0x80000001 | 0
>>     gUefiCpuPkgTokenSpaceGuid.PcdCpuHotPlugDataAddress|0x0|UINT64|0x60000011
>>   
>> +  ## Contains the pointer to a CPU Hot Eject Data structure if CPU hot-plug is supported.
>> +  # @Prompt The pointer to CPU Hot Eject Data.
>> +  # @ValidList   0x80000001 | 0
>> +  gUefiCpuPkgTokenSpaceGuid.PcdCpuHotEjectDataAddress|0x0|UINT64|0x60000017
>> +
>>     ## Indicates processor feature capabilities, each bit corresponding to a specific feature.
>>     # @Prompt Processor feature capabilities.
>>     # @ValidList   0x80000001 | 0
>> diff --git a/UefiCpuPkg/UefiCpuPkg.uni b/UefiCpuPkg/UefiCpuPkg.uni
>> index 219c1963bf08..b1721f256632 100644
>> --- a/UefiCpuPkg/UefiCpuPkg.uni
>> +++ b/UefiCpuPkg/UefiCpuPkg.uni
>> @@ -140,6 +140,10 @@
>>   
>>   #string STR_gUefiCpuPkgTokenSpaceGuid_PcdCpuHotPlugDataAddress_HELP  #language en-US "Contains the pointer to a CPU Hot Plug Data structure if CPU hot-plug is supported."
>>   
>> +#string STR_gUefiCpuPkgTokenSpaceGuid_PcdCpuHotEjectDataAddress_PROMPT  #language en-US "The pointer to CPU Hot Eject Data"
>> +
>> +#string STR_gUefiCpuPkgTokenSpaceGuid_PcdCpuHotEjectDataAddress_HELP  #language en-US "Contains the pointer to a CPU Hot Eject Data structure if CPU hot-plug is supported."
>> +
>>   #string STR_gUefiCpuPkgTokenSpaceGuid_PcdCpuNumberOfReservedVariableMtrrs_PROMPT  #language en-US "Number of reserved variable MTRRs"
>>   
>>   #string STR_gUefiCpuPkgTokenSpaceGuid_PcdCpuNumberOfReservedVariableMtrrs_HELP  #language en-US "Specifies the number of variable MTRRs reserved for OS use."
>>
>