[PATCH v7 09/10] OvmfPkg/CpuHotplugSmm: do actual CPU hot-eject

["Ankur Arora" <ankur.a.arora@oracle.com>]

Add logic in EjectCpu() to do the actual the CPU ejection.

On the BSP, ejection happens by first selecting the CPU via
its QemuSelector and then sending the QEMU "eject" command.
QEMU in-turn signals the remote VCPU thread which context-switches
the CPU out of the SMI handler.

Meanwhile the CPU being ejected, waits around in its holding
area until it is context-switched out. Note that it is possible
that a slow CPU gets ejected before it reaches the wait loop.
However, this would never happen before it has executed the
"AllCpusInSync" loop in SmiRendezvous().
It can mean that an ejected CPU does not execute code after
that point but given that the CPU state will be destroyed by
QEMU, the missed cleanup is no great loss.

Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Aaron Young <aaron.young@oracle.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3132
Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com>
---

Notes:
    Addresses the following reviewing comments from v6:
    (1) s/CpuEject/EjectCpu/g
    (2,2a,2c) Get rid of eject-worker and related.
    (2b,2d) Use the PlatformSmmBspElection() logic to find out IsBSP.
    (3,3b) Use CPU_HOT_EJECT_DATA->QemuSelector instead of ApicIdMap to
     do the actual ejection.
    (4,5a,5b) Fix the format etc in the final unplugged log message
    () Also as discussed elsewhere document the ordering requirements for
     mCpuHotEjectData->QemuSelector[] and mCpuHotEjectData->Handler.
    () [from patch 2] Move definition of QEMU_CPUHP_STAT_EJECTED to this
     patch.
    () s/QEMU_CPUHP_STAT_EJECTED/QEMU_CPUHP_STAT_EJECT/

 OvmfPkg/Include/IndustryStandard/QemuCpuHotplug.h  |   1 +
 OvmfPkg/CpuHotplugSmm/CpuHotplug.c                 | 122 +++++++++++++++++++--
 .../Library/SmmCpuFeaturesLib/SmmCpuFeaturesLib.c  |  31 ++++++
 3 files changed, 147 insertions(+), 7 deletions(-)

diff --git a/OvmfPkg/Include/IndustryStandard/QemuCpuHotplug.h b/OvmfPkg/Include/IndustryStandard/QemuCpuHotplug.h
index 2ec7a107a64d..d0e83102c13f 100644
--- a/OvmfPkg/Include/IndustryStandard/QemuCpuHotplug.h
+++ b/OvmfPkg/Include/IndustryStandard/QemuCpuHotplug.h
@@ -34,6 +34,7 @@
 #define QEMU_CPUHP_STAT_ENABLED                BIT0
 #define QEMU_CPUHP_STAT_INSERT                 BIT1
 #define QEMU_CPUHP_STAT_REMOVE                 BIT2
+#define QEMU_CPUHP_STAT_EJECT                  BIT3
 #define QEMU_CPUHP_STAT_FW_REMOVE              BIT4
 
 #define QEMU_CPUHP_RW_CMD_DATA               0x8
diff --git a/OvmfPkg/CpuHotplugSmm/CpuHotplug.c b/OvmfPkg/CpuHotplugSmm/CpuHotplug.c
index 5dcaca53d42f..b80d4a2283a4 100644
--- a/OvmfPkg/CpuHotplugSmm/CpuHotplug.c
+++ b/OvmfPkg/CpuHotplugSmm/CpuHotplug.c
@@ -18,6 +18,7 @@
 #include <Pcd/CpuHotEjectData.h>             // CPU_HOT_EJECT_DATA
 #include <Protocol/MmCpuIo.h>                // EFI_MM_CPU_IO_PROTOCOL
 #include <Protocol/SmmCpuService.h>          // EFI_SMM_CPU_SERVICE_PROTOCOL
+#include <Register/Intel/ArchitecturalMsr.h> // MSR_IA32_APIC_BASE_REGISTER
 #include <Uefi/UefiBaseType.h>               // EFI_STATUS
 
 #include "ApicId.h"                          // APIC_ID
@@ -191,12 +192,36 @@ RevokeNewSlot:
 }
 
 /**
+  EjectCpu needs to know the BSP at SMI exit at a point when
+  some of the EFI_SMM_CPU_SERVICE_PROTOCOL state has been torn
+  down.
+  Reuse the logic from OvmfPkg::PlatformSmmBspElection() to
+  do that.
+**/
+STATIC
+BOOLEAN
+CheckIfBsp (
+  IN UINTN ProcessorNum
+  )
+{
+  MSR_IA32_APIC_BASE_REGISTER ApicBaseMsr;
+  BOOLEAN IsBsp;
+
+  ApicBaseMsr.Uint64 = AsmReadMsr64 (MSR_IA32_APIC_BASE);
+  IsBsp = (BOOLEAN)(ApicBaseMsr.Bits.BSP == 1);
+  return IsBsp;
+}
+
+/**
   CPU Hot-eject handler, called from SmmCpuFeaturesRendezvousExit()
   on each CPU at exit from SMM.
 
-  If, the executing CPU is not being ejected, nothing to be done.
+  If, the executing CPU is neither the BSP, nor being ejected, nothing
+  to be done.
   If, the executing CPU is being ejected, wait in a halted loop
   until ejected.
+  If, the executing CPU is the BSP, set QEMU CPU status to eject
+  for CPUs being ejected.
 
   @param[in] ProcessorNum      ProcessorNum denotes the CPU exiting SMM,
                                and will be used as an index into
@@ -211,9 +236,97 @@ EjectCpu (
   )
 {
   UINT64 QemuSelector;
+  BOOLEAN IsBsp = CheckIfBsp (ProcessorNum);
 
+  //
+  // mCpuHotEjectData->QemuSelectorMap[ProcessorNum] is updated
+  // on the BSP in the ongoing SMI iteration at two places:
+  //
+  // - UnplugCpus() where the BSP determines if a CPU is under ejection
+  //   or not. As the comment where mCpuHotEjectData->Handler is set-up
+  //   describes any such updates are guaranteed to be ordered-before the
+  //   dereference below.
+  //
+  // - EjectCpu() on the BSP updates QemuSelectorMap[ProcessorNum] for
+  //   CPUs after they have been hot-ejected.
+  //
+  //   The CPU under ejection: might be executing anywhere between the
+  //   "AllCpusInSync" exit loop in SmiRendezvous() to about to
+  //   dereference QemuSelectorMap[ProcessorNum].
+  //   Given that the BSP ensures that this store only happens after the
+  //   CPU has been ejected, this CPU would never see the after value.
+  //   (Note that any CPU that is already executing the CpuSleep() loop
+  //   below never raced any updates and always saw the before value.)
+  //
+  //   CPUs not-under ejection: never see any changes so they are fine.
+  //
+  //   Lastly, note that we are also guaranteed that any dereferencing
+  //   CPU only sees the before or after value and not an intermediate
+  //   value. This is because QemuSelectorMap[ProcessorNum] is aligned at
+  //   a natural boundary.
+  //
   QemuSelector = mCpuHotEjectData->QemuSelectorMap[ProcessorNum];
-  if (QemuSelector == CPU_EJECT_QEMU_SELECTOR_INVALID) {
+  if (QemuSelector == CPU_EJECT_QEMU_SELECTOR_INVALID && !IsBsp) {
+    return;
+  }
+
+  if (IsBsp) {
+    UINT32 Idx;
+
+    for (Idx = 0; Idx < mCpuHotEjectData->ArrayLength; Idx++) {
+      UINT64 QemuSelector;
+
+      QemuSelector = mCpuHotEjectData->QemuSelectorMap[Idx];
+
+      if (QemuSelector != CPU_EJECT_QEMU_SELECTOR_INVALID) {
+        //
+        // This to-be-ejected-CPU has already received the BSP's SMI exit
+        // signal and, will execute SmmCpuFeaturesRendezvousExit()
+        // followed by this callback or is already waiting in the
+        // CpuSleep() loop below.
+        //
+        // Tell QEMU to context-switch it out.
+        //
+        QemuCpuhpWriteCpuSelector (mMmCpuIo, (UINT32) QemuSelector);
+        QemuCpuhpWriteCpuStatus (mMmCpuIo, QEMU_CPUHP_STAT_EJECT);
+
+        //
+        // We need a compiler barrier here to ensure that the compiler
+        // does not reorder the CpuStatus and QemuSelectorMap[Idx] stores.
+        //
+        // A store fence is not strictly necessary on x86 which has
+        // TSO; however, both of these stores are in different address spaces
+        // so also add a Store Fence here.
+        //
+        MemoryFence ();
+
+        //
+        // Clear the eject status for this CPU Idx to ensure that an invalid
+        // SMI later does not end up trying to eject it or a newly
+        // hotplugged CPU Idx does not go into the dead loop.
+        //
+        mCpuHotEjectData->QemuSelectorMap[Idx] =
+          CPU_EJECT_QEMU_SELECTOR_INVALID;
+
+        DEBUG ((DEBUG_INFO, "%a: Unplugged ProcessorNum %u, "
+          "QemuSelector 0x%Lx\n", __FUNCTION__, Idx, QemuSelector));
+      }
+    }
+
+    //
+    // We are done until the next hot-unplug; clear the handler.
+    //
+    // By virtue of the MemoryFence() in the ejection loop above, the
+    // following store is ordered-after all the ejections are done.
+    // (We know that there is at least one CPU hot-eject handler if this
+    // handler was installed.)
+    //
+    // As described in OvmfPkg::SmmCpuFeaturesRendezvousExit() this
+    // means that the only CPUs which might dereference
+    // mCpuHotEjectData->Handler are not under ejection, so we can
+    // safely reset.
+    //
+    mCpuHotEjectData->Handler = NULL;
     return;
   }
 
@@ -496,11 +609,6 @@ CpuHotplugMmi (
   if (EFI_ERROR (Status)) {
     goto Fatal;
   }
-  if (ToUnplugCount > 0) {
-    DEBUG ((DEBUG_ERROR, "%a: hot-unplug is not supported yet\n",
-      __FUNCTION__));
-    goto Fatal;
-  }
 
   if (PluggedCount > 0) {
     Status = ProcessHotAddedCpus (mPluggedApicIds, PluggedCount);
diff --git a/OvmfPkg/Library/SmmCpuFeaturesLib/SmmCpuFeaturesLib.c b/OvmfPkg/Library/SmmCpuFeaturesLib/SmmCpuFeaturesLib.c
index c5cfe2610c23..0345466ac5a5 100644
--- a/OvmfPkg/Library/SmmCpuFeaturesLib/SmmCpuFeaturesLib.c
+++ b/OvmfPkg/Library/SmmCpuFeaturesLib/SmmCpuFeaturesLib.c
@@ -472,6 +472,37 @@ SmmCpuFeaturesRendezvousExit (
   // (PcdCpuMaxLogicalProcessorNumber > 1), and hot-eject is needed
   // in this SMI exit (otherwise mCpuHotEjectData->Handler is not armed.)
   //
+  // mCpuHotEjectData itself is stable once setup so it can be
+  // dereferenced without needing any synchronization,
+  // but, mCpuHotEjectData->Handler is updated on the BSP in the
+  // ongoing SMI iteration at two places:
+  //
+  // - UnplugCpus() where the BSP determines if a CPU is under ejection
+  //   or not. As the comment where mCpuHotEjectData->Handler is set-up
+  //   describes any such updates are guaranteed to be ordered-before the
+  //   dereference below.
+  //
+  // - EjectCpu() (which is called via the Handler below), on the BSP
+  //   updates mCpuHotEjectData->Handler once it is done with all ejections.
+  //
+  //   The CPU under ejection: might be executing anywhere between the
+  //   "AllCpusInSync" exit loop in SmiRendezvous() to about to
+  //   dereference the Handler field.
+  //   Given that the BSP ensures that this store only happens after all
+  //   CPUs under ejection have been ejected, this CPU would never see
+  //   the after value.
+  //   (Note that any CPU that is already executing the CpuSleep() loop
+  //   below never raced any updates and always saw the before value.)
+  //
+  //   CPUs not-under ejection: might see either value of the Handler
+  //   which is fine, because the Handler is a NOP for CPUs not-under
+  //   ejection.
+  //
+  //   Lastly, note that we are also guaranteed that any dereferencing
+  //   CPU only sees the before or after value and not an intermediate
+  //   value. This is because mCpuHotEjectData->Handler is aligned at a
+  //   natural boundary.
+  //
 
   if (mCpuHotEjectData != NULL) {
     CPU_HOT_EJECT_HANDLER Handler = mCpuHotEjectData->Handler;
-- 
2.9.3