* [PATCH 01/16] MdeModulePkg/PiSmmCore: log SMM image start failure
2020-02-23 17:25 [PATCH 00/16] OvmfPkg: support VCPU hotplug with -D SMM_REQUIRE Laszlo Ersek
@ 2020-02-23 17:25 ` Laszlo Ersek
2020-02-23 17:25 ` [PATCH 02/16] UefiCpuPkg/PiSmmCpuDxeSmm: fix S3 Resume for CPU hotplug Laszlo Ersek
` (16 subsequent siblings)
17 siblings, 0 replies; 24+ messages in thread
From: Laszlo Ersek @ 2020-02-23 17:25 UTC (permalink / raw)
To: edk2-devel-groups-io
Cc: Ard Biesheuvel, Eric Dong, Hao A Wu, Igor Mammedov, Jian J Wang,
Jiewen Yao, Jordan Justen, Michael Kinney,
Philippe Mathieu-Daudé, Ray Ni
In the CoreStartImage() function [MdeModulePkg/Core/Dxe/Image/Image.c], if
the image entry point returns a failure code, then the DXE Core logs a
helpful DEBUG_ERROR message, with the following format string:
"Error: Image at %11p start failed: %r\n"
Do similarly in the SMM Core (update the message slightly).
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Eric Dong <eric.dong@intel.com>
Cc: Hao A Wu <hao.a.wu@intel.com>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Jian J Wang <jian.j.wang@intel.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Michael Kinney <michael.d.kinney@intel.com>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Cc: Ray Ni <ray.ni@intel.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=1512
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
---
MdeModulePkg/Core/PiSmmCore/Dispatcher.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/MdeModulePkg/Core/PiSmmCore/Dispatcher.c b/MdeModulePkg/Core/PiSmmCore/Dispatcher.c
index 9bec731e5312..76ee9e0b89cc 100644
--- a/MdeModulePkg/Core/PiSmmCore/Dispatcher.c
+++ b/MdeModulePkg/Core/PiSmmCore/Dispatcher.c
@@ -883,44 +883,50 @@ SmmDispatcher (
RemoveEntryList (&DriverEntry->ScheduledLink);
REPORT_STATUS_CODE_WITH_EXTENDED_DATA (
EFI_PROGRESS_CODE,
EFI_SOFTWARE_SMM_DRIVER | EFI_SW_PC_INIT_BEGIN,
&DriverEntry->ImageHandle,
sizeof (DriverEntry->ImageHandle)
);
//
// Cache state of SmmEntryPointRegistered before calling entry point
//
PreviousSmmEntryPointRegistered = gSmmCorePrivate->SmmEntryPointRegistered;
//
// For each SMM driver, pass NULL as ImageHandle
//
RegisterSmramProfileImage (DriverEntry, TRUE);
PERF_START_IMAGE_BEGIN (DriverEntry->ImageHandle);
Status = ((EFI_IMAGE_ENTRY_POINT)(UINTN)DriverEntry->ImageEntryPoint)(DriverEntry->ImageHandle, gST);
PERF_START_IMAGE_END (DriverEntry->ImageHandle);
if (EFI_ERROR(Status)){
+ DEBUG ((
+ DEBUG_ERROR,
+ "Error: SMM image at %11p start failed: %r\n",
+ DriverEntry->SmmLoadedImage.ImageBase,
+ Status
+ ));
UnregisterSmramProfileImage (DriverEntry, TRUE);
SmmFreePages(DriverEntry->ImageBuffer, DriverEntry->NumberOfPage);
//
// Uninstall LoadedImage
//
Status = gBS->UninstallProtocolInterface (
DriverEntry->ImageHandle,
&gEfiLoadedImageProtocolGuid,
DriverEntry->LoadedImage
);
if (!EFI_ERROR (Status)) {
if (DriverEntry->LoadedImage->FilePath != NULL) {
gBS->FreePool (DriverEntry->LoadedImage->FilePath);
}
gBS->FreePool (DriverEntry->LoadedImage);
}
Status = SmmUninstallProtocolInterface (
DriverEntry->SmmImageHandle,
&gEfiLoadedImageProtocolGuid,
&DriverEntry->SmmLoadedImage
);
if (!EFI_ERROR(Status)) {
--
2.19.1.3.g30247aa5d201
^ permalink raw reply related [flat|nested] 24+ messages in thread* [PATCH 02/16] UefiCpuPkg/PiSmmCpuDxeSmm: fix S3 Resume for CPU hotplug
2020-02-23 17:25 [PATCH 00/16] OvmfPkg: support VCPU hotplug with -D SMM_REQUIRE Laszlo Ersek
2020-02-23 17:25 ` [PATCH 01/16] MdeModulePkg/PiSmmCore: log SMM image start failure Laszlo Ersek
@ 2020-02-23 17:25 ` Laszlo Ersek
2020-02-23 17:25 ` [PATCH 03/16] OvmfPkg: clone SmmCpuPlatformHookLib from UefiCpuPkg Laszlo Ersek
` (15 subsequent siblings)
17 siblings, 0 replies; 24+ messages in thread
From: Laszlo Ersek @ 2020-02-23 17:25 UTC (permalink / raw)
To: edk2-devel-groups-io
Cc: Ard Biesheuvel, Eric Dong, Igor Mammedov, Jiewen Yao,
Jordan Justen, Michael Kinney, Philippe Mathieu-Daudé,
Ray Ni
The "ACPI_CPU_DATA.NumberOfCpus" field is specified as follows, in
"UefiCpuPkg/Include/AcpiCpuData.h" (rewrapped for this commit message):
//
// The number of CPUs. If a platform does not support hot plug CPUs,
// then this is the number of CPUs detected when the platform is booted,
// regardless of being enabled or disabled. If a platform does support
// hot plug CPUs, then this is the maximum number of CPUs that the
// platform supports.
//
The InitializeCpuBeforeRebase() and InitializeCpuAfterRebase() functions
in "UefiCpuPkg/PiSmmCpuDxeSmm/CpuS3.c" try to restore CPU configuration on
the S3 Resume path for *all* CPUs accounted for in
"ACPI_CPU_DATA.NumberOfCpus". This is wrong, as with CPU hotplug, not all
of the possible CPUs may be present at the time of S3 Suspend / Resume.
The symptom is an infinite wait.
Instead, the "mNumberOfCpus" variable should be used, which is properly
maintained through the EFI_SMM_CPU_SERVICE_PROTOCOL implementation (see
SmmAddProcessor(), SmmRemoveProcessor(), SmmCpuUpdate() in
"UefiCpuPkg/PiSmmCpuDxeSmm/CpuService.c").
When CPU hotplug is disabled, "mNumberOfCpus" is constant, and equals
"ACPI_CPU_DATA.NumberOfCpus" at all times.
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Eric Dong <eric.dong@intel.com>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Michael Kinney <michael.d.kinney@intel.com>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Cc: Ray Ni <ray.ni@intel.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=1512
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
---
UefiCpuPkg/PiSmmCpuDxeSmm/CpuS3.c | 14 ++++++++++++--
1 file changed, 12 insertions(+), 2 deletions(-)
diff --git a/UefiCpuPkg/PiSmmCpuDxeSmm/CpuS3.c b/UefiCpuPkg/PiSmmCpuDxeSmm/CpuS3.c
index ba5cc0194c2d..1e0840119724 100644
--- a/UefiCpuPkg/PiSmmCpuDxeSmm/CpuS3.c
+++ b/UefiCpuPkg/PiSmmCpuDxeSmm/CpuS3.c
@@ -597,75 +597,85 @@ PrepareApStartupVector (
}
/**
The function is invoked before SMBASE relocation in S3 path to restores CPU status.
The function is invoked before SMBASE relocation in S3 path. It does first time microcode load
and restores MTRRs for both BSP and APs.
**/
VOID
InitializeCpuBeforeRebase (
VOID
)
{
LoadMtrrData (mAcpiCpuData.MtrrTable);
SetRegister (TRUE);
ProgramVirtualWireMode ();
PrepareApStartupVector (mAcpiCpuData.StartupVector);
- mNumberToFinish = mAcpiCpuData.NumberOfCpus - 1;
+ if (FeaturePcdGet (PcdCpuHotPlugSupport)) {
+ ASSERT (mNumberOfCpus <= mAcpiCpuData.NumberOfCpus);
+ } else {
+ ASSERT (mNumberOfCpus == mAcpiCpuData.NumberOfCpus);
+ }
+ mNumberToFinish = mNumberOfCpus - 1;
mExchangeInfo->ApFunction = (VOID *) (UINTN) InitializeAp;
//
// Execute code for before SmmBaseReloc. Note: This flag is maintained across S3 boots.
//
mInitApsAfterSmmBaseReloc = FALSE;
//
// Send INIT IPI - SIPI to all APs
//
SendInitSipiSipiAllExcludingSelf ((UINT32)mAcpiCpuData.StartupVector);
while (mNumberToFinish > 0) {
CpuPause ();
}
}
/**
The function is invoked after SMBASE relocation in S3 path to restores CPU status.
The function is invoked after SMBASE relocation in S3 path. It restores configuration according to
data saved by normal boot path for both BSP and APs.
**/
VOID
InitializeCpuAfterRebase (
VOID
)
{
- mNumberToFinish = mAcpiCpuData.NumberOfCpus - 1;
+ if (FeaturePcdGet (PcdCpuHotPlugSupport)) {
+ ASSERT (mNumberOfCpus <= mAcpiCpuData.NumberOfCpus);
+ } else {
+ ASSERT (mNumberOfCpus == mAcpiCpuData.NumberOfCpus);
+ }
+ mNumberToFinish = mNumberOfCpus - 1;
//
// Signal that SMM base relocation is complete and to continue initialization for all APs.
//
mInitApsAfterSmmBaseReloc = TRUE;
//
// Must begin set register after all APs have continue their initialization.
// This is a requirement to support semaphore mechanism in register table.
// Because if semaphore's dependence type is package type, semaphore will wait
// for all Aps in one package finishing their tasks before set next register
// for all APs. If the Aps not begin its task during BSP doing its task, the
// BSP thread will hang because it is waiting for other Aps in the same
// package finishing their task.
//
SetRegister (FALSE);
while (mNumberToFinish > 0) {
CpuPause ();
}
}
--
2.19.1.3.g30247aa5d201
^ permalink raw reply related [flat|nested] 24+ messages in thread* [PATCH 03/16] OvmfPkg: clone SmmCpuPlatformHookLib from UefiCpuPkg
2020-02-23 17:25 [PATCH 00/16] OvmfPkg: support VCPU hotplug with -D SMM_REQUIRE Laszlo Ersek
2020-02-23 17:25 ` [PATCH 01/16] MdeModulePkg/PiSmmCore: log SMM image start failure Laszlo Ersek
2020-02-23 17:25 ` [PATCH 02/16] UefiCpuPkg/PiSmmCpuDxeSmm: fix S3 Resume for CPU hotplug Laszlo Ersek
@ 2020-02-23 17:25 ` Laszlo Ersek
2020-02-23 17:25 ` [PATCH 04/16] OvmfPkg: enable SMM Monarch Election in PiSmmCpuDxeSmm Laszlo Ersek
` (14 subsequent siblings)
17 siblings, 0 replies; 24+ messages in thread
From: Laszlo Ersek @ 2020-02-23 17:25 UTC (permalink / raw)
To: edk2-devel-groups-io
Cc: Ard Biesheuvel, Igor Mammedov, Jiewen Yao, Jordan Justen,
Michael Kinney, Philippe Mathieu-Daudé
Clone the Null instance of SmmCpuPlatformHookLib from UefiCpuPkg to
OvmfPkg. In this patch, customize the lib instance only with the following
no-op steps:
- Replace Null/NULL references in filenames and comments with Qemu/QEMU
references.
- Update copyright notices.
- Clean up and rewrap comment blocks.
- Update INF_VERSION to the latest INF spec version (1.29).
- Update FILE_GUID.
- Drop the UNI file.
This patch is best reviewed with:
$ git show --find-copies=43 --find-copies-harder
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Michael Kinney <michael.d.kinney@intel.com>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=1512
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
---
OvmfPkg/OvmfPkgIa32.dsc | 2 +-
OvmfPkg/OvmfPkgIa32X64.dsc | 2 +-
OvmfPkg/OvmfPkgX64.dsc | 2 +-
UefiCpuPkg/Library/SmmCpuPlatformHookLibNull/SmmCpuPlatformHookLibNull.inf => OvmfPkg/Library/SmmCpuPlatformHookLibQemu/SmmCpuPlatformHookLibQemu.inf | 21 +++++-------
UefiCpuPkg/Library/SmmCpuPlatformHookLibNull/SmmCpuPlatformHookLibNull.c => OvmfPkg/Library/SmmCpuPlatformHookLibQemu/SmmCpuPlatformHookLibQemu.c | 36 ++++++++++++--------
5 files changed, 32 insertions(+), 31 deletions(-)
diff --git a/OvmfPkg/OvmfPkgIa32.dsc b/OvmfPkg/OvmfPkgIa32.dsc
index 19728f20b34e..813995fefad8 100644
--- a/OvmfPkg/OvmfPkgIa32.dsc
+++ b/OvmfPkg/OvmfPkgIa32.dsc
@@ -858,45 +858,45 @@ [Components]
UefiCpuPkg/CpuS3DataDxe/CpuS3DataDxe.inf
#
# SMM Initial Program Load (a DXE_RUNTIME_DRIVER)
#
MdeModulePkg/Core/PiSmmCore/PiSmmIpl.inf
#
# SMM_CORE
#
MdeModulePkg/Core/PiSmmCore/PiSmmCore.inf
#
# Privileged drivers (DXE_SMM_DRIVER modules)
#
UefiCpuPkg/CpuIo2Smm/CpuIo2Smm.inf
MdeModulePkg/Universal/LockBox/SmmLockBox/SmmLockBox.inf {
<LibraryClasses>
LockBoxLib|MdeModulePkg/Library/SmmLockBoxLib/SmmLockBoxSmmLib.inf
}
UefiCpuPkg/PiSmmCpuDxeSmm/PiSmmCpuDxeSmm.inf {
<LibraryClasses>
- SmmCpuPlatformHookLib|UefiCpuPkg/Library/SmmCpuPlatformHookLibNull/SmmCpuPlatformHookLibNull.inf
+ SmmCpuPlatformHookLib|OvmfPkg/Library/SmmCpuPlatformHookLibQemu/SmmCpuPlatformHookLibQemu.inf
SmmCpuFeaturesLib|OvmfPkg/Library/SmmCpuFeaturesLib/SmmCpuFeaturesLib.inf
}
#
# Variable driver stack (SMM)
#
OvmfPkg/QemuFlashFvbServicesRuntimeDxe/FvbServicesSmm.inf
MdeModulePkg/Universal/FaultTolerantWriteDxe/FaultTolerantWriteSmm.inf
MdeModulePkg/Universal/Variable/RuntimeDxe/VariableSmm.inf {
<LibraryClasses>
NULL|MdeModulePkg/Library/VarCheckUefiLib/VarCheckUefiLib.inf
}
MdeModulePkg/Universal/Variable/RuntimeDxe/VariableSmmRuntimeDxe.inf
!else
#
# Variable driver stack (non-SMM)
#
OvmfPkg/QemuFlashFvbServicesRuntimeDxe/FvbServicesRuntimeDxe.inf
OvmfPkg/EmuVariableFvbRuntimeDxe/Fvb.inf {
<LibraryClasses>
diff --git a/OvmfPkg/OvmfPkgIa32X64.dsc b/OvmfPkg/OvmfPkgIa32X64.dsc
index 3c0c229e3a72..a256c7084a7e 100644
--- a/OvmfPkg/OvmfPkgIa32X64.dsc
+++ b/OvmfPkg/OvmfPkgIa32X64.dsc
@@ -872,45 +872,45 @@ [Components.X64]
UefiCpuPkg/CpuS3DataDxe/CpuS3DataDxe.inf
#
# SMM Initial Program Load (a DXE_RUNTIME_DRIVER)
#
MdeModulePkg/Core/PiSmmCore/PiSmmIpl.inf
#
# SMM_CORE
#
MdeModulePkg/Core/PiSmmCore/PiSmmCore.inf
#
# Privileged drivers (DXE_SMM_DRIVER modules)
#
UefiCpuPkg/CpuIo2Smm/CpuIo2Smm.inf
MdeModulePkg/Universal/LockBox/SmmLockBox/SmmLockBox.inf {
<LibraryClasses>
LockBoxLib|MdeModulePkg/Library/SmmLockBoxLib/SmmLockBoxSmmLib.inf
}
UefiCpuPkg/PiSmmCpuDxeSmm/PiSmmCpuDxeSmm.inf {
<LibraryClasses>
- SmmCpuPlatformHookLib|UefiCpuPkg/Library/SmmCpuPlatformHookLibNull/SmmCpuPlatformHookLibNull.inf
+ SmmCpuPlatformHookLib|OvmfPkg/Library/SmmCpuPlatformHookLibQemu/SmmCpuPlatformHookLibQemu.inf
SmmCpuFeaturesLib|OvmfPkg/Library/SmmCpuFeaturesLib/SmmCpuFeaturesLib.inf
}
#
# Variable driver stack (SMM)
#
OvmfPkg/QemuFlashFvbServicesRuntimeDxe/FvbServicesSmm.inf
MdeModulePkg/Universal/FaultTolerantWriteDxe/FaultTolerantWriteSmm.inf
MdeModulePkg/Universal/Variable/RuntimeDxe/VariableSmm.inf {
<LibraryClasses>
NULL|MdeModulePkg/Library/VarCheckUefiLib/VarCheckUefiLib.inf
}
MdeModulePkg/Universal/Variable/RuntimeDxe/VariableSmmRuntimeDxe.inf
!else
#
# Variable driver stack (non-SMM)
#
OvmfPkg/QemuFlashFvbServicesRuntimeDxe/FvbServicesRuntimeDxe.inf
OvmfPkg/EmuVariableFvbRuntimeDxe/Fvb.inf {
<LibraryClasses>
diff --git a/OvmfPkg/OvmfPkgX64.dsc b/OvmfPkg/OvmfPkgX64.dsc
index f6c1d8d228c6..78079b9f8e13 100644
--- a/OvmfPkg/OvmfPkgX64.dsc
+++ b/OvmfPkg/OvmfPkgX64.dsc
@@ -870,45 +870,45 @@ [Components]
UefiCpuPkg/CpuS3DataDxe/CpuS3DataDxe.inf
#
# SMM Initial Program Load (a DXE_RUNTIME_DRIVER)
#
MdeModulePkg/Core/PiSmmCore/PiSmmIpl.inf
#
# SMM_CORE
#
MdeModulePkg/Core/PiSmmCore/PiSmmCore.inf
#
# Privileged drivers (DXE_SMM_DRIVER modules)
#
UefiCpuPkg/CpuIo2Smm/CpuIo2Smm.inf
MdeModulePkg/Universal/LockBox/SmmLockBox/SmmLockBox.inf {
<LibraryClasses>
LockBoxLib|MdeModulePkg/Library/SmmLockBoxLib/SmmLockBoxSmmLib.inf
}
UefiCpuPkg/PiSmmCpuDxeSmm/PiSmmCpuDxeSmm.inf {
<LibraryClasses>
- SmmCpuPlatformHookLib|UefiCpuPkg/Library/SmmCpuPlatformHookLibNull/SmmCpuPlatformHookLibNull.inf
+ SmmCpuPlatformHookLib|OvmfPkg/Library/SmmCpuPlatformHookLibQemu/SmmCpuPlatformHookLibQemu.inf
SmmCpuFeaturesLib|OvmfPkg/Library/SmmCpuFeaturesLib/SmmCpuFeaturesLib.inf
}
#
# Variable driver stack (SMM)
#
OvmfPkg/QemuFlashFvbServicesRuntimeDxe/FvbServicesSmm.inf
MdeModulePkg/Universal/FaultTolerantWriteDxe/FaultTolerantWriteSmm.inf
MdeModulePkg/Universal/Variable/RuntimeDxe/VariableSmm.inf {
<LibraryClasses>
NULL|MdeModulePkg/Library/VarCheckUefiLib/VarCheckUefiLib.inf
}
MdeModulePkg/Universal/Variable/RuntimeDxe/VariableSmmRuntimeDxe.inf
!else
#
# Variable driver stack (non-SMM)
#
OvmfPkg/QemuFlashFvbServicesRuntimeDxe/FvbServicesRuntimeDxe.inf
OvmfPkg/EmuVariableFvbRuntimeDxe/Fvb.inf {
<LibraryClasses>
diff --git a/UefiCpuPkg/Library/SmmCpuPlatformHookLibNull/SmmCpuPlatformHookLibNull.inf b/OvmfPkg/Library/SmmCpuPlatformHookLibQemu/SmmCpuPlatformHookLibQemu.inf
similarity index 43%
copy from UefiCpuPkg/Library/SmmCpuPlatformHookLibNull/SmmCpuPlatformHookLibNull.inf
copy to OvmfPkg/Library/SmmCpuPlatformHookLibQemu/SmmCpuPlatformHookLibQemu.inf
index fab6b30b7a3f..82edeca3d12d 100644
--- a/UefiCpuPkg/Library/SmmCpuPlatformHookLibNull/SmmCpuPlatformHookLibNull.inf
+++ b/OvmfPkg/Library/SmmCpuPlatformHookLibQemu/SmmCpuPlatformHookLibQemu.inf
@@ -1,34 +1,29 @@
## @file
-# SMM CPU Platform Hook NULL library instance.
+# SMM CPU Platform Hook library instance for QEMU.
#
+# Copyright (c) 2020, Red Hat, Inc.
# Copyright (c) 2006 - 2015, Intel Corporation. All rights reserved.<BR>
# SPDX-License-Identifier: BSD-2-Clause-Patent
-#
##
-################################################################################
-#
-# Defines Section - statements that will be processed to create a Makefile.
-#
-################################################################################
[Defines]
- INF_VERSION = 0x00010005
- BASE_NAME = SmmCpuPlatformHookLibNull
- MODULE_UNI_FILE = SmmCpuPlatformHookLibNull.uni
- FILE_GUID = D6494E1B-E06F-4ab5-B64D-48B25AA9EB33
+ INF_VERSION = 1.29
+ BASE_NAME = SmmCpuPlatformHookLibQemu
+ FILE_GUID = 154D6D26-54B8-45BC-BA3A-CBAA20C02A6A
MODULE_TYPE = DXE_DRIVER
VERSION_STRING = 1.0
LIBRARY_CLASS = SmmCpuPlatformHookLib
#
-# The following information is for reference only and not required by the build tools.
+# The following information is for reference only and not required by the build
+# tools.
#
# VALID_ARCHITECTURES = IA32 X64
#
[Sources]
- SmmCpuPlatformHookLibNull.c
+ SmmCpuPlatformHookLibQemu.c
[Packages]
MdePkg/MdePkg.dec
UefiCpuPkg/UefiCpuPkg.dec
diff --git a/UefiCpuPkg/Library/SmmCpuPlatformHookLibNull/SmmCpuPlatformHookLibNull.c b/OvmfPkg/Library/SmmCpuPlatformHookLibQemu/SmmCpuPlatformHookLibQemu.c
similarity index 67%
copy from UefiCpuPkg/Library/SmmCpuPlatformHookLibNull/SmmCpuPlatformHookLibNull.c
copy to OvmfPkg/Library/SmmCpuPlatformHookLibQemu/SmmCpuPlatformHookLibQemu.c
index 6c2010dc0a67..257e1d399cc6 100644
--- a/UefiCpuPkg/Library/SmmCpuPlatformHookLibNull/SmmCpuPlatformHookLibNull.c
+++ b/OvmfPkg/Library/SmmCpuPlatformHookLibQemu/SmmCpuPlatformHookLibQemu.c
@@ -1,102 +1,108 @@
/** @file
-SMM CPU Platform Hook NULL library instance.
+SMM CPU Platform Hook library instance for QEMU.
+Copyright (c) 2020, Red Hat, Inc.
Copyright (c) 2006 - 2015, Intel Corporation. All rights reserved.<BR>
SPDX-License-Identifier: BSD-2-Clause-Patent
**/
#include <PiSmm.h>
#include <Library/SmmCpuPlatformHookLib.h>
/**
Checks if platform produces a valid SMI.
This function checks if platform produces a valid SMI. This function is
called at SMM entry to detect if this is a spurious SMI. This function
must be implemented in an MP safe way because it is called by multiple CPU
threads.
@retval TRUE There is a valid SMI
@retval FALSE There is no valid SMI
**/
BOOLEAN
EFIAPI
PlatformValidSmi (
VOID
)
{
return TRUE;
}
/**
Clears platform top level SMI status bit.
This function clears platform top level SMI status bit.
@retval TRUE The platform top level SMI status is cleared.
- @retval FALSE The platform top level SMI status cannot be cleared.
+ @retval FALSE The platform top level SMI status cannot be
+ cleared.
**/
BOOLEAN
EFIAPI
ClearTopLevelSmiStatus (
VOID
)
{
return TRUE;
}
/**
Performs platform specific way of SMM BSP election.
This function performs platform specific way of SMM BSP election.
- @param IsBsp Output parameter. TRUE: the CPU this function executes
- on is elected to be the SMM BSP. FALSE: the CPU this
- function executes on is to be SMM AP.
+ @param IsBsp Output parameter. TRUE: the CPU this function
+ executes on is elected to be the SMM BSP. FALSE:
+ the CPU this function executes on is to be SMM AP.
@retval EFI_SUCCESS The function executes successfully.
- @retval EFI_NOT_READY The function does not determine whether this CPU should be
- BSP or AP. This may occur if hardware init sequence to
- enable the determination is yet to be done, or the function
- chooses not to do BSP election and will let SMM CPU driver to
- use its default BSP election process.
- @retval EFI_DEVICE_ERROR The function cannot determine whether this CPU should be
- BSP or AP due to hardware error.
+ @retval EFI_NOT_READY The function does not determine whether this CPU
+ should be BSP or AP. This may occur if hardware
+ init sequence to enable the determination is yet to
+ be done, or the function chooses not to do BSP
+ election and will let SMM CPU driver to use its
+ default BSP election process.
+ @retval EFI_DEVICE_ERROR The function cannot determine whether this CPU
+ should be BSP or AP due to hardware error.
**/
EFI_STATUS
EFIAPI
PlatformSmmBspElection (
OUT BOOLEAN *IsBsp
)
{
return EFI_NOT_READY;
}
/**
Get platform page table attribute.
This function gets page table attribute of platform.
- @param Address Input parameter. Obtain the page table entries attribute on this address.
+ @param Address Input parameter. Obtain the page table entries
+ attribute on this address.
@param PageSize Output parameter. The size of the page.
@param NumOfPages Output parameter. Number of page.
@param PageAttribute Output parameter. Paging Attributes (WB, UC, etc).
- @retval EFI_SUCCESS The platform page table attribute from the address is determined.
- @retval EFI_UNSUPPORTED The platform does not support getting page table attribute for the address.
+ @retval EFI_SUCCESS The platform page table attribute from the address
+ is determined.
+ @retval EFI_UNSUPPORTED The platform does not support getting page table
+ attribute for the address.
**/
EFI_STATUS
EFIAPI
GetPlatformPageTableAttribute (
IN UINT64 Address,
IN OUT SMM_PAGE_SIZE_TYPE *PageSize,
IN OUT UINTN *NumOfPages,
IN OUT UINTN *PageAttribute
)
{
return EFI_UNSUPPORTED;
}
--
2.19.1.3.g30247aa5d201
^ permalink raw reply related [flat|nested] 24+ messages in thread* [PATCH 04/16] OvmfPkg: enable SMM Monarch Election in PiSmmCpuDxeSmm
2020-02-23 17:25 [PATCH 00/16] OvmfPkg: support VCPU hotplug with -D SMM_REQUIRE Laszlo Ersek
` (2 preceding siblings ...)
2020-02-23 17:25 ` [PATCH 03/16] OvmfPkg: clone SmmCpuPlatformHookLib from UefiCpuPkg Laszlo Ersek
@ 2020-02-23 17:25 ` Laszlo Ersek
2020-02-23 17:25 ` [PATCH 05/16] OvmfPkg: enable CPU hotplug support " Laszlo Ersek
` (13 subsequent siblings)
17 siblings, 0 replies; 24+ messages in thread
From: Laszlo Ersek @ 2020-02-23 17:25 UTC (permalink / raw)
To: edk2-devel-groups-io
Cc: Ard Biesheuvel, Igor Mammedov, Jiewen Yao, Jordan Justen,
Michael Kinney, Philippe Mathieu-Daudé
With "PcdCpuSmmEnableBspElection" set to FALSE, PiSmmCpuDxeSmm always
considers the processor with index 0 to be the SMM Monarch (a.k.a. the SMM
BSP). The SMM Monarch handles the SMI for real, while the other CPUs wait
in their SMM loops.
In a subsequent patch, we want to set "PcdCpuHotPlugSupport" to TRUE. For
that, PiCpuSmmEntry() [UefiCpuPkg/PiSmmCpuDxeSmm/PiSmmCpuDxeSmm.c] forces
us with an ASSERT() to set "PcdCpuSmmEnableBspElection" to TRUE as well.
To satisfy that expectation, we can simply remove our current
"PcdCpuSmmEnableBspElection|FALSE" setting, and inherit the default TRUE
value from "UefiCpuPkg.dec".
This causes "mSmmMpSyncData->BspIndex" in PiSmmCpuDxeSmm to lose its
static zero value (standing for CPU#0); instead it becomes (-1) in
general, and the SMM Monarch is elected anew on every SMI.
The default SMM Monarch Election is basically a race -- whichever CPU can
flip "mSmmMpSyncData->BspIndex" from (-1) to its own index, becomes king,
for handling that SMI. Refer to SmiRendezvous()
[UefiCpuPkg/PiSmmCpuDxeSmm/MpService.c].
I consider this non-determinism less than ideal on QEMU/KVM; it would be
nice to stick with a "mostly permanent" SMM Monarch even with the Election
enabled. We can do that by implementing the PlatformSmmBspElection() API
in the SmmCpuPlatformHookLibQemu instance:
The IA32 APIC Base MSR can be read on each CPU concurrently, and it will
report the BSP bit as set only on the current Boot Service Processor. QEMU
marks CPU#0 as the BSP, by default.
Elect the current BSP, as reported by QEMU, for the SMM Monarch role.
(Note that the QEMU commit history is not entirely consistent on whether
QEMU/KVM may mark a CPU with nonzero index as the BSP:
- At tag v4.2.0, "target/i386/cpu.c" has a comment saying "We hard-wire
the BSP to the first CPU". This comment goes back to commit 6cb2996cef5e
("x86: Extend validity of bsp_to_cpu", 2010-03-04).
- Compare commit 9cb11fd7539b ("target-i386: clear bsp bit when
designating bsp", 2015-04-02) though, especially considering KVM.
Either way, this OvmfPkg patch is *not* dependent on CPU index 0; it just
takes the race on every SMI out of the game.)
One benefit of using a "mostly permanent" SMM Monarch / BSP is that we can
continue testing the SMM CPU synchronization by deterministically entering
the firmware on the BSP, vs. on an AP, from Linux guests:
$ time taskset -c 0 efibootmgr
$ time taskset -c 1 efibootmgr
(See
<https://github.com/tianocore/tianocore.github.io/wiki/Testing-SMM-with-QEMU,-KVM-and-libvirt#uefi-variable-access-test>.)
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Michael Kinney <michael.d.kinney@intel.com>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Suggested-by: Igor Mammedov <imammedo@redhat.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=1512#c5
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
---
OvmfPkg/OvmfPkgIa32.dsc | 1 -
OvmfPkg/OvmfPkgIa32X64.dsc | 1 -
OvmfPkg/OvmfPkgX64.dsc | 1 -
OvmfPkg/Library/SmmCpuPlatformHookLibQemu/SmmCpuPlatformHookLibQemu.inf | 3 +++
OvmfPkg/Library/SmmCpuPlatformHookLibQemu/SmmCpuPlatformHookLibQemu.c | 9 ++++++++-
5 files changed, 11 insertions(+), 4 deletions(-)
diff --git a/OvmfPkg/OvmfPkgIa32.dsc b/OvmfPkg/OvmfPkgIa32.dsc
index 813995fefad8..60d8af185b9c 100644
--- a/OvmfPkg/OvmfPkgIa32.dsc
+++ b/OvmfPkg/OvmfPkgIa32.dsc
@@ -414,45 +414,44 @@ [LibraryClasses.common.SMM_CORE]
!endif
PciLib|OvmfPkg/Library/DxePciLibI440FxQ35/DxePciLibI440FxQ35.inf
################################################################################
#
# Pcd Section - list of all EDK II PCD Entries defined by this Platform.
#
################################################################################
[PcdsFeatureFlag]
gEfiMdeModulePkgTokenSpaceGuid.PcdHiiOsRuntimeSupport|FALSE
gEfiMdeModulePkgTokenSpaceGuid.PcdStatusCodeUseSerial|FALSE
gEfiMdeModulePkgTokenSpaceGuid.PcdStatusCodeUseMemory|TRUE
gEfiMdeModulePkgTokenSpaceGuid.PcdDxeIplSupportUefiDecompress|FALSE
gEfiMdeModulePkgTokenSpaceGuid.PcdDxeIplSwitchToLongMode|FALSE
gEfiMdeModulePkgTokenSpaceGuid.PcdConOutGopSupport|TRUE
gEfiMdeModulePkgTokenSpaceGuid.PcdConOutUgaSupport|FALSE
gEfiMdeModulePkgTokenSpaceGuid.PcdInstallAcpiSdtProtocol|TRUE
!ifdef $(CSM_ENABLE)
gUefiOvmfPkgTokenSpaceGuid.PcdCsmEnable|TRUE
!endif
!if $(SMM_REQUIRE) == TRUE
gUefiOvmfPkgTokenSpaceGuid.PcdSmmSmramRequire|TRUE
- gUefiCpuPkgTokenSpaceGuid.PcdCpuSmmEnableBspElection|FALSE
gEfiMdeModulePkgTokenSpaceGuid.PcdEnableVariableRuntimeCache|FALSE
!endif
[PcdsFixedAtBuild]
gEfiMdeModulePkgTokenSpaceGuid.PcdStatusCodeMemorySize|1
gEfiMdeModulePkgTokenSpaceGuid.PcdResetOnMemoryTypeInformationChange|FALSE
gEfiMdePkgTokenSpaceGuid.PcdMaximumGuidedExtractHandler|0x10
!if ($(FD_SIZE_IN_KB) == 1024) || ($(FD_SIZE_IN_KB) == 2048)
gEfiMdeModulePkgTokenSpaceGuid.PcdMaxVariableSize|0x2000
gEfiMdeModulePkgTokenSpaceGuid.PcdMaxAuthVariableSize|0x2800
!if $(NETWORK_TLS_ENABLE) == FALSE
# match PcdFlashNvStorageVariableSize purely for convenience
gEfiMdeModulePkgTokenSpaceGuid.PcdVariableStoreSize|0xe000
!endif
!endif
!if $(FD_SIZE_IN_KB) == 4096
gEfiMdeModulePkgTokenSpaceGuid.PcdMaxVariableSize|0x8400
gEfiMdeModulePkgTokenSpaceGuid.PcdMaxAuthVariableSize|0x8400
!if $(NETWORK_TLS_ENABLE) == FALSE
# match PcdFlashNvStorageVariableSize purely for convenience
gEfiMdeModulePkgTokenSpaceGuid.PcdVariableStoreSize|0x40000
!endif
diff --git a/OvmfPkg/OvmfPkgIa32X64.dsc b/OvmfPkg/OvmfPkgIa32X64.dsc
index a256c7084a7e..be6bc7bd88a7 100644
--- a/OvmfPkg/OvmfPkgIa32X64.dsc
+++ b/OvmfPkg/OvmfPkgIa32X64.dsc
@@ -419,45 +419,44 @@ [LibraryClasses.common.SMM_CORE]
!endif
PciLib|OvmfPkg/Library/DxePciLibI440FxQ35/DxePciLibI440FxQ35.inf
################################################################################
#
# Pcd Section - list of all EDK II PCD Entries defined by this Platform.
#
################################################################################
[PcdsFeatureFlag]
gEfiMdeModulePkgTokenSpaceGuid.PcdHiiOsRuntimeSupport|FALSE
gEfiMdeModulePkgTokenSpaceGuid.PcdStatusCodeUseSerial|FALSE
gEfiMdeModulePkgTokenSpaceGuid.PcdStatusCodeUseMemory|TRUE
gEfiMdeModulePkgTokenSpaceGuid.PcdDxeIplSupportUefiDecompress|FALSE
gEfiMdeModulePkgTokenSpaceGuid.PcdDxeIplSwitchToLongMode|TRUE
gEfiMdeModulePkgTokenSpaceGuid.PcdConOutGopSupport|TRUE
gEfiMdeModulePkgTokenSpaceGuid.PcdConOutUgaSupport|FALSE
gEfiMdeModulePkgTokenSpaceGuid.PcdInstallAcpiSdtProtocol|TRUE
!ifdef $(CSM_ENABLE)
gUefiOvmfPkgTokenSpaceGuid.PcdCsmEnable|TRUE
!endif
!if $(SMM_REQUIRE) == TRUE
gUefiOvmfPkgTokenSpaceGuid.PcdSmmSmramRequire|TRUE
- gUefiCpuPkgTokenSpaceGuid.PcdCpuSmmEnableBspElection|FALSE
gEfiMdeModulePkgTokenSpaceGuid.PcdEnableVariableRuntimeCache|FALSE
!endif
[PcdsFixedAtBuild]
gEfiMdeModulePkgTokenSpaceGuid.PcdStatusCodeMemorySize|1
gEfiMdeModulePkgTokenSpaceGuid.PcdResetOnMemoryTypeInformationChange|FALSE
gEfiMdePkgTokenSpaceGuid.PcdMaximumGuidedExtractHandler|0x10
!if ($(FD_SIZE_IN_KB) == 1024) || ($(FD_SIZE_IN_KB) == 2048)
gEfiMdeModulePkgTokenSpaceGuid.PcdMaxVariableSize|0x2000
gEfiMdeModulePkgTokenSpaceGuid.PcdMaxAuthVariableSize|0x2800
!if $(NETWORK_TLS_ENABLE) == FALSE
# match PcdFlashNvStorageVariableSize purely for convenience
gEfiMdeModulePkgTokenSpaceGuid.PcdVariableStoreSize|0xe000
!endif
!endif
!if $(FD_SIZE_IN_KB) == 4096
gEfiMdeModulePkgTokenSpaceGuid.PcdMaxVariableSize|0x8400
gEfiMdeModulePkgTokenSpaceGuid.PcdMaxAuthVariableSize|0x8400
!if $(NETWORK_TLS_ENABLE) == FALSE
# match PcdFlashNvStorageVariableSize purely for convenience
gEfiMdeModulePkgTokenSpaceGuid.PcdVariableStoreSize|0x40000
!endif
diff --git a/OvmfPkg/OvmfPkgX64.dsc b/OvmfPkg/OvmfPkgX64.dsc
index 78079b9f8e13..e258c474b60d 100644
--- a/OvmfPkg/OvmfPkgX64.dsc
+++ b/OvmfPkg/OvmfPkgX64.dsc
@@ -419,45 +419,44 @@ [LibraryClasses.common.SMM_CORE]
!endif
PciLib|OvmfPkg/Library/DxePciLibI440FxQ35/DxePciLibI440FxQ35.inf
################################################################################
#
# Pcd Section - list of all EDK II PCD Entries defined by this Platform.
#
################################################################################
[PcdsFeatureFlag]
gEfiMdeModulePkgTokenSpaceGuid.PcdHiiOsRuntimeSupport|FALSE
gEfiMdeModulePkgTokenSpaceGuid.PcdStatusCodeUseSerial|FALSE
gEfiMdeModulePkgTokenSpaceGuid.PcdStatusCodeUseMemory|TRUE
gEfiMdeModulePkgTokenSpaceGuid.PcdDxeIplSupportUefiDecompress|FALSE
gEfiMdeModulePkgTokenSpaceGuid.PcdDxeIplSwitchToLongMode|FALSE
gEfiMdeModulePkgTokenSpaceGuid.PcdConOutGopSupport|TRUE
gEfiMdeModulePkgTokenSpaceGuid.PcdConOutUgaSupport|FALSE
gEfiMdeModulePkgTokenSpaceGuid.PcdInstallAcpiSdtProtocol|TRUE
!ifdef $(CSM_ENABLE)
gUefiOvmfPkgTokenSpaceGuid.PcdCsmEnable|TRUE
!endif
!if $(SMM_REQUIRE) == TRUE
gUefiOvmfPkgTokenSpaceGuid.PcdSmmSmramRequire|TRUE
- gUefiCpuPkgTokenSpaceGuid.PcdCpuSmmEnableBspElection|FALSE
gEfiMdeModulePkgTokenSpaceGuid.PcdEnableVariableRuntimeCache|FALSE
!endif
[PcdsFixedAtBuild]
gEfiMdeModulePkgTokenSpaceGuid.PcdStatusCodeMemorySize|1
gEfiMdeModulePkgTokenSpaceGuid.PcdResetOnMemoryTypeInformationChange|FALSE
gEfiMdePkgTokenSpaceGuid.PcdMaximumGuidedExtractHandler|0x10
!if ($(FD_SIZE_IN_KB) == 1024) || ($(FD_SIZE_IN_KB) == 2048)
gEfiMdeModulePkgTokenSpaceGuid.PcdMaxVariableSize|0x2000
gEfiMdeModulePkgTokenSpaceGuid.PcdMaxAuthVariableSize|0x2800
!if $(NETWORK_TLS_ENABLE) == FALSE
# match PcdFlashNvStorageVariableSize purely for convenience
gEfiMdeModulePkgTokenSpaceGuid.PcdVariableStoreSize|0xe000
!endif
!endif
!if $(FD_SIZE_IN_KB) == 4096
gEfiMdeModulePkgTokenSpaceGuid.PcdMaxVariableSize|0x8400
gEfiMdeModulePkgTokenSpaceGuid.PcdMaxAuthVariableSize|0x8400
!if $(NETWORK_TLS_ENABLE) == FALSE
# match PcdFlashNvStorageVariableSize purely for convenience
gEfiMdeModulePkgTokenSpaceGuid.PcdVariableStoreSize|0x40000
!endif
diff --git a/OvmfPkg/Library/SmmCpuPlatformHookLibQemu/SmmCpuPlatformHookLibQemu.inf b/OvmfPkg/Library/SmmCpuPlatformHookLibQemu/SmmCpuPlatformHookLibQemu.inf
index 82edeca3d12d..413c56fce6e1 100644
--- a/OvmfPkg/Library/SmmCpuPlatformHookLibQemu/SmmCpuPlatformHookLibQemu.inf
+++ b/OvmfPkg/Library/SmmCpuPlatformHookLibQemu/SmmCpuPlatformHookLibQemu.inf
@@ -8,22 +8,25 @@
[Defines]
INF_VERSION = 1.29
BASE_NAME = SmmCpuPlatformHookLibQemu
FILE_GUID = 154D6D26-54B8-45BC-BA3A-CBAA20C02A6A
MODULE_TYPE = DXE_DRIVER
VERSION_STRING = 1.0
LIBRARY_CLASS = SmmCpuPlatformHookLib
#
# The following information is for reference only and not required by the build
# tools.
#
# VALID_ARCHITECTURES = IA32 X64
#
[Sources]
SmmCpuPlatformHookLibQemu.c
[Packages]
MdePkg/MdePkg.dec
UefiCpuPkg/UefiCpuPkg.dec
+
+[LibraryClasses]
+ BaseLib
diff --git a/OvmfPkg/Library/SmmCpuPlatformHookLibQemu/SmmCpuPlatformHookLibQemu.c b/OvmfPkg/Library/SmmCpuPlatformHookLibQemu/SmmCpuPlatformHookLibQemu.c
index 257e1d399cc6..c88a95c6deff 100644
--- a/OvmfPkg/Library/SmmCpuPlatformHookLibQemu/SmmCpuPlatformHookLibQemu.c
+++ b/OvmfPkg/Library/SmmCpuPlatformHookLibQemu/SmmCpuPlatformHookLibQemu.c
@@ -1,31 +1,34 @@
/** @file
SMM CPU Platform Hook library instance for QEMU.
Copyright (c) 2020, Red Hat, Inc.
Copyright (c) 2006 - 2015, Intel Corporation. All rights reserved.<BR>
SPDX-License-Identifier: BSD-2-Clause-Patent
**/
+#include <Library/BaseLib.h> // AsmReadMsr64()
#include <PiSmm.h>
+#include <Register/Intel/ArchitecturalMsr.h> // MSR_IA32_APIC_BASE_REGISTER
+
#include <Library/SmmCpuPlatformHookLib.h>
/**
Checks if platform produces a valid SMI.
This function checks if platform produces a valid SMI. This function is
called at SMM entry to detect if this is a spurious SMI. This function
must be implemented in an MP safe way because it is called by multiple CPU
threads.
@retval TRUE There is a valid SMI
@retval FALSE There is no valid SMI
**/
BOOLEAN
EFIAPI
PlatformValidSmi (
VOID
)
{
return TRUE;
}
@@ -56,45 +59,49 @@ ClearTopLevelSmiStatus (
@param IsBsp Output parameter. TRUE: the CPU this function
executes on is elected to be the SMM BSP. FALSE:
the CPU this function executes on is to be SMM AP.
@retval EFI_SUCCESS The function executes successfully.
@retval EFI_NOT_READY The function does not determine whether this CPU
should be BSP or AP. This may occur if hardware
init sequence to enable the determination is yet to
be done, or the function chooses not to do BSP
election and will let SMM CPU driver to use its
default BSP election process.
@retval EFI_DEVICE_ERROR The function cannot determine whether this CPU
should be BSP or AP due to hardware error.
**/
EFI_STATUS
EFIAPI
PlatformSmmBspElection (
OUT BOOLEAN *IsBsp
)
{
- return EFI_NOT_READY;
+ MSR_IA32_APIC_BASE_REGISTER ApicBaseMsr;
+
+ ApicBaseMsr.Uint64 = AsmReadMsr64 (MSR_IA32_APIC_BASE);
+ *IsBsp = (BOOLEAN)(ApicBaseMsr.Bits.BSP == 1);
+ return EFI_SUCCESS;
}
/**
Get platform page table attribute.
This function gets page table attribute of platform.
@param Address Input parameter. Obtain the page table entries
attribute on this address.
@param PageSize Output parameter. The size of the page.
@param NumOfPages Output parameter. Number of page.
@param PageAttribute Output parameter. Paging Attributes (WB, UC, etc).
@retval EFI_SUCCESS The platform page table attribute from the address
is determined.
@retval EFI_UNSUPPORTED The platform does not support getting page table
attribute for the address.
**/
EFI_STATUS
EFIAPI
GetPlatformPageTableAttribute (
--
2.19.1.3.g30247aa5d201
^ permalink raw reply related [flat|nested] 24+ messages in thread* [PATCH 05/16] OvmfPkg: enable CPU hotplug support in PiSmmCpuDxeSmm
2020-02-23 17:25 [PATCH 00/16] OvmfPkg: support VCPU hotplug with -D SMM_REQUIRE Laszlo Ersek
` (3 preceding siblings ...)
2020-02-23 17:25 ` [PATCH 04/16] OvmfPkg: enable SMM Monarch Election in PiSmmCpuDxeSmm Laszlo Ersek
@ 2020-02-23 17:25 ` Laszlo Ersek
2020-02-23 17:25 ` [PATCH 06/16] OvmfPkg/CpuHotplugSmm: introduce skeleton for CPU Hotplug SMM driver Laszlo Ersek
` (12 subsequent siblings)
17 siblings, 0 replies; 24+ messages in thread
From: Laszlo Ersek @ 2020-02-23 17:25 UTC (permalink / raw)
To: edk2-devel-groups-io
Cc: Ard Biesheuvel, Igor Mammedov, Jiewen Yao, Jordan Justen,
Michael Kinney, Philippe Mathieu-Daudé
Set "PcdCpuHotPlugSupport" to TRUE, when OVMF is built with SMM_REQUIRE.
Consequences:
(1) In PiCpuSmmEntry() [UefiCpuPkg/PiSmmCpuDxeSmm/PiSmmCpuDxeSmm.c],
resources are allocated and populated in advance for all possible
(i.e., potentially hot-added) processors, rather than only the
processors present at boot.
The possible count (called "mMaxNumberOfCpus") is set from
"PcdCpuMaxLogicalProcessorNumber"; we set the latter in
OvmfPkg/PlatformPei. (Refer to commit 83357313dd67,
"OvmfPkg/PlatformPei: rewrite MaxCpuCountInitialization() for CPU
hotplug", 2020-01-29).
(2) The AddProcessor() and RemoveProcessor() member functions of
EFI_SMM_CPU_SERVICE_PROTOCOL, implemented in
"UefiCpuPkg/PiSmmCpuDxeSmm/CpuService.c", are no longer
short-circuited to EFI_UNSUPPORTED.
We'll rely on these functions in the CPU hotplug SMI handler, in a
subsequent patch.
(3) In PiCpuSmmEntry(), the address of the CPU_HOT_PLUG_DATA structure (in
SMRAM) is exposed via the dynamic-only "PcdCpuHotPlugDataAddress".
This structure is an information channel between the CPU hotplug SMI
handler, and EFI_SMM_CPU_SERVICE_PROTOCOL. Namely, at the first
"Index" where the following equality holds:
CPU_HOT_PLUG_DATA.ApicId[Index] == INVALID_APIC_ID
a hot-plugged CPU can be accepted, with the steps below:
(3.1) The hotplug SMI handler has to overwrite INVALID_APIC_ID with the
new CPU's APIC ID.
(3.2) The new CPU's SMBASE has to be relocated to:
CPU_HOT_PLUG_DATA.SmBase[Index]
(which was precomputed in step (1) above).
(3.3) The hotplug SMI handler is supposed to call
EFI_SMM_CPU_SERVICE_PROTOCOL.AddProcessor().
Note: we need not spell out "PcdCpuHotPlugDataAddress" in the
[PcdsDynamicDefault] sections of the OVMF DSC files, just so the PCD
become dynamically settable. That's because "UefiCpuPkg.dec" declares this
PCD with [PcdsDynamic, PcdsDynamicEx] access methods *only*.
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Michael Kinney <michael.d.kinney@intel.com>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=1512
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
---
OvmfPkg/OvmfPkgIa32.dsc | 1 +
OvmfPkg/OvmfPkgIa32X64.dsc | 1 +
OvmfPkg/OvmfPkgX64.dsc | 1 +
3 files changed, 3 insertions(+)
diff --git a/OvmfPkg/OvmfPkgIa32.dsc b/OvmfPkg/OvmfPkgIa32.dsc
index 60d8af185b9c..8c065ca7cec9 100644
--- a/OvmfPkg/OvmfPkgIa32.dsc
+++ b/OvmfPkg/OvmfPkgIa32.dsc
@@ -414,44 +414,45 @@ [LibraryClasses.common.SMM_CORE]
!endif
PciLib|OvmfPkg/Library/DxePciLibI440FxQ35/DxePciLibI440FxQ35.inf
################################################################################
#
# Pcd Section - list of all EDK II PCD Entries defined by this Platform.
#
################################################################################
[PcdsFeatureFlag]
gEfiMdeModulePkgTokenSpaceGuid.PcdHiiOsRuntimeSupport|FALSE
gEfiMdeModulePkgTokenSpaceGuid.PcdStatusCodeUseSerial|FALSE
gEfiMdeModulePkgTokenSpaceGuid.PcdStatusCodeUseMemory|TRUE
gEfiMdeModulePkgTokenSpaceGuid.PcdDxeIplSupportUefiDecompress|FALSE
gEfiMdeModulePkgTokenSpaceGuid.PcdDxeIplSwitchToLongMode|FALSE
gEfiMdeModulePkgTokenSpaceGuid.PcdConOutGopSupport|TRUE
gEfiMdeModulePkgTokenSpaceGuid.PcdConOutUgaSupport|FALSE
gEfiMdeModulePkgTokenSpaceGuid.PcdInstallAcpiSdtProtocol|TRUE
!ifdef $(CSM_ENABLE)
gUefiOvmfPkgTokenSpaceGuid.PcdCsmEnable|TRUE
!endif
!if $(SMM_REQUIRE) == TRUE
gUefiOvmfPkgTokenSpaceGuid.PcdSmmSmramRequire|TRUE
+ gUefiCpuPkgTokenSpaceGuid.PcdCpuHotPlugSupport|TRUE
gEfiMdeModulePkgTokenSpaceGuid.PcdEnableVariableRuntimeCache|FALSE
!endif
[PcdsFixedAtBuild]
gEfiMdeModulePkgTokenSpaceGuid.PcdStatusCodeMemorySize|1
gEfiMdeModulePkgTokenSpaceGuid.PcdResetOnMemoryTypeInformationChange|FALSE
gEfiMdePkgTokenSpaceGuid.PcdMaximumGuidedExtractHandler|0x10
!if ($(FD_SIZE_IN_KB) == 1024) || ($(FD_SIZE_IN_KB) == 2048)
gEfiMdeModulePkgTokenSpaceGuid.PcdMaxVariableSize|0x2000
gEfiMdeModulePkgTokenSpaceGuid.PcdMaxAuthVariableSize|0x2800
!if $(NETWORK_TLS_ENABLE) == FALSE
# match PcdFlashNvStorageVariableSize purely for convenience
gEfiMdeModulePkgTokenSpaceGuid.PcdVariableStoreSize|0xe000
!endif
!endif
!if $(FD_SIZE_IN_KB) == 4096
gEfiMdeModulePkgTokenSpaceGuid.PcdMaxVariableSize|0x8400
gEfiMdeModulePkgTokenSpaceGuid.PcdMaxAuthVariableSize|0x8400
!if $(NETWORK_TLS_ENABLE) == FALSE
# match PcdFlashNvStorageVariableSize purely for convenience
gEfiMdeModulePkgTokenSpaceGuid.PcdVariableStoreSize|0x40000
!endif
diff --git a/OvmfPkg/OvmfPkgIa32X64.dsc b/OvmfPkg/OvmfPkgIa32X64.dsc
index be6bc7bd88a7..944b785e61a9 100644
--- a/OvmfPkg/OvmfPkgIa32X64.dsc
+++ b/OvmfPkg/OvmfPkgIa32X64.dsc
@@ -419,44 +419,45 @@ [LibraryClasses.common.SMM_CORE]
!endif
PciLib|OvmfPkg/Library/DxePciLibI440FxQ35/DxePciLibI440FxQ35.inf
################################################################################
#
# Pcd Section - list of all EDK II PCD Entries defined by this Platform.
#
################################################################################
[PcdsFeatureFlag]
gEfiMdeModulePkgTokenSpaceGuid.PcdHiiOsRuntimeSupport|FALSE
gEfiMdeModulePkgTokenSpaceGuid.PcdStatusCodeUseSerial|FALSE
gEfiMdeModulePkgTokenSpaceGuid.PcdStatusCodeUseMemory|TRUE
gEfiMdeModulePkgTokenSpaceGuid.PcdDxeIplSupportUefiDecompress|FALSE
gEfiMdeModulePkgTokenSpaceGuid.PcdDxeIplSwitchToLongMode|TRUE
gEfiMdeModulePkgTokenSpaceGuid.PcdConOutGopSupport|TRUE
gEfiMdeModulePkgTokenSpaceGuid.PcdConOutUgaSupport|FALSE
gEfiMdeModulePkgTokenSpaceGuid.PcdInstallAcpiSdtProtocol|TRUE
!ifdef $(CSM_ENABLE)
gUefiOvmfPkgTokenSpaceGuid.PcdCsmEnable|TRUE
!endif
!if $(SMM_REQUIRE) == TRUE
gUefiOvmfPkgTokenSpaceGuid.PcdSmmSmramRequire|TRUE
+ gUefiCpuPkgTokenSpaceGuid.PcdCpuHotPlugSupport|TRUE
gEfiMdeModulePkgTokenSpaceGuid.PcdEnableVariableRuntimeCache|FALSE
!endif
[PcdsFixedAtBuild]
gEfiMdeModulePkgTokenSpaceGuid.PcdStatusCodeMemorySize|1
gEfiMdeModulePkgTokenSpaceGuid.PcdResetOnMemoryTypeInformationChange|FALSE
gEfiMdePkgTokenSpaceGuid.PcdMaximumGuidedExtractHandler|0x10
!if ($(FD_SIZE_IN_KB) == 1024) || ($(FD_SIZE_IN_KB) == 2048)
gEfiMdeModulePkgTokenSpaceGuid.PcdMaxVariableSize|0x2000
gEfiMdeModulePkgTokenSpaceGuid.PcdMaxAuthVariableSize|0x2800
!if $(NETWORK_TLS_ENABLE) == FALSE
# match PcdFlashNvStorageVariableSize purely for convenience
gEfiMdeModulePkgTokenSpaceGuid.PcdVariableStoreSize|0xe000
!endif
!endif
!if $(FD_SIZE_IN_KB) == 4096
gEfiMdeModulePkgTokenSpaceGuid.PcdMaxVariableSize|0x8400
gEfiMdeModulePkgTokenSpaceGuid.PcdMaxAuthVariableSize|0x8400
!if $(NETWORK_TLS_ENABLE) == FALSE
# match PcdFlashNvStorageVariableSize purely for convenience
gEfiMdeModulePkgTokenSpaceGuid.PcdVariableStoreSize|0x40000
!endif
diff --git a/OvmfPkg/OvmfPkgX64.dsc b/OvmfPkg/OvmfPkgX64.dsc
index e258c474b60d..8de0f7179784 100644
--- a/OvmfPkg/OvmfPkgX64.dsc
+++ b/OvmfPkg/OvmfPkgX64.dsc
@@ -419,44 +419,45 @@ [LibraryClasses.common.SMM_CORE]
!endif
PciLib|OvmfPkg/Library/DxePciLibI440FxQ35/DxePciLibI440FxQ35.inf
################################################################################
#
# Pcd Section - list of all EDK II PCD Entries defined by this Platform.
#
################################################################################
[PcdsFeatureFlag]
gEfiMdeModulePkgTokenSpaceGuid.PcdHiiOsRuntimeSupport|FALSE
gEfiMdeModulePkgTokenSpaceGuid.PcdStatusCodeUseSerial|FALSE
gEfiMdeModulePkgTokenSpaceGuid.PcdStatusCodeUseMemory|TRUE
gEfiMdeModulePkgTokenSpaceGuid.PcdDxeIplSupportUefiDecompress|FALSE
gEfiMdeModulePkgTokenSpaceGuid.PcdDxeIplSwitchToLongMode|FALSE
gEfiMdeModulePkgTokenSpaceGuid.PcdConOutGopSupport|TRUE
gEfiMdeModulePkgTokenSpaceGuid.PcdConOutUgaSupport|FALSE
gEfiMdeModulePkgTokenSpaceGuid.PcdInstallAcpiSdtProtocol|TRUE
!ifdef $(CSM_ENABLE)
gUefiOvmfPkgTokenSpaceGuid.PcdCsmEnable|TRUE
!endif
!if $(SMM_REQUIRE) == TRUE
gUefiOvmfPkgTokenSpaceGuid.PcdSmmSmramRequire|TRUE
+ gUefiCpuPkgTokenSpaceGuid.PcdCpuHotPlugSupport|TRUE
gEfiMdeModulePkgTokenSpaceGuid.PcdEnableVariableRuntimeCache|FALSE
!endif
[PcdsFixedAtBuild]
gEfiMdeModulePkgTokenSpaceGuid.PcdStatusCodeMemorySize|1
gEfiMdeModulePkgTokenSpaceGuid.PcdResetOnMemoryTypeInformationChange|FALSE
gEfiMdePkgTokenSpaceGuid.PcdMaximumGuidedExtractHandler|0x10
!if ($(FD_SIZE_IN_KB) == 1024) || ($(FD_SIZE_IN_KB) == 2048)
gEfiMdeModulePkgTokenSpaceGuid.PcdMaxVariableSize|0x2000
gEfiMdeModulePkgTokenSpaceGuid.PcdMaxAuthVariableSize|0x2800
!if $(NETWORK_TLS_ENABLE) == FALSE
# match PcdFlashNvStorageVariableSize purely for convenience
gEfiMdeModulePkgTokenSpaceGuid.PcdVariableStoreSize|0xe000
!endif
!endif
!if $(FD_SIZE_IN_KB) == 4096
gEfiMdeModulePkgTokenSpaceGuid.PcdMaxVariableSize|0x8400
gEfiMdeModulePkgTokenSpaceGuid.PcdMaxAuthVariableSize|0x8400
!if $(NETWORK_TLS_ENABLE) == FALSE
# match PcdFlashNvStorageVariableSize purely for convenience
gEfiMdeModulePkgTokenSpaceGuid.PcdVariableStoreSize|0x40000
!endif
--
2.19.1.3.g30247aa5d201
^ permalink raw reply related [flat|nested] 24+ messages in thread* [PATCH 06/16] OvmfPkg/CpuHotplugSmm: introduce skeleton for CPU Hotplug SMM driver
2020-02-23 17:25 [PATCH 00/16] OvmfPkg: support VCPU hotplug with -D SMM_REQUIRE Laszlo Ersek
` (4 preceding siblings ...)
2020-02-23 17:25 ` [PATCH 05/16] OvmfPkg: enable CPU hotplug support " Laszlo Ersek
@ 2020-02-23 17:25 ` Laszlo Ersek
2020-02-23 17:25 ` [PATCH 07/16] OvmfPkg/CpuHotplugSmm: add hotplug register block helper functions Laszlo Ersek
` (11 subsequent siblings)
17 siblings, 0 replies; 24+ messages in thread
From: Laszlo Ersek @ 2020-02-23 17:25 UTC (permalink / raw)
To: edk2-devel-groups-io
Cc: Ard Biesheuvel, Igor Mammedov, Jiewen Yao, Jordan Justen,
Michael Kinney, Philippe Mathieu-Daudé
Add a new SMM driver skeleton that registers a root SMI handler, and
checks if the SMI control value (written to 0xB2) indicates a CPU hotplug
SMI.
QEMU's ACPI payload will cause the OS to raise a broadcast SMI when a CPU
hotplug event occurs, namely by writing value 4 to IO Port 0xB2. In other
words, control value 4 is now allocated for this purpose; introduce the
ICH9_APM_CNT_CPU_HOTPLUG macro for it.
The standard identifiers in this driver use the new MM (Management Mode)
terminology from the PI spec, not the earlier SMM (System Management Mode)
terms.
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Michael Kinney <michael.d.kinney@intel.com>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=1512
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
---
OvmfPkg/OvmfPkgIa32.dsc | 1 +
OvmfPkg/OvmfPkgIa32X64.dsc | 1 +
OvmfPkg/OvmfPkgX64.dsc | 1 +
OvmfPkg/OvmfPkgIa32.fdf | 1 +
OvmfPkg/OvmfPkgIa32X64.fdf | 1 +
OvmfPkg/OvmfPkgX64.fdf | 1 +
OvmfPkg/Include/IndustryStandard/Q35MchIch9.h | 5 +-
OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf | 48 +++++
OvmfPkg/CpuHotplugSmm/CpuHotplug.c | 191 ++++++++++++++++++++
9 files changed, 248 insertions(+), 2 deletions(-)
diff --git a/OvmfPkg/OvmfPkgIa32.dsc b/OvmfPkg/OvmfPkgIa32.dsc
index 8c065ca7cec9..78310da44a5f 100644
--- a/OvmfPkg/OvmfPkgIa32.dsc
+++ b/OvmfPkg/OvmfPkgIa32.dsc
@@ -851,44 +851,45 @@ [Components]
OvmfPkg/PlatformDxe/Platform.inf
OvmfPkg/IoMmuDxe/IoMmuDxe.inf
!if $(SMM_REQUIRE) == TRUE
OvmfPkg/SmmAccess/SmmAccess2Dxe.inf
OvmfPkg/SmmControl2Dxe/SmmControl2Dxe.inf
UefiCpuPkg/CpuS3DataDxe/CpuS3DataDxe.inf
#
# SMM Initial Program Load (a DXE_RUNTIME_DRIVER)
#
MdeModulePkg/Core/PiSmmCore/PiSmmIpl.inf
#
# SMM_CORE
#
MdeModulePkg/Core/PiSmmCore/PiSmmCore.inf
#
# Privileged drivers (DXE_SMM_DRIVER modules)
#
+ OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf
UefiCpuPkg/CpuIo2Smm/CpuIo2Smm.inf
MdeModulePkg/Universal/LockBox/SmmLockBox/SmmLockBox.inf {
<LibraryClasses>
LockBoxLib|MdeModulePkg/Library/SmmLockBoxLib/SmmLockBoxSmmLib.inf
}
UefiCpuPkg/PiSmmCpuDxeSmm/PiSmmCpuDxeSmm.inf {
<LibraryClasses>
SmmCpuPlatformHookLib|OvmfPkg/Library/SmmCpuPlatformHookLibQemu/SmmCpuPlatformHookLibQemu.inf
SmmCpuFeaturesLib|OvmfPkg/Library/SmmCpuFeaturesLib/SmmCpuFeaturesLib.inf
}
#
# Variable driver stack (SMM)
#
OvmfPkg/QemuFlashFvbServicesRuntimeDxe/FvbServicesSmm.inf
MdeModulePkg/Universal/FaultTolerantWriteDxe/FaultTolerantWriteSmm.inf
MdeModulePkg/Universal/Variable/RuntimeDxe/VariableSmm.inf {
<LibraryClasses>
NULL|MdeModulePkg/Library/VarCheckUefiLib/VarCheckUefiLib.inf
}
MdeModulePkg/Universal/Variable/RuntimeDxe/VariableSmmRuntimeDxe.inf
diff --git a/OvmfPkg/OvmfPkgIa32X64.dsc b/OvmfPkg/OvmfPkgIa32X64.dsc
index 944b785e61a9..428578a4f839 100644
--- a/OvmfPkg/OvmfPkgIa32X64.dsc
+++ b/OvmfPkg/OvmfPkgIa32X64.dsc
@@ -865,44 +865,45 @@ [Components.X64]
OvmfPkg/PlatformDxe/Platform.inf
OvmfPkg/AmdSevDxe/AmdSevDxe.inf
OvmfPkg/IoMmuDxe/IoMmuDxe.inf
!if $(SMM_REQUIRE) == TRUE
OvmfPkg/SmmAccess/SmmAccess2Dxe.inf
OvmfPkg/SmmControl2Dxe/SmmControl2Dxe.inf
UefiCpuPkg/CpuS3DataDxe/CpuS3DataDxe.inf
#
# SMM Initial Program Load (a DXE_RUNTIME_DRIVER)
#
MdeModulePkg/Core/PiSmmCore/PiSmmIpl.inf
#
# SMM_CORE
#
MdeModulePkg/Core/PiSmmCore/PiSmmCore.inf
#
# Privileged drivers (DXE_SMM_DRIVER modules)
#
+ OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf
UefiCpuPkg/CpuIo2Smm/CpuIo2Smm.inf
MdeModulePkg/Universal/LockBox/SmmLockBox/SmmLockBox.inf {
<LibraryClasses>
LockBoxLib|MdeModulePkg/Library/SmmLockBoxLib/SmmLockBoxSmmLib.inf
}
UefiCpuPkg/PiSmmCpuDxeSmm/PiSmmCpuDxeSmm.inf {
<LibraryClasses>
SmmCpuPlatformHookLib|OvmfPkg/Library/SmmCpuPlatformHookLibQemu/SmmCpuPlatformHookLibQemu.inf
SmmCpuFeaturesLib|OvmfPkg/Library/SmmCpuFeaturesLib/SmmCpuFeaturesLib.inf
}
#
# Variable driver stack (SMM)
#
OvmfPkg/QemuFlashFvbServicesRuntimeDxe/FvbServicesSmm.inf
MdeModulePkg/Universal/FaultTolerantWriteDxe/FaultTolerantWriteSmm.inf
MdeModulePkg/Universal/Variable/RuntimeDxe/VariableSmm.inf {
<LibraryClasses>
NULL|MdeModulePkg/Library/VarCheckUefiLib/VarCheckUefiLib.inf
}
MdeModulePkg/Universal/Variable/RuntimeDxe/VariableSmmRuntimeDxe.inf
diff --git a/OvmfPkg/OvmfPkgX64.dsc b/OvmfPkg/OvmfPkgX64.dsc
index 8de0f7179784..73b92f259201 100644
--- a/OvmfPkg/OvmfPkgX64.dsc
+++ b/OvmfPkg/OvmfPkgX64.dsc
@@ -863,44 +863,45 @@ [Components]
OvmfPkg/PlatformDxe/Platform.inf
OvmfPkg/AmdSevDxe/AmdSevDxe.inf
OvmfPkg/IoMmuDxe/IoMmuDxe.inf
!if $(SMM_REQUIRE) == TRUE
OvmfPkg/SmmAccess/SmmAccess2Dxe.inf
OvmfPkg/SmmControl2Dxe/SmmControl2Dxe.inf
UefiCpuPkg/CpuS3DataDxe/CpuS3DataDxe.inf
#
# SMM Initial Program Load (a DXE_RUNTIME_DRIVER)
#
MdeModulePkg/Core/PiSmmCore/PiSmmIpl.inf
#
# SMM_CORE
#
MdeModulePkg/Core/PiSmmCore/PiSmmCore.inf
#
# Privileged drivers (DXE_SMM_DRIVER modules)
#
+ OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf
UefiCpuPkg/CpuIo2Smm/CpuIo2Smm.inf
MdeModulePkg/Universal/LockBox/SmmLockBox/SmmLockBox.inf {
<LibraryClasses>
LockBoxLib|MdeModulePkg/Library/SmmLockBoxLib/SmmLockBoxSmmLib.inf
}
UefiCpuPkg/PiSmmCpuDxeSmm/PiSmmCpuDxeSmm.inf {
<LibraryClasses>
SmmCpuPlatformHookLib|OvmfPkg/Library/SmmCpuPlatformHookLibQemu/SmmCpuPlatformHookLibQemu.inf
SmmCpuFeaturesLib|OvmfPkg/Library/SmmCpuFeaturesLib/SmmCpuFeaturesLib.inf
}
#
# Variable driver stack (SMM)
#
OvmfPkg/QemuFlashFvbServicesRuntimeDxe/FvbServicesSmm.inf
MdeModulePkg/Universal/FaultTolerantWriteDxe/FaultTolerantWriteSmm.inf
MdeModulePkg/Universal/Variable/RuntimeDxe/VariableSmm.inf {
<LibraryClasses>
NULL|MdeModulePkg/Library/VarCheckUefiLib/VarCheckUefiLib.inf
}
MdeModulePkg/Universal/Variable/RuntimeDxe/VariableSmmRuntimeDxe.inf
diff --git a/OvmfPkg/OvmfPkgIa32.fdf b/OvmfPkg/OvmfPkgIa32.fdf
index 63607551ed75..61b891765c56 100644
--- a/OvmfPkg/OvmfPkgIa32.fdf
+++ b/OvmfPkg/OvmfPkgIa32.fdf
@@ -301,44 +301,45 @@ [FV.DXEFV]
INF MdeModulePkg/Bus/Usb/UsbKbDxe/UsbKbDxe.inf
INF MdeModulePkg/Bus/Usb/UsbMassStorageDxe/UsbMassStorageDxe.inf
!ifdef $(CSM_ENABLE)
INF OvmfPkg/Csm/BiosThunk/VideoDxe/VideoDxe.inf
INF OvmfPkg/Csm/LegacyBiosDxe/LegacyBiosDxe.inf
INF RuleOverride=CSM OvmfPkg/Csm/Csm16/Csm16.inf
!else
INF OvmfPkg/QemuVideoDxe/QemuVideoDxe.inf
!endif
INF OvmfPkg/QemuRamfbDxe/QemuRamfbDxe.inf
INF OvmfPkg/VirtioGpuDxe/VirtioGpu.inf
INF OvmfPkg/PlatformDxe/Platform.inf
INF OvmfPkg/IoMmuDxe/IoMmuDxe.inf
!if $(SMM_REQUIRE) == TRUE
INF OvmfPkg/SmmAccess/SmmAccess2Dxe.inf
INF OvmfPkg/SmmControl2Dxe/SmmControl2Dxe.inf
INF UefiCpuPkg/CpuS3DataDxe/CpuS3DataDxe.inf
INF MdeModulePkg/Core/PiSmmCore/PiSmmIpl.inf
INF MdeModulePkg/Core/PiSmmCore/PiSmmCore.inf
+INF OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf
INF UefiCpuPkg/CpuIo2Smm/CpuIo2Smm.inf
INF MdeModulePkg/Universal/LockBox/SmmLockBox/SmmLockBox.inf
INF UefiCpuPkg/PiSmmCpuDxeSmm/PiSmmCpuDxeSmm.inf
#
# Variable driver stack (SMM)
#
INF OvmfPkg/QemuFlashFvbServicesRuntimeDxe/FvbServicesSmm.inf
INF MdeModulePkg/Universal/FaultTolerantWriteDxe/FaultTolerantWriteSmm.inf
INF MdeModulePkg/Universal/Variable/RuntimeDxe/VariableSmm.inf
INF MdeModulePkg/Universal/Variable/RuntimeDxe/VariableSmmRuntimeDxe.inf
!else
#
# Variable driver stack (non-SMM)
#
INF OvmfPkg/QemuFlashFvbServicesRuntimeDxe/FvbServicesRuntimeDxe.inf
INF OvmfPkg/EmuVariableFvbRuntimeDxe/Fvb.inf
INF MdeModulePkg/Universal/FaultTolerantWriteDxe/FaultTolerantWriteDxe.inf
INF MdeModulePkg/Universal/Variable/RuntimeDxe/VariableRuntimeDxe.inf
!endif
diff --git a/OvmfPkg/OvmfPkgIa32X64.fdf b/OvmfPkg/OvmfPkgIa32X64.fdf
index 0488e5d95ffe..501b4fcb7b67 100644
--- a/OvmfPkg/OvmfPkgIa32X64.fdf
+++ b/OvmfPkg/OvmfPkgIa32X64.fdf
@@ -308,44 +308,45 @@ [FV.DXEFV]
INF MdeModulePkg/Bus/Usb/UsbMassStorageDxe/UsbMassStorageDxe.inf
!ifdef $(CSM_ENABLE)
INF OvmfPkg/Csm/BiosThunk/VideoDxe/VideoDxe.inf
INF OvmfPkg/Csm/LegacyBiosDxe/LegacyBiosDxe.inf
INF RuleOverride=CSM OvmfPkg/Csm/Csm16/Csm16.inf
!else
INF OvmfPkg/QemuVideoDxe/QemuVideoDxe.inf
!endif
INF OvmfPkg/QemuRamfbDxe/QemuRamfbDxe.inf
INF OvmfPkg/VirtioGpuDxe/VirtioGpu.inf
INF OvmfPkg/PlatformDxe/Platform.inf
INF OvmfPkg/AmdSevDxe/AmdSevDxe.inf
INF OvmfPkg/IoMmuDxe/IoMmuDxe.inf
!if $(SMM_REQUIRE) == TRUE
INF OvmfPkg/SmmAccess/SmmAccess2Dxe.inf
INF OvmfPkg/SmmControl2Dxe/SmmControl2Dxe.inf
INF UefiCpuPkg/CpuS3DataDxe/CpuS3DataDxe.inf
INF MdeModulePkg/Core/PiSmmCore/PiSmmIpl.inf
INF MdeModulePkg/Core/PiSmmCore/PiSmmCore.inf
+INF OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf
INF UefiCpuPkg/CpuIo2Smm/CpuIo2Smm.inf
INF MdeModulePkg/Universal/LockBox/SmmLockBox/SmmLockBox.inf
INF UefiCpuPkg/PiSmmCpuDxeSmm/PiSmmCpuDxeSmm.inf
#
# Variable driver stack (SMM)
#
INF OvmfPkg/QemuFlashFvbServicesRuntimeDxe/FvbServicesSmm.inf
INF MdeModulePkg/Universal/FaultTolerantWriteDxe/FaultTolerantWriteSmm.inf
INF MdeModulePkg/Universal/Variable/RuntimeDxe/VariableSmm.inf
INF MdeModulePkg/Universal/Variable/RuntimeDxe/VariableSmmRuntimeDxe.inf
!else
#
# Variable driver stack (non-SMM)
#
INF OvmfPkg/QemuFlashFvbServicesRuntimeDxe/FvbServicesRuntimeDxe.inf
INF OvmfPkg/EmuVariableFvbRuntimeDxe/Fvb.inf
INF MdeModulePkg/Universal/FaultTolerantWriteDxe/FaultTolerantWriteDxe.inf
INF MdeModulePkg/Universal/Variable/RuntimeDxe/VariableRuntimeDxe.inf
!endif
diff --git a/OvmfPkg/OvmfPkgX64.fdf b/OvmfPkg/OvmfPkgX64.fdf
index 0488e5d95ffe..501b4fcb7b67 100644
--- a/OvmfPkg/OvmfPkgX64.fdf
+++ b/OvmfPkg/OvmfPkgX64.fdf
@@ -308,44 +308,45 @@ [FV.DXEFV]
INF MdeModulePkg/Bus/Usb/UsbMassStorageDxe/UsbMassStorageDxe.inf
!ifdef $(CSM_ENABLE)
INF OvmfPkg/Csm/BiosThunk/VideoDxe/VideoDxe.inf
INF OvmfPkg/Csm/LegacyBiosDxe/LegacyBiosDxe.inf
INF RuleOverride=CSM OvmfPkg/Csm/Csm16/Csm16.inf
!else
INF OvmfPkg/QemuVideoDxe/QemuVideoDxe.inf
!endif
INF OvmfPkg/QemuRamfbDxe/QemuRamfbDxe.inf
INF OvmfPkg/VirtioGpuDxe/VirtioGpu.inf
INF OvmfPkg/PlatformDxe/Platform.inf
INF OvmfPkg/AmdSevDxe/AmdSevDxe.inf
INF OvmfPkg/IoMmuDxe/IoMmuDxe.inf
!if $(SMM_REQUIRE) == TRUE
INF OvmfPkg/SmmAccess/SmmAccess2Dxe.inf
INF OvmfPkg/SmmControl2Dxe/SmmControl2Dxe.inf
INF UefiCpuPkg/CpuS3DataDxe/CpuS3DataDxe.inf
INF MdeModulePkg/Core/PiSmmCore/PiSmmIpl.inf
INF MdeModulePkg/Core/PiSmmCore/PiSmmCore.inf
+INF OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf
INF UefiCpuPkg/CpuIo2Smm/CpuIo2Smm.inf
INF MdeModulePkg/Universal/LockBox/SmmLockBox/SmmLockBox.inf
INF UefiCpuPkg/PiSmmCpuDxeSmm/PiSmmCpuDxeSmm.inf
#
# Variable driver stack (SMM)
#
INF OvmfPkg/QemuFlashFvbServicesRuntimeDxe/FvbServicesSmm.inf
INF MdeModulePkg/Universal/FaultTolerantWriteDxe/FaultTolerantWriteSmm.inf
INF MdeModulePkg/Universal/Variable/RuntimeDxe/VariableSmm.inf
INF MdeModulePkg/Universal/Variable/RuntimeDxe/VariableSmmRuntimeDxe.inf
!else
#
# Variable driver stack (non-SMM)
#
INF OvmfPkg/QemuFlashFvbServicesRuntimeDxe/FvbServicesRuntimeDxe.inf
INF OvmfPkg/EmuVariableFvbRuntimeDxe/Fvb.inf
INF MdeModulePkg/Universal/FaultTolerantWriteDxe/FaultTolerantWriteDxe.inf
INF MdeModulePkg/Universal/Variable/RuntimeDxe/VariableRuntimeDxe.inf
!endif
diff --git a/OvmfPkg/Include/IndustryStandard/Q35MchIch9.h b/OvmfPkg/Include/IndustryStandard/Q35MchIch9.h
index cb705fee92ca..73db4b59a111 100644
--- a/OvmfPkg/Include/IndustryStandard/Q35MchIch9.h
+++ b/OvmfPkg/Include/IndustryStandard/Q35MchIch9.h
@@ -90,37 +90,38 @@
#define POWER_MGMT_REGISTER_Q35(Offset) \
PCI_LIB_ADDRESS (0, 0x1f, 0, (Offset))
#define POWER_MGMT_REGISTER_Q35_EFI_PCI_ADDRESS(Offset) \
EFI_PCI_ADDRESS (0, 0x1f, 0, (Offset))
#define ICH9_PMBASE 0x40
#define ICH9_PMBASE_MASK (BIT15 | BIT14 | BIT13 | BIT12 | BIT11 | \
BIT10 | BIT9 | BIT8 | BIT7)
#define ICH9_ACPI_CNTL 0x44
#define ICH9_ACPI_CNTL_ACPI_EN BIT7
#define ICH9_GEN_PMCON_1 0xA0
#define ICH9_GEN_PMCON_1_SMI_LOCK BIT4
#define ICH9_RCBA 0xF0
#define ICH9_RCBA_EN BIT0
//
// IO ports
//
-#define ICH9_APM_CNT 0xB2
-#define ICH9_APM_STS 0xB3
+#define ICH9_APM_CNT 0xB2
+#define ICH9_APM_CNT_CPU_HOTPLUG 0x04
+#define ICH9_APM_STS 0xB3
#define ICH9_CPU_HOTPLUG_BASE 0x0CD8
//
// IO ports relative to PMBASE
//
#define ICH9_PMBASE_OFS_SMI_EN 0x30
#define ICH9_SMI_EN_APMC_EN BIT5
#define ICH9_SMI_EN_GBL_SMI_EN BIT0
#define ICH9_ROOT_COMPLEX_BASE 0xFED1C000
#endif
diff --git a/OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf b/OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf
new file mode 100644
index 000000000000..fa70858a8dab
--- /dev/null
+++ b/OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf
@@ -0,0 +1,48 @@
+## @file
+# Root SMI handler for VCPU hotplug SMIs.
+#
+# Copyright (c) 2020, Red Hat, Inc.
+#
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+
+[Defines]
+ INF_VERSION = 1.29
+ PI_SPECIFICATION_VERSION = 0x00010046 # PI-1.7.0
+ BASE_NAME = CpuHotplugSmm
+ FILE_GUID = 84EEA114-C6BE-4445-8F90-51D97863E363
+ MODULE_TYPE = DXE_SMM_DRIVER
+ ENTRY_POINT = CpuHotplugEntry
+
+#
+# The following information is for reference only and not required by the build
+# tools.
+#
+# VALID_ARCHITECTURES = IA32 X64
+#
+
+[Sources]
+ CpuHotplug.c
+
+[Packages]
+ MdePkg/MdePkg.dec
+ OvmfPkg/OvmfPkg.dec
+
+[LibraryClasses]
+ BaseLib
+ DebugLib
+ MmServicesTableLib
+ PcdLib
+ UefiDriverEntryPoint
+
+[Protocols]
+ gEfiMmCpuIoProtocolGuid ## CONSUMES
+
+[Pcd]
+ gUefiOvmfPkgTokenSpaceGuid.PcdQ35SmramAtDefaultSmbase ## CONSUMES
+
+[FeaturePcd]
+ gUefiOvmfPkgTokenSpaceGuid.PcdSmmSmramRequire ## CONSUMES
+
+[Depex]
+ gEfiMmCpuIoProtocolGuid
diff --git a/OvmfPkg/CpuHotplugSmm/CpuHotplug.c b/OvmfPkg/CpuHotplugSmm/CpuHotplug.c
new file mode 100644
index 000000000000..fd09403eabf3
--- /dev/null
+++ b/OvmfPkg/CpuHotplugSmm/CpuHotplug.c
@@ -0,0 +1,191 @@
+/** @file
+ Root SMI handler for VCPU hotplug SMIs.
+
+ Copyright (c) 2020, Red Hat, Inc.
+
+ SPDX-License-Identifier: BSD-2-Clause-Patent
+**/
+
+#include <IndustryStandard/Q35MchIch9.h> // ICH9_APM_CNT
+#include <Library/BaseLib.h> // CpuDeadLoop()
+#include <Library/DebugLib.h> // ASSERT()
+#include <Library/MmServicesTableLib.h> // gMmst
+#include <Library/PcdLib.h> // PcdGetBool()
+#include <Protocol/MmCpuIo.h> // EFI_MM_CPU_IO_PROTOCOL
+#include <Uefi/UefiBaseType.h> // EFI_STATUS
+
+//
+// We use this protocol for accessing IO Ports.
+//
+STATIC EFI_MM_CPU_IO_PROTOCOL *mMmCpuIo;
+//
+// Represents the registration of the CPU Hotplug MMI handler.
+//
+STATIC EFI_HANDLE mDispatchHandle;
+
+
+/**
+ CPU Hotplug MMI handler function.
+
+ This is a root MMI handler.
+
+ @param[in] DispatchHandle The unique handle assigned to this handler by
+ EFI_MM_SYSTEM_TABLE.MmiHandlerRegister().
+
+ @param[in] Context Context passed in by
+ EFI_MM_SYSTEM_TABLE.MmiManage(). Due to
+ CpuHotplugMmi() being a root MMI handler,
+ Context is ASSERT()ed to be NULL.
+
+ @param[in,out] CommBuffer Ignored, due to CpuHotplugMmi() being a root
+ MMI handler.
+
+ @param[in,out] CommBufferSize Ignored, due to CpuHotplugMmi() being a root
+ MMI handler.
+
+ @retval EFI_SUCCESS The MMI was handled and the MMI
+ source was quiesced. When returned
+ by a non-root MMI handler,
+ EFI_SUCCESS terminates the
+ processing of MMI handlers in
+ EFI_MM_SYSTEM_TABLE.MmiManage().
+ For a root MMI handler (i.e., for
+ the present function too),
+ EFI_SUCCESS behaves identically to
+ EFI_WARN_INTERRUPT_SOURCE_QUIESCED,
+ as further root MMI handlers are
+ going to be called by
+ EFI_MM_SYSTEM_TABLE.MmiManage()
+ anyway.
+
+ @retval EFI_WARN_INTERRUPT_SOURCE_QUIESCED The MMI source has been quiesced,
+ but other handlers should still
+ be called.
+
+ @retval EFI_WARN_INTERRUPT_SOURCE_PENDING The MMI source is still pending,
+ and other handlers should still
+ be called.
+
+ @retval EFI_INTERRUPT_PENDING The MMI source could not be
+ quiesced.
+**/
+STATIC
+EFI_STATUS
+EFIAPI
+CpuHotplugMmi (
+ IN EFI_HANDLE DispatchHandle,
+ IN CONST VOID *Context OPTIONAL,
+ IN OUT VOID *CommBuffer OPTIONAL,
+ IN OUT UINTN *CommBufferSize OPTIONAL
+ )
+{
+ EFI_STATUS Status;
+ UINT8 ApmControl;
+
+ //
+ // Assert that we are entering this function due to our root MMI handler
+ // registration.
+ //
+ ASSERT (DispatchHandle == mDispatchHandle);
+ //
+ // When MmiManage() is invoked to process root MMI handlers, the caller (the
+ // MM Core) is expected to pass in a NULL Context. MmiManage() then passes
+ // the same NULL Context to individual handlers.
+ //
+ ASSERT (Context == NULL);
+ //
+ // Read the MMI command value from the APM Control Port, to see if this is an
+ // MMI we should care about.
+ //
+ Status = mMmCpuIo->Io.Read (mMmCpuIo, MM_IO_UINT8, ICH9_APM_CNT, 1,
+ &ApmControl);
+ if (EFI_ERROR (Status)) {
+ DEBUG ((DEBUG_ERROR, "%a: failed to read ICH9_APM_CNT: %r\n", __FUNCTION__,
+ Status));
+ //
+ // We couldn't even determine if the MMI was for us or not.
+ //
+ goto Fatal;
+ }
+
+ if (ApmControl != ICH9_APM_CNT_CPU_HOTPLUG) {
+ //
+ // The MMI is not for us.
+ //
+ return EFI_WARN_INTERRUPT_SOURCE_QUIESCED;
+ }
+
+ //
+ // We've handled this MMI.
+ //
+ return EFI_SUCCESS;
+
+Fatal:
+ ASSERT (FALSE);
+ CpuDeadLoop ();
+ //
+ // We couldn't handle this MMI.
+ //
+ return EFI_INTERRUPT_PENDING;
+}
+
+
+//
+// Entry point function of this driver.
+//
+EFI_STATUS
+EFIAPI
+CpuHotplugEntry (
+ IN EFI_HANDLE ImageHandle,
+ IN EFI_SYSTEM_TABLE *SystemTable
+ )
+{
+ EFI_STATUS Status;
+
+ //
+ // This module should only be included when SMM support is required.
+ //
+ ASSERT (FeaturePcdGet (PcdSmmSmramRequire));
+ //
+ // This driver depends on the dynamically detected "SMRAM at default SMBASE"
+ // feature.
+ //
+ if (!PcdGetBool (PcdQ35SmramAtDefaultSmbase)) {
+ return EFI_UNSUPPORTED;
+ }
+
+ //
+ // Errors from here on are fatal; we cannot allow the boot to proceed if we
+ // can't set up this driver to handle CPU hotplug.
+ //
+ // First, collect the protocols needed later. All of these protocols are
+ // listed in our module DEPEX.
+ //
+ Status = gMmst->MmLocateProtocol (&gEfiMmCpuIoProtocolGuid,
+ NULL /* Registration */, (VOID **)&mMmCpuIo);
+ if (EFI_ERROR (Status)) {
+ DEBUG ((DEBUG_ERROR, "%a: locate MmCpuIo: %r\n", __FUNCTION__, Status));
+ goto Fatal;
+ }
+
+ //
+ // Register the handler for the CPU Hotplug MMI.
+ //
+ Status = gMmst->MmiHandlerRegister (
+ CpuHotplugMmi,
+ NULL, // HandlerType: root MMI handler
+ &mDispatchHandle
+ );
+ if (EFI_ERROR (Status)) {
+ DEBUG ((DEBUG_ERROR, "%a: MmiHandlerRegister(): %r\n", __FUNCTION__,
+ Status));
+ goto Fatal;
+ }
+
+ return EFI_SUCCESS;
+
+Fatal:
+ ASSERT (FALSE);
+ CpuDeadLoop ();
+ return Status;
+}
--
2.19.1.3.g30247aa5d201
^ permalink raw reply related [flat|nested] 24+ messages in thread* [PATCH 07/16] OvmfPkg/CpuHotplugSmm: add hotplug register block helper functions
2020-02-23 17:25 [PATCH 00/16] OvmfPkg: support VCPU hotplug with -D SMM_REQUIRE Laszlo Ersek
` (5 preceding siblings ...)
2020-02-23 17:25 ` [PATCH 06/16] OvmfPkg/CpuHotplugSmm: introduce skeleton for CPU Hotplug SMM driver Laszlo Ersek
@ 2020-02-23 17:25 ` Laszlo Ersek
2020-02-23 17:25 ` [PATCH 08/16] OvmfPkg/CpuHotplugSmm: define the QEMU_CPUHP_CMD_GET_ARCH_ID macro Laszlo Ersek
` (10 subsequent siblings)
17 siblings, 0 replies; 24+ messages in thread
From: Laszlo Ersek @ 2020-02-23 17:25 UTC (permalink / raw)
To: edk2-devel-groups-io
Cc: Ard Biesheuvel, Igor Mammedov, Jiewen Yao, Jordan Justen,
Michael Kinney, Philippe Mathieu-Daudé
Add a handful of simple functions for accessing QEMU's hotplug registers
more conveniently. These functions thinly wrap some of the registers
described in "docs/specs/acpi_cpu_hotplug.txt" in the QEMU tree. The
functions hang (by design) if they encounter an internal failure.
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Michael Kinney <michael.d.kinney@intel.com>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=1512
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
---
OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf | 2 +
OvmfPkg/CpuHotplugSmm/QemuCpuhp.h | 47 +++++++
OvmfPkg/CpuHotplugSmm/QemuCpuhp.c | 136 ++++++++++++++++++++
3 files changed, 185 insertions(+)
diff --git a/OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf b/OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf
index fa70858a8dab..ac4ca4c1f4f2 100644
--- a/OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf
+++ b/OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf
@@ -4,44 +4,46 @@
# Copyright (c) 2020, Red Hat, Inc.
#
# SPDX-License-Identifier: BSD-2-Clause-Patent
##
[Defines]
INF_VERSION = 1.29
PI_SPECIFICATION_VERSION = 0x00010046 # PI-1.7.0
BASE_NAME = CpuHotplugSmm
FILE_GUID = 84EEA114-C6BE-4445-8F90-51D97863E363
MODULE_TYPE = DXE_SMM_DRIVER
ENTRY_POINT = CpuHotplugEntry
#
# The following information is for reference only and not required by the build
# tools.
#
# VALID_ARCHITECTURES = IA32 X64
#
[Sources]
CpuHotplug.c
+ QemuCpuhp.c
+ QemuCpuhp.h
[Packages]
MdePkg/MdePkg.dec
OvmfPkg/OvmfPkg.dec
[LibraryClasses]
BaseLib
DebugLib
MmServicesTableLib
PcdLib
UefiDriverEntryPoint
[Protocols]
gEfiMmCpuIoProtocolGuid ## CONSUMES
[Pcd]
gUefiOvmfPkgTokenSpaceGuid.PcdQ35SmramAtDefaultSmbase ## CONSUMES
[FeaturePcd]
gUefiOvmfPkgTokenSpaceGuid.PcdSmmSmramRequire ## CONSUMES
[Depex]
diff --git a/OvmfPkg/CpuHotplugSmm/QemuCpuhp.h b/OvmfPkg/CpuHotplugSmm/QemuCpuhp.h
new file mode 100644
index 000000000000..82f88f0b73bb
--- /dev/null
+++ b/OvmfPkg/CpuHotplugSmm/QemuCpuhp.h
@@ -0,0 +1,47 @@
+/** @file
+ Simple wrapper functions that access QEMU's modern CPU hotplug register
+ block.
+
+ These functions thinly wrap some of the registers described in
+ "docs/specs/acpi_cpu_hotplug.txt" in the QEMU source. IO Ports are accessed
+ via EFI_MM_CPU_IO_PROTOCOL. If a protocol call fails, these functions don't
+ return.
+
+ Copyright (c) 2020, Red Hat, Inc.
+
+ SPDX-License-Identifier: BSD-2-Clause-Patent
+**/
+
+#ifndef QEMU_CPUHP_H_
+#define QEMU_CPUHP_H_
+
+#include <Protocol/MmCpuIo.h> // EFI_MM_CPU_IO_PROTOCOL
+
+UINT32
+QemuCpuhpReadCommandData2 (
+ IN CONST EFI_MM_CPU_IO_PROTOCOL *MmCpuIo
+ );
+
+UINT8
+QemuCpuhpReadCpuStatus (
+ IN CONST EFI_MM_CPU_IO_PROTOCOL *MmCpuIo
+ );
+
+UINT32
+QemuCpuhpReadCommandData (
+ IN CONST EFI_MM_CPU_IO_PROTOCOL *MmCpuIo
+ );
+
+VOID
+QemuCpuhpWriteCpuSelector (
+ IN CONST EFI_MM_CPU_IO_PROTOCOL *MmCpuIo,
+ IN UINT32 Selector
+ );
+
+VOID
+QemuCpuhpWriteCommand (
+ IN CONST EFI_MM_CPU_IO_PROTOCOL *MmCpuIo,
+ IN UINT8 Command
+ );
+
+#endif // QEMU_CPUHP_H_
diff --git a/OvmfPkg/CpuHotplugSmm/QemuCpuhp.c b/OvmfPkg/CpuHotplugSmm/QemuCpuhp.c
new file mode 100644
index 000000000000..31e46f51934a
--- /dev/null
+++ b/OvmfPkg/CpuHotplugSmm/QemuCpuhp.c
@@ -0,0 +1,136 @@
+/** @file
+ Simple wrapper functions that access QEMU's modern CPU hotplug register
+ block.
+
+ These functions thinly wrap some of the registers described in
+ "docs/specs/acpi_cpu_hotplug.txt" in the QEMU source. IO Ports are accessed
+ via EFI_MM_CPU_IO_PROTOCOL. If a protocol call fails, these functions don't
+ return.
+
+ Copyright (c) 2020, Red Hat, Inc.
+
+ SPDX-License-Identifier: BSD-2-Clause-Patent
+**/
+
+#include <IndustryStandard/Q35MchIch9.h> // ICH9_CPU_HOTPLUG_BASE
+#include <IndustryStandard/QemuCpuHotplug.h> // QEMU_CPUHP_R_CMD_DATA2
+#include <Library/BaseLib.h> // CpuDeadLoop()
+#include <Library/DebugLib.h> // DEBUG()
+
+#include "QemuCpuhp.h"
+
+UINT32
+QemuCpuhpReadCommandData2 (
+ IN CONST EFI_MM_CPU_IO_PROTOCOL *MmCpuIo
+ )
+{
+ UINT32 CommandData2;
+ EFI_STATUS Status;
+
+ CommandData2 = 0;
+ Status = MmCpuIo->Io.Read (
+ MmCpuIo,
+ MM_IO_UINT32,
+ ICH9_CPU_HOTPLUG_BASE + QEMU_CPUHP_R_CMD_DATA2,
+ 1,
+ &CommandData2
+ );
+ if (EFI_ERROR (Status)) {
+ DEBUG ((DEBUG_ERROR, "%a: %r\n", __FUNCTION__, Status));
+ ASSERT (FALSE);
+ CpuDeadLoop ();
+ }
+ return CommandData2;
+}
+
+UINT8
+QemuCpuhpReadCpuStatus (
+ IN CONST EFI_MM_CPU_IO_PROTOCOL *MmCpuIo
+ )
+{
+ UINT8 CpuStatus;
+ EFI_STATUS Status;
+
+ CpuStatus = 0;
+ Status = MmCpuIo->Io.Read (
+ MmCpuIo,
+ MM_IO_UINT8,
+ ICH9_CPU_HOTPLUG_BASE + QEMU_CPUHP_R_CPU_STAT,
+ 1,
+ &CpuStatus
+ );
+ if (EFI_ERROR (Status)) {
+ DEBUG ((DEBUG_ERROR, "%a: %r\n", __FUNCTION__, Status));
+ ASSERT (FALSE);
+ CpuDeadLoop ();
+ }
+ return CpuStatus;
+}
+
+UINT32
+QemuCpuhpReadCommandData (
+ IN CONST EFI_MM_CPU_IO_PROTOCOL *MmCpuIo
+ )
+{
+ UINT32 CommandData;
+ EFI_STATUS Status;
+
+ CommandData = 0;
+ Status = MmCpuIo->Io.Read (
+ MmCpuIo,
+ MM_IO_UINT32,
+ ICH9_CPU_HOTPLUG_BASE + QEMU_CPUHP_RW_CMD_DATA,
+ 1,
+ &CommandData
+ );
+ if (EFI_ERROR (Status)) {
+ DEBUG ((DEBUG_ERROR, "%a: %r\n", __FUNCTION__, Status));
+ ASSERT (FALSE);
+ CpuDeadLoop ();
+ }
+ return CommandData;
+}
+
+VOID
+QemuCpuhpWriteCpuSelector (
+ IN CONST EFI_MM_CPU_IO_PROTOCOL *MmCpuIo,
+ IN UINT32 Selector
+ )
+{
+ EFI_STATUS Status;
+
+ Status = MmCpuIo->Io.Write (
+ MmCpuIo,
+ MM_IO_UINT32,
+ ICH9_CPU_HOTPLUG_BASE + QEMU_CPUHP_W_CPU_SEL,
+ 1,
+ &Selector
+ );
+ if (EFI_ERROR (Status)) {
+ DEBUG ((DEBUG_ERROR, "%a: %r\n", __FUNCTION__, Status));
+ ASSERT (FALSE);
+ CpuDeadLoop ();
+ }
+}
+
+VOID
+QemuCpuhpWriteCommand (
+ IN CONST EFI_MM_CPU_IO_PROTOCOL *MmCpuIo,
+ IN UINT8 Command
+ )
+{
+ EFI_STATUS Status;
+
+ Status = MmCpuIo->Io.Write (
+ MmCpuIo,
+ MM_IO_UINT8,
+ ICH9_CPU_HOTPLUG_BASE + QEMU_CPUHP_W_CMD,
+ 1,
+ &Command
+ );
+ if (EFI_ERROR (Status)) {
+ DEBUG ((DEBUG_ERROR, "%a: %r\n", __FUNCTION__, Status));
+ ASSERT (FALSE);
+ CpuDeadLoop ();
+ }
+}
--
2.19.1.3.g30247aa5d201
^ permalink raw reply related [flat|nested] 24+ messages in thread* [PATCH 08/16] OvmfPkg/CpuHotplugSmm: define the QEMU_CPUHP_CMD_GET_ARCH_ID macro
2020-02-23 17:25 [PATCH 00/16] OvmfPkg: support VCPU hotplug with -D SMM_REQUIRE Laszlo Ersek
` (6 preceding siblings ...)
2020-02-23 17:25 ` [PATCH 07/16] OvmfPkg/CpuHotplugSmm: add hotplug register block helper functions Laszlo Ersek
@ 2020-02-23 17:25 ` Laszlo Ersek
2020-02-23 17:25 ` [PATCH 09/16] OvmfPkg/CpuHotplugSmm: add function for collecting CPUs with events Laszlo Ersek
` (9 subsequent siblings)
17 siblings, 0 replies; 24+ messages in thread
From: Laszlo Ersek @ 2020-02-23 17:25 UTC (permalink / raw)
To: edk2-devel-groups-io
Cc: Ard Biesheuvel, Igor Mammedov, Jiewen Yao, Jordan Justen,
Michael Kinney, Philippe Mathieu-Daudé
QEMU commit 3a61c8db9d25 ("acpi: cpuhp: add CPHP_GET_CPU_ID_CMD command",
2020-01-22) introduced a new command in the modern CPU hotplug register
block that lets the firmware query the arch-specific IDs (on IA32/X64: the
APIC IDs) of CPUs. Add a macro for this command value, because we'll need
it later.
At the same time, add a sanity check for the modern hotplug interface to
CpuHotplugSmm.
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Michael Kinney <michael.d.kinney@intel.com>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=1512
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
---
OvmfPkg/Include/IndustryStandard/QemuCpuHotplug.h | 1 +
OvmfPkg/CpuHotplugSmm/CpuHotplug.c | 35 ++++++++++++++++++++
2 files changed, 36 insertions(+)
diff --git a/OvmfPkg/Include/IndustryStandard/QemuCpuHotplug.h b/OvmfPkg/Include/IndustryStandard/QemuCpuHotplug.h
index cf0745610f2c..3d013633501b 100644
--- a/OvmfPkg/Include/IndustryStandard/QemuCpuHotplug.h
+++ b/OvmfPkg/Include/IndustryStandard/QemuCpuHotplug.h
@@ -20,24 +20,25 @@
#define QEMU_CPU_HOTPLUG_H_
#include <Base.h>
//
// Each register offset is:
// - relative to the board-dependent IO base address of the register block,
// - named QEMU_CPUHP_(R|W|RW)_*, according to the possible access modes of the
// register,
// - followed by distinguished bitmasks or values in the register.
//
#define QEMU_CPUHP_R_CMD_DATA2 0x0
#define QEMU_CPUHP_R_CPU_STAT 0x4
#define QEMU_CPUHP_STAT_ENABLED BIT0
#define QEMU_CPUHP_RW_CMD_DATA 0x8
#define QEMU_CPUHP_W_CPU_SEL 0x0
#define QEMU_CPUHP_W_CMD 0x5
#define QEMU_CPUHP_CMD_GET_PENDING 0x0
+#define QEMU_CPUHP_CMD_GET_ARCH_ID 0x3
#endif // QEMU_CPU_HOTPLUG_H_
diff --git a/OvmfPkg/CpuHotplugSmm/CpuHotplug.c b/OvmfPkg/CpuHotplugSmm/CpuHotplug.c
index fd09403eabf3..5df8c689c63a 100644
--- a/OvmfPkg/CpuHotplugSmm/CpuHotplug.c
+++ b/OvmfPkg/CpuHotplugSmm/CpuHotplug.c
@@ -1,38 +1,41 @@
/** @file
Root SMI handler for VCPU hotplug SMIs.
Copyright (c) 2020, Red Hat, Inc.
SPDX-License-Identifier: BSD-2-Clause-Patent
**/
#include <IndustryStandard/Q35MchIch9.h> // ICH9_APM_CNT
+#include <IndustryStandard/QemuCpuHotplug.h> // QEMU_CPUHP_CMD_GET_PENDING
#include <Library/BaseLib.h> // CpuDeadLoop()
#include <Library/DebugLib.h> // ASSERT()
#include <Library/MmServicesTableLib.h> // gMmst
#include <Library/PcdLib.h> // PcdGetBool()
#include <Protocol/MmCpuIo.h> // EFI_MM_CPU_IO_PROTOCOL
#include <Uefi/UefiBaseType.h> // EFI_STATUS
+#include "QemuCpuhp.h" // QemuCpuhpWriteCpuSelector()
+
//
// We use this protocol for accessing IO Ports.
//
STATIC EFI_MM_CPU_IO_PROTOCOL *mMmCpuIo;
//
// Represents the registration of the CPU Hotplug MMI handler.
//
STATIC EFI_HANDLE mDispatchHandle;
/**
CPU Hotplug MMI handler function.
This is a root MMI handler.
@param[in] DispatchHandle The unique handle assigned to this handler by
EFI_MM_SYSTEM_TABLE.MmiHandlerRegister().
@param[in] Context Context passed in by
EFI_MM_SYSTEM_TABLE.MmiManage(). Due to
CpuHotplugMmi() being a root MMI handler,
Context is ASSERT()ed to be NULL.
@@ -149,43 +152,75 @@ CpuHotplugEntry (
//
// This driver depends on the dynamically detected "SMRAM at default SMBASE"
// feature.
//
if (!PcdGetBool (PcdQ35SmramAtDefaultSmbase)) {
return EFI_UNSUPPORTED;
}
//
// Errors from here on are fatal; we cannot allow the boot to proceed if we
// can't set up this driver to handle CPU hotplug.
//
// First, collect the protocols needed later. All of these protocols are
// listed in our module DEPEX.
//
Status = gMmst->MmLocateProtocol (&gEfiMmCpuIoProtocolGuid,
NULL /* Registration */, (VOID **)&mMmCpuIo);
if (EFI_ERROR (Status)) {
DEBUG ((DEBUG_ERROR, "%a: locate MmCpuIo: %r\n", __FUNCTION__, Status));
goto Fatal;
}
+ //
+ // Sanity-check the CPU hotplug interface.
+ //
+ // Both of the following features are part of QEMU 5.0, introduced primarily
+ // in commit range 3e08b2b9cb64..3a61c8db9d25:
+ //
+ // (a) the QEMU_CPUHP_CMD_GET_ARCH_ID command of the modern CPU hotplug
+ // interface,
+ //
+ // (b) the "SMRAM at default SMBASE" feature.
+ //
+ // From these, (b) is restricted to 5.0+ machine type versions, while (a)
+ // does not depend on machine type version. Because we ensured the stricter
+ // condition (b) through PcdQ35SmramAtDefaultSmbase above, the (a)
+ // QEMU_CPUHP_CMD_GET_ARCH_ID command must now be available too. While we
+ // can't verify the presence of precisely that command, we can still verify
+ // (sanity-check) that the modern interface is active, at least.
+ //
+ // Consult the "Typical usecases | Detecting and enabling modern CPU hotplug
+ // interface" section in QEMU's "docs/specs/acpi_cpu_hotplug.txt", on the
+ // following.
+ //
+ QemuCpuhpWriteCpuSelector (mMmCpuIo, 0);
+ QemuCpuhpWriteCpuSelector (mMmCpuIo, 0);
+ QemuCpuhpWriteCommand (mMmCpuIo, QEMU_CPUHP_CMD_GET_PENDING);
+ if (QemuCpuhpReadCommandData2 (mMmCpuIo) != 0) {
+ Status = EFI_NOT_FOUND;
+ DEBUG ((DEBUG_ERROR, "%a: modern CPU hotplug interface: %r\n",
+ __FUNCTION__, Status));
+ goto Fatal;
+ }
+
//
// Register the handler for the CPU Hotplug MMI.
//
Status = gMmst->MmiHandlerRegister (
CpuHotplugMmi,
NULL, // HandlerType: root MMI handler
&mDispatchHandle
);
if (EFI_ERROR (Status)) {
DEBUG ((DEBUG_ERROR, "%a: MmiHandlerRegister(): %r\n", __FUNCTION__,
Status));
goto Fatal;
}
return EFI_SUCCESS;
Fatal:
ASSERT (FALSE);
CpuDeadLoop ();
return Status;
}
--
2.19.1.3.g30247aa5d201
^ permalink raw reply related [flat|nested] 24+ messages in thread* [PATCH 09/16] OvmfPkg/CpuHotplugSmm: add function for collecting CPUs with events
2020-02-23 17:25 [PATCH 00/16] OvmfPkg: support VCPU hotplug with -D SMM_REQUIRE Laszlo Ersek
` (7 preceding siblings ...)
2020-02-23 17:25 ` [PATCH 08/16] OvmfPkg/CpuHotplugSmm: define the QEMU_CPUHP_CMD_GET_ARCH_ID macro Laszlo Ersek
@ 2020-02-23 17:25 ` Laszlo Ersek
2020-02-23 17:25 ` [PATCH 10/16] OvmfPkg/CpuHotplugSmm: collect " Laszlo Ersek
` (8 subsequent siblings)
17 siblings, 0 replies; 24+ messages in thread
From: Laszlo Ersek @ 2020-02-23 17:25 UTC (permalink / raw)
To: edk2-devel-groups-io
Cc: Ard Biesheuvel, Igor Mammedov, Jiewen Yao, Jordan Justen,
Michael Kinney, Philippe Mathieu-Daudé
Add a function that collects the APIC IDs of CPUs that have just been
hot-plugged, or are about to be hot-unplugged.
Pending events are only located and never cleared; QEMU's AML needs the
firmware to leave the status bits intact in the hotplug register block.
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Michael Kinney <michael.d.kinney@intel.com>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=1512
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
---
OvmfPkg/Include/IndustryStandard/QemuCpuHotplug.h | 2 +
OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf | 1 +
OvmfPkg/CpuHotplugSmm/ApicId.h | 23 +++
OvmfPkg/CpuHotplugSmm/QemuCpuhp.h | 20 ++-
OvmfPkg/CpuHotplugSmm/QemuCpuhp.c | 171 +++++++++++++++++++-
5 files changed, 211 insertions(+), 6 deletions(-)
diff --git a/OvmfPkg/Include/IndustryStandard/QemuCpuHotplug.h b/OvmfPkg/Include/IndustryStandard/QemuCpuHotplug.h
index 3d013633501b..a34a6d3fae61 100644
--- a/OvmfPkg/Include/IndustryStandard/QemuCpuHotplug.h
+++ b/OvmfPkg/Include/IndustryStandard/QemuCpuHotplug.h
@@ -13,32 +13,34 @@
The new ("modern") hotplug interface appeared in QEMU v2.7.0.
The macros in this header file map to the minimal subset of the modern
interface that OVMF needs.
**/
#ifndef QEMU_CPU_HOTPLUG_H_
#define QEMU_CPU_HOTPLUG_H_
#include <Base.h>
//
// Each register offset is:
// - relative to the board-dependent IO base address of the register block,
// - named QEMU_CPUHP_(R|W|RW)_*, according to the possible access modes of the
// register,
// - followed by distinguished bitmasks or values in the register.
//
#define QEMU_CPUHP_R_CMD_DATA2 0x0
#define QEMU_CPUHP_R_CPU_STAT 0x4
#define QEMU_CPUHP_STAT_ENABLED BIT0
+#define QEMU_CPUHP_STAT_INSERT BIT1
+#define QEMU_CPUHP_STAT_REMOVE BIT2
#define QEMU_CPUHP_RW_CMD_DATA 0x8
#define QEMU_CPUHP_W_CPU_SEL 0x0
#define QEMU_CPUHP_W_CMD 0x5
#define QEMU_CPUHP_CMD_GET_PENDING 0x0
#define QEMU_CPUHP_CMD_GET_ARCH_ID 0x3
#endif // QEMU_CPU_HOTPLUG_H_
diff --git a/OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf b/OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf
index ac4ca4c1f4f2..ab690a9e5e20 100644
--- a/OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf
+++ b/OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf
@@ -3,44 +3,45 @@
#
# Copyright (c) 2020, Red Hat, Inc.
#
# SPDX-License-Identifier: BSD-2-Clause-Patent
##
[Defines]
INF_VERSION = 1.29
PI_SPECIFICATION_VERSION = 0x00010046 # PI-1.7.0
BASE_NAME = CpuHotplugSmm
FILE_GUID = 84EEA114-C6BE-4445-8F90-51D97863E363
MODULE_TYPE = DXE_SMM_DRIVER
ENTRY_POINT = CpuHotplugEntry
#
# The following information is for reference only and not required by the build
# tools.
#
# VALID_ARCHITECTURES = IA32 X64
#
[Sources]
+ ApicId.h
CpuHotplug.c
QemuCpuhp.c
QemuCpuhp.h
[Packages]
MdePkg/MdePkg.dec
OvmfPkg/OvmfPkg.dec
[LibraryClasses]
BaseLib
DebugLib
MmServicesTableLib
PcdLib
UefiDriverEntryPoint
[Protocols]
gEfiMmCpuIoProtocolGuid ## CONSUMES
[Pcd]
gUefiOvmfPkgTokenSpaceGuid.PcdQ35SmramAtDefaultSmbase ## CONSUMES
[FeaturePcd]
diff --git a/OvmfPkg/CpuHotplugSmm/ApicId.h b/OvmfPkg/CpuHotplugSmm/ApicId.h
new file mode 100644
index 000000000000..3c365148ed02
--- /dev/null
+++ b/OvmfPkg/CpuHotplugSmm/ApicId.h
@@ -0,0 +1,23 @@
+/** @file
+ Type and macro definitions for representing and printing APIC IDs, compatibly
+ with the LocalApicLib and PrintLib classes, respectively.
+
+ Copyright (c) 2020, Red Hat, Inc.
+
+ SPDX-License-Identifier: BSD-2-Clause-Patent
+**/
+
+#ifndef APIC_ID_H_
+#define APIC_ID_H_
+
+//
+// The type that LocalApicLib represents an APIC ID with.
+//
+typedef UINT32 APIC_ID;
+
+//
+// The PrintLib conversion specification for formatting an APIC_ID.
+//
+#define FMT_APIC_ID "0x%08x"
+
+#endif // APIC_ID_H_
diff --git a/OvmfPkg/CpuHotplugSmm/QemuCpuhp.h b/OvmfPkg/CpuHotplugSmm/QemuCpuhp.h
index 82f88f0b73bb..8adaa0ad91f0 100644
--- a/OvmfPkg/CpuHotplugSmm/QemuCpuhp.h
+++ b/OvmfPkg/CpuHotplugSmm/QemuCpuhp.h
@@ -1,47 +1,61 @@
/** @file
- Simple wrapper functions that access QEMU's modern CPU hotplug register
- block.
+ Simple wrapper functions and utility functions that access QEMU's modern CPU
+ hotplug register block.
- These functions thinly wrap some of the registers described in
+ These functions manipulate some of the registers described in
"docs/specs/acpi_cpu_hotplug.txt" in the QEMU source. IO Ports are accessed
via EFI_MM_CPU_IO_PROTOCOL. If a protocol call fails, these functions don't
return.
Copyright (c) 2020, Red Hat, Inc.
SPDX-License-Identifier: BSD-2-Clause-Patent
**/
#ifndef QEMU_CPUHP_H_
#define QEMU_CPUHP_H_
#include <Protocol/MmCpuIo.h> // EFI_MM_CPU_IO_PROTOCOL
+#include <Uefi/UefiBaseType.h> // EFI_STATUS
+
+#include "ApicId.h" // APIC_ID
UINT32
QemuCpuhpReadCommandData2 (
IN CONST EFI_MM_CPU_IO_PROTOCOL *MmCpuIo
);
UINT8
QemuCpuhpReadCpuStatus (
IN CONST EFI_MM_CPU_IO_PROTOCOL *MmCpuIo
);
UINT32
QemuCpuhpReadCommandData (
IN CONST EFI_MM_CPU_IO_PROTOCOL *MmCpuIo
);
VOID
QemuCpuhpWriteCpuSelector (
IN CONST EFI_MM_CPU_IO_PROTOCOL *MmCpuIo,
IN UINT32 Selector
);
VOID
QemuCpuhpWriteCommand (
IN CONST EFI_MM_CPU_IO_PROTOCOL *MmCpuIo,
IN UINT8 Command
);
+EFI_STATUS
+QemuCpuhpCollectApicIds (
+ IN CONST EFI_MM_CPU_IO_PROTOCOL *MmCpuIo,
+ IN UINT32 PossibleCpuCount,
+ IN UINT32 ApicIdCount,
+ OUT APIC_ID *PluggedApicIds,
+ OUT UINT32 *PluggedCount,
+ OUT APIC_ID *ToUnplugApicIds,
+ OUT UINT32 *ToUnplugCount
+ );
+
#endif // QEMU_CPUHP_H_
diff --git a/OvmfPkg/CpuHotplugSmm/QemuCpuhp.c b/OvmfPkg/CpuHotplugSmm/QemuCpuhp.c
index 31e46f51934a..8d4a6693c8d6 100644
--- a/OvmfPkg/CpuHotplugSmm/QemuCpuhp.c
+++ b/OvmfPkg/CpuHotplugSmm/QemuCpuhp.c
@@ -1,27 +1,27 @@
/** @file
- Simple wrapper functions that access QEMU's modern CPU hotplug register
- block.
+ Simple wrapper functions and utility functions that access QEMU's modern CPU
+ hotplug register block.
- These functions thinly wrap some of the registers described in
+ These functions manipulate some of the registers described in
"docs/specs/acpi_cpu_hotplug.txt" in the QEMU source. IO Ports are accessed
via EFI_MM_CPU_IO_PROTOCOL. If a protocol call fails, these functions don't
return.
Copyright (c) 2020, Red Hat, Inc.
SPDX-License-Identifier: BSD-2-Clause-Patent
**/
#include <IndustryStandard/Q35MchIch9.h> // ICH9_CPU_HOTPLUG_BASE
#include <IndustryStandard/QemuCpuHotplug.h> // QEMU_CPUHP_R_CMD_DATA2
#include <Library/BaseLib.h> // CpuDeadLoop()
#include <Library/DebugLib.h> // DEBUG()
#include "QemuCpuhp.h"
UINT32
QemuCpuhpReadCommandData2 (
IN CONST EFI_MM_CPU_IO_PROTOCOL *MmCpuIo
)
{
UINT32 CommandData2;
@@ -115,22 +115,187 @@ QemuCpuhpWriteCpuSelector (
VOID
QemuCpuhpWriteCommand (
IN CONST EFI_MM_CPU_IO_PROTOCOL *MmCpuIo,
IN UINT8 Command
)
{
EFI_STATUS Status;
Status = MmCpuIo->Io.Write (
MmCpuIo,
MM_IO_UINT8,
ICH9_CPU_HOTPLUG_BASE + QEMU_CPUHP_W_CMD,
1,
&Command
);
if (EFI_ERROR (Status)) {
DEBUG ((DEBUG_ERROR, "%a: %r\n", __FUNCTION__, Status));
ASSERT (FALSE);
CpuDeadLoop ();
}
}
+
+/**
+ Collect the APIC IDs of
+ - the CPUs that have been hot-plugged,
+ - the CPUs that are about to be hot-unplugged.
+
+ This function only scans for events -- it does not modify them -- in the
+ hotplug registers.
+
+ On error, the contents of the output parameters are undefined.
+
+ @param[in] MmCpuIo The EFI_MM_CPU_IO_PROTOCOL instance for
+ accessing IO Ports.
+
+ @param[in] PossibleCpuCount The number of possible CPUs in the system. Must
+ be positive.
+
+ @param[in] ApicIdCount The number of elements each one of the
+ PluggedApicIds and ToUnplugApicIds arrays can
+ accommodate. Must be positive.
+
+ @param[out] PluggedApicIds The APIC IDs of the CPUs that have been
+ hot-plugged.
+
+ @param[out] PluggedCount The number of filled-in APIC IDs in
+ PluggedApicIds.
+
+ @param[out] ToUnplugApicIds The APIC IDs of the CPUs that are about to be
+ hot-unplugged.
+
+ @param[out] ToUnplugCount The number of filled-in APIC IDs in
+ ToUnplugApicIds.
+
+ @retval EFI_INVALID_PARAMETER PossibleCpuCount is zero, or ApicIdCount is
+ zero.
+
+ @retval EFI_PROTOCOL_ERROR Invalid bitmap detected in the
+ QEMU_CPUHP_R_CPU_STAT register.
+
+ @retval EFI_BUFFER_TOO_SMALL There was an attempt to place more than
+ ApicIdCount APIC IDs into one of the
+ PluggedApicIds and ToUnplugApicIds arrays.
+
+ @retval EFI_SUCCESS Output parameters have been set successfully.
+**/
+EFI_STATUS
+QemuCpuhpCollectApicIds (
+ IN CONST EFI_MM_CPU_IO_PROTOCOL *MmCpuIo,
+ IN UINT32 PossibleCpuCount,
+ IN UINT32 ApicIdCount,
+ OUT APIC_ID *PluggedApicIds,
+ OUT UINT32 *PluggedCount,
+ OUT APIC_ID *ToUnplugApicIds,
+ OUT UINT32 *ToUnplugCount
+ )
+{
+ UINT32 CurrentSelector;
+
+ if (PossibleCpuCount == 0 || ApicIdCount == 0) {
+ return EFI_INVALID_PARAMETER;
+ }
+
+ *PluggedCount = 0;
+ *ToUnplugCount = 0;
+
+ CurrentSelector = 0;
+ do {
+ UINT32 PendingSelector;
+ UINT8 CpuStatus;
+ APIC_ID *ExtendIds;
+ UINT32 *ExtendCount;
+ APIC_ID NewApicId;
+
+ //
+ // Write CurrentSelector (which is valid) to the CPU selector register.
+ // Consequences:
+ //
+ // - Other register accesses will be permitted.
+ //
+ // - The QEMU_CPUHP_CMD_GET_PENDING command will start scanning for a CPU
+ // with pending events at CurrentSelector (inclusive).
+ //
+ QemuCpuhpWriteCpuSelector (MmCpuIo, CurrentSelector);
+ //
+ // Write the QEMU_CPUHP_CMD_GET_PENDING command. Consequences
+ // (independently of each other):
+ //
+ // - If there is a CPU with pending events, starting at CurrentSelector
+ // (inclusive), the CPU selector will be updated to that CPU. Note that
+ // the scanning in QEMU may wrap around, because we must never clear the
+ // event bits.
+ //
+ // - The QEMU_CPUHP_RW_CMD_DATA register will return the (possibly updated)
+ // CPU selector value.
+ //
+ QemuCpuhpWriteCommand (MmCpuIo, QEMU_CPUHP_CMD_GET_PENDING);
+ PendingSelector = QemuCpuhpReadCommandData (MmCpuIo);
+ if (PendingSelector < CurrentSelector) {
+ DEBUG ((DEBUG_VERBOSE, "%a: CurrentSelector=%u PendingSelector=%u: "
+ "wrap-around\n", __FUNCTION__, CurrentSelector, PendingSelector));
+ break;
+ }
+ CurrentSelector = PendingSelector;
+
+ //
+ // Check the known status / event bits for the currently selected CPU.
+ //
+ CpuStatus = QemuCpuhpReadCpuStatus (MmCpuIo);
+ if ((CpuStatus & QEMU_CPUHP_STAT_INSERT) != 0) {
+ //
+ // The "insert" event guarantees the "enabled" status; plus it excludes
+ // the "remove" event.
+ //
+ if ((CpuStatus & QEMU_CPUHP_STAT_ENABLED) == 0 ||
+ (CpuStatus & QEMU_CPUHP_STAT_REMOVE) != 0) {
+ DEBUG ((DEBUG_ERROR, "%a: CurrentSelector=%u CpuStatus=0x%x: "
+ "inconsistent CPU status\n", __FUNCTION__, CurrentSelector,
+ CpuStatus));
+ return EFI_PROTOCOL_ERROR;
+ }
+
+ DEBUG ((DEBUG_VERBOSE, "%a: CurrentSelector=%u: insert\n", __FUNCTION__,
+ CurrentSelector));
+
+ ExtendIds = PluggedApicIds;
+ ExtendCount = PluggedCount;
+ } else if ((CpuStatus & QEMU_CPUHP_STAT_REMOVE) != 0) {
+ DEBUG ((DEBUG_VERBOSE, "%a: CurrentSelector=%u: remove\n", __FUNCTION__,
+ CurrentSelector));
+
+ ExtendIds = ToUnplugApicIds;
+ ExtendCount = ToUnplugCount;
+ } else {
+ DEBUG ((DEBUG_VERBOSE, "%a: CurrentSelector=%u: no event\n",
+ __FUNCTION__, CurrentSelector));
+ break;
+ }
+
+ //
+ // Save the APIC ID of the CPU with the pending event, to the corresponding
+ // APIC ID array.
+ //
+ if (*ExtendCount == ApicIdCount) {
+ DEBUG ((DEBUG_ERROR, "%a: APIC ID array too small\n", __FUNCTION__));
+ return EFI_BUFFER_TOO_SMALL;
+ }
+ QemuCpuhpWriteCommand (MmCpuIo, QEMU_CPUHP_CMD_GET_ARCH_ID);
+ NewApicId = QemuCpuhpReadCommandData (MmCpuIo);
+ DEBUG ((DEBUG_VERBOSE, "%a: ApicId=" FMT_APIC_ID "\n", __FUNCTION__,
+ NewApicId));
+ ExtendIds[(*ExtendCount)++] = NewApicId;
+
+ //
+ // We've processed the CPU with (known) pending events, but we must never
+ // clear events. Therefore we need to advance past this CPU manually;
+ // otherwise, QEMU_CPUHP_CMD_GET_PENDING would stick to the currently
+ // selected CPU.
+ //
+ CurrentSelector++;
+ } while (CurrentSelector < PossibleCpuCount);
+
+ DEBUG ((DEBUG_VERBOSE, "%a: PluggedCount=%u ToUnplugCount=%u\n",
+ __FUNCTION__, *PluggedCount, *ToUnplugCount));
+ return EFI_SUCCESS;
+}
--
2.19.1.3.g30247aa5d201
^ permalink raw reply related [flat|nested] 24+ messages in thread* [PATCH 10/16] OvmfPkg/CpuHotplugSmm: collect CPUs with events
2020-02-23 17:25 [PATCH 00/16] OvmfPkg: support VCPU hotplug with -D SMM_REQUIRE Laszlo Ersek
` (8 preceding siblings ...)
2020-02-23 17:25 ` [PATCH 09/16] OvmfPkg/CpuHotplugSmm: add function for collecting CPUs with events Laszlo Ersek
@ 2020-02-23 17:25 ` Laszlo Ersek
2020-02-23 17:25 ` [PATCH 11/16] OvmfPkg/CpuHotplugSmm: introduce Post-SMM Pen for hot-added CPUs Laszlo Ersek
` (7 subsequent siblings)
17 siblings, 0 replies; 24+ messages in thread
From: Laszlo Ersek @ 2020-02-23 17:25 UTC (permalink / raw)
To: edk2-devel-groups-io
Cc: Ard Biesheuvel, Igor Mammedov, Jiewen Yao, Jordan Justen,
Michael Kinney, Philippe Mathieu-Daudé
Call QemuCpuhpCollectApicIds() in the root MMI handler. The APIC IDs of
the hotplugged CPUs will be used for several purposes in subsequent
patches.
For calling QemuCpuhpCollectApicIds(), pre-allocate both of its output
arrays "PluggedApicIds" and "ToUnplugApicIds" in the driver's entry point
function. The allocation size is dictated by the possible CPU count, which
we fetch from "CPU_HOT_PLUG_DATA.ArrayLength".
The CPU_HOT_PLUG_DATA structure in SMRAM is an out-of-band information
channel between this driver and PiSmmCpuDxeSmm, underlying
EFI_SMM_CPU_SERVICE_PROTOCOL.
In order to consume "CPU_HOT_PLUG_DATA.ArrayLength", extend the driver's
DEPEX to EFI_SMM_CPU_SERVICE_PROTOCOL. PiSmmCpuDxeSmm stores the address
of CPU_HOT_PLUG_DATA to "PcdCpuHotPlugDataAddress", before it produces
EFI_SMM_CPU_SERVICE_PROTOCOL.
Stash the protocol at once, as it will be needed later.
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Michael Kinney <michael.d.kinney@intel.com>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=1512
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
---
OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf | 7 +-
OvmfPkg/CpuHotplugSmm/CpuHotplug.c | 111 +++++++++++++++++++-
2 files changed, 115 insertions(+), 3 deletions(-)
diff --git a/OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf b/OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf
index ab690a9e5e20..31c1ee1c9f6d 100644
--- a/OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf
+++ b/OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf
@@ -11,41 +11,46 @@ [Defines]
PI_SPECIFICATION_VERSION = 0x00010046 # PI-1.7.0
BASE_NAME = CpuHotplugSmm
FILE_GUID = 84EEA114-C6BE-4445-8F90-51D97863E363
MODULE_TYPE = DXE_SMM_DRIVER
ENTRY_POINT = CpuHotplugEntry
#
# The following information is for reference only and not required by the build
# tools.
#
# VALID_ARCHITECTURES = IA32 X64
#
[Sources]
ApicId.h
CpuHotplug.c
QemuCpuhp.c
QemuCpuhp.h
[Packages]
MdePkg/MdePkg.dec
OvmfPkg/OvmfPkg.dec
+ UefiCpuPkg/UefiCpuPkg.dec
[LibraryClasses]
BaseLib
DebugLib
MmServicesTableLib
PcdLib
+ SafeIntLib
UefiDriverEntryPoint
[Protocols]
gEfiMmCpuIoProtocolGuid ## CONSUMES
+ gEfiSmmCpuServiceProtocolGuid ## CONSUMES
[Pcd]
+ gUefiCpuPkgTokenSpaceGuid.PcdCpuHotPlugDataAddress ## CONSUMES
gUefiOvmfPkgTokenSpaceGuid.PcdQ35SmramAtDefaultSmbase ## CONSUMES
[FeaturePcd]
gUefiOvmfPkgTokenSpaceGuid.PcdSmmSmramRequire ## CONSUMES
[Depex]
- gEfiMmCpuIoProtocolGuid
+ gEfiMmCpuIoProtocolGuid AND
+ gEfiSmmCpuServiceProtocolGuid
diff --git a/OvmfPkg/CpuHotplugSmm/CpuHotplug.c b/OvmfPkg/CpuHotplugSmm/CpuHotplug.c
index 5df8c689c63a..42e023cb85c0 100644
--- a/OvmfPkg/CpuHotplugSmm/CpuHotplug.c
+++ b/OvmfPkg/CpuHotplugSmm/CpuHotplug.c
@@ -1,46 +1,76 @@
/** @file
Root SMI handler for VCPU hotplug SMIs.
Copyright (c) 2020, Red Hat, Inc.
SPDX-License-Identifier: BSD-2-Clause-Patent
**/
+#include <CpuHotPlugData.h> // CPU_HOT_PLUG_DATA
#include <IndustryStandard/Q35MchIch9.h> // ICH9_APM_CNT
#include <IndustryStandard/QemuCpuHotplug.h> // QEMU_CPUHP_CMD_GET_PENDING
#include <Library/BaseLib.h> // CpuDeadLoop()
#include <Library/DebugLib.h> // ASSERT()
#include <Library/MmServicesTableLib.h> // gMmst
#include <Library/PcdLib.h> // PcdGetBool()
+#include <Library/SafeIntLib.h> // SafeUintnSub()
#include <Protocol/MmCpuIo.h> // EFI_MM_CPU_IO_PROTOCOL
+#include <Protocol/SmmCpuService.h> // EFI_SMM_CPU_SERVICE_PROTOCOL
#include <Uefi/UefiBaseType.h> // EFI_STATUS
+#include "ApicId.h" // APIC_ID
#include "QemuCpuhp.h" // QemuCpuhpWriteCpuSelector()
//
// We use this protocol for accessing IO Ports.
//
STATIC EFI_MM_CPU_IO_PROTOCOL *mMmCpuIo;
//
+// The following protocol is used to report the addition or removal of a CPU to
+// the SMM CPU driver (PiSmmCpuDxeSmm).
+//
+STATIC EFI_SMM_CPU_SERVICE_PROTOCOL *mMmCpuService;
+//
+// This structure is a communication side-channel between the
+// EFI_SMM_CPU_SERVICE_PROTOCOL consumer (i.e., this driver) and provider
+// (i.e., PiSmmCpuDxeSmm).
+//
+STATIC CPU_HOT_PLUG_DATA *mCpuHotPlugData;
+//
+// SMRAM arrays for fetching the APIC IDs of processors with pending events (of
+// known event types), for the time of just one MMI.
+//
+// The lifetimes of these arrays match that of this driver only because we
+// don't want to allocate SMRAM at OS runtime, and potentially fail (or
+// fragment the SMRAM map).
+//
+// These arrays provide room for ("possible CPU count" minus one) APIC IDs
+// each, as we don't expect every possible CPU to appear, or disappear, in a
+// single MMI. The numbers of used (populated) elements in the arrays are
+// determined on every MMI separately.
+//
+STATIC APIC_ID *mPluggedApicIds;
+STATIC APIC_ID *mToUnplugApicIds;
+//
// Represents the registration of the CPU Hotplug MMI handler.
//
STATIC EFI_HANDLE mDispatchHandle;
/**
CPU Hotplug MMI handler function.
This is a root MMI handler.
@param[in] DispatchHandle The unique handle assigned to this handler by
EFI_MM_SYSTEM_TABLE.MmiHandlerRegister().
@param[in] Context Context passed in by
EFI_MM_SYSTEM_TABLE.MmiManage(). Due to
CpuHotplugMmi() being a root MMI handler,
Context is ASSERT()ed to be NULL.
@param[in,out] CommBuffer Ignored, due to CpuHotplugMmi() being a root
MMI handler.
@param[in,out] CommBufferSize Ignored, due to CpuHotplugMmi() being a root
@@ -65,162 +95,239 @@ STATIC EFI_HANDLE mDispatchHandle;
but other handlers should still
be called.
@retval EFI_WARN_INTERRUPT_SOURCE_PENDING The MMI source is still pending,
and other handlers should still
be called.
@retval EFI_INTERRUPT_PENDING The MMI source could not be
quiesced.
**/
STATIC
EFI_STATUS
EFIAPI
CpuHotplugMmi (
IN EFI_HANDLE DispatchHandle,
IN CONST VOID *Context OPTIONAL,
IN OUT VOID *CommBuffer OPTIONAL,
IN OUT UINTN *CommBufferSize OPTIONAL
)
{
EFI_STATUS Status;
UINT8 ApmControl;
+ UINT32 PluggedCount;
+ UINT32 ToUnplugCount;
//
// Assert that we are entering this function due to our root MMI handler
// registration.
//
ASSERT (DispatchHandle == mDispatchHandle);
//
// When MmiManage() is invoked to process root MMI handlers, the caller (the
// MM Core) is expected to pass in a NULL Context. MmiManage() then passes
// the same NULL Context to individual handlers.
//
ASSERT (Context == NULL);
//
// Read the MMI command value from the APM Control Port, to see if this is an
// MMI we should care about.
//
Status = mMmCpuIo->Io.Read (mMmCpuIo, MM_IO_UINT8, ICH9_APM_CNT, 1,
&ApmControl);
if (EFI_ERROR (Status)) {
DEBUG ((DEBUG_ERROR, "%a: failed to read ICH9_APM_CNT: %r\n", __FUNCTION__,
Status));
//
// We couldn't even determine if the MMI was for us or not.
//
goto Fatal;
}
if (ApmControl != ICH9_APM_CNT_CPU_HOTPLUG) {
//
// The MMI is not for us.
//
return EFI_WARN_INTERRUPT_SOURCE_QUIESCED;
}
+ //
+ // Collect the CPUs with pending events.
+ //
+ Status = QemuCpuhpCollectApicIds (
+ mMmCpuIo,
+ mCpuHotPlugData->ArrayLength, // PossibleCpuCount
+ mCpuHotPlugData->ArrayLength - 1, // ApicIdCount
+ mPluggedApicIds,
+ &PluggedCount,
+ mToUnplugApicIds,
+ &ToUnplugCount
+ );
+ if (EFI_ERROR (Status)) {
+ goto Fatal;
+ }
+ if (ToUnplugCount > 0) {
+ DEBUG ((DEBUG_ERROR, "%a: hot-unplug is not supported yet\n",
+ __FUNCTION__));
+ goto Fatal;
+ }
+
//
// We've handled this MMI.
//
return EFI_SUCCESS;
Fatal:
ASSERT (FALSE);
CpuDeadLoop ();
//
// We couldn't handle this MMI.
//
return EFI_INTERRUPT_PENDING;
}
//
// Entry point function of this driver.
//
EFI_STATUS
EFIAPI
CpuHotplugEntry (
IN EFI_HANDLE ImageHandle,
IN EFI_SYSTEM_TABLE *SystemTable
)
{
EFI_STATUS Status;
+ UINTN Size;
//
// This module should only be included when SMM support is required.
//
ASSERT (FeaturePcdGet (PcdSmmSmramRequire));
//
// This driver depends on the dynamically detected "SMRAM at default SMBASE"
// feature.
//
if (!PcdGetBool (PcdQ35SmramAtDefaultSmbase)) {
return EFI_UNSUPPORTED;
}
//
// Errors from here on are fatal; we cannot allow the boot to proceed if we
// can't set up this driver to handle CPU hotplug.
//
// First, collect the protocols needed later. All of these protocols are
// listed in our module DEPEX.
//
Status = gMmst->MmLocateProtocol (&gEfiMmCpuIoProtocolGuid,
NULL /* Registration */, (VOID **)&mMmCpuIo);
if (EFI_ERROR (Status)) {
DEBUG ((DEBUG_ERROR, "%a: locate MmCpuIo: %r\n", __FUNCTION__, Status));
goto Fatal;
}
+ Status = gMmst->MmLocateProtocol (&gEfiSmmCpuServiceProtocolGuid,
+ NULL /* Registration */, (VOID **)&mMmCpuService);
+ if (EFI_ERROR (Status)) {
+ DEBUG ((DEBUG_ERROR, "%a: locate MmCpuService: %r\n", __FUNCTION__,
+ Status));
+ goto Fatal;
+ }
+
+ //
+ // Our DEPEX on EFI_SMM_CPU_SERVICE_PROTOCOL guarantees that PiSmmCpuDxeSmm
+ // has pointed PcdCpuHotPlugDataAddress to CPU_HOT_PLUG_DATA in SMRAM.
+ //
+ mCpuHotPlugData = (VOID *)(UINTN)PcdGet64 (PcdCpuHotPlugDataAddress);
+ if (mCpuHotPlugData == NULL) {
+ Status = EFI_NOT_FOUND;
+ DEBUG ((DEBUG_ERROR, "%a: CPU_HOT_PLUG_DATA: %r\n", __FUNCTION__, Status));
+ goto Fatal;
+ }
+ //
+ // If the possible CPU count is 1, there's nothing for this driver to do.
+ //
+ if (mCpuHotPlugData->ArrayLength == 1) {
+ return EFI_UNSUPPORTED;
+ }
+ //
+ // Allocate the data structures that depend on the possible CPU count.
+ //
+ if (RETURN_ERROR (SafeUintnSub (mCpuHotPlugData->ArrayLength, 1, &Size)) ||
+ RETURN_ERROR (SafeUintnMult (sizeof (APIC_ID), Size, &Size))) {
+ Status = EFI_ABORTED;
+ DEBUG ((DEBUG_ERROR, "%a: invalid CPU_HOT_PLUG_DATA\n", __FUNCTION__));
+ goto Fatal;
+ }
+ Status = gMmst->MmAllocatePool (EfiRuntimeServicesData, Size,
+ (VOID **)&mPluggedApicIds);
+ if (EFI_ERROR (Status)) {
+ DEBUG ((DEBUG_ERROR, "%a: MmAllocatePool(): %r\n", __FUNCTION__, Status));
+ goto Fatal;
+ }
+ Status = gMmst->MmAllocatePool (EfiRuntimeServicesData, Size,
+ (VOID **)&mToUnplugApicIds);
+ if (EFI_ERROR (Status)) {
+ DEBUG ((DEBUG_ERROR, "%a: MmAllocatePool(): %r\n", __FUNCTION__, Status));
+ goto ReleasePluggedApicIds;
+ }
//
// Sanity-check the CPU hotplug interface.
//
// Both of the following features are part of QEMU 5.0, introduced primarily
// in commit range 3e08b2b9cb64..3a61c8db9d25:
//
// (a) the QEMU_CPUHP_CMD_GET_ARCH_ID command of the modern CPU hotplug
// interface,
//
// (b) the "SMRAM at default SMBASE" feature.
//
// From these, (b) is restricted to 5.0+ machine type versions, while (a)
// does not depend on machine type version. Because we ensured the stricter
// condition (b) through PcdQ35SmramAtDefaultSmbase above, the (a)
// QEMU_CPUHP_CMD_GET_ARCH_ID command must now be available too. While we
// can't verify the presence of precisely that command, we can still verify
// (sanity-check) that the modern interface is active, at least.
//
// Consult the "Typical usecases | Detecting and enabling modern CPU hotplug
// interface" section in QEMU's "docs/specs/acpi_cpu_hotplug.txt", on the
// following.
//
QemuCpuhpWriteCpuSelector (mMmCpuIo, 0);
QemuCpuhpWriteCpuSelector (mMmCpuIo, 0);
QemuCpuhpWriteCommand (mMmCpuIo, QEMU_CPUHP_CMD_GET_PENDING);
if (QemuCpuhpReadCommandData2 (mMmCpuIo) != 0) {
Status = EFI_NOT_FOUND;
DEBUG ((DEBUG_ERROR, "%a: modern CPU hotplug interface: %r\n",
__FUNCTION__, Status));
- goto Fatal;
+ goto ReleaseToUnplugApicIds;
}
//
// Register the handler for the CPU Hotplug MMI.
//
Status = gMmst->MmiHandlerRegister (
CpuHotplugMmi,
NULL, // HandlerType: root MMI handler
&mDispatchHandle
);
if (EFI_ERROR (Status)) {
DEBUG ((DEBUG_ERROR, "%a: MmiHandlerRegister(): %r\n", __FUNCTION__,
Status));
- goto Fatal;
+ goto ReleaseToUnplugApicIds;
}
return EFI_SUCCESS;
+ReleaseToUnplugApicIds:
+ gMmst->MmFreePool (mToUnplugApicIds);
+ mToUnplugApicIds = NULL;
+
+ReleasePluggedApicIds:
+ gMmst->MmFreePool (mPluggedApicIds);
+ mPluggedApicIds = NULL;
+
Fatal:
ASSERT (FALSE);
CpuDeadLoop ();
return Status;
}
--
2.19.1.3.g30247aa5d201
^ permalink raw reply related [flat|nested] 24+ messages in thread* [PATCH 11/16] OvmfPkg/CpuHotplugSmm: introduce Post-SMM Pen for hot-added CPUs
2020-02-23 17:25 [PATCH 00/16] OvmfPkg: support VCPU hotplug with -D SMM_REQUIRE Laszlo Ersek
` (9 preceding siblings ...)
2020-02-23 17:25 ` [PATCH 10/16] OvmfPkg/CpuHotplugSmm: collect " Laszlo Ersek
@ 2020-02-23 17:25 ` Laszlo Ersek
2020-02-23 17:25 ` [PATCH 12/16] OvmfPkg/CpuHotplugSmm: introduce First SMI Handler " Laszlo Ersek
` (6 subsequent siblings)
17 siblings, 0 replies; 24+ messages in thread
From: Laszlo Ersek @ 2020-02-23 17:25 UTC (permalink / raw)
To: edk2-devel-groups-io
Cc: Ard Biesheuvel, Igor Mammedov, Jiewen Yao, Jordan Justen,
Michael Kinney, Philippe Mathieu-Daudé
Once a hot-added CPU finishes the SMBASE relocation, we need to pen it in
a HLT loop. Add the NASM implementation (with just a handful of
instructions, but much documentation), and some C language helper
functions.
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Michael Kinney <michael.d.kinney@intel.com>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=1512
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
---
OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf | 4 +
OvmfPkg/CpuHotplugSmm/Smbase.h | 32 +++++
OvmfPkg/CpuHotplugSmm/PostSmmPen.nasm | 137 ++++++++++++++++++++
OvmfPkg/CpuHotplugSmm/Smbase.c | 110 ++++++++++++++++
4 files changed, 283 insertions(+)
diff --git a/OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf b/OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf
index 31c1ee1c9f6d..bf4162299c7c 100644
--- a/OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf
+++ b/OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf
@@ -5,52 +5,56 @@
#
# SPDX-License-Identifier: BSD-2-Clause-Patent
##
[Defines]
INF_VERSION = 1.29
PI_SPECIFICATION_VERSION = 0x00010046 # PI-1.7.0
BASE_NAME = CpuHotplugSmm
FILE_GUID = 84EEA114-C6BE-4445-8F90-51D97863E363
MODULE_TYPE = DXE_SMM_DRIVER
ENTRY_POINT = CpuHotplugEntry
#
# The following information is for reference only and not required by the build
# tools.
#
# VALID_ARCHITECTURES = IA32 X64
#
[Sources]
ApicId.h
CpuHotplug.c
+ PostSmmPen.nasm
QemuCpuhp.c
QemuCpuhp.h
+ Smbase.c
+ Smbase.h
[Packages]
MdePkg/MdePkg.dec
OvmfPkg/OvmfPkg.dec
UefiCpuPkg/UefiCpuPkg.dec
[LibraryClasses]
BaseLib
+ BaseMemoryLib
DebugLib
MmServicesTableLib
PcdLib
SafeIntLib
UefiDriverEntryPoint
[Protocols]
gEfiMmCpuIoProtocolGuid ## CONSUMES
gEfiSmmCpuServiceProtocolGuid ## CONSUMES
[Pcd]
gUefiCpuPkgTokenSpaceGuid.PcdCpuHotPlugDataAddress ## CONSUMES
gUefiOvmfPkgTokenSpaceGuid.PcdQ35SmramAtDefaultSmbase ## CONSUMES
[FeaturePcd]
gUefiOvmfPkgTokenSpaceGuid.PcdSmmSmramRequire ## CONSUMES
[Depex]
gEfiMmCpuIoProtocolGuid AND
gEfiSmmCpuServiceProtocolGuid
diff --git a/OvmfPkg/CpuHotplugSmm/Smbase.h b/OvmfPkg/CpuHotplugSmm/Smbase.h
new file mode 100644
index 000000000000..cb5aed98cdd3
--- /dev/null
+++ b/OvmfPkg/CpuHotplugSmm/Smbase.h
@@ -0,0 +1,32 @@
+/** @file
+ SMBASE relocation for hot-plugged CPUs.
+
+ Copyright (c) 2020, Red Hat, Inc.
+
+ SPDX-License-Identifier: BSD-2-Clause-Patent
+**/
+
+#ifndef SMBASE_H_
+#define SMBASE_H_
+
+#include <Uefi/UefiBaseType.h> // EFI_STATUS
+#include <Uefi/UefiSpec.h> // EFI_BOOT_SERVICES
+
+EFI_STATUS
+SmbaseAllocatePostSmmPen (
+ OUT UINT32 *PenAddress,
+ IN CONST EFI_BOOT_SERVICES *BootServices
+ );
+
+VOID
+SmbaseReinstallPostSmmPen (
+ IN UINT32 PenAddress
+ );
+
+VOID
+SmbaseReleasePostSmmPen (
+ IN UINT32 PenAddress,
+ IN CONST EFI_BOOT_SERVICES *BootServices
+ );
+
+#endif // SMBASE_H_
diff --git a/OvmfPkg/CpuHotplugSmm/PostSmmPen.nasm b/OvmfPkg/CpuHotplugSmm/PostSmmPen.nasm
new file mode 100644
index 000000000000..3a328be29b1f
--- /dev/null
+++ b/OvmfPkg/CpuHotplugSmm/PostSmmPen.nasm
@@ -0,0 +1,137 @@
+;------------------------------------------------------------------------------
+; @file
+; Pen any hot-added CPU in a 16-bit, real mode HLT loop, after it leaves SMM by
+; executing the RSM instruction.
+;
+; Copyright (c) 2020, Red Hat, Inc.
+;
+; SPDX-License-Identifier: BSD-2-Clause-Patent
+;
+; The routine implemented here is stored into normal RAM, under 1MB, at the
+; beginning of a page that is allocated as EfiReservedMemoryType. On any
+; hot-added CPU, it is executed after *at least* the first RSM (i.e., after
+; SMBASE relocation).
+;
+; The first execution of this code occurs as follows:
+;
+; - The hot-added CPU is in RESET state.
+;
+; - The ACPI CPU hotplug event handler triggers a broadcast SMI, from the OS.
+;
+; - Existent CPUs (BSP and APs) enter SMM.
+;
+; - The hot-added CPU remains in RESET state, but an SMI is pending for it now.
+; (See "SYSTEM MANAGEMENT INTERRUPT (SMI)" in the Intel SDM.)
+;
+; - In SMM, pre-existent CPUs that are not elected SMM Monarch, keep themselves
+; busy with their wait loops.
+;
+; - From the root MMI handler, the SMM Monarch:
+;
+; - places this routine in the reserved page,
+;
+; - clears the last byte of the reserved page,
+;
+; - sends an INIT-SIPI-SIPI sequence to the hot-added CPU,
+;
+; - un-gates the default SMI handler by APIC ID.
+;
+; - The startup vector in the SIPI that is sent by the SMM Monarch points to
+; this code; i.e., to the reserved page. (Example: 0x9_F000.)
+;
+; - The SMM Monarch starts polling the last byte in the reserved page.
+;
+; - The hot-added CPU boots, and immediately enters SMM due to the pending SMI.
+; It starts executing the default SMI handler.
+;
+; - Importantly, the SMRAM Save State Map captures the following information,
+; when the hot-added CPU enters SMM:
+;
+; - CS selector: assumes the 16 most significant bits of the 20-bit (i.e.,
+; below 1MB) startup vector from the SIPI. (Example: 0x9F00.)
+;
+; - CS attributes: Accessed, Readable, User (S=1), CodeSegment (bit#11),
+; Present.
+;
+; - CS limit: 0xFFFF.
+;
+; - CS base: the CS selector value shifted left by 4 bits. That is, the CS
+; base equals the SIPI startup vector. (Example: 0x9_F000.)
+;
+; - IP: the least significant 4 bits from the SIPI startup vector. Because
+; the routine is page-aligned, these bits are zero (hence IP is zero).
+;
+; - ES, SS, DS, FS, GS selectors: 0.
+;
+; - ES, SS, DS, FS, GS attributes: same as the CS attributes, minus
+; CodeSegment (bit#11).
+;
+; - ES, SS, DS, FS, GS limits: 0xFFFF.
+;
+; - ES, SS, DS, FS, GS bases: 0.
+;
+; - The hot-added CPU performs SMBASE relocation, then executes the RSM
+; instruction, leaving SMM.
+;
+; - The hot-added CPU jumps ("returns") to the code below (in the reserved
+; page), according to the register state listed in the SMRAM Save State Map.
+;
+; - The hot-added CPU sets the last byte of the reserved page, then halts
+; itself.
+;
+; - The SMM Monarch notices that the hot-added CPU is done with SMBASE
+; relocation.
+;
+; Note that, if the OS is malicious and sends INIT-SIPI-SIPI to the hot-added
+; CPU before allowing the ACPI CPU hotplug event handler to trigger a broadcast
+; SMI, then said broadcast SMI will yank the hot-added CPU directly into SMM,
+; without becoming pending for it (as the hot-added CPU is no longer in RESET
+; state). This is OK, because:
+;
+; - The default SMI handler copes with this, as it is gated by APIC ID. The
+; hot-added CPU won't start the actual SMBASE relocation until the SMM
+; Monarch lets it.
+;
+; - The INIT-SIPI-SIPI sequence that the SMM Monarch sends to the hot-added CPU
+; will be ignored in this sate (it won't even be latched). See "SMI HANDLER
+; EXECUTION ENVIRONMENT" in the Intel SDM: "INIT operations are inhibited
+; when the processor enters SMM".
+;
+; - When the hot-added CPU executes the RSM (having relocated SMBASE), it
+; returns to the OS.
+;
+; In other words, we do not / need not prevent a malicious OS from booting the
+; hot-added CPU early; instead we provide benign OSes with a pen for hot-added
+; CPUs.
+;------------------------------------------------------------------------------
+
+SECTION .data
+BITS 16
+
+GLOBAL ASM_PFX (mPostSmmPen) ; UINT8[]
+GLOBAL ASM_PFX (mPostSmmPenSize) ; UINT16
+
+ASM_PFX (mPostSmmPen):
+ ;
+ ; Point DS at the same reserved page.
+ ;
+ mov ax, cs
+ mov ds, ax
+
+ ;
+ ; Inform the SMM Monarch that we're done with SMBASE relocation, by setting
+ ; the last byte in the reserved page.
+ ;
+ mov byte [ds : word 0xFFF], 1
+
+ ;
+ ; Halt now, until we get woken by another SMI, or (more likely) the OS
+ ; reboots us with another INIT-SIPI-SIPI.
+ ;
+HltLoop:
+ cli
+ hlt
+ jmp HltLoop
+
+ASM_PFX (mPostSmmPenSize):
+ dw $ - ASM_PFX (mPostSmmPen)
diff --git a/OvmfPkg/CpuHotplugSmm/Smbase.c b/OvmfPkg/CpuHotplugSmm/Smbase.c
new file mode 100644
index 000000000000..ea21153d9145
--- /dev/null
+++ b/OvmfPkg/CpuHotplugSmm/Smbase.c
@@ -0,0 +1,110 @@
+/** @file
+ SMBASE relocation for hot-plugged CPUs.
+
+ Copyright (c) 2020, Red Hat, Inc.
+
+ SPDX-License-Identifier: BSD-2-Clause-Patent
+**/
+
+#include <Base.h> // BASE_1MB
+#include <Library/BaseMemoryLib.h> // CopyMem()
+#include <Library/DebugLib.h> // DEBUG()
+
+#include "Smbase.h"
+
+extern CONST UINT8 mPostSmmPen[];
+extern CONST UINT16 mPostSmmPenSize;
+
+/**
+ Allocate a non-SMRAM reserved memory page for the Post-SMM Pen for hot-added
+ CPUs.
+
+ This function may only be called from the entry point function of the driver.
+
+ @param[out] PenAddress The address of the allocated (normal RAM) reserved
+ page.
+
+ @param[in] BootServices Pointer to the UEFI boot services table. Used for
+ allocating the normal RAM (not SMRAM) reserved page.
+
+ @retval EFI_SUCCESS Allocation successful.
+
+ @retval EFI_BAD_BUFFER_SIZE The Post-SMM Pen template is not smaller than
+ EFI_PAGE_SIZE.
+
+ @return Error codes propagated from underlying services.
+ DEBUG_ERROR messages have been logged. No
+ resources have been allocated.
+**/
+EFI_STATUS
+SmbaseAllocatePostSmmPen (
+ OUT UINT32 *PenAddress,
+ IN CONST EFI_BOOT_SERVICES *BootServices
+ )
+{
+ EFI_STATUS Status;
+ EFI_PHYSICAL_ADDRESS Address;
+
+ //
+ // The pen code must fit in one page, and the last byte must remain free for
+ // signaling the SMM Monarch.
+ //
+ if (mPostSmmPenSize >= EFI_PAGE_SIZE) {
+ Status = EFI_BAD_BUFFER_SIZE;
+ DEBUG ((DEBUG_ERROR, "%a: mPostSmmPenSize=%u: %r\n", __FUNCTION__,
+ mPostSmmPenSize, Status));
+ return Status;
+ }
+
+ Address = BASE_1MB - 1;
+ Status = BootServices->AllocatePages (AllocateMaxAddress,
+ EfiReservedMemoryType, 1, &Address);
+ if (EFI_ERROR (Status)) {
+ DEBUG ((DEBUG_ERROR, "%a: AllocatePages(): %r\n", __FUNCTION__, Status));
+ return Status;
+ }
+
+ DEBUG ((DEBUG_INFO, "%a: Post-SMM Pen at 0x%Lx\n", __FUNCTION__, Address));
+ *PenAddress = (UINT32)Address;
+ return EFI_SUCCESS;
+}
+
+/**
+ Copy the Post-SMM Pen template code into the reserved page allocated with
+ SmbaseAllocatePostSmmPen().
+
+ Note that this effects an "SMRAM to normal RAM" copy.
+
+ The SMM Monarch is supposed to call this function from the root MMI handler.
+
+ @param[in] PenAddress The allocation address returned by
+ SmbaseAllocatePostSmmPen().
+**/
+VOID
+SmbaseReinstallPostSmmPen (
+ IN UINT32 PenAddress
+ )
+{
+ CopyMem ((VOID *)(UINTN)PenAddress, mPostSmmPen, mPostSmmPenSize);
+}
+
+/**
+ Release the reserved page allocated with SmbaseAllocatePostSmmPen().
+
+ This function may only be called from the entry point function of the driver,
+ on the error path.
+
+ @param[in] PenAddress The allocation address returned by
+ SmbaseAllocatePostSmmPen().
+
+ @param[in] BootServices Pointer to the UEFI boot services table. Used for
+ releasing the normal RAM (not SMRAM) reserved page.
+**/
+VOID
+SmbaseReleasePostSmmPen (
+ IN UINT32 PenAddress,
+ IN CONST EFI_BOOT_SERVICES *BootServices
+ )
+{
+ BootServices->FreePages (PenAddress, 1);
+}
--
2.19.1.3.g30247aa5d201
^ permalink raw reply related [flat|nested] 24+ messages in thread* [PATCH 12/16] OvmfPkg/CpuHotplugSmm: introduce First SMI Handler for hot-added CPUs
2020-02-23 17:25 [PATCH 00/16] OvmfPkg: support VCPU hotplug with -D SMM_REQUIRE Laszlo Ersek
` (10 preceding siblings ...)
2020-02-23 17:25 ` [PATCH 11/16] OvmfPkg/CpuHotplugSmm: introduce Post-SMM Pen for hot-added CPUs Laszlo Ersek
@ 2020-02-23 17:25 ` Laszlo Ersek
2020-02-24 9:10 ` [edk2-devel] " Laszlo Ersek
2020-02-23 17:25 ` [PATCH 13/16] OvmfPkg/CpuHotplugSmm: complete root MMI handler for CPU hotplug Laszlo Ersek
` (5 subsequent siblings)
17 siblings, 1 reply; 24+ messages in thread
From: Laszlo Ersek @ 2020-02-23 17:25 UTC (permalink / raw)
To: edk2-devel-groups-io
Cc: Ard Biesheuvel, Igor Mammedov, Jiewen Yao, Jordan Justen,
Michael Kinney, Philippe Mathieu-Daudé
Implement the First SMI Handler for hot-added CPUs, in NASM.
Add the interfacing C-language function that the SMM Monarch calls. This
function launches and coordinates SMBASE relocation for a hot-added CPU.
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Michael Kinney <michael.d.kinney@intel.com>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=1512
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
---
OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf | 4 +
OvmfPkg/CpuHotplugSmm/FirstSmiHandlerContext.h | 41 ++++++
OvmfPkg/CpuHotplugSmm/Smbase.h | 14 ++
OvmfPkg/CpuHotplugSmm/FirstSmiHandler.nasm | 149 ++++++++++++++++++++
OvmfPkg/CpuHotplugSmm/Smbase.c | 142 +++++++++++++++++++
5 files changed, 350 insertions(+)
diff --git a/OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf b/OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf
index bf4162299c7c..04322b0d7855 100644
--- a/OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf
+++ b/OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf
@@ -5,56 +5,60 @@
#
# SPDX-License-Identifier: BSD-2-Clause-Patent
##
[Defines]
INF_VERSION = 1.29
PI_SPECIFICATION_VERSION = 0x00010046 # PI-1.7.0
BASE_NAME = CpuHotplugSmm
FILE_GUID = 84EEA114-C6BE-4445-8F90-51D97863E363
MODULE_TYPE = DXE_SMM_DRIVER
ENTRY_POINT = CpuHotplugEntry
#
# The following information is for reference only and not required by the build
# tools.
#
# VALID_ARCHITECTURES = IA32 X64
#
[Sources]
ApicId.h
CpuHotplug.c
+ FirstSmiHandler.nasm
+ FirstSmiHandlerContext.h
PostSmmPen.nasm
QemuCpuhp.c
QemuCpuhp.h
Smbase.c
Smbase.h
[Packages]
MdePkg/MdePkg.dec
OvmfPkg/OvmfPkg.dec
UefiCpuPkg/UefiCpuPkg.dec
[LibraryClasses]
BaseLib
BaseMemoryLib
DebugLib
+ LocalApicLib
MmServicesTableLib
PcdLib
SafeIntLib
+ SynchronizationLib
UefiDriverEntryPoint
[Protocols]
gEfiMmCpuIoProtocolGuid ## CONSUMES
gEfiSmmCpuServiceProtocolGuid ## CONSUMES
[Pcd]
gUefiCpuPkgTokenSpaceGuid.PcdCpuHotPlugDataAddress ## CONSUMES
gUefiOvmfPkgTokenSpaceGuid.PcdQ35SmramAtDefaultSmbase ## CONSUMES
[FeaturePcd]
gUefiOvmfPkgTokenSpaceGuid.PcdSmmSmramRequire ## CONSUMES
[Depex]
gEfiMmCpuIoProtocolGuid AND
gEfiSmmCpuServiceProtocolGuid
diff --git a/OvmfPkg/CpuHotplugSmm/FirstSmiHandlerContext.h b/OvmfPkg/CpuHotplugSmm/FirstSmiHandlerContext.h
new file mode 100644
index 000000000000..7806a5b2ad03
--- /dev/null
+++ b/OvmfPkg/CpuHotplugSmm/FirstSmiHandlerContext.h
@@ -0,0 +1,41 @@
+/** @file
+ Define the FIRST_SMI_HANDLER_CONTEXT structure, which is an exchange area
+ between the SMM Monarch and the hot-added CPU, for relocating the SMBASE of
+ the hot-added CPU.
+
+ Copyright (c) 2020, Red Hat, Inc.
+
+ SPDX-License-Identifier: BSD-2-Clause-Patent
+**/
+
+#ifndef FIRST_SMI_HANDLER_CONTEXT_H_
+#define FIRST_SMI_HANDLER_CONTEXT_H_
+
+//
+// The following structure is used to communicate between the SMM Monarch
+// (running the root MMI handler) and the hot-added CPU (handling its first
+// SMI). It is placed at SMM_DEFAULT_SMBASE, which is in SMRAM under QEMU's
+// "SMRAM at default SMBASE" feature.
+//
+#pragma pack (1)
+typedef struct {
+ //
+ // When ApicIdGate is MAX_UINT64, then no hot-added CPU may proceed with
+ // SMBASE relocation.
+ //
+ // Otherwise, the hot-added CPU whose APIC ID equals ApicIdGate may proceed
+ // with SMBASE relocation.
+ //
+ // This field is intentionally wider than APIC_ID (UINT32) because we need a
+ // "gate locked" value that is different from all possible APIC_IDs.
+ //
+ UINT64 ApicIdGate;
+ //
+ // The new SMBASE value for the hot-added CPU to set in the SMRAM Save State
+ // Map, before leaving SMM with the RSM instruction.
+ //
+ UINT32 NewSmbase;
+} FIRST_SMI_HANDLER_CONTEXT;
+#pragma pack ()
+
+#endif // FIRST_SMI_HANDLER_CONTEXT_H_
diff --git a/OvmfPkg/CpuHotplugSmm/Smbase.h b/OvmfPkg/CpuHotplugSmm/Smbase.h
index cb5aed98cdd3..e73730d19926 100644
--- a/OvmfPkg/CpuHotplugSmm/Smbase.h
+++ b/OvmfPkg/CpuHotplugSmm/Smbase.h
@@ -1,32 +1,46 @@
/** @file
SMBASE relocation for hot-plugged CPUs.
Copyright (c) 2020, Red Hat, Inc.
SPDX-License-Identifier: BSD-2-Clause-Patent
**/
#ifndef SMBASE_H_
#define SMBASE_H_
#include <Uefi/UefiBaseType.h> // EFI_STATUS
#include <Uefi/UefiSpec.h> // EFI_BOOT_SERVICES
+#include "ApicId.h" // APIC_ID
+
EFI_STATUS
SmbaseAllocatePostSmmPen (
OUT UINT32 *PenAddress,
IN CONST EFI_BOOT_SERVICES *BootServices
);
VOID
SmbaseReinstallPostSmmPen (
IN UINT32 PenAddress
);
VOID
SmbaseReleasePostSmmPen (
IN UINT32 PenAddress,
IN CONST EFI_BOOT_SERVICES *BootServices
);
+VOID
+SmbaseInstallFirstSmiHandler (
+ VOID
+ );
+
+EFI_STATUS
+SmbaseRelocate (
+ IN APIC_ID ApicId,
+ IN UINTN Smbase,
+ IN UINT32 PenAddress
+ );
+
#endif // SMBASE_H_
diff --git a/OvmfPkg/CpuHotplugSmm/FirstSmiHandler.nasm b/OvmfPkg/CpuHotplugSmm/FirstSmiHandler.nasm
new file mode 100644
index 000000000000..d5ce3472bd14
--- /dev/null
+++ b/OvmfPkg/CpuHotplugSmm/FirstSmiHandler.nasm
@@ -0,0 +1,149 @@
+;------------------------------------------------------------------------------
+; @file
+; Relocate the SMBASE on a hot-added CPU when it services its first SMI.
+;
+; Copyright (c) 2020, Red Hat, Inc.
+;
+; SPDX-License-Identifier: BSD-2-Clause-Patent
+;
+; The routine runs on the hot-added CPU in the following "big real mode",
+; 16-bit environment; per "SMI HANDLER EXECUTION ENVIRONMENT" in the Intel SDM
+; (table "Processor Register Initialization in SMM"):
+;
+; - CS selector: 0x3000 (most significant 16 bits of SMM_DEFAULT_SMBASE).
+;
+; - CS limit: 0xFFFF_FFFF.
+;
+; - CS base: SMM_DEFAULT_SMBASE (0x3_0000).
+;
+; - IP: SMM_HANDLER_OFFSET (0x8000).
+;
+; - ES, SS, DS, FS, GS selectors: 0.
+;
+; - ES, SS, DS, FS, GS limits: 0xFFFF_FFFF.
+;
+; - ES, SS, DS, FS, GS bases: 0.
+;
+; - Operand-size and address-size override prefixes can be used to access the
+; address space beyond 1MB.
+;------------------------------------------------------------------------------
+
+SECTION .data
+BITS 16
+
+;
+; Bring in SMM_DEFAULT_SMBASE from
+; "MdePkg/Include/Register/Intel/SmramSaveStateMap.h".
+;
+SMM_DEFAULT_SMBASE: equ 0x3_0000
+
+;
+; Field offsets in FIRST_SMI_HANDLER_CONTEXT, which resides at
+; SMM_DEFAULT_SMBASE.
+;
+ApicIdGate: equ 0 ; UINT64
+NewSmbase: equ 8 ; UINT32
+
+;
+; SMRAM Save State Map field offsets, per the AMD (not Intel) layout that QEMU
+; implements. Relative to SMM_DEFAULT_SMBASE.
+;
+SaveStateRevId: equ 0xFEFC ; UINT32
+SaveStateSmbase: equ 0xFEF8 ; UINT32
+SaveStateSmbase64: equ 0xFF00 ; UINT32
+
+;
+; CPUID constants, from "MdePkg/Include/Register/Intel/Cpuid.h".
+;
+CPUID_SIGNATURE: equ 0x00
+CPUID_EXTENDED_TOPOLOGY: equ 0x0B
+CPUID_VERSION_INFO: equ 0x01
+
+GLOBAL ASM_PFX (mFirstSmiHandler) ; UINT8[]
+GLOBAL ASM_PFX (mFirstSmiHandlerSize) ; UINT16
+
+ASM_PFX (mFirstSmiHandler):
+ ;
+ ; Get our own APIC ID first, so we can contend for ApicIdGate.
+ ;
+ ; This basically reimplements GetInitialApicId() from
+ ; "UefiCpuPkg/Library/BaseXApicLib/BaseXApicLib.c".
+ ;
+ mov eax, CPUID_SIGNATURE
+ cpuid
+ cmp eax, CPUID_EXTENDED_TOPOLOGY
+ jb GetApicIdFromVersionInfo
+
+ mov eax, CPUID_EXTENDED_TOPOLOGY
+ mov ecx, 0
+ cpuid
+ test ebx, 0xFFFF
+ jz GetApicIdFromVersionInfo
+
+ ;
+ ; EDX has the APIC ID, save it to ESI.
+ ;
+ mov esi, edx
+ jmp KnockOnGate
+
+GetApicIdFromVersionInfo:
+ mov eax, CPUID_VERSION_INFO
+ cpuid
+ shr ebx, 24
+ ;
+ ; EBX has the APIC ID, save it to ESI.
+ ;
+ mov esi, ebx
+
+KnockOnGate:
+ ;
+ ; See if ApicIdGate shows our own APIC ID. If so, swap it to MAX_UINT64
+ ; (close the gate), and advance. Otherwise, keep knocking.
+ ;
+ ; InterlockedCompareExchange64():
+ ; - Value := &FIRST_SMI_HANDLER_CONTEXT.ApicIdGate
+ ; - CompareValue (EDX:EAX) := APIC ID (from ESI)
+ ; - ExchangeValue (ECX:EBX) := MAX_UINT64
+ ;
+ mov edx, 0
+ mov eax, esi
+ mov ecx, 0xFFFF_FFFF
+ mov ebx, 0xFFFF_FFFF
+ lock cmpxchg8b [ds : dword (SMM_DEFAULT_SMBASE + ApicIdGate)]
+ jz ApicIdMatch
+ pause
+ jmp KnockOnGate
+
+ApicIdMatch:
+ ;
+ ; Update the SMBASE field in the SMRAM Save State Map.
+ ;
+ ; First, calculate the address of the SMBASE field, based on the SMM Revision
+ ; ID; store the result in EBX.
+ ;
+ mov eax, dword [ds : dword (SMM_DEFAULT_SMBASE + SaveStateRevId)]
+ test eax, 0xFFFF
+ jz LegacySaveStateMap
+
+ mov ebx, SMM_DEFAULT_SMBASE + SaveStateSmbase64
+ jmp UpdateSmbase
+
+LegacySaveStateMap:
+ mov ebx, SMM_DEFAULT_SMBASE + SaveStateSmbase
+
+UpdateSmbase:
+ ;
+ ; Load the new SMBASE value into EAX.
+ ;
+ mov eax, dword [ds : dword (SMM_DEFAULT_SMBASE + NewSmbase)]
+ ;
+ ; Save it to the SMBASE field whose address we calculated in EBX.
+ ;
+ mov dword [ds : dword ebx], eax
+ ;
+ ; We're done; leave SMM and continue to the pen.
+ ;
+ rsm
+
+ASM_PFX (mFirstSmiHandlerSize):
+ dw $ - ASM_PFX (mFirstSmiHandler)
diff --git a/OvmfPkg/CpuHotplugSmm/Smbase.c b/OvmfPkg/CpuHotplugSmm/Smbase.c
index ea21153d9145..57c9e86f3e93 100644
--- a/OvmfPkg/CpuHotplugSmm/Smbase.c
+++ b/OvmfPkg/CpuHotplugSmm/Smbase.c
@@ -1,38 +1,46 @@
/** @file
SMBASE relocation for hot-plugged CPUs.
Copyright (c) 2020, Red Hat, Inc.
SPDX-License-Identifier: BSD-2-Clause-Patent
**/
#include <Base.h> // BASE_1MB
+#include <Library/BaseLib.h> // CpuPause()
#include <Library/BaseMemoryLib.h> // CopyMem()
#include <Library/DebugLib.h> // DEBUG()
+#include <Library/LocalApicLib.h> // SendInitSipiSipi()
+#include <Library/SynchronizationLib.h> // InterlockedCompareExchange64()
+#include <Register/Intel/SmramSaveStateMap.h> // SMM_DEFAULT_SMBASE
+
+#include "FirstSmiHandlerContext.h" // FIRST_SMI_HANDLER_CONTEXT
#include "Smbase.h"
extern CONST UINT8 mPostSmmPen[];
extern CONST UINT16 mPostSmmPenSize;
+extern CONST UINT8 mFirstSmiHandler[];
+extern CONST UINT16 mFirstSmiHandlerSize;
/**
Allocate a non-SMRAM reserved memory page for the Post-SMM Pen for hot-added
CPUs.
This function may only be called from the entry point function of the driver.
@param[out] PenAddress The address of the allocated (normal RAM) reserved
page.
@param[in] BootServices Pointer to the UEFI boot services table. Used for
allocating the normal RAM (not SMRAM) reserved page.
@retval EFI_SUCCESS Allocation successful.
@retval EFI_BAD_BUFFER_SIZE The Post-SMM Pen template is not smaller than
EFI_PAGE_SIZE.
@return Error codes propagated from underlying services.
DEBUG_ERROR messages have been logged. No
resources have been allocated.
**/
@@ -89,22 +97,156 @@ SmbaseReinstallPostSmmPen (
}
/**
Release the reserved page allocated with SmbaseAllocatePostSmmPen().
This function may only be called from the entry point function of the driver,
on the error path.
@param[in] PenAddress The allocation address returned by
SmbaseAllocatePostSmmPen().
@param[in] BootServices Pointer to the UEFI boot services table. Used for
releasing the normal RAM (not SMRAM) reserved page.
**/
VOID
SmbaseReleasePostSmmPen (
IN UINT32 PenAddress,
IN CONST EFI_BOOT_SERVICES *BootServices
)
{
BootServices->FreePages (PenAddress, 1);
}
+
+/**
+ Place the handler routine for the first SMIs of hot-added CPUs at
+ (SMM_DEFAULT_SMBASE + SMM_HANDLER_OFFSET).
+
+ Note that this effects an "SMRAM to SMRAM" copy.
+
+ Additionally, shut the APIC ID gate in FIRST_SMI_HANDLER_CONTEXT.
+
+ This function may only be called from the entry point function of the driver,
+ and only after PcdQ35SmramAtDefaultSmbase has been determined to be TRUE.
+**/
+VOID
+SmbaseInstallFirstSmiHandler (
+ VOID
+ )
+{
+ FIRST_SMI_HANDLER_CONTEXT *Context;
+
+ CopyMem ((VOID *)(UINTN)(SMM_DEFAULT_SMBASE + SMM_HANDLER_OFFSET),
+ mFirstSmiHandler, mFirstSmiHandlerSize);
+
+ Context = (VOID *)(UINTN)SMM_DEFAULT_SMBASE;
+ Context->ApicIdGate = MAX_UINT64;
+}
+
+/**
+ Relocate the SMBASE on a hot-added CPU. Then pen the hot-added CPU in the
+ normal RAM reserved memory page, set up earlier with
+ SmbaseAllocatePostSmmPen() and SmbaseReinstallPostSmmPen().
+
+ The SMM Monarch is supposed to call this function from the root MMI handler.
+
+ The SMM Monarch is responsible for calling SmbaseInstallFirstSmiHandler(),
+ SmbaseAllocatePostSmmPen(), and SmbaseReinstallPostSmmPen() before calling
+ this function.
+
+ If the OS maliciously boots the hot-added CPU ahead of letting the ACPI CPU
+ hotplug event handler broadcast the CPU hotplug MMI, then the hot-added CPU
+ returns to the OS rather than to the pen, upon RSM. In that case, this
+ function will hang forever (unless the OS happens to signal back through the
+ last byte of the pen page).
+
+ @param[in] ApicId The APIC ID of the hot-added CPU whose SMBASE should
+ be relocated.
+
+ @param[in] Smbase The new SMBASE address. The root MMI handler is
+ responsible for passing in a free ("unoccupied")
+ SMBASE address that was pre-configured by
+ PiSmmCpuDxeSmm in CPU_HOT_PLUG_DATA.
+
+ @param[in] PenAddress The address of the Post-SMM Pen for hot-added CPUs, as
+ returned by SmbaseAllocatePostSmmPen(), and installed
+ by SmbaseReinstallPostSmmPen().
+
+ @retval EFI_SUCCESS The SMBASE of the hot-added CPU with APIC ID
+ ApicId has been relocated to Smbase. The
+ hot-added CPU has reported back about leaving
+ SMM.
+
+ @retval EFI_PROTOCOL_ERROR Synchronization bug encountered around
+ FIRST_SMI_HANDLER_CONTEXT.ApicIdGate.
+
+ @retval EFI_INVALID_PARAMETER Smbase does not fit in 32 bits. No relocation
+ has been attempted.
+**/
+EFI_STATUS
+SmbaseRelocate (
+ IN APIC_ID ApicId,
+ IN UINTN Smbase,
+ IN UINT32 PenAddress
+ )
+{
+ EFI_STATUS Status;
+ volatile UINT8 *SmmVacated;
+ volatile FIRST_SMI_HANDLER_CONTEXT *Context;
+ UINT64 ExchangeResult;
+
+ if (Smbase > MAX_UINT32) {
+ Status = EFI_INVALID_PARAMETER;
+ DEBUG ((DEBUG_ERROR, "%a: ApicId=" FMT_APIC_ID " Smbase=0x%Lx: %r\n",
+ __FUNCTION__, ApicId, (UINT64)Smbase, Status));
+ return Status;
+ }
+
+ SmmVacated = (UINT8 *)(UINTN)PenAddress + (EFI_PAGE_SIZE - 1);
+ Context = (VOID *)(UINTN)SMM_DEFAULT_SMBASE;
+
+ //
+ // Clear the last byte of the reserved page, so we notice when the hot-added
+ // CPU checks back in from the pen.
+ //
+ *SmmVacated = 0;
+
+ //
+ // Boot the hot-added CPU.
+ //
+ // If the OS is benign, and so the hot-added CPU is still in RESET state,
+ // then the broadcast SMI is still pending for it; it will now launch
+ // directly into SMM.
+ //
+ // If the OS is malicious, the hot-added CPU has been booted already, and so
+ // it is already spinning on the APIC ID gate. In that case, the
+ // INIT-SIPI-SIPI below will be ignored.
+ //
+ SendInitSipiSipi (ApicId, PenAddress);
+
+ //
+ // Expose the desired new SMBASE value to the hot-added CPU.
+ //
+ Context->NewSmbase = (UINT32)Smbase;
+
+ //
+ // Un-gate SMBASE relocation for the hot-added CPU whose APIC ID is ApicId.
+ //
+ ExchangeResult = InterlockedCompareExchange64 (&Context->ApicIdGate,
+ MAX_UINT64, ApicId);
+ if (ExchangeResult != MAX_UINT64) {
+ Status = EFI_PROTOCOL_ERROR;
+ DEBUG ((DEBUG_ERROR, "%a: ApicId=" FMT_APIC_ID " ApicIdGate=0x%Lx: %r\n",
+ __FUNCTION__, ApicId, ExchangeResult, Status));
+ return Status;
+ }
+
+ //
+ // Now wait until the hot-added CPU reports back.
+ //
+ while (*SmmVacated == 0) {
+ CpuPause ();
+ }
+
+ Status = EFI_SUCCESS;
+ return Status;
+}
--
2.19.1.3.g30247aa5d201
^ permalink raw reply related [flat|nested] 24+ messages in thread* Re: [edk2-devel] [PATCH 12/16] OvmfPkg/CpuHotplugSmm: introduce First SMI Handler for hot-added CPUs
2020-02-23 17:25 ` [PATCH 12/16] OvmfPkg/CpuHotplugSmm: introduce First SMI Handler " Laszlo Ersek
@ 2020-02-24 9:10 ` Laszlo Ersek
2020-02-26 21:22 ` Laszlo Ersek
0 siblings, 1 reply; 24+ messages in thread
From: Laszlo Ersek @ 2020-02-24 9:10 UTC (permalink / raw)
To: edk2-devel-groups-io
Cc: Ard Biesheuvel, Igor Mammedov, Jiewen Yao, Jordan Justen,
Michael Kinney, Philippe Mathieu-Daudé
On 02/23/20 18:25, Laszlo Ersek wrote:
> Implement the First SMI Handler for hot-added CPUs, in NASM.
>
> Add the interfacing C-language function that the SMM Monarch calls. This
> function launches and coordinates SMBASE relocation for a hot-added CPU.
>
> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> Cc: Igor Mammedov <imammedo@redhat.com>
> Cc: Jiewen Yao <jiewen.yao@intel.com>
> Cc: Jordan Justen <jordan.l.justen@intel.com>
> Cc: Michael Kinney <michael.d.kinney@intel.com>
> Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
> Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=1512
> Signed-off-by: Laszlo Ersek <lersek@redhat.com>
> ---
> OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf | 4 +
> OvmfPkg/CpuHotplugSmm/FirstSmiHandlerContext.h | 41 ++++++
> OvmfPkg/CpuHotplugSmm/Smbase.h | 14 ++
> OvmfPkg/CpuHotplugSmm/FirstSmiHandler.nasm | 149 ++++++++++++++++++++
> OvmfPkg/CpuHotplugSmm/Smbase.c | 142 +++++++++++++++++++
> 5 files changed, 350 insertions(+)
>
> diff --git a/OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf b/OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf
> index bf4162299c7c..04322b0d7855 100644
> --- a/OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf
> +++ b/OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf
> @@ -5,56 +5,60 @@
> #
> # SPDX-License-Identifier: BSD-2-Clause-Patent
> ##
>
> [Defines]
> INF_VERSION = 1.29
> PI_SPECIFICATION_VERSION = 0x00010046 # PI-1.7.0
> BASE_NAME = CpuHotplugSmm
> FILE_GUID = 84EEA114-C6BE-4445-8F90-51D97863E363
> MODULE_TYPE = DXE_SMM_DRIVER
> ENTRY_POINT = CpuHotplugEntry
>
> #
> # The following information is for reference only and not required by the build
> # tools.
> #
> # VALID_ARCHITECTURES = IA32 X64
> #
>
> [Sources]
> ApicId.h
> CpuHotplug.c
> + FirstSmiHandler.nasm
> + FirstSmiHandlerContext.h
> PostSmmPen.nasm
> QemuCpuhp.c
> QemuCpuhp.h
> Smbase.c
> Smbase.h
>
> [Packages]
> MdePkg/MdePkg.dec
> OvmfPkg/OvmfPkg.dec
> UefiCpuPkg/UefiCpuPkg.dec
>
> [LibraryClasses]
> BaseLib
> BaseMemoryLib
> DebugLib
> + LocalApicLib
> MmServicesTableLib
> PcdLib
> SafeIntLib
> + SynchronizationLib
> UefiDriverEntryPoint
>
> [Protocols]
> gEfiMmCpuIoProtocolGuid ## CONSUMES
> gEfiSmmCpuServiceProtocolGuid ## CONSUMES
>
> [Pcd]
> gUefiCpuPkgTokenSpaceGuid.PcdCpuHotPlugDataAddress ## CONSUMES
> gUefiOvmfPkgTokenSpaceGuid.PcdQ35SmramAtDefaultSmbase ## CONSUMES
>
> [FeaturePcd]
> gUefiOvmfPkgTokenSpaceGuid.PcdSmmSmramRequire ## CONSUMES
>
> [Depex]
> gEfiMmCpuIoProtocolGuid AND
> gEfiSmmCpuServiceProtocolGuid
> diff --git a/OvmfPkg/CpuHotplugSmm/FirstSmiHandlerContext.h b/OvmfPkg/CpuHotplugSmm/FirstSmiHandlerContext.h
> new file mode 100644
> index 000000000000..7806a5b2ad03
> --- /dev/null
> +++ b/OvmfPkg/CpuHotplugSmm/FirstSmiHandlerContext.h
> @@ -0,0 +1,41 @@
> +/** @file
> + Define the FIRST_SMI_HANDLER_CONTEXT structure, which is an exchange area
> + between the SMM Monarch and the hot-added CPU, for relocating the SMBASE of
> + the hot-added CPU.
> +
> + Copyright (c) 2020, Red Hat, Inc.
> +
> + SPDX-License-Identifier: BSD-2-Clause-Patent
> +**/
> +
> +#ifndef FIRST_SMI_HANDLER_CONTEXT_H_
> +#define FIRST_SMI_HANDLER_CONTEXT_H_
> +
> +//
> +// The following structure is used to communicate between the SMM Monarch
> +// (running the root MMI handler) and the hot-added CPU (handling its first
> +// SMI). It is placed at SMM_DEFAULT_SMBASE, which is in SMRAM under QEMU's
> +// "SMRAM at default SMBASE" feature.
> +//
> +#pragma pack (1)
> +typedef struct {
> + //
> + // When ApicIdGate is MAX_UINT64, then no hot-added CPU may proceed with
> + // SMBASE relocation.
> + //
> + // Otherwise, the hot-added CPU whose APIC ID equals ApicIdGate may proceed
> + // with SMBASE relocation.
> + //
> + // This field is intentionally wider than APIC_ID (UINT32) because we need a
> + // "gate locked" value that is different from all possible APIC_IDs.
> + //
> + UINT64 ApicIdGate;
> + //
> + // The new SMBASE value for the hot-added CPU to set in the SMRAM Save State
> + // Map, before leaving SMM with the RSM instruction.
> + //
> + UINT32 NewSmbase;
> +} FIRST_SMI_HANDLER_CONTEXT;
> +#pragma pack ()
> +
> +#endif // FIRST_SMI_HANDLER_CONTEXT_H_
> diff --git a/OvmfPkg/CpuHotplugSmm/Smbase.h b/OvmfPkg/CpuHotplugSmm/Smbase.h
> index cb5aed98cdd3..e73730d19926 100644
> --- a/OvmfPkg/CpuHotplugSmm/Smbase.h
> +++ b/OvmfPkg/CpuHotplugSmm/Smbase.h
> @@ -1,32 +1,46 @@
> /** @file
> SMBASE relocation for hot-plugged CPUs.
>
> Copyright (c) 2020, Red Hat, Inc.
>
> SPDX-License-Identifier: BSD-2-Clause-Patent
> **/
>
> #ifndef SMBASE_H_
> #define SMBASE_H_
>
> #include <Uefi/UefiBaseType.h> // EFI_STATUS
> #include <Uefi/UefiSpec.h> // EFI_BOOT_SERVICES
>
> +#include "ApicId.h" // APIC_ID
> +
> EFI_STATUS
> SmbaseAllocatePostSmmPen (
> OUT UINT32 *PenAddress,
> IN CONST EFI_BOOT_SERVICES *BootServices
> );
>
> VOID
> SmbaseReinstallPostSmmPen (
> IN UINT32 PenAddress
> );
>
> VOID
> SmbaseReleasePostSmmPen (
> IN UINT32 PenAddress,
> IN CONST EFI_BOOT_SERVICES *BootServices
> );
>
> +VOID
> +SmbaseInstallFirstSmiHandler (
> + VOID
> + );
> +
> +EFI_STATUS
> +SmbaseRelocate (
> + IN APIC_ID ApicId,
> + IN UINTN Smbase,
> + IN UINT32 PenAddress
> + );
> +
> #endif // SMBASE_H_
> diff --git a/OvmfPkg/CpuHotplugSmm/FirstSmiHandler.nasm b/OvmfPkg/CpuHotplugSmm/FirstSmiHandler.nasm
> new file mode 100644
> index 000000000000..d5ce3472bd14
> --- /dev/null
> +++ b/OvmfPkg/CpuHotplugSmm/FirstSmiHandler.nasm
> @@ -0,0 +1,149 @@
> +;------------------------------------------------------------------------------
> +; @file
> +; Relocate the SMBASE on a hot-added CPU when it services its first SMI.
> +;
> +; Copyright (c) 2020, Red Hat, Inc.
> +;
> +; SPDX-License-Identifier: BSD-2-Clause-Patent
> +;
> +; The routine runs on the hot-added CPU in the following "big real mode",
> +; 16-bit environment; per "SMI HANDLER EXECUTION ENVIRONMENT" in the Intel SDM
> +; (table "Processor Register Initialization in SMM"):
> +;
> +; - CS selector: 0x3000 (most significant 16 bits of SMM_DEFAULT_SMBASE).
> +;
> +; - CS limit: 0xFFFF_FFFF.
> +;
> +; - CS base: SMM_DEFAULT_SMBASE (0x3_0000).
> +;
> +; - IP: SMM_HANDLER_OFFSET (0x8000).
> +;
> +; - ES, SS, DS, FS, GS selectors: 0.
> +;
> +; - ES, SS, DS, FS, GS limits: 0xFFFF_FFFF.
> +;
> +; - ES, SS, DS, FS, GS bases: 0.
> +;
> +; - Operand-size and address-size override prefixes can be used to access the
> +; address space beyond 1MB.
> +;------------------------------------------------------------------------------
> +
> +SECTION .data
> +BITS 16
> +
> +;
> +; Bring in SMM_DEFAULT_SMBASE from
> +; "MdePkg/Include/Register/Intel/SmramSaveStateMap.h".
> +;
> +SMM_DEFAULT_SMBASE: equ 0x3_0000
> +
> +;
> +; Field offsets in FIRST_SMI_HANDLER_CONTEXT, which resides at
> +; SMM_DEFAULT_SMBASE.
> +;
> +ApicIdGate: equ 0 ; UINT64
> +NewSmbase: equ 8 ; UINT32
> +
> +;
> +; SMRAM Save State Map field offsets, per the AMD (not Intel) layout that QEMU
> +; implements. Relative to SMM_DEFAULT_SMBASE.
> +;
> +SaveStateRevId: equ 0xFEFC ; UINT32
> +SaveStateSmbase: equ 0xFEF8 ; UINT32
> +SaveStateSmbase64: equ 0xFF00 ; UINT32
> +
> +;
> +; CPUID constants, from "MdePkg/Include/Register/Intel/Cpuid.h".
> +;
> +CPUID_SIGNATURE: equ 0x00
> +CPUID_EXTENDED_TOPOLOGY: equ 0x0B
> +CPUID_VERSION_INFO: equ 0x01
> +
> +GLOBAL ASM_PFX (mFirstSmiHandler) ; UINT8[]
> +GLOBAL ASM_PFX (mFirstSmiHandlerSize) ; UINT16
> +
> +ASM_PFX (mFirstSmiHandler):
> + ;
> + ; Get our own APIC ID first, so we can contend for ApicIdGate.
> + ;
> + ; This basically reimplements GetInitialApicId() from
> + ; "UefiCpuPkg/Library/BaseXApicLib/BaseXApicLib.c".
> + ;
> + mov eax, CPUID_SIGNATURE
> + cpuid
> + cmp eax, CPUID_EXTENDED_TOPOLOGY
> + jb GetApicIdFromVersionInfo
> +
> + mov eax, CPUID_EXTENDED_TOPOLOGY
> + mov ecx, 0
> + cpuid
> + test ebx, 0xFFFF
> + jz GetApicIdFromVersionInfo
> +
> + ;
> + ; EDX has the APIC ID, save it to ESI.
> + ;
> + mov esi, edx
> + jmp KnockOnGate
> +
> +GetApicIdFromVersionInfo:
> + mov eax, CPUID_VERSION_INFO
> + cpuid
> + shr ebx, 24
> + ;
> + ; EBX has the APIC ID, save it to ESI.
> + ;
> + mov esi, ebx
> +
> +KnockOnGate:
> + ;
> + ; See if ApicIdGate shows our own APIC ID. If so, swap it to MAX_UINT64
> + ; (close the gate), and advance. Otherwise, keep knocking.
> + ;
> + ; InterlockedCompareExchange64():
> + ; - Value := &FIRST_SMI_HANDLER_CONTEXT.ApicIdGate
> + ; - CompareValue (EDX:EAX) := APIC ID (from ESI)
> + ; - ExchangeValue (ECX:EBX) := MAX_UINT64
> + ;
> + mov edx, 0
> + mov eax, esi
> + mov ecx, 0xFFFF_FFFF
> + mov ebx, 0xFFFF_FFFF
> + lock cmpxchg8b [ds : dword (SMM_DEFAULT_SMBASE + ApicIdGate)]
> + jz ApicIdMatch
> + pause
> + jmp KnockOnGate
> +
> +ApicIdMatch:
> + ;
> + ; Update the SMBASE field in the SMRAM Save State Map.
> + ;
> + ; First, calculate the address of the SMBASE field, based on the SMM Revision
> + ; ID; store the result in EBX.
> + ;
> + mov eax, dword [ds : dword (SMM_DEFAULT_SMBASE + SaveStateRevId)]
> + test eax, 0xFFFF
> + jz LegacySaveStateMap
> +
> + mov ebx, SMM_DEFAULT_SMBASE + SaveStateSmbase64
> + jmp UpdateSmbase
> +
> +LegacySaveStateMap:
> + mov ebx, SMM_DEFAULT_SMBASE + SaveStateSmbase
> +
> +UpdateSmbase:
> + ;
> + ; Load the new SMBASE value into EAX.
> + ;
> + mov eax, dword [ds : dword (SMM_DEFAULT_SMBASE + NewSmbase)]
> + ;
> + ; Save it to the SMBASE field whose address we calculated in EBX.
> + ;
> + mov dword [ds : dword ebx], eax
> + ;
> + ; We're done; leave SMM and continue to the pen.
> + ;
> + rsm
> +
> +ASM_PFX (mFirstSmiHandlerSize):
> + dw $ - ASM_PFX (mFirstSmiHandler)
> diff --git a/OvmfPkg/CpuHotplugSmm/Smbase.c b/OvmfPkg/CpuHotplugSmm/Smbase.c
> index ea21153d9145..57c9e86f3e93 100644
> --- a/OvmfPkg/CpuHotplugSmm/Smbase.c
> +++ b/OvmfPkg/CpuHotplugSmm/Smbase.c
> @@ -1,38 +1,46 @@
> /** @file
> SMBASE relocation for hot-plugged CPUs.
>
> Copyright (c) 2020, Red Hat, Inc.
>
> SPDX-License-Identifier: BSD-2-Clause-Patent
> **/
>
> #include <Base.h> // BASE_1MB
> +#include <Library/BaseLib.h> // CpuPause()
> #include <Library/BaseMemoryLib.h> // CopyMem()
> #include <Library/DebugLib.h> // DEBUG()
> +#include <Library/LocalApicLib.h> // SendInitSipiSipi()
> +#include <Library/SynchronizationLib.h> // InterlockedCompareExchange64()
> +#include <Register/Intel/SmramSaveStateMap.h> // SMM_DEFAULT_SMBASE
> +
> +#include "FirstSmiHandlerContext.h" // FIRST_SMI_HANDLER_CONTEXT
>
> #include "Smbase.h"
>
> extern CONST UINT8 mPostSmmPen[];
> extern CONST UINT16 mPostSmmPenSize;
> +extern CONST UINT8 mFirstSmiHandler[];
> +extern CONST UINT16 mFirstSmiHandlerSize;
>
> /**
> Allocate a non-SMRAM reserved memory page for the Post-SMM Pen for hot-added
> CPUs.
>
> This function may only be called from the entry point function of the driver.
>
> @param[out] PenAddress The address of the allocated (normal RAM) reserved
> page.
>
> @param[in] BootServices Pointer to the UEFI boot services table. Used for
> allocating the normal RAM (not SMRAM) reserved page.
>
> @retval EFI_SUCCESS Allocation successful.
>
> @retval EFI_BAD_BUFFER_SIZE The Post-SMM Pen template is not smaller than
> EFI_PAGE_SIZE.
>
> @return Error codes propagated from underlying services.
> DEBUG_ERROR messages have been logged. No
> resources have been allocated.
> **/
> @@ -89,22 +97,156 @@ SmbaseReinstallPostSmmPen (
> }
>
> /**
> Release the reserved page allocated with SmbaseAllocatePostSmmPen().
>
> This function may only be called from the entry point function of the driver,
> on the error path.
>
> @param[in] PenAddress The allocation address returned by
> SmbaseAllocatePostSmmPen().
>
> @param[in] BootServices Pointer to the UEFI boot services table. Used for
> releasing the normal RAM (not SMRAM) reserved page.
> **/
> VOID
> SmbaseReleasePostSmmPen (
> IN UINT32 PenAddress,
> IN CONST EFI_BOOT_SERVICES *BootServices
> )
> {
> BootServices->FreePages (PenAddress, 1);
> }
> +
> +/**
> + Place the handler routine for the first SMIs of hot-added CPUs at
> + (SMM_DEFAULT_SMBASE + SMM_HANDLER_OFFSET).
> +
> + Note that this effects an "SMRAM to SMRAM" copy.
> +
> + Additionally, shut the APIC ID gate in FIRST_SMI_HANDLER_CONTEXT.
> +
> + This function may only be called from the entry point function of the driver,
> + and only after PcdQ35SmramAtDefaultSmbase has been determined to be TRUE.
> +**/
> +VOID
> +SmbaseInstallFirstSmiHandler (
> + VOID
> + )
> +{
> + FIRST_SMI_HANDLER_CONTEXT *Context;
> +
> + CopyMem ((VOID *)(UINTN)(SMM_DEFAULT_SMBASE + SMM_HANDLER_OFFSET),
> + mFirstSmiHandler, mFirstSmiHandlerSize);
> +
> + Context = (VOID *)(UINTN)SMM_DEFAULT_SMBASE;
> + Context->ApicIdGate = MAX_UINT64;
> +}
> +
> +/**
> + Relocate the SMBASE on a hot-added CPU. Then pen the hot-added CPU in the
> + normal RAM reserved memory page, set up earlier with
> + SmbaseAllocatePostSmmPen() and SmbaseReinstallPostSmmPen().
> +
> + The SMM Monarch is supposed to call this function from the root MMI handler.
> +
> + The SMM Monarch is responsible for calling SmbaseInstallFirstSmiHandler(),
> + SmbaseAllocatePostSmmPen(), and SmbaseReinstallPostSmmPen() before calling
> + this function.
> +
> + If the OS maliciously boots the hot-added CPU ahead of letting the ACPI CPU
> + hotplug event handler broadcast the CPU hotplug MMI, then the hot-added CPU
> + returns to the OS rather than to the pen, upon RSM. In that case, this
> + function will hang forever (unless the OS happens to signal back through the
> + last byte of the pen page).
> +
> + @param[in] ApicId The APIC ID of the hot-added CPU whose SMBASE should
> + be relocated.
> +
> + @param[in] Smbase The new SMBASE address. The root MMI handler is
> + responsible for passing in a free ("unoccupied")
> + SMBASE address that was pre-configured by
> + PiSmmCpuDxeSmm in CPU_HOT_PLUG_DATA.
> +
> + @param[in] PenAddress The address of the Post-SMM Pen for hot-added CPUs, as
> + returned by SmbaseAllocatePostSmmPen(), and installed
> + by SmbaseReinstallPostSmmPen().
> +
> + @retval EFI_SUCCESS The SMBASE of the hot-added CPU with APIC ID
> + ApicId has been relocated to Smbase. The
> + hot-added CPU has reported back about leaving
> + SMM.
> +
> + @retval EFI_PROTOCOL_ERROR Synchronization bug encountered around
> + FIRST_SMI_HANDLER_CONTEXT.ApicIdGate.
> +
> + @retval EFI_INVALID_PARAMETER Smbase does not fit in 32 bits. No relocation
> + has been attempted.
> +**/
> +EFI_STATUS
> +SmbaseRelocate (
> + IN APIC_ID ApicId,
> + IN UINTN Smbase,
> + IN UINT32 PenAddress
> + )
> +{
> + EFI_STATUS Status;
> + volatile UINT8 *SmmVacated;
> + volatile FIRST_SMI_HANDLER_CONTEXT *Context;
> + UINT64 ExchangeResult;
> +
> + if (Smbase > MAX_UINT32) {
> + Status = EFI_INVALID_PARAMETER;
> + DEBUG ((DEBUG_ERROR, "%a: ApicId=" FMT_APIC_ID " Smbase=0x%Lx: %r\n",
> + __FUNCTION__, ApicId, (UINT64)Smbase, Status));
> + return Status;
> + }
> +
> + SmmVacated = (UINT8 *)(UINTN)PenAddress + (EFI_PAGE_SIZE - 1);
> + Context = (VOID *)(UINTN)SMM_DEFAULT_SMBASE;
> +
> + //
> + // Clear the last byte of the reserved page, so we notice when the hot-added
> + // CPU checks back in from the pen.
> + //
> + *SmmVacated = 0;
> +
> + //
> + // Boot the hot-added CPU.
> + //
> + // If the OS is benign, and so the hot-added CPU is still in RESET state,
> + // then the broadcast SMI is still pending for it; it will now launch
> + // directly into SMM.
> + //
> + // If the OS is malicious, the hot-added CPU has been booted already, and so
> + // it is already spinning on the APIC ID gate. In that case, the
> + // INIT-SIPI-SIPI below will be ignored.
> + //
> + SendInitSipiSipi (ApicId, PenAddress);
> +
> + //
> + // Expose the desired new SMBASE value to the hot-added CPU.
> + //
> + Context->NewSmbase = (UINT32)Smbase;
> +
> + //
> + // Un-gate SMBASE relocation for the hot-added CPU whose APIC ID is ApicId.
> + //
> + ExchangeResult = InterlockedCompareExchange64 (&Context->ApicIdGate,
> + MAX_UINT64, ApicId);
> + if (ExchangeResult != MAX_UINT64) {
> + Status = EFI_PROTOCOL_ERROR;
> + DEBUG ((DEBUG_ERROR, "%a: ApicId=" FMT_APIC_ID " ApicIdGate=0x%Lx: %r\n",
> + __FUNCTION__, ApicId, ExchangeResult, Status));
> + return Status;
> + }
> +
> + //
> + // Now wait until the hot-added CPU reports back.
> + //
> + while (*SmmVacated == 0) {
> + CpuPause ();
> + }
> +
> + Status = EFI_SUCCESS;
> + return Status;
> +}
>
Overnight I managed to think up an attack, from the OS, against the
"SmmVacated" byte (the last byte of the reserved page, i.e. the last
byte of the Post-SMM Pen).
Here's how:
There are three CPUs being hotplugged in one SMI, CPU#1..CPU#3. The OS
boots them all before raising the SMI (i.e. before it allows the ACPI
GPE handler to take effect). After the first CPU (let's say CPU#1)
returns to the OS via the RSM, the OS uses it (=CPU#1) to attack
"SmmVacated", writing 1 to it in a loop.
Meanwhile CPU#2 and CPU#3 are still in SMM; let's say CPU#2 is
relocating SMBASE, while CPU#3 is spinning on the APIC ID gate. And the
SMM Monarch (CPU#0) is waiting for CPU#2 to report back in through
"SmmVacated", from the Post-SMM Pen.
Now, the OS writes 1 to "SmmVacated" early, pretending to be CPU#2. This
causes the SMM Monarch (CPU#0) to open the APIC ID gate for CPU#3 before
CPU#2 actually reaches the RSM. This may cause CPU#2 and CPU#3 to both
reach RSM with the same SMBASE value.
So why did I think to put SmmVacated in normal RAM (in the Post-SMM Pen
reserved page?) Because in the following message:
http://mid.mail-archive.com/20191004160948.72097f6c@redhat.com
(alt. link: <https://edk2.groups.io/g/devel/message/48475>)
Igor showed that putting "SmmVacated" in SMRAM is racy, even without a
malicious OS. The problem is that there is no way to flip a byte in
SMRAM *atomically* with RSM. So the CPU that has just completed SMBASE
relocation can only flip the byte before RSM (in SMRAM) or after RSM (in
normal RAM). In the latter case, the OS can attack SmmVacated -- but in
the former case, we get the same race *without* any OS involvement
(because the CPU just about to leave SMM via RSM actively allows the SMM
Monarch to ungate the relocation code for a different CPU).
So I don't think there's a 100% safe & secure way to do this. One thing
we could try -- I could update the code -- is to *combine* both
approaches; use two "SmmVacated" flags: one in SMRAM, set to 1 just
before the RSM instruction, and the other one in normal RAM (reserved
page) that this patch set already introduces. The normal RAM flag would
avoid the race completely for benign OSes (like the present patchset
already does), and the SMRAM flag would keep the racy window to a single
instruction when the OS is malicious and the normal RAM flag cannot be
trusted.
I'd appreciate feedback on this; I don't know how in physical firmware,
the initial SMI handler for hot-added CPUs deals with the same problem.
I guess on physical platforms, the platform manual could simply say,
"only hot-plug CPUs one by one". That eliminates the whole problem. In
such a case, we could safely stick with the current patchset, too.
--*--
BTW, I did try hot-plugging multiple CPUs at once, with libvirt:
> virsh setvcpu ovmf.fedora.q35 1,3 --enable --live
>
> error: Operation not supported: only one hotpluggable entity can be
> selected
I think this is because it would require multiple "device-add" commands
to be sent at the same time over the QMP monitor -- and that's something
that QEMU doesn't support. (Put alternatively, "device-add" does not
take a list of objects to create.) In that regard, the problem described
above is academic, because QEMU already seems like a platform that can
only hotplug one CPU at a time. In that sense, using APIC ID *arrays*
and *loops* per hotplug SMI is not really needed; I did that because we
had discussed this feature in terms of multiple CPUs from the beginning,
and because QEMU's ACPI GPE handler (the CSCN AML method) also loops
over multiple processors.
Again, comments would be most welcome... I wouldn't like to complicate
the SMI handler if it's not absolutely necessary.
Thanks!
Laszlo
^ permalink raw reply [flat|nested] 24+ messages in thread* Re: [edk2-devel] [PATCH 12/16] OvmfPkg/CpuHotplugSmm: introduce First SMI Handler for hot-added CPUs
2020-02-24 9:10 ` [edk2-devel] " Laszlo Ersek
@ 2020-02-26 21:22 ` Laszlo Ersek
0 siblings, 0 replies; 24+ messages in thread
From: Laszlo Ersek @ 2020-02-26 21:22 UTC (permalink / raw)
To: edk2-devel-groups-io
Cc: Ard Biesheuvel, Igor Mammedov, Jiewen Yao, Jordan Justen,
Michael Kinney, Philippe Mathieu-Daudé
On 02/24/20 10:10, Laszlo Ersek wrote:
> Overnight I managed to think up an attack, from the OS, against the
> "SmmVacated" byte (the last byte of the reserved page, i.e. the last
> byte of the Post-SMM Pen).
>
> Here's how:
>
> There are three CPUs being hotplugged in one SMI, CPU#1..CPU#3. The OS
> boots them all before raising the SMI (i.e. before it allows the ACPI
> GPE handler to take effect). After the first CPU (let's say CPU#1)
> returns to the OS via the RSM, the OS uses it (=CPU#1) to attack
> "SmmVacated", writing 1 to it in a loop.
>
> Meanwhile CPU#2 and CPU#3 are still in SMM; let's say CPU#2 is
> relocating SMBASE, while CPU#3 is spinning on the APIC ID gate. And the
> SMM Monarch (CPU#0) is waiting for CPU#2 to report back in through
> "SmmVacated", from the Post-SMM Pen.
>
> Now, the OS writes 1 to "SmmVacated" early, pretending to be CPU#2. This
> causes the SMM Monarch (CPU#0) to open the APIC ID gate for CPU#3 before
> CPU#2 actually reaches the RSM. This may cause CPU#2 and CPU#3 to both
> reach RSM with the same SMBASE value.
>
> So why did I think to put SmmVacated in normal RAM (in the Post-SMM Pen
> reserved page?) Because in the following message:
>
> http://mid.mail-archive.com/20191004160948.72097f6c@redhat.com
> (alt. link: <https://edk2.groups.io/g/devel/message/48475>)
>
> Igor showed that putting "SmmVacated" in SMRAM is racy, even without a
> malicious OS. The problem is that there is no way to flip a byte in
> SMRAM *atomically* with RSM. So the CPU that has just completed SMBASE
> relocation can only flip the byte before RSM (in SMRAM) or after RSM (in
> normal RAM). In the latter case, the OS can attack SmmVacated -- but in
> the former case, we get the same race *without* any OS involvement
> (because the CPU just about to leave SMM via RSM actively allows the SMM
> Monarch to ungate the relocation code for a different CPU).
>
> So I don't think there's a 100% safe & secure way to do this. One thing
> we could try -- I could update the code -- is to *combine* both
> approaches; use two "SmmVacated" flags: one in SMRAM, set to 1 just
> before the RSM instruction, and the other one in normal RAM (reserved
> page) that this patch set already introduces. The normal RAM flag would
> avoid the race completely for benign OSes (like the present patchset
> already does), and the SMRAM flag would keep the racy window to a single
> instruction when the OS is malicious and the normal RAM flag cannot be
> trusted.
I've implemented and tested this "combined" approach.
Thanks
Laszlo
>
> I'd appreciate feedback on this; I don't know how in physical firmware,
> the initial SMI handler for hot-added CPUs deals with the same problem.
>
> I guess on physical platforms, the platform manual could simply say,
> "only hot-plug CPUs one by one". That eliminates the whole problem. In
> such a case, we could safely stick with the current patchset, too.
>
> --*--
>
> BTW, I did try hot-plugging multiple CPUs at once, with libvirt:
>
>> virsh setvcpu ovmf.fedora.q35 1,3 --enable --live
>>
>> error: Operation not supported: only one hotpluggable entity can be
>> selected
>
> I think this is because it would require multiple "device-add" commands
> to be sent at the same time over the QMP monitor -- and that's something
> that QEMU doesn't support. (Put alternatively, "device-add" does not
> take a list of objects to create.) In that regard, the problem described
> above is academic, because QEMU already seems like a platform that can
> only hotplug one CPU at a time. In that sense, using APIC ID *arrays*
> and *loops* per hotplug SMI is not really needed; I did that because we
> had discussed this feature in terms of multiple CPUs from the beginning,
> and because QEMU's ACPI GPE handler (the CSCN AML method) also loops
> over multiple processors.
>
> Again, comments would be most welcome... I wouldn't like to complicate
> the SMI handler if it's not absolutely necessary.
^ permalink raw reply [flat|nested] 24+ messages in thread
* [PATCH 13/16] OvmfPkg/CpuHotplugSmm: complete root MMI handler for CPU hotplug
2020-02-23 17:25 [PATCH 00/16] OvmfPkg: support VCPU hotplug with -D SMM_REQUIRE Laszlo Ersek
` (11 preceding siblings ...)
2020-02-23 17:25 ` [PATCH 12/16] OvmfPkg/CpuHotplugSmm: introduce First SMI Handler " Laszlo Ersek
@ 2020-02-23 17:25 ` Laszlo Ersek
2020-02-23 17:25 ` [PATCH 14/16] OvmfPkg: clone CpuS3DataDxe from UefiCpuPkg Laszlo Ersek
` (4 subsequent siblings)
17 siblings, 0 replies; 24+ messages in thread
From: Laszlo Ersek @ 2020-02-23 17:25 UTC (permalink / raw)
To: edk2-devel-groups-io
Cc: Ard Biesheuvel, Igor Mammedov, Jiewen Yao, Jordan Justen,
Michael Kinney, Philippe Mathieu-Daudé
With the help of the Post-SMM Pen and the SMBASE relocation functions
added in the previous patches, we can now complete the root MMI handler
for CPU hotplug.
In the driver's entry point function:
- allocate the pen (in a reserved page in normal RAM),
- install the default ("first") SMI handler for hot-added CPUs (which
includes priming the exchange area between the MM Monarch and the
hot-added CPUs, i.e., shutting the APIC ID gate).
In the root MMI handler, for each hot-added CPU:
- record the APIC ID of the new CPU in CPU_HOT_PLUG_DATA,
- relocate the SMBASE of the new CPU,
- inform PiSmmCpuDxeSmm by calling
EFI_SMM_CPU_SERVICE_PROTOCOL.AddProcessor().
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Michael Kinney <michael.d.kinney@intel.com>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=1512
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
---
Notes:
(1) At this stage, CPU hotplug with SMM works.
Test CPU topology (libvirt domain XML snippet):
> <vcpus>
> <vcpu id='0' enabled='yes' hotpluggable='no'/>
> <vcpu id='1' enabled='no' hotpluggable='yes'/>
> <vcpu id='2' enabled='yes' hotpluggable='yes'/>
> <vcpu id='3' enabled='no' hotpluggable='yes'/>
> </vcpus>
The firmware logs the following during normal boot:
> SmbaseAllocatePostSmmPen: Post-SMM Pen at 0x9F000
CPU hotplug command on the host side:
> virsh setvcpu ovmf.fedora.q35 1 --enable --live
Firmware log in response:
> QemuCpuhpCollectApicIds: CurrentSelector=1: insert
> QemuCpuhpCollectApicIds: ApicId=0x00000001
> QemuCpuhpCollectApicIds: CurrentSelector=2 PendingSelector=1:
> wrap-around
> QemuCpuhpCollectApicIds: PluggedCount=1 ToUnplugCount=0
> CpuHotplugMmi: hot-added APIC ID 0x00000001, SMBASE 0x7D082000,
> EFI_SMM_CPU_SERVICE_PROTOCOL assigned number 2
Query the CPU register state on the host side (without onlining the
hot-added CPU in the OS just yet):
> virsh qemu-monitor-command ovmf.fedora.q35 --hmp 'info registers -a'
Output, confirming penned status (pen at 0x9F000):
> CPU#1
> EAX=00009f00 EBX=00000000 ECX=00000000 EDX=00000600
> ESI=00000000 EDI=00000000 EBP=00000000 ESP=00000000
> EIP=0000000c EFL=00000002 [-------] CPL=0 II=0 A20=1 SMM=0 HLT=1
> ES =0000 00000000 0000ffff 00009300
> CS =9f00 0009f000 0000ffff 00009b00
> SS =0000 00000000 0000ffff 00009300
> DS =9f00 0009f000 0000ffff 00009300
> FS =0000 00000000 0000ffff 00009300
> GS =0000 00000000 0000ffff 00009300
> LDT=0000 00000000 0000ffff 00008200
> TR =0000 00000000 0000ffff 00008b00
> GDT= 00000000 0000ffff
> IDT= 00000000 0000ffff
> CR0=60000010 CR2=00000000 CR3=00000000 CR4=00000000
> DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000
> DR3=0000000000000000
> DR6=00000000ffff0ff0 DR7=0000000000000400
> EFER=0000000000000000
> FCW=037f FSW=0000 [ST=0] FTW=00 MXCSR=00001f80
> FPR0=0000000000000000 0000 FPR1=0000000000000000 0000
> FPR2=0000000000000000 0000 FPR3=0000000000000000 0000
> FPR4=0000000000000000 0000 FPR5=0000000000000000 0000
> FPR6=0000000000000000 0000 FPR7=0000000000000000 0000
> XMM00=00000000000000000000000000000000
> XMM01=00000000000000000000000000000000
> XMM02=00000000000000000000000000000000
> XMM03=00000000000000000000000000000000
> XMM04=00000000000000000000000000000000
> XMM05=00000000000000000000000000000000
> XMM06=00000000000000000000000000000000
> XMM07=00000000000000000000000000000000
Additionally, the dmesg in the Linux guest contains:
> CPU2 has been hot-added
We can now boot the hot-addad CPU in the Linux guest:
> echo 1 > /sys/devices/system/cpu/cpu2/online
Dmesg in response:
> smpboot: Booting Node 0 Processor 2 APIC 0x1
> kvm-clock: cpu 2, msr 6e01081, secondary cpu clock
> KVM setup async PF for cpu 2
> kvm-stealtime: cpu 2, msr 7b52a040
> Will online and init hotplugged CPU: 2
Still in the guest, call UEFI variable services on each CPU (including
the hot-added one), per
<https://github.com/tianocore/tianocore.github.io/wiki/Testing-SMM-with-QEMU,-KVM-and-libvirt#uefi-variable-access-test>:
> taskset -c 0 efibootmgr
> taskset -c 1 efibootmgr
> taskset -c 2 efibootmgr
No delays, no instability.
Having booted the hot-added CPU in the Linux guest, the host-side
command
> virsh qemu-monitor-command ovmf.fedora.q35 --hmp 'info registers -a'
now shows:
> CPU#1
> RAX=ffffffffb39e4b50 RBX=0000000000000002 RCX=0000000000000000
> RDX=0000000000000002
> RSI=0000000000000002 RDI=ffff932cbb51c520 RBP=0000000000000002
> RSP=ffffbc5bc0083eb0
> R8 =000000cd42e4dffb R9 =0000005a22502599 R10=0000005a22502599
> R11=0000005a22502599
> R12=ffff932cbae25dc0 R13=0000000000000000 R14=0000000000000000
> R15=ffff932cbae25dc0
> RIP=ffffffffb39e4f2e RFL=00000246 [---Z-P-] CPL=0 II=0 A20=1 SMM=0
> HLT=1
> ES =0000 0000000000000000 ffffffff 00c00000
> CS =0010 0000000000000000 ffffffff 00a09b00 DPL=0 CS64 [-RA]
> SS =0018 0000000000000000 ffffffff 00c09300 DPL=0 DS [-WA]
> DS =0000 0000000000000000 ffffffff 00c00000
> FS =0000 0000000000000000 ffffffff 00c00000
> GS =0000 ffff932cbb500000 ffffffff 00c00000
> LDT=0000 0000000000000000 000fffff 00000000
> TR =0040 fffffe0000069000 0000206f 00008b00 DPL=0 TSS64-busy
> GDT= fffffe0000067000 0000007f
> IDT= fffffe0000000000 00000fff
> CR0=80050033 CR2=00007f54893cbf20 CR3=0000000078418001 CR4=001606e0
> DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000
> DR3=0000000000000000
> DR6=00000000ffff0ff0 DR7=0000000000000400
> EFER=0000000000000d01
> FCW=037f FSW=0000 [ST=0] FTW=00 MXCSR=00001fa0
> FPR0=0000000000000000 0000 FPR1=0000000000000000 0000
> FPR2=0000000000000000 0000 FPR3=0000000000000000 0000
> FPR4=0000000000000000 0000 FPR5=0000000000000000 0000
> FPR6=0000000000000000 0000 FPR7=0000000000000000 0000
> XMM00=000000000000000040ed2be000000000
> XMM01=0000000000000000404ddf1a9fbe76c9
> XMM02=00000000000000000000000000000000
> XMM03=00000000000000000000000000000000
> XMM04=00000000000000003ff0000000000000
> XMM05=00000000000000000000000000000000
> XMM06=00000000000000004078002d0ac487b7
> XMM07=0000000000000000404ddf1a9fbe76c9
> XMM08=72223d444955412022746f6f72223d44
> XMM09=00000000000000000000000000000000
> XMM10=00000000000000000000000000000000
> XMM11=00000000000000000000000000000000
> XMM12=00000000000000000000000000000000
> XMM13=00000000000000000000000000000000
> XMM14=00000000000000000000000000000000
> XMM15=00000000000000000000000000000000
Hotplug another CPU:
> virsh setvcpu ovmf.fedora.q35 3 --enable --live
Firmware log in response:
> QemuCpuhpCollectApicIds: CurrentSelector=3: insert
> QemuCpuhpCollectApicIds: ApicId=0x00000003
> QemuCpuhpCollectApicIds: PluggedCount=1 ToUnplugCount=0
> CpuHotplugMmi: hot-added APIC ID 0x00000003, SMBASE 0x7D084000,
> EFI_SMM_CPU_SERVICE_PROTOCOL assigned number 3
Online the hot-added CPU in the Linux guest:
> echo 1 > /sys/devices/system/cpu/cpu3/online
Guest dmesg in response:
> smpboot: Booting Node 0 Processor 3 APIC 0x3
> kvm-clock: cpu 3, msr 6e010c1, secondary cpu clock
> KVM setup async PF for cpu 3
> kvm-stealtime: cpu 3, msr 7b5aa040
> Will online and init hotplugged CPU: 3
(2) Alternative order of actions:
Hotplug both CPUs first, on the host:
> virsh setvcpu ovmf.fedora.q35 1 --enable --live
> virsh setvcpu ovmf.fedora.q35 3 --enable --live
*then* online both hot-added CPUs in the Linux guest:
> echo 1 > /sys/devices/system/cpu/cpu2/online
> echo 1 > /sys/devices/system/cpu/cpu3/online
The only difference here is that the broadcast SMI for the second
hotplug finds the first hotplugged CPU still in the pen (i.e., not
onlined by Linux). There is no difference in observable behavior (beyond
the matching log messages and register dumps).
(3) What doesn't work yet:
- CPU hotplug w/ SMM in SEV guests.
See <https://bugzilla.tianocore.org/show_bug.cgi?id=1512#c14>.
(NB. I'm uncertain if CPU hotplug works in SEV guests without
SMM_REQUIRE to begin with.)
- S3 resume w/ SMM after hot-adding a CPU.
The remaining patches in the series take care of that. (Only in the
absence of SEV; SEV+S3 looks broken regardless of hotplug.)
- CPU hot-unplug w/ SMM.
Needs separate investigation. EFI_SMM_REMOVE_PROCESSOR in
"UefiCpuPkg/Include/Protocol/SmmCpuService.h" is documented as
follows: "After this API is called, the removed processor must not
respond to SMIs in the coherence domain". This could raise challenging
ordering questions between QEMU, the firmware, and the OS.
OvmfPkg/CpuHotplugSmm/CpuHotplug.c | 97 +++++++++++++++++++-
1 file changed, 95 insertions(+), 2 deletions(-)
diff --git a/OvmfPkg/CpuHotplugSmm/CpuHotplug.c b/OvmfPkg/CpuHotplugSmm/CpuHotplug.c
index 42e023cb85c0..20e6bec04f41 100644
--- a/OvmfPkg/CpuHotplugSmm/CpuHotplug.c
+++ b/OvmfPkg/CpuHotplugSmm/CpuHotplug.c
@@ -1,76 +1,82 @@
/** @file
Root SMI handler for VCPU hotplug SMIs.
Copyright (c) 2020, Red Hat, Inc.
SPDX-License-Identifier: BSD-2-Clause-Patent
**/
#include <CpuHotPlugData.h> // CPU_HOT_PLUG_DATA
#include <IndustryStandard/Q35MchIch9.h> // ICH9_APM_CNT
#include <IndustryStandard/QemuCpuHotplug.h> // QEMU_CPUHP_CMD_GET_PENDING
#include <Library/BaseLib.h> // CpuDeadLoop()
#include <Library/DebugLib.h> // ASSERT()
#include <Library/MmServicesTableLib.h> // gMmst
#include <Library/PcdLib.h> // PcdGetBool()
#include <Library/SafeIntLib.h> // SafeUintnSub()
#include <Protocol/MmCpuIo.h> // EFI_MM_CPU_IO_PROTOCOL
#include <Protocol/SmmCpuService.h> // EFI_SMM_CPU_SERVICE_PROTOCOL
#include <Uefi/UefiBaseType.h> // EFI_STATUS
#include "ApicId.h" // APIC_ID
#include "QemuCpuhp.h" // QemuCpuhpWriteCpuSelector()
+#include "Smbase.h" // SmbaseAllocatePostSmmPen()
//
// We use this protocol for accessing IO Ports.
//
STATIC EFI_MM_CPU_IO_PROTOCOL *mMmCpuIo;
//
// The following protocol is used to report the addition or removal of a CPU to
// the SMM CPU driver (PiSmmCpuDxeSmm).
//
STATIC EFI_SMM_CPU_SERVICE_PROTOCOL *mMmCpuService;
//
// This structure is a communication side-channel between the
// EFI_SMM_CPU_SERVICE_PROTOCOL consumer (i.e., this driver) and provider
// (i.e., PiSmmCpuDxeSmm).
//
STATIC CPU_HOT_PLUG_DATA *mCpuHotPlugData;
//
// SMRAM arrays for fetching the APIC IDs of processors with pending events (of
// known event types), for the time of just one MMI.
//
// The lifetimes of these arrays match that of this driver only because we
// don't want to allocate SMRAM at OS runtime, and potentially fail (or
// fragment the SMRAM map).
//
// These arrays provide room for ("possible CPU count" minus one) APIC IDs
// each, as we don't expect every possible CPU to appear, or disappear, in a
// single MMI. The numbers of used (populated) elements in the arrays are
// determined on every MMI separately.
//
STATIC APIC_ID *mPluggedApicIds;
STATIC APIC_ID *mToUnplugApicIds;
//
+// Address of the non-SMRAM reserved memory page that contains the Post-SMM Pen
+// for hot-added CPUs.
+//
+STATIC UINT32 mPostSmmPenAddress;
+//
// Represents the registration of the CPU Hotplug MMI handler.
//
STATIC EFI_HANDLE mDispatchHandle;
/**
CPU Hotplug MMI handler function.
This is a root MMI handler.
@param[in] DispatchHandle The unique handle assigned to this handler by
EFI_MM_SYSTEM_TABLE.MmiHandlerRegister().
@param[in] Context Context passed in by
EFI_MM_SYSTEM_TABLE.MmiManage(). Due to
CpuHotplugMmi() being a root MMI handler,
Context is ASSERT()ed to be NULL.
@param[in,out] CommBuffer Ignored, due to CpuHotplugMmi() being a root
MMI handler.
@param[in,out] CommBufferSize Ignored, due to CpuHotplugMmi() being a root
@@ -97,44 +103,46 @@ STATIC EFI_HANDLE mDispatchHandle;
@retval EFI_WARN_INTERRUPT_SOURCE_PENDING The MMI source is still pending,
and other handlers should still
be called.
@retval EFI_INTERRUPT_PENDING The MMI source could not be
quiesced.
**/
STATIC
EFI_STATUS
EFIAPI
CpuHotplugMmi (
IN EFI_HANDLE DispatchHandle,
IN CONST VOID *Context OPTIONAL,
IN OUT VOID *CommBuffer OPTIONAL,
IN OUT UINTN *CommBufferSize OPTIONAL
)
{
EFI_STATUS Status;
UINT8 ApmControl;
UINT32 PluggedCount;
UINT32 ToUnplugCount;
+ UINT32 PluggedIdx;
+ UINT32 NewSlot;
//
// Assert that we are entering this function due to our root MMI handler
// registration.
//
ASSERT (DispatchHandle == mDispatchHandle);
//
// When MmiManage() is invoked to process root MMI handlers, the caller (the
// MM Core) is expected to pass in a NULL Context. MmiManage() then passes
// the same NULL Context to individual handlers.
//
ASSERT (Context == NULL);
//
// Read the MMI command value from the APM Control Port, to see if this is an
// MMI we should care about.
//
Status = mMmCpuIo->Io.Read (mMmCpuIo, MM_IO_UINT8, ICH9_APM_CNT, 1,
&ApmControl);
if (EFI_ERROR (Status)) {
DEBUG ((DEBUG_ERROR, "%a: failed to read ICH9_APM_CNT: %r\n", __FUNCTION__,
Status));
//
@@ -152,49 +160,116 @@ CpuHotplugMmi (
//
// Collect the CPUs with pending events.
//
Status = QemuCpuhpCollectApicIds (
mMmCpuIo,
mCpuHotPlugData->ArrayLength, // PossibleCpuCount
mCpuHotPlugData->ArrayLength - 1, // ApicIdCount
mPluggedApicIds,
&PluggedCount,
mToUnplugApicIds,
&ToUnplugCount
);
if (EFI_ERROR (Status)) {
goto Fatal;
}
if (ToUnplugCount > 0) {
DEBUG ((DEBUG_ERROR, "%a: hot-unplug is not supported yet\n",
__FUNCTION__));
goto Fatal;
}
+ //
+ // Process hot-added CPUs.
+ //
+ // The Post-SMM Pen need not be reinstalled multiple times within a single
+ // root MMI handling. Even reinstalling once per root MMI is only prudence;
+ // in theory installing the pen in the driver's entry point function should
+ // suffice.
+ //
+ SmbaseReinstallPostSmmPen (mPostSmmPenAddress);
+
+ PluggedIdx = 0;
+ NewSlot = 0;
+ while (PluggedIdx < PluggedCount) {
+ APIC_ID NewApicId;
+ UINTN NewProcessorNumberByProtocol;
+
+ NewApicId = mPluggedApicIds[PluggedIdx];
+ //
+ // Find the first empty slot in CPU_HOT_PLUG_DATA.
+ //
+ while (NewSlot < mCpuHotPlugData->ArrayLength &&
+ mCpuHotPlugData->ApicId[NewSlot] != MAX_UINT64) {
+ NewSlot++;
+ }
+ if (NewSlot == mCpuHotPlugData->ArrayLength) {
+ DEBUG ((DEBUG_ERROR, "%a: no room for APIC ID " FMT_APIC_ID "\n",
+ __FUNCTION__, NewApicId));
+ goto Fatal;
+ }
+
+ //
+ // Store the APIC ID of the new processor to the slot.
+ //
+ mCpuHotPlugData->ApicId[NewSlot] = NewApicId;
+
+ //
+ // Relocate the SMBASE of the new CPU.
+ //
+ Status = SmbaseRelocate (NewApicId, mCpuHotPlugData->SmBase[NewSlot],
+ mPostSmmPenAddress);
+ if (EFI_ERROR (Status)) {
+ goto RevokeNewSlot;
+ }
+
+ //
+ // Add the new CPU with EFI_SMM_CPU_SERVICE_PROTOCOL.
+ //
+ Status = mMmCpuService->AddProcessor (mMmCpuService, NewApicId,
+ &NewProcessorNumberByProtocol);
+ if (EFI_ERROR (Status)) {
+ DEBUG ((DEBUG_ERROR, "%a: AddProcessor(" FMT_APIC_ID "): %r\n",
+ __FUNCTION__, NewApicId, Status));
+ goto RevokeNewSlot;
+ }
+
+ DEBUG ((DEBUG_INFO, "%a: hot-added APIC ID " FMT_APIC_ID ", SMBASE 0x%Lx, "
+ "EFI_SMM_CPU_SERVICE_PROTOCOL assigned number %Lu\n", __FUNCTION__,
+ NewApicId, (UINT64)mCpuHotPlugData->SmBase[NewSlot],
+ (UINT64)NewProcessorNumberByProtocol));
+
+ NewSlot++;
+ PluggedIdx++;
+ }
+
//
// We've handled this MMI.
//
return EFI_SUCCESS;
+RevokeNewSlot:
+ mCpuHotPlugData->ApicId[NewSlot] = MAX_UINT64;
+
Fatal:
ASSERT (FALSE);
CpuDeadLoop ();
//
// We couldn't handle this MMI.
//
return EFI_INTERRUPT_PENDING;
}
//
// Entry point function of this driver.
//
EFI_STATUS
EFIAPI
CpuHotplugEntry (
IN EFI_HANDLE ImageHandle,
IN EFI_SYSTEM_TABLE *SystemTable
)
{
EFI_STATUS Status;
UINTN Size;
@@ -251,83 +326,101 @@ CpuHotplugEntry (
//
// Allocate the data structures that depend on the possible CPU count.
//
if (RETURN_ERROR (SafeUintnSub (mCpuHotPlugData->ArrayLength, 1, &Size)) ||
RETURN_ERROR (SafeUintnMult (sizeof (APIC_ID), Size, &Size))) {
Status = EFI_ABORTED;
DEBUG ((DEBUG_ERROR, "%a: invalid CPU_HOT_PLUG_DATA\n", __FUNCTION__));
goto Fatal;
}
Status = gMmst->MmAllocatePool (EfiRuntimeServicesData, Size,
(VOID **)&mPluggedApicIds);
if (EFI_ERROR (Status)) {
DEBUG ((DEBUG_ERROR, "%a: MmAllocatePool(): %r\n", __FUNCTION__, Status));
goto Fatal;
}
Status = gMmst->MmAllocatePool (EfiRuntimeServicesData, Size,
(VOID **)&mToUnplugApicIds);
if (EFI_ERROR (Status)) {
DEBUG ((DEBUG_ERROR, "%a: MmAllocatePool(): %r\n", __FUNCTION__, Status));
goto ReleasePluggedApicIds;
}
+ //
+ // Allocate the Post-SMM Pen for hot-added CPUs.
+ //
+ Status = SmbaseAllocatePostSmmPen (&mPostSmmPenAddress,
+ SystemTable->BootServices);
+ if (EFI_ERROR (Status)) {
+ goto ReleaseToUnplugApicIds;
+ }
+
//
// Sanity-check the CPU hotplug interface.
//
// Both of the following features are part of QEMU 5.0, introduced primarily
// in commit range 3e08b2b9cb64..3a61c8db9d25:
//
// (a) the QEMU_CPUHP_CMD_GET_ARCH_ID command of the modern CPU hotplug
// interface,
//
// (b) the "SMRAM at default SMBASE" feature.
//
// From these, (b) is restricted to 5.0+ machine type versions, while (a)
// does not depend on machine type version. Because we ensured the stricter
// condition (b) through PcdQ35SmramAtDefaultSmbase above, the (a)
// QEMU_CPUHP_CMD_GET_ARCH_ID command must now be available too. While we
// can't verify the presence of precisely that command, we can still verify
// (sanity-check) that the modern interface is active, at least.
//
// Consult the "Typical usecases | Detecting and enabling modern CPU hotplug
// interface" section in QEMU's "docs/specs/acpi_cpu_hotplug.txt", on the
// following.
//
QemuCpuhpWriteCpuSelector (mMmCpuIo, 0);
QemuCpuhpWriteCpuSelector (mMmCpuIo, 0);
QemuCpuhpWriteCommand (mMmCpuIo, QEMU_CPUHP_CMD_GET_PENDING);
if (QemuCpuhpReadCommandData2 (mMmCpuIo) != 0) {
Status = EFI_NOT_FOUND;
DEBUG ((DEBUG_ERROR, "%a: modern CPU hotplug interface: %r\n",
__FUNCTION__, Status));
- goto ReleaseToUnplugApicIds;
+ goto ReleasePostSmmPen;
}
//
// Register the handler for the CPU Hotplug MMI.
//
Status = gMmst->MmiHandlerRegister (
CpuHotplugMmi,
NULL, // HandlerType: root MMI handler
&mDispatchHandle
);
if (EFI_ERROR (Status)) {
DEBUG ((DEBUG_ERROR, "%a: MmiHandlerRegister(): %r\n", __FUNCTION__,
Status));
- goto ReleaseToUnplugApicIds;
+ goto ReleasePostSmmPen;
}
+ //
+ // Install the handler for the hot-added CPUs' first SMI.
+ //
+ SmbaseInstallFirstSmiHandler ();
+
return EFI_SUCCESS;
+ReleasePostSmmPen:
+ SmbaseReleasePostSmmPen (mPostSmmPenAddress, SystemTable->BootServices);
+ mPostSmmPenAddress = 0;
+
ReleaseToUnplugApicIds:
gMmst->MmFreePool (mToUnplugApicIds);
mToUnplugApicIds = NULL;
ReleasePluggedApicIds:
gMmst->MmFreePool (mPluggedApicIds);
mPluggedApicIds = NULL;
Fatal:
ASSERT (FALSE);
CpuDeadLoop ();
return Status;
}
--
2.19.1.3.g30247aa5d201
^ permalink raw reply related [flat|nested] 24+ messages in thread* [PATCH 14/16] OvmfPkg: clone CpuS3DataDxe from UefiCpuPkg
2020-02-23 17:25 [PATCH 00/16] OvmfPkg: support VCPU hotplug with -D SMM_REQUIRE Laszlo Ersek
` (12 preceding siblings ...)
2020-02-23 17:25 ` [PATCH 13/16] OvmfPkg/CpuHotplugSmm: complete root MMI handler for CPU hotplug Laszlo Ersek
@ 2020-02-23 17:25 ` Laszlo Ersek
2020-02-23 17:25 ` [PATCH 15/16] OvmfPkg/CpuS3DataDxe: superficial cleanups Laszlo Ersek
` (3 subsequent siblings)
17 siblings, 0 replies; 24+ messages in thread
From: Laszlo Ersek @ 2020-02-23 17:25 UTC (permalink / raw)
To: edk2-devel-groups-io
Cc: Ard Biesheuvel, Igor Mammedov, Jiewen Yao, Jordan Justen,
Michael Kinney, Philippe Mathieu-Daudé
The @file comments in UefiCpuPkg/CpuS3DataDxe say,
[...] It also only supports the number of CPUs reported by the MP
Services Protocol, so this module does not support hot plug CPUs. This
module can be copied into a CPU specific package and customized if these
additional features are required. [...]
The driver is so small that the simplest way to extend it with hotplug
support is indeed to clone it at first. In this patch, customize the
driver only with the following no-op steps:
- Update copyright notices.
- Update INF_VERSION to the latest INF spec version (1.29).
- Update FILE_GUID.
- Drop the UNI files.
- Replace EFI_D_VERBOSE with DEBUG_VERBOSE, to appease "PatchCheck.py".
This patch is best reviewed with:
$ git show --find-copies-harder
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Michael Kinney <michael.d.kinney@intel.com>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=1512
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
---
OvmfPkg/OvmfPkgIa32.dsc | 2 +-
OvmfPkg/OvmfPkgIa32X64.dsc | 2 +-
OvmfPkg/OvmfPkgX64.dsc | 2 +-
OvmfPkg/OvmfPkgIa32.fdf | 2 +-
OvmfPkg/OvmfPkgIa32X64.fdf | 2 +-
OvmfPkg/OvmfPkgX64.fdf | 2 +-
{UefiCpuPkg => OvmfPkg}/CpuS3DataDxe/CpuS3DataDxe.inf | 10 +++-------
{UefiCpuPkg => OvmfPkg}/CpuS3DataDxe/CpuS3Data.c | 4 ++--
8 files changed, 11 insertions(+), 15 deletions(-)
diff --git a/OvmfPkg/OvmfPkgIa32.dsc b/OvmfPkg/OvmfPkgIa32.dsc
index 78310da44a5f..8d8ca746ba03 100644
--- a/OvmfPkg/OvmfPkgIa32.dsc
+++ b/OvmfPkg/OvmfPkgIa32.dsc
@@ -836,45 +836,45 @@ [Components]
!endif
HandleParsingLib|ShellPkg/Library/UefiHandleParsingLib/UefiHandleParsingLib.inf
PrintLib|MdePkg/Library/BasePrintLib/BasePrintLib.inf
BcfgCommandLib|ShellPkg/Library/UefiShellBcfgCommandLib/UefiShellBcfgCommandLib.inf
<PcdsFixedAtBuild>
gEfiMdePkgTokenSpaceGuid.PcdDebugPropertyMask|0xFF
gEfiShellPkgTokenSpaceGuid.PcdShellLibAutoInitialize|FALSE
gEfiMdePkgTokenSpaceGuid.PcdUefiLibMaxPrintBufferSize|8000
}
!if $(SECURE_BOOT_ENABLE) == TRUE
SecurityPkg/VariableAuthenticated/SecureBootConfigDxe/SecureBootConfigDxe.inf
OvmfPkg/EnrollDefaultKeys/EnrollDefaultKeys.inf
!endif
OvmfPkg/PlatformDxe/Platform.inf
OvmfPkg/IoMmuDxe/IoMmuDxe.inf
!if $(SMM_REQUIRE) == TRUE
OvmfPkg/SmmAccess/SmmAccess2Dxe.inf
OvmfPkg/SmmControl2Dxe/SmmControl2Dxe.inf
- UefiCpuPkg/CpuS3DataDxe/CpuS3DataDxe.inf
+ OvmfPkg/CpuS3DataDxe/CpuS3DataDxe.inf
#
# SMM Initial Program Load (a DXE_RUNTIME_DRIVER)
#
MdeModulePkg/Core/PiSmmCore/PiSmmIpl.inf
#
# SMM_CORE
#
MdeModulePkg/Core/PiSmmCore/PiSmmCore.inf
#
# Privileged drivers (DXE_SMM_DRIVER modules)
#
OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf
UefiCpuPkg/CpuIo2Smm/CpuIo2Smm.inf
MdeModulePkg/Universal/LockBox/SmmLockBox/SmmLockBox.inf {
<LibraryClasses>
LockBoxLib|MdeModulePkg/Library/SmmLockBoxLib/SmmLockBoxSmmLib.inf
}
UefiCpuPkg/PiSmmCpuDxeSmm/PiSmmCpuDxeSmm.inf {
<LibraryClasses>
diff --git a/OvmfPkg/OvmfPkgIa32X64.dsc b/OvmfPkg/OvmfPkgIa32X64.dsc
index 428578a4f839..acba1f80a431 100644
--- a/OvmfPkg/OvmfPkgIa32X64.dsc
+++ b/OvmfPkg/OvmfPkgIa32X64.dsc
@@ -850,45 +850,45 @@ [Components.X64]
HandleParsingLib|ShellPkg/Library/UefiHandleParsingLib/UefiHandleParsingLib.inf
PrintLib|MdePkg/Library/BasePrintLib/BasePrintLib.inf
BcfgCommandLib|ShellPkg/Library/UefiShellBcfgCommandLib/UefiShellBcfgCommandLib.inf
<PcdsFixedAtBuild>
gEfiMdePkgTokenSpaceGuid.PcdDebugPropertyMask|0xFF
gEfiShellPkgTokenSpaceGuid.PcdShellLibAutoInitialize|FALSE
gEfiMdePkgTokenSpaceGuid.PcdUefiLibMaxPrintBufferSize|8000
}
!if $(SECURE_BOOT_ENABLE) == TRUE
SecurityPkg/VariableAuthenticated/SecureBootConfigDxe/SecureBootConfigDxe.inf
OvmfPkg/EnrollDefaultKeys/EnrollDefaultKeys.inf
!endif
OvmfPkg/PlatformDxe/Platform.inf
OvmfPkg/AmdSevDxe/AmdSevDxe.inf
OvmfPkg/IoMmuDxe/IoMmuDxe.inf
!if $(SMM_REQUIRE) == TRUE
OvmfPkg/SmmAccess/SmmAccess2Dxe.inf
OvmfPkg/SmmControl2Dxe/SmmControl2Dxe.inf
- UefiCpuPkg/CpuS3DataDxe/CpuS3DataDxe.inf
+ OvmfPkg/CpuS3DataDxe/CpuS3DataDxe.inf
#
# SMM Initial Program Load (a DXE_RUNTIME_DRIVER)
#
MdeModulePkg/Core/PiSmmCore/PiSmmIpl.inf
#
# SMM_CORE
#
MdeModulePkg/Core/PiSmmCore/PiSmmCore.inf
#
# Privileged drivers (DXE_SMM_DRIVER modules)
#
OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf
UefiCpuPkg/CpuIo2Smm/CpuIo2Smm.inf
MdeModulePkg/Universal/LockBox/SmmLockBox/SmmLockBox.inf {
<LibraryClasses>
LockBoxLib|MdeModulePkg/Library/SmmLockBoxLib/SmmLockBoxSmmLib.inf
}
UefiCpuPkg/PiSmmCpuDxeSmm/PiSmmCpuDxeSmm.inf {
<LibraryClasses>
diff --git a/OvmfPkg/OvmfPkgX64.dsc b/OvmfPkg/OvmfPkgX64.dsc
index 73b92f259201..621b27f80d4b 100644
--- a/OvmfPkg/OvmfPkgX64.dsc
+++ b/OvmfPkg/OvmfPkgX64.dsc
@@ -848,45 +848,45 @@ [Components]
HandleParsingLib|ShellPkg/Library/UefiHandleParsingLib/UefiHandleParsingLib.inf
PrintLib|MdePkg/Library/BasePrintLib/BasePrintLib.inf
BcfgCommandLib|ShellPkg/Library/UefiShellBcfgCommandLib/UefiShellBcfgCommandLib.inf
<PcdsFixedAtBuild>
gEfiMdePkgTokenSpaceGuid.PcdDebugPropertyMask|0xFF
gEfiShellPkgTokenSpaceGuid.PcdShellLibAutoInitialize|FALSE
gEfiMdePkgTokenSpaceGuid.PcdUefiLibMaxPrintBufferSize|8000
}
!if $(SECURE_BOOT_ENABLE) == TRUE
SecurityPkg/VariableAuthenticated/SecureBootConfigDxe/SecureBootConfigDxe.inf
OvmfPkg/EnrollDefaultKeys/EnrollDefaultKeys.inf
!endif
OvmfPkg/PlatformDxe/Platform.inf
OvmfPkg/AmdSevDxe/AmdSevDxe.inf
OvmfPkg/IoMmuDxe/IoMmuDxe.inf
!if $(SMM_REQUIRE) == TRUE
OvmfPkg/SmmAccess/SmmAccess2Dxe.inf
OvmfPkg/SmmControl2Dxe/SmmControl2Dxe.inf
- UefiCpuPkg/CpuS3DataDxe/CpuS3DataDxe.inf
+ OvmfPkg/CpuS3DataDxe/CpuS3DataDxe.inf
#
# SMM Initial Program Load (a DXE_RUNTIME_DRIVER)
#
MdeModulePkg/Core/PiSmmCore/PiSmmIpl.inf
#
# SMM_CORE
#
MdeModulePkg/Core/PiSmmCore/PiSmmCore.inf
#
# Privileged drivers (DXE_SMM_DRIVER modules)
#
OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf
UefiCpuPkg/CpuIo2Smm/CpuIo2Smm.inf
MdeModulePkg/Universal/LockBox/SmmLockBox/SmmLockBox.inf {
<LibraryClasses>
LockBoxLib|MdeModulePkg/Library/SmmLockBoxLib/SmmLockBoxSmmLib.inf
}
UefiCpuPkg/PiSmmCpuDxeSmm/PiSmmCpuDxeSmm.inf {
<LibraryClasses>
diff --git a/OvmfPkg/OvmfPkgIa32.fdf b/OvmfPkg/OvmfPkgIa32.fdf
index 61b891765c56..004aa318b222 100644
--- a/OvmfPkg/OvmfPkgIa32.fdf
+++ b/OvmfPkg/OvmfPkgIa32.fdf
@@ -298,45 +298,45 @@ [FV.DXEFV]
INF MdeModulePkg/Bus/Pci/EhciDxe/EhciDxe.inf
INF MdeModulePkg/Bus/Pci/XhciDxe/XhciDxe.inf
INF MdeModulePkg/Bus/Usb/UsbBusDxe/UsbBusDxe.inf
INF MdeModulePkg/Bus/Usb/UsbKbDxe/UsbKbDxe.inf
INF MdeModulePkg/Bus/Usb/UsbMassStorageDxe/UsbMassStorageDxe.inf
!ifdef $(CSM_ENABLE)
INF OvmfPkg/Csm/BiosThunk/VideoDxe/VideoDxe.inf
INF OvmfPkg/Csm/LegacyBiosDxe/LegacyBiosDxe.inf
INF RuleOverride=CSM OvmfPkg/Csm/Csm16/Csm16.inf
!else
INF OvmfPkg/QemuVideoDxe/QemuVideoDxe.inf
!endif
INF OvmfPkg/QemuRamfbDxe/QemuRamfbDxe.inf
INF OvmfPkg/VirtioGpuDxe/VirtioGpu.inf
INF OvmfPkg/PlatformDxe/Platform.inf
INF OvmfPkg/IoMmuDxe/IoMmuDxe.inf
!if $(SMM_REQUIRE) == TRUE
INF OvmfPkg/SmmAccess/SmmAccess2Dxe.inf
INF OvmfPkg/SmmControl2Dxe/SmmControl2Dxe.inf
-INF UefiCpuPkg/CpuS3DataDxe/CpuS3DataDxe.inf
+INF OvmfPkg/CpuS3DataDxe/CpuS3DataDxe.inf
INF MdeModulePkg/Core/PiSmmCore/PiSmmIpl.inf
INF MdeModulePkg/Core/PiSmmCore/PiSmmCore.inf
INF OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf
INF UefiCpuPkg/CpuIo2Smm/CpuIo2Smm.inf
INF MdeModulePkg/Universal/LockBox/SmmLockBox/SmmLockBox.inf
INF UefiCpuPkg/PiSmmCpuDxeSmm/PiSmmCpuDxeSmm.inf
#
# Variable driver stack (SMM)
#
INF OvmfPkg/QemuFlashFvbServicesRuntimeDxe/FvbServicesSmm.inf
INF MdeModulePkg/Universal/FaultTolerantWriteDxe/FaultTolerantWriteSmm.inf
INF MdeModulePkg/Universal/Variable/RuntimeDxe/VariableSmm.inf
INF MdeModulePkg/Universal/Variable/RuntimeDxe/VariableSmmRuntimeDxe.inf
!else
#
# Variable driver stack (non-SMM)
#
INF OvmfPkg/QemuFlashFvbServicesRuntimeDxe/FvbServicesRuntimeDxe.inf
INF OvmfPkg/EmuVariableFvbRuntimeDxe/Fvb.inf
diff --git a/OvmfPkg/OvmfPkgIa32X64.fdf b/OvmfPkg/OvmfPkgIa32X64.fdf
index 501b4fcb7b67..13da8b9dbe65 100644
--- a/OvmfPkg/OvmfPkgIa32X64.fdf
+++ b/OvmfPkg/OvmfPkgIa32X64.fdf
@@ -305,45 +305,45 @@ [FV.DXEFV]
INF MdeModulePkg/Bus/Pci/XhciDxe/XhciDxe.inf
INF MdeModulePkg/Bus/Usb/UsbBusDxe/UsbBusDxe.inf
INF MdeModulePkg/Bus/Usb/UsbKbDxe/UsbKbDxe.inf
INF MdeModulePkg/Bus/Usb/UsbMassStorageDxe/UsbMassStorageDxe.inf
!ifdef $(CSM_ENABLE)
INF OvmfPkg/Csm/BiosThunk/VideoDxe/VideoDxe.inf
INF OvmfPkg/Csm/LegacyBiosDxe/LegacyBiosDxe.inf
INF RuleOverride=CSM OvmfPkg/Csm/Csm16/Csm16.inf
!else
INF OvmfPkg/QemuVideoDxe/QemuVideoDxe.inf
!endif
INF OvmfPkg/QemuRamfbDxe/QemuRamfbDxe.inf
INF OvmfPkg/VirtioGpuDxe/VirtioGpu.inf
INF OvmfPkg/PlatformDxe/Platform.inf
INF OvmfPkg/AmdSevDxe/AmdSevDxe.inf
INF OvmfPkg/IoMmuDxe/IoMmuDxe.inf
!if $(SMM_REQUIRE) == TRUE
INF OvmfPkg/SmmAccess/SmmAccess2Dxe.inf
INF OvmfPkg/SmmControl2Dxe/SmmControl2Dxe.inf
-INF UefiCpuPkg/CpuS3DataDxe/CpuS3DataDxe.inf
+INF OvmfPkg/CpuS3DataDxe/CpuS3DataDxe.inf
INF MdeModulePkg/Core/PiSmmCore/PiSmmIpl.inf
INF MdeModulePkg/Core/PiSmmCore/PiSmmCore.inf
INF OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf
INF UefiCpuPkg/CpuIo2Smm/CpuIo2Smm.inf
INF MdeModulePkg/Universal/LockBox/SmmLockBox/SmmLockBox.inf
INF UefiCpuPkg/PiSmmCpuDxeSmm/PiSmmCpuDxeSmm.inf
#
# Variable driver stack (SMM)
#
INF OvmfPkg/QemuFlashFvbServicesRuntimeDxe/FvbServicesSmm.inf
INF MdeModulePkg/Universal/FaultTolerantWriteDxe/FaultTolerantWriteSmm.inf
INF MdeModulePkg/Universal/Variable/RuntimeDxe/VariableSmm.inf
INF MdeModulePkg/Universal/Variable/RuntimeDxe/VariableSmmRuntimeDxe.inf
!else
#
# Variable driver stack (non-SMM)
#
INF OvmfPkg/QemuFlashFvbServicesRuntimeDxe/FvbServicesRuntimeDxe.inf
INF OvmfPkg/EmuVariableFvbRuntimeDxe/Fvb.inf
diff --git a/OvmfPkg/OvmfPkgX64.fdf b/OvmfPkg/OvmfPkgX64.fdf
index 501b4fcb7b67..13da8b9dbe65 100644
--- a/OvmfPkg/OvmfPkgX64.fdf
+++ b/OvmfPkg/OvmfPkgX64.fdf
@@ -305,45 +305,45 @@ [FV.DXEFV]
INF MdeModulePkg/Bus/Pci/XhciDxe/XhciDxe.inf
INF MdeModulePkg/Bus/Usb/UsbBusDxe/UsbBusDxe.inf
INF MdeModulePkg/Bus/Usb/UsbKbDxe/UsbKbDxe.inf
INF MdeModulePkg/Bus/Usb/UsbMassStorageDxe/UsbMassStorageDxe.inf
!ifdef $(CSM_ENABLE)
INF OvmfPkg/Csm/BiosThunk/VideoDxe/VideoDxe.inf
INF OvmfPkg/Csm/LegacyBiosDxe/LegacyBiosDxe.inf
INF RuleOverride=CSM OvmfPkg/Csm/Csm16/Csm16.inf
!else
INF OvmfPkg/QemuVideoDxe/QemuVideoDxe.inf
!endif
INF OvmfPkg/QemuRamfbDxe/QemuRamfbDxe.inf
INF OvmfPkg/VirtioGpuDxe/VirtioGpu.inf
INF OvmfPkg/PlatformDxe/Platform.inf
INF OvmfPkg/AmdSevDxe/AmdSevDxe.inf
INF OvmfPkg/IoMmuDxe/IoMmuDxe.inf
!if $(SMM_REQUIRE) == TRUE
INF OvmfPkg/SmmAccess/SmmAccess2Dxe.inf
INF OvmfPkg/SmmControl2Dxe/SmmControl2Dxe.inf
-INF UefiCpuPkg/CpuS3DataDxe/CpuS3DataDxe.inf
+INF OvmfPkg/CpuS3DataDxe/CpuS3DataDxe.inf
INF MdeModulePkg/Core/PiSmmCore/PiSmmIpl.inf
INF MdeModulePkg/Core/PiSmmCore/PiSmmCore.inf
INF OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf
INF UefiCpuPkg/CpuIo2Smm/CpuIo2Smm.inf
INF MdeModulePkg/Universal/LockBox/SmmLockBox/SmmLockBox.inf
INF UefiCpuPkg/PiSmmCpuDxeSmm/PiSmmCpuDxeSmm.inf
#
# Variable driver stack (SMM)
#
INF OvmfPkg/QemuFlashFvbServicesRuntimeDxe/FvbServicesSmm.inf
INF MdeModulePkg/Universal/FaultTolerantWriteDxe/FaultTolerantWriteSmm.inf
INF MdeModulePkg/Universal/Variable/RuntimeDxe/VariableSmm.inf
INF MdeModulePkg/Universal/Variable/RuntimeDxe/VariableSmmRuntimeDxe.inf
!else
#
# Variable driver stack (non-SMM)
#
INF OvmfPkg/QemuFlashFvbServicesRuntimeDxe/FvbServicesRuntimeDxe.inf
INF OvmfPkg/EmuVariableFvbRuntimeDxe/Fvb.inf
diff --git a/UefiCpuPkg/CpuS3DataDxe/CpuS3DataDxe.inf b/OvmfPkg/CpuS3DataDxe/CpuS3DataDxe.inf
similarity index 83%
copy from UefiCpuPkg/CpuS3DataDxe/CpuS3DataDxe.inf
copy to OvmfPkg/CpuS3DataDxe/CpuS3DataDxe.inf
index 510133a614ba..0ad8a0b35d25 100644
--- a/UefiCpuPkg/CpuS3DataDxe/CpuS3DataDxe.inf
+++ b/OvmfPkg/CpuS3DataDxe/CpuS3DataDxe.inf
@@ -1,65 +1,61 @@
## @file
# ACPI CPU Data initialization module
#
# This module initializes the ACPI_CPU_DATA structure and registers the address
# of this structure in the PcdCpuS3DataAddress PCD. This is a generic/simple
# version of this module. It does not provide a machine check handler or CPU
# register initialization tables for ACPI S3 resume. It also only supports the
# number of CPUs reported by the MP Services Protocol, so this module does not
# support hot plug CPUs. This module can be copied into a CPU specific package
# and customized if these additional features are required.
#
# Copyright (c) 2013-2016, Intel Corporation. All rights reserved.<BR>
-# Copyright (c) 2015, Red Hat, Inc.
+# Copyright (c) 2015-2020, Red Hat, Inc.
#
# SPDX-License-Identifier: BSD-2-Clause-Patent
#
##
[Defines]
- INF_VERSION = 0x00010005
+ INF_VERSION = 1.29
BASE_NAME = CpuS3DataDxe
- MODULE_UNI_FILE = CpuS3DataDxe.uni
- FILE_GUID = 4D2E57EE-0E3F-44DD-93C4-D3B57E96945D
+ FILE_GUID = 229B7EFD-DA02-46B9-93F4-E20C009F94E9
MODULE_TYPE = DXE_DRIVER
VERSION_STRING = 1.0
ENTRY_POINT = CpuS3DataInitialize
# The following information is for reference only and not required by the build
# tools.
#
# VALID_ARCHITECTURES = IA32 X64
[Sources]
CpuS3Data.c
[Packages]
MdePkg/MdePkg.dec
MdeModulePkg/MdeModulePkg.dec
UefiCpuPkg/UefiCpuPkg.dec
[LibraryClasses]
UefiDriverEntryPoint
UefiBootServicesTableLib
BaseMemoryLib
DebugLib
BaseLib
MtrrLib
MemoryAllocationLib
[Guids]
gEfiEndOfDxeEventGroupGuid ## CONSUMES ## Event
[Protocols]
gEfiMpServiceProtocolGuid ## CONSUMES
[Pcd]
gUefiCpuPkgTokenSpaceGuid.PcdCpuApStackSize ## CONSUMES
gUefiCpuPkgTokenSpaceGuid.PcdCpuS3DataAddress ## PRODUCES
gEfiMdeModulePkgTokenSpaceGuid.PcdAcpiS3Enable ## CONSUMES
[Depex]
gEfiMpServiceProtocolGuid
-
-[UserExtensions.TianoCore."ExtraFiles"]
- CpuS3DataDxeExtra.uni
diff --git a/UefiCpuPkg/CpuS3DataDxe/CpuS3Data.c b/OvmfPkg/CpuS3DataDxe/CpuS3Data.c
similarity index 96%
copy from UefiCpuPkg/CpuS3DataDxe/CpuS3Data.c
copy to OvmfPkg/CpuS3DataDxe/CpuS3Data.c
index 2be335d91903..2bb60d591b1e 100644
--- a/UefiCpuPkg/CpuS3DataDxe/CpuS3Data.c
+++ b/OvmfPkg/CpuS3DataDxe/CpuS3Data.c
@@ -1,35 +1,35 @@
/** @file
ACPI CPU Data initialization module
This module initializes the ACPI_CPU_DATA structure and registers the address
of this structure in the PcdCpuS3DataAddress PCD. This is a generic/simple
version of this module. It does not provide a machine check handler or CPU
register initialization tables for ACPI S3 resume. It also only supports the
number of CPUs reported by the MP Services Protocol, so this module does not
support hot plug CPUs. This module can be copied into a CPU specific package
and customized if these additional features are required.
Copyright (c) 2013 - 2017, Intel Corporation. All rights reserved.<BR>
-Copyright (c) 2015, Red Hat, Inc.
+Copyright (c) 2015 - 2020, Red Hat, Inc.
SPDX-License-Identifier: BSD-2-Clause-Patent
**/
#include <PiDxe.h>
#include <AcpiCpuData.h>
#include <Library/BaseLib.h>
#include <Library/BaseMemoryLib.h>
#include <Library/UefiBootServicesTableLib.h>
#include <Library/DebugLib.h>
#include <Library/MtrrLib.h>
#include <Library/MemoryAllocationLib.h>
#include <Protocol/MpService.h>
#include <Guid/EventGroup.h>
//
// Data structure used to allocate ACPI_CPU_DATA and its supporting structures
//
@@ -107,45 +107,45 @@ VOID
EFIAPI
CpuS3DataOnEndOfDxe (
IN EFI_EVENT Event,
OUT VOID *Context
)
{
EFI_STATUS Status;
ACPI_CPU_DATA_EX *AcpiCpuDataEx;
AcpiCpuDataEx = (ACPI_CPU_DATA_EX *) Context;
//
// Allocate a 4KB reserved page below 1MB
//
AcpiCpuDataEx->AcpiCpuData.StartupVector = BASE_1MB - 1;
Status = gBS->AllocatePages (
AllocateMaxAddress,
EfiReservedMemoryType,
1,
&AcpiCpuDataEx->AcpiCpuData.StartupVector
);
ASSERT_EFI_ERROR (Status);
- DEBUG ((EFI_D_VERBOSE, "%a\n", __FUNCTION__));
+ DEBUG ((DEBUG_VERBOSE, "%a\n", __FUNCTION__));
MtrrGetAllMtrrs (&AcpiCpuDataEx->MtrrTable);
//
// Close event, so it will not be invoked again.
//
gBS->CloseEvent (Event);
}
/**
The entry function of the CpuS3Data driver.
Allocate and initialize all fields of the ACPI_CPU_DATA structure except the
MTRR settings. Register an event notification on gEfiEndOfDxeEventGroupGuid
to capture the ACPI_CPU_DATA MTRR settings. The PcdCpuS3DataAddress is set
to the address that ACPI_CPU_DATA is allocated at.
@param[in] ImageHandle The firmware allocated handle for the EFI image.
@param[in] SystemTable A pointer to the EFI System Table.
@retval EFI_SUCCESS The entry point is executed successfully.
@retval EFI_UNSUPPORTED Do not support ACPI S3.
@retval other Some error occurs when executing this entry point.
--
2.19.1.3.g30247aa5d201
^ permalink raw reply related [flat|nested] 24+ messages in thread* [PATCH 15/16] OvmfPkg/CpuS3DataDxe: superficial cleanups
2020-02-23 17:25 [PATCH 00/16] OvmfPkg: support VCPU hotplug with -D SMM_REQUIRE Laszlo Ersek
` (13 preceding siblings ...)
2020-02-23 17:25 ` [PATCH 14/16] OvmfPkg: clone CpuS3DataDxe from UefiCpuPkg Laszlo Ersek
@ 2020-02-23 17:25 ` Laszlo Ersek
2020-02-23 17:25 ` [PATCH 16/16] OvmfPkg/CpuS3DataDxe: enable S3 resume after CPU hotplug Laszlo Ersek
` (2 subsequent siblings)
17 siblings, 0 replies; 24+ messages in thread
From: Laszlo Ersek @ 2020-02-23 17:25 UTC (permalink / raw)
To: edk2-devel-groups-io
Cc: Ard Biesheuvel, Igor Mammedov, Jiewen Yao, Jordan Justen,
Michael Kinney, Philippe Mathieu-Daudé
Sort the [Packages], [LibraryClasses], and [Pcd] sections in the INF file.
Pad the usage notes (CONSUMES, PRODUCES) in the [Pcd] section.
Sort the Library #includes in the C file.
This patch is functionally a no-op.
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Michael Kinney <michael.d.kinney@intel.com>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=1512
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
---
OvmfPkg/CpuS3DataDxe/CpuS3DataDxe.inf | 16 ++++++++--------
OvmfPkg/CpuS3DataDxe/CpuS3Data.c | 4 ++--
2 files changed, 10 insertions(+), 10 deletions(-)
diff --git a/OvmfPkg/CpuS3DataDxe/CpuS3DataDxe.inf b/OvmfPkg/CpuS3DataDxe/CpuS3DataDxe.inf
index 0ad8a0b35d25..f9679e0c33b3 100644
--- a/OvmfPkg/CpuS3DataDxe/CpuS3DataDxe.inf
+++ b/OvmfPkg/CpuS3DataDxe/CpuS3DataDxe.inf
@@ -14,48 +14,48 @@
#
# SPDX-License-Identifier: BSD-2-Clause-Patent
#
##
[Defines]
INF_VERSION = 1.29
BASE_NAME = CpuS3DataDxe
FILE_GUID = 229B7EFD-DA02-46B9-93F4-E20C009F94E9
MODULE_TYPE = DXE_DRIVER
VERSION_STRING = 1.0
ENTRY_POINT = CpuS3DataInitialize
# The following information is for reference only and not required by the build
# tools.
#
# VALID_ARCHITECTURES = IA32 X64
[Sources]
CpuS3Data.c
[Packages]
- MdePkg/MdePkg.dec
MdeModulePkg/MdeModulePkg.dec
+ MdePkg/MdePkg.dec
UefiCpuPkg/UefiCpuPkg.dec
[LibraryClasses]
- UefiDriverEntryPoint
- UefiBootServicesTableLib
+ BaseLib
BaseMemoryLib
DebugLib
- BaseLib
- MtrrLib
MemoryAllocationLib
+ MtrrLib
+ UefiBootServicesTableLib
+ UefiDriverEntryPoint
[Guids]
gEfiEndOfDxeEventGroupGuid ## CONSUMES ## Event
[Protocols]
gEfiMpServiceProtocolGuid ## CONSUMES
[Pcd]
- gUefiCpuPkgTokenSpaceGuid.PcdCpuApStackSize ## CONSUMES
- gUefiCpuPkgTokenSpaceGuid.PcdCpuS3DataAddress ## PRODUCES
- gEfiMdeModulePkgTokenSpaceGuid.PcdAcpiS3Enable ## CONSUMES
+ gEfiMdeModulePkgTokenSpaceGuid.PcdAcpiS3Enable ## CONSUMES
+ gUefiCpuPkgTokenSpaceGuid.PcdCpuApStackSize ## CONSUMES
+ gUefiCpuPkgTokenSpaceGuid.PcdCpuS3DataAddress ## PRODUCES
[Depex]
gEfiMpServiceProtocolGuid
diff --git a/OvmfPkg/CpuS3DataDxe/CpuS3Data.c b/OvmfPkg/CpuS3DataDxe/CpuS3Data.c
index 2bb60d591b1e..8bb9807cd501 100644
--- a/OvmfPkg/CpuS3DataDxe/CpuS3Data.c
+++ b/OvmfPkg/CpuS3DataDxe/CpuS3Data.c
@@ -3,48 +3,48 @@ ACPI CPU Data initialization module
This module initializes the ACPI_CPU_DATA structure and registers the address
of this structure in the PcdCpuS3DataAddress PCD. This is a generic/simple
version of this module. It does not provide a machine check handler or CPU
register initialization tables for ACPI S3 resume. It also only supports the
number of CPUs reported by the MP Services Protocol, so this module does not
support hot plug CPUs. This module can be copied into a CPU specific package
and customized if these additional features are required.
Copyright (c) 2013 - 2017, Intel Corporation. All rights reserved.<BR>
Copyright (c) 2015 - 2020, Red Hat, Inc.
SPDX-License-Identifier: BSD-2-Clause-Patent
**/
#include <PiDxe.h>
#include <AcpiCpuData.h>
#include <Library/BaseLib.h>
#include <Library/BaseMemoryLib.h>
-#include <Library/UefiBootServicesTableLib.h>
#include <Library/DebugLib.h>
-#include <Library/MtrrLib.h>
#include <Library/MemoryAllocationLib.h>
+#include <Library/MtrrLib.h>
+#include <Library/UefiBootServicesTableLib.h>
#include <Protocol/MpService.h>
#include <Guid/EventGroup.h>
//
// Data structure used to allocate ACPI_CPU_DATA and its supporting structures
//
typedef struct {
ACPI_CPU_DATA AcpiCpuData;
MTRR_SETTINGS MtrrTable;
IA32_DESCRIPTOR GdtrProfile;
IA32_DESCRIPTOR IdtrProfile;
} ACPI_CPU_DATA_EX;
/**
Allocate EfiACPIMemoryNVS memory.
@param[in] Size Size of memory to allocate.
@return Allocated address for output.
**/
--
2.19.1.3.g30247aa5d201
^ permalink raw reply related [flat|nested] 24+ messages in thread* [PATCH 16/16] OvmfPkg/CpuS3DataDxe: enable S3 resume after CPU hotplug
2020-02-23 17:25 [PATCH 00/16] OvmfPkg: support VCPU hotplug with -D SMM_REQUIRE Laszlo Ersek
` (14 preceding siblings ...)
2020-02-23 17:25 ` [PATCH 15/16] OvmfPkg/CpuS3DataDxe: superficial cleanups Laszlo Ersek
@ 2020-02-23 17:25 ` Laszlo Ersek
2020-02-24 16:31 ` [PATCH 00/16] OvmfPkg: support VCPU hotplug with -D SMM_REQUIRE Ard Biesheuvel
2020-07-24 6:26 ` [edk2-devel] " Wu, Jiaxin
17 siblings, 0 replies; 24+ messages in thread
From: Laszlo Ersek @ 2020-02-23 17:25 UTC (permalink / raw)
To: edk2-devel-groups-io
Cc: Ard Biesheuvel, Igor Mammedov, Jiewen Yao, Jordan Justen,
Michael Kinney, Philippe Mathieu-Daudé
During normal boot, CpuS3DataDxe allocates
- an empty CPU_REGISTER_TABLE entry in the
"ACPI_CPU_DATA.PreSmmInitRegisterTable" array, and
- an empty CPU_REGISTER_TABLE entry in the "ACPI_CPU_DATA.RegisterTable"
array,
for every CPU whose APIC ID CpuS3DataDxe can learn.
Currently EFI_MP_SERVICES_PROTOCOL is used for both determining the number
of CPUs -- the protocol reports the present-at-boot CPU count --, and for
retrieving the APIC IDs of those CPUs.
Consequently, if a CPU is hot-plugged at OS runtime, then S3 resume
breaks. That's because PiSmmCpuDxeSmm will not find the hot-added CPU's
APIC ID associated with any CPU_REGISTER_TABLE object, in the SMRAM copies
of either of the "RegisterTable" and "PreSmmInitRegisterTable" arrays. The
failure to match the hot-added CPU's APIC ID trips the ASSERT() in
SetRegister() [UefiCpuPkg/PiSmmCpuDxeSmm/CpuS3.c].
If "PcdQ35SmramAtDefaultSmbase" is TRUE, then:
- prepare CPU_REGISTER_TABLE objects for all possible CPUs, not just the
present-at-boot CPUs (PlatformPei stored the possible CPU count to
"PcdCpuMaxLogicalProcessorNumber");
- use QEMU_CPUHP_CMD_GET_ARCH_ID for filling in the "InitialApicId" fields
of the CPU_REGISTER_TABLE objects.
This provides full APIC ID coverage for PiSmmCpuDxeSmm during S3 resume,
accommodating CPUs hot-added at OS runtime.
This patch is best reviewed with
$ git show -b
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Michael Kinney <michael.d.kinney@intel.com>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=1512
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
---
OvmfPkg/CpuS3DataDxe/CpuS3DataDxe.inf | 4 +
OvmfPkg/CpuS3DataDxe/CpuS3Data.c | 91 ++++++++++++++------
2 files changed, 70 insertions(+), 25 deletions(-)
diff --git a/OvmfPkg/CpuS3DataDxe/CpuS3DataDxe.inf b/OvmfPkg/CpuS3DataDxe/CpuS3DataDxe.inf
index f9679e0c33b3..ceae1d4078c7 100644
--- a/OvmfPkg/CpuS3DataDxe/CpuS3DataDxe.inf
+++ b/OvmfPkg/CpuS3DataDxe/CpuS3DataDxe.inf
@@ -16,46 +16,50 @@
#
##
[Defines]
INF_VERSION = 1.29
BASE_NAME = CpuS3DataDxe
FILE_GUID = 229B7EFD-DA02-46B9-93F4-E20C009F94E9
MODULE_TYPE = DXE_DRIVER
VERSION_STRING = 1.0
ENTRY_POINT = CpuS3DataInitialize
# The following information is for reference only and not required by the build
# tools.
#
# VALID_ARCHITECTURES = IA32 X64
[Sources]
CpuS3Data.c
[Packages]
MdeModulePkg/MdeModulePkg.dec
MdePkg/MdePkg.dec
+ OvmfPkg/OvmfPkg.dec
UefiCpuPkg/UefiCpuPkg.dec
[LibraryClasses]
BaseLib
BaseMemoryLib
DebugLib
+ IoLib
MemoryAllocationLib
MtrrLib
UefiBootServicesTableLib
UefiDriverEntryPoint
[Guids]
gEfiEndOfDxeEventGroupGuid ## CONSUMES ## Event
[Protocols]
gEfiMpServiceProtocolGuid ## CONSUMES
[Pcd]
gEfiMdeModulePkgTokenSpaceGuid.PcdAcpiS3Enable ## CONSUMES
gUefiCpuPkgTokenSpaceGuid.PcdCpuApStackSize ## CONSUMES
+ gUefiCpuPkgTokenSpaceGuid.PcdCpuMaxLogicalProcessorNumber ## CONSUMES
gUefiCpuPkgTokenSpaceGuid.PcdCpuS3DataAddress ## PRODUCES
+ gUefiOvmfPkgTokenSpaceGuid.PcdQ35SmramAtDefaultSmbase ## CONSUMES
[Depex]
gEfiMpServiceProtocolGuid
diff --git a/OvmfPkg/CpuS3DataDxe/CpuS3Data.c b/OvmfPkg/CpuS3DataDxe/CpuS3Data.c
index 8bb9807cd501..bac7285aa2f3 100644
--- a/OvmfPkg/CpuS3DataDxe/CpuS3Data.c
+++ b/OvmfPkg/CpuS3DataDxe/CpuS3Data.c
@@ -4,51 +4,55 @@ ACPI CPU Data initialization module
This module initializes the ACPI_CPU_DATA structure and registers the address
of this structure in the PcdCpuS3DataAddress PCD. This is a generic/simple
version of this module. It does not provide a machine check handler or CPU
register initialization tables for ACPI S3 resume. It also only supports the
number of CPUs reported by the MP Services Protocol, so this module does not
support hot plug CPUs. This module can be copied into a CPU specific package
and customized if these additional features are required.
Copyright (c) 2013 - 2017, Intel Corporation. All rights reserved.<BR>
Copyright (c) 2015 - 2020, Red Hat, Inc.
SPDX-License-Identifier: BSD-2-Clause-Patent
**/
#include <PiDxe.h>
#include <AcpiCpuData.h>
#include <Library/BaseLib.h>
#include <Library/BaseMemoryLib.h>
#include <Library/DebugLib.h>
+#include <Library/IoLib.h>
#include <Library/MemoryAllocationLib.h>
#include <Library/MtrrLib.h>
#include <Library/UefiBootServicesTableLib.h>
#include <Protocol/MpService.h>
#include <Guid/EventGroup.h>
+#include <IndustryStandard/Q35MchIch9.h>
+#include <IndustryStandard/QemuCpuHotplug.h>
+
//
// Data structure used to allocate ACPI_CPU_DATA and its supporting structures
//
typedef struct {
ACPI_CPU_DATA AcpiCpuData;
MTRR_SETTINGS MtrrTable;
IA32_DESCRIPTOR GdtrProfile;
IA32_DESCRIPTOR IdtrProfile;
} ACPI_CPU_DATA_EX;
/**
Allocate EfiACPIMemoryNVS memory.
@param[in] Size Size of memory to allocate.
@return Allocated address for output.
**/
VOID *
AllocateAcpiNvsMemory (
IN UINTN Size
)
@@ -144,89 +148,101 @@ CpuS3DataOnEndOfDxe (
to the address that ACPI_CPU_DATA is allocated at.
@param[in] ImageHandle The firmware allocated handle for the EFI image.
@param[in] SystemTable A pointer to the EFI System Table.
@retval EFI_SUCCESS The entry point is executed successfully.
@retval EFI_UNSUPPORTED Do not support ACPI S3.
@retval other Some error occurs when executing this entry point.
**/
EFI_STATUS
EFIAPI
CpuS3DataInitialize (
IN EFI_HANDLE ImageHandle,
IN EFI_SYSTEM_TABLE *SystemTable
)
{
EFI_STATUS Status;
ACPI_CPU_DATA_EX *AcpiCpuDataEx;
ACPI_CPU_DATA *AcpiCpuData;
EFI_MP_SERVICES_PROTOCOL *MpServices;
UINTN NumberOfCpus;
- UINTN NumberOfEnabledProcessors;
VOID *Stack;
UINTN TableSize;
CPU_REGISTER_TABLE *RegisterTable;
UINTN Index;
EFI_PROCESSOR_INFORMATION ProcessorInfoBuffer;
UINTN GdtSize;
UINTN IdtSize;
VOID *Gdt;
VOID *Idt;
EFI_EVENT Event;
ACPI_CPU_DATA *OldAcpiCpuData;
+ BOOLEAN FetchPossibleApicIds;
if (!PcdGetBool (PcdAcpiS3Enable)) {
return EFI_UNSUPPORTED;
}
//
// Set PcdCpuS3DataAddress to the base address of the ACPI_CPU_DATA structure
//
OldAcpiCpuData = (ACPI_CPU_DATA *) (UINTN) PcdGet64 (PcdCpuS3DataAddress);
AcpiCpuDataEx = AllocateZeroPages (sizeof (ACPI_CPU_DATA_EX));
ASSERT (AcpiCpuDataEx != NULL);
AcpiCpuData = &AcpiCpuDataEx->AcpiCpuData;
//
- // Get MP Services Protocol
+ // The "SMRAM at default SMBASE" feature guarantees that
+ // QEMU_CPUHP_CMD_GET_ARCH_ID too is available.
//
- Status = gBS->LocateProtocol (
- &gEfiMpServiceProtocolGuid,
- NULL,
- (VOID **)&MpServices
- );
- ASSERT_EFI_ERROR (Status);
+ FetchPossibleApicIds = PcdGetBool (PcdQ35SmramAtDefaultSmbase);
- //
- // Get the number of CPUs
- //
- Status = MpServices->GetNumberOfProcessors (
- MpServices,
- &NumberOfCpus,
- &NumberOfEnabledProcessors
- );
- ASSERT_EFI_ERROR (Status);
+ if (FetchPossibleApicIds) {
+ NumberOfCpus = PcdGet32 (PcdCpuMaxLogicalProcessorNumber);
+ } else {
+ UINTN NumberOfEnabledProcessors;
+
+ //
+ // Get MP Services Protocol
+ //
+ Status = gBS->LocateProtocol (
+ &gEfiMpServiceProtocolGuid,
+ NULL,
+ (VOID **)&MpServices
+ );
+ ASSERT_EFI_ERROR (Status);
+
+ //
+ // Get the number of CPUs
+ //
+ Status = MpServices->GetNumberOfProcessors (
+ MpServices,
+ &NumberOfCpus,
+ &NumberOfEnabledProcessors
+ );
+ ASSERT_EFI_ERROR (Status);
+ }
AcpiCpuData->NumberOfCpus = (UINT32)NumberOfCpus;
//
// Initialize ACPI_CPU_DATA fields
//
AcpiCpuData->StackSize = PcdGet32 (PcdCpuApStackSize);
AcpiCpuData->ApMachineCheckHandlerBase = 0;
AcpiCpuData->ApMachineCheckHandlerSize = 0;
AcpiCpuData->GdtrProfile = (EFI_PHYSICAL_ADDRESS)(UINTN)&AcpiCpuDataEx->GdtrProfile;
AcpiCpuData->IdtrProfile = (EFI_PHYSICAL_ADDRESS)(UINTN)&AcpiCpuDataEx->IdtrProfile;
AcpiCpuData->MtrrTable = (EFI_PHYSICAL_ADDRESS)(UINTN)&AcpiCpuDataEx->MtrrTable;
//
// Allocate stack space for all CPUs.
// Use ACPI NVS memory type because this data will be directly used by APs
// in S3 resume phase in long mode. Also during S3 resume, the stack buffer
// will only be used as scratch space. i.e. we won't read anything from it
// before we write to it, in PiSmmCpuDxeSmm.
//
Stack = AllocateAcpiNvsMemory (NumberOfCpus * AcpiCpuData->StackSize);
ASSERT (Stack != NULL);
AcpiCpuData->StackAddress = (EFI_PHYSICAL_ADDRESS)(UINTN)Stack;
@@ -244,58 +260,83 @@ CpuS3DataInitialize (
IdtSize = AcpiCpuDataEx->IdtrProfile.Limit + 1;
Gdt = AllocateZeroPages (GdtSize + IdtSize);
ASSERT (Gdt != NULL);
Idt = (VOID *)((UINTN)Gdt + GdtSize);
CopyMem (Gdt, (VOID *)AcpiCpuDataEx->GdtrProfile.Base, GdtSize);
CopyMem (Idt, (VOID *)AcpiCpuDataEx->IdtrProfile.Base, IdtSize);
AcpiCpuDataEx->GdtrProfile.Base = (UINTN)Gdt;
AcpiCpuDataEx->IdtrProfile.Base = (UINTN)Idt;
if (OldAcpiCpuData != NULL) {
AcpiCpuData->RegisterTable = OldAcpiCpuData->RegisterTable;
AcpiCpuData->PreSmmInitRegisterTable = OldAcpiCpuData->PreSmmInitRegisterTable;
AcpiCpuData->ApLocation = OldAcpiCpuData->ApLocation;
CopyMem (&AcpiCpuData->CpuStatus, &OldAcpiCpuData->CpuStatus, sizeof (CPU_STATUS_INFORMATION));
} else {
//
// Allocate buffer for empty RegisterTable and PreSmmInitRegisterTable for all CPUs
//
TableSize = 2 * NumberOfCpus * sizeof (CPU_REGISTER_TABLE);
RegisterTable = (CPU_REGISTER_TABLE *)AllocateZeroPages (TableSize);
ASSERT (RegisterTable != NULL);
+ if (FetchPossibleApicIds) {
+ //
+ // Write a valid selector so that other hotplug registers can be
+ // accessed.
+ //
+ IoWrite32 (ICH9_CPU_HOTPLUG_BASE + QEMU_CPUHP_W_CPU_SEL, 0);
+ //
+ // We'll be fetching the APIC IDs.
+ //
+ IoWrite8 (ICH9_CPU_HOTPLUG_BASE + QEMU_CPUHP_W_CMD,
+ QEMU_CPUHP_CMD_GET_ARCH_ID);
+ }
for (Index = 0; Index < NumberOfCpus; Index++) {
- Status = MpServices->GetProcessorInfo (
- MpServices,
- Index,
- &ProcessorInfoBuffer
- );
- ASSERT_EFI_ERROR (Status);
+ UINT32 InitialApicId;
- RegisterTable[Index].InitialApicId = (UINT32)ProcessorInfoBuffer.ProcessorId;
+ if (FetchPossibleApicIds) {
+ IoWrite32 (ICH9_CPU_HOTPLUG_BASE + QEMU_CPUHP_W_CPU_SEL,
+ (UINT32)Index);
+ InitialApicId = IoRead32 (
+ ICH9_CPU_HOTPLUG_BASE + QEMU_CPUHP_RW_CMD_DATA);
+ } else {
+ Status = MpServices->GetProcessorInfo (
+ MpServices,
+ Index,
+ &ProcessorInfoBuffer
+ );
+ ASSERT_EFI_ERROR (Status);
+ InitialApicId = (UINT32)ProcessorInfoBuffer.ProcessorId;
+ }
+
+ DEBUG ((DEBUG_VERBOSE, "%a: Index=%05Lu ApicId=0x%08x\n", __FUNCTION__,
+ (UINT64)Index, InitialApicId));
+
+ RegisterTable[Index].InitialApicId = InitialApicId;
RegisterTable[Index].TableLength = 0;
RegisterTable[Index].AllocatedSize = 0;
RegisterTable[Index].RegisterTableEntry = 0;
- RegisterTable[NumberOfCpus + Index].InitialApicId = (UINT32)ProcessorInfoBuffer.ProcessorId;
+ RegisterTable[NumberOfCpus + Index].InitialApicId = InitialApicId;
RegisterTable[NumberOfCpus + Index].TableLength = 0;
RegisterTable[NumberOfCpus + Index].AllocatedSize = 0;
RegisterTable[NumberOfCpus + Index].RegisterTableEntry = 0;
}
AcpiCpuData->RegisterTable = (EFI_PHYSICAL_ADDRESS)(UINTN)RegisterTable;
AcpiCpuData->PreSmmInitRegisterTable = (EFI_PHYSICAL_ADDRESS)(UINTN)(RegisterTable + NumberOfCpus);
}
//
// Set PcdCpuS3DataAddress to the base address of the ACPI_CPU_DATA structure
//
Status = PcdSet64S (PcdCpuS3DataAddress, (UINT64)(UINTN)AcpiCpuData);
ASSERT_EFI_ERROR (Status);
//
// Register EFI_END_OF_DXE_EVENT_GROUP_GUID event.
// The notification function allocates StartupVector and saves MTRRs for ACPI_CPU_DATA
//
Status = gBS->CreateEventEx (
EVT_NOTIFY_SIGNAL,
TPL_CALLBACK,
CpuS3DataOnEndOfDxe,
--
2.19.1.3.g30247aa5d201
^ permalink raw reply related [flat|nested] 24+ messages in thread* Re: [PATCH 00/16] OvmfPkg: support VCPU hotplug with -D SMM_REQUIRE
2020-02-23 17:25 [PATCH 00/16] OvmfPkg: support VCPU hotplug with -D SMM_REQUIRE Laszlo Ersek
` (15 preceding siblings ...)
2020-02-23 17:25 ` [PATCH 16/16] OvmfPkg/CpuS3DataDxe: enable S3 resume after CPU hotplug Laszlo Ersek
@ 2020-02-24 16:31 ` Ard Biesheuvel
2020-07-24 6:26 ` [edk2-devel] " Wu, Jiaxin
17 siblings, 0 replies; 24+ messages in thread
From: Ard Biesheuvel @ 2020-02-24 16:31 UTC (permalink / raw)
To: Laszlo Ersek
Cc: edk2-devel-groups-io, Eric Dong, Hao A Wu, Igor Mammedov,
Jian J Wang, Jiewen Yao, Jordan Justen, Michael Kinney,
Philippe Mathieu-Daudé, Ray Ni
On Sun, 23 Feb 2020 at 18:25, Laszlo Ersek <lersek@redhat.com> wrote:
>
> Bugzilla: https://bugzilla.tianocore.org/show_bug.cgi?id=1512
> Repo: https://github.com/lersek/edk2.git
> Branch: vcpu_hotplug_smm_bz_1512
>
> This series implements VCPU hotplug with SMM for OVMF, i.e., when OVMF
> is built with "-D SMM_REQUIRE".
>
> SEV support and hot-unplug support are out of scope for now.
>
> Patch#13 ("OvmfPkg/CpuHotplugSmm: complete root MMI handler for CPU
> hotplug") describes tests and results in the Notes section.
>
> Obviously this is not being proposed for the edk2-stable202002 tag
> (which is in hard feature freeze).
>
> QEMU needs patches for this feature, too. I've done the development
> against a QEMU patch that Igor hacked up quickly at my request. It was
> never posted (it needs some polish for upstreaming), but it has allowed
> me to write and test this feature.
>
> The key parts of the QEMU commit message are:
>
> > x68:acpi: trigger SMI before scanning for added/removed CPUs
> >
> > Firmware should only scan for new CPUs and not modify events in CPU
> > hotplug registers.
>
> Igor's patch is based on upstream QEMU commit 418fa86dd465. Until he
> decides to post or otherwise share the patch, its effect can be
> expressed with a diff, taken in the Linux guest, between decompiled
> before/after versions of the QEMU-generated DSDT:
>
> > @@ -81,6 +81,27 @@
> > Return (Arg3)
> > }
> > }
> > +
> > + Device (SMI0)
> > + {
> > + Name (_HID, "PNP0A06" /* Generic Container Device */) // _HID: Hardware ID
> > + Name (_UID, "SMI resources") // _UID: Unique ID
> > + Name (_STA, 0x0B) // _STA: Status
> > + Name (_CRS, ResourceTemplate () // _CRS: Current Resource Settings
> > + {
> > + IO (Decode16,
> > + 0x00B2, // Range Minimum
> > + 0x00B2, // Range Maximum
> > + 0x01, // Alignment
> > + 0x01, // Length
> > + )
> > + })
> > + OperationRegion (SMIR, SystemIO, 0xB2, One)
> > + Field (SMIR, ByteAcc, NoLock, WriteAsZeros)
> > + {
> > + SMIC, 8
> > + }
> > + }
> > }
> >
> > Scope (_SB)
> > @@ -3016,6 +3037,7 @@
> > Method (CSCN, 0, Serialized)
> > {
> > Acquire (\_SB.PCI0.PRES.CPLK, 0xFFFF)
> > + \_SB.SMI0.SMIC = 0x04
> > Local0 = One
> > While ((Local0 == One))
> > {
>
> where the CSCN ("CPU scan") method is the _E02 GPE ("CPU hotplug") event
> handler:
>
> > Method (\_GPE._E02, 0, NotSerialized) // _Exx: Edge-Triggered GPE, xx=0x00-0xFF
> > {
> > \_SB.CPUS.CSCN ()
> > }
>
> If you'd like to test this series, please ask Igor for the QEMU patch.
> :)
>
> The series has been formatted for review with the following options:
>
> --stat=1000 --stat-graph-width=20 \
> --unified=22 \
> --find-copies=43 --find-copies-harder \
> --base=master \
>
> At every stage in the series:
> - the tree builds,
> - "PatchCheck.py" is happy,
> - and OVMF works without regressions.
>
> (Hotplug is made functional at patch#13, and "S3 after hotplug" is
> completed at patch#16. So those actions should not be attempted before
> said respective patches.)
>
I skimmed these patches, and it all looks reasonable to me, but I am
by no means knowledgeable on x86 SMM internals.
So provided the @intel.com folks on cc are happy with these changes,
and ack the series,
Acked-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
>
> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> Cc: Eric Dong <eric.dong@intel.com>
> Cc: Hao A Wu <hao.a.wu@intel.com>
> Cc: Igor Mammedov <imammedo@redhat.com>
> Cc: Jian J Wang <jian.j.wang@intel.com>
> Cc: Jiewen Yao <jiewen.yao@intel.com>
> Cc: Jordan Justen <jordan.l.justen@intel.com>
> Cc: Michael Kinney <michael.d.kinney@intel.com>
> Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
> Cc: Ray Ni <ray.ni@intel.com>
>
> Thanks
> Laszlo
>
> Laszlo Ersek (16):
> MdeModulePkg/PiSmmCore: log SMM image start failure
> UefiCpuPkg/PiSmmCpuDxeSmm: fix S3 Resume for CPU hotplug
> OvmfPkg: clone SmmCpuPlatformHookLib from UefiCpuPkg
> OvmfPkg: enable SMM Monarch Election in PiSmmCpuDxeSmm
> OvmfPkg: enable CPU hotplug support in PiSmmCpuDxeSmm
> OvmfPkg/CpuHotplugSmm: introduce skeleton for CPU Hotplug SMM driver
> OvmfPkg/CpuHotplugSmm: add hotplug register block helper functions
> OvmfPkg/CpuHotplugSmm: define the QEMU_CPUHP_CMD_GET_ARCH_ID macro
> OvmfPkg/CpuHotplugSmm: add function for collecting CPUs with events
> OvmfPkg/CpuHotplugSmm: collect CPUs with events
> OvmfPkg/CpuHotplugSmm: introduce Post-SMM Pen for hot-added CPUs
> OvmfPkg/CpuHotplugSmm: introduce First SMI Handler for hot-added CPUs
> OvmfPkg/CpuHotplugSmm: complete root MMI handler for CPU hotplug
> OvmfPkg: clone CpuS3DataDxe from UefiCpuPkg
> OvmfPkg/CpuS3DataDxe: superficial cleanups
> OvmfPkg/CpuS3DataDxe: enable S3 resume after CPU hotplug
>
> MdeModulePkg/Core/PiSmmCore/Dispatcher.c | 6 +
> OvmfPkg/CpuHotplugSmm/ApicId.h | 23 ++
> OvmfPkg/CpuHotplugSmm/CpuHotplug.c | 426 ++++++++++++++++++++
> OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf | 64 +++
> OvmfPkg/CpuHotplugSmm/FirstSmiHandler.nasm | 149 +++++++
> OvmfPkg/CpuHotplugSmm/FirstSmiHandlerContext.h | 41 ++
> OvmfPkg/CpuHotplugSmm/PostSmmPen.nasm | 137 +++++++
> OvmfPkg/CpuHotplugSmm/QemuCpuhp.c | 301 ++++++++++++++
> OvmfPkg/CpuHotplugSmm/QemuCpuhp.h | 61 +++
> OvmfPkg/CpuHotplugSmm/Smbase.c | 252 ++++++++++++
> OvmfPkg/CpuHotplugSmm/Smbase.h | 46 +++
> OvmfPkg/Include/IndustryStandard/Q35MchIch9.h | 5 +-
> OvmfPkg/Include/IndustryStandard/QemuCpuHotplug.h | 3 +
> OvmfPkg/OvmfPkgIa32.dsc | 7 +-
> OvmfPkg/OvmfPkgIa32.fdf | 3 +-
> OvmfPkg/OvmfPkgIa32X64.dsc | 7 +-
> OvmfPkg/OvmfPkgIa32X64.fdf | 3 +-
> OvmfPkg/OvmfPkgX64.dsc | 7 +-
> OvmfPkg/OvmfPkgX64.fdf | 3 +-
> UefiCpuPkg/Library/SmmCpuPlatformHookLibNull/SmmCpuPlatformHookLibNull.c => OvmfPkg/Library/SmmCpuPlatformHookLibQemu/SmmCpuPlatformHookLibQemu.c | 45 ++-
> UefiCpuPkg/Library/SmmCpuPlatformHookLibNull/SmmCpuPlatformHookLibNull.inf => OvmfPkg/Library/SmmCpuPlatformHookLibQemu/SmmCpuPlatformHookLibQemu.inf | 24 +-
> UefiCpuPkg/PiSmmCpuDxeSmm/CpuS3.c | 14 +-
> {UefiCpuPkg => OvmfPkg}/CpuS3DataDxe/CpuS3Data.c | 99 +++--
> {UefiCpuPkg => OvmfPkg}/CpuS3DataDxe/CpuS3DataDxe.inf | 30 +-
> 24 files changed, 1667 insertions(+), 89 deletions(-)
> copy UefiCpuPkg/Library/SmmCpuPlatformHookLibNull/SmmCpuPlatformHookLibNull.c => OvmfPkg/Library/SmmCpuPlatformHookLibQemu/SmmCpuPlatformHookLibQemu.c (61%)
> copy UefiCpuPkg/Library/SmmCpuPlatformHookLibNull/SmmCpuPlatformHookLibNull.inf => OvmfPkg/Library/SmmCpuPlatformHookLibQemu/SmmCpuPlatformHookLibQemu.inf (43%)
> copy {UefiCpuPkg => OvmfPkg}/CpuS3DataDxe/CpuS3Data.c (77%)
> copy {UefiCpuPkg => OvmfPkg}/CpuS3DataDxe/CpuS3DataDxe.inf (69%)
> create mode 100644 OvmfPkg/CpuHotplugSmm/ApicId.h
> create mode 100644 OvmfPkg/CpuHotplugSmm/CpuHotplug.c
> create mode 100644 OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf
> create mode 100644 OvmfPkg/CpuHotplugSmm/FirstSmiHandler.nasm
> create mode 100644 OvmfPkg/CpuHotplugSmm/FirstSmiHandlerContext.h
> create mode 100644 OvmfPkg/CpuHotplugSmm/PostSmmPen.nasm
> create mode 100644 OvmfPkg/CpuHotplugSmm/QemuCpuhp.c
> create mode 100644 OvmfPkg/CpuHotplugSmm/QemuCpuhp.h
> create mode 100644 OvmfPkg/CpuHotplugSmm/Smbase.c
> create mode 100644 OvmfPkg/CpuHotplugSmm/Smbase.h
>
>
> base-commit: 1d3215fd24f47eaa4877542a59b4bbf5afc0cfe8
> --
> 2.19.1.3.g30247aa5d201
>
^ permalink raw reply [flat|nested] 24+ messages in thread* Re: [edk2-devel] [PATCH 00/16] OvmfPkg: support VCPU hotplug with -D SMM_REQUIRE
2020-02-23 17:25 [PATCH 00/16] OvmfPkg: support VCPU hotplug with -D SMM_REQUIRE Laszlo Ersek
` (16 preceding siblings ...)
2020-02-24 16:31 ` [PATCH 00/16] OvmfPkg: support VCPU hotplug with -D SMM_REQUIRE Ard Biesheuvel
@ 2020-07-24 6:26 ` Wu, Jiaxin
2020-07-24 16:01 ` Laszlo Ersek
17 siblings, 1 reply; 24+ messages in thread
From: Wu, Jiaxin @ 2020-07-24 6:26 UTC (permalink / raw)
To: devel@edk2.groups.io, lersek@redhat.com
Hi Laszlo,
Looks OVMF supports the CPU hotplug with those series patches.
Could you provide some guide how to enable the OVMF CPU hotplug verification? Is there any general work flow introduction how it works? For example, how to do the hot add CPU initialization (e.g. Register setting / Microcode update, etc.)? I'm very interested in this feature on OVMF.
Thank you very much.
Jiaxin
> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Laszlo
> Ersek
> Sent: Monday, February 24, 2020 1:25 AM
> To: edk2-devel-groups-io <devel@edk2.groups.io>
> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>; Dong, Eric
> <eric.dong@intel.com>; Wu, Hao A <hao.a.wu@intel.com>; Igor Mammedov
> <imammedo@redhat.com>; Wang, Jian J <jian.j.wang@intel.com>; Yao,
> Jiewen <jiewen.yao@intel.com>; Justen, Jordan L
> <jordan.l.justen@intel.com>; Kinney, Michael D
> <michael.d.kinney@intel.com>; Philippe Mathieu-Daudé
> <philmd@redhat.com>; Ni, Ray <ray.ni@intel.com>
> Subject: [edk2-devel] [PATCH 00/16] OvmfPkg: support VCPU hotplug with -
> D SMM_REQUIRE
>
> Bugzilla: https://bugzilla.tianocore.org/show_bug.cgi?id=1512
> Repo: https://github.com/lersek/edk2.git
> Branch: vcpu_hotplug_smm_bz_1512
>
> This series implements VCPU hotplug with SMM for OVMF, i.e., when OVMF
> is built with "-D SMM_REQUIRE".
>
> SEV support and hot-unplug support are out of scope for now.
>
> Patch#13 ("OvmfPkg/CpuHotplugSmm: complete root MMI handler for CPU
> hotplug") describes tests and results in the Notes section.
>
> Obviously this is not being proposed for the edk2-stable202002 tag
> (which is in hard feature freeze).
>
> QEMU needs patches for this feature, too. I've done the development
> against a QEMU patch that Igor hacked up quickly at my request. It was
> never posted (it needs some polish for upstreaming), but it has allowed
> me to write and test this feature.
>
> The key parts of the QEMU commit message are:
>
> > x68:acpi: trigger SMI before scanning for added/removed CPUs
> >
> > Firmware should only scan for new CPUs and not modify events in CPU
> > hotplug registers.
>
> Igor's patch is based on upstream QEMU commit 418fa86dd465. Until he
> decides to post or otherwise share the patch, its effect can be
> expressed with a diff, taken in the Linux guest, between decompiled
> before/after versions of the QEMU-generated DSDT:
>
> > @@ -81,6 +81,27 @@
> > Return (Arg3)
> > }
> > }
> > +
> > + Device (SMI0)
> > + {
> > + Name (_HID, "PNP0A06" /* Generic Container Device */) // _HID:
> Hardware ID
> > + Name (_UID, "SMI resources") // _UID: Unique ID
> > + Name (_STA, 0x0B) // _STA: Status
> > + Name (_CRS, ResourceTemplate () // _CRS: Current Resource
> Settings
> > + {
> > + IO (Decode16,
> > + 0x00B2, // Range Minimum
> > + 0x00B2, // Range Maximum
> > + 0x01, // Alignment
> > + 0x01, // Length
> > + )
> > + })
> > + OperationRegion (SMIR, SystemIO, 0xB2, One)
> > + Field (SMIR, ByteAcc, NoLock, WriteAsZeros)
> > + {
> > + SMIC, 8
> > + }
> > + }
> > }
> >
> > Scope (_SB)
> > @@ -3016,6 +3037,7 @@
> > Method (CSCN, 0, Serialized)
> > {
> > Acquire (\_SB.PCI0.PRES.CPLK, 0xFFFF)
> > + \_SB.SMI0.SMIC = 0x04
> > Local0 = One
> > While ((Local0 == One))
> > {
>
> where the CSCN ("CPU scan") method is the _E02 GPE ("CPU hotplug") event
> handler:
>
> > Method (\_GPE._E02, 0, NotSerialized) // _Exx: Edge-Triggered GPE,
> xx=0x00-0xFF
> > {
> > \_SB.CPUS.CSCN ()
> > }
>
> If you'd like to test this series, please ask Igor for the QEMU patch.
> :)
>
> The series has been formatted for review with the following options:
>
> --stat=1000 --stat-graph-width=20 \
> --unified=22 \
> --find-copies=43 --find-copies-harder \
> --base=master \
>
> At every stage in the series:
> - the tree builds,
> - "PatchCheck.py" is happy,
> - and OVMF works without regressions.
>
> (Hotplug is made functional at patch#13, and "S3 after hotplug" is
> completed at patch#16. So those actions should not be attempted before
> said respective patches.)
>
> Thanks,
> Laszlo
>
> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> Cc: Eric Dong <eric.dong@intel.com>
> Cc: Hao A Wu <hao.a.wu@intel.com>
> Cc: Igor Mammedov <imammedo@redhat.com>
> Cc: Jian J Wang <jian.j.wang@intel.com>
> Cc: Jiewen Yao <jiewen.yao@intel.com>
> Cc: Jordan Justen <jordan.l.justen@intel.com>
> Cc: Michael Kinney <michael.d.kinney@intel.com>
> Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
> Cc: Ray Ni <ray.ni@intel.com>
>
> Thanks
> Laszlo
>
> Laszlo Ersek (16):
> MdeModulePkg/PiSmmCore: log SMM image start failure
> UefiCpuPkg/PiSmmCpuDxeSmm: fix S3 Resume for CPU hotplug
> OvmfPkg: clone SmmCpuPlatformHookLib from UefiCpuPkg
> OvmfPkg: enable SMM Monarch Election in PiSmmCpuDxeSmm
> OvmfPkg: enable CPU hotplug support in PiSmmCpuDxeSmm
> OvmfPkg/CpuHotplugSmm: introduce skeleton for CPU Hotplug SMM driver
> OvmfPkg/CpuHotplugSmm: add hotplug register block helper functions
> OvmfPkg/CpuHotplugSmm: define the QEMU_CPUHP_CMD_GET_ARCH_ID
> macro
> OvmfPkg/CpuHotplugSmm: add function for collecting CPUs with events
> OvmfPkg/CpuHotplugSmm: collect CPUs with events
> OvmfPkg/CpuHotplugSmm: introduce Post-SMM Pen for hot-added CPUs
> OvmfPkg/CpuHotplugSmm: introduce First SMI Handler for hot-added CPUs
> OvmfPkg/CpuHotplugSmm: complete root MMI handler for CPU hotplug
> OvmfPkg: clone CpuS3DataDxe from UefiCpuPkg
> OvmfPkg/CpuS3DataDxe: superficial cleanups
> OvmfPkg/CpuS3DataDxe: enable S3 resume after CPU hotplug
>
> MdeModulePkg/Core/PiSmmCore/Dispatcher.c
> | 6 +
> OvmfPkg/CpuHotplugSmm/ApicId.h
> | 23 ++
> OvmfPkg/CpuHotplugSmm/CpuHotplug.c
> | 426 ++++++++++++++++++++
> OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf
> | 64 +++
> OvmfPkg/CpuHotplugSmm/FirstSmiHandler.nasm
> | 149 +++++++
> OvmfPkg/CpuHotplugSmm/FirstSmiHandlerContext.h
> | 41 ++
> OvmfPkg/CpuHotplugSmm/PostSmmPen.nasm
> | 137 +++++++
> OvmfPkg/CpuHotplugSmm/QemuCpuhp.c
> | 301 ++++++++++++++
> OvmfPkg/CpuHotplugSmm/QemuCpuhp.h
> | 61 +++
> OvmfPkg/CpuHotplugSmm/Smbase.c
> | 252 ++++++++++++
> OvmfPkg/CpuHotplugSmm/Smbase.h
> | 46 +++
> OvmfPkg/Include/IndustryStandard/Q35MchIch9.h
> | 5 +-
> OvmfPkg/Include/IndustryStandard/QemuCpuHotplug.h
> | 3 +
> OvmfPkg/OvmfPkgIa32.dsc
> | 7 +-
> OvmfPkg/OvmfPkgIa32.fdf
> | 3 +-
> OvmfPkg/OvmfPkgIa32X64.dsc
> | 7 +-
> OvmfPkg/OvmfPkgIa32X64.fdf
> | 3 +-
> OvmfPkg/OvmfPkgX64.dsc
> | 7 +-
> OvmfPkg/OvmfPkgX64.fdf
> | 3 +-
>
> UefiCpuPkg/Library/SmmCpuPlatformHookLibNull/SmmCpuPlatformHookLib
> Null.c =>
> OvmfPkg/Library/SmmCpuPlatformHookLibQemu/SmmCpuPlatformHookLib
> Qemu.c | 45 ++-
>
> UefiCpuPkg/Library/SmmCpuPlatformHookLibNull/SmmCpuPlatformHookLib
> Null.inf =>
> OvmfPkg/Library/SmmCpuPlatformHookLibQemu/SmmCpuPlatformHookLib
> Qemu.inf | 24 +-
> UefiCpuPkg/PiSmmCpuDxeSmm/CpuS3.c
> | 14 +-
> {UefiCpuPkg => OvmfPkg}/CpuS3DataDxe/CpuS3Data.c
> | 99 +++--
> {UefiCpuPkg => OvmfPkg}/CpuS3DataDxe/CpuS3DataDxe.inf
> | 30 +-
> 24 files changed, 1667 insertions(+), 89 deletions(-)
> copy
> UefiCpuPkg/Library/SmmCpuPlatformHookLibNull/SmmCpuPlatformHookLib
> Null.c =>
> OvmfPkg/Library/SmmCpuPlatformHookLibQemu/SmmCpuPlatformHookLib
> Qemu.c (61%)
> copy
> UefiCpuPkg/Library/SmmCpuPlatformHookLibNull/SmmCpuPlatformHookLib
> Null.inf =>
> OvmfPkg/Library/SmmCpuPlatformHookLibQemu/SmmCpuPlatformHookLib
> Qemu.inf (43%)
> copy {UefiCpuPkg => OvmfPkg}/CpuS3DataDxe/CpuS3Data.c (77%)
> copy {UefiCpuPkg => OvmfPkg}/CpuS3DataDxe/CpuS3DataDxe.inf (69%)
> create mode 100644 OvmfPkg/CpuHotplugSmm/ApicId.h
> create mode 100644 OvmfPkg/CpuHotplugSmm/CpuHotplug.c
> create mode 100644 OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf
> create mode 100644 OvmfPkg/CpuHotplugSmm/FirstSmiHandler.nasm
> create mode 100644 OvmfPkg/CpuHotplugSmm/FirstSmiHandlerContext.h
> create mode 100644 OvmfPkg/CpuHotplugSmm/PostSmmPen.nasm
> create mode 100644 OvmfPkg/CpuHotplugSmm/QemuCpuhp.c
> create mode 100644 OvmfPkg/CpuHotplugSmm/QemuCpuhp.h
> create mode 100644 OvmfPkg/CpuHotplugSmm/Smbase.c
> create mode 100644 OvmfPkg/CpuHotplugSmm/Smbase.h
>
>
> base-commit: 1d3215fd24f47eaa4877542a59b4bbf5afc0cfe8
> --
> 2.19.1.3.g30247aa5d201
>
>
>
^ permalink raw reply [flat|nested] 24+ messages in thread* Re: [edk2-devel] [PATCH 00/16] OvmfPkg: support VCPU hotplug with -D SMM_REQUIRE
2020-07-24 6:26 ` [edk2-devel] " Wu, Jiaxin
@ 2020-07-24 16:01 ` Laszlo Ersek
2020-07-29 8:37 ` Wu, Jiaxin
0 siblings, 1 reply; 24+ messages in thread
From: Laszlo Ersek @ 2020-07-24 16:01 UTC (permalink / raw)
To: Wu, Jiaxin; +Cc: devel@edk2.groups.io
On 07/24/20 08:26, Wu, Jiaxin wrote:
> Hi Laszlo,
>
> Looks OVMF supports the CPU hotplug with those series patches.
>
> Could you provide some guide how to enable the OVMF CPU hotplug
> verification? Is there any general work flow introduction how it
> works? For example, how to do the hot add CPU initialization (e.g.
> Register setting / Microcode update, etc.)? I'm very interested in
> this feature on OVMF.
Long version:
-------------
(1) There are three pieces missing:
(1a) The QEMU side changes for the ACPI (DSDT) content that QEMU
generates for the OS.
The ACPI GPE handler for CPU hotplug is being modified by my colleague
Igor Mammedov to raise the SMI (command value 4) on CPU hotplug.
For developing the OVMF series for TianoCore#1512 (which is now merged),
I used a prototype QEMU patch, from Igor. But that patch is not suitable
for upstreaming to QEMU. SO Igor is now developing the real patches for
QEMU's ACPI generator.
(1b) The related feature negotiation patches in QEMU.
In order for "CPU hotplug with SMM" to work, both OVMF and QEMU need to
perform specific things. In order to deal with cross-version
compatibility problems, the "CPU hotplug with SMI" feature is
dynamically negotiated between OVMF and QEMU. For this negotiation, both
QEMU and OVMF need additional patches. These patches are not related to
the actual plugging activities; instead they control whether plugging is
permitted at all, or not.
Igor's QEMU series covers both purposes (1a) and (1b). It's work in
progress. The first posting was an RFC series:
(1b1) [RFC 0/3] x86: fix cpu hotplug with secure boot
https://lists.gnu.org/archive/html/qemu-devel/2020-07/msg03746.html
http://mid.mail-archive.com/20200710161704.309824-1-imammedo@redhat.com
The latest posting has been a PATCH series:
(1b2) [qemu-devel] [PATCH 0/6] x86: fix cpu hotplug with secure boot
https://lists.gnu.org/archive/html/qemu-devel/2020-07/msg05850.html
http://mid.mail-archive.com/20200720141610.574308-1-imammedo@redhat.com
(1c) The feature negotiation patch for OVMF is here:
* [edk2-devel] [PATCH] OvmfPkg/SmmControl2Dxe: negotiate ICH9_LPC_SMI_F_CPU_HOTPLUG
https://edk2.groups.io/g/devel/message/62561
http://mid.mail-archive.com/20200714184305.9814-1-lersek@redhat.com
(2) Special register setting and microcode stuff are not needed.
(3) As I mentioned before, I strongly suggest using QEMU and OVMF with
libvirt. I had written an article about that here:
https://github.com/tianocore/tianocore.github.io/wiki/Testing-SMM-with-QEMU,-KVM-and-libvirt
I wrote this article specifically for "Windows-based" developers. The
article is written from such a perspective that you don't need a
personal Linux workstation, only a single Linux workstation *per team*.
So you can continue using a Windows workstation, just set up one Linux
box for your team (if you don't yet have one).
This article remains relevant.
(3a) In order to set up a guest for VCPU hotplug, simply go through the
article, initially.
(3b) Once you're done with that, power down the guest, and modify the
domain XML as follows:
virsh edit <DOMAIN_NAME>
(3b1) replace the "pc-q35-2.9" machine type with "pc-q35-5.1"
(3b2) replace the following stanza:
<vcpu placement='static'>4</vcpu>
with:
<vcpu placement='static' current='2'>4</vcpu>
<vcpus>
<vcpu id='0' enabled='yes' hotpluggable='no'/>
<vcpu id='1' enabled='no' hotpluggable='yes'/>
<vcpu id='2' enabled='yes' hotpluggable='yes'/>
<vcpu id='3' enabled='no' hotpluggable='yes'/>
</vcpus>
This will create a VCPU topology where:
- CPU#0 is present up-front, and is not hot-pluggable (this is a QEMU
requirement),
- CPU#1, CPU#2, and CPU#3 are hot-pluggable,
- CPU#2 is present up-front ("cold-plugged"), while CPU#1 and CPU#3 are
absent initially.
(4) Boot the guest. Once you have a root prompt in the guest, you can
use one of two libvirt commands for hot-plugging a CPU:
(4a) the singular "virsh setvcpu" command:
virsh setvcpu <DOMAIN_NAME> <PROCESSOR_ID> --enable --live
where you can pass in 1 or 3 for <PROCESSOR_ID>.
This command lets you specify the precise ID of the processor to be
hot-plugged; IOW, the command lets you control topology.
(4b) the plural "virsh setvcpus" command:
virsh setvcpus <GUEST_NAME> <PROCESSOR_COUNT> --live
This command lets you specify the desired number of total active CPUs.
It does not let you control topology. (My understanding is that it keeps
the topology populated at the "front".)
Regarding the current QEMU status, we need to do more work for
supporting (4b). The RFC series (1b1) enables (4a) to work. The PATCH
series (1b2) intended to make (4b) work, but unfortunately it broke even
(4a). So now we need at least one more version of the QEMU series (I've
given my feedback to Igor already, on qemu-devel).
(4c) Dependent on guest OS configuration, you might have to manually
online the newly plugged CPUs in the guest:
echo 1 > /sys/devices/system/cpu/cpu2/online
echo 1 > /sys/devices/system/cpu/cpu3/online
Note that the "cpuN" identifiers seen here are *neither* APIC IDs *nor*
the same IDs as seen in the libvirt domain XML. Instead, these IDs are
assigned in the order the Linux kernel learns about the CPUs (if I
understand correctly).
Short version:
--------------
- apply (1b1) on top of latest QEMU master from git, and build and
install it,
- apply (1c) on latest edk2, and build OVMF with "-D SMM_REQUIRE",
- install a Linux guest on a Linux host (using KVM!) as described in my
Wiki article (3),
- modify the domain XML for the guest as described in (3b),
- use the singular "virsh setvcpu" command (4a) for hot-plugging VCPU#1
and/or VCPU#3,
- if necessary, use (4c) in the guest.
You can do the same with Windows Server guests too, although I'm not
exactly sure what versions support CPU hotplug. For testing I've used
Windows Server 2012 R2. The Wiki article at (3) has a section dedicated
to installing Windows guests too.
Thanks,
Laszlo
^ permalink raw reply [flat|nested] 24+ messages in thread* Re: [edk2-devel] [PATCH 00/16] OvmfPkg: support VCPU hotplug with -D SMM_REQUIRE
2020-07-24 16:01 ` Laszlo Ersek
@ 2020-07-29 8:37 ` Wu, Jiaxin
2020-10-01 9:59 ` [ann] " Laszlo Ersek
0 siblings, 1 reply; 24+ messages in thread
From: Wu, Jiaxin @ 2020-07-29 8:37 UTC (permalink / raw)
To: Laszlo Ersek; +Cc: devel@edk2.groups.io
Hi Laszlo,
Appreciate your feedback! Thank you very much.
Jiaxin
> -----Original Message-----
> From: Laszlo Ersek <lersek@redhat.com>
> Sent: Saturday, July 25, 2020 12:02 AM
> To: Wu, Jiaxin <jiaxin.wu@intel.com>
> Cc: devel@edk2.groups.io
> Subject: Re: [edk2-devel] [PATCH 00/16] OvmfPkg: support VCPU hotplug
> with -D SMM_REQUIRE
>
> On 07/24/20 08:26, Wu, Jiaxin wrote:
> > Hi Laszlo,
> >
> > Looks OVMF supports the CPU hotplug with those series patches.
> >
> > Could you provide some guide how to enable the OVMF CPU hotplug
> > verification? Is there any general work flow introduction how it
> > works? For example, how to do the hot add CPU initialization (e.g.
> > Register setting / Microcode update, etc.)? I'm very interested in
> > this feature on OVMF.
>
> Long version:
> -------------
>
> (1) There are three pieces missing:
>
> (1a) The QEMU side changes for the ACPI (DSDT) content that QEMU
> generates for the OS.
>
> The ACPI GPE handler for CPU hotplug is being modified by my colleague
> Igor Mammedov to raise the SMI (command value 4) on CPU hotplug.
>
> For developing the OVMF series for TianoCore#1512 (which is now merged),
> I used a prototype QEMU patch, from Igor. But that patch is not suitable
> for upstreaming to QEMU. SO Igor is now developing the real patches for
> QEMU's ACPI generator.
>
> (1b) The related feature negotiation patches in QEMU.
>
> In order for "CPU hotplug with SMM" to work, both OVMF and QEMU need
> to
> perform specific things. In order to deal with cross-version
> compatibility problems, the "CPU hotplug with SMI" feature is
> dynamically negotiated between OVMF and QEMU. For this negotiation,
> both
> QEMU and OVMF need additional patches. These patches are not related to
> the actual plugging activities; instead they control whether plugging is
> permitted at all, or not.
>
> Igor's QEMU series covers both purposes (1a) and (1b). It's work in
> progress. The first posting was an RFC series:
>
> (1b1) [RFC 0/3] x86: fix cpu hotplug with secure boot
> https://lists.gnu.org/archive/html/qemu-devel/2020-07/msg03746.html
> http://mid.mail-archive.com/20200710161704.309824-1-
> imammedo@redhat.com
>
> The latest posting has been a PATCH series:
>
> (1b2) [qemu-devel] [PATCH 0/6] x86: fix cpu hotplug with secure boot
> https://lists.gnu.org/archive/html/qemu-devel/2020-07/msg05850.html
> http://mid.mail-archive.com/20200720141610.574308-1-
> imammedo@redhat.com
>
> (1c) The feature negotiation patch for OVMF is here:
>
> * [edk2-devel] [PATCH] OvmfPkg/SmmControl2Dxe: negotiate
> ICH9_LPC_SMI_F_CPU_HOTPLUG
> https://edk2.groups.io/g/devel/message/62561
> http://mid.mail-archive.com/20200714184305.9814-1-lersek@redhat.com
>
>
> (2) Special register setting and microcode stuff are not needed.
>
>
> (3) As I mentioned before, I strongly suggest using QEMU and OVMF with
> libvirt. I had written an article about that here:
>
> https://github.com/tianocore/tianocore.github.io/wiki/Testing-SMM-with-
> QEMU,-KVM-and-libvirt
>
> I wrote this article specifically for "Windows-based" developers. The
> article is written from such a perspective that you don't need a
> personal Linux workstation, only a single Linux workstation *per team*.
> So you can continue using a Windows workstation, just set up one Linux
> box for your team (if you don't yet have one).
>
> This article remains relevant.
>
> (3a) In order to set up a guest for VCPU hotplug, simply go through the
> article, initially.
>
> (3b) Once you're done with that, power down the guest, and modify the
> domain XML as follows:
>
> virsh edit <DOMAIN_NAME>
>
> (3b1) replace the "pc-q35-2.9" machine type with "pc-q35-5.1"
>
> (3b2) replace the following stanza:
>
> <vcpu placement='static'>4</vcpu>
>
> with:
>
> <vcpu placement='static' current='2'>4</vcpu>
> <vcpus>
> <vcpu id='0' enabled='yes' hotpluggable='no'/>
> <vcpu id='1' enabled='no' hotpluggable='yes'/>
> <vcpu id='2' enabled='yes' hotpluggable='yes'/>
> <vcpu id='3' enabled='no' hotpluggable='yes'/>
> </vcpus>
>
> This will create a VCPU topology where:
>
> - CPU#0 is present up-front, and is not hot-pluggable (this is a QEMU
> requirement),
>
> - CPU#1, CPU#2, and CPU#3 are hot-pluggable,
>
> - CPU#2 is present up-front ("cold-plugged"), while CPU#1 and CPU#3 are
> absent initially.
>
>
> (4) Boot the guest. Once you have a root prompt in the guest, you can
> use one of two libvirt commands for hot-plugging a CPU:
>
> (4a) the singular "virsh setvcpu" command:
>
> virsh setvcpu <DOMAIN_NAME> <PROCESSOR_ID> --enable --live
>
> where you can pass in 1 or 3 for <PROCESSOR_ID>.
>
> This command lets you specify the precise ID of the processor to be
> hot-plugged; IOW, the command lets you control topology.
>
> (4b) the plural "virsh setvcpus" command:
>
> virsh setvcpus <GUEST_NAME> <PROCESSOR_COUNT> --live
>
> This command lets you specify the desired number of total active CPUs.
> It does not let you control topology. (My understanding is that it keeps
> the topology populated at the "front".)
>
> Regarding the current QEMU status, we need to do more work for
> supporting (4b). The RFC series (1b1) enables (4a) to work. The PATCH
> series (1b2) intended to make (4b) work, but unfortunately it broke even
> (4a). So now we need at least one more version of the QEMU series (I've
> given my feedback to Igor already, on qemu-devel).
>
> (4c) Dependent on guest OS configuration, you might have to manually
> online the newly plugged CPUs in the guest:
>
> echo 1 > /sys/devices/system/cpu/cpu2/online
> echo 1 > /sys/devices/system/cpu/cpu3/online
>
> Note that the "cpuN" identifiers seen here are *neither* APIC IDs *nor*
> the same IDs as seen in the libvirt domain XML. Instead, these IDs are
> assigned in the order the Linux kernel learns about the CPUs (if I
> understand correctly).
>
>
> Short version:
> --------------
>
> - apply (1b1) on top of latest QEMU master from git, and build and
> install it,
>
> - apply (1c) on latest edk2, and build OVMF with "-D SMM_REQUIRE",
>
> - install a Linux guest on a Linux host (using KVM!) as described in my
> Wiki article (3),
>
> - modify the domain XML for the guest as described in (3b),
>
> - use the singular "virsh setvcpu" command (4a) for hot-plugging VCPU#1
> and/or VCPU#3,
>
> - if necessary, use (4c) in the guest.
>
>
> You can do the same with Windows Server guests too, although I'm not
> exactly sure what versions support CPU hotplug. For testing I've used
> Windows Server 2012 R2. The Wiki article at (3) has a section dedicated
> to installing Windows guests too.
>
> Thanks,
> Laszlo
^ permalink raw reply [flat|nested] 24+ messages in thread* [ann] OvmfPkg: support VCPU hotplug with -D SMM_REQUIRE
2020-07-29 8:37 ` Wu, Jiaxin
@ 2020-10-01 9:59 ` Laszlo Ersek
0 siblings, 0 replies; 24+ messages in thread
From: Laszlo Ersek @ 2020-10-01 9:59 UTC (permalink / raw)
To: devel, jiaxin.wu; +Cc: aaron.young, Boris Ostrovsky, Igor Mammedov
Hello Jiaxin,
On 07/29/20 10:37, Wu, Jiaxin wrote:
> Hi Laszlo,
>
> Appreciate your feedback! Thank you very much.
if you are still interested in using this feature, I have some good
news.
As of current edk2 master (to be entirely precise: as of edk2 commit
cbccf995920a), and as of upstream QEMU commit 6e2e2e8a4220, the "VCPU
hotplug with SMM" feature is considered complete.
Both the singular "virsh setvcpu" command (which gives you precise NUMA
affinity for the VCPU being hot-plugged), and the plural "virsh
setvcpus" command, are expected to work. (For plugging.)
I'm repeating a short summary here about the necessary configuration on
your end:
* install a Linux guest as described in my Wiki article:
https://github.com/tianocore/tianocore.github.io/wiki/Testing-SMM-with-QEMU,-KVM-and-libvirt
* modify the domain XML for the guest:,
- replace the "pc-q35-2.9" machine type with "pc-q35-5.2"
- replace the following stanza:
<vcpu placement='static'>4</vcpu>
with:
<vcpu placement='static' current='2'>4</vcpu>
<vcpus>
<vcpu id='0' enabled='yes' hotpluggable='no'/>
<vcpu id='1' enabled='no' hotpluggable='yes'/>
<vcpu id='2' enabled='yes' hotpluggable='yes'/>
<vcpu id='3' enabled='no' hotpluggable='yes'/>
</vcpus>
* With Linux guests, you may have to online the just-plugged VCPUs in
the guest as a separate step (dependent on your guest Linux
distribution). For this, you can either use (in the guest) commands
like
echo 1 > /sys/devices/system/cpu/cpu2/online
echo 1 > /sys/devices/system/cpu/cpu3/online
(note that the numbering is in order of the Linux guest learning of
the processor).
Alternatively, in case you used the plural "virsh setvcpus" command,
*and* you have the guest agent running in your guest, you can follow
up with
virsh setvcpus ... --guest
on the host side (the "--guest" option being the important one). This
will ask the QEMU guest agent to perform the same onlining as the
explicit "echo" commands above.
* Windows guests do not need separate onlining of the hot-added CPUs.
The oldest Windows release that (to my knowledge) supports CPU hotplug
is Windows Server 2008 R2 *Datacenter* edition.
Thanks
Laszlo
^ permalink raw reply [flat|nested] 24+ messages in thread