From: "Ard Biesheuvel" <ard.biesheuvel@linaro.org>
To: edk2-devel-groups-io <devel@edk2.groups.io>,
Laszlo Ersek <lersek@redhat.com>
Cc: "Igor Mammedov" <imammedo@redhat.com>,
"Jiewen Yao" <jiewen.yao@intel.com>,
"Jordan Justen" <jordan.l.justen@intel.com>,
"Michael Kinney" <michael.d.kinney@intel.com>,
"Philippe Mathieu-Daudé" <philmd@redhat.com>
Subject: Re: [edk2-devel] [PATCH v2 11/16] OvmfPkg/CpuHotplugSmm: introduce Post-SMM Pen for hot-added CPUs
Date: Mon, 2 Mar 2020 15:02:19 +0100 [thread overview]
Message-ID: <CAKv+Gu_R48_oW+LF0ofKSZXCs_Kj6b+VGtsscje73j4zw-OP2w@mail.gmail.com> (raw)
In-Reply-To: <20200226221156.29589-12-lersek@redhat.com>
On Wed, 26 Feb 2020 at 23:12, Laszlo Ersek <lersek@redhat.com> wrote:
>
> Once a hot-added CPU finishes the SMBASE relocation, we need to pen it in
> a HLT loop. Add the NASM implementation (with just a handful of
> instructions, but much documentation), and some C language helper
> functions.
>
> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> Cc: Igor Mammedov <imammedo@redhat.com>
> Cc: Jiewen Yao <jiewen.yao@intel.com>
> Cc: Jordan Justen <jordan.l.justen@intel.com>
> Cc: Michael Kinney <michael.d.kinney@intel.com>
> Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
> Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=1512
> Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Acked-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> ---
>
> Notes:
> v2:
>
> - document the combined approach described here:
>
> http://mid.mail-archive.com/111145fc-be3d-2a9a-a126-c14345a8a8a4@redhat.com
> https://edk2.groups.io/g/devel/message/54754
>
> by mentioning the "about to leave SMM" byte in SMRAM.
>
> OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf | 4 +
> OvmfPkg/CpuHotplugSmm/Smbase.h | 32 +++++
> OvmfPkg/CpuHotplugSmm/PostSmmPen.nasm | 151 ++++++++++++++++++++
> OvmfPkg/CpuHotplugSmm/Smbase.c | 110 ++++++++++++++
> 4 files changed, 297 insertions(+)
>
> diff --git a/OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf b/OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf
> index 31c1ee1c9f6d..bf4162299c7c 100644
> --- a/OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf
> +++ b/OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf
> @@ -5,52 +5,56 @@
> #
> # SPDX-License-Identifier: BSD-2-Clause-Patent
> ##
>
> [Defines]
> INF_VERSION = 1.29
> PI_SPECIFICATION_VERSION = 0x00010046 # PI-1.7.0
> BASE_NAME = CpuHotplugSmm
> FILE_GUID = 84EEA114-C6BE-4445-8F90-51D97863E363
> MODULE_TYPE = DXE_SMM_DRIVER
> ENTRY_POINT = CpuHotplugEntry
>
> #
> # The following information is for reference only and not required by the build
> # tools.
> #
> # VALID_ARCHITECTURES = IA32 X64
> #
>
> [Sources]
> ApicId.h
> CpuHotplug.c
> + PostSmmPen.nasm
> QemuCpuhp.c
> QemuCpuhp.h
> + Smbase.c
> + Smbase.h
>
> [Packages]
> MdePkg/MdePkg.dec
> OvmfPkg/OvmfPkg.dec
> UefiCpuPkg/UefiCpuPkg.dec
>
> [LibraryClasses]
> BaseLib
> + BaseMemoryLib
> DebugLib
> MmServicesTableLib
> PcdLib
> SafeIntLib
> UefiDriverEntryPoint
>
> [Protocols]
> gEfiMmCpuIoProtocolGuid ## CONSUMES
> gEfiSmmCpuServiceProtocolGuid ## CONSUMES
>
> [Pcd]
> gUefiCpuPkgTokenSpaceGuid.PcdCpuHotPlugDataAddress ## CONSUMES
> gUefiOvmfPkgTokenSpaceGuid.PcdQ35SmramAtDefaultSmbase ## CONSUMES
>
> [FeaturePcd]
> gUefiOvmfPkgTokenSpaceGuid.PcdSmmSmramRequire ## CONSUMES
>
> [Depex]
> gEfiMmCpuIoProtocolGuid AND
> gEfiSmmCpuServiceProtocolGuid
> diff --git a/OvmfPkg/CpuHotplugSmm/Smbase.h b/OvmfPkg/CpuHotplugSmm/Smbase.h
> new file mode 100644
> index 000000000000..cb5aed98cdd3
> --- /dev/null
> +++ b/OvmfPkg/CpuHotplugSmm/Smbase.h
> @@ -0,0 +1,32 @@
> +/** @file
> + SMBASE relocation for hot-plugged CPUs.
> +
> + Copyright (c) 2020, Red Hat, Inc.
> +
> + SPDX-License-Identifier: BSD-2-Clause-Patent
> +**/
> +
> +#ifndef SMBASE_H_
> +#define SMBASE_H_
> +
> +#include <Uefi/UefiBaseType.h> // EFI_STATUS
> +#include <Uefi/UefiSpec.h> // EFI_BOOT_SERVICES
> +
> +EFI_STATUS
> +SmbaseAllocatePostSmmPen (
> + OUT UINT32 *PenAddress,
> + IN CONST EFI_BOOT_SERVICES *BootServices
> + );
> +
> +VOID
> +SmbaseReinstallPostSmmPen (
> + IN UINT32 PenAddress
> + );
> +
> +VOID
> +SmbaseReleasePostSmmPen (
> + IN UINT32 PenAddress,
> + IN CONST EFI_BOOT_SERVICES *BootServices
> + );
> +
> +#endif // SMBASE_H_
> diff --git a/OvmfPkg/CpuHotplugSmm/PostSmmPen.nasm b/OvmfPkg/CpuHotplugSmm/PostSmmPen.nasm
> new file mode 100644
> index 000000000000..ef702689bdb5
> --- /dev/null
> +++ b/OvmfPkg/CpuHotplugSmm/PostSmmPen.nasm
> @@ -0,0 +1,151 @@
> +;------------------------------------------------------------------------------
> +; @file
> +; Pen any hot-added CPU in a 16-bit, real mode HLT loop, after it leaves SMM by
> +; executing the RSM instruction.
> +;
> +; Copyright (c) 2020, Red Hat, Inc.
> +;
> +; SPDX-License-Identifier: BSD-2-Clause-Patent
> +;
> +; The routine implemented here is stored into normal RAM, under 1MB, at the
> +; beginning of a page that is allocated as EfiReservedMemoryType. On any
> +; hot-added CPU, it is executed after *at least* the first RSM (i.e., after
> +; SMBASE relocation).
> +;
> +; The first execution of this code occurs as follows:
> +;
> +; - The hot-added CPU is in RESET state.
> +;
> +; - The ACPI CPU hotplug event handler triggers a broadcast SMI, from the OS.
> +;
> +; - Existent CPUs (BSP and APs) enter SMM.
> +;
> +; - The hot-added CPU remains in RESET state, but an SMI is pending for it now.
> +; (See "SYSTEM MANAGEMENT INTERRUPT (SMI)" in the Intel SDM.)
> +;
> +; - In SMM, pre-existent CPUs that are not elected SMM Monarch, keep themselves
> +; busy with their wait loops.
> +;
> +; - From the root MMI handler, the SMM Monarch:
> +;
> +; - places this routine in the reserved page,
> +;
> +; - clears the "about to leave SMM" byte in SMRAM,
> +;
> +; - clears the last byte of the reserved page,
> +;
> +; - sends an INIT-SIPI-SIPI sequence to the hot-added CPU,
> +;
> +; - un-gates the default SMI handler by APIC ID.
> +;
> +; - The startup vector in the SIPI that is sent by the SMM Monarch points to
> +; this code; i.e., to the reserved page. (Example: 0x9_F000.)
> +;
> +; - The SMM Monarch starts polling the "about to leave SMM" byte in SMRAM.
> +;
> +; - The hot-added CPU boots, and immediately enters SMM due to the pending SMI.
> +; It starts executing the default SMI handler.
> +;
> +; - Importantly, the SMRAM Save State Map captures the following information,
> +; when the hot-added CPU enters SMM:
> +;
> +; - CS selector: assumes the 16 most significant bits of the 20-bit (i.e.,
> +; below 1MB) startup vector from the SIPI. (Example: 0x9F00.)
> +;
> +; - CS attributes: Accessed, Readable, User (S=1), CodeSegment (bit#11),
> +; Present.
> +;
> +; - CS limit: 0xFFFF.
> +;
> +; - CS base: the CS selector value shifted left by 4 bits. That is, the CS
> +; base equals the SIPI startup vector. (Example: 0x9_F000.)
> +;
> +; - IP: the least significant 4 bits from the SIPI startup vector. Because
> +; the routine is page-aligned, these bits are zero (hence IP is zero).
> +;
> +; - ES, SS, DS, FS, GS selectors: 0.
> +;
> +; - ES, SS, DS, FS, GS attributes: same as the CS attributes, minus
> +; CodeSegment (bit#11).
> +;
> +; - ES, SS, DS, FS, GS limits: 0xFFFF.
> +;
> +; - ES, SS, DS, FS, GS bases: 0.
> +;
> +; - The hot-added CPU sets its new SMBASE value in the SMRAM Save State Map.
> +;
> +; - The hot-added CPU sets the "about to leave SMM" byte in SMRAM, then
> +; executes the RSM instruction immediately after, leaving SMM.
> +;
> +; - The SMM Monarch notices that the "about to leave SMM" byte in SMRAM has
> +; been set, and starts polling the last byte in the reserved page.
> +;
> +; - The hot-added CPU jumps ("returns") to the code below (in the reserved
> +; page), according to the register state listed in the SMRAM Save State Map.
> +;
> +; - The hot-added CPU sets the last byte of the reserved page, then halts
> +; itself.
> +;
> +; - The SMM Monarch notices that the hot-added CPU is done with SMBASE
> +; relocation.
> +;
> +; Note that, if the OS is malicious and sends INIT-SIPI-SIPI to the hot-added
> +; CPU before allowing the ACPI CPU hotplug event handler to trigger a broadcast
> +; SMI, then said broadcast SMI will yank the hot-added CPU directly into SMM,
> +; without becoming pending for it (as the hot-added CPU is no longer in RESET
> +; state). This is OK, because:
> +;
> +; - The default SMI handler copes with this, as it is gated by APIC ID. The
> +; hot-added CPU won't start the actual SMBASE relocation until the SMM
> +; Monarch lets it.
> +;
> +; - The INIT-SIPI-SIPI sequence that the SMM Monarch sends to the hot-added CPU
> +; will be ignored in this sate (it won't even be latched). See "SMI HANDLER
> +; EXECUTION ENVIRONMENT" in the Intel SDM: "INIT operations are inhibited
> +; when the processor enters SMM".
> +;
> +; - When the hot-added CPU (e.g., CPU#1) executes the RSM (having relocated
> +; SMBASE), it returns to the OS. The OS can use CPU#1 to attack the last byte
> +; of the reserved page, while another CPU (e.g., CPU#2) is relocating SMBASE,
> +; in order to trick the SMM Monarch (e.g., CPU#0) to open the APIC ID gate
> +; for yet another CPU (e.g., CPU#3). However, the SMM Monarch won't look at
> +; the last byte of the reserved page, until CPU#2 sets the "about to leave
> +; SMM" byte in SMRAM. This leaves a very small window (just one instruction's
> +; worth before the RSM) for CPU#3 to "catch up" with CPU#2, and overwrite
> +; CPU#2's SMBASE with its own.
> +;
> +; In other words, we do not / need not prevent a malicious OS from booting the
> +; hot-added CPU early; instead we provide benign OSes with a pen for hot-added
> +; CPUs.
> +;------------------------------------------------------------------------------
> +
> +SECTION .data
> +BITS 16
> +
> +GLOBAL ASM_PFX (mPostSmmPen) ; UINT8[]
> +GLOBAL ASM_PFX (mPostSmmPenSize) ; UINT16
> +
> +ASM_PFX (mPostSmmPen):
> + ;
> + ; Point DS at the same reserved page.
> + ;
> + mov ax, cs
> + mov ds, ax
> +
> + ;
> + ; Inform the SMM Monarch that we're done with SMBASE relocation, by setting
> + ; the last byte in the reserved page.
> + ;
> + mov byte [ds : word 0xFFF], 1
> +
> + ;
> + ; Halt now, until we get woken by another SMI, or (more likely) the OS
> + ; reboots us with another INIT-SIPI-SIPI.
> + ;
> +HltLoop:
> + cli
> + hlt
> + jmp HltLoop
> +
> +ASM_PFX (mPostSmmPenSize):
> + dw $ - ASM_PFX (mPostSmmPen)
> diff --git a/OvmfPkg/CpuHotplugSmm/Smbase.c b/OvmfPkg/CpuHotplugSmm/Smbase.c
> new file mode 100644
> index 000000000000..ea21153d9145
> --- /dev/null
> +++ b/OvmfPkg/CpuHotplugSmm/Smbase.c
> @@ -0,0 +1,110 @@
> +/** @file
> + SMBASE relocation for hot-plugged CPUs.
> +
> + Copyright (c) 2020, Red Hat, Inc.
> +
> + SPDX-License-Identifier: BSD-2-Clause-Patent
> +**/
> +
> +#include <Base.h> // BASE_1MB
> +#include <Library/BaseMemoryLib.h> // CopyMem()
> +#include <Library/DebugLib.h> // DEBUG()
> +
> +#include "Smbase.h"
> +
> +extern CONST UINT8 mPostSmmPen[];
> +extern CONST UINT16 mPostSmmPenSize;
> +
> +/**
> + Allocate a non-SMRAM reserved memory page for the Post-SMM Pen for hot-added
> + CPUs.
> +
> + This function may only be called from the entry point function of the driver.
> +
> + @param[out] PenAddress The address of the allocated (normal RAM) reserved
> + page.
> +
> + @param[in] BootServices Pointer to the UEFI boot services table. Used for
> + allocating the normal RAM (not SMRAM) reserved page.
> +
> + @retval EFI_SUCCESS Allocation successful.
> +
> + @retval EFI_BAD_BUFFER_SIZE The Post-SMM Pen template is not smaller than
> + EFI_PAGE_SIZE.
> +
> + @return Error codes propagated from underlying services.
> + DEBUG_ERROR messages have been logged. No
> + resources have been allocated.
> +**/
> +EFI_STATUS
> +SmbaseAllocatePostSmmPen (
> + OUT UINT32 *PenAddress,
> + IN CONST EFI_BOOT_SERVICES *BootServices
> + )
> +{
> + EFI_STATUS Status;
> + EFI_PHYSICAL_ADDRESS Address;
> +
> + //
> + // The pen code must fit in one page, and the last byte must remain free for
> + // signaling the SMM Monarch.
> + //
> + if (mPostSmmPenSize >= EFI_PAGE_SIZE) {
> + Status = EFI_BAD_BUFFER_SIZE;
> + DEBUG ((DEBUG_ERROR, "%a: mPostSmmPenSize=%u: %r\n", __FUNCTION__,
> + mPostSmmPenSize, Status));
> + return Status;
> + }
> +
> + Address = BASE_1MB - 1;
> + Status = BootServices->AllocatePages (AllocateMaxAddress,
> + EfiReservedMemoryType, 1, &Address);
> + if (EFI_ERROR (Status)) {
> + DEBUG ((DEBUG_ERROR, "%a: AllocatePages(): %r\n", __FUNCTION__, Status));
> + return Status;
> + }
> +
> + DEBUG ((DEBUG_INFO, "%a: Post-SMM Pen at 0x%Lx\n", __FUNCTION__, Address));
> + *PenAddress = (UINT32)Address;
> + return EFI_SUCCESS;
> +}
> +
> +/**
> + Copy the Post-SMM Pen template code into the reserved page allocated with
> + SmbaseAllocatePostSmmPen().
> +
> + Note that this effects an "SMRAM to normal RAM" copy.
> +
> + The SMM Monarch is supposed to call this function from the root MMI handler.
> +
> + @param[in] PenAddress The allocation address returned by
> + SmbaseAllocatePostSmmPen().
> +**/
> +VOID
> +SmbaseReinstallPostSmmPen (
> + IN UINT32 PenAddress
> + )
> +{
> + CopyMem ((VOID *)(UINTN)PenAddress, mPostSmmPen, mPostSmmPenSize);
> +}
> +
> +/**
> + Release the reserved page allocated with SmbaseAllocatePostSmmPen().
> +
> + This function may only be called from the entry point function of the driver,
> + on the error path.
> +
> + @param[in] PenAddress The allocation address returned by
> + SmbaseAllocatePostSmmPen().
> +
> + @param[in] BootServices Pointer to the UEFI boot services table. Used for
> + releasing the normal RAM (not SMRAM) reserved page.
> +**/
> +VOID
> +SmbaseReleasePostSmmPen (
> + IN UINT32 PenAddress,
> + IN CONST EFI_BOOT_SERVICES *BootServices
> + )
> +{
> + BootServices->FreePages (PenAddress, 1);
> +}
> --
> 2.19.1.3.g30247aa5d201
>
>
>
>
>
next prev parent reply other threads:[~2020-03-02 14:02 UTC|newest]
Thread overview: 53+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-02-26 22:11 [PATCH v2 00/16] OvmfPkg: support VCPU hotplug with -D SMM_REQUIRE Laszlo Ersek
2020-02-26 22:11 ` [PATCH v2 01/16] MdeModulePkg/PiSmmCore: log SMM image start failure Laszlo Ersek
2020-03-02 12:47 ` [edk2-devel] " Laszlo Ersek
2020-03-02 12:55 ` Liming Gao
2020-03-02 13:46 ` Philippe Mathieu-Daudé
2020-03-03 0:46 ` Dong, Eric
2020-02-26 22:11 ` [PATCH v2 02/16] UefiCpuPkg/PiSmmCpuDxeSmm: fix S3 Resume for CPU hotplug Laszlo Ersek
2020-02-28 3:05 ` [edk2-devel] " Dong, Eric
2020-02-28 10:50 ` Laszlo Ersek
2020-03-04 12:23 ` Laszlo Ersek
2020-03-04 14:36 ` Dong, Eric
2020-02-26 22:11 ` [PATCH v2 03/16] OvmfPkg: clone SmmCpuPlatformHookLib from UefiCpuPkg Laszlo Ersek
2020-03-02 13:27 ` Ard Biesheuvel
2020-03-02 13:49 ` Philippe Mathieu-Daudé
2020-02-26 22:11 ` [PATCH v2 04/16] OvmfPkg: enable SMM Monarch Election in PiSmmCpuDxeSmm Laszlo Ersek
2020-03-02 13:32 ` Ard Biesheuvel
2020-02-26 22:11 ` [PATCH v2 05/16] OvmfPkg: enable CPU hotplug support " Laszlo Ersek
2020-03-02 13:33 ` Ard Biesheuvel
2020-02-26 22:11 ` [PATCH v2 06/16] OvmfPkg/CpuHotplugSmm: introduce skeleton for CPU Hotplug SMM driver Laszlo Ersek
2020-03-02 13:44 ` Ard Biesheuvel
2020-02-26 22:11 ` [PATCH v2 07/16] OvmfPkg/CpuHotplugSmm: add hotplug register block helper functions Laszlo Ersek
2020-03-02 13:24 ` Philippe Mathieu-Daudé
2020-03-02 13:45 ` [edk2-devel] " Ard Biesheuvel
2020-02-26 22:11 ` [PATCH v2 08/16] OvmfPkg/CpuHotplugSmm: define the QEMU_CPUHP_CMD_GET_ARCH_ID macro Laszlo Ersek
2020-03-02 13:22 ` Philippe Mathieu-Daudé
2020-03-02 13:45 ` Ard Biesheuvel
2020-02-26 22:11 ` [PATCH v2 09/16] OvmfPkg/CpuHotplugSmm: add function for collecting CPUs with events Laszlo Ersek
2020-03-02 13:49 ` Ard Biesheuvel
2020-03-02 20:34 ` Philippe Mathieu-Daudé
2020-03-03 10:31 ` Laszlo Ersek
2020-02-26 22:11 ` [PATCH v2 10/16] OvmfPkg/CpuHotplugSmm: collect " Laszlo Ersek
2020-03-02 13:58 ` Ard Biesheuvel
2020-02-26 22:11 ` [PATCH v2 11/16] OvmfPkg/CpuHotplugSmm: introduce Post-SMM Pen for hot-added CPUs Laszlo Ersek
2020-03-02 14:02 ` Ard Biesheuvel [this message]
2020-02-26 22:11 ` [PATCH v2 12/16] OvmfPkg/CpuHotplugSmm: introduce First SMI Handler " Laszlo Ersek
2020-03-02 14:03 ` Ard Biesheuvel
2020-02-26 22:11 ` [PATCH v2 13/16] OvmfPkg/CpuHotplugSmm: complete root MMI handler for CPU hotplug Laszlo Ersek
2020-03-02 14:05 ` Ard Biesheuvel
2020-02-26 22:11 ` [PATCH v2 14/16] OvmfPkg: clone CpuS3DataDxe from UefiCpuPkg Laszlo Ersek
2020-03-02 13:44 ` Philippe Mathieu-Daudé
2020-03-02 14:06 ` Ard Biesheuvel
2020-02-26 22:11 ` [PATCH v2 15/16] OvmfPkg/CpuS3DataDxe: superficial cleanups Laszlo Ersek
2020-03-02 13:25 ` Philippe Mathieu-Daudé
2020-03-02 14:06 ` Ard Biesheuvel
2020-02-26 22:11 ` [PATCH v2 16/16] OvmfPkg/CpuS3DataDxe: enable S3 resume after CPU hotplug Laszlo Ersek
2020-03-02 14:16 ` Ard Biesheuvel
2020-03-02 15:46 ` [edk2-devel] [PATCH v2 00/16] OvmfPkg: support VCPU hotplug with -D SMM_REQUIRE Boris Ostrovsky
2020-03-02 19:22 ` Laszlo Ersek
2020-03-02 19:59 ` Laszlo Ersek
2020-03-04 13:29 ` Philippe Mathieu-Daudé
2020-03-04 18:09 ` Laszlo Ersek
2020-03-04 12:29 ` Laszlo Ersek
2020-03-05 8:32 ` Laszlo Ersek
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-list from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAKv+Gu_R48_oW+LF0ofKSZXCs_Kj6b+VGtsscje73j4zw-OP2w@mail.gmail.com \
--to=devel@edk2.groups.io \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox