From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-wr1-f49.google.com (mail-wr1-f49.google.com [209.85.221.49]) by mx.groups.io with SMTP id smtpd.web10.14597.1583157752788682471 for ; Mon, 02 Mar 2020 06:02:33 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@linaro.org header.s=google header.b=LvE6ZwQT; spf=pass (domain: linaro.org, ip: 209.85.221.49, mailfrom: ard.biesheuvel@linaro.org) Received: by mail-wr1-f49.google.com with SMTP id v4so12707874wrs.8 for ; Mon, 02 Mar 2020 06:02:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=8lWkF+LIJKd0zIaqTIv6sordxUAd8gUzi8px2ldo39k=; b=LvE6ZwQTAeM3DGarjG+ruAmgiyWuM5ra/q8WXzylprgVEAkpDtftuudlqLtROhHw7q Oma/06HznHzC5d1dTBmB8C54oQFShaaKIYK2Tuq74gzOYN9WWLNc88k9nO6Ga17ftF0A 4H10nPWNuXLZsQJH9o3f+dAd0/4R9shUpp52VN4cZbt3lsYKAO/q24St1MUtEtNbRvvq in0snL/cryRg2/lBs3YB3Y85aOkp5OAJg2SzU7J/588hdhKkEnd/c49T58Am4v9/Wfgr q155pzZvhbnGNLzi+uvdUKksi5/NrwL4ckA1HZcGqNDMv5T6wZISCHdsShlhSyQ7H3FN JyHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=8lWkF+LIJKd0zIaqTIv6sordxUAd8gUzi8px2ldo39k=; b=Hf2+EjXVJ9JmnjMhm2F8yqAD4GieK8EGc72aEH8Lbx1F0+McG/4H41t1IrpqjBiDs7 gYqcW9fzU6pdGfQaxcEFrKA9+GJWmizLTXJ4TGTU2PRtYnHn9ZA6n87LgVhCSJWcKzU8 yEHC2wWs3jCXCLHp7dMfRcrqzOiNbDd46Q/xfMzMhblUCOvsfkybY8DwnLolr9ZeuPcm 1r/dHMbBzdMCU+KqcNyKwkVeeiBKcD56VDQbLmLlh4VoiuwNSfS3XtLnwP/HVCNL3B0p YQ3B9q961qStOY9rY2TrAFJBUvZi0GeOEW+keIwrdpDOYgqvo7i24EIbTC9gtUwTgxpO tvBQ== X-Gm-Message-State: APjAAAVb/dBlndyXrfsaY5R/OI4OxKBdzTWL7yuzNarN9Ceci9VyLMwA lKnXv/LgMUfp2FlKz4SzuNLt5wtd63LLZkBgtufSIfbguZ3vuQ== X-Google-Smtp-Source: APXvYqxQ9Uih+OATQCzQB3NXgCU2rAAOBDZr94yt3Q1X1y+C4ePaZt7VBo5IejHDbnuIKQzDw7tm+naT2qKO8+SYiOU= X-Received: by 2002:adf:f84a:: with SMTP id d10mr22947368wrq.208.1583157750478; Mon, 02 Mar 2020 06:02:30 -0800 (PST) MIME-Version: 1.0 References: <20200226221156.29589-1-lersek@redhat.com> <20200226221156.29589-12-lersek@redhat.com> In-Reply-To: <20200226221156.29589-12-lersek@redhat.com> From: "Ard Biesheuvel" Date: Mon, 2 Mar 2020 15:02:19 +0100 Message-ID: Subject: Re: [edk2-devel] [PATCH v2 11/16] OvmfPkg/CpuHotplugSmm: introduce Post-SMM Pen for hot-added CPUs To: edk2-devel-groups-io , Laszlo Ersek Cc: Igor Mammedov , Jiewen Yao , Jordan Justen , Michael Kinney , =?UTF-8?Q?Philippe_Mathieu=2DDaud=C3=A9?= Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Wed, 26 Feb 2020 at 23:12, Laszlo Ersek wrote: > > Once a hot-added CPU finishes the SMBASE relocation, we need to pen it i= n > a HLT loop. Add the NASM implementation (with just a handful of > instructions, but much documentation), and some C language helper > functions. > > Cc: Ard Biesheuvel > Cc: Igor Mammedov > Cc: Jiewen Yao > Cc: Jordan Justen > Cc: Michael Kinney > Cc: Philippe Mathieu-Daud=C3=A9 > Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3D1512 > Signed-off-by: Laszlo Ersek Acked-by: Ard Biesheuvel > --- > > Notes: > v2: > > - document the combined approach described here: > > http://mid.mail-archive.com/111145fc-be3d-2a9a-a126-c14345a8a8a4@r= edhat.com > https://edk2.groups.io/g/devel/message/54754 > > by mentioning the "about to leave SMM" byte in SMRAM. > > OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf | 4 + > OvmfPkg/CpuHotplugSmm/Smbase.h | 32 +++++ > OvmfPkg/CpuHotplugSmm/PostSmmPen.nasm | 151 ++++++++++++++++++++ > OvmfPkg/CpuHotplugSmm/Smbase.c | 110 ++++++++++++++ > 4 files changed, 297 insertions(+) > > diff --git a/OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf b/OvmfPkg/CpuHotplu= gSmm/CpuHotplugSmm.inf > index 31c1ee1c9f6d..bf4162299c7c 100644 > --- a/OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf > +++ b/OvmfPkg/CpuHotplugSmm/CpuHotplugSmm.inf > @@ -5,52 +5,56 @@ > # > # SPDX-License-Identifier: BSD-2-Clause-Patent > ## > > [Defines] > INF_VERSION =3D 1.29 > PI_SPECIFICATION_VERSION =3D 0x00010046 = # PI-1.7.0 > BASE_NAME =3D CpuHotplugSmm > FILE_GUID =3D 84EEA114-C6BE-4445-8F90-51D97863E363 > MODULE_TYPE =3D DXE_SMM_DRIVER > ENTRY_POINT =3D CpuHotplugEntry > > # > # The following information is for reference only and not required by t= he build > # tools. > # > # VALID_ARCHITECTURES =3D IA32 X64 > # > > [Sources] > ApicId.h > CpuHotplug.c > + PostSmmPen.nasm > QemuCpuhp.c > QemuCpuhp.h > + Smbase.c > + Smbase.h > > [Packages] > MdePkg/MdePkg.dec > OvmfPkg/OvmfPkg.dec > UefiCpuPkg/UefiCpuPkg.dec > > [LibraryClasses] > BaseLib > + BaseMemoryLib > DebugLib > MmServicesTableLib > PcdLib > SafeIntLib > UefiDriverEntryPoint > > [Protocols] > gEfiMmCpuIoProtocolGuid ## = CONSUMES > gEfiSmmCpuServiceProtocolGuid ## = CONSUMES > > [Pcd] > gUefiCpuPkgTokenSpaceGuid.PcdCpuHotPlugDataAddress ## = CONSUMES > gUefiOvmfPkgTokenSpaceGuid.PcdQ35SmramAtDefaultSmbase ## = CONSUMES > > [FeaturePcd] > gUefiOvmfPkgTokenSpaceGuid.PcdSmmSmramRequire ## = CONSUMES > > [Depex] > gEfiMmCpuIoProtocolGuid AND > gEfiSmmCpuServiceProtocolGuid > diff --git a/OvmfPkg/CpuHotplugSmm/Smbase.h b/OvmfPkg/CpuHotplugSmm/Smba= se.h > new file mode 100644 > index 000000000000..cb5aed98cdd3 > --- /dev/null > +++ b/OvmfPkg/CpuHotplugSmm/Smbase.h > @@ -0,0 +1,32 @@ > +/** @file > + SMBASE relocation for hot-plugged CPUs. > + > + Copyright (c) 2020, Red Hat, Inc. > + > + SPDX-License-Identifier: BSD-2-Clause-Patent > +**/ > + > +#ifndef SMBASE_H_ > +#define SMBASE_H_ > + > +#include // EFI_STATUS > +#include // EFI_BOOT_SERVICES > + > +EFI_STATUS > +SmbaseAllocatePostSmmPen ( > + OUT UINT32 *PenAddress, > + IN CONST EFI_BOOT_SERVICES *BootServices > + ); > + > +VOID > +SmbaseReinstallPostSmmPen ( > + IN UINT32 PenAddress > + ); > + > +VOID > +SmbaseReleasePostSmmPen ( > + IN UINT32 PenAddress, > + IN CONST EFI_BOOT_SERVICES *BootServices > + ); > + > +#endif // SMBASE_H_ > diff --git a/OvmfPkg/CpuHotplugSmm/PostSmmPen.nasm b/OvmfPkg/CpuHotplugS= mm/PostSmmPen.nasm > new file mode 100644 > index 000000000000..ef702689bdb5 > --- /dev/null > +++ b/OvmfPkg/CpuHotplugSmm/PostSmmPen.nasm > @@ -0,0 +1,151 @@ > +;----------------------------------------------------------------------= -------- > +; @file > +; Pen any hot-added CPU in a 16-bit, real mode HLT loop, after it leave= s SMM by > +; executing the RSM instruction. > +; > +; Copyright (c) 2020, Red Hat, Inc. > +; > +; SPDX-License-Identifier: BSD-2-Clause-Patent > +; > +; The routine implemented here is stored into normal RAM, under 1MB, at= the > +; beginning of a page that is allocated as EfiReservedMemoryType. On an= y > +; hot-added CPU, it is executed after *at least* the first RSM (i.e., a= fter > +; SMBASE relocation). > +; > +; The first execution of this code occurs as follows: > +; > +; - The hot-added CPU is in RESET state. > +; > +; - The ACPI CPU hotplug event handler triggers a broadcast SMI, from t= he OS. > +; > +; - Existent CPUs (BSP and APs) enter SMM. > +; > +; - The hot-added CPU remains in RESET state, but an SMI is pending for= it now. > +; (See "SYSTEM MANAGEMENT INTERRUPT (SMI)" in the Intel SDM.) > +; > +; - In SMM, pre-existent CPUs that are not elected SMM Monarch, keep th= emselves > +; busy with their wait loops. > +; > +; - From the root MMI handler, the SMM Monarch: > +; > +; - places this routine in the reserved page, > +; > +; - clears the "about to leave SMM" byte in SMRAM, > +; > +; - clears the last byte of the reserved page, > +; > +; - sends an INIT-SIPI-SIPI sequence to the hot-added CPU, > +; > +; - un-gates the default SMI handler by APIC ID. > +; > +; - The startup vector in the SIPI that is sent by the SMM Monarch poin= ts to > +; this code; i.e., to the reserved page. (Example: 0x9_F000.) > +; > +; - The SMM Monarch starts polling the "about to leave SMM" byte in SMR= AM. > +; > +; - The hot-added CPU boots, and immediately enters SMM due to the pend= ing SMI. > +; It starts executing the default SMI handler. > +; > +; - Importantly, the SMRAM Save State Map captures the following inform= ation, > +; when the hot-added CPU enters SMM: > +; > +; - CS selector: assumes the 16 most significant bits of the 20-bit (= i.e., > +; below 1MB) startup vector from the SIPI. (Example: 0x9F00.) > +; > +; - CS attributes: Accessed, Readable, User (S=3D1), CodeSegment (bit= #11), > +; Present. > +; > +; - CS limit: 0xFFFF. > +; > +; - CS base: the CS selector value shifted left by 4 bits. That is, t= he CS > +; base equals the SIPI startup vector. (Example: 0x9_F000.) > +; > +; - IP: the least significant 4 bits from the SIPI startup vector. Be= cause > +; the routine is page-aligned, these bits are zero (hence IP is zer= o). > +; > +; - ES, SS, DS, FS, GS selectors: 0. > +; > +; - ES, SS, DS, FS, GS attributes: same as the CS attributes, minus > +; CodeSegment (bit#11). > +; > +; - ES, SS, DS, FS, GS limits: 0xFFFF. > +; > +; - ES, SS, DS, FS, GS bases: 0. > +; > +; - The hot-added CPU sets its new SMBASE value in the SMRAM Save State= Map. > +; > +; - The hot-added CPU sets the "about to leave SMM" byte in SMRAM, then > +; executes the RSM instruction immediately after, leaving SMM. > +; > +; - The SMM Monarch notices that the "about to leave SMM" byte in SMRAM= has > +; been set, and starts polling the last byte in the reserved page. > +; > +; - The hot-added CPU jumps ("returns") to the code below (in the reser= ved > +; page), according to the register state listed in the SMRAM Save Sta= te Map. > +; > +; - The hot-added CPU sets the last byte of the reserved page, then hal= ts > +; itself. > +; > +; - The SMM Monarch notices that the hot-added CPU is done with SMBASE > +; relocation. > +; > +; Note that, if the OS is malicious and sends INIT-SIPI-SIPI to the hot= -added > +; CPU before allowing the ACPI CPU hotplug event handler to trigger a b= roadcast > +; SMI, then said broadcast SMI will yank the hot-added CPU directly int= o SMM, > +; without becoming pending for it (as the hot-added CPU is no longer in= RESET > +; state). This is OK, because: > +; > +; - The default SMI handler copes with this, as it is gated by APIC ID.= The > +; hot-added CPU won't start the actual SMBASE relocation until the SM= M > +; Monarch lets it. > +; > +; - The INIT-SIPI-SIPI sequence that the SMM Monarch sends to the hot-a= dded CPU > +; will be ignored in this sate (it won't even be latched). See "SMI H= ANDLER > +; EXECUTION ENVIRONMENT" in the Intel SDM: "INIT operations are inhib= ited > +; when the processor enters SMM". > +; > +; - When the hot-added CPU (e.g., CPU#1) executes the RSM (having reloc= ated > +; SMBASE), it returns to the OS. The OS can use CPU#1 to attack the l= ast byte > +; of the reserved page, while another CPU (e.g., CPU#2) is relocating= SMBASE, > +; in order to trick the SMM Monarch (e.g., CPU#0) to open the APIC ID= gate > +; for yet another CPU (e.g., CPU#3). However, the SMM Monarch won't l= ook at > +; the last byte of the reserved page, until CPU#2 sets the "about to = leave > +; SMM" byte in SMRAM. This leaves a very small window (just one instr= uction's > +; worth before the RSM) for CPU#3 to "catch up" with CPU#2, and overw= rite > +; CPU#2's SMBASE with its own. > +; > +; In other words, we do not / need not prevent a malicious OS from boot= ing the > +; hot-added CPU early; instead we provide benign OSes with a pen for ho= t-added > +; CPUs. > +;----------------------------------------------------------------------= -------- > + > +SECTION .data > +BITS 16 > + > +GLOBAL ASM_PFX (mPostSmmPen) ; UINT8[] > +GLOBAL ASM_PFX (mPostSmmPenSize) ; UINT16 > + > +ASM_PFX (mPostSmmPen): > + ; > + ; Point DS at the same reserved page. > + ; > + mov ax, cs > + mov ds, ax > + > + ; > + ; Inform the SMM Monarch that we're done with SMBASE relocation, by s= etting > + ; the last byte in the reserved page. > + ; > + mov byte [ds : word 0xFFF], 1 > + > + ; > + ; Halt now, until we get woken by another SMI, or (more likely) the O= S > + ; reboots us with another INIT-SIPI-SIPI. > + ; > +HltLoop: > + cli > + hlt > + jmp HltLoop > + > +ASM_PFX (mPostSmmPenSize): > + dw $ - ASM_PFX (mPostSmmPen) > diff --git a/OvmfPkg/CpuHotplugSmm/Smbase.c b/OvmfPkg/CpuHotplugSmm/Smba= se.c > new file mode 100644 > index 000000000000..ea21153d9145 > --- /dev/null > +++ b/OvmfPkg/CpuHotplugSmm/Smbase.c > @@ -0,0 +1,110 @@ > +/** @file > + SMBASE relocation for hot-plugged CPUs. > + > + Copyright (c) 2020, Red Hat, Inc. > + > + SPDX-License-Identifier: BSD-2-Clause-Patent > +**/ > + > +#include // BASE_1MB > +#include // CopyMem() > +#include // DEBUG() > + > +#include "Smbase.h" > + > +extern CONST UINT8 mPostSmmPen[]; > +extern CONST UINT16 mPostSmmPenSize; > + > +/** > + Allocate a non-SMRAM reserved memory page for the Post-SMM Pen for ho= t-added > + CPUs. > + > + This function may only be called from the entry point function of the= driver. > + > + @param[out] PenAddress The address of the allocated (normal RAM) re= served > + page. > + > + @param[in] BootServices Pointer to the UEFI boot services table. Use= d for > + allocating the normal RAM (not SMRAM) reserv= ed page. > + > + @retval EFI_SUCCESS Allocation successful. > + > + @retval EFI_BAD_BUFFER_SIZE The Post-SMM Pen template is not smaller= than > + EFI_PAGE_SIZE. > + > + @return Error codes propagated from underlying s= ervices. > + DEBUG_ERROR messages have been logged. N= o > + resources have been allocated. > +**/ > +EFI_STATUS > +SmbaseAllocatePostSmmPen ( > + OUT UINT32 *PenAddress, > + IN CONST EFI_BOOT_SERVICES *BootServices > + ) > +{ > + EFI_STATUS Status; > + EFI_PHYSICAL_ADDRESS Address; > + > + // > + // The pen code must fit in one page, and the last byte must remain f= ree for > + // signaling the SMM Monarch. > + // > + if (mPostSmmPenSize >=3D EFI_PAGE_SIZE) { > + Status =3D EFI_BAD_BUFFER_SIZE; > + DEBUG ((DEBUG_ERROR, "%a: mPostSmmPenSize=3D%u: %r\n", __FUNCTION__= , > + mPostSmmPenSize, Status)); > + return Status; > + } > + > + Address =3D BASE_1MB - 1; > + Status =3D BootServices->AllocatePages (AllocateMaxAddress, > + EfiReservedMemoryType, 1, &Address); > + if (EFI_ERROR (Status)) { > + DEBUG ((DEBUG_ERROR, "%a: AllocatePages(): %r\n", __FUNCTION__, Sta= tus)); > + return Status; > + } > + > + DEBUG ((DEBUG_INFO, "%a: Post-SMM Pen at 0x%Lx\n", __FUNCTION__, Addr= ess)); > + *PenAddress =3D (UINT32)Address; > + return EFI_SUCCESS; > +} > + > +/** > + Copy the Post-SMM Pen template code into the reserved page allocated = with > + SmbaseAllocatePostSmmPen(). > + > + Note that this effects an "SMRAM to normal RAM" copy. > + > + The SMM Monarch is supposed to call this function from the root MMI h= andler. > + > + @param[in] PenAddress The allocation address returned by > + SmbaseAllocatePostSmmPen(). > +**/ > +VOID > +SmbaseReinstallPostSmmPen ( > + IN UINT32 PenAddress > + ) > +{ > + CopyMem ((VOID *)(UINTN)PenAddress, mPostSmmPen, mPostSmmPenSize); > +} > + > +/** > + Release the reserved page allocated with SmbaseAllocatePostSmmPen(). > + > + This function may only be called from the entry point function of the= driver, > + on the error path. > + > + @param[in] PenAddress The allocation address returned by > + SmbaseAllocatePostSmmPen(). > + > + @param[in] BootServices Pointer to the UEFI boot services table. Use= d for > + releasing the normal RAM (not SMRAM) reserve= d page. > +**/ > +VOID > +SmbaseReleasePostSmmPen ( > + IN UINT32 PenAddress, > + IN CONST EFI_BOOT_SERVICES *BootServices > + ) > +{ > + BootServices->FreePages (PenAddress, 1); > +} > -- > 2.19.1.3.g30247aa5d201 > > > >=20 >