[POC QEMU PATCH 0/2] CPU hotplug: use dedicated SMRAM at 0x30000 in SMM address space

[POC QEMU PATCH 0/2] CPU hotplug: use dedicated SMRAM at 0x30000 in SMM address space

[Igor Mammedov]

It's just a quick hack together with Seabios to show
that normal RAM at 0x30000 is not affected by SMM relocation
and dedicated SMRAM could be used for relocation without need to
care about untrusted RAM content at 0x30000.


CC: "Chen,  Yingwen" <yingwen.chen@intel.com>
CC: edk2-devel-groups-io <devel@edk2.groups.io>
CC: Phillip Goerl <phillip.goerl@oracle.com>
CC: qemu devel list <qemu-devel@nongnu.org>
CC: "Yao, Jiewen" <jiewen.yao@intel.com>
CC: "Nakajima, Jun" <jun.nakajima@intel.com>
CC: Boris Ostrovsky <boris.ostrovsky@oracle.com>
CC: edk2-rfc-groups-io <rfc@edk2.groups.io>
CC: Laszlo Ersek <lersek@redhat.com>
CC: Joao Marcal Lemos Martins <joao.m.martins@oracle.com>
CC: pbonzini@redhat.com

 include/hw/pci-host/q35.h |  1 +
 hw/pci-host/q35.c         | 10 ++++++++++
 2 files changed, 11 insertions(+)

-- 
2.18.1

[PATCH QEMU 1/1] q35: use dedicated SMRAM at default SMM_BASE

[Igor Mammedov]

it will allow us to hide sensetive SMM_BASE area from non SMM running env,
that will allow us to ensure that hotplugged CPU will run trusted
SMM BASE relocation code and we won't need to force all present CPUs
into SMM mode since we don not care about about 0x30000 content
in normal RAM address space.

it's a obviously a hack only to demo approach. for easy SMI initialization
on SMI entry point SMRAM is aliased in to hajaked normal RAM address space
at a0000.
Patch should be used with supplied SMBIOS patch, that drops
save/restore sequence and just inits SMI entry point.

to test to run:
  qemu-system-x86_64 -M q35 -bios /path_to_seabios/out/bios.bin \
    -nodefaults \
    -chardev stdio,id=seabios -device isa-debugcon,iobase=0x402,chardev=seabios

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
 include/hw/pci-host/q35.h |  1 +
 hw/pci-host/q35.c         | 10 ++++++++++
 2 files changed, 11 insertions(+)

diff --git a/include/hw/pci-host/q35.h b/include/hw/pci-host/q35.h
index 5ed77facd0..556ecb835a 100644
--- a/include/hw/pci-host/q35.h
+++ b/include/hw/pci-host/q35.h
@@ -55,6 +55,7 @@ typedef struct MCHPCIState {
     MemoryRegion smram_region, open_high_smram;
     MemoryRegion smram, low_smram, high_smram;
     MemoryRegion tseg_blackhole, tseg_window;
+    MemoryRegion smbase, smram_alias;
     Range pci_hole;
     uint64_t below_4g_mem_size;
     uint64_t above_4g_mem_size;
diff --git a/hw/pci-host/q35.c b/hw/pci-host/q35.c
index 0a010be4cf..6c4c0f308c 100644
--- a/hw/pci-host/q35.c
+++ b/hw/pci-host/q35.c
@@ -574,6 +574,16 @@ static void mch_realize(PCIDevice *d, Error **errp)
     memory_region_set_enabled(&mch->tseg_window, false);
     memory_region_add_subregion(&mch->smram, mch->below_4g_mem_size,
                                 &mch->tseg_window);
+
+    memory_region_init_ram(&mch->smm_base, OBJECT(mch), "SMM BASE", MCH_HOST_BRIDGE_SMRAM_C_SIZE, &error_fatal);
+    memory_region_set_enabled(&mch->smm_base, true);
+    memory_region_add_subregion(&mch->smram, 0x30000, &mch->smm_base);
+
+    memory_region_init_alias(&mch->smm_base_alias, OBJECT(mch), "smim_base_alias",
+                             &mch->smm_base, 0, MCH_HOST_BRIDGE_SMRAM_C_SIZE);
+    memory_region_set_enabled(&mch->smm_base_alias, true);
+    memory_region_add_subregion_overlap(mch->system_memory, MCH_HOST_BRIDGE_SMRAM_C_BASE, &mch->smm_base_alias, 1);
+
     object_property_add_const_link(qdev_get_machine(), "smram",
                                    OBJECT(&mch->smram), &error_abort);
 
-- 
2.18.1

[POC Seabios PATCH] seabios: use isolated SMM address space for relocation

[Igor Mammedov]

for purpose of demo SMRAM (at 0x30000) is aliased at a0000 in system address space
for easy initialization of SMI entry point.
Here is resulting debug output showing that RAM at 0x30000 is not affected
by SMM and only RAM in SMM adderss space is modified:

init smm
smm_relocate: before relocaten
smm_relocate: RAM codeentry 0
smm_relocate: RAM  cpu.i64.smm_base  0
smm_relocate: SMRAM  codeentry f000c831eac88c
smm_relocate: SMRAM  cpu.i64.smm_base  0
handle_smi cmd=0 smbase=0x00030000
smm_relocate: after relocaten
smm_relocate: RAM codeentry 0
smm_relocate: RAM  cpu.i64.smm_base  0
smm_relocate: SMRAM  codeentry f000c831eac88c
smm_relocate: SMRAM  cpu.i64.smm_base  a0000

Patch depends on QEMU POC patch that adds SMRAM at 0x30000 in SMM address space

PS:
configure bios with level 9 debugging and debug port

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
 src/fw/smm.c | 43 +++++++++++++++++++------------------------
 1 file changed, 19 insertions(+), 24 deletions(-)

diff --git a/src/fw/smm.c b/src/fw/smm.c
index d90e43a9..27f9747e 100644
--- a/src/fw/smm.c
+++ b/src/fw/smm.c
@@ -140,21 +140,22 @@ extern void entry_smi(void);
                   | ((u64)((u32)entry_smi - BUILD_BIOS_ADDR) << 24))
 
 static void
-smm_save_and_copy(void)
+smm_relocate(void)
 {
-    // save original memory content
     struct smm_layout *initsmm = (void*)BUILD_SMM_INIT_ADDR;
     struct smm_layout *smm = (void*)BUILD_SMM_ADDR;
-    memcpy(&smm->cpu, &initsmm->cpu, sizeof(smm->cpu));
-    memcpy(&smm->codeentry, &initsmm->codeentry, sizeof(smm->codeentry));
 
-    // Setup code entry point.
-    initsmm->codeentry = SMI_INSN;
-}
+    dprintf(3, "smm_relocate: before relocaten\n");
+    dprintf(3, "smm_relocate: RAM codeentry %llx\n", initsmm->codeentry);
+    dprintf(3, "smm_relocate: RAM  cpu.i64.smm_base  %lx\n", initsmm->cpu.i64.smm_base);
+
+
+    /* BUILD_SMM_ADDR aliased to BUILD_SMM_INIT_ADDR in SMM AS
+     * so we could set SMI entry point there */
+    smm->codeentry = SMI_INSN;
+    dprintf(3, "smm_relocate: SMRAM  codeentry %llx\n", smm->codeentry);
+    dprintf(3, "smm_relocate: SMRAM  cpu.i64.smm_base  %lx\n", smm->cpu.i64.smm_base);
 
-static void
-smm_relocate_and_restore(void)
-{
     /* init APM status port */
     outb(0x01, PORT_SMI_STATUS);
 
@@ -165,15 +166,13 @@ smm_relocate_and_restore(void)
     while (inb(PORT_SMI_STATUS) != 0x00)
         ;
 
-    /* restore original memory content */
-    struct smm_layout *initsmm = (void*)BUILD_SMM_INIT_ADDR;
-    struct smm_layout *smm = (void*)BUILD_SMM_ADDR;
-    memcpy(&initsmm->cpu, &smm->cpu, sizeof(initsmm->cpu));
-    memcpy(&initsmm->codeentry, &smm->codeentry, sizeof(initsmm->codeentry));
-
-    // Setup code entry point.
-    smm->codeentry = SMI_INSN;
     wbinvd();
+
+    dprintf(3, "smm_relocate: after relocaten\n");
+    dprintf(3, "smm_relocate: RAM codeentry %llx\n", initsmm->codeentry);
+    dprintf(3, "smm_relocate: RAM  cpu.i64.smm_base  %lx\n", initsmm->cpu.i64.smm_base);
+    dprintf(3, "smm_relocate: SMRAM  codeentry %llx\n", smm->codeentry);
+    dprintf(3, "smm_relocate: SMRAM  cpu.i64.smm_base  %lx\n", smm->cpu.i64.smm_base);
 }
 
 // This code is hardcoded for PIIX4 Power Management device.
@@ -187,8 +186,6 @@ static void piix4_apmc_smm_setup(int isabdf, int i440_bdf)
     /* enable the SMM memory window */
     pci_config_writeb(i440_bdf, I440FX_SMRAM, 0x02 | 0x48);
 
-    smm_save_and_copy();
-
     /* enable SMI generation when writing to the APMC register */
     pci_config_writel(isabdf, PIIX_DEVACTB, value | PIIX_DEVACTB_APMC_EN);
 
@@ -196,7 +193,7 @@ static void piix4_apmc_smm_setup(int isabdf, int i440_bdf)
     value = inl(acpi_pm_base + PIIX_PMIO_GLBCTL);
     outl(value | PIIX_PMIO_GLBCTL_SMI_EN, acpi_pm_base + PIIX_PMIO_GLBCTL);
 
-    smm_relocate_and_restore();
+    smm_relocate();
 
     /* close the SMM memory window and enable normal SMM */
     pci_config_writeb(i440_bdf, I440FX_SMRAM, 0x02 | 0x08);
@@ -213,8 +210,6 @@ void ich9_lpc_apmc_smm_setup(int isabdf, int mch_bdf)
     /* enable the SMM memory window */
     pci_config_writeb(mch_bdf, Q35_HOST_BRIDGE_SMRAM, 0x02 | 0x48);
 
-    smm_save_and_copy();
-
     /* enable SMI generation when writing to the APMC register */
     outl(value | ICH9_PMIO_SMI_EN_APMC_EN | ICH9_PMIO_SMI_EN_GLB_SMI_EN,
          acpi_pm_base + ICH9_PMIO_SMI_EN);
@@ -224,7 +219,7 @@ void ich9_lpc_apmc_smm_setup(int isabdf, int mch_bdf)
     pci_config_writel(isabdf, ICH9_LPC_GEN_PMCON_1,
                       value | ICH9_LPC_GEN_PMCON_1_SMI_LOCK);
 
-    smm_relocate_and_restore();
+    smm_relocate();
 
     /* close the SMM memory window and enable normal SMM */
     pci_config_writeb(mch_bdf, Q35_HOST_BRIDGE_SMRAM, 0x02 | 0x08);
-- 
2.18.1

Re: [POC Seabios PATCH] seabios: use isolated SMM address space for relocation

[Boris Ostrovsky]

On 8/16/19 7:24 AM, Igor Mammedov wrote:
> for purpose of demo SMRAM (at 0x30000) is aliased at a0000 in system address space
> for easy initialization of SMI entry point.
> Here is resulting debug output showing that RAM at 0x30000 is not affected
> by SMM and only RAM in SMM adderss space is modified:
>
> init smm
> smm_relocate: before relocaten
> smm_relocate: RAM codeentry 0
> smm_relocate: RAM  cpu.i64.smm_base  0
> smm_relocate: SMRAM  codeentry f000c831eac88c
> smm_relocate: SMRAM  cpu.i64.smm_base  0
> handle_smi cmd=0 smbase=0x00030000
> smm_relocate: after relocaten
> smm_relocate: RAM codeentry 0
> smm_relocate: RAM  cpu.i64.smm_base  0
> smm_relocate: SMRAM  codeentry f000c831eac88c
> smm_relocate: SMRAM  cpu.i64.smm_base  a0000


I most likely don't understand how this is supposed to work but aren't
we here successfully reading SMRAM from non-SMM context, something we
are not supposed to be able to do?


-boris

Re: [POC Seabios PATCH] seabios: use isolated SMM address space for relocation

[Igor Mammedov]

On Fri, 16 Aug 2019 18:43:11 -0400
Boris Ostrovsky <boris.ostrovsky@oracle.com> wrote:

> On 8/16/19 7:24 AM, Igor Mammedov wrote:
> > for purpose of demo SMRAM (at 0x30000) is aliased at a0000 in system address space
> > for easy initialization of SMI entry point.
> > Here is resulting debug output showing that RAM at 0x30000 is not affected
> > by SMM and only RAM in SMM adderss space is modified:
> >
> > init smm
> > smm_relocate: before relocaten
> > smm_relocate: RAM codeentry 0
> > smm_relocate: RAM  cpu.i64.smm_base  0
^^^ reads using 0x30000 base in non-SMM mode

> > smm_relocate: SMRAM  codeentry f000c831eac88c
> > smm_relocate: SMRAM  cpu.i64.smm_base  0
^^^ reads from SMRAM temporarily aliased at 0xa0000 in non-SMM mode 

> > handle_smi cmd=0 smbase=0x00030000
^^^ reads using 0x30000 base in SMM mode

> > smm_relocate: after relocaten
> > smm_relocate: RAM codeentry 0
> > smm_relocate: RAM  cpu.i64.smm_base  0
^^^ normal RAM at 0x30000 base hasn't been modified after SMM relocation
    without us taking care of saving/restoring it (2nd patch removes it altogether)

> > smm_relocate: SMRAM  codeentry f000c831eac88c
> > smm_relocate: SMRAM  cpu.i64.smm_base  a0000
^^^ but SMRAM has changed base to what out handler told it to
    (note we are reading it form non-SMM context only because we
     have an alias at a0000 which it there only for demo purposes)
> 
> 
> I most likely don't understand how this is supposed to work but aren't
> we here successfully reading SMRAM from non-SMM context, something we
> are not supposed to be able to do?

We are aren't reading SMRAM at 0x30000 base directly,
"RAM" marked log lines are non-SMM context reads using as base
  BUILD_SMM_INIT_ADDR       0x30000
and as you see, it isn't showing anything from SMRAM

For mgmt/demo purposes SMRAM (which is at 0x30000 in SMM address space)
is also aliased at
  BUILD_SMM_ADDR            0xa0000
into non-SMM address space to allow us to initialize SMM entry point
(log entries are marked as "SMRAM").

Aliased SMRAM also allows us to check that relocation worked
(i.e. smm_base was relocated from default "handle_smi cmd=0 smbase=0x00030000"
to a new one "smm_relocate: SMRAM  cpu.i64.smm_base  a0000").


It's similar to what we do with TSEG where QEMU steals RAM from
normal address space and puts MMIO region 'tseg_blackhole' over it
so non-SMM context reads 0xFF from TSEG window, while SMM context
accesses RAM hidden below tseg_blackhole.

These patches show that we can have normal usable RAM at 0x30000
which doesn't overlap with SMRAM at the same address and each can
be made accessible only from its own mode (no-SMM and SMM).
Preventing non-SMM mode from injecting attack on SMRAM via CPU
that hasn't been initialized yet once firmware locked down SMRAM.


> 
> 
> -boris
> 

Re: [POC Seabios PATCH] seabios: use isolated SMM address space for relocation

[Boris Ostrovsky]

On 8/26/19 9:57 AM, Igor Mammedov wrote:
>
>> I most likely don't understand how this is supposed to work but aren't
>> we here successfully reading SMRAM from non-SMM context, something we
>> are not supposed to be able to do?
> We are aren't reading SMRAM at 0x30000 base directly,
> "RAM" marked log lines are non-SMM context reads using as base
>   BUILD_SMM_INIT_ADDR       0x30000
> and as you see, it isn't showing anything from SMRAM
>
> For mgmt/demo purposes SMRAM (which is at 0x30000 in SMM address space)
> is also aliased at
>   BUILD_SMM_ADDR            0xa0000
> into non-SMM address space to allow us to initialize SMM entry point
> (log entries are marked as "SMRAM").



OK, I then misunderstood the purpose of this demo. I thought you were
not supposed to be able to read it from either location in non-SMM mode.

Thanks for the explanation.

-boris

>
> Aliased SMRAM also allows us to check that relocation worked
> (i.e. smm_base was relocated from default "handle_smi cmd=0 smbase=0x00030000"
> to a new one "smm_relocate: SMRAM  cpu.i64.smm_base  a0000").
>
>
> It's similar to what we do with TSEG where QEMU steals RAM from
> normal address space and puts MMIO region 'tseg_blackhole' over it
> so non-SMM context reads 0xFF from TSEG window, while SMM context
> accesses RAM hidden below tseg_blackhole.
>
> These patches show that we can have normal usable RAM at 0x30000
> which doesn't overlap with SMRAM at the same address and each can
> be made accessible only from its own mode (no-SMM and SMM).
> Preventing non-SMM mode from injecting attack on SMRAM via CPU
> that hasn't been initialized yet once firmware locked down SMRAM.
>
>
>>
>> -boris
>>