SMRAM sizes on large hosts

[Laszlo Ersek <lersek@redhat.com>]

Hi All,

in your experience, how much SMRAM do "big hosts" provide? (Machines
that have, say, ~300 CPU cores.)

With QEMU's Q35 board, which provides 8MB of SMRAM (TSEG), we're hitting
various out-of-SMRAM conditions with OVMF at around 230-240 VCPUs. We'd
like to go to a higher VCPU count than that.

* So, in your experience, how much SMRAM do physical boards, that carry
such a high number of cores, provide?

* Perhaps we'll have to do something about the SMRAM size on QEMU in the
longer term, but until then, can you guys recommend various "cheap
tricks" to decrease per-VCPU SMRAM usage?

For example, in OVMF we have a 16KB SMM stack per VCPU, and we also
enable the SMM stack overflow guard page -- we had been hit by an SMM
stack overflow with the original 8KB stack size, and so we increased
both the stack size and enabled the guard page; see commits

  509f8425b75d UefiCpuPkg: change PcdCpuSmmStackGuard default to TRUE
  0d0c245dfb14 OvmfPkg: set SMM stack size to 16KB

I've now tried to decrease the stack size to the "middle point" 12KB.
That stack size does not reproduce the SMM stack overflow seen
originally, but it also doesn't help with the SMRAM exhaustion -- we
cannot go to any higher VCPU count with it.

Are there any other "tweakables" (PCDs) we could massage to see the
per-VCPU SMRAM usage go down?

Here's a (somewhat indiscriminate) list of PCDs, from the OVMF Ia32X64
build report file, where each PCD's name contains "smm":

    PcdCpuSmmBlockStartupThisAp                 :   FLAG  (BOOLEAN) = 0
    PcdCpuSmmDebug                              :   FLAG  (BOOLEAN) = 0
    PcdCpuSmmFeatureControlMsrLock              :   FLAG  (BOOLEAN) = 1
    PcdCpuSmmProfileEnable                      :   FLAG  (BOOLEAN) = 0
    PcdCpuSmmProfileRingBuffer                  :   FLAG  (BOOLEAN) = 0
    PcdCpuSmmStackGuard                         :   FLAG  (BOOLEAN) = 1
 *P PcdCpuSmmEnableBspElection                  :   FLAG  (BOOLEAN) = 0
 *P PcdSmmSmramRequire                          :   FLAG  (BOOLEAN) = 1

    PcdCpuSmmCodeAccessCheckEnable              :  FIXED  (BOOLEAN) = 1
    PcdCpuSmmProfileSize                        :  FIXED   (UINT32) = 0x200000
    PcdCpuSmmStaticPageTable                    :  FIXED  (BOOLEAN) = 1
 *P PcdCpuSmmStackSize                          :  FIXED   (UINT32) = 0x4000

    PcdLoadFixAddressSmmCodePageNumber          :  PATCH   (UINT32) = 0

    PcdS3BootScriptTablePrivateSmmDataPtr       :    DYN   (UINT64) = 0x0
 *P PcdCpuSmmApSyncTimeout                      :    DYN   (UINT64) = 100000
 *P PcdCpuSmmSyncMode                           :    DYN    (UINT8) = 0x01

(BTW, security features should not be disabled, even if they saved some
SMRAM.)

Thank you,
Laszlo