From mboxrd@z Thu Jan  1 00:00:00 1970
Received: from bedivere.hansenpartnership.com (bedivere.hansenpartnership.com [96.44.175.130])
 by mx.groups.io with SMTP id smtpd.web08.1406.1649798523464807350
 for <devel@edk2.groups.io>;
 Tue, 12 Apr 2022 14:22:03 -0700
Authentication-Results: mx.groups.io;
 dkim=pass header.i=@hansenpartnership.com header.s=20151216 header.b=bw8ZyY16;
 spf=pass (domain: hansenpartnership.com, ip: 96.44.175.130, mailfrom: james.bottomley@hansenpartnership.com)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple;
	d=hansenpartnership.com; s=20151216; t=1649798522;
	bh=cxp5C2KqSqboZHVMFW5ZCBm7zwx3/BDvHasRTc/X8tM=;
	h=Message-ID:Subject:From:To:Date:From;
	b=bw8ZyY16r0YdjFrL11IYa/rlufgIOGliLkaVgk4o2I4s5inKlGGqMAnRjVjZLNZzb
	 +1ltRV60F9tU4vfpRxCdoaiXYdprRDz+qB0Rw+Rf6T71c2zuPOA/BMoJrVyhJoT9CF
	 w526fIgS57C+Bg/+LXLttxsizkwYeDaqOSq9RY78=
Received: from localhost (localhost [127.0.0.1])
	by bedivere.hansenpartnership.com (Postfix) with ESMTP id E40001280F85;
	Tue, 12 Apr 2022 17:22:02 -0400 (EDT)
Received: from bedivere.hansenpartnership.com ([127.0.0.1])
	by localhost (bedivere.hansenpartnership.com [127.0.0.1]) (amavisd-new, port 10024)
	with ESMTP id MRZezrrsuHP4; Tue, 12 Apr 2022 17:22:02 -0400 (EDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple;
	d=hansenpartnership.com; s=20151216; t=1649798522;
	bh=cxp5C2KqSqboZHVMFW5ZCBm7zwx3/BDvHasRTc/X8tM=;
	h=Message-ID:Subject:From:To:Date:From;
	b=bw8ZyY16r0YdjFrL11IYa/rlufgIOGliLkaVgk4o2I4s5inKlGGqMAnRjVjZLNZzb
	 +1ltRV60F9tU4vfpRxCdoaiXYdprRDz+qB0Rw+Rf6T71c2zuPOA/BMoJrVyhJoT9CF
	 w526fIgS57C+Bg/+LXLttxsizkwYeDaqOSq9RY78=
Received: from lingrow.int.hansenpartnership.com (unknown [IPv6:2601:5c4:4300:c551::c14])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits))
	(No client certificate requested)
	by bedivere.hansenpartnership.com (Postfix) with ESMTPSA id 57B501280EEF;
	Tue, 12 Apr 2022 17:22:02 -0400 (EDT)
Message-ID: <e69f65c5800014337d4a06bee96211e4b55bed43.camel@HansenPartnership.com>
Subject: Regression: 100x I/O performance slowdown in SEC phase caused by TDX
From: "James Bottomley" <James.Bottomley@HansenPartnership.com>
To: Min Xu <min.m.xu@intel.com>, Jiewen Yao <jiewen.yao@intel.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>, devel@edk2.groups.io
Date: Tue, 12 Apr 2022 17:22:01 -0400
User-Agent: Evolution 3.34.4 
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit

I'm using a SEC phase which has a TPM driver to experiment with sorting
out measured boot, which is how I noticed (usually SEC doesn't do MMIO)
.  What I'm seeing is after commit b6b2de884864 ("MdePkg: Support mmio
for Tdx guest in BaseIoLibIntrinsic") we get a massive slowdown of
about 100x in TPM performance.  The reason seems to be this addition to
the mmioreadX/mmiowriteX code:

     MemoryFence ();
-    *(volatile UINT16 *)Address = Value;
+
+    if (IsTdxGuest ()) {
+      TdMmioWrite16 (Address, Value);
+    } else {
+      *(volatile UINT16 *)Address = Value;
+    }
+
     MemoryFence ();


The problem is that IsTdxGuest () has this structure:

BOOLEAN
EFIAPI
IsTdxGuest (
  VOID
  )
{
  if (mTdxProbed) {
    return mTdxEnabled;
  }

  mTdxEnabled = TdIsEnabled ();
  mTdxProbed  = TRUE;

  return mTdxEnabled;
}

Which is trying to cache the result of the probe in the efi data
segment.  However, that doesn't work in SEC, because the data segment
is read only (so the write seems to succeed but a read will always
return the original value), leading to us calling TdIsEnabled() check
for every mmio we do, which is causing the slowdown because it's very
expensive.

James