From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by mx.groups.io with SMTP id smtpd.web11.13906.1671610129317706041 for ; Wed, 21 Dec 2022 00:08:49 -0800 Authentication-Results: mx.groups.io; dkim=fail reason="unable to parse pub key" header.i=@intel.com header.s=intel header.b=TF/cbCHk; spf=pass (domain: intel.com, ip: 134.134.136.65, mailfrom: jiewen.yao@intel.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1671610129; x=1703146129; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=ohrbYYz4H+/vV6cUpYCnFd/nkNk14rcRh/8RClq1Bwc=; b=TF/cbCHkppjfqAMHFuqTz2h2LxmCoZj7fWZcrUnh/WrJLGIThXSHrEP0 DcjE+jyXiLh7vdVDMY/BMDUeKpPJgKuPAagre8Bdr81jv61auKwJHmWLj YA5Oe7wclT6xhIPVrj6RmIZErwuX0J+3OArILEIdtq/8OB/1EjwsXXqDL E1jI1jpHH6cDwdyT0XZ3fHqqWU/CKS+TUUF2yCeNbmb4AsHTU+sBrHeb3 ucchwsgG0QNXq8NqJDTAVGN+ZUX3knrQIWd+jyUvdTUsDjMXwg9CgVkMU 6qQTCE0S0FcHawLRaOoWebphfJfzfb/XBhzBtGLeecNPvpYtMxO08u+ON Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10567"; a="321725117" X-IronPort-AV: E=Sophos;i="5.96,262,1665471600"; d="scan'208";a="321725117" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Dec 2022 00:07:27 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10567"; a="740090999" X-IronPort-AV: E=Sophos;i="5.96,261,1665471600"; d="scan'208";a="740090999" Received: from fmsmsx601.amr.corp.intel.com ([10.18.126.81]) by FMSMGA003.fm.intel.com with ESMTP; 21 Dec 2022 00:07:27 -0800 Received: from fmsmsx601.amr.corp.intel.com (10.18.126.81) by fmsmsx601.amr.corp.intel.com (10.18.126.81) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.16; Wed, 21 Dec 2022 00:07:27 -0800 Received: from FMSEDG603.ED.cps.intel.com (10.1.192.133) by fmsmsx601.amr.corp.intel.com (10.18.126.81) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.16 via Frontend Transport; Wed, 21 Dec 2022 00:07:27 -0800 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (104.47.57.169) by edgegateway.intel.com (192.55.55.68) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.16; Wed, 21 Dec 2022 00:07:25 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=a3Vv/p1BcWpLY3GCFobLRV3Fgu1V1vOtFcRgUTrtGsv2u6AruCawKVtcMIl9ETzbQSRvjMVymMJZ00I2OO9I5/JFgmhBZjb7+ScAZmqip573vpDw9mSmIyw5PhcJTscNWKb/CshFxvEWvW2GpWcHavdspT3f8K/ywyWB8ln02eO5M3K6rJg4uwM8WTbnSsiSHTlrzp964l1xhCTDIGgegiSH0lQ+CSGqXAcxImPeLSZ3sFK4FhXomA1EsfEJRFTYlq0+Yf/ofG9pPblxnlaW6zpdBp0AS/VWidtVVcbng19slIrOHuc33db91C/PMCvV6lnnXh0jrRTKC4xEwDvwBQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=oVItqYQFRRtNCYJ7hfPzW7IQA01AIXD2hdlgYLz4/u0=; b=Y5xT6qB4ZrdNzSnTU+pJaPKVdJk0jWPl708Pfs1hGRODLQbkonBJ6w1K+dPfvP7J0pn/De8btCa3Ty5lCJYFNrwpf2rkrY42eGvVOX+4oP7qtq4Q8Sn1jGG36Cl09T1SZcJharTuGqSN7L54BaKBi9r4NKzlUjprQsKCjek5YtN9Jp6qPNhkJEiT1wTxDtcbArMPRGhQ6nQrA9SkV5nEB0VVV/MSR/HI9pj3XiIPbNIgmibinflH4o853MRAeMwqa/p1DpN4KE3mlIw9afIgItgrAMoSPMnYHA6g8jcP/DkikX8qzTcrtiZGEHQd46z9NzN/1pNrli9u8vUXj9DZ8Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from MW4PR11MB5872.namprd11.prod.outlook.com (2603:10b6:303:169::14) by BL3PR11MB6409.namprd11.prod.outlook.com (2603:10b6:208:3b8::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5924.16; Wed, 21 Dec 2022 08:07:20 +0000 Received: from MW4PR11MB5872.namprd11.prod.outlook.com ([fe80::5f56:1bdc:2eae:c041]) by MW4PR11MB5872.namprd11.prod.outlook.com ([fe80::5f56:1bdc:2eae:c041%8]) with mapi id 15.20.5924.016; Wed, 21 Dec 2022 08:07:20 +0000 From: "Yao, Jiewen" To: "Xu, Min M" , "devel@edk2.groups.io" CC: "Aktas, Erdem" , Gerd Hoffmann , James Bottomley , Tom Lendacky Subject: Re: [PATCH V2 0/6] Enable Multi-core based lazy-accept for TDVF Thread-Topic: [PATCH V2 0/6] Enable Multi-core based lazy-accept for TDVF Thread-Index: AQHZFE8SZ+jbAwfU4kO1qmy/x1gqOK53/cgQ Date: Wed, 21 Dec 2022 08:07:20 +0000 Message-ID: References: <20221220084240.1922-1-min.m.xu@intel.com> In-Reply-To: <20221220084240.1922-1-min.m.xu@intel.com> Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: MW4PR11MB5872:EE_|BL3PR11MB6409:EE_ x-ms-office365-filtering-correlation-id: c4f0cb03-734c-4024-ab7d-08dae32a6278 x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: hr9pSpHbetK+Ov5ulwBzp2USE0KLItkI5QgMdxgipVkhCo71jcAwaHTgyAtSkLulO2/JPMiys4duN/dSp8dTT8ZsZU6+zDiNOsAsEFs2tqwXcFZEc0yZE94LEl8pEkJsiU4pDHOe3zNot1aN3GKJYnEsyEsfEDkk2+s9YIzqGE7WOc7cykSe+ukcBmjuaNZ4Y9P08n+PHmfhIJxxTH9PvR4x6/+1obors3gfMMXVf29ZUqo3ZyXaE/Tfqt0CJTWFhmPUGn5YsNCqxLIJ08fCrAIy73BGVJzOMsvs+INBB4kfkmxTlOb0VXL3RFM0SwG3gq5mbdgflyFLscC8Qc9Fzprbak0wQECfwkjN1NdtUZ5hteWibjg+21gTnPB9Qa6oIs2U2PewGTkbjHd1l/ZlfoGRhgbA9kA26nfwccApgRDPjd1HejEF8Hf4HpySCkdzVXKz0Djy3xS8ZocTbuGIxagMN1T+2KE2kNAG3IYqrjzfYoIeD09QUBXPc+tLvg3ImIuAokfKY8YxXVWZ50R47FD1TTAY8XB5cA9+Yn3wtegz/ma8oyPmJNlKcN8h7T10zljIwh3Q/va4LQZKu239tXJjOCzbfIlQsIzdUIL1yj0er03cneDpCWvfUjaEvSBdlIG0l/mOVBOmN7HL3J/pwy0CIsIemKgBQc7HF636Wr/Z0cl/hDjuKNIP/HSOhmtXb+lIM/1Oz5FOPfoO8HAVYpdP4OQJrIBznPi1AlfeDJ7SrWC4zVOrMog2HzHqeEzaOEy5c2EYiHOLyh7NItbBmQ== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MW4PR11MB5872.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230022)(346002)(39860400002)(136003)(396003)(366004)(376002)(451199015)(55016003)(86362001)(33656002)(966005)(71200400001)(478600001)(186003)(26005)(6506007)(53546011)(66476007)(66946007)(5660300002)(8676002)(66446008)(66556008)(4326008)(64756008)(316002)(8936002)(52536014)(2906002)(41300700001)(54906003)(19627235002)(110136005)(7696005)(76116006)(122000001)(38070700005)(82960400001)(38100700002)(9686003)(83380400001);DIR:OUT;SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?hIazOsaQeZF2MX9IpeI0RnxkPzX8LLng/5+MCGdxvnAsMUMjn5ANKB649VdD?= =?us-ascii?Q?WQTt7lSpaPzsY9rdGQpO+a97S8AsjZYqADfQHepcU5iVPvUqdheuvip9TpIH?= =?us-ascii?Q?XXflH69GxACRuQsFCUFJbDIgAicU+jkK4eZaOgqiseOCs1YgKPiVEqRzOGkq?= =?us-ascii?Q?h+3O6zcIRDFZRwLtfv8AFlKhHRLhsi50LZScRlTPSn7VMOXG3YGJngjqhKol?= =?us-ascii?Q?aFa02zbO6nHXIAE5YsDCObyLILMdaTquOU2RAD++rg7ZH97NChjeDcdQj3at?= =?us-ascii?Q?X0kzwMSUmaFZnqOCwnbZLAjiWqfm7QwxQUvkRrmIxrua0Zn6YQMc1C70DGwg?= =?us-ascii?Q?O1fVeEu1hA4i0AxAbJc3qF07HdA6CB2rKj4EeJIFosBJS8USBU0i/fLn7vtc?= =?us-ascii?Q?V2ut1Guw7CA3O9nLvQv4g1whFZTBLSydNtQUUTk6ygJB/L3PxvD26tNakVQv?= =?us-ascii?Q?3LUP4kw3PsHnR7SskUo2XkHMJEVdnAPN8fplywsbQvaVTbEy4VYkZ2CdQ/GT?= =?us-ascii?Q?RarltVOQ7mARYNDLOaCEEQjcvQIbYuv8ZEQc3bed8kGQepe/7QeRqbBcP748?= =?us-ascii?Q?L9okKvyRk0n9F2VBtBJaWpBgttBb1zOOXcIWd8M2L/GixDmGDTu3J6vQvMMo?= =?us-ascii?Q?bxFRxCpSqM6YV2O4k3dN5+D/SVdY4y2yWRmN4d0p6p+AhYvOueTOx+9qonJa?= =?us-ascii?Q?6FdhoVGNag9RSeZbnf4uzwcV/jvMTsR+ZUvgpGZlBHPFuiUHXkjoVOtWJPxO?= =?us-ascii?Q?AWwUD/wdaOdSFXZwMiw+3ruUzJamuA1aTJS2/Mo5ZiQ9cqgGECorftfnbB2/?= =?us-ascii?Q?RKHY4bZEROiSOboW14o1BbcSfPP3FHn1kzWxcSgN8DK/XcRVvI19focTbM07?= =?us-ascii?Q?9H3WsLrr4tcCnfXTupiVQyOI6QoXpG/7+W/AY8Gvs4BvgDM6BulClif9r/Xu?= =?us-ascii?Q?/3gPkR0/2M0qKbWcBqMgClEitZN5CPIpGPpLDboCnljm5nFk4ZpRU+pW0fFB?= =?us-ascii?Q?2Qd8hvyCPe/XbvfdFxMh/Yf2iuBvV00iH4WwfizqsVprhF5tn5fop/eqqYnf?= =?us-ascii?Q?KYZHKrdTWzaEJI/tBGrV/AOySV//k1EPy9xnInzT9+3aziX3gHH659p2K9u5?= =?us-ascii?Q?cC5dqyTCwpWLsrpK1G8XmP7ILjhS4MbzU1k1NujwlahXUEHpUF7yFZFmLrPa?= =?us-ascii?Q?AbMFtJ3zJDxqPKtGGZ3n+ztNFDn+xbygl5xZbH8eMsinFZoZffbLq6aPtBwY?= =?us-ascii?Q?pzmT8bpM5XLNrHTd80ajzstPJaeE8Pa9Ye1rhSE0Vab1ezP3yDo/8J9iX9Ei?= =?us-ascii?Q?z27yTdWQz03oCCzJCMulKL2/hegQUjAZ1KN0hWUW4I+dzqhXVIvJof1H2DeO?= =?us-ascii?Q?WJvNsgmIpDI2IIX1/ImkGNw1bRgZeCosDC6wt67NF3OYY+IBa8/gqT5FDLUy?= =?us-ascii?Q?MXhiYrgAX6n4pL/EnOvRZZiMtOSZH3UpIHKb6eF3Hbif/SR7PgklGOWfqwVe?= =?us-ascii?Q?OEsWF5aVotXl6q8YKkRKteiCXcdmXLJRtY2T78rsT/Idw9vTDuF5dsJ9MlTZ?= =?us-ascii?Q?KnCMZ6UP4m8X7ko2AtcKnKNWCEtj547YPrBlSX/Y?= MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MW4PR11MB5872.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: c4f0cb03-734c-4024-ab7d-08dae32a6278 X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Dec 2022 08:07:20.6397 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: nmBLeWkDOWv01eU0X5EphhB/aSLuAdFxgIHAxkff/nVEOe6VHLu9PiQyCZ/j8+J77oEEtXqdVQTK55hoQvxPyw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL3PR11MB6409 Return-Path: jiewen.yao@intel.com X-OriginatorOrg: intel.com Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Reviewed-by: Jiewen Yao Merged https://github.com/tianocore/edk2/pull/3813 > -----Original Message----- > From: Xu, Min M > Sent: Tuesday, December 20, 2022 4:43 PM > To: devel@edk2.groups.io > Cc: Xu, Min M ; Aktas, Erdem > ; Gerd Hoffmann ; James > Bottomley ; Yao, Jiewen ; Tom > Lendacky > Subject: [PATCH V2 0/6] Enable Multi-core based lazy-accept for TDVF >=20 > BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3D4172 >=20 > TDVF once accepts memory only by BSP. To improve the boot performance > this patch-set introduce the multi-core accpet memory. Multi-core means > BSP and APs work together to accept memory. >=20 > TDVF leverages mailbox to wake up APs. It is not enabled in MpInitLib > (Which requires SIPI). So multi-core accept memory cannot leverages > MpInitLib to coordinate BSP and APs to work together. >=20 > So TDVF split the accept memory into 2 phases. > - AcceptMemoryForAPsStack: > BSP accepts a small piece of memory which is then used by APs to setup > stack. We assign a 16KB stack for each AP. So a td-guest with 256 vCPU > requires 255*16KB =3D 4080KB. > - AcceptMemory: > After above small piece of memory is accepted, BSP commands APs to > accept memory by sending AcceptPages command in td-mailbox. Together > with the command and accpet-function, the APsStack address is send > as well. APs then set the stack and jump to accept-function to accept > memory. >=20 > Patch 1-3: > TdxMailboxLib is refactored to help multi-core lazy accept. > Patch 4-5: > Refactor APs' logical to support accpet page. > Patch 6: > Implement multi-core accept in TDVF. >=20 > Code: https://github.com/mxu9/edk2/tree/Multi-Core-Lazy-Accept.v2 >=20 > v2 changes: > - Fix an override error when vCPU number is larger than 64. >=20 > Cc: Erdem Aktas > Cc: Gerd Hoffmann > Cc: James Bottomley > Cc: Jiewen Yao > Cc: Tom Lendacky > Signed-off-by: Min Xu >=20 > Min M Xu (6): > OvmfPkg/TdxMailboxLib: Delete global variables > OvmfPkg/TdxMailboxLib: Add NULL instance of TdxMailboxLib > OvmfPkg: Add TdxMailboxLibNull in some platform dsc > OvmfPkg/Sec: Move TDX APs related nasm code to IntelTdxAPs.nasm > OvmfPkg: Enable APs to accept memory for TDVF > OvmfPkg/PlatformInitLib: Implement multi-core accept memory for TDVF >=20 > OvmfPkg/AmdSev/AmdSevX64.dsc | 1 + > OvmfPkg/CloudHv/CloudHvX64.dsc | 1 + > OvmfPkg/Include/TdxCommondefs.inc | 4 +- > OvmfPkg/IntelTdx/Sec/X64/IntelTdxAPs.nasm | 119 +++++ > OvmfPkg/IntelTdx/Sec/X64/SecEntry.nasm | 58 +-- > OvmfPkg/Library/PlatformInitLib/IntelTdx.c | 411 +++++++++++++++--- > .../PlatformInitLib/PlatformInitLib.inf | 1 + > OvmfPkg/Library/TdxMailboxLib/TdxMailbox.c | 22 +- > .../Library/TdxMailboxLib/TdxMailboxLib.inf | 7 +- > .../TdxMailboxLib/TdxMailboxLibNull.inf | 34 ++ > OvmfPkg/Microvm/MicrovmX64.dsc | 1 + > OvmfPkg/OvmfPkgIa32.dsc | 1 + > OvmfPkg/OvmfPkgIa32X64.dsc | 2 +- > OvmfPkg/Sec/X64/SecEntry.nasm | 58 +-- > 14 files changed, 545 insertions(+), 175 deletions(-) > create mode 100644 OvmfPkg/IntelTdx/Sec/X64/IntelTdxAPs.nasm > create mode 100644 OvmfPkg/Library/TdxMailboxLib/TdxMailboxLibNull.inf >=20 > -- > 2.29.2.windows.2