From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by mx.groups.io with SMTP id smtpd.web12.6347.1597309938472555584 for ; Thu, 13 Aug 2020 02:12:18 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@intel.onmicrosoft.com header.s=selector2-intel-onmicrosoft-com header.b=QuwYaDzO; spf=pass (domain: intel.com, ip: 192.55.52.93, mailfrom: liming.gao@intel.com) IronPort-SDR: X5p2eqktuaYDrI9Ljr1S0f6ngIYj21C3byjVtq+bLv8h8qJOHifjSKFb2yRYDgvvXJfdR7BXxj +m/++Gp1/iKA== X-IronPort-AV: E=McAfee;i="6000,8403,9711"; a="151832059" X-IronPort-AV: E=Sophos;i="5.76,307,1592895600"; d="scan'208";a="151832059" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Aug 2020 02:12:17 -0700 IronPort-SDR: XYvXfCnFMnjZi4zpP7/pXVMcY+WQV/ph2Tt5/bghudLbAVXDHcoWdCtT0vWcKfCPyamAHazo2v RCYPMWwTn4Gg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,307,1592895600"; d="scan'208";a="276800325" Received: from fmsmsx603-2.cps.intel.com (HELO fmsmsx603.amr.corp.intel.com) ([10.18.84.213]) by fmsmga007.fm.intel.com with ESMTP; 13 Aug 2020 02:12:17 -0700 Received: from fmsmsx606.amr.corp.intel.com (10.18.126.86) by fmsmsx603.amr.corp.intel.com (10.18.126.83) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Thu, 13 Aug 2020 02:12:17 -0700 Received: from FMSEDG001.ED.cps.intel.com (10.1.192.133) by fmsmsx606.amr.corp.intel.com (10.18.126.86) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.1713.5 via Frontend Transport; Thu, 13 Aug 2020 02:12:16 -0700 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (104.47.57.177) by edgegateway.intel.com (192.55.55.68) with Microsoft SMTP Server (TLS) id 14.3.439.0; Thu, 13 Aug 2020 02:12:16 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=hhb7qoqK5OZpPSIc+ja4Qa6bnXz2PgcVmJ2Q5YFJ89pD43z0CtIXtK4QTBTQfhdWEsLtITT+/qGekU6U13RQEDOWJEr9PNrkZBlFPfale0JhvYrLp8p3OMRHQ2VACzFeODagsWmFZhjJagOOOL45IbwiEl6AOu1FJhufppLhp/c2m1ojp/TrPWAGDc15ONw1VlrWTnNADA9Ul1blMaBl1KtRR7OwAiWBEHrKsOjZ5I0g0DjHwNJwyEFIFvy8+WPHABaVMxlfZ2ttHq2ZIcmRAvaqbGNZnkhudRAuJkKaw2PCGg0FHjGl16yb1a1KrKcULrs3HZuJvMkIs/yPsFTj8A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=YI+hW/4lVJui5TDISLDl6WewNVkQxISavQBiSDDLaEg=; b=YF2wttL/atJYm8PzXKUeuCg5VtczXklcaHKKQ97OjgiriLqzwAP761zRPBBDtaY4yH4bsA62KqgsCwHgxE7tvM+Sf1Pxuw46526HOY10qdGnuLpXebCRlzwYF5yRO5iXVYJBHFUMfdSZxIXAiuGS/KccYVOjQ+y1ilz+JnztjUzLnmKuNl63+lssNiFuTrKDpbzV+Bia6WdhaJd2L55yjx9Q3VaDCB6z22+87Yl3OXSlM5LlwJ/I1v9BpIFAM1Yo+JBJuGxMsQziWVtB+UxDWSGTnR9BEYXuovD0jn29v6j3CrKze0XS+9sMTELsX88eNJ0j6RyJ/FLZ201b8GgLTw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=YI+hW/4lVJui5TDISLDl6WewNVkQxISavQBiSDDLaEg=; b=QuwYaDzO8Mh+hBMRBGbnwgG7UaB9GMUTf2jCDVgk6DIEvpP8opSUcw3DlHXSbF/OPLkM0T9oMgrfcH/5lyZTCzIlQxQVbby1atu4R99skdlsUt+ts9Yg0Ga1IKCSSqEWNXQ1yBPjbXy+FLKtPrNXYeo1c1FcbFzyAlYTqXezMyA= Received: from CY4PR11MB1526.namprd11.prod.outlook.com (2603:10b6:910:7::10) by CY4PR11MB1944.namprd11.prod.outlook.com (2603:10b6:903:121::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3261.19; Thu, 13 Aug 2020 09:12:15 +0000 Received: from CY4PR11MB1526.namprd11.prod.outlook.com ([fe80::f8c4:711:84e3:4c97]) by CY4PR11MB1526.namprd11.prod.outlook.com ([fe80::f8c4:711:84e3:4c97%4]) with mapi id 15.20.3261.025; Thu, 13 Aug 2020 09:12:15 +0000 From: "Liming Gao" To: "Feng, Bob C" , "devel@edk2.groups.io" CC: "Chen, Christine" Subject: Re: [Patch] BaseTools: Move CreateAsBuiltInf into AutoGenWorker for parallel Thread-Topic: [Patch] BaseTools: Move CreateAsBuiltInf into AutoGenWorker for parallel Thread-Index: AQHWcG8bUEq+XXoFukqWpvRxiYUTtak0ipawgAE0yqCAAANBIA== Date: Thu, 13 Aug 2020 09:12:14 +0000 Message-ID: References: <20200812060903.23704-1-bob.c.feng@intel.com> In-Reply-To: Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-version: 11.5.1.3 dlp-product: dlpe-windows dlp-reaction: no-action authentication-results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=intel.com; x-originating-ip: [192.198.147.194] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 4e2811da-adce-4b52-6157-08d83f68f87b x-ms-traffictypediagnostic: CY4PR11MB1944: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:6790; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: oWk+ZcU9S2syKD4aiq0Xvn9Oe9YDjUKPKHI4OKbDvv5b62Qq6J47M/0c+CP26Isng39FvK3yHvIvsonmgPLofZE/RK063lSvZqo+X/jIvTpmBw1VsB4s/7Dstn1HORvzt2/Hi6Gd9uZ0VxR5YKwtDMboPojtbSG5Fnp50OwQt/w5LfDgJAzvKMDzLaJJekTKgci68m+V5xBNrrJ6XUZFT8G1Dzwwl5p8T/nwCwvFUKrzvoq0061F2+oJacwCS2DrOwFmGiGkpJwKUPipt8IHJeMfVnblLj85Bm+7TIqEB3XCfqKg+AXSlExr6yEhU38m x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CY4PR11MB1526.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFTY:;SFS:(6029001)(4636009)(396003)(39860400002)(346002)(376002)(136003)(366004)(4326008)(186003)(55016002)(26005)(33656002)(316002)(9686003)(2906002)(53546011)(110136005)(86362001)(6506007)(8936002)(5660300002)(66446008)(66556008)(64756008)(66476007)(71200400001)(52536014)(478600001)(83380400001)(66946007)(7696005)(107886003)(76116006)(8676002);DIR:OUT;SFP:1102; x-ms-exchange-antispam-messagedata: jq+9Q2ldgHaJDOHRlIC6C3QeuKGfUpV42p5MomwOEudZbEDTAJrSJOWHRCnBNvhBOS4CFZb4TENlKl5++t6+o/qW8Z4dmBzZcLCS+5vt5b+5s26OE+zmHAqycTqOlQYvrXSyALnxd/G8EVXVy6Bx283Xgo4ZYOx2nmZMyuJzQ3VrdU7qXidD1a9EPlcaW/9KqpKe3HzFweBdIbbUJ9qm1P/2cPRQv2DeRmWBqrgz8p5lOkOWxc/oGwLENCRWda7bzqmBVZcZJQVRBKwVXcvHc4WALbcuJ/2nueDukzDNKWFJTSw27fDT31JjZbgOzma0IDJ6hv+2pNTlSGLMs/KVXkcef/AoegrDljSFzNjCyLwpmCiXGB2xcXErnM/KIcYCCXaKMUHI8yU56SVFQp3uhvPRIRSmkFauslnKNarRElbvvYi0XT+U0IfOKTTrTBC5IQk/+5HHOgtijqdXsf7OR4OISHvQT3FUmxtFzn7WB+8hDAqG8LMxc2T9uVuKafbBA2bXNDaCU1oy4GivRvXGZsDRHDLLoGXVPgvuSPTptp/+KRcH6WRFj2SnAOV7nmtAo/LqpxF4UNz958SOLYdGHzexWsyJZXJK4d372ofejLvVhOYnfy6budigWBYql/+QTfVZqdsaCanO8/e2vzi5Iw== MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: CY4PR11MB1526.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4e2811da-adce-4b52-6157-08d83f68f87b X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Aug 2020 09:12:14.9579 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: jdmVY2yB0cI3nX5c0TvCrcKTjlPg282kDgMBEhQRG5nCzSGNRQHitCSg5vJcmi5Y0VRyupT0sThOpxJg3W5PPw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR11MB1944 Return-Path: liming.gao@intel.com X-OriginatorOrg: intel.com Content-Language: en-US Content-Type: text/plain; charset="iso-2022-jp" Content-Transfer-Encoding: quoted-printable A little better is also OK. Reviewed-by: Liming Gao -----Original Message----- From: Feng, Bob C =20 Sent: 2020=1B$BG/=1B(B8=1B$B7n=1B(B13=1B$BF|=1B(B 17:01 To: Gao, Liming ; devel@edk2.groups.io Cc: Chen, Christine Subject: RE: [Patch] BaseTools: Move CreateAsBuiltInf into AutoGenWorker fo= r parallel Liming, I did the test on Ovmf, there is some improvement but not much. Thanks, Bob -----Original Message----- From: Gao, Liming Sent: Wednesday, August 12, 2020 10:35 PM To: Feng, Bob C ; devel@edk2.groups.io Cc: Chen, Christine Subject: RE: [Patch] BaseTools: Move CreateAsBuiltInf into AutoGenWorker fo= r parallel I like this idea. Is there any build performance improvement? > -----Original Message----- > From: Feng, Bob C > Sent: Wednesday, August 12, 2020 2:09 PM > To: devel@edk2.groups.io > Cc: Gao, Liming ; Chen, Christine=20 > > Subject: [Patch] BaseTools: Move CreateAsBuiltInf into AutoGenWorker=20 > for parallel >=20 > AsBuiltInf can be created during AutoGen phase. Move CreateAsBuiltInf=20 > into AutoGenWorker to make this step run in parallel. >=20 > Signed-off-by: Bob Feng > Cc: Liming Gao > Cc: Yuwei Chen > --- > BaseTools/Source/Python/AutoGen/AutoGenWorker.py | 2 +- > BaseTools/Source/Python/build/build.py | 4 +--- > 2 files changed, 2 insertions(+), 4 deletions(-) >=20 > diff --git a/BaseTools/Source/Python/AutoGen/AutoGenWorker.py > b/BaseTools/Source/Python/AutoGen/AutoGenWorker.py > index 017f676399..0425f1ab0b 100755 > --- a/BaseTools/Source/Python/AutoGen/AutoGenWorker.py > +++ b/BaseTools/Source/Python/AutoGen/AutoGenWorker.py > @@ -278,11 +278,11 @@ class AutoGenWorkerInProcess(mp.Process): > else: > self.cache_q.put((Ma.MetaFile.Path, Ma.Arch,=20 > "PreMakeCache", False)) >=20 > Ma.CreateCodeFile(False) > =20 > Ma.CreateMakeFile(False,GenFfsList=3DFfsCmd.get((Ma.MetaFile.Path, > Ma.Arch),[])) > - > + Ma.CreateAsBuiltInf() > if GlobalData.gBinCacheSource and CommandTarget in [None= , "", "all"]: > try: > CacheResult =3D Ma.CanSkipbyMakeCache() > except: > CacheResult =3D False diff --git=20 > a/BaseTools/Source/Python/build/build.py > b/BaseTools/Source/Python/build/build.py > index 59ceacfed0..c4cfe38ad9 100755 > --- a/BaseTools/Source/Python/build/build.py > +++ b/BaseTools/Source/Python/build/build.py > @@ -871,11 +871,11 @@ class Build(): > if GlobalData.gUseHashCache and not GlobalData.gBinC= acheDest and self.Target in [None, "", "all"]: > cqueue.put((PcdMa.MetaFile.Path, PcdMa.Arch,=20 > "PreMakeCache", False)) >=20 > PcdMa.CreateCodeFile(False) > PcdMa.CreateMakeFile(False,GenFfsList =3D=20 > DataPipe.Get("FfsCommand").get((PcdMa.MetaFile.Path, PcdMa.Arch),[])) > - > + PcdMa.CreateAsBuiltInf() > # Force cache miss for PCD driver > if GlobalData.gBinCacheSource and self.Target in [No= ne, "", "all"]: > cqueue.put((PcdMa.MetaFile.Path, PcdMa.Arch,=20 > "MakeCache", False)) >=20 > self.AutoGenMgr.join() > @@ -1263,11 +1263,10 @@ class Build(): >=20 > # build modules > if BuildModule: > BuildCommand =3D BuildCommand + [Target] > LaunchCommand(BuildCommand, AutoGenObject.MakeFileDir) > - self.CreateAsBuiltInf() > if GlobalData.gBinCacheDest: > self.GenDestCache() > elif GlobalData.gUseHashCache and not GlobalData.gBinCacheSo= urce: > # Only for --hash > # Update PreMakeCacheChain files @@ -2274,11 +2273,10=20 > @@ class Build(): > # All modules have been put in build tasks queue. Tell t= ask scheduler > # to exit if all tasks are completed > # > ExitFlag.set() > BuildTask.WaitForComplete() > - self.CreateAsBuiltInf() > if GlobalData.gBinCacheDest: > self.GenDestCache() > elif GlobalData.gUseHashCache and not GlobalData.gBinCac= heSource: > # Only for --hash > # Update PreMakeCacheChain files > -- > 2.20.1.windows.1