From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by mx.groups.io with SMTP id smtpd.web11.16161.1597242910692971809 for ; Wed, 12 Aug 2020 07:35:10 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@intel.onmicrosoft.com header.s=selector2-intel-onmicrosoft-com header.b=orLa/QVb; spf=pass (domain: intel.com, ip: 134.134.136.31, mailfrom: liming.gao@intel.com) IronPort-SDR: vi81KfTd1iAkTpHd9Ir8zdfpfy6IVKOCNJyxNLSGPmmVFV3zfotB2hST3QIs69L1T4Ofpi7MzF YMXDC0u6PqUA== X-IronPort-AV: E=McAfee;i="6000,8403,9711"; a="215489560" X-IronPort-AV: E=Sophos;i="5.76,304,1592895600"; d="scan'208";a="215489560" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Aug 2020 07:35:10 -0700 IronPort-SDR: 2D2R3BaX5gJd+F0newZWkYQMbt0MuRif71dDJSieGi/ImCH+hxZWLrJ/0YIkz3Jmj/Aw0S6T9S WVhr+EpNUyqw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,304,1592895600"; d="scan'208";a="318123282" Received: from fmsmsx603-2.cps.intel.com (HELO fmsmsx603.amr.corp.intel.com) ([10.18.84.213]) by fmsmga004.fm.intel.com with ESMTP; 12 Aug 2020 07:35:09 -0700 Received: from fmsmsx603.amr.corp.intel.com (10.18.126.83) by fmsmsx603.amr.corp.intel.com (10.18.126.83) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Wed, 12 Aug 2020 07:35:09 -0700 Received: from fmsmsx105.amr.corp.intel.com (10.18.124.203) by fmsmsx603.amr.corp.intel.com (10.18.126.83) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.1713.5 via Frontend Transport; Wed, 12 Aug 2020 07:35:09 -0700 Received: from fmsedg601.ED.cps.intel.com (10.1.192.135) by FMSMSX105.amr.corp.intel.com (10.18.124.203) with Microsoft SMTP Server (TLS) id 14.3.439.0; Wed, 12 Aug 2020 07:35:09 -0700 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (104.47.59.171) by edgegateway.intel.com (192.55.55.70) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.1713.5; Wed, 12 Aug 2020 07:35:09 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=c9yrDbQat+vyIf//g3kLqQ4xXAOumxSGnfs1qBH7yuCc5/SAyw+FmpSsLzA2gsqdyxe58RAXzhF5hNL9MGip6CPIhNtkerFyNjfsqqpDLdsEA5AxzTN8gbrpceoWOEb8Xfd7DVAnQObpKVbICg0TTYGbaJl+DDwaQ1wLipukNhSRANiunhUS44q5/EHsIFebApsah6iN3uGuBvhcl3t1jKIH54yKAsRrLwuZXjkLperysN/IAfoJfl7W+Zqxr5ytfTvtJDdKWiZZFBrAF7YpaYLfY7mou5Ey103r7ZYsBnmLOmmxQFuXP4nfTEOjUQfqSKCQEFwBdsHkHWLJWXGYYA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=G+vfpKQde88FOxjHPSs7SOqD9/hwFVEUlXlO3j5Zh+U=; b=NO/OFeBuZSEZtJkmq4uhestxx5o9nKmdFjjxWXgWuUlM5RxLhGOwlXM1kwrRxzZMCErvP5Kgg/31wbGcBfGa/nf6DA7JIkdbyxIs8hGgd7ArYFTWzLRj1ppavxPaIcKJKBTQhsnGUjQMgFFgXuYBMfdUTNbXTvdQMvp8meSPrDlLtoPIOqYJmSm+T1rxXzPc5Col+4Ql5+AOIlfqwez4w78IqCba4Hz0FgRj9+8JN82+2RapF4KN8IGLsKxrHaQVF3T74OQXebdtaCd9b1v928F1yHdnS/ghGZwZ6PMHVZI4Wi7UCrve9YPLwlgS2lJ1Irh1qLPPJnRSV0sxM0orDA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=G+vfpKQde88FOxjHPSs7SOqD9/hwFVEUlXlO3j5Zh+U=; b=orLa/QVbuZxmB3mk3s6CrLnj65O47T+Io/OYBq14kdaZsrv2kt2g25Odz0pPK4/nhhZtLXn9d2iVtEKCAD57FH+2kQiLcc463xXqQgxDTkp37tm1Hi1qK47TU2hY6s5+qyy4YpkeBq3tjZ/hbyENakfL1W/4F7JCfwzvLU7Fq1Q= Received: from CY4PR11MB1526.namprd11.prod.outlook.com (2603:10b6:910:7::10) by CY4PR11MB1350.namprd11.prod.outlook.com (2603:10b6:903:2e::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3261.16; Wed, 12 Aug 2020 14:35:08 +0000 Received: from CY4PR11MB1526.namprd11.prod.outlook.com ([fe80::f8c4:711:84e3:4c97]) by CY4PR11MB1526.namprd11.prod.outlook.com ([fe80::f8c4:711:84e3:4c97%4]) with mapi id 15.20.3261.025; Wed, 12 Aug 2020 14:35:08 +0000 From: "Liming Gao" To: "Feng, Bob C" , "devel@edk2.groups.io" CC: "Chen, Christine" Subject: Re: [Patch] BaseTools: Move CreateAsBuiltInf into AutoGenWorker for parallel Thread-Topic: [Patch] BaseTools: Move CreateAsBuiltInf into AutoGenWorker for parallel Thread-Index: AQHWcG8bUEq+XXoFukqWpvRxiYUTtak0ipaw Date: Wed, 12 Aug 2020 14:35:07 +0000 Message-ID: References: <20200812060903.23704-1-bob.c.feng@intel.com> In-Reply-To: <20200812060903.23704-1-bob.c.feng@intel.com> Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-version: 11.5.1.3 dlp-product: dlpe-windows dlp-reaction: no-action authentication-results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=intel.com; x-originating-ip: [192.198.147.222] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 51f6f384-811b-48f9-a748-08d83ecce951 x-ms-traffictypediagnostic: CY4PR11MB1350: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:2803; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: yR4HxU/Wn3mkfCUa8kZx84ofaMid9F+RMXJq0nuTOPrfxH39JK6oP1fp9XJ108G3lHt5VowvqVCNiYs6PJBL3B1/ChwiszILkYVWCfyaZp5qE5kyZdeF7DDbAen9YTxfGe17R110dS3CE3QEeyn05STS8hxEW8lCGMj99P9fulaXW7xhFHe9xQFHIpCNmQ7OtS/DR1r6XIAWy0vA8iR87NLgIBmM02ewDNLpKhPtVx+DyXk4RlF8Eevt7WfIe3i5KsDpP1Zusfq+fu/ugxxq5VA5ppJ8krvC08yvKGa3op3s9NYagEWxOZvb0enGayOu x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CY4PR11MB1526.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFTY:;SFS:(6029001)(4636009)(396003)(366004)(136003)(39860400002)(376002)(346002)(55016002)(9686003)(52536014)(53546011)(33656002)(6506007)(8936002)(8676002)(7696005)(66476007)(316002)(110136005)(66446008)(86362001)(64756008)(2906002)(5660300002)(71200400001)(478600001)(76116006)(66556008)(107886003)(66946007)(186003)(4326008)(26005)(83380400001);DIR:OUT;SFP:1102; x-ms-exchange-antispam-messagedata: SevF9AblGUhKRyO9Q+0Oq8QaWPj1vRuwnaGLYxCpqNJS+BdegtwmFwRRy9822lbFXGYwN/Y+dDlBGDbfFHHNRbbamLM9DTJABYHh/4K++tXxMQLihuk0Iry/61F+5Tcm7Xgjoig24yGdXLlSi4BdOIoAGwwtJKuPndlvcpX2qA2D6emAMJRg34S8rE8QLVF+4QPEUettBURCuLjNAzysIFlD3pG3aUEql5ASWcCeV8QGom/dUgxrBDRRm7A0AmpaoI8UWuLaSQ3AtoYtQihBAKz7b9kgLc/PHEeqEVkF+06WmeVxNEIrJBgfg09PcYSXfM3AFUYpfSnbC5Pp1N4CRjYKcXtAvCOLn+WF+utflkjPrz/4G2q5lEG6FRMMNjvHMl33ICa8MLM8SnIu/vG7ZkH0CTMwhcxg8JvSr56daDW/1ExFfZXA3N8gRjyGi3Bk1v59NHhTtodz85+vYt2SjvDpDlN3mlOCqq1D933qbOsuqk0d1g+fZmPo3Mx1xgI+AMLsNjEzTEYerruPHGJ23u6F9nLflsYgDWr6x5zdtBnqUoW9PMvJIF/R5yGC/BVMnuOUD60oxa2B4DkpB+PD2iQaC84CPlgyK2fcAknaBA3qYTXKo92xrwP3caOvc4AMsuJMZW8K0BwC/Klkqka3yg== MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: CY4PR11MB1526.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 51f6f384-811b-48f9-a748-08d83ecce951 X-MS-Exchange-CrossTenant-originalarrivaltime: 12 Aug 2020 14:35:08.0238 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: Q59bmjPyR5zEui9QRKlusY+KE3EIeVrHbEOQ5WHK9SnS4HvTD+Y8lEomtui6B/l6UFjPE/rWBdVTBJRr3HcQXA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR11MB1350 Return-Path: liming.gao@intel.com X-OriginatorOrg: intel.com Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable I like this idea. Is there any build performance improvement? > -----Original Message----- > From: Feng, Bob C > Sent: Wednesday, August 12, 2020 2:09 PM > To: devel@edk2.groups.io > Cc: Gao, Liming ; Chen, Christine > Subject: [Patch] BaseTools: Move CreateAsBuiltInf into AutoGenWorker for = parallel >=20 > AsBuiltInf can be created during AutoGen phase. Move CreateAsBuiltInf > into AutoGenWorker to make this step run in parallel. >=20 > Signed-off-by: Bob Feng > Cc: Liming Gao > Cc: Yuwei Chen > --- > BaseTools/Source/Python/AutoGen/AutoGenWorker.py | 2 +- > BaseTools/Source/Python/build/build.py | 4 +--- > 2 files changed, 2 insertions(+), 4 deletions(-) >=20 > diff --git a/BaseTools/Source/Python/AutoGen/AutoGenWorker.py b/BaseTools= /Source/Python/AutoGen/AutoGenWorker.py > index 017f676399..0425f1ab0b 100755 > --- a/BaseTools/Source/Python/AutoGen/AutoGenWorker.py > +++ b/BaseTools/Source/Python/AutoGen/AutoGenWorker.py > @@ -278,11 +278,11 @@ class AutoGenWorkerInProcess(mp.Process): > else: > self.cache_q.put((Ma.MetaFile.Path, Ma.Arch, "Pr= eMakeCache", False)) >=20 > Ma.CreateCodeFile(False) > Ma.CreateMakeFile(False,GenFfsList=3DFfsCmd.get((Ma.Meta= File.Path, Ma.Arch),[])) > - > + Ma.CreateAsBuiltInf() > if GlobalData.gBinCacheSource and CommandTarget in [None= , "", "all"]: > try: > CacheResult =3D Ma.CanSkipbyMakeCache() > except: > CacheResult =3D False > diff --git a/BaseTools/Source/Python/build/build.py b/BaseTools/Source/Py= thon/build/build.py > index 59ceacfed0..c4cfe38ad9 100755 > --- a/BaseTools/Source/Python/build/build.py > +++ b/BaseTools/Source/Python/build/build.py > @@ -871,11 +871,11 @@ class Build(): > if GlobalData.gUseHashCache and not GlobalData.gBinC= acheDest and self.Target in [None, "", "all"]: > cqueue.put((PcdMa.MetaFile.Path, PcdMa.Arch, "Pr= eMakeCache", False)) >=20 > PcdMa.CreateCodeFile(False) > PcdMa.CreateMakeFile(False,GenFfsList =3D DataPipe.G= et("FfsCommand").get((PcdMa.MetaFile.Path, PcdMa.Arch),[])) > - > + PcdMa.CreateAsBuiltInf() > # Force cache miss for PCD driver > if GlobalData.gBinCacheSource and self.Target in [No= ne, "", "all"]: > cqueue.put((PcdMa.MetaFile.Path, PcdMa.Arch, "Ma= keCache", False)) >=20 > self.AutoGenMgr.join() > @@ -1263,11 +1263,10 @@ class Build(): >=20 > # build modules > if BuildModule: > BuildCommand =3D BuildCommand + [Target] > LaunchCommand(BuildCommand, AutoGenObject.MakeFileDir) > - self.CreateAsBuiltInf() > if GlobalData.gBinCacheDest: > self.GenDestCache() > elif GlobalData.gUseHashCache and not GlobalData.gBinCacheSo= urce: > # Only for --hash > # Update PreMakeCacheChain files > @@ -2274,11 +2273,10 @@ class Build(): > # All modules have been put in build tasks queue. Tell t= ask scheduler > # to exit if all tasks are completed > # > ExitFlag.set() > BuildTask.WaitForComplete() > - self.CreateAsBuiltInf() > if GlobalData.gBinCacheDest: > self.GenDestCache() > elif GlobalData.gUseHashCache and not GlobalData.gBinCac= heSource: > # Only for --hash > # Update PreMakeCacheChain files > -- > 2.20.1.windows.1