From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by mx.groups.io with SMTP id smtpd.web12.6250.1597309268558567488 for ; Thu, 13 Aug 2020 02:01:08 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@intel.onmicrosoft.com header.s=selector2-intel-onmicrosoft-com header.b=PkGtBtAc; spf=pass (domain: intel.com, ip: 192.55.52.120, mailfrom: bob.c.feng@intel.com) IronPort-SDR: cetOoJOBpjlyjTgHRBYqsR+eXHWifqVpZfMLHvtFixDIx/3dUHmARwvryjW/j/tDYP30bFsiX0 ev8ev6mSgQ1Q== X-IronPort-AV: E=McAfee;i="6000,8403,9711"; a="151604681" X-IronPort-AV: E=Sophos;i="5.76,307,1592895600"; d="scan'208";a="151604681" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Aug 2020 02:01:07 -0700 IronPort-SDR: RQXFkvNg9lkAt9bYcFcmG9tuK9UCv2e1zWMW/vs9z9eaJGag/9YrI3x55LoPEuczLoBbGbn7rW UvPzS120FH5g== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,307,1592895600"; d="scan'208";a="276797569" Received: from fmsmsx603-2.cps.intel.com (HELO fmsmsx603.amr.corp.intel.com) ([10.18.84.213]) by fmsmga007.fm.intel.com with ESMTP; 13 Aug 2020 02:01:07 -0700 Received: from fmsmsx611.amr.corp.intel.com (10.18.126.91) by fmsmsx603.amr.corp.intel.com (10.18.126.83) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Thu, 13 Aug 2020 02:01:06 -0700 Received: from FMSEDG002.ED.cps.intel.com (10.1.192.134) by fmsmsx611.amr.corp.intel.com (10.18.126.91) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.1713.5 via Frontend Transport; Thu, 13 Aug 2020 02:01:06 -0700 Received: from NAM02-BL2-obe.outbound.protection.outlook.com (104.47.38.56) by edgegateway.intel.com (192.55.55.69) with Microsoft SMTP Server (TLS) id 14.3.439.0; Thu, 13 Aug 2020 02:01:05 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=NCc2PixYNSirWkrxK0g5HeTNG6eC/U+b0Hl8nAuQzgHcsoLdWoi28qYaWoTXDtZYmNQWxVU0aWXTSzk5blzFbrq0/L/0zOMl5zlIZkIRFA0mcN4h482wCgMLlqB4YRQKmjmp5V7/oCMdRkiyXpenYEn7cffmmac7M6SQd1Z5plsWwzv0FldkTmXn7NnXmnO8atJLdfd11tBOHoDArkqfz9JY8KykEJugeCAKghOpNrqMg/3C5V17EfJ8AZvbeHCHV85bpCdjDeWeGHsbzxBtijnAjmgTUHRrLvSSPd4s6dzHyagVYvwN8lEy7U2Il+JeugmvEsUkwJajkQyvc5iP1g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=e1/haTyu4krRSXhR8BCYjkwhuBg+hV1Mr3PnHK7kdTY=; b=UKXFWRvE+MrVq5PebaCb8WlO0PdiR8p3JlJu+xR2yLabNBoEq0QGXvPuw39xARAvvL6M0hgiD6mGOjW4w9qfSj4QZTmap84Ai3jsLTBmnMm+QOHTjUEoejIJT/GYliY818sP1U6+o6JqC2HRuHXfGVB1NckjsIhd8zLX2Zf+Puu11+BxquNIjwbHImX7FAOUuxkd79Tr0+y5mPCHic+KfzblScvsl2GoDZ4rpCXyWctaFyAxFFSBCdCmERFqj+b8i+W1m4o7ksyXfaW2D2KDfa7YJETbbqNdrxQRjKO3Mtwtv0NBGt+rHHNgvfkB2Q7E2omNXN2HiW/woWW0rufuBg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=e1/haTyu4krRSXhR8BCYjkwhuBg+hV1Mr3PnHK7kdTY=; b=PkGtBtAcXSOGW7E3J5WTfxRrYAoqxJDMFMiEGNvpXh6+DC4eglz/OXdakSagDmJqJJT1a/OjDhg8WO4/sxVoCGU7C9uKBEwJ+AZKbVwHM1D54jwSdyT8aHmbymOCPSxa9fP8w33yRYNN+yNBaUPCXdjH6fML2XZhmFqd5x2ZAOs= Received: from DM6PR11MB4073.namprd11.prod.outlook.com (2603:10b6:5:19f::22) by DM6PR11MB2604.namprd11.prod.outlook.com (2603:10b6:5:c8::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3261.16; Thu, 13 Aug 2020 09:01:02 +0000 Received: from DM6PR11MB4073.namprd11.prod.outlook.com ([fe80::88bc:d834:df24:ea90]) by DM6PR11MB4073.namprd11.prod.outlook.com ([fe80::88bc:d834:df24:ea90%6]) with mapi id 15.20.3261.026; Thu, 13 Aug 2020 09:01:02 +0000 From: "Bob Feng" To: "Gao, Liming" , "devel@edk2.groups.io" CC: "Chen, Christine" Subject: Re: [Patch] BaseTools: Move CreateAsBuiltInf into AutoGenWorker for parallel Thread-Topic: [Patch] BaseTools: Move CreateAsBuiltInf into AutoGenWorker for parallel Thread-Index: AQHWcG8bUEq+XXoFukqWpvRxiYUTtak0ipawgAE0yqA= Date: Thu, 13 Aug 2020 09:01:02 +0000 Message-ID: References: <20200812060903.23704-1-bob.c.feng@intel.com> In-Reply-To: Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=intel.com; x-originating-ip: [192.102.204.38] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: dc2c7e4f-e572-4c94-dee6-08d83f676770 x-ms-traffictypediagnostic: DM6PR11MB2604: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:6790; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: SioPLDMNmThb0VeI+YnxwjL6y67gW9iXC1lY5IuqaT5AuhVosmMw0hw7W5wPcGsDd0TZW54v7ohvJed992+SeTzne4mFKT08NBZ+I4b6vhltEuWwZqsuviOokzfnQ9FgIvU8u6ZDQwDKMxNjwFgK3n6G2icrHkATXmdkzHZcY+zSyyyLe1gus6gLEZ6yNHX9GucwqjzdNJgav59aquCaykUliUfn5tY3Eb7nnCumsV+pQyeCOht8M9Crvn5i3O/JIO6y9KLys96EWEH5D1Umgtr6sAMd4wFFM670LODnaRz0F0dwqlR6Ly3Q+TJLK8B1 x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR11MB4073.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFTY:;SFS:(6029001)(4636009)(346002)(366004)(39860400002)(376002)(136003)(396003)(66476007)(66556008)(66446008)(76116006)(66946007)(7696005)(4326008)(107886003)(2906002)(33656002)(110136005)(53546011)(8936002)(83380400001)(9686003)(55016002)(86362001)(6506007)(478600001)(8676002)(5660300002)(64756008)(316002)(71200400001)(186003)(26005)(52536014);DIR:OUT;SFP:1102; x-ms-exchange-antispam-messagedata: SkMMGyc/Ns7hGynlljeKvhce/QfGWLHPfE3OhTpzaVTlGDLScC62b+9UmRilulQHS6lG0QpSqa4MpH++ucRNIU1YKZDRdqccQoURrCmS/eCxnqiF38J265KabTasFOcEJyy1K08Gm8i83IhEAkWo8ahXfNKhYbMQlew2VQuKQJP/XOopyCEBoNDAXGqZRcXuU4VoU11GDDiDoKrYO7Xsux8Ywiw7mJzSHunfX+rBGdfLwh/jpGQjjV6h6FktvbEoOM0KYjC2HvmTD2TzByPgnkURiTgM2jOh9rwA/ZrxCY0Lwo/g9wKU2Ok+0Zho5ZYnKg/vGoi4Jwu8O4ZugBsEpnMWT7JiOzQBUJYShlPeecrkA+ZAf4NRD2l5/t3RpyBQIzPrJ3PWT7hMqPQ5/XEKUGT69LFOAJYeFm7fUk3frgRB3+eD1eBBWXRjyx4OrMxja5uygNMJutPU0iHmwFzet43GnKajOqdPCzQK8X0AI+4pdp/RTdvsKTkPJyxRY2vhDvniLjlDDbOAsM7doXmDgNWupnk8Hu9X9c5zz3Y/24tZPkpfIQXm0Ogmvq9cLNuVbhJwdBnt7oWQg1nnQTcVuKuEprtm7MQjZuOXSdluM1pEcTRfUDmYgVyJuLZISU3CPmzpWHn0+UxX5WXU9KbF3A== MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM6PR11MB4073.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: dc2c7e4f-e572-4c94-dee6-08d83f676770 X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Aug 2020 09:01:02.0667 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: T23OGCQc4NxOPzJb9GiUs9SwocwIiNZbc51vb6plQ5AXlZO9tpKQyOuEztuY0nrz8LkD9QQ6AFfJFAnVMtJboA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR11MB2604 Return-Path: bob.c.feng@intel.com X-OriginatorOrg: intel.com Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Liming, I did the test on Ovmf, there is some improvement but not much. Thanks, Bob -----Original Message----- From: Gao, Liming =20 Sent: Wednesday, August 12, 2020 10:35 PM To: Feng, Bob C ; devel@edk2.groups.io Cc: Chen, Christine Subject: RE: [Patch] BaseTools: Move CreateAsBuiltInf into AutoGenWorker fo= r parallel I like this idea. Is there any build performance improvement? > -----Original Message----- > From: Feng, Bob C > Sent: Wednesday, August 12, 2020 2:09 PM > To: devel@edk2.groups.io > Cc: Gao, Liming ; Chen, Christine=20 > > Subject: [Patch] BaseTools: Move CreateAsBuiltInf into AutoGenWorker=20 > for parallel >=20 > AsBuiltInf can be created during AutoGen phase. Move CreateAsBuiltInf=20 > into AutoGenWorker to make this step run in parallel. >=20 > Signed-off-by: Bob Feng > Cc: Liming Gao > Cc: Yuwei Chen > --- > BaseTools/Source/Python/AutoGen/AutoGenWorker.py | 2 +- > BaseTools/Source/Python/build/build.py | 4 +--- > 2 files changed, 2 insertions(+), 4 deletions(-) >=20 > diff --git a/BaseTools/Source/Python/AutoGen/AutoGenWorker.py=20 > b/BaseTools/Source/Python/AutoGen/AutoGenWorker.py > index 017f676399..0425f1ab0b 100755 > --- a/BaseTools/Source/Python/AutoGen/AutoGenWorker.py > +++ b/BaseTools/Source/Python/AutoGen/AutoGenWorker.py > @@ -278,11 +278,11 @@ class AutoGenWorkerInProcess(mp.Process): > else: > self.cache_q.put((Ma.MetaFile.Path, Ma.Arch,=20 > "PreMakeCache", False)) >=20 > Ma.CreateCodeFile(False) > =20 > Ma.CreateMakeFile(False,GenFfsList=3DFfsCmd.get((Ma.MetaFile.Path,=20 > Ma.Arch),[])) > - > + Ma.CreateAsBuiltInf() > if GlobalData.gBinCacheSource and CommandTarget in [None= , "", "all"]: > try: > CacheResult =3D Ma.CanSkipbyMakeCache() > except: > CacheResult =3D False diff --git=20 > a/BaseTools/Source/Python/build/build.py=20 > b/BaseTools/Source/Python/build/build.py > index 59ceacfed0..c4cfe38ad9 100755 > --- a/BaseTools/Source/Python/build/build.py > +++ b/BaseTools/Source/Python/build/build.py > @@ -871,11 +871,11 @@ class Build(): > if GlobalData.gUseHashCache and not GlobalData.gBinC= acheDest and self.Target in [None, "", "all"]: > cqueue.put((PcdMa.MetaFile.Path, PcdMa.Arch,=20 > "PreMakeCache", False)) >=20 > PcdMa.CreateCodeFile(False) > PcdMa.CreateMakeFile(False,GenFfsList =3D=20 > DataPipe.Get("FfsCommand").get((PcdMa.MetaFile.Path, PcdMa.Arch),[])) > - > + PcdMa.CreateAsBuiltInf() > # Force cache miss for PCD driver > if GlobalData.gBinCacheSource and self.Target in [No= ne, "", "all"]: > cqueue.put((PcdMa.MetaFile.Path, PcdMa.Arch,=20 > "MakeCache", False)) >=20 > self.AutoGenMgr.join() > @@ -1263,11 +1263,10 @@ class Build(): >=20 > # build modules > if BuildModule: > BuildCommand =3D BuildCommand + [Target] > LaunchCommand(BuildCommand, AutoGenObject.MakeFileDir) > - self.CreateAsBuiltInf() > if GlobalData.gBinCacheDest: > self.GenDestCache() > elif GlobalData.gUseHashCache and not GlobalData.gBinCacheSo= urce: > # Only for --hash > # Update PreMakeCacheChain files @@ -2274,11 +2273,10=20 > @@ class Build(): > # All modules have been put in build tasks queue. Tell t= ask scheduler > # to exit if all tasks are completed > # > ExitFlag.set() > BuildTask.WaitForComplete() > - self.CreateAsBuiltInf() > if GlobalData.gBinCacheDest: > self.GenDestCache() > elif GlobalData.gUseHashCache and not GlobalData.gBinCac= heSource: > # Only for --hash > # Update PreMakeCacheChain files > -- > 2.20.1.windows.1