From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail02.groups.io (mail02.groups.io [66.175.222.108]) by spool.mail.gandi.net (Postfix) with ESMTPS id 2DB21D80D0C for ; Tue, 23 Jan 2024 01:27:52 +0000 (UTC) DKIM-Signature: a=rsa-sha256; bh=c+92DtIC3OPL5PTaA92QkRaktEUISScV2BmhtWqvso0=; c=relaxed/simple; d=groups.io; h=ARC-Seal:ARC-Message-Signature:ARC-Authentication-Results:From:To:CC:Subject:Thread-Topic:Thread-Index:Date:Message-ID:References:In-Reply-To:Accept-Language:MIME-Version:Precedence:List-Subscribe:List-Help:Sender:List-Id:Mailing-List:Delivered-To:Reply-To:List-Unsubscribe-Post:List-Unsubscribe:Content-Language:Content-Type:Content-Transfer-Encoding; s=20140610; t=1705973270; v=1; b=tV7UoIypILYHB6TeFzCgyCHzHwqQRcKlh4oLtdcknlRNWXR5sEGDUGBclI/XMapuBwijnedI Xb5r8O0V5szzqWzypM0F6cfT7+KsYMwozSRVAW5Twx2m3vl0N6TNtmPTHmaL7Sw1zYVScT+w1Mf n0apOgRkCYM7kDFYKYA/ACkY= X-Received: by 127.0.0.2 with SMTP id V4YCYY7687511xNoqJI7OjIH; Mon, 22 Jan 2024 17:27:50 -0800 X-Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.14]) by mx.groups.io with SMTP id smtpd.web10.1161.1705973269911202782 for ; Mon, 22 Jan 2024 17:27:50 -0800 X-IronPort-AV: E=McAfee;i="6600,9927,10961"; a="1271518" X-IronPort-AV: E=Sophos;i="6.05,212,1701158400"; d="scan'208";a="1271518" X-Received: from orviesa005.jf.intel.com ([10.64.159.145]) by orvoesa106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Jan 2024 17:27:49 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,212,1701158400"; d="scan'208";a="1373825" X-Received: from fmsmsx602.amr.corp.intel.com ([10.18.126.82]) by orviesa005.jf.intel.com with ESMTP/TLS/AES256-GCM-SHA384; 22 Jan 2024 17:27:49 -0800 X-Received: from fmsmsx611.amr.corp.intel.com (10.18.126.91) by fmsmsx602.amr.corp.intel.com (10.18.126.82) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 22 Jan 2024 17:27:48 -0800 X-Received: from fmsedg601.ED.cps.intel.com (10.1.192.135) by fmsmsx611.amr.corp.intel.com (10.18.126.91) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35 via Frontend Transport; Mon, 22 Jan 2024 17:27:48 -0800 X-Received: from NAM04-DM6-obe.outbound.protection.outlook.com (104.47.73.41) by edgegateway.intel.com (192.55.55.70) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.35; Mon, 22 Jan 2024 17:27:48 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=MyCVrw6Jg9IxmV34ExHK8HDKEvfmmPkjVlwoBbZl2OwKzKLh9K3LnpiufnUrGyOpNdMoHkmSc9hoBPDHVVbb0wZVcL+BW04RdyZ4qLJEZEsp2zofPqOiLilV4XifrSx6b8j9ye4ZPbYX/La6RPk4kKrlu5XZ6ONXcE+RWmX0B5IQONIzSUBro9c9ES+H9BVX1yM8+y5eXwn4zHBgrJ4UpIIoDshAoOipXAUJM4joK5D+rJ0ThdAnNxyxgBbRSJgBJE31pBGk1bzwsKY6V0RfRwQBjZoWNS5dcYbJWwMifQM0CfpsGy8WTYzKwFEuCC539tGrR9Rf46RHSLil9ddSrQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=bH1nJ3qeGBdH5TG3TBxO4ZpJ7pfXRGsJYd99LTGvhZA=; b=TKDTPGC4yZAQxL3Ku7j8rWmLvT+F4Fia23B4u2FzsLkyWejJJuirTyynIveoJVHVESR4KBevdrXglwJXvvYITiqA5joIoQqNSwkXZS745H2YUun4wLVZaJCRob7t//MKqQfCcm89f0WJx2p1dwDk4pOF2vZZ3M03ugPmQAja9Nr+jfU+qYr8Rxe7bjOhjjeRyMr6C4MQtP05hyT1mxX/V+5K6NqYJ268WYSuDkgsxQIP+Wx/z6JipU8OJzxbwcpnBUhai72yF4cvQTYGl9rkuCY2IMX3Vh5gh4+4hYccsMMXKPSIgYFARUeUQzSRY1wRkB2+D2MDAc0y3H4qBozuAg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none X-Received: from CO1PR11MB4929.namprd11.prod.outlook.com (2603:10b6:303:6d::19) by SN7PR11MB7708.namprd11.prod.outlook.com (2603:10b6:806:352::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7202.34; Tue, 23 Jan 2024 01:27:45 +0000 X-Received: from CO1PR11MB4929.namprd11.prod.outlook.com ([fe80::7e42:1633:d0a6:9c00]) by CO1PR11MB4929.namprd11.prod.outlook.com ([fe80::7e42:1633:d0a6:9c00%6]) with mapi id 15.20.7202.035; Tue, 23 Jan 2024 01:27:45 +0000 From: "Michael D Kinney" To: "S, Ashraf Ali" , "devel@edk2.groups.io" CC: "Chen, Christine" , Rebecca Cran , Liming Gao , "Feng, Bob C" , "Chan, Amy" , "Chaganty, Rangasai V" , "Solanki, Digant H" , "Kinney, Michael D" Subject: Re: [edk2-devel] [PATCH] BaseTools: Optimize GenerateByteArrayValue and CollectPlatformGuids APIs Thread-Topic: [edk2-devel] [PATCH] BaseTools: Optimize GenerateByteArrayValue and CollectPlatformGuids APIs Thread-Index: AQHaSRr4zjZxN4xxmU2bdFdODZOPTLDh92OQgAI85YCAAm7Q4A== Date: Tue, 23 Jan 2024 01:27:45 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-traffictypediagnostic: CO1PR11MB4929:EE_|SN7PR11MB7708:EE_ x-ms-office365-filtering-correlation-id: 54df532d-8ebd-4e20-b179-08dc1bb2806e x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam-message-info: rzHD3I1sXDNvt8Yx8szmgviCykIFA3ZZWDHYuVfSlTFOardY93IYVayLPmEhGyUezjiIY4yN+gJ9JfIa9qripBjFIFwYrws0Gd1Ahc1FcSHBiHrW0yXKGH3G4+i4rFwSS71d14a1Au06ufZQqGno416HkVNQJv/cHs3tm7DXF504DzzehewuPZRP9mSHjAHc+YbcO6Y/JecBijbc/HzcPgzq5FX0f3IT6W+chThytEwYQ9kQ2JD0xGTrQlxHVIqa2cJVJ9f0kJfED9IxDwNJqw5t8XeZXOx/cPWsWDx7PChviPevUfG1S7wqPoHlX+Zw1NTy2J8sdp0Szuaihz3jtzpUJVGX7W1BR1ch3NEPFhaHiwfkJKtrigheJNnWvSBw2H3V2MYN9gIex41xL5jzhqqqHSAEYnvecgNicy6s7myTSwDQ0v8f7CpHkusnD1XQLexNCy3Y5U7/z9OdVKJ5CUP5cMuF+goWUjrt7JHgmWpqw6G41s9d+JE4MZ1t7C2t0HKVYy9g0DsXvZTjNF0tuSJWGKmLBlCx3zgKPffqbuBTOZ03BHwwy9A28H2YtXcwtkpqgj7s5DEKyr8eb5lMp4YJQz6Tqk0uEH4BLymW6Lq0Xz6H4YweJIaY7rmm0ElWZI5jrY96Oin2LgMG2B1WnxYKmgN2i9vm3LSTeGc/Bxc= x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?BmqjLAf8oKqWTjP6okmstd3a0/8EELn9DwFw0uHUwTMdJhUVS0hySeOPGNid?= =?us-ascii?Q?10KcAmvFMZCVK3i5OCA+SLJzldHNxLpsDgDCj8otEFKsqzrB3dP8UyHpLf/c?= =?us-ascii?Q?Ylpw5kQUOhSVhiUIvjaHtdyyV9N95P9KkrG7JC8KF6wpnE74OrcSEVBtIOwg?= =?us-ascii?Q?cOlp/6el6EY+XnvOS1iI+d9Q/4bxySyazdJf/LmJPS7co/Cao2szu1sZxDF5?= =?us-ascii?Q?2iNFve0FeHmoHmaKhjNPHHvjNxiomOmlPftAeBhkEw/2PWU2rJct+zgK82DO?= =?us-ascii?Q?HxOuqJhHedraz7FVwLA2SlChK1q8IAKWG9AFBnV+d4Ropt3JfJDd62B6Peia?= =?us-ascii?Q?uayWKK5fgOIuF8xTHCC9VlvCdKAlOMgnuMG/4WePUrk0trRCMAqKfzDtAHo2?= =?us-ascii?Q?6t2r2oLa7a0/4wg8kyUoSM4ahwtUsKa55AGAlsaV+hemhOKxvFFlACaGjzMv?= =?us-ascii?Q?+eAxEsS8p9cgy3vjTwl5CK8V3d4wif4l5+WX0xlPGZXuLZVTuPK/0RKJw/bb?= =?us-ascii?Q?EQ/zTl8i+7t/gqjPkF4d848C5Pxflz5Ce4khK6q0AUDNicjnkDkm2VIuNt6M?= =?us-ascii?Q?pDebbF29diMwusab8KuuKeN+WewrO+lM4vcIoN2IFL1Sb7TbA3DrUQQlEY6C?= =?us-ascii?Q?UupT0YkuCJrWChJQcyjEPlSsYNvBdZiSFfYZ98+/NYkpoECNvooD9PnLyf/I?= =?us-ascii?Q?ZyL+SL6RPNHbLJyiRX9Jn8GZ4O34/gf5b3JGouLwxKQMBXu4aVIPAS08XI8H?= =?us-ascii?Q?v6oijBCI+pyNOU3QJnCYp+tKVOzDWwldY6NfB3cQW7pPmi0wKvyQU3RDlXUt?= =?us-ascii?Q?GiJWOU/wg75vnEb0VOqTE7ixp1c8vjvVL/RNgfGhguzadOwBvIlziHrtb4ih?= =?us-ascii?Q?DM2Ufya0Pgk650F16ERGRsW30R/boDdIB6989Mr16a5wP387kCaAKNqbsQVp?= =?us-ascii?Q?NI0ibuZLJHzWfa95/RIsnFcFz8px+5kH1ShCGNE6o6HDY8pMFWWu+dhBRDMr?= =?us-ascii?Q?DiJ3AIZyqBMZp6aEb3ENYmL7vm1qPTqF4P/BHnJf50ytqCQ71IdAl6ptYM53?= =?us-ascii?Q?eouYxBGSoc6wKYO+IXawK6Tp+hRbAbFzvZAoliC3DL/iP5+CJY3PppYfdhdX?= =?us-ascii?Q?bbmgN9FjAUylrIH42sdKkoXPBFO8m2f3cFGQOohPCiyjju/3ZptAe5MdoPIx?= =?us-ascii?Q?HJ24BV1pl0U6CX9xhtS11KYc1MCIKnxRQTS7nsOvA25/VVEUYBuKIjeyIrDa?= =?us-ascii?Q?tX4Xk/lv4IoH1/7Cq+adVqZr0trwj3ZbI5I3g/IaLckdQiryrdaXmlmcdtGt?= =?us-ascii?Q?IK9CUSNBRbsj+Er6TQAuG31PJxdcrq/7GuqtU64wd/5MDlVM6XwAp1rmpt1P?= =?us-ascii?Q?lJaoZTolqxL29lWOlNdRQcrKfTpnKl4l1ewq9bcU3tS1cGNyclYrAOeLjwNF?= =?us-ascii?Q?GTQeu0JDtUv5aZpBqIBywce2IE8qPI48GL1xV9wp79IZIwV5jnwNNdacfK+A?= =?us-ascii?Q?nJIVjdKT+jfF9NF7It5a59ZutpON5PfcStel9EOPXn+/g7WVEq2jpzdPHCR4?= =?us-ascii?Q?aIvdtyQTR8b6fvSzNUD4MigMp8ATtIaToWsOHea+?= MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: CO1PR11MB4929.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 54df532d-8ebd-4e20-b179-08dc1bb2806e X-MS-Exchange-CrossTenant-originalarrivaltime: 23 Jan 2024 01:27:45.2090 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 4Gx2DBjzrIsgB8L2p4Czeo8A3xKj0G9eT5Cyehz2varZ1vUBQz0ZXfAcnMNOV7f75JVr1kkutpCyQjZOmgyL73bnUKKi1e8rpQK3XJLOwaM= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR11MB7708 X-OriginatorOrg: intel.com Precedence: Bulk List-Subscribe: List-Help: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,michael.d.kinney@intel.com List-Unsubscribe-Post: List-Unsubscribe=One-Click List-Unsubscribe: X-Gm-Message-State: wOHX0gw7Ck955Iu9pBo1oVuIx7686176AA= Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-GND-Status: LEGIT Authentication-Results: spool.mail.gandi.net; dkim=pass header.d=groups.io header.s=20140610 header.b=tV7UoIyp; spf=pass (spool.mail.gandi.net: domain of bounce@groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce@groups.io; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=intel.com (policy=none); arc=reject ("signature check failed: fail, {[1] = sig:microsoft.com:reject}") Hi Ashraf, The PcdValueInit feature is not limited to only PCDs of type VPD. It is for any structured PCDs. Did you test PCDs with all types (e.g. PcdsFixedA= tBuild PcdsPactahbleInModule, PcdsDynamicHii, PcdsDynamicDatabase, PcdsDynamicVpd, PcdsDynamicExHii, PcdsDynamicExDatabase, PcdsDynamicExVpd) And did you DSC test cases cover both changes in PCD types and changes in P= CD values? Thanks, Mike > -----Original Message----- > From: S, Ashraf Ali > Sent: Sunday, January 21, 2024 4:14 AM > To: Kinney, Michael D ; devel@edk2.groups.io > Cc: Chen, Christine ; Rebecca Cran > ; Liming Gao ; Feng, Bob C > ; Chan, Amy ; Chaganty, > Rangasai V ; Solanki, Digant H > > Subject: RE: [edk2-devel] [PATCH] BaseTools: Optimize > GenerateByteArrayValue and CollectPlatformGuids APIs >=20 > Hi., @Kinney, Michael D >=20 > The main API which is modified in this Patch is GenerateByteArrayValue. >=20 > This API is used to return the list of SKUID.TokenSpaceGuid.VpdName|VPD > STRUCT|Binary Data which will be stored in Output.txt > Example: > SAMPLESKUID.STANDARD.gEfiMdePkgTokenSpaceGuid.SampleVpd|SAMPLE_STRUCT[]| > {0x01,0x01,0x05,0x09,0x02} >=20 > This VPD/PCD is coming from either the DEC file or the DSC file. >=20 > GenerateByteArrayValue API is used to create the PcdValueInit.exe and > it's equivalent C and Make File. >=20 > When there is no change in DEC or DSC VPD/PCD still it used to do the > same process again in the incremental build which is time consuming. > In my project this API is used to take 3min of time with High end server > (64Threads) and 3.5Min in Laptop (16 threads with performance mode > enabled) only for GenerateByteArrayValue API. >=20 > This patch will check if there are any change in the DEC/DSC > VPDs/PCDs2, if there is any change in VPD the current flow is not > affect. (it will create the PcdRecordList.json and copy Output.txt for > Arch folder with the project. Ex: > Build\{Project}Pkg\DEBUG_VS2019\PcdValueInit\{IA32/X64/Any}\PcdRecordLis > t.json and > Build\{Project}Pkg\DEBUG_VS2019\PcdValueInit\{IA32/X64/Any}\Output.txt) > If there is no change the DSC/DEC VPDs it will read the data from > Output.txt and return the same data. (we will compare the Input > structure and PcdRecordList.json) if there is any mismatch then it there > change in VPDs if not then no change in VPDs). >=20 > Unit testing from my side as follows: > * Build the Firmware. > * PcdRecordList.json and Output.txt will be created based on the > Arch. > * Build the Firmware again without any change. > * It will read the Output.txt and return the data. (it will match > Input PCD/VPD list and compare with existing PcdRecordList.json) > * I verified the return length and content GenerateByteArrayValue > (it's same with and without my changes) > * Build the Firmware again my modifying the VPDs in DSC file > * Change in VPDs, it will regenerate PcdRecordList.json file > * it will create the Exe file and Output.txt > * New Output.txt will be copied to above path. > * Build the Firmware again by modifying the VPDs in DEC file. > * Change in VPDs, it will regenerate PcdRecordList.json file > * it will create the Exe file and Output.txt > * New Output.txt will be copied to above path. > * Build the Firmware again without modifying the code. > * It will read the Output.txt and return the data. > * I verified the return length and content GenerateByteArrayValue > (it's same with and without my changes) >=20 >=20 > There is no impact on the Boot/cache. > This patch is used to reduce the incremental build time by checking if > the VPDs/PCDs are changed or not, if not changed then it will return the > previous stored data. >=20 > Thanks., > S, Ashraf Ali >=20 > -----Original Message----- > From: Kinney, Michael D > Sent: Saturday, January 20, 2024 7:36 AM > To: devel@edk2.groups.io; S, Ashraf Ali > Cc: Chen, Christine ; Rebecca Cran > ; Liming Gao ; Feng, Bob C > ; Chan, Amy ; Chaganty, > Rangasai V ; Solanki, Digant H > ; Kinney, Michael D > > Subject: RE: [edk2-devel] [PATCH] BaseTools: Optimize > GenerateByteArrayValue and CollectPlatformGuids APIs >=20 > Hi Ashraf, >=20 > What is captured in the file? >=20 > What PCD/VPD changes will invalidate the cache? Just the number and > type of PCD/VPD elements or their default values/sizes? >=20 > How was this tested? Were all conditions that invalidate the cache > tested? I ask because incremental build is a very important feature and > if there is any logic error in the cache management of a file like this, > it will cause unexpected behavior and developers will not trust > incremental builds. >=20 > Mike >=20 > > -----Original Message----- > > From: devel@edk2.groups.io On Behalf Of Ashraf > > Ali S > > Sent: Tuesday, January 16, 2024 11:55 PM > > To: devel@edk2.groups.io > > Cc: S, Ashraf Ali ; Chen, Christine > > ; Rebecca Cran ; Liming Gao > > ; Feng, Bob C ; Chan, > > Amy ; Chaganty, Rangasai V > > ; Solanki, Digant H > > > > Subject: [edk2-devel] [PATCH] BaseTools: Optimize > > GenerateByteArrayValue and CollectPlatformGuids APIs > > > > During the Incremental build GenerateByteArrayValue used to generate > > the ByteArrayValue even when there is no change in the PCD/VPDs. which > > is time consuming API based on the number of PCD/VPDs and SKU IDs. > > > > The optimization is that GenerateByteArrayValue is used to store the > > PcdRecordList in a JSON file for each of the arch. and during the > > Incremental build this API will check if there is any change in the > > PCD /VPDs then rest of the flow remains the same. if there is no > > change then it will return the provious build data. > > > > Flow: > > during the 1st build PcdRecordList.json is not exists, PcdRecordList > > will be dumped to json file. and it will copy the output.txt as well. > > Note: as the output.txt are different for different Arch, so it will > > be stored in the Arch folder. > > During the Incremental build check if there is any change in PCD/VPD. > > if there is a change in VPD/PCD then recreate the PcdRecordList.json. > > and rest of the flow remains same. > > if there is no change in VPD/PCD read the output.txt and return the > > data > > > > Cc: Yuwei Chen > > Cc: Rebecca Cran > > Cc: Liming Gao > > Cc: Bob Feng > > Cc: Amy Chan > > Cc: Sai Chaganty > > Cc: Digant H Solanki > > Signed-off-by: Ashraf Ali S > > --- > > .../Source/Python/AutoGen/WorkspaceAutoGen.py | 16 ++--- > > .../Source/Python/Workspace/DscBuildData.py | 72 +++++++++++++++--- > - > > 2 files changed, 64 insertions(+), 24 deletions(-) > > > > diff --git a/BaseTools/Source/Python/AutoGen/WorkspaceAutoGen.py > > b/BaseTools/Source/Python/AutoGen/WorkspaceAutoGen.py > > index 160e3a3cd3..eec9280c8e 100644 > > --- a/BaseTools/Source/Python/AutoGen/WorkspaceAutoGen.py > > +++ b/BaseTools/Source/Python/AutoGen/WorkspaceAutoGen.py > > @@ -160,22 +160,18 @@ class WorkspaceAutoGen(AutoGen): > > > > def CollectPlatformGuids(self): > > oriInfList =3D [] > > - oriPkgSet =3D set() > > - PlatformPkg =3D set() > > + pkgSet =3D set() > > for Arch in self.ArchList: > > Platform =3D self.BuildDatabase[self.MetaFile, Arch, > > self.BuildTarget, self.ToolChain] > > oriInfList =3D Platform.Modules > > for ModuleFile in oriInfList: > > ModuleData =3D self.BuildDatabase[ModuleFile, > > Platform._Arch, Platform._Target, Platform._Toolchain] > > - oriPkgSet.update(ModuleData.Packages) > > - for Pkg in oriPkgSet: > > - Guids =3D Pkg.Guids > > - GlobalData.gGuidDict.update(Guids) > > + pkgSet.update(ModuleData.Packages) > > if Platform.Packages: > > - PlatformPkg.update(Platform.Packages) > > - for Pkg in PlatformPkg: > > - Guids =3D Pkg.Guids > > - GlobalData.gGuidDict.update(Guids) > > + pkgSet.update(Platform.Packages) > > + for Pkg in pkgSet: > > + Guids =3D Pkg.Guids > > + GlobalData.gGuidDict.update(Guids) > > > > @cached_property > > def FdfProfile(self): > > diff --git a/BaseTools/Source/Python/Workspace/DscBuildData.py > > b/BaseTools/Source/Python/Workspace/DscBuildData.py > > index 4768099343..740b8e22be 100644 > > --- a/BaseTools/Source/Python/Workspace/DscBuildData.py > > +++ b/BaseTools/Source/Python/Workspace/DscBuildData.py > > @@ -37,6 +37,8 @@ from functools import reduce from Common.Misc > > import SaveFileOnChange from Workspace.BuildClassObject import > > PlatformBuildClassObject, StructurePcd, PcdClassObject, > > ModuleBuildClassObject from collections import OrderedDict, > > defaultdict > > +import json > > +import shutil > > > > def _IsFieldValueAnArray (Value): > > Value =3D Value.strip() > > @@ -56,6 +58,7 @@ def _IsFieldValueAnArray (Value): > > > > PcdValueInitName =3D 'PcdValueInit' > > PcdValueCommonName =3D 'PcdValueCommon' > > +PcdRecordListName =3D 'PcdRecordList.json' > > > > PcdMainCHeader =3D ''' > > /** > > @@ -1599,7 +1602,7 @@ class DscBuildData(PlatformBuildClassObject): > > S_pcd_set =3D DscBuildData.OverrideByComm(S_pcd_set) > > > > # Create a tool to caculate structure pcd value > > - Str_Pcd_Values =3D self.GenerateByteArrayValue(S_pcd_set) > > + Str_Pcd_Values =3D self.GenerateByteArrayValue(S_pcd_set, > > RecordList) > > > > if Str_Pcd_Values: > > for (skuname, StoreName, PcdGuid, PcdName, PcdValue) in > > Str_Pcd_Values: > > @@ -2750,12 +2753,61 @@ class DscBuildData(PlatformBuildClassObject): > > ccflags.add(item) > > i +=3D1 > > return ccflags > > - def GenerateByteArrayValue (self, StructuredPcds): > > + > > + def GetStructurePcdSet (self, OutputValueFile): > > + if not os.path.isfile(OutputValueFile): > > + EdkLogger.error("GetStructurePcdSet", FILE_NOT_FOUND, > > "Output.txt doesn't exist", ExtraData=3DOutputValueFile) > > + return [] > > + File =3D open (OutputValueFile, 'r') > > + FileBuffer =3D File.readlines() > > + File.close() > > + > > + #start update structure pcd final value > > + StructurePcdSet =3D [] > > + for Pcd in FileBuffer: > > + PcdValue =3D Pcd.split ('|') > > + PcdInfo =3D PcdValue[0].split ('.') > > + StructurePcdSet.append((PcdInfo[0], PcdInfo[1], > > + PcdInfo[2], > > PcdInfo[3], PcdValue[2].strip())) > > + return StructurePcdSet > > + > > + def GenerateByteArrayValue (self, StructuredPcds, PcdRecordList): > > # > > # Generate/Compile/Run C application to determine if there > > are any flexible array members > > # > > if not StructuredPcds: > > return > > + # > > + # If the output path doesn't exists then create it > > + # > > + if not os.path.exists(self.OutputPath): > > + os.makedirs(self.OutputPath) > > + > > + PcdRecordListPath =3D os.path.join(self.OutputPath, self._Arch= , > > PcdRecordListName) > > + PcdRecordOutputValueFile =3D os.path.join(self.OutputPath, > > self._Arch, 'Output.txt') > > + > > + if not os.path.exists(os.path.dirname(PcdRecordListPath)): > > + os.makedirs(os.path.dirname(PcdRecordListPath)) > > + # > > + # Check if the PcdRecordList.json exists or not > > + # if exits then it might be a incremental build then check if > > the PcdRecord list has been changed or not. > > + # if changed then proceed further, if not changed then return > > the stored data from earlier build > > + # > > + if os.path.isfile(PcdRecordListPath): > > + with open(PcdRecordListPath, 'r') as file: > > + file_content_str =3D file.read() > > + if file_content_str: > > + # Use json.loads to convert the string back to a > > list > > + file_content =3D json.loads(file_content_str) > > + # Check if all PcdRecordList in record_set are > > present in file_content > > + # and if there are no extra PcdRecordList in > > file_content > > + if set(map(tuple, PcdRecordList)) =3D=3D > > + set(map(tuple, > > file_content)): > > + return > > self.GetStructurePcdSet(PcdRecordOutputValueFile) > > + # > > + # 1st build, create the PcdRecordList.json > > + # update the record as PCD Input has been changed in > > incremental build > > + # > > + with open(PcdRecordListPath, 'w') as file: > > + json.dump(PcdRecordList, file) > > > > InitByteValue =3D "" > > CApp =3D PcdMainCHeader > > @@ -2832,8 +2884,6 @@ class DscBuildData(PlatformBuildClassObject): > > > > CApp =3D CApp + PcdMainCEntry + '\n' > > > > - if not os.path.exists(self.OutputPath): > > - os.makedirs(self.OutputPath) > > CAppBaseFileName =3D os.path.join(self.OutputPath, > > PcdValueInitName) > > SaveFileOnChange(CAppBaseFileName + '.c', CApp, False) > > > > @@ -3042,17 +3092,11 @@ class DscBuildData(PlatformBuildClassObject): > > if returncode !=3D 0: > > EdkLogger.warn('Build', COMMAND_FAILURE, 'Can not > > collect output from command: %s\n%s\n%s\n' % (Command, StdOut, > > StdErr)) > > > > - #start update structure pcd final value > > - File =3D open (OutputValueFile, 'r') > > - FileBuffer =3D File.readlines() > > - File.close() > > + # Copy output file for each Arch > > + shutil.copyfile(OutputValueFile, PcdRecordOutputValueFile) > > > > - StructurePcdSet =3D [] > > - for Pcd in FileBuffer: > > - PcdValue =3D Pcd.split ('|') > > - PcdInfo =3D PcdValue[0].split ('.') > > - StructurePcdSet.append((PcdInfo[0], PcdInfo[1], > PcdInfo[2], > > PcdInfo[3], PcdValue[2].strip())) > > - return StructurePcdSet > > + # return structure pcd final value > > + return self.GetStructurePcdSet(OutputValueFile) > > > > @staticmethod > > def NeedUpdateOutput(OutputFile, ValueCFile, StructureInput): > > -- > > 2.39.1.windows.1 > > > > > > > >=20 > > -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#114160): https://edk2.groups.io/g/devel/message/114160 Mute This Topic: https://groups.io/mt/103781798/7686176 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/leave/12367111/7686176/19134562= 12/xyzzy [rebecca@openfw.io] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-