From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail02.groups.io (mail02.groups.io [66.175.222.108]) by spool.mail.gandi.net (Postfix) with ESMTPS id 1DDA0AC0060 for ; Tue, 23 Jan 2024 05:13:37 +0000 (UTC) DKIM-Signature: a=rsa-sha256; bh=NYgJRlUs0VWliBML7WRroQCs/2Hkl+kv+voTKYq1ivI=; c=relaxed/simple; d=groups.io; h=ARC-Seal:ARC-Message-Signature:ARC-Authentication-Results:From:To:CC:Subject:Thread-Topic:Thread-Index:Date:Message-ID:References:In-Reply-To:Accept-Language:MIME-Version:Precedence:List-Subscribe:List-Help:Sender:List-Id:Mailing-List:Delivered-To:Reply-To:List-Unsubscribe-Post:List-Unsubscribe:Content-Language:Content-Type:Content-Transfer-Encoding; s=20140610; t=1705986816; v=1; b=ZxxOWS2CEx+FyuF5RZdx93l2pQ1KHWqmbmjzOWj0G1dBqN0eaO2GhI881+CnljWmynjqwBeO ws3WqkUWFYutLFyeAA0hmOCVa+T9aUR70PWMQbJEPvJNRYjVH/lm2DU242+uGQs0ZzcoF6ZCkoB Oq29DtQWnK1lBRJX0IVbubgo= X-Received: by 127.0.0.2 with SMTP id 5rFxYY7687511xHVEysQKuJp; Mon, 22 Jan 2024 21:13:36 -0800 X-Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.12]) by mx.groups.io with SMTP id smtpd.web11.4346.1705983833494271677 for ; Mon, 22 Jan 2024 20:23:53 -0800 X-IronPort-AV: E=McAfee;i="6600,9927,10961"; a="1265272" X-IronPort-AV: E=Sophos;i="6.05,213,1701158400"; d="scan'208";a="1265272" X-Received: from fmviesa001.fm.intel.com ([10.60.135.141]) by fmvoesa106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Jan 2024 20:23:54 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,213,1701158400"; d="scan'208";a="27878999" X-Received: from fmsmsx602.amr.corp.intel.com ([10.18.126.82]) by fmviesa001.fm.intel.com with ESMTP/TLS/AES256-GCM-SHA384; 22 Jan 2024 20:23:53 -0800 X-Received: from fmsmsx610.amr.corp.intel.com (10.18.126.90) by fmsmsx602.amr.corp.intel.com (10.18.126.82) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 22 Jan 2024 20:23:52 -0800 X-Received: from FMSEDG603.ED.cps.intel.com (10.1.192.133) by fmsmsx610.amr.corp.intel.com (10.18.126.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35 via Frontend Transport; Mon, 22 Jan 2024 20:23:52 -0800 X-Received: from NAM11-BN8-obe.outbound.protection.outlook.com (104.47.58.168) by edgegateway.intel.com (192.55.55.68) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.35; Mon, 22 Jan 2024 20:23:52 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=gkscx/C7K/MpP7On0k34hqwqUy+bldDwAqbU2BiuBAYozI+iTurspgOYfHO21G5DJEp06wpekda18H2ikwaQSq+F7c4G/BEsC3Jf1yc04AjclJzKzSjDhlc7PWSBVxL06saOzH2b+6bGbdB5emjP914VDZhE6jqCmqEMvVm5zH9q3qWXSaazGa6IxEZR4M0yg8FVQ0nz4z9y+29+f8BUssHTAWXS58LEoIIkVOI8rb+/PHeV0sv1KdeYCTXMR7+wY+jic1eiaZoS098ePiFQTngYDrWRI/lR92OSsy/r/DQuJNUDGkg94uAHuNaXbkPHUMdJ4+Vok3N6ulNO3EBkeA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=XeDjq8xt12iZzYyDmkfDnXRXrHDffAFN5Hqe60ZmR6M=; b=hQrNTIMUF9rC4n8Crf6pHn3/B8RuTOUDOoIdsiG3p8Nss/6Q3qCfCen+5hTSh+6Je5A9034dt4enVwNHUn+P+Lic2cADn1H2YUbB9GdD0uFXNzVc5ARcQjpX32TGZ3gdPx4PAmoSqX1BzNoKxZ9bVy6BgjzrA8a34J3xwcnfdnK8vLRmvNJzB6VauUKen62Dmt0aWM1F3unBYbd1PR6KM2/c5DKKUs0jsxUGOriAXSMxhGEC+zrBHoEac0tkpFvRUKcUn+2nJvqCMxndk6H3Ps6nCoJCU1QEC50iO7A6Ip5GLVkAlkHAvoRXW/awO2amGR41yUOt9wWsq5Ua0B/JRA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none X-Received: from PH7PR11MB5863.namprd11.prod.outlook.com (2603:10b6:510:135::11) by SA1PR11MB5924.namprd11.prod.outlook.com (2603:10b6:806:23b::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7202.32; Tue, 23 Jan 2024 04:23:44 +0000 X-Received: from PH7PR11MB5863.namprd11.prod.outlook.com ([fe80::e4d6:ab6:b4c7:8830]) by PH7PR11MB5863.namprd11.prod.outlook.com ([fe80::e4d6:ab6:b4c7:8830%6]) with mapi id 15.20.7202.031; Tue, 23 Jan 2024 04:23:44 +0000 From: "Feng, Bob C" To: "Kinney, Michael D" , "S, Ashraf Ali" , "devel@edk2.groups.io" CC: "Chen, Christine" , Rebecca Cran , Liming Gao , "Chan, Amy" , "Chaganty, Rangasai V" , "Solanki, Digant H" Subject: Re: [edk2-devel] [PATCH] BaseTools: Optimize GenerateByteArrayValue and CollectPlatformGuids APIs Thread-Topic: [edk2-devel] [PATCH] BaseTools: Optimize GenerateByteArrayValue and CollectPlatformGuids APIs Thread-Index: AQHaSRqFQFXQj6y89026+gih3V6ohrDh+COAgAI8JoCAAnA9gIAAL6gg Date: Tue, 23 Jan 2024 04:23:44 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-traffictypediagnostic: PH7PR11MB5863:EE_|SA1PR11MB5924:EE_ x-ms-office365-filtering-correlation-id: 993c2ce9-1ba3-4657-ea5c-08dc1bcb161a x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam-message-info: tsLkFScnJ7NVx/sjGA6390RGZ6Tlpkt2iR8NVvZy0e5PK+Obhk0glHO8ppslZmz9yPMZ1x7eiO6t6x6I+k+e7bWTIPH8W+e8z42/iEzJJVwdYPIgvAGPHDpXlb+PyyBd9fAn5cFcVKCtqkEvQ4WN42vlmEnAiru7yu/eUOqA3NL52inyOUz51eV5RekMxdKJCIGmS6j0Z4sVoBPVZYQAvPw6TvdT7YWE0fgJzrY/nuaIsbqVZULZyh4ZPYKK5QF9FWu+/TY3mKz5hDyB4IfZfjiXt9iOlBRphUWxivhBxLsYi4MozFUZz2dgBh4HlZt7KbLJ9GAvMgz6dD5jCTLYCyLsepTb0gCX9fjdl2nh4Umx9Y5TQULB4p0vW8zpYp7tghap086S3QwujBWTwdJ9fszBaN4kk1fgN8h62ayTzUosOlwjHCmpqh+UKFqvfiDkAw8e/9+s3U0USyKId/NdUTOK3Vr9IdimVd8A2mhqya+JDsUtR83cL9/K2/70JpYelK4Y7ltIvgUKU00VOhNr+rTTuht4FVj250Pe7JVGIGW7/2+mw/QgeTfqaX2lKyDWysTUfz5VWZLpt6h8DA1LBWEEoJnkioXK7NtAz6ZfiJr2CmYU/tAME3fOXGjk7DMflJANvLBjdiFBlvNgG8+vMmLnY3jFLIznVDyAFq6Ya+8= x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?1Asvoka1lmEVjJH1nlM7SulJs320Sp8Z9regYVmYIQjbbo2ujLU8X/nNqfpV?= =?us-ascii?Q?X/Z0JAXJuyp864Kv0kaKCRmE9gtyK3rg9Rtz0IkYcIumrM97puKfoGNabFEh?= =?us-ascii?Q?Q0/pbJJjT4FuDW0d4QigFjSrWykLpdXK85fo0jrRuZaHKQ5KCEJNttUk0M9F?= =?us-ascii?Q?ExyVqAKOY3u5vwL0ER68iPFWQIMzGYRiRcJznXfg1efXbQ4At3o/4yxnbdNu?= =?us-ascii?Q?WOHBYC7r8XidYBroyl9t71ApV7SQLcUYWnpq6Ou/uwcbOX/LoedxDnhWOXp2?= =?us-ascii?Q?TmMRK6UnaJ90b52Bv5CRFThBLPpSiA6kByOsRG8ZuD/y+mYQeQit+FVOSKzJ?= =?us-ascii?Q?u5lupYBhHnH34ImQfQ81t6lW1y89Ce04Z9mI1erboWhd0XAQn8/2redoVSFi?= =?us-ascii?Q?hbjbX7SjMW6rAccNzcydMOpokd2yoOwZDpRm9aYiVUdPbY6SELsJ4KR6YIgf?= =?us-ascii?Q?MfCXPxX6u6Pnuk2eC2Ki4lvq+uZwXiwKW1Y+rrOOSPDNL7Z7+VmrlZk6PhSV?= =?us-ascii?Q?+MDYHeBj3fM0vxNDvgTgoMzHlO//HIx/56M5/k/lXvkWei8Rdhca1f3Gc+HE?= =?us-ascii?Q?P4+vwt6mGly/LJxKFBhhv6YnVdugRwQfSaLsxkE/fHSJWeuqhBmcivmo/irX?= =?us-ascii?Q?lnL6LlFOKb8qs1toMX3aUZy6tp1jhEVo/keg8E//JL7fucuDQmx4TVi7oOGA?= =?us-ascii?Q?/zyVwDRHbkWMkxO+SIDkoQ7nwiLCJbn6AYIfm0AB0cYqFl5IHpms17kCRKdV?= =?us-ascii?Q?2J8EqsfQakP0XGeywXeTQcUBFLU4BSaqX0kdJJxDhQvhrXmSUjq0boz3Nrx5?= =?us-ascii?Q?9ETTKdSGUm27a6pEzm+gaJWrjgiiwc4HrPLu7fXgMXPlUhOQ84BV9CueTXki?= =?us-ascii?Q?lHGdSH+WkX77eCY8UWSPWGReuxrafc6wUQd0Jr+Lz/8nDcsNnJMvU1F06lhd?= =?us-ascii?Q?p3Hj3Vs+LcumJnujs485zVOF30VeOTsyisTpw5xWDGK4BBzMnORJmeWrz/pt?= =?us-ascii?Q?0jZB+vB5H53KH7Xva6OoUWh9qhZY3NIkt1PddsytVRLEufHFrzcgScyc7RXV?= =?us-ascii?Q?n7uXGJVjmfvP6HyAwUC2UZUKk23KPQCIejk9P+ND1lsbNlRixNGCgQb0qZVE?= =?us-ascii?Q?4V3/84/Y6LdQovR7/1dkcWsXxnyB+uNPqE/mCzw/MK/dQmUHWFjqKqsDb3IX?= =?us-ascii?Q?CPyN+YZdkRk8lrCa8GMyFmVMGde7b1yDgntLewcFpgs9jaEnR+mNCIY1abKS?= =?us-ascii?Q?KSzs2/bz0hEbuMoLHcR9iRL4qZq06F4c1YnWWYQ2MfIGUSm3P9I1wPEuSZ++?= =?us-ascii?Q?9I18493ZXnfBieQf2roq30N6h49MtTGcSUwYmUeFiIdT+b0YuWxPn9hIY5qx?= =?us-ascii?Q?+9If+89+x7EKkMHx0c/QXks4UQ4U0rtGbkn7TU7UBo/+8qe4lnGcK99Hthag?= =?us-ascii?Q?2zwAYz6rwv31kZqjjIRuDRiGy9Vr92w8M9AjjvdUEH86Jhad6DUvQE5PUys+?= =?us-ascii?Q?FJoSYlxiEcY49eQDYsSd+iefYNXJslu5NeQLaavzRsDftHpQJAwoehyiv2w1?= =?us-ascii?Q?M3vZPcqJ1wULZUsWvFVSeNjXx89k09DrYlgKBHU6?= MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: PH7PR11MB5863.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 993c2ce9-1ba3-4657-ea5c-08dc1bcb161a X-MS-Exchange-CrossTenant-originalarrivaltime: 23 Jan 2024 04:23:44.2343 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 1DGxiHPBnRO+Oqa97pQixgDmzlzCP+pWk7eG3Y2JUnR2GBgLzilNpqmbrbfo6Mxba5YYi3Lpn/dYxfiIT3H0LA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR11MB5924 X-OriginatorOrg: intel.com Precedence: Bulk List-Subscribe: List-Help: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,bob.c.feng@intel.com List-Unsubscribe-Post: List-Unsubscribe=One-Click List-Unsubscribe: X-Gm-Message-State: WFSvLIDLr0lLzD8e2pwPxyZax7686176AA= Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-GND-Status: LEGIT Authentication-Results: spool.mail.gandi.net; dkim=pass header.d=groups.io header.s=20140610 header.b=ZxxOWS2C; arc=reject ("signature check failed: fail, {[1] = sig:microsoft.com:reject}"); dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=intel.com (policy=none); spf=pass (spool.mail.gandi.net: domain of bounce@groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce@groups.io Hi Ashraf, Besides DSC, PCD value also comes from the build tool command line option, = DEC and INF. This patch looks only check the PCD changes from DSC, it's not= cover all the cases.=20 Thanks, Bob -----Original Message----- From: Kinney, Michael D =20 Sent: Tuesday, January 23, 2024 9:28 AM To: S, Ashraf Ali ; devel@edk2.groups.io Cc: Chen, Christine ; Rebecca Cran ; Liming Gao ; Feng, Bob C ; Chan, Amy ; Chaganty, Rangasai V ; Solanki, Digant H ; Kinney, Micha= el D Subject: RE: [edk2-devel] [PATCH] BaseTools: Optimize GenerateByteArrayValu= e and CollectPlatformGuids APIs Hi Ashraf, The PcdValueInit feature is not limited to only PCDs of type VPD. It is fo= r any structured PCDs. Did you test PCDs with all types (e.g. PcdsFixedAtB= uild PcdsPactahbleInModule, PcdsDynamicHii, PcdsDynamicDatabase, PcdsDynami= cVpd, PcdsDynamicExHii, PcdsDynamicExDatabase, PcdsDynamicExVpd) And did you DSC test cases cover both changes in PCD types and changes in P= CD values? Thanks, Mike > -----Original Message----- > From: S, Ashraf Ali > Sent: Sunday, January 21, 2024 4:14 AM > To: Kinney, Michael D ;=20 > devel@edk2.groups.io > Cc: Chen, Christine ; Rebecca Cran=20 > ; Liming Gao ; Feng, Bob=20 > C ; Chan, Amy ; Chaganty,=20 > Rangasai V ; Solanki, Digant H=20 > > Subject: RE: [edk2-devel] [PATCH] BaseTools: Optimize=20 > GenerateByteArrayValue and CollectPlatformGuids APIs >=20 > Hi., @Kinney, Michael D >=20 > The main API which is modified in this Patch is GenerateByteArrayValue. >=20 > This API is used to return the list of=20 > SKUID.TokenSpaceGuid.VpdName|VPD > STRUCT|Binary Data which will be stored in Output.txt > Example: > SAMPLESKUID.STANDARD.gEfiMdePkgTokenSpaceGuid.SampleVpd|SAMPLE_STRUCT[ > ]| > {0x01,0x01,0x05,0x09,0x02} >=20 > This VPD/PCD is coming from either the DEC file or the DSC file. >=20 > GenerateByteArrayValue API is used to create the PcdValueInit.exe and=20 > it's equivalent C and Make File. >=20 > When there is no change in DEC or DSC VPD/PCD still it used to do the=20 > same process again in the incremental build which is time consuming. > In my project this API is used to take 3min of time with High end=20 > server > (64Threads) and 3.5Min in Laptop (16 threads with performance mode > enabled) only for GenerateByteArrayValue API. >=20 > This patch will check if there are any change in the DEC/DSC=20 > VPDs/PCDs2, if there is any change in VPD the current flow is not=20 > affect. (it will create the PcdRecordList.json and copy Output.txt for=20 > Arch folder with the project. Ex: > Build\{Project}Pkg\DEBUG_VS2019\PcdValueInit\{IA32/X64/Any}\PcdRecordL > is > t.json and > Build\{Project}Pkg\DEBUG_VS2019\PcdValueInit\{IA32/X64/Any}\Output.txt > ) If there is no change the DSC/DEC VPDs it will read the data from=20 > Output.txt and return the same data. (we will compare the Input=20 > structure and PcdRecordList.json) if there is any mismatch then it=20 > there change in VPDs if not then no change in VPDs). >=20 > Unit testing from my side as follows: > * Build the Firmware. > * PcdRecordList.json and Output.txt will be created based on the=20 > Arch. > * Build the Firmware again without any change. > * It will read the Output.txt and return the data. (it will match=20 > Input PCD/VPD list and compare with existing PcdRecordList.json) > * I verified the return length and content GenerateByteArrayValue=20 > (it's same with and without my changes) > * Build the Firmware again my modifying the VPDs in DSC file > * Change in VPDs, it will regenerate PcdRecordList.json file > * it will create the Exe file and Output.txt > * New Output.txt will be copied to above path. > * Build the Firmware again by modifying the VPDs in DEC file. > * Change in VPDs, it will regenerate PcdRecordList.json file > * it will create the Exe file and Output.txt > * New Output.txt will be copied to above path. > * Build the Firmware again without modifying the code. > * It will read the Output.txt and return the data. > * I verified the return length and content GenerateByteArrayValue=20 > (it's same with and without my changes) >=20 >=20 > There is no impact on the Boot/cache. > This patch is used to reduce the incremental build time by checking if=20 > the VPDs/PCDs are changed or not, if not changed then it will return=20 > the previous stored data. >=20 > Thanks., > S, Ashraf Ali >=20 > -----Original Message----- > From: Kinney, Michael D > Sent: Saturday, January 20, 2024 7:36 AM > To: devel@edk2.groups.io; S, Ashraf Ali > Cc: Chen, Christine ; Rebecca Cran=20 > ; Liming Gao ; Feng, Bob=20 > C ; Chan, Amy ; Chaganty,=20 > Rangasai V ; Solanki, Digant H=20 > ; Kinney, Michael D=20 > > Subject: RE: [edk2-devel] [PATCH] BaseTools: Optimize=20 > GenerateByteArrayValue and CollectPlatformGuids APIs >=20 > Hi Ashraf, >=20 > What is captured in the file? >=20 > What PCD/VPD changes will invalidate the cache? Just the number and=20 > type of PCD/VPD elements or their default values/sizes? >=20 > How was this tested? Were all conditions that invalidate the cache=20 > tested? I ask because incremental build is a very important feature=20 > and if there is any logic error in the cache management of a file like=20 > this, it will cause unexpected behavior and developers will not trust=20 > incremental builds. >=20 > Mike >=20 > > -----Original Message----- > > From: devel@edk2.groups.io On Behalf Of=20 > > Ashraf Ali S > > Sent: Tuesday, January 16, 2024 11:55 PM > > To: devel@edk2.groups.io > > Cc: S, Ashraf Ali ; Chen, Christine=20 > > ; Rebecca Cran ; Liming Gao=20 > > ; Feng, Bob C ;=20 > > Chan, Amy ; Chaganty, Rangasai V=20 > > ; Solanki, Digant H=20 > > > > Subject: [edk2-devel] [PATCH] BaseTools: Optimize=20 > > GenerateByteArrayValue and CollectPlatformGuids APIs > > > > During the Incremental build GenerateByteArrayValue used to generate=20 > > the ByteArrayValue even when there is no change in the PCD/VPDs.=20 > > which is time consuming API based on the number of PCD/VPDs and SKU IDs= . > > > > The optimization is that GenerateByteArrayValue is used to store the=20 > > PcdRecordList in a JSON file for each of the arch. and during the=20 > > Incremental build this API will check if there is any change in the=20 > > PCD /VPDs then rest of the flow remains the same. if there is no=20 > > change then it will return the provious build data. > > > > Flow: > > during the 1st build PcdRecordList.json is not exists, PcdRecordList=20 > > will be dumped to json file. and it will copy the output.txt as well. > > Note: as the output.txt are different for different Arch, so it will=20 > > be stored in the Arch folder. > > During the Incremental build check if there is any change in PCD/VPD. > > if there is a change in VPD/PCD then recreate the PcdRecordList.json. > > and rest of the flow remains same. > > if there is no change in VPD/PCD read the output.txt and return the=20 > > data > > > > Cc: Yuwei Chen > > Cc: Rebecca Cran > > Cc: Liming Gao > > Cc: Bob Feng > > Cc: Amy Chan > > Cc: Sai Chaganty > > Cc: Digant H Solanki > > Signed-off-by: Ashraf Ali S > > --- > > .../Source/Python/AutoGen/WorkspaceAutoGen.py | 16 ++--- > > .../Source/Python/Workspace/DscBuildData.py | 72 +++++++++++++++--- > - > > 2 files changed, 64 insertions(+), 24 deletions(-) > > > > diff --git a/BaseTools/Source/Python/AutoGen/WorkspaceAutoGen.py > > b/BaseTools/Source/Python/AutoGen/WorkspaceAutoGen.py > > index 160e3a3cd3..eec9280c8e 100644 > > --- a/BaseTools/Source/Python/AutoGen/WorkspaceAutoGen.py > > +++ b/BaseTools/Source/Python/AutoGen/WorkspaceAutoGen.py > > @@ -160,22 +160,18 @@ class WorkspaceAutoGen(AutoGen): > > > > def CollectPlatformGuids(self): > > oriInfList =3D [] > > - oriPkgSet =3D set() > > - PlatformPkg =3D set() > > + pkgSet =3D set() > > for Arch in self.ArchList: > > Platform =3D self.BuildDatabase[self.MetaFile, Arch,=20 > > self.BuildTarget, self.ToolChain] > > oriInfList =3D Platform.Modules > > for ModuleFile in oriInfList: > > ModuleData =3D self.BuildDatabase[ModuleFile,=20 > > Platform._Arch, Platform._Target, Platform._Toolchain] > > - oriPkgSet.update(ModuleData.Packages) > > - for Pkg in oriPkgSet: > > - Guids =3D Pkg.Guids > > - GlobalData.gGuidDict.update(Guids) > > + pkgSet.update(ModuleData.Packages) > > if Platform.Packages: > > - PlatformPkg.update(Platform.Packages) > > - for Pkg in PlatformPkg: > > - Guids =3D Pkg.Guids > > - GlobalData.gGuidDict.update(Guids) > > + pkgSet.update(Platform.Packages) > > + for Pkg in pkgSet: > > + Guids =3D Pkg.Guids > > + GlobalData.gGuidDict.update(Guids) > > > > @cached_property > > def FdfProfile(self): > > diff --git a/BaseTools/Source/Python/Workspace/DscBuildData.py > > b/BaseTools/Source/Python/Workspace/DscBuildData.py > > index 4768099343..740b8e22be 100644 > > --- a/BaseTools/Source/Python/Workspace/DscBuildData.py > > +++ b/BaseTools/Source/Python/Workspace/DscBuildData.py > > @@ -37,6 +37,8 @@ from functools import reduce from Common.Misc=20 > > import SaveFileOnChange from Workspace.BuildClassObject import=20 > > PlatformBuildClassObject, StructurePcd, PcdClassObject,=20 > > ModuleBuildClassObject from collections import OrderedDict,=20 > > defaultdict > > +import json > > +import shutil > > > > def _IsFieldValueAnArray (Value): > > Value =3D Value.strip() > > @@ -56,6 +58,7 @@ def _IsFieldValueAnArray (Value): > > > > PcdValueInitName =3D 'PcdValueInit' > > PcdValueCommonName =3D 'PcdValueCommon' > > +PcdRecordListName =3D 'PcdRecordList.json' > > > > PcdMainCHeader =3D ''' > > /** > > @@ -1599,7 +1602,7 @@ class DscBuildData(PlatformBuildClassObject): > > S_pcd_set =3D DscBuildData.OverrideByComm(S_pcd_set) > > > > # Create a tool to caculate structure pcd value > > - Str_Pcd_Values =3D self.GenerateByteArrayValue(S_pcd_set) > > + Str_Pcd_Values =3D self.GenerateByteArrayValue(S_pcd_set, > > RecordList) > > > > if Str_Pcd_Values: > > for (skuname, StoreName, PcdGuid, PcdName, PcdValue) in > > Str_Pcd_Values: > > @@ -2750,12 +2753,61 @@ class DscBuildData(PlatformBuildClassObject): > > ccflags.add(item) > > i +=3D1 > > return ccflags > > - def GenerateByteArrayValue (self, StructuredPcds): > > + > > + def GetStructurePcdSet (self, OutputValueFile): > > + if not os.path.isfile(OutputValueFile): > > + EdkLogger.error("GetStructurePcdSet", FILE_NOT_FOUND, > > "Output.txt doesn't exist", ExtraData=3DOutputValueFile) > > + return [] > > + File =3D open (OutputValueFile, 'r') > > + FileBuffer =3D File.readlines() > > + File.close() > > + > > + #start update structure pcd final value > > + StructurePcdSet =3D [] > > + for Pcd in FileBuffer: > > + PcdValue =3D Pcd.split ('|') > > + PcdInfo =3D PcdValue[0].split ('.') > > + StructurePcdSet.append((PcdInfo[0], PcdInfo[1],=20 > > + PcdInfo[2], > > PcdInfo[3], PcdValue[2].strip())) > > + return StructurePcdSet > > + > > + def GenerateByteArrayValue (self, StructuredPcds, PcdRecordList): > > # > > # Generate/Compile/Run C application to determine if there=20 > > are any flexible array members > > # > > if not StructuredPcds: > > return > > + # > > + # If the output path doesn't exists then create it > > + # > > + if not os.path.exists(self.OutputPath): > > + os.makedirs(self.OutputPath) > > + > > + PcdRecordListPath =3D os.path.join(self.OutputPath,=20 > > + self._Arch, > > PcdRecordListName) > > + PcdRecordOutputValueFile =3D os.path.join(self.OutputPath, > > self._Arch, 'Output.txt') > > + > > + if not os.path.exists(os.path.dirname(PcdRecordListPath)): > > + os.makedirs(os.path.dirname(PcdRecordListPath)) > > + # > > + # Check if the PcdRecordList.json exists or not > > + # if exits then it might be a incremental build then check=20 > > + if > > the PcdRecord list has been changed or not. > > + # if changed then proceed further, if not changed then=20 > > + return > > the stored data from earlier build > > + # > > + if os.path.isfile(PcdRecordListPath): > > + with open(PcdRecordListPath, 'r') as file: > > + file_content_str =3D file.read() > > + if file_content_str: > > + # Use json.loads to convert the string back to=20 > > + a > > list > > + file_content =3D json.loads(file_content_str) > > + # Check if all PcdRecordList in record_set are > > present in file_content > > + # and if there are no extra PcdRecordList in > > file_content > > + if set(map(tuple, PcdRecordList)) =3D=3D=20 > > + set(map(tuple, > > file_content)): > > + return > > self.GetStructurePcdSet(PcdRecordOutputValueFile) > > + # > > + # 1st build, create the PcdRecordList.json > > + # update the record as PCD Input has been changed in > > incremental build > > + # > > + with open(PcdRecordListPath, 'w') as file: > > + json.dump(PcdRecordList, file) > > > > InitByteValue =3D "" > > CApp =3D PcdMainCHeader > > @@ -2832,8 +2884,6 @@ class DscBuildData(PlatformBuildClassObject): > > > > CApp =3D CApp + PcdMainCEntry + '\n' > > > > - if not os.path.exists(self.OutputPath): > > - os.makedirs(self.OutputPath) > > CAppBaseFileName =3D os.path.join(self.OutputPath, > > PcdValueInitName) > > SaveFileOnChange(CAppBaseFileName + '.c', CApp, False) > > > > @@ -3042,17 +3092,11 @@ class DscBuildData(PlatformBuildClassObject): > > if returncode !=3D 0: > > EdkLogger.warn('Build', COMMAND_FAILURE, 'Can not=20 > > collect output from command: %s\n%s\n%s\n' % (Command, StdOut, > > StdErr)) > > > > - #start update structure pcd final value > > - File =3D open (OutputValueFile, 'r') > > - FileBuffer =3D File.readlines() > > - File.close() > > + # Copy output file for each Arch > > + shutil.copyfile(OutputValueFile, PcdRecordOutputValueFile) > > > > - StructurePcdSet =3D [] > > - for Pcd in FileBuffer: > > - PcdValue =3D Pcd.split ('|') > > - PcdInfo =3D PcdValue[0].split ('.') > > - StructurePcdSet.append((PcdInfo[0], PcdInfo[1], > PcdInfo[2], > > PcdInfo[3], PcdValue[2].strip())) > > - return StructurePcdSet > > + # return structure pcd final value > > + return self.GetStructurePcdSet(OutputValueFile) > > > > @staticmethod > > def NeedUpdateOutput(OutputFile, ValueCFile, StructureInput): > > -- > > 2.39.1.windows.1 > > > > > > > >=20 > > -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#114176): https://edk2.groups.io/g/devel/message/114176 Mute This Topic: https://groups.io/mt/103781798/7686176 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-