From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail02.groups.io (mail02.groups.io [66.175.222.108]) by spool.mail.gandi.net (Postfix) with ESMTPS id 34B51AC14A6 for ; Sun, 21 Jan 2024 12:13:39 +0000 (UTC) DKIM-Signature: a=rsa-sha256; bh=LbWu+9spczpImmfOiwF0t0pRvxDSfYNnlnCUbOVasKw=; c=relaxed/simple; d=groups.io; h=ARC-Seal:ARC-Message-Signature:ARC-Authentication-Results:From:To:CC:Subject:Thread-Topic:Thread-Index:Date:Message-ID:References:In-Reply-To:Accept-Language:MIME-Version:Precedence:List-Subscribe:List-Help:Sender:List-Id:Mailing-List:Delivered-To:Reply-To:List-Unsubscribe-Post:List-Unsubscribe:Content-Language:Content-Type:Content-Transfer-Encoding; s=20140610; t=1705839217; v=1; b=NtaUIbFSN+Tx30WaIQS5kquu6ukrJHTi1TocbhH+TxCg4cZUOBGcGVEylAFZyWW7YxixNn5E 9I8uSEMQ8wJ29cctgf7QESrN7pljC6er/n0RoZ/a3mtEpLn+BQI2kHd/rHl9KucszV/6nVybEmR 8jQG7af0/cN2YhnIpcnq/qTA= X-Received: by 127.0.0.2 with SMTP id 9TSeYY7687511xYkkEOgfnYc; Sun, 21 Jan 2024 04:13:37 -0800 X-Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.65]) by mx.groups.io with SMTP id smtpd.web11.45623.1705839216749976663 for ; Sun, 21 Jan 2024 04:13:37 -0800 X-IronPort-AV: E=McAfee;i="6600,9927,10959"; a="404786281" X-IronPort-AV: E=Sophos;i="6.05,209,1701158400"; d="scan'208";a="404786281" X-Received: from fmviesa001.fm.intel.com ([10.60.135.141]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Jan 2024 04:13:35 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,209,1701158400"; d="scan'208";a="27406331" X-Received: from orsmsx601.amr.corp.intel.com ([10.22.229.14]) by fmviesa001.fm.intel.com with ESMTP/TLS/AES256-GCM-SHA384; 21 Jan 2024 04:13:35 -0800 X-Received: from orsmsx610.amr.corp.intel.com (10.22.229.23) by ORSMSX601.amr.corp.intel.com (10.22.229.14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Sun, 21 Jan 2024 04:13:34 -0800 X-Received: from ORSEDG602.ED.cps.intel.com (10.7.248.7) by orsmsx610.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35 via Frontend Transport; Sun, 21 Jan 2024 04:13:34 -0800 X-Received: from NAM11-DM6-obe.outbound.protection.outlook.com (104.47.57.169) by edgegateway.intel.com (134.134.137.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.35; Sun, 21 Jan 2024 04:13:34 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ie2S/RKLyPEdCVKIreo26hH05nLWqwDmaGF4AXFEUOEPINtR/asf7mlof8UbQprr7YlKyZSql/5acDSDzQwesbJ6ACpDAWgyBXpNFE9v/IxAogg1ct8LjU62dn7yZm7nqsEfj98ZwXIZZEHTJcSjs2OmTQkC8oh7a5P1xa5U5v63gLKTrh4kvAtUV4Jl5TPGdao6Fg4/ccMb2YWOdmJ5g6Y/rsTVstIbGsw07bVs3S7xDrTzoxGIRRddSJg8dL4t5z0WeY20gv8Wlz/2ONWEZurkz2qSoUVhTAv21ikhuzy8cEe0yRNLEUR6aYCuj6eXUGVPogV8yKdIKERrC1CnsQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Pbi9fx/c4coKEdPNOcE6B4ZhhT55wnPa3EmFQTpbwBw=; b=OpxgNjvvmZO4i0HCxt5FHqj6nZtIBqcl79P6Xy6V7tGQLdxD3ODT0sH6otZQtFgtV0wDbqPjH2SyzHNOx+ZQnFdsseXQ2NmVFDvsOyhbHI+ggCqiPAcXay/qRztYbpvDZp/dBsKAJokGAhxEehwDJ0qa8QwRhsS3gHdxY+VNtFq0GtCiATLWA84o2kC5ELl2ERTWS5xfIh5F+SODlmIvoU94w9QUHkViQB2DHUr7SBSPjSaMlu3oX3R4L4IqtyqAsvufQpOIOtDC4VM8snRGue7QbuPJi7gsBdXoXogXo2X0tzT/zn6O3R1adl8pNjVWYrcSxzhkmyzC3d/iNj08yw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none X-Received: from DM4PR11MB5280.namprd11.prod.outlook.com (2603:10b6:5:38b::18) by IA1PR11MB7677.namprd11.prod.outlook.com (2603:10b6:208:3fd::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7202.31; Sun, 21 Jan 2024 12:13:32 +0000 X-Received: from DM4PR11MB5280.namprd11.prod.outlook.com ([fe80::13e7:7c59:f658:63ba]) by DM4PR11MB5280.namprd11.prod.outlook.com ([fe80::13e7:7c59:f658:63ba%4]) with mapi id 15.20.7202.028; Sun, 21 Jan 2024 12:13:32 +0000 From: "Ashraf Ali S" To: "Kinney, Michael D" , "devel@edk2.groups.io" CC: "Chen, Christine" , Rebecca Cran , Liming Gao , "Feng, Bob C" , "Chan, Amy" , "Chaganty, Rangasai V" , "Solanki, Digant H" Subject: Re: [edk2-devel] [PATCH] BaseTools: Optimize GenerateByteArrayValue and CollectPlatformGuids APIs Thread-Topic: [edk2-devel] [PATCH] BaseTools: Optimize GenerateByteArrayValue and CollectPlatformGuids APIs Thread-Index: AQHaSRqF3r14eWumqkeZ1CVqlRGnJ7Dh+COAgAIPRmA= Date: Sun, 21 Jan 2024 12:13:31 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US X-Mentions: michael.d.kinney@intel.com X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-traffictypediagnostic: DM4PR11MB5280:EE_|IA1PR11MB7677:EE_ x-ms-office365-filtering-correlation-id: 4757e69b-442a-4e7c-9a4f-08dc1a7a626a x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam-message-info: Kbxs8nqXknsY9vQkvQuhA628txIpVpxf/PKzDEfHkoAI9TbhVyVsLQ4Qexb4571ckQHzoCYx0QOJScoLW6isGqMngrqg8D2fajbSbeJ4Z9lf4fQKxNaw9c8WbXKwk/gBhHywQlwF51FF7LHU417jZmvi+6UyABr+/p+cMtwkUSDjkx6TFm1WmQI2xII6yzrhsQtN8+vVmOzY4H1jilgjqzPJOjTUr04wlnq5t7SmvHjcMX3JzEcf/dcxafSW6i6sAXm1hoXcds7wv/jW7WqqRWQyIkzsl3VNc88yfH0KZLFl+w+W1Mr9uCmHUfBEx98wmGgKotoOMtFIHmnsLneBHE0Vlr+3dTikOJfsgXCYQqIy/Elhp0TFopbb/a1aB1ZUpaBfNdkH/piN3CSrjB4txgKpFN6FQq0rwOpDsIY3xJGCXxQzIUtVmUjn+5L/FdnFQMfgqXM28K9UrjqjdJuIe/fvah9L2BIANmLBJyL5e1Wnu26VZBQ0tKt3mvm4wClyLQ5TIw3YHGVCvpzqFzZWqlaNoHCBtlryO5D0+Y6unt0eq2CPxmOzaNyWtnpef66+eUmRa+dhaTPYSiV9oqmuoR2VLkEQCABbuBZDhCld9AB5KpMfVTWrCuLSPY2JYk1LKL+T+rfm+zlCjv64dJZ+/tDiSXgY/BFr8SxIzs7vvzo= x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?3cLPMGX50gwY2pONrUmCrATwanXiJIrLL65NkpfqWnETM8TVejtZRGDcMh7/?= =?us-ascii?Q?H9Wcoy8I6+ESUMTMCcS/oAdZxeOpG9M2v4u6ueosIanFQK6XGETlfciGuGgu?= =?us-ascii?Q?73TpXIzvFCLeulRas/erbw+82OaGYdrpGyep/Fczr9LDF9G2GLiuvX8IoUKB?= =?us-ascii?Q?pnN+tLCzZkQ4dDtwnHBSZLQN1sTzdrX/oJqL+JRc9c5KWbTwxkHn9vjMqIBZ?= =?us-ascii?Q?F4WH4bApliHHO/Yeze5QEZ5xMwszVK3khtVvqIC3XqhM4qLCIKuLBC0ZQdXU?= =?us-ascii?Q?1DmT197twsnTWkQa3wtCCwCGa33gz9f1FvnvyCM2j5Oh9zNIcSFgTgBxt6BR?= =?us-ascii?Q?nmt0HVj3CSWofX3XAUhDjjWmmrvxSZPNhNvk/AqoYeh8Ntl5JGetdJN8tTms?= =?us-ascii?Q?95bk7g1D70e1C2Im39fjF8SjECWypddjTlfuetZzWeYPn8kKhQmYaEmoTiI2?= =?us-ascii?Q?n0+s9I3JKeu4sXAyiSkXOYvZXvkwTlkk424WjugvrDEEi+YjIMQbc4mOGlcy?= =?us-ascii?Q?7FcjWaafZU/nNATSRnyxq/97YSnYLVf7GW2pDF6sZENyk8faN8pFYqZjjFFX?= =?us-ascii?Q?xz95+3B43SSnnjajp58DpHo7APLOy2BBchTtY69v3vcC0nqnkvUIHi08xETd?= =?us-ascii?Q?W+6khibg63t7Yej8iPehZZoouYeVHOz+RzlNGDts2J2B5CWAW46QBqPSIoQ/?= =?us-ascii?Q?VN8y/1huW6MCZZuD9f+7tkVOoeiYxvYEfM9m9O8JJJWlN/w6C1uhu4+ydAE5?= =?us-ascii?Q?fHNbDexaUsQXzdA+mMygtVUATX2PhCUVqlhpi/l2oecXX4+c3zf5SHLJCsmg?= =?us-ascii?Q?zAVEDra3ytfOef60gbVpOn/7euFVJ7k7U8rtFhCawX7bJ2WDkI6r9qZoOGeM?= =?us-ascii?Q?KfNdbVomJoMkeZduGmY3LFiAy9lT8R/oTgFFALOJIKdyiUMFLMqiOgdq/PiK?= =?us-ascii?Q?4xoca5/99WQU7pSkrTjYSIJ+ddPK88ALQV4ARCg+NvoMgiw4nIN7A1GupUjS?= =?us-ascii?Q?J1DIEbgxR96QxX7LzH2UgRs42lLRP1UglaYWKK4kZAzY48GGvGkwWyhB/BIM?= =?us-ascii?Q?GQ10lfVA4AsXndglSOUagvgYPtn3P2L9M0TE5V1JCUR9F9e/3IW5livGzdsk?= =?us-ascii?Q?eD+TZ9YLxpwy30CFjkSFj/Wu0k3X1I65/+o88TnrEtXJ7FHJF4co2IItGwsa?= =?us-ascii?Q?XuI5cBferimLa0D9y8Ck0au+gmZkPo1ztwvsz/0iwTX1CZ8JOyy67R3qfbAI?= =?us-ascii?Q?5qtSSLlDuaZJof2/lxJzyR4JuZeeBZ0AXcS0n9WC9LjvxvIPFg8zuYlr+WZb?= =?us-ascii?Q?Pnsdu/tzpa6AxjYH0cSg3LEB4CR5bOcjcuIDs4UOYNX6n19Qa+9Z3RwSaiD2?= =?us-ascii?Q?Si3fYJSpCBm3MG8sQtyu7wCcko8hvG1Vit34NHauUiR9BIu3oQ39cyeCU0cP?= =?us-ascii?Q?KU4JxgbTVpHYoSINBVHZ+mn2qatKxjQsmjfqPALirNlbbQE6so/pgLtOXt3C?= =?us-ascii?Q?sVn7b1P3yABOg+yjl7N78JjTUhfD2GprEIC9/irIQhweuVWO9uOiG9/No1k9?= =?us-ascii?Q?UAqSd8Y/AIz2QO5HDtmpraYFZLbUUcyUYXOSi9Id?= MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM4PR11MB5280.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4757e69b-442a-4e7c-9a4f-08dc1a7a626a X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Jan 2024 12:13:31.8628 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: PK7CY2nk9LmRV5uU+fI9czDd9uicPOtFNdHVxbJK8OzsaRPramIKxy6TJipV6imhT4VL7XyMWnwwR5rHPw+Y4A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR11MB7677 X-OriginatorOrg: intel.com Precedence: Bulk List-Subscribe: List-Help: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,ashraf.ali.s@intel.com List-Unsubscribe-Post: List-Unsubscribe=One-Click List-Unsubscribe: X-Gm-Message-State: IblwIUPDrfFLl47g6BuHOJkXx7686176AA= Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-GND-Status: LEGIT Authentication-Results: spool.mail.gandi.net; dkim=pass header.d=groups.io header.s=20140610 header.b=NtaUIbFS; arc=reject ("signature check failed: fail, {[1] = sig:microsoft.com:reject}"); dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=intel.com (policy=none); spf=pass (spool.mail.gandi.net: domain of bounce@groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce@groups.io Hi., @Kinney, Michael D The main API which is modified in this Patch is GenerateByteArrayValue. This API is used to return the list of SKUID.TokenSpaceGuid.VpdName|VPD STR= UCT|Binary Data which will be stored in Output.txt Example: SAMPLESKUID.STANDARD.gEfiMdePkgTokenSpaceGuid.SampleVpd|SAMPLE_STR= UCT[]|{0x01,0x01,0x05,0x09,0x02} This VPD/PCD is coming from either the DEC file or the DSC file. GenerateByteArrayValue API is used to create the PcdValueInit.exe and it's = equivalent C and Make File. When there is no change in DEC or DSC VPD/PCD still it used to do the same = process again in the incremental build which is time consuming. In my project this API is used to take 3min of time with High end server (6= 4Threads) and 3.5Min in Laptop (16 threads with performance mode enabled) o= nly for GenerateByteArrayValue API. This patch will check if there are any change in the DEC/DSC VPDs/PCDs2, i= f there is any change in VPD the current flow is not affect. (it will creat= e the PcdRecordList.json and copy Output.txt for Arch folder with the proje= ct. Ex: Build\{Project}Pkg\DEBUG_VS2019\PcdValueInit\{IA32/X64/Any}\PcdReco= rdList.json and Build\{Project}Pkg\DEBUG_VS2019\PcdValueInit\{IA32/X64/Any}= \Output.txt) If there is no change the DSC/DEC VPDs it will read the data from Output.tx= t and return the same data. (we will compare the Input structure and PcdRec= ordList.json) if there is any mismatch then it there change in VPDs if not = then no change in VPDs). Unit testing from my side as follows: * Build the Firmware. * PcdRecordList.json and Output.txt will be created based on the Arch. * Build the Firmware again without any change. * It will read the Output.txt and return the data. (it will match Input PC= D/VPD list and compare with existing PcdRecordList.json) * I verified the return length and content GenerateByteArrayValue (it's sa= me with and without my changes) * Build the Firmware again my modifying the VPDs in DSC file * Change in VPDs, it will regenerate PcdRecordList.json file * it will create the Exe file and Output.txt * New Output.txt will be copied to above path. * Build the Firmware again by modifying the VPDs in DEC file. * Change in VPDs, it will regenerate PcdRecordList.json file * it will create the Exe file and Output.txt * New Output.txt will be copied to above path. * Build the Firmware again without modifying the code. * It will read the Output.txt and return the data. * I verified the return length and content GenerateByteArrayValue (it's sa= me with and without my changes) There is no impact on the Boot/cache. This patch is used to reduce the incremental build time by checking if the = VPDs/PCDs are changed or not, if not changed then it will return the previo= us stored data. Thanks., S, Ashraf Ali -----Original Message----- From: Kinney, Michael D =20 Sent: Saturday, January 20, 2024 7:36 AM To: devel@edk2.groups.io; S, Ashraf Ali Cc: Chen, Christine ; Rebecca Cran ; Liming Gao ; Feng, Bob C ; Chan, Amy ; Chaganty, Rangasai V ; Solanki, Digant H ; Kinney, Micha= el D Subject: RE: [edk2-devel] [PATCH] BaseTools: Optimize GenerateByteArrayValu= e and CollectPlatformGuids APIs Hi Ashraf, What is captured in the file? What PCD/VPD changes will invalidate the cache? Just the number and type o= f PCD/VPD elements or their default values/sizes? How was this tested? Were all conditions that invalidate the cache tested?= I ask because incremental build is a very important feature and if there = is any logic error in the cache management of a file like this, it will cau= se unexpected behavior and developers will not trust incremental builds. Mike > -----Original Message----- > From: devel@edk2.groups.io On Behalf Of Ashraf=20 > Ali S > Sent: Tuesday, January 16, 2024 11:55 PM > To: devel@edk2.groups.io > Cc: S, Ashraf Ali ; Chen, Christine=20 > ; Rebecca Cran ; Liming Gao=20 > ; Feng, Bob C ; Chan,=20 > Amy ; Chaganty, Rangasai V=20 > ; Solanki, Digant H=20 > > Subject: [edk2-devel] [PATCH] BaseTools: Optimize=20 > GenerateByteArrayValue and CollectPlatformGuids APIs >=20 > During the Incremental build GenerateByteArrayValue used to generate=20 > the ByteArrayValue even when there is no change in the PCD/VPDs. which=20 > is time consuming API based on the number of PCD/VPDs and SKU IDs. >=20 > The optimization is that GenerateByteArrayValue is used to store the=20 > PcdRecordList in a JSON file for each of the arch. and during the=20 > Incremental build this API will check if there is any change in the=20 > PCD /VPDs then rest of the flow remains the same. if there is no=20 > change then it will return the provious build data. >=20 > Flow: > during the 1st build PcdRecordList.json is not exists, PcdRecordList=20 > will be dumped to json file. and it will copy the output.txt as well. > Note: as the output.txt are different for different Arch, so it will=20 > be stored in the Arch folder. > During the Incremental build check if there is any change in PCD/VPD. > if there is a change in VPD/PCD then recreate the PcdRecordList.json. > and rest of the flow remains same. > if there is no change in VPD/PCD read the output.txt and return the=20 > data >=20 > Cc: Yuwei Chen > Cc: Rebecca Cran > Cc: Liming Gao > Cc: Bob Feng > Cc: Amy Chan > Cc: Sai Chaganty > Cc: Digant H Solanki > Signed-off-by: Ashraf Ali S > --- > .../Source/Python/AutoGen/WorkspaceAutoGen.py | 16 ++--- > .../Source/Python/Workspace/DscBuildData.py | 72 +++++++++++++++---- > 2 files changed, 64 insertions(+), 24 deletions(-) >=20 > diff --git a/BaseTools/Source/Python/AutoGen/WorkspaceAutoGen.py > b/BaseTools/Source/Python/AutoGen/WorkspaceAutoGen.py > index 160e3a3cd3..eec9280c8e 100644 > --- a/BaseTools/Source/Python/AutoGen/WorkspaceAutoGen.py > +++ b/BaseTools/Source/Python/AutoGen/WorkspaceAutoGen.py > @@ -160,22 +160,18 @@ class WorkspaceAutoGen(AutoGen): >=20 > def CollectPlatformGuids(self): > oriInfList =3D [] > - oriPkgSet =3D set() > - PlatformPkg =3D set() > + pkgSet =3D set() > for Arch in self.ArchList: > Platform =3D self.BuildDatabase[self.MetaFile, Arch,=20 > self.BuildTarget, self.ToolChain] > oriInfList =3D Platform.Modules > for ModuleFile in oriInfList: > ModuleData =3D self.BuildDatabase[ModuleFile,=20 > Platform._Arch, Platform._Target, Platform._Toolchain] > - oriPkgSet.update(ModuleData.Packages) > - for Pkg in oriPkgSet: > - Guids =3D Pkg.Guids > - GlobalData.gGuidDict.update(Guids) > + pkgSet.update(ModuleData.Packages) > if Platform.Packages: > - PlatformPkg.update(Platform.Packages) > - for Pkg in PlatformPkg: > - Guids =3D Pkg.Guids > - GlobalData.gGuidDict.update(Guids) > + pkgSet.update(Platform.Packages) > + for Pkg in pkgSet: > + Guids =3D Pkg.Guids > + GlobalData.gGuidDict.update(Guids) >=20 > @cached_property > def FdfProfile(self): > diff --git a/BaseTools/Source/Python/Workspace/DscBuildData.py > b/BaseTools/Source/Python/Workspace/DscBuildData.py > index 4768099343..740b8e22be 100644 > --- a/BaseTools/Source/Python/Workspace/DscBuildData.py > +++ b/BaseTools/Source/Python/Workspace/DscBuildData.py > @@ -37,6 +37,8 @@ from functools import reduce from Common.Misc=20 > import SaveFileOnChange from Workspace.BuildClassObject import=20 > PlatformBuildClassObject, StructurePcd, PcdClassObject,=20 > ModuleBuildClassObject from collections import OrderedDict,=20 > defaultdict > +import json > +import shutil >=20 > def _IsFieldValueAnArray (Value): > Value =3D Value.strip() > @@ -56,6 +58,7 @@ def _IsFieldValueAnArray (Value): >=20 > PcdValueInitName =3D 'PcdValueInit' > PcdValueCommonName =3D 'PcdValueCommon' > +PcdRecordListName =3D 'PcdRecordList.json' >=20 > PcdMainCHeader =3D ''' > /** > @@ -1599,7 +1602,7 @@ class DscBuildData(PlatformBuildClassObject): > S_pcd_set =3D DscBuildData.OverrideByComm(S_pcd_set) >=20 > # Create a tool to caculate structure pcd value > - Str_Pcd_Values =3D self.GenerateByteArrayValue(S_pcd_set) > + Str_Pcd_Values =3D self.GenerateByteArrayValue(S_pcd_set, > RecordList) >=20 > if Str_Pcd_Values: > for (skuname, StoreName, PcdGuid, PcdName, PcdValue) in > Str_Pcd_Values: > @@ -2750,12 +2753,61 @@ class DscBuildData(PlatformBuildClassObject): > ccflags.add(item) > i +=3D1 > return ccflags > - def GenerateByteArrayValue (self, StructuredPcds): > + > + def GetStructurePcdSet (self, OutputValueFile): > + if not os.path.isfile(OutputValueFile): > + EdkLogger.error("GetStructurePcdSet", FILE_NOT_FOUND, > "Output.txt doesn't exist", ExtraData=3DOutputValueFile) > + return [] > + File =3D open (OutputValueFile, 'r') > + FileBuffer =3D File.readlines() > + File.close() > + > + #start update structure pcd final value > + StructurePcdSet =3D [] > + for Pcd in FileBuffer: > + PcdValue =3D Pcd.split ('|') > + PcdInfo =3D PcdValue[0].split ('.') > + StructurePcdSet.append((PcdInfo[0], PcdInfo[1],=20 > + PcdInfo[2], > PcdInfo[3], PcdValue[2].strip())) > + return StructurePcdSet > + > + def GenerateByteArrayValue (self, StructuredPcds, PcdRecordList): > # > # Generate/Compile/Run C application to determine if there=20 > are any flexible array members > # > if not StructuredPcds: > return > + # > + # If the output path doesn't exists then create it > + # > + if not os.path.exists(self.OutputPath): > + os.makedirs(self.OutputPath) > + > + PcdRecordListPath =3D os.path.join(self.OutputPath, self._Arch, > PcdRecordListName) > + PcdRecordOutputValueFile =3D os.path.join(self.OutputPath, > self._Arch, 'Output.txt') > + > + if not os.path.exists(os.path.dirname(PcdRecordListPath)): > + os.makedirs(os.path.dirname(PcdRecordListPath)) > + # > + # Check if the PcdRecordList.json exists or not > + # if exits then it might be a incremental build then check if > the PcdRecord list has been changed or not. > + # if changed then proceed further, if not changed then return > the stored data from earlier build > + # > + if os.path.isfile(PcdRecordListPath): > + with open(PcdRecordListPath, 'r') as file: > + file_content_str =3D file.read() > + if file_content_str: > + # Use json.loads to convert the string back to a > list > + file_content =3D json.loads(file_content_str) > + # Check if all PcdRecordList in record_set are > present in file_content > + # and if there are no extra PcdRecordList in > file_content > + if set(map(tuple, PcdRecordList)) =3D=3D=20 > + set(map(tuple, > file_content)): > + return > self.GetStructurePcdSet(PcdRecordOutputValueFile) > + # > + # 1st build, create the PcdRecordList.json > + # update the record as PCD Input has been changed in > incremental build > + # > + with open(PcdRecordListPath, 'w') as file: > + json.dump(PcdRecordList, file) >=20 > InitByteValue =3D "" > CApp =3D PcdMainCHeader > @@ -2832,8 +2884,6 @@ class DscBuildData(PlatformBuildClassObject): >=20 > CApp =3D CApp + PcdMainCEntry + '\n' >=20 > - if not os.path.exists(self.OutputPath): > - os.makedirs(self.OutputPath) > CAppBaseFileName =3D os.path.join(self.OutputPath, > PcdValueInitName) > SaveFileOnChange(CAppBaseFileName + '.c', CApp, False) >=20 > @@ -3042,17 +3092,11 @@ class DscBuildData(PlatformBuildClassObject): > if returncode !=3D 0: > EdkLogger.warn('Build', COMMAND_FAILURE, 'Can not=20 > collect output from command: %s\n%s\n%s\n' % (Command, StdOut,=20 > StdErr)) >=20 > - #start update structure pcd final value > - File =3D open (OutputValueFile, 'r') > - FileBuffer =3D File.readlines() > - File.close() > + # Copy output file for each Arch > + shutil.copyfile(OutputValueFile, PcdRecordOutputValueFile) >=20 > - StructurePcdSet =3D [] > - for Pcd in FileBuffer: > - PcdValue =3D Pcd.split ('|') > - PcdInfo =3D PcdValue[0].split ('.') > - StructurePcdSet.append((PcdInfo[0], PcdInfo[1], PcdInfo[2], > PcdInfo[3], PcdValue[2].strip())) > - return StructurePcdSet > + # return structure pcd final value > + return self.GetStructurePcdSet(OutputValueFile) >=20 > @staticmethod > def NeedUpdateOutput(OutputFile, ValueCFile, StructureInput): > -- > 2.39.1.windows.1 >=20 >=20 >=20 >=20 >=20 -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#114124): https://edk2.groups.io/g/devel/message/114124 Mute This Topic: https://groups.io/mt/103781798/7686176 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-