From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail02.groups.io (mail02.groups.io [66.175.222.108]) by spool.mail.gandi.net (Postfix) with ESMTPS id 0041E740032 for ; Thu, 8 Feb 2024 03:19:43 +0000 (UTC) DKIM-Signature: a=rsa-sha256; bh=IrbxA6Mj17M1HUF7+85qq5NuGDXkryFJ7Vq+WDHwVnU=; c=relaxed/simple; d=groups.io; h=ARC-Seal:ARC-Message-Signature:ARC-Authentication-Results:From:To:CC:Subject:Thread-Topic:Thread-Index:Date:Message-ID:References:In-Reply-To:Accept-Language:MIME-Version:Precedence:List-Subscribe:List-Help:Sender:List-Id:Mailing-List:Delivered-To:Reply-To:List-Unsubscribe-Post:List-Unsubscribe:Content-Language:Content-Type:Content-Transfer-Encoding; s=20140610; t=1707362382; v=1; b=OOKuRJJLTCAPRZjVPF1Pr2tsaWeIvn5EV2bomW2BjLlBpeTfhcy2MJy1DBn3KXXOh+UazHAx f3WHkQXJhkLp2VxcT5wH5PElscsfhBsjOuUgef/J/2UuI3QU8zLhtJyYyfncgNxtzlHZ5ZQZ/dY atvWxpUjwQJtCDKtvRxPO3UU= X-Received: by 127.0.0.2 with SMTP id TfoTYY7687511xKczDFTgH1S; Wed, 07 Feb 2024 19:19:42 -0800 X-Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.17]) by mx.groups.io with SMTP id smtpd.web11.10216.1707362381433721165 for ; Wed, 07 Feb 2024 19:19:41 -0800 X-IronPort-AV: E=McAfee;i="6600,9927,10977"; a="1274449" X-IronPort-AV: E=Sophos;i="6.05,252,1701158400"; d="scan'208";a="1274449" X-Received: from fmviesa010.fm.intel.com ([10.60.135.150]) by orvoesa109.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Feb 2024 19:19:41 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,252,1701158400"; d="scan'208";a="1528947" X-Received: from fmsmsx601.amr.corp.intel.com ([10.18.126.81]) by fmviesa010.fm.intel.com with ESMTP/TLS/AES256-GCM-SHA384; 07 Feb 2024 19:19:40 -0800 X-Received: from fmsmsx610.amr.corp.intel.com (10.18.126.90) by fmsmsx601.amr.corp.intel.com (10.18.126.81) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 7 Feb 2024 19:19:39 -0800 X-Received: from fmsmsx612.amr.corp.intel.com (10.18.126.92) by fmsmsx610.amr.corp.intel.com (10.18.126.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 7 Feb 2024 19:19:39 -0800 X-Received: from fmsedg601.ED.cps.intel.com (10.1.192.135) by fmsmsx612.amr.corp.intel.com (10.18.126.92) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35 via Frontend Transport; Wed, 7 Feb 2024 19:19:39 -0800 X-Received: from NAM10-BN7-obe.outbound.protection.outlook.com (104.47.70.101) by edgegateway.intel.com (192.55.55.70) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.35; Wed, 7 Feb 2024 19:19:38 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=OuUBQhnAe0hdWyGo8Ugw2Z1Yklm3FZ7VvnK/5Sp1NNu7jJCXNOWSNLhTVe7sR+6MD/uyhzk8NUeWAqGMixwK0LNhJV9k8en7LDJ7TtMHjP1Z+olPL5p5cSYPzWQc0zAfYlJv4fO7kZsZudxtG68vK5+/+zVTzrJRWKfj0IpN37DSTzbIzEE3ZhXRZ1O+wuMatpDFVVQ4XQf0wKL9+FUAnGwijtjFcG5ZvHpiG9ZFs0JcTfN6hWoom56fLB9jNTr5b4pb7e+BfVKbiMncUUcyaUI5Nq/+KvEBtjQmMkXoe3MpEvkE6TLLEqfUqP9yNlsLteagS/P4FxQq+UijxAlPMg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=BAsS2gY/iS7Lcoe0J3jLlxrj+LubrsMR14F+3K1XZfI=; b=COWrdgeSeORvuOkGm3LGoaLHTayPnvKUFqGYfTLZPHyUeLfRngwHumZ3TxRTxvX5mxXzpeIi+PWMSb01GhbZnmVRpzrKcnMcQAH/X0Ire+XO7IQ4owUfsGnx+wezchqyNbGu0Mzi0ia2+bvbcKC5EqvPdwPv3HMWHGIXXH3W5JPXTGsFdTQjM4/aPqkBoEpspAA2f1R5TiB/CsVXGonzttbqpHksN/PcwoTUFN1I7WV4bxjGfJf1OKGg8dSxazb5ZuFCVv0DSIJQ8XDxK//3+JXPHCpQGVCiacvill0OCOqSAsjM6g0pJVZIFhjBiYwe1rSUeqt68sTTMwAoeVUycg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none X-Received: from CO1PR11MB4929.namprd11.prod.outlook.com (2603:10b6:303:6d::19) by IA1PR11MB6098.namprd11.prod.outlook.com (2603:10b6:208:3d6::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.36; Thu, 8 Feb 2024 03:19:30 +0000 X-Received: from CO1PR11MB4929.namprd11.prod.outlook.com ([fe80::7e42:1633:d0a6:9c00]) by CO1PR11MB4929.namprd11.prod.outlook.com ([fe80::7e42:1633:d0a6:9c00%6]) with mapi id 15.20.7249.037; Thu, 8 Feb 2024 03:19:30 +0000 From: "Michael D Kinney" To: "devel@edk2.groups.io" , "Chen, Christine" , "S, Ashraf Ali" CC: Rebecca Cran , Liming Gao , "Feng, Bob C" , "Chan, Amy" , "Chaganty, Rangasai V" , "Solanki, Digant H" , "Kinney, Michael D" Subject: Re: [edk2-devel] [PATCH V4] BaseTools: Optimize GenerateByteArrayValue and CollectPlatformGuids APIs Thread-Topic: [edk2-devel] [PATCH V4] BaseTools: Optimize GenerateByteArrayValue and CollectPlatformGuids APIs Thread-Index: AQHaWepzNX5m48k67Eu78wdHQxssRrD/u2QAgAALaHA= Date: Thu, 8 Feb 2024 03:19:30 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-traffictypediagnostic: CO1PR11MB4929:EE_|IA1PR11MB6098:EE_ x-ms-office365-filtering-correlation-id: 01a3000b-e849-421d-949f-08dc2854c3a6 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam-message-info: do1hgwSU1fqQKEyrtqHoYeZmeSV/UOR4Ke7wFLaSqeu1/f83m8IpBDZLy0XS/EXsVtbvxvUfYSqB/kgI9W2JztnPxCtpwZQ9dGkpKEJ1aFLMv40iFtfWhvPGNwWhG6wKHutTijEPQ1kiNFm3DM5BG6MTgzPmGmIeihSJOzqjmkbgRH6+IiA13nIquCLohXkteNmgLtvJrRTJr8ndGqk1bfTx3iMMRYyvusl1WZsuE4w0Sad/z7WftuddDhzVmhLNCoN0VsX3pDNwGiLCw/gKbh/L0KPdNy+CQtEKmRnTbnBYar9GoNrYjI+12s8KRn/lp5IP/Nf5H4PZ5SdUHJk2CfPybPc6dg9gv+yLvV9YH6naYBH5PYGTlENSEByUI6ReCJ32tWI/TV20oa825SNVFxlAm5VyPpxgCP7NLqM0iZd/7DlH6wjbdr2rspU2vV28UFTmendIXe7ou5+RZmQIRZqAOEj0TpZGqxYdV7hgK7GJOVnuNMrIH+i1frjav2Aon/u0UwvAkgw2Rr2Yv3bvZI6OmDEdHVOdLaFApo4sAiZPF6kyyai/r4pwzDggJ3vw40qRN1vRZuIDkrIVXMw1tnv3bb98UuAawcTqDN09xGEAFPr7Zsgttw61oo3QThoLiQ+9OPH05UIFMkVo2czrdQ== x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?o/l8QQ5rD4ALhpS+6E7XZM1acOM5jxn3Uw6mqWqA/zcncLMYVQ854nuc3LZS?= =?us-ascii?Q?uaAixssBHuFvAujqe3DkYJf00pGDpkKkkmeKy+bHkTlVkJ3780Myg1XkvIP0?= =?us-ascii?Q?DOUDIzJPwGjdPvCQLRwLEyUSFVyVNRc6sYuJuaZGVYc6zDlvYRTifcI4rRar?= =?us-ascii?Q?Tf3FBmKEHqAoE7KKOdsmYJaGvqQq9J32Ie8+f9xs9w+8rKIf73K6N7vt5AVF?= =?us-ascii?Q?bC4mHefXRAr4p7ZxWhDuRlKWsmPKL72Dn2mBzNsycWf+JDrOa/AuWdbK6AdK?= =?us-ascii?Q?KOubn43+ORLW+CXWPOOxTWs9ahqXLRaqWhRKe1OAjq+MYboBsN3+BNCLK7iG?= =?us-ascii?Q?Ah9STA7ZzKa043GfwtwEmUuKM+UvYIlRNzORUNYgFWCoUd9akvbqxiTuR3tZ?= =?us-ascii?Q?hGd6NnLNaEb+OjJ/aMCG4R1Eo2xAoM1aDyTQlHZeGwfdShP09IBsGKm9KJ8P?= =?us-ascii?Q?LE5G7C/2raAXRZD5RCnXMuwjG6kWHSnGPGHVQx5qQ524L2V14qAfaM9qlj83?= =?us-ascii?Q?VPUmi0zx0jOy/2MlNfuwXw9H5HCsREICzzTc23NvAZlk9kkd3wUUoK8Xa8fa?= =?us-ascii?Q?ChzRT0OK0iiuqxGW++nwvvanuN8/P9hE8jI8vW4iJrMEoKp5z+PrtyrR+dem?= =?us-ascii?Q?wLlswDekLPRy/DyQ92XR0ZRCkq+zaUbc5gVU4p2FN5iSSrrMn35fI0r51XPi?= =?us-ascii?Q?Fj0HFoHfA1+ofvAPLOO8vHQTtFpU9E+4Mu90+1hOfhy1Yfu8plRUwZR8Mpny?= =?us-ascii?Q?wzXhs6G0Yza3QxSYnG2yOM9qb3+w3kj1n90j3GsIl2DBnwdNppBoJ26pkEur?= =?us-ascii?Q?h1NDsYrCzeZmkYbmb9G+bZv15WQ3y5bo49vpwlM/R/7P7r6zk1xprLKclHaj?= =?us-ascii?Q?f7+xQxmciomEPtMDtOir2Llkz3tBVI527991Jemo7RUKishKeprHyLEFimnn?= =?us-ascii?Q?hGsyPLTU/4ZKYlcg5x2SWmpETo/ae7gweKpDDsGedzf2M9y5XNgRYJtEDa8o?= =?us-ascii?Q?oEYS/CwhuU8S/bTBbnOuWxbNZLG1dDiShUSqSTn+ZnDBCsnJ7mq7SLPCecZL?= =?us-ascii?Q?ohgHB+GlZ75jSnUT9crJ/bwxQECYafrJS79E5h0zwjXsw8IPITCyaT9I+PRM?= =?us-ascii?Q?+7kqE8RIeI87wKk2D730tPpomZ8XBB5A5Y8r8FcYz3sQCO5S+UupxhkvDdtc?= =?us-ascii?Q?RuvIv8jnwtG04fMIafbQfmzp4oW2K4OhLDrJ/ISbqbuZIbj6v3DN6ZiU/nb2?= =?us-ascii?Q?PGAQdiYZiq5EHp+lvj31ZtnkPOLtdqBHZcB5UdvWa+mkIpu+R5HnXdBTrTHm?= =?us-ascii?Q?uaqaIm7nAfMYx440RUevRTgJwlPQabjokoS17SpWzhs1srPLNgAGP1cVPjza?= =?us-ascii?Q?xYFrp7+wogeVT6/9k1dhWVAvO5AIYmIfmBtRUtIE6iNJFwRsc4LK9RTaebUg?= =?us-ascii?Q?uqWjD3CX9RDtJ9N5qUYHEpsj8f0hKRQFg46crbcxQBnCwtn4FUZ8FpOhSjm7?= =?us-ascii?Q?EDj16kjUi57mpfVG876+rsjcOMAGxK4xjawrEeNwROV1aP575wcoSJM8hDOe?= =?us-ascii?Q?vxedYEL2ApgZFlvl8rAxqV0uR9ipQp+Ga3L+WWpC?= MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: CO1PR11MB4929.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 01a3000b-e849-421d-949f-08dc2854c3a6 X-MS-Exchange-CrossTenant-originalarrivaltime: 08 Feb 2024 03:19:30.4323 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: GyP0Jy7OF4Ili8DVfZ+TKtPGmh5iT0Q3IOnrXA42mK8u7MxybS5guMwZYtYDQ31dtlBKIvaPc3ss5TGuT/7vnc8f9EkADPBdMtOqtZTdIRA= X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR11MB6098 X-OriginatorOrg: intel.com Precedence: Bulk List-Subscribe: List-Help: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,michael.d.kinney@intel.com List-Unsubscribe-Post: List-Unsubscribe=One-Click List-Unsubscribe: X-Gm-Message-State: 8EX9cWKgsZYqzHZbmyVQCGuEx7686176AA= Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-GND-Status: LEGIT Authentication-Results: spool.mail.gandi.net; dkim=pass header.d=groups.io header.s=20140610 header.b=OOKuRJJL; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=intel.com (policy=none); spf=pass (spool.mail.gandi.net: domain of bounce@groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce@groups.io; arc=reject ("signature check failed: fail, {[1] = sig:microsoft.com:reject}") Hi Christine, I have created a PR with your Rb https://github.com/tianocore/edk2/pull/5356 This review started before the soft freeze, so if it passes EDK II CI,=20 I will merge it. I do see that there are some issues with this same change being applied to the edk2-basetools repo. Please work with Ashraf to get those resolved and keep the edk2-basetools and edk2/BaseTools repos aligned. Thanks, Mike > -----Original Message----- > From: devel@edk2.groups.io On Behalf Of Yuwei > Chen > Sent: Wednesday, February 7, 2024 6:36 PM > To: S, Ashraf Ali ; devel@edk2.groups.io > Cc: Rebecca Cran ; Liming Gao > ; Feng, Bob C ; Chan, > Amy ; Chaganty, Rangasai V > ; Solanki, Digant H > > Subject: Re: [edk2-devel] [PATCH V4] BaseTools: Optimize > GenerateByteArrayValue and CollectPlatformGuids APIs >=20 > Reviewed-by: Yuwei Chen >=20 > > -----Original Message----- > > From: S, Ashraf Ali > > Sent: Thursday, February 8, 2024 1:21 AM > > To: devel@edk2.groups.io > > Cc: S, Ashraf Ali ; Chen, Christine > > ; Rebecca Cran ; Liming Gao > > ; Feng, Bob C ; Chan, > > Amy ; Chaganty, Rangasai V > > ; Solanki, Digant H > > > > Subject: [PATCH V4] BaseTools: Optimize GenerateByteArrayValue and > > CollectPlatformGuids APIs > > > > During the Incremental build GenerateByteArrayValue used to generate > the > > ByteArrayValue even when there is no change in the PCD/VPDs. which is > time > > consuming API based on the number of PCD/VPDs and SKU IDs. > > > > The optimization is that GenerateByteArrayValue is used to store the > > StructuredPcdsData in a JSON file for each of the arch. and during > the > > Incremental build this API will check, if there is any change in the > Structured > > PCD/VPDs then rest of the flow remains the same. > > if there is no change then it will return the provious build data. > > > > Flow: > > during the 1st build StructuredPcdsData.json is not exists, > StructuredPcdsData > > will be dumped to json file. and it will copy the output.txt as well. > > Note: as the output.txt are different for different Arch, so it will > be stored in > > the Arch folder. > > During the Incremental build check if there is any change in > Structured > > PCD/VPD. if there is a change in Structured VPD/PCD then recreate the > > StructuredPcdsData.json, and rest of the flow remains same. > > if there is no change in VPD/PCD read the output.txt and return the > data > > > > Unit Test: > > Test1: Modified the Structured Pcds default from DEC file. current > flow is > > executing. > > Test2: Override the default value of the PCD from DEC file. current > flow is > > executing. > > Test3: Modified/Override the PCD from DSC file. current flow > executing > > Test4: Modified/Override the FDF from DSC file. current flow > executing > > Test5: update the default value from Command Line.current flow > executing > > Test6: Build without change in PCD in DSC, FDF, DEC and Command Line > the > > proposed changes will be executing, and the return data remains the > same > > with and without the changes. > > Test7: Build with and without modified the include headers of > Structured > > PCDs. if there is any change in those Structured PCD header then > current flow > > will be executed. > > > > With these changes it's helping to save around ~2.5min to ~3.5min of > > Incremental build time in my build environment. > > > > Sample PR: https://github.com/tianocore/edk2-basetools/pull/113 > > > > Cc: Yuwei Chen > > Cc: Rebecca Cran > > Cc: Liming Gao > > Cc: Bob Feng > > Cc: Amy Chan > > Cc: Sai Chaganty > > Cc: Digant H Solanki > > Signed-off-by: Ashraf Ali S > > --- > > .../Source/Python/AutoGen/WorkspaceAutoGen.py | 16 +- > > .../Source/Python/Workspace/DscBuildData.py | 181 +++++++++++++--- > -- > > 2 files changed, 139 insertions(+), 58 deletions(-) > > > > diff --git a/BaseTools/Source/Python/AutoGen/WorkspaceAutoGen.py > > b/BaseTools/Source/Python/AutoGen/WorkspaceAutoGen.py > > index 160e3a3cd3..eec9280c8e 100644 > > --- a/BaseTools/Source/Python/AutoGen/WorkspaceAutoGen.py > > +++ b/BaseTools/Source/Python/AutoGen/WorkspaceAutoGen.py > > @@ -160,22 +160,18 @@ class WorkspaceAutoGen(AutoGen): > > > > def CollectPlatformGuids(self): > > oriInfList =3D [] > > - oriPkgSet =3D set() > > - PlatformPkg =3D set() > > + pkgSet =3D set() > > for Arch in self.ArchList: > > Platform =3D self.BuildDatabase[self.MetaFile, Arch, > self.BuildTarget, > > self.ToolChain] > > oriInfList =3D Platform.Modules > > for ModuleFile in oriInfList: > > ModuleData =3D self.BuildDatabase[ModuleFile, > Platform._Arch, > > Platform._Target, Platform._Toolchain] > > - oriPkgSet.update(ModuleData.Packages) > > - for Pkg in oriPkgSet: > > - Guids =3D Pkg.Guids > > - GlobalData.gGuidDict.update(Guids) > > + pkgSet.update(ModuleData.Packages) > > if Platform.Packages: > > - PlatformPkg.update(Platform.Packages) > > - for Pkg in PlatformPkg: > > - Guids =3D Pkg.Guids > > - GlobalData.gGuidDict.update(Guids) > > + pkgSet.update(Platform.Packages) > > + for Pkg in pkgSet: > > + Guids =3D Pkg.Guids > > + GlobalData.gGuidDict.update(Guids) > > > > @cached_property > > def FdfProfile(self): > > diff --git a/BaseTools/Source/Python/Workspace/DscBuildData.py > > b/BaseTools/Source/Python/Workspace/DscBuildData.py > > index 4768099343..ddcd157c73 100644 > > --- a/BaseTools/Source/Python/Workspace/DscBuildData.py > > +++ b/BaseTools/Source/Python/Workspace/DscBuildData.py > > @@ -37,6 +37,8 @@ from functools import reduce from Common.Misc > > import SaveFileOnChange from Workspace.BuildClassObject import > > PlatformBuildClassObject, StructurePcd, PcdClassObject, > > ModuleBuildClassObject from collections import OrderedDict, > defaultdict > > +import json > > +import shutil > > > > def _IsFieldValueAnArray (Value): > > Value =3D Value.strip() > > @@ -56,6 +58,7 @@ def _IsFieldValueAnArray (Value): > > > > PcdValueInitName =3D 'PcdValueInit' > > PcdValueCommonName =3D 'PcdValueCommon' > > +StructuredPcdsDataName =3D 'StructuredPcdsData.json' > > > > PcdMainCHeader =3D ''' > > /** > > @@ -2750,6 +2753,63 @@ class DscBuildData(PlatformBuildClassObject): > > ccflags.add(item) > > i +=3D1 > > return ccflags > > + > > + def GetStructurePcdSet (self, OutputValueFile): > > + if not os.path.isfile(OutputValueFile): > > + EdkLogger.error("GetStructurePcdSet", FILE_NOT_FOUND, > "Output.txt > > doesn't exist", ExtraData=3DOutputValueFile) > > + return [] > > + File =3D open (OutputValueFile, 'r') > > + FileBuffer =3D File.readlines() > > + File.close() > > + > > + #start update structure pcd final value > > + StructurePcdSet =3D [] > > + for Pcd in FileBuffer: > > + PcdValue =3D Pcd.split ('|') > > + PcdInfo =3D PcdValue[0].split ('.') > > + StructurePcdSet.append((PcdInfo[0], PcdInfo[1], > PcdInfo[2], > > PcdInfo[3], PcdValue[2].strip())) > > + return StructurePcdSet > > + > > + def GetBuildOptionsValueList(self): > > + CC_FLAGS =3D LinuxCFLAGS > > + if sys.platform =3D=3D "win32": > > + CC_FLAGS =3D WindowsCFLAGS > > + BuildOptions =3D OrderedDict() > > + for Options in self.BuildOptions: > > + if Options[2] !=3D EDKII_NAME: > > + continue > > + Family =3D Options[0] > > + if Family and Family !=3D self.ToolChainFamily: > > + continue > > + Target, Tag, Arch, Tool, Attr =3D Options[1].split("_") > > + if Tool !=3D 'CC': > > + continue > > + if Attr !=3D "FLAGS": > > + continue > > + if Target =3D=3D TAB_STAR or Target =3D=3D self._Target: > > + if Tag =3D=3D TAB_STAR or Tag =3D=3D self._Toolchain: > > + if 'COMMON' not in BuildOptions: > > + BuildOptions['COMMON'] =3D set() > > + if Arch =3D=3D TAB_STAR: > > + BuildOptions['COMMON']|=3D > > self.ParseCCFlags(self.BuildOptions[Options]) > > + if Arch in self.SupArchList: > > + if Arch not in BuildOptions: > > + BuildOptions[Arch] =3D set() > > + BuildOptions[Arch] |=3D > > + self.ParseCCFlags(self.BuildOptions[Options]) > > + > > + if BuildOptions: > > + ArchBuildOptions =3D {arch:flags for arch,flags in > BuildOptions.items() if > > arch !=3D 'COMMON'} > > + if len(ArchBuildOptions.keys()) =3D=3D 1: > > + BuildOptions['COMMON'] |=3D > (list(ArchBuildOptions.values())[0]) > > + elif len(ArchBuildOptions.keys()) > 1: > > + CommonBuildOptions =3D reduce(lambda x,y: x&y, > > ArchBuildOptions.values()) > > + BuildOptions['COMMON'] |=3D CommonBuildOptions > > + ValueList =3D [item for item in BuildOptions['COMMON'] if > > item.startswith((r"/U","-U"))] > > + ValueList.extend([item for item in > BuildOptions['COMMON'] if > > item.startswith((r"/D", "-D"))]) > > + CC_FLAGS +=3D " ".join(ValueList) > > + return CC_FLAGS > > + > > + > > def GenerateByteArrayValue (self, StructuredPcds): > > # > > # Generate/Compile/Run C application to determine if there > are any > > flexible array members @@ -2757,6 +2817,66 @@ class > > DscBuildData(PlatformBuildClassObject): > > if not StructuredPcds: > > return > > > > + StructuredPcdsData =3D {} > > + StoredStructuredPcdObjectPaths =3D {} > > + SkipPcdValueInit =3D False > > + > > + CC_FLAGS =3D self.GetBuildOptionsValueList() > > + > > + for PcdName in StructuredPcds: > > + Pcd =3D StructuredPcds[PcdName] > > + TokenSpaceGuidCName =3D Pcd.TokenSpaceGuidCName > > + TokenCName =3D Pcd.TokenCName > > + > > + # Create a key using TokenSpaceGuidCName and TokenCName > > + > StructuredPcdsData[f"{TokenSpaceGuidCName}_{TokenCName}"] =3D { > > + "DefaultValueFromDec": Pcd.DefaultValueFromDec, > > + "DefaultValues": Pcd.DefaultValues, > > + "PcdFieldValueFromComm": Pcd.PcdFieldValueFromComm, > > + "PcdFieldValueFromFdf": Pcd.PcdFieldValueFromFdf, > > + "DefaultFromDSC": Pcd.DefaultFromDSC, > > + "PcdFiledValueFromDscComponent": > > Pcd.PcdFiledValueFromDscComponent > > + } > > + > > + # Store the CC Flags > > + StructuredPcdsData["CC_FLAGS"] =3D CC_FLAGS > > + # > > + # If the output path doesn't exists then create it > > + # > > + if not os.path.exists(self.OutputPath): > > + os.makedirs(self.OutputPath) > > + > > + StructuredPcdsDataPath =3D os.path.join(self.OutputPath, > self._Arch, > > StructuredPcdsDataName) > > + PcdRecordOutputValueFile =3D os.path.join(self.OutputPath, > > + self._Arch, 'Output.txt') > > + > > + if not > os.path.exists(os.path.dirname(StructuredPcdsDataPath)): > > + os.makedirs(os.path.dirname(StructuredPcdsDataPath)) > > + # > > + # Check if the StructuredPcdsData.json exists or not > > + # if exits then it might be a incremental build then check > if the > > StructuredPcdsData has been changed or not. > > + # if changed then proceed further, if not changed then > return the stored > > data from earlier build > > + # > > + if os.path.isfile(StructuredPcdsDataPath): > > + with open(StructuredPcdsDataPath, 'r') as file: > > + StoredStructuredPcdsData =3D json.load(file) > > + # OBJECTS will have the modified time, which needs > to be checked > > later > > + StoredStructuredPcdObjectPaths =3D > > + StoredStructuredPcdsData.pop("OBJECTS", {}) > > + > > + if StructuredPcdsData =3D=3D StoredStructuredPcdsData: > > + SkipPcdValueInit =3D True > > + for filename, file_mtime in > > StoredStructuredPcdObjectPaths.items(): > > + f_mtime =3D os.path.getmtime(filename) > > + # > > + # check if the include_file are modified or > not, > > + # if modified then generate the PcdValueInit > > + # > > + if f_mtime !=3D file_mtime: > > + SkipPcdValueInit =3D False > > + break > > + > > + if SkipPcdValueInit: > > + return self.GetStructurePcdSet(PcdRecordOutputValueFile) > > + > > InitByteValue =3D "" > > CApp =3D PcdMainCHeader > > > > @@ -2832,8 +2952,6 @@ class DscBuildData(PlatformBuildClassObject): > > > > CApp =3D CApp + PcdMainCEntry + '\n' > > > > - if not os.path.exists(self.OutputPath): > > - os.makedirs(self.OutputPath) > > CAppBaseFileName =3D os.path.join(self.OutputPath, > PcdValueInitName) > > SaveFileOnChange(CAppBaseFileName + '.c', CApp, False) > > > > @@ -2890,42 +3008,6 @@ class DscBuildData(PlatformBuildClassObject): > > IncSearchList.append(inc) > > MakeApp =3D MakeApp + '\n' > > > > - CC_FLAGS =3D LinuxCFLAGS > > - if sys.platform =3D=3D "win32": > > - CC_FLAGS =3D WindowsCFLAGS > > - BuildOptions =3D OrderedDict() > > - for Options in self.BuildOptions: > > - if Options[2] !=3D EDKII_NAME: > > - continue > > - Family =3D Options[0] > > - if Family and Family !=3D self.ToolChainFamily: > > - continue > > - Target, Tag, Arch, Tool, Attr =3D Options[1].split("_") > > - if Tool !=3D 'CC': > > - continue > > - if Attr !=3D "FLAGS": > > - continue > > - if Target =3D=3D TAB_STAR or Target =3D=3D self._Target: > > - if Tag =3D=3D TAB_STAR or Tag =3D=3D self._Toolchain: > > - if 'COMMON' not in BuildOptions: > > - BuildOptions['COMMON'] =3D set() > > - if Arch =3D=3D TAB_STAR: > > - BuildOptions['COMMON']|=3D > > self.ParseCCFlags(self.BuildOptions[Options]) > > - if Arch in self.SupArchList: > > - if Arch not in BuildOptions: > > - BuildOptions[Arch] =3D set() > > - BuildOptions[Arch] |=3D > > self.ParseCCFlags(self.BuildOptions[Options]) > > - > > - if BuildOptions: > > - ArchBuildOptions =3D {arch:flags for arch,flags in > BuildOptions.items() if > > arch !=3D 'COMMON'} > > - if len(ArchBuildOptions.keys()) =3D=3D 1: > > - BuildOptions['COMMON'] |=3D > (list(ArchBuildOptions.values())[0]) > > - elif len(ArchBuildOptions.keys()) > 1: > > - CommonBuildOptions =3D reduce(lambda x,y: x&y, > > ArchBuildOptions.values()) > > - BuildOptions['COMMON'] |=3D CommonBuildOptions > > - ValueList =3D [item for item in BuildOptions['COMMON'] if > > item.startswith((r"/U","-U"))] > > - ValueList.extend([item for item in > BuildOptions['COMMON'] if > > item.startswith((r"/D", "-D"))]) > > - CC_FLAGS +=3D " ".join(ValueList) > > MakeApp +=3D CC_FLAGS > > > > if sys.platform =3D=3D "win32": > > @@ -2946,7 +3028,9 @@ class DscBuildData(PlatformBuildClassObject): > > > > > SearchPathList.append(os.path.normpath(mws.join(GlobalData.gGlobalDefin > > es["EDK_TOOLS_PATH"], "BaseTools/Source/C/Common"))) > > SearchPathList.extend(str(item) for item in IncSearchList) > > IncFileList =3D GetDependencyList(IncludeFileFullPaths, > SearchPathList) > > + StructuredPcdsData["OBJECTS"] =3D {} > > for include_file in IncFileList: > > + StructuredPcdsData["OBJECTS"][include_file] =3D > > + os.path.getmtime(include_file) > > MakeApp +=3D "$(OBJECTS) : %s\n" % include_file > > if sys.platform =3D=3D "win32": > > PcdValueCommonPath =3D > > os.path.normpath(mws.join(GlobalData.gGlobalDefines["EDK_TOOLS_PATH"] > > , "Source\C\Common\PcdValueCommon.c")) > > @@ -3042,17 +3126,18 @@ class DscBuildData(PlatformBuildClassObject): > > if returncode !=3D 0: > > EdkLogger.warn('Build', COMMAND_FAILURE, 'Can not > collect > > output from command: %s\n%s\n%s\n' % (Command, StdOut, StdErr)) > > > > - #start update structure pcd final value > > - File =3D open (OutputValueFile, 'r') > > - FileBuffer =3D File.readlines() > > - File.close() > > + # > > + # In 1st build create the StructuredPcdsData.json > > + # update the record as PCD Input has been changed if its > incremental > > build > > + # > > + with open(StructuredPcdsDataPath, 'w') as file: > > + json.dump(StructuredPcdsData, file, indent=3D2) > > > > - StructurePcdSet =3D [] > > - for Pcd in FileBuffer: > > - PcdValue =3D Pcd.split ('|') > > - PcdInfo =3D PcdValue[0].split ('.') > > - StructurePcdSet.append((PcdInfo[0], PcdInfo[1], > PcdInfo[2], PcdInfo[3], > > PcdValue[2].strip())) > > - return StructurePcdSet > > + # Copy update output file for each Arch > > + shutil.copyfile(OutputValueFile, PcdRecordOutputValueFile) > > + > > + #start update structure pcd final value > > + return self.GetStructurePcdSet(OutputValueFile) > > > > @staticmethod > > def NeedUpdateOutput(OutputFile, ValueCFile, StructureInput): > > -- > > 2.39.1.windows.1 >=20 >=20 >=20 >=20 >=20 -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#115262): https://edk2.groups.io/g/devel/message/115262 Mute This Topic: https://groups.io/mt/104222804/7686176 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-