From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail02.groups.io (mail02.groups.io [66.175.222.108]) by spool.mail.gandi.net (Postfix) with ESMTPS id 4A88778003C for ; Fri, 9 Feb 2024 10:08:43 +0000 (UTC) DKIM-Signature: a=rsa-sha256; bh=PrxB0gOIGbbhCeS2BkJHGAYLdVjDL70QMH+dmigSf7U=; c=relaxed/simple; d=groups.io; h=ARC-Seal:ARC-Message-Signature:ARC-Authentication-Results:From:To:CC:Subject:Thread-Topic:Thread-Index:Date:Message-ID:References:In-Reply-To:Accept-Language:MIME-Version:Precedence:List-Subscribe:List-Help:Sender:List-Id:Mailing-List:Delivered-To:Reply-To:List-Unsubscribe-Post:List-Unsubscribe:Content-Language:Content-Type:Content-Transfer-Encoding; s=20140610; t=1707473322; v=1; b=Lb7G9hwBPOQE6lwfQ4YZo4wUBSa0kn2Mf8g7zZlQM9cwx3gwFuufdSgzBQiDj7vo+S1ceEv1 DG24dQ943FT+Z8pa2C8NT70qmCigmglDFwdSNQmQUn1lv/lrEoQekPnEe/JXtQMrQhr9PzzJ5cv hCvXYdy9Cl2xKp0QjRWrSuGc= X-Received: by 127.0.0.2 with SMTP id AH26YY7687511xg2RC13pIPz; Fri, 09 Feb 2024 02:08:42 -0800 X-Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.20]) by mx.groups.io with SMTP id smtpd.web10.6834.1707352376773559811 for ; Wed, 07 Feb 2024 16:32:57 -0800 X-IronPort-AV: E=McAfee;i="6600,9927,10977"; a="1006315" X-IronPort-AV: E=Sophos;i="6.05,252,1701158400"; d="scan'208";a="1006315" X-Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orvoesa112.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Feb 2024 16:32:56 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10977"; a="824675733" X-IronPort-AV: E=Sophos;i="6.05,252,1701158400"; d="scan'208";a="824675733" X-Received: from orsmsx603.amr.corp.intel.com ([10.22.229.16]) by orsmga001.jf.intel.com with ESMTP/TLS/AES256-GCM-SHA384; 07 Feb 2024 16:32:55 -0800 X-Received: from orsmsx611.amr.corp.intel.com (10.22.229.24) by ORSMSX603.amr.corp.intel.com (10.22.229.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 7 Feb 2024 16:32:55 -0800 X-Received: from orsmsx610.amr.corp.intel.com (10.22.229.23) by ORSMSX611.amr.corp.intel.com (10.22.229.24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 7 Feb 2024 16:32:55 -0800 X-Received: from ORSEDG601.ED.cps.intel.com (10.7.248.6) by orsmsx610.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35 via Frontend Transport; Wed, 7 Feb 2024 16:32:55 -0800 X-Received: from NAM12-DM6-obe.outbound.protection.outlook.com (104.47.59.168) by edgegateway.intel.com (134.134.137.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.35; Wed, 7 Feb 2024 16:32:54 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ZcNk6Vq8/8IiAJfmyCMt+W6dG4T0Y0t/Y/MPODSMSY8LyH4TDZtcNhOhEZQSOIjMyrjljSdDkyNGJaSogfVM34KVKy7y6cpQXgnfAkEprefrBvIPmL7i3vSf1vSLcIU/AZ/qfvLsnZbFmm1TkySPAdOQGX+BAMg2h0Bj+4Du4oBDsX75hDsssYK3VHvUPGSx++46HeddXVmJly/8IQuNBS7AfB8SgX2xAfm9h0gvUuwVIjcFvvpKGPygnoca/tZg9h98PIplsge2LmnXfD66ADPYIvdWFtl8oxhFNbunKileliMhjwDbru0n5oxKWqtddOAham/rB5XN+9W9OMVGnQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=b+M98FdtPy8iinWAcNxtAG8RfQNdkDZe4jV4hQ0cH8c=; b=fvoaEDHYgqvfjDKeQOgpDQbESVtz8vvjFe8p3KcEFM0x2IQibJw1/OnK0zXFStolu0I3Xlb8nYj3dQ9Uh3Ysnv/8lemP3kc4yw/2ZD3JEBiDEAtb+A3gTGjhQEH+iq8FEFpvi2gWnA1wdr9ADmL85dsJ6f1Et2minjQorlqcHa9+oQzo2ZcKfFyfKE/lfrdrpwczCVgiyG8IOu9X7xvZ9kVv7qnK468I/2JALtoeV+vMZH0ioaZ80OhoOibaITd5yVT3c2DPthrSgcCoITg2LS2A9JsLu0LxzPKDvMkosLEtbVXvrynaWZ0Q8jwPvfzJOwynvYieAF8cw2hJs8PM8A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none X-Received: from PH7PR11MB5863.namprd11.prod.outlook.com (2603:10b6:510:135::11) by CO1PR11MB4787.namprd11.prod.outlook.com (2603:10b6:303:95::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.39; Thu, 8 Feb 2024 00:32:52 +0000 X-Received: from PH7PR11MB5863.namprd11.prod.outlook.com ([fe80::e4d6:ab6:b4c7:8830]) by PH7PR11MB5863.namprd11.prod.outlook.com ([fe80::e4d6:ab6:b4c7:8830%7]) with mapi id 15.20.7249.035; Thu, 8 Feb 2024 00:32:52 +0000 From: "Feng, Bob C" To: "S, Ashraf Ali" , "devel@edk2.groups.io" CC: "Chen, Christine" , Rebecca Cran , Liming Gao , "Chan, Amy" , "Chaganty, Rangasai V" , "Solanki, Digant H" Subject: Re: [edk2-devel] [PATCH V4] BaseTools: Optimize GenerateByteArrayValue and CollectPlatformGuids APIs Thread-Topic: [PATCH V4] BaseTools: Optimize GenerateByteArrayValue and CollectPlatformGuids APIs Thread-Index: AQHaWeoJbd1XFSsmVU203y05LeRZy7D/mMog Date: Thu, 8 Feb 2024 00:32:52 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-traffictypediagnostic: PH7PR11MB5863:EE_|CO1PR11MB4787:EE_ x-ms-office365-filtering-correlation-id: 35793255-d87f-429e-611b-08dc283d7c2a x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam-message-info: 7A44nzd9VFtagq2KuS9jM8uCNXA7BDbq9nDp983rINl6P00Hr/x8kRoSDTXjRtGu3MlOh3YeZizRGCC6ccLfQbIWz2opDwlpXDOLAtp4O496JFY7BCoK04CdwuUY5zfGe3UaMwF81Ig1gBw2g5yPl+CmC2xnCHbN60UcOKfnpwKZ1Qaxww+hitZOGkGaIZXLBfonLe8Gy5zwUa+Dtk8GqHQ+6jLEGcMWz4i7H8Cfk3CvJl2TGmakGsYCoWDJnoG7VNgsmALVpehvhUe3rV33QryJlcd4T8H5NO00HNSKELWG/Ed56olfLUCEq6BvB8RpX4W7fNTRxCuueRTBSMu7QKnnTq3bcaDq9qB3zDsUkVzzUli9qcn+pcE2eu2mcqLwQTBR1kzpOW6q8WWzhglgR1CEKnWdoXW89RFD/nS9zTwcVAewZv7j63ZNtetXO5GG6HhJojRaJ/I6Z1NPXH+jlEVmZ9ebc/6YRqbVaI+689LFveqzjxX6RFOL6wXLce3+QBgbnpNzRnRaDPrPtj8y57blUEuQZUbKUJUF+AuikkzEyiOw6/KdHOjrW3dRrxPngZJboqadgG+xxpQIchz0jIfc+ls7GyEbrrBhTXB01NY= x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?j9YUHDMWzlMIqNGENsdE+mn5Cjz1MfC1Bj1JwapWyWnf50/hnpzeC9/HkDoc?= =?us-ascii?Q?6VuPm5/g/fNMJnq/nujkyushQ7kVHmDLS3n0FAH4/qa/G7Ei9vmr6lKp4Gsn?= =?us-ascii?Q?h4YEJgEH9IF/pJ3122t6096VPmh9BMkQLpAwH7Rcyrmkxg4/8PEoBNm+P/b2?= =?us-ascii?Q?650IVoGHaCzrXkeZE6aF6sbxpB38uxalvhx5KXCD52I+RdRRXPTmN8SMvTLv?= =?us-ascii?Q?6Yo0B4kJNcVgULhfkvDcD1d5sUKPF3SyZdb9qObn5QI9W2b36WDcUWN2XvBa?= =?us-ascii?Q?PcoXrrSp316131sTGPw1LZEje/uVZf9+gzJFBZ0yx7MdWKoLzFzfuhnoLXuo?= =?us-ascii?Q?byxWAdNZLvURuEqJKDu3yTPKYmkBXxMBdv9AS3n4wmNa/mZKLzvYM1h6t1iL?= =?us-ascii?Q?S8x0GtHc6fbWAvZd7xLWazoouh0CTu4KLgmcb60JssIQoW84rGaw8uldQ6dr?= =?us-ascii?Q?NjXbQxGv0uBUjfRoVuHWct2Iz6zn8hKYh5lTmrE2wBI6Oxw3cUM66fj55i55?= =?us-ascii?Q?Hl9hmPkLRbIiPd9ecnqWEIkc5oQfOYbYO7ylkCCx6WN7Y6jd+ngVLIA7J4Js?= =?us-ascii?Q?+nm1/EiF7RWDgCryXQ92JRGX+efDirzKEoqsXyBxND8C5K8epHU0BHMVNdgN?= =?us-ascii?Q?9C27CnrwzKv1LcEC3uyYZ5pOXnjS0A3Ngd6QN6CWxQDxEqkOIJHF1I5Mda5v?= =?us-ascii?Q?YNKDYDbXNlLH4evTDOkUV4zzt5CbUsNtM5Qjo7iiUc/zPgloWSNhAdmjg2P4?= =?us-ascii?Q?81t9hHu+5EZHBiOwFtGVCjY9G5XILWWcmEqUW86h51/B4/pc6A+AwgeRis16?= =?us-ascii?Q?XqwRnRAMtobYCHreZDW8WdHd0Y5vW8f1pVjidprbpoNyC8XRqDt39Kw2hnww?= =?us-ascii?Q?FcnhtdLlWK3DyHSqtjuFLT8QlQlRHR0cyIGzzYk31zg7HyTxi8omHSfV+Xla?= =?us-ascii?Q?7rj+sQrpAVvHecS5eQw+ke7H08GK6i2IPuguFhLuCqUiz0x5W8Yt2SisNF9I?= =?us-ascii?Q?u5YJMfXySqyTl79u50GqEOvp3beQVYYsrtwWhDFI+L2yQUcm9Seet8p5HNxN?= =?us-ascii?Q?5f+COaHqdD2B4JW9mfU/yhGQdxyaXD1eXNGY2y4k6U7RfL66TS5CGuxywWdt?= =?us-ascii?Q?wHa7Zo+X/q0AFm7TNcJfpWCAA0nuv4bxK36CKKT4YJbpJpaznETGaCwajfAF?= =?us-ascii?Q?rsddI9K3SqaufxpzaimjvwhZ23CXFkF8w7kE8uLPtmk5S9INOdTI7QzrVShs?= =?us-ascii?Q?BhSw53dXQycnWuiqv247Lk2nG9MuGgtCRR4BFIZ/3r3nfXR8RO8GTVrCdUf2?= =?us-ascii?Q?3vJSUuh1WizvW7L5dijbFuIadm8bG7PsWGgSXxG7eUjACKouC0tdaEHyj/jG?= =?us-ascii?Q?g8CwkxAgn2N+YEOXruLF/KV6riIN76qX/ODx3byZARorr2A/36K2KddnuorU?= =?us-ascii?Q?bRgGyQIbkZCp4akhsh/8UjA8sXL8pktlB32qh4nG0t0Le3+o0jIH0fwuPXdU?= =?us-ascii?Q?0/ZwTkE3XtLpA7rhH3N+5v5AGlFL4u43xc23wR2SwDQme6a+BR9aEBeTsDVH?= =?us-ascii?Q?thaARjomxaCTrJoMPQkeVvsvVVbrwO1FF/xWYoLV?= MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: PH7PR11MB5863.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 35793255-d87f-429e-611b-08dc283d7c2a X-MS-Exchange-CrossTenant-originalarrivaltime: 08 Feb 2024 00:32:52.0586 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: uys/wl8QXIZL86aFr0KjQgOEajx2UPpgX520bXv45WRF3qQ2qDOhShhsnnt06qiGmQyvV30eU44gtWpRmtkv3g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR11MB4787 X-OriginatorOrg: intel.com Precedence: Bulk List-Subscribe: List-Help: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,bob.c.feng@intel.com List-Unsubscribe-Post: List-Unsubscribe=One-Click List-Unsubscribe: X-Gm-Message-State: 4ny8xOPqfeq7WQIfy9s9g5tXx7686176AA= Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-GND-Status: LEGIT Authentication-Results: spool.mail.gandi.net; dkim=pass header.d=groups.io header.s=20140610 header.b=Lb7G9hwB; arc=reject ("signature check failed: fail, {[1] = sig:microsoft.com:reject}"); dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=intel.com (policy=none); spf=pass (spool.mail.gandi.net: domain of bounce@groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce@groups.io Reviewed-by: Bob Feng -----Original Message----- From: S, Ashraf Ali =20 Sent: Thursday, February 8, 2024 1:21 AM To: devel@edk2.groups.io Cc: S, Ashraf Ali ; Chen, Christine ; Rebecca Cran ; Liming Gao ; Feng, Bob C ; Chan, Amy ; = Chaganty, Rangasai V ; Solanki, Digant H Subject: [PATCH V4] BaseTools: Optimize GenerateByteArrayValue and CollectP= latformGuids APIs During the Incremental build GenerateByteArrayValue used to generate the By= teArrayValue even when there is no change in the PCD/VPDs. which is time co= nsuming API based on the number of PCD/VPDs and SKU IDs. The optimization is that GenerateByteArrayValue is used to store the Struct= uredPcdsData in a JSON file for each of the arch. and during the Incrementa= l build this API will check, if there is any change in the Structured PCD/V= PDs then rest of the flow remains the same. if there is no change then it will return the provious build data. Flow: during the 1st build StructuredPcdsData.json is not exists, StructuredPcdsD= ata will be dumped to json file. and it will copy the output.txt as well. Note: as the output.txt are different for different Arch, so it will be sto= red in the Arch folder. During the Incremental build check if there is any change in Structured PCD= /VPD. if there is a change in Structured VPD/PCD then recreate the Structur= edPcdsData.json, and rest of the flow remains same. if there is no change in VPD/PCD read the output.txt and return the data Unit Test: Test1: Modified the Structured Pcds default from DEC file. current flow is = executing. Test2: Override the default value of the PCD from DEC file. current flow is= executing. Test3: Modified/Override the PCD from DSC file. current flow executing Test4: Modified/Override the FDF from DSC file. current flow executing Test5: update the default value from Command Line.current flow executing Test6: Build without change in PCD in DSC, FDF, DEC and Command Line the pr= oposed changes will be executing, and the return data remains the same with= and without the changes. Test7: Build with and without modified the include headers of Structured PC= Ds. if there is any change in those Structured PCD header then current flow= will be executed. With these changes it's helping to save around ~2.5min to ~3.5min of Increm= ental build time in my build environment. Sample PR: https://github.com/tianocore/edk2-basetools/pull/113 Cc: Yuwei Chen Cc: Rebecca Cran Cc: Liming Gao Cc: Bob Feng Cc: Amy Chan Cc: Sai Chaganty Cc: Digant H Solanki Signed-off-by: Ashraf Ali S --- .../Source/Python/AutoGen/WorkspaceAutoGen.py | 16 +- .../Source/Python/Workspace/DscBuildData.py | 181 +++++++++++++----- 2 files changed, 139 insertions(+), 58 deletions(-) diff --git a/BaseTools/Source/Python/AutoGen/WorkspaceAutoGen.py b/BaseTool= s/Source/Python/AutoGen/WorkspaceAutoGen.py index 160e3a3cd3..eec9280c8e 100644 --- a/BaseTools/Source/Python/AutoGen/WorkspaceAutoGen.py +++ b/BaseTools/Source/Python/AutoGen/WorkspaceAutoGen.py @@ -160,22 +160,18 @@ class WorkspaceAutoGen(AutoGen): =20 def CollectPlatformGuids(self): oriInfList =3D [] - oriPkgSet =3D set() - PlatformPkg =3D set() + pkgSet =3D set() for Arch in self.ArchList: Platform =3D self.BuildDatabase[self.MetaFile, Arch, self.Buil= dTarget, self.ToolChain] oriInfList =3D Platform.Modules for ModuleFile in oriInfList: ModuleData =3D self.BuildDatabase[ModuleFile, Platform._Ar= ch, Platform._Target, Platform._Toolchain] - oriPkgSet.update(ModuleData.Packages) - for Pkg in oriPkgSet: - Guids =3D Pkg.Guids - GlobalData.gGuidDict.update(Guids) + pkgSet.update(ModuleData.Packages) if Platform.Packages: - PlatformPkg.update(Platform.Packages) - for Pkg in PlatformPkg: - Guids =3D Pkg.Guids - GlobalData.gGuidDict.update(Guids) + pkgSet.update(Platform.Packages) + for Pkg in pkgSet: + Guids =3D Pkg.Guids + GlobalData.gGuidDict.update(Guids) =20 @cached_property def FdfProfile(self): diff --git a/BaseTools/Source/Python/Workspace/DscBuildData.py b/BaseTools/= Source/Python/Workspace/DscBuildData.py index 4768099343..ddcd157c73 100644 --- a/BaseTools/Source/Python/Workspace/DscBuildData.py +++ b/BaseTools/Source/Python/Workspace/DscBuildData.py @@ -37,6 +37,8 @@ from functools import reduce from Common.Misc import Sav= eFileOnChange from Workspace.BuildClassObject import PlatformBuildClassObj= ect, StructurePcd, PcdClassObject, ModuleBuildClassObject from collections= import OrderedDict, defaultdict +import json +import shutil =20 def _IsFieldValueAnArray (Value): Value =3D Value.strip() @@ -56,6 +58,7 @@ def _IsFieldValueAnArray (Value): =20 PcdValueInitName =3D 'PcdValueInit' PcdValueCommonName =3D 'PcdValueCommon' +StructuredPcdsDataName =3D 'StructuredPcdsData.json' =20 PcdMainCHeader =3D ''' /** @@ -2750,6 +2753,63 @@ class DscBuildData(PlatformBuildClassObject): ccflags.add(item) i +=3D1 return ccflags + + def GetStructurePcdSet (self, OutputValueFile): + if not os.path.isfile(OutputValueFile): + EdkLogger.error("GetStructurePcdSet", FILE_NOT_FOUND, "Output.= txt doesn't exist", ExtraData=3DOutputValueFile) + return [] + File =3D open (OutputValueFile, 'r') + FileBuffer =3D File.readlines() + File.close() + + #start update structure pcd final value + StructurePcdSet =3D [] + for Pcd in FileBuffer: + PcdValue =3D Pcd.split ('|') + PcdInfo =3D PcdValue[0].split ('.') + StructurePcdSet.append((PcdInfo[0], PcdInfo[1], PcdInfo[2], Pc= dInfo[3], PcdValue[2].strip())) + return StructurePcdSet + + def GetBuildOptionsValueList(self): + CC_FLAGS =3D LinuxCFLAGS + if sys.platform =3D=3D "win32": + CC_FLAGS =3D WindowsCFLAGS + BuildOptions =3D OrderedDict() + for Options in self.BuildOptions: + if Options[2] !=3D EDKII_NAME: + continue + Family =3D Options[0] + if Family and Family !=3D self.ToolChainFamily: + continue + Target, Tag, Arch, Tool, Attr =3D Options[1].split("_") + if Tool !=3D 'CC': + continue + if Attr !=3D "FLAGS": + continue + if Target =3D=3D TAB_STAR or Target =3D=3D self._Target: + if Tag =3D=3D TAB_STAR or Tag =3D=3D self._Toolchain: + if 'COMMON' not in BuildOptions: + BuildOptions['COMMON'] =3D set() + if Arch =3D=3D TAB_STAR: + BuildOptions['COMMON']|=3D self.ParseCCFlags(self.= BuildOptions[Options]) + if Arch in self.SupArchList: + if Arch not in BuildOptions: + BuildOptions[Arch] =3D set() + BuildOptions[Arch] |=3D=20 + self.ParseCCFlags(self.BuildOptions[Options]) + + if BuildOptions: + ArchBuildOptions =3D {arch:flags for arch,flags in BuildOption= s.items() if arch !=3D 'COMMON'} + if len(ArchBuildOptions.keys()) =3D=3D 1: + BuildOptions['COMMON'] |=3D (list(ArchBuildOptions.values(= ))[0]) + elif len(ArchBuildOptions.keys()) > 1: + CommonBuildOptions =3D reduce(lambda x,y: x&y, ArchBuildOp= tions.values()) + BuildOptions['COMMON'] |=3D CommonBuildOptions + ValueList =3D [item for item in BuildOptions['COMMON'] if item= .startswith((r"/U","-U"))] + ValueList.extend([item for item in BuildOptions['COMMON'] if i= tem.startswith((r"/D", "-D"))]) + CC_FLAGS +=3D " ".join(ValueList) + return CC_FLAGS + + def GenerateByteArrayValue (self, StructuredPcds): # # Generate/Compile/Run C application to determine if there are any= flexible array members @@ -2757,6 +2817,66 @@ class DscBuildData(PlatformB= uildClassObject): if not StructuredPcds: return =20 + StructuredPcdsData =3D {} + StoredStructuredPcdObjectPaths =3D {} + SkipPcdValueInit =3D False + + CC_FLAGS =3D self.GetBuildOptionsValueList() + + for PcdName in StructuredPcds: + Pcd =3D StructuredPcds[PcdName] + TokenSpaceGuidCName =3D Pcd.TokenSpaceGuidCName + TokenCName =3D Pcd.TokenCName + + # Create a key using TokenSpaceGuidCName and TokenCName + StructuredPcdsData[f"{TokenSpaceGuidCName}_{TokenCName}"] =3D = { + "DefaultValueFromDec": Pcd.DefaultValueFromDec, + "DefaultValues": Pcd.DefaultValues, + "PcdFieldValueFromComm": Pcd.PcdFieldValueFromComm, + "PcdFieldValueFromFdf": Pcd.PcdFieldValueFromFdf, + "DefaultFromDSC": Pcd.DefaultFromDSC, + "PcdFiledValueFromDscComponent": Pcd.PcdFiledValueFromDscC= omponent + } + + # Store the CC Flags + StructuredPcdsData["CC_FLAGS"] =3D CC_FLAGS + # + # If the output path doesn't exists then create it + # + if not os.path.exists(self.OutputPath): + os.makedirs(self.OutputPath) + + StructuredPcdsDataPath =3D os.path.join(self.OutputPath, self._Arc= h, StructuredPcdsDataName) + PcdRecordOutputValueFile =3D os.path.join(self.OutputPath,=20 + self._Arch, 'Output.txt') + + if not os.path.exists(os.path.dirname(StructuredPcdsDataPath)): + os.makedirs(os.path.dirname(StructuredPcdsDataPath)) + # + # Check if the StructuredPcdsData.json exists or not + # if exits then it might be a incremental build then check if the = StructuredPcdsData has been changed or not. + # if changed then proceed further, if not changed then return the = stored data from earlier build + # + if os.path.isfile(StructuredPcdsDataPath): + with open(StructuredPcdsDataPath, 'r') as file: + StoredStructuredPcdsData =3D json.load(file) + # OBJECTS will have the modified time, which needs to be c= hecked later + StoredStructuredPcdObjectPaths =3D=20 + StoredStructuredPcdsData.pop("OBJECTS", {}) + + if StructuredPcdsData =3D=3D StoredStructuredPcdsData: + SkipPcdValueInit =3D True + for filename, file_mtime in StoredStructuredPcdObjectP= aths.items(): + f_mtime =3D os.path.getmtime(filename) + # + # check if the include_file are modified or not, + # if modified then generate the PcdValueInit + # + if f_mtime !=3D file_mtime: + SkipPcdValueInit =3D False + break + + if SkipPcdValueInit: + return self.GetStructurePcdSet(PcdRecordOutputValueFile) + InitByteValue =3D "" CApp =3D PcdMainCHeader =20 @@ -2832,8 +2952,6 @@ class DscBuildData(PlatformBuildClassObject): =20 CApp =3D CApp + PcdMainCEntry + '\n' =20 - if not os.path.exists(self.OutputPath): - os.makedirs(self.OutputPath) CAppBaseFileName =3D os.path.join(self.OutputPath, PcdValueInitNam= e) SaveFileOnChange(CAppBaseFileName + '.c', CApp, False) =20 @@ -2890,42 +3008,6 @@ class DscBuildData(PlatformBuildClassObject): IncSearchList.append(inc) MakeApp =3D MakeApp + '\n' =20 - CC_FLAGS =3D LinuxCFLAGS - if sys.platform =3D=3D "win32": - CC_FLAGS =3D WindowsCFLAGS - BuildOptions =3D OrderedDict() - for Options in self.BuildOptions: - if Options[2] !=3D EDKII_NAME: - continue - Family =3D Options[0] - if Family and Family !=3D self.ToolChainFamily: - continue - Target, Tag, Arch, Tool, Attr =3D Options[1].split("_") - if Tool !=3D 'CC': - continue - if Attr !=3D "FLAGS": - continue - if Target =3D=3D TAB_STAR or Target =3D=3D self._Target: - if Tag =3D=3D TAB_STAR or Tag =3D=3D self._Toolchain: - if 'COMMON' not in BuildOptions: - BuildOptions['COMMON'] =3D set() - if Arch =3D=3D TAB_STAR: - BuildOptions['COMMON']|=3D self.ParseCCFlags(self.= BuildOptions[Options]) - if Arch in self.SupArchList: - if Arch not in BuildOptions: - BuildOptions[Arch] =3D set() - BuildOptions[Arch] |=3D self.ParseCCFlags(self.Bui= ldOptions[Options]) - - if BuildOptions: - ArchBuildOptions =3D {arch:flags for arch,flags in BuildOption= s.items() if arch !=3D 'COMMON'} - if len(ArchBuildOptions.keys()) =3D=3D 1: - BuildOptions['COMMON'] |=3D (list(ArchBuildOptions.values(= ))[0]) - elif len(ArchBuildOptions.keys()) > 1: - CommonBuildOptions =3D reduce(lambda x,y: x&y, ArchBuildOp= tions.values()) - BuildOptions['COMMON'] |=3D CommonBuildOptions - ValueList =3D [item for item in BuildOptions['COMMON'] if item= .startswith((r"/U","-U"))] - ValueList.extend([item for item in BuildOptions['COMMON'] if i= tem.startswith((r"/D", "-D"))]) - CC_FLAGS +=3D " ".join(ValueList) MakeApp +=3D CC_FLAGS =20 if sys.platform =3D=3D "win32": @@ -2946,7 +3028,9 @@ class DscBuildData(PlatformBuildClassObject): SearchPathList.append(os.path.normpath(mws.join(GlobalData.gGlobal= Defines["EDK_TOOLS_PATH"], "BaseTools/Source/C/Common"))) SearchPathList.extend(str(item) for item in IncSearchList) IncFileList =3D GetDependencyList(IncludeFileFullPaths, SearchPath= List) + StructuredPcdsData["OBJECTS"] =3D {} for include_file in IncFileList: + StructuredPcdsData["OBJECTS"][include_file] =3D=20 + os.path.getmtime(include_file) MakeApp +=3D "$(OBJECTS) : %s\n" % include_file if sys.platform =3D=3D "win32": PcdValueCommonPath =3D os.path.normpath(mws.join(GlobalData.gG= lobalDefines["EDK_TOOLS_PATH"], "Source\C\Common\PcdValueCommon.c")) @@ -3042,17 +3126,18 @@ class DscBuildData(PlatformBuildClassObject): if returncode !=3D 0: EdkLogger.warn('Build', COMMAND_FAILURE, 'Can not collect = output from command: %s\n%s\n%s\n' % (Command, StdOut, StdErr)) =20 - #start update structure pcd final value - File =3D open (OutputValueFile, 'r') - FileBuffer =3D File.readlines() - File.close() + # + # In 1st build create the StructuredPcdsData.json + # update the record as PCD Input has been changed if its increment= al build + # + with open(StructuredPcdsDataPath, 'w') as file: + json.dump(StructuredPcdsData, file, indent=3D2) =20 - StructurePcdSet =3D [] - for Pcd in FileBuffer: - PcdValue =3D Pcd.split ('|') - PcdInfo =3D PcdValue[0].split ('.') - StructurePcdSet.append((PcdInfo[0], PcdInfo[1], PcdInfo[2], Pc= dInfo[3], PcdValue[2].strip())) - return StructurePcdSet + # Copy update output file for each Arch + shutil.copyfile(OutputValueFile, PcdRecordOutputValueFile) + + #start update structure pcd final value + return self.GetStructurePcdSet(OutputValueFile) =20 @staticmethod def NeedUpdateOutput(OutputFile, ValueCFile, StructureInput): -- 2.39.1.windows.1 -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#115295): https://edk2.groups.io/g/devel/message/115295 Mute This Topic: https://groups.io/mt/104222804/7686176 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-