From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail02.groups.io (mail02.groups.io [66.175.222.108]) by spool.mail.gandi.net (Postfix) with ESMTPS id 47A72780091 for ; Thu, 8 Feb 2024 02:36:38 +0000 (UTC) DKIM-Signature: a=rsa-sha256; bh=WR8Dh/wegGLK9wjyM0ZLeWBsUsnp1PktaA2iyCplriU=; c=relaxed/simple; d=groups.io; h=ARC-Seal:ARC-Message-Signature:ARC-Authentication-Results:From:To:CC:Subject:Thread-Topic:Thread-Index:Date:Message-ID:References:In-Reply-To:Accept-Language:MIME-Version:Precedence:List-Subscribe:List-Help:Sender:List-Id:Mailing-List:Delivered-To:Reply-To:List-Unsubscribe-Post:List-Unsubscribe:Content-Language:Content-Type:Content-Transfer-Encoding; s=20140610; t=1707359796; v=1; b=XTIIL+P6A9aY8L30mkEUa4mpxKf6lJkqYT2Wi8YZbmTPlaAAa6xGRzdgP6Oli3SG+BKMkteZ 3MAZwM1649/6NhBIeQ1hbQxVMkK+FRqGlKSy6T5AFbNJzcRKYV5qn4uByLmxHqwb023uidOFsrr eHnRdkxW23AxATsbM8wXfZ1E= X-Received: by 127.0.0.2 with SMTP id r8dmYY7687511xML3ZuOCJoQ; Wed, 07 Feb 2024 18:36:36 -0800 X-Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.10]) by mx.groups.io with SMTP id smtpd.web10.9364.1707359795901499973 for ; Wed, 07 Feb 2024 18:36:35 -0800 X-IronPort-AV: E=McAfee;i="6600,9927,10977"; a="12486704" X-IronPort-AV: E=Sophos;i="6.05,252,1701158400"; d="scan'208";a="12486704" X-Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by fmvoesa104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Feb 2024 18:36:35 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,252,1701158400"; d="scan'208";a="1529684" X-Received: from fmsmsx602.amr.corp.intel.com ([10.18.126.82]) by fmviesa007.fm.intel.com with ESMTP/TLS/AES256-GCM-SHA384; 07 Feb 2024 18:36:35 -0800 X-Received: from fmsmsx612.amr.corp.intel.com (10.18.126.92) by fmsmsx602.amr.corp.intel.com (10.18.126.82) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 7 Feb 2024 18:36:34 -0800 X-Received: from fmsmsx610.amr.corp.intel.com (10.18.126.90) by fmsmsx612.amr.corp.intel.com (10.18.126.92) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 7 Feb 2024 18:36:34 -0800 X-Received: from FMSEDG603.ED.cps.intel.com (10.1.192.133) by fmsmsx610.amr.corp.intel.com (10.18.126.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35 via Frontend Transport; Wed, 7 Feb 2024 18:36:34 -0800 X-Received: from NAM04-MW2-obe.outbound.protection.outlook.com (104.47.73.168) by edgegateway.intel.com (192.55.55.68) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.35; Wed, 7 Feb 2024 18:36:29 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=eOFxZlhPS6vmQMSxnLWAGa+ikcyAqKasboA+suPlWjxZzDlVLza6QQsqR3qCER+WLcmbMmgnu0AWBfNYQBEQaD4dz0EOFjP0VDQWHioSzdRx7mNFax8shweQTqSQikaOriQ6p27BxWi6SW5PYQzEBcBL4upKRkaX5hDpTPWzQ+W8/OBpXc3b4A1osT37uefsmDHYTFgrNnspyeORtW7GqGEp/V5Twb9cfb7zaQHOADLlwRB3WWVuxCPvqnmcEJ7nfTORzagyEciH1kGBvQCVCyrILctcNB/jzx+WozVDYFF273Cv7Cq7M2MokIJs4bs+M6TcHmnBqQ8NfahWRrVpkA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=s6UExIBKmCdHIJQoTilhbi6NYfYTmtnmsIdL/nSSZyQ=; b=JAopRb81CJe516rZbMFTXHy1dUmijB93pp9s1gdScG5MvFt44t+N28OZkojjk5MbNzyVn0uYMe1bhfZhA5B5vd6iBQ04DM+fjs78Ihbmkycm6TbojSTQokC8oAoQKd9BzfkEd8o6Jfw14fZ2ScEvW46eZB8+U/cVxVfpmB28hRokn2l4990RLOkr93ZawduQJoiJZiwUgy5TIRHWDLvDmRViWCZD2mvxBRn3J1zbv8s9CWxAONtZu8/rRyugJsSgDAN8bRTpzEQP+DduIpkBXbu1RlM0fioylkdQ+WdrObSplX/ncx61NHV+wBwjH0CTglDAm0JuxSEC++z/VEBplA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none X-Received: from MW5PR11MB5906.namprd11.prod.outlook.com (2603:10b6:303:1a0::21) by CY8PR11MB7394.namprd11.prod.outlook.com (2603:10b6:930:85::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.38; Thu, 8 Feb 2024 02:36:27 +0000 X-Received: from MW5PR11MB5906.namprd11.prod.outlook.com ([fe80::295d:9865:fc32:12d1]) by MW5PR11MB5906.namprd11.prod.outlook.com ([fe80::295d:9865:fc32:12d1%7]) with mapi id 15.20.7249.038; Thu, 8 Feb 2024 02:36:27 +0000 From: "Yuwei Chen" To: "S, Ashraf Ali" , "devel@edk2.groups.io" CC: Rebecca Cran , Liming Gao , "Feng, Bob C" , "Chan, Amy" , "Chaganty, Rangasai V" , "Solanki, Digant H" Subject: Re: [edk2-devel] [PATCH V4] BaseTools: Optimize GenerateByteArrayValue and CollectPlatformGuids APIs Thread-Topic: [PATCH V4] BaseTools: Optimize GenerateByteArrayValue and CollectPlatformGuids APIs Thread-Index: AQHaWeoJAnDEmYjpW0S6SLwN61spXrD/u12g Date: Thu, 8 Feb 2024 02:36:26 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-traffictypediagnostic: MW5PR11MB5906:EE_|CY8PR11MB7394:EE_ x-ms-office365-filtering-correlation-id: da3006da-1ca1-4100-d9e6-08dc284ebfcb x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam-message-info: 3sMgWvuxqgXvnYln8JNkjsfq7rlsgygiPYQj4xqi5dcqcoOO3ZWE6m1ZEUnkn2apeL6sZsiVa8pK933hGxc9+sp3ttEskDzh+kmnZVh8k0dAEG62PFnNMmV3XxAhQLH4z/tPgr/u/0gi2YABhKRRD2lIbZY5ouQKmw0qmrq7o6pVaE2iCW+Rp6h2ffmkY1aYDTHueO+o0PxJ4PjaDXs5OR0VV8WFdp2mHSFDpC3nlKhW94RSVyoeAD3X5swaNHlEXxh5xaioGTNAN+jozgP9OLGOxchgvvSzkPv+7lfmDvUNU3fYBFcwVA203mYGFIW4ZKr07BQWEoH2tSDoZhWXaR+4OnO10yLteXFUb1pM7BHQIAicQObOLYWzKl2wxirFqu2Xy/UNDCLairf6VwhZnmjFQ/Lf+MNEpFqYXTtiIy4XFvXzjIRbrEHPCxK5ZexCQi5J3nxWy+bEU2uC0jtvhr7xPpky2z4KL03d6DzvrPgCNKjSRwfNRPXL4HtTX7sn0lmo0f8mc4IzL2uDwRslr8LjpQ9FJ1S+Eozm9rVzewApiDbPgrRnmq9gqL+xEUDvw3PMBSRwMEmtIxB57JW5jcGWBBANA5vCsfSWodVXjVY= x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?XTsiamMb6duGNEStgHflJGdyv6YK984Z5/PlO3Z55s9pFVeDSTWmH4eDirxp?= =?us-ascii?Q?2MHcvecGzbAgqudOCxaGsBOpyMbzObGozSH2lPlYuM8pjj92D17qByE9Cwzj?= =?us-ascii?Q?Jd9Al2Ug/C9K7o36RDC7F2sU7/d1ZAUTCsu/0yOCqWQpPbdGrTN65oJ0KdOx?= =?us-ascii?Q?G6k8BHqbDXa89s1OmBiwRpoqQ2mMEF1YoOc0u2Lz9B6BGrSWC92dpwpf6Ss/?= =?us-ascii?Q?ryL9LJIcepVQ2VQaUvje9Zy+/nv6DTx5LsExLtCZBOFIigYZEzPHHRDCrzKL?= =?us-ascii?Q?Ff4oDGewbyo5QmFHemMqaOvVflZlnb002mrbXCh0e3ueDqqWBkNh27ufDyGw?= =?us-ascii?Q?TQkT/E1MqN5Wadw2MsftZOMw+Aw1B75VacOk+K/1oxaXcy1LIs4PKQu78evS?= =?us-ascii?Q?d9+GZWcjEjsSxi216Tvmes6XqWF9LOahvRM3WeqtEp0a/FAWFaw3zJ2ZNnbK?= =?us-ascii?Q?xko5ch/paU97n6K6h4cjxiBThSTYHekOKnf5EtxjB87F7s1AG5q6UqFbfLhV?= =?us-ascii?Q?IvkzRobuJ+jv6Z2GKltkXKQjSB7/MgxsUl0IAjqLylFb8EtJjWyccD5RwMjA?= =?us-ascii?Q?/Zo8Fkrqch3BzvFKGWLhQELuyl0ljG/KRKmWRI1ip6fJUc36EyPTXnd/lf3H?= =?us-ascii?Q?4cGQKwm7KQ5ON80Xi4JaUwmKpQuNBfiFY+Xzyag19kHhtjeiJqGBGPKAdO1y?= =?us-ascii?Q?Cf7U9r+11qeG5EPu2N6ZheYusyOComheJF910fCT6xXjxuRzLFMSvpumSCJP?= =?us-ascii?Q?usRQaaUkPbO7tS3cp05/VQhLU9AscC+/CtMQPhzenb6Q775Mm5sEkrcci1wI?= =?us-ascii?Q?fT+drylbDKG7UToFD0P2/rrHXLVayVgmTt/Gn7v5LwXqQWDLLyAPfwdesyFh?= =?us-ascii?Q?jvQH2sSMA4FILTfpz38nHhPPrz2TVlo3OvZVzhSenTB+tlfPTAGYC0XPuOn1?= =?us-ascii?Q?DtfvbSNCog6QND11QD0CA1ptSc15BYCbV2POd1hV0Zy7STPrtC2xzf0qATZC?= =?us-ascii?Q?W4Lv0CGdoqRt5SdkQpiU8lmmdhMWOGCCWx6KnnuiGBFJB3qVqiDLtHdynUiv?= =?us-ascii?Q?KRn1CS/oXMFD/zB93cvmua+gkXRiR6TlBx4MLQaau2fi4nZeWv1wVi3F3gFA?= =?us-ascii?Q?+o9f0OV6kSmNJtg+z0JxYg6ktRAIx5vmFjn+vQI1BjXi8s/FP75mZCF3Zx71?= =?us-ascii?Q?FBZZtDEfOS9gndPimSPgeFams2YpXmPALozsGGw+waFcuSrjYG+baizf4usS?= =?us-ascii?Q?LSHgPSmV7x1UCF9iA0Ot2I757Em8hlw3L2XNOv6OiJW3yeDG/ftNq8ON5cAr?= =?us-ascii?Q?PKJr8g9i6vDz+MriJufuqmyXyYQi/g0ieFUw54xDlSdRVKs2paF5pvi0T11H?= =?us-ascii?Q?ZJkfdOUC55sx78qopf1h5msNjCJAB9kbJbG6fIXf2+CY0YJGNLfAVEwYOSdN?= =?us-ascii?Q?hfIb3Ty9tYSym9P8zWcQiGtTDlaCtgW4il1svaEc3JP8j9CZn+oKfCI4XjiV?= =?us-ascii?Q?7gTycQz/5TQmsAiYp4vdxbI0xADck2im5YtMh/Sf6T6A7e1tFMMtDTKrriYQ?= =?us-ascii?Q?ZBHln0zMCS0iZ/zJ5ILjmmqbATKbz1Tx2Y/CSW0h?= MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MW5PR11MB5906.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: da3006da-1ca1-4100-d9e6-08dc284ebfcb X-MS-Exchange-CrossTenant-originalarrivaltime: 08 Feb 2024 02:36:26.9960 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: rflCwazP/ZhRTRqO0wXte8BJk/7GiFRfqHuQsMANxxxBNe0yxXOmlEVl6hZt1Y7S6AbWKRChWkwk9KT4q/S3ow== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR11MB7394 X-OriginatorOrg: intel.com Precedence: Bulk List-Subscribe: List-Help: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,yuwei.chen@intel.com List-Unsubscribe-Post: List-Unsubscribe=One-Click List-Unsubscribe: X-Gm-Message-State: pQoBpStEfa25ciJqbCM7Fe27x7686176AA= Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-GND-Status: LEGIT Authentication-Results: spool.mail.gandi.net; dkim=pass header.d=groups.io header.s=20140610 header.b=XTIIL+P6; arc=reject ("signature check failed: fail, {[1] = sig:microsoft.com:reject}"); dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=intel.com (policy=none); spf=pass (spool.mail.gandi.net: domain of bounce@groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce@groups.io Reviewed-by: Yuwei Chen > -----Original Message----- > From: S, Ashraf Ali > Sent: Thursday, February 8, 2024 1:21 AM > To: devel@edk2.groups.io > Cc: S, Ashraf Ali ; Chen, Christine > ; Rebecca Cran ; Liming Gao > ; Feng, Bob C ; Chan, > Amy ; Chaganty, Rangasai V > ; Solanki, Digant H > > Subject: [PATCH V4] BaseTools: Optimize GenerateByteArrayValue and > CollectPlatformGuids APIs >=20 > During the Incremental build GenerateByteArrayValue used to generate the > ByteArrayValue even when there is no change in the PCD/VPDs. which is tim= e > consuming API based on the number of PCD/VPDs and SKU IDs. >=20 > The optimization is that GenerateByteArrayValue is used to store the > StructuredPcdsData in a JSON file for each of the arch. and during the > Incremental build this API will check, if there is any change in the Stru= ctured > PCD/VPDs then rest of the flow remains the same. > if there is no change then it will return the provious build data. >=20 > Flow: > during the 1st build StructuredPcdsData.json is not exists, StructuredPcd= sData > will be dumped to json file. and it will copy the output.txt as well. > Note: as the output.txt are different for different Arch, so it will be s= tored in > the Arch folder. > During the Incremental build check if there is any change in Structured > PCD/VPD. if there is a change in Structured VPD/PCD then recreate the > StructuredPcdsData.json, and rest of the flow remains same. > if there is no change in VPD/PCD read the output.txt and return the data >=20 > Unit Test: > Test1: Modified the Structured Pcds default from DEC file. current flow i= s > executing. > Test2: Override the default value of the PCD from DEC file. current flow = is > executing. > Test3: Modified/Override the PCD from DSC file. current flow executing > Test4: Modified/Override the FDF from DSC file. current flow executing > Test5: update the default value from Command Line.current flow executing > Test6: Build without change in PCD in DSC, FDF, DEC and Command Line the > proposed changes will be executing, and the return data remains the same > with and without the changes. > Test7: Build with and without modified the include headers of Structured > PCDs. if there is any change in those Structured PCD header then current = flow > will be executed. >=20 > With these changes it's helping to save around ~2.5min to ~3.5min of > Incremental build time in my build environment. >=20 > Sample PR: https://github.com/tianocore/edk2-basetools/pull/113 >=20 > Cc: Yuwei Chen > Cc: Rebecca Cran > Cc: Liming Gao > Cc: Bob Feng > Cc: Amy Chan > Cc: Sai Chaganty > Cc: Digant H Solanki > Signed-off-by: Ashraf Ali S > --- > .../Source/Python/AutoGen/WorkspaceAutoGen.py | 16 +- > .../Source/Python/Workspace/DscBuildData.py | 181 +++++++++++++----- > 2 files changed, 139 insertions(+), 58 deletions(-) >=20 > diff --git a/BaseTools/Source/Python/AutoGen/WorkspaceAutoGen.py > b/BaseTools/Source/Python/AutoGen/WorkspaceAutoGen.py > index 160e3a3cd3..eec9280c8e 100644 > --- a/BaseTools/Source/Python/AutoGen/WorkspaceAutoGen.py > +++ b/BaseTools/Source/Python/AutoGen/WorkspaceAutoGen.py > @@ -160,22 +160,18 @@ class WorkspaceAutoGen(AutoGen): >=20 > def CollectPlatformGuids(self): > oriInfList =3D [] > - oriPkgSet =3D set() > - PlatformPkg =3D set() > + pkgSet =3D set() > for Arch in self.ArchList: > Platform =3D self.BuildDatabase[self.MetaFile, Arch, self.Bu= ildTarget, > self.ToolChain] > oriInfList =3D Platform.Modules > for ModuleFile in oriInfList: > ModuleData =3D self.BuildDatabase[ModuleFile, Platform._= Arch, > Platform._Target, Platform._Toolchain] > - oriPkgSet.update(ModuleData.Packages) > - for Pkg in oriPkgSet: > - Guids =3D Pkg.Guids > - GlobalData.gGuidDict.update(Guids) > + pkgSet.update(ModuleData.Packages) > if Platform.Packages: > - PlatformPkg.update(Platform.Packages) > - for Pkg in PlatformPkg: > - Guids =3D Pkg.Guids > - GlobalData.gGuidDict.update(Guids) > + pkgSet.update(Platform.Packages) > + for Pkg in pkgSet: > + Guids =3D Pkg.Guids > + GlobalData.gGuidDict.update(Guids) >=20 > @cached_property > def FdfProfile(self): > diff --git a/BaseTools/Source/Python/Workspace/DscBuildData.py > b/BaseTools/Source/Python/Workspace/DscBuildData.py > index 4768099343..ddcd157c73 100644 > --- a/BaseTools/Source/Python/Workspace/DscBuildData.py > +++ b/BaseTools/Source/Python/Workspace/DscBuildData.py > @@ -37,6 +37,8 @@ from functools import reduce from Common.Misc > import SaveFileOnChange from Workspace.BuildClassObject import > PlatformBuildClassObject, StructurePcd, PcdClassObject, > ModuleBuildClassObject from collections import OrderedDict, defaultdict > +import json > +import shutil >=20 > def _IsFieldValueAnArray (Value): > Value =3D Value.strip() > @@ -56,6 +58,7 @@ def _IsFieldValueAnArray (Value): >=20 > PcdValueInitName =3D 'PcdValueInit' > PcdValueCommonName =3D 'PcdValueCommon' > +StructuredPcdsDataName =3D 'StructuredPcdsData.json' >=20 > PcdMainCHeader =3D ''' > /** > @@ -2750,6 +2753,63 @@ class DscBuildData(PlatformBuildClassObject): > ccflags.add(item) > i +=3D1 > return ccflags > + > + def GetStructurePcdSet (self, OutputValueFile): > + if not os.path.isfile(OutputValueFile): > + EdkLogger.error("GetStructurePcdSet", FILE_NOT_FOUND, "Outpu= t.txt > doesn't exist", ExtraData=3DOutputValueFile) > + return [] > + File =3D open (OutputValueFile, 'r') > + FileBuffer =3D File.readlines() > + File.close() > + > + #start update structure pcd final value > + StructurePcdSet =3D [] > + for Pcd in FileBuffer: > + PcdValue =3D Pcd.split ('|') > + PcdInfo =3D PcdValue[0].split ('.') > + StructurePcdSet.append((PcdInfo[0], PcdInfo[1], PcdInfo[2], > PcdInfo[3], PcdValue[2].strip())) > + return StructurePcdSet > + > + def GetBuildOptionsValueList(self): > + CC_FLAGS =3D LinuxCFLAGS > + if sys.platform =3D=3D "win32": > + CC_FLAGS =3D WindowsCFLAGS > + BuildOptions =3D OrderedDict() > + for Options in self.BuildOptions: > + if Options[2] !=3D EDKII_NAME: > + continue > + Family =3D Options[0] > + if Family and Family !=3D self.ToolChainFamily: > + continue > + Target, Tag, Arch, Tool, Attr =3D Options[1].split("_") > + if Tool !=3D 'CC': > + continue > + if Attr !=3D "FLAGS": > + continue > + if Target =3D=3D TAB_STAR or Target =3D=3D self._Target: > + if Tag =3D=3D TAB_STAR or Tag =3D=3D self._Toolchain: > + if 'COMMON' not in BuildOptions: > + BuildOptions['COMMON'] =3D set() > + if Arch =3D=3D TAB_STAR: > + BuildOptions['COMMON']|=3D > self.ParseCCFlags(self.BuildOptions[Options]) > + if Arch in self.SupArchList: > + if Arch not in BuildOptions: > + BuildOptions[Arch] =3D set() > + BuildOptions[Arch] |=3D > + self.ParseCCFlags(self.BuildOptions[Options]) > + > + if BuildOptions: > + ArchBuildOptions =3D {arch:flags for arch,flags in BuildOpti= ons.items() if > arch !=3D 'COMMON'} > + if len(ArchBuildOptions.keys()) =3D=3D 1: > + BuildOptions['COMMON'] |=3D (list(ArchBuildOptions.value= s())[0]) > + elif len(ArchBuildOptions.keys()) > 1: > + CommonBuildOptions =3D reduce(lambda x,y: x&y, > ArchBuildOptions.values()) > + BuildOptions['COMMON'] |=3D CommonBuildOptions > + ValueList =3D [item for item in BuildOptions['COMMON'] if > item.startswith((r"/U","-U"))] > + ValueList.extend([item for item in BuildOptions['COMMON'] if > item.startswith((r"/D", "-D"))]) > + CC_FLAGS +=3D " ".join(ValueList) > + return CC_FLAGS > + > + > def GenerateByteArrayValue (self, StructuredPcds): > # > # Generate/Compile/Run C application to determine if there are a= ny > flexible array members @@ -2757,6 +2817,66 @@ class > DscBuildData(PlatformBuildClassObject): > if not StructuredPcds: > return >=20 > + StructuredPcdsData =3D {} > + StoredStructuredPcdObjectPaths =3D {} > + SkipPcdValueInit =3D False > + > + CC_FLAGS =3D self.GetBuildOptionsValueList() > + > + for PcdName in StructuredPcds: > + Pcd =3D StructuredPcds[PcdName] > + TokenSpaceGuidCName =3D Pcd.TokenSpaceGuidCName > + TokenCName =3D Pcd.TokenCName > + > + # Create a key using TokenSpaceGuidCName and TokenCName > + StructuredPcdsData[f"{TokenSpaceGuidCName}_{TokenCName}"] = =3D { > + "DefaultValueFromDec": Pcd.DefaultValueFromDec, > + "DefaultValues": Pcd.DefaultValues, > + "PcdFieldValueFromComm": Pcd.PcdFieldValueFromComm, > + "PcdFieldValueFromFdf": Pcd.PcdFieldValueFromFdf, > + "DefaultFromDSC": Pcd.DefaultFromDSC, > + "PcdFiledValueFromDscComponent": > Pcd.PcdFiledValueFromDscComponent > + } > + > + # Store the CC Flags > + StructuredPcdsData["CC_FLAGS"] =3D CC_FLAGS > + # > + # If the output path doesn't exists then create it > + # > + if not os.path.exists(self.OutputPath): > + os.makedirs(self.OutputPath) > + > + StructuredPcdsDataPath =3D os.path.join(self.OutputPath, self._A= rch, > StructuredPcdsDataName) > + PcdRecordOutputValueFile =3D os.path.join(self.OutputPath, > + self._Arch, 'Output.txt') > + > + if not os.path.exists(os.path.dirname(StructuredPcdsDataPath)): > + os.makedirs(os.path.dirname(StructuredPcdsDataPath)) > + # > + # Check if the StructuredPcdsData.json exists or not > + # if exits then it might be a incremental build then check if th= e > StructuredPcdsData has been changed or not. > + # if changed then proceed further, if not changed then return th= e stored > data from earlier build > + # > + if os.path.isfile(StructuredPcdsDataPath): > + with open(StructuredPcdsDataPath, 'r') as file: > + StoredStructuredPcdsData =3D json.load(file) > + # OBJECTS will have the modified time, which needs to be= checked > later > + StoredStructuredPcdObjectPaths =3D > + StoredStructuredPcdsData.pop("OBJECTS", {}) > + > + if StructuredPcdsData =3D=3D StoredStructuredPcdsData: > + SkipPcdValueInit =3D True > + for filename, file_mtime in > StoredStructuredPcdObjectPaths.items(): > + f_mtime =3D os.path.getmtime(filename) > + # > + # check if the include_file are modified or not, > + # if modified then generate the PcdValueInit > + # > + if f_mtime !=3D file_mtime: > + SkipPcdValueInit =3D False > + break > + > + if SkipPcdValueInit: > + return self.GetStructurePcdSet(PcdRecordOutputValueFile) > + > InitByteValue =3D "" > CApp =3D PcdMainCHeader >=20 > @@ -2832,8 +2952,6 @@ class DscBuildData(PlatformBuildClassObject): >=20 > CApp =3D CApp + PcdMainCEntry + '\n' >=20 > - if not os.path.exists(self.OutputPath): > - os.makedirs(self.OutputPath) > CAppBaseFileName =3D os.path.join(self.OutputPath, PcdValueInitN= ame) > SaveFileOnChange(CAppBaseFileName + '.c', CApp, False) >=20 > @@ -2890,42 +3008,6 @@ class DscBuildData(PlatformBuildClassObject): > IncSearchList.append(inc) > MakeApp =3D MakeApp + '\n' >=20 > - CC_FLAGS =3D LinuxCFLAGS > - if sys.platform =3D=3D "win32": > - CC_FLAGS =3D WindowsCFLAGS > - BuildOptions =3D OrderedDict() > - for Options in self.BuildOptions: > - if Options[2] !=3D EDKII_NAME: > - continue > - Family =3D Options[0] > - if Family and Family !=3D self.ToolChainFamily: > - continue > - Target, Tag, Arch, Tool, Attr =3D Options[1].split("_") > - if Tool !=3D 'CC': > - continue > - if Attr !=3D "FLAGS": > - continue > - if Target =3D=3D TAB_STAR or Target =3D=3D self._Target: > - if Tag =3D=3D TAB_STAR or Tag =3D=3D self._Toolchain: > - if 'COMMON' not in BuildOptions: > - BuildOptions['COMMON'] =3D set() > - if Arch =3D=3D TAB_STAR: > - BuildOptions['COMMON']|=3D > self.ParseCCFlags(self.BuildOptions[Options]) > - if Arch in self.SupArchList: > - if Arch not in BuildOptions: > - BuildOptions[Arch] =3D set() > - BuildOptions[Arch] |=3D > self.ParseCCFlags(self.BuildOptions[Options]) > - > - if BuildOptions: > - ArchBuildOptions =3D {arch:flags for arch,flags in BuildOpti= ons.items() if > arch !=3D 'COMMON'} > - if len(ArchBuildOptions.keys()) =3D=3D 1: > - BuildOptions['COMMON'] |=3D (list(ArchBuildOptions.value= s())[0]) > - elif len(ArchBuildOptions.keys()) > 1: > - CommonBuildOptions =3D reduce(lambda x,y: x&y, > ArchBuildOptions.values()) > - BuildOptions['COMMON'] |=3D CommonBuildOptions > - ValueList =3D [item for item in BuildOptions['COMMON'] if > item.startswith((r"/U","-U"))] > - ValueList.extend([item for item in BuildOptions['COMMON'] if > item.startswith((r"/D", "-D"))]) > - CC_FLAGS +=3D " ".join(ValueList) > MakeApp +=3D CC_FLAGS >=20 > if sys.platform =3D=3D "win32": > @@ -2946,7 +3028,9 @@ class DscBuildData(PlatformBuildClassObject): >=20 > SearchPathList.append(os.path.normpath(mws.join(GlobalData.gGlobalDefin > es["EDK_TOOLS_PATH"], "BaseTools/Source/C/Common"))) > SearchPathList.extend(str(item) for item in IncSearchList) > IncFileList =3D GetDependencyList(IncludeFileFullPaths, SearchPa= thList) > + StructuredPcdsData["OBJECTS"] =3D {} > for include_file in IncFileList: > + StructuredPcdsData["OBJECTS"][include_file] =3D > + os.path.getmtime(include_file) > MakeApp +=3D "$(OBJECTS) : %s\n" % include_file > if sys.platform =3D=3D "win32": > PcdValueCommonPath =3D > os.path.normpath(mws.join(GlobalData.gGlobalDefines["EDK_TOOLS_PATH"] > , "Source\C\Common\PcdValueCommon.c")) > @@ -3042,17 +3126,18 @@ class DscBuildData(PlatformBuildClassObject): > if returncode !=3D 0: > EdkLogger.warn('Build', COMMAND_FAILURE, 'Can not collec= t > output from command: %s\n%s\n%s\n' % (Command, StdOut, StdErr)) >=20 > - #start update structure pcd final value > - File =3D open (OutputValueFile, 'r') > - FileBuffer =3D File.readlines() > - File.close() > + # > + # In 1st build create the StructuredPcdsData.json > + # update the record as PCD Input has been changed if its increme= ntal > build > + # > + with open(StructuredPcdsDataPath, 'w') as file: > + json.dump(StructuredPcdsData, file, indent=3D2) >=20 > - StructurePcdSet =3D [] > - for Pcd in FileBuffer: > - PcdValue =3D Pcd.split ('|') > - PcdInfo =3D PcdValue[0].split ('.') > - StructurePcdSet.append((PcdInfo[0], PcdInfo[1], PcdInfo[2], = PcdInfo[3], > PcdValue[2].strip())) > - return StructurePcdSet > + # Copy update output file for each Arch > + shutil.copyfile(OutputValueFile, PcdRecordOutputValueFile) > + > + #start update structure pcd final value > + return self.GetStructurePcdSet(OutputValueFile) >=20 > @staticmethod > def NeedUpdateOutput(OutputFile, ValueCFile, StructureInput): > -- > 2.39.1.windows.1 -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#115261): https://edk2.groups.io/g/devel/message/115261 Mute This Topic: https://groups.io/mt/104222804/7686176 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-