From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by mx.groups.io with SMTP id smtpd.web12.1808.1574217956256943278 for ; Tue, 19 Nov 2019 18:45:56 -0800 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: intel.com, ip: 134.134.136.100, mailfrom: zhijux.fan@intel.com) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga105.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 19 Nov 2019 18:45:50 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.69,220,1571727600"; d="scan'208";a="289805557" Received: from fmsmsx107.amr.corp.intel.com ([10.18.124.205]) by orsmga001.jf.intel.com with ESMTP; 19 Nov 2019 18:45:50 -0800 Received: from fmsmsx604.amr.corp.intel.com (10.18.126.84) by fmsmsx107.amr.corp.intel.com (10.18.124.205) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 19 Nov 2019 18:45:50 -0800 Received: from fmsmsx604.amr.corp.intel.com (10.18.126.84) by fmsmsx604.amr.corp.intel.com (10.18.126.84) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Tue, 19 Nov 2019 18:45:47 -0800 Received: from shsmsx104.ccr.corp.intel.com (10.239.4.70) by fmsmsx604.amr.corp.intel.com (10.18.126.84) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.1713.5 via Frontend Transport; Tue, 19 Nov 2019 18:45:46 -0800 Received: from shsmsx101.ccr.corp.intel.com ([169.254.1.213]) by SHSMSX104.ccr.corp.intel.com ([169.254.5.127]) with mapi id 14.03.0439.000; Wed, 20 Nov 2019 10:45:45 +0800 From: "Fan, ZhijuX" To: "Gao, Liming" , "devel@edk2.groups.io" CC: "Feng, Bob C" Subject: Re: [PATCH V2] BaseTools:Add [packages] section in dsc file Thread-Topic: [PATCH V2] BaseTools:Add [packages] section in dsc file Thread-Index: AdWairM9JYS36Rs6QsuXhur7pWjW8QAzOi0wAPwmZkA= Date: Wed, 20 Nov 2019 02:45:45 +0000 Message-ID: References: <4A89E2EF3DFEDB4C8BFDE51014F606A14E54145B@SHSMSX104.ccr.corp.intel.com> In-Reply-To: <4A89E2EF3DFEDB4C8BFDE51014F606A14E54145B@SHSMSX104.ccr.corp.intel.com> Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ctpclassification: CTP_NT x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiMTU5NmQyMjQtMzAzOS00YjA5LWIxMGYtZDJiM2U5M2M3OTFmIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoic28rYTRGRjIzOFhFXC9hZEsxM2dOano4V2hzWHIzNmhNczMzQlwvNHZxYUF2S29hQndTTndzSFVpcEpndFJ1bE1VIn0= dlp-product: dlpe-windows dlp-version: 11.2.0.6 dlp-reaction: no-action x-originating-ip: [10.239.127.40] MIME-Version: 1.0 Return-Path: zhijux.fan@intel.com Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi: regression test: pass build for minplatforms, WhitleyRp, TigerLake functional test: Define a PCD in a dec file (e.g. StandaloneMmPkg.dec) that none of the modu= les depend on, it is defined in a [PcdsFixedAtBuild] or [PcdsFeatureFlag] section Create the [Packages] section in the DSC file and add " StandaloneMmPkg.dec= " in [Packages] section Test whether this PCD can be used in conditional statements in DSC/FDF file= s Any question, please let me know. Thanks. Best Regards Fan Zhiju > -----Original Message----- > From: Gao, Liming > Sent: Friday, November 15, 2019 9:55 AM > To: Fan, ZhijuX ; devel@edk2.groups.io > Cc: Feng, Bob C > Subject: RE: [PATCH V2] BaseTools:Add [packages] section in dsc file >=20 > Zhiju: > Can you show what test are done for this new support? >=20 > Thanks > Liming > >-----Original Message----- > >From: Fan, ZhijuX > >Sent: Thursday, November 14, 2019 9:28 AM > >To: devel@edk2.groups.io > >Cc: Gao, Liming ; Feng, Bob C > > > >Subject: [PATCH V2] BaseTools:Add [packages] section in dsc file > > > >BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=3D2270 > > > >Currently a PCD (e.g. FeaturePCD) cannot be used in a conditional > >statement in a DSC/FDF file without a module in the build referencing > >the PCD package DEC file. > > > >An example implementation that to support this is to allow a [Packages] > >section in the DSC file to list additional package dependencies for PCD > >references in the package DSC/FDF files. > > > >this patch is going to add the ability to have the [packages] section > >defined in the DSC file > > > >Cc: Liming Gao > >Cc: Bob Feng > >Signed-off-by: Zhiju.Fan > >--- > > BaseTools/Source/Python/AutoGen/ModuleAutoGen.py | 29 > >+++++++++++++++++----- > > BaseTools/Source/Python/AutoGen/PlatformAutoGen.py | 1 + > > .../Source/Python/AutoGen/WorkspaceAutoGen.py | 1 + > > BaseTools/Source/Python/Common/DataType.py | 1 + > > BaseTools/Source/Python/Workspace/DscBuildData.py | 23 > >++++++++++++++++- > > .../Source/Python/Workspace/MetaFileParser.py | 14 +++++++++++ > > .../Source/Python/Workspace/WorkspaceCommon.py | 2 ++ > > .../Source/Python/Workspace/WorkspaceDatabase.py | 4 +++ > > 8 files changed, 68 insertions(+), 7 deletions(-) > > > >diff --git a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py > >b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py > >index f0812b6887..e6d6c43810 100755 > >--- a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py > >+++ b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py > >@@ -462,14 +462,31 @@ class ModuleAutoGen(AutoGen): > > def BuildCommand(self): > > return self.PlatformInfo.BuildCommand > > > >- ## Get object list of all packages the module and its dependent lib= raries > >belong to > >+ ## Get Module package and Platform package > >+ # > >+ # @retval list The list of package object > >+ # > >+ @cached_property > >+ def PackageList(self): > >+ PkagList =3D [] > >+ if self.Module.Packages: > >+ PkagList.extend(self.Module.Packages) > >+ Platform =3D self.BuildDatabase[self.PlatformInfo.MetaFile, > >+ self.Arch, > >self.BuildTarget, self.ToolChain] > >+ for Package in Platform.Packages: > >+ if Package in PkagList: > >+ continue > >+ PkagList.append(Package) > >+ return PkagList > >+ > >+ ## Get object list of all packages the module and its dependent > >+ libraries > >belong to and the Platform depends on > > # > > # @retval list The list of package object > > # > > @cached_property > > def DerivedPackageList(self): > > PackageList =3D [] > >- for M in [self.Module] + self.DependentLibraryList: > >+ PackageList.extend(self.PackageList) > >+ for M in self.DependentLibraryList: > > for Package in M.Packages: > > if Package in PackageList: > > continue > >@@ -938,13 +955,13 @@ class ModuleAutoGen(AutoGen): > > self.Targets > > return self._FileTypes > > > >- ## Get the list of package object the module depends on > >+ ## Get the list of package object the module depends on and the > >+ Platform > >depends on > > # > > # @retval list The package object list > > # > > @cached_property > > def DependentPackageList(self): > >- return self.Module.Packages > >+ return self.PackageList > > > > ## Return the list of auto-generated code file > > # > >@@ -1101,7 +1118,7 @@ class ModuleAutoGen(AutoGen): > > RetVal.append(self.MetaFile.Dir) > > RetVal.append(self.DebugDir) > > > >- for Package in self.Module.Packages: > >+ for Package in self.PackageList: > > PackageDir =3D mws.join(self.WorkspaceDir, Package.MetaFile= .Dir) > > if PackageDir not in RetVal: > > RetVal.append(PackageDir) @@ -1125,7 +1142,7 @@ class > >ModuleAutoGen(AutoGen): > > @cached_property > > def PackageIncludePathList(self): > > IncludesList =3D [] > >- for Package in self.Module.Packages: > >+ for Package in self.PackageList: > > PackageDir =3D mws.join(self.WorkspaceDir, Package.MetaFile= .Dir) > > IncludesList =3D Package.Includes > > if Package._PrivateIncludes: > >diff --git a/BaseTools/Source/Python/AutoGen/PlatformAutoGen.py > >b/BaseTools/Source/Python/AutoGen/PlatformAutoGen.py > >index debeb46f58..4c3cdf82d5 100644 > >--- a/BaseTools/Source/Python/AutoGen/PlatformAutoGen.py > >+++ b/BaseTools/Source/Python/AutoGen/PlatformAutoGen.py > >@@ -975,6 +975,7 @@ class PlatformAutoGen(AutoGen): > > continue > > ModuleData =3D self.BuildDatabase[ModuleFile, self.Arch, > >self.BuildTarget, self.ToolChain] > > RetVal.update(ModuleData.Packages) > >+ RetVal.update(self.Platform.Packages) > > return list(RetVal) > > > > @cached_property > >diff --git a/BaseTools/Source/Python/AutoGen/WorkspaceAutoGen.py > >b/BaseTools/Source/Python/AutoGen/WorkspaceAutoGen.py > >index 9d8040905e..fde48b4b27 100644 > >--- a/BaseTools/Source/Python/AutoGen/WorkspaceAutoGen.py > >+++ b/BaseTools/Source/Python/AutoGen/WorkspaceAutoGen.py > >@@ -420,6 +420,7 @@ class WorkspaceAutoGen(AutoGen): > > continue > > ModuleData =3D self.BuildDatabase[ModuleFile, Arch, > >self.BuildTarget, self.ToolChain] > > PkgSet.update(ModuleData.Packages) > >+ PkgSet.update(Platform.Packages) > > Pkgs[Arch] =3D list(PkgSet) > > return Pkgs > > > >diff --git a/BaseTools/Source/Python/Common/DataType.py > >b/BaseTools/Source/Python/Common/DataType.py > >index 5d49afb0a9..8d80b41089 100644 > >--- a/BaseTools/Source/Python/Common/DataType.py > >+++ b/BaseTools/Source/Python/Common/DataType.py > >@@ -519,6 +519,7 @@ SECTIONS_HAVE_ITEM_AFTER_ARCH_SET =3D > >{TAB_LIBRARY_CLASSES.upper(), TAB_DEPEX.uppe > > PCDS_DYNAMICEX_VPD.upper(), > > PCDS_DYNAMICEX_HII.upper(), > > TAB_BUILD_OPTIONS.upper(), > >+ TAB_PACKAGES.upper(), > > TAB_INCLUDES.upper()} > > > > # > >diff --git a/BaseTools/Source/Python/Workspace/DscBuildData.py > >b/BaseTools/Source/Python/Workspace/DscBuildData.py > >index 9192077f90..03a15bbf3e 100644 > >--- a/BaseTools/Source/Python/Workspace/DscBuildData.py > >+++ b/BaseTools/Source/Python/Workspace/DscBuildData.py > >@@ -719,6 +719,24 @@ class DscBuildData(PlatformBuildClassObject): > > > >self._RawData.DisableOverrideComponent(Components[(file_guid_str,str( > M > >oduleFile))]) > > Components[(file_guid_str,str(ModuleFile))] =3D ModuleId > > self._RawData._PostProcessed =3D False > >+ > >+ ## Retrieve packages this Platform depends on > >+ @cached_property > >+ def Packages(self): > >+ RetVal =3D set() > >+ RecordList =3D self._RawData[MODEL_META_DATA_PACKAGE, > self._Arch] > >+ Macros =3D self._Macros > >+ for Record in RecordList: > >+ File =3D PathClass(NormPath(Record[0], Macros), > >GlobalData.gWorkspace, Arch=3Dself._Arch) > >+ # check the file validation > >+ ErrorCode, ErrorInfo =3D File.Validate('.dec') > >+ if ErrorCode !=3D 0: > >+ LineNo =3D Record[-1] > >+ EdkLogger.error('build', ErrorCode, > >+ ExtraData=3DErrorInfo, > >File=3Dself.MetaFile, Line=3DLineNo) > >+ # parse this package now. we need it to get protocol/ppi/gu= id value > >+ RetVal.add(self._Bdb[File, self._Arch, self._Target, self._= Toolchain]) > >+ return RetVal > >+ > > ## Retrieve [Components] section information > > @property > > def Modules(self): > >@@ -896,7 +914,8 @@ class DscBuildData(PlatformBuildClassObject): > > continue > > ModuleData =3D self._Bdb[ModuleFile, self._Arch, > >self._Target, self._Toolchain] > > PkgSet.update(ModuleData.Packages) > >- > >+ if self.Packages: > >+ PkgSet.update(self.Packages) > > self._DecPcds, self._GuidDict =3D GetDeclaredPcd(self, > >self._Bdb, self._Arch, self._Target, self._Toolchain, PkgSet) > > self._GuidDict.update(GlobalData.gPlatformPcds) > > > >@@ -3320,6 +3339,8 @@ class DscBuildData(PlatformBuildClassObject): > > continue > > ModuleData =3D self._Bdb[ModuleFile, self._Arch, > >self._Target, self._Toolchain] > > PkgSet.update(ModuleData.Packages) > >+ if self.Packages: > >+ PkgSet.update(self.Packages) > > self._DecPcds, self._GuidDict =3D GetDeclaredPcd(self, > >self._Bdb, self._Arch, self._Target, self._Toolchain, PkgSet) > > self._GuidDict.update(GlobalData.gPlatformPcds) > > return self._DecPcds > >diff --git a/BaseTools/Source/Python/Workspace/MetaFileParser.py > >b/BaseTools/Source/Python/Workspace/MetaFileParser.py > >index 806fc322c5..3f96ce0564 100644 > >--- a/BaseTools/Source/Python/Workspace/MetaFileParser.py > >+++ b/BaseTools/Source/Python/Workspace/MetaFileParser.py > >@@ -160,6 +160,7 @@ class MetaFileParser(object): > > self.MetaFile =3D FilePath > > self._FileDir =3D self.MetaFile.Dir > > self._Defines =3D {} > >+ self._Packages =3D [] > > self._FileLocalMacros =3D {} > > self._SectionsMacroDict =3D defaultdict(dict) > > > >@@ -351,6 +352,13 @@ class MetaFileParser(object): > > # If the section information is needed later, it should be > >stored in database > > self._ValueList[0] =3D self._SectionName > > > >+ ## [packages] section parser > >+ @ParseMacro > >+ def _PackageParser(self): > >+ self._CurrentLine =3D CleanString(self._CurrentLine) > >+ self._Packages.append(self._CurrentLine) > >+ self._ValueList[0] =3D self._CurrentLine > >+ > > ## [defines] section parser > > @ParseMacro > > def _DefineParser(self): > >@@ -848,6 +856,7 @@ class DscParser(MetaFileParser): > > TAB_LIBRARIES.upper() : MODEL_EFI_LIBRA= RY_INSTANCE, > > TAB_LIBRARY_CLASSES.upper() : MODEL_EFI_LIBRA= RY_CLASS, > > TAB_BUILD_OPTIONS.upper() : > >MODEL_META_DATA_BUILD_OPTION, > >+ TAB_PACKAGES.upper() : > MODEL_META_DATA_PACKAGE, > > TAB_PCDS_FIXED_AT_BUILD_NULL.upper() : > >MODEL_PCD_FIXED_AT_BUILD, > > TAB_PCDS_PATCHABLE_IN_MODULE_NULL.upper() : > >MODEL_PCD_PATCHABLE_IN_MODULE, > > TAB_PCDS_FEATURE_FLAG_NULL.upper() : > >MODEL_PCD_FEATURE_FLAG, > >@@ -1339,6 +1348,7 @@ class DscParser(MetaFileParser): > > MODEL_META_DATA_DEFINE : self.__= ProcessDefine, > > MODEL_META_DATA_GLOBAL_DEFINE : > >self.__ProcessDefine, > > MODEL_META_DATA_INCLUDE : > self.__ProcessDirective, > >+ MODEL_META_DATA_PACKAGE : > self.__ProcessPackages, > > MODEL_META_DATA_CONDITIONAL_STATEMENT_IF : > >self.__ProcessDirective, > > MODEL_META_DATA_CONDITIONAL_STATEMENT_ELSE : > >self.__ProcessDirective, > > MODEL_META_DATA_CONDITIONAL_STATEMENT_IFDEF : > >self.__ProcessDirective, > >@@ -1642,6 +1652,9 @@ class DscParser(MetaFileParser): > > self._ValueList =3D None > > self._ContentIndex -=3D 1 > > > >+ def __ProcessPackages(self): > >+ self._ValueList[0] =3D ReplaceMacro(self._ValueList[0], > >+ self._Macros) > >+ > > def __ProcessSkuId(self): > > self._ValueList =3D [ReplaceMacro(Value, self._Macros, RaiseErr= or=3DTrue) > > for Value in self._ValueList] @@ -1720,6 > >+1733,7 @@ class DscParser(MetaFileParser): > > MODEL_META_DATA_COMPONENT : _ComponentP= arser, > > MODEL_META_DATA_BUILD_OPTION : _BuildOptio= nParser, > > MODEL_UNKNOWN : MetaFilePar= ser._Skip, > >+ MODEL_META_DATA_PACKAGE : > >MetaFileParser._PackageParser, > > MODEL_META_DATA_USER_EXTENSION : > >MetaFileParser._SkipUserExtension, > > MODEL_META_DATA_SECTION_HEADER : > >MetaFileParser._SectionHeaderParser, > > MODEL_META_DATA_SUBSECTION_HEADER : > >_SubsectionHeaderParser, > >diff --git a/BaseTools/Source/Python/Workspace/WorkspaceCommon.py > >b/BaseTools/Source/Python/Workspace/WorkspaceCommon.py > >index 0b11ec2d59..913e710fd9 100644 > >--- a/BaseTools/Source/Python/Workspace/WorkspaceCommon.py > >+++ b/BaseTools/Source/Python/Workspace/WorkspaceCommon.py > >@@ -37,6 +37,8 @@ class OrderedListDict(OrderedDict): > > # > > def GetPackageList(Platform, BuildDatabase, Arch, Target, Toolchain): > > PkgSet =3D set() > >+ if Platform.Packages: > >+ PkgSet.update(Platform.Packages) > > for ModuleFile in Platform.Modules: > > Data =3D BuildDatabase[ModuleFile, Arch, Target, Toolchain] > > PkgSet.update(Data.Packages) > >diff --git a/BaseTools/Source/Python/Workspace/WorkspaceDatabase.py > >b/BaseTools/Source/Python/Workspace/WorkspaceDatabase.py > >index ab7b4506c1..9420eaa608 100644 > >--- a/BaseTools/Source/Python/Workspace/WorkspaceDatabase.py > >+++ b/BaseTools/Source/Python/Workspace/WorkspaceDatabase.py > >@@ -186,6 +186,10 @@ class WorkspaceDatabase(object): > > for Package in LibObj.Packages: > > if Package not in PackageList: > > PackageList.append(Package) > >+ for Package in Pa.Packages: > >+ if Package in PackageList: > >+ continue > >+ PackageList.append(Package) > > > > return PackageList > > > >-- > >2.14.1.windows.1