From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by mx.groups.io with SMTP id smtpd.web11.3086.1574226783840216899 for ; Tue, 19 Nov 2019 21:13:03 -0800 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: intel.com, ip: 192.55.52.136, mailfrom: liming.gao@intel.com) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 19 Nov 2019 21:13:02 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.69,220,1571727600"; d="scan'208";a="289833840" Received: from fmsmsx105.amr.corp.intel.com ([10.18.124.203]) by orsmga001.jf.intel.com with ESMTP; 19 Nov 2019 21:13:03 -0800 Received: from fmsmsx602.amr.corp.intel.com (10.18.126.82) by FMSMSX105.amr.corp.intel.com (10.18.124.203) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 19 Nov 2019 21:13:02 -0800 Received: from fmsmsx602.amr.corp.intel.com (10.18.126.82) by fmsmsx602.amr.corp.intel.com (10.18.126.82) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Tue, 19 Nov 2019 21:13:02 -0800 Received: from shsmsx101.ccr.corp.intel.com (10.239.4.153) by fmsmsx602.amr.corp.intel.com (10.18.126.82) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.1713.5 via Frontend Transport; Tue, 19 Nov 2019 21:13:02 -0800 Received: from shsmsx104.ccr.corp.intel.com ([169.254.5.127]) by SHSMSX101.ccr.corp.intel.com ([169.254.1.213]) with mapi id 14.03.0439.000; Wed, 20 Nov 2019 13:13:01 +0800 From: "Liming Gao" To: "Fan, ZhijuX" , "devel@edk2.groups.io" CC: "Feng, Bob C" Subject: Re: [PATCH V2] BaseTools:Add [packages] section in dsc file Thread-Topic: [PATCH V2] BaseTools:Add [packages] section in dsc file Thread-Index: AdWairM9JYS36Rs6QsuXhur7pWjW8QAzOi0wAPwmZkAABjjCEA== Date: Wed, 20 Nov 2019 05:13:00 +0000 Message-ID: <4A89E2EF3DFEDB4C8BFDE51014F606A14E543E1F@SHSMSX104.ccr.corp.intel.com> References: <4A89E2EF3DFEDB4C8BFDE51014F606A14E54145B@SHSMSX104.ccr.corp.intel.com> In-Reply-To: Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.239.127.40] MIME-Version: 1.0 Return-Path: liming.gao@intel.com Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable The test case is ok to me.=20 Acked-by: Liming Gao Thanks Liming >-----Original Message----- >From: Fan, ZhijuX >Sent: Wednesday, November 20, 2019 10:46 AM >To: Gao, Liming ; devel@edk2.groups.io >Cc: Feng, Bob C >Subject: RE: [PATCH V2] BaseTools:Add [packages] section in dsc file > >Hi: > >regression test: >pass build for minplatforms, WhitleyRp, TigerLake > >functional test: > >Define a PCD in a dec file (e.g. StandaloneMmPkg.dec) that none of the >modules depend on, >it is defined in a [PcdsFixedAtBuild] or [PcdsFeatureFlag] section >Create the [Packages] section in the DSC file and add " >StandaloneMmPkg.dec" in [Packages] section > >Test whether this PCD can be used in conditional statements in DSC/FDF fil= es > > >Any question, please let me know. Thanks. > >Best Regards >Fan Zhiju > > > >> -----Original Message----- >> From: Gao, Liming >> Sent: Friday, November 15, 2019 9:55 AM >> To: Fan, ZhijuX ; devel@edk2.groups.io >> Cc: Feng, Bob C >> Subject: RE: [PATCH V2] BaseTools:Add [packages] section in dsc file >> >> Zhiju: >> Can you show what test are done for this new support? >> >> Thanks >> Liming >> >-----Original Message----- >> >From: Fan, ZhijuX >> >Sent: Thursday, November 14, 2019 9:28 AM >> >To: devel@edk2.groups.io >> >Cc: Gao, Liming ; Feng, Bob C >> > >> >Subject: [PATCH V2] BaseTools:Add [packages] section in dsc file >> > >> >BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=3D2270 >> > >> >Currently a PCD (e.g. FeaturePCD) cannot be used in a conditional >> >statement in a DSC/FDF file without a module in the build referencing >> >the PCD package DEC file. >> > >> >An example implementation that to support this is to allow a [Packages] >> >section in the DSC file to list additional package dependencies for PCD >> >references in the package DSC/FDF files. >> > >> >this patch is going to add the ability to have the [packages] section >> >defined in the DSC file >> > >> >Cc: Liming Gao >> >Cc: Bob Feng >> >Signed-off-by: Zhiju.Fan >> >--- >> > BaseTools/Source/Python/AutoGen/ModuleAutoGen.py | 29 >> >+++++++++++++++++----- >> > BaseTools/Source/Python/AutoGen/PlatformAutoGen.py | 1 + >> > .../Source/Python/AutoGen/WorkspaceAutoGen.py | 1 + >> > BaseTools/Source/Python/Common/DataType.py | 1 + >> > BaseTools/Source/Python/Workspace/DscBuildData.py | 23 >> >++++++++++++++++- >> > .../Source/Python/Workspace/MetaFileParser.py | 14 +++++++++++ >> > .../Source/Python/Workspace/WorkspaceCommon.py | 2 ++ >> > .../Source/Python/Workspace/WorkspaceDatabase.py | 4 +++ >> > 8 files changed, 68 insertions(+), 7 deletions(-) >> > >> >diff --git a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py >> >b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py >> >index f0812b6887..e6d6c43810 100755 >> >--- a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py >> >+++ b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py >> >@@ -462,14 +462,31 @@ class ModuleAutoGen(AutoGen): >> > def BuildCommand(self): >> > return self.PlatformInfo.BuildCommand >> > >> >- ## Get object list of all packages the module and its dependent li= braries >> >belong to >> >+ ## Get Module package and Platform package >> >+ # >> >+ # @retval list The list of package object >> >+ # >> >+ @cached_property >> >+ def PackageList(self): >> >+ PkagList =3D [] >> >+ if self.Module.Packages: >> >+ PkagList.extend(self.Module.Packages) >> >+ Platform =3D self.BuildDatabase[self.PlatformInfo.MetaFile, >> >+ self.Arch, >> >self.BuildTarget, self.ToolChain] >> >+ for Package in Platform.Packages: >> >+ if Package in PkagList: >> >+ continue >> >+ PkagList.append(Package) >> >+ return PkagList >> >+ >> >+ ## Get object list of all packages the module and its dependent >> >+ libraries >> >belong to and the Platform depends on >> > # >> > # @retval list The list of package object >> > # >> > @cached_property >> > def DerivedPackageList(self): >> > PackageList =3D [] >> >- for M in [self.Module] + self.DependentLibraryList: >> >+ PackageList.extend(self.PackageList) >> >+ for M in self.DependentLibraryList: >> > for Package in M.Packages: >> > if Package in PackageList: >> > continue >> >@@ -938,13 +955,13 @@ class ModuleAutoGen(AutoGen): >> > self.Targets >> > return self._FileTypes >> > >> >- ## Get the list of package object the module depends on >> >+ ## Get the list of package object the module depends on and the >> >+ Platform >> >depends on >> > # >> > # @retval list The package object list >> > # >> > @cached_property >> > def DependentPackageList(self): >> >- return self.Module.Packages >> >+ return self.PackageList >> > >> > ## Return the list of auto-generated code file >> > # >> >@@ -1101,7 +1118,7 @@ class ModuleAutoGen(AutoGen): >> > RetVal.append(self.MetaFile.Dir) >> > RetVal.append(self.DebugDir) >> > >> >- for Package in self.Module.Packages: >> >+ for Package in self.PackageList: >> > PackageDir =3D mws.join(self.WorkspaceDir, Package.MetaFil= e.Dir) >> > if PackageDir not in RetVal: >> > RetVal.append(PackageDir) @@ -1125,7 +1142,7 @@ class >> >ModuleAutoGen(AutoGen): >> > @cached_property >> > def PackageIncludePathList(self): >> > IncludesList =3D [] >> >- for Package in self.Module.Packages: >> >+ for Package in self.PackageList: >> > PackageDir =3D mws.join(self.WorkspaceDir, Package.MetaFil= e.Dir) >> > IncludesList =3D Package.Includes >> > if Package._PrivateIncludes: >> >diff --git a/BaseTools/Source/Python/AutoGen/PlatformAutoGen.py >> >b/BaseTools/Source/Python/AutoGen/PlatformAutoGen.py >> >index debeb46f58..4c3cdf82d5 100644 >> >--- a/BaseTools/Source/Python/AutoGen/PlatformAutoGen.py >> >+++ b/BaseTools/Source/Python/AutoGen/PlatformAutoGen.py >> >@@ -975,6 +975,7 @@ class PlatformAutoGen(AutoGen): >> > continue >> > ModuleData =3D self.BuildDatabase[ModuleFile, self.Arch, >> >self.BuildTarget, self.ToolChain] >> > RetVal.update(ModuleData.Packages) >> >+ RetVal.update(self.Platform.Packages) >> > return list(RetVal) >> > >> > @cached_property >> >diff --git a/BaseTools/Source/Python/AutoGen/WorkspaceAutoGen.py >> >b/BaseTools/Source/Python/AutoGen/WorkspaceAutoGen.py >> >index 9d8040905e..fde48b4b27 100644 >> >--- a/BaseTools/Source/Python/AutoGen/WorkspaceAutoGen.py >> >+++ b/BaseTools/Source/Python/AutoGen/WorkspaceAutoGen.py >> >@@ -420,6 +420,7 @@ class WorkspaceAutoGen(AutoGen): >> > continue >> > ModuleData =3D self.BuildDatabase[ModuleFile, Arch, >> >self.BuildTarget, self.ToolChain] >> > PkgSet.update(ModuleData.Packages) >> >+ PkgSet.update(Platform.Packages) >> > Pkgs[Arch] =3D list(PkgSet) >> > return Pkgs >> > >> >diff --git a/BaseTools/Source/Python/Common/DataType.py >> >b/BaseTools/Source/Python/Common/DataType.py >> >index 5d49afb0a9..8d80b41089 100644 >> >--- a/BaseTools/Source/Python/Common/DataType.py >> >+++ b/BaseTools/Source/Python/Common/DataType.py >> >@@ -519,6 +519,7 @@ SECTIONS_HAVE_ITEM_AFTER_ARCH_SET =3D >> >{TAB_LIBRARY_CLASSES.upper(), TAB_DEPEX.uppe >> > PCDS_DYNAMICEX_VPD.upper(), >> > PCDS_DYNAMICEX_HII.upper(), >> > TAB_BUILD_OPTIONS.upper(), >> >+ TAB_PACKAGES.upper(), >> > TAB_INCLUDES.upper()} >> > >> > # >> >diff --git a/BaseTools/Source/Python/Workspace/DscBuildData.py >> >b/BaseTools/Source/Python/Workspace/DscBuildData.py >> >index 9192077f90..03a15bbf3e 100644 >> >--- a/BaseTools/Source/Python/Workspace/DscBuildData.py >> >+++ b/BaseTools/Source/Python/Workspace/DscBuildData.py >> >@@ -719,6 +719,24 @@ class DscBuildData(PlatformBuildClassObject): >> > >> >self._RawData.DisableOverrideComponent(Components[(file_guid_str,str >( >> M >> >oduleFile))]) >> > Components[(file_guid_str,str(ModuleFile))] =3D ModuleId >> > self._RawData._PostProcessed =3D False >> >+ >> >+ ## Retrieve packages this Platform depends on >> >+ @cached_property >> >+ def Packages(self): >> >+ RetVal =3D set() >> >+ RecordList =3D self._RawData[MODEL_META_DATA_PACKAGE, >> self._Arch] >> >+ Macros =3D self._Macros >> >+ for Record in RecordList: >> >+ File =3D PathClass(NormPath(Record[0], Macros), >> >GlobalData.gWorkspace, Arch=3Dself._Arch) >> >+ # check the file validation >> >+ ErrorCode, ErrorInfo =3D File.Validate('.dec') >> >+ if ErrorCode !=3D 0: >> >+ LineNo =3D Record[-1] >> >+ EdkLogger.error('build', ErrorCode, >> >+ ExtraData=3DErrorInfo, >> >File=3Dself.MetaFile, Line=3DLineNo) >> >+ # parse this package now. we need it to get protocol/ppi/g= uid value >> >+ RetVal.add(self._Bdb[File, self._Arch, self._Target, self.= _Toolchain]) >> >+ return RetVal >> >+ >> > ## Retrieve [Components] section information >> > @property >> > def Modules(self): >> >@@ -896,7 +914,8 @@ class DscBuildData(PlatformBuildClassObject): >> > continue >> > ModuleData =3D self._Bdb[ModuleFile, self._Arch, >> >self._Target, self._Toolchain] >> > PkgSet.update(ModuleData.Packages) >> >- >> >+ if self.Packages: >> >+ PkgSet.update(self.Packages) >> > self._DecPcds, self._GuidDict =3D GetDeclaredPcd(self, >> >self._Bdb, self._Arch, self._Target, self._Toolchain, PkgSet) >> > self._GuidDict.update(GlobalData.gPlatformPcds) >> > >> >@@ -3320,6 +3339,8 @@ class DscBuildData(PlatformBuildClassObject): >> > continue >> > ModuleData =3D self._Bdb[ModuleFile, self._Arch, >> >self._Target, self._Toolchain] >> > PkgSet.update(ModuleData.Packages) >> >+ if self.Packages: >> >+ PkgSet.update(self.Packages) >> > self._DecPcds, self._GuidDict =3D GetDeclaredPcd(self, >> >self._Bdb, self._Arch, self._Target, self._Toolchain, PkgSet) >> > self._GuidDict.update(GlobalData.gPlatformPcds) >> > return self._DecPcds >> >diff --git a/BaseTools/Source/Python/Workspace/MetaFileParser.py >> >b/BaseTools/Source/Python/Workspace/MetaFileParser.py >> >index 806fc322c5..3f96ce0564 100644 >> >--- a/BaseTools/Source/Python/Workspace/MetaFileParser.py >> >+++ b/BaseTools/Source/Python/Workspace/MetaFileParser.py >> >@@ -160,6 +160,7 @@ class MetaFileParser(object): >> > self.MetaFile =3D FilePath >> > self._FileDir =3D self.MetaFile.Dir >> > self._Defines =3D {} >> >+ self._Packages =3D [] >> > self._FileLocalMacros =3D {} >> > self._SectionsMacroDict =3D defaultdict(dict) >> > >> >@@ -351,6 +352,13 @@ class MetaFileParser(object): >> > # If the section information is needed later, it should be >> >stored in database >> > self._ValueList[0] =3D self._SectionName >> > >> >+ ## [packages] section parser >> >+ @ParseMacro >> >+ def _PackageParser(self): >> >+ self._CurrentLine =3D CleanString(self._CurrentLine) >> >+ self._Packages.append(self._CurrentLine) >> >+ self._ValueList[0] =3D self._CurrentLine >> >+ >> > ## [defines] section parser >> > @ParseMacro >> > def _DefineParser(self): >> >@@ -848,6 +856,7 @@ class DscParser(MetaFileParser): >> > TAB_LIBRARIES.upper() : MODEL_EFI_LIBR= ARY_INSTANCE, >> > TAB_LIBRARY_CLASSES.upper() : MODEL_EFI_LIBR= ARY_CLASS, >> > TAB_BUILD_OPTIONS.upper() : >> >MODEL_META_DATA_BUILD_OPTION, >> >+ TAB_PACKAGES.upper() : >> MODEL_META_DATA_PACKAGE, >> > TAB_PCDS_FIXED_AT_BUILD_NULL.upper() : >> >MODEL_PCD_FIXED_AT_BUILD, >> > TAB_PCDS_PATCHABLE_IN_MODULE_NULL.upper() : >> >MODEL_PCD_PATCHABLE_IN_MODULE, >> > TAB_PCDS_FEATURE_FLAG_NULL.upper() : >> >MODEL_PCD_FEATURE_FLAG, >> >@@ -1339,6 +1348,7 @@ class DscParser(MetaFileParser): >> > MODEL_META_DATA_DEFINE : self._= _ProcessDefine, >> > MODEL_META_DATA_GLOBAL_DEFINE : >> >self.__ProcessDefine, >> > MODEL_META_DATA_INCLUDE : >> self.__ProcessDirective, >> >+ MODEL_META_DATA_PACKAGE : >> self.__ProcessPackages, >> > MODEL_META_DATA_CONDITIONAL_STATEMENT_IF : >> >self.__ProcessDirective, >> > MODEL_META_DATA_CONDITIONAL_STATEMENT_ELSE : >> >self.__ProcessDirective, >> > MODEL_META_DATA_CONDITIONAL_STATEMENT_IFDEF : >> >self.__ProcessDirective, >> >@@ -1642,6 +1652,9 @@ class DscParser(MetaFileParser): >> > self._ValueList =3D None >> > self._ContentIndex -=3D 1 >> > >> >+ def __ProcessPackages(self): >> >+ self._ValueList[0] =3D ReplaceMacro(self._ValueList[0], >> >+ self._Macros) >> >+ >> > def __ProcessSkuId(self): >> > self._ValueList =3D [ReplaceMacro(Value, self._Macros, RaiseEr= ror=3DTrue) >> > for Value in self._ValueList] @@ -1720,6 >> >+1733,7 @@ class DscParser(MetaFileParser): >> > MODEL_META_DATA_COMPONENT : _Component= Parser, >> > MODEL_META_DATA_BUILD_OPTION : _BuildOpti= onParser, >> > MODEL_UNKNOWN : MetaFilePa= rser._Skip, >> >+ MODEL_META_DATA_PACKAGE : >> >MetaFileParser._PackageParser, >> > MODEL_META_DATA_USER_EXTENSION : >> >MetaFileParser._SkipUserExtension, >> > MODEL_META_DATA_SECTION_HEADER : >> >MetaFileParser._SectionHeaderParser, >> > MODEL_META_DATA_SUBSECTION_HEADER : >> >_SubsectionHeaderParser, >> >diff --git a/BaseTools/Source/Python/Workspace/WorkspaceCommon.py >> >b/BaseTools/Source/Python/Workspace/WorkspaceCommon.py >> >index 0b11ec2d59..913e710fd9 100644 >> >--- a/BaseTools/Source/Python/Workspace/WorkspaceCommon.py >> >+++ b/BaseTools/Source/Python/Workspace/WorkspaceCommon.py >> >@@ -37,6 +37,8 @@ class OrderedListDict(OrderedDict): >> > # >> > def GetPackageList(Platform, BuildDatabase, Arch, Target, Toolchain): >> > PkgSet =3D set() >> >+ if Platform.Packages: >> >+ PkgSet.update(Platform.Packages) >> > for ModuleFile in Platform.Modules: >> > Data =3D BuildDatabase[ModuleFile, Arch, Target, Toolchain] >> > PkgSet.update(Data.Packages) >> >diff --git a/BaseTools/Source/Python/Workspace/WorkspaceDatabase.py >> >b/BaseTools/Source/Python/Workspace/WorkspaceDatabase.py >> >index ab7b4506c1..9420eaa608 100644 >> >--- a/BaseTools/Source/Python/Workspace/WorkspaceDatabase.py >> >+++ b/BaseTools/Source/Python/Workspace/WorkspaceDatabase.py >> >@@ -186,6 +186,10 @@ class WorkspaceDatabase(object): >> > for Package in LibObj.Packages: >> > if Package not in PackageList: >> > PackageList.append(Package) >> >+ for Package in Pa.Packages: >> >+ if Package in PackageList: >> >+ continue >> >+ PackageList.append(Package) >> > >> > return PackageList >> > >> >-- >> >2.14.1.windows.1