From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=2607:f8b0:4864:20::d35; helo=mail-io1-xd35.google.com; envelope-from=ard.biesheuvel@linaro.org; receiver=edk2-devel@lists.01.org Received: from mail-io1-xd35.google.com (mail-io1-xd35.google.com [IPv6:2607:f8b0:4864:20::d35]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 564DD21A07A80 for ; Sun, 13 Jan 2019 22:57:43 -0800 (PST) Received: by mail-io1-xd35.google.com with SMTP id v10so16803751ios.13 for ; Sun, 13 Jan 2019 22:57:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=RlkwAbUEJueL45CZRwWedqZkj1dqk5y4TpHxk4DCgRg=; b=BTIn6sb4BICR9RcNlMNiEnYzI/UeN06KmcjIKAtse/6Fs83WsAfnZO633B4dTNCswk YDnEWz1tYGNP5Fi6NNDVXpWY1ABfJwtAtZl8Q6dlX6JOwmONM5Bw4+4vfY4BDZ+Gset0 s8+qxBBtRhb+lXevCqzNKosKkHM9OUxxnr7cA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=RlkwAbUEJueL45CZRwWedqZkj1dqk5y4TpHxk4DCgRg=; b=Qsg5gqH+u/NtEcZJaaQdUcIwz1kDD85bGPxTCcd0EwBdIqzYFOS4AJDXivQPfefKCi VvB/U2MGWcACgFzFFL6wo91SBCHniaQHJX9RD4lenU2UlLBm//9YcE7Cth+aymbEiNYi gDO94inn5mpx4Yh0/Ij2RjEMHPjbEeYyK118AI/+qybo2/ZQZQndPuYkowXRjeoNYUWJ jK9bNy1pzLmHnwsZpli7SNtHBIelqSnfaEf8zt4ga3/6DrEBUIkA0VGOs1INw9aDpHrf fx40Rk+ji7vYa+Oj2lQBgrH60YQZ2a4OGB78YwTPoagyHi+MTQ+ac1l3nqGZPpq7r0ir cbRA== X-Gm-Message-State: AJcUukf09ShAsgX4UYCWS6TwwlzTsC3tmFsUSIIW4LVwQGeWRlkHBXaJ BbKHCk4IHjCfOuo37wgvyDz45uUIq5MBMMIy3/vPDw== X-Google-Smtp-Source: ALg8bN783rbNegtzkpLX2Bv/UT7eo7ycXDHREGyyZ1SlViAeqOpzCwPb/qAzyrBUEl+xRyUElwNz7j1svI38+iWENRg= X-Received: by 2002:a5d:8410:: with SMTP id i16mr15582927ion.173.1547449062511; Sun, 13 Jan 2019 22:57:42 -0800 (PST) MIME-Version: 1.0 References: <20190111023851.9172-1-bob.c.feng@intel.com> <4A89E2EF3DFEDB4C8BFDE51014F606A14E3B0FD1@SHSMSX152.ccr.corp.intel.com> In-Reply-To: <4A89E2EF3DFEDB4C8BFDE51014F606A14E3B0FD1@SHSMSX152.ccr.corp.intel.com> From: Ard Biesheuvel Date: Mon, 14 Jan 2019 07:57:31 +0100 Message-ID: To: "Gao, Liming" Cc: "Feng, Bob C" , "edk2-devel@lists.01.org" , "Carsey, Jaben" Subject: Re: [Patch] BaseTools: Enable component override functionality X-BeenThere: edk2-devel@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: EDK II Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Jan 2019 06:57:44 -0000 Content-Type: text/plain; charset="UTF-8" On Mon, 14 Jan 2019 at 04:00, Gao, Liming wrote: > > Reviewed-by: Liming Gao > This patch breaks the build on AArch64/GGC: (Python 2.7.13 on linux2) Traceback (most recent call last): File " line 2403, in Main MyBuild.Launch() File " line 2137, in Launch self._MultiThreadBuildPlatform() File " line 1917, in _MultiThreadBuildPlatform self.Progress File " line 239, in __init__ self._InitWorker(Workspace, MetaFile, Target, Toolchain, Arch, *args, **kwargs) File " line 403, in _InitWorker PlatformPcds = Platform.Pcds File " line 1194, in Pcds self._Pcds = self.UpdateStructuredPcds(MODEL_PCD_TYPE_LIST, self._Pcds) File " line 1486, in UpdateStructuredPcds for Pcd in self.DecPcds: File " line 3183, in DecPcds self._DecPcds, self._GuidDict = GetDeclaredPcd(self, self._Bdb, self._Arch, self._Target, self._Toolchain, PkgSet) File " line 62, in GetDeclaredPcd PkgList = GetPackageList(Platform, BuildDatabase, Arch, Target, Toolchain) File " line 44, in GetPackageList for ModuleFile in Platform.Modules: File " line 728, in Modules self.OverrideDuplicateModule() File " line 712, in OverrideDuplicateModule Macros["EDK_SOURCE"] = GlobalData.gEcpSource AttributeError: 'module' object has no attribute 'gEcpSource' > >-----Original Message----- > >From: Feng, Bob C > >Sent: Friday, January 11, 2019 10:39 AM > >To: edk2-devel@lists.01.org > >Cc: Feng, Bob C ; Gao, Liming > >; Carsey, Jaben > >Subject: [Patch] BaseTools: Enable component override functionality > > > >https://bugzilla.tianocore.org/show_bug.cgi?id=1449 > >This patch enable build tools to recognize that > >when two given files have the same GUID, file path and ARCH in Dsc, > >The later one's definition will be used. > > > >Contributed-under: TianoCore Contribution Agreement 1.1 > >Signed-off-by: Bob Feng > >Cc: Liming Gao > >Cc: Carsey Jaben > >--- > > .../Source/Python/Workspace/DscBuildData.py | 24 ++++++++++++------- > > .../Source/Python/Workspace/MetaFileParser.py | 5 ++++ > > .../Source/Python/Workspace/MetaFileTable.py | 7 ++++-- > > 3 files changed, 25 insertions(+), 11 deletions(-) > > > >diff --git a/BaseTools/Source/Python/Workspace/DscBuildData.py > >b/BaseTools/Source/Python/Workspace/DscBuildData.py > >index 7e82e8e934..f9805f58f5 100644 > >--- a/BaseTools/Source/Python/Workspace/DscBuildData.py > >+++ b/BaseTools/Source/Python/Workspace/DscBuildData.py > >@@ -704,36 +704,44 @@ class DscBuildData(PlatformBuildClassObject): > > if TAB_DEFAULT_STORES_DEFAULT not in self.DefaultStores: > > self.DefaultStores[TAB_DEFAULT_STORES_DEFAULT] = (0, > >TAB_DEFAULT_STORES_DEFAULT) > > GlobalData.gDefaultStores = sorted(self.DefaultStores.keys()) > > return self.DefaultStores > > > >+ def OverrideDuplicateModule(self): > >+ RecordList = self._RawData[MODEL_META_DATA_COMPONENT, > >self._Arch] > >+ Macros = self._Macros > >+ Macros["EDK_SOURCE"] = GlobalData.gEcpSource > >+ Components = {} > >+ for Record in RecordList: > >+ ModuleId = Record[6] > >+ file_guid = self._RawData[MODEL_META_DATA_HEADER, self._Arch, > >None, ModuleId] > >+ file_guid_str = file_guid[0][2] if file_guid else "NULL" > >+ ModuleFile = PathClass(NormPath(Record[0], Macros), > >GlobalData.gWorkspace, Arch=self._Arch) > >+ if self._Arch != TAB_ARCH_COMMON and > >(file_guid_str,str(ModuleFile)) in Components: > >+ > >self._RawData.DisableOverrideComponent(Components[(file_guid_str,str(M > >oduleFile))]) > >+ Components[(file_guid_str,str(ModuleFile))] = ModuleId > >+ self._RawData._PostProcessed = False > > ## Retrieve [Components] section information > > @property > > def Modules(self): > > if self._Modules is not None: > > return self._Modules > >- > >+ self.OverrideDuplicateModule() > > self._Modules = OrderedDict() > > RecordList = self._RawData[MODEL_META_DATA_COMPONENT, > >self._Arch] > > Macros = self._Macros > > Macros["EDK_SOURCE"] = GlobalData.gEcpSource > > for Record in RecordList: > >- DuplicatedFile = False > >- > > ModuleFile = PathClass(NormPath(Record[0], Macros), > >GlobalData.gWorkspace, Arch=self._Arch) > > ModuleId = Record[6] > > LineNo = Record[7] > > > > # check the file validation > > ErrorCode, ErrorInfo = ModuleFile.Validate('.inf') > > if ErrorCode != 0: > > EdkLogger.error('build', ErrorCode, File=self.MetaFile, Line=LineNo, > > ExtraData=ErrorInfo) > >- # Check duplication > >- # If arch is COMMON, no duplicate module is checked since all modules > >in all component sections are selected > >- if self._Arch != TAB_ARCH_COMMON and ModuleFile in self._Modules: > >- DuplicatedFile = True > > > > Module = ModuleBuildClassObject() > > Module.MetaFile = ModuleFile > > > > # get module private library instance > >@@ -792,12 +800,10 @@ class DscBuildData(PlatformBuildClassObject): > > else: > > OptionString = Module.BuildOptions[ToolChainFamily, ToolChain] > > Module.BuildOptions[ToolChainFamily, ToolChain] = OptionString + > >" " + Option > > > > RecordList = self._RawData[MODEL_META_DATA_HEADER, self._Arch, > >None, ModuleId] > >- if DuplicatedFile and not RecordList: > >- EdkLogger.error('build', FILE_DUPLICATED, File=self.MetaFile, > >ExtraData=str(ModuleFile), Line=LineNo) > > if RecordList: > > if len(RecordList) != 1: > > EdkLogger.error('build', OPTION_UNKNOWN, 'Only FILE_GUID can > >be listed in section.', > > File=self.MetaFile, ExtraData=str(ModuleFile), Line=LineNo) > > ModuleFile = ProcessDuplicatedInf(ModuleFile, RecordList[0][2], > >GlobalData.gWorkspace) > >diff --git a/BaseTools/Source/Python/Workspace/MetaFileParser.py > >b/BaseTools/Source/Python/Workspace/MetaFileParser.py > >index 032220813b..a52e9229df 100644 > >--- a/BaseTools/Source/Python/Workspace/MetaFileParser.py > >+++ b/BaseTools/Source/Python/Workspace/MetaFileParser.py > >@@ -1711,10 +1711,15 @@ class DscParser(MetaFileParser): > > > > def __ProcessBuildOption(self): > > self._ValueList = [ReplaceMacro(Value, self._Macros, RaiseError=False) > > for Value in self._ValueList] > > > >+ def DisableOverrideComponent(self,module_id): > >+ for ori_id in self._IdMapping: > >+ if self._IdMapping[ori_id] == module_id: > >+ self._RawTable.DisableComponent(ori_id) > >+ > > _SectionParser = { > > MODEL_META_DATA_HEADER : _DefineParser, > > MODEL_EFI_SKU_ID : _SkuIdParser, > > MODEL_EFI_DEFAULT_STORES : _DefaultStoresParser, > > MODEL_EFI_LIBRARY_INSTANCE : _LibraryInstanceParser, > >diff --git a/BaseTools/Source/Python/Workspace/MetaFileTable.py > >b/BaseTools/Source/Python/Workspace/MetaFileTable.py > >index 004e9494c3..823a87e057 100644 > >--- a/BaseTools/Source/Python/Workspace/MetaFileTable.py > >+++ b/BaseTools/Source/Python/Workspace/MetaFileTable.py > >@@ -74,11 +74,11 @@ class MetaFileTable(): > > > > def SetEndFlag(self): > > self.CurrentContent.append(self._DUMMY_) > > > > def GetAll(self): > >- return [item for item in self.CurrentContent if item[0] >= 0 ] > >+ return [item for item in self.CurrentContent if item[0] >= 0 and item[- > >1]>=0] > > > > ## Python class representation of table storing module data > > class ModuleTable(MetaFileTable): > > _COLUMN_ = ''' > > ID REAL PRIMARY KEY, > >@@ -371,11 +371,10 @@ class PlatformTable(MetaFileTable): > > # > > def Query(self, Model, Scope1=None, Scope2=None, > >BelongsToItem=None, FromItem=None): > > > > QueryTab = self.CurrentContent > > result = [item for item in QueryTab if item[1] == Model and item[-1]>0 ] > >- > > if Scope1 is not None and Scope1 != TAB_ARCH_COMMON: > > Sc1 = set(['COMMON']) > > Sc1.add(Scope1) > > result = [item for item in result if item[5] in Sc1] > > Sc2 = set( ['COMMON','DEFAULT']) > >@@ -395,10 +394,14 @@ class PlatformTable(MetaFileTable): > > result = [item for item in result if item[9] == FromItem] > > > > result = [ [r[2],r[3],r[4],r[5],r[6],r[7],r[0],r[9]] for r in result ] > > return result > > > >+ def DisableComponent(self,comp_id): > >+ for item in self.CurrentContent: > >+ if item[0] == comp_id or item[8] == comp_id: > >+ item[-1] = -1 > > > > ## Factory class to produce different storage for different type of meta-file > > class MetaFileStorage(object): > > _FILE_TABLE_ = { > > MODEL_FILE_INF : ModuleTable, > >-- > >2.19.1.windows.1 > > _______________________________________________ > edk2-devel mailing list > edk2-devel@lists.01.org > https://lists.01.org/mailman/listinfo/edk2-devel