From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by mx.groups.io with SMTP id smtpd.web09.6909.1580907599906076925 for ; Wed, 05 Feb 2020 05:00:00 -0800 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: intel.com, ip: 134.134.136.24, mailfrom: bob.c.feng@intel.com) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 05 Feb 2020 04:59:58 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,405,1574150400"; d="scan'208";a="264213457" Received: from fmsmsx108.amr.corp.intel.com ([10.18.124.206]) by fmsmga002.fm.intel.com with ESMTP; 05 Feb 2020 04:59:58 -0800 Received: from shsmsx603.ccr.corp.intel.com (10.109.6.143) by FMSMSX108.amr.corp.intel.com (10.18.124.206) with Microsoft SMTP Server (TLS) id 14.3.439.0; Wed, 5 Feb 2020 04:59:57 -0800 Received: from shsmsx601.ccr.corp.intel.com (10.109.6.141) by SHSMSX603.ccr.corp.intel.com (10.109.6.143) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Wed, 5 Feb 2020 20:59:55 +0800 Received: from shsmsx601.ccr.corp.intel.com ([10.109.6.141]) by SHSMSX601.ccr.corp.intel.com ([10.109.6.141]) with mapi id 15.01.1713.004; Wed, 5 Feb 2020 20:59:55 +0800 From: "Bob Feng" To: "devel@edk2.groups.io" , "pierre.gondois@arm.com" CC: "ard.biesheuvel@linaro.org" , "Gao, Liming" , "sami.mujawar@arm.com" , "nd@arm.com" Subject: Re: [edk2-devel] [PATCH v3 1/1] BaseTools: Build ASL files before C files Thread-Topic: [edk2-devel] [PATCH v3 1/1] BaseTools: Build ASL files before C files Thread-Index: AQHV23+dJ6D9/1DHV0iHaJhFCT9DVqgMkDZA Date: Wed, 5 Feb 2020 12:59:55 +0000 Message-ID: References: <20200204172138.48052-1-pierre.gondois@arm.com> In-Reply-To: <20200204172138.48052-1-pierre.gondois@arm.com> Accept-Language: zh-CN, en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiZjQ5MWU5OTEtZDA1ZC00OWFlLThmMDktNGE5MWMzMGNiNTYyIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoibTZ6ZXlSWERzZCtLcTZKUENCQ0hnVWRZenZmMjNKa1pob3ZZZEp2RW5zcjNac0VrMjN2S2Z1MHlLVVhlc1ZtbyJ9 dlp-version: 11.2.0.6 dlp-product: dlpe-windows x-ctpclassification: CTP_NT dlp-reaction: no-action x-originating-ip: [10.239.127.36] MIME-Version: 1.0 Return-Path: bob.c.feng@intel.com Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Pierre, This patch looks good to me. Is there a Bugzilla for this patch? And will you send the patch for INF sp= ec?=20 There may be a typo in the log message: [Sources] FileName1.X FileName2.Y : FileName1.X FileName3.Z : FileName1.X FileName3.Z # here the FileName3.Z should = be FileName2.Y ? Thanks, Bob -----Original Message----- From: devel@edk2.groups.io On Behalf Of PierreGondo= is Sent: Wednesday, February 5, 2020 1:22 AM To: devel@edk2.groups.io Cc: Pierre Gondois ; ard.biesheuvel@linaro.org; Fe= ng, Bob C ; Gao, Liming ; sami.= mujawar@arm.com; nd@arm.com Subject: [edk2-devel] [PATCH v3 1/1] BaseTools: Build ASL files before C f= iles From: Pierre Gondois The dependencies for C files are satisfied by the build system. However, there are use cases where source files with different languages a= re inter-dependent. The EDKII build framework currently doesn't have option= s to specify such dependencies. E.g. It may be necessary to specify the build order between ASL files and C files. The use case being that the AML blob generated by compiling the ASL files is loaded and parsed by some C code. This patch allows to describe dependencies between files listed in the '[S= ources]' section of '.inf' files. The list of source files to build prior s= tarts with a colon (':'), e.g.: [Sources] FileName1.X FileName2.Y : FileName1.X FileName3.Z : FileName1.X FileName3.Z In the example above: * FileName1.X will be built prior to FileName2.Y. * FileName1.X and FileName2.Y will be built prior to FileName3.Z. This does not affect the file specific build options, which are pipe separ= ated, e.g.: FileName2.Y | GCC : FileName1.X The list of colon separated files must be part of the module, i.e. they mu= st be present in the [Sources] section. Their file extension must be described in the BaseTools/Conf/build_rule.te= mplate file, and generate an output file, i.e. have a non-empty '' section. Declaring a dependency in the [Sources] sections leads to the creation of = makefile rules between these files in the build. The makefile rule is betwe= en the generated files. E.g.: FileName1.X generates FileName1.objx FileName2.Y generates FileName2.objy Then FileName2.Y : FileName1.X will create the following makefile rule: FileName2.objy : FileName1.objx INF Specification updates: s3.2.1, "Common Definitions": ::=3D ":" ::=3D s2.5, "[Sources] Section": To describe a build order dependency between source files, specify the source files to build prior by listing them, colon separated. ::=3D [] [] ::=3D [FileNameDependency]* Signed-off-by: Pierre Gondois --- The changes can be seen at https://github.com/PierreARM/edk2/tree/676_buil= d_asl_first_v3 Notes: v3: - In .inf files, after the first colon starting the list of file dependencies, the list of files are space separated. In v2, a colon was required before each file. Cf the example in the commit message. [Pierre] BaseTools/Source/Python/AutoGen/BuildEngine.py | 6 ++++ BaseTools/Source/Python/AutoGen/GenMake.py | 7 +++++ BaseTools/Source/Python/AutoGen/ModuleAutoGen.py | 17 +++++++++++ BaseTools/Source/Python/Common/DataType.py | 3 +- BaseTools/Source/Python/Common/Misc.py | 3 ++ BaseTools/Source/Python/Workspace/InfBuildData.py | 32 ++++++++++++++++= ++-- BaseTools/Source/Python/Workspace/MetaFileParser.py | 18 +++++++++-- 7 files changed, 81 insertions(+), 5 deletions(-) diff --git a/BaseTools/Source/Python/AutoGen/BuildEngine.py b/BaseTools/So= urce/Python/AutoGen/BuildEngine.py index d602414ca41f37155c9c6d00eec54ea3918840c3..687617756619a5b547f18c99c4= 773842a8328ae8 100644 --- a/BaseTools/Source/Python/AutoGen/BuildEngine.py +++ b/BaseTools/Source/Python/AutoGen/BuildEngine.py @@ -2,6 +2,8 @@ # The engine for building files # # Copyright (c) 2007 - 2018, Intel Corporation. All rights reserved.
+# Copyright (c) 2020, ARM Limited. All rights reserved.
# # SPDX-License-Identifier: BSD-2-Clause-Patent # =20 @@ -53,6 +55,7 @@ class TargetDescBlock(object): self.Outputs =3D Outputs self.Commands =3D Commands self.Dependencies =3D Dependencies + self.SourceFileDependencies =3D None if self.Outputs: self.Target =3D self.Outputs[0] else: @@ -277,6 +280,9 @@ class FileBuildRule: TargetDesc.GenListFile =3D self.GenListFile TargetDesc.GenIncListFile =3D self.GenIncListFile self.BuildTargets[DstFile[0]] =3D TargetDesc + + TargetDesc.SourceFileDependencies =3D=20 + SourceFile.SourceFileDependencies + return TargetDesc =20 def _BuildCommand(self, Macros): diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py b/BaseTools/Source= /Python/AutoGen/GenMake.py index ba199c1aa73dc46856b41c13e07e3d9770081acd..9d9ac328b1f1fd781e3057d341= 16485e5ac99afd 100755 --- a/BaseTools/Source/Python/AutoGen/GenMake.py +++ b/BaseTools/Source/Python/AutoGen/GenMake.py @@ -2,6 +2,8 @@ # Create makefile for MS nmake and GNU make # # Copyright (c) 2007 - 20= 20, Intel Corporation. All rights reserved.
+# Copyright (c) 2020, ARM Limited. All rights reserved.
# # SPDX-License-Identifier: BSD-2-Clause-Patent # =20 @@ -1002,6 +1004,11 @@ cleanlib: =20 Deps =3D [] CCodeDeps =3D [] + + if T.SourceFileDependencies: + for File in T.SourceFileDependencies: + Deps.append(self.PlaceMacro(str(File),=20 + self.Macros)) + # Add force-dependencies for Dep in T.Dependencies: Deps.append(self.PlaceMacro(str(Dep), self.Macros)) d= iff --git a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py b/BaseTools/So= urce/Python/AutoGen/ModuleAutoGen.py index aad591de65f086043d55aeea5661f59c53792e7c..b1cba5178e88db0855f5d0979b= b7aa355093eade 100755 --- a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py +++ b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py @@ -2,6 +2,8 @@ # Create makefile for MS nmake and GNU make # # Copyright (c) 2019, Int= el Corporation. All rights reserved.
+# Copyright (c) 2020, ARM Limited. All rights reserved.
# # SPDX-License-Identifier: BSD-2-Clause-Patent # from __future__ import= absolute_import @@ -876,6 +878,21 @@ class ModuleAutoGen(AutoGen): if Source !=3D File: CreateDirectory(Source.Dir) =20 + # The Makefile rule created from with source file dependency = must + # have an object file as prerequisite. + # This gets the ouput format of sources. + OutPutSourceFileDependencies =3D [] + if Source.SourceFileDependencies: + for File in Source.SourceFileDependencies: + if File.Type in self.BuildRules: + DepRuleObject =3D self.BuildRules[File.Type] + DepTarget =3D DepRuleObject.Apply(File, self.Buil= dRuleOrder) + # Files not producing outputs like '.h files don'= t create a target. + if DepTarget: + + OutPutSourceFileDependencies.extend(DepTarget.Outputs) + + Source.SourceFileDependencies =3D=20 + OutPutSourceFileDependencies + if File.IsBinary and File =3D=3D Source and File in BinaryFil= eList: # Skip all files that are not binary libraries if not self.IsLibrary: diff --git a/BaseTools/Source/Python/Common/DataType.py b/BaseTools/Source= /Python/Common/DataType.py index 8d80b410892946c8b862e060b0bf5f630b409825..8f21e1060446982f866c653aae= a70cd662e307a0 100644 --- a/BaseTools/Source/Python/Common/DataType.py +++ b/BaseTools/Source/Python/Common/DataType.py @@ -2,7 +2,7 @@ # This file is used to define common static strings used by INF/DEC/DSC f= iles # # Copyright (c) 2007 - 2018, Intel Corporation. All rights reserve= d.
-# Portions copyright (c) 2011 - 2013, ARM Ltd. All rights reserved.=
+# Portions copyright (c) 2011 - 2020, ARM Ltd. All rights reserved.
# SPDX-License-Identifier: BSD-2-Clause-Patent =20 ## @@ -19,6 +19,7 @@ TAB_VALUE_SPLIT =3D '|' TAB_COMMA_SPLIT =3D ',' TAB_SPACE_SPLIT =3D ' ' TAB_SEMI_COLON_SPLIT =3D ';' +TAB_COLON_SPLIT =3D ':' TAB_SECTION_START =3D '[' TAB_SECTION_END =3D ']' TAB_OPTION_START =3D '<' diff --git a/BaseTools/Source/Python/Common/Misc.py b/BaseTools/Source/Pyt= hon/Common/Misc.py index da5fb380f0354d6e885aa716d2c9b2fead249d63..fb7a8f62087cfe7e0e007a9864= 832dc1eefc3068 100755 --- a/BaseTools/Source/Python/Common/Misc.py +++ b/BaseTools/Source/Python/Common/Misc.py @@ -2,6 +2,8 @@ # Common routines used by all tools # # Copyright (c) 2007 - 2019, Intel Corporation. All rights reserved.
+# Copyright (c) 2020, ARM Limited. All rights reserved.
# # SPDX-License-Identifier: BSD-2-Clause-Patent # =20 @@ -1438,6 +1440,7 @@ class PathClass(object): Arch=3D'COMMON', ToolChainFamily=3D'', Target=3D'', TagN= ame=3D'', ToolCode=3D''): self.Arch =3D Arch self.File =3D str(File) + self.SourceFileDependencies =3D [] if os.path.isabs(self.File): self.Root =3D '' self.AlterRoot =3D '' diff --git a/BaseTools/Source/Python/Workspace/InfBuildData.py b/BaseTools= /Source/Python/Workspace/InfBuildData.py index 7675b0ea00ebd6a5fc3e823c965e32066f66f650..3c9aa366994bac89cfc72e8f11= 75a62c17c47fc3 100644 --- a/BaseTools/Source/Python/Workspace/InfBuildData.py +++ b/BaseTools/Source/Python/Workspace/InfBuildData.py @@ -3,6 +3,8 @@ # # Copyright (c) 2008 - 2018, Intel Corporation. All rights reserved.
= # (C) Copyright 2016 Hewlett Packard Enterprise Development LP
+# Copyright (c) 2020, ARM Limited. All rights reserved.
# # SPDX-License-Identifier: BSD-2-Clause-Patent # =20 @@ -536,12 +538,19 @@ class InfBuildData(ModuleBuildClassObject): return [] =20 RetVal =3D [] + PendingSourceFileDependencies =3D dict() RecordList =3D self._RawData[MODEL_EFI_SOURCE_FILE, self._Arch, s= elf._Platform] Macros =3D self._Macros for Record in RecordList: LineNo =3D Record[-1] - ToolChainFamily =3D Record[1] - TagName =3D Record[2] + + BuildOptions =3D ['', ''] + SplittedValueList =3D GetSplitValueList(Record[1], TAB_VALUE_= SPLIT, 1) + BuildOptions[0:len(SplittedValueList)] =3D SplittedValueList + SourceFileDependencies =3D=20 + list(set(GetSplitValueList(Record[2], TAB_SPACE_SPLIT))) + + ToolChainFamily =3D BuildOptions[0] + TagName =3D BuildOptions[1] ToolCode =3D Record[3] =20 File =3D PathClass(NormPath(Record[0], Macros), self._ModuleD= ir, '', @@ -552,9 +561,28 @@ class InfBuildData(ModuleBuildClassObject): EdkLogger.error('build', ErrorCode, ExtraData=3DErrorInfo= , File=3Dself.MetaFile, Line=3DLineNo) =20 RetVal.append(File) + + # If there are source file dependencies, resolve them after a= ll + # the PathClass instances of the source files have been creat= ed. + if SourceFileDependencies[0] !=3D '': + PendingSourceFileDependencies[File] =3D=20 + SourceFileDependencies + # add any previously found dependency files to the source list if self._DependencyFileList: RetVal.extend(self._DependencyFileList) + + # Resolve the dependencies between the PathClass instances. + for SourceFile, SourceFileDepList in PendingSourceFileDependencie= s.items(): + for SourceFileDepFile in SourceFileDepList: + Found =3D False + for Val in RetVal: + if SourceFileDepFile =3D=3D Val.File: + SourceFile.SourceFileDependencies.append(Val) + Found =3D True + break + if not Found: + EdkLogger.error("build", FILE_NOT_FOUND,=20 + ExtraData=3DSourceFileDepFile) + return RetVal =20 ## Retrieve library classes employed by this module diff --git a/Base= Tools/Source/Python/Workspace/MetaFileParser.py b/BaseTools/Source/Python/W= orkspace/MetaFileParser.py index a3b6edbd15ee5bf79cef0ac1fc5e53db30356c91..04a656b8ba1e55addc5f37ec8e= e2b0439cef0ef1 100644 --- a/BaseTools/Source/Python/Workspace/MetaFileParser.py +++ b/BaseTools/Source/Python/Workspace/MetaFileParser.py @@ -3,6 +3,8 @@ # # Copyright (c) 2008 - 2018, Intel Corporation. All rights reserved.
= # (C) Copyright 2015-2018 Hewlett Packard Enterprise Development LP
+# Copyright (c) 2020, ARM Limited. All rights reserved.
# # SPDX-License-Identifier: BSD-2-Clause-Patent # =20 @@ -735,8 +737,20 @@ class InfParser(MetaFileParser): # @ParseMacro def _SourceFileParser(self): - TokenList =3D GetSplitValueList(self._CurrentLine, TAB_VALUE_SPLI= T) - self._ValueList[0:len(TokenList)] =3D TokenList + SourceFileDependencySplit =3D GetSplitValueList(self._CurrentLine= , TAB_COLON_SPLIT, 1) + BuildOptionSplit =3D=20 + GetSplitValueList(SourceFileDependencySplit[0], TAB_VALUE_SPLIT, 1) + + # Get the filename. + self._ValueList[0] =3D BuildOptionSplit[0] + + # Get the build options. + if len(BuildOptionSplit) > 1: + self._ValueList[1] =3D BuildOptionSplit[1] + + # Get the dependencies. + if len(SourceFileDependencySplit) > 1: + self._ValueList[2] =3D SourceFileDependencySplit[1] + Macros =3D self._Macros # For Acpi tables, remove macro like ' TABLE_NAME=3DSata1' if 'COMPONENT_TYPE' in Macros: -- 'Guid(CE165669-3EF3-493F-B85D-6190EE5B9759)'