From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by mx.groups.io with SMTP id smtpd.web10.6352.1580824309146584702 for ; Tue, 04 Feb 2020 05:51:49 -0800 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: intel.com, ip: 134.134.136.24, mailfrom: liming.gao@intel.com) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 04 Feb 2020 05:51:48 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,402,1574150400"; d="scan'208";a="231373475" Received: from fmsmsx105.amr.corp.intel.com ([10.18.124.203]) by orsmga003.jf.intel.com with ESMTP; 04 Feb 2020 05:51:48 -0800 Received: from shsmsx602.ccr.corp.intel.com (10.109.6.142) by FMSMSX105.amr.corp.intel.com (10.18.124.203) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 4 Feb 2020 05:51:47 -0800 Received: from shsmsx606.ccr.corp.intel.com (10.109.6.216) by SHSMSX602.ccr.corp.intel.com (10.109.6.142) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Tue, 4 Feb 2020 21:51:43 +0800 Received: from shsmsx606.ccr.corp.intel.com ([10.109.6.216]) by SHSMSX606.ccr.corp.intel.com ([10.109.6.216]) with mapi id 15.01.1713.004; Tue, 4 Feb 2020 21:51:43 +0800 From: "Liming Gao" To: "devel@edk2.groups.io" , "pierre.gondois@arm.com" CC: "ard.biesheuvel@linaro.org" , "Feng, Bob C" , Sami Mujawar , nd Subject: Re: [edk2-devel] [PATCH v2 1/1] BaseTools: Build ASL files before C files Thread-Topic: [edk2-devel] [PATCH v2 1/1] BaseTools: Build ASL files before C files Thread-Index: AQHVywr0LRu3BntM506huQJeJToEq6f2d+MAgBMfdgA= Date: Tue, 4 Feb 2020 13:51:43 +0000 Message-ID: References: <20200114184615.9832-1-pierre.gondois@arm.com> In-Reply-To: Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-version: 11.2.0.6 dlp-product: dlpe-windows dlp-reaction: no-action x-originating-ip: [10.239.127.36] MIME-Version: 1.0 Return-Path: liming.gao@intel.com Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Pierre: I see you use ':' to describe the dependency between the different sourc= e file. This is makefile syntax for the dependency. If one file depends on = more than one files, other files will list after this file with the separat= or space ' '. So, I think the example should be like below. > [Sources] > FileName1.X > FileName2.Y : FileName1.X > FileName3.Z : FileName1.X FileName3.Y Thanks Liming > -----Original Message----- > From: devel@edk2.groups.io On Behalf Of PierreGon= dois > Sent: Thursday, January 23, 2020 1:33 AM > To: Pierre Gondois ; devel@edk2.groups.io > Cc: ard.biesheuvel@linaro.org; Feng, Bob C ; Gao, = Liming ; Sami Mujawar > ; nd > Subject: Re: [edk2-devel] [PATCH v2 1/1] BaseTools: Build ASL files befo= re C files >=20 > Hello everyone, > Do you have any input on the patch? >=20 > Regards, > Pierre >=20 > -----Original Message----- > From: PierreGondois > Sent: 14 January 2020 18:46 > To: devel@edk2.groups.io > Cc: Pierre Gondois ; ard.biesheuvel@linaro.org; = bob.c.feng@intel.com; liming.gao@intel.com; Sami > Mujawar ; nd > Subject: [PATCH v2 1/1] BaseTools: Build ASL files before C files >=20 > From: Pierre Gondois >=20 > The dependencies for C files are satisfied by the build system. > However, there are use cases where source files with different languages= are inter-dependent. The EDKII build framework currently > doesn't have options to specify such dependencies. > E.g. It may be necessary to specify the build order between > ASL files and C files. The use case being that the AML > blob generated by compiling the ASL files is loaded and > parsed by some C code. >=20 > This patch allows to describe dependencies between files listed in the '= [Sources]' section of '.inf' files. The list of source files to build > prior are colon separated (':'), e.g.: > [Sources] > FileName1.X > FileName2.Y : FileName1.X > FileName3.Z : FileName1.X : FileName3.Z >=20 > In the example above: > * FileName1.X will be built prior to FileName2.Y. > * FileName1.X and FileName2.Y will be built prior to > FileName3.Z. >=20 > This does not affect the file specific build options, which are pipe sep= arated, e.g.: > FileName2.Y | GCC : FileName1.X >=20 > The list of colon separated files must be part of the module, i.e. they = must be present in the [Sources] section. >=20 > Their file extension must be described in the BaseTools/Conf/build_rule.= template file, and generate an output file, i.e. have a non- > empty '' section. >=20 > Declaring a dependency in the [Sources] sections leads to the creation o= f makefile rules between these files in the build. The makefile > rule is between the generated files. > E.g.: > FileName1.X generates FileName1.objx > FileName2.Y generates FileName2.objy > Then > FileName2.Y : FileName1.X > will create the following makefile rule: > FileName2.objy : FileName1.objx >=20 > INF Specification updates: > s3.2.1, "Common Definitions": > ::=3D ":" > ::=3D >=20 > s2.5, "[Sources] Section": > To describe a build order dependency between source > files, specify the source files to build prior by > listing them, colon separated. > ::=3D [] []* > ::=3D [FileNameDependency] >=20 > Signed-off-by: Pierre Gondois > --- >=20 > The changes can be seen at https://github.com/PierreARM/edk2/tree/676_bu= ild_asl_first_v2 >=20 > Notes: > v2: > - Enable file dependencies in .if files [Pierre] >=20 > BaseTools/Source/Python/AutoGen/BuildEngine.py | 6 ++++ > BaseTools/Source/Python/AutoGen/GenMake.py | 7 +++++ > BaseTools/Source/Python/AutoGen/ModuleAutoGen.py | 17 +++++++++++ > BaseTools/Source/Python/Common/DataType.py | 3 +- > BaseTools/Source/Python/Common/Misc.py | 3 ++ > BaseTools/Source/Python/Workspace/InfBuildData.py | 32 ++++++++++++++= ++++-- > BaseTools/Source/Python/Workspace/MetaFileParser.py | 18 +++++++++-- > 7 files changed, 81 insertions(+), 5 deletions(-) >=20 > diff --git a/BaseTools/Source/Python/AutoGen/BuildEngine.py b/BaseTools/= Source/Python/AutoGen/BuildEngine.py > index d602414ca41f37155c9c6d00eec54ea3918840c3..687617756619a5b547f18c99= c4773842a8328ae8 100644 > --- a/BaseTools/Source/Python/AutoGen/BuildEngine.py > +++ b/BaseTools/Source/Python/AutoGen/BuildEngine.py > @@ -2,6 +2,8 @@ > # The engine for building files > # > # Copyright (c) 2007 - 2018, Intel Corporation. All rights reserved. > +# Copyright (c) 2020, ARM Limited. All rights reserved.
# > # SPDX-License-Identifier: BSD-2-Clause-Patent # >=20 > @@ -53,6 +55,7 @@ class TargetDescBlock(object): > self.Outputs =3D Outputs > self.Commands =3D Commands > self.Dependencies =3D Dependencies > + self.SourceFileDependencies =3D None > if self.Outputs: > self.Target =3D self.Outputs[0] > else: > @@ -277,6 +280,9 @@ class FileBuildRule: > TargetDesc.GenListFile =3D self.GenListFile > TargetDesc.GenIncListFile =3D self.GenIncListFile > self.BuildTargets[DstFile[0]] =3D TargetDesc > + > + TargetDesc.SourceFileDependencies =3D > + SourceFile.SourceFileDependencies > + > return TargetDesc >=20 > def _BuildCommand(self, Macros): > diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py b/BaseTools/Sour= ce/Python/AutoGen/GenMake.py > index fe94f9a4c232bb599a59563444c3985700c78ec6..5d932b66fb68baa06570a2eb= 421d717f0dab8187 100755 > --- a/BaseTools/Source/Python/AutoGen/GenMake.py > +++ b/BaseTools/Source/Python/AutoGen/GenMake.py > @@ -2,6 +2,8 @@ > # Create makefile for MS nmake and GNU make # # Copyright (c) 2007 - = 2018, Intel Corporation. All rights reserved.
> +# Copyright (c) 2020, ARM Limited. All rights reserved.
# > # SPDX-License-Identifier: BSD-2-Clause-Patent # >=20 > @@ -997,6 +999,11 @@ cleanlib: >=20 > Deps =3D [] > CCodeDeps =3D [] > + > + if T.SourceFileDependencies: > + for File in T.SourceFileDependencies: > + Deps.append(self.PlaceMacro(str(File), > + self.Macros)) > + > # Add force-dependencies > for Dep in T.Dependencies: > Deps.append(self.PlaceMacro(str(Dep), self.Macros))= diff --git a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py > b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py > index aad591de65f086043d55aeea5661f59c53792e7c..b1cba5178e88db0855f5d097= 9bb7aa355093eade 100755 > --- a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py > +++ b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py > @@ -2,6 +2,8 @@ > # Create makefile for MS nmake and GNU make # # Copyright (c) 2019, I= ntel Corporation. All rights reserved.
> +# Copyright (c) 2020, ARM Limited. All rights reserved.
# > # SPDX-License-Identifier: BSD-2-Clause-Patent # from __future__ impo= rt absolute_import @@ -876,6 +878,21 @@ class > ModuleAutoGen(AutoGen): > if Source !=3D File: > CreateDirectory(Source.Dir) >=20 > + # The Makefile rule created from with source file dependenc= y must > + # have an object file as prerequisite. > + # This gets the ouput format of sources. > + OutPutSourceFileDependencies =3D [] > + if Source.SourceFileDependencies: > + for File in Source.SourceFileDependencies: > + if File.Type in self.BuildRules: > + DepRuleObject =3D self.BuildRules[File.Type] > + DepTarget =3D DepRuleObject.Apply(File, self.Bu= ildRuleOrder) > + # Files not producing outputs like '.h files do= n't create a target. > + if DepTarget: > + > + OutPutSourceFileDependencies.extend(DepTarget.Outputs) > + > + Source.SourceFileDependencies =3D > + OutPutSourceFileDependencies > + > if File.IsBinary and File =3D=3D Source and File in BinaryF= ileList: > # Skip all files that are not binary libraries > if not self.IsLibrary: > diff --git a/BaseTools/Source/Python/Common/DataType.py b/BaseTools/Sour= ce/Python/Common/DataType.py > index 8d80b410892946c8b862e060b0bf5f630b409825..8f21e1060446982f866c653a= aea70cd662e307a0 100644 > --- a/BaseTools/Source/Python/Common/DataType.py > +++ b/BaseTools/Source/Python/Common/DataType.py > @@ -2,7 +2,7 @@ > # This file is used to define common static strings used by INF/DEC/DSC= files # # Copyright (c) 2007 - 2018, Intel Corporation. All > rights reserved.
-# Portions copyright (c) 2011 - 2013, ARM Ltd. All= rights reserved.
> +# Portions copyright (c) 2011 - 2020, ARM Ltd. All rights reserved.
> # SPDX-License-Identifier: BSD-2-Clause-Patent >=20 > ## > @@ -19,6 +19,7 @@ TAB_VALUE_SPLIT =3D '|' > TAB_COMMA_SPLIT =3D ',' > TAB_SPACE_SPLIT =3D ' ' > TAB_SEMI_COLON_SPLIT =3D ';' > +TAB_COLON_SPLIT =3D ':' > TAB_SECTION_START =3D '[' > TAB_SECTION_END =3D ']' > TAB_OPTION_START =3D '<' > diff --git a/BaseTools/Source/Python/Common/Misc.py b/BaseTools/Source/P= ython/Common/Misc.py > index da5fb380f0354d6e885aa716d2c9b2fead249d63..fb7a8f62087cfe7e0e007a98= 64832dc1eefc3068 100755 > --- a/BaseTools/Source/Python/Common/Misc.py > +++ b/BaseTools/Source/Python/Common/Misc.py > @@ -2,6 +2,8 @@ > # Common routines used by all tools > # > # Copyright (c) 2007 - 2019, Intel Corporation. All rights reserved. > +# Copyright (c) 2020, ARM Limited. All rights reserved.
# > # SPDX-License-Identifier: BSD-2-Clause-Patent # >=20 > @@ -1438,6 +1440,7 @@ class PathClass(object): > Arch=3D'COMMON', ToolChainFamily=3D'', Target=3D'', Ta= gName=3D'', ToolCode=3D''): > self.Arch =3D Arch > self.File =3D str(File) > + self.SourceFileDependencies =3D [] > if os.path.isabs(self.File): > self.Root =3D '' > self.AlterRoot =3D '' > diff --git a/BaseTools/Source/Python/Workspace/InfBuildData.py b/BaseToo= ls/Source/Python/Workspace/InfBuildData.py > index 7675b0ea00ebd6a5fc3e823c965e32066f66f650..02e5c578b426e66b343df937= a56ccea319b9f646 100644 > --- a/BaseTools/Source/Python/Workspace/InfBuildData.py > +++ b/BaseTools/Source/Python/Workspace/InfBuildData.py > @@ -3,6 +3,8 @@ > # > # Copyright (c) 2008 - 2018, Intel Corporation. All rights reserved. # (C) Copyright 2016 Hewlett Packard Enterprise Development > LP
> +# Copyright (c) 2020, ARM Limited. All rights reserved.
# > # SPDX-License-Identifier: BSD-2-Clause-Patent # >=20 > @@ -536,12 +538,19 @@ class InfBuildData(ModuleBuildClassObject): > return [] >=20 > RetVal =3D [] > + PendingSourceFileDependencies =3D dict() > RecordList =3D self._RawData[MODEL_EFI_SOURCE_FILE, self._Arch,= self._Platform] > Macros =3D self._Macros > for Record in RecordList: > LineNo =3D Record[-1] > - ToolChainFamily =3D Record[1] > - TagName =3D Record[2] > + > + BuildOptions =3D ['', ''] > + SplittedValueList =3D GetSplitValueList(Record[1], TAB_VALU= E_SPLIT, 1) > + BuildOptions[0:len(SplittedValueList)] =3D SplittedValueLis= t > + SourceFileDependencies =3D > + list(set(GetSplitValueList(Record[2], TAB_COLON_SPLIT))) > + > + ToolChainFamily =3D BuildOptions[0] > + TagName =3D BuildOptions[1] > ToolCode =3D Record[3] >=20 > File =3D PathClass(NormPath(Record[0], Macros), self._Modul= eDir, '', @@ -552,9 +561,28 @@ class > InfBuildData(ModuleBuildClassObject): > EdkLogger.error('build', ErrorCode, ExtraData=3DErrorIn= fo, File=3Dself.MetaFile, Line=3DLineNo) >=20 > RetVal.append(File) > + > + # If there are source file dependencies, resolve them after= all > + # the PathClass instances of the source files have been cre= ated. > + if SourceFileDependencies[0] !=3D '': > + PendingSourceFileDependencies[File] =3D > + SourceFileDependencies > + > # add any previously found dependency files to the source list > if self._DependencyFileList: > RetVal.extend(self._DependencyFileList) > + > + # Resolve the dependencies between the PathClass instances. > + for SourceFile, SourceFileDepList in PendingSourceFileDependenc= ies.items(): > + for SourceFileDepFile in SourceFileDepList: > + Found =3D False > + for Val in RetVal: > + if SourceFileDepFile =3D=3D Val.File: > + SourceFile.SourceFileDependencies.append(Val) > + Found =3D True > + break > + if not Found: > + EdkLogger.error("build", FILE_NOT_FOUND, > + ExtraData=3DSourceFileDepFile) > + > return RetVal >=20 > ## Retrieve library classes employed by this module diff --git a/Ba= seTools/Source/Python/Workspace/MetaFileParser.py > b/BaseTools/Source/Python/Workspace/MetaFileParser.py > index a3b6edbd15ee5bf79cef0ac1fc5e53db30356c91..04a656b8ba1e55addc5f37ec= 8ee2b0439cef0ef1 100644 > --- a/BaseTools/Source/Python/Workspace/MetaFileParser.py > +++ b/BaseTools/Source/Python/Workspace/MetaFileParser.py > @@ -3,6 +3,8 @@ > # > # Copyright (c) 2008 - 2018, Intel Corporation. All rights reserved. # (C) Copyright 2015-2018 Hewlett Packard Enterprise > Development LP
> +# Copyright (c) 2020, ARM Limited. All rights reserved.
# > # SPDX-License-Identifier: BSD-2-Clause-Patent # >=20 > @@ -735,8 +737,20 @@ class InfParser(MetaFileParser): > # > @ParseMacro > def _SourceFileParser(self): > - TokenList =3D GetSplitValueList(self._CurrentLine, TAB_VALUE_SP= LIT) > - self._ValueList[0:len(TokenList)] =3D TokenList > + SourceFileDependencySplit =3D GetSplitValueList(self._CurrentLi= ne, TAB_COLON_SPLIT, 1) > + BuildOptionSplit =3D > + GetSplitValueList(SourceFileDependencySplit[0], TAB_VALUE_SPLIT, 1) > + > + # Get the filename. > + self._ValueList[0] =3D BuildOptionSplit[0] > + > + # Get the build options. > + if len(BuildOptionSplit) > 1: > + self._ValueList[1] =3D BuildOptionSplit[1] > + > + # Get the dependencies. > + if len(SourceFileDependencySplit) > 1: > + self._ValueList[2] =3D SourceFileDependencySplit[1] > + > Macros =3D self._Macros > # For Acpi tables, remove macro like ' TABLE_NAME=3DSata1' > if 'COMPONENT_TYPE' in Macros: > -- > 'Guid(CE165669-3EF3-493F-B85D-6190EE5B9759)' >=20 >=20 >=20 >=20