* [PATCH v2 1/1] BaseTools: Build ASL files before C files
@ 2020-01-14 18:46 PierreGondois
2020-01-22 17:33 ` PierreGondois
0 siblings, 1 reply; 3+ messages in thread
From: PierreGondois @ 2020-01-14 18:46 UTC (permalink / raw)
To: devel
Cc: Pierre Gondois, ard.biesheuvel, bob.c.feng, liming.gao,
sami.mujawar, nd
From: Pierre Gondois <pierre.gondois@arm.com>
The dependencies for C files are satisfied by the build system.
However, there are use cases where source files with different
languages are inter-dependent. The EDKII build framework
currently doesn't have options to specify such dependencies.
E.g. It may be necessary to specify the build order between
ASL files and C files. The use case being that the AML
blob generated by compiling the ASL files is loaded and
parsed by some C code.
This patch allows to describe dependencies between files listed
in the '[Sources]' section of '.inf' files. The list of source
files to build prior are colon separated (':'), e.g.:
[Sources]
FileName1.X
FileName2.Y : FileName1.X
FileName3.Z : FileName1.X : FileName3.Z
In the example above:
* FileName1.X will be built prior to FileName2.Y.
* FileName1.X and FileName2.Y will be built prior to
FileName3.Z.
This does not affect the file specific build options,
which are pipe separated, e.g.:
FileName2.Y | GCC : FileName1.X
The list of colon separated files must be part of the module,
i.e. they must be present in the [Sources] section.
Their file extension must be described in the
BaseTools/Conf/build_rule.template file, and generate
an output file, i.e. have a non-empty '<OutputFile>' section.
Declaring a dependency in the [Sources] sections leads to
the creation of makefile rules between these files in the
build. The makefile rule is between the generated files.
E.g.:
FileName1.X generates FileName1.objx
FileName2.Y generates FileName2.objy
Then
FileName2.Y : FileName1.X
will create the following makefile rule:
FileName2.objy : FileName1.objx
INF Specification updates:
s3.2.1, "Common Definitions":
<SrcDependencySeperator> ::= ":"
<DepS> ::= <TS> <SrcDependencySeperator> <TS>
s2.5, "[Sources] Section":
To describe a build order dependency between source
files, specify the source files to build prior by
listing them, colon separated.
<opt3> ::= <FS> [<FeatureFlagExpress>] [<opt4>]*
<opt4> ::= <DepS> [FileNameDependency]
Signed-off-by: Pierre Gondois <pierre.gondois@arm.com>
---
The changes can be seen at https://github.com/PierreARM/edk2/tree/676_build_asl_first_v2
Notes:
v2:
- Enable file dependencies in .if files [Pierre]
BaseTools/Source/Python/AutoGen/BuildEngine.py | 6 ++++
BaseTools/Source/Python/AutoGen/GenMake.py | 7 +++++
BaseTools/Source/Python/AutoGen/ModuleAutoGen.py | 17 +++++++++++
BaseTools/Source/Python/Common/DataType.py | 3 +-
BaseTools/Source/Python/Common/Misc.py | 3 ++
BaseTools/Source/Python/Workspace/InfBuildData.py | 32 ++++++++++++++++++--
BaseTools/Source/Python/Workspace/MetaFileParser.py | 18 +++++++++--
7 files changed, 81 insertions(+), 5 deletions(-)
diff --git a/BaseTools/Source/Python/AutoGen/BuildEngine.py b/BaseTools/Source/Python/AutoGen/BuildEngine.py
index d602414ca41f37155c9c6d00eec54ea3918840c3..687617756619a5b547f18c99c4773842a8328ae8 100644
--- a/BaseTools/Source/Python/AutoGen/BuildEngine.py
+++ b/BaseTools/Source/Python/AutoGen/BuildEngine.py
@@ -2,6 +2,8 @@
# The engine for building files
#
# Copyright (c) 2007 - 2018, Intel Corporation. All rights reserved.<BR>
+# Copyright (c) 2020, ARM Limited. All rights reserved.<BR>
+#
# SPDX-License-Identifier: BSD-2-Clause-Patent
#
@@ -53,6 +55,7 @@ class TargetDescBlock(object):
self.Outputs = Outputs
self.Commands = Commands
self.Dependencies = Dependencies
+ self.SourceFileDependencies = None
if self.Outputs:
self.Target = self.Outputs[0]
else:
@@ -277,6 +280,9 @@ class FileBuildRule:
TargetDesc.GenListFile = self.GenListFile
TargetDesc.GenIncListFile = self.GenIncListFile
self.BuildTargets[DstFile[0]] = TargetDesc
+
+ TargetDesc.SourceFileDependencies = SourceFile.SourceFileDependencies
+
return TargetDesc
def _BuildCommand(self, Macros):
diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py b/BaseTools/Source/Python/AutoGen/GenMake.py
index fe94f9a4c232bb599a59563444c3985700c78ec6..5d932b66fb68baa06570a2eb421d717f0dab8187 100755
--- a/BaseTools/Source/Python/AutoGen/GenMake.py
+++ b/BaseTools/Source/Python/AutoGen/GenMake.py
@@ -2,6 +2,8 @@
# Create makefile for MS nmake and GNU make
#
# Copyright (c) 2007 - 2018, Intel Corporation. All rights reserved.<BR>
+# Copyright (c) 2020, ARM Limited. All rights reserved.<BR>
+#
# SPDX-License-Identifier: BSD-2-Clause-Patent
#
@@ -997,6 +999,11 @@ cleanlib:
Deps = []
CCodeDeps = []
+
+ if T.SourceFileDependencies:
+ for File in T.SourceFileDependencies:
+ Deps.append(self.PlaceMacro(str(File), self.Macros))
+
# Add force-dependencies
for Dep in T.Dependencies:
Deps.append(self.PlaceMacro(str(Dep), self.Macros))
diff --git a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
index aad591de65f086043d55aeea5661f59c53792e7c..b1cba5178e88db0855f5d0979bb7aa355093eade 100755
--- a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
@@ -2,6 +2,8 @@
# Create makefile for MS nmake and GNU make
#
# Copyright (c) 2019, Intel Corporation. All rights reserved.<BR>
+# Copyright (c) 2020, ARM Limited. All rights reserved.<BR>
+#
# SPDX-License-Identifier: BSD-2-Clause-Patent
#
from __future__ import absolute_import
@@ -876,6 +878,21 @@ class ModuleAutoGen(AutoGen):
if Source != File:
CreateDirectory(Source.Dir)
+ # The Makefile rule created from with source file dependency must
+ # have an object file as prerequisite.
+ # This gets the ouput format of sources.
+ OutPutSourceFileDependencies = []
+ if Source.SourceFileDependencies:
+ for File in Source.SourceFileDependencies:
+ if File.Type in self.BuildRules:
+ DepRuleObject = self.BuildRules[File.Type]
+ DepTarget = DepRuleObject.Apply(File, self.BuildRuleOrder)
+ # Files not producing outputs like '.h files don't create a target.
+ if DepTarget:
+ OutPutSourceFileDependencies.extend(DepTarget.Outputs)
+
+ Source.SourceFileDependencies = OutPutSourceFileDependencies
+
if File.IsBinary and File == Source and File in BinaryFileList:
# Skip all files that are not binary libraries
if not self.IsLibrary:
diff --git a/BaseTools/Source/Python/Common/DataType.py b/BaseTools/Source/Python/Common/DataType.py
index 8d80b410892946c8b862e060b0bf5f630b409825..8f21e1060446982f866c653aaea70cd662e307a0 100644
--- a/BaseTools/Source/Python/Common/DataType.py
+++ b/BaseTools/Source/Python/Common/DataType.py
@@ -2,7 +2,7 @@
# This file is used to define common static strings used by INF/DEC/DSC files
#
# Copyright (c) 2007 - 2018, Intel Corporation. All rights reserved.<BR>
-# Portions copyright (c) 2011 - 2013, ARM Ltd. All rights reserved.<BR>
+# Portions copyright (c) 2011 - 2020, ARM Ltd. All rights reserved.<BR>
# SPDX-License-Identifier: BSD-2-Clause-Patent
##
@@ -19,6 +19,7 @@ TAB_VALUE_SPLIT = '|'
TAB_COMMA_SPLIT = ','
TAB_SPACE_SPLIT = ' '
TAB_SEMI_COLON_SPLIT = ';'
+TAB_COLON_SPLIT = ':'
TAB_SECTION_START = '['
TAB_SECTION_END = ']'
TAB_OPTION_START = '<'
diff --git a/BaseTools/Source/Python/Common/Misc.py b/BaseTools/Source/Python/Common/Misc.py
index da5fb380f0354d6e885aa716d2c9b2fead249d63..fb7a8f62087cfe7e0e007a9864832dc1eefc3068 100755
--- a/BaseTools/Source/Python/Common/Misc.py
+++ b/BaseTools/Source/Python/Common/Misc.py
@@ -2,6 +2,8 @@
# Common routines used by all tools
#
# Copyright (c) 2007 - 2019, Intel Corporation. All rights reserved.<BR>
+# Copyright (c) 2020, ARM Limited. All rights reserved.<BR>
+#
# SPDX-License-Identifier: BSD-2-Clause-Patent
#
@@ -1438,6 +1440,7 @@ class PathClass(object):
Arch='COMMON', ToolChainFamily='', Target='', TagName='', ToolCode=''):
self.Arch = Arch
self.File = str(File)
+ self.SourceFileDependencies = []
if os.path.isabs(self.File):
self.Root = ''
self.AlterRoot = ''
diff --git a/BaseTools/Source/Python/Workspace/InfBuildData.py b/BaseTools/Source/Python/Workspace/InfBuildData.py
index 7675b0ea00ebd6a5fc3e823c965e32066f66f650..02e5c578b426e66b343df937a56ccea319b9f646 100644
--- a/BaseTools/Source/Python/Workspace/InfBuildData.py
+++ b/BaseTools/Source/Python/Workspace/InfBuildData.py
@@ -3,6 +3,8 @@
#
# Copyright (c) 2008 - 2018, Intel Corporation. All rights reserved.<BR>
# (C) Copyright 2016 Hewlett Packard Enterprise Development LP<BR>
+# Copyright (c) 2020, ARM Limited. All rights reserved.<BR>
+#
# SPDX-License-Identifier: BSD-2-Clause-Patent
#
@@ -536,12 +538,19 @@ class InfBuildData(ModuleBuildClassObject):
return []
RetVal = []
+ PendingSourceFileDependencies = dict()
RecordList = self._RawData[MODEL_EFI_SOURCE_FILE, self._Arch, self._Platform]
Macros = self._Macros
for Record in RecordList:
LineNo = Record[-1]
- ToolChainFamily = Record[1]
- TagName = Record[2]
+
+ BuildOptions = ['', '']
+ SplittedValueList = GetSplitValueList(Record[1], TAB_VALUE_SPLIT, 1)
+ BuildOptions[0:len(SplittedValueList)] = SplittedValueList
+ SourceFileDependencies = list(set(GetSplitValueList(Record[2], TAB_COLON_SPLIT)))
+
+ ToolChainFamily = BuildOptions[0]
+ TagName = BuildOptions[1]
ToolCode = Record[3]
File = PathClass(NormPath(Record[0], Macros), self._ModuleDir, '',
@@ -552,9 +561,28 @@ class InfBuildData(ModuleBuildClassObject):
EdkLogger.error('build', ErrorCode, ExtraData=ErrorInfo, File=self.MetaFile, Line=LineNo)
RetVal.append(File)
+
+ # If there are source file dependencies, resolve them after all
+ # the PathClass instances of the source files have been created.
+ if SourceFileDependencies[0] != '':
+ PendingSourceFileDependencies[File] = SourceFileDependencies
+
# add any previously found dependency files to the source list
if self._DependencyFileList:
RetVal.extend(self._DependencyFileList)
+
+ # Resolve the dependencies between the PathClass instances.
+ for SourceFile, SourceFileDepList in PendingSourceFileDependencies.items():
+ for SourceFileDepFile in SourceFileDepList:
+ Found = False
+ for Val in RetVal:
+ if SourceFileDepFile == Val.File:
+ SourceFile.SourceFileDependencies.append(Val)
+ Found = True
+ break
+ if not Found:
+ EdkLogger.error("build", FILE_NOT_FOUND, ExtraData=SourceFileDepFile)
+
return RetVal
## Retrieve library classes employed by this module
diff --git a/BaseTools/Source/Python/Workspace/MetaFileParser.py b/BaseTools/Source/Python/Workspace/MetaFileParser.py
index a3b6edbd15ee5bf79cef0ac1fc5e53db30356c91..04a656b8ba1e55addc5f37ec8ee2b0439cef0ef1 100644
--- a/BaseTools/Source/Python/Workspace/MetaFileParser.py
+++ b/BaseTools/Source/Python/Workspace/MetaFileParser.py
@@ -3,6 +3,8 @@
#
# Copyright (c) 2008 - 2018, Intel Corporation. All rights reserved.<BR>
# (C) Copyright 2015-2018 Hewlett Packard Enterprise Development LP<BR>
+# Copyright (c) 2020, ARM Limited. All rights reserved.<BR>
+#
# SPDX-License-Identifier: BSD-2-Clause-Patent
#
@@ -735,8 +737,20 @@ class InfParser(MetaFileParser):
#
@ParseMacro
def _SourceFileParser(self):
- TokenList = GetSplitValueList(self._CurrentLine, TAB_VALUE_SPLIT)
- self._ValueList[0:len(TokenList)] = TokenList
+ SourceFileDependencySplit = GetSplitValueList(self._CurrentLine, TAB_COLON_SPLIT, 1)
+ BuildOptionSplit = GetSplitValueList(SourceFileDependencySplit[0], TAB_VALUE_SPLIT, 1)
+
+ # Get the filename.
+ self._ValueList[0] = BuildOptionSplit[0]
+
+ # Get the build options.
+ if len(BuildOptionSplit) > 1:
+ self._ValueList[1] = BuildOptionSplit[1]
+
+ # Get the dependencies.
+ if len(SourceFileDependencySplit) > 1:
+ self._ValueList[2] = SourceFileDependencySplit[1]
+
Macros = self._Macros
# For Acpi tables, remove macro like ' TABLE_NAME=Sata1'
if 'COMPONENT_TYPE' in Macros:
--
'Guid(CE165669-3EF3-493F-B85D-6190EE5B9759)'
^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [PATCH v2 1/1] BaseTools: Build ASL files before C files
2020-01-14 18:46 [PATCH v2 1/1] BaseTools: Build ASL files before C files PierreGondois
@ 2020-01-22 17:33 ` PierreGondois
2020-02-04 13:51 ` [edk2-devel] " Liming Gao
0 siblings, 1 reply; 3+ messages in thread
From: PierreGondois @ 2020-01-22 17:33 UTC (permalink / raw)
To: Pierre Gondois, devel@edk2.groups.io
Cc: ard.biesheuvel@linaro.org, bob.c.feng@intel.com,
liming.gao@intel.com, Sami Mujawar, nd
Hello everyone,
Do you have any input on the patch?
Regards,
Pierre
-----Original Message-----
From: PierreGondois <pierre.gondois@arm.com>
Sent: 14 January 2020 18:46
To: devel@edk2.groups.io
Cc: Pierre Gondois <Pierre.Gondois@arm.com>; ard.biesheuvel@linaro.org; bob.c.feng@intel.com; liming.gao@intel.com; Sami Mujawar <Sami.Mujawar@arm.com>; nd <nd@arm.com>
Subject: [PATCH v2 1/1] BaseTools: Build ASL files before C files
From: Pierre Gondois <pierre.gondois@arm.com>
The dependencies for C files are satisfied by the build system.
However, there are use cases where source files with different languages are inter-dependent. The EDKII build framework currently doesn't have options to specify such dependencies.
E.g. It may be necessary to specify the build order between
ASL files and C files. The use case being that the AML
blob generated by compiling the ASL files is loaded and
parsed by some C code.
This patch allows to describe dependencies between files listed in the '[Sources]' section of '.inf' files. The list of source files to build prior are colon separated (':'), e.g.:
[Sources]
FileName1.X
FileName2.Y : FileName1.X
FileName3.Z : FileName1.X : FileName3.Z
In the example above:
* FileName1.X will be built prior to FileName2.Y.
* FileName1.X and FileName2.Y will be built prior to
FileName3.Z.
This does not affect the file specific build options, which are pipe separated, e.g.:
FileName2.Y | GCC : FileName1.X
The list of colon separated files must be part of the module, i.e. they must be present in the [Sources] section.
Their file extension must be described in the BaseTools/Conf/build_rule.template file, and generate an output file, i.e. have a non-empty '<OutputFile>' section.
Declaring a dependency in the [Sources] sections leads to the creation of makefile rules between these files in the build. The makefile rule is between the generated files.
E.g.:
FileName1.X generates FileName1.objx
FileName2.Y generates FileName2.objy
Then
FileName2.Y : FileName1.X
will create the following makefile rule:
FileName2.objy : FileName1.objx
INF Specification updates:
s3.2.1, "Common Definitions":
<SrcDependencySeperator> ::= ":"
<DepS> ::= <TS> <SrcDependencySeperator> <TS>
s2.5, "[Sources] Section":
To describe a build order dependency between source
files, specify the source files to build prior by
listing them, colon separated.
<opt3> ::= <FS> [<FeatureFlagExpress>] [<opt4>]*
<opt4> ::= <DepS> [FileNameDependency]
Signed-off-by: Pierre Gondois <pierre.gondois@arm.com>
---
The changes can be seen at https://github.com/PierreARM/edk2/tree/676_build_asl_first_v2
Notes:
v2:
- Enable file dependencies in .if files [Pierre]
BaseTools/Source/Python/AutoGen/BuildEngine.py | 6 ++++
BaseTools/Source/Python/AutoGen/GenMake.py | 7 +++++
BaseTools/Source/Python/AutoGen/ModuleAutoGen.py | 17 +++++++++++
BaseTools/Source/Python/Common/DataType.py | 3 +-
BaseTools/Source/Python/Common/Misc.py | 3 ++
BaseTools/Source/Python/Workspace/InfBuildData.py | 32 ++++++++++++++++++--
BaseTools/Source/Python/Workspace/MetaFileParser.py | 18 +++++++++--
7 files changed, 81 insertions(+), 5 deletions(-)
diff --git a/BaseTools/Source/Python/AutoGen/BuildEngine.py b/BaseTools/Source/Python/AutoGen/BuildEngine.py
index d602414ca41f37155c9c6d00eec54ea3918840c3..687617756619a5b547f18c99c4773842a8328ae8 100644
--- a/BaseTools/Source/Python/AutoGen/BuildEngine.py
+++ b/BaseTools/Source/Python/AutoGen/BuildEngine.py
@@ -2,6 +2,8 @@
# The engine for building files
#
# Copyright (c) 2007 - 2018, Intel Corporation. All rights reserved.<BR>
+# Copyright (c) 2020, ARM Limited. All rights reserved.<BR> #
# SPDX-License-Identifier: BSD-2-Clause-Patent #
@@ -53,6 +55,7 @@ class TargetDescBlock(object):
self.Outputs = Outputs
self.Commands = Commands
self.Dependencies = Dependencies
+ self.SourceFileDependencies = None
if self.Outputs:
self.Target = self.Outputs[0]
else:
@@ -277,6 +280,9 @@ class FileBuildRule:
TargetDesc.GenListFile = self.GenListFile
TargetDesc.GenIncListFile = self.GenIncListFile
self.BuildTargets[DstFile[0]] = TargetDesc
+
+ TargetDesc.SourceFileDependencies =
+ SourceFile.SourceFileDependencies
+
return TargetDesc
def _BuildCommand(self, Macros):
diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py b/BaseTools/Source/Python/AutoGen/GenMake.py
index fe94f9a4c232bb599a59563444c3985700c78ec6..5d932b66fb68baa06570a2eb421d717f0dab8187 100755
--- a/BaseTools/Source/Python/AutoGen/GenMake.py
+++ b/BaseTools/Source/Python/AutoGen/GenMake.py
@@ -2,6 +2,8 @@
# Create makefile for MS nmake and GNU make # # Copyright (c) 2007 - 2018, Intel Corporation. All rights reserved.<BR>
+# Copyright (c) 2020, ARM Limited. All rights reserved.<BR> #
# SPDX-License-Identifier: BSD-2-Clause-Patent #
@@ -997,6 +999,11 @@ cleanlib:
Deps = []
CCodeDeps = []
+
+ if T.SourceFileDependencies:
+ for File in T.SourceFileDependencies:
+ Deps.append(self.PlaceMacro(str(File),
+ self.Macros))
+
# Add force-dependencies
for Dep in T.Dependencies:
Deps.append(self.PlaceMacro(str(Dep), self.Macros)) diff --git a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
index aad591de65f086043d55aeea5661f59c53792e7c..b1cba5178e88db0855f5d0979bb7aa355093eade 100755
--- a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
@@ -2,6 +2,8 @@
# Create makefile for MS nmake and GNU make # # Copyright (c) 2019, Intel Corporation. All rights reserved.<BR>
+# Copyright (c) 2020, ARM Limited. All rights reserved.<BR> #
# SPDX-License-Identifier: BSD-2-Clause-Patent # from __future__ import absolute_import @@ -876,6 +878,21 @@ class ModuleAutoGen(AutoGen):
if Source != File:
CreateDirectory(Source.Dir)
+ # The Makefile rule created from with source file dependency must
+ # have an object file as prerequisite.
+ # This gets the ouput format of sources.
+ OutPutSourceFileDependencies = []
+ if Source.SourceFileDependencies:
+ for File in Source.SourceFileDependencies:
+ if File.Type in self.BuildRules:
+ DepRuleObject = self.BuildRules[File.Type]
+ DepTarget = DepRuleObject.Apply(File, self.BuildRuleOrder)
+ # Files not producing outputs like '.h files don't create a target.
+ if DepTarget:
+
+ OutPutSourceFileDependencies.extend(DepTarget.Outputs)
+
+ Source.SourceFileDependencies =
+ OutPutSourceFileDependencies
+
if File.IsBinary and File == Source and File in BinaryFileList:
# Skip all files that are not binary libraries
if not self.IsLibrary:
diff --git a/BaseTools/Source/Python/Common/DataType.py b/BaseTools/Source/Python/Common/DataType.py
index 8d80b410892946c8b862e060b0bf5f630b409825..8f21e1060446982f866c653aaea70cd662e307a0 100644
--- a/BaseTools/Source/Python/Common/DataType.py
+++ b/BaseTools/Source/Python/Common/DataType.py
@@ -2,7 +2,7 @@
# This file is used to define common static strings used by INF/DEC/DSC files # # Copyright (c) 2007 - 2018, Intel Corporation. All rights reserved.<BR> -# Portions copyright (c) 2011 - 2013, ARM Ltd. All rights reserved.<BR>
+# Portions copyright (c) 2011 - 2020, ARM Ltd. All rights reserved.<BR>
# SPDX-License-Identifier: BSD-2-Clause-Patent
##
@@ -19,6 +19,7 @@ TAB_VALUE_SPLIT = '|'
TAB_COMMA_SPLIT = ','
TAB_SPACE_SPLIT = ' '
TAB_SEMI_COLON_SPLIT = ';'
+TAB_COLON_SPLIT = ':'
TAB_SECTION_START = '['
TAB_SECTION_END = ']'
TAB_OPTION_START = '<'
diff --git a/BaseTools/Source/Python/Common/Misc.py b/BaseTools/Source/Python/Common/Misc.py
index da5fb380f0354d6e885aa716d2c9b2fead249d63..fb7a8f62087cfe7e0e007a9864832dc1eefc3068 100755
--- a/BaseTools/Source/Python/Common/Misc.py
+++ b/BaseTools/Source/Python/Common/Misc.py
@@ -2,6 +2,8 @@
# Common routines used by all tools
#
# Copyright (c) 2007 - 2019, Intel Corporation. All rights reserved.<BR>
+# Copyright (c) 2020, ARM Limited. All rights reserved.<BR> #
# SPDX-License-Identifier: BSD-2-Clause-Patent #
@@ -1438,6 +1440,7 @@ class PathClass(object):
Arch='COMMON', ToolChainFamily='', Target='', TagName='', ToolCode=''):
self.Arch = Arch
self.File = str(File)
+ self.SourceFileDependencies = []
if os.path.isabs(self.File):
self.Root = ''
self.AlterRoot = ''
diff --git a/BaseTools/Source/Python/Workspace/InfBuildData.py b/BaseTools/Source/Python/Workspace/InfBuildData.py
index 7675b0ea00ebd6a5fc3e823c965e32066f66f650..02e5c578b426e66b343df937a56ccea319b9f646 100644
--- a/BaseTools/Source/Python/Workspace/InfBuildData.py
+++ b/BaseTools/Source/Python/Workspace/InfBuildData.py
@@ -3,6 +3,8 @@
#
# Copyright (c) 2008 - 2018, Intel Corporation. All rights reserved.<BR> # (C) Copyright 2016 Hewlett Packard Enterprise Development LP<BR>
+# Copyright (c) 2020, ARM Limited. All rights reserved.<BR> #
# SPDX-License-Identifier: BSD-2-Clause-Patent #
@@ -536,12 +538,19 @@ class InfBuildData(ModuleBuildClassObject):
return []
RetVal = []
+ PendingSourceFileDependencies = dict()
RecordList = self._RawData[MODEL_EFI_SOURCE_FILE, self._Arch, self._Platform]
Macros = self._Macros
for Record in RecordList:
LineNo = Record[-1]
- ToolChainFamily = Record[1]
- TagName = Record[2]
+
+ BuildOptions = ['', '']
+ SplittedValueList = GetSplitValueList(Record[1], TAB_VALUE_SPLIT, 1)
+ BuildOptions[0:len(SplittedValueList)] = SplittedValueList
+ SourceFileDependencies =
+ list(set(GetSplitValueList(Record[2], TAB_COLON_SPLIT)))
+
+ ToolChainFamily = BuildOptions[0]
+ TagName = BuildOptions[1]
ToolCode = Record[3]
File = PathClass(NormPath(Record[0], Macros), self._ModuleDir, '', @@ -552,9 +561,28 @@ class InfBuildData(ModuleBuildClassObject):
EdkLogger.error('build', ErrorCode, ExtraData=ErrorInfo, File=self.MetaFile, Line=LineNo)
RetVal.append(File)
+
+ # If there are source file dependencies, resolve them after all
+ # the PathClass instances of the source files have been created.
+ if SourceFileDependencies[0] != '':
+ PendingSourceFileDependencies[File] =
+ SourceFileDependencies
+
# add any previously found dependency files to the source list
if self._DependencyFileList:
RetVal.extend(self._DependencyFileList)
+
+ # Resolve the dependencies between the PathClass instances.
+ for SourceFile, SourceFileDepList in PendingSourceFileDependencies.items():
+ for SourceFileDepFile in SourceFileDepList:
+ Found = False
+ for Val in RetVal:
+ if SourceFileDepFile == Val.File:
+ SourceFile.SourceFileDependencies.append(Val)
+ Found = True
+ break
+ if not Found:
+ EdkLogger.error("build", FILE_NOT_FOUND,
+ ExtraData=SourceFileDepFile)
+
return RetVal
## Retrieve library classes employed by this module diff --git a/BaseTools/Source/Python/Workspace/MetaFileParser.py b/BaseTools/Source/Python/Workspace/MetaFileParser.py
index a3b6edbd15ee5bf79cef0ac1fc5e53db30356c91..04a656b8ba1e55addc5f37ec8ee2b0439cef0ef1 100644
--- a/BaseTools/Source/Python/Workspace/MetaFileParser.py
+++ b/BaseTools/Source/Python/Workspace/MetaFileParser.py
@@ -3,6 +3,8 @@
#
# Copyright (c) 2008 - 2018, Intel Corporation. All rights reserved.<BR> # (C) Copyright 2015-2018 Hewlett Packard Enterprise Development LP<BR>
+# Copyright (c) 2020, ARM Limited. All rights reserved.<BR> #
# SPDX-License-Identifier: BSD-2-Clause-Patent #
@@ -735,8 +737,20 @@ class InfParser(MetaFileParser):
#
@ParseMacro
def _SourceFileParser(self):
- TokenList = GetSplitValueList(self._CurrentLine, TAB_VALUE_SPLIT)
- self._ValueList[0:len(TokenList)] = TokenList
+ SourceFileDependencySplit = GetSplitValueList(self._CurrentLine, TAB_COLON_SPLIT, 1)
+ BuildOptionSplit =
+ GetSplitValueList(SourceFileDependencySplit[0], TAB_VALUE_SPLIT, 1)
+
+ # Get the filename.
+ self._ValueList[0] = BuildOptionSplit[0]
+
+ # Get the build options.
+ if len(BuildOptionSplit) > 1:
+ self._ValueList[1] = BuildOptionSplit[1]
+
+ # Get the dependencies.
+ if len(SourceFileDependencySplit) > 1:
+ self._ValueList[2] = SourceFileDependencySplit[1]
+
Macros = self._Macros
# For Acpi tables, remove macro like ' TABLE_NAME=Sata1'
if 'COMPONENT_TYPE' in Macros:
--
'Guid(CE165669-3EF3-493F-B85D-6190EE5B9759)'
^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [edk2-devel] [PATCH v2 1/1] BaseTools: Build ASL files before C files
2020-01-22 17:33 ` PierreGondois
@ 2020-02-04 13:51 ` Liming Gao
0 siblings, 0 replies; 3+ messages in thread
From: Liming Gao @ 2020-02-04 13:51 UTC (permalink / raw)
To: devel@edk2.groups.io, pierre.gondois@arm.com
Cc: ard.biesheuvel@linaro.org, Feng, Bob C, Sami Mujawar, nd
Pierre:
I see you use ':' to describe the dependency between the different source file. This is makefile syntax for the dependency. If one file depends on more than one files, other files will list after this file with the separator space ' '. So, I think the example should be like below.
> [Sources]
> FileName1.X
> FileName2.Y : FileName1.X
> FileName3.Z : FileName1.X FileName3.Y
Thanks
Liming
> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of PierreGondois
> Sent: Thursday, January 23, 2020 1:33 AM
> To: Pierre Gondois <Pierre.Gondois@arm.com>; devel@edk2.groups.io
> Cc: ard.biesheuvel@linaro.org; Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming <liming.gao@intel.com>; Sami Mujawar
> <Sami.Mujawar@arm.com>; nd <nd@arm.com>
> Subject: Re: [edk2-devel] [PATCH v2 1/1] BaseTools: Build ASL files before C files
>
> Hello everyone,
> Do you have any input on the patch?
>
> Regards,
> Pierre
>
> -----Original Message-----
> From: PierreGondois <pierre.gondois@arm.com>
> Sent: 14 January 2020 18:46
> To: devel@edk2.groups.io
> Cc: Pierre Gondois <Pierre.Gondois@arm.com>; ard.biesheuvel@linaro.org; bob.c.feng@intel.com; liming.gao@intel.com; Sami
> Mujawar <Sami.Mujawar@arm.com>; nd <nd@arm.com>
> Subject: [PATCH v2 1/1] BaseTools: Build ASL files before C files
>
> From: Pierre Gondois <pierre.gondois@arm.com>
>
> The dependencies for C files are satisfied by the build system.
> However, there are use cases where source files with different languages are inter-dependent. The EDKII build framework currently
> doesn't have options to specify such dependencies.
> E.g. It may be necessary to specify the build order between
> ASL files and C files. The use case being that the AML
> blob generated by compiling the ASL files is loaded and
> parsed by some C code.
>
> This patch allows to describe dependencies between files listed in the '[Sources]' section of '.inf' files. The list of source files to build
> prior are colon separated (':'), e.g.:
> [Sources]
> FileName1.X
> FileName2.Y : FileName1.X
> FileName3.Z : FileName1.X : FileName3.Z
>
> In the example above:
> * FileName1.X will be built prior to FileName2.Y.
> * FileName1.X and FileName2.Y will be built prior to
> FileName3.Z.
>
> This does not affect the file specific build options, which are pipe separated, e.g.:
> FileName2.Y | GCC : FileName1.X
>
> The list of colon separated files must be part of the module, i.e. they must be present in the [Sources] section.
>
> Their file extension must be described in the BaseTools/Conf/build_rule.template file, and generate an output file, i.e. have a non-
> empty '<OutputFile>' section.
>
> Declaring a dependency in the [Sources] sections leads to the creation of makefile rules between these files in the build. The makefile
> rule is between the generated files.
> E.g.:
> FileName1.X generates FileName1.objx
> FileName2.Y generates FileName2.objy
> Then
> FileName2.Y : FileName1.X
> will create the following makefile rule:
> FileName2.objy : FileName1.objx
>
> INF Specification updates:
> s3.2.1, "Common Definitions":
> <SrcDependencySeperator> ::= ":"
> <DepS> ::= <TS> <SrcDependencySeperator> <TS>
>
> s2.5, "[Sources] Section":
> To describe a build order dependency between source
> files, specify the source files to build prior by
> listing them, colon separated.
> <opt3> ::= <FS> [<FeatureFlagExpress>] [<opt4>]*
> <opt4> ::= <DepS> [FileNameDependency]
>
> Signed-off-by: Pierre Gondois <pierre.gondois@arm.com>
> ---
>
> The changes can be seen at https://github.com/PierreARM/edk2/tree/676_build_asl_first_v2
>
> Notes:
> v2:
> - Enable file dependencies in .if files [Pierre]
>
> BaseTools/Source/Python/AutoGen/BuildEngine.py | 6 ++++
> BaseTools/Source/Python/AutoGen/GenMake.py | 7 +++++
> BaseTools/Source/Python/AutoGen/ModuleAutoGen.py | 17 +++++++++++
> BaseTools/Source/Python/Common/DataType.py | 3 +-
> BaseTools/Source/Python/Common/Misc.py | 3 ++
> BaseTools/Source/Python/Workspace/InfBuildData.py | 32 ++++++++++++++++++--
> BaseTools/Source/Python/Workspace/MetaFileParser.py | 18 +++++++++--
> 7 files changed, 81 insertions(+), 5 deletions(-)
>
> diff --git a/BaseTools/Source/Python/AutoGen/BuildEngine.py b/BaseTools/Source/Python/AutoGen/BuildEngine.py
> index d602414ca41f37155c9c6d00eec54ea3918840c3..687617756619a5b547f18c99c4773842a8328ae8 100644
> --- a/BaseTools/Source/Python/AutoGen/BuildEngine.py
> +++ b/BaseTools/Source/Python/AutoGen/BuildEngine.py
> @@ -2,6 +2,8 @@
> # The engine for building files
> #
> # Copyright (c) 2007 - 2018, Intel Corporation. All rights reserved.<BR>
> +# Copyright (c) 2020, ARM Limited. All rights reserved.<BR> #
> # SPDX-License-Identifier: BSD-2-Clause-Patent #
>
> @@ -53,6 +55,7 @@ class TargetDescBlock(object):
> self.Outputs = Outputs
> self.Commands = Commands
> self.Dependencies = Dependencies
> + self.SourceFileDependencies = None
> if self.Outputs:
> self.Target = self.Outputs[0]
> else:
> @@ -277,6 +280,9 @@ class FileBuildRule:
> TargetDesc.GenListFile = self.GenListFile
> TargetDesc.GenIncListFile = self.GenIncListFile
> self.BuildTargets[DstFile[0]] = TargetDesc
> +
> + TargetDesc.SourceFileDependencies =
> + SourceFile.SourceFileDependencies
> +
> return TargetDesc
>
> def _BuildCommand(self, Macros):
> diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py b/BaseTools/Source/Python/AutoGen/GenMake.py
> index fe94f9a4c232bb599a59563444c3985700c78ec6..5d932b66fb68baa06570a2eb421d717f0dab8187 100755
> --- a/BaseTools/Source/Python/AutoGen/GenMake.py
> +++ b/BaseTools/Source/Python/AutoGen/GenMake.py
> @@ -2,6 +2,8 @@
> # Create makefile for MS nmake and GNU make # # Copyright (c) 2007 - 2018, Intel Corporation. All rights reserved.<BR>
> +# Copyright (c) 2020, ARM Limited. All rights reserved.<BR> #
> # SPDX-License-Identifier: BSD-2-Clause-Patent #
>
> @@ -997,6 +999,11 @@ cleanlib:
>
> Deps = []
> CCodeDeps = []
> +
> + if T.SourceFileDependencies:
> + for File in T.SourceFileDependencies:
> + Deps.append(self.PlaceMacro(str(File),
> + self.Macros))
> +
> # Add force-dependencies
> for Dep in T.Dependencies:
> Deps.append(self.PlaceMacro(str(Dep), self.Macros)) diff --git a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
> b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
> index aad591de65f086043d55aeea5661f59c53792e7c..b1cba5178e88db0855f5d0979bb7aa355093eade 100755
> --- a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
> +++ b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
> @@ -2,6 +2,8 @@
> # Create makefile for MS nmake and GNU make # # Copyright (c) 2019, Intel Corporation. All rights reserved.<BR>
> +# Copyright (c) 2020, ARM Limited. All rights reserved.<BR> #
> # SPDX-License-Identifier: BSD-2-Clause-Patent # from __future__ import absolute_import @@ -876,6 +878,21 @@ class
> ModuleAutoGen(AutoGen):
> if Source != File:
> CreateDirectory(Source.Dir)
>
> + # The Makefile rule created from with source file dependency must
> + # have an object file as prerequisite.
> + # This gets the ouput format of sources.
> + OutPutSourceFileDependencies = []
> + if Source.SourceFileDependencies:
> + for File in Source.SourceFileDependencies:
> + if File.Type in self.BuildRules:
> + DepRuleObject = self.BuildRules[File.Type]
> + DepTarget = DepRuleObject.Apply(File, self.BuildRuleOrder)
> + # Files not producing outputs like '.h files don't create a target.
> + if DepTarget:
> +
> + OutPutSourceFileDependencies.extend(DepTarget.Outputs)
> +
> + Source.SourceFileDependencies =
> + OutPutSourceFileDependencies
> +
> if File.IsBinary and File == Source and File in BinaryFileList:
> # Skip all files that are not binary libraries
> if not self.IsLibrary:
> diff --git a/BaseTools/Source/Python/Common/DataType.py b/BaseTools/Source/Python/Common/DataType.py
> index 8d80b410892946c8b862e060b0bf5f630b409825..8f21e1060446982f866c653aaea70cd662e307a0 100644
> --- a/BaseTools/Source/Python/Common/DataType.py
> +++ b/BaseTools/Source/Python/Common/DataType.py
> @@ -2,7 +2,7 @@
> # This file is used to define common static strings used by INF/DEC/DSC files # # Copyright (c) 2007 - 2018, Intel Corporation. All
> rights reserved.<BR> -# Portions copyright (c) 2011 - 2013, ARM Ltd. All rights reserved.<BR>
> +# Portions copyright (c) 2011 - 2020, ARM Ltd. All rights reserved.<BR>
> # SPDX-License-Identifier: BSD-2-Clause-Patent
>
> ##
> @@ -19,6 +19,7 @@ TAB_VALUE_SPLIT = '|'
> TAB_COMMA_SPLIT = ','
> TAB_SPACE_SPLIT = ' '
> TAB_SEMI_COLON_SPLIT = ';'
> +TAB_COLON_SPLIT = ':'
> TAB_SECTION_START = '['
> TAB_SECTION_END = ']'
> TAB_OPTION_START = '<'
> diff --git a/BaseTools/Source/Python/Common/Misc.py b/BaseTools/Source/Python/Common/Misc.py
> index da5fb380f0354d6e885aa716d2c9b2fead249d63..fb7a8f62087cfe7e0e007a9864832dc1eefc3068 100755
> --- a/BaseTools/Source/Python/Common/Misc.py
> +++ b/BaseTools/Source/Python/Common/Misc.py
> @@ -2,6 +2,8 @@
> # Common routines used by all tools
> #
> # Copyright (c) 2007 - 2019, Intel Corporation. All rights reserved.<BR>
> +# Copyright (c) 2020, ARM Limited. All rights reserved.<BR> #
> # SPDX-License-Identifier: BSD-2-Clause-Patent #
>
> @@ -1438,6 +1440,7 @@ class PathClass(object):
> Arch='COMMON', ToolChainFamily='', Target='', TagName='', ToolCode=''):
> self.Arch = Arch
> self.File = str(File)
> + self.SourceFileDependencies = []
> if os.path.isabs(self.File):
> self.Root = ''
> self.AlterRoot = ''
> diff --git a/BaseTools/Source/Python/Workspace/InfBuildData.py b/BaseTools/Source/Python/Workspace/InfBuildData.py
> index 7675b0ea00ebd6a5fc3e823c965e32066f66f650..02e5c578b426e66b343df937a56ccea319b9f646 100644
> --- a/BaseTools/Source/Python/Workspace/InfBuildData.py
> +++ b/BaseTools/Source/Python/Workspace/InfBuildData.py
> @@ -3,6 +3,8 @@
> #
> # Copyright (c) 2008 - 2018, Intel Corporation. All rights reserved.<BR> # (C) Copyright 2016 Hewlett Packard Enterprise Development
> LP<BR>
> +# Copyright (c) 2020, ARM Limited. All rights reserved.<BR> #
> # SPDX-License-Identifier: BSD-2-Clause-Patent #
>
> @@ -536,12 +538,19 @@ class InfBuildData(ModuleBuildClassObject):
> return []
>
> RetVal = []
> + PendingSourceFileDependencies = dict()
> RecordList = self._RawData[MODEL_EFI_SOURCE_FILE, self._Arch, self._Platform]
> Macros = self._Macros
> for Record in RecordList:
> LineNo = Record[-1]
> - ToolChainFamily = Record[1]
> - TagName = Record[2]
> +
> + BuildOptions = ['', '']
> + SplittedValueList = GetSplitValueList(Record[1], TAB_VALUE_SPLIT, 1)
> + BuildOptions[0:len(SplittedValueList)] = SplittedValueList
> + SourceFileDependencies =
> + list(set(GetSplitValueList(Record[2], TAB_COLON_SPLIT)))
> +
> + ToolChainFamily = BuildOptions[0]
> + TagName = BuildOptions[1]
> ToolCode = Record[3]
>
> File = PathClass(NormPath(Record[0], Macros), self._ModuleDir, '', @@ -552,9 +561,28 @@ class
> InfBuildData(ModuleBuildClassObject):
> EdkLogger.error('build', ErrorCode, ExtraData=ErrorInfo, File=self.MetaFile, Line=LineNo)
>
> RetVal.append(File)
> +
> + # If there are source file dependencies, resolve them after all
> + # the PathClass instances of the source files have been created.
> + if SourceFileDependencies[0] != '':
> + PendingSourceFileDependencies[File] =
> + SourceFileDependencies
> +
> # add any previously found dependency files to the source list
> if self._DependencyFileList:
> RetVal.extend(self._DependencyFileList)
> +
> + # Resolve the dependencies between the PathClass instances.
> + for SourceFile, SourceFileDepList in PendingSourceFileDependencies.items():
> + for SourceFileDepFile in SourceFileDepList:
> + Found = False
> + for Val in RetVal:
> + if SourceFileDepFile == Val.File:
> + SourceFile.SourceFileDependencies.append(Val)
> + Found = True
> + break
> + if not Found:
> + EdkLogger.error("build", FILE_NOT_FOUND,
> + ExtraData=SourceFileDepFile)
> +
> return RetVal
>
> ## Retrieve library classes employed by this module diff --git a/BaseTools/Source/Python/Workspace/MetaFileParser.py
> b/BaseTools/Source/Python/Workspace/MetaFileParser.py
> index a3b6edbd15ee5bf79cef0ac1fc5e53db30356c91..04a656b8ba1e55addc5f37ec8ee2b0439cef0ef1 100644
> --- a/BaseTools/Source/Python/Workspace/MetaFileParser.py
> +++ b/BaseTools/Source/Python/Workspace/MetaFileParser.py
> @@ -3,6 +3,8 @@
> #
> # Copyright (c) 2008 - 2018, Intel Corporation. All rights reserved.<BR> # (C) Copyright 2015-2018 Hewlett Packard Enterprise
> Development LP<BR>
> +# Copyright (c) 2020, ARM Limited. All rights reserved.<BR> #
> # SPDX-License-Identifier: BSD-2-Clause-Patent #
>
> @@ -735,8 +737,20 @@ class InfParser(MetaFileParser):
> #
> @ParseMacro
> def _SourceFileParser(self):
> - TokenList = GetSplitValueList(self._CurrentLine, TAB_VALUE_SPLIT)
> - self._ValueList[0:len(TokenList)] = TokenList
> + SourceFileDependencySplit = GetSplitValueList(self._CurrentLine, TAB_COLON_SPLIT, 1)
> + BuildOptionSplit =
> + GetSplitValueList(SourceFileDependencySplit[0], TAB_VALUE_SPLIT, 1)
> +
> + # Get the filename.
> + self._ValueList[0] = BuildOptionSplit[0]
> +
> + # Get the build options.
> + if len(BuildOptionSplit) > 1:
> + self._ValueList[1] = BuildOptionSplit[1]
> +
> + # Get the dependencies.
> + if len(SourceFileDependencySplit) > 1:
> + self._ValueList[2] = SourceFileDependencySplit[1]
> +
> Macros = self._Macros
> # For Acpi tables, remove macro like ' TABLE_NAME=Sata1'
> if 'COMPONENT_TYPE' in Macros:
> --
> 'Guid(CE165669-3EF3-493F-B85D-6190EE5B9759)'
>
>
>
>
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2020-02-04 13:51 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2020-01-14 18:46 [PATCH v2 1/1] BaseTools: Build ASL files before C files PierreGondois
2020-01-22 17:33 ` PierreGondois
2020-02-04 13:51 ` [edk2-devel] " Liming Gao
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox