public inbox for devel@edk2.groups.io
 help / color / mirror / Atom feed
* [PATCH v1 0/3] Compile AML bytecode array into OBJ file
@ 2020-05-18 14:11 PierreGondois
  2020-05-18 14:11 ` [PATCH v1 1/3] BaseTools: Generate multiple rules when multiple output files PierreGondois
                   ` (5 more replies)
  0 siblings, 6 replies; 17+ messages in thread
From: PierreGondois @ 2020-05-18 14:11 UTC (permalink / raw)
  To: devel; +Cc: Pierre Gondois, bob.c.feng, liming.gao, Sami.Mujawar, Tomas.Pilar,
	nd

Following the BZ at https://bugzilla.tianocore.org/show_bug.cgi?id=2425
This patch serie is a another way to solve the dependency
of C files over ASL files. With this new method, the
dependency is resolved at the linking stage.

The last method to solve this dependency was to add
the possibility to modify INF files to depict such a
dependency. This method was not accepted. The discussion
is available at https://edk2.groups.io/g/devel/topic/72655342#56658

The last patch modifying the INF specification and INF
parsing are available at:
https://edk2.groups.io/g/devel/topic/72655342#56658
https://edk2.groups.io/g/devel/topic/72656060#56662

Pierre Gondois (3):
  BaseTools: Generate multiple rules when multiple output files
  BaseTools: Rename AmlToHex script to AmlToC
  BaseTools: Compile AML bytecode arrays into .obj file

 BaseTools/BinWrappers/PosixLike/{AmlToHex => AmlToC}               | 28 +++----
 BaseTools/BinWrappers/WindowsLike/{AmlToHex.bat => AmlToC.bat}     |  0
 BaseTools/Conf/build_rule.template                                 | 15 +++-
 BaseTools/Source/Python/{AmlToHex/AmlToHex.py => AmlToC/AmlToC.py} | 82 ++++++++------------
 BaseTools/Source/Python/AutoGen/GenMake.py                         |  6 ++
 BaseTools/Source/Python/AutoGen/ModuleAutoGen.py                   | 40 +++++-----
 6 files changed, 86 insertions(+), 85 deletions(-)
 rename BaseTools/BinWrappers/PosixLike/{AmlToHex => AmlToC} (97%)
 rename BaseTools/BinWrappers/WindowsLike/{AmlToHex.bat => AmlToC.bat} (100%)
 rename BaseTools/Source/Python/{AmlToHex/AmlToHex.py => AmlToC/AmlToC.py} (52%)

-- 
'Guid(CE165669-3EF3-493F-B85D-6190EE5B9759)'


^ permalink raw reply	[flat|nested] 17+ messages in thread

* [PATCH v1 1/3] BaseTools: Generate multiple rules when multiple output files
  2020-05-18 14:11 [PATCH v1 0/3] Compile AML bytecode array into OBJ file PierreGondois
@ 2020-05-18 14:11 ` PierreGondois
  2020-06-02 11:15   ` [edk2-devel] " Bob Feng
  2020-05-18 14:11 ` [PATCH v1 2/3] BaseTools: Rename AmlToHex script to AmlToC PierreGondois
                   ` (4 subsequent siblings)
  5 siblings, 1 reply; 17+ messages in thread
From: PierreGondois @ 2020-05-18 14:11 UTC (permalink / raw)
  To: devel; +Cc: Pierre Gondois, bob.c.feng, liming.gao, Sami.Mujawar, Tomas.Pilar,
	nd

From: Pierre Gondois <pierre.gondois@arm.com>

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2425

This patch modifies the Makefile generation not to stop
adding Makfile rules when the first final target is found.
E.g.:
If the following rules are described in build_rule.txt:
 -[Rule1]: .X files generate .Y and .Z files;
 -[Rule2]: .Z files generate .Z1 files.
Currently, if a File1.X file was part of the sources of a
module, only [Rule1] would be generated in the Makefile.
Indeed, there are no rules to apply to .Y files: .Y files
are a final target. However, there is still [Rule2] to
apply to .Z files.

This patch also adds a dependency between the first
ouput file of a rule and the other output files.
For instance, with the same example as above, File1.Y
and File1.Z are generated by the following rule:
File1.Y: File1.X
    <Generate File1.Y>
    <Generate File1.Z>

and the new dependency is:
File1.Z: File1.Y

This is necessary to keep a dependency order during the
execution of the Makefile. Indeed, .Y and .Z files are
generated by the execution of a common set of commands,
and without this rule, there is no explicit dependency
relation between them.

Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
---

The changes can be seen at: https://github.com/PierreARM/edk2/tree/803_Compile_AML_bytecode_array_into_OBJ_file_v1

Notes:
    v1:
     - Generate multiple rules when multiple output files
       are specified in the build_rule.txt file. [Pierre]

 BaseTools/Source/Python/AutoGen/GenMake.py       |  6 +++
 BaseTools/Source/Python/AutoGen/ModuleAutoGen.py | 40 ++++++++++----------
 2 files changed, 25 insertions(+), 21 deletions(-)

diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py b/BaseTools/Source/Python/AutoGen/GenMake.py
index bbb3c29446f53fa7f2cb61a216a5b119f72c3fbc..0314d0ea34d99a014379e8d30c46ac0f0a7068ce 100755
--- a/BaseTools/Source/Python/AutoGen/GenMake.py
+++ b/BaseTools/Source/Python/AutoGen/GenMake.py
@@ -1054,6 +1054,12 @@ cleanlib:
                     TargetDict = {"target": self.PlaceMacro(T.Target.Path, self.Macros), "cmd": "\n\t".join(T.Commands),"deps": Deps}
                     self.BuildTargetList.append(self._BUILD_TARGET_TEMPLATE.Replace(TargetDict))
 
+                    # Add a Makefile rule for targets generating multiple files.
+                    # The main output is a prerequisite for the other output files.
+                    for i in T.Outputs[1:]:
+                        AnnexeTargetDict = {"target": self.PlaceMacro(i.Path, self.Macros), "cmd": "", "deps": self.PlaceMacro(T.Target.Path, self.Macros)}
+                        self.BuildTargetList.append(self._BUILD_TARGET_TEMPLATE.Replace(AnnexeTargetDict))
+
     def ParserCCodeFile(self, T, Type, CmdSumDict, CmdTargetDict, CmdCppDict, DependencyDict):
         if not CmdSumDict:
             for item in self._AutoGenObject.Targets[Type]:
diff --git a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
index aad591de65f086043d55aeea5661f59c53792e7c..82facdb2774c56833718317f4414e0c1861de47f 100755
--- a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
@@ -860,7 +860,8 @@ class ModuleAutoGen(AutoGen):
         SubDirectory = os.path.join(self.OutputDir, File.SubDir)
         if not os.path.exists(SubDirectory):
             CreateDirectory(SubDirectory)
-        LastTarget = None
+        TargetList = set()
+        FinalTargetName = set()
         RuleChain = set()
         SourceList = [File]
         Index = 0
@@ -872,7 +873,7 @@ class ModuleAutoGen(AutoGen):
         while Index < len(SourceList):
             Source = SourceList[Index]
             Index = Index + 1
-
+            FileType = TAB_UNKNOWN_FILE
             if Source != File:
                 CreateDirectory(Source.Dir)
 
@@ -881,34 +882,27 @@ class ModuleAutoGen(AutoGen):
                 if not self.IsLibrary:
                     continue
                 RuleObject = self.BuildRules[TAB_DEFAULT_BINARY_FILE]
-            elif FileType in self.BuildRules:
-                RuleObject = self.BuildRules[FileType]
             elif Source.Ext in self.BuildRules:
                 RuleObject = self.BuildRules[Source.Ext]
             else:
-                # stop at no more rules
-                if LastTarget:
-                    self._FinalBuildTargetList.add(LastTarget)
-                break
-
+                # No more rule to apply: Source is a final target.
+                FinalTargetName.add(Source)
+                continue
             FileType = RuleObject.SourceFileType
             self._FileTypes[FileType].add(Source)
 
             # stop at STATIC_LIBRARY for library
             if self.IsLibrary and FileType == TAB_STATIC_LIBRARY:
-                if LastTarget:
-                    self._FinalBuildTargetList.add(LastTarget)
-                break
+                FinalTargetName.add(Source)
+                continue
 
             Target = RuleObject.Apply(Source, self.BuildRuleOrder)
             if not Target:
-                if LastTarget:
-                    self._FinalBuildTargetList.add(LastTarget)
-                break
-            elif not Target.Outputs:
-                # Only do build for target with outputs
-                self._FinalBuildTargetList.add(Target)
+                # No Target: Source is a final target.
+                FinalTargetName.add(Source)
+                continue
 
+            TargetList.add(Target)
             self._BuildTargets[FileType].add(Target)
 
             if not Source.IsBinary and Source == File:
@@ -916,12 +910,16 @@ class ModuleAutoGen(AutoGen):
 
             # to avoid cyclic rule
             if FileType in RuleChain:
-                break
+                EdkLogger.error("build", ERROR_STATEMENT, "Cyclic dependency detected while generating rule for %s" % str(Source))
 
             RuleChain.add(FileType)
             SourceList.extend(Target.Outputs)
-            LastTarget = Target
-            FileType = TAB_UNKNOWN_FILE
+
+        # For each final target, retrieve the TargetDescBlock instance.
+        for FTargetName in FinalTargetName:
+            for Target in TargetList:
+                if FTargetName == Target.Target:
+                    self._FinalBuildTargetList.add(Target)
 
     @cached_property
     def Targets(self):
-- 
'Guid(CE165669-3EF3-493F-B85D-6190EE5B9759)'


^ permalink raw reply related	[flat|nested] 17+ messages in thread

* [PATCH v1 2/3] BaseTools: Rename AmlToHex script to AmlToC
  2020-05-18 14:11 [PATCH v1 0/3] Compile AML bytecode array into OBJ file PierreGondois
  2020-05-18 14:11 ` [PATCH v1 1/3] BaseTools: Generate multiple rules when multiple output files PierreGondois
@ 2020-05-18 14:11 ` PierreGondois
  2020-05-18 14:11 ` [PATCH v1 3/3] BaseTools: Compile AML bytecode arrays into .obj file PierreGondois
                   ` (3 subsequent siblings)
  5 siblings, 0 replies; 17+ messages in thread
From: PierreGondois @ 2020-05-18 14:11 UTC (permalink / raw)
  To: devel; +Cc: Pierre Gondois, bob.c.feng, liming.gao, Sami.Mujawar, Tomas.Pilar,
	nd

From: Pierre Gondois <pierre.gondois@arm.com>

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2425

The AmlToHex script and Posix/WindowsLike wrappers convert
an AML file to a .hex file, containing a C array storing
AML bytecode. This ".hex" file can then be included in a
C file, allowing to access the AML bytecode from this C
file.

The EDK2 build system doesn't allow to a depict dependency
orders between files of different languages. For instance,
in a module containing a ".c" file and a ".asl", the ".c"
file may or may not be built prior to the ".asl" file.
This prevents any inclusion of a generated ".hex" in a
".c" file since this later ".hex" file may or may not
have been created yet.

This patch renames the script as AmlToC. It is posted as
a separate patch to prevent git from seeing the renaming
as a deletion plus addition of a new file.
The ending line of the posix-like bin-wrapper script has
also been corrected.

This is a first step toward generating a C file containing
the AML bytecode from an ASL file. This C file will then
be handled by the EDK2 build system to generate an object
file.
Thus, no file inclusion will be required anymore. The C file
requiring the AML bytecode as a C array, and the ASL file,
will be compiled independently. The C array must be defined
as an external symbol. The linker is resolving the
reference to the C array symbol.

To summarize, the flow goes as:
 -1. ASL file is compiled to AML;
 -2. AML file is copied to a ".amli" intermediate file;
 -3. EDK2 build system applies the rule relevant to ".amli"
     files. This is, calling the "AmlToC" script, generating
     a C file from the ".amli" file;
 -4. EDK2 build system applies the rule relevant to C files.
     This is creating an object file.
 -5. EDK2 build system links the object file containing the
     AML bytecode with the object file requiring it.

Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
Signed-off-by: Tomas Pilar <Tomas.Pilar@arm.com>
---

The changes can be seen at: https://github.com/PierreARM/edk2/tree/803_Compile_AML_bytecode_array_into_OBJ_file_v1

Notes:
    v1:
     - Rename AmlToHex scripts to AmlToC, and change line
       endings of the PosixLike bin-wrapper. [Pierre]

 BaseTools/BinWrappers/PosixLike/{AmlToHex => AmlToC}               | 28 ++++++++++----------
 BaseTools/BinWrappers/WindowsLike/{AmlToHex.bat => AmlToC.bat}     |  0
 BaseTools/Source/Python/{AmlToHex/AmlToHex.py => AmlToC/AmlToC.py} |  0
 3 files changed, 14 insertions(+), 14 deletions(-)

diff --git a/BaseTools/BinWrappers/PosixLike/AmlToHex b/BaseTools/BinWrappers/PosixLike/AmlToC
similarity index 97%
rename from BaseTools/BinWrappers/PosixLike/AmlToHex
rename to BaseTools/BinWrappers/PosixLike/AmlToC
index 9fb68299e4c67d1f332cd883fd348a896f1bdc50..1dd28e966288f6ea4fc52d42e2dc7b1f74226c23 100755
--- a/BaseTools/BinWrappers/PosixLike/AmlToHex
+++ b/BaseTools/BinWrappers/PosixLike/AmlToC
@@ -1,14 +1,14 @@
-#!/usr/bin/env bash
-#python `dirname $0`/RunToolFromSource.py `basename $0` $*
-
-# If a ${PYTHON_COMMAND} command is available, use it in preference to python
-if command -v ${PYTHON_COMMAND} >/dev/null 2>&1; then
-    python_exe=${PYTHON_COMMAND}
-fi
-
-full_cmd=${BASH_SOURCE:-$0} # see http://mywiki.wooledge.org/BashFAQ/028 for a discussion of why $0 is not a good choice here
-dir=$(dirname "$full_cmd")
-exe=$(basename "$full_cmd")
-
-export PYTHONPATH="$dir/../../Source/Python${PYTHONPATH:+:"$PYTHONPATH"}"
-exec "${python_exe:-python}" "$dir/../../Source/Python/$exe/$exe.py" "$@"
+#!/usr/bin/env bash
+#python `dirname $0`/RunToolFromSource.py `basename $0` $*
+
+# If a ${PYTHON_COMMAND} command is available, use it in preference to python
+if command -v ${PYTHON_COMMAND} >/dev/null 2>&1; then
+    python_exe=${PYTHON_COMMAND}
+fi
+
+full_cmd=${BASH_SOURCE:-$0} # see http://mywiki.wooledge.org/BashFAQ/028 for a discussion of why $0 is not a good choice here
+dir=$(dirname "$full_cmd")
+exe=$(basename "$full_cmd")
+
+export PYTHONPATH="$dir/../../Source/Python${PYTHONPATH:+:"$PYTHONPATH"}"
+exec "${python_exe:-python}" "$dir/../../Source/Python/$exe/$exe.py" "$@"
diff --git a/BaseTools/BinWrappers/WindowsLike/AmlToHex.bat b/BaseTools/BinWrappers/WindowsLike/AmlToC.bat
similarity index 100%
rename from BaseTools/BinWrappers/WindowsLike/AmlToHex.bat
rename to BaseTools/BinWrappers/WindowsLike/AmlToC.bat
diff --git a/BaseTools/Source/Python/AmlToHex/AmlToHex.py b/BaseTools/Source/Python/AmlToC/AmlToC.py
similarity index 100%
rename from BaseTools/Source/Python/AmlToHex/AmlToHex.py
rename to BaseTools/Source/Python/AmlToC/AmlToC.py
-- 
'Guid(CE165669-3EF3-493F-B85D-6190EE5B9759)'


^ permalink raw reply related	[flat|nested] 17+ messages in thread

* [PATCH v1 3/3] BaseTools: Compile AML bytecode arrays into .obj file
  2020-05-18 14:11 [PATCH v1 0/3] Compile AML bytecode array into OBJ file PierreGondois
  2020-05-18 14:11 ` [PATCH v1 1/3] BaseTools: Generate multiple rules when multiple output files PierreGondois
  2020-05-18 14:11 ` [PATCH v1 2/3] BaseTools: Rename AmlToHex script to AmlToC PierreGondois
@ 2020-05-18 14:11 ` PierreGondois
  2020-05-18 14:31 ` [PATCH v1 0/3] Compile AML bytecode array into OBJ file PierreGondois
                   ` (2 subsequent siblings)
  5 siblings, 0 replies; 17+ messages in thread
From: PierreGondois @ 2020-05-18 14:11 UTC (permalink / raw)
  To: devel; +Cc: Pierre Gondois, bob.c.feng, liming.gao, Sami.Mujawar, Tomas.Pilar,
	nd

From: Pierre Gondois <pierre.gondois@arm.com>

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2425

The AmlToHex script and Posix/WindowsLike wrappers convert
an AML file to a .hex file, containing a C array storing
AML bytecode. This ".hex" file can then be included in a
C file, allowing to access the AML bytecode from this C
file.

The EDK2 build system doesn't allow to a depict dependency
orders between files of different languages. For instance,
in a module containing a ".c" file and a ".asl", the ".c"
file may or may not be built prior to the ".asl" file.
This prevents any inclusion of a generated ".hex" in a
".c" file since this later ".hex" file may or may not
have been created yet.

This patch modifies the AmlToC script to generate a C file
instead of a ".hex" file.
It also adds the generation of an intermediate ".amli" file
when compiling an ASL file, and adds a rule to convert this
".amli" to a C file.

This allows to generate a C file containing the AML bytecode
from an ASL file. This C file will then be handled by the EDK2
build system to generate an object file.
Thus, no file inclusion will be required anymore. The C file
requiring the AML bytecode as a C array, and the ASL file,
will be compiled independently. The C array must be defined
as an external symbol. The linker is resolving the
reference to the C array symbol.

To summarize, the flow goes as:
 -1. ASL file is compiled to AML;
 -2. AML file is copied to a ".amli" intermediate file;
 -3. EDK2 build system applies the rule relevant to ".amli"
     files. This is, calling the "AmlToC" script, generating
     a C file from the ".amli" file;
 -4. EDK2 build system applies the rule relevant to C files.
     This is creating an object file.
 -5. EDK2 build system links the object file containing the
     AML bytecode with the object file requiring it.

Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
Signed-off-by: Tomas Pilar <Tomas.Pilar@arm.com>
---

The changes can be seen at: https://github.com/PierreARM/edk2/tree/803_Compile_AML_bytecode_array_into_OBJ_file_v1

Notes:
    v1:
     - Add a new rule to the build_rule.template file to
       generate ".obj" files from .asl files, and modify
       the AmlToC script accordingly. [Pierre]

 BaseTools/Conf/build_rule.template       | 15 +++-
 BaseTools/Source/Python/AmlToC/AmlToC.py | 82 ++++++++------------
 2 files changed, 47 insertions(+), 50 deletions(-)

diff --git a/BaseTools/Conf/build_rule.template b/BaseTools/Conf/build_rule.template
index 0822b681fcd9f61c6508e6f93ffc31fa70fd7059..a467d9097650435dab9b6d0b3c575be31f0c84e0 100755
--- a/BaseTools/Conf/build_rule.template
+++ b/BaseTools/Conf/build_rule.template
@@ -419,6 +419,7 @@
 
     <OutputFile>
         $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.aml
+        $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.amli
 
     <ExtraDependency>
         $(MAKE_FILE)
@@ -428,14 +429,24 @@
         "$(ASLPP)" $(DEPS_FLAGS) $(ASLPP_FLAGS) $(INC) /I${s_path} $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.i > $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.iii
         Trim --source-code -l -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.iiii $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.iii 
         "$(ASL)" $(ASL_FLAGS) $(ASL_OUTFLAGS)${dst} $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.iiii
-        -AmlToHex $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.aml
+        $(CP) $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.aml $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.amli
 
     <Command.GCC>
         Trim --asl-file --asl-deps -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.i -i $(INC_LIST) ${src}
         "$(ASLPP)" $(DEPS_FLAGS) $(ASLPP_FLAGS) $(INC) -I${s_path} $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.i > $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.iii
         Trim --source-code -l -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.iiii $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.iii 
         "$(ASL)" $(ASL_FLAGS) $(ASL_OUTFLAGS)${dst} $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.iiii
-        -AmlToHex $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.aml
+        $(CP) $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.aml $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.amli
+
+[Acpi-Machine-Language-File-to-C]
+    <InputFile>
+        ?.amli
+
+    <OutputFile>
+        ${s_path}(+)${s_base}.c
+
+    <Command>
+        -AmlToC ${src}
 
 [C-Code-File.AcpiTable]
     <InputFile>
diff --git a/BaseTools/Source/Python/AmlToC/AmlToC.py b/BaseTools/Source/Python/AmlToC/AmlToC.py
index 643db2910e37acfdd80ac18d288c921320a79ce1..346de7159de702d860bbd809ddbe8175f1493cfb 100644
--- a/BaseTools/Source/Python/AmlToC/AmlToC.py
+++ b/BaseTools/Source/Python/AmlToC/AmlToC.py
@@ -1,9 +1,9 @@
 ## @file
 #
-# Convert an AML file to a .hex file containing the AML bytecode stored in a
+# Convert an AML file to a .c file containing the AML bytecode stored in a
 # C array.
-# By default, "Tables\Dsdt.aml" will generate "Tables\Dsdt.hex".
-# "Tables\Dsdt.hex" will contain a C array named "dsdt_aml_code" that contains
+# By default, "Tables\Dsdt.aml" will generate "Tables\Dsdt.c".
+# "Tables\Dsdt.c" will contain a C array named "dsdt_aml_code" that contains
 # the AML bytecode.
 #
 # Copyright (c) 2020, ARM Limited. All rights reserved.<BR>
@@ -17,31 +17,26 @@ from Common.BuildToolError import *
 import sys
 import os
 
+__description__ = """
+Convert an AML file to a .c file containing the AML bytecode stored in a C
+array. By default, Tables\Dsdt.aml will generate Tables\Dsdt.c.
+Tables\Dsdt.c will contain a C array named "dsdt_aml_code" that contains
+the AML bytecode.
+"""
+
 ## Parse the command line arguments.
 #
 # @retval A argparse.NameSpace instance, containing parsed values.
 #
 def ParseArgs():
     # Initialize the parser.
-    Parser = argparse.ArgumentParser(
-      description="Convert an AML file to a .hex file containing the AML " + \
-                  "bytecode stored in a C array. By default, " + \
-                  "\"Tables\\Dsdt.aml\" will generate" + \
-                  "\"Tables\\Dsdt.hex\". \"Tables\\Dsdt.hex\" will " + \
-                  "contain a C array named \"dsdt_aml_code\" that " + \
-                  "contains the AML bytecode."
-      )
+    Parser = argparse.ArgumentParser(description=__description__)
 
     # Define the possible arguments.
-    Parser.add_argument(
-      dest="InputFile",
-      help="Path to an input AML file to generate a .hex file from."
-      )
-    Parser.add_argument(
-      "-o", "--out-dir", dest="OutDir",
-      help="Output directory where the .hex file will be generated. " + \
-           "Default is the input file's directory."
-      )
+    Parser.add_argument(dest="InputFile",
+                        help="Path to an input AML file to generate a .c file from.")
+    Parser.add_argument("-o", "--out-dir", dest="OutDir",
+                        help="Output directory where the .c file will be generated. Default is the input file's directory.")
 
     # Parse the input arguments.
     Args = Parser.parse_args()
@@ -55,9 +50,7 @@ def ParseArgs():
         with open(Args.InputFile, "rb") as fIn:
             Signature = str(fIn.read(4))
             if ("DSDT" not in Signature) and ("SSDT" not in Signature):
-                EdkLogger.info("Invalid file type. " + \
-                                "File does not have a valid " + \
-                                "DSDT or SSDT signature: %s" % Args.InputFile)
+                EdkLogger.info("Invalid file type. File does not have a valid DSDT or SSDT signature: {}".format(Args.InputFile))
                 return None
 
     # Get the basename of the input file.
@@ -66,42 +59,39 @@ def ParseArgs():
 
     # If no output directory is specified, output to the input directory.
     if not Args.OutDir:
-        Args.OutputFile = os.path.join(
-          os.path.dirname(Args.InputFile),
-          BaseName + ".hex"
-          )
+        Args.OutputFile = os.path.join(os.path.dirname(Args.InputFile),
+                                       BaseName + ".c")
     else:
         if not os.path.exists(Args.OutDir):
             os.mkdir(Args.OutDir)
-        Args.OutputFile = os.path.join(Args.OutDir, BaseName + ".hex")
+        Args.OutputFile = os.path.join(Args.OutDir, BaseName + ".c")
 
     Args.BaseName = BaseName
 
     return Args
 
-## Convert an AML file to a .hex file containing the AML bytecode stored
+## Convert an AML file to a .c file containing the AML bytecode stored
 #  in a C array.
 #
 # @param  InputFile     Path to the input AML file.
-# @param  OutputFile    Path to the output .hex file to generate.
+# @param  OutputFile    Path to the output .c file to generate.
 # @param  BaseName      Base name of the input file.
-#                       This is also the name of the generated .hex file.
+#                       This is also the name of the generated .c file.
 #
-def AmlToHex(InputFile, OutputFile, BaseName):
+def AmlToC(InputFile, OutputFile, BaseName):
 
-    MacroName = "__{}_HEX__".format(BaseName.upper())
     ArrayName = BaseName.lower() + "_aml_code"
+    FileHeader =\
+"""
+// This file has been generated from:
+//   -Python script: {}
+//   -Input AML file: {}
+
+"""
 
     with open(InputFile, "rb") as fIn, open(OutputFile, "w") as fOut:
         # Write header.
-        fOut.write("// This file has been generated from:\n" + \
-                   "// \tPython script: " + \
-                   os.path.abspath(__file__) + "\n" + \
-                   "// \tInput AML file: " + \
-                   os.path.abspath(InputFile) + "\n\n" + \
-                   "#ifndef {}\n".format(MacroName) + \
-                   "#define {}\n\n".format(MacroName)
-                   )
+        fOut.write(FileHeader.format(os.path.abspath(InputFile), os.path.abspath(__file__)))
 
         # Write the array and its content.
         fOut.write("unsigned char {}[] = {{\n  ".format(ArrayName))
@@ -115,15 +105,12 @@ def AmlToHex(InputFile, OutputFile, BaseName):
             byte = fIn.read(1)
         fOut.write("\n};\n")
 
-        # Write footer.
-        fOut.write("#endif // {}\n".format(MacroName))
-
 ## Main method
 #
 # This method:
 #   1-  Initialize an EdkLogger instance.
 #   2-  Parses the input arguments.
-#   3-  Converts an AML file to a .hex file containing the AML bytecode stored
+#   3-  Converts an AML file to a .c file containing the AML bytecode stored
 #       in a C array.
 #
 # @retval 0     Success.
@@ -139,10 +126,9 @@ def Main():
         if not CommandArguments:
             return 1
 
-        # Convert an AML file to a .hex file containing the AML bytecode stored
+        # Convert an AML file to a .c file containing the AML bytecode stored
         # in a C array.
-        AmlToHex(CommandArguments.InputFile, CommandArguments.OutputFile,
-                          CommandArguments.BaseName)
+        AmlToC(CommandArguments.InputFile, CommandArguments.OutputFile, CommandArguments.BaseName)
     except Exception as e:
         print(e)
         return 1
-- 
'Guid(CE165669-3EF3-493F-B85D-6190EE5B9759)'


^ permalink raw reply related	[flat|nested] 17+ messages in thread

* Re: [PATCH v1 0/3] Compile AML bytecode array into OBJ file
  2020-05-18 14:11 [PATCH v1 0/3] Compile AML bytecode array into OBJ file PierreGondois
                   ` (2 preceding siblings ...)
  2020-05-18 14:11 ` [PATCH v1 3/3] BaseTools: Compile AML bytecode arrays into .obj file PierreGondois
@ 2020-05-18 14:31 ` PierreGondois
       [not found] ` <161025E2E32EBB17.23434@groups.io>
  2020-06-13  9:37 ` Ard Biesheuvel
  5 siblings, 0 replies; 17+ messages in thread
From: PierreGondois @ 2020-05-18 14:31 UTC (permalink / raw)
  To: Pierre Gondois, devel@edk2.groups.io
  Cc: bob.c.feng@intel.com, liming.gao@intel.com, Sami Mujawar,
	Tomas Pilar, nd

[Bob] https://edk2.groups.io/g/devel/message/58782
I'd prefer the option 2#. I think that is the easiest way to meet the requirement. Since the aml file is the definite output of asl file, we can use it to descript the dependency relationship between c source file and asl source code.
[Pierre]
We have found what might be a better way to address the issue. If not convenient, we can still follow the option 2# that we discussed.

Regards,
Pierre

-----Original Message-----
From: PierreGondois <pierre.gondois@arm.com> 
Sent: Monday, May 18, 2020 3:11 PM
To: devel@edk2.groups.io
Cc: Pierre Gondois <Pierre.Gondois@arm.com>; bob.c.feng@intel.com; liming.gao@intel.com; Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; nd <nd@arm.com>
Subject: [PATCH v1 0/3] Compile AML bytecode array into OBJ file

Following the BZ at https://bugzilla.tianocore.org/show_bug.cgi?id=2425
This patch serie is a another way to solve the dependency of C files over ASL files. With this new method, the dependency is resolved at the linking stage.

The last method to solve this dependency was to add the possibility to modify INF files to depict such a dependency. This method was not accepted. The discussion is available at https://edk2.groups.io/g/devel/topic/72655342#56658

The last patch modifying the INF specification and INF parsing are available at:
https://edk2.groups.io/g/devel/topic/72655342#56658
https://edk2.groups.io/g/devel/topic/72656060#56662

Pierre Gondois (3):
  BaseTools: Generate multiple rules when multiple output files
  BaseTools: Rename AmlToHex script to AmlToC
  BaseTools: Compile AML bytecode arrays into .obj file

 BaseTools/BinWrappers/PosixLike/{AmlToHex => AmlToC}               | 28 +++----
 BaseTools/BinWrappers/WindowsLike/{AmlToHex.bat => AmlToC.bat}     |  0
 BaseTools/Conf/build_rule.template                                 | 15 +++-
 BaseTools/Source/Python/{AmlToHex/AmlToHex.py => AmlToC/AmlToC.py} | 82 ++++++++------------
 BaseTools/Source/Python/AutoGen/GenMake.py                         |  6 ++
 BaseTools/Source/Python/AutoGen/ModuleAutoGen.py                   | 40 +++++-----
 6 files changed, 86 insertions(+), 85 deletions(-)  rename BaseTools/BinWrappers/PosixLike/{AmlToHex => AmlToC} (97%)  rename BaseTools/BinWrappers/WindowsLike/{AmlToHex.bat => AmlToC.bat} (100%)  rename BaseTools/Source/Python/{AmlToHex/AmlToHex.py => AmlToC/AmlToC.py} (52%)

--
'Guid(CE165669-3EF3-493F-B85D-6190EE5B9759)'


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [edk2-devel] [PATCH v1 0/3] Compile AML bytecode array into OBJ file
       [not found] ` <161025E2E32EBB17.23434@groups.io>
@ 2020-05-22 12:01   ` PierreGondois
       [not found]   ` <161157FAC70B55F8.21259@groups.io>
  1 sibling, 0 replies; 17+ messages in thread
From: PierreGondois @ 2020-05-22 12:01 UTC (permalink / raw)
  To: devel@edk2.groups.io, Pierre Gondois
  Cc: bob.c.feng@intel.com, liming.gao@intel.com, Sami Mujawar,
	Tomas Pilar, nd

Hello Bob and Liming,
What do you think about the patch? Do you think it goes in the right direction?

Regards,
Pierre

-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of PierreGondois via groups.io
Sent: Monday, May 18, 2020 3:32 PM
To: Pierre Gondois <Pierre.Gondois@arm.com>; devel@edk2.groups.io
Cc: bob.c.feng@intel.com; liming.gao@intel.com; Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; nd <nd@arm.com>
Subject: Re: [edk2-devel] [PATCH v1 0/3] Compile AML bytecode array into OBJ file

[Bob] https://edk2.groups.io/g/devel/message/58782
I'd prefer the option 2#. I think that is the easiest way to meet the requirement. Since the aml file is the definite output of asl file, we can use it to descript the dependency relationship between c source file and asl source code.
[Pierre]
We have found what might be a better way to address the issue. If not convenient, we can still follow the option 2# that we discussed.

Regards,
Pierre

-----Original Message-----
From: PierreGondois <pierre.gondois@arm.com> 
Sent: Monday, May 18, 2020 3:11 PM
To: devel@edk2.groups.io
Cc: Pierre Gondois <Pierre.Gondois@arm.com>; bob.c.feng@intel.com; liming.gao@intel.com; Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; nd <nd@arm.com>
Subject: [PATCH v1 0/3] Compile AML bytecode array into OBJ file

Following the BZ at https://bugzilla.tianocore.org/show_bug.cgi?id=2425
This patch serie is a another way to solve the dependency of C files over ASL files. With this new method, the dependency is resolved at the linking stage.

The last method to solve this dependency was to add the possibility to modify INF files to depict such a dependency. This method was not accepted. The discussion is available at https://edk2.groups.io/g/devel/topic/72655342#56658

The last patch modifying the INF specification and INF parsing are available at:
https://edk2.groups.io/g/devel/topic/72655342#56658
https://edk2.groups.io/g/devel/topic/72656060#56662

Pierre Gondois (3):
  BaseTools: Generate multiple rules when multiple output files
  BaseTools: Rename AmlToHex script to AmlToC
  BaseTools: Compile AML bytecode arrays into .obj file

 BaseTools/BinWrappers/PosixLike/{AmlToHex => AmlToC}               | 28 +++----
 BaseTools/BinWrappers/WindowsLike/{AmlToHex.bat => AmlToC.bat}     |  0
 BaseTools/Conf/build_rule.template                                 | 15 +++-
 BaseTools/Source/Python/{AmlToHex/AmlToHex.py => AmlToC/AmlToC.py} | 82 ++++++++------------
 BaseTools/Source/Python/AutoGen/GenMake.py                         |  6 ++
 BaseTools/Source/Python/AutoGen/ModuleAutoGen.py                   | 40 +++++-----
 6 files changed, 86 insertions(+), 85 deletions(-)  rename BaseTools/BinWrappers/PosixLike/{AmlToHex => AmlToC} (97%)  rename BaseTools/BinWrappers/WindowsLike/{AmlToHex.bat => AmlToC.bat} (100%)  rename BaseTools/Source/Python/{AmlToHex/AmlToHex.py => AmlToC/AmlToC.py} (52%)

--
'Guid(CE165669-3EF3-493F-B85D-6190EE5B9759)'





^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [edk2-devel] [PATCH v1 0/3] Compile AML bytecode array into OBJ file
       [not found]   ` <161157FAC70B55F8.21259@groups.io>
@ 2020-05-28 11:03     ` PierreGondois
  2020-05-28 11:10       ` Bob Feng
  2020-06-02 11:15       ` Bob Feng
  0 siblings, 2 replies; 17+ messages in thread
From: PierreGondois @ 2020-05-28 11:03 UTC (permalink / raw)
  To: devel@edk2.groups.io, Pierre Gondois
  Cc: bob.c.feng@intel.com, liming.gao@intel.com, Sami Mujawar,
	Tomas Pilar, nd

Hello Bob and Liming,
Just a message in case you missed the patch set,

Regards,
Pierre

-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of PierreGondois via groups.io
Sent: Friday, May 22, 2020 1:01 PM
To: devel@edk2.groups.io; Pierre Gondois <Pierre.Gondois@arm.com>
Cc: bob.c.feng@intel.com; liming.gao@intel.com; Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; nd <nd@arm.com>
Subject: Re: [edk2-devel] [PATCH v1 0/3] Compile AML bytecode array into OBJ file

Hello Bob and Liming,
What do you think about the patch? Do you think it goes in the right direction?

Regards,
Pierre

-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of PierreGondois via groups.io
Sent: Monday, May 18, 2020 3:32 PM
To: Pierre Gondois <Pierre.Gondois@arm.com>; devel@edk2.groups.io
Cc: bob.c.feng@intel.com; liming.gao@intel.com; Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; nd <nd@arm.com>
Subject: Re: [edk2-devel] [PATCH v1 0/3] Compile AML bytecode array into OBJ file

[Bob] https://edk2.groups.io/g/devel/message/58782
I'd prefer the option 2#. I think that is the easiest way to meet the requirement. Since the aml file is the definite output of asl file, we can use it to descript the dependency relationship between c source file and asl source code.
[Pierre]
We have found what might be a better way to address the issue. If not convenient, we can still follow the option 2# that we discussed.

Regards,
Pierre

-----Original Message-----
From: PierreGondois <pierre.gondois@arm.com> 
Sent: Monday, May 18, 2020 3:11 PM
To: devel@edk2.groups.io
Cc: Pierre Gondois <Pierre.Gondois@arm.com>; bob.c.feng@intel.com; liming.gao@intel.com; Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; nd <nd@arm.com>
Subject: [PATCH v1 0/3] Compile AML bytecode array into OBJ file

Following the BZ at https://bugzilla.tianocore.org/show_bug.cgi?id=2425
This patch serie is a another way to solve the dependency of C files over ASL files. With this new method, the dependency is resolved at the linking stage.

The last method to solve this dependency was to add the possibility to modify INF files to depict such a dependency. This method was not accepted. The discussion is available at https://edk2.groups.io/g/devel/topic/72655342#56658

The last patch modifying the INF specification and INF parsing are available at:
https://edk2.groups.io/g/devel/topic/72655342#56658
https://edk2.groups.io/g/devel/topic/72656060#56662

Pierre Gondois (3):
  BaseTools: Generate multiple rules when multiple output files
  BaseTools: Rename AmlToHex script to AmlToC
  BaseTools: Compile AML bytecode arrays into .obj file

 BaseTools/BinWrappers/PosixLike/{AmlToHex => AmlToC}               | 28 +++----
 BaseTools/BinWrappers/WindowsLike/{AmlToHex.bat => AmlToC.bat}     |  0
 BaseTools/Conf/build_rule.template                                 | 15 +++-
 BaseTools/Source/Python/{AmlToHex/AmlToHex.py => AmlToC/AmlToC.py} | 82 ++++++++------------
 BaseTools/Source/Python/AutoGen/GenMake.py                         |  6 ++
 BaseTools/Source/Python/AutoGen/ModuleAutoGen.py                   | 40 +++++-----
 6 files changed, 86 insertions(+), 85 deletions(-)  rename BaseTools/BinWrappers/PosixLike/{AmlToHex => AmlToC} (97%)  rename BaseTools/BinWrappers/WindowsLike/{AmlToHex.bat => AmlToC.bat} (100%)  rename BaseTools/Source/Python/{AmlToHex/AmlToHex.py => AmlToC/AmlToC.py} (52%)

--
'Guid(CE165669-3EF3-493F-B85D-6190EE5B9759)'








^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [edk2-devel] [PATCH v1 0/3] Compile AML bytecode array into OBJ file
  2020-05-28 11:03     ` PierreGondois
@ 2020-05-28 11:10       ` Bob Feng
  2020-06-02 11:15       ` Bob Feng
  1 sibling, 0 replies; 17+ messages in thread
From: Bob Feng @ 2020-05-28 11:10 UTC (permalink / raw)
  To: devel@edk2.groups.io, pierre.gondois@arm.com
  Cc: Gao, Liming, Sami Mujawar, Tomas Pilar, nd

Hi Pierre,

Sorry for late response. I'll review the patch in this week and response early in the next week.

Thanks,
Bob

-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of PierreGondois
Sent: Thursday, May 28, 2020 7:04 PM
To: devel@edk2.groups.io; Pierre Gondois <Pierre.Gondois@arm.com>
Cc: Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming <liming.gao@intel.com>; Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; nd <nd@arm.com>
Subject: Re: [edk2-devel] [PATCH v1 0/3] Compile AML bytecode array into OBJ file

Hello Bob and Liming,
Just a message in case you missed the patch set,

Regards,
Pierre

-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of PierreGondois via groups.io
Sent: Friday, May 22, 2020 1:01 PM
To: devel@edk2.groups.io; Pierre Gondois <Pierre.Gondois@arm.com>
Cc: bob.c.feng@intel.com; liming.gao@intel.com; Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; nd <nd@arm.com>
Subject: Re: [edk2-devel] [PATCH v1 0/3] Compile AML bytecode array into OBJ file

Hello Bob and Liming,
What do you think about the patch? Do you think it goes in the right direction?

Regards,
Pierre

-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of PierreGondois via groups.io
Sent: Monday, May 18, 2020 3:32 PM
To: Pierre Gondois <Pierre.Gondois@arm.com>; devel@edk2.groups.io
Cc: bob.c.feng@intel.com; liming.gao@intel.com; Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; nd <nd@arm.com>
Subject: Re: [edk2-devel] [PATCH v1 0/3] Compile AML bytecode array into OBJ file

[Bob] https://edk2.groups.io/g/devel/message/58782
I'd prefer the option 2#. I think that is the easiest way to meet the requirement. Since the aml file is the definite output of asl file, we can use it to descript the dependency relationship between c source file and asl source code.
[Pierre]
We have found what might be a better way to address the issue. If not convenient, we can still follow the option 2# that we discussed.

Regards,
Pierre

-----Original Message-----
From: PierreGondois <pierre.gondois@arm.com> 
Sent: Monday, May 18, 2020 3:11 PM
To: devel@edk2.groups.io
Cc: Pierre Gondois <Pierre.Gondois@arm.com>; bob.c.feng@intel.com; liming.gao@intel.com; Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; nd <nd@arm.com>
Subject: [PATCH v1 0/3] Compile AML bytecode array into OBJ file

Following the BZ at https://bugzilla.tianocore.org/show_bug.cgi?id=2425
This patch serie is a another way to solve the dependency of C files over ASL files. With this new method, the dependency is resolved at the linking stage.

The last method to solve this dependency was to add the possibility to modify INF files to depict such a dependency. This method was not accepted. The discussion is available at https://edk2.groups.io/g/devel/topic/72655342#56658

The last patch modifying the INF specification and INF parsing are available at:
https://edk2.groups.io/g/devel/topic/72655342#56658
https://edk2.groups.io/g/devel/topic/72656060#56662

Pierre Gondois (3):
  BaseTools: Generate multiple rules when multiple output files
  BaseTools: Rename AmlToHex script to AmlToC
  BaseTools: Compile AML bytecode arrays into .obj file

 BaseTools/BinWrappers/PosixLike/{AmlToHex => AmlToC}               | 28 +++----
 BaseTools/BinWrappers/WindowsLike/{AmlToHex.bat => AmlToC.bat}     |  0
 BaseTools/Conf/build_rule.template                                 | 15 +++-
 BaseTools/Source/Python/{AmlToHex/AmlToHex.py => AmlToC/AmlToC.py} | 82 ++++++++------------
 BaseTools/Source/Python/AutoGen/GenMake.py                         |  6 ++
 BaseTools/Source/Python/AutoGen/ModuleAutoGen.py                   | 40 +++++-----
 6 files changed, 86 insertions(+), 85 deletions(-)  rename BaseTools/BinWrappers/PosixLike/{AmlToHex => AmlToC} (97%)  rename BaseTools/BinWrappers/WindowsLike/{AmlToHex.bat => AmlToC.bat} (100%)  rename BaseTools/Source/Python/{AmlToHex/AmlToHex.py => AmlToC/AmlToC.py} (52%)

--
'Guid(CE165669-3EF3-493F-B85D-6190EE5B9759)'











^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [edk2-devel] [PATCH v1 0/3] Compile AML bytecode array into OBJ file
  2020-05-28 11:03     ` PierreGondois
  2020-05-28 11:10       ` Bob Feng
@ 2020-06-02 11:15       ` Bob Feng
  1 sibling, 0 replies; 17+ messages in thread
From: Bob Feng @ 2020-06-02 11:15 UTC (permalink / raw)
  To: devel@edk2.groups.io, pierre.gondois@arm.com
  Cc: Gao, Liming, Sami Mujawar, Tomas Pilar, nd

Hi Pierre,

I agree with your solution.  My comments are in the 1/3 patch mail.

Thanks,
Bob

-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of PierreGondois
Sent: Thursday, May 28, 2020 7:04 PM
To: devel@edk2.groups.io; Pierre Gondois <Pierre.Gondois@arm.com>
Cc: Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming <liming.gao@intel.com>; Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; nd <nd@arm.com>
Subject: Re: [edk2-devel] [PATCH v1 0/3] Compile AML bytecode array into OBJ file

Hello Bob and Liming,
Just a message in case you missed the patch set,

Regards,
Pierre

-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of PierreGondois via groups.io
Sent: Friday, May 22, 2020 1:01 PM
To: devel@edk2.groups.io; Pierre Gondois <Pierre.Gondois@arm.com>
Cc: bob.c.feng@intel.com; liming.gao@intel.com; Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; nd <nd@arm.com>
Subject: Re: [edk2-devel] [PATCH v1 0/3] Compile AML bytecode array into OBJ file

Hello Bob and Liming,
What do you think about the patch? Do you think it goes in the right direction?

Regards,
Pierre

-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of PierreGondois via groups.io
Sent: Monday, May 18, 2020 3:32 PM
To: Pierre Gondois <Pierre.Gondois@arm.com>; devel@edk2.groups.io
Cc: bob.c.feng@intel.com; liming.gao@intel.com; Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; nd <nd@arm.com>
Subject: Re: [edk2-devel] [PATCH v1 0/3] Compile AML bytecode array into OBJ file

[Bob] https://edk2.groups.io/g/devel/message/58782
I'd prefer the option 2#. I think that is the easiest way to meet the requirement. Since the aml file is the definite output of asl file, we can use it to descript the dependency relationship between c source file and asl source code.
[Pierre]
We have found what might be a better way to address the issue. If not convenient, we can still follow the option 2# that we discussed.

Regards,
Pierre

-----Original Message-----
From: PierreGondois <pierre.gondois@arm.com> 
Sent: Monday, May 18, 2020 3:11 PM
To: devel@edk2.groups.io
Cc: Pierre Gondois <Pierre.Gondois@arm.com>; bob.c.feng@intel.com; liming.gao@intel.com; Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; nd <nd@arm.com>
Subject: [PATCH v1 0/3] Compile AML bytecode array into OBJ file

Following the BZ at https://bugzilla.tianocore.org/show_bug.cgi?id=2425
This patch serie is a another way to solve the dependency of C files over ASL files. With this new method, the dependency is resolved at the linking stage.

The last method to solve this dependency was to add the possibility to modify INF files to depict such a dependency. This method was not accepted. The discussion is available at https://edk2.groups.io/g/devel/topic/72655342#56658

The last patch modifying the INF specification and INF parsing are available at:
https://edk2.groups.io/g/devel/topic/72655342#56658
https://edk2.groups.io/g/devel/topic/72656060#56662

Pierre Gondois (3):
  BaseTools: Generate multiple rules when multiple output files
  BaseTools: Rename AmlToHex script to AmlToC
  BaseTools: Compile AML bytecode arrays into .obj file

 BaseTools/BinWrappers/PosixLike/{AmlToHex => AmlToC}               | 28 +++----
 BaseTools/BinWrappers/WindowsLike/{AmlToHex.bat => AmlToC.bat}     |  0
 BaseTools/Conf/build_rule.template                                 | 15 +++-
 BaseTools/Source/Python/{AmlToHex/AmlToHex.py => AmlToC/AmlToC.py} | 82 ++++++++------------
 BaseTools/Source/Python/AutoGen/GenMake.py                         |  6 ++
 BaseTools/Source/Python/AutoGen/ModuleAutoGen.py                   | 40 +++++-----
 6 files changed, 86 insertions(+), 85 deletions(-)  rename BaseTools/BinWrappers/PosixLike/{AmlToHex => AmlToC} (97%)  rename BaseTools/BinWrappers/WindowsLike/{AmlToHex.bat => AmlToC.bat} (100%)  rename BaseTools/Source/Python/{AmlToHex/AmlToHex.py => AmlToC/AmlToC.py} (52%)

--
'Guid(CE165669-3EF3-493F-B85D-6190EE5B9759)'











^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [edk2-devel] [PATCH v1 1/3] BaseTools: Generate multiple rules when multiple output files
  2020-05-18 14:11 ` [PATCH v1 1/3] BaseTools: Generate multiple rules when multiple output files PierreGondois
@ 2020-06-02 11:15   ` Bob Feng
  2020-06-02 13:03     ` PierreGondois
  0 siblings, 1 reply; 17+ messages in thread
From: Bob Feng @ 2020-06-02 11:15 UTC (permalink / raw)
  To: devel@edk2.groups.io, pierre.gondois@arm.com
  Cc: Gao, Liming, Sami.Mujawar@arm.com, Tomas.Pilar@arm.com,
	nd@arm.com

My comments are inline marked as [Bob].

Thanks,
Bob

-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of PierreGondois
Sent: Monday, May 18, 2020 10:11 PM
To: devel@edk2.groups.io
Cc: Pierre Gondois <Pierre.Gondois@arm.com>; Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming <liming.gao@intel.com>; Sami.Mujawar@arm.com; Tomas.Pilar@arm.com; nd@arm.com
Subject: [edk2-devel] [PATCH v1 1/3] BaseTools: Generate multiple rules when multiple output files

From: Pierre Gondois <pierre.gondois@arm.com>

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2425

This patch modifies the Makefile generation not to stop adding Makfile rules when the first final target is found.
E.g.:
If the following rules are described in build_rule.txt:
 -[Rule1]: .X files generate .Y and .Z files;
 -[Rule2]: .Z files generate .Z1 files.
Currently, if a File1.X file was part of the sources of a module, only [Rule1] would be generated in the Makefile.
Indeed, there are no rules to apply to .Y files: .Y files are a final target. However, there is still [Rule2] to apply to .Z files.

[Bob] I think currently a rule's output file will be added back to source file list, and in the later loop, that output file will be handled by another rule.  Doesn't that algorithm handle your case above?

This patch also adds a dependency between the first ouput file of a rule and the other output files.
For instance, with the same example as above, File1.Y and File1.Z are generated by the following rule:
File1.Y: File1.X
    <Generate File1.Y>
    <Generate File1.Z>

and the new dependency is:
File1.Z: File1.Y

This is necessary to keep a dependency order during the execution of the Makefile. Indeed, .Y and .Z files are generated by the execution of a common set of commands, and without this rule, there is no explicit dependency relation between them.

[Bob] If there are 3 output files, for example: 
<OutputFile>
        A
        B
        C
Will the dependency relationship be 
B: A
C: B

Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
---

The changes can be seen at: https://github.com/PierreARM/edk2/tree/803_Compile_AML_bytecode_array_into_OBJ_file_v1

Notes:
    v1:
     - Generate multiple rules when multiple output files
       are specified in the build_rule.txt file. [Pierre]

 BaseTools/Source/Python/AutoGen/GenMake.py       |  6 +++
 BaseTools/Source/Python/AutoGen/ModuleAutoGen.py | 40 ++++++++++----------
 2 files changed, 25 insertions(+), 21 deletions(-)

diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py b/BaseTools/Source/Python/AutoGen/GenMake.py
index bbb3c29446f53fa7f2cb61a216a5b119f72c3fbc..0314d0ea34d99a014379e8d30c46ac0f0a7068ce 100755
--- a/BaseTools/Source/Python/AutoGen/GenMake.py
+++ b/BaseTools/Source/Python/AutoGen/GenMake.py
@@ -1054,6 +1054,12 @@ cleanlib:
                     TargetDict = {"target": self.PlaceMacro(T.Target.Path, self.Macros), "cmd": "\n\t".join(T.Commands),"deps": Deps}
                     self.BuildTargetList.append(self._BUILD_TARGET_TEMPLATE.Replace(TargetDict))
 
+                    # Add a Makefile rule for targets generating multiple files.
+                    # The main output is a prerequisite for the other output files.
+                    for i in T.Outputs[1:]:
+                        AnnexeTargetDict = {"target": self.PlaceMacro(i.Path, self.Macros), "cmd": "", "deps": self.PlaceMacro(T.Target.Path, self.Macros)}
+
+ self.BuildTargetList.append(self._BUILD_TARGET_TEMPLATE.Replace(Annexe
+ TargetDict))
+
     def ParserCCodeFile(self, T, Type, CmdSumDict, CmdTargetDict, CmdCppDict, DependencyDict):
         if not CmdSumDict:
             for item in self._AutoGenObject.Targets[Type]:
diff --git a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
index aad591de65f086043d55aeea5661f59c53792e7c..82facdb2774c56833718317f4414e0c1861de47f 100755
--- a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
@@ -860,7 +860,8 @@ class ModuleAutoGen(AutoGen):
         SubDirectory = os.path.join(self.OutputDir, File.SubDir)
         if not os.path.exists(SubDirectory):
             CreateDirectory(SubDirectory)
-        LastTarget = None
+        TargetList = set()
+        FinalTargetName = set()
         RuleChain = set()
         SourceList = [File]
         Index = 0
@@ -872,7 +873,7 @@ class ModuleAutoGen(AutoGen):
         while Index < len(SourceList):
             Source = SourceList[Index]
             Index = Index + 1
-
+            FileType = TAB_UNKNOWN_FILE
             if Source != File:
                 CreateDirectory(Source.Dir)
 
@@ -881,34 +882,27 @@ class ModuleAutoGen(AutoGen):
                 if not self.IsLibrary:
                     continue
                 RuleObject = self.BuildRules[TAB_DEFAULT_BINARY_FILE]
-            elif FileType in self.BuildRules:
-                RuleObject = self.BuildRules[FileType]
             elif Source.Ext in self.BuildRules:
                 RuleObject = self.BuildRules[Source.Ext]
             else:
-                # stop at no more rules
-                if LastTarget:
-                    self._FinalBuildTargetList.add(LastTarget)
-                break
-
+                # No more rule to apply: Source is a final target.
+                FinalTargetName.add(Source)
+                continue
             FileType = RuleObject.SourceFileType
             self._FileTypes[FileType].add(Source)
 
             # stop at STATIC_LIBRARY for library
             if self.IsLibrary and FileType == TAB_STATIC_LIBRARY:
-                if LastTarget:
-                    self._FinalBuildTargetList.add(LastTarget)
-                break
+                FinalTargetName.add(Source)
+                continue
 
             Target = RuleObject.Apply(Source, self.BuildRuleOrder)
             if not Target:
-                if LastTarget:
-                    self._FinalBuildTargetList.add(LastTarget)
-                break
-            elif not Target.Outputs:
-                # Only do build for target with outputs
-                self._FinalBuildTargetList.add(Target)
+                # No Target: Source is a final target.
+                FinalTargetName.add(Source)
+                continue
 
+            TargetList.add(Target)
             self._BuildTargets[FileType].add(Target)
 
             if not Source.IsBinary and Source == File:
@@ -916,12 +910,16 @@ class ModuleAutoGen(AutoGen):
 
             # to avoid cyclic rule
             if FileType in RuleChain:
-                break
+                EdkLogger.error("build", ERROR_STATEMENT, "Cyclic 
+ dependency detected while generating rule for %s" % str(Source))
 
             RuleChain.add(FileType)
             SourceList.extend(Target.Outputs)
-            LastTarget = Target
-            FileType = TAB_UNKNOWN_FILE
+
+        # For each final target, retrieve the TargetDescBlock instance.
+        for FTargetName in FinalTargetName:
+            for Target in TargetList:
+                if FTargetName == Target.Target:
+                    self._FinalBuildTargetList.add(Target)
 
     @cached_property
     def Targets(self):
--
'Guid(CE165669-3EF3-493F-B85D-6190EE5B9759)'





^ permalink raw reply related	[flat|nested] 17+ messages in thread

* Re: [edk2-devel] [PATCH v1 1/3] BaseTools: Generate multiple rules when multiple output files
  2020-06-02 11:15   ` [edk2-devel] " Bob Feng
@ 2020-06-02 13:03     ` PierreGondois
  2020-06-08 14:00       ` PierreGondois
  0 siblings, 1 reply; 17+ messages in thread
From: PierreGondois @ 2020-06-02 13:03 UTC (permalink / raw)
  To: Feng, Bob C, devel@edk2.groups.io
  Cc: Gao, Liming, Sami Mujawar, Tomas Pilar, nd

Hello Bob,
Thank you for your answer,
I put my comments as [Pierre],

Regards,
Pierre


-----Original Message-----
From: Feng, Bob C <bob.c.feng@intel.com> 
Sent: 02 June 2020 12:16
To: devel@edk2.groups.io; Pierre Gondois <Pierre.Gondois@arm.com>
Cc: Gao, Liming <liming.gao@intel.com>; Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; nd <nd@arm.com>
Subject: RE: [edk2-devel] [PATCH v1 1/3] BaseTools: Generate multiple rules when multiple output files

My comments are inline marked as [Bob].

Thanks,
Bob

-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of PierreGondois
Sent: Monday, May 18, 2020 10:11 PM
To: devel@edk2.groups.io
Cc: Pierre Gondois <Pierre.Gondois@arm.com>; Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming <liming.gao@intel.com>; Sami.Mujawar@arm.com; Tomas.Pilar@arm.com; nd@arm.com
Subject: [edk2-devel] [PATCH v1 1/3] BaseTools: Generate multiple rules when multiple output files

From: Pierre Gondois <pierre.gondois@arm.com>

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2425

This patch modifies the Makefile generation not to stop adding Makfile rules when the first final target is found.
E.g.:
If the following rules are described in build_rule.txt:
 -[Rule1]: .X files generate .Y and .Z files;
 -[Rule2]: .Z files generate .Z1 files.
Currently, if a File1.X file was part of the sources of a module, only [Rule1] would be generated in the Makefile.
Indeed, there are no rules to apply to .Y files: .Y files are a final target. However, there is still [Rule2] to apply to .Z files.

[Bob] I think currently a rule's output file will be added back to source file list, and in the later loop, that output file will be handled by another rule.  Doesn't that algorithm handle your case above?
[Pierre]
The rule's output file was effectively added to the list of source files, and a rule was searched  for this output file. However, the loop stopped when the first final target was found. By final target I mean "a file that isn't the input of a rule".
For the asl/aml/amli case, this meant that:
 -(first loop iteration: treating the ASL file) The rule for ASL files was found and applied to the input ASL file. The AML and ".amli" files were added to the list of source files to look for a rule to apply to them. The loop continues.
 -(second loop iteration: treating the AML file) There is no rule for AML files. This means that the AML file is a final target. The loop ends without having applied the rule on the ".amli" file.
This is why all the "break" instructions of the loop have been replaced with "continue". This prevents the loop from exiting without having treated all the files in the "SourceList".

This patch also adds a dependency between the first ouput file of a rule and the other output files.
For instance, with the same example as above, File1.Y and File1.Z are generated by the following rule:
File1.Y: File1.X
    <Generate File1.Y>
    <Generate File1.Z>

and the new dependency is:
File1.Z: File1.Y

This is necessary to keep a dependency order during the execution of the Makefile. Indeed, .Y and .Z files are generated by the execution of a common set of commands, and without this rule, there is no explicit dependency relation between them.

[Bob] If there are 3 output files, for example: 
<OutputFile>
        A
        B
        C
Will the dependency relationship be
B: A
C: B

[Pierre]
Currently, the dependency relation that would be created is:
B: A
C: A
This can be changed to the dependency you described.

Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
---

The changes can be seen at: https://github.com/PierreARM/edk2/tree/803_Compile_AML_bytecode_array_into_OBJ_file_v1

Notes:
    v1:
     - Generate multiple rules when multiple output files
       are specified in the build_rule.txt file. [Pierre]

 BaseTools/Source/Python/AutoGen/GenMake.py       |  6 +++
 BaseTools/Source/Python/AutoGen/ModuleAutoGen.py | 40 ++++++++++----------
 2 files changed, 25 insertions(+), 21 deletions(-)

diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py b/BaseTools/Source/Python/AutoGen/GenMake.py
index bbb3c29446f53fa7f2cb61a216a5b119f72c3fbc..0314d0ea34d99a014379e8d30c46ac0f0a7068ce 100755
--- a/BaseTools/Source/Python/AutoGen/GenMake.py
+++ b/BaseTools/Source/Python/AutoGen/GenMake.py
@@ -1054,6 +1054,12 @@ cleanlib:
                     TargetDict = {"target": self.PlaceMacro(T.Target.Path, self.Macros), "cmd": "\n\t".join(T.Commands),"deps": Deps}
                     self.BuildTargetList.append(self._BUILD_TARGET_TEMPLATE.Replace(TargetDict))
 
+                    # Add a Makefile rule for targets generating multiple files.
+                    # The main output is a prerequisite for the other output files.
+                    for i in T.Outputs[1:]:
+                        AnnexeTargetDict = {"target": 
+ self.PlaceMacro(i.Path, self.Macros), "cmd": "", "deps": 
+ self.PlaceMacro(T.Target.Path, self.Macros)}
+
+ self.BuildTargetList.append(self._BUILD_TARGET_TEMPLATE.Replace(Annexe
+ TargetDict))
+
     def ParserCCodeFile(self, T, Type, CmdSumDict, CmdTargetDict, CmdCppDict, DependencyDict):
         if not CmdSumDict:
             for item in self._AutoGenObject.Targets[Type]:
diff --git a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
index aad591de65f086043d55aeea5661f59c53792e7c..82facdb2774c56833718317f4414e0c1861de47f 100755
--- a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
@@ -860,7 +860,8 @@ class ModuleAutoGen(AutoGen):
         SubDirectory = os.path.join(self.OutputDir, File.SubDir)
         if not os.path.exists(SubDirectory):
             CreateDirectory(SubDirectory)
-        LastTarget = None
+        TargetList = set()
+        FinalTargetName = set()
         RuleChain = set()
         SourceList = [File]
         Index = 0
@@ -872,7 +873,7 @@ class ModuleAutoGen(AutoGen):
         while Index < len(SourceList):
             Source = SourceList[Index]
             Index = Index + 1
-
+            FileType = TAB_UNKNOWN_FILE
             if Source != File:
                 CreateDirectory(Source.Dir)
 
@@ -881,34 +882,27 @@ class ModuleAutoGen(AutoGen):
                 if not self.IsLibrary:
                     continue
                 RuleObject = self.BuildRules[TAB_DEFAULT_BINARY_FILE]
-            elif FileType in self.BuildRules:
-                RuleObject = self.BuildRules[FileType]
             elif Source.Ext in self.BuildRules:
                 RuleObject = self.BuildRules[Source.Ext]
             else:
-                # stop at no more rules
-                if LastTarget:
-                    self._FinalBuildTargetList.add(LastTarget)
-                break
-
+                # No more rule to apply: Source is a final target.
+                FinalTargetName.add(Source)
+                continue
             FileType = RuleObject.SourceFileType
             self._FileTypes[FileType].add(Source)
 
             # stop at STATIC_LIBRARY for library
             if self.IsLibrary and FileType == TAB_STATIC_LIBRARY:
-                if LastTarget:
-                    self._FinalBuildTargetList.add(LastTarget)
-                break
+                FinalTargetName.add(Source)
+                continue
 
             Target = RuleObject.Apply(Source, self.BuildRuleOrder)
             if not Target:
-                if LastTarget:
-                    self._FinalBuildTargetList.add(LastTarget)
-                break
-            elif not Target.Outputs:
-                # Only do build for target with outputs
-                self._FinalBuildTargetList.add(Target)
+                # No Target: Source is a final target.
+                FinalTargetName.add(Source)
+                continue
 
+            TargetList.add(Target)
             self._BuildTargets[FileType].add(Target)
 
             if not Source.IsBinary and Source == File:
@@ -916,12 +910,16 @@ class ModuleAutoGen(AutoGen):
 
             # to avoid cyclic rule
             if FileType in RuleChain:
-                break
+                EdkLogger.error("build", ERROR_STATEMENT, "Cyclic 
+ dependency detected while generating rule for %s" % str(Source))
 
             RuleChain.add(FileType)
             SourceList.extend(Target.Outputs)
-            LastTarget = Target
-            FileType = TAB_UNKNOWN_FILE
+
+        # For each final target, retrieve the TargetDescBlock instance.
+        for FTargetName in FinalTargetName:
+            for Target in TargetList:
+                if FTargetName == Target.Target:
+                    self._FinalBuildTargetList.add(Target)
 
     @cached_property
     def Targets(self):
--
'Guid(CE165669-3EF3-493F-B85D-6190EE5B9759)'





^ permalink raw reply related	[flat|nested] 17+ messages in thread

* Re: [edk2-devel] [PATCH v1 1/3] BaseTools: Generate multiple rules when multiple output files
  2020-06-02 13:03     ` PierreGondois
@ 2020-06-08 14:00       ` PierreGondois
  2020-06-10  3:58         ` Bob Feng
  0 siblings, 1 reply; 17+ messages in thread
From: PierreGondois @ 2020-06-08 14:00 UTC (permalink / raw)
  To: Feng, Bob C, devel@edk2.groups.io
  Cc: Gao, Liming, Sami Mujawar, Tomas Pilar, nd

Hello Bob,
Should I modify the patch ?

Regards,
Pierre

-----Original Message-----
From: Pierre Gondois 
Sent: Tuesday, June 2, 2020 2:04 PM
To: Feng, Bob C <bob.c.feng@intel.com>; devel@edk2.groups.io
Cc: Gao, Liming <liming.gao@intel.com>; Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; nd <nd@arm.com>
Subject: RE: [edk2-devel] [PATCH v1 1/3] BaseTools: Generate multiple rules when multiple output files

Hello Bob,
Thank you for your answer,
I put my comments as [Pierre],

Regards,
Pierre


-----Original Message-----
From: Feng, Bob C <bob.c.feng@intel.com>
Sent: 02 June 2020 12:16
To: devel@edk2.groups.io; Pierre Gondois <Pierre.Gondois@arm.com>
Cc: Gao, Liming <liming.gao@intel.com>; Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; nd <nd@arm.com>
Subject: RE: [edk2-devel] [PATCH v1 1/3] BaseTools: Generate multiple rules when multiple output files

My comments are inline marked as [Bob].

Thanks,
Bob

-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of PierreGondois
Sent: Monday, May 18, 2020 10:11 PM
To: devel@edk2.groups.io
Cc: Pierre Gondois <Pierre.Gondois@arm.com>; Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming <liming.gao@intel.com>; Sami.Mujawar@arm.com; Tomas.Pilar@arm.com; nd@arm.com
Subject: [edk2-devel] [PATCH v1 1/3] BaseTools: Generate multiple rules when multiple output files

From: Pierre Gondois <pierre.gondois@arm.com>

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2425

This patch modifies the Makefile generation not to stop adding Makfile rules when the first final target is found.
E.g.:
If the following rules are described in build_rule.txt:
 -[Rule1]: .X files generate .Y and .Z files;
 -[Rule2]: .Z files generate .Z1 files.
Currently, if a File1.X file was part of the sources of a module, only [Rule1] would be generated in the Makefile.
Indeed, there are no rules to apply to .Y files: .Y files are a final target. However, there is still [Rule2] to apply to .Z files.

[Bob] I think currently a rule's output file will be added back to source file list, and in the later loop, that output file will be handled by another rule.  Doesn't that algorithm handle your case above?
[Pierre]
The rule's output file was effectively added to the list of source files, and a rule was searched  for this output file. However, the loop stopped when the first final target was found. By final target I mean "a file that isn't the input of a rule".
For the asl/aml/amli case, this meant that:
 -(first loop iteration: treating the ASL file) The rule for ASL files was found and applied to the input ASL file. The AML and ".amli" files were added to the list of source files to look for a rule to apply to them. The loop continues.
 -(second loop iteration: treating the AML file) There is no rule for AML files. This means that the AML file is a final target. The loop ends without having applied the rule on the ".amli" file.
This is why all the "break" instructions of the loop have been replaced with "continue". This prevents the loop from exiting without having treated all the files in the "SourceList".

This patch also adds a dependency between the first ouput file of a rule and the other output files.
For instance, with the same example as above, File1.Y and File1.Z are generated by the following rule:
File1.Y: File1.X
    <Generate File1.Y>
    <Generate File1.Z>

and the new dependency is:
File1.Z: File1.Y

This is necessary to keep a dependency order during the execution of the Makefile. Indeed, .Y and .Z files are generated by the execution of a common set of commands, and without this rule, there is no explicit dependency relation between them.

[Bob] If there are 3 output files, for example: 
<OutputFile>
        A
        B
        C
Will the dependency relationship be
B: A
C: B

[Pierre]
Currently, the dependency relation that would be created is:
B: A
C: A
This can be changed to the dependency you described.

Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
---

The changes can be seen at: https://github.com/PierreARM/edk2/tree/803_Compile_AML_bytecode_array_into_OBJ_file_v1

Notes:
    v1:
     - Generate multiple rules when multiple output files
       are specified in the build_rule.txt file. [Pierre]

 BaseTools/Source/Python/AutoGen/GenMake.py       |  6 +++
 BaseTools/Source/Python/AutoGen/ModuleAutoGen.py | 40 ++++++++++----------
 2 files changed, 25 insertions(+), 21 deletions(-)

diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py b/BaseTools/Source/Python/AutoGen/GenMake.py
index bbb3c29446f53fa7f2cb61a216a5b119f72c3fbc..0314d0ea34d99a014379e8d30c46ac0f0a7068ce 100755
--- a/BaseTools/Source/Python/AutoGen/GenMake.py
+++ b/BaseTools/Source/Python/AutoGen/GenMake.py
@@ -1054,6 +1054,12 @@ cleanlib:
                     TargetDict = {"target": self.PlaceMacro(T.Target.Path, self.Macros), "cmd": "\n\t".join(T.Commands),"deps": Deps}
                     self.BuildTargetList.append(self._BUILD_TARGET_TEMPLATE.Replace(TargetDict))
 
+                    # Add a Makefile rule for targets generating multiple files.
+                    # The main output is a prerequisite for the other output files.
+                    for i in T.Outputs[1:]:
+                        AnnexeTargetDict = {"target": 
+ self.PlaceMacro(i.Path, self.Macros), "cmd": "", "deps": 
+ self.PlaceMacro(T.Target.Path, self.Macros)}
+
+ self.BuildTargetList.append(self._BUILD_TARGET_TEMPLATE.Replace(Annexe
+ TargetDict))
+
     def ParserCCodeFile(self, T, Type, CmdSumDict, CmdTargetDict, CmdCppDict, DependencyDict):
         if not CmdSumDict:
             for item in self._AutoGenObject.Targets[Type]:
diff --git a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
index aad591de65f086043d55aeea5661f59c53792e7c..82facdb2774c56833718317f4414e0c1861de47f 100755
--- a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
@@ -860,7 +860,8 @@ class ModuleAutoGen(AutoGen):
         SubDirectory = os.path.join(self.OutputDir, File.SubDir)
         if not os.path.exists(SubDirectory):
             CreateDirectory(SubDirectory)
-        LastTarget = None
+        TargetList = set()
+        FinalTargetName = set()
         RuleChain = set()
         SourceList = [File]
         Index = 0
@@ -872,7 +873,7 @@ class ModuleAutoGen(AutoGen):
         while Index < len(SourceList):
             Source = SourceList[Index]
             Index = Index + 1
-
+            FileType = TAB_UNKNOWN_FILE
             if Source != File:
                 CreateDirectory(Source.Dir)
 
@@ -881,34 +882,27 @@ class ModuleAutoGen(AutoGen):
                 if not self.IsLibrary:
                     continue
                 RuleObject = self.BuildRules[TAB_DEFAULT_BINARY_FILE]
-            elif FileType in self.BuildRules:
-                RuleObject = self.BuildRules[FileType]
             elif Source.Ext in self.BuildRules:
                 RuleObject = self.BuildRules[Source.Ext]
             else:
-                # stop at no more rules
-                if LastTarget:
-                    self._FinalBuildTargetList.add(LastTarget)
-                break
-
+                # No more rule to apply: Source is a final target.
+                FinalTargetName.add(Source)
+                continue
             FileType = RuleObject.SourceFileType
             self._FileTypes[FileType].add(Source)
 
             # stop at STATIC_LIBRARY for library
             if self.IsLibrary and FileType == TAB_STATIC_LIBRARY:
-                if LastTarget:
-                    self._FinalBuildTargetList.add(LastTarget)
-                break
+                FinalTargetName.add(Source)
+                continue
 
             Target = RuleObject.Apply(Source, self.BuildRuleOrder)
             if not Target:
-                if LastTarget:
-                    self._FinalBuildTargetList.add(LastTarget)
-                break
-            elif not Target.Outputs:
-                # Only do build for target with outputs
-                self._FinalBuildTargetList.add(Target)
+                # No Target: Source is a final target.
+                FinalTargetName.add(Source)
+                continue
 
+            TargetList.add(Target)
             self._BuildTargets[FileType].add(Target)
 
             if not Source.IsBinary and Source == File:
@@ -916,12 +910,16 @@ class ModuleAutoGen(AutoGen):
 
             # to avoid cyclic rule
             if FileType in RuleChain:
-                break
+                EdkLogger.error("build", ERROR_STATEMENT, "Cyclic 
+ dependency detected while generating rule for %s" % str(Source))
 
             RuleChain.add(FileType)
             SourceList.extend(Target.Outputs)
-            LastTarget = Target
-            FileType = TAB_UNKNOWN_FILE
+
+        # For each final target, retrieve the TargetDescBlock instance.
+        for FTargetName in FinalTargetName:
+            for Target in TargetList:
+                if FTargetName == Target.Target:
+                    self._FinalBuildTargetList.add(Target)
 
     @cached_property
     def Targets(self):
--
'Guid(CE165669-3EF3-493F-B85D-6190EE5B9759)'





^ permalink raw reply related	[flat|nested] 17+ messages in thread

* Re: [edk2-devel] [PATCH v1 1/3] BaseTools: Generate multiple rules when multiple output files
  2020-06-08 14:00       ` PierreGondois
@ 2020-06-10  3:58         ` Bob Feng
  2020-06-15 15:57           ` PierreGondois
  0 siblings, 1 reply; 17+ messages in thread
From: Bob Feng @ 2020-06-10  3:58 UTC (permalink / raw)
  To: Pierre Gondois, devel@edk2.groups.io
  Cc: Gao, Liming, Sami Mujawar, Tomas Pilar, nd

Hi Pierre,

I found there is an incorrect target generated in the OvmfPkg/AcpiTables Makefile when I tried to build Ovmf. That incorrect target causes ovmf to build failed.


$(DEBUG_DIR)\PlatformAcpiTables : $(MAKE_FILE)
$(DEBUG_DIR)\PlatformAcpiTables : $(STATIC_LIBRARY_FILES)
$(DEBUG_DIR)\PlatformAcpiTables : $(STATIC_LIBRARY_FILES_LIST)
	"$(DLINK)" $(DLINK_FLAGS) $(DLINK_SPATH) @$(STATIC_LIBRARY_FILES_LIST)

Regarding this patch, I think it changes the logic of the _ApplyBuildRule, replacing "break" with "continue" and removing some if and elseif blocks, so it would be hard for me to make sure your current logic can cover all the original cases.  Would you show me how many testing you have done?

Thanks,
Bob 

-----Original Message-----
From: Pierre Gondois <Pierre.Gondois@arm.com> 
Sent: Monday, June 8, 2020 10:01 PM
To: Feng, Bob C <bob.c.feng@intel.com>; devel@edk2.groups.io
Cc: Gao, Liming <liming.gao@intel.com>; Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; nd <nd@arm.com>
Subject: RE: [edk2-devel] [PATCH v1 1/3] BaseTools: Generate multiple rules when multiple output files

Hello Bob,
Should I modify the patch ?

Regards,
Pierre

-----Original Message-----
From: Pierre Gondois
Sent: Tuesday, June 2, 2020 2:04 PM
To: Feng, Bob C <bob.c.feng@intel.com>; devel@edk2.groups.io
Cc: Gao, Liming <liming.gao@intel.com>; Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; nd <nd@arm.com>
Subject: RE: [edk2-devel] [PATCH v1 1/3] BaseTools: Generate multiple rules when multiple output files

Hello Bob,
Thank you for your answer,
I put my comments as [Pierre],

Regards,
Pierre


-----Original Message-----
From: Feng, Bob C <bob.c.feng@intel.com>
Sent: 02 June 2020 12:16
To: devel@edk2.groups.io; Pierre Gondois <Pierre.Gondois@arm.com>
Cc: Gao, Liming <liming.gao@intel.com>; Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; nd <nd@arm.com>
Subject: RE: [edk2-devel] [PATCH v1 1/3] BaseTools: Generate multiple rules when multiple output files

My comments are inline marked as [Bob].

Thanks,
Bob

-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of PierreGondois
Sent: Monday, May 18, 2020 10:11 PM
To: devel@edk2.groups.io
Cc: Pierre Gondois <Pierre.Gondois@arm.com>; Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming <liming.gao@intel.com>; Sami.Mujawar@arm.com; Tomas.Pilar@arm.com; nd@arm.com
Subject: [edk2-devel] [PATCH v1 1/3] BaseTools: Generate multiple rules when multiple output files

From: Pierre Gondois <pierre.gondois@arm.com>

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2425

This patch modifies the Makefile generation not to stop adding Makfile rules when the first final target is found.
E.g.:
If the following rules are described in build_rule.txt:
 -[Rule1]: .X files generate .Y and .Z files;
 -[Rule2]: .Z files generate .Z1 files.
Currently, if a File1.X file was part of the sources of a module, only [Rule1] would be generated in the Makefile.
Indeed, there are no rules to apply to .Y files: .Y files are a final target. However, there is still [Rule2] to apply to .Z files.

[Bob] I think currently a rule's output file will be added back to source file list, and in the later loop, that output file will be handled by another rule.  Doesn't that algorithm handle your case above?
[Pierre]
The rule's output file was effectively added to the list of source files, and a rule was searched  for this output file. However, the loop stopped when the first final target was found. By final target I mean "a file that isn't the input of a rule".
For the asl/aml/amli case, this meant that:
 -(first loop iteration: treating the ASL file) The rule for ASL files was found and applied to the input ASL file. The AML and ".amli" files were added to the list of source files to look for a rule to apply to them. The loop continues.
 -(second loop iteration: treating the AML file) There is no rule for AML files. This means that the AML file is a final target. The loop ends without having applied the rule on the ".amli" file.
This is why all the "break" instructions of the loop have been replaced with "continue". This prevents the loop from exiting without having treated all the files in the "SourceList".

This patch also adds a dependency between the first ouput file of a rule and the other output files.
For instance, with the same example as above, File1.Y and File1.Z are generated by the following rule:
File1.Y: File1.X
    <Generate File1.Y>
    <Generate File1.Z>

and the new dependency is:
File1.Z: File1.Y

This is necessary to keep a dependency order during the execution of the Makefile. Indeed, .Y and .Z files are generated by the execution of a common set of commands, and without this rule, there is no explicit dependency relation between them.

[Bob] If there are 3 output files, for example: 
<OutputFile>
        A
        B
        C
Will the dependency relationship be
B: A
C: B

[Pierre]
Currently, the dependency relation that would be created is:
B: A
C: A
This can be changed to the dependency you described.

Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
---

The changes can be seen at: https://github.com/PierreARM/edk2/tree/803_Compile_AML_bytecode_array_into_OBJ_file_v1

Notes:
    v1:
     - Generate multiple rules when multiple output files
       are specified in the build_rule.txt file. [Pierre]

 BaseTools/Source/Python/AutoGen/GenMake.py       |  6 +++
 BaseTools/Source/Python/AutoGen/ModuleAutoGen.py | 40 ++++++++++----------
 2 files changed, 25 insertions(+), 21 deletions(-)

diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py b/BaseTools/Source/Python/AutoGen/GenMake.py
index bbb3c29446f53fa7f2cb61a216a5b119f72c3fbc..0314d0ea34d99a014379e8d30c46ac0f0a7068ce 100755
--- a/BaseTools/Source/Python/AutoGen/GenMake.py
+++ b/BaseTools/Source/Python/AutoGen/GenMake.py
@@ -1054,6 +1054,12 @@ cleanlib:
                     TargetDict = {"target": self.PlaceMacro(T.Target.Path, self.Macros), "cmd": "\n\t".join(T.Commands),"deps": Deps}
                     self.BuildTargetList.append(self._BUILD_TARGET_TEMPLATE.Replace(TargetDict))
 
+                    # Add a Makefile rule for targets generating multiple files.
+                    # The main output is a prerequisite for the other output files.
+                    for i in T.Outputs[1:]:
+                        AnnexeTargetDict = {"target": 
+ self.PlaceMacro(i.Path, self.Macros), "cmd": "", "deps": 
+ self.PlaceMacro(T.Target.Path, self.Macros)}
+
+ self.BuildTargetList.append(self._BUILD_TARGET_TEMPLATE.Replace(Annexe
+ TargetDict))
+
     def ParserCCodeFile(self, T, Type, CmdSumDict, CmdTargetDict, CmdCppDict, DependencyDict):
         if not CmdSumDict:
             for item in self._AutoGenObject.Targets[Type]:
diff --git a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
index aad591de65f086043d55aeea5661f59c53792e7c..82facdb2774c56833718317f4414e0c1861de47f 100755
--- a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
@@ -860,7 +860,8 @@ class ModuleAutoGen(AutoGen):
         SubDirectory = os.path.join(self.OutputDir, File.SubDir)
         if not os.path.exists(SubDirectory):
             CreateDirectory(SubDirectory)
-        LastTarget = None
+        TargetList = set()
+        FinalTargetName = set()
         RuleChain = set()
         SourceList = [File]
         Index = 0
@@ -872,7 +873,7 @@ class ModuleAutoGen(AutoGen):
         while Index < len(SourceList):
             Source = SourceList[Index]
             Index = Index + 1
-
+            FileType = TAB_UNKNOWN_FILE
             if Source != File:
                 CreateDirectory(Source.Dir)
 
@@ -881,34 +882,27 @@ class ModuleAutoGen(AutoGen):
                 if not self.IsLibrary:
                     continue
                 RuleObject = self.BuildRules[TAB_DEFAULT_BINARY_FILE]
-            elif FileType in self.BuildRules:
-                RuleObject = self.BuildRules[FileType]
             elif Source.Ext in self.BuildRules:
                 RuleObject = self.BuildRules[Source.Ext]
             else:
-                # stop at no more rules
-                if LastTarget:
-                    self._FinalBuildTargetList.add(LastTarget)
-                break
-
+                # No more rule to apply: Source is a final target.
+                FinalTargetName.add(Source)
+                continue
             FileType = RuleObject.SourceFileType
             self._FileTypes[FileType].add(Source)
 
             # stop at STATIC_LIBRARY for library
             if self.IsLibrary and FileType == TAB_STATIC_LIBRARY:
-                if LastTarget:
-                    self._FinalBuildTargetList.add(LastTarget)
-                break
+                FinalTargetName.add(Source)
+                continue
 
             Target = RuleObject.Apply(Source, self.BuildRuleOrder)
             if not Target:
-                if LastTarget:
-                    self._FinalBuildTargetList.add(LastTarget)
-                break
-            elif not Target.Outputs:
-                # Only do build for target with outputs
-                self._FinalBuildTargetList.add(Target)
+                # No Target: Source is a final target.
+                FinalTargetName.add(Source)
+                continue
 
+            TargetList.add(Target)
             self._BuildTargets[FileType].add(Target)
 
             if not Source.IsBinary and Source == File:
@@ -916,12 +910,16 @@ class ModuleAutoGen(AutoGen):
 
             # to avoid cyclic rule
             if FileType in RuleChain:
-                break
+                EdkLogger.error("build", ERROR_STATEMENT, "Cyclic 
+ dependency detected while generating rule for %s" % str(Source))
 
             RuleChain.add(FileType)
             SourceList.extend(Target.Outputs)
-            LastTarget = Target
-            FileType = TAB_UNKNOWN_FILE
+
+        # For each final target, retrieve the TargetDescBlock instance.
+        for FTargetName in FinalTargetName:
+            for Target in TargetList:
+                if FTargetName == Target.Target:
+                    self._FinalBuildTargetList.add(Target)
 
     @cached_property
     def Targets(self):
--
'Guid(CE165669-3EF3-493F-B85D-6190EE5B9759)'





^ permalink raw reply related	[flat|nested] 17+ messages in thread

* Re: [edk2-devel] [PATCH v1 0/3] Compile AML bytecode array into OBJ file
  2020-05-18 14:11 [PATCH v1 0/3] Compile AML bytecode array into OBJ file PierreGondois
                   ` (4 preceding siblings ...)
       [not found] ` <161025E2E32EBB17.23434@groups.io>
@ 2020-06-13  9:37 ` Ard Biesheuvel
  2020-06-15 16:08   ` PierreGondois
  5 siblings, 1 reply; 17+ messages in thread
From: Ard Biesheuvel @ 2020-06-13  9:37 UTC (permalink / raw)
  To: devel, pierre.gondois
  Cc: bob.c.feng, liming.gao, Sami.Mujawar, Tomas.Pilar, nd

On 5/18/20 4:11 PM, PierreGondois via groups.io wrote:
> Following the BZ at https://bugzilla.tianocore.org/show_bug.cgi?id=2425
> This patch serie is a another way to solve the dependency
> of C files over ASL files. With this new method, the
> dependency is resolved at the linking stage.
> 
> The last method to solve this dependency was to add
> the possibility to modify INF files to depict such a
> dependency. This method was not accepted. The discussion
> is available at https://edk2.groups.io/g/devel/topic/72655342#56658
> 
> The last patch modifying the INF specification and INF
> parsing are available at:
> https://edk2.groups.io/g/devel/topic/72655342#56658
> https://edk2.groups.io/g/devel/topic/72656060#56662
> 
> Pierre Gondois (3):
>    BaseTools: Generate multiple rules when multiple output files
>    BaseTools: Rename AmlToHex script to AmlToC
>    BaseTools: Compile AML bytecode arrays into .obj file
> 


Hello Pierre,

I have no comments on this series, but I noticed that AmlToHex was 
checked into the repository with CRLF line endings:

$ file BaseTools/BinWrappers/PosixLike/AmlToHex
Bourne-Again shell script, ASCII text executable, with CRLF line terminators

resulting in Build output like

Intel ACPI Component Architecture
ASL+ Optimizing Compiler/Disassembler version 20181213
Copyright (c) 2000 - 2018 Intel Corporation

ASL Input: 
/home/ard/build/edk2-workspace/Build/ArmVirtQemuKernel-ARM/RELEASE_CLANG38/ARM/MdeModulePkg/Universal/Disk/RamDiskDxe/RamDiskDxe/OUTPUT/./RamDisk.iiii 
- 42 lines, 305 bytes, 6 keywords
AML Output: 
/home/ard/build/edk2-workspace/Build/ArmVirtQemuKernel-ARM/RELEASE_CLANG38/ARM/MdeModulePkg/Universal/Disk/RamDiskDxe/RamDiskDxe/OUTPUT/./RamDisk.aml 
- 124 bytes, 5 named objects, 1 executable opcodes

Compilation complete. 0 Errors, 0 Warnings, 0 Remarks, 0 Optimizations
/usr/bin/env: ‘bash\r’: No such file or directory


Note that this platform has no use for the hex-dumped output of the AML, 
so it would be better if we only call AmlToHex when necessary.

Thanks,
Ard.

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [edk2-devel] [PATCH v1 1/3] BaseTools: Generate multiple rules when multiple output files
  2020-06-10  3:58         ` Bob Feng
@ 2020-06-15 15:57           ` PierreGondois
  2020-06-16  8:48             ` Bob Feng
  0 siblings, 1 reply; 17+ messages in thread
From: PierreGondois @ 2020-06-15 15:57 UTC (permalink / raw)
  To: Feng, Bob C, devel@edk2.groups.io
  Cc: Gao, Liming, Sami Mujawar, Tomas Pilar, nd

Hello Bob,
I have locally corrected the patch. On non-arm architecture, I had previously tested it for the following configurations, but this wasn't enough apparently. I am currently testing it on more platforms and I will send you the list of the tested platforms along with the V2.
Plarform="Platform/Intel/MinPlatformPkg/MinPlatformPkg.dsc"|"Platform/Intel/BoardModulePkg/BoardModulePkg.dsc"
  Host=Windows|Linux
    BuildType=DEBUG|RELEASE
      Toolchain=VS2017|GCC5
        Arch=X64|IA32
Before submitting a V2, I am planning to create a pull request on the edk2 github repository, allowing me to check whether it passes the CI tests. Would it be fine for you?

About your comment about changing the logic of ApplyBuildRule, the logic of _ApplyBuildRule currently breaks out of the loop whenever a final target is found. The new rule on ".amli" files needs to continue looking for rules to apply, otherwise the processing on the initial ASL file stops when the AML file is generated (thus, not continuing the path ".amli"->".c"->".obj"). I agree this is hard to check, but I believe the edk2 CI tests should allow to put more trust in the patch serie.

Regards,
Pierre

-----Original Message-----
From: Feng, Bob C <bob.c.feng@intel.com> 
Sent: 10 June 2020 04:58
To: Pierre Gondois <Pierre.Gondois@arm.com>; devel@edk2.groups.io
Cc: Gao, Liming <liming.gao@intel.com>; Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; nd <nd@arm.com>
Subject: RE: [edk2-devel] [PATCH v1 1/3] BaseTools: Generate multiple rules when multiple output files

Hi Pierre,

I found there is an incorrect target generated in the OvmfPkg/AcpiTables Makefile when I tried to build Ovmf. That incorrect target causes ovmf to build failed.


$(DEBUG_DIR)\PlatformAcpiTables : $(MAKE_FILE) $(DEBUG_DIR)\PlatformAcpiTables : $(STATIC_LIBRARY_FILES) $(DEBUG_DIR)\PlatformAcpiTables : $(STATIC_LIBRARY_FILES_LIST)
	"$(DLINK)" $(DLINK_FLAGS) $(DLINK_SPATH) @$(STATIC_LIBRARY_FILES_LIST)

Regarding this patch, I think it changes the logic of the _ApplyBuildRule, replacing "break" with "continue" and removing some if and elseif blocks, so it would be hard for me to make sure your current logic can cover all the original cases.  Would you show me how many testing you have done?

Thanks,
Bob 

-----Original Message-----
From: Pierre Gondois <Pierre.Gondois@arm.com>
Sent: Monday, June 8, 2020 10:01 PM
To: Feng, Bob C <bob.c.feng@intel.com>; devel@edk2.groups.io
Cc: Gao, Liming <liming.gao@intel.com>; Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; nd <nd@arm.com>
Subject: RE: [edk2-devel] [PATCH v1 1/3] BaseTools: Generate multiple rules when multiple output files

Hello Bob,
Should I modify the patch ?

Regards,
Pierre

-----Original Message-----
From: Pierre Gondois
Sent: Tuesday, June 2, 2020 2:04 PM
To: Feng, Bob C <bob.c.feng@intel.com>; devel@edk2.groups.io
Cc: Gao, Liming <liming.gao@intel.com>; Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; nd <nd@arm.com>
Subject: RE: [edk2-devel] [PATCH v1 1/3] BaseTools: Generate multiple rules when multiple output files

Hello Bob,
Thank you for your answer,
I put my comments as [Pierre],

Regards,
Pierre


-----Original Message-----
From: Feng, Bob C <bob.c.feng@intel.com>
Sent: 02 June 2020 12:16
To: devel@edk2.groups.io; Pierre Gondois <Pierre.Gondois@arm.com>
Cc: Gao, Liming <liming.gao@intel.com>; Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; nd <nd@arm.com>
Subject: RE: [edk2-devel] [PATCH v1 1/3] BaseTools: Generate multiple rules when multiple output files

My comments are inline marked as [Bob].

Thanks,
Bob

-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of PierreGondois
Sent: Monday, May 18, 2020 10:11 PM
To: devel@edk2.groups.io
Cc: Pierre Gondois <Pierre.Gondois@arm.com>; Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming <liming.gao@intel.com>; Sami.Mujawar@arm.com; Tomas.Pilar@arm.com; nd@arm.com
Subject: [edk2-devel] [PATCH v1 1/3] BaseTools: Generate multiple rules when multiple output files

From: Pierre Gondois <pierre.gondois@arm.com>

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2425

This patch modifies the Makefile generation not to stop adding Makfile rules when the first final target is found.
E.g.:
If the following rules are described in build_rule.txt:
 -[Rule1]: .X files generate .Y and .Z files;
 -[Rule2]: .Z files generate .Z1 files.
Currently, if a File1.X file was part of the sources of a module, only [Rule1] would be generated in the Makefile.
Indeed, there are no rules to apply to .Y files: .Y files are a final target. However, there is still [Rule2] to apply to .Z files.

[Bob] I think currently a rule's output file will be added back to source file list, and in the later loop, that output file will be handled by another rule.  Doesn't that algorithm handle your case above?
[Pierre]
The rule's output file was effectively added to the list of source files, and a rule was searched  for this output file. However, the loop stopped when the first final target was found. By final target I mean "a file that isn't the input of a rule".
For the asl/aml/amli case, this meant that:
 -(first loop iteration: treating the ASL file) The rule for ASL files was found and applied to the input ASL file. The AML and ".amli" files were added to the list of source files to look for a rule to apply to them. The loop continues.
 -(second loop iteration: treating the AML file) There is no rule for AML files. This means that the AML file is a final target. The loop ends without having applied the rule on the ".amli" file.
This is why all the "break" instructions of the loop have been replaced with "continue". This prevents the loop from exiting without having treated all the files in the "SourceList".

This patch also adds a dependency between the first ouput file of a rule and the other output files.
For instance, with the same example as above, File1.Y and File1.Z are generated by the following rule:
File1.Y: File1.X
    <Generate File1.Y>
    <Generate File1.Z>

and the new dependency is:
File1.Z: File1.Y

This is necessary to keep a dependency order during the execution of the Makefile. Indeed, .Y and .Z files are generated by the execution of a common set of commands, and without this rule, there is no explicit dependency relation between them.

[Bob] If there are 3 output files, for example: 
<OutputFile>
        A
        B
        C
Will the dependency relationship be
B: A
C: B

[Pierre]
Currently, the dependency relation that would be created is:
B: A
C: A
This can be changed to the dependency you described.

Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
---

The changes can be seen at: https://github.com/PierreARM/edk2/tree/803_Compile_AML_bytecode_array_into_OBJ_file_v1

Notes:
    v1:
     - Generate multiple rules when multiple output files
       are specified in the build_rule.txt file. [Pierre]

 BaseTools/Source/Python/AutoGen/GenMake.py       |  6 +++
 BaseTools/Source/Python/AutoGen/ModuleAutoGen.py | 40 ++++++++++----------
 2 files changed, 25 insertions(+), 21 deletions(-)

diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py b/BaseTools/Source/Python/AutoGen/GenMake.py
index bbb3c29446f53fa7f2cb61a216a5b119f72c3fbc..0314d0ea34d99a014379e8d30c46ac0f0a7068ce 100755
--- a/BaseTools/Source/Python/AutoGen/GenMake.py
+++ b/BaseTools/Source/Python/AutoGen/GenMake.py
@@ -1054,6 +1054,12 @@ cleanlib:
                     TargetDict = {"target": self.PlaceMacro(T.Target.Path, self.Macros), "cmd": "\n\t".join(T.Commands),"deps": Deps}
                     self.BuildTargetList.append(self._BUILD_TARGET_TEMPLATE.Replace(TargetDict))
 
+                    # Add a Makefile rule for targets generating multiple files.
+                    # The main output is a prerequisite for the other output files.
+                    for i in T.Outputs[1:]:
+                        AnnexeTargetDict = {"target": 
+ self.PlaceMacro(i.Path, self.Macros), "cmd": "", "deps": 
+ self.PlaceMacro(T.Target.Path, self.Macros)}
+
+ self.BuildTargetList.append(self._BUILD_TARGET_TEMPLATE.Replace(Annexe
+ TargetDict))
+
     def ParserCCodeFile(self, T, Type, CmdSumDict, CmdTargetDict, CmdCppDict, DependencyDict):
         if not CmdSumDict:
             for item in self._AutoGenObject.Targets[Type]:
diff --git a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
index aad591de65f086043d55aeea5661f59c53792e7c..82facdb2774c56833718317f4414e0c1861de47f 100755
--- a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
@@ -860,7 +860,8 @@ class ModuleAutoGen(AutoGen):
         SubDirectory = os.path.join(self.OutputDir, File.SubDir)
         if not os.path.exists(SubDirectory):
             CreateDirectory(SubDirectory)
-        LastTarget = None
+        TargetList = set()
+        FinalTargetName = set()
         RuleChain = set()
         SourceList = [File]
         Index = 0
@@ -872,7 +873,7 @@ class ModuleAutoGen(AutoGen):
         while Index < len(SourceList):
             Source = SourceList[Index]
             Index = Index + 1
-
+            FileType = TAB_UNKNOWN_FILE
             if Source != File:
                 CreateDirectory(Source.Dir)
 
@@ -881,34 +882,27 @@ class ModuleAutoGen(AutoGen):
                 if not self.IsLibrary:
                     continue
                 RuleObject = self.BuildRules[TAB_DEFAULT_BINARY_FILE]
-            elif FileType in self.BuildRules:
-                RuleObject = self.BuildRules[FileType]
             elif Source.Ext in self.BuildRules:
                 RuleObject = self.BuildRules[Source.Ext]
             else:
-                # stop at no more rules
-                if LastTarget:
-                    self._FinalBuildTargetList.add(LastTarget)
-                break
-
+                # No more rule to apply: Source is a final target.
+                FinalTargetName.add(Source)
+                continue
             FileType = RuleObject.SourceFileType
             self._FileTypes[FileType].add(Source)
 
             # stop at STATIC_LIBRARY for library
             if self.IsLibrary and FileType == TAB_STATIC_LIBRARY:
-                if LastTarget:
-                    self._FinalBuildTargetList.add(LastTarget)
-                break
+                FinalTargetName.add(Source)
+                continue
 
             Target = RuleObject.Apply(Source, self.BuildRuleOrder)
             if not Target:
-                if LastTarget:
-                    self._FinalBuildTargetList.add(LastTarget)
-                break
-            elif not Target.Outputs:
-                # Only do build for target with outputs
-                self._FinalBuildTargetList.add(Target)
+                # No Target: Source is a final target.
+                FinalTargetName.add(Source)
+                continue
 
+            TargetList.add(Target)
             self._BuildTargets[FileType].add(Target)
 
             if not Source.IsBinary and Source == File:
@@ -916,12 +910,16 @@ class ModuleAutoGen(AutoGen):
 
             # to avoid cyclic rule
             if FileType in RuleChain:
-                break
+                EdkLogger.error("build", ERROR_STATEMENT, "Cyclic 
+ dependency detected while generating rule for %s" % str(Source))
 
             RuleChain.add(FileType)
             SourceList.extend(Target.Outputs)
-            LastTarget = Target
-            FileType = TAB_UNKNOWN_FILE
+
+        # For each final target, retrieve the TargetDescBlock instance.
+        for FTargetName in FinalTargetName:
+            for Target in TargetList:
+                if FTargetName == Target.Target:
+                    self._FinalBuildTargetList.add(Target)
 
     @cached_property
     def Targets(self):
--
'Guid(CE165669-3EF3-493F-B85D-6190EE5B9759)'





^ permalink raw reply related	[flat|nested] 17+ messages in thread

* Re: [edk2-devel] [PATCH v1 0/3] Compile AML bytecode array into OBJ file
  2020-06-13  9:37 ` Ard Biesheuvel
@ 2020-06-15 16:08   ` PierreGondois
  0 siblings, 0 replies; 17+ messages in thread
From: PierreGondois @ 2020-06-15 16:08 UTC (permalink / raw)
  To: Ard Biesheuvel, devel@edk2.groups.io
  Cc: bob.c.feng@intel.com, liming.gao@intel.com, Sami Mujawar,
	Tomas Pilar, nd

Hello Ard and Bob,

I also noticed the line ending is incorrect for BaseTools/BinWrappers/PosixLike/AmlToHex. The patch "[PATCH v1 2/3] BaseTools: Rename AmlToHex script to AmlToC" is correcting the line ending. This script is generating an optional output file and doesn't prevent edk2 from building, due to the dash present at the beginning of the command line in Conf/build_rule.txt.
Do you want the line ending to be fixed as soon as possible, or can it wait a bit to go with this patch serie?

About the fact that the file generated by AmlToHex is not need by all platforms:
A solution would be to add an "enable" flag for this script. The script would not generate any output file unless this "enable" flag is provided to the script. The platforms willing to have the file generated by AmlToHex would provide it.
However this would add one more flag to the build system, in the Conf/tools_def.txt file.

Regards,
Pierre

-----Original Message-----
From: Ard Biesheuvel <ard.biesheuvel@arm.com> 
Sent: 13 June 2020 10:38
To: devel@edk2.groups.io; Pierre Gondois <Pierre.Gondois@arm.com>
Cc: bob.c.feng@intel.com; liming.gao@intel.com; Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; nd <nd@arm.com>
Subject: Re: [edk2-devel] [PATCH v1 0/3] Compile AML bytecode array into OBJ file

On 5/18/20 4:11 PM, PierreGondois via groups.io wrote:
> Following the BZ at 
> https://bugzilla.tianocore.org/show_bug.cgi?id=2425
> This patch serie is a another way to solve the dependency of C files 
> over ASL files. With this new method, the dependency is resolved at 
> the linking stage.
> 
> The last method to solve this dependency was to add the possibility to 
> modify INF files to depict such a dependency. This method was not 
> accepted. The discussion is available at 
> https://edk2.groups.io/g/devel/topic/72655342#56658
> 
> The last patch modifying the INF specification and INF parsing are 
> available at:
> https://edk2.groups.io/g/devel/topic/72655342#56658
> https://edk2.groups.io/g/devel/topic/72656060#56662
> 
> Pierre Gondois (3):
>    BaseTools: Generate multiple rules when multiple output files
>    BaseTools: Rename AmlToHex script to AmlToC
>    BaseTools: Compile AML bytecode arrays into .obj file
> 


Hello Pierre,

I have no comments on this series, but I noticed that AmlToHex was checked into the repository with CRLF line endings:

$ file BaseTools/BinWrappers/PosixLike/AmlToHex
Bourne-Again shell script, ASCII text executable, with CRLF line terminators

resulting in Build output like

Intel ACPI Component Architecture
ASL+ Optimizing Compiler/Disassembler version 20181213
Copyright (c) 2000 - 2018 Intel Corporation

ASL Input: 
/home/ard/build/edk2-workspace/Build/ArmVirtQemuKernel-ARM/RELEASE_CLANG38/ARM/MdeModulePkg/Universal/Disk/RamDiskDxe/RamDiskDxe/OUTPUT/./RamDisk.iiii
- 42 lines, 305 bytes, 6 keywords
AML Output: 
/home/ard/build/edk2-workspace/Build/ArmVirtQemuKernel-ARM/RELEASE_CLANG38/ARM/MdeModulePkg/Universal/Disk/RamDiskDxe/RamDiskDxe/OUTPUT/./RamDisk.aml
- 124 bytes, 5 named objects, 1 executable opcodes

Compilation complete. 0 Errors, 0 Warnings, 0 Remarks, 0 Optimizations
/usr/bin/env: ‘bash\r’: No such file or directory


Note that this platform has no use for the hex-dumped output of the AML, so it would be better if we only call AmlToHex when necessary.

Thanks,
Ard.

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [edk2-devel] [PATCH v1 1/3] BaseTools: Generate multiple rules when multiple output files
  2020-06-15 15:57           ` PierreGondois
@ 2020-06-16  8:48             ` Bob Feng
  0 siblings, 0 replies; 17+ messages in thread
From: Bob Feng @ 2020-06-16  8:48 UTC (permalink / raw)
  To: Pierre Gondois, devel@edk2.groups.io
  Cc: Gao, Liming, Sami Mujawar, Tomas Pilar, nd

Hi Pierre,

Yes. It's fine for me. 


Thanks,
Bob

-----Original Message-----
From: Pierre Gondois <Pierre.Gondois@arm.com> 
Sent: Monday, June 15, 2020 11:58 PM
To: Feng, Bob C <bob.c.feng@intel.com>; devel@edk2.groups.io
Cc: Gao, Liming <liming.gao@intel.com>; Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; nd <nd@arm.com>
Subject: RE: [edk2-devel] [PATCH v1 1/3] BaseTools: Generate multiple rules when multiple output files

Hello Bob,
I have locally corrected the patch. On non-arm architecture, I had previously tested it for the following configurations, but this wasn't enough apparently. I am currently testing it on more platforms and I will send you the list of the tested platforms along with the V2.
Plarform="Platform/Intel/MinPlatformPkg/MinPlatformPkg.dsc"|"Platform/Intel/BoardModulePkg/BoardModulePkg.dsc"
  Host=Windows|Linux
    BuildType=DEBUG|RELEASE
      Toolchain=VS2017|GCC5
        Arch=X64|IA32
Before submitting a V2, I am planning to create a pull request on the edk2 github repository, allowing me to check whether it passes the CI tests. Would it be fine for you?

About your comment about changing the logic of ApplyBuildRule, the logic of _ApplyBuildRule currently breaks out of the loop whenever a final target is found. The new rule on ".amli" files needs to continue looking for rules to apply, otherwise the processing on the initial ASL file stops when the AML file is generated (thus, not continuing the path ".amli"->".c"->".obj"). I agree this is hard to check, but I believe the edk2 CI tests should allow to put more trust in the patch serie.

Regards,
Pierre

-----Original Message-----
From: Feng, Bob C <bob.c.feng@intel.com>
Sent: 10 June 2020 04:58
To: Pierre Gondois <Pierre.Gondois@arm.com>; devel@edk2.groups.io
Cc: Gao, Liming <liming.gao@intel.com>; Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; nd <nd@arm.com>
Subject: RE: [edk2-devel] [PATCH v1 1/3] BaseTools: Generate multiple rules when multiple output files

Hi Pierre,

I found there is an incorrect target generated in the OvmfPkg/AcpiTables Makefile when I tried to build Ovmf. That incorrect target causes ovmf to build failed.


$(DEBUG_DIR)\PlatformAcpiTables : $(MAKE_FILE) $(DEBUG_DIR)\PlatformAcpiTables : $(STATIC_LIBRARY_FILES) $(DEBUG_DIR)\PlatformAcpiTables : $(STATIC_LIBRARY_FILES_LIST)
	"$(DLINK)" $(DLINK_FLAGS) $(DLINK_SPATH) @$(STATIC_LIBRARY_FILES_LIST)

Regarding this patch, I think it changes the logic of the _ApplyBuildRule, replacing "break" with "continue" and removing some if and elseif blocks, so it would be hard for me to make sure your current logic can cover all the original cases.  Would you show me how many testing you have done?

Thanks,
Bob 

-----Original Message-----
From: Pierre Gondois <Pierre.Gondois@arm.com>
Sent: Monday, June 8, 2020 10:01 PM
To: Feng, Bob C <bob.c.feng@intel.com>; devel@edk2.groups.io
Cc: Gao, Liming <liming.gao@intel.com>; Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; nd <nd@arm.com>
Subject: RE: [edk2-devel] [PATCH v1 1/3] BaseTools: Generate multiple rules when multiple output files

Hello Bob,
Should I modify the patch ?

Regards,
Pierre

-----Original Message-----
From: Pierre Gondois
Sent: Tuesday, June 2, 2020 2:04 PM
To: Feng, Bob C <bob.c.feng@intel.com>; devel@edk2.groups.io
Cc: Gao, Liming <liming.gao@intel.com>; Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; nd <nd@arm.com>
Subject: RE: [edk2-devel] [PATCH v1 1/3] BaseTools: Generate multiple rules when multiple output files

Hello Bob,
Thank you for your answer,
I put my comments as [Pierre],

Regards,
Pierre


-----Original Message-----
From: Feng, Bob C <bob.c.feng@intel.com>
Sent: 02 June 2020 12:16
To: devel@edk2.groups.io; Pierre Gondois <Pierre.Gondois@arm.com>
Cc: Gao, Liming <liming.gao@intel.com>; Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; nd <nd@arm.com>
Subject: RE: [edk2-devel] [PATCH v1 1/3] BaseTools: Generate multiple rules when multiple output files

My comments are inline marked as [Bob].

Thanks,
Bob

-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of PierreGondois
Sent: Monday, May 18, 2020 10:11 PM
To: devel@edk2.groups.io
Cc: Pierre Gondois <Pierre.Gondois@arm.com>; Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming <liming.gao@intel.com>; Sami.Mujawar@arm.com; Tomas.Pilar@arm.com; nd@arm.com
Subject: [edk2-devel] [PATCH v1 1/3] BaseTools: Generate multiple rules when multiple output files

From: Pierre Gondois <pierre.gondois@arm.com>

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2425

This patch modifies the Makefile generation not to stop adding Makfile rules when the first final target is found.
E.g.:
If the following rules are described in build_rule.txt:
 -[Rule1]: .X files generate .Y and .Z files;
 -[Rule2]: .Z files generate .Z1 files.
Currently, if a File1.X file was part of the sources of a module, only [Rule1] would be generated in the Makefile.
Indeed, there are no rules to apply to .Y files: .Y files are a final target. However, there is still [Rule2] to apply to .Z files.

[Bob] I think currently a rule's output file will be added back to source file list, and in the later loop, that output file will be handled by another rule.  Doesn't that algorithm handle your case above?
[Pierre]
The rule's output file was effectively added to the list of source files, and a rule was searched  for this output file. However, the loop stopped when the first final target was found. By final target I mean "a file that isn't the input of a rule".
For the asl/aml/amli case, this meant that:
 -(first loop iteration: treating the ASL file) The rule for ASL files was found and applied to the input ASL file. The AML and ".amli" files were added to the list of source files to look for a rule to apply to them. The loop continues.
 -(second loop iteration: treating the AML file) There is no rule for AML files. This means that the AML file is a final target. The loop ends without having applied the rule on the ".amli" file.
This is why all the "break" instructions of the loop have been replaced with "continue". This prevents the loop from exiting without having treated all the files in the "SourceList".

This patch also adds a dependency between the first ouput file of a rule and the other output files.
For instance, with the same example as above, File1.Y and File1.Z are generated by the following rule:
File1.Y: File1.X
    <Generate File1.Y>
    <Generate File1.Z>

and the new dependency is:
File1.Z: File1.Y

This is necessary to keep a dependency order during the execution of the Makefile. Indeed, .Y and .Z files are generated by the execution of a common set of commands, and without this rule, there is no explicit dependency relation between them.

[Bob] If there are 3 output files, for example: 
<OutputFile>
        A
        B
        C
Will the dependency relationship be
B: A
C: B

[Pierre]
Currently, the dependency relation that would be created is:
B: A
C: A
This can be changed to the dependency you described.

Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
---

The changes can be seen at: https://github.com/PierreARM/edk2/tree/803_Compile_AML_bytecode_array_into_OBJ_file_v1

Notes:
    v1:
     - Generate multiple rules when multiple output files
       are specified in the build_rule.txt file. [Pierre]

 BaseTools/Source/Python/AutoGen/GenMake.py       |  6 +++
 BaseTools/Source/Python/AutoGen/ModuleAutoGen.py | 40 ++++++++++----------
 2 files changed, 25 insertions(+), 21 deletions(-)

diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py b/BaseTools/Source/Python/AutoGen/GenMake.py
index bbb3c29446f53fa7f2cb61a216a5b119f72c3fbc..0314d0ea34d99a014379e8d30c46ac0f0a7068ce 100755
--- a/BaseTools/Source/Python/AutoGen/GenMake.py
+++ b/BaseTools/Source/Python/AutoGen/GenMake.py
@@ -1054,6 +1054,12 @@ cleanlib:
                     TargetDict = {"target": self.PlaceMacro(T.Target.Path, self.Macros), "cmd": "\n\t".join(T.Commands),"deps": Deps}
                     self.BuildTargetList.append(self._BUILD_TARGET_TEMPLATE.Replace(TargetDict))
 
+                    # Add a Makefile rule for targets generating multiple files.
+                    # The main output is a prerequisite for the other output files.
+                    for i in T.Outputs[1:]:
+                        AnnexeTargetDict = {"target": 
+ self.PlaceMacro(i.Path, self.Macros), "cmd": "", "deps": 
+ self.PlaceMacro(T.Target.Path, self.Macros)}
+
+ self.BuildTargetList.append(self._BUILD_TARGET_TEMPLATE.Replace(Annexe
+ TargetDict))
+
     def ParserCCodeFile(self, T, Type, CmdSumDict, CmdTargetDict, CmdCppDict, DependencyDict):
         if not CmdSumDict:
             for item in self._AutoGenObject.Targets[Type]:
diff --git a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
index aad591de65f086043d55aeea5661f59c53792e7c..82facdb2774c56833718317f4414e0c1861de47f 100755
--- a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
@@ -860,7 +860,8 @@ class ModuleAutoGen(AutoGen):
         SubDirectory = os.path.join(self.OutputDir, File.SubDir)
         if not os.path.exists(SubDirectory):
             CreateDirectory(SubDirectory)
-        LastTarget = None
+        TargetList = set()
+        FinalTargetName = set()
         RuleChain = set()
         SourceList = [File]
         Index = 0
@@ -872,7 +873,7 @@ class ModuleAutoGen(AutoGen):
         while Index < len(SourceList):
             Source = SourceList[Index]
             Index = Index + 1
-
+            FileType = TAB_UNKNOWN_FILE
             if Source != File:
                 CreateDirectory(Source.Dir)
 
@@ -881,34 +882,27 @@ class ModuleAutoGen(AutoGen):
                 if not self.IsLibrary:
                     continue
                 RuleObject = self.BuildRules[TAB_DEFAULT_BINARY_FILE]
-            elif FileType in self.BuildRules:
-                RuleObject = self.BuildRules[FileType]
             elif Source.Ext in self.BuildRules:
                 RuleObject = self.BuildRules[Source.Ext]
             else:
-                # stop at no more rules
-                if LastTarget:
-                    self._FinalBuildTargetList.add(LastTarget)
-                break
-
+                # No more rule to apply: Source is a final target.
+                FinalTargetName.add(Source)
+                continue
             FileType = RuleObject.SourceFileType
             self._FileTypes[FileType].add(Source)
 
             # stop at STATIC_LIBRARY for library
             if self.IsLibrary and FileType == TAB_STATIC_LIBRARY:
-                if LastTarget:
-                    self._FinalBuildTargetList.add(LastTarget)
-                break
+                FinalTargetName.add(Source)
+                continue
 
             Target = RuleObject.Apply(Source, self.BuildRuleOrder)
             if not Target:
-                if LastTarget:
-                    self._FinalBuildTargetList.add(LastTarget)
-                break
-            elif not Target.Outputs:
-                # Only do build for target with outputs
-                self._FinalBuildTargetList.add(Target)
+                # No Target: Source is a final target.
+                FinalTargetName.add(Source)
+                continue
 
+            TargetList.add(Target)
             self._BuildTargets[FileType].add(Target)
 
             if not Source.IsBinary and Source == File:
@@ -916,12 +910,16 @@ class ModuleAutoGen(AutoGen):
 
             # to avoid cyclic rule
             if FileType in RuleChain:
-                break
+                EdkLogger.error("build", ERROR_STATEMENT, "Cyclic 
+ dependency detected while generating rule for %s" % str(Source))
 
             RuleChain.add(FileType)
             SourceList.extend(Target.Outputs)
-            LastTarget = Target
-            FileType = TAB_UNKNOWN_FILE
+
+        # For each final target, retrieve the TargetDescBlock instance.
+        for FTargetName in FinalTargetName:
+            for Target in TargetList:
+                if FTargetName == Target.Target:
+                    self._FinalBuildTargetList.add(Target)
 
     @cached_property
     def Targets(self):
--
'Guid(CE165669-3EF3-493F-B85D-6190EE5B9759)'





^ permalink raw reply related	[flat|nested] 17+ messages in thread

end of thread, other threads:[~2020-06-16  8:48 UTC | newest]

Thread overview: 17+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2020-05-18 14:11 [PATCH v1 0/3] Compile AML bytecode array into OBJ file PierreGondois
2020-05-18 14:11 ` [PATCH v1 1/3] BaseTools: Generate multiple rules when multiple output files PierreGondois
2020-06-02 11:15   ` [edk2-devel] " Bob Feng
2020-06-02 13:03     ` PierreGondois
2020-06-08 14:00       ` PierreGondois
2020-06-10  3:58         ` Bob Feng
2020-06-15 15:57           ` PierreGondois
2020-06-16  8:48             ` Bob Feng
2020-05-18 14:11 ` [PATCH v1 2/3] BaseTools: Rename AmlToHex script to AmlToC PierreGondois
2020-05-18 14:11 ` [PATCH v1 3/3] BaseTools: Compile AML bytecode arrays into .obj file PierreGondois
2020-05-18 14:31 ` [PATCH v1 0/3] Compile AML bytecode array into OBJ file PierreGondois
     [not found] ` <161025E2E32EBB17.23434@groups.io>
2020-05-22 12:01   ` [edk2-devel] " PierreGondois
     [not found]   ` <161157FAC70B55F8.21259@groups.io>
2020-05-28 11:03     ` PierreGondois
2020-05-28 11:10       ` Bob Feng
2020-06-02 11:15       ` Bob Feng
2020-06-13  9:37 ` Ard Biesheuvel
2020-06-15 16:08   ` PierreGondois

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox