Re: [edk2-devel] [PATCH v2 3/4] BaseTools: Compile AML bytecode arrays into .obj file

["Bob Feng" <bob.c.feng@intel.com>]

Hi Pierre,

The commit message should not have more than one Signed-off-by.

Use of Suggested-by may be more appropriate here.

Thanks,
Bob

-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of PierreGondois
Sent: Thursday, June 18, 2020 10:27 PM
To: devel@edk2.groups.io
Cc: Pierre Gondois <Pierre.Gondois@arm.com>; sami.mujawar@arm.com; tomas.pilar@arm.com; Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming <liming.gao@intel.com>; nd@arm.com
Subject: [edk2-devel] [PATCH v2 3/4] BaseTools: Compile AML bytecode arrays into .obj file

From: Pierre Gondois <pierre.gondois@arm.com>

The AmlToHex script and Posix/WindowsLike wrappers convert an AML file to a .hex file, containing a C array storing AML bytecode. This ".hex" file can then be included in a C file, allowing to access the AML bytecode from this C file.

The EDK2 build system doesn't allow to a depict dependency orders between files of different languages. For instance, in a module containing a ".c" file and a ".asl", the ".c"
file may or may not be built prior to the ".asl" file.
This prevents any inclusion of a generated ".hex" in a ".c" file since this later ".hex" file may or may not have been created yet.

This patch modifies the AmlToC script to generate a C file instead of a ".hex" file.
It also adds the generation of an intermediate ".amli" file when compiling an ASL file, and adds a rule to convert this ".amli" to a C file.

This allows to generate a C file containing the AML bytecode from an ASL file. This C file will then be handled by the EDK2 build system to generate an object file.
Thus, no file inclusion will be required anymore. The C file requiring the AML bytecode as a C array, and the ASL file, will be compiled independently. The C array must be defined as an external symbol. The linker is resolving the reference to the C array symbol.

To summarize, the flow goes as:
 -1. ASL file is compiled to AML;
 -2. AML file is copied to a ".amli" intermediate file;  -3. EDK2 build system applies the rule relevant to ".amli"
     files. This is, calling the "AmlToC" script, generating
     a C file from the ".amli" file;
 -4. EDK2 build system applies the rule relevant to C files.
     This is creating an object file.
 -5. EDK2 build system links the object file containing the
     AML bytecode with the object file requiring it.

Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
Signed-off-by: Tomas Pilar <Tomas.Pilar@arm.com>
---

The changes can be seen at https://github.com/PierreARM/edk2/commits/pg/803_Compile_AML_bytecode_array_into_OBJ_file_v2

Notes:
    Notes:
      v1:
       - Add a new rule to the build_rule.template file to
         generate ".obj" files from .asl files, and modify
         the AmlToC script accordingly. [Pierre]
      v2:
       - Restrict the rule to DXE_DRIVER. This allows to build
         the OvmfPkg, which was not the case in v1. [Pierre]

 BaseTools/Conf/build_rule.template       | 15 +++-
 BaseTools/Source/Python/AmlToC/AmlToC.py | 82 ++++++++------------
 2 files changed, 47 insertions(+), 50 deletions(-)

diff --git a/BaseTools/Conf/build_rule.template b/BaseTools/Conf/build_rule.template
index 0822b681fcd9f61c6508e6f93ffc31fa70fd7059..c034869915914936e28f64a6aadba08e0169da44 100755
--- a/BaseTools/Conf/build_rule.template
+++ b/BaseTools/Conf/build_rule.template
@@ -419,6 +419,7 @@
 
     <OutputFile>
         $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.aml
+        $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.amli
 
     <ExtraDependency>
         $(MAKE_FILE)
@@ -428,14 +429,24 @@
         "$(ASLPP)" $(DEPS_FLAGS) $(ASLPP_FLAGS) $(INC) /I${s_path} $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.i > $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.iii
         Trim --source-code -l -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.iiii $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.iii 
         "$(ASL)" $(ASL_FLAGS) $(ASL_OUTFLAGS)${dst} $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.iiii
-        -AmlToHex $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.aml
+        $(CP) $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.aml 
+ $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.amli
 
     <Command.GCC>
         Trim --asl-file --asl-deps -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.i -i $(INC_LIST) ${src}
         "$(ASLPP)" $(DEPS_FLAGS) $(ASLPP_FLAGS) $(INC) -I${s_path} $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.i > $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.iii
         Trim --source-code -l -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.iiii $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.iii 
         "$(ASL)" $(ASL_FLAGS) $(ASL_OUTFLAGS)${dst} $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.iiii
-        -AmlToHex $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.aml
+        $(CP) $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.aml 
+ $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.amli
+
+[Acpi-Machine-Language-File-to-C.DXE_DRIVER]
+    <InputFile>
+        ?.amli
+
+    <OutputFile>
+        ${s_path}(+)${s_base}.c
+
+    <Command>
+        -AmlToC ${src}
 
 [C-Code-File.AcpiTable]
     <InputFile>
diff --git a/BaseTools/Source/Python/AmlToC/AmlToC.py b/BaseTools/Source/Python/AmlToC/AmlToC.py
index 643db2910e37acfdd80ac18d288c921320a79ce1..346de7159de702d860bbd809ddbe8175f1493cfb 100644
--- a/BaseTools/Source/Python/AmlToC/AmlToC.py
+++ b/BaseTools/Source/Python/AmlToC/AmlToC.py
@@ -1,9 +1,9 @@
 ## @file
 #
-# Convert an AML file to a .hex file containing the AML bytecode stored in a
+# Convert an AML file to a .c file containing the AML bytecode stored 
+in a
 # C array.
-# By default, "Tables\Dsdt.aml" will generate "Tables\Dsdt.hex".
-# "Tables\Dsdt.hex" will contain a C array named "dsdt_aml_code" that contains
+# By default, "Tables\Dsdt.aml" will generate "Tables\Dsdt.c".
+# "Tables\Dsdt.c" will contain a C array named "dsdt_aml_code" that 
+contains
 # the AML bytecode.
 #
 # Copyright (c) 2020, ARM Limited. All rights reserved.<BR> @@ -17,31 +17,26 @@ from Common.BuildToolError import *  import sys  import os
 
+__description__ = """
+Convert an AML file to a .c file containing the AML bytecode stored in 
+a C array. By default, Tables\Dsdt.aml will generate Tables\Dsdt.c.
+Tables\Dsdt.c will contain a C array named "dsdt_aml_code" that 
+contains the AML bytecode.
+"""
+
 ## Parse the command line arguments.
 #
 # @retval A argparse.NameSpace instance, containing parsed values.
 #
 def ParseArgs():
     # Initialize the parser.
-    Parser = argparse.ArgumentParser(
-      description="Convert an AML file to a .hex file containing the AML " + \
-                  "bytecode stored in a C array. By default, " + \
-                  "\"Tables\\Dsdt.aml\" will generate" + \
-                  "\"Tables\\Dsdt.hex\". \"Tables\\Dsdt.hex\" will " + \
-                  "contain a C array named \"dsdt_aml_code\" that " + \
-                  "contains the AML bytecode."
-      )
+    Parser = argparse.ArgumentParser(description=__description__)
 
     # Define the possible arguments.
-    Parser.add_argument(
-      dest="InputFile",
-      help="Path to an input AML file to generate a .hex file from."
-      )
-    Parser.add_argument(
-      "-o", "--out-dir", dest="OutDir",
-      help="Output directory where the .hex file will be generated. " + \
-           "Default is the input file's directory."
-      )
+    Parser.add_argument(dest="InputFile",
+                        help="Path to an input AML file to generate a .c file from.")
+    Parser.add_argument("-o", "--out-dir", dest="OutDir",
+                        help="Output directory where the .c file will 
+ be generated. Default is the input file's directory.")
 
     # Parse the input arguments.
     Args = Parser.parse_args()
@@ -55,9 +50,7 @@ def ParseArgs():
         with open(Args.InputFile, "rb") as fIn:
             Signature = str(fIn.read(4))
             if ("DSDT" not in Signature) and ("SSDT" not in Signature):
-                EdkLogger.info("Invalid file type. " + \
-                                "File does not have a valid " + \
-                                "DSDT or SSDT signature: %s" % Args.InputFile)
+                EdkLogger.info("Invalid file type. File does not have a 
+ valid DSDT or SSDT signature: {}".format(Args.InputFile))
                 return None
 
     # Get the basename of the input file.
@@ -66,42 +59,39 @@ def ParseArgs():
 
     # If no output directory is specified, output to the input directory.
     if not Args.OutDir:
-        Args.OutputFile = os.path.join(
-          os.path.dirname(Args.InputFile),
-          BaseName + ".hex"
-          )
+        Args.OutputFile = os.path.join(os.path.dirname(Args.InputFile),
+                                       BaseName + ".c")
     else:
         if not os.path.exists(Args.OutDir):
             os.mkdir(Args.OutDir)
-        Args.OutputFile = os.path.join(Args.OutDir, BaseName + ".hex")
+        Args.OutputFile = os.path.join(Args.OutDir, BaseName + ".c")
 
     Args.BaseName = BaseName
 
     return Args
 
-## Convert an AML file to a .hex file containing the AML bytecode stored
+## Convert an AML file to a .c file containing the AML bytecode stored
 #  in a C array.
 #
 # @param  InputFile     Path to the input AML file.
-# @param  OutputFile    Path to the output .hex file to generate.
+# @param  OutputFile    Path to the output .c file to generate.
 # @param  BaseName      Base name of the input file.
-#                       This is also the name of the generated .hex file.
+#                       This is also the name of the generated .c file.
 #
-def AmlToHex(InputFile, OutputFile, BaseName):
+def AmlToC(InputFile, OutputFile, BaseName):
 
-    MacroName = "__{}_HEX__".format(BaseName.upper())
     ArrayName = BaseName.lower() + "_aml_code"
+    FileHeader =\
+"""
+// This file has been generated from:
+//   -Python script: {}
+//   -Input AML file: {}
+
+"""
 
     with open(InputFile, "rb") as fIn, open(OutputFile, "w") as fOut:
         # Write header.
-        fOut.write("// This file has been generated from:\n" + \
-                   "// \tPython script: " + \
-                   os.path.abspath(__file__) + "\n" + \
-                   "// \tInput AML file: " + \
-                   os.path.abspath(InputFile) + "\n\n" + \
-                   "#ifndef {}\n".format(MacroName) + \
-                   "#define {}\n\n".format(MacroName)
-                   )
+        fOut.write(FileHeader.format(os.path.abspath(InputFile), 
+ os.path.abspath(__file__)))
 
         # Write the array and its content.
         fOut.write("unsigned char {}[] = {{\n  ".format(ArrayName)) @@ -115,15 +105,12 @@ def AmlToHex(InputFile, OutputFile, BaseName):
             byte = fIn.read(1)
         fOut.write("\n};\n")
 
-        # Write footer.
-        fOut.write("#endif // {}\n".format(MacroName))
-
 ## Main method
 #
 # This method:
 #   1-  Initialize an EdkLogger instance.
 #   2-  Parses the input arguments.
-#   3-  Converts an AML file to a .hex file containing the AML bytecode stored
+#   3-  Converts an AML file to a .c file containing the AML bytecode stored
 #       in a C array.
 #
 # @retval 0     Success.
@@ -139,10 +126,9 @@ def Main():
         if not CommandArguments:
             return 1
 
-        # Convert an AML file to a .hex file containing the AML bytecode stored
+        # Convert an AML file to a .c file containing the AML bytecode 
+ stored
         # in a C array.
-        AmlToHex(CommandArguments.InputFile, CommandArguments.OutputFile,
-                          CommandArguments.BaseName)
+        AmlToC(CommandArguments.InputFile, CommandArguments.OutputFile, 
+ CommandArguments.BaseName)
     except Exception as e:
         print(e)
         return 1
--
'Guid(CE165669-3EF3-493F-B85D-6190EE5B9759)'