* [PATCH v3 0/4] Compile AML bytecode array into OBJ file
@ 2020-06-24 9:08 PierreGondois
2020-06-24 9:08 ` [PATCH v3 1/4] BaseTools: Generate multiple rules when multiple output files PierreGondois
` (4 more replies)
0 siblings, 5 replies; 14+ messages in thread
From: PierreGondois @ 2020-06-24 9:08 UTC (permalink / raw)
To: devel; +Cc: Pierre Gondois, sami.mujawar, tomas.pilar, bob.c.feng, liming.gao,
nd
Following the BZ at https://bugzilla.tianocore.org/show_bug.cgi?id=2425
This patch serie is a another way to solve the dependency
of C files over ASL files. With this new method, the
dependency is resolved at the linking stage.
The last method to solve this dependency was to add
the possibility to modify INF files to depict such a
dependency. This method was not accepted. The discussion
is available at https://edk2.groups.io/g/devel/topic/72655342#56658
The last patch modifying the INF specification and INF
parsing are available at:
https://edk2.groups.io/g/devel/topic/72655342#56658
https://edk2.groups.io/g/devel/topic/72656060#56662
Pierre Gondois (4):
BaseTools: Generate multiple rules when multiple output files
BaseTools: Rename AmlToHex script to AmlToC
BaseTools: Compile AML bytecode arrays into .obj file
BaseTools: Fix string concatenation
BaseTools/BinWrappers/PosixLike/{AmlToHex => AmlToC} | 28 +++----
BaseTools/BinWrappers/WindowsLike/{AmlToHex.bat => AmlToC.bat} | 0
BaseTools/Conf/build_rule.template | 15 +++-
BaseTools/Source/Python/{AmlToHex/AmlToHex.py => AmlToC/AmlToC.py} | 82 ++++++++------------
BaseTools/Source/Python/AutoGen/BuildEngine.py | 2 +-
BaseTools/Source/Python/AutoGen/GenMake.py | 6 ++
BaseTools/Source/Python/AutoGen/ModuleAutoGen.py | 38 +++++----
7 files changed, 89 insertions(+), 82 deletions(-)
rename BaseTools/BinWrappers/PosixLike/{AmlToHex => AmlToC} (97%)
rename BaseTools/BinWrappers/WindowsLike/{AmlToHex.bat => AmlToC.bat} (100%)
rename BaseTools/Source/Python/{AmlToHex/AmlToHex.py => AmlToC/AmlToC.py} (52%)
--
'Guid(CE165669-3EF3-493F-B85D-6190EE5B9759)'
^ permalink raw reply [flat|nested] 14+ messages in thread* [PATCH v3 1/4] BaseTools: Generate multiple rules when multiple output files 2020-06-24 9:08 [PATCH v3 0/4] Compile AML bytecode array into OBJ file PierreGondois @ 2020-06-24 9:08 ` PierreGondois 2020-06-24 9:08 ` [PATCH v3 2/4] BaseTools: Rename AmlToHex script to AmlToC PierreGondois ` (3 subsequent siblings) 4 siblings, 0 replies; 14+ messages in thread From: PierreGondois @ 2020-06-24 9:08 UTC (permalink / raw) To: devel; +Cc: Pierre Gondois, sami.mujawar, tomas.pilar, bob.c.feng, liming.gao, nd From: Pierre Gondois <pierre.gondois@arm.com> This patch modifies the Makefile generation not to stop adding Makfile rules when the first final target is found. E.g.: If the following rules are described in build_rule.txt: -[Rule1]: .X files generate .Y and .Z files; -[Rule2]: .Z files generate .Z1 files. Currently, if a File1.X file was part of the sources of a module, only [Rule1] would be generated in the Makefile. Indeed, there are no rules to apply to .Y files: .Y files are a final target. However, there is still [Rule2] to apply to .Z files. This patch also adds a dependency between the first ouput file of a rule and the other output files. For instance, with the same example as above, File1.Y and File1.Z are generated by the following rule: File1.Y: File1.X <Generate File1.Y> <Generate File1.Z> and the new dependency is: File1.Z: File1.Y This is necessary to keep a dependency order during the execution of the Makefile. Indeed, .Y and .Z files are generated by the execution of a common set of commands, and without this rule, there is no explicit dependency relation between them. Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com> Suggested-by: Tomas Pilar <Tomas.Pilar@arm.com> --- The changes can be seen at https://github.com/PierreARM/edk2/commits/803_Compile_AML_bytecode_array_into_OBJ_file_v3 Notes: Notes: v1: - Generate multiple rules when multiple output files are specified in the build_rule.txt file. [Pierre] v2: - Use the "FileType" variable in the _ApplyBuildRule function as it is in the current state. [Pierre] v3: - Adding Suggested-by [Pierre] BaseTools/Source/Python/AutoGen/GenMake.py | 6 ++++ BaseTools/Source/Python/AutoGen/ModuleAutoGen.py | 38 +++++++++++--------- 2 files changed, 27 insertions(+), 17 deletions(-) diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py b/BaseTools/Source/Python/AutoGen/GenMake.py index bbb3c29446f53fa7f2cb61a216a5b119f72c3fbc..0314d0ea34d99a014379e8d30c46ac0f0a7068ce 100755 --- a/BaseTools/Source/Python/AutoGen/GenMake.py +++ b/BaseTools/Source/Python/AutoGen/GenMake.py @@ -1054,6 +1054,12 @@ cleanlib: TargetDict = {"target": self.PlaceMacro(T.Target.Path, self.Macros), "cmd": "\n\t".join(T.Commands),"deps": Deps} self.BuildTargetList.append(self._BUILD_TARGET_TEMPLATE.Replace(TargetDict)) + # Add a Makefile rule for targets generating multiple files. + # The main output is a prerequisite for the other output files. + for i in T.Outputs[1:]: + AnnexeTargetDict = {"target": self.PlaceMacro(i.Path, self.Macros), "cmd": "", "deps": self.PlaceMacro(T.Target.Path, self.Macros)} + self.BuildTargetList.append(self._BUILD_TARGET_TEMPLATE.Replace(AnnexeTargetDict)) + def ParserCCodeFile(self, T, Type, CmdSumDict, CmdTargetDict, CmdCppDict, DependencyDict): if not CmdSumDict: for item in self._AutoGenObject.Targets[Type]: diff --git a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py index aad591de65f086043d55aeea5661f59c53792e7c..dc8b1fe3d160cac2da22227fc233e3aa0d92cb1e 100755 --- a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py +++ b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py @@ -860,7 +860,8 @@ class ModuleAutoGen(AutoGen): SubDirectory = os.path.join(self.OutputDir, File.SubDir) if not os.path.exists(SubDirectory): CreateDirectory(SubDirectory) - LastTarget = None + TargetList = set() + FinalTargetName = set() RuleChain = set() SourceList = [File] Index = 0 @@ -870,6 +871,9 @@ class ModuleAutoGen(AutoGen): self.BuildOption while Index < len(SourceList): + # Reset the FileType if not the first iteration. + if Index > 0: + FileType = TAB_UNKNOWN_FILE Source = SourceList[Index] Index = Index + 1 @@ -886,29 +890,25 @@ class ModuleAutoGen(AutoGen): elif Source.Ext in self.BuildRules: RuleObject = self.BuildRules[Source.Ext] else: - # stop at no more rules - if LastTarget: - self._FinalBuildTargetList.add(LastTarget) - break + # No more rule to apply: Source is a final target. + FinalTargetName.add(Source) + continue FileType = RuleObject.SourceFileType self._FileTypes[FileType].add(Source) # stop at STATIC_LIBRARY for library if self.IsLibrary and FileType == TAB_STATIC_LIBRARY: - if LastTarget: - self._FinalBuildTargetList.add(LastTarget) - break + FinalTargetName.add(Source) + continue Target = RuleObject.Apply(Source, self.BuildRuleOrder) if not Target: - if LastTarget: - self._FinalBuildTargetList.add(LastTarget) - break - elif not Target.Outputs: - # Only do build for target with outputs - self._FinalBuildTargetList.add(Target) + # No Target: Source is a final target. + FinalTargetName.add(Source) + continue + TargetList.add(Target) self._BuildTargets[FileType].add(Target) if not Source.IsBinary and Source == File: @@ -916,12 +916,16 @@ class ModuleAutoGen(AutoGen): # to avoid cyclic rule if FileType in RuleChain: - break + EdkLogger.error("build", ERROR_STATEMENT, "Cyclic dependency detected while generating rule for %s" % str(Source)) RuleChain.add(FileType) SourceList.extend(Target.Outputs) - LastTarget = Target - FileType = TAB_UNKNOWN_FILE + + # For each final target name, retrieve the corresponding TargetDescBlock instance. + for FTargetName in FinalTargetName: + for Target in TargetList: + if FTargetName == Target.Target: + self._FinalBuildTargetList.add(Target) @cached_property def Targets(self): -- 'Guid(CE165669-3EF3-493F-B85D-6190EE5B9759)' ^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH v3 2/4] BaseTools: Rename AmlToHex script to AmlToC 2020-06-24 9:08 [PATCH v3 0/4] Compile AML bytecode array into OBJ file PierreGondois 2020-06-24 9:08 ` [PATCH v3 1/4] BaseTools: Generate multiple rules when multiple output files PierreGondois @ 2020-06-24 9:08 ` PierreGondois 2020-06-24 9:08 ` [PATCH v3 3/4] BaseTools: Compile AML bytecode arrays into .obj file PierreGondois ` (2 subsequent siblings) 4 siblings, 0 replies; 14+ messages in thread From: PierreGondois @ 2020-06-24 9:08 UTC (permalink / raw) To: devel; +Cc: Pierre Gondois, sami.mujawar, tomas.pilar, bob.c.feng, liming.gao, nd From: Pierre Gondois <pierre.gondois@arm.com> The AmlToHex script and Posix/WindowsLike wrappers convert an AML file to a .hex file, containing a C array storing AML bytecode. This ".hex" file can then be included in a C file, allowing to access the AML bytecode from this C file. The EDK2 build system doesn't allow to a depict dependency orders between files of different languages. For instance, in a module containing a ".c" file and a ".asl", the ".c" file may or may not be built prior to the ".asl" file. This prevents any inclusion of a generated ".hex" in a ".c" file since this later ".hex" file may or may not have been created yet. This patch renames the script as AmlToC. It is posted as a separate patch to prevent git from seeing the renaming as a deletion plus addition of a new file. The ending line of the posix-like bin-wrapper script has also been corrected. This is a first step toward generating a C file containing the AML bytecode from an ASL file. This C file will then be handled by the EDK2 build system to generate an object file. Thus, no file inclusion will be required anymore. The C file requiring the AML bytecode as a C array, and the ASL file, will be compiled independently. The C array must be defined as an external symbol. The linker is resolving the reference to the C array symbol. To summarize, the flow goes as: -1. ASL file is compiled to AML; -2. AML file is copied to a ".amli" intermediate file; -3. EDK2 build system applies the rule relevant to ".amli" files. This is, calling the "AmlToC" script, generating a C file from the ".amli" file; -4. EDK2 build system applies the rule relevant to C files. This is creating an object file. -5. EDK2 build system links the object file containing the AML bytecode with the object file requiring it. Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com> Suggested-by: Tomas Pilar <Tomas.Pilar@arm.com> --- The changes can be seen at https://github.com/PierreARM/edk2/commits/803_Compile_AML_bytecode_array_into_OBJ_file_v3 Notes: Notes: v1: - Rename AmlToHex scripts to AmlToC, and change line endings of the PosixLike bin-wrapper. [Pierre] v2: - No modification. [Pierre] v3: - Changed "Signed-off-by" to "Suggested-by". [Bob] BaseTools/BinWrappers/PosixLike/{AmlToHex => AmlToC} | 28 ++++++++++---------- BaseTools/BinWrappers/WindowsLike/{AmlToHex.bat => AmlToC.bat} | 0 BaseTools/Source/Python/{AmlToHex/AmlToHex.py => AmlToC/AmlToC.py} | 0 3 files changed, 14 insertions(+), 14 deletions(-) diff --git a/BaseTools/BinWrappers/PosixLike/AmlToHex b/BaseTools/BinWrappers/PosixLike/AmlToC similarity index 97% rename from BaseTools/BinWrappers/PosixLike/AmlToHex rename to BaseTools/BinWrappers/PosixLike/AmlToC index 9fb68299e4c67d1f332cd883fd348a896f1bdc50..1dd28e966288f6ea4fc52d42e2dc7b1f74226c23 100755 --- a/BaseTools/BinWrappers/PosixLike/AmlToHex +++ b/BaseTools/BinWrappers/PosixLike/AmlToC @@ -1,14 +1,14 @@ -#!/usr/bin/env bash -#python `dirname $0`/RunToolFromSource.py `basename $0` $* - -# If a ${PYTHON_COMMAND} command is available, use it in preference to python -if command -v ${PYTHON_COMMAND} >/dev/null 2>&1; then - python_exe=${PYTHON_COMMAND} -fi - -full_cmd=${BASH_SOURCE:-$0} # see http://mywiki.wooledge.org/BashFAQ/028 for a discussion of why $0 is not a good choice here -dir=$(dirname "$full_cmd") -exe=$(basename "$full_cmd") - -export PYTHONPATH="$dir/../../Source/Python${PYTHONPATH:+:"$PYTHONPATH"}" -exec "${python_exe:-python}" "$dir/../../Source/Python/$exe/$exe.py" "$@" +#!/usr/bin/env bash +#python `dirname $0`/RunToolFromSource.py `basename $0` $* + +# If a ${PYTHON_COMMAND} command is available, use it in preference to python +if command -v ${PYTHON_COMMAND} >/dev/null 2>&1; then + python_exe=${PYTHON_COMMAND} +fi + +full_cmd=${BASH_SOURCE:-$0} # see http://mywiki.wooledge.org/BashFAQ/028 for a discussion of why $0 is not a good choice here +dir=$(dirname "$full_cmd") +exe=$(basename "$full_cmd") + +export PYTHONPATH="$dir/../../Source/Python${PYTHONPATH:+:"$PYTHONPATH"}" +exec "${python_exe:-python}" "$dir/../../Source/Python/$exe/$exe.py" "$@" diff --git a/BaseTools/BinWrappers/WindowsLike/AmlToHex.bat b/BaseTools/BinWrappers/WindowsLike/AmlToC.bat similarity index 100% rename from BaseTools/BinWrappers/WindowsLike/AmlToHex.bat rename to BaseTools/BinWrappers/WindowsLike/AmlToC.bat diff --git a/BaseTools/Source/Python/AmlToHex/AmlToHex.py b/BaseTools/Source/Python/AmlToC/AmlToC.py similarity index 100% rename from BaseTools/Source/Python/AmlToHex/AmlToHex.py rename to BaseTools/Source/Python/AmlToC/AmlToC.py -- 'Guid(CE165669-3EF3-493F-B85D-6190EE5B9759)' ^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH v3 3/4] BaseTools: Compile AML bytecode arrays into .obj file 2020-06-24 9:08 [PATCH v3 0/4] Compile AML bytecode array into OBJ file PierreGondois 2020-06-24 9:08 ` [PATCH v3 1/4] BaseTools: Generate multiple rules when multiple output files PierreGondois 2020-06-24 9:08 ` [PATCH v3 2/4] BaseTools: Rename AmlToHex script to AmlToC PierreGondois @ 2020-06-24 9:08 ` PierreGondois 2020-06-24 9:08 ` [PATCH v3 4/4] BaseTools: Fix string concatenation PierreGondois 2020-06-24 15:16 ` [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file Bob Feng 4 siblings, 0 replies; 14+ messages in thread From: PierreGondois @ 2020-06-24 9:08 UTC (permalink / raw) To: devel; +Cc: Pierre Gondois, sami.mujawar, tomas.pilar, bob.c.feng, liming.gao, nd From: Pierre Gondois <pierre.gondois@arm.com> The AmlToHex script and Posix/WindowsLike wrappers convert an AML file to a .hex file, containing a C array storing AML bytecode. This ".hex" file can then be included in a C file, allowing to access the AML bytecode from this C file. The EDK2 build system doesn't allow to a depict dependency orders between files of different languages. For instance, in a module containing a ".c" file and a ".asl", the ".c" file may or may not be built prior to the ".asl" file. This prevents any inclusion of a generated ".hex" in a ".c" file since this later ".hex" file may or may not have been created yet. This patch modifies the AmlToC script to generate a C file instead of a ".hex" file. It also adds the generation of an intermediate ".amli" file when compiling an ASL file, and adds a rule to convert this ".amli" to a C file. This allows to generate a C file containing the AML bytecode from an ASL file. This C file will then be handled by the EDK2 build system to generate an object file. Thus, no file inclusion will be required anymore. The C file requiring the AML bytecode as a C array, and the ASL file, will be compiled independently. The C array must be defined as an external symbol. The linker is resolving the reference to the C array symbol. To summarize, the flow goes as: -1. ASL file is compiled to AML; -2. AML file is copied to a ".amli" intermediate file; -3. EDK2 build system applies the rule relevant to ".amli" files. This is, calling the "AmlToC" script, generating a C file from the ".amli" file; -4. EDK2 build system applies the rule relevant to C files. This is creating an object file. -5. EDK2 build system links the object file containing the AML bytecode with the object file requiring it. Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com> Suggested-by: Tomas Pilar <Tomas.Pilar@arm.com> --- The changes can be seen at https://github.com/PierreARM/edk2/commits/803_Compile_AML_bytecode_array_into_OBJ_file_v3 Notes: Notes: v1: - Add a new rule to the build_rule.template file to generate ".obj" files from .asl files, and modify the AmlToC script accordingly. [Pierre] v2: - Restrict the rule to DXE_DRIVER. This allows to build the OvmfPkg, which was not the case in v1. [Pierre] v3: - Changed "Signed-off-by" to "Suggested-by". [Bob] BaseTools/Conf/build_rule.template | 15 +++- BaseTools/Source/Python/AmlToC/AmlToC.py | 82 ++++++++------------ 2 files changed, 47 insertions(+), 50 deletions(-) diff --git a/BaseTools/Conf/build_rule.template b/BaseTools/Conf/build_rule.template index 0822b681fcd9f61c6508e6f93ffc31fa70fd7059..c034869915914936e28f64a6aadba08e0169da44 100755 --- a/BaseTools/Conf/build_rule.template +++ b/BaseTools/Conf/build_rule.template @@ -419,6 +419,7 @@ <OutputFile> $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.aml + $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.amli <ExtraDependency> $(MAKE_FILE) @@ -428,14 +429,24 @@ "$(ASLPP)" $(DEPS_FLAGS) $(ASLPP_FLAGS) $(INC) /I${s_path} $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.i > $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.iii Trim --source-code -l -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.iiii $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.iii "$(ASL)" $(ASL_FLAGS) $(ASL_OUTFLAGS)${dst} $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.iiii - -AmlToHex $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.aml + $(CP) $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.aml $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.amli <Command.GCC> Trim --asl-file --asl-deps -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.i -i $(INC_LIST) ${src} "$(ASLPP)" $(DEPS_FLAGS) $(ASLPP_FLAGS) $(INC) -I${s_path} $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.i > $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.iii Trim --source-code -l -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.iiii $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.iii "$(ASL)" $(ASL_FLAGS) $(ASL_OUTFLAGS)${dst} $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.iiii - -AmlToHex $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.aml + $(CP) $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.aml $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.amli + +[Acpi-Machine-Language-File-to-C.DXE_DRIVER] + <InputFile> + ?.amli + + <OutputFile> + ${s_path}(+)${s_base}.c + + <Command> + -AmlToC ${src} [C-Code-File.AcpiTable] <InputFile> diff --git a/BaseTools/Source/Python/AmlToC/AmlToC.py b/BaseTools/Source/Python/AmlToC/AmlToC.py index 643db2910e37acfdd80ac18d288c921320a79ce1..346de7159de702d860bbd809ddbe8175f1493cfb 100644 --- a/BaseTools/Source/Python/AmlToC/AmlToC.py +++ b/BaseTools/Source/Python/AmlToC/AmlToC.py @@ -1,9 +1,9 @@ ## @file # -# Convert an AML file to a .hex file containing the AML bytecode stored in a +# Convert an AML file to a .c file containing the AML bytecode stored in a # C array. -# By default, "Tables\Dsdt.aml" will generate "Tables\Dsdt.hex". -# "Tables\Dsdt.hex" will contain a C array named "dsdt_aml_code" that contains +# By default, "Tables\Dsdt.aml" will generate "Tables\Dsdt.c". +# "Tables\Dsdt.c" will contain a C array named "dsdt_aml_code" that contains # the AML bytecode. # # Copyright (c) 2020, ARM Limited. All rights reserved.<BR> @@ -17,31 +17,26 @@ from Common.BuildToolError import * import sys import os +__description__ = """ +Convert an AML file to a .c file containing the AML bytecode stored in a C +array. By default, Tables\Dsdt.aml will generate Tables\Dsdt.c. +Tables\Dsdt.c will contain a C array named "dsdt_aml_code" that contains +the AML bytecode. +""" + ## Parse the command line arguments. # # @retval A argparse.NameSpace instance, containing parsed values. # def ParseArgs(): # Initialize the parser. - Parser = argparse.ArgumentParser( - description="Convert an AML file to a .hex file containing the AML " + \ - "bytecode stored in a C array. By default, " + \ - "\"Tables\\Dsdt.aml\" will generate" + \ - "\"Tables\\Dsdt.hex\". \"Tables\\Dsdt.hex\" will " + \ - "contain a C array named \"dsdt_aml_code\" that " + \ - "contains the AML bytecode." - ) + Parser = argparse.ArgumentParser(description=__description__) # Define the possible arguments. - Parser.add_argument( - dest="InputFile", - help="Path to an input AML file to generate a .hex file from." - ) - Parser.add_argument( - "-o", "--out-dir", dest="OutDir", - help="Output directory where the .hex file will be generated. " + \ - "Default is the input file's directory." - ) + Parser.add_argument(dest="InputFile", + help="Path to an input AML file to generate a .c file from.") + Parser.add_argument("-o", "--out-dir", dest="OutDir", + help="Output directory where the .c file will be generated. Default is the input file's directory.") # Parse the input arguments. Args = Parser.parse_args() @@ -55,9 +50,7 @@ def ParseArgs(): with open(Args.InputFile, "rb") as fIn: Signature = str(fIn.read(4)) if ("DSDT" not in Signature) and ("SSDT" not in Signature): - EdkLogger.info("Invalid file type. " + \ - "File does not have a valid " + \ - "DSDT or SSDT signature: %s" % Args.InputFile) + EdkLogger.info("Invalid file type. File does not have a valid DSDT or SSDT signature: {}".format(Args.InputFile)) return None # Get the basename of the input file. @@ -66,42 +59,39 @@ def ParseArgs(): # If no output directory is specified, output to the input directory. if not Args.OutDir: - Args.OutputFile = os.path.join( - os.path.dirname(Args.InputFile), - BaseName + ".hex" - ) + Args.OutputFile = os.path.join(os.path.dirname(Args.InputFile), + BaseName + ".c") else: if not os.path.exists(Args.OutDir): os.mkdir(Args.OutDir) - Args.OutputFile = os.path.join(Args.OutDir, BaseName + ".hex") + Args.OutputFile = os.path.join(Args.OutDir, BaseName + ".c") Args.BaseName = BaseName return Args -## Convert an AML file to a .hex file containing the AML bytecode stored +## Convert an AML file to a .c file containing the AML bytecode stored # in a C array. # # @param InputFile Path to the input AML file. -# @param OutputFile Path to the output .hex file to generate. +# @param OutputFile Path to the output .c file to generate. # @param BaseName Base name of the input file. -# This is also the name of the generated .hex file. +# This is also the name of the generated .c file. # -def AmlToHex(InputFile, OutputFile, BaseName): +def AmlToC(InputFile, OutputFile, BaseName): - MacroName = "__{}_HEX__".format(BaseName.upper()) ArrayName = BaseName.lower() + "_aml_code" + FileHeader =\ +""" +// This file has been generated from: +// -Python script: {} +// -Input AML file: {} + +""" with open(InputFile, "rb") as fIn, open(OutputFile, "w") as fOut: # Write header. - fOut.write("// This file has been generated from:\n" + \ - "// \tPython script: " + \ - os.path.abspath(__file__) + "\n" + \ - "// \tInput AML file: " + \ - os.path.abspath(InputFile) + "\n\n" + \ - "#ifndef {}\n".format(MacroName) + \ - "#define {}\n\n".format(MacroName) - ) + fOut.write(FileHeader.format(os.path.abspath(InputFile), os.path.abspath(__file__))) # Write the array and its content. fOut.write("unsigned char {}[] = {{\n ".format(ArrayName)) @@ -115,15 +105,12 @@ def AmlToHex(InputFile, OutputFile, BaseName): byte = fIn.read(1) fOut.write("\n};\n") - # Write footer. - fOut.write("#endif // {}\n".format(MacroName)) - ## Main method # # This method: # 1- Initialize an EdkLogger instance. # 2- Parses the input arguments. -# 3- Converts an AML file to a .hex file containing the AML bytecode stored +# 3- Converts an AML file to a .c file containing the AML bytecode stored # in a C array. # # @retval 0 Success. @@ -139,10 +126,9 @@ def Main(): if not CommandArguments: return 1 - # Convert an AML file to a .hex file containing the AML bytecode stored + # Convert an AML file to a .c file containing the AML bytecode stored # in a C array. - AmlToHex(CommandArguments.InputFile, CommandArguments.OutputFile, - CommandArguments.BaseName) + AmlToC(CommandArguments.InputFile, CommandArguments.OutputFile, CommandArguments.BaseName) except Exception as e: print(e) return 1 -- 'Guid(CE165669-3EF3-493F-B85D-6190EE5B9759)' ^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH v3 4/4] BaseTools: Fix string concatenation 2020-06-24 9:08 [PATCH v3 0/4] Compile AML bytecode array into OBJ file PierreGondois ` (2 preceding siblings ...) 2020-06-24 9:08 ` [PATCH v3 3/4] BaseTools: Compile AML bytecode arrays into .obj file PierreGondois @ 2020-06-24 9:08 ` PierreGondois 2020-06-24 15:16 ` [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file Bob Feng 4 siblings, 0 replies; 14+ messages in thread From: PierreGondois @ 2020-06-24 9:08 UTC (permalink / raw) To: devel; +Cc: Pierre Gondois, sami.mujawar, tomas.pilar, bob.c.feng, liming.gao, nd From: Pierre Gondois <pierre.gondois@arm.com> Using Python 3.7.2 on win32, when printing a FileBuildRule instance, the following error occurs: File "edk2\BaseTools\Source\Python\AutoGen\BuildEngine.py", line 177, in __str__ DestString = ", ".join(self.DestFileList) TypeError: sequence item 0: expected str instance, PathClass found This patch converts each PathClass element of the list to a string instance before concatenating them. Signed-off-by: Pierre Gondois <pierre.gondois@arm.com> --- The changes can be seen at https://github.com/PierreARM/edk2/commits/803_Compile_AML_bytecode_array_into_OBJ_file_v3 Notes: Notes: v2: - No v1 for this patch. Fix a __str__ method. [Pierre] v3: - No modification. [Pierre] BaseTools/Source/Python/AutoGen/BuildEngine.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/BaseTools/Source/Python/AutoGen/BuildEngine.py b/BaseTools/Source/Python/AutoGen/BuildEngine.py index d602414ca41f37155c9c6d00eec54ea3918840c3..722fead75af6d60aa82365d999837cd5ac3299af 100644 --- a/BaseTools/Source/Python/AutoGen/BuildEngine.py +++ b/BaseTools/Source/Python/AutoGen/BuildEngine.py @@ -172,7 +172,7 @@ class FileBuildRule: def __str__(self): SourceString = "" SourceString += " %s %s %s" % (self.SourceFileType, " ".join(self.SourceFileExtList), self.ExtraSourceFileList) - DestString = ", ".join(self.DestFileList) + DestString = ", ".join([str(i) for i in self.DestFileList]) CommandString = "\n\t".join(self.CommandList) return "%s : %s\n\t%s" % (DestString, SourceString, CommandString) -- 'Guid(CE165669-3EF3-493F-B85D-6190EE5B9759)' ^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file 2020-06-24 9:08 [PATCH v3 0/4] Compile AML bytecode array into OBJ file PierreGondois ` (3 preceding siblings ...) 2020-06-24 9:08 ` [PATCH v3 4/4] BaseTools: Fix string concatenation PierreGondois @ 2020-06-24 15:16 ` Bob Feng 2020-06-25 9:30 ` PierreGondois 4 siblings, 1 reply; 14+ messages in thread From: Bob Feng @ 2020-06-24 15:16 UTC (permalink / raw) To: devel@edk2.groups.io, pierre.gondois@arm.com Cc: sami.mujawar@arm.com, tomas.pilar@arm.com, Gao, Liming, nd@arm.com Hi Pierre, There are some build failed in OpenCI. Would you please check it? https://github.com/tianocore/edk2/pull/727 Thanks, Bob -----Original Message----- From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of PierreGondois Sent: Wednesday, June 24, 2020 5:09 PM To: devel@edk2.groups.io Cc: Pierre Gondois <Pierre.Gondois@arm.com>; sami.mujawar@arm.com; tomas.pilar@arm.com; Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming <liming.gao@intel.com>; nd@arm.com Subject: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file Following the BZ at https://bugzilla.tianocore.org/show_bug.cgi?id=2425 This patch serie is a another way to solve the dependency of C files over ASL files. With this new method, the dependency is resolved at the linking stage. The last method to solve this dependency was to add the possibility to modify INF files to depict such a dependency. This method was not accepted. The discussion is available at https://edk2.groups.io/g/devel/topic/72655342#56658 The last patch modifying the INF specification and INF parsing are available at: https://edk2.groups.io/g/devel/topic/72655342#56658 https://edk2.groups.io/g/devel/topic/72656060#56662 Pierre Gondois (4): BaseTools: Generate multiple rules when multiple output files BaseTools: Rename AmlToHex script to AmlToC BaseTools: Compile AML bytecode arrays into .obj file BaseTools: Fix string concatenation BaseTools/BinWrappers/PosixLike/{AmlToHex => AmlToC} | 28 +++---- BaseTools/BinWrappers/WindowsLike/{AmlToHex.bat => AmlToC.bat} | 0 BaseTools/Conf/build_rule.template | 15 +++- BaseTools/Source/Python/{AmlToHex/AmlToHex.py => AmlToC/AmlToC.py} | 82 ++++++++------------ BaseTools/Source/Python/AutoGen/BuildEngine.py | 2 +- BaseTools/Source/Python/AutoGen/GenMake.py | 6 ++ BaseTools/Source/Python/AutoGen/ModuleAutoGen.py | 38 +++++---- 7 files changed, 89 insertions(+), 82 deletions(-) rename BaseTools/BinWrappers/PosixLike/{AmlToHex => AmlToC} (97%) rename BaseTools/BinWrappers/WindowsLike/{AmlToHex.bat => AmlToC.bat} (100%) rename BaseTools/Source/Python/{AmlToHex/AmlToHex.py => AmlToC/AmlToC.py} (52%) -- 'Guid(CE165669-3EF3-493F-B85D-6190EE5B9759)' ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file 2020-06-24 15:16 ` [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file Bob Feng @ 2020-06-25 9:30 ` PierreGondois 2020-06-25 11:51 ` Bob Feng 2020-06-28 7:34 ` Bob Feng 0 siblings, 2 replies; 14+ messages in thread From: PierreGondois @ 2020-06-25 9:30 UTC (permalink / raw) To: devel@edk2.groups.io, bob.c.feng@intel.com Cc: Sami Mujawar, Tomas Pilar, Gao, Liming, nd Hello Bob, I believe the line endings of the BaseTools/BinWrappers/PosixLike/AmlToC script have been modified to CRLF when I sent the patch. I created a pull request from the linked github branch noted in the patches. It is available at https://github.com/PierreARM/edk2/commits/803_Compile_AML_bytecode_array_into_OBJ_file_v3 . The pull request is available here (to show the result of the CI tests) https://github.com/tianocore/edk2/pull/729 . Do you want a v4 or is it possible to pull the patches from the github repository? Sorry for the inconvenience. Regards, Pierre -----Original Message----- From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Bob Feng via groups.io Sent: 24 June 2020 16:16 To: devel@edk2.groups.io; Pierre Gondois <Pierre.Gondois@arm.com> Cc: Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; Gao, Liming <liming.gao@intel.com>; nd <nd@arm.com> Subject: Re: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file Hi Pierre, There are some build failed in OpenCI. Would you please check it? https://github.com/tianocore/edk2/pull/727 Thanks, Bob -----Original Message----- From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of PierreGondois Sent: Wednesday, June 24, 2020 5:09 PM To: devel@edk2.groups.io Cc: Pierre Gondois <Pierre.Gondois@arm.com>; sami.mujawar@arm.com; tomas.pilar@arm.com; Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming <liming.gao@intel.com>; nd@arm.com Subject: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file Following the BZ at https://bugzilla.tianocore.org/show_bug.cgi?id=2425 This patch serie is a another way to solve the dependency of C files over ASL files. With this new method, the dependency is resolved at the linking stage. The last method to solve this dependency was to add the possibility to modify INF files to depict such a dependency. This method was not accepted. The discussion is available at https://edk2.groups.io/g/devel/topic/72655342#56658 The last patch modifying the INF specification and INF parsing are available at: https://edk2.groups.io/g/devel/topic/72655342#56658 https://edk2.groups.io/g/devel/topic/72656060#56662 Pierre Gondois (4): BaseTools: Generate multiple rules when multiple output files BaseTools: Rename AmlToHex script to AmlToC BaseTools: Compile AML bytecode arrays into .obj file BaseTools: Fix string concatenation BaseTools/BinWrappers/PosixLike/{AmlToHex => AmlToC} | 28 +++---- BaseTools/BinWrappers/WindowsLike/{AmlToHex.bat => AmlToC.bat} | 0 BaseTools/Conf/build_rule.template | 15 +++- BaseTools/Source/Python/{AmlToHex/AmlToHex.py => AmlToC/AmlToC.py} | 82 ++++++++------------ BaseTools/Source/Python/AutoGen/BuildEngine.py | 2 +- BaseTools/Source/Python/AutoGen/GenMake.py | 6 ++ BaseTools/Source/Python/AutoGen/ModuleAutoGen.py | 38 +++++---- 7 files changed, 89 insertions(+), 82 deletions(-) rename BaseTools/BinWrappers/PosixLike/{AmlToHex => AmlToC} (97%) rename BaseTools/BinWrappers/WindowsLike/{AmlToHex.bat => AmlToC.bat} (100%) rename BaseTools/Source/Python/{AmlToHex/AmlToHex.py => AmlToC/AmlToC.py} (52%) -- 'Guid(CE165669-3EF3-493F-B85D-6190EE5B9759)' ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file 2020-06-25 9:30 ` PierreGondois @ 2020-06-25 11:51 ` Bob Feng 2020-06-28 7:34 ` Bob Feng 1 sibling, 0 replies; 14+ messages in thread From: Bob Feng @ 2020-06-25 11:51 UTC (permalink / raw) To: devel@edk2.groups.io, pierre.gondois@arm.com Cc: Sami Mujawar, Tomas Pilar, Gao, Liming, nd Hi Pierre, Please send patch v4. Thanks, Bob -----Original Message----- From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of PierreGondois Sent: Thursday, June 25, 2020 5:31 PM To: devel@edk2.groups.io; Feng, Bob C <bob.c.feng@intel.com> Cc: Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; Gao, Liming <liming.gao@intel.com>; nd <nd@arm.com> Subject: Re: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file Hello Bob, I believe the line endings of the BaseTools/BinWrappers/PosixLike/AmlToC script have been modified to CRLF when I sent the patch. I created a pull request from the linked github branch noted in the patches. It is available at https://github.com/PierreARM/edk2/commits/803_Compile_AML_bytecode_array_into_OBJ_file_v3 . The pull request is available here (to show the result of the CI tests) https://github.com/tianocore/edk2/pull/729 . Do you want a v4 or is it possible to pull the patches from the github repository? Sorry for the inconvenience. Regards, Pierre -----Original Message----- From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Bob Feng via groups.io Sent: 24 June 2020 16:16 To: devel@edk2.groups.io; Pierre Gondois <Pierre.Gondois@arm.com> Cc: Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; Gao, Liming <liming.gao@intel.com>; nd <nd@arm.com> Subject: Re: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file Hi Pierre, There are some build failed in OpenCI. Would you please check it? https://github.com/tianocore/edk2/pull/727 Thanks, Bob -----Original Message----- From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of PierreGondois Sent: Wednesday, June 24, 2020 5:09 PM To: devel@edk2.groups.io Cc: Pierre Gondois <Pierre.Gondois@arm.com>; sami.mujawar@arm.com; tomas.pilar@arm.com; Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming <liming.gao@intel.com>; nd@arm.com Subject: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file Following the BZ at https://bugzilla.tianocore.org/show_bug.cgi?id=2425 This patch serie is a another way to solve the dependency of C files over ASL files. With this new method, the dependency is resolved at the linking stage. The last method to solve this dependency was to add the possibility to modify INF files to depict such a dependency. This method was not accepted. The discussion is available at https://edk2.groups.io/g/devel/topic/72655342#56658 The last patch modifying the INF specification and INF parsing are available at: https://edk2.groups.io/g/devel/topic/72655342#56658 https://edk2.groups.io/g/devel/topic/72656060#56662 Pierre Gondois (4): BaseTools: Generate multiple rules when multiple output files BaseTools: Rename AmlToHex script to AmlToC BaseTools: Compile AML bytecode arrays into .obj file BaseTools: Fix string concatenation BaseTools/BinWrappers/PosixLike/{AmlToHex => AmlToC} | 28 +++---- BaseTools/BinWrappers/WindowsLike/{AmlToHex.bat => AmlToC.bat} | 0 BaseTools/Conf/build_rule.template | 15 +++- BaseTools/Source/Python/{AmlToHex/AmlToHex.py => AmlToC/AmlToC.py} | 82 ++++++++------------ BaseTools/Source/Python/AutoGen/BuildEngine.py | 2 +- BaseTools/Source/Python/AutoGen/GenMake.py | 6 ++ BaseTools/Source/Python/AutoGen/ModuleAutoGen.py | 38 +++++---- 7 files changed, 89 insertions(+), 82 deletions(-) rename BaseTools/BinWrappers/PosixLike/{AmlToHex => AmlToC} (97%) rename BaseTools/BinWrappers/WindowsLike/{AmlToHex.bat => AmlToC.bat} (100%) rename BaseTools/Source/Python/{AmlToHex/AmlToHex.py => AmlToC/AmlToC.py} (52%) -- 'Guid(CE165669-3EF3-493F-B85D-6190EE5B9759)' ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file 2020-06-25 9:30 ` PierreGondois 2020-06-25 11:51 ` Bob Feng @ 2020-06-28 7:34 ` Bob Feng 2020-06-28 22:12 ` PierreGondois 2020-06-29 14:32 ` PierreGondois 1 sibling, 2 replies; 14+ messages in thread From: Bob Feng @ 2020-06-28 7:34 UTC (permalink / raw) To: devel@edk2.groups.io, pierre.gondois@arm.com Cc: Sami Mujawar, Tomas Pilar, Gao, Liming, nd Hi Pierre, I met a problem when I push your patch set. If I change the CRLF to a unix format EOL, the patch can't pass PatchCheck.py. If I don't change the CRLF, The build on Linux will fail. So I can't make the patch set pass the CI. Could you share me how did you do to make your PR 729 pass? Thanks, Bob -----Original Message----- From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of PierreGondois Sent: Thursday, June 25, 2020 5:31 PM To: devel@edk2.groups.io; Feng, Bob C <bob.c.feng@intel.com> Cc: Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; Gao, Liming <liming.gao@intel.com>; nd <nd@arm.com> Subject: Re: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file Hello Bob, I believe the line endings of the BaseTools/BinWrappers/PosixLike/AmlToC script have been modified to CRLF when I sent the patch. I created a pull request from the linked github branch noted in the patches. It is available at https://github.com/PierreARM/edk2/commits/803_Compile_AML_bytecode_array_into_OBJ_file_v3 . The pull request is available here (to show the result of the CI tests) https://github.com/tianocore/edk2/pull/729 . Do you want a v4 or is it possible to pull the patches from the github repository? Sorry for the inconvenience. Regards, Pierre -----Original Message----- From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Bob Feng via groups.io Sent: 24 June 2020 16:16 To: devel@edk2.groups.io; Pierre Gondois <Pierre.Gondois@arm.com> Cc: Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; Gao, Liming <liming.gao@intel.com>; nd <nd@arm.com> Subject: Re: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file Hi Pierre, There are some build failed in OpenCI. Would you please check it? https://github.com/tianocore/edk2/pull/727 Thanks, Bob -----Original Message----- From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of PierreGondois Sent: Wednesday, June 24, 2020 5:09 PM To: devel@edk2.groups.io Cc: Pierre Gondois <Pierre.Gondois@arm.com>; sami.mujawar@arm.com; tomas.pilar@arm.com; Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming <liming.gao@intel.com>; nd@arm.com Subject: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file Following the BZ at https://bugzilla.tianocore.org/show_bug.cgi?id=2425 This patch serie is a another way to solve the dependency of C files over ASL files. With this new method, the dependency is resolved at the linking stage. The last method to solve this dependency was to add the possibility to modify INF files to depict such a dependency. This method was not accepted. The discussion is available at https://edk2.groups.io/g/devel/topic/72655342#56658 The last patch modifying the INF specification and INF parsing are available at: https://edk2.groups.io/g/devel/topic/72655342#56658 https://edk2.groups.io/g/devel/topic/72656060#56662 Pierre Gondois (4): BaseTools: Generate multiple rules when multiple output files BaseTools: Rename AmlToHex script to AmlToC BaseTools: Compile AML bytecode arrays into .obj file BaseTools: Fix string concatenation BaseTools/BinWrappers/PosixLike/{AmlToHex => AmlToC} | 28 +++---- BaseTools/BinWrappers/WindowsLike/{AmlToHex.bat => AmlToC.bat} | 0 BaseTools/Conf/build_rule.template | 15 +++- BaseTools/Source/Python/{AmlToHex/AmlToHex.py => AmlToC/AmlToC.py} | 82 ++++++++------------ BaseTools/Source/Python/AutoGen/BuildEngine.py | 2 +- BaseTools/Source/Python/AutoGen/GenMake.py | 6 ++ BaseTools/Source/Python/AutoGen/ModuleAutoGen.py | 38 +++++---- 7 files changed, 89 insertions(+), 82 deletions(-) rename BaseTools/BinWrappers/PosixLike/{AmlToHex => AmlToC} (97%) rename BaseTools/BinWrappers/WindowsLike/{AmlToHex.bat => AmlToC.bat} (100%) rename BaseTools/Source/Python/{AmlToHex/AmlToHex.py => AmlToC/AmlToC.py} (52%) -- 'Guid(CE165669-3EF3-493F-B85D-6190EE5B9759)' ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file 2020-06-28 7:34 ` Bob Feng @ 2020-06-28 22:12 ` PierreGondois 2020-06-29 14:32 ` PierreGondois 1 sibling, 0 replies; 14+ messages in thread From: PierreGondois @ 2020-06-28 22:12 UTC (permalink / raw) To: Feng, Bob C, devel@edk2.groups.io Cc: Sami Mujawar, Tomas Pilar, Gao, Liming, nd Hello Bob, The PR729 didn't pass the PatchCheck for the patch adding the LF line endings to the script in BaseTools/BinWrappers/PosixLike, this is for the patch named "BaseTools: Rename AmlToHex script to AmlToC" (cf the following link: https://dev.azure.com/tianocore/edk2-ci/_build/results?buildId=8860&view=logs&j=12f1170f-54f2-53f3-20dd-22fc7dff55f9&t=9c939e41-62c2-5605-5e05-fc3554afc9f5&l=76). However, I thought it was a normal behaviour. Indeed, PatchCheck.py is triggering an error for any file with LF line endings, except when: - the file has a ".sh" extension - the filename is in .gitmodules (it is coming from another git repo) - the file mode of the file is 160000 (it is coming from another git repo) The new AmlToC posix script doesn't match any of these conditions. However some other files are in the same conditions. I searched the files containing the "#!usr/bin/env" string and which don't have a ".sh" extension. I tried to run the PatchCheck.py script on their respective commit with the version of PatchCheck.py that existed at that time and the same CRLF error appeared. The list of these files is at the end of the mail. Renaming them with a ".sh" extension would break the build on linux: the extension of the file is important to locate a file. This is not the case on windows (and this explains why all the scripts in BaseTools\BinWrappers\WindowsLike have a .bat extension). All the bash scripts not having a ".sh" extension are located in BaseTools/Bin/CYGWIN_NT-5.1-i686/ or BaseTools/BinWrappers/PosixLike/ . A solution would be to add an exception to the CRLF check for these two folders. A small question: when applying the patch set, did BaseTools\BinWrappers\PosixLike\AmlToC have CRLF line endings and you had to manually modify them to linux line endings, or where the line endings already correct (linux LF line endings)? Regards, Pierre The list of bash scripts without a ".sh" extension: BaseTools/Bin/CYGWIN_NT-5.1-i686/BootSectImage commit 5e407648358d639ed19d228303527133e4c95c25 Date: 2016-01-20T05:12:02+00:00 BaseTools/Bin/CYGWIN_NT-5.1-i686/build commit 7aef7b7cbf16f79fb17c5ace98b1bc7f15bb90fa Date: 2018-12-28T16:25:04+08:00 BaseTools/Bin/CYGWIN_NT-5.1-i686/BuildEnv commit 66a2dc96d3dff90b4243c4ed3e7eaa33abdcdf3c Date: 2012-02-15T08:06:01+00:00 BaseTools/Bin/CYGWIN_NT-5.1-i686/Ecc commit 7aef7b7cbf16f79fb17c5ace98b1bc7f15bb90fa Date: 2018-12-28T16:25:04+08:00 BaseTools/Bin/CYGWIN_NT-5.1-i686/EfiLdrImage commit 5e407648358d639ed19d228303527133e4c95c25 Date: 2016-01-20T05:12:02+00:00 BaseTools/Bin/CYGWIN_NT-5.1-i686/EfiRom commit 5e407648358d639ed19d228303527133e4c95c25 Date: 2016-01-20T05:12:02+00:00 BaseTools/Bin/CYGWIN_NT-5.1-i686/GenCrc32 commit 5e407648358d639ed19d228303527133e4c95c25 Date: 2016-01-20T05:12:02+00:00 BaseTools/Bin/CYGWIN_NT-5.1-i686/GenDepex commit 7aef7b7cbf16f79fb17c5ace98b1bc7f15bb90fa Date: 2018-12-28T16:25:04+08:00 BaseTools/Bin/CYGWIN_NT-5.1-i686/GenFds commit 7aef7b7cbf16f79fb17c5ace98b1bc7f15bb90fa Date: 2018-12-28T16:25:04+08:00 BaseTools/Bin/CYGWIN_NT-5.1-i686/GenFfs commit 5e407648358d639ed19d228303527133e4c95c25 Date: 2016-01-20T05:12:02+00:00 BaseTools/Bin/CYGWIN_NT-5.1-i686/GenFv commit 5e407648358d639ed19d228303527133e4c95c25 Date: 2016-01-20T05:12:02+00:00 BaseTools/Bin/CYGWIN_NT-5.1-i686/GenFw commit 5e407648358d639ed19d228303527133e4c95c25 Date: 2016-01-20T05:12:02+00:00 BaseTools/Bin/CYGWIN_NT-5.1-i686/GenPage commit 5e407648358d639ed19d228303527133e4c95c25 Date: 2016-01-20T05:12:02+00:00 BaseTools/Bin/CYGWIN_NT-5.1-i686/GenSec commit 5e407648358d639ed19d228303527133e4c95c25 Date: 2016-01-20T05:12:02+00:00 BaseTools/Bin/CYGWIN_NT-5.1-i686/GenVtf commit 5e407648358d639ed19d228303527133e4c95c25 Date: 2016-01-20T05:12:02+00:00 BaseTools/Bin/CYGWIN_NT-5.1-i686/GnuGenBootSector commit 5e407648358d639ed19d228303527133e4c95c25 Date: 2016-01-20T05:12:02+00:00 BaseTools/Bin/CYGWIN_NT-5.1-i686/LzmaCompress commit 5e407648358d639ed19d228303527133e4c95c25 Date: 2016-01-20T05:12:02+00:00 BaseTools/Bin/CYGWIN_NT-5.1-i686/LzmaF86Compress commit 2e351cbe8e190271b3716284fc1076551d005472 Date: 2019-04-03T16:03:11-07:00 BaseTools/Bin/CYGWIN_NT-5.1-i686/RunBinToolFromBuildDir commit 66a2dc96d3dff90b4243c4ed3e7eaa33abdcdf3c Date: 2012-02-15T08:06:01+00:00 BaseTools/Bin/CYGWIN_NT-5.1-i686/RunToolFromSource commit 66a2dc96d3dff90b4243c4ed3e7eaa33abdcdf3c Date: 2012-02-15T08:06:01+00:00 BaseTools/Bin/CYGWIN_NT-5.1-i686/Split commit 5e407648358d639ed19d228303527133e4c95c25 Date: 2016-01-20T05:12:02+00:00 BaseTools/Bin/CYGWIN_NT-5.1-i686/TargetTool commit 7aef7b7cbf16f79fb17c5ace98b1bc7f15bb90fa Date: 2018-12-28T16:25:04+08:00 BaseTools/Bin/CYGWIN_NT-5.1-i686/TianoCompress commit 5e407648358d639ed19d228303527133e4c95c25 Date: 2016-01-20T05:12:02+00:00 BaseTools/Bin/CYGWIN_NT-5.1-i686/Trim commit 7aef7b7cbf16f79fb17c5ace98b1bc7f15bb90fa Date: 2018-12-28T16:25:04+08:00 BaseTools/Bin/CYGWIN_NT-5.1-i686/VfrCompile commit 5e407648358d639ed19d228303527133e4c95c25 Date: 2016-01-20T05:12:02+00:00 BaseTools/Bin/CYGWIN_NT-5.1-i686/VolInfo commit 5e407648358d639ed19d228303527133e4c95c25 Date: 2016-01-20T05:12:02+00:00 BaseTools/BinWrappers/PosixLike/AmlToC commit 68fe454598fea8472f65283656c4e9abe06dadb3 (HEAD -> pg/803_Compile_AML_bytecode_array_into_OBJ_file) Date: 2020-06-25T13:44:09+01:00 BaseTools/BinWrappers/PosixLike/BPDG commit 7aef7b7cbf16f79fb17c5ace98b1bc7f15bb90fa Date: 2018-12-28T16:25:04+08:00 BaseTools/BinWrappers/PosixLike/Brotli commit de87f23291620d36d69ec55ea53a1c38b8780f0b Date: 2017-03-30T16:23:10+08:00 BaseTools/BinWrappers/PosixLike/BrotliCompress commit 2e351cbe8e190271b3716284fc1076551d005472 Date: 2019-04-03T16:03:11-07:00 BaseTools/BinWrappers/PosixLike/build commit 7aef7b7cbf16f79fb17c5ace98b1bc7f15bb90fa Date: 2018-12-28T16:25:04+08:00 BaseTools/BinWrappers/PosixLike/DevicePath commit 7dbc50bd244d95fdc1741b9cfc561f0bfd724de1 Date: 2017-12-27T14:12:29+08:00 BaseTools/BinWrappers/PosixLike/Ecc commit 7aef7b7cbf16f79fb17c5ace98b1bc7f15bb90fa Date: 2018-12-28T16:25:04+08:00 BaseTools/BinWrappers/PosixLike/EfiRom commit 5e407648358d639ed19d228303527133e4c95c25 Date: 2016-01-20T05:12:02+00:00 BaseTools/BinWrappers/PosixLike/GenCrc32 commit 5e407648358d639ed19d228303527133e4c95c25 Date: 2016-01-20T05:12:02+00:00 BaseTools/BinWrappers/PosixLike/GenDepex commit 7aef7b7cbf16f79fb17c5ace98b1bc7f15bb90fa Date: 2018-12-28T16:25:04+08:00 BaseTools/BinWrappers/PosixLike/GenerateCapsule commit 7aef7b7cbf16f79fb17c5ace98b1bc7f15bb90fa Date: 2018-12-28T16:25:04+08:00 BaseTools/BinWrappers/PosixLike/GenFds commit 7aef7b7cbf16f79fb17c5ace98b1bc7f15bb90fa Date: 2018-12-28T16:25:04+08:00 BaseTools/BinWrappers/PosixLike/GenFfs commit 5e407648358d639ed19d228303527133e4c95c25 Date: 2016-01-20T05:12:02+00:00 BaseTools/BinWrappers/PosixLike/GenFv commit 5e407648358d639ed19d228303527133e4c95c25 Date: 2016-01-20T05:12:02+00:00 BaseTools/BinWrappers/PosixLike/GenFw commit 5e407648358d639ed19d228303527133e4c95c25 Date: 2016-01-20T05:12:02+00:00 BaseTools/BinWrappers/PosixLike/GenPatchPcdTable commit 7aef7b7cbf16f79fb17c5ace98b1bc7f15bb90fa Date: 2018-12-28T16:25:04+08:00 BaseTools/BinWrappers/PosixLike/GenSec commit 5e407648358d639ed19d228303527133e4c95c25 Date: 2016-01-20T05:12:02+00:00 BaseTools/BinWrappers/PosixLike/LzmaCompress commit 5e407648358d639ed19d228303527133e4c95c25 Date: 2016-01-20T05:12:02+00:00 BaseTools/BinWrappers/PosixLike/LzmaF86Compress commit 2e351cbe8e190271b3716284fc1076551d005472 Date: 2019-04-03T16:03:11-07:00 BaseTools/BinWrappers/PosixLike/PatchPcdValue commit 7aef7b7cbf16f79fb17c5ace98b1bc7f15bb90fa Date: 2018-12-28T16:25:04+08:00 BaseTools/BinWrappers/PosixLike/Pkcs7Sign commit 7aef7b7cbf16f79fb17c5ace98b1bc7f15bb90fa Date: 2018-12-28T16:25:04+08:00 BaseTools/BinWrappers/PosixLike/Rsa2048Sha256GenerateKeys commit 7aef7b7cbf16f79fb17c5ace98b1bc7f15bb90fa Date: 2018-12-28T16:25:04+08:00 BaseTools/BinWrappers/PosixLike/Rsa2048Sha256Sign commit 7aef7b7cbf16f79fb17c5ace98b1bc7f15bb90fa Date: 2018-12-28T16:25:04+08:00 BaseTools/BinWrappers/PosixLike/Split commit 5e407648358d639ed19d228303527133e4c95c25 Date: 2016-01-20T05:12:02+00:00 BaseTools/BinWrappers/PosixLike/TargetTool commit 7aef7b7cbf16f79fb17c5ace98b1bc7f15bb90fa Date: 2018-12-28T16:25:04+08:00 BaseTools/BinWrappers/PosixLike/TianoCompress commit 5e407648358d639ed19d228303527133e4c95c25 Date: 2016-01-20T05:12:02+00:00 BaseTools/BinWrappers/PosixLike/Trim commit 7aef7b7cbf16f79fb17c5ace98b1bc7f15bb90fa Date: 2018-12-28T16:25:04+08:00 BaseTools/BinWrappers/PosixLike/UPT commit 7aef7b7cbf16f79fb17c5ace98b1bc7f15bb90fa Date: 2018-12-28T16:25:04+08:00 BaseTools/BinWrappers/PosixLike/VfrCompile commit 5e407648358d639ed19d228303527133e4c95c25 Date: 2016-01-20T05:12:02+00:00 BaseTools/BinWrappers/PosixLike/VolInfo commit 5e407648358d639ed19d228303527133e4c95c25 Date: 2016-01-20T05:12:02+00:00 -----Original Message----- From: Feng, Bob C <bob.c.feng@intel.com> Sent: 28 June 2020 08:34 To: devel@edk2.groups.io; Pierre Gondois <Pierre.Gondois@arm.com> Cc: Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; Gao, Liming <liming.gao@intel.com>; nd <nd@arm.com> Subject: RE: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file Hi Pierre, I met a problem when I push your patch set. If I change the CRLF to a unix format EOL, the patch can't pass PatchCheck.py. If I don't change the CRLF, The build on Linux will fail. So I can't make the patch set pass the CI. Could you share me how did you do to make your PR 729 pass? Thanks, Bob -----Original Message----- From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of PierreGondois Sent: Thursday, June 25, 2020 5:31 PM To: devel@edk2.groups.io; Feng, Bob C <bob.c.feng@intel.com> Cc: Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; Gao, Liming <liming.gao@intel.com>; nd <nd@arm.com> Subject: Re: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file Hello Bob, I believe the line endings of the BaseTools/BinWrappers/PosixLike/AmlToC script have been modified to CRLF when I sent the patch. I created a pull request from the linked github branch noted in the patches. It is available at https://github.com/PierreARM/edk2/commits/803_Compile_AML_bytecode_array_into_OBJ_file_v3 . The pull request is available here (to show the result of the CI tests) https://github.com/tianocore/edk2/pull/729 . Do you want a v4 or is it possible to pull the patches from the github repository? Sorry for the inconvenience. Regards, Pierre -----Original Message----- From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Bob Feng via groups.io Sent: 24 June 2020 16:16 To: devel@edk2.groups.io; Pierre Gondois <Pierre.Gondois@arm.com> Cc: Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; Gao, Liming <liming.gao@intel.com>; nd <nd@arm.com> Subject: Re: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file Hi Pierre, There are some build failed in OpenCI. Would you please check it? https://github.com/tianocore/edk2/pull/727 Thanks, Bob -----Original Message----- From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of PierreGondois Sent: Wednesday, June 24, 2020 5:09 PM To: devel@edk2.groups.io Cc: Pierre Gondois <Pierre.Gondois@arm.com>; sami.mujawar@arm.com; tomas.pilar@arm.com; Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming <liming.gao@intel.com>; nd@arm.com Subject: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file Following the BZ at https://bugzilla.tianocore.org/show_bug.cgi?id=2425 This patch serie is a another way to solve the dependency of C files over ASL files. With this new method, the dependency is resolved at the linking stage. The last method to solve this dependency was to add the possibility to modify INF files to depict such a dependency. This method was not accepted. The discussion is available at https://edk2.groups.io/g/devel/topic/72655342#56658 The last patch modifying the INF specification and INF parsing are available at: https://edk2.groups.io/g/devel/topic/72655342#56658 https://edk2.groups.io/g/devel/topic/72656060#56662 Pierre Gondois (4): BaseTools: Generate multiple rules when multiple output files BaseTools: Rename AmlToHex script to AmlToC BaseTools: Compile AML bytecode arrays into .obj file BaseTools: Fix string concatenation BaseTools/BinWrappers/PosixLike/{AmlToHex => AmlToC} | 28 +++---- BaseTools/BinWrappers/WindowsLike/{AmlToHex.bat => AmlToC.bat} | 0 BaseTools/Conf/build_rule.template | 15 +++- BaseTools/Source/Python/{AmlToHex/AmlToHex.py => AmlToC/AmlToC.py} | 82 ++++++++------------ BaseTools/Source/Python/AutoGen/BuildEngine.py | 2 +- BaseTools/Source/Python/AutoGen/GenMake.py | 6 ++ BaseTools/Source/Python/AutoGen/ModuleAutoGen.py | 38 +++++---- 7 files changed, 89 insertions(+), 82 deletions(-) rename BaseTools/BinWrappers/PosixLike/{AmlToHex => AmlToC} (97%) rename BaseTools/BinWrappers/WindowsLike/{AmlToHex.bat => AmlToC.bat} (100%) rename BaseTools/Source/Python/{AmlToHex/AmlToHex.py => AmlToC/AmlToC.py} (52%) -- 'Guid(CE165669-3EF3-493F-B85D-6190EE5B9759)' ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file 2020-06-28 7:34 ` Bob Feng 2020-06-28 22:12 ` PierreGondois @ 2020-06-29 14:32 ` PierreGondois 2020-06-30 10:58 ` Bob Feng 1 sibling, 1 reply; 14+ messages in thread From: PierreGondois @ 2020-06-29 14:32 UTC (permalink / raw) To: Feng, Bob C, devel@edk2.groups.io Cc: Sami Mujawar, Tomas Pilar, Gao, Liming, nd Hello Bob, I actually found more files having a LF line ending by running the following command: for file in `git ls-files`; do if [ "${file##*.}" != bin ] && [ "${file##*.}" != bmp ] && [ "${file##*.}" != a ] && [ "${file##*.}" != lib ] && [ "${file##*.}" != cer ] && [ "${file##*.}" != pyd ] &&[ "${file##*.}" != pem ] && [ "${file##*.}" != raw ]; then file $file | grep -v "CRLF" | grep -v ": directory"; fi; done The command list files referenced in git, then excludes files with specific extensions, excludes directories, and excludes files with CRLF line endings. I believe some of them should have CRLF line endings if this is the default in edk2. I am mainly thinking about the ".rtf", ".py" and ".txt" files. For the actual bash scripts, maybe a ".gitattributes" file can be created in edk2 to explicitly make them use LF line endings. This can be done by setting this attribute: "eol=lf". This ".gitattributes" file would then be parsed by the PatchCHeck.py script not to trigger an error for files having the later attribute. Regards, Pierre List of files obtained by running the command above, (maybe it needs some more filters): BaseTools/Bin/CYGWIN_NT-5.1-i686/BootSectImage: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/BuildEnv: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/Ecc: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/EfiLdrImage: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/EfiRom: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/GenCrc32: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/GenDepex: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/GenFds: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/GenFfs: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/GenFv: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/GenFw: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/GenPage: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/GenSec: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/GenVtf: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/GnuGenBootSector: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/LzmaCompress: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/LzmaF86Compress: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/RunBinToolFromBuildDir: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/RunToolFromSource: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/Split: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/TargetTool: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/TianoCompress: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/Trim: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/VfrCompile: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/VolInfo: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/build: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/BPDG: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/Brotli: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/BrotliCompress: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/DevicePath: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/Ecc: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/EfiRom: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/GenCrc32: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/GenDepex: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/GenFds: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/GenFfs: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/GenFv: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/GenFw: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/GenPatchPcdTable: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/GenSec: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/GenerateCapsule: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/LzmaCompress: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/LzmaF86Compress: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/PatchPcdValue: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/Pkcs7Sign: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/Rsa2048Sha256GenerateKeys: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/Rsa2048Sha256Sign: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/Split: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/TargetTool: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/TianoCompress: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/Trim: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/UPT: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/VfrCompile: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/VolInfo: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/build: Bourne-Again shell script, ASCII text executable BaseTools/BuildEnv: ASCII text BaseTools/Source/C/VfrCompile/Pccts/KNOWN_PROBLEMS.txt: data BaseTools/Source/C/VfrCompile/Pccts/history.ps: PostScript document text conforming DSC level 3.0 BaseTools/Source/Python/Ecc/CParser3/__init__.py: empty BaseTools/Source/Python/Ecc/CParser4/__init__.py: empty BaseTools/Source/Python/Eot/CParser3/__init__.py: empty BaseTools/Source/Python/Eot/CParser4/__init__.py: empty BaseTools/Source/Python/Pkcs7Sign/TestRoot.cer.gEfiSecurityPkgTokenSpaceGuid.PcdPkcs7CertBuffer.inc: ASCII text, with very long lines, with no line terminators BaseTools/Source/Python/Pkcs7Sign/TestRoot.cer.gFmpDevicePkgTokenSpaceGuid.PcdFmpDevicePkcs7CertBufferXdr.inc: ASCII text, with very long lines, with no line terminators BaseTools/UserManuals/Build_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/EfiRom_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/GenCrc32_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/GenDepex_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/GenFds_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/GenFfs_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/GenFv_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/GenFw_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/GenPatchPcdTable_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/GenSec_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/Intel_UEFI_Packaging_Tool_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/LzmaCompress_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/PatchPcdValue_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/SplitFile_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/TargetTool_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/TianoCompress_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/Trim_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/UtilityManPage_template.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/VfrCompiler_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/VolInfo_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set CryptoPkg/Library/Include/internal/dso_conf.h: C source, ASCII text CryptoPkg/Library/OpensslLib/process_files.pl: Perl script text executable EmbeddedPkg/Library/FdtLib/TODO: ASCII text EmbeddedPkg/Library/FdtLib/fdt_overlay.c: C source, ASCII text EmulatorPkg/Unix/.gdbinit: ASCII text EmulatorPkg/Unix/GdbRun.sh: ASCII text EmulatorPkg/Unix/Host/X11IncludeHack: ASCII text, with no line terminators EmulatorPkg/Unix/Xcode/xcode_project32/XcodeBuild.sh: Bourne-Again shell script, ASCII text executable EmulatorPkg/Unix/Xcode/xcode_project32/xcode_project.xcodeproj/default.pbxuser: ASCII text EmulatorPkg/Unix/Xcode/xcode_project32/xcode_project.xcodeproj/project.pbxproj: ASCII text EmulatorPkg/Unix/Xcode/xcode_project64/XcodeBuild.sh: Bourne-Again shell script, ASCII text executable EmulatorPkg/Unix/Xcode/xcode_project64/xcode_project.xcodeproj/default.pbxuser: ASCII text EmulatorPkg/Unix/Xcode/xcode_project64/xcode_project.xcodeproj/project.pbxproj: ASCII text EmulatorPkg/Unix/lldbinit: ASCII text EmulatorPkg/build.sh: Bourne-Again shell script, ASCII text executable OvmfPkg/QemuVideoDxe/VbeShim.sh: POSIX shell script, ASCII text executable OvmfPkg/build.sh: Bourne-Again shell script, ASCII text executable edksetup.sh: ASCII text -----Original Message----- From: Feng, Bob C <bob.c.feng@intel.com> Sent: 28 June 2020 08:34 To: devel@edk2.groups.io; Pierre Gondois <Pierre.Gondois@arm.com> Cc: Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; Gao, Liming <liming.gao@intel.com>; nd <nd@arm.com> Subject: RE: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file Hi Pierre, I met a problem when I push your patch set. If I change the CRLF to a unix format EOL, the patch can't pass PatchCheck.py. If I don't change the CRLF, The build on Linux will fail. So I can't make the patch set pass the CI. Could you share me how did you do to make your PR 729 pass? Thanks, Bob -----Original Message----- From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of PierreGondois Sent: Thursday, June 25, 2020 5:31 PM To: devel@edk2.groups.io; Feng, Bob C <bob.c.feng@intel.com> Cc: Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; Gao, Liming <liming.gao@intel.com>; nd <nd@arm.com> Subject: Re: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file Hello Bob, I believe the line endings of the BaseTools/BinWrappers/PosixLike/AmlToC script have been modified to CRLF when I sent the patch. I created a pull request from the linked github branch noted in the patches. It is available at https://github.com/PierreARM/edk2/commits/803_Compile_AML_bytecode_array_into_OBJ_file_v3 . The pull request is available here (to show the result of the CI tests) https://github.com/tianocore/edk2/pull/729 . Do you want a v4 or is it possible to pull the patches from the github repository? Sorry for the inconvenience. Regards, Pierre -----Original Message----- From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Bob Feng via groups.io Sent: 24 June 2020 16:16 To: devel@edk2.groups.io; Pierre Gondois <Pierre.Gondois@arm.com> Cc: Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; Gao, Liming <liming.gao@intel.com>; nd <nd@arm.com> Subject: Re: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file Hi Pierre, There are some build failed in OpenCI. Would you please check it? https://github.com/tianocore/edk2/pull/727 Thanks, Bob -----Original Message----- From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of PierreGondois Sent: Wednesday, June 24, 2020 5:09 PM To: devel@edk2.groups.io Cc: Pierre Gondois <Pierre.Gondois@arm.com>; sami.mujawar@arm.com; tomas.pilar@arm.com; Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming <liming.gao@intel.com>; nd@arm.com Subject: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file Following the BZ at https://bugzilla.tianocore.org/show_bug.cgi?id=2425 This patch serie is a another way to solve the dependency of C files over ASL files. With this new method, the dependency is resolved at the linking stage. The last method to solve this dependency was to add the possibility to modify INF files to depict such a dependency. This method was not accepted. The discussion is available at https://edk2.groups.io/g/devel/topic/72655342#56658 The last patch modifying the INF specification and INF parsing are available at: https://edk2.groups.io/g/devel/topic/72655342#56658 https://edk2.groups.io/g/devel/topic/72656060#56662 Pierre Gondois (4): BaseTools: Generate multiple rules when multiple output files BaseTools: Rename AmlToHex script to AmlToC BaseTools: Compile AML bytecode arrays into .obj file BaseTools: Fix string concatenation BaseTools/BinWrappers/PosixLike/{AmlToHex => AmlToC} | 28 +++---- BaseTools/BinWrappers/WindowsLike/{AmlToHex.bat => AmlToC.bat} | 0 BaseTools/Conf/build_rule.template | 15 +++- BaseTools/Source/Python/{AmlToHex/AmlToHex.py => AmlToC/AmlToC.py} | 82 ++++++++------------ BaseTools/Source/Python/AutoGen/BuildEngine.py | 2 +- BaseTools/Source/Python/AutoGen/GenMake.py | 6 ++ BaseTools/Source/Python/AutoGen/ModuleAutoGen.py | 38 +++++---- 7 files changed, 89 insertions(+), 82 deletions(-) rename BaseTools/BinWrappers/PosixLike/{AmlToHex => AmlToC} (97%) rename BaseTools/BinWrappers/WindowsLike/{AmlToHex.bat => AmlToC.bat} (100%) rename BaseTools/Source/Python/{AmlToHex/AmlToHex.py => AmlToC/AmlToC.py} (52%) -- 'Guid(CE165669-3EF3-493F-B85D-6190EE5B9759)' ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file 2020-06-29 14:32 ` PierreGondois @ 2020-06-30 10:58 ` Bob Feng 2020-06-30 13:55 ` PierreGondois 0 siblings, 1 reply; 14+ messages in thread From: Bob Feng @ 2020-06-30 10:58 UTC (permalink / raw) To: devel@edk2.groups.io, pierre.gondois@arm.com Cc: Sami Mujawar, Tomas Pilar, Gao, Liming, nd Hi Pierre, I think we need to update the PatchCheck.py script. The comments says, "Do not enforce CR/LF line endings for linux shell scripts.", but that only check the file ext is .sh is not enough. if self.filename.endswith('.sh'): # # Do not enforce CR/LF line endings for linux shell scripts. # self.force_crlf = False Thanks, Bob -----Original Message----- From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of PierreGondois Sent: Monday, June 29, 2020 10:33 PM To: Feng, Bob C <bob.c.feng@intel.com>; devel@edk2.groups.io Cc: Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; Gao, Liming <liming.gao@intel.com>; nd <nd@arm.com> Subject: Re: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file Hello Bob, I actually found more files having a LF line ending by running the following command: for file in `git ls-files`; do if [ "${file##*.}" != bin ] && [ "${file##*.}" != bmp ] && [ "${file##*.}" != a ] && [ "${file##*.}" != lib ] && [ "${file##*.}" != cer ] && [ "${file##*.}" != pyd ] &&[ "${file##*.}" != pem ] && [ "${file##*.}" != raw ]; then file $file | grep -v "CRLF" | grep -v ": directory"; fi; done The command list files referenced in git, then excludes files with specific extensions, excludes directories, and excludes files with CRLF line endings. I believe some of them should have CRLF line endings if this is the default in edk2. I am mainly thinking about the ".rtf", ".py" and ".txt" files. For the actual bash scripts, maybe a ".gitattributes" file can be created in edk2 to explicitly make them use LF line endings. This can be done by setting this attribute: "eol=lf". This ".gitattributes" file would then be parsed by the PatchCHeck.py script not to trigger an error for files having the later attribute. Regards, Pierre List of files obtained by running the command above, (maybe it needs some more filters): BaseTools/Bin/CYGWIN_NT-5.1-i686/BootSectImage: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/BuildEnv: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/Ecc: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/EfiLdrImage: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/EfiRom: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/GenCrc32: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/GenDepex: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/GenFds: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/GenFfs: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/GenFv: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/GenFw: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/GenPage: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/GenSec: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/GenVtf: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/GnuGenBootSector: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/LzmaCompress: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/LzmaF86Compress: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/RunBinToolFromBuildDir: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/RunToolFromSource: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/Split: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/TargetTool: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/TianoCompress: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/Trim: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/VfrCompile: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/VolInfo: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/build: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/BPDG: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/Brotli: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/BrotliCompress: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/DevicePath: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/Ecc: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/EfiRom: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/GenCrc32: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/GenDepex: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/GenFds: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/GenFfs: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/GenFv: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/GenFw: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/GenPatchPcdTable: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/GenSec: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/GenerateCapsule: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/LzmaCompress: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/LzmaF86Compress: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/PatchPcdValue: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/Pkcs7Sign: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/Rsa2048Sha256GenerateKeys: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/Rsa2048Sha256Sign: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/Split: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/TargetTool: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/TianoCompress: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/Trim: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/UPT: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/VfrCompile: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/VolInfo: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/build: Bourne-Again shell script, ASCII text executable BaseTools/BuildEnv: ASCII text BaseTools/Source/C/VfrCompile/Pccts/KNOWN_PROBLEMS.txt: data BaseTools/Source/C/VfrCompile/Pccts/history.ps: PostScript document text conforming DSC level 3.0 BaseTools/Source/Python/Ecc/CParser3/__init__.py: empty BaseTools/Source/Python/Ecc/CParser4/__init__.py: empty BaseTools/Source/Python/Eot/CParser3/__init__.py: empty BaseTools/Source/Python/Eot/CParser4/__init__.py: empty BaseTools/Source/Python/Pkcs7Sign/TestRoot.cer.gEfiSecurityPkgTokenSpaceGuid.PcdPkcs7CertBuffer.inc: ASCII text, with very long lines, with no line terminators BaseTools/Source/Python/Pkcs7Sign/TestRoot.cer.gFmpDevicePkgTokenSpaceGuid.PcdFmpDevicePkcs7CertBufferXdr.inc: ASCII text, with very long lines, with no line terminators BaseTools/UserManuals/Build_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/EfiRom_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/GenCrc32_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/GenDepex_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/GenFds_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/GenFfs_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/GenFv_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/GenFw_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/GenPatchPcdTable_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/GenSec_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/Intel_UEFI_Packaging_Tool_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/LzmaCompress_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/PatchPcdValue_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/SplitFile_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/TargetTool_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/TianoCompress_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/Trim_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/UtilityManPage_template.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/VfrCompiler_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/VolInfo_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set CryptoPkg/Library/Include/internal/dso_conf.h: C source, ASCII text CryptoPkg/Library/OpensslLib/process_files.pl: Perl script text executable EmbeddedPkg/Library/FdtLib/TODO: ASCII text EmbeddedPkg/Library/FdtLib/fdt_overlay.c: C source, ASCII text EmulatorPkg/Unix/.gdbinit: ASCII text EmulatorPkg/Unix/GdbRun.sh: ASCII text EmulatorPkg/Unix/Host/X11IncludeHack: ASCII text, with no line terminators EmulatorPkg/Unix/Xcode/xcode_project32/XcodeBuild.sh: Bourne-Again shell script, ASCII text executable EmulatorPkg/Unix/Xcode/xcode_project32/xcode_project.xcodeproj/default.pbxuser: ASCII text EmulatorPkg/Unix/Xcode/xcode_project32/xcode_project.xcodeproj/project.pbxproj: ASCII text EmulatorPkg/Unix/Xcode/xcode_project64/XcodeBuild.sh: Bourne-Again shell script, ASCII text executable EmulatorPkg/Unix/Xcode/xcode_project64/xcode_project.xcodeproj/default.pbxuser: ASCII text EmulatorPkg/Unix/Xcode/xcode_project64/xcode_project.xcodeproj/project.pbxproj: ASCII text EmulatorPkg/Unix/lldbinit: ASCII text EmulatorPkg/build.sh: Bourne-Again shell script, ASCII text executable OvmfPkg/QemuVideoDxe/VbeShim.sh: POSIX shell script, ASCII text executable OvmfPkg/build.sh: Bourne-Again shell script, ASCII text executable edksetup.sh: ASCII text -----Original Message----- From: Feng, Bob C <bob.c.feng@intel.com> Sent: 28 June 2020 08:34 To: devel@edk2.groups.io; Pierre Gondois <Pierre.Gondois@arm.com> Cc: Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; Gao, Liming <liming.gao@intel.com>; nd <nd@arm.com> Subject: RE: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file Hi Pierre, I met a problem when I push your patch set. If I change the CRLF to a unix format EOL, the patch can't pass PatchCheck.py. If I don't change the CRLF, The build on Linux will fail. So I can't make the patch set pass the CI. Could you share me how did you do to make your PR 729 pass? Thanks, Bob -----Original Message----- From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of PierreGondois Sent: Thursday, June 25, 2020 5:31 PM To: devel@edk2.groups.io; Feng, Bob C <bob.c.feng@intel.com> Cc: Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; Gao, Liming <liming.gao@intel.com>; nd <nd@arm.com> Subject: Re: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file Hello Bob, I believe the line endings of the BaseTools/BinWrappers/PosixLike/AmlToC script have been modified to CRLF when I sent the patch. I created a pull request from the linked github branch noted in the patches. It is available at https://github.com/PierreARM/edk2/commits/803_Compile_AML_bytecode_array_into_OBJ_file_v3 . The pull request is available here (to show the result of the CI tests) https://github.com/tianocore/edk2/pull/729 . Do you want a v4 or is it possible to pull the patches from the github repository? Sorry for the inconvenience. Regards, Pierre -----Original Message----- From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Bob Feng via groups.io Sent: 24 June 2020 16:16 To: devel@edk2.groups.io; Pierre Gondois <Pierre.Gondois@arm.com> Cc: Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; Gao, Liming <liming.gao@intel.com>; nd <nd@arm.com> Subject: Re: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file Hi Pierre, There are some build failed in OpenCI. Would you please check it? https://github.com/tianocore/edk2/pull/727 Thanks, Bob -----Original Message----- From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of PierreGondois Sent: Wednesday, June 24, 2020 5:09 PM To: devel@edk2.groups.io Cc: Pierre Gondois <Pierre.Gondois@arm.com>; sami.mujawar@arm.com; tomas.pilar@arm.com; Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming <liming.gao@intel.com>; nd@arm.com Subject: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file Following the BZ at https://bugzilla.tianocore.org/show_bug.cgi?id=2425 This patch serie is a another way to solve the dependency of C files over ASL files. With this new method, the dependency is resolved at the linking stage. The last method to solve this dependency was to add the possibility to modify INF files to depict such a dependency. This method was not accepted. The discussion is available at https://edk2.groups.io/g/devel/topic/72655342#56658 The last patch modifying the INF specification and INF parsing are available at: https://edk2.groups.io/g/devel/topic/72655342#56658 https://edk2.groups.io/g/devel/topic/72656060#56662 Pierre Gondois (4): BaseTools: Generate multiple rules when multiple output files BaseTools: Rename AmlToHex script to AmlToC BaseTools: Compile AML bytecode arrays into .obj file BaseTools: Fix string concatenation BaseTools/BinWrappers/PosixLike/{AmlToHex => AmlToC} | 28 +++---- BaseTools/BinWrappers/WindowsLike/{AmlToHex.bat => AmlToC.bat} | 0 BaseTools/Conf/build_rule.template | 15 +++- BaseTools/Source/Python/{AmlToHex/AmlToHex.py => AmlToC/AmlToC.py} | 82 ++++++++------------ BaseTools/Source/Python/AutoGen/BuildEngine.py | 2 +- BaseTools/Source/Python/AutoGen/GenMake.py | 6 ++ BaseTools/Source/Python/AutoGen/ModuleAutoGen.py | 38 +++++---- 7 files changed, 89 insertions(+), 82 deletions(-) rename BaseTools/BinWrappers/PosixLike/{AmlToHex => AmlToC} (97%) rename BaseTools/BinWrappers/WindowsLike/{AmlToHex.bat => AmlToC.bat} (100%) rename BaseTools/Source/Python/{AmlToHex/AmlToHex.py => AmlToC/AmlToC.py} (52%) -- 'Guid(CE165669-3EF3-493F-B85D-6190EE5B9759)' ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file 2020-06-30 10:58 ` Bob Feng @ 2020-06-30 13:55 ` PierreGondois 2020-07-01 1:03 ` Bob Feng 0 siblings, 1 reply; 14+ messages in thread From: PierreGondois @ 2020-06-30 13:55 UTC (permalink / raw) To: Feng, Bob C, devel@edk2.groups.io Cc: Sami Mujawar, Tomas Pilar, Gao, Liming, nd I agree that PatchCheck.py needs to be updated. However, this is difficult for PatchCheck.py to know whether a file should be allowed to have LF line endings or not. Some files currently in edk2 have LF line endings and: - have a ".sh" extension (EmulatorPkg/build.sh), some other don't (BaseTools/BinWrappers/PosixLike/GenFv); - start with "#!/usr/bin/env bash" (BaseTools/BinWrappers/PosixLike/GenFv), some other don't (edksetup.sh); I don't see any criteria that would allow to identify whether a file is supposed to have LF line endings or not. This is why I think creating a ".gitattribute" file in edk2 would help. In this file, people could identify files/folders being allowed to have LF line endings. PatchCheck.py would parse this file to exclude them from the CRLF check. Plus it would enforce the line ending on local copies of edk2, thus preventing mistakes like I did when sending a bash script with CRLF line endings. Do you agree with this solution? Another question: does the CRLF error triggered by PatchCheck.py imply delaying this current patch set? Indeed I'm scared trying to add a ".gitattribute" file might start long discussions. Regards, Pierre -----Original Message----- From: Feng, Bob C <bob.c.feng@intel.com> Sent: 30 June 2020 11:59 To: devel@edk2.groups.io; Pierre Gondois <Pierre.Gondois@arm.com> Cc: Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; Gao, Liming <liming.gao@intel.com>; nd <nd@arm.com> Subject: RE: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file Hi Pierre, I think we need to update the PatchCheck.py script. The comments says, "Do not enforce CR/LF line endings for linux shell scripts.", but that only check the file ext is .sh is not enough. if self.filename.endswith('.sh'): # # Do not enforce CR/LF line endings for linux shell scripts. # self.force_crlf = False Thanks, Bob -----Original Message----- From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of PierreGondois Sent: Monday, June 29, 2020 10:33 PM To: Feng, Bob C <bob.c.feng@intel.com>; devel@edk2.groups.io Cc: Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; Gao, Liming <liming.gao@intel.com>; nd <nd@arm.com> Subject: Re: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file Hello Bob, I actually found more files having a LF line ending by running the following command: for file in `git ls-files`; do if [ "${file##*.}" != bin ] && [ "${file##*.}" != bmp ] && [ "${file##*.}" != a ] && [ "${file##*.}" != lib ] && [ "${file##*.}" != cer ] && [ "${file##*.}" != pyd ] &&[ "${file##*.}" != pem ] && [ "${file##*.}" != raw ]; then file $file | grep -v "CRLF" | grep -v ": directory"; fi; done The command list files referenced in git, then excludes files with specific extensions, excludes directories, and excludes files with CRLF line endings. I believe some of them should have CRLF line endings if this is the default in edk2. I am mainly thinking about the ".rtf", ".py" and ".txt" files. For the actual bash scripts, maybe a ".gitattributes" file can be created in edk2 to explicitly make them use LF line endings. This can be done by setting this attribute: "eol=lf". This ".gitattributes" file would then be parsed by the PatchCHeck.py script not to trigger an error for files having the later attribute. Regards, Pierre List of files obtained by running the command above, (maybe it needs some more filters): BaseTools/Bin/CYGWIN_NT-5.1-i686/BootSectImage: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/BuildEnv: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/Ecc: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/EfiLdrImage: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/EfiRom: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/GenCrc32: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/GenDepex: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/GenFds: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/GenFfs: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/GenFv: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/GenFw: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/GenPage: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/GenSec: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/GenVtf: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/GnuGenBootSector: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/LzmaCompress: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/LzmaF86Compress: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/RunBinToolFromBuildDir: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/RunToolFromSource: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/Split: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/TargetTool: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/TianoCompress: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/Trim: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/VfrCompile: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/VolInfo: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/build: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/BPDG: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/Brotli: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/BrotliCompress: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/DevicePath: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/Ecc: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/EfiRom: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/GenCrc32: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/GenDepex: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/GenFds: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/GenFfs: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/GenFv: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/GenFw: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/GenPatchPcdTable: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/GenSec: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/GenerateCapsule: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/LzmaCompress: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/LzmaF86Compress: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/PatchPcdValue: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/Pkcs7Sign: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/Rsa2048Sha256GenerateKeys: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/Rsa2048Sha256Sign: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/Split: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/TargetTool: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/TianoCompress: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/Trim: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/UPT: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/VfrCompile: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/VolInfo: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/build: Bourne-Again shell script, ASCII text executable BaseTools/BuildEnv: ASCII text BaseTools/Source/C/VfrCompile/Pccts/KNOWN_PROBLEMS.txt: data BaseTools/Source/C/VfrCompile/Pccts/history.ps: PostScript document text conforming DSC level 3.0 BaseTools/Source/Python/Ecc/CParser3/__init__.py: empty BaseTools/Source/Python/Ecc/CParser4/__init__.py: empty BaseTools/Source/Python/Eot/CParser3/__init__.py: empty BaseTools/Source/Python/Eot/CParser4/__init__.py: empty BaseTools/Source/Python/Pkcs7Sign/TestRoot.cer.gEfiSecurityPkgTokenSpaceGuid.PcdPkcs7CertBuffer.inc: ASCII text, with very long lines, with no line terminators BaseTools/Source/Python/Pkcs7Sign/TestRoot.cer.gFmpDevicePkgTokenSpaceGuid.PcdFmpDevicePkcs7CertBufferXdr.inc: ASCII text, with very long lines, with no line terminators BaseTools/UserManuals/Build_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/EfiRom_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/GenCrc32_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/GenDepex_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/GenFds_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/GenFfs_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/GenFv_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/GenFw_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/GenPatchPcdTable_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/GenSec_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/Intel_UEFI_Packaging_Tool_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/LzmaCompress_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/PatchPcdValue_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/SplitFile_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/TargetTool_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/TianoCompress_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/Trim_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/UtilityManPage_template.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/VfrCompiler_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/VolInfo_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set CryptoPkg/Library/Include/internal/dso_conf.h: C source, ASCII text CryptoPkg/Library/OpensslLib/process_files.pl: Perl script text executable EmbeddedPkg/Library/FdtLib/TODO: ASCII text EmbeddedPkg/Library/FdtLib/fdt_overlay.c: C source, ASCII text EmulatorPkg/Unix/.gdbinit: ASCII text EmulatorPkg/Unix/GdbRun.sh: ASCII text EmulatorPkg/Unix/Host/X11IncludeHack: ASCII text, with no line terminators EmulatorPkg/Unix/Xcode/xcode_project32/XcodeBuild.sh: Bourne-Again shell script, ASCII text executable EmulatorPkg/Unix/Xcode/xcode_project32/xcode_project.xcodeproj/default.pbxuser: ASCII text EmulatorPkg/Unix/Xcode/xcode_project32/xcode_project.xcodeproj/project.pbxproj: ASCII text EmulatorPkg/Unix/Xcode/xcode_project64/XcodeBuild.sh: Bourne-Again shell script, ASCII text executable EmulatorPkg/Unix/Xcode/xcode_project64/xcode_project.xcodeproj/default.pbxuser: ASCII text EmulatorPkg/Unix/Xcode/xcode_project64/xcode_project.xcodeproj/project.pbxproj: ASCII text EmulatorPkg/Unix/lldbinit: ASCII text EmulatorPkg/build.sh: Bourne-Again shell script, ASCII text executable OvmfPkg/QemuVideoDxe/VbeShim.sh: POSIX shell script, ASCII text executable OvmfPkg/build.sh: Bourne-Again shell script, ASCII text executable edksetup.sh: ASCII text -----Original Message----- From: Feng, Bob C <bob.c.feng@intel.com> Sent: 28 June 2020 08:34 To: devel@edk2.groups.io; Pierre Gondois <Pierre.Gondois@arm.com> Cc: Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; Gao, Liming <liming.gao@intel.com>; nd <nd@arm.com> Subject: RE: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file Hi Pierre, I met a problem when I push your patch set. If I change the CRLF to a unix format EOL, the patch can't pass PatchCheck.py. If I don't change the CRLF, The build on Linux will fail. So I can't make the patch set pass the CI. Could you share me how did you do to make your PR 729 pass? Thanks, Bob -----Original Message----- From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of PierreGondois Sent: Thursday, June 25, 2020 5:31 PM To: devel@edk2.groups.io; Feng, Bob C <bob.c.feng@intel.com> Cc: Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; Gao, Liming <liming.gao@intel.com>; nd <nd@arm.com> Subject: Re: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file Hello Bob, I believe the line endings of the BaseTools/BinWrappers/PosixLike/AmlToC script have been modified to CRLF when I sent the patch. I created a pull request from the linked github branch noted in the patches. It is available at https://github.com/PierreARM/edk2/commits/803_Compile_AML_bytecode_array_into_OBJ_file_v3 . The pull request is available here (to show the result of the CI tests) https://github.com/tianocore/edk2/pull/729 . Do you want a v4 or is it possible to pull the patches from the github repository? Sorry for the inconvenience. Regards, Pierre -----Original Message----- From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Bob Feng via groups.io Sent: 24 June 2020 16:16 To: devel@edk2.groups.io; Pierre Gondois <Pierre.Gondois@arm.com> Cc: Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; Gao, Liming <liming.gao@intel.com>; nd <nd@arm.com> Subject: Re: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file Hi Pierre, There are some build failed in OpenCI. Would you please check it? https://github.com/tianocore/edk2/pull/727 Thanks, Bob -----Original Message----- From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of PierreGondois Sent: Wednesday, June 24, 2020 5:09 PM To: devel@edk2.groups.io Cc: Pierre Gondois <Pierre.Gondois@arm.com>; sami.mujawar@arm.com; tomas.pilar@arm.com; Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming <liming.gao@intel.com>; nd@arm.com Subject: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file Following the BZ at https://bugzilla.tianocore.org/show_bug.cgi?id=2425 This patch serie is a another way to solve the dependency of C files over ASL files. With this new method, the dependency is resolved at the linking stage. The last method to solve this dependency was to add the possibility to modify INF files to depict such a dependency. This method was not accepted. The discussion is available at https://edk2.groups.io/g/devel/topic/72655342#56658 The last patch modifying the INF specification and INF parsing are available at: https://edk2.groups.io/g/devel/topic/72655342#56658 https://edk2.groups.io/g/devel/topic/72656060#56662 Pierre Gondois (4): BaseTools: Generate multiple rules when multiple output files BaseTools: Rename AmlToHex script to AmlToC BaseTools: Compile AML bytecode arrays into .obj file BaseTools: Fix string concatenation BaseTools/BinWrappers/PosixLike/{AmlToHex => AmlToC} | 28 +++---- BaseTools/BinWrappers/WindowsLike/{AmlToHex.bat => AmlToC.bat} | 0 BaseTools/Conf/build_rule.template | 15 +++- BaseTools/Source/Python/{AmlToHex/AmlToHex.py => AmlToC/AmlToC.py} | 82 ++++++++------------ BaseTools/Source/Python/AutoGen/BuildEngine.py | 2 +- BaseTools/Source/Python/AutoGen/GenMake.py | 6 ++ BaseTools/Source/Python/AutoGen/ModuleAutoGen.py | 38 +++++---- 7 files changed, 89 insertions(+), 82 deletions(-) rename BaseTools/BinWrappers/PosixLike/{AmlToHex => AmlToC} (97%) rename BaseTools/BinWrappers/WindowsLike/{AmlToHex.bat => AmlToC.bat} (100%) rename BaseTools/Source/Python/{AmlToHex/AmlToHex.py => AmlToC/AmlToC.py} (52%) -- 'Guid(CE165669-3EF3-493F-B85D-6190EE5B9759)' ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file 2020-06-30 13:55 ` PierreGondois @ 2020-07-01 1:03 ` Bob Feng 0 siblings, 0 replies; 14+ messages in thread From: Bob Feng @ 2020-07-01 1:03 UTC (permalink / raw) To: devel@edk2.groups.io, pierre.gondois@arm.com Cc: Sami Mujawar, Tomas Pilar, Gao, Liming, nd Hi Pierre, Yes. if the patch set can't pass CI, it can't be merged. I think we can add a check for the file path. The scripts under BinWrappers\PosixLike are all linux shell scripts. Thanks, Bob -----Original Message----- From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of PierreGondois Sent: Tuesday, June 30, 2020 9:56 PM To: Feng, Bob C <bob.c.feng@intel.com>; devel@edk2.groups.io Cc: Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; Gao, Liming <liming.gao@intel.com>; nd <nd@arm.com> Subject: Re: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file I agree that PatchCheck.py needs to be updated. However, this is difficult for PatchCheck.py to know whether a file should be allowed to have LF line endings or not. Some files currently in edk2 have LF line endings and: - have a ".sh" extension (EmulatorPkg/build.sh), some other don't (BaseTools/BinWrappers/PosixLike/GenFv); - start with "#!/usr/bin/env bash" (BaseTools/BinWrappers/PosixLike/GenFv), some other don't (edksetup.sh); I don't see any criteria that would allow to identify whether a file is supposed to have LF line endings or not. This is why I think creating a ".gitattribute" file in edk2 would help. In this file, people could identify files/folders being allowed to have LF line endings. PatchCheck.py would parse this file to exclude them from the CRLF check. Plus it would enforce the line ending on local copies of edk2, thus preventing mistakes like I did when sending a bash script with CRLF line endings. Do you agree with this solution? Another question: does the CRLF error triggered by PatchCheck.py imply delaying this current patch set? Indeed I'm scared trying to add a ".gitattribute" file might start long discussions. Regards, Pierre -----Original Message----- From: Feng, Bob C <bob.c.feng@intel.com> Sent: 30 June 2020 11:59 To: devel@edk2.groups.io; Pierre Gondois <Pierre.Gondois@arm.com> Cc: Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; Gao, Liming <liming.gao@intel.com>; nd <nd@arm.com> Subject: RE: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file Hi Pierre, I think we need to update the PatchCheck.py script. The comments says, "Do not enforce CR/LF line endings for linux shell scripts.", but that only check the file ext is .sh is not enough. if self.filename.endswith('.sh'): # # Do not enforce CR/LF line endings for linux shell scripts. # self.force_crlf = False Thanks, Bob -----Original Message----- From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of PierreGondois Sent: Monday, June 29, 2020 10:33 PM To: Feng, Bob C <bob.c.feng@intel.com>; devel@edk2.groups.io Cc: Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; Gao, Liming <liming.gao@intel.com>; nd <nd@arm.com> Subject: Re: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file Hello Bob, I actually found more files having a LF line ending by running the following command: for file in `git ls-files`; do if [ "${file##*.}" != bin ] && [ "${file##*.}" != bmp ] && [ "${file##*.}" != a ] && [ "${file##*.}" != lib ] && [ "${file##*.}" != cer ] && [ "${file##*.}" != pyd ] &&[ "${file##*.}" != pem ] && [ "${file##*.}" != raw ]; then file $file | grep -v "CRLF" | grep -v ": directory"; fi; done The command list files referenced in git, then excludes files with specific extensions, excludes directories, and excludes files with CRLF line endings. I believe some of them should have CRLF line endings if this is the default in edk2. I am mainly thinking about the ".rtf", ".py" and ".txt" files. For the actual bash scripts, maybe a ".gitattributes" file can be created in edk2 to explicitly make them use LF line endings. This can be done by setting this attribute: "eol=lf". This ".gitattributes" file would then be parsed by the PatchCHeck.py script not to trigger an error for files having the later attribute. Regards, Pierre List of files obtained by running the command above, (maybe it needs some more filters): BaseTools/Bin/CYGWIN_NT-5.1-i686/BootSectImage: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/BuildEnv: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/Ecc: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/EfiLdrImage: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/EfiRom: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/GenCrc32: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/GenDepex: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/GenFds: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/GenFfs: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/GenFv: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/GenFw: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/GenPage: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/GenSec: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/GenVtf: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/GnuGenBootSector: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/LzmaCompress: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/LzmaF86Compress: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/RunBinToolFromBuildDir: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/RunToolFromSource: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/Split: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/TargetTool: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/TianoCompress: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/Trim: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/VfrCompile: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/VolInfo: Bourne-Again shell script, ASCII text executable BaseTools/Bin/CYGWIN_NT-5.1-i686/build: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/BPDG: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/Brotli: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/BrotliCompress: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/DevicePath: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/Ecc: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/EfiRom: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/GenCrc32: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/GenDepex: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/GenFds: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/GenFfs: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/GenFv: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/GenFw: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/GenPatchPcdTable: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/GenSec: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/GenerateCapsule: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/LzmaCompress: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/LzmaF86Compress: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/PatchPcdValue: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/Pkcs7Sign: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/Rsa2048Sha256GenerateKeys: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/Rsa2048Sha256Sign: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/Split: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/TargetTool: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/TianoCompress: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/Trim: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/UPT: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/VfrCompile: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/VolInfo: Bourne-Again shell script, ASCII text executable BaseTools/BinWrappers/PosixLike/build: Bourne-Again shell script, ASCII text executable BaseTools/BuildEnv: ASCII text BaseTools/Source/C/VfrCompile/Pccts/KNOWN_PROBLEMS.txt: data BaseTools/Source/C/VfrCompile/Pccts/history.ps: PostScript document text conforming DSC level 3.0 BaseTools/Source/Python/Ecc/CParser3/__init__.py: empty BaseTools/Source/Python/Ecc/CParser4/__init__.py: empty BaseTools/Source/Python/Eot/CParser3/__init__.py: empty BaseTools/Source/Python/Eot/CParser4/__init__.py: empty BaseTools/Source/Python/Pkcs7Sign/TestRoot.cer.gEfiSecurityPkgTokenSpaceGuid.PcdPkcs7CertBuffer.inc: ASCII text, with very long lines, with no line terminators BaseTools/Source/Python/Pkcs7Sign/TestRoot.cer.gFmpDevicePkgTokenSpaceGuid.PcdFmpDevicePkcs7CertBufferXdr.inc: ASCII text, with very long lines, with no line terminators BaseTools/UserManuals/Build_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/EfiRom_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/GenCrc32_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/GenDepex_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/GenFds_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/GenFfs_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/GenFv_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/GenFw_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/GenPatchPcdTable_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/GenSec_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/Intel_UEFI_Packaging_Tool_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/LzmaCompress_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/PatchPcdValue_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/SplitFile_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/TargetTool_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/TianoCompress_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/Trim_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/UtilityManPage_template.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/VfrCompiler_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set BaseTools/UserManuals/VolInfo_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set CryptoPkg/Library/Include/internal/dso_conf.h: C source, ASCII text CryptoPkg/Library/OpensslLib/process_files.pl: Perl script text executable EmbeddedPkg/Library/FdtLib/TODO: ASCII text EmbeddedPkg/Library/FdtLib/fdt_overlay.c: C source, ASCII text EmulatorPkg/Unix/.gdbinit: ASCII text EmulatorPkg/Unix/GdbRun.sh: ASCII text EmulatorPkg/Unix/Host/X11IncludeHack: ASCII text, with no line terminators EmulatorPkg/Unix/Xcode/xcode_project32/XcodeBuild.sh: Bourne-Again shell script, ASCII text executable EmulatorPkg/Unix/Xcode/xcode_project32/xcode_project.xcodeproj/default.pbxuser: ASCII text EmulatorPkg/Unix/Xcode/xcode_project32/xcode_project.xcodeproj/project.pbxproj: ASCII text EmulatorPkg/Unix/Xcode/xcode_project64/XcodeBuild.sh: Bourne-Again shell script, ASCII text executable EmulatorPkg/Unix/Xcode/xcode_project64/xcode_project.xcodeproj/default.pbxuser: ASCII text EmulatorPkg/Unix/Xcode/xcode_project64/xcode_project.xcodeproj/project.pbxproj: ASCII text EmulatorPkg/Unix/lldbinit: ASCII text EmulatorPkg/build.sh: Bourne-Again shell script, ASCII text executable OvmfPkg/QemuVideoDxe/VbeShim.sh: POSIX shell script, ASCII text executable OvmfPkg/build.sh: Bourne-Again shell script, ASCII text executable edksetup.sh: ASCII text -----Original Message----- From: Feng, Bob C <bob.c.feng@intel.com> Sent: 28 June 2020 08:34 To: devel@edk2.groups.io; Pierre Gondois <Pierre.Gondois@arm.com> Cc: Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; Gao, Liming <liming.gao@intel.com>; nd <nd@arm.com> Subject: RE: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file Hi Pierre, I met a problem when I push your patch set. If I change the CRLF to a unix format EOL, the patch can't pass PatchCheck.py. If I don't change the CRLF, The build on Linux will fail. So I can't make the patch set pass the CI. Could you share me how did you do to make your PR 729 pass? Thanks, Bob -----Original Message----- From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of PierreGondois Sent: Thursday, June 25, 2020 5:31 PM To: devel@edk2.groups.io; Feng, Bob C <bob.c.feng@intel.com> Cc: Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; Gao, Liming <liming.gao@intel.com>; nd <nd@arm.com> Subject: Re: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file Hello Bob, I believe the line endings of the BaseTools/BinWrappers/PosixLike/AmlToC script have been modified to CRLF when I sent the patch. I created a pull request from the linked github branch noted in the patches. It is available at https://github.com/PierreARM/edk2/commits/803_Compile_AML_bytecode_array_into_OBJ_file_v3 . The pull request is available here (to show the result of the CI tests) https://github.com/tianocore/edk2/pull/729 . Do you want a v4 or is it possible to pull the patches from the github repository? Sorry for the inconvenience. Regards, Pierre -----Original Message----- From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Bob Feng via groups.io Sent: 24 June 2020 16:16 To: devel@edk2.groups.io; Pierre Gondois <Pierre.Gondois@arm.com> Cc: Sami Mujawar <Sami.Mujawar@arm.com>; Tomas Pilar <Tomas.Pilar@arm.com>; Gao, Liming <liming.gao@intel.com>; nd <nd@arm.com> Subject: Re: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file Hi Pierre, There are some build failed in OpenCI. Would you please check it? https://github.com/tianocore/edk2/pull/727 Thanks, Bob -----Original Message----- From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of PierreGondois Sent: Wednesday, June 24, 2020 5:09 PM To: devel@edk2.groups.io Cc: Pierre Gondois <Pierre.Gondois@arm.com>; sami.mujawar@arm.com; tomas.pilar@arm.com; Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming <liming.gao@intel.com>; nd@arm.com Subject: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file Following the BZ at https://bugzilla.tianocore.org/show_bug.cgi?id=2425 This patch serie is a another way to solve the dependency of C files over ASL files. With this new method, the dependency is resolved at the linking stage. The last method to solve this dependency was to add the possibility to modify INF files to depict such a dependency. This method was not accepted. The discussion is available at https://edk2.groups.io/g/devel/topic/72655342#56658 The last patch modifying the INF specification and INF parsing are available at: https://edk2.groups.io/g/devel/topic/72655342#56658 https://edk2.groups.io/g/devel/topic/72656060#56662 Pierre Gondois (4): BaseTools: Generate multiple rules when multiple output files BaseTools: Rename AmlToHex script to AmlToC BaseTools: Compile AML bytecode arrays into .obj file BaseTools: Fix string concatenation BaseTools/BinWrappers/PosixLike/{AmlToHex => AmlToC} | 28 +++---- BaseTools/BinWrappers/WindowsLike/{AmlToHex.bat => AmlToC.bat} | 0 BaseTools/Conf/build_rule.template | 15 +++- BaseTools/Source/Python/{AmlToHex/AmlToHex.py => AmlToC/AmlToC.py} | 82 ++++++++------------ BaseTools/Source/Python/AutoGen/BuildEngine.py | 2 +- BaseTools/Source/Python/AutoGen/GenMake.py | 6 ++ BaseTools/Source/Python/AutoGen/ModuleAutoGen.py | 38 +++++---- 7 files changed, 89 insertions(+), 82 deletions(-) rename BaseTools/BinWrappers/PosixLike/{AmlToHex => AmlToC} (97%) rename BaseTools/BinWrappers/WindowsLike/{AmlToHex.bat => AmlToC.bat} (100%) rename BaseTools/Source/Python/{AmlToHex/AmlToHex.py => AmlToC/AmlToC.py} (52%) -- 'Guid(CE165669-3EF3-493F-B85D-6190EE5B9759)' ^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2020-07-01 1:03 UTC | newest] Thread overview: 14+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2020-06-24 9:08 [PATCH v3 0/4] Compile AML bytecode array into OBJ file PierreGondois 2020-06-24 9:08 ` [PATCH v3 1/4] BaseTools: Generate multiple rules when multiple output files PierreGondois 2020-06-24 9:08 ` [PATCH v3 2/4] BaseTools: Rename AmlToHex script to AmlToC PierreGondois 2020-06-24 9:08 ` [PATCH v3 3/4] BaseTools: Compile AML bytecode arrays into .obj file PierreGondois 2020-06-24 9:08 ` [PATCH v3 4/4] BaseTools: Fix string concatenation PierreGondois 2020-06-24 15:16 ` [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file Bob Feng 2020-06-25 9:30 ` PierreGondois 2020-06-25 11:51 ` Bob Feng 2020-06-28 7:34 ` Bob Feng 2020-06-28 22:12 ` PierreGondois 2020-06-29 14:32 ` PierreGondois 2020-06-30 10:58 ` Bob Feng 2020-06-30 13:55 ` PierreGondois 2020-07-01 1:03 ` Bob Feng
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox