* [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile information in build report
@ 2023-02-07 3:07 Guillermo Antonio Palomino Sosa
2023-02-15 2:43 ` Yuwei Chen
0 siblings, 1 reply; 17+ messages in thread
From: Guillermo Antonio Palomino Sosa @ 2023-02-07 3:07 UTC (permalink / raw)
To: devel; +Cc: bob.c.feng, gaoliming, yuwei.chen
REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2850
Add "-Y REPORT_INFO" option to build command to generate compile
information as part of BuildReport.
This option generates files to be used by external tools as IDE's
to enhance functionality.
Files are created inside build folder:
<Build>/<BuildTarget>/<ToolChain>/CompileInfo
Files created:
* compile_commands.json - Compilation Database. To be used by IDE's
to enable advance features
* cscope.files - List of files used in compilation. Used by Cscope to parse
C code and provide browse functionality.
* module_report.json - Module data form buildReport in Json format.
Signed-off-by: Guillermo Antonio Palomino Sosa <guillermo.a.palomino.sosa@intel.com>
---
BaseTools/Source/Python/build/BuildReport.py | 139 +++++++++++++++++++-
BaseTools/Source/Python/build/buildoptions.py | 4 +-
2 files changed, 140 insertions(+), 3 deletions(-)
diff --git a/BaseTools/Source/Python/build/BuildReport.py b/BaseTools/Source/Python/build/BuildReport.py
index 468772930c..33b43d471f 100644
--- a/BaseTools/Source/Python/build/BuildReport.py
+++ b/BaseTools/Source/Python/build/BuildReport.py
@@ -10,6 +10,8 @@
## Import Modules
#
+import json
+from pathlib import Path
import Common.LongFilePathOs as os
import re
import platform
@@ -41,6 +43,7 @@ from Common.DataType import *
import collections
from Common.Expression import *
from GenFds.AprioriSection import DXE_APRIORI_GUID, PEI_APRIORI_GUID
+from AutoGen.IncludesAutoGen import IncludesAutoGen
## Pattern to extract contents in EDK DXS files
gDxsDependencyPattern = re.compile(r"DEPENDENCY_START(.+)DEPENDENCY_END", re.DOTALL)
@@ -2298,6 +2301,10 @@ class BuildReport(object):
def GenerateReport(self, BuildDuration, AutoGenTime, MakeTime, GenFdsTime):
if self.ReportFile:
try:
+
+ if "COMPILE_INFO" in self.ReportType:
+ self.GenerateCompileInfo()
+
File = []
for (Wa, MaList) in self.ReportList:
PlatformReport(Wa, MaList, self.ReportType).GenerateReport(File, BuildDuration, AutoGenTime, MakeTime, GenFdsTime, self.ReportType)
@@ -2310,7 +2317,137 @@ class BuildReport(object):
EdkLogger.error("BuildReport", CODE_ERROR, "Unknown fatal error when generating build report", ExtraData=self.ReportFile, RaiseError=False)
EdkLogger.quiet("(Python %s on %s\n%s)" % (platform.python_version(), sys.platform, traceback.format_exc()))
+
+ ##
+ # Generates compile data files to be used by external tools.
+ # Compile information will be generated in <Build>/<BuildTarget>/<ToolChain>/CompileInfo
+ # Files generated: compile_commands.json, cscope.files, modules_report.json
+ #
+ # @param self The object pointer
+ #
+ def GenerateCompileInfo(self):
+ try:
+ # Lists for the output elements
+ compile_commands = []
+ used_files = set()
+ module_report = []
+
+ for (Wa, MaList) in self.ReportList:
+ # Obtain list of all processed Workspace files
+ for file_path in Wa._GetMetaFiles(Wa.BuildTarget, Wa.ToolChain):
+ used_files.add(file_path)
+
+ for autoGen in Wa.AutoGenObjectList:
+
+ # Loop through all modules
+ for module in (autoGen.LibraryAutoGenList + autoGen.ModuleAutoGenList):
+
+ used_files.add(module.MetaFile.Path)
+
+ # Main elements of module report
+ module_report_data = {}
+ module_report_data["Name"] = module.Name
+ module_report_data["Arch"] = module.Arch
+ module_report_data["Path"] = module.MetaFile.Path
+ module_report_data["Guid"] = module.Guid
+ module_report_data["BuildType"] = module.BuildType
+ module_report_data["IsLibrary"] = module.IsLibrary
+ module_report_data["SourceDir"] = module.SourceDir
+ module_report_data["Files"] = []
+
+ # Files used by module
+ for data_file in module.SourceFileList:
+ module_report_data["Files"].append({"Name": data_file.Name, "Path": data_file.Path})
+
+ # Libraries used by module
+ module_report_data["Libraries"] = []
+ for data_library in module.LibraryAutoGenList:
+ module_report_data["Libraries"].append({"Path": data_library.MetaFile.Path})
+
+ # Packages used by module
+ module_report_data["Packages"] = []
+ for data_package in module.PackageList:
+ module_report_data["Packages"].append({"Path": data_package.MetaFile.Path, "Includes": []})
+ # Includes path used in package
+ for data_package_include in data_package.Includes:
+ module_report_data["Packages"][-1]["Includes"].append(data_package_include.Path)
+
+ # PPI's in module
+ module_report_data["PPI"] = []
+ for data_ppi in module.PpiList.keys():
+ module_report_data["PPI"].append({"Name": data_ppi, "Guid": module.PpiList[data_ppi]})
+
+ # Protocol's in module
+ module_report_data["Protocol"] = []
+ for data_protocol in module.ProtocolList.keys():
+ module_report_data["Protocol"].append({"Name": data_protocol, "Guid": module.ProtocolList[data_protocol]})
+
+ # PCD's in module
+ module_report_data["Pcd"] = []
+ for data_pcd in module.LibraryPcdList:
+ module_report_data["Pcd"].append({"Space": data_pcd.TokenSpaceGuidCName,
+ "Name": data_pcd.TokenCName,
+ "Value": data_pcd.TokenValue,
+ "Guid": data_pcd.TokenSpaceGuidValue,
+ "DatumType": data_pcd.DatumType,
+ "Type": data_pcd.Type,
+ "DefaultValue": data_pcd.DefaultValue})
+ # Add module to report
+ module_report.append(module_report_data)
+
+ # Include file dependencies to used files
+ includes_autogen = IncludesAutoGen(module.MakeFileDir, module)
+ for dep in includes_autogen.DepsCollection:
+ used_files.add(dep)
+
+ inc_flag = "-I" # Default include flag
+ if module.BuildRuleFamily == TAB_COMPILER_MSFT:
+ inc_flag = "/I"
+
+ for source in module.SourceFileList:
+ used_files.add(source.Path)
+ compile_command = {}
+ if source.Ext in [".c", ".cc", ".cpp"]:
+ #
+ # Generate compile command for each c file
+ #
+ compile_command["file"] = source.Path
+ compile_command["directory"] = source.Dir
+ build_command = module.BuildRules[source.Ext].CommandList[0]
+ build_command_variables = re.findall(r"\$\((.*?)\)", build_command)
+ for var in build_command_variables:
+ var_tokens = var.split("_")
+ var_main = var_tokens[0]
+ if len(var_tokens) == 1:
+ var_value = module.BuildOption[var_main]["PATH"]
+ else:
+ var_value = module.BuildOption[var_main][var_tokens[1]]
+ build_command = build_command.replace(f"$({var})", var_value)
+ include_files = f" {inc_flag}".join(module.IncludePathList)
+ build_command = build_command.replace("${src}", include_files)
+ build_command = build_command.replace("${dst}", module.OutputDir)
+
+ # Remove un defined macros
+ compile_command["command"] = re.sub(r"\$\(.*?\)", "", build_command)
+ compile_commands.append(compile_command)
+
+ # Create output folder if doesn't exist
+ compile_info_folder = Path(Wa.BuildDir).joinpath("CompileInfo")
+ compile_info_folder.mkdir(exist_ok=True)
+
+ # Sort and save files
+ compile_commands.sort(key=lambda x: x["file"])
+ SaveFileOnChange(compile_info_folder.joinpath(f"compile_commands.json"),json.dumps(compile_commands, indent=2), False)
+
+ SaveFileOnChange(compile_info_folder.joinpath(f"cscope.files"), "\n".join(sorted(used_files)), False)
+
+ module_report.sort(key=lambda x: x["Path"])
+ SaveFileOnChange(compile_info_folder.joinpath(f"module_report.json"), json.dumps(module_report, indent=2), False)
+
+ except:
+ EdkLogger.error("BuildReport", CODE_ERROR, "Unknown fatal error when generating build report compile information", ExtraData=self.ReportFile, RaiseError=False)
+ EdkLogger.quiet("(Python %s on %s\n%s)" % (platform.python_version(), sys.platform, traceback.format_exc()))
+
# This acts like the main() function for the script, unless it is 'import'ed into another script.
if __name__ == '__main__':
pass
-
diff --git a/BaseTools/Source/Python/build/buildoptions.py b/BaseTools/Source/Python/build/buildoptions.py
index 8334604b46..5ec561f7ec 100644
--- a/BaseTools/Source/Python/build/buildoptions.py
+++ b/BaseTools/Source/Python/build/buildoptions.py
@@ -84,8 +84,8 @@ class MyOptionParser():
Parser.add_option("-D", "--define", action="append", type="string", dest="Macros", help="Macro: \"Name [= Value]\".")
Parser.add_option("-y", "--report-file", action="store", dest="ReportFile", help="Create/overwrite the report to the specified filename.")
- Parser.add_option("-Y", "--report-type", action="append", type="choice", choices=['PCD', 'LIBRARY', 'FLASH', 'DEPEX', 'BUILD_FLAGS', 'FIXED_ADDRESS', 'HASH', 'EXECUTION_ORDER'], dest="ReportType", default=[],
- help="Flags that control the type of build report to generate. Must be one of: [PCD, LIBRARY, FLASH, DEPEX, BUILD_FLAGS, FIXED_ADDRESS, HASH, EXECUTION_ORDER]. "\
+ Parser.add_option("-Y", "--report-type", action="append", type="choice", choices=['PCD', 'LIBRARY', 'FLASH', 'DEPEX', 'BUILD_FLAGS', 'FIXED_ADDRESS', 'HASH', 'EXECUTION_ORDER', 'COMPILE_INFO'], dest="ReportType", default=[],
+ help="Flags that control the type of build report to generate. Must be one of: [PCD, LIBRARY, FLASH, DEPEX, BUILD_FLAGS, FIXED_ADDRESS, HASH, EXECUTION_ORDER, COMPILE_INFO]. "\
"To specify more than one flag, repeat this option on the command line and the default flag set is [PCD, LIBRARY, FLASH, DEPEX, HASH, BUILD_FLAGS, FIXED_ADDRESS]")
Parser.add_option("-F", "--flag", action="store", type="string", dest="Flag",
help="Specify the specific option to parse EDK UNI file. Must be one of: [-c, -s]. -c is for EDK framework UNI file, and -s is for EDK UEFI UNI file. "\
--
2.28.0.windows.1
^ permalink raw reply related [flat|nested] 17+ messages in thread
* Re: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile information in build report
2023-02-07 3:07 [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile information in build report Guillermo Antonio Palomino Sosa
@ 2023-02-15 2:43 ` Yuwei Chen
2023-02-15 3:42 ` Michael D Kinney
0 siblings, 1 reply; 17+ messages in thread
From: Yuwei Chen @ 2023-02-15 2:43 UTC (permalink / raw)
To: Palomino Sosa, Guillermo A, devel@edk2.groups.io; +Cc: Feng, Bob C, Gao, Liming
Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
> -----Original Message-----
> From: Palomino Sosa, Guillermo A <guillermo.a.palomino.sosa@intel.com>
> Sent: Tuesday, February 7, 2023 11:07 AM
> To: devel@edk2.groups.io
> Cc: Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming
> <gaoliming@byosoft.com.cn>; Chen, Christine <yuwei.chen@intel.com>
> Subject: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile
> information in build report
>
> REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2850
>
> Add "-Y REPORT_INFO" option to build command to generate compile
> information as part of BuildReport.
> This option generates files to be used by external tools as IDE's to enhance
> functionality.
> Files are created inside build folder:
> <Build>/<BuildTarget>/<ToolChain>/CompileInfo
>
> Files created:
> * compile_commands.json - Compilation Database. To be used by IDE's
> to enable advance features
> * cscope.files - List of files used in compilation. Used by Cscope to parse
> C code and provide browse functionality.
> * module_report.json - Module data form buildReport in Json format.
>
> Signed-off-by: Guillermo Antonio Palomino Sosa
> <guillermo.a.palomino.sosa@intel.com>
> ---
> BaseTools/Source/Python/build/BuildReport.py | 139
> +++++++++++++++++++-
> BaseTools/Source/Python/build/buildoptions.py | 4 +-
> 2 files changed, 140 insertions(+), 3 deletions(-)
>
> diff --git a/BaseTools/Source/Python/build/BuildReport.py
> b/BaseTools/Source/Python/build/BuildReport.py
> index 468772930c..33b43d471f 100644
> --- a/BaseTools/Source/Python/build/BuildReport.py
> +++ b/BaseTools/Source/Python/build/BuildReport.py
> @@ -10,6 +10,8 @@
>
> ## Import Modules
> #
> +import json
> +from pathlib import Path
> import Common.LongFilePathOs as os
> import re
> import platform
> @@ -41,6 +43,7 @@ from Common.DataType import * import collections
> from Common.Expression import * from GenFds.AprioriSection import
> DXE_APRIORI_GUID, PEI_APRIORI_GUID
> +from AutoGen.IncludesAutoGen import IncludesAutoGen
>
> ## Pattern to extract contents in EDK DXS files gDxsDependencyPattern =
> re.compile(r"DEPENDENCY_START(.+)DEPENDENCY_END", re.DOTALL) @@ -
> 2298,6 +2301,10 @@ class BuildReport(object):
> def GenerateReport(self, BuildDuration, AutoGenTime, MakeTime,
> GenFdsTime):
> if self.ReportFile:
> try:
> +
> + if "COMPILE_INFO" in self.ReportType:
> + self.GenerateCompileInfo()
> +
> File = []
> for (Wa, MaList) in self.ReportList:
> PlatformReport(Wa, MaList,
> self.ReportType).GenerateReport(File, BuildDuration, AutoGenTime,
> MakeTime, GenFdsTime, self.ReportType) @@ -2310,7 +2317,137 @@ class
> BuildReport(object):
> EdkLogger.error("BuildReport", CODE_ERROR, "Unknown fatal
> error when generating build report", ExtraData=self.ReportFile,
> RaiseError=False)
> EdkLogger.quiet("(Python %s on %s\n%s)" %
> (platform.python_version(), sys.platform, traceback.format_exc()))
>
> +
> + ##
> + # Generates compile data files to be used by external tools.
> + # Compile information will be generated in
> <Build>/<BuildTarget>/<ToolChain>/CompileInfo
> + # Files generated: compile_commands.json, cscope.files,
> modules_report.json
> + #
> + # @param self The object pointer
> + #
> + def GenerateCompileInfo(self):
> + try:
> + # Lists for the output elements
> + compile_commands = []
> + used_files = set()
> + module_report = []
> +
> + for (Wa, MaList) in self.ReportList:
> + # Obtain list of all processed Workspace files
> + for file_path in Wa._GetMetaFiles(Wa.BuildTarget, Wa.ToolChain):
> + used_files.add(file_path)
> +
> + for autoGen in Wa.AutoGenObjectList:
> +
> + # Loop through all modules
> + for module in (autoGen.LibraryAutoGenList +
> autoGen.ModuleAutoGenList):
> +
> + used_files.add(module.MetaFile.Path)
> +
> + # Main elements of module report
> + module_report_data = {}
> + module_report_data["Name"] = module.Name
> + module_report_data["Arch"] = module.Arch
> + module_report_data["Path"] = module.MetaFile.Path
> + module_report_data["Guid"] = module.Guid
> + module_report_data["BuildType"] = module.BuildType
> + module_report_data["IsLibrary"] = module.IsLibrary
> + module_report_data["SourceDir"] = module.SourceDir
> + module_report_data["Files"] = []
> +
> + # Files used by module
> + for data_file in module.SourceFileList:
> + module_report_data["Files"].append({"Name":
> + data_file.Name, "Path": data_file.Path})
> +
> + # Libraries used by module
> + module_report_data["Libraries"] = []
> + for data_library in module.LibraryAutoGenList:
> +
> + module_report_data["Libraries"].append({"Path":
> + data_library.MetaFile.Path})
> +
> + # Packages used by module
> + module_report_data["Packages"] = []
> + for data_package in module.PackageList:
> + module_report_data["Packages"].append({"Path":
> data_package.MetaFile.Path, "Includes": []})
> + # Includes path used in package
> + for data_package_include in data_package.Includes:
> +
> + module_report_data["Packages"][-
> 1]["Includes"].append(data_package_inc
> + lude.Path)
> +
> + # PPI's in module
> + module_report_data["PPI"] = []
> + for data_ppi in module.PpiList.keys():
> + module_report_data["PPI"].append({"Name":
> + data_ppi, "Guid": module.PpiList[data_ppi]})
> +
> + # Protocol's in module
> + module_report_data["Protocol"] = []
> + for data_protocol in module.ProtocolList.keys():
> +
> + module_report_data["Protocol"].append({"Name": data_protocol, "Guid":
> + module.ProtocolList[data_protocol]})
> +
> + # PCD's in module
> + module_report_data["Pcd"] = []
> + for data_pcd in module.LibraryPcdList:
> + module_report_data["Pcd"].append({"Space":
> data_pcd.TokenSpaceGuidCName,
> + "Name": data_pcd.TokenCName,
> + "Value": data_pcd.TokenValue,
> + "Guid": data_pcd.TokenSpaceGuidValue,
> + "DatumType": data_pcd.DatumType,
> + "Type": data_pcd.Type,
> + "DefaultValue": data_pcd.DefaultValue})
> + # Add module to report
> + module_report.append(module_report_data)
> +
> + # Include file dependencies to used files
> + includes_autogen = IncludesAutoGen(module.MakeFileDir,
> module)
> + for dep in includes_autogen.DepsCollection:
> + used_files.add(dep)
> +
> + inc_flag = "-I" # Default include flag
> + if module.BuildRuleFamily == TAB_COMPILER_MSFT:
> + inc_flag = "/I"
> +
> + for source in module.SourceFileList:
> + used_files.add(source.Path)
> + compile_command = {}
> + if source.Ext in [".c", ".cc", ".cpp"]:
> + #
> + # Generate compile command for each c file
> + #
> + compile_command["file"] = source.Path
> + compile_command["directory"] = source.Dir
> + build_command =
> module.BuildRules[source.Ext].CommandList[0]
> + build_command_variables = re.findall(r"\$\((.*?)\)",
> build_command)
> + for var in build_command_variables:
> + var_tokens = var.split("_")
> + var_main = var_tokens[0]
> + if len(var_tokens) == 1:
> + var_value = module.BuildOption[var_main]["PATH"]
> + else:
> + var_value =
> module.BuildOption[var_main][var_tokens[1]]
> + build_command = build_command.replace(f"$({var})",
> var_value)
> + include_files = f" {inc_flag}".join(module.IncludePathList)
> + build_command = build_command.replace("${src}",
> include_files)
> + build_command =
> + build_command.replace("${dst}", module.OutputDir)
> +
> + # Remove un defined macros
> + compile_command["command"] = re.sub(r"\$\(.*?\)", "",
> build_command)
> +
> + compile_commands.append(compile_command)
> +
> + # Create output folder if doesn't exist
> + compile_info_folder = Path(Wa.BuildDir).joinpath("CompileInfo")
> + compile_info_folder.mkdir(exist_ok=True)
> +
> + # Sort and save files
> + compile_commands.sort(key=lambda x: x["file"])
> +
> +
> SaveFileOnChange(compile_info_folder.joinpath(f"compile_commands.json"
> + ),json.dumps(compile_commands, indent=2), False)
> +
> +
> + SaveFileOnChange(compile_info_folder.joinpath(f"cscope.files"),
> + "\n".join(sorted(used_files)), False)
> +
> + module_report.sort(key=lambda x: x["Path"])
> +
> + SaveFileOnChange(compile_info_folder.joinpath(f"module_report.json"),
> + json.dumps(module_report, indent=2), False)
> +
> + except:
> + EdkLogger.error("BuildReport", CODE_ERROR, "Unknown fatal error
> when generating build report compile information",
> ExtraData=self.ReportFile, RaiseError=False)
> + EdkLogger.quiet("(Python %s on %s\n%s)" %
> + (platform.python_version(), sys.platform, traceback.format_exc()))
> +
> # This acts like the main() function for the script, unless it is 'import'ed into
> another script.
> if __name__ == '__main__':
> pass
> -
> diff --git a/BaseTools/Source/Python/build/buildoptions.py
> b/BaseTools/Source/Python/build/buildoptions.py
> index 8334604b46..5ec561f7ec 100644
> --- a/BaseTools/Source/Python/build/buildoptions.py
> +++ b/BaseTools/Source/Python/build/buildoptions.py
> @@ -84,8 +84,8 @@ class MyOptionParser():
> Parser.add_option("-D", "--define", action="append", type="string",
> dest="Macros", help="Macro: \"Name [= Value]\".")
>
> Parser.add_option("-y", "--report-file", action="store",
> dest="ReportFile", help="Create/overwrite the report to the specified
> filename.")
> - Parser.add_option("-Y", "--report-type", action="append",
> type="choice", choices=['PCD', 'LIBRARY', 'FLASH', 'DEPEX', 'BUILD_FLAGS',
> 'FIXED_ADDRESS', 'HASH', 'EXECUTION_ORDER'], dest="ReportType",
> default=[],
> - help="Flags that control the type of build report to generate. Must be
> one of: [PCD, LIBRARY, FLASH, DEPEX, BUILD_FLAGS, FIXED_ADDRESS, HASH,
> EXECUTION_ORDER]. "\
> + Parser.add_option("-Y", "--report-type", action="append",
> type="choice", choices=['PCD', 'LIBRARY', 'FLASH', 'DEPEX', 'BUILD_FLAGS',
> 'FIXED_ADDRESS', 'HASH', 'EXECUTION_ORDER', 'COMPILE_INFO'],
> dest="ReportType", default=[],
> + help="Flags that control the type of build report to
> + generate. Must be one of: [PCD, LIBRARY, FLASH, DEPEX, BUILD_FLAGS,
> + FIXED_ADDRESS, HASH, EXECUTION_ORDER, COMPILE_INFO]. "\
> "To specify more than one flag, repeat this option on the command
> line and the default flag set is [PCD, LIBRARY, FLASH, DEPEX, HASH,
> BUILD_FLAGS, FIXED_ADDRESS]")
> Parser.add_option("-F", "--flag", action="store", type="string",
> dest="Flag",
> help="Specify the specific option to parse EDK UNI file. Must be one of:
> [-c, -s]. -c is for EDK framework UNI file, and -s is for EDK UEFI UNI file. "\
> --
> 2.28.0.windows.1
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile information in build report
2023-02-15 2:43 ` Yuwei Chen
@ 2023-02-15 3:42 ` Michael D Kinney
2023-02-15 5:38 ` Yuwei Chen
0 siblings, 1 reply; 17+ messages in thread
From: Michael D Kinney @ 2023-02-15 3:42 UTC (permalink / raw)
To: devel@edk2.groups.io, Chen, Christine, Palomino Sosa, Guillermo A
Cc: Feng, Bob C, Gao, Liming, Kinney, Michael D
Has this been reviewed for edk2-basetools repo?
Mike
> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Yuwei Chen
> Sent: Tuesday, February 14, 2023 6:44 PM
> To: Palomino Sosa, Guillermo A <guillermo.a.palomino.sosa@intel.com>; devel@edk2.groups.io
> Cc: Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming <gaoliming@byosoft.com.cn>
> Subject: Re: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile information in build report
>
> Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
>
> > -----Original Message-----
> > From: Palomino Sosa, Guillermo A <guillermo.a.palomino.sosa@intel.com>
> > Sent: Tuesday, February 7, 2023 11:07 AM
> > To: devel@edk2.groups.io
> > Cc: Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming
> > <gaoliming@byosoft.com.cn>; Chen, Christine <yuwei.chen@intel.com>
> > Subject: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile
> > information in build report
> >
> > REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2850
> >
> > Add "-Y REPORT_INFO" option to build command to generate compile
> > information as part of BuildReport.
> > This option generates files to be used by external tools as IDE's to enhance
> > functionality.
> > Files are created inside build folder:
> > <Build>/<BuildTarget>/<ToolChain>/CompileInfo
> >
> > Files created:
> > * compile_commands.json - Compilation Database. To be used by IDE's
> > to enable advance features
> > * cscope.files - List of files used in compilation. Used by Cscope to parse
> > C code and provide browse functionality.
> > * module_report.json - Module data form buildReport in Json format.
> >
> > Signed-off-by: Guillermo Antonio Palomino Sosa
> > <guillermo.a.palomino.sosa@intel.com>
> > ---
> > BaseTools/Source/Python/build/BuildReport.py | 139
> > +++++++++++++++++++-
> > BaseTools/Source/Python/build/buildoptions.py | 4 +-
> > 2 files changed, 140 insertions(+), 3 deletions(-)
> >
> > diff --git a/BaseTools/Source/Python/build/BuildReport.py
> > b/BaseTools/Source/Python/build/BuildReport.py
> > index 468772930c..33b43d471f 100644
> > --- a/BaseTools/Source/Python/build/BuildReport.py
> > +++ b/BaseTools/Source/Python/build/BuildReport.py
> > @@ -10,6 +10,8 @@
> >
> > ## Import Modules
> > #
> > +import json
> > +from pathlib import Path
> > import Common.LongFilePathOs as os
> > import re
> > import platform
> > @@ -41,6 +43,7 @@ from Common.DataType import * import collections
> > from Common.Expression import * from GenFds.AprioriSection import
> > DXE_APRIORI_GUID, PEI_APRIORI_GUID
> > +from AutoGen.IncludesAutoGen import IncludesAutoGen
> >
> > ## Pattern to extract contents in EDK DXS files gDxsDependencyPattern =
> > re.compile(r"DEPENDENCY_START(.+)DEPENDENCY_END", re.DOTALL) @@ -
> > 2298,6 +2301,10 @@ class BuildReport(object):
> > def GenerateReport(self, BuildDuration, AutoGenTime, MakeTime,
> > GenFdsTime):
> > if self.ReportFile:
> > try:
> > +
> > + if "COMPILE_INFO" in self.ReportType:
> > + self.GenerateCompileInfo()
> > +
> > File = []
> > for (Wa, MaList) in self.ReportList:
> > PlatformReport(Wa, MaList,
> > self.ReportType).GenerateReport(File, BuildDuration, AutoGenTime,
> > MakeTime, GenFdsTime, self.ReportType) @@ -2310,7 +2317,137 @@ class
> > BuildReport(object):
> > EdkLogger.error("BuildReport", CODE_ERROR, "Unknown fatal
> > error when generating build report", ExtraData=self.ReportFile,
> > RaiseError=False)
> > EdkLogger.quiet("(Python %s on %s\n%s)" %
> > (platform.python_version(), sys.platform, traceback.format_exc()))
> >
> > +
> > + ##
> > + # Generates compile data files to be used by external tools.
> > + # Compile information will be generated in
> > <Build>/<BuildTarget>/<ToolChain>/CompileInfo
> > + # Files generated: compile_commands.json, cscope.files,
> > modules_report.json
> > + #
> > + # @param self The object pointer
> > + #
> > + def GenerateCompileInfo(self):
> > + try:
> > + # Lists for the output elements
> > + compile_commands = []
> > + used_files = set()
> > + module_report = []
> > +
> > + for (Wa, MaList) in self.ReportList:
> > + # Obtain list of all processed Workspace files
> > + for file_path in Wa._GetMetaFiles(Wa.BuildTarget, Wa.ToolChain):
> > + used_files.add(file_path)
> > +
> > + for autoGen in Wa.AutoGenObjectList:
> > +
> > + # Loop through all modules
> > + for module in (autoGen.LibraryAutoGenList +
> > autoGen.ModuleAutoGenList):
> > +
> > + used_files.add(module.MetaFile.Path)
> > +
> > + # Main elements of module report
> > + module_report_data = {}
> > + module_report_data["Name"] = module.Name
> > + module_report_data["Arch"] = module.Arch
> > + module_report_data["Path"] = module.MetaFile.Path
> > + module_report_data["Guid"] = module.Guid
> > + module_report_data["BuildType"] = module.BuildType
> > + module_report_data["IsLibrary"] = module.IsLibrary
> > + module_report_data["SourceDir"] = module.SourceDir
> > + module_report_data["Files"] = []
> > +
> > + # Files used by module
> > + for data_file in module.SourceFileList:
> > + module_report_data["Files"].append({"Name":
> > + data_file.Name, "Path": data_file.Path})
> > +
> > + # Libraries used by module
> > + module_report_data["Libraries"] = []
> > + for data_library in module.LibraryAutoGenList:
> > +
> > + module_report_data["Libraries"].append({"Path":
> > + data_library.MetaFile.Path})
> > +
> > + # Packages used by module
> > + module_report_data["Packages"] = []
> > + for data_package in module.PackageList:
> > + module_report_data["Packages"].append({"Path":
> > data_package.MetaFile.Path, "Includes": []})
> > + # Includes path used in package
> > + for data_package_include in data_package.Includes:
> > +
> > + module_report_data["Packages"][-
> > 1]["Includes"].append(data_package_inc
> > + lude.Path)
> > +
> > + # PPI's in module
> > + module_report_data["PPI"] = []
> > + for data_ppi in module.PpiList.keys():
> > + module_report_data["PPI"].append({"Name":
> > + data_ppi, "Guid": module.PpiList[data_ppi]})
> > +
> > + # Protocol's in module
> > + module_report_data["Protocol"] = []
> > + for data_protocol in module.ProtocolList.keys():
> > +
> > + module_report_data["Protocol"].append({"Name": data_protocol, "Guid":
> > + module.ProtocolList[data_protocol]})
> > +
> > + # PCD's in module
> > + module_report_data["Pcd"] = []
> > + for data_pcd in module.LibraryPcdList:
> > + module_report_data["Pcd"].append({"Space":
> > data_pcd.TokenSpaceGuidCName,
> > + "Name": data_pcd.TokenCName,
> > + "Value": data_pcd.TokenValue,
> > + "Guid": data_pcd.TokenSpaceGuidValue,
> > + "DatumType": data_pcd.DatumType,
> > + "Type": data_pcd.Type,
> > + "DefaultValue": data_pcd.DefaultValue})
> > + # Add module to report
> > + module_report.append(module_report_data)
> > +
> > + # Include file dependencies to used files
> > + includes_autogen = IncludesAutoGen(module.MakeFileDir,
> > module)
> > + for dep in includes_autogen.DepsCollection:
> > + used_files.add(dep)
> > +
> > + inc_flag = "-I" # Default include flag
> > + if module.BuildRuleFamily == TAB_COMPILER_MSFT:
> > + inc_flag = "/I"
> > +
> > + for source in module.SourceFileList:
> > + used_files.add(source.Path)
> > + compile_command = {}
> > + if source.Ext in [".c", ".cc", ".cpp"]:
> > + #
> > + # Generate compile command for each c file
> > + #
> > + compile_command["file"] = source.Path
> > + compile_command["directory"] = source.Dir
> > + build_command =
> > module.BuildRules[source.Ext].CommandList[0]
> > + build_command_variables = re.findall(r"\$\((.*?)\)",
> > build_command)
> > + for var in build_command_variables:
> > + var_tokens = var.split("_")
> > + var_main = var_tokens[0]
> > + if len(var_tokens) == 1:
> > + var_value = module.BuildOption[var_main]["PATH"]
> > + else:
> > + var_value =
> > module.BuildOption[var_main][var_tokens[1]]
> > + build_command = build_command.replace(f"$({var})",
> > var_value)
> > + include_files = f" {inc_flag}".join(module.IncludePathList)
> > + build_command = build_command.replace("${src}",
> > include_files)
> > + build_command =
> > + build_command.replace("${dst}", module.OutputDir)
> > +
> > + # Remove un defined macros
> > + compile_command["command"] = re.sub(r"\$\(.*?\)", "",
> > build_command)
> > +
> > + compile_commands.append(compile_command)
> > +
> > + # Create output folder if doesn't exist
> > + compile_info_folder = Path(Wa.BuildDir).joinpath("CompileInfo")
> > + compile_info_folder.mkdir(exist_ok=True)
> > +
> > + # Sort and save files
> > + compile_commands.sort(key=lambda x: x["file"])
> > +
> > +
> > SaveFileOnChange(compile_info_folder.joinpath(f"compile_commands.json"
> > + ),json.dumps(compile_commands, indent=2), False)
> > +
> > +
> > + SaveFileOnChange(compile_info_folder.joinpath(f"cscope.files"),
> > + "\n".join(sorted(used_files)), False)
> > +
> > + module_report.sort(key=lambda x: x["Path"])
> > +
> > + SaveFileOnChange(compile_info_folder.joinpath(f"module_report.json"),
> > + json.dumps(module_report, indent=2), False)
> > +
> > + except:
> > + EdkLogger.error("BuildReport", CODE_ERROR, "Unknown fatal error
> > when generating build report compile information",
> > ExtraData=self.ReportFile, RaiseError=False)
> > + EdkLogger.quiet("(Python %s on %s\n%s)" %
> > + (platform.python_version(), sys.platform, traceback.format_exc()))
> > +
> > # This acts like the main() function for the script, unless it is 'import'ed into
> > another script.
> > if __name__ == '__main__':
> > pass
> > -
> > diff --git a/BaseTools/Source/Python/build/buildoptions.py
> > b/BaseTools/Source/Python/build/buildoptions.py
> > index 8334604b46..5ec561f7ec 100644
> > --- a/BaseTools/Source/Python/build/buildoptions.py
> > +++ b/BaseTools/Source/Python/build/buildoptions.py
> > @@ -84,8 +84,8 @@ class MyOptionParser():
> > Parser.add_option("-D", "--define", action="append", type="string",
> > dest="Macros", help="Macro: \"Name [= Value]\".")
> >
> > Parser.add_option("-y", "--report-file", action="store",
> > dest="ReportFile", help="Create/overwrite the report to the specified
> > filename.")
> > - Parser.add_option("-Y", "--report-type", action="append",
> > type="choice", choices=['PCD', 'LIBRARY', 'FLASH', 'DEPEX', 'BUILD_FLAGS',
> > 'FIXED_ADDRESS', 'HASH', 'EXECUTION_ORDER'], dest="ReportType",
> > default=[],
> > - help="Flags that control the type of build report to generate. Must be
> > one of: [PCD, LIBRARY, FLASH, DEPEX, BUILD_FLAGS, FIXED_ADDRESS, HASH,
> > EXECUTION_ORDER]. "\
> > + Parser.add_option("-Y", "--report-type", action="append",
> > type="choice", choices=['PCD', 'LIBRARY', 'FLASH', 'DEPEX', 'BUILD_FLAGS',
> > 'FIXED_ADDRESS', 'HASH', 'EXECUTION_ORDER', 'COMPILE_INFO'],
> > dest="ReportType", default=[],
> > + help="Flags that control the type of build report to
> > + generate. Must be one of: [PCD, LIBRARY, FLASH, DEPEX, BUILD_FLAGS,
> > + FIXED_ADDRESS, HASH, EXECUTION_ORDER, COMPILE_INFO]. "\
> > "To specify more than one flag, repeat this option on the command
> > line and the default flag set is [PCD, LIBRARY, FLASH, DEPEX, HASH,
> > BUILD_FLAGS, FIXED_ADDRESS]")
> > Parser.add_option("-F", "--flag", action="store", type="string",
> > dest="Flag",
> > help="Specify the specific option to parse EDK UNI file. Must be one of:
> > [-c, -s]. -c is for EDK framework UNI file, and -s is for EDK UEFI UNI file. "\
> > --
> > 2.28.0.windows.1
>
>
>
>
>
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile information in build report
2023-02-15 3:42 ` Michael D Kinney
@ 2023-02-15 5:38 ` Yuwei Chen
2023-02-15 8:17 ` Ni, Ray
0 siblings, 1 reply; 17+ messages in thread
From: Yuwei Chen @ 2023-02-15 5:38 UTC (permalink / raw)
To: Kinney, Michael D, devel@edk2.groups.io,
Palomino Sosa, Guillermo A
Cc: Feng, Bob C, Gao, Liming
Hi Mike, thanks for reminder.
Hi Willy, currently, BaseTools related changes will be implemented on the edk2-basetools repo. Please send the patch based on the edk2-basetools repo~
Thanks,
Christine
> -----Original Message-----
> From: Kinney, Michael D <michael.d.kinney@intel.com>
> Sent: Wednesday, February 15, 2023 11:43 AM
> To: devel@edk2.groups.io; Chen, Christine <yuwei.chen@intel.com>;
> Palomino Sosa, Guillermo A <guillermo.a.palomino.sosa@intel.com>
> Cc: Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming
> <gaoliming@byosoft.com.cn>; Kinney, Michael D
> <michael.d.kinney@intel.com>
> Subject: RE: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile
> information in build report
>
> Has this been reviewed for edk2-basetools repo?
>
> Mike
>
> > -----Original Message-----
> > From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Yuwei
> > Chen
> > Sent: Tuesday, February 14, 2023 6:44 PM
> > To: Palomino Sosa, Guillermo A <guillermo.a.palomino.sosa@intel.com>;
> > devel@edk2.groups.io
> > Cc: Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming
> > <gaoliming@byosoft.com.cn>
> > Subject: Re: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile
> > information in build report
> >
> > Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
> >
> > > -----Original Message-----
> > > From: Palomino Sosa, Guillermo A
> > > <guillermo.a.palomino.sosa@intel.com>
> > > Sent: Tuesday, February 7, 2023 11:07 AM
> > > To: devel@edk2.groups.io
> > > Cc: Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming
> > > <gaoliming@byosoft.com.cn>; Chen, Christine <yuwei.chen@intel.com>
> > > Subject: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile
> > > information in build report
> > >
> > > REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2850
> > >
> > > Add "-Y REPORT_INFO" option to build command to generate compile
> > > information as part of BuildReport.
> > > This option generates files to be used by external tools as IDE's to
> > > enhance functionality.
> > > Files are created inside build folder:
> > > <Build>/<BuildTarget>/<ToolChain>/CompileInfo
> > >
> > > Files created:
> > > * compile_commands.json - Compilation Database. To be used by IDE's
> > > to enable advance features
> > > * cscope.files - List of files used in compilation. Used by Cscope to parse
> > > C code and provide browse functionality.
> > > * module_report.json - Module data form buildReport in Json format.
> > >
> > > Signed-off-by: Guillermo Antonio Palomino Sosa
> > > <guillermo.a.palomino.sosa@intel.com>
> > > ---
> > > BaseTools/Source/Python/build/BuildReport.py | 139
> > > +++++++++++++++++++-
> > > BaseTools/Source/Python/build/buildoptions.py | 4 +-
> > > 2 files changed, 140 insertions(+), 3 deletions(-)
> > >
> > > diff --git a/BaseTools/Source/Python/build/BuildReport.py
> > > b/BaseTools/Source/Python/build/BuildReport.py
> > > index 468772930c..33b43d471f 100644
> > > --- a/BaseTools/Source/Python/build/BuildReport.py
> > > +++ b/BaseTools/Source/Python/build/BuildReport.py
> > > @@ -10,6 +10,8 @@
> > >
> > > ## Import Modules
> > > #
> > > +import json
> > > +from pathlib import Path
> > > import Common.LongFilePathOs as os
> > > import re
> > > import platform
> > > @@ -41,6 +43,7 @@ from Common.DataType import * import
> collections
> > > from Common.Expression import * from GenFds.AprioriSection import
> > > DXE_APRIORI_GUID, PEI_APRIORI_GUID
> > > +from AutoGen.IncludesAutoGen import IncludesAutoGen
> > >
> > > ## Pattern to extract contents in EDK DXS files
> > > gDxsDependencyPattern =
> > > re.compile(r"DEPENDENCY_START(.+)DEPENDENCY_END", re.DOTALL)
> @@ -
> > > 2298,6 +2301,10 @@ class BuildReport(object):
> > > def GenerateReport(self, BuildDuration, AutoGenTime, MakeTime,
> > > GenFdsTime):
> > > if self.ReportFile:
> > > try:
> > > +
> > > + if "COMPILE_INFO" in self.ReportType:
> > > + self.GenerateCompileInfo()
> > > +
> > > File = []
> > > for (Wa, MaList) in self.ReportList:
> > > PlatformReport(Wa, MaList,
> > > self.ReportType).GenerateReport(File, BuildDuration, AutoGenTime,
> > > MakeTime, GenFdsTime, self.ReportType) @@ -2310,7 +2317,137 @@
> class
> > > BuildReport(object):
> > > EdkLogger.error("BuildReport", CODE_ERROR, "Unknown
> > > fatal error when generating build report",
> > > ExtraData=self.ReportFile,
> > > RaiseError=False)
> > > EdkLogger.quiet("(Python %s on %s\n%s)" %
> > > (platform.python_version(), sys.platform, traceback.format_exc()))
> > >
> > > +
> > > + ##
> > > + # Generates compile data files to be used by external tools.
> > > + # Compile information will be generated in
> > > <Build>/<BuildTarget>/<ToolChain>/CompileInfo
> > > + # Files generated: compile_commands.json, cscope.files,
> > > modules_report.json
> > > + #
> > > + # @param self The object pointer
> > > + #
> > > + def GenerateCompileInfo(self):
> > > + try:
> > > + # Lists for the output elements
> > > + compile_commands = []
> > > + used_files = set()
> > > + module_report = []
> > > +
> > > + for (Wa, MaList) in self.ReportList:
> > > + # Obtain list of all processed Workspace files
> > > + for file_path in Wa._GetMetaFiles(Wa.BuildTarget,
> Wa.ToolChain):
> > > + used_files.add(file_path)
> > > +
> > > + for autoGen in Wa.AutoGenObjectList:
> > > +
> > > + # Loop through all modules
> > > + for module in (autoGen.LibraryAutoGenList +
> > > autoGen.ModuleAutoGenList):
> > > +
> > > + used_files.add(module.MetaFile.Path)
> > > +
> > > + # Main elements of module report
> > > + module_report_data = {}
> > > + module_report_data["Name"] = module.Name
> > > + module_report_data["Arch"] = module.Arch
> > > + module_report_data["Path"] = module.MetaFile.Path
> > > + module_report_data["Guid"] = module.Guid
> > > + module_report_data["BuildType"] = module.BuildType
> > > + module_report_data["IsLibrary"] = module.IsLibrary
> > > + module_report_data["SourceDir"] = module.SourceDir
> > > + module_report_data["Files"] = []
> > > +
> > > + # Files used by module
> > > + for data_file in module.SourceFileList:
> > > + module_report_data["Files"].append({"Name":
> > > + data_file.Name, "Path": data_file.Path})
> > > +
> > > + # Libraries used by module
> > > + module_report_data["Libraries"] = []
> > > + for data_library in module.LibraryAutoGenList:
> > > +
> > > + module_report_data["Libraries"].append({"Path":
> > > + data_library.MetaFile.Path})
> > > +
> > > + # Packages used by module
> > > + module_report_data["Packages"] = []
> > > + for data_package in module.PackageList:
> > > + module_report_data["Packages"].append({"Path":
> > > data_package.MetaFile.Path, "Includes": []})
> > > + # Includes path used in package
> > > + for data_package_include in data_package.Includes:
> > > +
> > > + module_report_data["Packages"][-
> > > 1]["Includes"].append(data_package_inc
> > > + lude.Path)
> > > +
> > > + # PPI's in module
> > > + module_report_data["PPI"] = []
> > > + for data_ppi in module.PpiList.keys():
> > > + module_report_data["PPI"].append({"Name":
> > > + data_ppi, "Guid": module.PpiList[data_ppi]})
> > > +
> > > + # Protocol's in module
> > > + module_report_data["Protocol"] = []
> > > + for data_protocol in module.ProtocolList.keys():
> > > +
> > > + module_report_data["Protocol"].append({"Name": data_protocol,
> "Guid":
> > > + module.ProtocolList[data_protocol]})
> > > +
> > > + # PCD's in module
> > > + module_report_data["Pcd"] = []
> > > + for data_pcd in module.LibraryPcdList:
> > > + module_report_data["Pcd"].append({"Space":
> > > data_pcd.TokenSpaceGuidCName,
> > > + "Name": data_pcd.TokenCName,
> > > + "Value": data_pcd.TokenValue,
> > > + "Guid":
> data_pcd.TokenSpaceGuidValue,
> > > + "DatumType": data_pcd.DatumType,
> > > + "Type": data_pcd.Type,
> > > + "DefaultValue":
> data_pcd.DefaultValue})
> > > + # Add module to report
> > > + module_report.append(module_report_data)
> > > +
> > > + # Include file dependencies to used files
> > > + includes_autogen =
> > > + IncludesAutoGen(module.MakeFileDir,
> > > module)
> > > + for dep in includes_autogen.DepsCollection:
> > > + used_files.add(dep)
> > > +
> > > + inc_flag = "-I" # Default include flag
> > > + if module.BuildRuleFamily == TAB_COMPILER_MSFT:
> > > + inc_flag = "/I"
> > > +
> > > + for source in module.SourceFileList:
> > > + used_files.add(source.Path)
> > > + compile_command = {}
> > > + if source.Ext in [".c", ".cc", ".cpp"]:
> > > + #
> > > + # Generate compile command for each c file
> > > + #
> > > + compile_command["file"] = source.Path
> > > + compile_command["directory"] = source.Dir
> > > + build_command =
> > > module.BuildRules[source.Ext].CommandList[0]
> > > + build_command_variables =
> > > + re.findall(r"\$\((.*?)\)",
> > > build_command)
> > > + for var in build_command_variables:
> > > + var_tokens = var.split("_")
> > > + var_main = var_tokens[0]
> > > + if len(var_tokens) == 1:
> > > + var_value =
> module.BuildOption[var_main]["PATH"]
> > > + else:
> > > + var_value =
> > > module.BuildOption[var_main][var_tokens[1]]
> > > + build_command =
> > > + build_command.replace(f"$({var})",
> > > var_value)
> > > + include_files = f"
> {inc_flag}".join(module.IncludePathList)
> > > + build_command =
> > > + build_command.replace("${src}",
> > > include_files)
> > > + build_command =
> > > + build_command.replace("${dst}", module.OutputDir)
> > > +
> > > + # Remove un defined macros
> > > + compile_command["command"] =
> > > + re.sub(r"\$\(.*?\)", "",
> > > build_command)
> > > +
> > > + compile_commands.append(compile_command)
> > > +
> > > + # Create output folder if doesn't exist
> > > + compile_info_folder =
> Path(Wa.BuildDir).joinpath("CompileInfo")
> > > + compile_info_folder.mkdir(exist_ok=True)
> > > +
> > > + # Sort and save files
> > > + compile_commands.sort(key=lambda x: x["file"])
> > > +
> > > +
> > >
> SaveFileOnChange(compile_info_folder.joinpath(f"compile_commands.json"
> > > + ),json.dumps(compile_commands, indent=2), False)
> > > +
> > > +
> > > + SaveFileOnChange(compile_info_folder.joinpath(f"cscope.files"),
> > > + "\n".join(sorted(used_files)), False)
> > > +
> > > + module_report.sort(key=lambda x: x["Path"])
> > > +
> > > + SaveFileOnChange(compile_info_folder.joinpath(f"module_report.json
> > > + "), json.dumps(module_report, indent=2), False)
> > > +
> > > + except:
> > > + EdkLogger.error("BuildReport", CODE_ERROR, "Unknown
> > > + fatal error
> > > when generating build report compile information",
> > > ExtraData=self.ReportFile, RaiseError=False)
> > > + EdkLogger.quiet("(Python %s on %s\n%s)" %
> > > + (platform.python_version(), sys.platform, traceback.format_exc()))
> > > +
> > > # This acts like the main() function for the script, unless it is
> > > 'import'ed into another script.
> > > if __name__ == '__main__':
> > > pass
> > > -
> > > diff --git a/BaseTools/Source/Python/build/buildoptions.py
> > > b/BaseTools/Source/Python/build/buildoptions.py
> > > index 8334604b46..5ec561f7ec 100644
> > > --- a/BaseTools/Source/Python/build/buildoptions.py
> > > +++ b/BaseTools/Source/Python/build/buildoptions.py
> > > @@ -84,8 +84,8 @@ class MyOptionParser():
> > > Parser.add_option("-D", "--define", action="append",
> > > type="string", dest="Macros", help="Macro: \"Name [= Value]\".")
> > >
> > > Parser.add_option("-y", "--report-file", action="store",
> > > dest="ReportFile", help="Create/overwrite the report to the
> > > specified
> > > filename.")
> > > - Parser.add_option("-Y", "--report-type", action="append",
> > > type="choice", choices=['PCD', 'LIBRARY', 'FLASH', 'DEPEX',
> > > 'BUILD_FLAGS', 'FIXED_ADDRESS', 'HASH', 'EXECUTION_ORDER'],
> > > dest="ReportType", default=[],
> > > - help="Flags that control the type of build report to generate.
> Must be
> > > one of: [PCD, LIBRARY, FLASH, DEPEX, BUILD_FLAGS, FIXED_ADDRESS,
> > > HASH, EXECUTION_ORDER]. "\
> > > + Parser.add_option("-Y", "--report-type", action="append",
> > > type="choice", choices=['PCD', 'LIBRARY', 'FLASH', 'DEPEX',
> > > 'BUILD_FLAGS', 'FIXED_ADDRESS', 'HASH', 'EXECUTION_ORDER',
> > > 'COMPILE_INFO'], dest="ReportType", default=[],
> > > + help="Flags that control the type of build report to
> > > + generate. Must be one of: [PCD, LIBRARY, FLASH, DEPEX,
> > > + BUILD_FLAGS, FIXED_ADDRESS, HASH, EXECUTION_ORDER,
> COMPILE_INFO].
> > > + "\
> > > "To specify more than one flag, repeat this option
> > > on the command line and the default flag set is [PCD, LIBRARY,
> > > FLASH, DEPEX, HASH, BUILD_FLAGS, FIXED_ADDRESS]")
> > > Parser.add_option("-F", "--flag", action="store",
> > > type="string", dest="Flag",
> > > help="Specify the specific option to parse EDK UNI file. Must be
> one of:
> > > [-c, -s]. -c is for EDK framework UNI file, and -s is for EDK UEFI
> > > UNI file. "\
> > > --
> > > 2.28.0.windows.1
> >
> >
> >
> >
> >
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile information in build report
2023-02-15 5:38 ` Yuwei Chen
@ 2023-02-15 8:17 ` Ni, Ray
2023-02-15 23:12 ` Michael D Kinney
0 siblings, 1 reply; 17+ messages in thread
From: Ni, Ray @ 2023-02-15 8:17 UTC (permalink / raw)
To: devel@edk2.groups.io, Chen, Christine, Kinney, Michael D,
Palomino Sosa, Guillermo A
Cc: Feng, Bob C, Gao, Liming
Christine,
If BaseTools related changes is implemented in edk2-basetools repo,
does that mean if I only checkout edk2 repo, I am using an older version of BaseTools?
Thanks,
Ray
> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Yuwei
> Chen
> Sent: Wednesday, February 15, 2023 1:38 PM
> To: Kinney, Michael D <michael.d.kinney@intel.com>; devel@edk2.groups.io;
> Palomino Sosa, Guillermo A <guillermo.a.palomino.sosa@intel.com>
> Cc: Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming
> <gaoliming@byosoft.com.cn>
> Subject: Re: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile
> information in build report
>
> Hi Mike, thanks for reminder.
>
> Hi Willy, currently, BaseTools related changes will be implemented on the
> edk2-basetools repo. Please send the patch based on the edk2-basetools
> repo~
>
> Thanks,
> Christine
>
> > -----Original Message-----
> > From: Kinney, Michael D <michael.d.kinney@intel.com>
> > Sent: Wednesday, February 15, 2023 11:43 AM
> > To: devel@edk2.groups.io; Chen, Christine <yuwei.chen@intel.com>;
> > Palomino Sosa, Guillermo A <guillermo.a.palomino.sosa@intel.com>
> > Cc: Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming
> > <gaoliming@byosoft.com.cn>; Kinney, Michael D
> > <michael.d.kinney@intel.com>
> > Subject: RE: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile
> > information in build report
> >
> > Has this been reviewed for edk2-basetools repo?
> >
> > Mike
> >
> > > -----Original Message-----
> > > From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of
> Yuwei
> > > Chen
> > > Sent: Tuesday, February 14, 2023 6:44 PM
> > > To: Palomino Sosa, Guillermo A <guillermo.a.palomino.sosa@intel.com>;
> > > devel@edk2.groups.io
> > > Cc: Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming
> > > <gaoliming@byosoft.com.cn>
> > > Subject: Re: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile
> > > information in build report
> > >
> > > Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
> > >
> > > > -----Original Message-----
> > > > From: Palomino Sosa, Guillermo A
> > > > <guillermo.a.palomino.sosa@intel.com>
> > > > Sent: Tuesday, February 7, 2023 11:07 AM
> > > > To: devel@edk2.groups.io
> > > > Cc: Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming
> > > > <gaoliming@byosoft.com.cn>; Chen, Christine <yuwei.chen@intel.com>
> > > > Subject: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile
> > > > information in build report
> > > >
> > > > REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2850
> > > >
> > > > Add "-Y REPORT_INFO" option to build command to generate compile
> > > > information as part of BuildReport.
> > > > This option generates files to be used by external tools as IDE's to
> > > > enhance functionality.
> > > > Files are created inside build folder:
> > > > <Build>/<BuildTarget>/<ToolChain>/CompileInfo
> > > >
> > > > Files created:
> > > > * compile_commands.json - Compilation Database. To be used by IDE's
> > > > to enable advance features
> > > > * cscope.files - List of files used in compilation. Used by Cscope to parse
> > > > C code and provide browse functionality.
> > > > * module_report.json - Module data form buildReport in Json format.
> > > >
> > > > Signed-off-by: Guillermo Antonio Palomino Sosa
> > > > <guillermo.a.palomino.sosa@intel.com>
> > > > ---
> > > > BaseTools/Source/Python/build/BuildReport.py | 139
> > > > +++++++++++++++++++-
> > > > BaseTools/Source/Python/build/buildoptions.py | 4 +-
> > > > 2 files changed, 140 insertions(+), 3 deletions(-)
> > > >
> > > > diff --git a/BaseTools/Source/Python/build/BuildReport.py
> > > > b/BaseTools/Source/Python/build/BuildReport.py
> > > > index 468772930c..33b43d471f 100644
> > > > --- a/BaseTools/Source/Python/build/BuildReport.py
> > > > +++ b/BaseTools/Source/Python/build/BuildReport.py
> > > > @@ -10,6 +10,8 @@
> > > >
> > > > ## Import Modules
> > > > #
> > > > +import json
> > > > +from pathlib import Path
> > > > import Common.LongFilePathOs as os
> > > > import re
> > > > import platform
> > > > @@ -41,6 +43,7 @@ from Common.DataType import * import
> > collections
> > > > from Common.Expression import * from GenFds.AprioriSection import
> > > > DXE_APRIORI_GUID, PEI_APRIORI_GUID
> > > > +from AutoGen.IncludesAutoGen import IncludesAutoGen
> > > >
> > > > ## Pattern to extract contents in EDK DXS files
> > > > gDxsDependencyPattern =
> > > > re.compile(r"DEPENDENCY_START(.+)DEPENDENCY_END", re.DOTALL)
> > @@ -
> > > > 2298,6 +2301,10 @@ class BuildReport(object):
> > > > def GenerateReport(self, BuildDuration, AutoGenTime, MakeTime,
> > > > GenFdsTime):
> > > > if self.ReportFile:
> > > > try:
> > > > +
> > > > + if "COMPILE_INFO" in self.ReportType:
> > > > + self.GenerateCompileInfo()
> > > > +
> > > > File = []
> > > > for (Wa, MaList) in self.ReportList:
> > > > PlatformReport(Wa, MaList,
> > > > self.ReportType).GenerateReport(File, BuildDuration, AutoGenTime,
> > > > MakeTime, GenFdsTime, self.ReportType) @@ -2310,7 +2317,137 @@
> > class
> > > > BuildReport(object):
> > > > EdkLogger.error("BuildReport", CODE_ERROR, "Unknown
> > > > fatal error when generating build report",
> > > > ExtraData=self.ReportFile,
> > > > RaiseError=False)
> > > > EdkLogger.quiet("(Python %s on %s\n%s)" %
> > > > (platform.python_version(), sys.platform, traceback.format_exc()))
> > > >
> > > > +
> > > > + ##
> > > > + # Generates compile data files to be used by external tools.
> > > > + # Compile information will be generated in
> > > > <Build>/<BuildTarget>/<ToolChain>/CompileInfo
> > > > + # Files generated: compile_commands.json, cscope.files,
> > > > modules_report.json
> > > > + #
> > > > + # @param self The object pointer
> > > > + #
> > > > + def GenerateCompileInfo(self):
> > > > + try:
> > > > + # Lists for the output elements
> > > > + compile_commands = []
> > > > + used_files = set()
> > > > + module_report = []
> > > > +
> > > > + for (Wa, MaList) in self.ReportList:
> > > > + # Obtain list of all processed Workspace files
> > > > + for file_path in Wa._GetMetaFiles(Wa.BuildTarget,
> > Wa.ToolChain):
> > > > + used_files.add(file_path)
> > > > +
> > > > + for autoGen in Wa.AutoGenObjectList:
> > > > +
> > > > + # Loop through all modules
> > > > + for module in (autoGen.LibraryAutoGenList +
> > > > autoGen.ModuleAutoGenList):
> > > > +
> > > > + used_files.add(module.MetaFile.Path)
> > > > +
> > > > + # Main elements of module report
> > > > + module_report_data = {}
> > > > + module_report_data["Name"] = module.Name
> > > > + module_report_data["Arch"] = module.Arch
> > > > + module_report_data["Path"] = module.MetaFile.Path
> > > > + module_report_data["Guid"] = module.Guid
> > > > + module_report_data["BuildType"] = module.BuildType
> > > > + module_report_data["IsLibrary"] = module.IsLibrary
> > > > + module_report_data["SourceDir"] = module.SourceDir
> > > > + module_report_data["Files"] = []
> > > > +
> > > > + # Files used by module
> > > > + for data_file in module.SourceFileList:
> > > > + module_report_data["Files"].append({"Name":
> > > > + data_file.Name, "Path": data_file.Path})
> > > > +
> > > > + # Libraries used by module
> > > > + module_report_data["Libraries"] = []
> > > > + for data_library in module.LibraryAutoGenList:
> > > > +
> > > > + module_report_data["Libraries"].append({"Path":
> > > > + data_library.MetaFile.Path})
> > > > +
> > > > + # Packages used by module
> > > > + module_report_data["Packages"] = []
> > > > + for data_package in module.PackageList:
> > > > + module_report_data["Packages"].append({"Path":
> > > > data_package.MetaFile.Path, "Includes": []})
> > > > + # Includes path used in package
> > > > + for data_package_include in data_package.Includes:
> > > > +
> > > > + module_report_data["Packages"][-
> > > > 1]["Includes"].append(data_package_inc
> > > > + lude.Path)
> > > > +
> > > > + # PPI's in module
> > > > + module_report_data["PPI"] = []
> > > > + for data_ppi in module.PpiList.keys():
> > > > + module_report_data["PPI"].append({"Name":
> > > > + data_ppi, "Guid": module.PpiList[data_ppi]})
> > > > +
> > > > + # Protocol's in module
> > > > + module_report_data["Protocol"] = []
> > > > + for data_protocol in module.ProtocolList.keys():
> > > > +
> > > > + module_report_data["Protocol"].append({"Name": data_protocol,
> > "Guid":
> > > > + module.ProtocolList[data_protocol]})
> > > > +
> > > > + # PCD's in module
> > > > + module_report_data["Pcd"] = []
> > > > + for data_pcd in module.LibraryPcdList:
> > > > + module_report_data["Pcd"].append({"Space":
> > > > data_pcd.TokenSpaceGuidCName,
> > > > + "Name": data_pcd.TokenCName,
> > > > + "Value": data_pcd.TokenValue,
> > > > + "Guid":
> > data_pcd.TokenSpaceGuidValue,
> > > > + "DatumType": data_pcd.DatumType,
> > > > + "Type": data_pcd.Type,
> > > > + "DefaultValue":
> > data_pcd.DefaultValue})
> > > > + # Add module to report
> > > > + module_report.append(module_report_data)
> > > > +
> > > > + # Include file dependencies to used files
> > > > + includes_autogen =
> > > > + IncludesAutoGen(module.MakeFileDir,
> > > > module)
> > > > + for dep in includes_autogen.DepsCollection:
> > > > + used_files.add(dep)
> > > > +
> > > > + inc_flag = "-I" # Default include flag
> > > > + if module.BuildRuleFamily == TAB_COMPILER_MSFT:
> > > > + inc_flag = "/I"
> > > > +
> > > > + for source in module.SourceFileList:
> > > > + used_files.add(source.Path)
> > > > + compile_command = {}
> > > > + if source.Ext in [".c", ".cc", ".cpp"]:
> > > > + #
> > > > + # Generate compile command for each c file
> > > > + #
> > > > + compile_command["file"] = source.Path
> > > > + compile_command["directory"] = source.Dir
> > > > + build_command =
> > > > module.BuildRules[source.Ext].CommandList[0]
> > > > + build_command_variables =
> > > > + re.findall(r"\$\((.*?)\)",
> > > > build_command)
> > > > + for var in build_command_variables:
> > > > + var_tokens = var.split("_")
> > > > + var_main = var_tokens[0]
> > > > + if len(var_tokens) == 1:
> > > > + var_value =
> > module.BuildOption[var_main]["PATH"]
> > > > + else:
> > > > + var_value =
> > > > module.BuildOption[var_main][var_tokens[1]]
> > > > + build_command =
> > > > + build_command.replace(f"$({var})",
> > > > var_value)
> > > > + include_files = f"
> > {inc_flag}".join(module.IncludePathList)
> > > > + build_command =
> > > > + build_command.replace("${src}",
> > > > include_files)
> > > > + build_command =
> > > > + build_command.replace("${dst}", module.OutputDir)
> > > > +
> > > > + # Remove un defined macros
> > > > + compile_command["command"] =
> > > > + re.sub(r"\$\(.*?\)", "",
> > > > build_command)
> > > > +
> > > > + compile_commands.append(compile_command)
> > > > +
> > > > + # Create output folder if doesn't exist
> > > > + compile_info_folder =
> > Path(Wa.BuildDir).joinpath("CompileInfo")
> > > > + compile_info_folder.mkdir(exist_ok=True)
> > > > +
> > > > + # Sort and save files
> > > > + compile_commands.sort(key=lambda x: x["file"])
> > > > +
> > > > +
> > > >
> >
> SaveFileOnChange(compile_info_folder.joinpath(f"compile_commands.json
> "
> > > > + ),json.dumps(compile_commands, indent=2), False)
> > > > +
> > > > +
> > > > + SaveFileOnChange(compile_info_folder.joinpath(f"cscope.files"),
> > > > + "\n".join(sorted(used_files)), False)
> > > > +
> > > > + module_report.sort(key=lambda x: x["Path"])
> > > > +
> > > > +
> SaveFileOnChange(compile_info_folder.joinpath(f"module_report.json
> > > > + "), json.dumps(module_report, indent=2), False)
> > > > +
> > > > + except:
> > > > + EdkLogger.error("BuildReport", CODE_ERROR, "Unknown
> > > > + fatal error
> > > > when generating build report compile information",
> > > > ExtraData=self.ReportFile, RaiseError=False)
> > > > + EdkLogger.quiet("(Python %s on %s\n%s)" %
> > > > + (platform.python_version(), sys.platform, traceback.format_exc()))
> > > > +
> > > > # This acts like the main() function for the script, unless it is
> > > > 'import'ed into another script.
> > > > if __name__ == '__main__':
> > > > pass
> > > > -
> > > > diff --git a/BaseTools/Source/Python/build/buildoptions.py
> > > > b/BaseTools/Source/Python/build/buildoptions.py
> > > > index 8334604b46..5ec561f7ec 100644
> > > > --- a/BaseTools/Source/Python/build/buildoptions.py
> > > > +++ b/BaseTools/Source/Python/build/buildoptions.py
> > > > @@ -84,8 +84,8 @@ class MyOptionParser():
> > > > Parser.add_option("-D", "--define", action="append",
> > > > type="string", dest="Macros", help="Macro: \"Name [= Value]\".")
> > > >
> > > > Parser.add_option("-y", "--report-file", action="store",
> > > > dest="ReportFile", help="Create/overwrite the report to the
> > > > specified
> > > > filename.")
> > > > - Parser.add_option("-Y", "--report-type", action="append",
> > > > type="choice", choices=['PCD', 'LIBRARY', 'FLASH', 'DEPEX',
> > > > 'BUILD_FLAGS', 'FIXED_ADDRESS', 'HASH', 'EXECUTION_ORDER'],
> > > > dest="ReportType", default=[],
> > > > - help="Flags that control the type of build report to generate.
> > Must be
> > > > one of: [PCD, LIBRARY, FLASH, DEPEX, BUILD_FLAGS, FIXED_ADDRESS,
> > > > HASH, EXECUTION_ORDER]. "\
> > > > + Parser.add_option("-Y", "--report-type", action="append",
> > > > type="choice", choices=['PCD', 'LIBRARY', 'FLASH', 'DEPEX',
> > > > 'BUILD_FLAGS', 'FIXED_ADDRESS', 'HASH', 'EXECUTION_ORDER',
> > > > 'COMPILE_INFO'], dest="ReportType", default=[],
> > > > + help="Flags that control the type of build report to
> > > > + generate. Must be one of: [PCD, LIBRARY, FLASH, DEPEX,
> > > > + BUILD_FLAGS, FIXED_ADDRESS, HASH, EXECUTION_ORDER,
> > COMPILE_INFO].
> > > > + "\
> > > > "To specify more than one flag, repeat this option
> > > > on the command line and the default flag set is [PCD, LIBRARY,
> > > > FLASH, DEPEX, HASH, BUILD_FLAGS, FIXED_ADDRESS]")
> > > > Parser.add_option("-F", "--flag", action="store",
> > > > type="string", dest="Flag",
> > > > help="Specify the specific option to parse EDK UNI file. Must be
> > one of:
> > > > [-c, -s]. -c is for EDK framework UNI file, and -s is for EDK UEFI
> > > > UNI file. "\
> > > > --
> > > > 2.28.0.windows.1
> > >
> > >
> > >
> > >
> > >
>
>
>
>
>
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile information in build report
2023-02-15 8:17 ` Ni, Ray
@ 2023-02-15 23:12 ` Michael D Kinney
2023-02-16 0:50 ` Ni, Ray
0 siblings, 1 reply; 17+ messages in thread
From: Michael D Kinney @ 2023-02-15 23:12 UTC (permalink / raw)
To: Ni, Ray, devel@edk2.groups.io, Chen, Christine,
Palomino Sosa, Guillermo A
Cc: Feng, Bob C, Gao, Liming, Kinney, Michael D
Hi Ray,
Right now we want the commit in both places.
The priority is edk2-basetools first. It has more CI checks than edk2 repo for tools and
packages up as a pip module.
As soon as edk2-basetools change is merged, the edk2 repo change can be submitted and
merged because the reviews have already been completed by the BaseTools maintainers.
We do not like commit in 2 places. As soon as possible, we would like to see the
python sources removed from edk2/BaseTools and all devs/CI only use edk2-basetools
for python based tools.
Mike
> -----Original Message-----
> From: Ni, Ray <ray.ni@intel.com>
> Sent: Wednesday, February 15, 2023 12:17 AM
> To: devel@edk2.groups.io; Chen, Christine <yuwei.chen@intel.com>; Kinney, Michael D <michael.d.kinney@intel.com>; Palomino Sosa,
> Guillermo A <guillermo.a.palomino.sosa@intel.com>
> Cc: Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming <gaoliming@byosoft.com.cn>
> Subject: RE: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile information in build report
>
> Christine,
> If BaseTools related changes is implemented in edk2-basetools repo,
> does that mean if I only checkout edk2 repo, I am using an older version of BaseTools?
>
> Thanks,
> Ray
>
> > -----Original Message-----
> > From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Yuwei
> > Chen
> > Sent: Wednesday, February 15, 2023 1:38 PM
> > To: Kinney, Michael D <michael.d.kinney@intel.com>; devel@edk2.groups.io;
> > Palomino Sosa, Guillermo A <guillermo.a.palomino.sosa@intel.com>
> > Cc: Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming
> > <gaoliming@byosoft.com.cn>
> > Subject: Re: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile
> > information in build report
> >
> > Hi Mike, thanks for reminder.
> >
> > Hi Willy, currently, BaseTools related changes will be implemented on the
> > edk2-basetools repo. Please send the patch based on the edk2-basetools
> > repo~
> >
> > Thanks,
> > Christine
> >
> > > -----Original Message-----
> > > From: Kinney, Michael D <michael.d.kinney@intel.com>
> > > Sent: Wednesday, February 15, 2023 11:43 AM
> > > To: devel@edk2.groups.io; Chen, Christine <yuwei.chen@intel.com>;
> > > Palomino Sosa, Guillermo A <guillermo.a.palomino.sosa@intel.com>
> > > Cc: Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming
> > > <gaoliming@byosoft.com.cn>; Kinney, Michael D
> > > <michael.d.kinney@intel.com>
> > > Subject: RE: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile
> > > information in build report
> > >
> > > Has this been reviewed for edk2-basetools repo?
> > >
> > > Mike
> > >
> > > > -----Original Message-----
> > > > From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of
> > Yuwei
> > > > Chen
> > > > Sent: Tuesday, February 14, 2023 6:44 PM
> > > > To: Palomino Sosa, Guillermo A <guillermo.a.palomino.sosa@intel.com>;
> > > > devel@edk2.groups.io
> > > > Cc: Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming
> > > > <gaoliming@byosoft.com.cn>
> > > > Subject: Re: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile
> > > > information in build report
> > > >
> > > > Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
> > > >
> > > > > -----Original Message-----
> > > > > From: Palomino Sosa, Guillermo A
> > > > > <guillermo.a.palomino.sosa@intel.com>
> > > > > Sent: Tuesday, February 7, 2023 11:07 AM
> > > > > To: devel@edk2.groups.io
> > > > > Cc: Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming
> > > > > <gaoliming@byosoft.com.cn>; Chen, Christine <yuwei.chen@intel.com>
> > > > > Subject: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile
> > > > > information in build report
> > > > >
> > > > > REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2850
> > > > >
> > > > > Add "-Y REPORT_INFO" option to build command to generate compile
> > > > > information as part of BuildReport.
> > > > > This option generates files to be used by external tools as IDE's to
> > > > > enhance functionality.
> > > > > Files are created inside build folder:
> > > > > <Build>/<BuildTarget>/<ToolChain>/CompileInfo
> > > > >
> > > > > Files created:
> > > > > * compile_commands.json - Compilation Database. To be used by IDE's
> > > > > to enable advance features
> > > > > * cscope.files - List of files used in compilation. Used by Cscope to parse
> > > > > C code and provide browse functionality.
> > > > > * module_report.json - Module data form buildReport in Json format.
> > > > >
> > > > > Signed-off-by: Guillermo Antonio Palomino Sosa
> > > > > <guillermo.a.palomino.sosa@intel.com>
> > > > > ---
> > > > > BaseTools/Source/Python/build/BuildReport.py | 139
> > > > > +++++++++++++++++++-
> > > > > BaseTools/Source/Python/build/buildoptions.py | 4 +-
> > > > > 2 files changed, 140 insertions(+), 3 deletions(-)
> > > > >
> > > > > diff --git a/BaseTools/Source/Python/build/BuildReport.py
> > > > > b/BaseTools/Source/Python/build/BuildReport.py
> > > > > index 468772930c..33b43d471f 100644
> > > > > --- a/BaseTools/Source/Python/build/BuildReport.py
> > > > > +++ b/BaseTools/Source/Python/build/BuildReport.py
> > > > > @@ -10,6 +10,8 @@
> > > > >
> > > > > ## Import Modules
> > > > > #
> > > > > +import json
> > > > > +from pathlib import Path
> > > > > import Common.LongFilePathOs as os
> > > > > import re
> > > > > import platform
> > > > > @@ -41,6 +43,7 @@ from Common.DataType import * import
> > > collections
> > > > > from Common.Expression import * from GenFds.AprioriSection import
> > > > > DXE_APRIORI_GUID, PEI_APRIORI_GUID
> > > > > +from AutoGen.IncludesAutoGen import IncludesAutoGen
> > > > >
> > > > > ## Pattern to extract contents in EDK DXS files
> > > > > gDxsDependencyPattern =
> > > > > re.compile(r"DEPENDENCY_START(.+)DEPENDENCY_END", re.DOTALL)
> > > @@ -
> > > > > 2298,6 +2301,10 @@ class BuildReport(object):
> > > > > def GenerateReport(self, BuildDuration, AutoGenTime, MakeTime,
> > > > > GenFdsTime):
> > > > > if self.ReportFile:
> > > > > try:
> > > > > +
> > > > > + if "COMPILE_INFO" in self.ReportType:
> > > > > + self.GenerateCompileInfo()
> > > > > +
> > > > > File = []
> > > > > for (Wa, MaList) in self.ReportList:
> > > > > PlatformReport(Wa, MaList,
> > > > > self.ReportType).GenerateReport(File, BuildDuration, AutoGenTime,
> > > > > MakeTime, GenFdsTime, self.ReportType) @@ -2310,7 +2317,137 @@
> > > class
> > > > > BuildReport(object):
> > > > > EdkLogger.error("BuildReport", CODE_ERROR, "Unknown
> > > > > fatal error when generating build report",
> > > > > ExtraData=self.ReportFile,
> > > > > RaiseError=False)
> > > > > EdkLogger.quiet("(Python %s on %s\n%s)" %
> > > > > (platform.python_version(), sys.platform, traceback.format_exc()))
> > > > >
> > > > > +
> > > > > + ##
> > > > > + # Generates compile data files to be used by external tools.
> > > > > + # Compile information will be generated in
> > > > > <Build>/<BuildTarget>/<ToolChain>/CompileInfo
> > > > > + # Files generated: compile_commands.json, cscope.files,
> > > > > modules_report.json
> > > > > + #
> > > > > + # @param self The object pointer
> > > > > + #
> > > > > + def GenerateCompileInfo(self):
> > > > > + try:
> > > > > + # Lists for the output elements
> > > > > + compile_commands = []
> > > > > + used_files = set()
> > > > > + module_report = []
> > > > > +
> > > > > + for (Wa, MaList) in self.ReportList:
> > > > > + # Obtain list of all processed Workspace files
> > > > > + for file_path in Wa._GetMetaFiles(Wa.BuildTarget,
> > > Wa.ToolChain):
> > > > > + used_files.add(file_path)
> > > > > +
> > > > > + for autoGen in Wa.AutoGenObjectList:
> > > > > +
> > > > > + # Loop through all modules
> > > > > + for module in (autoGen.LibraryAutoGenList +
> > > > > autoGen.ModuleAutoGenList):
> > > > > +
> > > > > + used_files.add(module.MetaFile.Path)
> > > > > +
> > > > > + # Main elements of module report
> > > > > + module_report_data = {}
> > > > > + module_report_data["Name"] = module.Name
> > > > > + module_report_data["Arch"] = module.Arch
> > > > > + module_report_data["Path"] = module.MetaFile.Path
> > > > > + module_report_data["Guid"] = module.Guid
> > > > > + module_report_data["BuildType"] = module.BuildType
> > > > > + module_report_data["IsLibrary"] = module.IsLibrary
> > > > > + module_report_data["SourceDir"] = module.SourceDir
> > > > > + module_report_data["Files"] = []
> > > > > +
> > > > > + # Files used by module
> > > > > + for data_file in module.SourceFileList:
> > > > > + module_report_data["Files"].append({"Name":
> > > > > + data_file.Name, "Path": data_file.Path})
> > > > > +
> > > > > + # Libraries used by module
> > > > > + module_report_data["Libraries"] = []
> > > > > + for data_library in module.LibraryAutoGenList:
> > > > > +
> > > > > + module_report_data["Libraries"].append({"Path":
> > > > > + data_library.MetaFile.Path})
> > > > > +
> > > > > + # Packages used by module
> > > > > + module_report_data["Packages"] = []
> > > > > + for data_package in module.PackageList:
> > > > > + module_report_data["Packages"].append({"Path":
> > > > > data_package.MetaFile.Path, "Includes": []})
> > > > > + # Includes path used in package
> > > > > + for data_package_include in data_package.Includes:
> > > > > +
> > > > > + module_report_data["Packages"][-
> > > > > 1]["Includes"].append(data_package_inc
> > > > > + lude.Path)
> > > > > +
> > > > > + # PPI's in module
> > > > > + module_report_data["PPI"] = []
> > > > > + for data_ppi in module.PpiList.keys():
> > > > > + module_report_data["PPI"].append({"Name":
> > > > > + data_ppi, "Guid": module.PpiList[data_ppi]})
> > > > > +
> > > > > + # Protocol's in module
> > > > > + module_report_data["Protocol"] = []
> > > > > + for data_protocol in module.ProtocolList.keys():
> > > > > +
> > > > > + module_report_data["Protocol"].append({"Name": data_protocol,
> > > "Guid":
> > > > > + module.ProtocolList[data_protocol]})
> > > > > +
> > > > > + # PCD's in module
> > > > > + module_report_data["Pcd"] = []
> > > > > + for data_pcd in module.LibraryPcdList:
> > > > > + module_report_data["Pcd"].append({"Space":
> > > > > data_pcd.TokenSpaceGuidCName,
> > > > > + "Name": data_pcd.TokenCName,
> > > > > + "Value": data_pcd.TokenValue,
> > > > > + "Guid":
> > > data_pcd.TokenSpaceGuidValue,
> > > > > + "DatumType": data_pcd.DatumType,
> > > > > + "Type": data_pcd.Type,
> > > > > + "DefaultValue":
> > > data_pcd.DefaultValue})
> > > > > + # Add module to report
> > > > > + module_report.append(module_report_data)
> > > > > +
> > > > > + # Include file dependencies to used files
> > > > > + includes_autogen =
> > > > > + IncludesAutoGen(module.MakeFileDir,
> > > > > module)
> > > > > + for dep in includes_autogen.DepsCollection:
> > > > > + used_files.add(dep)
> > > > > +
> > > > > + inc_flag = "-I" # Default include flag
> > > > > + if module.BuildRuleFamily == TAB_COMPILER_MSFT:
> > > > > + inc_flag = "/I"
> > > > > +
> > > > > + for source in module.SourceFileList:
> > > > > + used_files.add(source.Path)
> > > > > + compile_command = {}
> > > > > + if source.Ext in [".c", ".cc", ".cpp"]:
> > > > > + #
> > > > > + # Generate compile command for each c file
> > > > > + #
> > > > > + compile_command["file"] = source.Path
> > > > > + compile_command["directory"] = source.Dir
> > > > > + build_command =
> > > > > module.BuildRules[source.Ext].CommandList[0]
> > > > > + build_command_variables =
> > > > > + re.findall(r"\$\((.*?)\)",
> > > > > build_command)
> > > > > + for var in build_command_variables:
> > > > > + var_tokens = var.split("_")
> > > > > + var_main = var_tokens[0]
> > > > > + if len(var_tokens) == 1:
> > > > > + var_value =
> > > module.BuildOption[var_main]["PATH"]
> > > > > + else:
> > > > > + var_value =
> > > > > module.BuildOption[var_main][var_tokens[1]]
> > > > > + build_command =
> > > > > + build_command.replace(f"$({var})",
> > > > > var_value)
> > > > > + include_files = f"
> > > {inc_flag}".join(module.IncludePathList)
> > > > > + build_command =
> > > > > + build_command.replace("${src}",
> > > > > include_files)
> > > > > + build_command =
> > > > > + build_command.replace("${dst}", module.OutputDir)
> > > > > +
> > > > > + # Remove un defined macros
> > > > > + compile_command["command"] =
> > > > > + re.sub(r"\$\(.*?\)", "",
> > > > > build_command)
> > > > > +
> > > > > + compile_commands.append(compile_command)
> > > > > +
> > > > > + # Create output folder if doesn't exist
> > > > > + compile_info_folder =
> > > Path(Wa.BuildDir).joinpath("CompileInfo")
> > > > > + compile_info_folder.mkdir(exist_ok=True)
> > > > > +
> > > > > + # Sort and save files
> > > > > + compile_commands.sort(key=lambda x: x["file"])
> > > > > +
> > > > > +
> > > > >
> > >
> > SaveFileOnChange(compile_info_folder.joinpath(f"compile_commands.json
> > "
> > > > > + ),json.dumps(compile_commands, indent=2), False)
> > > > > +
> > > > > +
> > > > > + SaveFileOnChange(compile_info_folder.joinpath(f"cscope.files"),
> > > > > + "\n".join(sorted(used_files)), False)
> > > > > +
> > > > > + module_report.sort(key=lambda x: x["Path"])
> > > > > +
> > > > > +
> > SaveFileOnChange(compile_info_folder.joinpath(f"module_report.json
> > > > > + "), json.dumps(module_report, indent=2), False)
> > > > > +
> > > > > + except:
> > > > > + EdkLogger.error("BuildReport", CODE_ERROR, "Unknown
> > > > > + fatal error
> > > > > when generating build report compile information",
> > > > > ExtraData=self.ReportFile, RaiseError=False)
> > > > > + EdkLogger.quiet("(Python %s on %s\n%s)" %
> > > > > + (platform.python_version(), sys.platform, traceback.format_exc()))
> > > > > +
> > > > > # This acts like the main() function for the script, unless it is
> > > > > 'import'ed into another script.
> > > > > if __name__ == '__main__':
> > > > > pass
> > > > > -
> > > > > diff --git a/BaseTools/Source/Python/build/buildoptions.py
> > > > > b/BaseTools/Source/Python/build/buildoptions.py
> > > > > index 8334604b46..5ec561f7ec 100644
> > > > > --- a/BaseTools/Source/Python/build/buildoptions.py
> > > > > +++ b/BaseTools/Source/Python/build/buildoptions.py
> > > > > @@ -84,8 +84,8 @@ class MyOptionParser():
> > > > > Parser.add_option("-D", "--define", action="append",
> > > > > type="string", dest="Macros", help="Macro: \"Name [= Value]\".")
> > > > >
> > > > > Parser.add_option("-y", "--report-file", action="store",
> > > > > dest="ReportFile", help="Create/overwrite the report to the
> > > > > specified
> > > > > filename.")
> > > > > - Parser.add_option("-Y", "--report-type", action="append",
> > > > > type="choice", choices=['PCD', 'LIBRARY', 'FLASH', 'DEPEX',
> > > > > 'BUILD_FLAGS', 'FIXED_ADDRESS', 'HASH', 'EXECUTION_ORDER'],
> > > > > dest="ReportType", default=[],
> > > > > - help="Flags that control the type of build report to generate.
> > > Must be
> > > > > one of: [PCD, LIBRARY, FLASH, DEPEX, BUILD_FLAGS, FIXED_ADDRESS,
> > > > > HASH, EXECUTION_ORDER]. "\
> > > > > + Parser.add_option("-Y", "--report-type", action="append",
> > > > > type="choice", choices=['PCD', 'LIBRARY', 'FLASH', 'DEPEX',
> > > > > 'BUILD_FLAGS', 'FIXED_ADDRESS', 'HASH', 'EXECUTION_ORDER',
> > > > > 'COMPILE_INFO'], dest="ReportType", default=[],
> > > > > + help="Flags that control the type of build report to
> > > > > + generate. Must be one of: [PCD, LIBRARY, FLASH, DEPEX,
> > > > > + BUILD_FLAGS, FIXED_ADDRESS, HASH, EXECUTION_ORDER,
> > > COMPILE_INFO].
> > > > > + "\
> > > > > "To specify more than one flag, repeat this option
> > > > > on the command line and the default flag set is [PCD, LIBRARY,
> > > > > FLASH, DEPEX, HASH, BUILD_FLAGS, FIXED_ADDRESS]")
> > > > > Parser.add_option("-F", "--flag", action="store",
> > > > > type="string", dest="Flag",
> > > > > help="Specify the specific option to parse EDK UNI file. Must be
> > > one of:
> > > > > [-c, -s]. -c is for EDK framework UNI file, and -s is for EDK UEFI
> > > > > UNI file. "\
> > > > > --
> > > > > 2.28.0.windows.1
> > > >
> > > >
> > > >
> > > >
> > > >
> >
> >
> >
> >
> >
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile information in build report
2023-02-15 23:12 ` Michael D Kinney
@ 2023-02-16 0:50 ` Ni, Ray
2023-02-18 4:42 ` [edk2-devel] [PATCH " Guillermo Antonio Palomino Sosa
2023-02-27 17:40 ` [edk2-devel][PATCH " Guillermo Antonio Palomino Sosa
0 siblings, 2 replies; 17+ messages in thread
From: Ni, Ray @ 2023-02-16 0:50 UTC (permalink / raw)
To: Kinney, Michael D, devel@edk2.groups.io, Chen, Christine,
Palomino Sosa, Guillermo A
Cc: Feng, Bob C, Gao, Liming
Mike,
Thanks for the explanation of status and plan.
> -----Original Message-----
> From: Kinney, Michael D <michael.d.kinney@intel.com>
> Sent: Thursday, February 16, 2023 7:13 AM
> To: Ni, Ray <ray.ni@intel.com>; devel@edk2.groups.io; Chen, Christine
> <yuwei.chen@intel.com>; Palomino Sosa, Guillermo A
> <guillermo.a.palomino.sosa@intel.com>
> Cc: Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming
> <gaoliming@byosoft.com.cn>; Kinney, Michael D
> <michael.d.kinney@intel.com>
> Subject: RE: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile
> information in build report
>
> Hi Ray,
>
> Right now we want the commit in both places.
>
> The priority is edk2-basetools first. It has more CI checks than edk2 repo for
> tools and
> packages up as a pip module.
>
> As soon as edk2-basetools change is merged, the edk2 repo change can be
> submitted and
> merged because the reviews have already been completed by the BaseTools
> maintainers.
>
> We do not like commit in 2 places. As soon as possible, we would like to see
> the
> python sources removed from edk2/BaseTools and all devs/CI only use edk2-
> basetools
> for python based tools.
>
> Mike
>
> > -----Original Message-----
> > From: Ni, Ray <ray.ni@intel.com>
> > Sent: Wednesday, February 15, 2023 12:17 AM
> > To: devel@edk2.groups.io; Chen, Christine <yuwei.chen@intel.com>;
> Kinney, Michael D <michael.d.kinney@intel.com>; Palomino Sosa,
> > Guillermo A <guillermo.a.palomino.sosa@intel.com>
> > Cc: Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming
> <gaoliming@byosoft.com.cn>
> > Subject: RE: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile
> information in build report
> >
> > Christine,
> > If BaseTools related changes is implemented in edk2-basetools repo,
> > does that mean if I only checkout edk2 repo, I am using an older version of
> BaseTools?
> >
> > Thanks,
> > Ray
> >
> > > -----Original Message-----
> > > From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of
> Yuwei
> > > Chen
> > > Sent: Wednesday, February 15, 2023 1:38 PM
> > > To: Kinney, Michael D <michael.d.kinney@intel.com>;
> devel@edk2.groups.io;
> > > Palomino Sosa, Guillermo A <guillermo.a.palomino.sosa@intel.com>
> > > Cc: Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming
> > > <gaoliming@byosoft.com.cn>
> > > Subject: Re: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile
> > > information in build report
> > >
> > > Hi Mike, thanks for reminder.
> > >
> > > Hi Willy, currently, BaseTools related changes will be implemented on the
> > > edk2-basetools repo. Please send the patch based on the edk2-basetools
> > > repo~
> > >
> > > Thanks,
> > > Christine
> > >
> > > > -----Original Message-----
> > > > From: Kinney, Michael D <michael.d.kinney@intel.com>
> > > > Sent: Wednesday, February 15, 2023 11:43 AM
> > > > To: devel@edk2.groups.io; Chen, Christine <yuwei.chen@intel.com>;
> > > > Palomino Sosa, Guillermo A <guillermo.a.palomino.sosa@intel.com>
> > > > Cc: Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming
> > > > <gaoliming@byosoft.com.cn>; Kinney, Michael D
> > > > <michael.d.kinney@intel.com>
> > > > Subject: RE: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile
> > > > information in build report
> > > >
> > > > Has this been reviewed for edk2-basetools repo?
> > > >
> > > > Mike
> > > >
> > > > > -----Original Message-----
> > > > > From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of
> > > Yuwei
> > > > > Chen
> > > > > Sent: Tuesday, February 14, 2023 6:44 PM
> > > > > To: Palomino Sosa, Guillermo A
> <guillermo.a.palomino.sosa@intel.com>;
> > > > > devel@edk2.groups.io
> > > > > Cc: Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming
> > > > > <gaoliming@byosoft.com.cn>
> > > > > Subject: Re: [edk2-devel][PATCH V1 1/1] BaseTools: Generate
> compile
> > > > > information in build report
> > > > >
> > > > > Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: Palomino Sosa, Guillermo A
> > > > > > <guillermo.a.palomino.sosa@intel.com>
> > > > > > Sent: Tuesday, February 7, 2023 11:07 AM
> > > > > > To: devel@edk2.groups.io
> > > > > > Cc: Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming
> > > > > > <gaoliming@byosoft.com.cn>; Chen, Christine
> <yuwei.chen@intel.com>
> > > > > > Subject: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile
> > > > > > information in build report
> > > > > >
> > > > > > REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2850
> > > > > >
> > > > > > Add "-Y REPORT_INFO" option to build command to generate
> compile
> > > > > > information as part of BuildReport.
> > > > > > This option generates files to be used by external tools as IDE's to
> > > > > > enhance functionality.
> > > > > > Files are created inside build folder:
> > > > > > <Build>/<BuildTarget>/<ToolChain>/CompileInfo
> > > > > >
> > > > > > Files created:
> > > > > > * compile_commands.json - Compilation Database. To be used by
> IDE's
> > > > > > to enable advance features
> > > > > > * cscope.files - List of files used in compilation. Used by Cscope to
> parse
> > > > > > C code and provide browse functionality.
> > > > > > * module_report.json - Module data form buildReport in Json
> format.
> > > > > >
> > > > > > Signed-off-by: Guillermo Antonio Palomino Sosa
> > > > > > <guillermo.a.palomino.sosa@intel.com>
> > > > > > ---
> > > > > > BaseTools/Source/Python/build/BuildReport.py | 139
> > > > > > +++++++++++++++++++-
> > > > > > BaseTools/Source/Python/build/buildoptions.py | 4 +-
> > > > > > 2 files changed, 140 insertions(+), 3 deletions(-)
> > > > > >
> > > > > > diff --git a/BaseTools/Source/Python/build/BuildReport.py
> > > > > > b/BaseTools/Source/Python/build/BuildReport.py
> > > > > > index 468772930c..33b43d471f 100644
> > > > > > --- a/BaseTools/Source/Python/build/BuildReport.py
> > > > > > +++ b/BaseTools/Source/Python/build/BuildReport.py
> > > > > > @@ -10,6 +10,8 @@
> > > > > >
> > > > > > ## Import Modules
> > > > > > #
> > > > > > +import json
> > > > > > +from pathlib import Path
> > > > > > import Common.LongFilePathOs as os
> > > > > > import re
> > > > > > import platform
> > > > > > @@ -41,6 +43,7 @@ from Common.DataType import * import
> > > > collections
> > > > > > from Common.Expression import * from GenFds.AprioriSection
> import
> > > > > > DXE_APRIORI_GUID, PEI_APRIORI_GUID
> > > > > > +from AutoGen.IncludesAutoGen import IncludesAutoGen
> > > > > >
> > > > > > ## Pattern to extract contents in EDK DXS files
> > > > > > gDxsDependencyPattern =
> > > > > > re.compile(r"DEPENDENCY_START(.+)DEPENDENCY_END",
> re.DOTALL)
> > > > @@ -
> > > > > > 2298,6 +2301,10 @@ class BuildReport(object):
> > > > > > def GenerateReport(self, BuildDuration, AutoGenTime,
> MakeTime,
> > > > > > GenFdsTime):
> > > > > > if self.ReportFile:
> > > > > > try:
> > > > > > +
> > > > > > + if "COMPILE_INFO" in self.ReportType:
> > > > > > + self.GenerateCompileInfo()
> > > > > > +
> > > > > > File = []
> > > > > > for (Wa, MaList) in self.ReportList:
> > > > > > PlatformReport(Wa, MaList,
> > > > > > self.ReportType).GenerateReport(File, BuildDuration,
> AutoGenTime,
> > > > > > MakeTime, GenFdsTime, self.ReportType) @@ -2310,7 +2317,137
> @@
> > > > class
> > > > > > BuildReport(object):
> > > > > > EdkLogger.error("BuildReport", CODE_ERROR, "Unknown
> > > > > > fatal error when generating build report",
> > > > > > ExtraData=self.ReportFile,
> > > > > > RaiseError=False)
> > > > > > EdkLogger.quiet("(Python %s on %s\n%s)" %
> > > > > > (platform.python_version(), sys.platform, traceback.format_exc()))
> > > > > >
> > > > > > +
> > > > > > + ##
> > > > > > + # Generates compile data files to be used by external tools.
> > > > > > + # Compile information will be generated in
> > > > > > <Build>/<BuildTarget>/<ToolChain>/CompileInfo
> > > > > > + # Files generated: compile_commands.json, cscope.files,
> > > > > > modules_report.json
> > > > > > + #
> > > > > > + # @param self The object pointer
> > > > > > + #
> > > > > > + def GenerateCompileInfo(self):
> > > > > > + try:
> > > > > > + # Lists for the output elements
> > > > > > + compile_commands = []
> > > > > > + used_files = set()
> > > > > > + module_report = []
> > > > > > +
> > > > > > + for (Wa, MaList) in self.ReportList:
> > > > > > + # Obtain list of all processed Workspace files
> > > > > > + for file_path in Wa._GetMetaFiles(Wa.BuildTarget,
> > > > Wa.ToolChain):
> > > > > > + used_files.add(file_path)
> > > > > > +
> > > > > > + for autoGen in Wa.AutoGenObjectList:
> > > > > > +
> > > > > > + # Loop through all modules
> > > > > > + for module in (autoGen.LibraryAutoGenList +
> > > > > > autoGen.ModuleAutoGenList):
> > > > > > +
> > > > > > + used_files.add(module.MetaFile.Path)
> > > > > > +
> > > > > > + # Main elements of module report
> > > > > > + module_report_data = {}
> > > > > > + module_report_data["Name"] = module.Name
> > > > > > + module_report_data["Arch"] = module.Arch
> > > > > > + module_report_data["Path"] = module.MetaFile.Path
> > > > > > + module_report_data["Guid"] = module.Guid
> > > > > > + module_report_data["BuildType"] =
> module.BuildType
> > > > > > + module_report_data["IsLibrary"] = module.IsLibrary
> > > > > > + module_report_data["SourceDir"] = module.SourceDir
> > > > > > + module_report_data["Files"] = []
> > > > > > +
> > > > > > + # Files used by module
> > > > > > + for data_file in module.SourceFileList:
> > > > > > + module_report_data["Files"].append({"Name":
> > > > > > + data_file.Name, "Path": data_file.Path})
> > > > > > +
> > > > > > + # Libraries used by module
> > > > > > + module_report_data["Libraries"] = []
> > > > > > + for data_library in module.LibraryAutoGenList:
> > > > > > +
> > > > > > + module_report_data["Libraries"].append({"Path":
> > > > > > + data_library.MetaFile.Path})
> > > > > > +
> > > > > > + # Packages used by module
> > > > > > + module_report_data["Packages"] = []
> > > > > > + for data_package in module.PackageList:
> > > > > > + module_report_data["Packages"].append({"Path":
> > > > > > data_package.MetaFile.Path, "Includes": []})
> > > > > > + # Includes path used in package
> > > > > > + for data_package_include in data_package.Includes:
> > > > > > +
> > > > > > + module_report_data["Packages"][-
> > > > > > 1]["Includes"].append(data_package_inc
> > > > > > + lude.Path)
> > > > > > +
> > > > > > + # PPI's in module
> > > > > > + module_report_data["PPI"] = []
> > > > > > + for data_ppi in module.PpiList.keys():
> > > > > > + module_report_data["PPI"].append({"Name":
> > > > > > + data_ppi, "Guid": module.PpiList[data_ppi]})
> > > > > > +
> > > > > > + # Protocol's in module
> > > > > > + module_report_data["Protocol"] = []
> > > > > > + for data_protocol in module.ProtocolList.keys():
> > > > > > +
> > > > > > + module_report_data["Protocol"].append({"Name":
> data_protocol,
> > > > "Guid":
> > > > > > + module.ProtocolList[data_protocol]})
> > > > > > +
> > > > > > + # PCD's in module
> > > > > > + module_report_data["Pcd"] = []
> > > > > > + for data_pcd in module.LibraryPcdList:
> > > > > > + module_report_data["Pcd"].append({"Space":
> > > > > > data_pcd.TokenSpaceGuidCName,
> > > > > > + "Name": data_pcd.TokenCName,
> > > > > > + "Value": data_pcd.TokenValue,
> > > > > > + "Guid":
> > > > data_pcd.TokenSpaceGuidValue,
> > > > > > + "DatumType":
> data_pcd.DatumType,
> > > > > > + "Type": data_pcd.Type,
> > > > > > + "DefaultValue":
> > > > data_pcd.DefaultValue})
> > > > > > + # Add module to report
> > > > > > + module_report.append(module_report_data)
> > > > > > +
> > > > > > + # Include file dependencies to used files
> > > > > > + includes_autogen =
> > > > > > + IncludesAutoGen(module.MakeFileDir,
> > > > > > module)
> > > > > > + for dep in includes_autogen.DepsCollection:
> > > > > > + used_files.add(dep)
> > > > > > +
> > > > > > + inc_flag = "-I" # Default include flag
> > > > > > + if module.BuildRuleFamily == TAB_COMPILER_MSFT:
> > > > > > + inc_flag = "/I"
> > > > > > +
> > > > > > + for source in module.SourceFileList:
> > > > > > + used_files.add(source.Path)
> > > > > > + compile_command = {}
> > > > > > + if source.Ext in [".c", ".cc", ".cpp"]:
> > > > > > + #
> > > > > > + # Generate compile command for each c file
> > > > > > + #
> > > > > > + compile_command["file"] = source.Path
> > > > > > + compile_command["directory"] = source.Dir
> > > > > > + build_command =
> > > > > > module.BuildRules[source.Ext].CommandList[0]
> > > > > > + build_command_variables =
> > > > > > + re.findall(r"\$\((.*?)\)",
> > > > > > build_command)
> > > > > > + for var in build_command_variables:
> > > > > > + var_tokens = var.split("_")
> > > > > > + var_main = var_tokens[0]
> > > > > > + if len(var_tokens) == 1:
> > > > > > + var_value =
> > > > module.BuildOption[var_main]["PATH"]
> > > > > > + else:
> > > > > > + var_value =
> > > > > > module.BuildOption[var_main][var_tokens[1]]
> > > > > > + build_command =
> > > > > > + build_command.replace(f"$({var})",
> > > > > > var_value)
> > > > > > + include_files = f"
> > > > {inc_flag}".join(module.IncludePathList)
> > > > > > + build_command =
> > > > > > + build_command.replace("${src}",
> > > > > > include_files)
> > > > > > + build_command =
> > > > > > + build_command.replace("${dst}", module.OutputDir)
> > > > > > +
> > > > > > + # Remove un defined macros
> > > > > > + compile_command["command"] =
> > > > > > + re.sub(r"\$\(.*?\)", "",
> > > > > > build_command)
> > > > > > +
> > > > > > + compile_commands.append(compile_command)
> > > > > > +
> > > > > > + # Create output folder if doesn't exist
> > > > > > + compile_info_folder =
> > > > Path(Wa.BuildDir).joinpath("CompileInfo")
> > > > > > + compile_info_folder.mkdir(exist_ok=True)
> > > > > > +
> > > > > > + # Sort and save files
> > > > > > + compile_commands.sort(key=lambda x: x["file"])
> > > > > > +
> > > > > > +
> > > > > >
> > > >
> > >
> SaveFileOnChange(compile_info_folder.joinpath(f"compile_commands.json
> > > "
> > > > > > + ),json.dumps(compile_commands, indent=2), False)
> > > > > > +
> > > > > > +
> > > > > > + SaveFileOnChange(compile_info_folder.joinpath(f"cscope.files"),
> > > > > > + "\n".join(sorted(used_files)), False)
> > > > > > +
> > > > > > + module_report.sort(key=lambda x: x["Path"])
> > > > > > +
> > > > > > +
> > > SaveFileOnChange(compile_info_folder.joinpath(f"module_report.json
> > > > > > + "), json.dumps(module_report, indent=2), False)
> > > > > > +
> > > > > > + except:
> > > > > > + EdkLogger.error("BuildReport", CODE_ERROR, "Unknown
> > > > > > + fatal error
> > > > > > when generating build report compile information",
> > > > > > ExtraData=self.ReportFile, RaiseError=False)
> > > > > > + EdkLogger.quiet("(Python %s on %s\n%s)" %
> > > > > > + (platform.python_version(), sys.platform,
> traceback.format_exc()))
> > > > > > +
> > > > > > # This acts like the main() function for the script, unless it is
> > > > > > 'import'ed into another script.
> > > > > > if __name__ == '__main__':
> > > > > > pass
> > > > > > -
> > > > > > diff --git a/BaseTools/Source/Python/build/buildoptions.py
> > > > > > b/BaseTools/Source/Python/build/buildoptions.py
> > > > > > index 8334604b46..5ec561f7ec 100644
> > > > > > --- a/BaseTools/Source/Python/build/buildoptions.py
> > > > > > +++ b/BaseTools/Source/Python/build/buildoptions.py
> > > > > > @@ -84,8 +84,8 @@ class MyOptionParser():
> > > > > > Parser.add_option("-D", "--define", action="append",
> > > > > > type="string", dest="Macros", help="Macro: \"Name [= Value]\".")
> > > > > >
> > > > > > Parser.add_option("-y", "--report-file", action="store",
> > > > > > dest="ReportFile", help="Create/overwrite the report to the
> > > > > > specified
> > > > > > filename.")
> > > > > > - Parser.add_option("-Y", "--report-type", action="append",
> > > > > > type="choice", choices=['PCD', 'LIBRARY', 'FLASH', 'DEPEX',
> > > > > > 'BUILD_FLAGS', 'FIXED_ADDRESS', 'HASH', 'EXECUTION_ORDER'],
> > > > > > dest="ReportType", default=[],
> > > > > > - help="Flags that control the type of build report to generate.
> > > > Must be
> > > > > > one of: [PCD, LIBRARY, FLASH, DEPEX, BUILD_FLAGS,
> FIXED_ADDRESS,
> > > > > > HASH, EXECUTION_ORDER]. "\
> > > > > > + Parser.add_option("-Y", "--report-type", action="append",
> > > > > > type="choice", choices=['PCD', 'LIBRARY', 'FLASH', 'DEPEX',
> > > > > > 'BUILD_FLAGS', 'FIXED_ADDRESS', 'HASH', 'EXECUTION_ORDER',
> > > > > > 'COMPILE_INFO'], dest="ReportType", default=[],
> > > > > > + help="Flags that control the type of build report to
> > > > > > + generate. Must be one of: [PCD, LIBRARY, FLASH, DEPEX,
> > > > > > + BUILD_FLAGS, FIXED_ADDRESS, HASH, EXECUTION_ORDER,
> > > > COMPILE_INFO].
> > > > > > + "\
> > > > > > "To specify more than one flag, repeat this option
> > > > > > on the command line and the default flag set is [PCD, LIBRARY,
> > > > > > FLASH, DEPEX, HASH, BUILD_FLAGS, FIXED_ADDRESS]")
> > > > > > Parser.add_option("-F", "--flag", action="store",
> > > > > > type="string", dest="Flag",
> > > > > > help="Specify the specific option to parse EDK UNI file. Must
> be
> > > > one of:
> > > > > > [-c, -s]. -c is for EDK framework UNI file, and -s is for EDK UEFI
> > > > > > UNI file. "\
> > > > > > --
> > > > > > 2.28.0.windows.1
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > >
> > >
> > >
> > >
> > >
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [edk2-devel] [PATCH V1 1/1] BaseTools: Generate compile information in build report
2023-02-16 0:50 ` Ni, Ray
@ 2023-02-18 4:42 ` Guillermo Antonio Palomino Sosa
2023-02-27 17:40 ` [edk2-devel][PATCH " Guillermo Antonio Palomino Sosa
1 sibling, 0 replies; 17+ messages in thread
From: Guillermo Antonio Palomino Sosa @ 2023-02-18 4:42 UTC (permalink / raw)
To: Ni, Ray, devel
[-- Attachment #1: Type: text/plain, Size: 75 bytes --]
Thanks guys. I will create a new patch based on edk2-basetools next week.
[-- Attachment #2: Type: text/html, Size: 124 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile information in build report
2023-02-16 0:50 ` Ni, Ray
2023-02-18 4:42 ` [edk2-devel] [PATCH " Guillermo Antonio Palomino Sosa
@ 2023-02-27 17:40 ` Guillermo Antonio Palomino Sosa
2023-02-27 18:57 ` Ard Biesheuvel
1 sibling, 1 reply; 17+ messages in thread
From: Guillermo Antonio Palomino Sosa @ 2023-02-27 17:40 UTC (permalink / raw)
To: Ni, Ray, Kinney, Michael D, devel@edk2.groups.io, Chen, Christine
Cc: Feng, Bob C, Gao, Liming, Oram, Isaac W
[-- Attachment #1: Type: text/plain, Size: 21836 bytes --]
Hi. I have submitted a pull request to edk2-basetools repository: https://github.com/tianocore/edk2-basetools/pull/88
This is the feature request for it: https://github.com/tianocore/edk2-basetools/issues/87
I'm also attaching the patch here: (0001-BaseTools-Generate-compile-information-in-build-repo.patch)
On a side note, seems like tip of edk2-basetools is broken due this commit that makes direct import of Common package:
https://github.com/tianocore/edk2-basetools/commit/8e6018d3ea4c1aae7185f589d129cea14a5d89fd
edk2-basetools\edk2basetools\GenFds\SubTypeGuidSection.py:
import Common.LongFilePathOs as os
-----Original Message-----
From: Ni, Ray <ray.ni@intel.com>
Sent: Wednesday, February 15, 2023 6:51 PM
To: Kinney, Michael D <michael.d.kinney@intel.com>; devel@edk2.groups.io; Chen, Christine <yuwei.chen@intel.com>; Palomino Sosa, Guillermo A <guillermo.a.palomino.sosa@intel.com>
Cc: Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming <gaoliming@byosoft.com.cn>
Subject: RE: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile information in build report
Mike,
Thanks for the explanation of status and plan.
> -----Original Message-----
> From: Kinney, Michael D <michael.d.kinney@intel.com>
> Sent: Thursday, February 16, 2023 7:13 AM
> To: Ni, Ray <ray.ni@intel.com>; devel@edk2.groups.io; Chen, Christine
> <yuwei.chen@intel.com>; Palomino Sosa, Guillermo A
> <guillermo.a.palomino.sosa@intel.com>
> Cc: Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming
> <gaoliming@byosoft.com.cn>; Kinney, Michael D
> <michael.d.kinney@intel.com>
> Subject: RE: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile
> information in build report
>
> Hi Ray,
>
> Right now we want the commit in both places.
>
> The priority is edk2-basetools first. It has more CI checks than edk2
> repo for tools and packages up as a pip module.
>
> As soon as edk2-basetools change is merged, the edk2 repo change can
> be submitted and merged because the reviews have already been
> completed by the BaseTools maintainers.
>
> We do not like commit in 2 places. As soon as possible, we would like
> to see the python sources removed from edk2/BaseTools and all devs/CI
> only use edk2- basetools for python based tools.
>
> Mike
>
> > -----Original Message-----
> > From: Ni, Ray <ray.ni@intel.com>
> > Sent: Wednesday, February 15, 2023 12:17 AM
> > To: devel@edk2.groups.io; Chen, Christine <yuwei.chen@intel.com>;
> Kinney, Michael D <michael.d.kinney@intel.com>; Palomino Sosa,
> > Guillermo A <guillermo.a.palomino.sosa@intel.com>
> > Cc: Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming
> <gaoliming@byosoft.com.cn>
> > Subject: RE: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile
> information in build report
> >
> > Christine,
> > If BaseTools related changes is implemented in edk2-basetools repo,
> > does that mean if I only checkout edk2 repo, I am using an older
> > version of
> BaseTools?
> >
> > Thanks,
> > Ray
> >
> > > -----Original Message-----
> > > From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of
> Yuwei
> > > Chen
> > > Sent: Wednesday, February 15, 2023 1:38 PM
> > > To: Kinney, Michael D <michael.d.kinney@intel.com>;
> devel@edk2.groups.io;
> > > Palomino Sosa, Guillermo A <guillermo.a.palomino.sosa@intel.com>
> > > Cc: Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming
> > > <gaoliming@byosoft.com.cn>
> > > Subject: Re: [edk2-devel][PATCH V1 1/1] BaseTools: Generate
> > > compile information in build report
> > >
> > > Hi Mike, thanks for reminder.
> > >
> > > Hi Willy, currently, BaseTools related changes will be implemented
> > > on the edk2-basetools repo. Please send the patch based on the
> > > edk2-basetools repo~
> > >
> > > Thanks,
> > > Christine
> > >
> > > > -----Original Message-----
> > > > From: Kinney, Michael D <michael.d.kinney@intel.com>
> > > > Sent: Wednesday, February 15, 2023 11:43 AM
> > > > To: devel@edk2.groups.io; Chen, Christine
> > > > <yuwei.chen@intel.com>; Palomino Sosa, Guillermo A
> > > > <guillermo.a.palomino.sosa@intel.com>
> > > > Cc: Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming
> > > > <gaoliming@byosoft.com.cn>; Kinney, Michael D
> > > > <michael.d.kinney@intel.com>
> > > > Subject: RE: [edk2-devel][PATCH V1 1/1] BaseTools: Generate
> > > > compile information in build report
> > > >
> > > > Has this been reviewed for edk2-basetools repo?
> > > >
> > > > Mike
> > > >
> > > > > -----Original Message-----
> > > > > From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of
> > > Yuwei
> > > > > Chen
> > > > > Sent: Tuesday, February 14, 2023 6:44 PM
> > > > > To: Palomino Sosa, Guillermo A
> <guillermo.a.palomino.sosa@intel.com>;
> > > > > devel@edk2.groups.io
> > > > > Cc: Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming
> > > > > <gaoliming@byosoft.com.cn>
> > > > > Subject: Re: [edk2-devel][PATCH V1 1/1] BaseTools: Generate
> compile
> > > > > information in build report
> > > > >
> > > > > Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: Palomino Sosa, Guillermo A
> > > > > > <guillermo.a.palomino.sosa@intel.com>
> > > > > > Sent: Tuesday, February 7, 2023 11:07 AM
> > > > > > To: devel@edk2.groups.io
> > > > > > Cc: Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming
> > > > > > <gaoliming@byosoft.com.cn>; Chen, Christine
> <yuwei.chen@intel.com>
> > > > > > Subject: [edk2-devel][PATCH V1 1/1] BaseTools: Generate
> > > > > > compile information in build report
> > > > > >
> > > > > > REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2850
> > > > > >
> > > > > > Add "-Y REPORT_INFO" option to build command to generate
> compile
> > > > > > information as part of BuildReport.
> > > > > > This option generates files to be used by external tools as
> > > > > > IDE's to enhance functionality.
> > > > > > Files are created inside build folder:
> > > > > > <Build>/<BuildTarget>/<ToolChain>/CompileInfo
> > > > > >
> > > > > > Files created:
> > > > > > * compile_commands.json - Compilation Database. To be used
> > > > > > by
> IDE's
> > > > > > to enable advance features
> > > > > > * cscope.files - List of files used in compilation. Used by
> > > > > > Cscope to
> parse
> > > > > > C code and provide browse functionality.
> > > > > > * module_report.json - Module data form buildReport in Json
> format.
> > > > > >
> > > > > > Signed-off-by: Guillermo Antonio Palomino Sosa
> > > > > > <guillermo.a.palomino.sosa@intel.com>
> > > > > > ---
> > > > > > BaseTools/Source/Python/build/BuildReport.py | 139
> > > > > > +++++++++++++++++++-
> > > > > > BaseTools/Source/Python/build/buildoptions.py | 4 +-
> > > > > > 2 files changed, 140 insertions(+), 3 deletions(-)
> > > > > >
> > > > > > diff --git a/BaseTools/Source/Python/build/BuildReport.py
> > > > > > b/BaseTools/Source/Python/build/BuildReport.py
> > > > > > index 468772930c..33b43d471f 100644
> > > > > > --- a/BaseTools/Source/Python/build/BuildReport.py
> > > > > > +++ b/BaseTools/Source/Python/build/BuildReport.py
> > > > > > @@ -10,6 +10,8 @@
> > > > > >
> > > > > > ## Import Modules
> > > > > > #
> > > > > > +import json
> > > > > > +from pathlib import Path
> > > > > > import Common.LongFilePathOs as os import re import
> > > > > > platform @@ -41,6 +43,7 @@ from Common.DataType import *
> > > > > > import
> > > > collections
> > > > > > from Common.Expression import * from GenFds.AprioriSection
> import
> > > > > > DXE_APRIORI_GUID, PEI_APRIORI_GUID
> > > > > > +from AutoGen.IncludesAutoGen import IncludesAutoGen
> > > > > >
> > > > > > ## Pattern to extract contents in EDK DXS files
> > > > > > gDxsDependencyPattern =
> > > > > > re.compile(r"DEPENDENCY_START(.+)DEPENDENCY_END",
> re.DOTALL)
> > > > @@ -
> > > > > > 2298,6 +2301,10 @@ class BuildReport(object):
> > > > > > def GenerateReport(self, BuildDuration, AutoGenTime,
> MakeTime,
> > > > > > GenFdsTime):
> > > > > > if self.ReportFile:
> > > > > > try:
> > > > > > +
> > > > > > + if "COMPILE_INFO" in self.ReportType:
> > > > > > + self.GenerateCompileInfo()
> > > > > > +
> > > > > > File = []
> > > > > > for (Wa, MaList) in self.ReportList:
> > > > > > PlatformReport(Wa, MaList,
> > > > > > self.ReportType).GenerateReport(File, BuildDuration,
> AutoGenTime,
> > > > > > MakeTime, GenFdsTime, self.ReportType) @@ -2310,7 +2317,137
> @@
> > > > class
> > > > > > BuildReport(object):
> > > > > > EdkLogger.error("BuildReport", CODE_ERROR,
> > > > > > "Unknown fatal error when generating build report",
> > > > > > ExtraData=self.ReportFile,
> > > > > > RaiseError=False)
> > > > > > EdkLogger.quiet("(Python %s on %s\n%s)" %
> > > > > > (platform.python_version(), sys.platform,
> > > > > > traceback.format_exc()))
> > > > > >
> > > > > > +
> > > > > > + ##
> > > > > > + # Generates compile data files to be used by external tools.
> > > > > > + # Compile information will be generated in
> > > > > > <Build>/<BuildTarget>/<ToolChain>/CompileInfo
> > > > > > + # Files generated: compile_commands.json, cscope.files,
> > > > > > modules_report.json
> > > > > > + #
> > > > > > + # @param self The object pointer
> > > > > > + #
> > > > > > + def GenerateCompileInfo(self):
> > > > > > + try:
> > > > > > + # Lists for the output elements
> > > > > > + compile_commands = []
> > > > > > + used_files = set()
> > > > > > + module_report = []
> > > > > > +
> > > > > > + for (Wa, MaList) in self.ReportList:
> > > > > > + # Obtain list of all processed Workspace files
> > > > > > + for file_path in
> > > > > > + Wa._GetMetaFiles(Wa.BuildTarget,
> > > > Wa.ToolChain):
> > > > > > + used_files.add(file_path)
> > > > > > +
> > > > > > + for autoGen in Wa.AutoGenObjectList:
> > > > > > +
> > > > > > + # Loop through all modules
> > > > > > + for module in
> > > > > > + (autoGen.LibraryAutoGenList +
> > > > > > autoGen.ModuleAutoGenList):
> > > > > > +
> > > > > > +
> > > > > > + used_files.add(module.MetaFile.Path)
> > > > > > +
> > > > > > + # Main elements of module report
> > > > > > + module_report_data = {}
> > > > > > + module_report_data["Name"] = module.Name
> > > > > > + module_report_data["Arch"] = module.Arch
> > > > > > + module_report_data["Path"] = module.MetaFile.Path
> > > > > > + module_report_data["Guid"] = module.Guid
> > > > > > + module_report_data["BuildType"] =
> module.BuildType
> > > > > > + module_report_data["IsLibrary"] = module.IsLibrary
> > > > > > + module_report_data["SourceDir"] = module.SourceDir
> > > > > > + module_report_data["Files"] = []
> > > > > > +
> > > > > > + # Files used by module
> > > > > > + for data_file in module.SourceFileList:
> > > > > > + module_report_data["Files"].append({"Name":
> > > > > > + data_file.Name, "Path": data_file.Path})
> > > > > > +
> > > > > > + # Libraries used by module
> > > > > > + module_report_data["Libraries"] = []
> > > > > > + for data_library in module.LibraryAutoGenList:
> > > > > > +
> > > > > > + module_report_data["Libraries"].append({"Path":
> > > > > > + data_library.MetaFile.Path})
> > > > > > +
> > > > > > + # Packages used by module
> > > > > > + module_report_data["Packages"] = []
> > > > > > + for data_package in module.PackageList:
> > > > > > + module_report_data["Packages"].append({"Path":
> > > > > > data_package.MetaFile.Path, "Includes": []})
> > > > > > + # Includes path used in package
> > > > > > + for data_package_include in data_package.Includes:
> > > > > > +
> > > > > > + module_report_data["Packages"][-
> > > > > > 1]["Includes"].append(data_package_inc
> > > > > > + lude.Path)
> > > > > > +
> > > > > > + # PPI's in module
> > > > > > + module_report_data["PPI"] = []
> > > > > > + for data_ppi in module.PpiList.keys():
> > > > > > + module_report_data["PPI"].append({"Name":
> > > > > > + data_ppi, "Guid": module.PpiList[data_ppi]})
> > > > > > +
> > > > > > + # Protocol's in module
> > > > > > + module_report_data["Protocol"] = []
> > > > > > + for data_protocol in module.ProtocolList.keys():
> > > > > > +
> > > > > > + module_report_data["Protocol"].append({"Name":
> data_protocol,
> > > > "Guid":
> > > > > > + module.ProtocolList[data_protocol]})
> > > > > > +
> > > > > > + # PCD's in module
> > > > > > + module_report_data["Pcd"] = []
> > > > > > + for data_pcd in module.LibraryPcdList:
> > > > > > + module_report_data["Pcd"].append({"Space":
> > > > > > data_pcd.TokenSpaceGuidCName,
> > > > > > + "Name": data_pcd.TokenCName,
> > > > > > + "Value": data_pcd.TokenValue,
> > > > > > + "Guid":
> > > > data_pcd.TokenSpaceGuidValue,
> > > > > > + "DatumType":
> data_pcd.DatumType,
> > > > > > + "Type": data_pcd.Type,
> > > > > > + "DefaultValue":
> > > > data_pcd.DefaultValue})
> > > > > > + # Add module to report
> > > > > > +
> > > > > > + module_report.append(module_report_data)
> > > > > > +
> > > > > > + # Include file dependencies to used files
> > > > > > + includes_autogen =
> > > > > > + IncludesAutoGen(module.MakeFileDir,
> > > > > > module)
> > > > > > + for dep in includes_autogen.DepsCollection:
> > > > > > + used_files.add(dep)
> > > > > > +
> > > > > > + inc_flag = "-I" # Default include flag
> > > > > > + if module.BuildRuleFamily == TAB_COMPILER_MSFT:
> > > > > > + inc_flag = "/I"
> > > > > > +
> > > > > > + for source in module.SourceFileList:
> > > > > > + used_files.add(source.Path)
> > > > > > + compile_command = {}
> > > > > > + if source.Ext in [".c", ".cc", ".cpp"]:
> > > > > > + #
> > > > > > + # Generate compile command for each c file
> > > > > > + #
> > > > > > + compile_command["file"] = source.Path
> > > > > > + compile_command["directory"] = source.Dir
> > > > > > + build_command =
> > > > > > module.BuildRules[source.Ext].CommandList[0]
> > > > > > + build_command_variables =
> > > > > > + re.findall(r"\$\((.*?)\)",
> > > > > > build_command)
> > > > > > + for var in build_command_variables:
> > > > > > + var_tokens = var.split("_")
> > > > > > + var_main = var_tokens[0]
> > > > > > + if len(var_tokens) == 1:
> > > > > > + var_value =
> > > > module.BuildOption[var_main]["PATH"]
> > > > > > + else:
> > > > > > + var_value =
> > > > > > module.BuildOption[var_main][var_tokens[1]]
> > > > > > + build_command =
> > > > > > + build_command.replace(f"$({var})",
> > > > > > var_value)
> > > > > > + include_files = f"
> > > > {inc_flag}".join(module.IncludePathList)
> > > > > > + build_command =
> > > > > > + build_command.replace("${src}",
> > > > > > include_files)
> > > > > > + build_command =
> > > > > > + build_command.replace("${dst}", module.OutputDir)
> > > > > > +
> > > > > > + # Remove un defined macros
> > > > > > + compile_command["command"]
> > > > > > + = re.sub(r"\$\(.*?\)", "",
> > > > > > build_command)
> > > > > > +
> > > > > > + compile_commands.append(compile_command)
> > > > > > +
> > > > > > + # Create output folder if doesn't exist
> > > > > > + compile_info_folder =
> > > > Path(Wa.BuildDir).joinpath("CompileInfo")
> > > > > > + compile_info_folder.mkdir(exist_ok=True)
> > > > > > +
> > > > > > + # Sort and save files
> > > > > > + compile_commands.sort(key=lambda x:
> > > > > > + x["file"])
> > > > > > +
> > > > > > +
> > > > > >
> > > >
> > >
> SaveFileOnChange(compile_info_folder.joinpath(f"compile_commands.json
> > > "
> > > > > > + ),json.dumps(compile_commands, indent=2), False)
> > > > > > +
> > > > > > +
> > > > > > + SaveFileOnChange(compile_info_folder.joinpath(f"cscope.fil
> > > > > > + es"), "\n".join(sorted(used_files)), False)
> > > > > > +
> > > > > > + module_report.sort(key=lambda x: x["Path"])
> > > > > > +
> > > > > > +
> > > SaveFileOnChange(compile_info_folder.joinpath(f"module_report.json
> > > > > > + "), json.dumps(module_report, indent=2), False)
> > > > > > +
> > > > > > + except:
> > > > > > + EdkLogger.error("BuildReport", CODE_ERROR,
> > > > > > + "Unknown fatal error
> > > > > > when generating build report compile information",
> > > > > > ExtraData=self.ReportFile, RaiseError=False)
> > > > > > + EdkLogger.quiet("(Python %s on %s\n%s)" %
> > > > > > + (platform.python_version(), sys.platform,
> traceback.format_exc()))
> > > > > > +
> > > > > > # This acts like the main() function for the script, unless
> > > > > > it is 'import'ed into another script.
> > > > > > if __name__ == '__main__':
> > > > > > pass
> > > > > > -
> > > > > > diff --git a/BaseTools/Source/Python/build/buildoptions.py
> > > > > > b/BaseTools/Source/Python/build/buildoptions.py
> > > > > > index 8334604b46..5ec561f7ec 100644
> > > > > > --- a/BaseTools/Source/Python/build/buildoptions.py
> > > > > > +++ b/BaseTools/Source/Python/build/buildoptions.py
> > > > > > @@ -84,8 +84,8 @@ class MyOptionParser():
> > > > > > Parser.add_option("-D", "--define",
> > > > > > action="append", type="string", dest="Macros", help="Macro:
> > > > > > \"Name [= Value]\".")
> > > > > >
> > > > > > Parser.add_option("-y", "--report-file",
> > > > > > action="store", dest="ReportFile", help="Create/overwrite
> > > > > > the report to the specified
> > > > > > filename.")
> > > > > > - Parser.add_option("-Y", "--report-type", action="append",
> > > > > > type="choice", choices=['PCD', 'LIBRARY', 'FLASH', 'DEPEX',
> > > > > > 'BUILD_FLAGS', 'FIXED_ADDRESS', 'HASH', 'EXECUTION_ORDER'],
> > > > > > dest="ReportType", default=[],
> > > > > > - help="Flags that control the type of build report to generate.
> > > > Must be
> > > > > > one of: [PCD, LIBRARY, FLASH, DEPEX, BUILD_FLAGS,
> FIXED_ADDRESS,
> > > > > > HASH, EXECUTION_ORDER]. "\
> > > > > > + Parser.add_option("-Y", "--report-type",
> > > > > > + action="append",
> > > > > > type="choice", choices=['PCD', 'LIBRARY', 'FLASH', 'DEPEX',
> > > > > > 'BUILD_FLAGS', 'FIXED_ADDRESS', 'HASH', 'EXECUTION_ORDER',
> > > > > > 'COMPILE_INFO'], dest="ReportType", default=[],
> > > > > > + help="Flags that control the type of build
> > > > > > + report to generate. Must be one of: [PCD, LIBRARY, FLASH,
> > > > > > + DEPEX, BUILD_FLAGS, FIXED_ADDRESS, HASH, EXECUTION_ORDER,
> > > > COMPILE_INFO].
> > > > > > + "\
> > > > > > "To specify more than one flag, repeat
> > > > > > this option on the command line and the default flag set is
> > > > > > [PCD, LIBRARY, FLASH, DEPEX, HASH, BUILD_FLAGS, FIXED_ADDRESS]")
> > > > > > Parser.add_option("-F", "--flag", action="store",
> > > > > > type="string", dest="Flag",
> > > > > > help="Specify the specific option to parse EDK
> > > > > > UNI file. Must
> be
> > > > one of:
> > > > > > [-c, -s]. -c is for EDK framework UNI file, and -s is for
> > > > > > EDK UEFI UNI file. "\
> > > > > > --
> > > > > > 2.28.0.windows.1
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > >
> > >
> > >
> > >
> > >
[-- Attachment #2: 0001-BaseTools-Generate-compile-information-in-build-repo.patch --]
[-- Type: application/octet-stream, Size: 12488 bytes --]
From ddb21cdaca196f740a50fdd6524730c3e2cb76fb Mon Sep 17 00:00:00 2001
From: Guillermo Antonio Palomino Sosa <guillermo.a.palomino.sosa@intel.com>
Date: Mon, 27 Feb 2023 10:29:53 -0600
Subject: [PATCH 1/1] BaseTools: Generate compile information in build report
REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2850
Add "-Y REPORT_INFO" option to build command to generate compile
information as part of BuildReport.
This option generates files to be used by external tools as IDE's
to enhance functionality.
Files are created inside build folder:
<Build>/<BuildTarget>/<ToolChain>/CompileInfo
Files created:
* compile_commands.json - Compilation Database. To be used by IDE's
to enable advance features
* cscope.files - List of files used in compilation. Used by Cscope to parse
C code and provide browse functionality.
* module_report.json - Module data form buildReport in Json format.
Signed-off-by: Guillermo Antonio Palomino Sosa <guillermo.a.palomino.sosa@intel.com>
---
edk2basetools/build/BuildReport.py | 138 ++++++++++++++++++++++++++++
edk2basetools/build/buildoptions.py | 4 +-
2 files changed, 140 insertions(+), 2 deletions(-)
diff --git a/edk2basetools/build/BuildReport.py b/edk2basetools/build/BuildReport.py
index 553bfe5..9cfe9dc 100644
--- a/edk2basetools/build/BuildReport.py
+++ b/edk2basetools/build/BuildReport.py
@@ -21,6 +21,8 @@
import hashlib
import subprocess
import threading
+import json
+from pathlib import Path
from datetime import datetime
from io import BytesIO
from edk2basetools.Common import EdkLogger
@@ -41,6 +43,7 @@
import collections
from edk2basetools.Common.Expression import *
from edk2basetools.GenFds.AprioriSection import DXE_APRIORI_GUID, PEI_APRIORI_GUID
+from edk2basetools.AutoGen.IncludesAutoGen import IncludesAutoGen
## Pattern to extract contents in EDK DXS files
gDxsDependencyPattern = re.compile(r"DEPENDENCY_START(.+)DEPENDENCY_END", re.DOTALL)
@@ -2298,6 +2301,10 @@ def AddPlatformReport(self, Wa, MaList=None):
def GenerateReport(self, BuildDuration, AutoGenTime, MakeTime, GenFdsTime):
if self.ReportFile:
try:
+
+ if "COMPILE_INFO" in self.ReportType:
+ self.GenerateCompileInfo()
+
File = []
for (Wa, MaList) in self.ReportList:
PlatformReport(Wa, MaList, self.ReportType).GenerateReport(File, BuildDuration, AutoGenTime, MakeTime, GenFdsTime, self.ReportType)
@@ -2310,6 +2317,137 @@ def GenerateReport(self, BuildDuration, AutoGenTime, MakeTime, GenFdsTime):
EdkLogger.error("BuildReport", CODE_ERROR, "Unknown fatal error when generating build report", ExtraData=self.ReportFile, RaiseError=False)
EdkLogger.quiet("(Python %s on %s\n%s)" % (platform.python_version(), sys.platform, traceback.format_exc()))
+
+ ##
+ # Generates compile data files to be used by external tools.
+ # Compile information will be generated in <Build>/<BuildTarget>/<ToolChain>/CompileInfo
+ # Files generated: compile_commands.json, cscope.files, modules_report.json
+ #
+ # @param self The object pointer
+ #
+ def GenerateCompileInfo(self):
+ try:
+ # Lists for the output elements
+ compile_commands = []
+ used_files = set()
+ module_report = []
+
+ for (Wa, MaList) in self.ReportList:
+ # Obtain list of all processed Workspace files
+ for file_path in Wa._GetMetaFiles(Wa.BuildTarget, Wa.ToolChain):
+ used_files.add(file_path)
+
+ for autoGen in Wa.AutoGenObjectList:
+
+ # Loop through all modules
+ for module in (autoGen.LibraryAutoGenList + autoGen.ModuleAutoGenList):
+
+ used_files.add(module.MetaFile.Path)
+
+ # Main elements of module report
+ module_report_data = {}
+ module_report_data["Name"] = module.Name
+ module_report_data["Arch"] = module.Arch
+ module_report_data["Path"] = module.MetaFile.Path
+ module_report_data["Guid"] = module.Guid
+ module_report_data["BuildType"] = module.BuildType
+ module_report_data["IsLibrary"] = module.IsLibrary
+ module_report_data["SourceDir"] = module.SourceDir
+ module_report_data["Files"] = []
+
+ # Files used by module
+ for data_file in module.SourceFileList:
+ module_report_data["Files"].append({"Name": data_file.Name, "Path": data_file.Path})
+
+ # Libraries used by module
+ module_report_data["Libraries"] = []
+ for data_library in module.LibraryAutoGenList:
+ module_report_data["Libraries"].append({"Path": data_library.MetaFile.Path})
+
+ # Packages used by module
+ module_report_data["Packages"] = []
+ for data_package in module.PackageList:
+ module_report_data["Packages"].append({"Path": data_package.MetaFile.Path, "Includes": []})
+ # Includes path used in package
+ for data_package_include in data_package.Includes:
+ module_report_data["Packages"][-1]["Includes"].append(data_package_include.Path)
+
+ # PPIs in module
+ module_report_data["PPI"] = []
+ for data_ppi in module.PpiList.keys():
+ module_report_data["PPI"].append({"Name": data_ppi, "Guid": module.PpiList[data_ppi]})
+
+ # Protocols in module
+ module_report_data["Protocol"] = []
+ for data_protocol in module.ProtocolList.keys():
+ module_report_data["Protocol"].append({"Name": data_protocol, "Guid": module.ProtocolList[data_protocol]})
+
+ # PCDs in module
+ module_report_data["Pcd"] = []
+ for data_pcd in module.LibraryPcdList:
+ module_report_data["Pcd"].append({"Space": data_pcd.TokenSpaceGuidCName,
+ "Name": data_pcd.TokenCName,
+ "Value": data_pcd.TokenValue,
+ "Guid": data_pcd.TokenSpaceGuidValue,
+ "DatumType": data_pcd.DatumType,
+ "Type": data_pcd.Type,
+ "DefaultValue": data_pcd.DefaultValue})
+ # Add module to report
+ module_report.append(module_report_data)
+
+ # Include file dependencies to used files
+ includes_autogen = IncludesAutoGen(module.MakeFileDir, module)
+ for dep in includes_autogen.DepsCollection:
+ used_files.add(dep)
+
+ inc_flag = "-I" # Default include flag
+ if module.BuildRuleFamily == TAB_COMPILER_MSFT:
+ inc_flag = "/I"
+
+ for source in module.SourceFileList:
+ used_files.add(source.Path)
+ compile_command = {}
+ if source.Ext in [".c", ".cc", ".cpp"]:
+ #
+ # Generate compile command for each c file
+ #
+ compile_command["file"] = source.Path
+ compile_command["directory"] = source.Dir
+ build_command = module.BuildRules[source.Ext].CommandList[0]
+ build_command_variables = re.findall(r"\$\((.*?)\)", build_command)
+ for var in build_command_variables:
+ var_tokens = var.split("_")
+ var_main = var_tokens[0]
+ if len(var_tokens) == 1:
+ var_value = module.BuildOption[var_main]["PATH"]
+ else:
+ var_value = module.BuildOption[var_main][var_tokens[1]]
+ build_command = build_command.replace(f"$({var})", var_value)
+ include_files = f" {inc_flag}".join(module.IncludePathList)
+ build_command = build_command.replace("${src}", include_files)
+ build_command = build_command.replace("${dst}", module.OutputDir)
+
+ # Remove un defined macros
+ compile_command["command"] = re.sub(r"\$\(.*?\)", "", build_command)
+ compile_commands.append(compile_command)
+
+ # Create output folder if doesn't exist
+ compile_info_folder = Path(Wa.BuildDir).joinpath("CompileInfo")
+ compile_info_folder.mkdir(exist_ok=True)
+
+ # Sort and save files
+ compile_commands.sort(key=lambda x: x["file"])
+ SaveFileOnChange(compile_info_folder.joinpath(f"compile_commands.json"),json.dumps(compile_commands, indent=2), False)
+
+ SaveFileOnChange(compile_info_folder.joinpath(f"cscope.files"), "\n".join(sorted(used_files)), False)
+
+ module_report.sort(key=lambda x: x["Path"])
+ SaveFileOnChange(compile_info_folder.joinpath(f"module_report.json"), json.dumps(module_report, indent=2), False)
+
+ except:
+ EdkLogger.error("BuildReport", CODE_ERROR, "Unknown fatal error when generating build report compile information", ExtraData=self.ReportFile, RaiseError=False)
+ EdkLogger.quiet("(Python %s on %s\n%s)" % (platform.python_version(), sys.platform, traceback.format_exc()))
+
# This acts like the main() function for the script, unless it is 'import'ed into another script.
if __name__ == '__main__':
pass
diff --git a/edk2basetools/build/buildoptions.py b/edk2basetools/build/buildoptions.py
index 2afeb3a..d96c60c 100644
--- a/edk2basetools/build/buildoptions.py
+++ b/edk2basetools/build/buildoptions.py
@@ -83,8 +83,8 @@ def GetOption(self):
Parser.add_option("-D", "--define", action="append", type="string", dest="Macros", help="Macro: \"Name [= Value]\".")
Parser.add_option("-y", "--report-file", action="store", dest="ReportFile", help="Create/overwrite the report to the specified filename.")
- Parser.add_option("-Y", "--report-type", action="append", type="choice", choices=['PCD', 'LIBRARY', 'FLASH', 'DEPEX', 'BUILD_FLAGS', 'FIXED_ADDRESS', 'HASH', 'EXECUTION_ORDER'], dest="ReportType", default=[],
- help="Flags that control the type of build report to generate. Must be one of: [PCD, LIBRARY, FLASH, DEPEX, BUILD_FLAGS, FIXED_ADDRESS, HASH, EXECUTION_ORDER]. "\
+ Parser.add_option("-Y", "--report-type", action="append", type="choice", choices=['PCD', 'LIBRARY', 'FLASH', 'DEPEX', 'BUILD_FLAGS', 'FIXED_ADDRESS', 'HASH', 'EXECUTION_ORDER', 'COMPILE_INFO'], dest="ReportType", default=[],
+ help="Flags that control the type of build report to generate. Must be one of: [PCD, LIBRARY, FLASH, DEPEX, BUILD_FLAGS, FIXED_ADDRESS, HASH, EXECUTION_ORDER, COMPILE_INFO]. "\
"To specify more than one flag, repeat this option on the command line and the default flag set is [PCD, LIBRARY, FLASH, DEPEX, HASH, BUILD_FLAGS, FIXED_ADDRESS]")
Parser.add_option("-F", "--flag", action="store", type="string", dest="Flag",
help="Specify the specific option to parse EDK UNI file. Must be one of: [-c, -s]. -c is for EDK framework UNI file, and -s is for EDK UEFI UNI file. "\
--
2.39.1.windows.1
^ permalink raw reply related [flat|nested] 17+ messages in thread
* Re: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile information in build report
2023-02-27 17:40 ` [edk2-devel][PATCH " Guillermo Antonio Palomino Sosa
@ 2023-02-27 18:57 ` Ard Biesheuvel
2023-02-27 22:35 ` Michael D Kinney
0 siblings, 1 reply; 17+ messages in thread
From: Ard Biesheuvel @ 2023-02-27 18:57 UTC (permalink / raw)
To: devel, guillermo.a.palomino.sosa
Cc: Ni, Ray, Kinney, Michael D, Chen, Christine, Feng, Bob C,
Gao, Liming, Oram, Isaac W, Sean Brogan
On Mon, 27 Feb 2023 at 18:40, Guillermo Antonio Palomino Sosa
<guillermo.a.palomino.sosa@intel.com> wrote:
>
> Hi. I have submitted a pull request to edk2-basetools repository: https://github.com/tianocore/edk2-basetools/pull/88
> This is the feature request for it: https://github.com/tianocore/edk2-basetools/issues/87
> I'm also attaching the patch here: (0001-BaseTools-Generate-compile-information-in-build-repo.patch)
>
> On a side note, seems like tip of edk2-basetools is broken due this commit that makes direct import of Common package:
> https://github.com/tianocore/edk2-basetools/commit/8e6018d3ea4c1aae7185f589d129cea14a5d89fd
> edk2-basetools\edk2basetools\GenFds\SubTypeGuidSection.py:
> import Common.LongFilePathOs as os
>
>
I sent a fix and a PR for this about a month ago:
https://github.com/tianocore/edk2-basetools/pull/86
but CodeCov seems to take issue with it, for reasons I don't
understand (and I can't be bothered to spend more time on yet another
CI automation pass that adds to my workload rather than reduce it)
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile information in build report
2023-02-27 18:57 ` Ard Biesheuvel
@ 2023-02-27 22:35 ` Michael D Kinney
2023-02-27 23:42 ` Guillermo Antonio Palomino Sosa
0 siblings, 1 reply; 17+ messages in thread
From: Michael D Kinney @ 2023-02-27 22:35 UTC (permalink / raw)
To: devel@edk2.groups.io, ardb@kernel.org, Palomino Sosa, Guillermo A
Cc: Ni, Ray, Chen, Christine, Feng, Bob C, Gao, Liming, Oram, Isaac W,
Sean Brogan, Kinney, Michael D
Hi Guillermo,
Can you please look at Ards PR and make sure his fix is included in your PR.
Also, please work with Christine and Bob to see what is going on with the
Code Coverage check. We do want it to be easy for all community members to
submit change requests. We may need support from the edk2-bastools maintainers
to help with CI issues and help with changes to address.
Thanks,
Mike
> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Ard Biesheuvel
> Sent: Monday, February 27, 2023 10:58 AM
> To: devel@edk2.groups.io; Palomino Sosa, Guillermo A <guillermo.a.palomino.sosa@intel.com>
> Cc: Ni, Ray <ray.ni@intel.com>; Kinney, Michael D <michael.d.kinney@intel.com>; Chen, Christine <yuwei.chen@intel.com>; Feng,
> Bob C <bob.c.feng@intel.com>; Gao, Liming <gaoliming@byosoft.com.cn>; Oram, Isaac W <isaac.w.oram@intel.com>; Sean Brogan
> <sean.brogan@microsoft.com>
> Subject: Re: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile information in build report
>
> On Mon, 27 Feb 2023 at 18:40, Guillermo Antonio Palomino Sosa
> <guillermo.a.palomino.sosa@intel.com> wrote:
> >
> > Hi. I have submitted a pull request to edk2-basetools repository: https://github.com/tianocore/edk2-basetools/pull/88
> > This is the feature request for it: https://github.com/tianocore/edk2-basetools/issues/87
> > I'm also attaching the patch here: (0001-BaseTools-Generate-compile-information-in-build-repo.patch)
> >
> > On a side note, seems like tip of edk2-basetools is broken due this commit that makes direct import of Common package:
> > https://github.com/tianocore/edk2-basetools/commit/8e6018d3ea4c1aae7185f589d129cea14a5d89fd
> > edk2-basetools\edk2basetools\GenFds\SubTypeGuidSection.py:
> > import Common.LongFilePathOs as os
> >
> >
>
> I sent a fix and a PR for this about a month ago:
>
> https://github.com/tianocore/edk2-basetools/pull/86
>
> but CodeCov seems to take issue with it, for reasons I don't
> understand (and I can't be bothered to spend more time on yet another
> CI automation pass that adds to my workload rather than reduce it)
>
>
>
>
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile information in build report
2023-02-27 22:35 ` Michael D Kinney
@ 2023-02-27 23:42 ` Guillermo Antonio Palomino Sosa
2023-03-01 0:52 ` Ni, Ray
0 siblings, 1 reply; 17+ messages in thread
From: Guillermo Antonio Palomino Sosa @ 2023-02-27 23:42 UTC (permalink / raw)
To: Kinney, Michael D, devel@edk2.groups.io, ardb@kernel.org
Cc: Ni, Ray, Chen, Christine, Feng, Bob C, Gao, Liming, Oram, Isaac W,
Sean Brogan
I have updated the pull based on Sean feedback. I added following fields to module_report.json:
* LibraryClass
* ModuleEntryPointList
* ConstructorList
* DestructorList
I have also added commit from Ard based on this request to fix build issue:
https://github.com/tianocore/edk2-basetools/pull/88
Thanks
-----Original Message-----
From: Kinney, Michael D <michael.d.kinney@intel.com>
Sent: Monday, February 27, 2023 4:36 PM
To: devel@edk2.groups.io; ardb@kernel.org; Palomino Sosa, Guillermo A <guillermo.a.palomino.sosa@intel.com>
Cc: Ni, Ray <ray.ni@intel.com>; Chen, Christine <yuwei.chen@intel.com>; Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming <gaoliming@byosoft.com.cn>; Oram, Isaac W <isaac.w.oram@intel.com>; Sean Brogan <sean.brogan@microsoft.com>; Kinney, Michael D <michael.d.kinney@intel.com>
Subject: RE: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile information in build report
Hi Guillermo,
Can you please look at Ards PR and make sure his fix is included in your PR.
Also, please work with Christine and Bob to see what is going on with the Code Coverage check. We do want it to be easy for all community members to submit change requests. We may need support from the edk2-bastools maintainers to help with CI issues and help with changes to address.
Thanks,
Mike
> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Ard
> Biesheuvel
> Sent: Monday, February 27, 2023 10:58 AM
> To: devel@edk2.groups.io; Palomino Sosa, Guillermo A
> <guillermo.a.palomino.sosa@intel.com>
> Cc: Ni, Ray <ray.ni@intel.com>; Kinney, Michael D
> <michael.d.kinney@intel.com>; Chen, Christine <yuwei.chen@intel.com>;
> Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming
> <gaoliming@byosoft.com.cn>; Oram, Isaac W <isaac.w.oram@intel.com>;
> Sean Brogan <sean.brogan@microsoft.com>
> Subject: Re: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile
> information in build report
>
> On Mon, 27 Feb 2023 at 18:40, Guillermo Antonio Palomino Sosa
> <guillermo.a.palomino.sosa@intel.com> wrote:
> >
> > Hi. I have submitted a pull request to edk2-basetools repository:
> > https://github.com/tianocore/edk2-basetools/pull/88
> > This is the feature request for it:
> > https://github.com/tianocore/edk2-basetools/issues/87
> > I'm also attaching the patch here:
> > (0001-BaseTools-Generate-compile-information-in-build-repo.patch)
> >
> > On a side note, seems like tip of edk2-basetools is broken due this commit that makes direct import of Common package:
> > https://github.com/tianocore/edk2-basetools/commit/8e6018d3ea4c1aae7
> > 185f589d129cea14a5d89fd
> > edk2-basetools\edk2basetools\GenFds\SubTypeGuidSection.py:
> > import Common.LongFilePathOs as os
> >
> >
>
> I sent a fix and a PR for this about a month ago:
>
> https://github.com/tianocore/edk2-basetools/pull/86
>
> but CodeCov seems to take issue with it, for reasons I don't
> understand (and I can't be bothered to spend more time on yet another
> CI automation pass that adds to my workload rather than reduce it)
>
>
>
>
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile information in build report
2023-02-27 23:42 ` Guillermo Antonio Palomino Sosa
@ 2023-03-01 0:52 ` Ni, Ray
2023-03-01 4:00 ` Guillermo Antonio Palomino Sosa
0 siblings, 1 reply; 17+ messages in thread
From: Ni, Ray @ 2023-03-01 0:52 UTC (permalink / raw)
To: Palomino Sosa, Guillermo A, Kinney, Michael D,
devel@edk2.groups.io, ardb@kernel.org
Cc: Chen, Christine, Feng, Bob C, Gao, Liming, Oram, Isaac W,
Sean Brogan
What's the status of this patch?
Does report generation take time? If no, why not generate them by default without individual flag control.
I really like the feature to generate "compile_commands.json"
> -----Original Message-----
> From: Palomino Sosa, Guillermo A <guillermo.a.palomino.sosa@intel.com>
> Sent: Tuesday, February 28, 2023 7:42 AM
> To: Kinney, Michael D <michael.d.kinney@intel.com>; devel@edk2.groups.io;
> ardb@kernel.org
> Cc: Ni, Ray <ray.ni@intel.com>; Chen, Christine <yuwei.chen@intel.com>;
> Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming
> <gaoliming@byosoft.com.cn>; Oram, Isaac W <isaac.w.oram@intel.com>;
> Sean Brogan <sean.brogan@microsoft.com>
> Subject: RE: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile
> information in build report
>
> I have updated the pull based on Sean feedback. I added following fields to
> module_report.json:
> * LibraryClass
> * ModuleEntryPointList
> * ConstructorList
> * DestructorList
>
> I have also added commit from Ard based on this request to fix build issue:
> https://github.com/tianocore/edk2-basetools/pull/88
>
> Thanks
>
> -----Original Message-----
> From: Kinney, Michael D <michael.d.kinney@intel.com>
> Sent: Monday, February 27, 2023 4:36 PM
> To: devel@edk2.groups.io; ardb@kernel.org; Palomino Sosa, Guillermo A
> <guillermo.a.palomino.sosa@intel.com>
> Cc: Ni, Ray <ray.ni@intel.com>; Chen, Christine <yuwei.chen@intel.com>;
> Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming
> <gaoliming@byosoft.com.cn>; Oram, Isaac W <isaac.w.oram@intel.com>;
> Sean Brogan <sean.brogan@microsoft.com>; Kinney, Michael D
> <michael.d.kinney@intel.com>
> Subject: RE: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile
> information in build report
>
> Hi Guillermo,
>
> Can you please look at Ards PR and make sure his fix is included in your PR.
>
> Also, please work with Christine and Bob to see what is going on with the
> Code Coverage check. We do want it to be easy for all community members
> to submit change requests. We may need support from the edk2-bastools
> maintainers to help with CI issues and help with changes to address.
>
> Thanks,
>
> Mike
>
> > -----Original Message-----
> > From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Ard
> > Biesheuvel
> > Sent: Monday, February 27, 2023 10:58 AM
> > To: devel@edk2.groups.io; Palomino Sosa, Guillermo A
> > <guillermo.a.palomino.sosa@intel.com>
> > Cc: Ni, Ray <ray.ni@intel.com>; Kinney, Michael D
> > <michael.d.kinney@intel.com>; Chen, Christine <yuwei.chen@intel.com>;
> > Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming
> > <gaoliming@byosoft.com.cn>; Oram, Isaac W <isaac.w.oram@intel.com>;
> > Sean Brogan <sean.brogan@microsoft.com>
> > Subject: Re: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile
> > information in build report
> >
> > On Mon, 27 Feb 2023 at 18:40, Guillermo Antonio Palomino Sosa
> > <guillermo.a.palomino.sosa@intel.com> wrote:
> > >
> > > Hi. I have submitted a pull request to edk2-basetools repository:
> > > https://github.com/tianocore/edk2-basetools/pull/88
> > > This is the feature request for it:
> > > https://github.com/tianocore/edk2-basetools/issues/87
> > > I'm also attaching the patch here:
> > > (0001-BaseTools-Generate-compile-information-in-build-repo.patch)
> > >
> > > On a side note, seems like tip of edk2-basetools is broken due this
> commit that makes direct import of Common package:
> > > https://github.com/tianocore/edk2-
> basetools/commit/8e6018d3ea4c1aae7
> > > 185f589d129cea14a5d89fd
> > > edk2-basetools\edk2basetools\GenFds\SubTypeGuidSection.py:
> > > import Common.LongFilePathOs as os
> > >
> > >
> >
> > I sent a fix and a PR for this about a month ago:
> >
> > https://github.com/tianocore/edk2-basetools/pull/86
> >
> > but CodeCov seems to take issue with it, for reasons I don't
> > understand (and I can't be bothered to spend more time on yet another
> > CI automation pass that adds to my workload rather than reduce it)
> >
> >
> >
> >
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile information in build report
2023-03-01 0:52 ` Ni, Ray
@ 2023-03-01 4:00 ` Guillermo Antonio Palomino Sosa
2023-03-06 15:40 ` Guillermo Antonio Palomino Sosa
0 siblings, 1 reply; 17+ messages in thread
From: Guillermo Antonio Palomino Sosa @ 2023-03-01 4:00 UTC (permalink / raw)
To: Ni, Ray, Kinney, Michael D, devel@edk2.groups.io, ardb@kernel.org
Cc: Chen, Christine, Feng, Bob C, Gao, Liming, Oram, Isaac W,
Sean Brogan
It takes same time as original build report to be generated as it constructed using the same data structures as build report. So I think its OK to not have it enabled by default.
Patch is ready in the pull request to be reviews.
-----Original Message-----
From: Ni, Ray <ray.ni@intel.com>
Sent: Tuesday, February 28, 2023 6:52 PM
To: Palomino Sosa, Guillermo A <guillermo.a.palomino.sosa@intel.com>; Kinney, Michael D <michael.d.kinney@intel.com>; devel@edk2.groups.io; ardb@kernel.org
Cc: Chen, Christine <yuwei.chen@intel.com>; Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming <gaoliming@byosoft.com.cn>; Oram, Isaac W <isaac.w.oram@intel.com>; Sean Brogan <sean.brogan@microsoft.com>
Subject: RE: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile information in build report
What's the status of this patch?
Does report generation take time? If no, why not generate them by default without individual flag control.
I really like the feature to generate "compile_commands.json"
> -----Original Message-----
> From: Palomino Sosa, Guillermo A <guillermo.a.palomino.sosa@intel.com>
> Sent: Tuesday, February 28, 2023 7:42 AM
> To: Kinney, Michael D <michael.d.kinney@intel.com>;
> devel@edk2.groups.io; ardb@kernel.org
> Cc: Ni, Ray <ray.ni@intel.com>; Chen, Christine
> <yuwei.chen@intel.com>; Feng, Bob C <bob.c.feng@intel.com>; Gao,
> Liming <gaoliming@byosoft.com.cn>; Oram, Isaac W
> <isaac.w.oram@intel.com>; Sean Brogan <sean.brogan@microsoft.com>
> Subject: RE: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile
> information in build report
>
> I have updated the pull based on Sean feedback. I added following
> fields to
> module_report.json:
> * LibraryClass
> * ModuleEntryPointList
> * ConstructorList
> * DestructorList
>
> I have also added commit from Ard based on this request to fix build issue:
> https://github.com/tianocore/edk2-basetools/pull/88
>
> Thanks
>
> -----Original Message-----
> From: Kinney, Michael D <michael.d.kinney@intel.com>
> Sent: Monday, February 27, 2023 4:36 PM
> To: devel@edk2.groups.io; ardb@kernel.org; Palomino Sosa, Guillermo A
> <guillermo.a.palomino.sosa@intel.com>
> Cc: Ni, Ray <ray.ni@intel.com>; Chen, Christine
> <yuwei.chen@intel.com>; Feng, Bob C <bob.c.feng@intel.com>; Gao,
> Liming <gaoliming@byosoft.com.cn>; Oram, Isaac W
> <isaac.w.oram@intel.com>; Sean Brogan <sean.brogan@microsoft.com>;
> Kinney, Michael D <michael.d.kinney@intel.com>
> Subject: RE: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile
> information in build report
>
> Hi Guillermo,
>
> Can you please look at Ards PR and make sure his fix is included in your PR.
>
> Also, please work with Christine and Bob to see what is going on with
> the Code Coverage check. We do want it to be easy for all community
> members to submit change requests. We may need support from the
> edk2-bastools maintainers to help with CI issues and help with changes to address.
>
> Thanks,
>
> Mike
>
> > -----Original Message-----
> > From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Ard
> > Biesheuvel
> > Sent: Monday, February 27, 2023 10:58 AM
> > To: devel@edk2.groups.io; Palomino Sosa, Guillermo A
> > <guillermo.a.palomino.sosa@intel.com>
> > Cc: Ni, Ray <ray.ni@intel.com>; Kinney, Michael D
> > <michael.d.kinney@intel.com>; Chen, Christine
> > <yuwei.chen@intel.com>; Feng, Bob C <bob.c.feng@intel.com>; Gao,
> > Liming <gaoliming@byosoft.com.cn>; Oram, Isaac W
> > <isaac.w.oram@intel.com>; Sean Brogan <sean.brogan@microsoft.com>
> > Subject: Re: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile
> > information in build report
> >
> > On Mon, 27 Feb 2023 at 18:40, Guillermo Antonio Palomino Sosa
> > <guillermo.a.palomino.sosa@intel.com> wrote:
> > >
> > > Hi. I have submitted a pull request to edk2-basetools repository:
> > > https://github.com/tianocore/edk2-basetools/pull/88
> > > This is the feature request for it:
> > > https://github.com/tianocore/edk2-basetools/issues/87
> > > I'm also attaching the patch here:
> > > (0001-BaseTools-Generate-compile-information-in-build-repo.patch)
> > >
> > > On a side note, seems like tip of edk2-basetools is broken due
> > > this
> commit that makes direct import of Common package:
> > > https://github.com/tianocore/edk2-
> basetools/commit/8e6018d3ea4c1aae7
> > > 185f589d129cea14a5d89fd
> > > edk2-basetools\edk2basetools\GenFds\SubTypeGuidSection.py:
> > > import Common.LongFilePathOs as os
> > >
> > >
> >
> > I sent a fix and a PR for this about a month ago:
> >
> > https://github.com/tianocore/edk2-basetools/pull/86
> >
> > but CodeCov seems to take issue with it, for reasons I don't
> > understand (and I can't be bothered to spend more time on yet
> > another CI automation pass that adds to my workload rather than
> > reduce it)
> >
> >
> >
> >
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile information in build report
2023-03-01 4:00 ` Guillermo Antonio Palomino Sosa
@ 2023-03-06 15:40 ` Guillermo Antonio Palomino Sosa
2023-03-15 1:12 ` Ni, Ray
0 siblings, 1 reply; 17+ messages in thread
From: Guillermo Antonio Palomino Sosa @ 2023-03-06 15:40 UTC (permalink / raw)
To: Kinney, Michael D, Feng, Bob C, Gao, Liming, devel@edk2.groups.io
Cc: Chen, Christine, Ni, Ray, Oram, Isaac W, Sean Brogan,
ardb@kernel.org
Hi guys, can we submit the pull request or do you have comments on it?
https://github.com/tianocore/edk2-basetools/pull/88
Thanks
-----Original Message-----
From: Palomino Sosa, Guillermo A
Sent: Tuesday, February 28, 2023 10:00 PM
To: Ni, Ray <ray.ni@intel.com>; Kinney, Michael D <michael.d.kinney@intel.com>; devel@edk2.groups.io; ardb@kernel.org
Cc: Chen, Christine <Yuwei.Chen@intel.com>; Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming <gaoliming@byosoft.com.cn>; Oram, Isaac W <isaac.w.oram@intel.com>; Sean Brogan <sean.brogan@microsoft.com>
Subject: RE: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile information in build report
It takes same time as original build report to be generated as it constructed using the same data structures as build report. So I think its OK to not have it enabled by default.
Patch is ready in the pull request to be reviews.
-----Original Message-----
From: Ni, Ray <ray.ni@intel.com>
Sent: Tuesday, February 28, 2023 6:52 PM
To: Palomino Sosa, Guillermo A <guillermo.a.palomino.sosa@intel.com>; Kinney, Michael D <michael.d.kinney@intel.com>; devel@edk2.groups.io; ardb@kernel.org
Cc: Chen, Christine <yuwei.chen@intel.com>; Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming <gaoliming@byosoft.com.cn>; Oram, Isaac W <isaac.w.oram@intel.com>; Sean Brogan <sean.brogan@microsoft.com>
Subject: RE: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile information in build report
What's the status of this patch?
Does report generation take time? If no, why not generate them by default without individual flag control.
I really like the feature to generate "compile_commands.json"
> -----Original Message-----
> From: Palomino Sosa, Guillermo A <guillermo.a.palomino.sosa@intel.com>
> Sent: Tuesday, February 28, 2023 7:42 AM
> To: Kinney, Michael D <michael.d.kinney@intel.com>;
> devel@edk2.groups.io; ardb@kernel.org
> Cc: Ni, Ray <ray.ni@intel.com>; Chen, Christine
> <yuwei.chen@intel.com>; Feng, Bob C <bob.c.feng@intel.com>; Gao,
> Liming <gaoliming@byosoft.com.cn>; Oram, Isaac W
> <isaac.w.oram@intel.com>; Sean Brogan <sean.brogan@microsoft.com>
> Subject: RE: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile
> information in build report
>
> I have updated the pull based on Sean feedback. I added following
> fields to
> module_report.json:
> * LibraryClass
> * ModuleEntryPointList
> * ConstructorList
> * DestructorList
>
> I have also added commit from Ard based on this request to fix build issue:
> https://github.com/tianocore/edk2-basetools/pull/88
>
> Thanks
>
> -----Original Message-----
> From: Kinney, Michael D <michael.d.kinney@intel.com>
> Sent: Monday, February 27, 2023 4:36 PM
> To: devel@edk2.groups.io; ardb@kernel.org; Palomino Sosa, Guillermo A
> <guillermo.a.palomino.sosa@intel.com>
> Cc: Ni, Ray <ray.ni@intel.com>; Chen, Christine
> <yuwei.chen@intel.com>; Feng, Bob C <bob.c.feng@intel.com>; Gao,
> Liming <gaoliming@byosoft.com.cn>; Oram, Isaac W
> <isaac.w.oram@intel.com>; Sean Brogan <sean.brogan@microsoft.com>;
> Kinney, Michael D <michael.d.kinney@intel.com>
> Subject: RE: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile
> information in build report
>
> Hi Guillermo,
>
> Can you please look at Ards PR and make sure his fix is included in your PR.
>
> Also, please work with Christine and Bob to see what is going on with
> the Code Coverage check. We do want it to be easy for all community
> members to submit change requests. We may need support from the
> edk2-bastools maintainers to help with CI issues and help with changes to address.
>
> Thanks,
>
> Mike
>
> > -----Original Message-----
> > From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Ard
> > Biesheuvel
> > Sent: Monday, February 27, 2023 10:58 AM
> > To: devel@edk2.groups.io; Palomino Sosa, Guillermo A
> > <guillermo.a.palomino.sosa@intel.com>
> > Cc: Ni, Ray <ray.ni@intel.com>; Kinney, Michael D
> > <michael.d.kinney@intel.com>; Chen, Christine
> > <yuwei.chen@intel.com>; Feng, Bob C <bob.c.feng@intel.com>; Gao,
> > Liming <gaoliming@byosoft.com.cn>; Oram, Isaac W
> > <isaac.w.oram@intel.com>; Sean Brogan <sean.brogan@microsoft.com>
> > Subject: Re: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile
> > information in build report
> >
> > On Mon, 27 Feb 2023 at 18:40, Guillermo Antonio Palomino Sosa
> > <guillermo.a.palomino.sosa@intel.com> wrote:
> > >
> > > Hi. I have submitted a pull request to edk2-basetools repository:
> > > https://github.com/tianocore/edk2-basetools/pull/88
> > > This is the feature request for it:
> > > https://github.com/tianocore/edk2-basetools/issues/87
> > > I'm also attaching the patch here:
> > > (0001-BaseTools-Generate-compile-information-in-build-repo.patch)
> > >
> > > On a side note, seems like tip of edk2-basetools is broken due
> > > this
> commit that makes direct import of Common package:
> > > https://github.com/tianocore/edk2-
> basetools/commit/8e6018d3ea4c1aae7
> > > 185f589d129cea14a5d89fd
> > > edk2-basetools\edk2basetools\GenFds\SubTypeGuidSection.py:
> > > import Common.LongFilePathOs as os
> > >
> > >
> >
> > I sent a fix and a PR for this about a month ago:
> >
> > https://github.com/tianocore/edk2-basetools/pull/86
> >
> > but CodeCov seems to take issue with it, for reasons I don't
> > understand (and I can't be bothered to spend more time on yet
> > another CI automation pass that adds to my workload rather than
> > reduce it)
> >
> >
> >
> >
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile information in build report
2023-03-06 15:40 ` Guillermo Antonio Palomino Sosa
@ 2023-03-15 1:12 ` Ni, Ray
2023-03-15 10:28 ` Bob Feng
0 siblings, 1 reply; 17+ messages in thread
From: Ni, Ray @ 2023-03-15 1:12 UTC (permalink / raw)
To: Palomino Sosa, Guillermo A, Kinney, Michael D, Feng, Bob C,
Gao, Liming, devel@edk2.groups.io
Cc: Chen, Christine, Oram, Isaac W, Sean Brogan, ardb@kernel.org
I saw the code was merged to edk2-basetools repo 5 days ago.
When will the change be in edk2 repo? I am really interested in using the json files😊
Thanks,
Ray
> -----Original Message-----
> From: Palomino Sosa, Guillermo A <guillermo.a.palomino.sosa@intel.com>
> Sent: Monday, March 6, 2023 11:41 PM
> To: Kinney, Michael D <michael.d.kinney@intel.com>; Feng, Bob C
> <bob.c.feng@intel.com>; Gao, Liming <gaoliming@byosoft.com.cn>;
> devel@edk2.groups.io
> Cc: Chen, Christine <yuwei.chen@intel.com>; Ni, Ray <ray.ni@intel.com>;
> Oram, Isaac W <isaac.w.oram@intel.com>; Sean Brogan
> <sean.brogan@microsoft.com>; ardb@kernel.org
> Subject: RE: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile
> information in build report
>
> Hi guys, can we submit the pull request or do you have comments on it?
> https://github.com/tianocore/edk2-basetools/pull/88
>
> Thanks
>
>
> -----Original Message-----
> From: Palomino Sosa, Guillermo A
> Sent: Tuesday, February 28, 2023 10:00 PM
> To: Ni, Ray <ray.ni@intel.com>; Kinney, Michael D
> <michael.d.kinney@intel.com>; devel@edk2.groups.io; ardb@kernel.org
> Cc: Chen, Christine <Yuwei.Chen@intel.com>; Feng, Bob C
> <bob.c.feng@intel.com>; Gao, Liming <gaoliming@byosoft.com.cn>; Oram,
> Isaac W <isaac.w.oram@intel.com>; Sean Brogan
> <sean.brogan@microsoft.com>
> Subject: RE: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile
> information in build report
>
> It takes same time as original build report to be generated as it constructed
> using the same data structures as build report. So I think its OK to not have it
> enabled by default.
>
> Patch is ready in the pull request to be reviews.
>
>
>
> -----Original Message-----
> From: Ni, Ray <ray.ni@intel.com>
> Sent: Tuesday, February 28, 2023 6:52 PM
> To: Palomino Sosa, Guillermo A <guillermo.a.palomino.sosa@intel.com>;
> Kinney, Michael D <michael.d.kinney@intel.com>; devel@edk2.groups.io;
> ardb@kernel.org
> Cc: Chen, Christine <yuwei.chen@intel.com>; Feng, Bob C
> <bob.c.feng@intel.com>; Gao, Liming <gaoliming@byosoft.com.cn>; Oram,
> Isaac W <isaac.w.oram@intel.com>; Sean Brogan
> <sean.brogan@microsoft.com>
> Subject: RE: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile
> information in build report
>
> What's the status of this patch?
> Does report generation take time? If no, why not generate them by default
> without individual flag control.
> I really like the feature to generate "compile_commands.json"
>
> > -----Original Message-----
> > From: Palomino Sosa, Guillermo A <guillermo.a.palomino.sosa@intel.com>
> > Sent: Tuesday, February 28, 2023 7:42 AM
> > To: Kinney, Michael D <michael.d.kinney@intel.com>;
> > devel@edk2.groups.io; ardb@kernel.org
> > Cc: Ni, Ray <ray.ni@intel.com>; Chen, Christine
> > <yuwei.chen@intel.com>; Feng, Bob C <bob.c.feng@intel.com>; Gao,
> > Liming <gaoliming@byosoft.com.cn>; Oram, Isaac W
> > <isaac.w.oram@intel.com>; Sean Brogan <sean.brogan@microsoft.com>
> > Subject: RE: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile
> > information in build report
> >
> > I have updated the pull based on Sean feedback. I added following
> > fields to
> > module_report.json:
> > * LibraryClass
> > * ModuleEntryPointList
> > * ConstructorList
> > * DestructorList
> >
> > I have also added commit from Ard based on this request to fix build issue:
> > https://github.com/tianocore/edk2-basetools/pull/88
> >
> > Thanks
> >
> > -----Original Message-----
> > From: Kinney, Michael D <michael.d.kinney@intel.com>
> > Sent: Monday, February 27, 2023 4:36 PM
> > To: devel@edk2.groups.io; ardb@kernel.org; Palomino Sosa, Guillermo A
> > <guillermo.a.palomino.sosa@intel.com>
> > Cc: Ni, Ray <ray.ni@intel.com>; Chen, Christine
> > <yuwei.chen@intel.com>; Feng, Bob C <bob.c.feng@intel.com>; Gao,
> > Liming <gaoliming@byosoft.com.cn>; Oram, Isaac W
> > <isaac.w.oram@intel.com>; Sean Brogan <sean.brogan@microsoft.com>;
> > Kinney, Michael D <michael.d.kinney@intel.com>
> > Subject: RE: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile
> > information in build report
> >
> > Hi Guillermo,
> >
> > Can you please look at Ards PR and make sure his fix is included in your PR.
> >
> > Also, please work with Christine and Bob to see what is going on with
> > the Code Coverage check. We do want it to be easy for all community
> > members to submit change requests. We may need support from the
> > edk2-bastools maintainers to help with CI issues and help with changes to
> address.
> >
> > Thanks,
> >
> > Mike
> >
> > > -----Original Message-----
> > > From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Ard
> > > Biesheuvel
> > > Sent: Monday, February 27, 2023 10:58 AM
> > > To: devel@edk2.groups.io; Palomino Sosa, Guillermo A
> > > <guillermo.a.palomino.sosa@intel.com>
> > > Cc: Ni, Ray <ray.ni@intel.com>; Kinney, Michael D
> > > <michael.d.kinney@intel.com>; Chen, Christine
> > > <yuwei.chen@intel.com>; Feng, Bob C <bob.c.feng@intel.com>; Gao,
> > > Liming <gaoliming@byosoft.com.cn>; Oram, Isaac W
> > > <isaac.w.oram@intel.com>; Sean Brogan <sean.brogan@microsoft.com>
> > > Subject: Re: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile
> > > information in build report
> > >
> > > On Mon, 27 Feb 2023 at 18:40, Guillermo Antonio Palomino Sosa
> > > <guillermo.a.palomino.sosa@intel.com> wrote:
> > > >
> > > > Hi. I have submitted a pull request to edk2-basetools repository:
> > > > https://github.com/tianocore/edk2-basetools/pull/88
> > > > This is the feature request for it:
> > > > https://github.com/tianocore/edk2-basetools/issues/87
> > > > I'm also attaching the patch here:
> > > > (0001-BaseTools-Generate-compile-information-in-build-repo.patch)
> > > >
> > > > On a side note, seems like tip of edk2-basetools is broken due
> > > > this
> > commit that makes direct import of Common package:
> > > > https://github.com/tianocore/edk2-
> > basetools/commit/8e6018d3ea4c1aae7
> > > > 185f589d129cea14a5d89fd
> > > > edk2-basetools\edk2basetools\GenFds\SubTypeGuidSection.py:
> > > > import Common.LongFilePathOs as os
> > > >
> > > >
> > >
> > > I sent a fix and a PR for this about a month ago:
> > >
> > > https://github.com/tianocore/edk2-basetools/pull/86
> > >
> > > but CodeCov seems to take issue with it, for reasons I don't
> > > understand (and I can't be bothered to spend more time on yet
> > > another CI automation pass that adds to my workload rather than
> > > reduce it)
> > >
> > >
> > >
> > >
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile information in build report
2023-03-15 1:12 ` Ni, Ray
@ 2023-03-15 10:28 ` Bob Feng
0 siblings, 0 replies; 17+ messages in thread
From: Bob Feng @ 2023-03-15 10:28 UTC (permalink / raw)
To: Ni, Ray, Palomino Sosa, Guillermo A, Kinney, Michael D,
Gao, Liming, devel@edk2.groups.io
Cc: Chen, Christine, Oram, Isaac W, Sean Brogan, ardb@kernel.org
Ray, it's merged to edk2 repo.
-----Original Message-----
From: Ni, Ray <ray.ni@intel.com>
Sent: Wednesday, March 15, 2023 9:12 AM
To: Palomino Sosa, Guillermo A <guillermo.a.palomino.sosa@intel.com>; Kinney, Michael D <michael.d.kinney@intel.com>; Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming <gaoliming@byosoft.com.cn>; devel@edk2.groups.io
Cc: Chen, Christine <yuwei.chen@intel.com>; Oram, Isaac W <isaac.w.oram@intel.com>; Sean Brogan <sean.brogan@microsoft.com>; ardb@kernel.org
Subject: RE: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile information in build report
I saw the code was merged to edk2-basetools repo 5 days ago.
When will the change be in edk2 repo? I am really interested in using the json files😊
Thanks,
Ray
> -----Original Message-----
> From: Palomino Sosa, Guillermo A <guillermo.a.palomino.sosa@intel.com>
> Sent: Monday, March 6, 2023 11:41 PM
> To: Kinney, Michael D <michael.d.kinney@intel.com>; Feng, Bob C
> <bob.c.feng@intel.com>; Gao, Liming <gaoliming@byosoft.com.cn>;
> devel@edk2.groups.io
> Cc: Chen, Christine <yuwei.chen@intel.com>; Ni, Ray
> <ray.ni@intel.com>; Oram, Isaac W <isaac.w.oram@intel.com>; Sean
> Brogan <sean.brogan@microsoft.com>; ardb@kernel.org
> Subject: RE: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile
> information in build report
>
> Hi guys, can we submit the pull request or do you have comments on it?
> https://github.com/tianocore/edk2-basetools/pull/88
>
> Thanks
>
>
> -----Original Message-----
> From: Palomino Sosa, Guillermo A
> Sent: Tuesday, February 28, 2023 10:00 PM
> To: Ni, Ray <ray.ni@intel.com>; Kinney, Michael D
> <michael.d.kinney@intel.com>; devel@edk2.groups.io; ardb@kernel.org
> Cc: Chen, Christine <Yuwei.Chen@intel.com>; Feng, Bob C
> <bob.c.feng@intel.com>; Gao, Liming <gaoliming@byosoft.com.cn>; Oram,
> Isaac W <isaac.w.oram@intel.com>; Sean Brogan
> <sean.brogan@microsoft.com>
> Subject: RE: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile
> information in build report
>
> It takes same time as original build report to be generated as it
> constructed using the same data structures as build report. So I think
> its OK to not have it enabled by default.
>
> Patch is ready in the pull request to be reviews.
>
>
>
> -----Original Message-----
> From: Ni, Ray <ray.ni@intel.com>
> Sent: Tuesday, February 28, 2023 6:52 PM
> To: Palomino Sosa, Guillermo A <guillermo.a.palomino.sosa@intel.com>;
> Kinney, Michael D <michael.d.kinney@intel.com>; devel@edk2.groups.io;
> ardb@kernel.org
> Cc: Chen, Christine <yuwei.chen@intel.com>; Feng, Bob C
> <bob.c.feng@intel.com>; Gao, Liming <gaoliming@byosoft.com.cn>; Oram,
> Isaac W <isaac.w.oram@intel.com>; Sean Brogan
> <sean.brogan@microsoft.com>
> Subject: RE: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile
> information in build report
>
> What's the status of this patch?
> Does report generation take time? If no, why not generate them by
> default without individual flag control.
> I really like the feature to generate "compile_commands.json"
>
> > -----Original Message-----
> > From: Palomino Sosa, Guillermo A
> > <guillermo.a.palomino.sosa@intel.com>
> > Sent: Tuesday, February 28, 2023 7:42 AM
> > To: Kinney, Michael D <michael.d.kinney@intel.com>;
> > devel@edk2.groups.io; ardb@kernel.org
> > Cc: Ni, Ray <ray.ni@intel.com>; Chen, Christine
> > <yuwei.chen@intel.com>; Feng, Bob C <bob.c.feng@intel.com>; Gao,
> > Liming <gaoliming@byosoft.com.cn>; Oram, Isaac W
> > <isaac.w.oram@intel.com>; Sean Brogan <sean.brogan@microsoft.com>
> > Subject: RE: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile
> > information in build report
> >
> > I have updated the pull based on Sean feedback. I added following
> > fields to
> > module_report.json:
> > * LibraryClass
> > * ModuleEntryPointList
> > * ConstructorList
> > * DestructorList
> >
> > I have also added commit from Ard based on this request to fix build issue:
> > https://github.com/tianocore/edk2-basetools/pull/88
> >
> > Thanks
> >
> > -----Original Message-----
> > From: Kinney, Michael D <michael.d.kinney@intel.com>
> > Sent: Monday, February 27, 2023 4:36 PM
> > To: devel@edk2.groups.io; ardb@kernel.org; Palomino Sosa, Guillermo
> > A <guillermo.a.palomino.sosa@intel.com>
> > Cc: Ni, Ray <ray.ni@intel.com>; Chen, Christine
> > <yuwei.chen@intel.com>; Feng, Bob C <bob.c.feng@intel.com>; Gao,
> > Liming <gaoliming@byosoft.com.cn>; Oram, Isaac W
> > <isaac.w.oram@intel.com>; Sean Brogan <sean.brogan@microsoft.com>;
> > Kinney, Michael D <michael.d.kinney@intel.com>
> > Subject: RE: [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile
> > information in build report
> >
> > Hi Guillermo,
> >
> > Can you please look at Ards PR and make sure his fix is included in your PR.
> >
> > Also, please work with Christine and Bob to see what is going on
> > with the Code Coverage check. We do want it to be easy for all
> > community members to submit change requests. We may need support
> > from the edk2-bastools maintainers to help with CI issues and help
> > with changes to
> address.
> >
> > Thanks,
> >
> > Mike
> >
> > > -----Original Message-----
> > > From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Ard
> > > Biesheuvel
> > > Sent: Monday, February 27, 2023 10:58 AM
> > > To: devel@edk2.groups.io; Palomino Sosa, Guillermo A
> > > <guillermo.a.palomino.sosa@intel.com>
> > > Cc: Ni, Ray <ray.ni@intel.com>; Kinney, Michael D
> > > <michael.d.kinney@intel.com>; Chen, Christine
> > > <yuwei.chen@intel.com>; Feng, Bob C <bob.c.feng@intel.com>; Gao,
> > > Liming <gaoliming@byosoft.com.cn>; Oram, Isaac W
> > > <isaac.w.oram@intel.com>; Sean Brogan <sean.brogan@microsoft.com>
> > > Subject: Re: [edk2-devel][PATCH V1 1/1] BaseTools: Generate
> > > compile information in build report
> > >
> > > On Mon, 27 Feb 2023 at 18:40, Guillermo Antonio Palomino Sosa
> > > <guillermo.a.palomino.sosa@intel.com> wrote:
> > > >
> > > > Hi. I have submitted a pull request to edk2-basetools repository:
> > > > https://github.com/tianocore/edk2-basetools/pull/88
> > > > This is the feature request for it:
> > > > https://github.com/tianocore/edk2-basetools/issues/87
> > > > I'm also attaching the patch here:
> > > > (0001-BaseTools-Generate-compile-information-in-build-repo.patch
> > > > )
> > > >
> > > > On a side note, seems like tip of edk2-basetools is broken due
> > > > this
> > commit that makes direct import of Common package:
> > > > https://github.com/tianocore/edk2-
> > basetools/commit/8e6018d3ea4c1aae7
> > > > 185f589d129cea14a5d89fd
> > > > edk2-basetools\edk2basetools\GenFds\SubTypeGuidSection.py:
> > > > import Common.LongFilePathOs as os
> > > >
> > > >
> > >
> > > I sent a fix and a PR for this about a month ago:
> > >
> > > https://github.com/tianocore/edk2-basetools/pull/86
> > >
> > > but CodeCov seems to take issue with it, for reasons I don't
> > > understand (and I can't be bothered to spend more time on yet
> > > another CI automation pass that adds to my workload rather than
> > > reduce it)
> > >
> > >
> > >
> > >
^ permalink raw reply [flat|nested] 17+ messages in thread
end of thread, other threads:[~2023-03-15 10:28 UTC | newest]
Thread overview: 17+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-02-07 3:07 [edk2-devel][PATCH V1 1/1] BaseTools: Generate compile information in build report Guillermo Antonio Palomino Sosa
2023-02-15 2:43 ` Yuwei Chen
2023-02-15 3:42 ` Michael D Kinney
2023-02-15 5:38 ` Yuwei Chen
2023-02-15 8:17 ` Ni, Ray
2023-02-15 23:12 ` Michael D Kinney
2023-02-16 0:50 ` Ni, Ray
2023-02-18 4:42 ` [edk2-devel] [PATCH " Guillermo Antonio Palomino Sosa
2023-02-27 17:40 ` [edk2-devel][PATCH " Guillermo Antonio Palomino Sosa
2023-02-27 18:57 ` Ard Biesheuvel
2023-02-27 22:35 ` Michael D Kinney
2023-02-27 23:42 ` Guillermo Antonio Palomino Sosa
2023-03-01 0:52 ` Ni, Ray
2023-03-01 4:00 ` Guillermo Antonio Palomino Sosa
2023-03-06 15:40 ` Guillermo Antonio Palomino Sosa
2023-03-15 1:12 ` Ni, Ray
2023-03-15 10:28 ` Bob Feng
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox