[PATCH v1 0/1] Improve Edk2ToolsBuild.py compiler logging

[PATCH v1 0/1] Improve Edk2ToolsBuild.py compiler logging

[Joey Vagedes]

Improve the logging of compiler errors when building basetools with
Edk2ToolsBuild.py by attaching an output stream, running the command,
then scanning that output stream. This is the same logic that
stuart_build uses to log make errors from an edk2 build to the
terminal.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>

Joey Vagedes (1):
  BaseTools: scan Edk2ToolsBuild.py make output

 BaseTools/Edk2ToolsBuild.py | 11 +++++++++++
 1 file changed, 11 insertions(+)

-- 
2.41.0.windows.1

[PATCH v1 1/1] BaseTools: scan Edk2ToolsBuild.py make output

[Joey Vagedes]

Adds edk2_logging.scan_compiler_output() to Edk2ToolsBuild.py to catch
some compilation errors and log them as an error.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Joey Vagedes <joeyvagedes@gmail.com>
---
 BaseTools/Edk2ToolsBuild.py | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/BaseTools/Edk2ToolsBuild.py b/BaseTools/Edk2ToolsBuild.py
index f862468ce275..425bb1b63963 100644
--- a/BaseTools/Edk2ToolsBuild.py
+++ b/BaseTools/Edk2ToolsBuild.py
@@ -133,8 +133,13 @@ class Edk2ToolsBuild(BaseAbstractInvocable):
             shell_env.insert_path(self.OutputDir)
 
             # Actually build the tools.
+            output_stream = edk2_logging.create_output_stream()
             ret = RunCmd('nmake.exe', None,
                          workingdir=shell_env.get_shell_var("EDK_TOOLS_PATH"))
+            edk2_logging.remove_output_stream(output_stream)
+            problems = edk2_logging.scan_compiler_output(output_stream)
+            for level, problem in problems:
+                logging.log(level, problem)
             if ret != 0:
                 raise Exception("Failed to build.")
 
@@ -143,7 +148,13 @@ class Edk2ToolsBuild(BaseAbstractInvocable):
 
         elif self.tool_chain_tag.lower().startswith("gcc"):
             cpu_count = self.GetCpuThreads()
+
+            output_stream = edk2_logging.create_output_stream()
             ret = RunCmd("make", f"-C .  -j {cpu_count}", workingdir=shell_env.get_shell_var("EDK_TOOLS_PATH"))
+            edk2_logging.remove_output_stream(output_stream)
+            problems = edk2_logging.scan_compiler_output(output_stream)
+            for level, problem in problems:
+                logging.log(level, problem)
             if ret != 0:
                 raise Exception("Failed to build.")
 
-- 
2.41.0.windows.1

Re: [PATCH v1 1/1] BaseTools: scan Edk2ToolsBuild.py make output

[Joey Vagedes]

[-- Attachment #1: Type: text/plain, Size: 2410 bytes --]

Hi All,

Any concerns over this patch? It merely adds the ability to detect some
compiler errors when building BaseTools with Edk2ToolsBuild.py and report
them as logging level ERROR to get a quick glimpse at the error without
needing to review the entire log.

Thanks,
Joey

On Wed, Jun 21, 2023 at 8:59 AM Joey Vagedes <joey.vagedes@gmail.com> wrote:

> Adds edk2_logging.scan_compiler_output() to Edk2ToolsBuild.py to catch
> some compilation errors and log them as an error.
>
> Cc: Rebecca Cran <rebecca@bsdio.com>
> Cc: Liming Gao <gaoliming@byosoft.com.cn>
> Cc: Bob Feng <bob.c.feng@intel.com>
> Cc: Yuwei Chen <yuwei.chen@intel.com>
> Signed-off-by: Joey Vagedes <joeyvagedes@gmail.com>
> ---
>  BaseTools/Edk2ToolsBuild.py | 11 +++++++++++
>  1 file changed, 11 insertions(+)
>
> diff --git a/BaseTools/Edk2ToolsBuild.py b/BaseTools/Edk2ToolsBuild.py
> index f862468ce275..425bb1b63963 100644
> --- a/BaseTools/Edk2ToolsBuild.py
> +++ b/BaseTools/Edk2ToolsBuild.py
> @@ -133,8 +133,13 @@ class Edk2ToolsBuild(BaseAbstractInvocable):
>              shell_env.insert_path(self.OutputDir)
>
>              # Actually build the tools.
> +            output_stream = edk2_logging.create_output_stream()
>              ret = RunCmd('nmake.exe', None,
>
> workingdir=shell_env.get_shell_var("EDK_TOOLS_PATH"))
> +            edk2_logging.remove_output_stream(output_stream)
> +            problems = edk2_logging.scan_compiler_output(output_stream)
> +            for level, problem in problems:
> +                logging.log(level, problem)
>              if ret != 0:
>                  raise Exception("Failed to build.")
>
> @@ -143,7 +148,13 @@ class Edk2ToolsBuild(BaseAbstractInvocable):
>
>          elif self.tool_chain_tag.lower().startswith("gcc"):
>              cpu_count = self.GetCpuThreads()
> +
> +            output_stream = edk2_logging.create_output_stream()
>              ret = RunCmd("make", f"-C .  -j {cpu_count}",
> workingdir=shell_env.get_shell_var("EDK_TOOLS_PATH"))
> +            edk2_logging.remove_output_stream(output_stream)
> +            problems = edk2_logging.scan_compiler_output(output_stream)
> +            for level, problem in problems:
> +                logging.log(level, problem)
>              if ret != 0:
>                  raise Exception("Failed to build.")
>
> --
> 2.41.0.windows.1
>
>

[-- Attachment #2: Type: text/html, Size: 3409 bytes --]

Re: [PATCH v1 1/1] BaseTools: scan Edk2ToolsBuild.py make output

[Rebecca Cran]

Sorry for the delay.


Reviewed-by: Rebecca Cran <rebecca@bsdio.com>


On 6/21/23 9:59 AM, Joey Vagedes wrote:
> Adds edk2_logging.scan_compiler_output() to Edk2ToolsBuild.py to catch
> some compilation errors and log them as an error.
>
> Cc: Rebecca Cran <rebecca@bsdio.com>
> Cc: Liming Gao <gaoliming@byosoft.com.cn>
> Cc: Bob Feng <bob.c.feng@intel.com>
> Cc: Yuwei Chen <yuwei.chen@intel.com>
> Signed-off-by: Joey Vagedes <joeyvagedes@gmail.com>
> ---
>   BaseTools/Edk2ToolsBuild.py | 11 +++++++++++
>   1 file changed, 11 insertions(+)
>
> diff --git a/BaseTools/Edk2ToolsBuild.py b/BaseTools/Edk2ToolsBuild.py
> index f862468ce275..425bb1b63963 100644
> --- a/BaseTools/Edk2ToolsBuild.py
> +++ b/BaseTools/Edk2ToolsBuild.py
> @@ -133,8 +133,13 @@ class Edk2ToolsBuild(BaseAbstractInvocable):
>               shell_env.insert_path(self.OutputDir)
>   
>               # Actually build the tools.
> +            output_stream = edk2_logging.create_output_stream()
>               ret = RunCmd('nmake.exe', None,
>                            workingdir=shell_env.get_shell_var("EDK_TOOLS_PATH"))
> +            edk2_logging.remove_output_stream(output_stream)
> +            problems = edk2_logging.scan_compiler_output(output_stream)
> +            for level, problem in problems:
> +                logging.log(level, problem)
>               if ret != 0:
>                   raise Exception("Failed to build.")
>   
> @@ -143,7 +148,13 @@ class Edk2ToolsBuild(BaseAbstractInvocable):
>   
>           elif self.tool_chain_tag.lower().startswith("gcc"):
>               cpu_count = self.GetCpuThreads()
> +
> +            output_stream = edk2_logging.create_output_stream()
>               ret = RunCmd("make", f"-C .  -j {cpu_count}", workingdir=shell_env.get_shell_var("EDK_TOOLS_PATH"))
> +            edk2_logging.remove_output_stream(output_stream)
> +            problems = edk2_logging.scan_compiler_output(output_stream)
> +            for level, problem in problems:
> +                logging.log(level, problem)
>               if ret != 0:
>                   raise Exception("Failed to build.")
>   

Re: [PATCH v1 1/1] BaseTools: scan Edk2ToolsBuild.py make output

[Joey Vagedes]

[-- Attachment #1: Type: text/plain, Size: 2441 bytes --]

Thanks Rebecca! Let me know if you need anything else from me before you
get it merged. I appreciate your time!

Joey

On Sun, Jul 9, 2023 at 4:18 PM Rebecca Cran <rebecca@bsdio.com> wrote:

> Sorry for the delay.
>
>
> Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
>
>
> On 6/21/23 9:59 AM, Joey Vagedes wrote:
> > Adds edk2_logging.scan_compiler_output() to Edk2ToolsBuild.py to catch
> > some compilation errors and log them as an error.
> >
> > Cc: Rebecca Cran <rebecca@bsdio.com>
> > Cc: Liming Gao <gaoliming@byosoft.com.cn>
> > Cc: Bob Feng <bob.c.feng@intel.com>
> > Cc: Yuwei Chen <yuwei.chen@intel.com>
> > Signed-off-by: Joey Vagedes <joeyvagedes@gmail.com>
> > ---
> >   BaseTools/Edk2ToolsBuild.py | 11 +++++++++++
> >   1 file changed, 11 insertions(+)
> >
> > diff --git a/BaseTools/Edk2ToolsBuild.py b/BaseTools/Edk2ToolsBuild.py
> > index f862468ce275..425bb1b63963 100644
> > --- a/BaseTools/Edk2ToolsBuild.py
> > +++ b/BaseTools/Edk2ToolsBuild.py
> > @@ -133,8 +133,13 @@ class Edk2ToolsBuild(BaseAbstractInvocable):
> >               shell_env.insert_path(self.OutputDir)
> >
> >               # Actually build the tools.
> > +            output_stream = edk2_logging.create_output_stream()
> >               ret = RunCmd('nmake.exe', None,
> >
> workingdir=shell_env.get_shell_var("EDK_TOOLS_PATH"))
> > +            edk2_logging.remove_output_stream(output_stream)
> > +            problems = edk2_logging.scan_compiler_output(output_stream)
> > +            for level, problem in problems:
> > +                logging.log(level, problem)
> >               if ret != 0:
> >                   raise Exception("Failed to build.")
> >
> > @@ -143,7 +148,13 @@ class Edk2ToolsBuild(BaseAbstractInvocable):
> >
> >           elif self.tool_chain_tag.lower().startswith("gcc"):
> >               cpu_count = self.GetCpuThreads()
> > +
> > +            output_stream = edk2_logging.create_output_stream()
> >               ret = RunCmd("make", f"-C .  -j {cpu_count}",
> workingdir=shell_env.get_shell_var("EDK_TOOLS_PATH"))
> > +            edk2_logging.remove_output_stream(output_stream)
> > +            problems = edk2_logging.scan_compiler_output(output_stream)
> > +            for level, problem in problems:
> > +                logging.log(level, problem)
> >               if ret != 0:
> >                   raise Exception("Failed to build.")
> >
>

[-- Attachment #2: Type: text/html, Size: 3637 bytes --]