Re: [PATCH v1 1/1] BaseTools/Plugin: Clarify code coverage failure message

["Rebecca Cran" <rebecca@bsdio.com>]

Reviewed-by: Rebecca Cran <rebecca@bsdio.com>


On 4/18/23 10:11 AM, mikuback@linux.microsoft.com wrote:
> From: Michael Kubacki <michael.kubacki@microsoft.com>
>
> HostBasedUnitTestRunner.py is a build plugin responsible for locating
> and executing host-based unit tests.
>
> Recently, commit 6bb00aa introduced support for the plugin to
> generate code coverage reports via lcov and OpenCppCoverage.
>
> The plugin has discovered unit tests by searching for executables
> with "Test" in the name for a while. However, the test coverage
> change makes assumptions about test presence when crafting the
> OpenCppCoverage command that ultimately fails with an ambiguous error
> message if no host-based unit tests are discovered (see "ERROR").
>
> ```
> SECTION - Run Host based Unit Tests
> SUBSECTION - Testing for architecture: X64
> ERROR - UnitTest Coverage: Failed to generate cobertura format xml in
>          single package.
> PROGRESS - --->Test Success: Host Unit Test Compiler Plugin NOOPT
> ```
>
> This change preempts that message with a check in the plugin to
> determine if any host-based tests were discovered. If not, a message
> is printed with more guidance about how the developer should proceed
> to either (1) fix their tests so code coverage is generated as
> expected or (2) prevent the error message.
>
> New message:
>
> ```
> SECTION - Run Host based Unit Tests
> SUBSECTION - Testing for architecture: X64
> WARNING - UnitTest Coverage:
>    No unit tests discovered. Test coverage will not be generated.
>
>    Prevent this message by:
>    1. Adding host-based unit tests to this package
>    2. Ensuring tests have the word "Test" in their name
>    3. Disabling HostUnitTestCompilerPlugin in the package CI YAML file
> PROGRESS - --->Test Success: Host Unit Test Compiler Plugin NOOPT
> ```
>
> Cc: Rebecca Cran <rebecca@bsdio.com>
> Cc: Liming Gao <gaoliming@byosoft.com.cn>
> Cc: Bob Feng <bob.c.feng@intel.com>
> Cc: Yuwei Chen <yuwei.chen@intel.com>
> Cc: Michael D Kinney <michael.d.kinney@intel.com>
> Cc: Sean Brogan <sean.brogan@microsoft.com>
> Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
> ---
>   BaseTools/Plugin/HostBasedUnitTestRunner/HostBasedUnitTestRunner.py | 13 +++++++++++++
>   1 file changed, 13 insertions(+)
>
> diff --git a/BaseTools/Plugin/HostBasedUnitTestRunner/HostBasedUnitTestRunner.py b/BaseTools/Plugin/HostBasedUnitTestRunner/HostBasedUnitTestRunner.py
> index 0e013c5f1a7f..a384b556294c 100644
> --- a/BaseTools/Plugin/HostBasedUnitTestRunner/HostBasedUnitTestRunner.py
> +++ b/BaseTools/Plugin/HostBasedUnitTestRunner/HostBasedUnitTestRunner.py
> @@ -16,6 +16,7 @@ import edk2toollib.windows.locate_tools as locate_tools
>   from edk2toolext.environment import shell_environment
>   from edk2toollib.utility_functions import RunCmd
>   from edk2toollib.utility_functions import GetHostInfo
> +from textwrap import dedent
>   
>   
>   class HostBasedUnitTestRunner(IUefiBuildPlugin):
> @@ -84,6 +85,18 @@ class HostBasedUnitTestRunner(IUefiBuildPlugin):
>               else:
>                   raise NotImplementedError("Unsupported Operating System")
>   
> +            if not testList:
> +                logging.warning(dedent("""
> +                    UnitTest Coverage:
> +                      No unit tests discovered. Test coverage will not be generated.
> +
> +                      Prevent this message by:
> +                      1. Adding host-based unit tests to this package
> +                      2. Ensuring tests have the word "Test" in their name
> +                      3. Disabling HostUnitTestCompilerPlugin in the package CI YAML file
> +                    """).strip())
> +                return 0
> +
>               for test in testList:
>                   # Configure output name if test uses cmocka.
>                   shell_env.set_shell_var(