[PATCH v2] UnitTestFrameworkPkg: Add code coverage support for GCC

public inbox for devel@edk2.groups.io
 help / color / mirror / Atom feed

* [PATCH v2] UnitTestFrameworkPkg: Add code coverage support for GCC
@ 2022-09-14  1:01 Guo, Gua
  2022-09-14  1:17 ` [edk2-devel] " Sean
  0 siblings, 1 reply; 9+ messages in thread
From: Guo, Gua @ 2022-09-14  1:01 UTC (permalink / raw)
  To: devel
  Cc: Gua Guo, Michael D Kinney, Sean Brogan, Bret Barkelew,
	Michael Kubacki

From: Gua Guo <gua.guo@intel.com>

In order to collect code coverage after running executable
file, generate *.gcda and *.gcno file that require by lcov tool
to generate code coverage report.

Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Michael Kubacki <mikuback@linux.microsoft.com>
Signed-off-by: Gua Guo <gua.guo@intel.com>
---
 UnitTestFrameworkPkg/ReadMe.md                        | 116 +++++++++++++++++++-
 UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc |   2 +-
 2 files changed, 116 insertions(+), 2 deletions(-)

diff --git a/UnitTestFrameworkPkg/ReadMe.md b/UnitTestFrameworkPkg/ReadMe.md
index e696412cb3..93fdacfc4a 100644
--- a/UnitTestFrameworkPkg/ReadMe.md
+++ b/UnitTestFrameworkPkg/ReadMe.md
@@ -361,7 +361,7 @@ RUNNING TEST SUITE: Int Safe Conversions Test Suite
 ```
 
 You can also, if you are so inclined, read the output from the exact instance of the test that was run during
-`stuart_ci_build`. The ouput file can be found on a path that looks like:
+`stuart_ci_build`. The output file can be found on a path that looks like:
 
 `Build/<Package>/HostTest/<Arch>/<TestName>.<TestSuiteName>.<Arch>.result.xml`
 
@@ -399,6 +399,120 @@ CMOCKA_XML_FILE=<absolute or relative path to output file>
 
 This mode is used by the test running plugin to aggregate the results for CI test status reporting in the web view.
 
+### Enable Code Coverage for GCC host-based tests
+
+LCOV is an extension of GCOV, a GNU tool which provides information about what parts of a program are actually executed (i.e. "covered") while running a particular test case. The extension consists of a set of scripts which build on the textual GCOV output to implement it.
+
+Documentation for LCOV can be found here:
+https://lcov.readthedocs.io/
+
+You can follow below steps to generate your code coverage report.
+
+
+
+```bash
+user@linux_machine:~/_uefi$stuart_ci_build -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5 -p MdePkg -t NOOPT
+...
+user@linux_machine:~/_uefi$lcov -c --rc lcov_branch_coverage=1 -o UnitTest.info -d Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/
+Capturing coverage data from Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/
+Found gcov version: 9.4.0
+Using intermediate gcov format
+Scanning Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/ for .gcda files ...
+Found 38 data files in Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/
+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/RShiftU64.gcda
+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/Math64.gcda
+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/SafeString.gcda
+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/SwapBytes32.gcda
+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/CpuDeadLoop.gcda
+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/MultU64x64.gcda
+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/LongJump.gcda
+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/SetJump.gcda
+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/BitField.gcda
+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/String.gcda
+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/DivU64x32Remainder.gcda
+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/X64/GccInline.gcda
+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/MultU64x32.gcda
+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/MultS64x64.gcda
+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/LinkedList.gcda
+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/LShiftU64.gcda
+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/DivU64x32.gcda
+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/SwapBytes16.gcda
+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/Unaligned.gcda
+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/CheckSum.gcda
+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/CopyMem.gcda
+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/MemLibGuid.gcda
+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/CompareMemWrapper.gcda
+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/SetMemWrapper.gcda
+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/SetMem64Wrapper.gcda
+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/MemLibGeneric.gcda
+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/CopyMemWrapper.gcda
+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/SetMem.gcda
+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/ZeroMemWrapper.gcda
+Processing Library/BasePrintLib/BasePrintLib/OUTPUT/PrintLibInternal.gcda
+Processing Library/BasePrintLib/BasePrintLib/OUTPUT/PrintLib.gcda
+Processing Library/BaseSafeIntLib/BaseSafeIntLib/OUTPUT/SafeIntLib64.gcda
+Processing Library/BaseSafeIntLib/BaseSafeIntLib/OUTPUT/SafeIntLib.gcda
+Processing Test/UnitTest/Library/BaseLib/BaseLibUnitTestsHost/OUTPUT/Base64UnitTest.gcda
+Processing Test/UnitTest/Library/BaseLib/BaseLibUnitTestsHost/OUTPUT/AutoGen.gcda
+Processing Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLibHost/OUTPUT/SafeIntLibUintnIntnUnitTests64.gcda
+Processing Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLibHost/OUTPUT/TestBaseSafeIntLib.gcda
+Processing Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLibHost/OUTPUT/AutoGen.gcda
+
+user@linux_machine:~/_uefi$genhtml UnitTest.info --branch-coverage -o Build/UnitTestReport
+
+Reading data file UnitTest.info
+Found 38 entries.
+Found common filename prefix "~/_uefi/MdePkg/Library"
+Writing .css and .png files.
+Generating output.
+Processing file ~/_uefi/Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/Test/UnitTest/Library/BaseLib/BaseLibUnitTestsHost/DEBUG/AutoGen.c
+Processing file ~/_uefi/Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLibHost/DEBUG/AutoGen.c
+Processing file BaseLib/Unaligned.c
+Processing file BaseLib/CheckSum.c
+Processing file BaseLib/BitField.c
+Processing file BaseLib/DivU64x32.c
+Processing file BaseLib/CpuDeadLoop.c
+Processing file BaseLib/LongJump.c
+Processing file BaseLib/MultU64x64.c
+Processing file BaseLib/RShiftU64.c
+Processing file BaseLib/SwapBytes32.c
+Processing file BaseLib/SwapBytes16.c
+Processing file BaseLib/SetJump.c
+Processing file BaseLib/LinkedList.c
+Processing file BaseLib/LShiftU64.c
+Processing file BaseLib/SafeString.c
+Processing file BaseLib/String.c
+Processing file BaseLib/DivU64x32Remainder.c
+Processing file BaseLib/MultS64x64.c
+Processing file BaseLib/MultU64x32.c
+Processing file BaseLib/Math64.c
+Processing file BaseLib/X64/GccInline.c
+Processing file BaseMemoryLib/SetMemWrapper.c
+Processing file BaseMemoryLib/CompareMemWrapper.c
+Processing file BaseMemoryLib/SetMem64Wrapper.c
+Processing file BaseMemoryLib/ZeroMemWrapper.c
+Processing file BaseMemoryLib/CopyMemWrapper.c
+Processing file BaseMemoryLib/CopyMem.c
+Processing file BaseMemoryLib/SetMem.c
+Processing file BaseMemoryLib/MemLibGuid.c
+Processing file BaseMemoryLib/MemLibGeneric.c
+Processing file BasePrintLib/PrintLib.c
+Processing file BasePrintLib/PrintLibInternal.c
+Processing file BaseSafeIntLib/SafeIntLib64.c
+Processing file BaseSafeIntLib/SafeIntLib.c
+Processing file ~/_uefi/MdePkg/Test/UnitTest/Library/BaseLib/Base64UnitTest.c
+Processing file ~/_uefi/MdePkg/Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLib.c
+Processing file ~/_uefi/MdePkg/Test/UnitTest/Library/BaseSafeIntLib/SafeIntLibUintnIntnUnitTests64.c
+Writing directory view page.
+Overall coverage rate:
+  lines......: 58.6% (2688 of 4589 lines)
+  functions..: 57.1% (249 of 436 functions)
+  branches...: 25.3% (1062 of 4195 branches)
+
+...
+
+```
+
 ### Important Note
 
 This works on both Windows and Linux, but is currently limited to x64 architectures. Working on getting others, but we
diff --git a/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc b/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc
index 4dd8d4ac67..8623aecb33 100644
--- a/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc
+++ b/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc
@@ -19,7 +19,7 @@
   MemoryAllocationLib|UnitTestFrameworkPkg/Library/Posix/MemoryAllocationLibPosix/MemoryAllocationLibPosix.inf
 
 [BuildOptions]
-  GCC:*_*_*_CC_FLAGS = -fno-pie
+  GCC:*_*_*_CC_FLAGS = -fno-pie -ftest-coverage -fprofile-arcs
 !ifdef $(UNIT_TESTING_DEBUG)
   MSFT:*_*_*_CC_FLAGS  = -D UNIT_TESTING_DEBUG=1
   GCC:*_*_*_CC_FLAGS   = -D UNIT_TESTING_DEBUG=1
-- 
2.31.1.windows.1


^ permalink raw reply related	[flat|nested] 9+ messages in thread

* Re: [edk2-devel] [PATCH v2] UnitTestFrameworkPkg: Add code coverage support for GCC
  2022-09-14  1:01 [PATCH v2] UnitTestFrameworkPkg: Add code coverage support for GCC Guo, Gua
@ 2022-09-14  1:17 ` Sean
  2022-09-14  1:51   ` Michael D Kinney
  0 siblings, 1 reply; 9+ messages in thread
From: Sean @ 2022-09-14  1:17 UTC (permalink / raw)
  To: devel, gua.guo; +Cc: Michael D Kinney, Sean Brogan, Michael Kubacki

[-- Attachment #1: Type: text/plain, Size: 9627 bytes --]

Hi Gua Guo,

I would propose that we make this easier for both the local user and the 
CI system to get coverage metrics.

Can you review this commit as it uses the same Edk2 plugins but adds 
code coverage support with GCC5:

Merged PR 5008: Initial implementation or code coverage on basecore u… · 
microsoft/mu_basecore@f6af51f (github.com) 
<https://github.com/microsoft/mu_basecore/commit/f6af51f856747faec5a9f78f5604a69211b5af2d#diff-52e599c27ab877ef616f244e8b26826a53853872acb894df4aeff99f48df373d>


Thanks

Sean



On 9/13/2022 6:01 PM, Guo, Gua wrote:
> From: Gua Guo<gua.guo@intel.com>
>
> In order to collect code coverage after running executable
> file, generate *.gcda and *.gcno file that require by lcov tool
> to generate code coverage report.
>
> Cc: Michael D Kinney<michael.d.kinney@intel.com>
> Cc: Sean Brogan<sean.brogan@microsoft.com>
> Cc: Bret Barkelew<Bret.Barkelew@microsoft.com>
> Cc: Michael Kubacki<mikuback@linux.microsoft.com>
> Signed-off-by: Gua Guo<gua.guo@intel.com>
> ---
>   UnitTestFrameworkPkg/ReadMe.md                        | 116 +++++++++++++++++++-
>   UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc |   2 +-
>   2 files changed, 116 insertions(+), 2 deletions(-)
>
> diff --git a/UnitTestFrameworkPkg/ReadMe.md b/UnitTestFrameworkPkg/ReadMe.md
> index e696412cb3..93fdacfc4a 100644
> --- a/UnitTestFrameworkPkg/ReadMe.md
> +++ b/UnitTestFrameworkPkg/ReadMe.md
> @@ -361,7 +361,7 @@ RUNNING TEST SUITE: Int Safe Conversions Test Suite
>   ```
>
>   
>
>   You can also, if you are so inclined, read the output from the exact instance of the test that was run during
>
> -`stuart_ci_build`. The ouput file can be found on a path that looks like:
>
> +`stuart_ci_build`. The output file can be found on a path that looks like:
>
>   
>
>   `Build/<Package>/HostTest/<Arch>/<TestName>.<TestSuiteName>.<Arch>.result.xml`
>
>   
>
> @@ -399,6 +399,120 @@ CMOCKA_XML_FILE=<absolute or relative path to output file>
>   
>
>   This mode is used by the test running plugin to aggregate the results for CI test status reporting in the web view.
>
>   
>
> +### Enable Code Coverage for GCC host-based tests
>
> +
>
> +LCOV is an extension of GCOV, a GNU tool which provides information about what parts of a program are actually executed (i.e. "covered") while running a particular test case. The extension consists of a set of scripts which build on the textual GCOV output to implement it.
>
> +
>
> +Documentation for LCOV can be found here:
>
> +https://lcov.readthedocs.io/
>
> +
>
> +You can follow below steps to generate your code coverage report.
>
> +
>
> +
>
> +
>
> +```bash
>
> +user@linux_machine:~/_uefi$stuart_ci_build -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5 -p MdePkg -t NOOPT
>
> +...
>
> +user@linux_machine:~/_uefi$lcov -c --rc lcov_branch_coverage=1 -o UnitTest.info -d Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/
>
> +Capturing coverage data from Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/
>
> +Found gcov version: 9.4.0
>
> +Using intermediate gcov format
>
> +Scanning Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/ for .gcda files ...
>
> +Found 38 data files in Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/
>
> +Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/RShiftU64.gcda
>
> +Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/Math64.gcda
>
> +Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/SafeString.gcda
>
> +Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/SwapBytes32.gcda
>
> +Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/CpuDeadLoop.gcda
>
> +Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/MultU64x64.gcda
>
> +Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/LongJump.gcda
>
> +Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/SetJump.gcda
>
> +Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/BitField.gcda
>
> +Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/String.gcda
>
> +Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/DivU64x32Remainder.gcda
>
> +Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/X64/GccInline.gcda
>
> +Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/MultU64x32.gcda
>
> +Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/MultS64x64.gcda
>
> +Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/LinkedList.gcda
>
> +Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/LShiftU64.gcda
>
> +Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/DivU64x32.gcda
>
> +Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/SwapBytes16.gcda
>
> +Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/Unaligned.gcda
>
> +Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/CheckSum.gcda
>
> +Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/CopyMem.gcda
>
> +Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/MemLibGuid.gcda
>
> +Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/CompareMemWrapper.gcda
>
> +Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/SetMemWrapper.gcda
>
> +Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/SetMem64Wrapper.gcda
>
> +Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/MemLibGeneric.gcda
>
> +Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/CopyMemWrapper.gcda
>
> +Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/SetMem.gcda
>
> +Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/ZeroMemWrapper.gcda
>
> +Processing Library/BasePrintLib/BasePrintLib/OUTPUT/PrintLibInternal.gcda
>
> +Processing Library/BasePrintLib/BasePrintLib/OUTPUT/PrintLib.gcda
>
> +Processing Library/BaseSafeIntLib/BaseSafeIntLib/OUTPUT/SafeIntLib64.gcda
>
> +Processing Library/BaseSafeIntLib/BaseSafeIntLib/OUTPUT/SafeIntLib.gcda
>
> +Processing Test/UnitTest/Library/BaseLib/BaseLibUnitTestsHost/OUTPUT/Base64UnitTest.gcda
>
> +Processing Test/UnitTest/Library/BaseLib/BaseLibUnitTestsHost/OUTPUT/AutoGen.gcda
>
> +Processing Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLibHost/OUTPUT/SafeIntLibUintnIntnUnitTests64.gcda
>
> +Processing Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLibHost/OUTPUT/TestBaseSafeIntLib.gcda
>
> +Processing Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLibHost/OUTPUT/AutoGen.gcda
>
> +
>
> +user@linux_machine:~/_uefi$genhtml UnitTest.info --branch-coverage -o Build/UnitTestReport
>
> +
>
> +Reading data file UnitTest.info
>
> +Found 38 entries.
>
> +Found common filename prefix "~/_uefi/MdePkg/Library"
>
> +Writing .css and .png files.
>
> +Generating output.
>
> +Processing file ~/_uefi/Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/Test/UnitTest/Library/BaseLib/BaseLibUnitTestsHost/DEBUG/AutoGen.c
>
> +Processing file ~/_uefi/Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLibHost/DEBUG/AutoGen.c
>
> +Processing file BaseLib/Unaligned.c
>
> +Processing file BaseLib/CheckSum.c
>
> +Processing file BaseLib/BitField.c
>
> +Processing file BaseLib/DivU64x32.c
>
> +Processing file BaseLib/CpuDeadLoop.c
>
> +Processing file BaseLib/LongJump.c
>
> +Processing file BaseLib/MultU64x64.c
>
> +Processing file BaseLib/RShiftU64.c
>
> +Processing file BaseLib/SwapBytes32.c
>
> +Processing file BaseLib/SwapBytes16.c
>
> +Processing file BaseLib/SetJump.c
>
> +Processing file BaseLib/LinkedList.c
>
> +Processing file BaseLib/LShiftU64.c
>
> +Processing file BaseLib/SafeString.c
>
> +Processing file BaseLib/String.c
>
> +Processing file BaseLib/DivU64x32Remainder.c
>
> +Processing file BaseLib/MultS64x64.c
>
> +Processing file BaseLib/MultU64x32.c
>
> +Processing file BaseLib/Math64.c
>
> +Processing file BaseLib/X64/GccInline.c
>
> +Processing file BaseMemoryLib/SetMemWrapper.c
>
> +Processing file BaseMemoryLib/CompareMemWrapper.c
>
> +Processing file BaseMemoryLib/SetMem64Wrapper.c
>
> +Processing file BaseMemoryLib/ZeroMemWrapper.c
>
> +Processing file BaseMemoryLib/CopyMemWrapper.c
>
> +Processing file BaseMemoryLib/CopyMem.c
>
> +Processing file BaseMemoryLib/SetMem.c
>
> +Processing file BaseMemoryLib/MemLibGuid.c
>
> +Processing file BaseMemoryLib/MemLibGeneric.c
>
> +Processing file BasePrintLib/PrintLib.c
>
> +Processing file BasePrintLib/PrintLibInternal.c
>
> +Processing file BaseSafeIntLib/SafeIntLib64.c
>
> +Processing file BaseSafeIntLib/SafeIntLib.c
>
> +Processing file ~/_uefi/MdePkg/Test/UnitTest/Library/BaseLib/Base64UnitTest.c
>
> +Processing file ~/_uefi/MdePkg/Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLib.c
>
> +Processing file ~/_uefi/MdePkg/Test/UnitTest/Library/BaseSafeIntLib/SafeIntLibUintnIntnUnitTests64.c
>
> +Writing directory view page.
>
> +Overall coverage rate:
>
> +  lines......: 58.6% (2688 of 4589 lines)
>
> +  functions..: 57.1% (249 of 436 functions)
>
> +  branches...: 25.3% (1062 of 4195 branches)
>
> +
>
> +...
>
> +
>
> +```
>
> +
>
>   ### Important Note
>
>   
>
>   This works on both Windows and Linux, but is currently limited to x64 architectures. Working on getting others, but we
>
> diff --git a/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc b/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc
> index 4dd8d4ac67..8623aecb33 100644
> --- a/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc
> +++ b/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc
> @@ -19,7 +19,7 @@
>     MemoryAllocationLib|UnitTestFrameworkPkg/Library/Posix/MemoryAllocationLibPosix/MemoryAllocationLibPosix.inf
>
>   
>
>   [BuildOptions]
>
> -  GCC:*_*_*_CC_FLAGS = -fno-pie
>
> +  GCC:*_*_*_CC_FLAGS = -fno-pie -ftest-coverage -fprofile-arcs
>
>   !ifdef $(UNIT_TESTING_DEBUG)
>
>     MSFT:*_*_*_CC_FLAGS  = -D UNIT_TESTING_DEBUG=1
>
>     GCC:*_*_*_CC_FLAGS   = -D UNIT_TESTING_DEBUG=1
>

[-- Attachment #2: Type: text/html, Size: 10196 bytes --]

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [edk2-devel] [PATCH v2] UnitTestFrameworkPkg: Add code coverage support for GCC
  2022-09-14  1:17 ` [edk2-devel] " Sean
@ 2022-09-14  1:51   ` Michael D Kinney
  2022-09-14  2:10     ` Sean
  2022-09-14  2:11     ` Guo, Gua
  0 siblings, 2 replies; 9+ messages in thread
From: Michael D Kinney @ 2022-09-14  1:51 UTC (permalink / raw)
  To: Sean Brogan, devel@edk2.groups.io, Guo, Gua, Kinney, Michael D
  Cc: Sean Brogan, Michael Kubacki

[-- Attachment #1: Type: text/plain, Size: 10838 bytes --]

Does that pytool plugin support viewing the code coverage visually in C source files?

Either HTML or preferably an Visual Studio Code plugin?

Can it be updated to support VS compilers tool?

Mie

From: Sean Brogan <spbrogan@outlook.com>
Sent: Tuesday, September 13, 2022 6:18 PM
To: devel@edk2.groups.io; Guo, Gua <gua.guo@intel.com>
Cc: Kinney, Michael D <michael.d.kinney@intel.com>; Sean Brogan <sean.brogan@microsoft.com>; Michael Kubacki <mikuback@linux.microsoft.com>
Subject: Re: [edk2-devel] [PATCH v2] UnitTestFrameworkPkg: Add code coverage support for GCC


Hi Gua Guo,

I would propose that we make this easier for both the local user and the CI system to get coverage metrics.

Can you review this commit as it uses the same Edk2 plugins but adds code coverage support with GCC5:

Merged PR 5008: Initial implementation or code coverage on basecore u… · microsoft/mu_basecore@f6af51f (github.com)<https://github.com/microsoft/mu_basecore/commit/f6af51f856747faec5a9f78f5604a69211b5af2d#diff-52e599c27ab877ef616f244e8b26826a53853872acb894df4aeff99f48df373d>



Thanks

Sean




On 9/13/2022 6:01 PM, Guo, Gua wrote:

From: Gua Guo <gua.guo@intel.com><mailto:gua.guo@intel.com>



In order to collect code coverage after running executable

file, generate *.gcda and *.gcno file that require by lcov tool

to generate code coverage report.



Cc: Michael D Kinney <michael.d.kinney@intel.com><mailto:michael.d.kinney@intel.com>

Cc: Sean Brogan <sean.brogan@microsoft.com><mailto:sean.brogan@microsoft.com>

Cc: Bret Barkelew <Bret.Barkelew@microsoft.com><mailto:Bret.Barkelew@microsoft.com>

Cc: Michael Kubacki <mikuback@linux.microsoft.com><mailto:mikuback@linux.microsoft.com>

Signed-off-by: Gua Guo <gua.guo@intel.com><mailto:gua.guo@intel.com>

---

 UnitTestFrameworkPkg/ReadMe.md                        | 116 +++++++++++++++++++-

 UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc |   2 +-

 2 files changed, 116 insertions(+), 2 deletions(-)



diff --git a/UnitTestFrameworkPkg/ReadMe.md b/UnitTestFrameworkPkg/ReadMe.md

index e696412cb3..93fdacfc4a 100644

--- a/UnitTestFrameworkPkg/ReadMe.md

+++ b/UnitTestFrameworkPkg/ReadMe.md

@@ -361,7 +361,7 @@ RUNNING TEST SUITE: Int Safe Conversions Test Suite

 ```







 You can also, if you are so inclined, read the output from the exact instance of the test that was run during



-`stuart_ci_build`. The ouput file can be found on a path that looks like:



+`stuart_ci_build`. The output file can be found on a path that looks like:







 `Build/<Package>/HostTest/<Arch>/<TestName>.<TestSuiteName>.<Arch>.result.xml`







@@ -399,6 +399,120 @@ CMOCKA_XML_FILE=<absolute or relative path to output file>





 This mode is used by the test running plugin to aggregate the results for CI test status reporting in the web view.







+### Enable Code Coverage for GCC host-based tests



+



+LCOV is an extension of GCOV, a GNU tool which provides information about what parts of a program are actually executed (i.e. "covered") while running a particular test case. The extension consists of a set of scripts which build on the textual GCOV output to implement it.



+



+Documentation for LCOV can be found here:



+https://lcov.readthedocs.io/



+



+You can follow below steps to generate your code coverage report.



+



+



+



+```bash



+user@linux_machine:~/_uefi$stuart_ci_build -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5 -p MdePkg -t NOOPT



+...



+user@linux_machine:~/_uefi$lcov -c --rc lcov_branch_coverage=1 -o UnitTest.info -d Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/



+Capturing coverage data from Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/



+Found gcov version: 9.4.0



+Using intermediate gcov format



+Scanning Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/ for .gcda files ...



+Found 38 data files in Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/RShiftU64.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/Math64.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/SafeString.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/SwapBytes32.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/CpuDeadLoop.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/MultU64x64.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/LongJump.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/SetJump.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/BitField.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/String.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/DivU64x32Remainder.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/X64/GccInline.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/MultU64x32.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/MultS64x64.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/LinkedList.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/LShiftU64.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/DivU64x32.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/SwapBytes16.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/Unaligned.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/CheckSum.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/CopyMem.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/MemLibGuid.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/CompareMemWrapper.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/SetMemWrapper.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/SetMem64Wrapper.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/MemLibGeneric.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/CopyMemWrapper.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/SetMem.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/ZeroMemWrapper.gcda



+Processing Library/BasePrintLib/BasePrintLib/OUTPUT/PrintLibInternal.gcda



+Processing Library/BasePrintLib/BasePrintLib/OUTPUT/PrintLib.gcda



+Processing Library/BaseSafeIntLib/BaseSafeIntLib/OUTPUT/SafeIntLib64.gcda



+Processing Library/BaseSafeIntLib/BaseSafeIntLib/OUTPUT/SafeIntLib.gcda



+Processing Test/UnitTest/Library/BaseLib/BaseLibUnitTestsHost/OUTPUT/Base64UnitTest.gcda



+Processing Test/UnitTest/Library/BaseLib/BaseLibUnitTestsHost/OUTPUT/AutoGen.gcda



+Processing Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLibHost/OUTPUT/SafeIntLibUintnIntnUnitTests64.gcda



+Processing Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLibHost/OUTPUT/TestBaseSafeIntLib.gcda



+Processing Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLibHost/OUTPUT/AutoGen.gcda



+



+user@linux_machine:~/_uefi$genhtml UnitTest.info --branch-coverage -o Build/UnitTestReport



+



+Reading data file UnitTest.info



+Found 38 entries.



+Found common filename prefix "~/_uefi/MdePkg/Library"



+Writing .css and .png files.



+Generating output.



+Processing file ~/_uefi/Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/Test/UnitTest/Library/BaseLib/BaseLibUnitTestsHost/DEBUG/AutoGen.c



+Processing file ~/_uefi/Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLibHost/DEBUG/AutoGen.c



+Processing file BaseLib/Unaligned.c



+Processing file BaseLib/CheckSum.c



+Processing file BaseLib/BitField.c



+Processing file BaseLib/DivU64x32.c



+Processing file BaseLib/CpuDeadLoop.c



+Processing file BaseLib/LongJump.c



+Processing file BaseLib/MultU64x64.c



+Processing file BaseLib/RShiftU64.c



+Processing file BaseLib/SwapBytes32.c



+Processing file BaseLib/SwapBytes16.c



+Processing file BaseLib/SetJump.c



+Processing file BaseLib/LinkedList.c



+Processing file BaseLib/LShiftU64.c



+Processing file BaseLib/SafeString.c



+Processing file BaseLib/String.c



+Processing file BaseLib/DivU64x32Remainder.c



+Processing file BaseLib/MultS64x64.c



+Processing file BaseLib/MultU64x32.c



+Processing file BaseLib/Math64.c



+Processing file BaseLib/X64/GccInline.c



+Processing file BaseMemoryLib/SetMemWrapper.c



+Processing file BaseMemoryLib/CompareMemWrapper.c



+Processing file BaseMemoryLib/SetMem64Wrapper.c



+Processing file BaseMemoryLib/ZeroMemWrapper.c



+Processing file BaseMemoryLib/CopyMemWrapper.c



+Processing file BaseMemoryLib/CopyMem.c



+Processing file BaseMemoryLib/SetMem.c



+Processing file BaseMemoryLib/MemLibGuid.c



+Processing file BaseMemoryLib/MemLibGeneric.c



+Processing file BasePrintLib/PrintLib.c



+Processing file BasePrintLib/PrintLibInternal.c



+Processing file BaseSafeIntLib/SafeIntLib64.c



+Processing file BaseSafeIntLib/SafeIntLib.c



+Processing file ~/_uefi/MdePkg/Test/UnitTest/Library/BaseLib/Base64UnitTest.c



+Processing file ~/_uefi/MdePkg/Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLib.c



+Processing file ~/_uefi/MdePkg/Test/UnitTest/Library/BaseSafeIntLib/SafeIntLibUintnIntnUnitTests64.c



+Writing directory view page.



+Overall coverage rate:



+  lines......: 58.6% (2688 of 4589 lines)



+  functions..: 57.1% (249 of 436 functions)



+  branches...: 25.3% (1062 of 4195 branches)



+



+...



+



+```



+



 ### Important Note







 This works on both Windows and Linux, but is currently limited to x64 architectures. Working on getting others, but we



diff --git a/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc b/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc

index 4dd8d4ac67..8623aecb33 100644

--- a/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc

+++ b/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc

@@ -19,7 +19,7 @@

   MemoryAllocationLib|UnitTestFrameworkPkg/Library/Posix/MemoryAllocationLibPosix/MemoryAllocationLibPosix.inf







 [BuildOptions]



-  GCC:*_*_*_CC_FLAGS = -fno-pie



+  GCC:*_*_*_CC_FLAGS = -fno-pie -ftest-coverage -fprofile-arcs



 !ifdef $(UNIT_TESTING_DEBUG)



   MSFT:*_*_*_CC_FLAGS  = -D UNIT_TESTING_DEBUG=1



   GCC:*_*_*_CC_FLAGS   = -D UNIT_TESTING_DEBUG=1



[-- Attachment #2: Type: text/html, Size: 58822 bytes --]

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [edk2-devel] [PATCH v2] UnitTestFrameworkPkg: Add code coverage support for GCC
  2022-09-14  1:51   ` Michael D Kinney
@ 2022-09-14  2:10     ` Sean
  2022-09-14 16:25       ` Michael D Kinney
  2022-09-14  2:11     ` Guo, Gua
  1 sibling, 1 reply; 9+ messages in thread
From: Sean @ 2022-09-14  2:10 UTC (permalink / raw)
  To: Kinney, Michael D, devel@edk2.groups.io, Guo, Gua
  Cc: Sean Brogan, Michael Kubacki

[-- Attachment #1: Type: text/plain, Size: 12256 bytes --]

For local scenarios the plug-in generates the html report for viewing the c code with coverage annotations.  The intermediate files are available so I would think they could be loaded in all sorts of viewers but I haven’t tried. I have just used the html and it seemed acceptable.

For ci the pipeline files have been updated to publish the data to the build. This allows using the online viewer to navigate c code and coverage. The html report is also uploaded and can be downloaded for local viewing.

Regarding using visual studio tool chain/compilers we had discussed that for ci this is unnecessary ( since we already have coverage from gcc tool chain). For local users this is important. There is an easy tool you shared with me that gives similar results. I didn’t see a great way to integrate into the plug-in for auto invocation but I plan to write up usage details for local users.

Thanks
Sean
________________________________
From: Kinney, Michael D <michael.d.kinney@intel.com>
Sent: Tuesday, September 13, 2022 6:51:43 PM
To: Sean Brogan <spbrogan@outlook.com>; devel@edk2.groups.io <devel@edk2.groups.io>; Guo, Gua <gua.guo@intel.com>; Kinney, Michael D <michael.d.kinney@intel.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>; Michael Kubacki <mikuback@linux.microsoft.com>
Subject: RE: [edk2-devel] [PATCH v2] UnitTestFrameworkPkg: Add code coverage support for GCC


Does that pytool plugin support viewing the code coverage visually in C source files?



Either HTML or preferably an Visual Studio Code plugin?



Can it be updated to support VS compilers tool?



Mie



From: Sean Brogan <spbrogan@outlook.com>
Sent: Tuesday, September 13, 2022 6:18 PM
To: devel@edk2.groups.io; Guo, Gua <gua.guo@intel.com>
Cc: Kinney, Michael D <michael.d.kinney@intel.com>; Sean Brogan <sean.brogan@microsoft.com>; Michael Kubacki <mikuback@linux.microsoft.com>
Subject: Re: [edk2-devel] [PATCH v2] UnitTestFrameworkPkg: Add code coverage support for GCC



Hi Gua Guo,

I would propose that we make this easier for both the local user and the CI system to get coverage metrics.

Can you review this commit as it uses the same Edk2 plugins but adds code coverage support with GCC5:

Merged PR 5008: Initial implementation or code coverage on basecore u… · microsoft/mu_basecore@f6af51f (github.com)<https://github.com/microsoft/mu_basecore/commit/f6af51f856747faec5a9f78f5604a69211b5af2d#diff-52e599c27ab877ef616f244e8b26826a53853872acb894df4aeff99f48df373d>



Thanks

Sean





On 9/13/2022 6:01 PM, Guo, Gua wrote:

From: Gua Guo <gua.guo@intel.com><mailto:gua.guo@intel.com>



In order to collect code coverage after running executable

file, generate *.gcda and *.gcno file that require by lcov tool

to generate code coverage report.



Cc: Michael D Kinney <michael.d.kinney@intel.com><mailto:michael.d.kinney@intel.com>

Cc: Sean Brogan <sean.brogan@microsoft.com><mailto:sean.brogan@microsoft.com>

Cc: Bret Barkelew <Bret.Barkelew@microsoft.com><mailto:Bret.Barkelew@microsoft.com>

Cc: Michael Kubacki <mikuback@linux.microsoft.com><mailto:mikuback@linux.microsoft.com>

Signed-off-by: Gua Guo <gua.guo@intel.com><mailto:gua.guo@intel.com>

---

 UnitTestFrameworkPkg/ReadMe.md                        | 116 +++++++++++++++++++-

 UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc |   2 +-

 2 files changed, 116 insertions(+), 2 deletions(-)



diff --git a/UnitTestFrameworkPkg/ReadMe.md b/UnitTestFrameworkPkg/ReadMe.md

index e696412cb3..93fdacfc4a 100644

--- a/UnitTestFrameworkPkg/ReadMe.md

+++ b/UnitTestFrameworkPkg/ReadMe.md

@@ -361,7 +361,7 @@ RUNNING TEST SUITE: Int Safe Conversions Test Suite

 ```







 You can also, if you are so inclined, read the output from the exact instance of the test that was run during



-`stuart_ci_build`. The ouput file can be found on a path that looks like:



+`stuart_ci_build`. The output file can be found on a path that looks like:







 `Build/<Package>/HostTest/<Arch>/<TestName>.<TestSuiteName>.<Arch>.result.xml`







@@ -399,6 +399,120 @@ CMOCKA_XML_FILE=<absolute or relative path to output file>





 This mode is used by the test running plugin to aggregate the results for CI test status reporting in the web view.







+### Enable Code Coverage for GCC host-based tests



+



+LCOV is an extension of GCOV, a GNU tool which provides information about what parts of a program are actually executed (i.e. "covered") while running a particular test case. The extension consists of a set of scripts which build on the textual GCOV output to implement it.



+



+Documentation for LCOV can be found here:



+https://lcov.readthedocs.io/



+



+You can follow below steps to generate your code coverage report.



+



+



+



+```bash



+user@linux_machine:~/_uefi$stuart_ci_build -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5 -p MdePkg -t NOOPT



+...



+user@linux_machine:~/_uefi$lcov -c --rc lcov_branch_coverage=1 -o UnitTest.info -d Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/



+Capturing coverage data from Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/



+Found gcov version: 9.4.0



+Using intermediate gcov format



+Scanning Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/ for .gcda files ...



+Found 38 data files in Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/RShiftU64.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/Math64.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/SafeString.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/SwapBytes32.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/CpuDeadLoop.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/MultU64x64.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/LongJump.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/SetJump.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/BitField.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/String.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/DivU64x32Remainder.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/X64/GccInline.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/MultU64x32.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/MultS64x64.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/LinkedList.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/LShiftU64.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/DivU64x32.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/SwapBytes16.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/Unaligned.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/CheckSum.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/CopyMem.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/MemLibGuid.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/CompareMemWrapper.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/SetMemWrapper.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/SetMem64Wrapper.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/MemLibGeneric.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/CopyMemWrapper.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/SetMem.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/ZeroMemWrapper.gcda



+Processing Library/BasePrintLib/BasePrintLib/OUTPUT/PrintLibInternal.gcda



+Processing Library/BasePrintLib/BasePrintLib/OUTPUT/PrintLib.gcda



+Processing Library/BaseSafeIntLib/BaseSafeIntLib/OUTPUT/SafeIntLib64.gcda



+Processing Library/BaseSafeIntLib/BaseSafeIntLib/OUTPUT/SafeIntLib.gcda



+Processing Test/UnitTest/Library/BaseLib/BaseLibUnitTestsHost/OUTPUT/Base64UnitTest.gcda



+Processing Test/UnitTest/Library/BaseLib/BaseLibUnitTestsHost/OUTPUT/AutoGen.gcda



+Processing Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLibHost/OUTPUT/SafeIntLibUintnIntnUnitTests64.gcda



+Processing Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLibHost/OUTPUT/TestBaseSafeIntLib.gcda



+Processing Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLibHost/OUTPUT/AutoGen.gcda



+



+user@linux_machine:~/_uefi$genhtml UnitTest.info --branch-coverage -o Build/UnitTestReport



+



+Reading data file UnitTest.info



+Found 38 entries.



+Found common filename prefix "~/_uefi/MdePkg/Library"



+Writing .css and .png files.



+Generating output.



+Processing file ~/_uefi/Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/Test/UnitTest/Library/BaseLib/BaseLibUnitTestsHost/DEBUG/AutoGen.c



+Processing file ~/_uefi/Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLibHost/DEBUG/AutoGen.c



+Processing file BaseLib/Unaligned.c



+Processing file BaseLib/CheckSum.c



+Processing file BaseLib/BitField.c



+Processing file BaseLib/DivU64x32.c



+Processing file BaseLib/CpuDeadLoop.c



+Processing file BaseLib/LongJump.c



+Processing file BaseLib/MultU64x64.c



+Processing file BaseLib/RShiftU64.c



+Processing file BaseLib/SwapBytes32.c



+Processing file BaseLib/SwapBytes16.c



+Processing file BaseLib/SetJump.c



+Processing file BaseLib/LinkedList.c



+Processing file BaseLib/LShiftU64.c



+Processing file BaseLib/SafeString.c



+Processing file BaseLib/String.c



+Processing file BaseLib/DivU64x32Remainder.c



+Processing file BaseLib/MultS64x64.c



+Processing file BaseLib/MultU64x32.c



+Processing file BaseLib/Math64.c



+Processing file BaseLib/X64/GccInline.c



+Processing file BaseMemoryLib/SetMemWrapper.c



+Processing file BaseMemoryLib/CompareMemWrapper.c



+Processing file BaseMemoryLib/SetMem64Wrapper.c



+Processing file BaseMemoryLib/ZeroMemWrapper.c



+Processing file BaseMemoryLib/CopyMemWrapper.c



+Processing file BaseMemoryLib/CopyMem.c



+Processing file BaseMemoryLib/SetMem.c



+Processing file BaseMemoryLib/MemLibGuid.c



+Processing file BaseMemoryLib/MemLibGeneric.c



+Processing file BasePrintLib/PrintLib.c



+Processing file BasePrintLib/PrintLibInternal.c



+Processing file BaseSafeIntLib/SafeIntLib64.c



+Processing file BaseSafeIntLib/SafeIntLib.c



+Processing file ~/_uefi/MdePkg/Test/UnitTest/Library/BaseLib/Base64UnitTest.c



+Processing file ~/_uefi/MdePkg/Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLib.c



+Processing file ~/_uefi/MdePkg/Test/UnitTest/Library/BaseSafeIntLib/SafeIntLibUintnIntnUnitTests64.c



+Writing directory view page.



+Overall coverage rate:



+  lines......: 58.6% (2688 of 4589 lines)



+  functions..: 57.1% (249 of 436 functions)



+  branches...: 25.3% (1062 of 4195 branches)



+



+...



+



+```



+



 ### Important Note







 This works on both Windows and Linux, but is currently limited to x64 architectures. Working on getting others, but we



diff --git a/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc b/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc

index 4dd8d4ac67..8623aecb33 100644

--- a/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc

+++ b/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc

@@ -19,7 +19,7 @@

   MemoryAllocationLib|UnitTestFrameworkPkg/Library/Posix/MemoryAllocationLibPosix/MemoryAllocationLibPosix.inf







 [BuildOptions]



-  GCC:*_*_*_CC_FLAGS = -fno-pie



+  GCC:*_*_*_CC_FLAGS = -fno-pie -ftest-coverage -fprofile-arcs



 !ifdef $(UNIT_TESTING_DEBUG)



   MSFT:*_*_*_CC_FLAGS  = -D UNIT_TESTING_DEBUG=1



   GCC:*_*_*_CC_FLAGS   = -D UNIT_TESTING_DEBUG=1



[-- Attachment #2: Type: text/html, Size: 19198 bytes --]

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [edk2-devel] [PATCH v2] UnitTestFrameworkPkg: Add code coverage support for GCC
  2022-09-14  1:51   ` Michael D Kinney
  2022-09-14  2:10     ` Sean
@ 2022-09-14  2:11     ` Guo, Gua
  2022-09-14 16:28       ` Michael D Kinney
  2022-09-14 16:34       ` Michael D Kinney
  1 sibling, 2 replies; 9+ messages in thread
From: Guo, Gua @ 2022-09-14  2:11 UTC (permalink / raw)
  To: Kinney, Michael D, Sean Brogan, devel@edk2.groups.io
  Cc: Sean Brogan, Michael Kubacki

[-- Attachment #1: Type: text/plain, Size: 13324 bytes --]

@Sean Brogan<mailto:spbrogan@outlook.com>

I go through the code Merged PR 5008: Initial implementation or code coverage on basecore u… · microsoft/mu_basecore@f6af51f (github.com)<https://github.com/microsoft/mu_basecore/commit/f6af51f856747faec5a9f78f5604a69211b5af2d#diff-52e599c27ab877ef616f244e8b26826a53853872acb894df4aeff99f48df373d>
It looks good from my view point.

@Kinney, Michael D<mailto:michael.d.kinney@intel.com>

Does that pytool plugin support viewing the code coverage visually in C source files? Yes

Either HTML or preferably an Visual Studio Code plugin? I think it can be supported by both based on below change. Due to VScode plugin is auto scan *.gcda and show the result in VScode source tree.
Merged PR 5008: Initial implementation or code coverage on basecore u… · microsoft/mu_basecore@f6af51f (github.com)<https://github.com/microsoft/mu_basecore/commit/f6af51f856747faec5a9f78f5604a69211b5af2d#diff-52e599c27ab877ef616f244e8b26826a53853872acb894df4aeff99f48df373d>

Flow for stuart_ci_build: (a) -> (b) -> (c) -> (d) -> (e) -> (f)
Flow for vscode extension Gcov Viewer - Visual Studio Marketplace<https://marketplace.visualstudio.com/items?itemName=JacquesLucke.gcov-viewer>:  (a) -> (b) -> (c) -> (d) -> (g)

(a) stuart_ci_build -p XXXPkg
(b) Call Edk2/.pytool/Plugin/HostUnitTestCompilerPlugin/HostUnitTestCompilerPlugin.py
(c) Call Edk2/BaseTools/Plugin/HostBasedUnitTestRunner/HostBasedUnitTestRunner.py
(d)       execute Build/XXXPkg/**/*Test* -> generate *.gcda files
(e)       use “lcov” tool to parse *.gcda
(f)        use “pycobertura” tool to generate index.html

(g) vscode extension to parse *.gcda and show it on source code directly.

Can it be updated to support VS compilers tool? No, currently I don’t find a good way to generate code coverage for MSVC compiler

Thanks,
Gua
From: Kinney, Michael D <michael.d.kinney@intel.com>
Sent: Wednesday, September 14, 2022 9:52 AM
To: Sean Brogan <spbrogan@outlook.com>; devel@edk2.groups.io; Guo, Gua <gua.guo@intel.com>; Kinney, Michael D <michael.d.kinney@intel.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>; Michael Kubacki <mikuback@linux.microsoft.com>
Subject: RE: [edk2-devel] [PATCH v2] UnitTestFrameworkPkg: Add code coverage support for GCC

Does that pytool plugin support viewing the code coverage visually in C source files?

Either HTML or preferably an Visual Studio Code plugin?

Can it be updated to support VS compilers tool?

Mie

From: Sean Brogan <spbrogan@outlook.com<mailto:spbrogan@outlook.com>>
Sent: Tuesday, September 13, 2022 6:18 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Guo, Gua <gua.guo@intel.com<mailto:gua.guo@intel.com>>
Cc: Kinney, Michael D <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>; Sean Brogan <sean.brogan@microsoft.com<mailto:sean.brogan@microsoft.com>>; Michael Kubacki <mikuback@linux.microsoft.com<mailto:mikuback@linux.microsoft.com>>
Subject: Re: [edk2-devel] [PATCH v2] UnitTestFrameworkPkg: Add code coverage support for GCC


Hi Gua Guo,

I would propose that we make this easier for both the local user and the CI system to get coverage metrics.

Can you review this commit as it uses the same Edk2 plugins but adds code coverage support with GCC5:

Merged PR 5008: Initial implementation or code coverage on basecore u… · microsoft/mu_basecore@f6af51f (github.com)<https://github.com/microsoft/mu_basecore/commit/f6af51f856747faec5a9f78f5604a69211b5af2d#diff-52e599c27ab877ef616f244e8b26826a53853872acb894df4aeff99f48df373d>



Thanks

Sean




On 9/13/2022 6:01 PM, Guo, Gua wrote:

From: Gua Guo <gua.guo@intel.com><mailto:gua.guo@intel.com>



In order to collect code coverage after running executable

file, generate *.gcda and *.gcno file that require by lcov tool

to generate code coverage report.



Cc: Michael D Kinney <michael.d.kinney@intel.com><mailto:michael.d.kinney@intel.com>

Cc: Sean Brogan <sean.brogan@microsoft.com><mailto:sean.brogan@microsoft.com>

Cc: Bret Barkelew <Bret.Barkelew@microsoft.com><mailto:Bret.Barkelew@microsoft.com>

Cc: Michael Kubacki <mikuback@linux.microsoft.com><mailto:mikuback@linux.microsoft.com>

Signed-off-by: Gua Guo <gua.guo@intel.com><mailto:gua.guo@intel.com>

---

 UnitTestFrameworkPkg/ReadMe.md                        | 116 +++++++++++++++++++-

 UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc |   2 +-

 2 files changed, 116 insertions(+), 2 deletions(-)



diff --git a/UnitTestFrameworkPkg/ReadMe.md b/UnitTestFrameworkPkg/ReadMe.md

index e696412cb3..93fdacfc4a 100644

--- a/UnitTestFrameworkPkg/ReadMe.md

+++ b/UnitTestFrameworkPkg/ReadMe.md

@@ -361,7 +361,7 @@ RUNNING TEST SUITE: Int Safe Conversions Test Suite

 ```







 You can also, if you are so inclined, read the output from the exact instance of the test that was run during



-`stuart_ci_build`. The ouput file can be found on a path that looks like:



+`stuart_ci_build`. The output file can be found on a path that looks like:







 `Build/<Package>/HostTest/<Arch>/<TestName>.<TestSuiteName>.<Arch>.result.xml`







@@ -399,6 +399,120 @@ CMOCKA_XML_FILE=<absolute or relative path to output file>





 This mode is used by the test running plugin to aggregate the results for CI test status reporting in the web view.







+### Enable Code Coverage for GCC host-based tests



+



+LCOV is an extension of GCOV, a GNU tool which provides information about what parts of a program are actually executed (i.e. "covered") while running a particular test case. The extension consists of a set of scripts which build on the textual GCOV output to implement it.



+



+Documentation for LCOV can be found here:



+https://lcov.readthedocs.io/



+



+You can follow below steps to generate your code coverage report.



+



+



+



+```bash



+user@linux_machine:~/_uefi$stuart_ci_build -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5 -p MdePkg -t NOOPT



+...



+user@linux_machine:~/_uefi$lcov -c --rc lcov_branch_coverage=1 -o UnitTest.info -d Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/



+Capturing coverage data from Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/



+Found gcov version: 9.4.0



+Using intermediate gcov format



+Scanning Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/ for .gcda files ...



+Found 38 data files in Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/RShiftU64.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/Math64.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/SafeString.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/SwapBytes32.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/CpuDeadLoop.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/MultU64x64.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/LongJump.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/SetJump.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/BitField.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/String.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/DivU64x32Remainder.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/X64/GccInline.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/MultU64x32.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/MultS64x64.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/LinkedList.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/LShiftU64.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/DivU64x32.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/SwapBytes16.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/Unaligned.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/CheckSum.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/CopyMem.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/MemLibGuid.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/CompareMemWrapper.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/SetMemWrapper.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/SetMem64Wrapper.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/MemLibGeneric.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/CopyMemWrapper.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/SetMem.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/ZeroMemWrapper.gcda



+Processing Library/BasePrintLib/BasePrintLib/OUTPUT/PrintLibInternal.gcda



+Processing Library/BasePrintLib/BasePrintLib/OUTPUT/PrintLib.gcda



+Processing Library/BaseSafeIntLib/BaseSafeIntLib/OUTPUT/SafeIntLib64.gcda



+Processing Library/BaseSafeIntLib/BaseSafeIntLib/OUTPUT/SafeIntLib.gcda



+Processing Test/UnitTest/Library/BaseLib/BaseLibUnitTestsHost/OUTPUT/Base64UnitTest.gcda



+Processing Test/UnitTest/Library/BaseLib/BaseLibUnitTestsHost/OUTPUT/AutoGen.gcda



+Processing Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLibHost/OUTPUT/SafeIntLibUintnIntnUnitTests64.gcda



+Processing Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLibHost/OUTPUT/TestBaseSafeIntLib.gcda



+Processing Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLibHost/OUTPUT/AutoGen.gcda



+



+user@linux_machine:~/_uefi$genhtml UnitTest.info --branch-coverage -o Build/UnitTestReport



+



+Reading data file UnitTest.info



+Found 38 entries.



+Found common filename prefix "~/_uefi/MdePkg/Library"



+Writing .css and .png files.



+Generating output.



+Processing file ~/_uefi/Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/Test/UnitTest/Library/BaseLib/BaseLibUnitTestsHost/DEBUG/AutoGen.c



+Processing file ~/_uefi/Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLibHost/DEBUG/AutoGen.c



+Processing file BaseLib/Unaligned.c



+Processing file BaseLib/CheckSum.c



+Processing file BaseLib/BitField.c



+Processing file BaseLib/DivU64x32.c



+Processing file BaseLib/CpuDeadLoop.c



+Processing file BaseLib/LongJump.c



+Processing file BaseLib/MultU64x64.c



+Processing file BaseLib/RShiftU64.c



+Processing file BaseLib/SwapBytes32.c



+Processing file BaseLib/SwapBytes16.c



+Processing file BaseLib/SetJump.c



+Processing file BaseLib/LinkedList.c



+Processing file BaseLib/LShiftU64.c



+Processing file BaseLib/SafeString.c



+Processing file BaseLib/String.c



+Processing file BaseLib/DivU64x32Remainder.c



+Processing file BaseLib/MultS64x64.c



+Processing file BaseLib/MultU64x32.c



+Processing file BaseLib/Math64.c



+Processing file BaseLib/X64/GccInline.c



+Processing file BaseMemoryLib/SetMemWrapper.c



+Processing file BaseMemoryLib/CompareMemWrapper.c



+Processing file BaseMemoryLib/SetMem64Wrapper.c



+Processing file BaseMemoryLib/ZeroMemWrapper.c



+Processing file BaseMemoryLib/CopyMemWrapper.c



+Processing file BaseMemoryLib/CopyMem.c



+Processing file BaseMemoryLib/SetMem.c



+Processing file BaseMemoryLib/MemLibGuid.c



+Processing file BaseMemoryLib/MemLibGeneric.c



+Processing file BasePrintLib/PrintLib.c



+Processing file BasePrintLib/PrintLibInternal.c



+Processing file BaseSafeIntLib/SafeIntLib64.c



+Processing file BaseSafeIntLib/SafeIntLib.c



+Processing file ~/_uefi/MdePkg/Test/UnitTest/Library/BaseLib/Base64UnitTest.c



+Processing file ~/_uefi/MdePkg/Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLib.c



+Processing file ~/_uefi/MdePkg/Test/UnitTest/Library/BaseSafeIntLib/SafeIntLibUintnIntnUnitTests64.c



+Writing directory view page.



+Overall coverage rate:



+  lines......: 58.6% (2688 of 4589 lines)



+  functions..: 57.1% (249 of 436 functions)



+  branches...: 25.3% (1062 of 4195 branches)



+



+...



+



+```



+



 ### Important Note







 This works on both Windows and Linux, but is currently limited to x64 architectures. Working on getting others, but we



diff --git a/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc b/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc

index 4dd8d4ac67..8623aecb33 100644

--- a/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc

+++ b/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc

@@ -19,7 +19,7 @@

   MemoryAllocationLib|UnitTestFrameworkPkg/Library/Posix/MemoryAllocationLibPosix/MemoryAllocationLibPosix.inf







 [BuildOptions]



-  GCC:*_*_*_CC_FLAGS = -fno-pie



+  GCC:*_*_*_CC_FLAGS = -fno-pie -ftest-coverage -fprofile-arcs



 !ifdef $(UNIT_TESTING_DEBUG)



   MSFT:*_*_*_CC_FLAGS  = -D UNIT_TESTING_DEBUG=1



   GCC:*_*_*_CC_FLAGS   = -D UNIT_TESTING_DEBUG=1



[-- Attachment #2: Type: text/html, Size: 26097 bytes --]

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [edk2-devel] [PATCH v2] UnitTestFrameworkPkg: Add code coverage support for GCC
  2022-09-14  2:10     ` Sean
@ 2022-09-14 16:25       ` Michael D Kinney
  0 siblings, 0 replies; 9+ messages in thread
From: Michael D Kinney @ 2022-09-14 16:25 UTC (permalink / raw)
  To: Sean Brogan, devel@edk2.groups.io, Guo, Gua, Kinney, Michael D
  Cc: Sean Brogan, Michael Kubacki

[-- Attachment #1: Type: text/plain, Size: 13488 bytes --]

Does that mean that the patch under discussion here is still valuable for local users?

I want to make sure it is easy for local users to see the coverage as unit tests are implemented locally for Windows and Linux OSes.

The ci plugin you have looks valuable for CI testing to provide feedback to the entire community on coverage changes after an initial set of unit tests are in place.

Mike

From: Sean Brogan <spbrogan@outlook.com>
Sent: Tuesday, September 13, 2022 7:10 PM
To: Kinney, Michael D <michael.d.kinney@intel.com>; devel@edk2.groups.io; Guo, Gua <gua.guo@intel.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>; Michael Kubacki <mikuback@linux.microsoft.com>
Subject: Re: [edk2-devel] [PATCH v2] UnitTestFrameworkPkg: Add code coverage support for GCC

For local scenarios the plug-in generates the html report for viewing the c code with coverage annotations.  The intermediate files are available so I would think they could be loaded in all sorts of viewers but I haven’t tried. I have just used the html and it seemed acceptable.

For ci the pipeline files have been updated to publish the data to the build. This allows using the online viewer to navigate c code and coverage. The html report is also uploaded and can be downloaded for local viewing.

Regarding using visual studio tool chain/compilers we had discussed that for ci this is unnecessary ( since we already have coverage from gcc tool chain). For local users this is important. There is an easy tool you shared with me that gives similar results. I didn’t see a great way to integrate into the plug-in for auto invocation but I plan to write up usage details for local users.

Thanks
Sean
________________________________
From: Kinney, Michael D <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>
Sent: Tuesday, September 13, 2022 6:51:43 PM
To: Sean Brogan <spbrogan@outlook.com<mailto:spbrogan@outlook.com>>; devel@edk2.groups.io<mailto:devel@edk2.groups.io> <devel@edk2.groups.io<mailto:devel@edk2.groups.io>>; Guo, Gua <gua.guo@intel.com<mailto:gua.guo@intel.com>>; Kinney, Michael D <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>
Cc: Sean Brogan <sean.brogan@microsoft.com<mailto:sean.brogan@microsoft.com>>; Michael Kubacki <mikuback@linux.microsoft.com<mailto:mikuback@linux.microsoft.com>>
Subject: RE: [edk2-devel] [PATCH v2] UnitTestFrameworkPkg: Add code coverage support for GCC


Does that pytool plugin support viewing the code coverage visually in C source files?



Either HTML or preferably an Visual Studio Code plugin?



Can it be updated to support VS compilers tool?



Mie



From: Sean Brogan <spbrogan@outlook.com<mailto:spbrogan@outlook.com>>
Sent: Tuesday, September 13, 2022 6:18 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Guo, Gua <gua.guo@intel.com<mailto:gua.guo@intel.com>>
Cc: Kinney, Michael D <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>; Sean Brogan <sean.brogan@microsoft.com<mailto:sean.brogan@microsoft.com>>; Michael Kubacki <mikuback@linux.microsoft.com<mailto:mikuback@linux.microsoft.com>>
Subject: Re: [edk2-devel] [PATCH v2] UnitTestFrameworkPkg: Add code coverage support for GCC



Hi Gua Guo,

I would propose that we make this easier for both the local user and the CI system to get coverage metrics.

Can you review this commit as it uses the same Edk2 plugins but adds code coverage support with GCC5:

Merged PR 5008: Initial implementation or code coverage on basecore u… · microsoft/mu_basecore@f6af51f (github.com)<https://github.com/microsoft/mu_basecore/commit/f6af51f856747faec5a9f78f5604a69211b5af2d#diff-52e599c27ab877ef616f244e8b26826a53853872acb894df4aeff99f48df373d>



Thanks

Sean





On 9/13/2022 6:01 PM, Guo, Gua wrote:

From: Gua Guo <gua.guo@intel.com><mailto:gua.guo@intel.com>



In order to collect code coverage after running executable

file, generate *.gcda and *.gcno file that require by lcov tool

to generate code coverage report.



Cc: Michael D Kinney <michael.d.kinney@intel.com><mailto:michael.d.kinney@intel.com>

Cc: Sean Brogan <sean.brogan@microsoft.com><mailto:sean.brogan@microsoft.com>

Cc: Bret Barkelew <Bret.Barkelew@microsoft.com><mailto:Bret.Barkelew@microsoft.com>

Cc: Michael Kubacki <mikuback@linux.microsoft.com><mailto:mikuback@linux.microsoft.com>

Signed-off-by: Gua Guo <gua.guo@intel.com><mailto:gua.guo@intel.com>

---

 UnitTestFrameworkPkg/ReadMe.md                        | 116 +++++++++++++++++++-

 UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc |   2 +-

 2 files changed, 116 insertions(+), 2 deletions(-)



diff --git a/UnitTestFrameworkPkg/ReadMe.md b/UnitTestFrameworkPkg/ReadMe.md

index e696412cb3..93fdacfc4a 100644

--- a/UnitTestFrameworkPkg/ReadMe.md

+++ b/UnitTestFrameworkPkg/ReadMe.md

@@ -361,7 +361,7 @@ RUNNING TEST SUITE: Int Safe Conversions Test Suite

 ```







 You can also, if you are so inclined, read the output from the exact instance of the test that was run during



-`stuart_ci_build`. The ouput file can be found on a path that looks like:



+`stuart_ci_build`. The output file can be found on a path that looks like:







 `Build/<Package>/HostTest/<Arch>/<TestName>.<TestSuiteName>.<Arch>.result.xml`







@@ -399,6 +399,120 @@ CMOCKA_XML_FILE=<absolute or relative path to output file>





 This mode is used by the test running plugin to aggregate the results for CI test status reporting in the web view.







+### Enable Code Coverage for GCC host-based tests



+



+LCOV is an extension of GCOV, a GNU tool which provides information about what parts of a program are actually executed (i.e. "covered") while running a particular test case. The extension consists of a set of scripts which build on the textual GCOV output to implement it.



+



+Documentation for LCOV can be found here:



+https://lcov.readthedocs.io/



+



+You can follow below steps to generate your code coverage report.



+



+



+



+```bash



+user@linux_machine:~/_uefi$stuart_ci_build -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5 -p MdePkg -t NOOPT



+...



+user@linux_machine:~/_uefi$lcov -c --rc lcov_branch_coverage=1 -o UnitTest.info -d Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/



+Capturing coverage data from Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/



+Found gcov version: 9.4.0



+Using intermediate gcov format



+Scanning Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/ for .gcda files ...



+Found 38 data files in Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/RShiftU64.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/Math64.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/SafeString.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/SwapBytes32.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/CpuDeadLoop.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/MultU64x64.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/LongJump.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/SetJump.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/BitField.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/String.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/DivU64x32Remainder.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/X64/GccInline.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/MultU64x32.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/MultS64x64.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/LinkedList.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/LShiftU64.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/DivU64x32.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/SwapBytes16.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/Unaligned.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/CheckSum.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/CopyMem.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/MemLibGuid.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/CompareMemWrapper.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/SetMemWrapper.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/SetMem64Wrapper.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/MemLibGeneric.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/CopyMemWrapper.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/SetMem.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/ZeroMemWrapper.gcda



+Processing Library/BasePrintLib/BasePrintLib/OUTPUT/PrintLibInternal.gcda



+Processing Library/BasePrintLib/BasePrintLib/OUTPUT/PrintLib.gcda



+Processing Library/BaseSafeIntLib/BaseSafeIntLib/OUTPUT/SafeIntLib64.gcda



+Processing Library/BaseSafeIntLib/BaseSafeIntLib/OUTPUT/SafeIntLib.gcda



+Processing Test/UnitTest/Library/BaseLib/BaseLibUnitTestsHost/OUTPUT/Base64UnitTest.gcda



+Processing Test/UnitTest/Library/BaseLib/BaseLibUnitTestsHost/OUTPUT/AutoGen.gcda



+Processing Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLibHost/OUTPUT/SafeIntLibUintnIntnUnitTests64.gcda



+Processing Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLibHost/OUTPUT/TestBaseSafeIntLib.gcda



+Processing Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLibHost/OUTPUT/AutoGen.gcda



+



+user@linux_machine:~/_uefi$genhtml UnitTest.info --branch-coverage -o Build/UnitTestReport



+



+Reading data file UnitTest.info



+Found 38 entries.



+Found common filename prefix "~/_uefi/MdePkg/Library"



+Writing .css and .png files.



+Generating output.



+Processing file ~/_uefi/Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/Test/UnitTest/Library/BaseLib/BaseLibUnitTestsHost/DEBUG/AutoGen.c



+Processing file ~/_uefi/Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLibHost/DEBUG/AutoGen.c



+Processing file BaseLib/Unaligned.c



+Processing file BaseLib/CheckSum.c



+Processing file BaseLib/BitField.c



+Processing file BaseLib/DivU64x32.c



+Processing file BaseLib/CpuDeadLoop.c



+Processing file BaseLib/LongJump.c



+Processing file BaseLib/MultU64x64.c



+Processing file BaseLib/RShiftU64.c



+Processing file BaseLib/SwapBytes32.c



+Processing file BaseLib/SwapBytes16.c



+Processing file BaseLib/SetJump.c



+Processing file BaseLib/LinkedList.c



+Processing file BaseLib/LShiftU64.c



+Processing file BaseLib/SafeString.c



+Processing file BaseLib/String.c



+Processing file BaseLib/DivU64x32Remainder.c



+Processing file BaseLib/MultS64x64.c



+Processing file BaseLib/MultU64x32.c



+Processing file BaseLib/Math64.c



+Processing file BaseLib/X64/GccInline.c



+Processing file BaseMemoryLib/SetMemWrapper.c



+Processing file BaseMemoryLib/CompareMemWrapper.c



+Processing file BaseMemoryLib/SetMem64Wrapper.c



+Processing file BaseMemoryLib/ZeroMemWrapper.c



+Processing file BaseMemoryLib/CopyMemWrapper.c



+Processing file BaseMemoryLib/CopyMem.c



+Processing file BaseMemoryLib/SetMem.c



+Processing file BaseMemoryLib/MemLibGuid.c



+Processing file BaseMemoryLib/MemLibGeneric.c



+Processing file BasePrintLib/PrintLib.c



+Processing file BasePrintLib/PrintLibInternal.c



+Processing file BaseSafeIntLib/SafeIntLib64.c



+Processing file BaseSafeIntLib/SafeIntLib.c



+Processing file ~/_uefi/MdePkg/Test/UnitTest/Library/BaseLib/Base64UnitTest.c



+Processing file ~/_uefi/MdePkg/Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLib.c



+Processing file ~/_uefi/MdePkg/Test/UnitTest/Library/BaseSafeIntLib/SafeIntLibUintnIntnUnitTests64.c



+Writing directory view page.



+Overall coverage rate:



+  lines......: 58.6% (2688 of 4589 lines)



+  functions..: 57.1% (249 of 436 functions)



+  branches...: 25.3% (1062 of 4195 branches)



+



+...



+



+```



+



 ### Important Note







 This works on both Windows and Linux, but is currently limited to x64 architectures. Working on getting others, but we



diff --git a/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc b/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc

index 4dd8d4ac67..8623aecb33 100644

--- a/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc

+++ b/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc

@@ -19,7 +19,7 @@

   MemoryAllocationLib|UnitTestFrameworkPkg/Library/Posix/MemoryAllocationLibPosix/MemoryAllocationLibPosix.inf







 [BuildOptions]



-  GCC:*_*_*_CC_FLAGS = -fno-pie



+  GCC:*_*_*_CC_FLAGS = -fno-pie -ftest-coverage -fprofile-arcs



 !ifdef $(UNIT_TESTING_DEBUG)



   MSFT:*_*_*_CC_FLAGS  = -D UNIT_TESTING_DEBUG=1



   GCC:*_*_*_CC_FLAGS   = -D UNIT_TESTING_DEBUG=1



[-- Attachment #2: Type: text/html, Size: 65552 bytes --]

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [edk2-devel] [PATCH v2] UnitTestFrameworkPkg: Add code coverage support for GCC
  2022-09-14  2:11     ` Guo, Gua
@ 2022-09-14 16:28       ` Michael D Kinney
  2022-09-14 16:34       ` Michael D Kinney
  1 sibling, 0 replies; 9+ messages in thread
From: Michael D Kinney @ 2022-09-14 16:28 UTC (permalink / raw)
  To: Guo, Gua, Sean Brogan, devel@edk2.groups.io, Kinney, Michael D
  Cc: Sean Brogan, Michael Kubacki

[-- Attachment #1: Type: text/plain, Size: 14185 bytes --]

For VS builds, I have heard that OpenCppCoverage is a good option:

https://github.com/OpenCppCoverage/OpenCppCoverage

I think Sean has done some initial evaluation.  Would be great to see this enabled for local users for VS20xx tool chains.

Mike


From: Guo, Gua <gua.guo@intel.com>
Sent: Tuesday, September 13, 2022 7:11 PM
To: Kinney, Michael D <michael.d.kinney@intel.com>; Sean Brogan <spbrogan@outlook.com>; devel@edk2.groups.io
Cc: Sean Brogan <sean.brogan@microsoft.com>; Michael Kubacki <mikuback@linux.microsoft.com>
Subject: RE: [edk2-devel] [PATCH v2] UnitTestFrameworkPkg: Add code coverage support for GCC

@Sean Brogan<mailto:spbrogan@outlook.com>

I go through the code Merged PR 5008: Initial implementation or code coverage on basecore u… · microsoft/mu_basecore@f6af51f (github.com)<https://github.com/microsoft/mu_basecore/commit/f6af51f856747faec5a9f78f5604a69211b5af2d#diff-52e599c27ab877ef616f244e8b26826a53853872acb894df4aeff99f48df373d>
It looks good from my view point.

@Kinney, Michael D<mailto:michael.d.kinney@intel.com>

Does that pytool plugin support viewing the code coverage visually in C source files? Yes

Either HTML or preferably an Visual Studio Code plugin? I think it can be supported by both based on below change. Due to VScode plugin is auto scan *.gcda and show the result in VScode source tree.
Merged PR 5008: Initial implementation or code coverage on basecore u… · microsoft/mu_basecore@f6af51f (github.com)<https://github.com/microsoft/mu_basecore/commit/f6af51f856747faec5a9f78f5604a69211b5af2d#diff-52e599c27ab877ef616f244e8b26826a53853872acb894df4aeff99f48df373d>

Flow for stuart_ci_build: (a) -> (b) -> (c) -> (d) -> (e) -> (f)
Flow for vscode extension Gcov Viewer - Visual Studio Marketplace<https://marketplace.visualstudio.com/items?itemName=JacquesLucke.gcov-viewer>:  (a) -> (b) -> (c) -> (d) -> (g)

(a) stuart_ci_build -p XXXPkg
(b) Call Edk2/.pytool/Plugin/HostUnitTestCompilerPlugin/HostUnitTestCompilerPlugin.py
(c) Call Edk2/BaseTools/Plugin/HostBasedUnitTestRunner/HostBasedUnitTestRunner.py
(d)       execute Build/XXXPkg/**/*Test* -> generate *.gcda files
(e)       use “lcov” tool to parse *.gcda
(f)        use “pycobertura” tool to generate index.html
(g) vscode extension to parse *.gcda and show it on source code directly.

Can it be updated to support VS compilers tool? No, currently I don’t find a good way to generate code coverage for MSVC compiler

Thanks,
Gua
From: Kinney, Michael D <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>
Sent: Wednesday, September 14, 2022 9:52 AM
To: Sean Brogan <spbrogan@outlook.com<mailto:spbrogan@outlook.com>>; devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Guo, Gua <gua.guo@intel.com<mailto:gua.guo@intel.com>>; Kinney, Michael D <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>
Cc: Sean Brogan <sean.brogan@microsoft.com<mailto:sean.brogan@microsoft.com>>; Michael Kubacki <mikuback@linux.microsoft.com<mailto:mikuback@linux.microsoft.com>>
Subject: RE: [edk2-devel] [PATCH v2] UnitTestFrameworkPkg: Add code coverage support for GCC

Does that pytool plugin support viewing the code coverage visually in C source files?

Either HTML or preferably an Visual Studio Code plugin?

Can it be updated to support VS compilers tool?

Mie

From: Sean Brogan <spbrogan@outlook.com<mailto:spbrogan@outlook.com>>
Sent: Tuesday, September 13, 2022 6:18 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Guo, Gua <gua.guo@intel.com<mailto:gua.guo@intel.com>>
Cc: Kinney, Michael D <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>; Sean Brogan <sean.brogan@microsoft.com<mailto:sean.brogan@microsoft.com>>; Michael Kubacki <mikuback@linux.microsoft.com<mailto:mikuback@linux.microsoft.com>>
Subject: Re: [edk2-devel] [PATCH v2] UnitTestFrameworkPkg: Add code coverage support for GCC


Hi Gua Guo,

I would propose that we make this easier for both the local user and the CI system to get coverage metrics.

Can you review this commit as it uses the same Edk2 plugins but adds code coverage support with GCC5:

Merged PR 5008: Initial implementation or code coverage on basecore u… · microsoft/mu_basecore@f6af51f (github.com)<https://github.com/microsoft/mu_basecore/commit/f6af51f856747faec5a9f78f5604a69211b5af2d#diff-52e599c27ab877ef616f244e8b26826a53853872acb894df4aeff99f48df373d>



Thanks

Sean




On 9/13/2022 6:01 PM, Guo, Gua wrote:

From: Gua Guo <gua.guo@intel.com><mailto:gua.guo@intel.com>



In order to collect code coverage after running executable

file, generate *.gcda and *.gcno file that require by lcov tool

to generate code coverage report.



Cc: Michael D Kinney <michael.d.kinney@intel.com><mailto:michael.d.kinney@intel.com>

Cc: Sean Brogan <sean.brogan@microsoft.com><mailto:sean.brogan@microsoft.com>

Cc: Bret Barkelew <Bret.Barkelew@microsoft.com><mailto:Bret.Barkelew@microsoft.com>

Cc: Michael Kubacki <mikuback@linux.microsoft.com><mailto:mikuback@linux.microsoft.com>

Signed-off-by: Gua Guo <gua.guo@intel.com><mailto:gua.guo@intel.com>

---

 UnitTestFrameworkPkg/ReadMe.md                        | 116 +++++++++++++++++++-

 UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc |   2 +-

 2 files changed, 116 insertions(+), 2 deletions(-)



diff --git a/UnitTestFrameworkPkg/ReadMe.md b/UnitTestFrameworkPkg/ReadMe.md

index e696412cb3..93fdacfc4a 100644

--- a/UnitTestFrameworkPkg/ReadMe.md

+++ b/UnitTestFrameworkPkg/ReadMe.md

@@ -361,7 +361,7 @@ RUNNING TEST SUITE: Int Safe Conversions Test Suite

 ```







 You can also, if you are so inclined, read the output from the exact instance of the test that was run during



-`stuart_ci_build`. The ouput file can be found on a path that looks like:



+`stuart_ci_build`. The output file can be found on a path that looks like:







 `Build/<Package>/HostTest/<Arch>/<TestName>.<TestSuiteName>.<Arch>.result.xml`







@@ -399,6 +399,120 @@ CMOCKA_XML_FILE=<absolute or relative path to output file>





 This mode is used by the test running plugin to aggregate the results for CI test status reporting in the web view.







+### Enable Code Coverage for GCC host-based tests



+



+LCOV is an extension of GCOV, a GNU tool which provides information about what parts of a program are actually executed (i.e. "covered") while running a particular test case. The extension consists of a set of scripts which build on the textual GCOV output to implement it.



+



+Documentation for LCOV can be found here:



+https://lcov.readthedocs.io/



+



+You can follow below steps to generate your code coverage report.



+



+



+



+```bash



+user@linux_machine:~/_uefi$stuart_ci_build -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5 -p MdePkg -t NOOPT



+...



+user@linux_machine:~/_uefi$lcov -c --rc lcov_branch_coverage=1 -o UnitTest.info -d Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/



+Capturing coverage data from Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/



+Found gcov version: 9.4.0



+Using intermediate gcov format



+Scanning Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/ for .gcda files ...



+Found 38 data files in Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/RShiftU64.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/Math64.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/SafeString.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/SwapBytes32.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/CpuDeadLoop.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/MultU64x64.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/LongJump.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/SetJump.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/BitField.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/String.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/DivU64x32Remainder.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/X64/GccInline.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/MultU64x32.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/MultS64x64.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/LinkedList.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/LShiftU64.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/DivU64x32.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/SwapBytes16.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/Unaligned.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/CheckSum.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/CopyMem.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/MemLibGuid.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/CompareMemWrapper.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/SetMemWrapper.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/SetMem64Wrapper.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/MemLibGeneric.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/CopyMemWrapper.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/SetMem.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/ZeroMemWrapper.gcda



+Processing Library/BasePrintLib/BasePrintLib/OUTPUT/PrintLibInternal.gcda



+Processing Library/BasePrintLib/BasePrintLib/OUTPUT/PrintLib.gcda



+Processing Library/BaseSafeIntLib/BaseSafeIntLib/OUTPUT/SafeIntLib64.gcda



+Processing Library/BaseSafeIntLib/BaseSafeIntLib/OUTPUT/SafeIntLib.gcda



+Processing Test/UnitTest/Library/BaseLib/BaseLibUnitTestsHost/OUTPUT/Base64UnitTest.gcda



+Processing Test/UnitTest/Library/BaseLib/BaseLibUnitTestsHost/OUTPUT/AutoGen.gcda



+Processing Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLibHost/OUTPUT/SafeIntLibUintnIntnUnitTests64.gcda



+Processing Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLibHost/OUTPUT/TestBaseSafeIntLib.gcda



+Processing Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLibHost/OUTPUT/AutoGen.gcda



+



+user@linux_machine:~/_uefi$genhtml UnitTest.info --branch-coverage -o Build/UnitTestReport



+



+Reading data file UnitTest.info



+Found 38 entries.



+Found common filename prefix "~/_uefi/MdePkg/Library"



+Writing .css and .png files.



+Generating output.



+Processing file ~/_uefi/Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/Test/UnitTest/Library/BaseLib/BaseLibUnitTestsHost/DEBUG/AutoGen.c



+Processing file ~/_uefi/Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLibHost/DEBUG/AutoGen.c



+Processing file BaseLib/Unaligned.c



+Processing file BaseLib/CheckSum.c



+Processing file BaseLib/BitField.c



+Processing file BaseLib/DivU64x32.c



+Processing file BaseLib/CpuDeadLoop.c



+Processing file BaseLib/LongJump.c



+Processing file BaseLib/MultU64x64.c



+Processing file BaseLib/RShiftU64.c



+Processing file BaseLib/SwapBytes32.c



+Processing file BaseLib/SwapBytes16.c



+Processing file BaseLib/SetJump.c



+Processing file BaseLib/LinkedList.c



+Processing file BaseLib/LShiftU64.c



+Processing file BaseLib/SafeString.c



+Processing file BaseLib/String.c



+Processing file BaseLib/DivU64x32Remainder.c



+Processing file BaseLib/MultS64x64.c



+Processing file BaseLib/MultU64x32.c



+Processing file BaseLib/Math64.c



+Processing file BaseLib/X64/GccInline.c



+Processing file BaseMemoryLib/SetMemWrapper.c



+Processing file BaseMemoryLib/CompareMemWrapper.c



+Processing file BaseMemoryLib/SetMem64Wrapper.c



+Processing file BaseMemoryLib/ZeroMemWrapper.c



+Processing file BaseMemoryLib/CopyMemWrapper.c



+Processing file BaseMemoryLib/CopyMem.c



+Processing file BaseMemoryLib/SetMem.c



+Processing file BaseMemoryLib/MemLibGuid.c



+Processing file BaseMemoryLib/MemLibGeneric.c



+Processing file BasePrintLib/PrintLib.c



+Processing file BasePrintLib/PrintLibInternal.c



+Processing file BaseSafeIntLib/SafeIntLib64.c



+Processing file BaseSafeIntLib/SafeIntLib.c



+Processing file ~/_uefi/MdePkg/Test/UnitTest/Library/BaseLib/Base64UnitTest.c



+Processing file ~/_uefi/MdePkg/Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLib.c



+Processing file ~/_uefi/MdePkg/Test/UnitTest/Library/BaseSafeIntLib/SafeIntLibUintnIntnUnitTests64.c



+Writing directory view page.



+Overall coverage rate:



+  lines......: 58.6% (2688 of 4589 lines)



+  functions..: 57.1% (249 of 436 functions)



+  branches...: 25.3% (1062 of 4195 branches)



+



+...



+



+```



+



 ### Important Note







 This works on both Windows and Linux, but is currently limited to x64 architectures. Working on getting others, but we



diff --git a/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc b/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc

index 4dd8d4ac67..8623aecb33 100644

--- a/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc

+++ b/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc

@@ -19,7 +19,7 @@

   MemoryAllocationLib|UnitTestFrameworkPkg/Library/Posix/MemoryAllocationLibPosix/MemoryAllocationLibPosix.inf







 [BuildOptions]



-  GCC:*_*_*_CC_FLAGS = -fno-pie



+  GCC:*_*_*_CC_FLAGS = -fno-pie -ftest-coverage -fprofile-arcs



 !ifdef $(UNIT_TESTING_DEBUG)



   MSFT:*_*_*_CC_FLAGS  = -D UNIT_TESTING_DEBUG=1



   GCC:*_*_*_CC_FLAGS   = -D UNIT_TESTING_DEBUG=1



[-- Attachment #2: Type: text/html, Size: 65355 bytes --]

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [edk2-devel] [PATCH v2] UnitTestFrameworkPkg: Add code coverage support for GCC
  2022-09-14  2:11     ` Guo, Gua
  2022-09-14 16:28       ` Michael D Kinney
@ 2022-09-14 16:34       ` Michael D Kinney
  2022-09-15  3:10         ` Guo, Gua
  1 sibling, 1 reply; 9+ messages in thread
From: Michael D Kinney @ 2022-09-14 16:34 UTC (permalink / raw)
  To: Guo, Gua, Sean Brogan, devel@edk2.groups.io, Kinney, Michael D
  Cc: Sean Brogan, Michael Kubacki

[-- Attachment #1: Type: text/plain, Size: 14432 bytes --]

So if you run the stuart command with that plugin available on Linux/GCC, then both the  .gcda and HTML content is always generated and local user can either view HTML content with browser or view the results in VS Code with the .gcda files.  Correct?

I think there ways to run the stuart command for just that single pluigin specified on the command line, so the builds are as fast as possible during unit test development.  Would be good to capture that in the ReadMe.md as well.

Thanks,

Mike

From: Guo, Gua <gua.guo@intel.com>
Sent: Tuesday, September 13, 2022 7:11 PM
To: Kinney, Michael D <michael.d.kinney@intel.com>; Sean Brogan <spbrogan@outlook.com>; devel@edk2.groups.io
Cc: Sean Brogan <sean.brogan@microsoft.com>; Michael Kubacki <mikuback@linux.microsoft.com>
Subject: RE: [edk2-devel] [PATCH v2] UnitTestFrameworkPkg: Add code coverage support for GCC

@Sean Brogan<mailto:spbrogan@outlook.com>

I go through the code Merged PR 5008: Initial implementation or code coverage on basecore u… · microsoft/mu_basecore@f6af51f (github.com)<https://github.com/microsoft/mu_basecore/commit/f6af51f856747faec5a9f78f5604a69211b5af2d#diff-52e599c27ab877ef616f244e8b26826a53853872acb894df4aeff99f48df373d>
It looks good from my view point.

@Kinney, Michael D<mailto:michael.d.kinney@intel.com>

Does that pytool plugin support viewing the code coverage visually in C source files? Yes

Either HTML or preferably an Visual Studio Code plugin? I think it can be supported by both based on below change. Due to VScode plugin is auto scan *.gcda and show the result in VScode source tree.
Merged PR 5008: Initial implementation or code coverage on basecore u… · microsoft/mu_basecore@f6af51f (github.com)<https://github.com/microsoft/mu_basecore/commit/f6af51f856747faec5a9f78f5604a69211b5af2d#diff-52e599c27ab877ef616f244e8b26826a53853872acb894df4aeff99f48df373d>

Flow for stuart_ci_build: (a) -> (b) -> (c) -> (d) -> (e) -> (f)
Flow for vscode extension Gcov Viewer - Visual Studio Marketplace<https://marketplace.visualstudio.com/items?itemName=JacquesLucke.gcov-viewer>:  (a) -> (b) -> (c) -> (d) -> (g)

(a) stuart_ci_build -p XXXPkg
(b) Call Edk2/.pytool/Plugin/HostUnitTestCompilerPlugin/HostUnitTestCompilerPlugin.py
(c) Call Edk2/BaseTools/Plugin/HostBasedUnitTestRunner/HostBasedUnitTestRunner.py
(d)       execute Build/XXXPkg/**/*Test* -> generate *.gcda files
(e)       use “lcov” tool to parse *.gcda
(f)        use “pycobertura” tool to generate index.html
(g) vscode extension to parse *.gcda and show it on source code directly.

Can it be updated to support VS compilers tool? No, currently I don’t find a good way to generate code coverage for MSVC compiler

Thanks,
Gua
From: Kinney, Michael D <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>
Sent: Wednesday, September 14, 2022 9:52 AM
To: Sean Brogan <spbrogan@outlook.com<mailto:spbrogan@outlook.com>>; devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Guo, Gua <gua.guo@intel.com<mailto:gua.guo@intel.com>>; Kinney, Michael D <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>
Cc: Sean Brogan <sean.brogan@microsoft.com<mailto:sean.brogan@microsoft.com>>; Michael Kubacki <mikuback@linux.microsoft.com<mailto:mikuback@linux.microsoft.com>>
Subject: RE: [edk2-devel] [PATCH v2] UnitTestFrameworkPkg: Add code coverage support for GCC

Does that pytool plugin support viewing the code coverage visually in C source files?

Either HTML or preferably an Visual Studio Code plugin?

Can it be updated to support VS compilers tool?

Mie

From: Sean Brogan <spbrogan@outlook.com<mailto:spbrogan@outlook.com>>
Sent: Tuesday, September 13, 2022 6:18 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Guo, Gua <gua.guo@intel.com<mailto:gua.guo@intel.com>>
Cc: Kinney, Michael D <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>; Sean Brogan <sean.brogan@microsoft.com<mailto:sean.brogan@microsoft.com>>; Michael Kubacki <mikuback@linux.microsoft.com<mailto:mikuback@linux.microsoft.com>>
Subject: Re: [edk2-devel] [PATCH v2] UnitTestFrameworkPkg: Add code coverage support for GCC


Hi Gua Guo,

I would propose that we make this easier for both the local user and the CI system to get coverage metrics.

Can you review this commit as it uses the same Edk2 plugins but adds code coverage support with GCC5:

Merged PR 5008: Initial implementation or code coverage on basecore u… · microsoft/mu_basecore@f6af51f (github.com)<https://github.com/microsoft/mu_basecore/commit/f6af51f856747faec5a9f78f5604a69211b5af2d#diff-52e599c27ab877ef616f244e8b26826a53853872acb894df4aeff99f48df373d>



Thanks

Sean




On 9/13/2022 6:01 PM, Guo, Gua wrote:

From: Gua Guo <gua.guo@intel.com><mailto:gua.guo@intel.com>



In order to collect code coverage after running executable

file, generate *.gcda and *.gcno file that require by lcov tool

to generate code coverage report.



Cc: Michael D Kinney <michael.d.kinney@intel.com><mailto:michael.d.kinney@intel.com>

Cc: Sean Brogan <sean.brogan@microsoft.com><mailto:sean.brogan@microsoft.com>

Cc: Bret Barkelew <Bret.Barkelew@microsoft.com><mailto:Bret.Barkelew@microsoft.com>

Cc: Michael Kubacki <mikuback@linux.microsoft.com><mailto:mikuback@linux.microsoft.com>

Signed-off-by: Gua Guo <gua.guo@intel.com><mailto:gua.guo@intel.com>

---

 UnitTestFrameworkPkg/ReadMe.md                        | 116 +++++++++++++++++++-

 UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc |   2 +-

 2 files changed, 116 insertions(+), 2 deletions(-)



diff --git a/UnitTestFrameworkPkg/ReadMe.md b/UnitTestFrameworkPkg/ReadMe.md

index e696412cb3..93fdacfc4a 100644

--- a/UnitTestFrameworkPkg/ReadMe.md

+++ b/UnitTestFrameworkPkg/ReadMe.md

@@ -361,7 +361,7 @@ RUNNING TEST SUITE: Int Safe Conversions Test Suite

 ```







 You can also, if you are so inclined, read the output from the exact instance of the test that was run during



-`stuart_ci_build`. The ouput file can be found on a path that looks like:



+`stuart_ci_build`. The output file can be found on a path that looks like:







 `Build/<Package>/HostTest/<Arch>/<TestName>.<TestSuiteName>.<Arch>.result.xml`







@@ -399,6 +399,120 @@ CMOCKA_XML_FILE=<absolute or relative path to output file>





 This mode is used by the test running plugin to aggregate the results for CI test status reporting in the web view.







+### Enable Code Coverage for GCC host-based tests



+



+LCOV is an extension of GCOV, a GNU tool which provides information about what parts of a program are actually executed (i.e. "covered") while running a particular test case. The extension consists of a set of scripts which build on the textual GCOV output to implement it.



+



+Documentation for LCOV can be found here:



+https://lcov.readthedocs.io/



+



+You can follow below steps to generate your code coverage report.



+



+



+



+```bash



+user@linux_machine:~/_uefi$stuart_ci_build -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5 -p MdePkg -t NOOPT



+...



+user@linux_machine:~/_uefi$lcov -c --rc lcov_branch_coverage=1 -o UnitTest.info -d Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/



+Capturing coverage data from Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/



+Found gcov version: 9.4.0



+Using intermediate gcov format



+Scanning Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/ for .gcda files ...



+Found 38 data files in Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/RShiftU64.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/Math64.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/SafeString.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/SwapBytes32.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/CpuDeadLoop.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/MultU64x64.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/LongJump.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/SetJump.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/BitField.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/String.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/DivU64x32Remainder.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/X64/GccInline.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/MultU64x32.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/MultS64x64.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/LinkedList.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/LShiftU64.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/DivU64x32.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/SwapBytes16.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/Unaligned.gcda



+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/CheckSum.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/CopyMem.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/MemLibGuid.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/CompareMemWrapper.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/SetMemWrapper.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/SetMem64Wrapper.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/MemLibGeneric.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/CopyMemWrapper.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/SetMem.gcda



+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/ZeroMemWrapper.gcda



+Processing Library/BasePrintLib/BasePrintLib/OUTPUT/PrintLibInternal.gcda



+Processing Library/BasePrintLib/BasePrintLib/OUTPUT/PrintLib.gcda



+Processing Library/BaseSafeIntLib/BaseSafeIntLib/OUTPUT/SafeIntLib64.gcda



+Processing Library/BaseSafeIntLib/BaseSafeIntLib/OUTPUT/SafeIntLib.gcda



+Processing Test/UnitTest/Library/BaseLib/BaseLibUnitTestsHost/OUTPUT/Base64UnitTest.gcda



+Processing Test/UnitTest/Library/BaseLib/BaseLibUnitTestsHost/OUTPUT/AutoGen.gcda



+Processing Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLibHost/OUTPUT/SafeIntLibUintnIntnUnitTests64.gcda



+Processing Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLibHost/OUTPUT/TestBaseSafeIntLib.gcda



+Processing Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLibHost/OUTPUT/AutoGen.gcda



+



+user@linux_machine:~/_uefi$genhtml UnitTest.info --branch-coverage -o Build/UnitTestReport



+



+Reading data file UnitTest.info



+Found 38 entries.



+Found common filename prefix "~/_uefi/MdePkg/Library"



+Writing .css and .png files.



+Generating output.



+Processing file ~/_uefi/Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/Test/UnitTest/Library/BaseLib/BaseLibUnitTestsHost/DEBUG/AutoGen.c



+Processing file ~/_uefi/Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLibHost/DEBUG/AutoGen.c



+Processing file BaseLib/Unaligned.c



+Processing file BaseLib/CheckSum.c



+Processing file BaseLib/BitField.c



+Processing file BaseLib/DivU64x32.c



+Processing file BaseLib/CpuDeadLoop.c



+Processing file BaseLib/LongJump.c



+Processing file BaseLib/MultU64x64.c



+Processing file BaseLib/RShiftU64.c



+Processing file BaseLib/SwapBytes32.c



+Processing file BaseLib/SwapBytes16.c



+Processing file BaseLib/SetJump.c



+Processing file BaseLib/LinkedList.c



+Processing file BaseLib/LShiftU64.c



+Processing file BaseLib/SafeString.c



+Processing file BaseLib/String.c



+Processing file BaseLib/DivU64x32Remainder.c



+Processing file BaseLib/MultS64x64.c



+Processing file BaseLib/MultU64x32.c



+Processing file BaseLib/Math64.c



+Processing file BaseLib/X64/GccInline.c



+Processing file BaseMemoryLib/SetMemWrapper.c



+Processing file BaseMemoryLib/CompareMemWrapper.c



+Processing file BaseMemoryLib/SetMem64Wrapper.c



+Processing file BaseMemoryLib/ZeroMemWrapper.c



+Processing file BaseMemoryLib/CopyMemWrapper.c



+Processing file BaseMemoryLib/CopyMem.c



+Processing file BaseMemoryLib/SetMem.c



+Processing file BaseMemoryLib/MemLibGuid.c



+Processing file BaseMemoryLib/MemLibGeneric.c



+Processing file BasePrintLib/PrintLib.c



+Processing file BasePrintLib/PrintLibInternal.c



+Processing file BaseSafeIntLib/SafeIntLib64.c



+Processing file BaseSafeIntLib/SafeIntLib.c



+Processing file ~/_uefi/MdePkg/Test/UnitTest/Library/BaseLib/Base64UnitTest.c



+Processing file ~/_uefi/MdePkg/Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLib.c



+Processing file ~/_uefi/MdePkg/Test/UnitTest/Library/BaseSafeIntLib/SafeIntLibUintnIntnUnitTests64.c



+Writing directory view page.



+Overall coverage rate:



+  lines......: 58.6% (2688 of 4589 lines)



+  functions..: 57.1% (249 of 436 functions)



+  branches...: 25.3% (1062 of 4195 branches)



+



+...



+



+```



+



 ### Important Note







 This works on both Windows and Linux, but is currently limited to x64 architectures. Working on getting others, but we



diff --git a/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc b/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc

index 4dd8d4ac67..8623aecb33 100644

--- a/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc

+++ b/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc

@@ -19,7 +19,7 @@

   MemoryAllocationLib|UnitTestFrameworkPkg/Library/Posix/MemoryAllocationLibPosix/MemoryAllocationLibPosix.inf







 [BuildOptions]



-  GCC:*_*_*_CC_FLAGS = -fno-pie



+  GCC:*_*_*_CC_FLAGS = -fno-pie -ftest-coverage -fprofile-arcs



 !ifdef $(UNIT_TESTING_DEBUG)



   MSFT:*_*_*_CC_FLAGS  = -D UNIT_TESTING_DEBUG=1



   GCC:*_*_*_CC_FLAGS   = -D UNIT_TESTING_DEBUG=1



[-- Attachment #2: Type: text/html, Size: 65667 bytes --]

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [edk2-devel] [PATCH v2] UnitTestFrameworkPkg: Add code coverage support for GCC
  2022-09-14 16:34       ` Michael D Kinney
@ 2022-09-15  3:10         ` Guo, Gua
  0 siblings, 0 replies; 9+ messages in thread
From: Guo, Gua @ 2022-09-15  3:10 UTC (permalink / raw)
  To: Kinney, Michael D, Sean Brogan, devel@edk2.groups.io
  Cc: Sean Brogan, Michael Kubacki

[-- Attachment #1: Type: text/plain, Size: 18227 bytes --]

So if you run the stuart command with that plugin available on Linux/GCC, then both the  .gcda and HTML content is always generated and local user can either view HTML content with browser or view the results in VS Code with the .gcda files.  Correct? Yes, Correct.

I think there ways to run the stuart command for just that single pluigin specified on the command line, so the builds are as fast as possible during unit test development.  Would be good to capture that in the ReadMe.md as well. Let me think how to integrated into Edk2 appropriately

For VS builds, I have heard that OpenCppCoverage is a good option:

https://github.com/OpenCppCoverage/OpenCppCoverage

Wow, Don’t know the tool before.

Let me summary as below hope can eliminate your concern.

Windows
Linux
Compiler
VS2019
GCC5
Coverage Tool
https://github.com/OpenCppCoverage/OpenCppCoverage
https://github.com/linux-test-project/lcov
VS Code Extension Support
No, For long term, I may need to enough time to study OpenCppCoverage behavior.
And try to integrated into BIOS-CAT - Visual Studio Marketplace<https://marketplace.visualstudio.com/items?itemName=CatMaster.bios-cat>
Yes, Gcov Viewer - Visual Studio Marketplace<https://marketplace.visualstudio.com/items?itemName=JacquesLucke.gcov-viewer>
Coverage File type
*.pdb
*.gcda, *.gcno
Currently Behavior
(Will require [PATCH v2] UnitTestFrameworkPkg: Add code coverage support for GCC)
Step1. wget https://github.com/OpenCppCoverage/OpenCppCoverage/releases/download/release-0.9.9.0/OpenCppCoverageSetup-x64-0.9.9.0.exe
Step2. OpenCppCoverageSetup-x64-0.9.9.0.exe /SILENT
Step3. stuart_ci_build -c .pytool/CISettings.py  -t NOOPT TOOL_CHAIN_TAG=VS2019 -p MdePkg
Step4. OpenCppCoverage.exe --sources %cd% --export_type html:%cd%\Build\UnitTestReport -- %cd%\Build\MdePkg\HostTest\NOOPT_VS2019\X64\BaseLibUnitTestsHost.exe
Step5. Open "Build/UnitTestReport/index.html"
Step1. sudo apt-get install lcov
Step2. stuart_ci_build -c .pytool/CISettings.py -p MdePkg -t NOOPT TOOL_CHAIN_TAG=GCC5
Step3. lcov -c --rc lcov_branch_coverage=1 -o UnitTest.info -d Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/
Step4. genhtml UnitTest.info --branch-coverage -o .Build/UnitTestReport
Step5. Open "Build/UnitTestReport/index.html"
Ideally Behavior (For Local Build)
(It may give me some time to think how to integrate the code into edk2\BaseTools\Plugin\HostBasedUnitTestRunner\HostBasedUnitTestRunner.py)
Step1. wget https://github.com/OpenCppCoverage/OpenCppCoverage/releases/download/release-0.9.9.0/OpenCppCoverageSetup-x64-0.9.9.0.exe
Step2. OpenCppCoverageSetup-x64-0.9.9.0.exe /SILENT
Step3. stuart_ci_build -c .pytool/CISettings.py  -t NOOPT TOOL_CHAIN_TAG=VS2019 -p MdePkg
Step4. Open "Build/Coverage.html"

Step1. sudo apt-get install lcov
Step2. stuart_ci_build -c .pytool/CISettings.py -p MdePkg -t NOOPT TOOL_CHAIN_TAG=GCC5
Step3. Open "Build/Coverage.html"
Ideally Behavior (For CI/CD)

It may give me some time to think how to modify.
edk2\.azurepipelines\Windows-VS2019.yml
It may give me some time to think how to modify.
edk2\.azurepipelines\Ubuntu-GCC5.yml

Thanks,
Gua

From: Kinney, Michael D <michael.d.kinney@intel.com>
Sent: Thursday, September 15, 2022 12:35 AM
To: Guo, Gua <gua.guo@intel.com>; Sean Brogan <spbrogan@outlook.com>; devel@edk2.groups.io; Kinney, Michael D <michael.d.kinney@intel.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>; Michael Kubacki <mikuback@linux.microsoft.com>
Subject: RE: [edk2-devel] [PATCH v2] UnitTestFrameworkPkg: Add code coverage support for GCC

So if you run the stuart command with that plugin available on Linux/GCC, then both the  .gcda and HTML content is always generated and local user can either view HTML content with browser or view the results in VS Code with the .gcda files.  Correct?

I think there ways to run the stuart command for just that single pluigin specified on the command line, so the builds are as fast as possible during unit test development.  Would be good to capture that in the ReadMe.md as well.

Thanks,

Mike

From: Guo, Gua <gua.guo@intel.com<mailto:gua.guo@intel.com>>
Sent: Tuesday, September 13, 2022 7:11 PM
To: Kinney, Michael D <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>; Sean Brogan <spbrogan@outlook.com<mailto:spbrogan@outlook.com>>; devel@edk2.groups.io<mailto:devel@edk2.groups.io>
Cc: Sean Brogan <sean.brogan@microsoft.com<mailto:sean.brogan@microsoft.com>>; Michael Kubacki <mikuback@linux.microsoft.com<mailto:mikuback@linux.microsoft.com>>
Subject: RE: [edk2-devel] [PATCH v2] UnitTestFrameworkPkg: Add code coverage support for GCC

@Sean Brogan<mailto:spbrogan@outlook.com>

I go through the code Merged PR 5008: Initial implementation or code coverage on basecore u… · microsoft/mu_basecore@f6af51f (github.com)<https://github.com/microsoft/mu_basecore/commit/f6af51f856747faec5a9f78f5604a69211b5af2d#diff-52e599c27ab877ef616f244e8b26826a53853872acb894df4aeff99f48df373d>
It looks good from my view point.

@Kinney, Michael D<mailto:michael.d.kinney@intel.com>

Does that pytool plugin support viewing the code coverage visually in C source files? Yes

Either HTML or preferably an Visual Studio Code plugin? I think it can be supported by both based on below change. Due to VScode plugin is auto scan *.gcda and show the result in VScode source tree.
Merged PR 5008: Initial implementation or code coverage on basecore u… · microsoft/mu_basecore@f6af51f (github.com)<https://github.com/microsoft/mu_basecore/commit/f6af51f856747faec5a9f78f5604a69211b5af2d#diff-52e599c27ab877ef616f244e8b26826a53853872acb894df4aeff99f48df373d>

Flow for stuart_ci_build: (a) -> (b) -> (c) -> (d) -> (e) -> (f)
Flow for vscode extension Gcov Viewer - Visual Studio Marketplace<https://marketplace.visualstudio.com/items?itemName=JacquesLucke.gcov-viewer>:  (a) -> (b) -> (c) -> (d) -> (g)

(a) stuart_ci_build -p XXXPkg
(b) Call Edk2/.pytool/Plugin/HostUnitTestCompilerPlugin/HostUnitTestCompilerPlugin.py
(c) Call Edk2/BaseTools/Plugin/HostBasedUnitTestRunner/HostBasedUnitTestRunner.py
(d)       execute Build/XXXPkg/**/*Test* -> generate *.gcda files
(e)       use “lcov” tool to parse *.gcda
(f)        use “pycobertura” tool to generate index.html
(g) vscode extension to parse *.gcda and show it on source code directly.

Can it be updated to support VS compilers tool? No, currently I don’t find a good way to generate code coverage for MSVC compiler

Thanks,
Gua
From: Kinney, Michael D <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>
Sent: Wednesday, September 14, 2022 9:52 AM
To: Sean Brogan <spbrogan@outlook.com<mailto:spbrogan@outlook.com>>; devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Guo, Gua <gua.guo@intel.com<mailto:gua.guo@intel.com>>; Kinney, Michael D <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>
Cc: Sean Brogan <sean.brogan@microsoft.com<mailto:sean.brogan@microsoft.com>>; Michael Kubacki <mikuback@linux.microsoft.com<mailto:mikuback@linux.microsoft.com>>
Subject: RE: [edk2-devel] [PATCH v2] UnitTestFrameworkPkg: Add code coverage support for GCC

Does that pytool plugin support viewing the code coverage visually in C source files?

Either HTML or preferably an Visual Studio Code plugin?

Can it be updated to support VS compilers tool?

Mie

From: Sean Brogan <spbrogan@outlook.com<mailto:spbrogan@outlook.com>>
Sent: Tuesday, September 13, 2022 6:18 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Guo, Gua <gua.guo@intel.com<mailto:gua.guo@intel.com>>
Cc: Kinney, Michael D <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>; Sean Brogan <sean.brogan@microsoft.com<mailto:sean.brogan@microsoft.com>>; Michael Kubacki <mikuback@linux.microsoft.com<mailto:mikuback@linux.microsoft.com>>
Subject: Re: [edk2-devel] [PATCH v2] UnitTestFrameworkPkg: Add code coverage support for GCC

Hi Gua Guo,

I would propose that we make this easier for both the local user and the CI system to get coverage metrics.

Can you review this commit as it uses the same Edk2 plugins but adds code coverage support with GCC5:

Merged PR 5008: Initial implementation or code coverage on basecore u… · microsoft/mu_basecore@f6af51f (github.com)<https://github.com/microsoft/mu_basecore/commit/f6af51f856747faec5a9f78f5604a69211b5af2d#diff-52e599c27ab877ef616f244e8b26826a53853872acb894df4aeff99f48df373d>

Thanks

Sean

On 9/13/2022 6:01 PM, Guo, Gua wrote:

From: Gua Guo <gua.guo@intel.com><mailto:gua.guo@intel.com>

In order to collect code coverage after running executable

file, generate *.gcda and *.gcno file that require by lcov tool

to generate code coverage report.

Cc: Michael D Kinney <michael.d.kinney@intel.com><mailto:michael.d.kinney@intel.com>

Cc: Sean Brogan <sean.brogan@microsoft.com><mailto:sean.brogan@microsoft.com>

Cc: Bret Barkelew <Bret.Barkelew@microsoft.com><mailto:Bret.Barkelew@microsoft.com>

Cc: Michael Kubacki <mikuback@linux.microsoft.com><mailto:mikuback@linux.microsoft.com>

Signed-off-by: Gua Guo <gua.guo@intel.com><mailto:gua.guo@intel.com>

---

 UnitTestFrameworkPkg/ReadMe.md                        | 116 +++++++++++++++++++-

 UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc |   2 +-

 2 files changed, 116 insertions(+), 2 deletions(-)

diff --git a/UnitTestFrameworkPkg/ReadMe.md b/UnitTestFrameworkPkg/ReadMe.md

index e696412cb3..93fdacfc4a 100644

--- a/UnitTestFrameworkPkg/ReadMe.md

+++ b/UnitTestFrameworkPkg/ReadMe.md

@@ -361,7 +361,7 @@ RUNNING TEST SUITE: Int Safe Conversions Test Suite

 ```

 You can also, if you are so inclined, read the output from the exact instance of the test that was run during

-`stuart_ci_build`. The ouput file can be found on a path that looks like:

+`stuart_ci_build`. The output file can be found on a path that looks like:

 `Build/<Package>/HostTest/<Arch>/<TestName>.<TestSuiteName>.<Arch>.result.xml`

@@ -399,6 +399,120 @@ CMOCKA_XML_FILE=<absolute or relative path to output file>

 This mode is used by the test running plugin to aggregate the results for CI test status reporting in the web view.

+### Enable Code Coverage for GCC host-based tests

+

+LCOV is an extension of GCOV, a GNU tool which provides information about what parts of a program are actually executed (i.e. "covered") while running a particular test case. The extension consists of a set of scripts which build on the textual GCOV output to implement it.

+

+Documentation for LCOV can be found here:

+https://lcov.readthedocs.io/

+

+You can follow below steps to generate your code coverage report.

+

+

+

+```bash

+user@linux_machine:~/_uefi$stuart_ci_build -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5 -p MdePkg -t NOOPT

+...

+user@linux_machine:~/_uefi$lcov -c --rc lcov_branch_coverage=1 -o UnitTest.info -d Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/

+Capturing coverage data from Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/

+Found gcov version: 9.4.0

+Using intermediate gcov format

+Scanning Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/ for .gcda files ...

+Found 38 data files in Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/

+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/RShiftU64.gcda

+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/Math64.gcda

+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/SafeString.gcda

+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/SwapBytes32.gcda

+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/CpuDeadLoop.gcda

+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/MultU64x64.gcda

+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/LongJump.gcda

+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/SetJump.gcda

+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/BitField.gcda

+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/String.gcda

+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/DivU64x32Remainder.gcda

+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/X64/GccInline.gcda

+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/MultU64x32.gcda

+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/MultS64x64.gcda

+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/LinkedList.gcda

+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/LShiftU64.gcda

+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/DivU64x32.gcda

+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/SwapBytes16.gcda

+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/Unaligned.gcda

+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/CheckSum.gcda

+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/CopyMem.gcda

+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/MemLibGuid.gcda

+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/CompareMemWrapper.gcda

+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/SetMemWrapper.gcda

+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/SetMem64Wrapper.gcda

+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/MemLibGeneric.gcda

+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/CopyMemWrapper.gcda

+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/SetMem.gcda

+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/ZeroMemWrapper.gcda

+Processing Library/BasePrintLib/BasePrintLib/OUTPUT/PrintLibInternal.gcda

+Processing Library/BasePrintLib/BasePrintLib/OUTPUT/PrintLib.gcda

+Processing Library/BaseSafeIntLib/BaseSafeIntLib/OUTPUT/SafeIntLib64.gcda

+Processing Library/BaseSafeIntLib/BaseSafeIntLib/OUTPUT/SafeIntLib.gcda

+Processing Test/UnitTest/Library/BaseLib/BaseLibUnitTestsHost/OUTPUT/Base64UnitTest.gcda

+Processing Test/UnitTest/Library/BaseLib/BaseLibUnitTestsHost/OUTPUT/AutoGen.gcda

+Processing Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLibHost/OUTPUT/SafeIntLibUintnIntnUnitTests64.gcda

+Processing Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLibHost/OUTPUT/TestBaseSafeIntLib.gcda

+Processing Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLibHost/OUTPUT/AutoGen.gcda

+

+user@linux_machine:~/_uefi$genhtml UnitTest.info --branch-coverage -o Build/UnitTestReport

+

+Reading data file UnitTest.info

+Found 38 entries.

+Found common filename prefix "~/_uefi/MdePkg/Library"

+Writing .css and .png files.

+Generating output.

+Processing file ~/_uefi/Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/Test/UnitTest/Library/BaseLib/BaseLibUnitTestsHost/DEBUG/AutoGen.c

+Processing file ~/_uefi/Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLibHost/DEBUG/AutoGen.c

+Processing file BaseLib/Unaligned.c

+Processing file BaseLib/CheckSum.c

+Processing file BaseLib/BitField.c

+Processing file BaseLib/DivU64x32.c

+Processing file BaseLib/CpuDeadLoop.c

+Processing file BaseLib/LongJump.c

+Processing file BaseLib/MultU64x64.c

+Processing file BaseLib/RShiftU64.c

+Processing file BaseLib/SwapBytes32.c

+Processing file BaseLib/SwapBytes16.c

+Processing file BaseLib/SetJump.c

+Processing file BaseLib/LinkedList.c

+Processing file BaseLib/LShiftU64.c

+Processing file BaseLib/SafeString.c

+Processing file BaseLib/String.c

+Processing file BaseLib/DivU64x32Remainder.c

+Processing file BaseLib/MultS64x64.c

+Processing file BaseLib/MultU64x32.c

+Processing file BaseLib/Math64.c

+Processing file BaseLib/X64/GccInline.c

+Processing file BaseMemoryLib/SetMemWrapper.c

+Processing file BaseMemoryLib/CompareMemWrapper.c

+Processing file BaseMemoryLib/SetMem64Wrapper.c

+Processing file BaseMemoryLib/ZeroMemWrapper.c

+Processing file BaseMemoryLib/CopyMemWrapper.c

+Processing file BaseMemoryLib/CopyMem.c

+Processing file BaseMemoryLib/SetMem.c

+Processing file BaseMemoryLib/MemLibGuid.c

+Processing file BaseMemoryLib/MemLibGeneric.c

+Processing file BasePrintLib/PrintLib.c

+Processing file BasePrintLib/PrintLibInternal.c

+Processing file BaseSafeIntLib/SafeIntLib64.c

+Processing file BaseSafeIntLib/SafeIntLib.c

+Processing file ~/_uefi/MdePkg/Test/UnitTest/Library/BaseLib/Base64UnitTest.c

+Processing file ~/_uefi/MdePkg/Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLib.c

+Processing file ~/_uefi/MdePkg/Test/UnitTest/Library/BaseSafeIntLib/SafeIntLibUintnIntnUnitTests64.c

+Writing directory view page.

+Overall coverage rate:

+  lines......: 58.6% (2688 of 4589 lines)

+  functions..: 57.1% (249 of 436 functions)

+  branches...: 25.3% (1062 of 4195 branches)

+

+...

+

+```

+

 ### Important Note

 This works on both Windows and Linux, but is currently limited to x64 architectures. Working on getting others, but we

diff --git a/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc b/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc

index 4dd8d4ac67..8623aecb33 100644

--- a/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc

+++ b/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc

@@ -19,7 +19,7 @@

   MemoryAllocationLib|UnitTestFrameworkPkg/Library/Posix/MemoryAllocationLibPosix/MemoryAllocationLibPosix.inf

 [BuildOptions]

-  GCC:*_*_*_CC_FLAGS = -fno-pie

+  GCC:*_*_*_CC_FLAGS = -fno-pie -ftest-coverage -fprofile-arcs

 !ifdef $(UNIT_TESTING_DEBUG)

   MSFT:*_*_*_CC_FLAGS  = -D UNIT_TESTING_DEBUG=1

   GCC:*_*_*_CC_FLAGS   = -D UNIT_TESTING_DEBUG=1

[-- Attachment #2: Type: text/html, Size: 40775 bytes --]

^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2022-09-15  3:10 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2022-09-14  1:01 [PATCH v2] UnitTestFrameworkPkg: Add code coverage support for GCC Guo, Gua
2022-09-14  1:17 ` [edk2-devel] " Sean
2022-09-14  1:51   ` Michael D Kinney
2022-09-14  2:10     ` Sean
2022-09-14 16:25       ` Michael D Kinney
2022-09-14  2:11     ` Guo, Gua
2022-09-14 16:28       ` Michael D Kinney
2022-09-14 16:34       ` Michael D Kinney
2022-09-15  3:10         ` Guo, Gua

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox