* [PATCH v8 1/3] UnitTestFrameworkPkg: Add code coverage support for GCC
2023-01-02 11:24 [PATCH v8 0/3] Add code coverage support for GCC/MSVC Guo, Gua
@ 2023-01-02 11:25 ` Guo, Gua
2023-01-02 11:25 ` [PATCH v8 2/3] BaseTools/Plugin: Add coverage support for Unit Test Guo, Gua
` (4 subsequent siblings)
5 siblings, 0 replies; 10+ messages in thread
From: Guo, Gua @ 2023-01-02 11:25 UTC (permalink / raw)
To: devel
Cc: Gua Guo, Michael D Kinney, Sean Brogan, Bret Barkelew,
Michael Kubacki
From: Gua Guo <gua.guo@intel.com>
In order to collect code coverage after running executable
file, generate *.gcda and *.gcno file that require by lcov tool
to generate code coverage report.
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Michael Kubacki <mikuback@linux.microsoft.com>
Signed-off-by: Gua Guo <gua.guo@intel.com>
---
UnitTestFrameworkPkg/ReadMe.md | 41 +++++++++++++++++++
.../UnitTestFrameworkPkg.ci.yaml | 1 +
.../UnitTestFrameworkPkgHost.dsc.inc | 3 +-
3 files changed, 44 insertions(+), 1 deletion(-)
diff --git a/UnitTestFrameworkPkg/ReadMe.md b/UnitTestFrameworkPkg/ReadMe.md
index 9ce04b7f3e..3fa4e1910f 100644
--- a/UnitTestFrameworkPkg/ReadMe.md
+++ b/UnitTestFrameworkPkg/ReadMe.md
@@ -583,6 +583,47 @@ GTEST_OUTPUT=xml:<absolute or relative path to output file>
This mode is used by the test running plugin to aggregate the results for CI test status reporting in the web view.
+### Code Coverage
+
+Host based Unit Tests will automatically enable coverage data.
+
+For Windows, This is primarily leverage for pipeline builds, but this can be leveraged locally using the
+OpenCppCoverage windows tool to parse coverage data to cobertura xml format.
+
+- Windows Prerequisite
+ ```bash
+ Download and install https://github.com/OpenCppCoverage/OpenCppCoverage/releases
+ python -m pip install --upgrade -r ./pip-requirements.txt
+ stuart_ci_build -c .pytool/CISettings.py -t NOOPT TOOL_CHAIN_TAG=VS2019 -p MdeModulePkg
+ Open Build/coverage.xml
+ ```
+
+ - How to see code coverage data on IDE Visual Studio
+ ```
+ Open Visual Studio VS2019 or above version
+ Click "Tools" -> "OpenCppCoverage Settings"
+ Fill your execute file into "Program to run:"
+ Click "Tools" -> "Run OpenCppCoverage"
+ ```
+
+
+For Linux, This is primarily leveraged for pipeline builds, but this can be leveraged locally using the
+lcov linux tool, and parsed using the lcov_cobertura python tool to parse it to cobertura xml format.
+
+- Linux Prerequisite
+ ```bash
+ sudo apt-get install -y lcov
+ python -m pip install --upgrade -r ./pip-requirements.txt
+ stuart_ci_build -c .pytool/CISettings.py -t NOOPT TOOL_CHAIN_TAG=GCC5 -p MdeModulePkg
+ Open Build/coverage.xml
+ ```
+ - How to see code coverage data on IDE Visual Studio Code
+ ```
+ Download plugin "Coverage Gutters"
+ Press Hot Key "Ctrl + Shift + P" and click option "Coverage Gutters: Display Coverage"
+ ```
+
+
### Important Note
This works on both Windows and Linux but is currently limited to x64 architectures. Working on getting others, but we
diff --git a/UnitTestFrameworkPkg/UnitTestFrameworkPkg.ci.yaml b/UnitTestFrameworkPkg/UnitTestFrameworkPkg.ci.yaml
index 072df6208c..d8f8e024c4 100644
--- a/UnitTestFrameworkPkg/UnitTestFrameworkPkg.ci.yaml
+++ b/UnitTestFrameworkPkg/UnitTestFrameworkPkg.ci.yaml
@@ -91,6 +91,7 @@
"pytools",
"NOFAILURE",
"cmockery",
+ "cobertura",
"DHAVE", # build flag for cmocka in the INF
"gtest", # file name in GoogleTestLib.inf
"corthon", # Contact GitHub account in Readme
diff --git a/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc b/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc
index 8009337552..7f5dfa30ed 100644
--- a/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc
+++ b/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc
@@ -27,7 +27,8 @@
GCC:*_*_*_CC_FLAGS = -D UNIT_TESTING_DEBUG=1
XCODE:*_*_*_CC_FLAGS = -D UNIT_TESTING_DEBUG=1
!endif
-
+ GCC:*_GCC5_*_CC_FLAGS = --coverage
+ GCC:*_GCC5_*_DLINK_FLAGS = --coverage
[BuildOptions.common.EDKII.HOST_APPLICATION]
#
# MSFT
--
2.31.1.windows.1
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH v8 2/3] BaseTools/Plugin: Add coverage support for Unit Test
2023-01-02 11:24 [PATCH v8 0/3] Add code coverage support for GCC/MSVC Guo, Gua
2023-01-02 11:25 ` [PATCH v8 1/3] UnitTestFrameworkPkg: Add code coverage support for GCC Guo, Gua
@ 2023-01-02 11:25 ` Guo, Gua
2023-01-02 11:25 ` [PATCH v8 3/3] .azurepipelines: Install code coverage tool Guo, Gua
` (3 subsequent siblings)
5 siblings, 0 replies; 10+ messages in thread
From: Guo, Gua @ 2023-01-02 11:25 UTC (permalink / raw)
To: devel
Cc: Gua Guo, Bob Feng, Bret Barkelew, Liming Gao, Michael D Kinney,
Sean Brogan
From: Gua Guo <gua.guo@intel.com>
For GCC, use lcov to generate Unit Test code coverage
report
For VS2019, use OpenCppCoverage to generate code
coverage report
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Liming Gao <gaoliming@bysoft.com.cn>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>
Signed-off-by: Gua Guo <gua.guo@intel.com>
---
.../HostBasedUnitTestRunner.py | 101 +++++++++++++++++-
1 file changed, 100 insertions(+), 1 deletion(-)
diff --git a/BaseTools/Plugin/HostBasedUnitTestRunner/HostBasedUnitTestRunner.py b/BaseTools/Plugin/HostBasedUnitTestRunner/HostBasedUnitTestRunner.py
index a8220aacd3..0e013c5f1a 100644
--- a/BaseTools/Plugin/HostBasedUnitTestRunner/HostBasedUnitTestRunner.py
+++ b/BaseTools/Plugin/HostBasedUnitTestRunner/HostBasedUnitTestRunner.py
@@ -94,7 +94,7 @@ class HostBasedUnitTestRunner(IUefiBuildPlugin):
# Run the test.
ret = RunCmd('"' + test + '"', "", workingdir=cp)
- if(ret != 0):
+ if ret != 0:
logging.error("UnitTest Execution Error: " +
os.path.basename(test))
else:
@@ -115,4 +115,103 @@ class HostBasedUnitTestRunner(IUefiBuildPlugin):
" %s - %s" % (case.attrib['name'], result.text))
failure_count += 1
+ if thebuilder.env.GetValue("CODE_COVERAGE") != "FALSE":
+ if thebuilder.env.GetValue("TOOL_CHAIN_TAG") == "GCC5":
+ self.gen_code_coverage_gcc(thebuilder)
+ elif thebuilder.env.GetValue("TOOL_CHAIN_TAG").startswith ("VS"):
+ self.gen_code_coverage_msvc(thebuilder)
+ else:
+ logging.info("Skipping code coverage. Currently, support GCC and MSVC compiler.")
+
return failure_count
+
+ def gen_code_coverage_gcc(self, thebuilder):
+ logging.info("Generating UnitTest code coverage")
+
+ buildOutputBase = thebuilder.env.GetValue("BUILD_OUTPUT_BASE")
+ workspace = thebuilder.env.GetValue("WORKSPACE")
+
+ # Generate base code coverage for all source files
+ ret = RunCmd("lcov", f"--no-external --capture --initial --directory {buildOutputBase} --output-file {buildOutputBase}/cov-base.info --rc lcov_branch_coverage=1")
+ if ret != 0:
+ logging.error("UnitTest Coverage: Failed to build initial coverage data.")
+ return 1
+
+ # Coverage data for tested files only
+ ret = RunCmd("lcov", f"--capture --directory {buildOutputBase}/ --output-file {buildOutputBase}/coverage-test.info --rc lcov_branch_coverage=1")
+ if ret != 0:
+ logging.error("UnitTest Coverage: Failed to build coverage data for tested files.")
+ return 1
+
+ # Aggregate all coverage data
+ ret = RunCmd("lcov", f"--add-tracefile {buildOutputBase}/cov-base.info --add-tracefile {buildOutputBase}/coverage-test.info --output-file {buildOutputBase}/total-coverage.info --rc lcov_branch_coverage=1")
+ if ret != 0:
+ logging.error("UnitTest Coverage: Failed to aggregate coverage data.")
+ return 1
+
+ # Generate coverage XML
+ ret = RunCmd("lcov_cobertura",f"{buildOutputBase}/total-coverage.info -o {buildOutputBase}/compare.xml")
+ if ret != 0:
+ logging.error("UnitTest Coverage: Failed to generate coverage XML.")
+ return 1
+
+ # Filter out auto-generated and test code
+ ret = RunCmd("lcov_cobertura",f"{buildOutputBase}/total-coverage.info --excludes ^.*UnitTest\|^.*MU\|^.*Mock\|^.*DEBUG -o {buildOutputBase}/coverage.xml")
+ if ret != 0:
+ logging.error("UnitTest Coverage: Failed generate filtered coverage XML.")
+ return 1
+
+ # Generate all coverage file
+ testCoverageList = glob.glob (f"{workspace}/Build/**/total-coverage.info", recursive=True)
+
+ coverageFile = ""
+ for testCoverage in testCoverageList:
+ coverageFile += " --add-tracefile " + testCoverage
+ ret = RunCmd("lcov", f"{coverageFile} --output-file {workspace}/Build/all-coverage.info --rc lcov_branch_coverage=1")
+ if ret != 0:
+ logging.error("UnitTest Coverage: Failed generate all coverage file.")
+ return 1
+
+ # Generate and XML file if requested.for all package
+ if os.path.isfile(f"{workspace}/Build/coverage.xml"):
+ os.remove(f"{workspace}/Build/coverage.xml")
+ ret = RunCmd("lcov_cobertura",f"{workspace}/Build/all-coverage.info --excludes ^.*UnitTest\|^.*MU\|^.*Mock\|^.*DEBUG -o {workspace}/Build/coverage.xml")
+
+ return 0
+
+
+ def gen_code_coverage_msvc(self, thebuilder):
+ logging.info("Generating UnitTest code coverage")
+
+
+ buildOutputBase = thebuilder.env.GetValue("BUILD_OUTPUT_BASE")
+ testList = glob.glob(os.path.join(buildOutputBase, "**","*Test*.exe"), recursive=True)
+ workspace = thebuilder.env.GetValue("WORKSPACE")
+ workspace = (workspace + os.sep) if workspace[-1] != os.sep else workspace
+ # Generate coverage file
+ coverageFile = ""
+ for testFile in testList:
+ ret = RunCmd("OpenCppCoverage", f"--source {workspace} --export_type binary:{testFile}.cov -- {testFile}")
+ coverageFile += " --input_coverage=" + testFile + ".cov"
+ if ret != 0:
+ logging.error("UnitTest Coverage: Failed to collect coverage data.")
+ return 1
+
+ # Generate and XML file if requested.by each package
+ ret = RunCmd("OpenCppCoverage", f"--export_type cobertura:{os.path.join(buildOutputBase, 'coverage.xml')} --working_dir={workspace}Build {coverageFile}")
+ if ret != 0:
+ logging.error("UnitTest Coverage: Failed to generate cobertura format xml in single package.")
+ return 1
+
+ # Generate total report XML file for all package
+ testCoverageList = glob.glob(os.path.join(workspace, "Build", "**","*Test*.exe.cov"), recursive=True)
+ coverageFile = ""
+ for testCoverage in testCoverageList:
+ coverageFile += " --input_coverage=" + testCoverage
+
+ ret = RunCmd("OpenCppCoverage", f"--export_type cobertura:{workspace}Build/coverage.xml --working_dir={workspace}Build {coverageFile}")
+ if ret != 0:
+ logging.error("UnitTest Coverage: Failed to generate cobertura format xml.")
+ return 1
+
+ return 0
--
2.31.1.windows.1
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH v8 3/3] .azurepipelines: Install code coverage tool
2023-01-02 11:24 [PATCH v8 0/3] Add code coverage support for GCC/MSVC Guo, Gua
2023-01-02 11:25 ` [PATCH v8 1/3] UnitTestFrameworkPkg: Add code coverage support for GCC Guo, Gua
2023-01-02 11:25 ` [PATCH v8 2/3] BaseTools/Plugin: Add coverage support for Unit Test Guo, Gua
@ 2023-01-02 11:25 ` Guo, Gua
2023-01-02 18:05 ` [edk2-devel] [PATCH v8 0/3] Add code coverage support for GCC/MSVC Michael D Kinney
` (2 subsequent siblings)
5 siblings, 0 replies; 10+ messages in thread
From: Guo, Gua @ 2023-01-02 11:25 UTC (permalink / raw)
To: devel; +Cc: Gua Guo, Sean Brogan, Bret Barkelew, Michael D Kinney, Liming Gao
From: Gua Guo <gua.guo@intel.com>
For Windows add below tool for code coverage
1. OpenCppCoverage: parsing pdb file to generate coverage
data
2. pycobertura: show up html format data for coverage data
For Linux add below tool for code coverage
1. lcov: parsing gcda gcno file to generate coverage data
2. lcov-cobertura: convert coverage data to cobertura format
3. pycobertura: show up html format data for coverage data
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Signed-off-by: Gua Guo <gua.guo@intel.com>
---
.azurepipelines/Ubuntu-GCC5.yml | 5 ++-
.azurepipelines/Windows-VS2019.yml | 5 +++
.../templates/pr-gate-build-job.yml | 36 +++++++++++++++++++
.azurepipelines/templates/pr-gate-steps.yml | 4 +++
pip-requirements.txt | 2 ++
5 files changed, 51 insertions(+), 1 deletion(-)
diff --git a/.azurepipelines/Ubuntu-GCC5.yml b/.azurepipelines/Ubuntu-GCC5.yml
index 1acd8d2a46..f83951eeaf 100644
--- a/.azurepipelines/Ubuntu-GCC5.yml
+++ b/.azurepipelines/Ubuntu-GCC5.yml
@@ -19,4 +19,7 @@ jobs:
tool_chain_tag: 'GCC5'
vm_image: 'ubuntu-latest'
arch_list: "IA32,X64,ARM,AARCH64,RISCV64,LOONGARCH64"
-
+ extra_install_step:
+ - bash: sudo apt-get install -y lcov
+ displayName: Install Code Coverage Tools
+ condition: and(gt(variables.pkg_count, 0), succeeded())
diff --git a/.azurepipelines/Windows-VS2019.yml b/.azurepipelines/Windows-VS2019.yml
index e4bd4b1d22..c07e5bb434 100644
--- a/.azurepipelines/Windows-VS2019.yml
+++ b/.azurepipelines/Windows-VS2019.yml
@@ -18,3 +18,8 @@ jobs:
tool_chain_tag: 'VS2019'
vm_image: 'windows-2019'
arch_list: "IA32,X64"
+ extra_install_step:
+ - powershell: choco install opencppcoverage; Write-Host "##vso[task.prependpath]C:\Program Files\OpenCppCoverage"
+ displayName: Install Code Coverage Tool
+ condition: and(gt(variables.pkg_count, 0), succeeded())
+
diff --git a/.azurepipelines/templates/pr-gate-build-job.yml b/.azurepipelines/templates/pr-gate-build-job.yml
index 7f88b41dc8..f148ec23e2 100644
--- a/.azurepipelines/templates/pr-gate-build-job.yml
+++ b/.azurepipelines/templates/pr-gate-build-job.yml
@@ -12,6 +12,7 @@ parameters:
tool_chain_tag: ''
vm_image: ''
arch_list: ''
+ extra_install_step: []
# Build step
jobs:
@@ -77,3 +78,38 @@ jobs:
build_pkgs: $(Build.Pkgs)
build_targets: $(Build.Targets)
build_archs: ${{ parameters.arch_list }}
+ extra_install_step: ${{ parameters.extra_install_step }}
+
+- job: Build_${{ parameters.tool_chain_tag }}_TARGET_CODE_COVERAGE
+ dependsOn: Build_${{ parameters.tool_chain_tag }}
+ workspace:
+ clean: all
+
+ pool:
+ vmImage: 'windows-2019'
+
+ steps:
+ - checkout: self
+ clean: true
+ fetchDepth: 1
+ submodules: true
+
+ - task: DownloadPipelineArtifact@2
+ displayName: 'Download Build Artifacts'
+ inputs:
+ buildType: 'current'
+ targetPath: '$(Build.ArtifactStagingDirectory)'
+
+ - task: CmdLine@2
+ displayName: Create code coverage report
+ inputs:
+ script: |
+ dotnet tool install -g dotnet-reportgenerator-globaltool
+ reportgenerator -reports:$(Build.ArtifactStagingDirectory)/**/coverage.xml -targetdir:$(Build.ArtifactStagingDirectory)/Coverage -reporttypes:Cobertura -filefilters:-*Build*;-*UnitTest*;-*Mock*;*usr/*
+
+ - task: PublishCodeCoverageResults@1
+ displayName: 'Publish code coverage'
+ inputs:
+ codeCoverageTool: Cobertura
+ summaryFileLocation: '$(Build.ArtifactStagingDirectory)/Coverage/Cobertura.xml'
+
diff --git a/.azurepipelines/templates/pr-gate-steps.yml b/.azurepipelines/templates/pr-gate-steps.yml
index cb431e53fc..080f60aea6 100644
--- a/.azurepipelines/templates/pr-gate-steps.yml
+++ b/.azurepipelines/templates/pr-gate-steps.yml
@@ -12,6 +12,7 @@ parameters:
build_pkgs: ''
build_targets: ''
build_archs: ''
+ extra_install_step: []
steps:
- checkout: self
@@ -37,6 +38,8 @@ steps:
displayName: fetch target branch
condition: eq(variables['Build.Reason'], 'PullRequest')
+- ${{ parameters.extra_install_step }}
+
# trim the package list if this is a PR
- task: CmdLine@1
displayName: Check if ${{ parameters.build_pkgs }} need testing
@@ -125,6 +128,7 @@ steps:
TestSuites.xml
**/BUILD_TOOLS_REPORT.html
**/OVERRIDELOG.TXT
+ coverage.xml
flattenFolders: true
condition: succeededOrFailed()
diff --git a/pip-requirements.txt b/pip-requirements.txt
index be8c7a1c37..4ffcadddd8 100644
--- a/pip-requirements.txt
+++ b/pip-requirements.txt
@@ -16,3 +16,5 @@ edk2-pytool-library==0.12.1
edk2-pytool-extensions~=0.20.0
edk2-basetools==0.1.39
antlr4-python3-runtime==4.7.1
+lcov-cobertura==2.0.2
+
--
2.31.1.windows.1
^ permalink raw reply related [flat|nested] 10+ messages in thread* Re: [edk2-devel] [PATCH v8 0/3] Add code coverage support for GCC/MSVC
2023-01-02 11:24 [PATCH v8 0/3] Add code coverage support for GCC/MSVC Guo, Gua
` (2 preceding siblings ...)
2023-01-02 11:25 ` [PATCH v8 3/3] .azurepipelines: Install code coverage tool Guo, Gua
@ 2023-01-02 18:05 ` Michael D Kinney
2023-01-03 16:14 ` Michael Kubacki
2023-01-04 2:49 ` Sean
5 siblings, 0 replies; 10+ messages in thread
From: Michael D Kinney @ 2023-01-02 18:05 UTC (permalink / raw)
To: devel@edk2.groups.io, Guo, Gua; +Cc: Kinney, Michael D
Series Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com>
> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Guo, Gua
> Sent: Monday, January 2, 2023 3:25 AM
> To: devel@edk2.groups.io
> Cc: Guo, Gua <gua.guo@intel.com>
> Subject: [edk2-devel] [PATCH v8 0/3] Add code coverage support for GCC/MSVC
>
> From: Gua Guo <gua.guo@intel.com>
>
> V1: Add coverage option for GCC
> V2: Add ReadMe.md for how to generate coverage report
> V3: Add VS2019 and GCC code coverage support
> V4: Add VS2019 and GCC Azure CI/CD support
> V5: Fix some typo and some flow issue
> V6: Remove html coverage information
> - Due to python 3.11 install lxml will be failure,
> pycobertura need it to convert cobertura format to
> html file.
> - Add section for developer how to use OpenCppCoverage
> on IDE Visual Studio
> V7: Remove redundant code and add code coverage pipeline support
> - Remove redundant code on HostBasedUnitTestRunner.py
> - Unify coding rule on HostBasedUnitTestRunner.py
> - Add CodeCoverage Azure pipeline support for GCC5 and VS2019
>
> Gua Guo (3):
> UnitTestFrameworkPkg: Add code coverage support for GCC
> BaseTools/Plugin: Add coverage support for Unit Test
> .azurepipelines: Install code coverage tool
>
> .azurepipelines/Ubuntu-GCC5.yml | 5 +-
> .azurepipelines/Windows-VS2019.yml | 5 +
> .../templates/pr-gate-build-job.yml | 36 +++++++
> .azurepipelines/templates/pr-gate-steps.yml | 4 +
> .../HostBasedUnitTestRunner.py | 101 +++++++++++++++++-
> UnitTestFrameworkPkg/ReadMe.md | 41 +++++++
> .../UnitTestFrameworkPkg.ci.yaml | 1 +
> .../UnitTestFrameworkPkgHost.dsc.inc | 3 +-
> pip-requirements.txt | 2 +
> 9 files changed, 195 insertions(+), 3 deletions(-)
>
> --
> 2.31.1.windows.1
>
>
>
>
>
^ permalink raw reply [flat|nested] 10+ messages in thread* Re: [edk2-devel] [PATCH v8 0/3] Add code coverage support for GCC/MSVC
2023-01-02 11:24 [PATCH v8 0/3] Add code coverage support for GCC/MSVC Guo, Gua
` (3 preceding siblings ...)
2023-01-02 18:05 ` [edk2-devel] [PATCH v8 0/3] Add code coverage support for GCC/MSVC Michael D Kinney
@ 2023-01-03 16:14 ` Michael Kubacki
2023-01-04 2:49 ` Sean
5 siblings, 0 replies; 10+ messages in thread
From: Michael Kubacki @ 2023-01-03 16:14 UTC (permalink / raw)
To: devel, gua.guo
For the series:
Reviewed-by: Michael Kubacki <michael.kubacki@microsoft.com>
On 1/2/2023 6:24 AM, Guo, Gua wrote:
> From: Gua Guo <gua.guo@intel.com>
>
> V1: Add coverage option for GCC
> V2: Add ReadMe.md for how to generate coverage report
> V3: Add VS2019 and GCC code coverage support
> V4: Add VS2019 and GCC Azure CI/CD support
> V5: Fix some typo and some flow issue
> V6: Remove html coverage information
> - Due to python 3.11 install lxml will be failure,
> pycobertura need it to convert cobertura format to
> html file.
> - Add section for developer how to use OpenCppCoverage
> on IDE Visual Studio
> V7: Remove redundant code and add code coverage pipeline support
> - Remove redundant code on HostBasedUnitTestRunner.py
> - Unify coding rule on HostBasedUnitTestRunner.py
> - Add CodeCoverage Azure pipeline support for GCC5 and VS2019
>
> Gua Guo (3):
> UnitTestFrameworkPkg: Add code coverage support for GCC
> BaseTools/Plugin: Add coverage support for Unit Test
> .azurepipelines: Install code coverage tool
>
> .azurepipelines/Ubuntu-GCC5.yml | 5 +-
> .azurepipelines/Windows-VS2019.yml | 5 +
> .../templates/pr-gate-build-job.yml | 36 +++++++
> .azurepipelines/templates/pr-gate-steps.yml | 4 +
> .../HostBasedUnitTestRunner.py | 101 +++++++++++++++++-
> UnitTestFrameworkPkg/ReadMe.md | 41 +++++++
> .../UnitTestFrameworkPkg.ci.yaml | 1 +
> .../UnitTestFrameworkPkgHost.dsc.inc | 3 +-
> pip-requirements.txt | 2 +
> 9 files changed, 195 insertions(+), 3 deletions(-)
>
^ permalink raw reply [flat|nested] 10+ messages in thread* Re: [edk2-devel] [PATCH v8 0/3] Add code coverage support for GCC/MSVC
2023-01-02 11:24 [PATCH v8 0/3] Add code coverage support for GCC/MSVC Guo, Gua
` (4 preceding siblings ...)
2023-01-03 16:14 ` Michael Kubacki
@ 2023-01-04 2:49 ` Sean
5 siblings, 0 replies; 10+ messages in thread
From: Sean @ 2023-01-04 2:49 UTC (permalink / raw)
To: devel, gua.guo
Looks like a great first step for code coverage collection.
The container support patch series will need to be updated (since it
hasn't yet been merged) and it will need to account for the new
requirements for code coverage.
In the future i would like to see the local experience improved (HTML
reports generated) and metrics tracked per package instead of build
wide. These topics can be discussed at the Monday tools meeting but I
think this series as is should be merged now to start collecting this data.
Thanks for all your efforts.
Series Reviewed-by: Sean Brogan<sean.brogan@microsoft.com>
On 1/2/2023 3:24 AM, Guo, Gua wrote:
> From: Gua Guo <gua.guo@intel.com>
>
> V1: Add coverage option for GCC
> V2: Add ReadMe.md for how to generate coverage report
> V3: Add VS2019 and GCC code coverage support
> V4: Add VS2019 and GCC Azure CI/CD support
> V5: Fix some typo and some flow issue
> V6: Remove html coverage information
> - Due to python 3.11 install lxml will be failure,
> pycobertura need it to convert cobertura format to
> html file.
> - Add section for developer how to use OpenCppCoverage
> on IDE Visual Studio
> V7: Remove redundant code and add code coverage pipeline support
> - Remove redundant code on HostBasedUnitTestRunner.py
> - Unify coding rule on HostBasedUnitTestRunner.py
> - Add CodeCoverage Azure pipeline support for GCC5 and VS2019
>
> Gua Guo (3):
> UnitTestFrameworkPkg: Add code coverage support for GCC
> BaseTools/Plugin: Add coverage support for Unit Test
> .azurepipelines: Install code coverage tool
>
> .azurepipelines/Ubuntu-GCC5.yml | 5 +-
> .azurepipelines/Windows-VS2019.yml | 5 +
> .../templates/pr-gate-build-job.yml | 36 +++++++
> .azurepipelines/templates/pr-gate-steps.yml | 4 +
> .../HostBasedUnitTestRunner.py | 101 +++++++++++++++++-
> UnitTestFrameworkPkg/ReadMe.md | 41 +++++++
> .../UnitTestFrameworkPkg.ci.yaml | 1 +
> .../UnitTestFrameworkPkgHost.dsc.inc | 3 +-
> pip-requirements.txt | 2 +
> 9 files changed, 195 insertions(+), 3 deletions(-)
>
^ permalink raw reply [flat|nested] 10+ messages in thread
![[-- Attachment #2: image001.png --] [-- Type: image/png, Size: 35196 bytes --]](../../BL1PR11MB5478154BCE7558EA95573CC1EFF79@BL1PR11MB5478.namprd11.prod.outlook.com/2-image001.png){kind=link}
![[-- Attachment #3: image002.png --] [-- Type: image/png, Size: 35672 bytes --]](../../BL1PR11MB5478154BCE7558EA95573CC1EFF79@BL1PR11MB5478.namprd11.prod.outlook.com/3-image002.png){kind=link}
![[-- Attachment #4: image003.png --] [-- Type: image/png, Size: 49263 bytes --]](../../BL1PR11MB5478154BCE7558EA95573CC1EFF79@BL1PR11MB5478.namprd11.prod.outlook.com/4-image003.png){kind=link}
![[-- Attachment #5: image004.png --] [-- Type: image/png, Size: 54363 bytes --]](../../BL1PR11MB5478154BCE7558EA95573CC1EFF79@BL1PR11MB5478.namprd11.prod.outlook.com/5-image004.png){kind=link}
![[-- Attachment #2: image003.png --] [-- Type: image/png, Size: 49263 bytes --]](../../CO1PR11MB4929488CE7F00A54A34484C5D2F79@CO1PR11MB4929.namprd11.prod.outlook.com/2-image003.png){kind=link}
![[-- Attachment #3: image005.png --] [-- Type: image/png, Size: 35196 bytes --]](../../CO1PR11MB4929488CE7F00A54A34484C5D2F79@CO1PR11MB4929.namprd11.prod.outlook.com/3-image005.png){kind=link}
![[-- Attachment #4: image006.png --] [-- Type: image/png, Size: 54363 bytes --]](../../CO1PR11MB4929488CE7F00A54A34484C5D2F79@CO1PR11MB4929.namprd11.prod.outlook.com/4-image006.png){kind=link}
![[-- Attachment #5: image007.png --] [-- Type: image/png, Size: 35672 bytes --]](../../CO1PR11MB4929488CE7F00A54A34484C5D2F79@CO1PR11MB4929.namprd11.prod.outlook.com/5-image007.png){kind=link}