From: "Guo, Gua" <gua.guo@intel.com>
To: Michael Kubacki <mikuback@linux.microsoft.com>,
"devel@edk2.groups.io" <devel@edk2.groups.io>,
"Kinney, Michael D" <michael.d.kinney@intel.com>,
Sean Brogan <sean.brogan@microsoft.com>,
"Kubacki, Michael" <michael.kubacki@microsoft.com>
Cc: "Barkelew, Bret" <bret.barkelew@microsoft.com>,
"Gao, Liming" <gaoliming@byosoft.com.cn>
Subject: Re: [edk2-devel] [PATCH v6 3/3] .azurepipelines: Install code coverage tool
Date: Wed, 21 Dec 2022 04:20:51 +0000 [thread overview]
Message-ID: <BL1PR11MB5478EBC66140F57EACB2D067EFEB9@BL1PR11MB5478.namprd11.prod.outlook.com> (raw)
In-Reply-To: <b96e5fdf-1816-9452-71fa-cae7ebdfdde6@linux.microsoft.com>
[-- Attachment #1.1: Type: text/plain, Size: 12425 bytes --]
@Kubacki, Michael<mailto:michael.kubacki@microsoft.com>
Try to see the block diagram "Current behavior" vs "Expected behavior". We find a way to transfer “Current behavior” to “Expected behavior” if we want to support pipeline tab “Code Coverage”. Hope it’s helpful for you to realize the limitation.
[cid:image001.png@01D91536.A4A7ADD0]
Thanks,
Gua
-----Original Message-----
From: Michael Kubacki <mikuback@linux.microsoft.com>
Sent: Wednesday, December 21, 2022 11:11 AM
To: devel@edk2.groups.io; Guo, Gua <gua.guo@intel.com>; Kinney, Michael D <michael.d.kinney@intel.com>; Sean Brogan <sean.brogan@microsoft.com>; Kubacki, Michael <michael.kubacki@microsoft.com>
Cc: Barkelew, Bret <bret.barkelew@microsoft.com>; Gao, Liming <gaoliming@byosoft.com.cn>
Subject: Re: [edk2-devel] [PATCH v6 3/3] .azurepipelines: Install code coverage tool
I might be misunderstanding the problem, but can you merge the XML file from each package into one and then pass that to the "PublishCodeCoverageResult" task?
Perhaps with something like this
https://github.com/danielpalme/ReportGenerator
Thanks,
Michael
On 12/20/2022 8:32 PM, Guo, Gua wrote:
> @Kinney, Michael D <mailto:michael.d.kinney@intel.com>
>
> I assume you want to see pipeline "Code Coverage" like below “3. Azure
> Pipeline Change” diagram.
>
> But it will have a limitation. Current Edk2 CI/CD design is use
> “Ubuntu GCC PR5” and “Windows VS2019 PR” that include each subtask for
> each package. And each package will have itself Coverage Data, but the
> design can not be covered by Azure API “PublishCodeCoverageResult”.
>
> That’s why I don’t integrated it into the change.
>
> 1. Azure API doc:
> https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/reference/publish-code-coverage-results-v1?view=azure-pipelines
>
> <https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/refere
> nce/publish-code-coverage-results-v1?view=azure-pipelines>
>
> 2. Code Change:
>
> 3. Azure Pipeline Change:
>
> VS2019
>
>
>
> GCC5
>
> https://github.com/tianocore/edk2/pull/3349/checks?check_run_id=102204
> 22452
> <https://github.com/tianocore/edk2/pull/3349/checks?check_run_id=10220
> 422452>
>
>
>
> https://github.com/tianocore/edk2/pull/3349/checks?check_run_id=102206
> 15640
> <https://github.com/tianocore/edk2/pull/3349/checks?check_run_id=10220
> 615640>
>
> https://dev.azure.com/tianocore/edk2-ci/_build/results?buildId=73952&_
> a=summary&tab=ms.vss-codecoverage-web.code-coverage-details&view=codec
> overage-tab
> <https://dev.azure.com/tianocore/edk2-ci/_build/results?buildId=73952&
> _a=summary&tab=ms.vss-codecoverage-web.code-coverage-details&view=code
> coverage-tab>
>
>
>
> https://dev.azure.com/tianocore/edk2-ci/_build/results?buildId=73951&_
> a=summary&tab=ms.vss-codecoverage-web.code-coverage-details&view=codec
> overage-tab
> <https://dev.azure.com/tianocore/edk2-ci/_build/results?buildId=73951&
> _a=summary&tab=ms.vss-codecoverage-web.code-coverage-details&view=code
> coverage-tab>
>
> Thanks,
>
> Gua
>
> -----Original Message-----
> From: Kinney, Michael D <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>
> Sent: Wednesday, December 21, 2022 5:23 AM
> To: Guo, Gua <gua.guo@intel.com<mailto:gua.guo@intel.com>>; devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Kinney,
> Michael D <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>; Sean Brogan
> <sean.brogan@microsoft.com<mailto:sean.brogan@microsoft.com>>; Kubacki, Michael
> <michael.kubacki@microsoft.com<mailto:michael.kubacki@microsoft.com>>
> Cc: Sean Brogan <sean.brogan@microsoft.com<mailto:sean.brogan@microsoft.com>>; Barkelew, Bret
> <bret.barkelew@microsoft.com<mailto:bret.barkelew@microsoft.com>>; Gao, Liming <gaoliming@byosoft.com.cn<mailto:gaoliming@byosoft.com.cn>>;
> Kinney, Michael D <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>
> Subject: RE: [PATCH v6 3/3] .azurepipelines: Install code coverage
> tool
>
> Hi Gua,
>
> I created a PR with this series.
>
> https://github.com/tianocore/edk2/pull/3795
> <https://github.com/tianocore/edk2/pull/3795>
>
> When I follow link to Azure Pipelines for the GCC builds, I can see
> the coverage.xml file in the ZIP file of artifacts.
>
> https://dev.azure.com/tianocore/edk2-ci/_build/results?buildId=73664&v
> iew=results
> <https://dev.azure.com/tianocore/edk2-ci/_build/results?buildId=73664&
> view=results>
>
> But I do not see an information from Azure Pielines view on code
> coverage or a way to view the lines covered from the Azure Pipelines
> web pages.
>
> Are there additional items to enable?
>
> Mike
>
> > -----Original Message-----
>
> > From: Guo, Gua <gua.guo@intel.com <mailto:gua.guo@intel.com<mailto:gua.guo@intel.com%20%3cmailto:gua.guo@intel.com>>>
>
> > Sent: Tuesday, December 13, 2022 7:20 PM
>
> > To: devel@edk2.groups.io<mailto:devel@edk2.groups.io> <mailto:devel@edk2.groups.io>
>
> > Cc: Guo, Gua <gua.guo@intel.com <mailto:gua.guo@intel.com<mailto:gua.guo@intel.com%20%3cmailto:gua.guo@intel.com>>>; Sean
> Brogan
>
> > <sean.brogan@microsoft.com <mailto:sean.brogan@microsoft.com<mailto:sean.brogan@microsoft.com%20%3cmailto:sean.brogan@microsoft.com>>>;
> Barkelew, Bret
>
> > <bret.barkelew@microsoft.com <mailto:bret.barkelew@microsoft.com<mailto:bret.barkelew@microsoft.com%20%3cmailto:bret.barkelew@microsoft.com>>>;
> Kinney, Michael D
>
> > <michael.d.kinney@intel.com <mailto:michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com%20%3cmailto:michael.d.kinney@intel.com>>>;
> Gao, Liming <gaoliming@byosoft.com.cn
> <mailto:gaoliming@byosoft.com.cn>>
>
> > Subject: [PATCH v6 3/3] .azurepipelines: Install code coverage tool
>
> >
>
> > From: Gua Guo <gua.guo@intel.com <mailto:gua.guo@intel.com<mailto:gua.guo@intel.com%20%3cmailto:gua.guo@intel.com>>>
>
> >
>
> > For Windows add below tool for code coverage 1. OpenCppCoverage:
>
> > parsing pdb file to generate coverage data 2. pycobertura: show up
>
> > html format data for coverage data
>
> >
>
> > For Linux add below tool for code coverage 1. lcov: parsing gcda
> gcno
>
> > file to generate coverage data 2. lcov-cobertura: convert coverage
>
> > data to cobertura format 3. pycobertura: show up html format data
> for
>
> > coverage data
>
> >
>
> > Cc: Sean Brogan <sean.brogan@microsoft.com
> <mailto:sean.brogan@microsoft.com>>
>
> > Cc: Bret Barkelew <Bret.Barkelew@microsoft.com
> <mailto:Bret.Barkelew@microsoft.com>>
>
> > Cc: Michael D Kinney <michael.d.kinney@intel.com
> <mailto:michael.d.kinney@intel.com>>
>
> > Cc: Liming Gao <gaoliming@byosoft.com.cn
> <mailto:gaoliming@byosoft.com.cn>>
>
> > Signed-off-by: Gua Guo <gua.guo@intel.com
> <mailto:gua.guo@intel.com>>
>
> > ---
>
> > .azurepipelines/Ubuntu-GCC5.yml | 5 ++++-
>
> > .azurepipelines/Windows-VS2019.yml | 5 +++++
>
> > .azurepipelines/templates/pr-gate-build-job.yml | 2 ++
>
> > .azurepipelines/templates/pr-gate-steps.yml | 4 ++++
>
> > pip-requirements.txt | 2 ++
>
> > 5 files changed, 17 insertions(+), 1 deletion(-)
>
> >
>
> > diff --git a/.azurepipelines/Ubuntu-GCC5.yml
>
> > b/.azurepipelines/Ubuntu-GCC5.yml index 1acd8d2a46..f83951eeaf
> 100644
>
> > --- a/.azurepipelines/Ubuntu-GCC5.yml
>
> > +++ b/.azurepipelines/Ubuntu-GCC5.yml
>
> > @@ -19,4 +19,7 @@ jobs:
>
> > tool_chain_tag: 'GCC5'
>
> >
>
> > vm_image: 'ubuntu-latest'
>
> >
>
> > arch_list: "IA32,X64,ARM,AARCH64,RISCV64,LOONGARCH64"
>
> >
>
> > -
>
> >
>
> > + extra_install_step:
>
> >
>
> > + - bash: sudo apt-get install -y lcov
>
> >
>
> > + displayName: Install Code Coverage Tools
>
> >
>
> > + condition: and(gt(variables.pkg_count, 0), succeeded())
>
> >
>
> > diff --git a/.azurepipelines/Windows-VS2019.yml
>
> > b/.azurepipelines/Windows-VS2019.yml
>
> > index e4bd4b1d22..c07e5bb434 100644
>
> > --- a/.azurepipelines/Windows-VS2019.yml
>
> > +++ b/.azurepipelines/Windows-VS2019.yml
>
> > @@ -18,3 +18,8 @@ jobs:
>
> > tool_chain_tag: 'VS2019'
>
> >
>
> > vm_image: 'windows-2019'
>
> >
>
> > arch_list: "IA32,X64"
>
> >
>
> > + extra_install_step:
>
> >
>
> > + - powershell: choco install opencppcoverage; Write-Host
> "##vso[task.prependpath]C:\Program Files\OpenCppCoverage"
>
> >
>
> > + displayName: Install Code Coverage Tool
>
> >
>
> > + condition: and(gt(variables.pkg_count, 0), succeeded())
>
> >
>
> > +
>
> >
>
> > diff --git a/.azurepipelines/templates/pr-gate-build-job.yml
>
> > b/.azurepipelines/templates/pr-gate-build-job.yml
>
> > index 7f88b41dc8..0ff1ce2952 100644
>
> > --- a/.azurepipelines/templates/pr-gate-build-job.yml
>
> > +++ b/.azurepipelines/templates/pr-gate-build-job.yml
>
> > @@ -12,6 +12,7 @@ parameters:
>
> > tool_chain_tag: ''
>
> >
>
> > vm_image: ''
>
> >
>
> > arch_list: ''
>
> >
>
> > + extra_install_step: []
>
> >
>
> >
>
> >
>
> > # Build step
>
> >
>
> > jobs:
>
> >
>
> > @@ -77,3 +78,4 @@ jobs:
>
> > build_pkgs: $(Build.Pkgs)
>
> >
>
> > build_targets: $(Build.Targets)
>
> >
>
> > build_archs: ${{ parameters.arch_list }}
>
> >
>
> > + extra_install_step: ${{ parameters.extra_install_step }}
>
> >
>
> > diff --git a/.azurepipelines/templates/pr-gate-steps.yml
>
> > b/.azurepipelines/templates/pr-gate-steps.yml
>
> > index cb431e53fc..080f60aea6 100644
>
> > --- a/.azurepipelines/templates/pr-gate-steps.yml
>
> > +++ b/.azurepipelines/templates/pr-gate-steps.yml
>
> > @@ -12,6 +12,7 @@ parameters:
>
> > build_pkgs: ''
>
> >
>
> > build_targets: ''
>
> >
>
> > build_archs: ''
>
> >
>
> > + extra_install_step: []
>
> >
>
> >
>
> >
>
> > steps:
>
> >
>
> > - checkout: self
>
> >
>
> > @@ -37,6 +38,8 @@ steps:
>
> > displayName: fetch target branch
>
> >
>
> > condition: eq(variables['Build.Reason'], 'PullRequest')
>
> >
>
> >
>
> >
>
> > +- ${{ parameters.extra_install_step }}
>
> >
>
> > +
>
> >
>
> > # trim the package list if this is a PR
>
> >
>
> > - task: CmdLine@1
>
> >
>
> > displayName: Check if ${{ parameters.build_pkgs }} need testing
>
> >
>
> > @@ -125,6 +128,7 @@ steps:
>
> > TestSuites.xml
>
> >
>
> > **/BUILD_TOOLS_REPORT.html
>
> >
>
> > **/OVERRIDELOG.TXT
>
> >
>
> > + coverage.xml
>
> >
>
> > flattenFolders: true
>
> >
>
> > condition: succeededOrFailed()
>
> >
>
> >
>
> >
>
> > diff --git a/pip-requirements.txt b/pip-requirements.txt index
>
> > be8c7a1c37..4ffcadddd8 100644
>
> > --- a/pip-requirements.txt
>
> > +++ b/pip-requirements.txt
>
> > @@ -16,3 +16,5 @@ edk2-pytool-library==0.12.1
>
> > edk2-pytool-extensions~=0.20.0
>
> >
>
> > edk2-basetools==0.1.39
>
> >
>
> > antlr4-python3-runtime==4.7.1
>
> >
>
> > +lcov-cobertura==2.0.2
>
> >
>
> > +
>
> >
>
> > --
>
> > 2.31.1.windows.1
>
>
[-- Attachment #1.2: Type: text/html, Size: 40574 bytes --]
[-- Attachment #2: image001.png --]
[-- Type: image/png, Size: 66947 bytes --]
next prev parent reply other threads:[~2022-12-21 4:20 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-12-14 3:19 [PATCH v6 0/3] Add code coverage support for GCC/MSVC Guo, Gua
2022-12-14 3:19 ` [PATCH v6 1/3] UnitTestFrameworkPkg: Add code coverage support for GCC Guo, Gua
2022-12-14 3:19 ` [PATCH v6 2/3] BaseTools/Plugin: Add coverage support for Unit Test Guo, Gua
2022-12-14 3:19 ` [PATCH v6 3/3] .azurepipelines: Install code coverage tool Guo, Gua
2022-12-20 21:23 ` Michael D Kinney
2022-12-21 1:32 ` Guo, Gua
2022-12-21 3:11 ` [edk2-devel] " Michael Kubacki
2022-12-21 4:20 ` Guo, Gua [this message]
2022-12-21 4:29 ` Michael Kubacki
2022-12-21 6:19 ` Guo, Gua
2022-12-21 3:17 ` Michael D Kinney
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-list from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=BL1PR11MB5478EBC66140F57EACB2D067EFEB9@BL1PR11MB5478.namprd11.prod.outlook.com \
--to=devel@edk2.groups.io \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox