From: "Guo, Gua" <gua.guo@intel.com>
To: Michael Kubacki <mikuback@linux.microsoft.com>,
"devel@edk2.groups.io" <devel@edk2.groups.io>,
"Kinney, Michael D" <michael.d.kinney@intel.com>,
Sean Brogan <sean.brogan@microsoft.com>,
"Kubacki, Michael" <michael.kubacki@microsoft.com>
Cc: "Barkelew, Bret" <bret.barkelew@microsoft.com>,
"Gao, Liming" <gaoliming@byosoft.com.cn>
Subject: Re: [edk2-devel] [PATCH v6 3/3] .azurepipelines: Install code coverage tool
Date: Wed, 21 Dec 2022 06:19:13 +0000 [thread overview]
Message-ID: <BL1PR11MB54783E6AD409258F91069EB7EFEB9@BL1PR11MB5478.namprd11.prod.outlook.com> (raw)
In-Reply-To: <1b877ac3-186b-a12d-a040-99595aa83164@linux.microsoft.com>
[-- Attachment #1.1: Type: text/plain, Size: 19256 bytes --]
I think I may need to have the access right to install pipeline by the step. I guess I need to have enough access right to control https://dev.azure.com/tianocore/edk2-ci to install DevOps Server Extensions.
[cid:image001.png@01D91547.2CAA9B60]
Thanks,
Gua
-----Original Message-----
From: Michael Kubacki <mikuback@linux.microsoft.com>
Sent: Wednesday, December 21, 2022 12:30 PM
To: devel@edk2.groups.io; Guo, Gua <gua.guo@intel.com>; Kinney, Michael D <michael.d.kinney@intel.com>; Sean Brogan <sean.brogan@microsoft.com>; Kubacki, Michael <michael.kubacki@microsoft.com>
Cc: Barkelew, Bret <bret.barkelew@microsoft.com>; Gao, Liming <gaoliming@byosoft.com.cn>
Subject: Re: [edk2-devel] [PATCH v6 3/3] .azurepipelines: Install code coverage tool
I understand the problem. Have you looked into tools like the one I linked? It also has a DevOps extension - https://marketplace.visualstudio.com/items?itemName=Palmmedia.reportgenerator.
I have not looked at it closely, but at a high-level, it appears to at least attempt to address this problem.
Thanks,
Michael
On 12/20/2022 11:20 PM, Guo, Gua wrote:
> @Kubacki, Michael <mailto:michael.kubacki@microsoft.com>
>
> Try to see the block diagram "Current behavior" vs "Expected behavior".
> We find a way to transfer “Current behavior” to “Expected behavior” if
> we want to support pipeline tab “Code Coverage”. Hope it’s helpful for
> you to realize the limitation.
>
> Thanks,
>
> Gua
>
> -----Original Message-----
> From: Michael Kubacki <mikuback@linux.microsoft.com<mailto:mikuback@linux.microsoft.com>>
> Sent: Wednesday, December 21, 2022 11:11 AM
> To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Guo, Gua <gua.guo@intel.com<mailto:gua.guo@intel.com>>; Kinney,
> Michael D <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>; Sean Brogan
> <sean.brogan@microsoft.com<mailto:sean.brogan@microsoft.com>>; Kubacki, Michael
> <michael.kubacki@microsoft.com<mailto:michael.kubacki@microsoft.com>>
> Cc: Barkelew, Bret <bret.barkelew@microsoft.com<mailto:bret.barkelew@microsoft.com>>; Gao, Liming
> <gaoliming@byosoft.com.cn<mailto:gaoliming@byosoft.com.cn>>
> Subject: Re: [edk2-devel] [PATCH v6 3/3] .azurepipelines: Install code
> coverage tool
>
> I might be misunderstanding the problem, but can you merge the XML
> file from each package into one and then pass that to the
> "PublishCodeCoverageResult" task?
>
> Perhaps with something like this
>
> https://github.com/danielpalme/ReportGenerator
> <https://github.com/danielpalme/ReportGenerator>
>
> Thanks,
>
> Michael
>
> On 12/20/2022 8:32 PM, Guo, Gua wrote:
>
> > @Kinney, Michael D <mailto:michael.d.kinney@intel.com
> <mailto:michael.d.kinney@intel.com>>
>
> >
>
> > I assume you want to see pipeline "Code Coverage" like below “3.
> Azure
>
> > Pipeline Change” diagram.
>
> >
>
> > But it will have a limitation. Current Edk2 CI/CD design is use
>
> > “Ubuntu GCC PR5” and “Windows VS2019 PR” that include each subtask
> for
>
> > each package. And each package will have itself Coverage Data, but
> the
>
> > design can not be covered by Azure API “PublishCodeCoverageResult”.
>
> >
>
> > That’s why I don’t integrated it into the change.
>
> >
>
> > 1. Azure API doc:
>
> >
> https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/referen
> ce/publish-code-coverage-results-v1?view=azure-pipelines
> <https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/refere
> nce/publish-code-coverage-results-v1?view=azure-pipelines>
>
> >
>
> >
> <https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/refere
>
> > nce/publish-code-coverage-results-v1?view=azure-pipelines>
>
> >
>
> > 2. Code Change:
>
> >
>
> > 3. Azure Pipeline Change:
>
> >
>
> > VS2019
>
> >
>
> >
>
> >
>
> > GCC5
>
> >
>
> >
> https://github.com/tianocore/edk2/pull/3349/checks?check_run_id=102204
> <https://github.com/tianocore/edk2/pull/3349/checks?check_run_id=10220
> 4>
>
> > 22452
>
> >
> <https://github.com/tianocore/edk2/pull/3349/checks?check_run_id=10220
>
> > 422452>
>
> >
>
> >
>
> >
>
> >
> https://github.com/tianocore/edk2/pull/3349/checks?check_run_id=102206
> <https://github.com/tianocore/edk2/pull/3349/checks?check_run_id=10220
> 6>
>
> > 15640
>
> >
> <https://github.com/tianocore/edk2/pull/3349/checks?check_run_id=10220
>
> > 615640>
>
> >
>
> >
> https://dev.azure.com/tianocore/edk2-ci/_build/results?buildId=73952&_
> <https://dev.azure.com/tianocore/edk2-ci/_build/results?buildId=73952&
> _>
>
> >
> a=summary&tab=ms.vss-codecoverage-web.code-coverage-details&view=codec
>
> > overage-tab
>
> >
> <https://dev.azure.com/tianocore/edk2-ci/_build/results?buildId=73952&
>
> >
> _a=summary&tab=ms.vss-codecoverage-web.code-coverage-details&view=code
>
> > coverage-tab>
>
> >
>
> >
>
> >
>
> >
> https://dev.azure.com/tianocore/edk2-ci/_build/results?buildId=73951&_
> <https://dev.azure.com/tianocore/edk2-ci/_build/results?buildId=73951&
> _>
>
> >
> a=summary&tab=ms.vss-codecoverage-web.code-coverage-details&view=codec
>
> > overage-tab
>
> >
> <https://dev.azure.com/tianocore/edk2-ci/_build/results?buildId=73951&
>
> >
> _a=summary&tab=ms.vss-codecoverage-web.code-coverage-details&view=code
>
> > coverage-tab>
>
> >
>
> > Thanks,
>
> >
>
> > Gua
>
> >
>
> > -----Original Message-----
>
> > From: Kinney, Michael D <michael.d.kinney@intel.com
> <mailto:michael.d.kinney@intel.com>>
>
> > Sent: Wednesday, December 21, 2022 5:23 AM
>
> > To: Guo, Gua <gua.guo@intel.com <mailto:gua.guo@intel.com<mailto:gua.guo@intel.com%20%3cmailto:gua.guo@intel.com>>>;
> devel@edk2.groups.io<mailto:devel@edk2.groups.io> <mailto:devel@edk2.groups.io>; Kinney,
>
> > Michael D <michael.d.kinney@intel.com
> <mailto:michael.d.kinney@intel.com>>; Sean Brogan
>
> > <sean.brogan@microsoft.com <mailto:sean.brogan@microsoft.com<mailto:sean.brogan@microsoft.com%20%3cmailto:sean.brogan@microsoft.com>>>;
> Kubacki, Michael
>
> > <michael.kubacki@microsoft.com
> <mailto:michael.kubacki@microsoft.com>>
>
> > Cc: Sean Brogan <sean.brogan@microsoft.com
> <mailto:sean.brogan@microsoft.com>>; Barkelew, Bret
>
> > <bret.barkelew@microsoft.com <mailto:bret.barkelew@microsoft.com<mailto:bret.barkelew@microsoft.com%20%3cmailto:bret.barkelew@microsoft.com>>>;
> Gao, Liming <gaoliming@byosoft.com.cn
> <mailto:gaoliming@byosoft.com.cn>>;
>
> > Kinney, Michael D <michael.d.kinney@intel.com
> <mailto:michael.d.kinney@intel.com>>
>
> > Subject: RE: [PATCH v6 3/3] .azurepipelines: Install code coverage
>
> > tool
>
> >
>
> > Hi Gua,
>
> >
>
> > I created a PR with this series.
>
> >
>
> > https://github.com/tianocore/edk2/pull/3795
> <https://github.com/tianocore/edk2/pull/3795>
>
> > <https://github.com/tianocore/edk2/pull/3795
> <https://github.com/tianocore/edk2/pull/3795>>
>
> >
>
> > When I follow link to Azure Pipelines for the GCC builds, I can see
>
> > the coverage.xml file in the ZIP file of artifacts.
>
> >
>
> >
> https://dev.azure.com/tianocore/edk2-ci/_build/results?buildId=73664&v
> <https://dev.azure.com/tianocore/edk2-ci/_build/results?buildId=73664&
> v>
>
> > iew=results
>
> >
> <https://dev.azure.com/tianocore/edk2-ci/_build/results?buildId=73664&
>
> > view=results>
>
> >
>
> > But I do not see an information from Azure Pielines view on code
>
> > coverage or a way to view the lines covered from the Azure
> Pipelines
>
> > web pages.
>
> >
>
> > Are there additional items to enable?
>
> >
>
> > Mike
>
> >
>
> > > -----Original Message-----
>
> >
>
> > > From: Guo, Gua <gua.guo@intel.com <mailto:gua.guo@intel.com
> <mailto:gua.guo@intel.com%20%3cmailto:gua.guo@intel.com>>>
>
> >
>
> > > Sent: Tuesday, December 13, 2022 7:20 PM
>
> >
>
> > > To: devel@edk2.groups.io<mailto:devel@edk2.groups.io> <mailto:devel@edk2.groups.io>
> <mailto:devel@edk2.groups.io <mailto:devel@edk2.groups.io<mailto:devel@edk2.groups.io%20%3cmailto:devel@edk2.groups.io>>>
>
> >
>
> > > Cc: Guo, Gua <gua.guo@intel.com <mailto:gua.guo@intel.com
> <mailto:gua.guo@intel.com%20%3cmailto:gua.guo@intel.com>>>; Sean
>
> > Brogan
>
> >
>
> > > <sean.brogan@microsoft.com <mailto:sean.brogan@microsoft.com
> <mailto:sean.brogan@microsoft.com%20%3cmailto:sean.brogan@microsoft.co
> m>>>;
>
> > Barkelew, Bret
>
> >
>
> > > <bret.barkelew@microsoft.com <mailto:bret.barkelew@microsoft.com
> <mailto:bret.barkelew@microsoft.com%20%3cmailto:bret.barkelew@microsof
> t.com>>>;
>
> > Kinney, Michael D
>
> >
>
> > > <michael.d.kinney@intel.com <mailto:michael.d.kinney@intel.com
> <mailto:michael.d.kinney@intel.com%20%3cmailto:michael.d.kinney@intel.
> com>>>;
>
> > Gao, Liming <gaoliming@byosoft.com.cn
>
> > <mailto:gaoliming@byosoft.com.cn
> <mailto:gaoliming@byosoft.com.cn>>>
>
> >
>
> > > Subject: [PATCH v6 3/3] .azurepipelines: Install code coverage
> tool
>
> >
>
> > >
>
> >
>
> > > From: Gua Guo <gua.guo@intel.com <mailto:gua.guo@intel.com
> <mailto:gua.guo@intel.com%20%3cmailto:gua.guo@intel.com>>>
>
> >
>
> > >
>
> >
>
> > > For Windows add below tool for code coverage 1. OpenCppCoverage:
>
> >
>
> > > parsing pdb file to generate coverage data 2. pycobertura: show
> up
>
> >
>
> > > html format data for coverage data
>
> >
>
> > >
>
> >
>
> > > For Linux add below tool for code coverage 1. lcov: parsing gcda
>
> > gcno
>
> >
>
> > > file to generate coverage data 2. lcov-cobertura: convert
> coverage
>
> >
>
> > > data to cobertura format 3. pycobertura: show up html format
> data
>
> > for
>
> >
>
> > > coverage data
>
> >
>
> > >
>
> >
>
> > > Cc: Sean Brogan <sean.brogan@microsoft.com
>
> > <mailto:sean.brogan@microsoft.com
> <mailto:sean.brogan@microsoft.com>>>
>
> >
>
> > > Cc: Bret Barkelew <Bret.Barkelew@microsoft.com
>
> > <mailto:Bret.Barkelew@microsoft.com
> <mailto:Bret.Barkelew@microsoft.com>>>
>
> >
>
> > > Cc: Michael D Kinney <michael.d.kinney@intel.com
>
> > <mailto:michael.d.kinney@intel.com
> <mailto:michael.d.kinney@intel.com>>>
>
> >
>
> > > Cc: Liming Gao <gaoliming@byosoft.com.cn
>
> > <mailto:gaoliming@byosoft.com.cn
> <mailto:gaoliming@byosoft.com.cn>>>
>
> >
>
> > > Signed-off-by: Gua Guo <gua.guo@intel.com
>
> > <mailto:gua.guo@intel.com <mailto:gua.guo@intel.com<mailto:gua.guo@intel.com%20%3cmailto:gua.guo@intel.com>>>>
>
> >
>
> > > ---
>
> >
>
> > > .azurepipelines/Ubuntu-GCC5.yml | 5 ++++-
>
> >
>
> > > .azurepipelines/Windows-VS2019.yml | 5 +++++
>
> >
>
> > > .azurepipelines/templates/pr-gate-build-job.yml | 2 ++
>
> >
>
> > > .azurepipelines/templates/pr-gate-steps.yml | 4 ++++
>
> >
>
> > > pip-requirements.txt | 2 ++
>
> >
>
> > > 5 files changed, 17 insertions(+), 1 deletion(-)
>
> >
>
> > >
>
> >
>
> > > diff --git a/.azurepipelines/Ubuntu-GCC5.yml
>
> >
>
> > > b/.azurepipelines/Ubuntu-GCC5.yml index 1acd8d2a46..f83951eeaf
>
> > 100644
>
> >
>
> > > --- a/.azurepipelines/Ubuntu-GCC5.yml
>
> >
>
> > > +++ b/.azurepipelines/Ubuntu-GCC5.yml
>
> >
>
> > > @@ -19,4 +19,7 @@ jobs:
>
> >
>
> > > tool_chain_tag: 'GCC5'
>
> >
>
> > >
>
> >
>
> > > vm_image: 'ubuntu-latest'
>
> >
>
> > >
>
> >
>
> > > arch_list: "IA32,X64,ARM,AARCH64,RISCV64,LOONGARCH64"
>
> >
>
> > >
>
> >
>
> > > -
>
> >
>
> > >
>
> >
>
> > > + extra_install_step:
>
> >
>
> > >
>
> >
>
> > > + - bash: sudo apt-get install -y lcov
>
> >
>
> > >
>
> >
>
> > > + displayName: Install Code Coverage Tools
>
> >
>
> > >
>
> >
>
> > > + condition: and(gt(variables.pkg_count, 0), succeeded())
>
> >
>
> > >
>
> >
>
> > > diff --git a/.azurepipelines/Windows-VS2019.yml
>
> >
>
> > > b/.azurepipelines/Windows-VS2019.yml
>
> >
>
> > > index e4bd4b1d22..c07e5bb434 100644
>
> >
>
> > > --- a/.azurepipelines/Windows-VS2019.yml
>
> >
>
> > > +++ b/.azurepipelines/Windows-VS2019.yml
>
> >
>
> > > @@ -18,3 +18,8 @@ jobs:
>
> >
>
> > > tool_chain_tag: 'VS2019'
>
> >
>
> > >
>
> >
>
> > > vm_image: 'windows-2019'
>
> >
>
> > >
>
> >
>
> > > arch_list: "IA32,X64"
>
> >
>
> > >
>
> >
>
> > > + extra_install_step:
>
> >
>
> > >
>
> >
>
> > > + - powershell: choco install opencppcoverage; Write-Host
>
> > "##vso[task.prependpath]C:\Program Files\OpenCppCoverage"
>
> >
>
> > >
>
> >
>
> > > + displayName: Install Code Coverage Tool
>
> >
>
> > >
>
> >
>
> > > + condition: and(gt(variables.pkg_count, 0), succeeded())
>
> >
>
> > >
>
> >
>
> > > +
>
> >
>
> > >
>
> >
>
> > > diff --git a/.azurepipelines/templates/pr-gate-build-job.yml
>
> >
>
> > > b/.azurepipelines/templates/pr-gate-build-job.yml
>
> >
>
> > > index 7f88b41dc8..0ff1ce2952 100644
>
> >
>
> > > --- a/.azurepipelines/templates/pr-gate-build-job.yml
>
> >
>
> > > +++ b/.azurepipelines/templates/pr-gate-build-job.yml
>
> >
>
> > > @@ -12,6 +12,7 @@ parameters:
>
> >
>
> > > tool_chain_tag: ''
>
> >
>
> > >
>
> >
>
> > > vm_image: ''
>
> >
>
> > >
>
> >
>
> > > arch_list: ''
>
> >
>
> > >
>
> >
>
> > > + extra_install_step: []
>
> >
>
> > >
>
> >
>
> > >
>
> >
>
> > >
>
> >
>
> > > # Build step
>
> >
>
> > >
>
> >
>
> > > jobs:
>
> >
>
> > >
>
> >
>
> > > @@ -77,3 +78,4 @@ jobs:
>
> >
>
> > > build_pkgs: $(Build.Pkgs)
>
> >
>
> > >
>
> >
>
> > > build_targets: $(Build.Targets)
>
> >
>
> > >
>
> >
>
> > > build_archs: ${{ parameters.arch_list }}
>
> >
>
> > >
>
> >
>
> > > + extra_install_step: ${{ parameters.extra_install_step }}
>
> >
>
> > >
>
> >
>
> > > diff --git a/.azurepipelines/templates/pr-gate-steps.yml
>
> >
>
> > > b/.azurepipelines/templates/pr-gate-steps.yml
>
> >
>
> > > index cb431e53fc..080f60aea6 100644
>
> >
>
> > > --- a/.azurepipelines/templates/pr-gate-steps.yml
>
> >
>
> > > +++ b/.azurepipelines/templates/pr-gate-steps.yml
>
> >
>
> > > @@ -12,6 +12,7 @@ parameters:
>
> >
>
> > > build_pkgs: ''
>
> >
>
> > >
>
> >
>
> > > build_targets: ''
>
> >
>
> > >
>
> >
>
> > > build_archs: ''
>
> >
>
> > >
>
> >
>
> > > + extra_install_step: []
>
> >
>
> > >
>
> >
>
> > >
>
> >
>
> > >
>
> >
>
> > > steps:
>
> >
>
> > >
>
> >
>
> > > - checkout: self
>
> >
>
> > >
>
> >
>
> > > @@ -37,6 +38,8 @@ steps:
>
> >
>
> > > displayName: fetch target branch
>
> >
>
> > >
>
> >
>
> > > condition: eq(variables['Build.Reason'], 'PullRequest')
>
> >
>
> > >
>
> >
>
> > >
>
> >
>
> > >
>
> >
>
> > > +- ${{ parameters.extra_install_step }}
>
> >
>
> > >
>
> >
>
> > > +
>
> >
>
> > >
>
> >
>
> > > # trim the package list if this is a PR
>
> >
>
> > >
>
> >
>
> > > - task: CmdLine@1
>
> >
>
> > >
>
> >
>
> > > displayName: Check if ${{ parameters.build_pkgs }} need
> testing
>
> >
>
> > >
>
> >
>
> > > @@ -125,6 +128,7 @@ steps:
>
> >
>
> > > TestSuites.xml
>
> >
>
> > >
>
> >
>
> > > **/BUILD_TOOLS_REPORT.html
>
> >
>
> > >
>
> >
>
> > > **/OVERRIDELOG.TXT
>
> >
>
> > >
>
> >
>
> > > + coverage.xml
>
> >
>
> > >
>
> >
>
> > > flattenFolders: true
>
> >
>
> > >
>
> >
>
> > > condition: succeededOrFailed()
>
> >
>
> > >
>
> >
>
> > >
>
> >
>
> > >
>
> >
>
> > > diff --git a/pip-requirements.txt b/pip-requirements.txt index
>
> >
>
> > > be8c7a1c37..4ffcadddd8 100644
>
> >
>
> > > --- a/pip-requirements.txt
>
> >
>
> > > +++ b/pip-requirements.txt
>
> >
>
> > > @@ -16,3 +16,5 @@ edk2-pytool-library==0.12.1
>
> >
>
> > > edk2-pytool-extensions~=0.20.0
>
> >
>
> > >
>
> >
>
> > > edk2-basetools==0.1.39
>
> >
>
> > >
>
> >
>
> > > antlr4-python3-runtime==4.7.1
>
> >
>
> > >
>
> >
>
> > > +lcov-cobertura==2.0.2
>
> >
>
> > >
>
> >
>
> > > +
>
> >
>
> > >
>
> >
>
> > > --
>
> >
>
> > > 2.31.1.windows.1
>
> >
>
>
[-- Attachment #1.2: Type: text/html, Size: 75437 bytes --]
[-- Attachment #2: image001.png --]
[-- Type: image/png, Size: 45068 bytes --]
next prev parent reply other threads:[~2022-12-21 6:19 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-12-14 3:19 [PATCH v6 0/3] Add code coverage support for GCC/MSVC Guo, Gua
2022-12-14 3:19 ` [PATCH v6 1/3] UnitTestFrameworkPkg: Add code coverage support for GCC Guo, Gua
2022-12-14 3:19 ` [PATCH v6 2/3] BaseTools/Plugin: Add coverage support for Unit Test Guo, Gua
2022-12-14 3:19 ` [PATCH v6 3/3] .azurepipelines: Install code coverage tool Guo, Gua
2022-12-20 21:23 ` Michael D Kinney
2022-12-21 1:32 ` Guo, Gua
2022-12-21 3:11 ` [edk2-devel] " Michael Kubacki
2022-12-21 4:20 ` Guo, Gua
2022-12-21 4:29 ` Michael Kubacki
2022-12-21 6:19 ` Guo, Gua [this message]
2022-12-21 3:17 ` Michael D Kinney
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-list from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=BL1PR11MB54783E6AD409258F91069EB7EFEB9@BL1PR11MB5478.namprd11.prod.outlook.com \
--to=devel@edk2.groups.io \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox