From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by mx.groups.io with SMTP id smtpd.web11.10231.1671592277081452791 for ; Tue, 20 Dec 2022 19:11:17 -0800 Authentication-Results: mx.groups.io; dkim=fail reason="body hash did not verify" header.i=@linux.microsoft.com header.s=default header.b=cD7zk5cv; spf=pass (domain: linux.microsoft.com, ip: 13.77.154.182, mailfrom: mikuback@linux.microsoft.com) Received: from [192.168.4.22] (unknown [47.201.8.94]) by linux.microsoft.com (Postfix) with ESMTPSA id E16C620C3364; Tue, 20 Dec 2022 19:11:15 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com E16C620C3364 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1671592276; bh=UgM4ODRsomVIXnZRQaMedeiHNQnJAyJE7Bf3UFp/vHQ=; h=Date:Subject:To:Cc:References:From:In-Reply-To:From; b=cD7zk5cvG8CLNgRkRIzEafHyg22P7TU7fw/li979PMOLP8uq1E093N5ocs4herH5r fJt1/G/z4JdEOgWFIkFRXvzwJfDoo5TUIENFPCMBYM/aNftviXQxHDpprbHTZsgDVo cppvTps0J6M0h6gUsoCMT1+5ztsolYfrozxhIEzk= Message-ID: Date: Tue, 20 Dec 2022 22:11:14 -0500 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 6.2; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.13.1 Subject: Re: [edk2-devel] [PATCH v6 3/3] .azurepipelines: Install code coverage tool To: devel@edk2.groups.io, gua.guo@intel.com, "Kinney, Michael D" , Sean Brogan , "Kubacki, Michael" Cc: "Barkelew, Bret" , "Gao, Liming" References: <411e740a92fe660f0e173c6c91896ff274237391.1670987653.git.gua.guo@intel.com> From: "Michael Kubacki" In-Reply-To: Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable I might be misunderstanding the problem, but can you merge the XML file=20 from each package into one and then pass that to the=20 "PublishCodeCoverageResult" task? Perhaps with something like this=20 https://github.com/danielpalme/ReportGenerator Thanks, Michael On 12/20/2022 8:32 PM, Guo, Gua wrote: > @Kinney, Michael D >=20 > I assume you want to see pipeline "Code Coverage" like below =E2=80=9C3. = Azure=20 > Pipeline Change=E2=80=9D diagram. >=20 > But it will have a limitation. Current Edk2 CI/CD design is use =E2=80=9C= Ubuntu=20 > GCC PR5=E2=80=9D and =E2=80=9CWindows VS2019 PR=E2=80=9D that include eac= h subtask for each=20 > package. And each package will have itself Coverage Data, but the design= =20 > can not be covered by Azure API =E2=80=9CPublishCodeCoverageResult=E2=80= =9D. >=20 > That=E2=80=99s why I don=E2=80=99t integrated it into the change. >=20 > 1. Azure API doc: > https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/refere= nce/publish-code-coverage-results-v1?view=3Dazure-pipelines > >=20 > 2. Code Change: >=20 > 3. Azure Pipeline Change: >=20 > VS2019 >=20 > =09 >=20 > GCC5 >=20 > https://github.com/tianocore/edk2/pull/3349/checks?check_run_id=3D1022042= 2452=20 > >=20 > =09 >=20 > https://github.com/tianocore/edk2/pull/3349/checks?check_run_id=3D1022061= 5640=20 > >=20 > https://dev.azure.com/tianocore/edk2-ci/_build/results?buildId=3D73952&_a= =3Dsummary&tab=3Dms.vss-codecoverage-web.code-coverage-details&view=3Dcodec= overage-tab=20 > >=20 > =09 >=20 > https://dev.azure.com/tianocore/edk2-ci/_build/results?buildId=3D73951&_a= =3Dsummary&tab=3Dms.vss-codecoverage-web.code-coverage-details&view=3Dcodec= overage-tab=20 > >=20 > Thanks, >=20 > Gua >=20 > -----Original Message----- > From: Kinney, Michael D > Sent: Wednesday, December 21, 2022 5:23 AM > To: Guo, Gua ; devel@edk2.groups.io; Kinney, Michael= =20 > D ; Sean Brogan ;= =20 > Kubacki, Michael > Cc: Sean Brogan ; Barkelew, Bret=20 > ; Gao, Liming ;=20 > Kinney, Michael D > Subject: RE: [PATCH v6 3/3] .azurepipelines: Install code coverage tool >=20 > Hi Gua, >=20 > I created a PR with this series. >=20 > https://github.com/tianocore/edk2/pull/3795=20 > >=20 > When I follow link to Azure Pipelines for the GCC builds, I can see the= =20 > coverage.xml file in the ZIP file of artifacts. >=20 > https://dev.azure.com/tianocore/edk2-ci/_build/results?buildId=3D73664&vi= ew=3Dresults=20 > >=20 > But I do not see an information from Azure Pielines view on code=20 > coverage or a way to view the lines covered from the Azure Pipelines web= =20 > pages. >=20 > Are there additional items to enable? >=20 > Mike >=20 > > -----Original Message----- >=20 > > From: Guo, Gua > >=20 > > Sent: Tuesday, December 13, 2022 7:20 PM >=20 > > To: devel@edk2.groups.io >=20 > > Cc: Guo, Gua >; Sean Brog= an >=20 > > >;=20 > Barkelew, Bret >=20 > > >;=20 > Kinney, Michael D >=20 > > >;=20 > Gao, Liming > >=20 > > Subject: [PATCH v6 3/3] .azurepipelines: Install code coverage tool >=20 > > >=20 > > From: Gua Guo > >=20 > > >=20 > > For Windows add below tool for code coverage 1. OpenCppCoverage: >=20 > > parsing pdb file to generate coverage data 2. pycobertura: show up >=20 > > html format data for coverage data >=20 > > >=20 > > For Linux add below tool for code coverage 1. lcov: parsing gcda gcno >=20 > > file to generate coverage data 2. lcov-cobertura: convert coverage >=20 > > data to cobertura format 3. pycobertura: show up html format data for >=20 > > coverage data >=20 > > >=20 > > Cc: Sean Brogan > >=20 > > Cc: Bret Barkelew > >=20 > > Cc: Michael D Kinney > >=20 > > Cc: Liming Gao > >=20 > > Signed-off-by: Gua Guo > >=20 > > --- >=20 > >=C2=A0 .azurepipelines/Ubuntu-GCC5.yml=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 | 5 ++++- >=20 > >=C2=A0 .azurepipelines/Windows-VS2019.yml=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 | 5 +++++ >=20 > >=C2=A0 .azurepipelines/templates/pr-gate-build-job.yml | 2 ++ >=20 > >=C2=A0 .azurepipelines/templates/pr-gate-steps.yml=C2=A0=C2=A0=C2=A0=C2= =A0 | 4 ++++ >=20 > >=C2=A0 pip-requirements.txt=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 | 2 ++ >=20 > >=C2=A0 5 files changed, 17 insertions(+), 1 deletion(-) >=20 > > >=20 > > diff --git a/.azurepipelines/Ubuntu-GCC5.yml >=20 > > b/.azurepipelines/Ubuntu-GCC5.yml index 1acd8d2a46..f83951eeaf 100644 >=20 > > --- a/.azurepipelines/Ubuntu-GCC5.yml >=20 > > +++ b/.azurepipelines/Ubuntu-GCC5.yml >=20 > > @@ -19,4 +19,7 @@ jobs: >=20 > >=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 tool_chain_tag: 'GCC5' >=20 > > >=20 > >=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 vm_image: 'ubuntu-latest' >=20 > > >=20 > >=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 arch_list: "IA32,X64,ARM,AARCH64,RISCV64= ,LOONGARCH64" >=20 > > >=20 > > - >=20 > > >=20 > > +=C2=A0=C2=A0=C2=A0 extra_install_step: >=20 > > >=20 > > +=C2=A0=C2=A0=C2=A0 - bash: sudo apt-get install -y lcov >=20 > > >=20 > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 displayName: Install Code Coverage Too= ls >=20 > > >=20 > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 condition: and(gt(variables.pkg_count,= 0), succeeded()) >=20 > > >=20 > > diff --git a/.azurepipelines/Windows-VS2019.yml >=20 > > b/.azurepipelines/Windows-VS2019.yml >=20 > > index e4bd4b1d22..c07e5bb434 100644 >=20 > > --- a/.azurepipelines/Windows-VS2019.yml >=20 > > +++ b/.azurepipelines/Windows-VS2019.yml >=20 > > @@ -18,3 +18,8 @@ jobs: >=20 > >=C2=A0 =C2=A0=C2=A0=C2=A0=C2=A0tool_chain_tag: 'VS2019' >=20 > > >=20 > >=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 vm_image: 'windows-2019' >=20 > > >=20 > >=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 arch_list: "IA32,X64" >=20 > > >=20 > > +=C2=A0=C2=A0=C2=A0 extra_install_step: >=20 > > >=20 > > +=C2=A0=C2=A0=C2=A0 - powershell: choco install opencppcoverage; Write= -Host=20 > "##vso[task.prependpath]C:\Program Files\OpenCppCoverage" >=20 > > >=20 > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 displayName: Install Code Coverage Too= l >=20 > > >=20 > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 condition: and(gt(variables.pkg_count,= 0), succeeded()) >=20 > > >=20 > > + >=20 > > >=20 > > diff --git a/.azurepipelines/templates/pr-gate-build-job.yml >=20 > > b/.azurepipelines/templates/pr-gate-build-job.yml >=20 > > index 7f88b41dc8..0ff1ce2952 100644 >=20 > > --- a/.azurepipelines/templates/pr-gate-build-job.yml >=20 > > +++ b/.azurepipelines/templates/pr-gate-build-job.yml >=20 > > @@ -12,6 +12,7 @@ parameters: >=20 > >=C2=A0=C2=A0=C2=A0 tool_chain_tag: '' >=20 > > >=20 > >=C2=A0=C2=A0=C2=A0 vm_image: '' >=20 > > >=20 > >=C2=A0=C2=A0=C2=A0 arch_list: '' >=20 > > >=20 > > +=C2=A0 extra_install_step: [] >=20 > > >=20 > > >=20 > > >=20 > >=C2=A0 # Build step >=20 > > >=20 > >=C2=A0 jobs: >=20 > > >=20 > > @@ -77,3 +78,4 @@ jobs: >=20 > >=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 build_pkgs: $(Build.Pkgs) >=20 > > >=20 > >=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 build_targets: $(Build.Targe= ts) >=20 > > >=20 > >=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 build_archs: ${{ parameters.= arch_list }} >=20 > > >=20 > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 extra_install_step: ${{ parameters.ext= ra_install_step }} >=20 > > >=20 > > diff --git a/.azurepipelines/templates/pr-gate-steps.yml >=20 > > b/.azurepipelines/templates/pr-gate-steps.yml >=20 > > index cb431e53fc..080f60aea6 100644 >=20 > > --- a/.azurepipelines/templates/pr-gate-steps.yml >=20 > > +++ b/.azurepipelines/templates/pr-gate-steps.yml >=20 > > @@ -12,6 +12,7 @@ parameters: >=20 > >=C2=A0=C2=A0=C2=A0 build_pkgs: '' >=20 > > >=20 > >=C2=A0=C2=A0=C2=A0 build_targets: '' >=20 > > >=20 > >=C2=A0=C2=A0=C2=A0 build_archs: '' >=20 > > >=20 > > +=C2=A0 extra_install_step: [] >=20 > > >=20 > > >=20 > > >=20 > >=C2=A0 steps: >=20 > > >=20 > >=C2=A0 - checkout: self >=20 > > >=20 > > @@ -37,6 +38,8 @@ steps: >=20 > >=C2=A0=C2=A0=C2=A0 displayName: fetch target branch >=20 > > >=20 > >=C2=A0=C2=A0=C2=A0 condition: eq(variables['Build.Reason'], 'PullReques= t') >=20 > > >=20 > > >=20 > > >=20 > > +- ${{ parameters.extra_install_step }} >=20 > > >=20 > > + >=20 > > >=20 > >=C2=A0 # trim the package list if this is a PR >=20 > > >=20 > >=C2=A0 - task: CmdLine@1 >=20 > > >=20 > >=C2=A0=C2=A0=C2=A0 displayName: Check if ${{ parameters.build_pkgs }} n= eed testing >=20 > > >=20 > > @@ -125,6 +128,7 @@ steps: >=20 > >=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 TestSuites.xml >=20 > > >=20 > >=C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0**/BUILD_TOOLS_REPORT.html >=20 > > >=20 > >=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 **/OVERRIDELOG.TXT >=20 > > >=20 > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 coverage.xml >=20 > > >=20 > >=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 flattenFolders: true >=20 > > >=20 > >=C2=A0=C2=A0=C2=A0 condition: succeededOrFailed() >=20 > > >=20 > > >=20 > > >=20 > > diff --git a/pip-requirements.txt b/pip-requirements.txt index >=20 > > be8c7a1c37..4ffcadddd8 100644 >=20 > > --- a/pip-requirements.txt >=20 > > +++ b/pip-requirements.txt >=20 > > @@ -16,3 +16,5 @@ edk2-pytool-library=3D=3D0.12.1 >=20 > >=C2=A0 edk2-pytool-extensions~=3D0.20.0 >=20 > > >=20 > >=C2=A0 edk2-basetools=3D=3D0.1.39 >=20 > > >=20 > >=C2=A0 antlr4-python3-runtime=3D=3D4.7.1 >=20 > > >=20 > > +lcov-cobertura=3D=3D2.0.2 >=20 > > >=20 > > + >=20 > > >=20 > > -- >=20 > > 2.31.1.windows.1 >=20 >=20