From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (NAM11-DM6-obe.outbound.protection.outlook.com [40.92.19.97]) by mx.groups.io with SMTP id smtpd.web10.1656.1663121404094754968 for ; Tue, 13 Sep 2022 19:10:04 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@outlook.com header.s=selector1 header.b=nvXrr5pe; spf=pass (domain: outlook.com, ip: 40.92.19.97, mailfrom: spbrogan@outlook.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Q744g/sg6PZpgEGQGEMQHJk+zsIBNGiua1kF02bMVWI005zFHHePKCRijcogQFLC89qbOIzD7A+YH5Ms0so1hZyNJTuFt2o/p5KJm2e+cutTxoTTkJ8ZLAl6AmhjUi0ZX93l7RHcoCwGD6MEvhROizCsNgW/AR1IFur7180RdaSzhrsvnPI3bDdjJE+AtMMxcQch5rvorgzLVV7M7opZkeOVm7dsndHeFGXob1gZvgbRWdBooSk/HXe2be2S0lE3IKOQqnYyhM1PwZYQL/IeI/2X4jM1UGUl4f0MrMTlCUcSEddlQHOwVwLhzWtdRWmWPjyEa2cc4NuN/AR0X/ElXg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=JRR6XnHeGJuGAwWHwiOkTYB9IsCHQMNFqhahJ130ukw=; b=F3ncEDBPtWYZBTswUFH6kbWYRvlTGIEea3z2vec/AREBhHVvixNmRjNKDVsk5sJOT1QawuNtRiO5C0idSnVSCEOfd2fsrrWioKRW4Fm/Lx/6SLriUBYTPwkN6BUZg2zrfBkjfr3JV3mrw2YSiBWazX8yUtChfbwZh+88/ljRtO9X9wFiLScAYnI270TuFDaLAVk87lEZQ/34fiRmw40mVa1KkGn4bj3mVMnV0A3J53VQvBaQErahi/FAM+lCypmv+fiDvIlXXJd2fa/r50tmJd6eq5V+8tBs+VoYsPJNooTMPJfe8cjDMxd+drstzNfa3wACUwX9c0FwV//qficN+g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JRR6XnHeGJuGAwWHwiOkTYB9IsCHQMNFqhahJ130ukw=; b=nvXrr5pe6penpqaiHanXxd81evmoW1DnLNWMITShLiCUaMG05TOX65R7FWjaeebaP4gHRORKYmU0VCt01dHfzPIgIoipU9Y9gdREFkZqvuQKDdMagMz7MRwqHEk5rqhi3AnGN/WI5F5phgRasNccLTXJ5VBMSbyVkXwS7zYiQlQlp90P8TIYuytlsRFFdMcdSp2JKwgyxTZyTkvRyOrTALK8HLyIBf/fh2pF/ndGxMgW/d+RkIbfegXIMxqD4KC6ihaTHrN1UBy/+QHtpilhoyzWRhi4YFMyFbEVdLxs9vyA30Bb4CPVsHODMryqKL+KcbdcI34W8myfuEJ0RcfhOQ== Received: from BY3PR19MB4900.namprd19.prod.outlook.com (2603:10b6:a03:354::11) by MN2PR19MB4013.namprd19.prod.outlook.com (2603:10b6:208:1e1::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5612.22; Wed, 14 Sep 2022 02:10:02 +0000 Received: from BY3PR19MB4900.namprd19.prod.outlook.com ([fe80::5db6:2c20:44c4:42bc]) by BY3PR19MB4900.namprd19.prod.outlook.com ([fe80::5db6:2c20:44c4:42bc%2]) with mapi id 15.20.5612.022; Wed, 14 Sep 2022 02:10:01 +0000 From: "Sean" To: "Kinney, Michael D" , "devel@edk2.groups.io" , "Guo, Gua" CC: Sean Brogan , Michael Kubacki Subject: Re: [edk2-devel] [PATCH v2] UnitTestFrameworkPkg: Add code coverage support for GCC Thread-Topic: [edk2-devel] [PATCH v2] UnitTestFrameworkPkg: Add code coverage support for GCC Thread-Index: AQHYx9WFGfNR0gPzp0CPxt8zR0N4nq3eH9kAgAAJkICAAAK1Rg== Date: Wed, 14 Sep 2022 02:10:01 +0000 Message-ID: References: <20220914010118.1643-1-gua.guo@intel.com> In-Reply-To: Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-tmn: [5hlQ0V7MB7OY+rnnzKVVwPkWy9Uiaa37] x-ms-publictraffictype: Email x-ms-traffictypediagnostic: BY3PR19MB4900:EE_|MN2PR19MB4013:EE_ x-ms-office365-filtering-correlation-id: 3e057084-f040-4ff9-66f2-08da95f63b7f x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: PNIjfell3uhTBqSqC3bEq7WFlmo/YPBVDEIlt3akNIesa7z9WjCpcsAi/9KMSgz/j/IMb+09utJnf0OTAvEn5SumvjLvMISMRQu0T4/pxSkGw9VFUx3+Fq7JHmumfV1gWgue/72tbksL0wyXqDQRjlaCML80fNA0mX943RhCvyjt4jMWfjLk37kLa9laFDSkxMBgKQXhAZlH6IsWRNZjrgWOsm/T2A+kJJza4qEc41HdPkSW1A/hm/ZlAp9yw0uvX4tfZj4DXrCW7zpKYGqu6OLl7DgH5LwpTB9cCa/czEJf7DTUvFWYuveMiE0svvPvB95wj/k2A+ue6aMi3L1t4+3ETgM11dmUBPXfVLuG9/cUDCV7Qt7G/NyAZ6HSRBCnu4k30sAG2ULezJhkwM/glugVPW1X+4AfGP+mOK+kwZqVtjO8shQpr0z8CFzf3NvgKZflcWKKpK+HusktzWDb3g+kgR9pgxh7RLHI3qZhLmfhYLKdIQXvcfzfLdv/yN61JkMDYhiYQrWn4+/+yvgzMiOhfGJq1yv3pOmkI6Hf+eyaEQeWKoCLGL2THXwxzIsTbiZcrk8+oYrB7aRyfum5d7MlQmraZgDp64zT6qO0sW0Fg4NnsnzChvmWatoIy8MRgtract4mzBh91zEnmPSqbsLV4fzYQLhK/iUnFarhhgDe5nIeM2LH+v5DfvIipO6c x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?Windows-1252?Q?6XS11SigKffaXoaQDA5DxC0hSMkog8yUpCnuWrdYesZXbBP6vVf2+hEs?= =?Windows-1252?Q?wKqWZpe2OqtNIRdBkg+A99+LdDINBZqtTrLUXunxluIOQcml1CDSRNZ9?= =?Windows-1252?Q?3kMaWKbgvpes9TKmYhuxM4m19xJwZzN0/zbNFPisBHCNrrWNhyY3I8vl?= =?Windows-1252?Q?ub2gmk1dgkT7wOR9JHDC6Kqh8O+Tfc3AdXQJH2rf/NkBCGwKTd+f6f4M?= =?Windows-1252?Q?BHFevNgrddekTsEHxRMlIhwpaT6IKW+KDNJ9n8jziYBLHGM0VJpAg58Z?= =?Windows-1252?Q?7X14m4NuK8N/AsCE702SeuKNdI/zNnHMuQrywu5Mruu3Gy2H63W7Sylb?= =?Windows-1252?Q?CpI1weoqzr/dzEdqdPhfVw5Q0HBz8VFt/KQx7UtlKrsAy27xV3b9aSDj?= =?Windows-1252?Q?wo0CP9ifxmnCKYvrzYmrvvOIf/a9Y7emDaYnuSswy1PpkcEu7Gfaa7jO?= =?Windows-1252?Q?DHe55W9mgmV0QUUbia0v1+VOIQFMKaDBvAkw5XJu0thmwe1fqVDlVlM4?= =?Windows-1252?Q?SUFUHoc48aPUS+8rVsh7YeNkv8zi11VdBdvfXvwW23sHh4yKSl6ERJZG?= =?Windows-1252?Q?XUn4ykEEJjOL7NAseQRA0WTwVEUZKFu3wkpHHU3ER9Lwp9lh2ggFX4ql?= =?Windows-1252?Q?ixr9Jh9lK5LY1fNzAP0cF+bHYYsWIw+Sy2Yr2uGVRPQANZsaMNeobxZI?= =?Windows-1252?Q?OTr5VkgOoXjv4KVKEy5885WwNhX7ulMYYzcDiDf2omy+WaFnybIIqzSe?= =?Windows-1252?Q?wa500Hg3Kk5bQegjlVLe4Assq6WRu9G/CflZL6nMoBUjZQYTM+KXb9Qo?= =?Windows-1252?Q?rIH6nhk9FYVrNYwEkIUgZyOlDAjwczSBzWlNdofsi6BH0lRQwQclfBfm?= =?Windows-1252?Q?dXNsaoW7LfOz2/rtgWMZWOu9nLdhSByAaXwIJg/4Y6cLgAgQqT6p4CP3?= =?Windows-1252?Q?jKKg7KRoNZuF3ki5O4bqZkl8AwEBoeBk5Jw+TCijN8V7sLQB32xCuDBp?= =?Windows-1252?Q?W+Pu/AwRK5R//XyTpqoTxIUT+VVvNrBJ9K0fitkc0z5Yqyym0E9iiBdr?= =?Windows-1252?Q?z5hmeUps1qBixTWAos6pba1GIpjQEAPypGYnU0u+sttvpojuO3Mjitp5?= =?Windows-1252?Q?dBWCdw9zYLykcn1iOAG8IEgU+VgYtUcv17bZt0IPrxs7O0a77iYRKU3o?= =?Windows-1252?Q?T4xbfG8qoDxxBme4NJ0Hc7h+S2CJtCWhPkLB3254TvyXL1NNW3uI8zMZ?= =?Windows-1252?Q?gB90KyOzv/tK8+gFsQnTMnpi3K5KBgKunT8WiEcVPX16bx1hGjirve/Z?= =?Windows-1252?Q?PZWJ8r8QOpGNCLtziEh9gMQdS6dV7fXn2mfoBJrmqhBobS9+15f4Bfbt?= =?Windows-1252?Q?hDZezb/jzOgxA2HiMqKUBwBF1ZMHc8AXmLE=3D?= MIME-Version: 1.0 X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: BY3PR19MB4900.namprd19.prod.outlook.com X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: 3e057084-f040-4ff9-66f2-08da95f63b7f X-MS-Exchange-CrossTenant-originalarrivaltime: 14 Sep 2022 02:10:01.8733 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR19MB4013 Content-Language: en-US Content-Type: multipart/alternative; boundary="_000_BY3PR19MB4900C77DCFA59D0C255BDDE2C8469BY3PR19MB4900namp_" --_000_BY3PR19MB4900C77DCFA59D0C255BDDE2C8469BY3PR19MB4900namp_ Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable For local scenarios the plug-in generates the html report for viewing the c= code with coverage annotations. The intermediate files are available so I= would think they could be loaded in all sorts of viewers but I haven=92t t= ried. I have just used the html and it seemed acceptable. For ci the pipeline files have been updated to publish the data to the buil= d. This allows using the online viewer to navigate c code and coverage. The= html report is also uploaded and can be downloaded for local viewing. Regarding using visual studio tool chain/compilers we had discussed that fo= r ci this is unnecessary ( since we already have coverage from gcc tool cha= in). For local users this is important. There is an easy tool you shared wi= th me that gives similar results. I didn=92t see a great way to integrate i= nto the plug-in for auto invocation but I plan to write up usage details fo= r local users. Thanks Sean ________________________________ From: Kinney, Michael D Sent: Tuesday, September 13, 2022 6:51:43 PM To: Sean Brogan ; devel@edk2.groups.io ; Guo, Gua ; Kinney, Michael D Cc: Sean Brogan ; Michael Kubacki Subject: RE: [edk2-devel] [PATCH v2] UnitTestFrameworkPkg: Add code coverag= e support for GCC Does that pytool plugin support viewing the code coverage visually in C sou= rce files? Either HTML or preferably an Visual Studio Code plugin? Can it be updated to support VS compilers tool? Mie From: Sean Brogan Sent: Tuesday, September 13, 2022 6:18 PM To: devel@edk2.groups.io; Guo, Gua Cc: Kinney, Michael D ; Sean Brogan ; Michael Kubacki Subject: Re: [edk2-devel] [PATCH v2] UnitTestFrameworkPkg: Add code coverag= e support for GCC Hi Gua Guo, I would propose that we make this easier for both the local user and the CI= system to get coverage metrics. Can you review this commit as it uses the same Edk2 plugins but adds code c= overage support with GCC5: Merged PR 5008: Initial implementation or code coverage on basecore u=85 = =B7 microsoft/mu_basecore@f6af51f (github.com) Thanks Sean On 9/13/2022 6:01 PM, Guo, Gua wrote: From: Gua Guo In order to collect code coverage after running executable file, generate *.gcda and *.gcno file that require by lcov tool to generate code coverage report. Cc: Michael D Kinney Cc: Sean Brogan Cc: Bret Barkelew Cc: Michael Kubacki Signed-off-by: Gua Guo --- UnitTestFrameworkPkg/ReadMe.md | 116 ++++++++++++++= +++++- UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc | 2 +- 2 files changed, 116 insertions(+), 2 deletions(-) diff --git a/UnitTestFrameworkPkg/ReadMe.md b/UnitTestFrameworkPkg/ReadMe.m= d index e696412cb3..93fdacfc4a 100644 --- a/UnitTestFrameworkPkg/ReadMe.md +++ b/UnitTestFrameworkPkg/ReadMe.md @@ -361,7 +361,7 @@ RUNNING TEST SUITE: Int Safe Conversions Test Suite ``` You can also, if you are so inclined, read the output from the exact insta= nce of the test that was run during -`stuart_ci_build`. The ouput file can be found on a path that looks like: +`stuart_ci_build`. The output file can be found on a path that looks like: `Build//HostTest//...result.= xml` @@ -399,6 +399,120 @@ CMOCKA_XML_FILE=3D This mode is used by the test running plugin to aggregate the results for = CI test status reporting in the web view. +### Enable Code Coverage for GCC host-based tests + +LCOV is an extension of GCOV, a GNU tool which provides information about = what parts of a program are actually executed (i.e. "covered") while runnin= g a particular test case. The extension consists of a set of scripts which = build on the textual GCOV output to implement it. + +Documentation for LCOV can be found here: +https://lcov.readthedocs.io/ + +You can follow below steps to generate your code coverage report. + + + +```bash +user@linux_machine:~/_uefi$stuart_ci_build -c .pytool/CISettings.py TOOL_C= HAIN_TAG=3DGCC5 -p MdePkg -t NOOPT +... +user@linux_machine:~/_uefi$lcov -c --rc lcov_branch_coverage=3D1 -o UnitTe= st.info -d Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/ +Capturing coverage data from Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/ +Found gcov version: 9.4.0 +Using intermediate gcov format +Scanning Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/ for .gcda files ... +Found 38 data files in Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/ +Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/RShiftU64.gcda +Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/Math64.gcda +Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/SafeString.gcda +Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/SwapBytes32.gcda +Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/CpuDeadLoop.gcda +Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/MultU64x64.gcda +Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/LongJump.gcda +Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/SetJump.gcda +Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/BitField.gcda +Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/String.gcda +Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/DivU64x32Remainder.g= cda +Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/X64/GccInline.gcda +Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/MultU64x32.gcda +Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/MultS64x64.gcda +Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/LinkedList.gcda +Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/LShiftU64.gcda +Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/DivU64x32.gcda +Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/SwapBytes16.gcda +Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/Unaligned.gcda +Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/CheckSum.gcda +Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/CopyMem.gcda +Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/MemLibGuid.gcda +Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/CompareMemWrapper.gc= da +Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/SetMemWrapper.gcda +Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/SetMem64Wrapper.gcda +Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/MemLibGeneric.gcda +Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/CopyMemWrapper.gcda +Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/SetMem.gcda +Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/ZeroMemWrapper.gcda +Processing Library/BasePrintLib/BasePrintLib/OUTPUT/PrintLibInternal.gcda +Processing Library/BasePrintLib/BasePrintLib/OUTPUT/PrintLib.gcda +Processing Library/BaseSafeIntLib/BaseSafeIntLib/OUTPUT/SafeIntLib64.gcda +Processing Library/BaseSafeIntLib/BaseSafeIntLib/OUTPUT/SafeIntLib.gcda +Processing Test/UnitTest/Library/BaseLib/BaseLibUnitTestsHost/OUTPUT/Base6= 4UnitTest.gcda +Processing Test/UnitTest/Library/BaseLib/BaseLibUnitTestsHost/OUTPUT/AutoG= en.gcda +Processing Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLibHost/OUT= PUT/SafeIntLibUintnIntnUnitTests64.gcda +Processing Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLibHost/OUT= PUT/TestBaseSafeIntLib.gcda +Processing Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLibHost/OUT= PUT/AutoGen.gcda + +user@linux_machine:~/_uefi$genhtml UnitTest.info --branch-coverage -o Buil= d/UnitTestReport + +Reading data file UnitTest.info +Found 38 entries. +Found common filename prefix "~/_uefi/MdePkg/Library" +Writing .css and .png files. +Generating output. +Processing file ~/_uefi/Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/Test/U= nitTest/Library/BaseLib/BaseLibUnitTestsHost/DEBUG/AutoGen.c +Processing file ~/_uefi/Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/Test/U= nitTest/Library/BaseSafeIntLib/TestBaseSafeIntLibHost/DEBUG/AutoGen.c +Processing file BaseLib/Unaligned.c +Processing file BaseLib/CheckSum.c +Processing file BaseLib/BitField.c +Processing file BaseLib/DivU64x32.c +Processing file BaseLib/CpuDeadLoop.c +Processing file BaseLib/LongJump.c +Processing file BaseLib/MultU64x64.c +Processing file BaseLib/RShiftU64.c +Processing file BaseLib/SwapBytes32.c +Processing file BaseLib/SwapBytes16.c +Processing file BaseLib/SetJump.c +Processing file BaseLib/LinkedList.c +Processing file BaseLib/LShiftU64.c +Processing file BaseLib/SafeString.c +Processing file BaseLib/String.c +Processing file BaseLib/DivU64x32Remainder.c +Processing file BaseLib/MultS64x64.c +Processing file BaseLib/MultU64x32.c +Processing file BaseLib/Math64.c +Processing file BaseLib/X64/GccInline.c +Processing file BaseMemoryLib/SetMemWrapper.c +Processing file BaseMemoryLib/CompareMemWrapper.c +Processing file BaseMemoryLib/SetMem64Wrapper.c +Processing file BaseMemoryLib/ZeroMemWrapper.c +Processing file BaseMemoryLib/CopyMemWrapper.c +Processing file BaseMemoryLib/CopyMem.c +Processing file BaseMemoryLib/SetMem.c +Processing file BaseMemoryLib/MemLibGuid.c +Processing file BaseMemoryLib/MemLibGeneric.c +Processing file BasePrintLib/PrintLib.c +Processing file BasePrintLib/PrintLibInternal.c +Processing file BaseSafeIntLib/SafeIntLib64.c +Processing file BaseSafeIntLib/SafeIntLib.c +Processing file ~/_uefi/MdePkg/Test/UnitTest/Library/BaseLib/Base64UnitTes= t.c +Processing file ~/_uefi/MdePkg/Test/UnitTest/Library/BaseSafeIntLib/TestBa= seSafeIntLib.c +Processing file ~/_uefi/MdePkg/Test/UnitTest/Library/BaseSafeIntLib/SafeIn= tLibUintnIntnUnitTests64.c +Writing directory view page. +Overall coverage rate: + lines......: 58.6% (2688 of 4589 lines) + functions..: 57.1% (249 of 436 functions) + branches...: 25.3% (1062 of 4195 branches) + +... + +``` + ### Important Note This works on both Windows and Linux, but is currently limited to x64 arch= itectures. Working on getting others, but we diff --git a/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc b/UnitTe= stFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc index 4dd8d4ac67..8623aecb33 100644 --- a/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc +++ b/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc @@ -19,7 +19,7 @@ MemoryAllocationLib|UnitTestFrameworkPkg/Library/Posix/MemoryAllocationL= ibPosix/MemoryAllocationLibPosix.inf [BuildOptions] - GCC:*_*_*_CC_FLAGS =3D -fno-pie + GCC:*_*_*_CC_FLAGS =3D -fno-pie -ftest-coverage -fprofile-arcs !ifdef $(UNIT_TESTING_DEBUG) MSFT:*_*_*_CC_FLAGS =3D -D UNIT_TESTING_DEBUG=3D1 GCC:*_*_*_CC_FLAGS =3D -D UNIT_TESTING_DEBUG=3D1 --_000_BY3PR19MB4900C77DCFA59D0C255BDDE2C8469BY3PR19MB4900namp_ Content-Type: text/html; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable
For local scenarios the plug-in generates the html report for viewing = the c code with coverage annotations.  The intermediate files are avai= lable so I would think they could be loaded in all sorts of viewers but I h= aven=92t tried. I have just used the html and it seemed acceptable. 

For ci the pipeline files have been updated to publish the= data to the build. This allows using the online viewer to navigate c code = and coverage. The html report is also uploaded and can be downloaded for lo= cal viewing. 

Regarding using visual studio tool chain/compilers we had = discussed that for ci this is unnecessary ( since we already have coverage = from gcc tool chain). For local users this is important. There is an easy t= ool you shared with me that gives similar results. I didn=92t see a great way to integrate into the plug-in = for auto invocation but I plan to write up usage details for local users.&n= bsp;

Thanks
Sean
From: Kinney, Michael D <= ;michael.d.kinney@intel.com>
Sent: Tuesday, September 13, 2022 6:51:43 PM
To: Sean Brogan <spbrogan@outlook.com>; devel@edk2.groups.io &= lt;devel@edk2.groups.io>; Guo, Gua <gua.guo@intel.com>; Kinney, Mi= chael D <michael.d.kinney@intel.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>; Michael Kubacki &= lt;mikuback@linux.microsoft.com>
Subject: RE: [edk2-devel] [PATCH v2] UnitTestFrameworkPkg: Add code = coverage support for GCC
 

Does that pytool plugin support viewing the code coverage visually in C sour= ce files?

 

Either HTML or preferably an Visu= al Studio Code plugin?

 

Can it be updated to support VS c= ompilers tool?

 

Mie

 

From: Sean Brogan <spbrogan@outlook.com>
Sent: Tuesday, September 13, 2022 6:18 PM
To: devel@edk2.groups.io; Guo, Gua <gua.guo@intel.com>
Cc: Kinney, Michael D <michael.d.kinney@intel.com>; Sean Broga= n <sean.brogan@microsoft.com>; Michael Kubacki <mikuback@linux.mic= rosoft.com>
Subject: Re: [edk2-devel] [PATCH v2] UnitTestFrameworkPkg: Add code = coverage support for GCC

 

Hi Gua Guo,

I would propose that we make this easier for both the local user and the= CI system to get coverage metrics. 

Can you review this commit as it uses the same Edk2 plugins but adds cod= e coverage support with GCC5:

Merged PR 5008: Initial implementation or code cove= rage on basecore u=85 =B7 microsoft/mu_basecore@f6af51f (github.com)

 

Thanks

Sean

 

 

On 9/13/2022 6:01 PM, Guo, Gua wr= ote:

From: Gua Guo <gua.=
guo@intel.com>
 
In order to collect code coverage after running executable
file, generate *.gcda and *.gcno file that require by lcov tool
to generate code coverage report.
 
Cc: Michael D Kinney <=
;michael.d.kinney@intel.com>
Cc: Sean Brogan <sean.=
brogan@microsoft.com>
Cc: Bret Barkelew <B=
ret.Barkelew@microsoft.com>
Cc: Michael Kubacki &l=
t;mikuback@linux.microsoft.com>
Signed-off-by: Gua Guo <gua.gu=
o@intel.com>
---
 UnitTestFrameworkPkg/ReadMe.md   &nbs=
p;            &=
nbsp;       | 116 +++++++++++++++++++-=

 UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc |   2 +-
 2 files changed, 116 insertions(+), 2 deletions(-)
 
diff --git a/UnitTestFrameworkPkg/ReadMe.md b/UnitTestFrameworkPkg/Rea=
dMe.md
index e696412cb3..93fdacfc4a 100644
--- a/UnitTestFrameworkPkg/ReadMe.md
+++ b/UnitTestFrameworkPkg/ReadMe.md
@@ -361,7 +361,7 @@ RUNNING TEST SUITE: Int Safe Conversions Test Suit=
e
 ```
 
 
 
 You can also, if you are so inclined, read the output from the exact =
instance of the test that was run during
 
-`stuart_ci_build`. The ouput file can be found on a path that looks l=
ike:
 
+`stuart_ci_build`. The output file can be found on a path that looks =
like:
 
 
 
 `Build/<Package>/HostTest/<Arch>/<TestName>.<Tes=
tSuiteName>.<Arch>.result.xml`
 
 
 
@@ -399,6 +399,120 @@ CMOCKA_XML_FILE=3D<absolute or relative path =
to output file>
 
 
 This mode is used by the test running plugin to aggregate the results=
 for CI test status reporting in the web view.
 
 
 
+### Enable Code Coverage for GCC host-based tests
 
+
 
+LCOV is an extension of GCOV, a GNU tool which provides information a=
bout what parts of a program are actually executed (i.e. "covered"=
;) while running a particular test case. The extension consists of a set of=
 scripts which build on the textual GCOV output to implement it.
 
+
 
+Documentation for LCOV can be found here:
 
+https://lcov.readthedocs.io/
 
+
 
+You can follow below steps to generate your code coverage report.

 


+


 


+


 


+


 


+```bash


 


+user@linux_machine:~/_uefi$stuart_ci_build -c .pytool/CISettings.py T=
OOL_CHAIN_TAG=3DGCC5 -p MdePkg -t NOOPT


 


+...


 


+user@linux_machine:~/_uefi$lcov -c --rc lcov_branch_coverage=3D1 -o U=
nitTest.info -d Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/


 


+Capturing coverage data from Build/MdePkg/HostTest/NOOPT_GCC5/X64/Mde=
Pkg/


 


+Found gcov version: 9.4.0


 


+Using intermediate gcov format


 


+Scanning Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/ for .gcda files=
 ...


 


+Found 38 data files in Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/

 


+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/RShiftU64.gcda<=
/pre>

 


+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/Math64.gcda

 


+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/SafeString.gcda=



 


+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/SwapBytes32.gcd=
a


 


+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/CpuDeadLoop.gcd=
a


 


+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/MultU64x64.gcda=



 


+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/LongJump.gcda

 


+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/SetJump.gcda

 


+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/BitField.gcda

 


+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/String.gcda

 


+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/DivU64x32Remain=
der.gcda


 


+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/X64/GccInline.g=
cda


 


+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/MultU64x32.gcda=



 


+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/MultS64x64.gcda=



 


+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/LinkedList.gcda=



 


+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/LShiftU64.gcda<=
/pre>

 


+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/DivU64x32.gcda<=
/pre>

 


+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/SwapBytes16.gcd=
a


 


+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/Unaligned.gcda<=
/pre>

 


+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/CheckSum.gcda

 


+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/CopyMem.gcda

 


+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/MemLibGuid.gcda=



 


+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/CompareMemWrapp=
er.gcda


 


+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/SetMemWrapper.g=
cda


 


+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/SetMem64Wrapper=
.gcda


 


+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/MemLibGeneric.g=
cda


 


+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/CopyMemWrapper.=
gcda


 


+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/SetMem.gcda

 


+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/ZeroMemWrapper.=
gcda


 


+Processing Library/BasePrintLib/BasePrintLib/OUTPUT/PrintLibInternal.=
gcda


 


+Processing Library/BasePrintLib/BasePrintLib/OUTPUT/PrintLib.gcda

 


+Processing Library/BaseSafeIntLib/BaseSafeIntLib/OUTPUT/SafeIntLib64.=
gcda


 


+Processing Library/BaseSafeIntLib/BaseSafeIntLib/OUTPUT/SafeIntLib.gc=
da


 


+Processing Test/UnitTest/Library/BaseLib/BaseLibUnitTestsHost/OUTPUT/=
Base64UnitTest.gcda


 


+Processing Test/UnitTest/Library/BaseLib/BaseLibUnitTestsHost/OUTPUT/=
AutoGen.gcda


 


+Processing Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLibHos=
t/OUTPUT/SafeIntLibUintnIntnUnitTests64.gcda


 


+Processing Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLibHos=
t/OUTPUT/TestBaseSafeIntLib.gcda


 


+Processing Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLibHos=
t/OUTPUT/AutoGen.gcda


 


+


 


+user@linux_machine:~/_uefi$genhtml UnitTest.info --branch-coverage -o=
 Build/UnitTestReport


 


+


 


+Reading data file UnitTest.info


 


+Found 38 entries.


 


+Found common filename prefix "~/_uefi/MdePkg/Library"


 


+Writing .css and .png files.


 


+Generating output.


 


+Processing file ~/_uefi/Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/T=
est/UnitTest/Library/BaseLib/BaseLibUnitTestsHost/DEBUG/AutoGen.c


 


+Processing file ~/_uefi/Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/T=
est/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLibHost/DEBUG/AutoGen.c<=
/pre>

 


+Processing file BaseLib/Unaligned.c


 


+Processing file BaseLib/CheckSum.c


 


+Processing file BaseLib/BitField.c


 


+Processing file BaseLib/DivU64x32.c


 


+Processing file BaseLib/CpuDeadLoop.c


 


+Processing file BaseLib/LongJump.c


 


+Processing file BaseLib/MultU64x64.c


 


+Processing file BaseLib/RShiftU64.c


 


+Processing file BaseLib/SwapBytes32.c


 


+Processing file BaseLib/SwapBytes16.c


 


+Processing file BaseLib/SetJump.c


 


+Processing file BaseLib/LinkedList.c


 


+Processing file BaseLib/LShiftU64.c


 


+Processing file BaseLib/SafeString.c


 


+Processing file BaseLib/String.c


 


+Processing file BaseLib/DivU64x32Remainder.c


 


+Processing file BaseLib/MultS64x64.c


 


+Processing file BaseLib/MultU64x32.c


 


+Processing file BaseLib/Math64.c


 


+Processing file BaseLib/X64/GccInline.c


 


+Processing file BaseMemoryLib/SetMemWrapper.c


 


+Processing file BaseMemoryLib/CompareMemWrapper.c


 


+Processing file BaseMemoryLib/SetMem64Wrapper.c


 


+Processing file BaseMemoryLib/ZeroMemWrapper.c


 


+Processing file BaseMemoryLib/CopyMemWrapper.c


 


+Processing file BaseMemoryLib/CopyMem.c


 


+Processing file BaseMemoryLib/SetMem.c


 


+Processing file BaseMemoryLib/MemLibGuid.c


 


+Processing file BaseMemoryLib/MemLibGeneric.c


 


+Processing file BasePrintLib/PrintLib.c


 


+Processing file BasePrintLib/PrintLibInternal.c


 


+Processing file BaseSafeIntLib/SafeIntLib64.c


 


+Processing file BaseSafeIntLib/SafeIntLib.c


 


+Processing file ~/_uefi/MdePkg/Test/UnitTest/Library/BaseLib/Base64Un=
itTest.c


 


+Processing file ~/_uefi/MdePkg/Test/UnitTest/Library/BaseSafeIntLib/T=
estBaseSafeIntLib.c


 


+Processing file ~/_uefi/MdePkg/Test/UnitTest/Library/BaseSafeIntLib/S=
afeIntLibUintnIntnUnitTests64.c


 


+Writing directory view page.


 


+Overall coverage rate:


 


+  lines......: 58.6% (2688 of 4589 lines=
)


 


+  functions..: 57.1% (249 of 436 functio=
ns)


 


+  branches...: 25.3% (1062 of 4195 branc=
hes)


 


+


 


+...


 


+


 


+```


 


+


 


 ### Important Note


 


 


 


 This works on both Windows and Linux, but is currently limited to x64=
 architectures. Working on getting others, but we


 


diff --git a/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc b/U=
nitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc


index 4dd8d4ac67..8623aecb33 100644


--- a/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc


+++ b/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc


@@ -19,7 +19,7 @@


   MemoryAllocationLib|UnitTestFrame=
workPkg/Library/Posix/MemoryAllocationLibPosix/MemoryAllocationLibPosix.inf=



 


 


 


 [BuildOptions]


 


-  GCC:*_*_*_CC_FLAGS =3D -fno-pie


 


+  GCC:*_*_*_CC_FLAGS =3D -fno-pie -ftest=
-coverage -fprofile-arcs


 


 !ifdef $(UNIT_TESTING_DEBUG)


 


   MSFT:*_*_*_CC_FLAGS  =3D -D UNIT_TESTING_DEBUG=3D1


 


   GCC:*_*_*_CC_FLAGS   =3D -D UNIT_TESTING_DEBUG=3D1


 
--_000_BY3PR19MB4900C77DCFA59D0C255BDDE2C8469BY3PR19MB4900namp_--