From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by mx.groups.io with SMTP id smtpd.web11.11076.1671595123077640876 for ; Tue, 20 Dec 2022 19:58:43 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@linux.microsoft.com header.s=default header.b=Uxz+wQB8; spf=pass (domain: linux.microsoft.com, ip: 13.77.154.182, mailfrom: mikuback@linux.microsoft.com) Received: from [192.168.4.22] (unknown [47.201.8.94]) by linux.microsoft.com (Postfix) with ESMTPSA id 0C07320C3364; Tue, 20 Dec 2022 19:58:41 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 0C07320C3364 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1671595122; bh=VH5qMB8KoRSTaqvBBY9LyEo8+MXxzDiYjlQ/BRDeSR4=; h=Date:Subject:To:Cc:References:From:In-Reply-To:From; b=Uxz+wQB8INK4IGlYOg1zt9SRLy1j5BTmatqYmWdrxLMD+KcWgycwgxlsBRPdxttRm Hh+JCIGeCM/8Mo8M6DabtgricdEfFUEvB4lnIpRh7M0GGm8NM3IfaEKO10CnD0zX9J fyyUBm4NIIFKYaw6H0pyK8vyRzJAAbfPChh8oxFU= Message-ID: Date: Tue, 20 Dec 2022 22:58:40 -0500 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 6.2; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.13.1 Subject: Re: [PATCH 1/3] UnitTestFrameworkPkg: Add code coverage support for GCC To: "Guo, Gua" , "devel@edk2.groups.io" Cc: "Kinney, Michael D" , Sean Brogan , "Barkelew, Bret" References: <3e761ca731604e3eb08664309a2c16a56c5e472f.1664502701.git.gua.guo@intel.com> <74525b37-5cff-9a95-4ce4-af1955b3aa17@linux.microsoft.com> From: "Michael Kubacki" In-Reply-To: Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit It might cause problems depending on markdown linter settings, but if it passes CI, it's fine. Thanks, Michael On 12/20/2022 10:53 PM, Guo, Gua wrote: > You means "```" to "```inf ", it just make markdown file more readability only. > > Thanks, > Gua > -----Original Message----- > From: Michael Kubacki > Sent: Wednesday, December 21, 2022 11:41 AM > To: Guo, Gua ; devel@edk2.groups.io > Cc: Kinney, Michael D ; Sean Brogan ; Barkelew, Bret > Subject: Re: [PATCH 1/3] UnitTestFrameworkPkg: Add code coverage support for GCC > > I'm not sure if you added some extra new lines in the markdown file to help with readability in the raw markdown or they were accidental, but I think they can be eliminated. > > Reviewed-by: Michael Kubacki > > On 9/29/2022 9:53 PM, gua.guo@intel.com wrote: >> From: Gua Guo >> >> In order to collect code coverage after running executable file, >> generate *.gcda and *.gcno file that require by lcov tool to generate >> code coverage report. >> >> Cc: Michael D Kinney >> Cc: Sean Brogan >> Cc: Bret Barkelew >> Cc: Michael Kubacki >> Signed-off-by: Gua Guo >> --- >> UnitTestFrameworkPkg/ReadMe.md | 37 +++++++++++++++++-- >> .../UnitTestFrameworkPkg.ci.yaml | 2 + >> .../UnitTestFrameworkPkgHost.dsc.inc | 3 +- >> 3 files changed, 38 insertions(+), 4 deletions(-) >> >> diff --git a/UnitTestFrameworkPkg/ReadMe.md >> b/UnitTestFrameworkPkg/ReadMe.md index e696412cb3..c00e2411f8 100644 >> --- a/UnitTestFrameworkPkg/ReadMe.md >> +++ b/UnitTestFrameworkPkg/ReadMe.md >> @@ -60,7 +60,7 @@ you should be good to go. >> >> >> See this example in 'SampleUnitTestUefiShell.inf'... >> >> >> >> -``` >> >> +```inf >> >> [Packages] >> >> MdePkg/MdePkg.dec >> >> >> >> @@ -75,7 +75,7 @@ See this example in 'SampleUnitTestUefiShell.inf'... >> Also, if you want you test to automatically be picked up by the Test >> Runner plugin, you will need >> >> to make sure that the module `BASE_NAME` contains the word `Test`... >> >> >> >> -``` >> >> +```inf >> >> [Defines] >> >> BASE_NAME = SampleUnitTestUefiShell >> >> ``` >> >> @@ -361,7 +361,7 @@ RUNNING TEST SUITE: Int Safe Conversions Test Suite >> ``` >> >> >> >> You can also, if you are so inclined, read the output from the exact >> instance of the test that was run during >> >> -`stuart_ci_build`. The ouput file can be found on a path that looks like: >> >> +`stuart_ci_build`. The output file can be found on a path that looks like: >> >> >> >> >> `Build//HostTest//...res >> ult.xml` >> >> >> >> @@ -399,6 +399,37 @@ CMOCKA_XML_FILE=> output file> >> >> >> This mode is used by the test running plugin to aggregate the results for CI test status reporting in the web view. >> >> >> >> +### Code Coverage >> >> + >> >> +Host based Unit Tests will automatically enable coverage data. >> >> + >> >> +For Windows, This is primarily leverage for pipeline builds, but this >> +can be leveraged locally using the >> >> +OpenCppCoverage windows tool, and pycobertura is used to covert this >> +coverage data to a human readable HTML >> >> +file. These tools must be installed to parse code coverage. >> >> + >> >> +- Windows Prerequisite >> >> + ```bash >> >> + Download and install >> + https://github.com/OpenCppCoverage/OpenCppCoverage/tags >> >> + python -m pip install --upgrade -r ./pip-requirements.txt >> >> + stuart_ci_build -c .pytool/CISettings.py -t NOOPT >> + TOOL_CHAIN_TAG=VS2019 -p MdeModulePkg >> >> + Open Build/coverage.html >> >> + ``` >> >> + >> >> + >> >> +For Linux, This is primarily leveraged for pipeline builds, but this >> +can be leveraged locally using the >> >> +lcov linux tool, and parsed using the lcov_cobertura python tool. >> +pycobertura is used to >> >> +covert this coverage data to a human readable HTML file. These tools >> +must be installed >> >> +to parse code coverage. >> >> + >> >> +- Linux Prerequisite >> >> + ```bash >> >> + sudo apt-get install -y lcov >> >> + python -m pip install --upgrade -r ./pip-requirements.txt >> >> + stuart_ci_build -c .pytool/CISettings.py -t NOOPT >> + TOOL_CHAIN_TAG=GCC5 -p MdeModulePkg >> >> + Open Build/coverage.html >> >> + ``` >> >> + >> >> + >> >> ### Important Note >> >> >> >> This works on both Windows and Linux, but is currently limited to >> x64 architectures. Working on getting others, but we >> >> diff --git a/UnitTestFrameworkPkg/UnitTestFrameworkPkg.ci.yaml >> b/UnitTestFrameworkPkg/UnitTestFrameworkPkg.ci.yaml >> index 77d51e1348..f68882e1cf 100644 >> --- a/UnitTestFrameworkPkg/UnitTestFrameworkPkg.ci.yaml >> +++ b/UnitTestFrameworkPkg/UnitTestFrameworkPkg.ci.yaml >> @@ -90,6 +90,8 @@ >> "pytools", >> >> "NOFAILURE", >> >> "cmockery", >> >> + "cobertura", >> >> + "pycobertura", >> >> "DHAVE", # build flag for cmocka in the INF >> >> "corthon", # Contact GitHub account in Readme >> >> "mdkinney", # Contact GitHub account in Readme >> >> diff --git a/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc >> b/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc >> index 4dd8d4ac67..ca46b8513a 100644 >> --- a/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc >> +++ b/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc >> @@ -25,7 +25,8 @@ >> GCC:*_*_*_CC_FLAGS = -D UNIT_TESTING_DEBUG=1 >> >> XCODE:*_*_*_CC_FLAGS = -D UNIT_TESTING_DEBUG=1 >> >> !endif >> >> - >> >> + GCC:*_GCC5_*_CC_FLAGS = --coverage >> >> + GCC:*_GCC5_*_DLINK_FLAGS = --coverage >> >> [BuildOptions.common.EDKII.HOST_APPLICATION] >> >> # >> >> # MSFT >>