From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by mx.groups.io with SMTP id smtpd.web11.10770.1671594066028682329 for ; Tue, 20 Dec 2022 19:41:06 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@linux.microsoft.com header.s=default header.b=LrctY/8D; spf=pass (domain: linux.microsoft.com, ip: 13.77.154.182, mailfrom: mikuback@linux.microsoft.com) Received: from [192.168.4.22] (unknown [47.201.8.94]) by linux.microsoft.com (Postfix) with ESMTPSA id EAC7A20C3360; Tue, 20 Dec 2022 19:41:04 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com EAC7A20C3360 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1671594065; bh=kJ2YEzVReFSJ/mbu7Ow986ed0/EUdGDoA3cv2HyK+q8=; h=Date:Subject:To:Cc:References:From:In-Reply-To:From; b=LrctY/8DSC2IG+UCWugVb21nkyLr/iwtbEO6CZg/ZvD/fcv/jZqjh59411fpZO/RM rKZeL6aXE2sDwgMeT7HCRxZv0aA5hUpUwm8Ny5Rlggk7wRIYTerARCLqiBzrRHJNVw oqvcy80qEK3uzMekMF3FlUNF/znlTNh25jbjeWyw= Message-ID: <74525b37-5cff-9a95-4ce4-af1955b3aa17@linux.microsoft.com> Date: Tue, 20 Dec 2022 22:41:03 -0500 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 6.2; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.13.1 Subject: Re: [PATCH 1/3] UnitTestFrameworkPkg: Add code coverage support for GCC To: gua.guo@intel.com, devel@edk2.groups.io Cc: Michael D Kinney , Sean Brogan , Bret Barkelew References: <3e761ca731604e3eb08664309a2c16a56c5e472f.1664502701.git.gua.guo@intel.com> From: "Michael Kubacki" In-Reply-To: <3e761ca731604e3eb08664309a2c16a56c5e472f.1664502701.git.gua.guo@intel.com> Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit I'm not sure if you added some extra new lines in the markdown file to help with readability in the raw markdown or they were accidental, but I think they can be eliminated. Reviewed-by: Michael Kubacki On 9/29/2022 9:53 PM, gua.guo@intel.com wrote: > From: Gua Guo > > In order to collect code coverage after running executable > file, generate *.gcda and *.gcno file that require by lcov tool > to generate code coverage report. > > Cc: Michael D Kinney > Cc: Sean Brogan > Cc: Bret Barkelew > Cc: Michael Kubacki > Signed-off-by: Gua Guo > --- > UnitTestFrameworkPkg/ReadMe.md | 37 +++++++++++++++++-- > .../UnitTestFrameworkPkg.ci.yaml | 2 + > .../UnitTestFrameworkPkgHost.dsc.inc | 3 +- > 3 files changed, 38 insertions(+), 4 deletions(-) > > diff --git a/UnitTestFrameworkPkg/ReadMe.md b/UnitTestFrameworkPkg/ReadMe.md > index e696412cb3..c00e2411f8 100644 > --- a/UnitTestFrameworkPkg/ReadMe.md > +++ b/UnitTestFrameworkPkg/ReadMe.md > @@ -60,7 +60,7 @@ you should be good to go. > > > See this example in 'SampleUnitTestUefiShell.inf'... > > > > -``` > > +```inf > > [Packages] > > MdePkg/MdePkg.dec > > > > @@ -75,7 +75,7 @@ See this example in 'SampleUnitTestUefiShell.inf'... > Also, if you want you test to automatically be picked up by the Test Runner plugin, you will need > > to make sure that the module `BASE_NAME` contains the word `Test`... > > > > -``` > > +```inf > > [Defines] > > BASE_NAME = SampleUnitTestUefiShell > > ``` > > @@ -361,7 +361,7 @@ RUNNING TEST SUITE: Int Safe Conversions Test Suite > ``` > > > > You can also, if you are so inclined, read the output from the exact instance of the test that was run during > > -`stuart_ci_build`. The ouput file can be found on a path that looks like: > > +`stuart_ci_build`. The output file can be found on a path that looks like: > > > > `Build//HostTest//...result.xml` > > > > @@ -399,6 +399,37 @@ CMOCKA_XML_FILE= > > > This mode is used by the test running plugin to aggregate the results for CI test status reporting in the web view. > > > > +### Code Coverage > > + > > +Host based Unit Tests will automatically enable coverage data. > > + > > +For Windows, This is primarily leverage for pipeline builds, but this can be leveraged locally using the > > +OpenCppCoverage windows tool, and pycobertura is used to covert this coverage data to a human readable HTML > > +file. These tools must be installed to parse code coverage. > > + > > +- Windows Prerequisite > > + ```bash > > + Download and install https://github.com/OpenCppCoverage/OpenCppCoverage/tags > > + python -m pip install --upgrade -r ./pip-requirements.txt > > + stuart_ci_build -c .pytool/CISettings.py -t NOOPT TOOL_CHAIN_TAG=VS2019 -p MdeModulePkg > > + Open Build/coverage.html > > + ``` > > + > > + > > +For Linux, This is primarily leveraged for pipeline builds, but this can be leveraged locally using the > > +lcov linux tool, and parsed using the lcov_cobertura python tool. pycobertura is used to > > +covert this coverage data to a human readable HTML file. These tools must be installed > > +to parse code coverage. > > + > > +- Linux Prerequisite > > + ```bash > > + sudo apt-get install -y lcov > > + python -m pip install --upgrade -r ./pip-requirements.txt > > + stuart_ci_build -c .pytool/CISettings.py -t NOOPT TOOL_CHAIN_TAG=GCC5 -p MdeModulePkg > > + Open Build/coverage.html > > + ``` > > + > > + > > ### Important Note > > > > This works on both Windows and Linux, but is currently limited to x64 architectures. Working on getting others, but we > > diff --git a/UnitTestFrameworkPkg/UnitTestFrameworkPkg.ci.yaml b/UnitTestFrameworkPkg/UnitTestFrameworkPkg.ci.yaml > index 77d51e1348..f68882e1cf 100644 > --- a/UnitTestFrameworkPkg/UnitTestFrameworkPkg.ci.yaml > +++ b/UnitTestFrameworkPkg/UnitTestFrameworkPkg.ci.yaml > @@ -90,6 +90,8 @@ > "pytools", > > "NOFAILURE", > > "cmockery", > > + "cobertura", > > + "pycobertura", > > "DHAVE", # build flag for cmocka in the INF > > "corthon", # Contact GitHub account in Readme > > "mdkinney", # Contact GitHub account in Readme > > diff --git a/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc b/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc > index 4dd8d4ac67..ca46b8513a 100644 > --- a/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc > +++ b/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc > @@ -25,7 +25,8 @@ > GCC:*_*_*_CC_FLAGS = -D UNIT_TESTING_DEBUG=1 > > XCODE:*_*_*_CC_FLAGS = -D UNIT_TESTING_DEBUG=1 > > !endif > > - > > + GCC:*_GCC5_*_CC_FLAGS = --coverage > > + GCC:*_GCC5_*_DLINK_FLAGS = --coverage > > [BuildOptions.common.EDKII.HOST_APPLICATION] > > # > > # MSFT >