From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by mx.groups.io with SMTP id smtpd.web11.4494.1663830549554291597 for ; Thu, 22 Sep 2022 00:09:10 -0700 Authentication-Results: mx.groups.io; dkim=fail reason="unable to parse pub key" header.i=@intel.com header.s=intel header.b=EK/y8JM7; spf=pass (domain: intel.com, ip: 134.134.136.100, mailfrom: gua.guo@intel.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1663830548; x=1695366548; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=a25fnNtxBwZk3cN1zKJUuFi6dOLGIgL7Wyw+s2BInAE=; b=EK/y8JM7kjrvQQ5BVfK2qCAtJ2ZXC/3BPRwcAYQUvkXf8VO/sU7MCJ+x +lsV9+AX/Coa7qtg0GsYrCht5xbzRE4L2gESapDNuo34i6i1aBuSbjB7M 0FhPudnk0t7T3Pu5KjGCR06LaTqbnNxptcFQMgyoUQrQVqkEUe5C1Mr4G 8dVw02bFLzfAzLjL7IBh5W1ExiTPIQwYgKFPVPd/H8W6Zd/IckVBPel3/ EwSmDsulxMHa9jBvBwveS0hGt1LsHSbRugFKohHys4m5ejldpyNe8o7Qs +xEizPOz/ClJ900Um+edXIs5TJ/IBzVw8napIMI/Of3Flq+ZlEcJibnSY Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10477"; a="364198666" X-IronPort-AV: E=Sophos;i="5.93,335,1654585200"; d="scan'208";a="364198666" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Sep 2022 00:09:08 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,335,1654585200"; d="scan'208";a="621986200" Received: from gguo-desk.gar.corp.intel.com ([10.5.215.23]) by fmsmga007.fm.intel.com with ESMTP; 22 Sep 2022 00:09:08 -0700 From: "Guo, Gua" To: devel@edk2.groups.io Cc: Gua Guo , Michael D Kinney , Sean Brogan , Bret Barkelew , Michael Kubacki Subject: [PATCH v3 1/2] UnitTestFrameworkPkg: Add code coverage support for GCC Date: Thu, 22 Sep 2022 15:09:04 +0800 Message-Id: <3a060305c0eaf9a996043795744c4ada4f83914b.1663830230.git.gua.guo@intel.com> X-Mailer: git-send-email 2.31.1.windows.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable From: Gua Guo In order to collect code coverage after running executable file, generate *.gcda and *.gcno file that require by lcov tool to generate code coverage report. Cc: Michael D Kinney Cc: Sean Brogan Cc: Bret Barkelew Cc: Michael Kubacki Signed-off-by: Gua Guo --- UnitTestFrameworkPkg/ReadMe.md | 42 +++++++++++++++++-- .../UnitTestFrameworkPkg.ci.yaml | 2 + .../UnitTestFrameworkPkgHost.dsc.inc | 3 +- 3 files changed, 43 insertions(+), 4 deletions(-) diff --git a/UnitTestFrameworkPkg/ReadMe.md b/UnitTestFrameworkPkg/ReadMe.md index e696412cb3..3c69cfdd97 100644 --- a/UnitTestFrameworkPkg/ReadMe.md +++ b/UnitTestFrameworkPkg/ReadMe.md @@ -60,7 +60,7 @@ you should be good to go. =0D See this example in 'SampleUnitTestUefiShell.inf'...=0D =0D -```=0D +```inf=0D [Packages]=0D MdePkg/MdePkg.dec=0D =0D @@ -75,7 +75,7 @@ See this example in 'SampleUnitTestUefiShell.inf'... Also, if you want you test to automatically be picked up by the Test Runne= r plugin, you will need=0D to make sure that the module `BASE_NAME` contains the word `Test`...=0D =0D -```=0D +```inf=0D [Defines]=0D BASE_NAME =3D SampleUnitTestUefiShell=0D ```=0D @@ -361,7 +361,7 @@ RUNNING TEST SUITE: Int Safe Conversions Test Suite ```=0D =0D You can also, if you are so inclined, read the output from the exact insta= nce of the test that was run during=0D -`stuart_ci_build`. The ouput file can be found on a path that looks like:= =0D +`stuart_ci_build`. The output file can be found on a path that looks like:= =0D =0D `Build//HostTest//...result.= xml`=0D =0D @@ -399,6 +399,42 @@ CMOCKA_XML_FILE=3D =0D This mode is used by the test running plugin to aggregate the results for = CI test status reporting in the web view.=0D =0D +### Code Coverage=0D +=0D +Host based Unit Tests will automatically i enable coverage data.=0D +This is primarily leveraged for pipeline builds, but this can be leveraged= locally using the=0D +lcov linux tool, and parsed using the lcov_cobertura python tool. pycobert= ura is used to=0D +covert this coverage data to a human readable HTML file. These tools must = be installed=0D +to parse code coverage.=0D +=0D +- Windows Prerequisite=0D + ```bash=0D + Download and install https://github.com/OpenCppCoverage/OpenCppCoverage/= tags=0D + pip install pycobertura=0D + ```=0D +=0D +- Linux Prerequisite=0D + ```bash=0D + sudo apt-get install -y lcov=0D + pip install lcov_cobertura=0D + pip install pycobertura=0D + ```=0D +=0D +During CI builds, use the ```CODE_COVERAGE=3DTRUE``` flag to generate the= code coverage XML files,=0D +and additionally use the ```CC_HTML=3DTRUE``` flag to generate the HTML fi= le. This will be generated=0D +in Build/coverage.html.=0D +=0D +There is currently no official guidance or support for code coverage when = compiling=0D +in Visual Studio at this time.=0D +During CI builds, use the ```CODE_COVERAGE=3DTRUE``` flag to generate the= code coverage XML files,=0D +and additionally use the ```CC_HTML=3DTRUE``` flag to generate the HTML fi= le. This will be generated=0D +in Build/coverage.html.=0D +=0D +Example=0D +```bash=0D + stuart_ci_build -c .pytool/CISettings.py -t NOOPT TOOL_CHAIN_TAG=3DVS20= 19 -p MdeModulePkg CC_HTML=3DTRUE CODE_COVERAGE=3DTRUE=0D + ```=0D +=0D ### Important Note=0D =0D This works on both Windows and Linux, but is currently limited to x64 arch= itectures. Working on getting others, but we=0D diff --git a/UnitTestFrameworkPkg/UnitTestFrameworkPkg.ci.yaml b/UnitTestFr= ameworkPkg/UnitTestFrameworkPkg.ci.yaml index 77d51e1348..f68882e1cf 100644 --- a/UnitTestFrameworkPkg/UnitTestFrameworkPkg.ci.yaml +++ b/UnitTestFrameworkPkg/UnitTestFrameworkPkg.ci.yaml @@ -90,6 +90,8 @@ "pytools",=0D "NOFAILURE",=0D "cmockery",=0D + "cobertura",=0D + "pycobertura",=0D "DHAVE", # build flag for cmocka in the INF=0D "corthon", # Contact GitHub account in Readme=0D "mdkinney", # Contact GitHub account in Readme=0D diff --git a/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc b/UnitTe= stFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc index 4dd8d4ac67..ca46b8513a 100644 --- a/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc +++ b/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc @@ -25,7 +25,8 @@ GCC:*_*_*_CC_FLAGS =3D -D UNIT_TESTING_DEBUG=3D1=0D XCODE:*_*_*_CC_FLAGS =3D -D UNIT_TESTING_DEBUG=3D1=0D !endif=0D -=0D + GCC:*_GCC5_*_CC_FLAGS =3D --coverage=0D + GCC:*_GCC5_*_DLINK_FLAGS =3D --coverage=0D [BuildOptions.common.EDKII.HOST_APPLICATION]=0D #=0D # MSFT=0D --=20 2.31.1.windows.1