Re: [PATCH 1/3] UnitTestFrameworkPkg: Add code coverage support for GCC

["Michael Kubacki" <mikuback@linux.microsoft.com>]

It might cause problems depending on markdown linter settings, but if it 
passes CI, it's fine.

Thanks,
Michael

On 12/20/2022 10:53 PM, Guo, Gua wrote:
> You means "```" to "```inf ", it just make markdown file more readability only.
> 
> Thanks,
> Gua
> -----Original Message-----
> From: Michael Kubacki <mikuback@linux.microsoft.com>
> Sent: Wednesday, December 21, 2022 11:41 AM
> To: Guo, Gua <gua.guo@intel.com>; devel@edk2.groups.io
> Cc: Kinney, Michael D <michael.d.kinney@intel.com>; Sean Brogan <sean.brogan@microsoft.com>; Barkelew, Bret <bret.barkelew@microsoft.com>
> Subject: Re: [PATCH 1/3] UnitTestFrameworkPkg: Add code coverage support for GCC
> 
> I'm not sure if you added some extra new lines in the markdown file to help with readability in the raw markdown or they were accidental, but I think they can be eliminated.
> 
> Reviewed-by: Michael Kubacki <michael.kubacki@microsoft.com>
> 
> On 9/29/2022 9:53 PM, gua.guo@intel.com wrote:
>> From: Gua Guo <gua.guo@intel.com>
>>
>> In order to collect code coverage after running executable file,
>> generate *.gcda and *.gcno file that require by lcov tool to generate
>> code coverage report.
>>
>> Cc: Michael D Kinney <michael.d.kinney@intel.com>
>> Cc: Sean Brogan <sean.brogan@microsoft.com>
>> Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
>> Cc: Michael Kubacki <mikuback@linux.microsoft.com>
>> Signed-off-by: Gua Guo <gua.guo@intel.com>
>> ---
>>    UnitTestFrameworkPkg/ReadMe.md                | 37 +++++++++++++++++--
>>    .../UnitTestFrameworkPkg.ci.yaml              |  2 +
>>    .../UnitTestFrameworkPkgHost.dsc.inc          |  3 +-
>>    3 files changed, 38 insertions(+), 4 deletions(-)
>>
>> diff --git a/UnitTestFrameworkPkg/ReadMe.md
>> b/UnitTestFrameworkPkg/ReadMe.md index e696412cb3..c00e2411f8 100644
>> --- a/UnitTestFrameworkPkg/ReadMe.md
>> +++ b/UnitTestFrameworkPkg/ReadMe.md
>> @@ -60,7 +60,7 @@ you should be good to go.
>>    
>>
>>    See this example in 'SampleUnitTestUefiShell.inf'...
>>
>>    
>>
>> -```
>>
>> +```inf
>>
>>    [Packages]
>>
>>      MdePkg/MdePkg.dec
>>
>>    
>>
>> @@ -75,7 +75,7 @@ See this example in 'SampleUnitTestUefiShell.inf'...
>>    Also, if you want you test to automatically be picked up by the Test
>> Runner plugin, you will need
>>
>>    to make sure that the module `BASE_NAME` contains the word `Test`...
>>
>>    
>>
>> -```
>>
>> +```inf
>>
>>    [Defines]
>>
>>      BASE_NAME      = SampleUnitTestUefiShell
>>
>>    ```
>>
>> @@ -361,7 +361,7 @@ RUNNING TEST SUITE: Int Safe Conversions Test Suite
>>    ```
>>
>>    
>>
>>    You can also, if you are so inclined, read the output from the exact
>> instance of the test that was run during
>>
>> -`stuart_ci_build`. The ouput file can be found on a path that looks like:
>>
>> +`stuart_ci_build`. The output file can be found on a path that looks like:
>>
>>    
>>
>>    
>> `Build/<Package>/HostTest/<Arch>/<TestName>.<TestSuiteName>.<Arch>.res
>> ult.xml`
>>
>>    
>>
>> @@ -399,6 +399,37 @@ CMOCKA_XML_FILE=<absolute or relative path to
>> output file>
>>    
>>
>>    This mode is used by the test running plugin to aggregate the results for CI test status reporting in the web view.
>>
>>    
>>
>> +### Code Coverage
>>
>> +
>>
>> +Host based Unit Tests will automatically enable coverage data.
>>
>> +
>>
>> +For Windows, This is primarily leverage for pipeline builds, but this
>> +can be leveraged locally using the
>>
>> +OpenCppCoverage windows tool, and pycobertura is used to covert this
>> +coverage data to a human readable HTML
>>
>> +file. These tools must be installed to parse code coverage.
>>
>> +
>>
>> +- Windows Prerequisite
>>
>> +  ```bash
>>
>> +  Download and install
>> + https://github.com/OpenCppCoverage/OpenCppCoverage/tags
>>
>> +  python -m pip install --upgrade -r ./pip-requirements.txt
>>
>> +  stuart_ci_build -c .pytool/CISettings.py  -t NOOPT
>> + TOOL_CHAIN_TAG=VS2019 -p MdeModulePkg
>>
>> +  Open Build/coverage.html
>>
>> +  ```
>>
>> +
>>
>> +
>>
>> +For Linux, This is primarily leveraged for pipeline builds, but this
>> +can be leveraged locally using the
>>
>> +lcov linux tool, and parsed using the lcov_cobertura python tool.
>> +pycobertura is used to
>>
>> +covert this coverage data to a human readable HTML file. These tools
>> +must be installed
>>
>> +to parse code coverage.
>>
>> +
>>
>> +- Linux Prerequisite
>>
>> +  ```bash
>>
>> +  sudo apt-get install -y lcov
>>
>> +  python -m pip install --upgrade -r ./pip-requirements.txt
>>
>> +  stuart_ci_build -c .pytool/CISettings.py  -t NOOPT
>> + TOOL_CHAIN_TAG=GCC5 -p MdeModulePkg
>>
>> +  Open Build/coverage.html
>>
>> +  ```
>>
>> +
>>
>> +
>>
>>    ### Important Note
>>
>>    
>>
>>    This works on both Windows and Linux, but is currently limited to
>> x64 architectures. Working on getting others, but we
>>
>> diff --git a/UnitTestFrameworkPkg/UnitTestFrameworkPkg.ci.yaml
>> b/UnitTestFrameworkPkg/UnitTestFrameworkPkg.ci.yaml
>> index 77d51e1348..f68882e1cf 100644
>> --- a/UnitTestFrameworkPkg/UnitTestFrameworkPkg.ci.yaml
>> +++ b/UnitTestFrameworkPkg/UnitTestFrameworkPkg.ci.yaml
>> @@ -90,6 +90,8 @@
>>                "pytools",
>>
>>                "NOFAILURE",
>>
>>                "cmockery",
>>
>> +            "cobertura",
>>
>> +            "pycobertura",
>>
>>                "DHAVE", # build flag for cmocka in the INF
>>
>>                "corthon",      # Contact GitHub account in Readme
>>
>>                "mdkinney",     # Contact GitHub account in Readme
>>
>> diff --git a/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc
>> b/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc
>> index 4dd8d4ac67..ca46b8513a 100644
>> --- a/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc
>> +++ b/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc
>> @@ -25,7 +25,8 @@
>>      GCC:*_*_*_CC_FLAGS   = -D UNIT_TESTING_DEBUG=1
>>
>>      XCODE:*_*_*_CC_FLAGS = -D UNIT_TESTING_DEBUG=1
>>
>>    !endif
>>
>> -
>>
>> +  GCC:*_GCC5_*_CC_FLAGS = --coverage
>>
>> +  GCC:*_GCC5_*_DLINK_FLAGS = --coverage
>>
>>    [BuildOptions.common.EDKII.HOST_APPLICATION]
>>
>>      #
>>
>>      # MSFT
>>