From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from alexa-out-sd-02.qualcomm.com (alexa-out-sd-02.qualcomm.com [199.106.114.39]) by mx.groups.io with SMTP id smtpd.web08.1048.1668020125630672356 for ; Wed, 09 Nov 2022 10:55:25 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@quicinc.com header.s=qcdkim header.b=reYbvei7; spf=permerror, err=parse error for token &{10 18 %{ir}.%{v}.%{d}.spf.has.pphosted.com}: invalid domain name (domain: quicinc.com, ip: 199.106.114.39, mailfrom: quic_llindhol@quicinc.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1668020125; x=1699556125; h=date:from:to:cc:subject:message-id:references: mime-version:in-reply-to; bh=mp6yTlecxQY8xdOH/XKn5W9g776NAZiOTkR2mMWHDHU=; b=reYbvei74pfWFi9oiLMtMAYsVw5N3CvN3QjDBPiFb7vGHLpIQX5BFJRh 6xouiikBseGohBumnj9YA48ieV66XJmfDYBVa3WctE3t0V+hydAF00NWC if7G8FjHszXI+UH3c+cjjKVxgQzxsRUykl1tjmJn1mU3gBb0/ZrLkOx0P s=; Received: from unknown (HELO ironmsg04-sd.qualcomm.com) ([10.53.140.144]) by alexa-out-sd-02.qualcomm.com with ESMTP; 09 Nov 2022 10:55:25 -0800 X-QCInternal: smtphost Received: from nasanex01c.na.qualcomm.com ([10.45.79.139]) by ironmsg04-sd.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Nov 2022 10:55:24 -0800 Received: from qc-i7.hemma.eciton.net (10.80.80.8) by nasanex01c.na.qualcomm.com (10.45.79.139) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.29; Wed, 9 Nov 2022 10:55:22 -0800 Date: Wed, 9 Nov 2022 18:55:19 +0000 From: "Leif Lindholm" To: Michael D Kinney CC: , Michael Kubacki , Sean Brogan , Andrew Fish Subject: Re: [Patch v3 3/7] UnitTestFrameworkPkg: Add googletest submodule and GoogleTestLib Message-ID: References: <20221108231252.1864-1-michael.d.kinney@intel.com> <20221108231252.1864-4-michael.d.kinney@intel.com> MIME-Version: 1.0 In-Reply-To: <20221108231252.1864-4-michael.d.kinney@intel.com> Return-Path: quic_llindhol@quicinc.com X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01a.na.qualcomm.com (10.52.223.231) To nasanex01c.na.qualcomm.com (10.45.79.139) Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline On Tue, Nov 08, 2022 at 15:12:48 -0800, Michael D Kinney wrote: > REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4134 > > Add submodule for googletest and add GoogleTestLib that is > required for GoogleTest based unit tests. Add GoogleTest > documentation to Readme.md along with a port of the sample > unit test to the GoogleTest style. > > Cc: Michael Kubacki > Cc: Sean Brogan > Cc: Andrew Fish > Cc: Leif Lindholm > Signed-off-by: Michael D Kinney > Reviewed-by: Michael Kubacki If I was to nitpick, I would point out this patch adds documentation changes/fixes *not* related to this addition. Address if you feel appropriate. Regardless: Acked-by: Leif Lindholm > --- > .gitmodules | 3 + > ReadMe.rst | 1 + > .../Include/Library/GoogleTestLib.h | 14 + > .../Library/GoogleTestLib/GoogleTestLib.inf | 36 +++ > .../Library/GoogleTestLib/GoogleTestLib.uni | 14 + > .../Library/GoogleTestLib/googletest | 1 + > UnitTestFrameworkPkg/ReadMe.md | 255 +++++++++++++++-- > .../SampleGoogleTest/SampleGoogleTest.cpp | 263 ++++++++++++++++++ > .../SampleGoogleTest/SampleGoogleTestHost.inf | 35 +++ > .../Test/UnitTestFrameworkPkgHostTest.dsc | 4 +- > .../UnitTestFrameworkPkg.ci.yaml | 4 +- > UnitTestFrameworkPkg/UnitTestFrameworkPkg.dec | 8 + > .../UnitTestFrameworkPkgHost.dsc.inc | 4 +- > 13 files changed, 611 insertions(+), 31 deletions(-) > create mode 100644 UnitTestFrameworkPkg/Include/Library/GoogleTestLib.h > create mode 100644 UnitTestFrameworkPkg/Library/GoogleTestLib/GoogleTestLib.inf > create mode 100644 UnitTestFrameworkPkg/Library/GoogleTestLib/GoogleTestLib.uni > create mode 160000 UnitTestFrameworkPkg/Library/GoogleTestLib/googletest > create mode 100644 UnitTestFrameworkPkg/Test/GoogleTest/Sample/SampleGoogleTest/SampleGoogleTest.cpp > create mode 100644 UnitTestFrameworkPkg/Test/GoogleTest/Sample/SampleGoogleTest/SampleGoogleTestHost.inf > > diff --git a/.gitmodules b/.gitmodules > index b845c9ee3ff0..8011a88d9d25 100644 > --- a/.gitmodules > +++ b/.gitmodules > @@ -20,3 +20,6 @@ > [submodule "RedfishPkg/Library/JsonLib/jansson"] > path = RedfishPkg/Library/JsonLib/jansson > url = https://github.com/akheron/jansson > +[submodule "UnitTestFrameworkPkg/Library/GoogleTestLib/googletest"] > + path = UnitTestFrameworkPkg/Library/GoogleTestLib/googletest > + url = https://github.com/google/googletest.git > diff --git a/ReadMe.rst b/ReadMe.rst > index 8f5db11281bf..497d96355908 100644 > --- a/ReadMe.rst > +++ b/ReadMe.rst > @@ -93,6 +93,7 @@ that are covered by additional licenses. > - `MdeModulePkg/Library/BrotliCustomDecompressLib/brotli `__ > - `MdeModulePkg/Universal/RegularExpressionDxe/oniguruma `__ > - `UnitTestFrameworkPkg/Library/CmockaLib/cmocka `__ > +- `UnitTestFrameworkPkg/Library/GoogleTestLib/googletest `__ > - `RedfishPkg/Library/JsonLib/jansson `__ > > The EDK II Project is composed of packages. The maintainers for each package > diff --git a/UnitTestFrameworkPkg/Include/Library/GoogleTestLib.h b/UnitTestFrameworkPkg/Include/Library/GoogleTestLib.h > new file mode 100644 > index 000000000000..ebec766d4cf7 > --- /dev/null > +++ b/UnitTestFrameworkPkg/Include/Library/GoogleTestLib.h > @@ -0,0 +1,14 @@ > +/** @file > + GoogleTestLib class with APIs from the googletest project > + > + Copyright (c) 2022, Intel Corporation. All rights reserved.
> + SPDX-License-Identifier: BSD-2-Clause-Patent > + > +**/ > + > +#ifndef GOOGLE_TEST_LIB_H_ > +#define GOOGLE_TEST_LIB_H_ > + > +#include > + > +#endif > diff --git a/UnitTestFrameworkPkg/Library/GoogleTestLib/GoogleTestLib.inf b/UnitTestFrameworkPkg/Library/GoogleTestLib/GoogleTestLib.inf > new file mode 100644 > index 000000000000..68db75d7023f > --- /dev/null > +++ b/UnitTestFrameworkPkg/Library/GoogleTestLib/GoogleTestLib.inf > @@ -0,0 +1,36 @@ > +## @file > +# This module provides GoogleTest Library implementation. > +# > +# Copyright (c) 2022, Intel Corporation. All rights reserved.
> +# SPDX-License-Identifier: BSD-2-Clause-Patent > +# > +## > + > +[Defines] > + INF_VERSION = 0x00010005 > + BASE_NAME = GoogleTestLib > + MODULE_UNI_FILE = GoogleTestLib.uni > + FILE_GUID = A90E4751-AD30-43CC-980B-01E356B49ADF > + MODULE_TYPE = BASE > + VERSION_STRING = 0.1 > + LIBRARY_CLASS = GoogleTestLib|HOST_APPLICATION > + > +# > +# VALID_ARCHITECTURES = IA32 X64 ARM AARCH64 > +# > + > +[Sources] > + googletest/googletest/src/gtest-all.cc > + > +[Packages] > + UnitTestFrameworkPkg/UnitTestFrameworkPkg.dec > + > +[BuildOptions] > + MSFT:*_*_*_CC_FLAGS == /c /EHsc /Zi > + MSFT:NOOPT_*_*_CC_FLAGS = /Od > + > + GCC:*_*_*_CC_FLAGS == -g -c > + > + GCC:NOOPT_*_*_CC_FLAGS = -O0 > + GCC:*_*_IA32_CC_FLAGS = -m32 > + GCC:*_*_X64_CC_FLAGS = -m64 > diff --git a/UnitTestFrameworkPkg/Library/GoogleTestLib/GoogleTestLib.uni b/UnitTestFrameworkPkg/Library/GoogleTestLib/GoogleTestLib.uni > new file mode 100644 > index 000000000000..14c862a23744 > --- /dev/null > +++ b/UnitTestFrameworkPkg/Library/GoogleTestLib/GoogleTestLib.uni > @@ -0,0 +1,14 @@ > +// /** @file > +// This module provides GoogleTest Library implementation. > +// > +// This module provides GoogleTest Library implementation. > +// > +// Copyright (c) 2022, Intel Corporation. All rights reserved.
> +// > +// SPDX-License-Identifier: BSD-2-Clause-Patent > +// > +// **/ > + > +#string STR_MODULE_ABSTRACT #language en-US "GoogleTest Library implementation" > + > +#string STR_MODULE_DESCRIPTION #language en-US "This module provides GoogleTest Library implementation." > diff --git a/UnitTestFrameworkPkg/Library/GoogleTestLib/googletest b/UnitTestFrameworkPkg/Library/GoogleTestLib/googletest > new file mode 160000 > index 000000000000..86add13493e5 > --- /dev/null > +++ b/UnitTestFrameworkPkg/Library/GoogleTestLib/googletest > @@ -0,0 +1 @@ > +Subproject commit 86add13493e5c881d7e4ba77fb91c1f57752b3a4 > diff --git a/UnitTestFrameworkPkg/ReadMe.md b/UnitTestFrameworkPkg/ReadMe.md > index e696412cb3cf..9ce04b7f3eb6 100644 > --- a/UnitTestFrameworkPkg/ReadMe.md > +++ b/UnitTestFrameworkPkg/ReadMe.md > @@ -2,12 +2,67 @@ > > ## About > > -This package adds a unit test framework capable of building tests for multiple contexts including > +This package provides unit test frameworks capable of building tests for multiple contexts including > the UEFI shell environment and host-based environments. It allows for unit test development to focus > -on the tests and leave error logging, result formatting, context persistance, and test running to the framework. > +on the tests and leave error logging, result formatting, context persistence, and test running to the framework. > The unit test framework works well for low level unit tests as well as system level tests and > fits easily in automation frameworks. > > +### Framework > + > +The first unit test framework is called **Framework** and is implemented as a set of EDK II libraries. > +The Framework supports both host-based unit tests and target-based unit tests that share the same > +source style, macros, and APIs. In some scenarios, the same unit test case sources can be built > +for both host-based unit test execution and target-based unit test execution. Host-based unit tests > +that require mocked interfaces can use the mocking infrastructure provided by > +[cmocka](https://api.cmocka.org/) that is included in the UnitTestFrameworkPkg as a submodule. > + > +### GoogleTest > + > +The second unit test framework supported by the UnitTestFrameworkPkg is > +[GoogleTest](http://google.github.io/googletest/) that can be used to implement host-based unit tests. > +Use of GoogleTest for target-based unit tests of EDK II components is not supported. If a > +host-based unit test requires mocked interfaces, then the Framework with cmocka support should be > +used instead. Enabling support for mocked interfaces with GoogleTest is being actively investigated. > +[GoogleTest on GitHub](https://github.com/google/googletest) is included in the UnitTestFrameworkPkg > +as a submodule. > + > +GoogleTest requires less overhead to register test suites and test cases compared to the Framework. > +There are also a number of tools that layer on top of GoogleTest that improve developer productivity. > +One example is the VS Code extension > +[C++ TestMate](https://marketplace.visualstudio.com/items?itemName=matepek.vscode-catch2-test-adapter) > +that may be used to implement, run, and debug unit tests implemented using GoogleTest. > + > +If a component can be tested with host-based unit tests without support for mocked interfaces, > +then GoogleTest is recommended. The MdePkg contains a port of the BaseSafeIntLib unit tests in > +the GoogleTest style so the differences between GoogleTest and Framework unit tests can be reviewed. > +The paths to the BaseSafeIntLib unit tests are: > + > +* MdePkg\Test\UnitTest\Library\BaseSafeIntLib > +* MdePkg\Test\GoogleTest\Library\BaseSafeIntLib > + > +## Framework and GoogleTest Feature Comparison > + > +| Feature | Framework | GoogleTest | > +|:----------------------------|:---------:|:----------:| > +| Host Based Unit Tests | YES | YES | > +| Target Based Unit Tests | YES | NO | > +| Unit Test Source Language | C | C++ | > +| Register Test Suite | YES | Auto | > +| Register Test Case | YES | Auto | > +| Death/Expected Assert Tests | YES | YES | > +| Setup/Teardown Hooks | YES | YES | > +| Value-Parameterized Tests | NO | YES | > +| Typed Tests | NO | YES | > +| Type-Parameterized Tests | NO | YES | > +| Timeout Support | NO | YES | > +| Mocking Support | Cmocka | NO | > +| JUNIT XML Reports | YES | YES | > +| Execute subset of tests | NO | YES | > +| VS Code Extensions | NO | YES | > + > +## Framework Libraries > + > ### UnitTestLib > > The main "framework" library. The core of the framework is the Framework object, which can have any number > @@ -31,10 +86,10 @@ in supporting a system reboot in the middle of a test run. > > Library provides function to run at the end of a framework test run and handles formatting the report. > This is a common customization point and allows the unit test framework to fit its output reports into > -other test infrastructure. In this package a simple library instances has been supplied to output test > +other test infrastructure. In this package simple library instances have been supplied to output test > results to the console as plain text. > > -## Samples > +## Framework Samples > > There is a sample unit test provided as both an example of how to write a unit test and leverage > many of the features of the framework. This sample can be found in the `Test/UnitTest/Sample/SampleUnitTest` > @@ -43,7 +98,7 @@ directory. > The sample is provided in PEI, SMM, DXE, and UEFI App flavors. It also has a flavor for the HOST_APPLICATION > build type, which can be run on a host system without needing a target. > > -## Usage > +## Framework Usage > > This section is built a lot like a "Getting Started". We'll go through some of the components that are needed > when constructing a unit test and some of the decisions that are made by the test writer. We'll also describe > @@ -51,7 +106,7 @@ how to check for expected conditions in test cases and a bit of the logging char > > Most of these examples will refer to the SampleUnitTestUefiShell app found in this package. > > -### Requirements - INF > +### Framework Requirements - INF > > In our INF file, we'll need to bring in the `UnitTestLib` library. Conveniently, the interface > header for the `UnitTestLib` is located in `MdePkg`, so you shouldn't need to depend on any other > @@ -80,7 +135,7 @@ to make sure that the module `BASE_NAME` contains the word `Test`... > BASE_NAME = SampleUnitTestUefiShell > ``` > > -### Requirements - Code > +### Framework Requirements - Code > > Not to state the obvious, but let's make sure we have the following include before getting too far along... > > @@ -90,9 +145,9 @@ Not to state the obvious, but let's make sure we have the following include befo > > Now that we've got that squared away, let's look at our 'Main()'' routine (or DriverEntryPoint() or whatever). > > -### Configuring the Framework > +### Framework Configuration > > -Everything in the UnitTestPkg framework is built around an object called -- conveniently -- the Framework. > +Everything in the UnitTestFrameworkPkg framework is built around an object called -- conveniently -- the Framework. > This Framework object will contain all the information about our test, the test suites and test cases associated > with it, the current location within the test pass, and any results that have been recorded so far. > > @@ -102,7 +157,7 @@ The long name and version strings are just for user presentation and relatively > will be used to name any cache files and/or test results, so should be a name that makes sense in that context. > These strings are copied internally to the Framework, so using stack-allocated or literal strings is fine. > > -In the 'SampleUnitTestUefiShell' app, the module name is used as the short name, so the init looks like this. > +In the 'SampleUnitTestUefiShell' app, the module name is used as the short name, so the initialization looks like this. > > ```c > DEBUG(( DEBUG_INFO, "%a v%a\n", UNIT_TEST_APP_NAME, UNIT_TEST_APP_VERSION )); > @@ -144,11 +199,11 @@ will be used when adding test cases. > Great! Now we've finished some of the cruft, red tape, and busy work. We're ready to add some tests. Adding a test > to a test suite is accomplished with the -- you guessed it -- `AddTestCase` function. It takes in the suite handle; > a `CHAR8` string for the description and class name; a function pointer for the test case itself; additional, optional > -function pointers for prerequisite check and cleanup routines; and and optional pointer to a context structure. > +function pointers for prerequisite check and cleanup routines; and an optional pointer to a context structure. > > Okay, that's a lot. Let's take it one piece at a time. The description and class name strings are very similar in > usage to the suite title and package name strings in the test suites. The former is for user presentation and the > -latter is for xUnit parsing. The test case function pointer is what is actually executed as the "test" and the > +latter is for xUnit parsing. The test case function pointer is what is executed as the "test" and the > prototype should be `UNIT_TEST_FUNCTION`. The last three parameters require a little bit more explaining. > > The prerequisite check function has a prototype of `UNIT_TEST_PREREQUISITE` and -- if provided -- will be called > @@ -180,7 +235,7 @@ Once all the suites and cases are added, it's time to run the Framework. > Status = RunAllTestSuites( Framework ); > ``` > > -### A Simple Test Case > +### Framework - A Simple Test Case > > We'll take a look at the below test case from 'SampleUnitTestApp'... > > @@ -217,9 +272,9 @@ _Note_ that this early return can have implications for memory leakage. > > At the end, if all test criteria pass, you should return `UNIT_TEST_PASSED`. > > -### More Complex Cases > +### Framework - More Complex Cases > > -To write more advanced tests, first take a look at all the Assertion and Logging macros provided in the framework. > +To write more advanced tests, first look at all the Assertion and Logging macros provided in the framework. > > Beyond that, if you're writing host-based tests and want to take a dependency on the UnitTestFrameworkPkg, you can > leverage the `cmocka.h` interface and write tests with all the features of the Cmocka framework. > @@ -227,6 +282,125 @@ leverage the `cmocka.h` interface and write tests with all the features of the C > Documentation for Cmocka can be found here: > https://api.cmocka.org/ > > +## GoogleTest Samples > + > +There is a sample unit test provided as both an example of how to write a unit test and leverage > +many of the GoogleTest features. This sample can be found in the `Test/GoogleTest/Sample/SampleGoogleTest` > +directory. > + > +The sample is provided for the HOST_APPLICATION build type, which can be run on a host system without > +needing a target. > + > +## GoogleTest Usage > + > +This section is built a lot like a "Getting Started". We'll go through some of the components that are needed > +when constructing a unit test and some of the decisions that are made by the test writer. We'll also describe > +how to check for expected conditions in test cases and a bit of the logging characteristics. > + > +Most of these examples will refer to the SampleGoogleTestHost app found in this package. > + > +### GoogleTest Requirements - INF > + > +In our INF file, we'll need to bring in the `GoogleTest` library. Conveniently, the interface > +header for the `GoogleTest` is in `UnitTestFrameworkPkg`, so you shouldn't need to depend on any other > +packages. As long as your DSC file knows where to find the lib implementation that you want to use, > +you should be good to go. > + > +See this example in 'SampleGoogleTestHost.inf'... > + > +``` > +[Packages] > + MdePkg/MdePkg.dec > + UnitTestFrameworkPkg/UnitTestFrameworkPkg.dec > + > +[LibraryClasses] > + GoogleTestLib > + BaseLib > + DebugLib > +``` > + > +Also, if you want you test to automatically be picked up by the Test Runner plugin, you will need > +to make sure that the module `BASE_NAME` contains the word `Test`... > + > +``` > +[Defines] > + BASE_NAME = SampleGoogleTestHost > +``` > + > +### GoogleTest Requirements - Code > + > +Not to state the obvious, but let's make sure we have the following include before getting too far along... > + > +``` > +#include > +extern "C" { > + #include > + #include > + #include > +} > +``` > + > +GoogleTest applications are implemented in C++. The first include brings in the > +GoogleTest definitions. Other EDK II related include files must be wrapped in > +`extern "C" {}` because they are C include files. Link failures will occur if > +this is not done. > + > +Now that we've got that squared away, let's look at our 'Main()'' routine (or DriverEntryPoint() or whatever). > + > +### GoogleTest Configuration > + > +Unlike the Framework, GoogleTest does not require test suites or test cases to > +be registered. Instead, the test cases declare the test suite name and test > +case name as part of their implementation. The only requirement for GoogleTest > +is to have a `main()` function that initialize the GoogleTest infrastructure and > +call the service `RUN_ALL_TESTS()` to run all the unit tests. > + > +```c > +int main(int argc, char* argv[]) { > + testing::InitGoogleTest(&argc, argv); > + return RUN_ALL_TESTS(); > +} > +``` > + > +### GoogleTest - A Simple Test Case > + > +We'll look at the below test case from 'SampleGoogleTestHost'... > + > +```c > +TEST(SimpleMathTests, OnePlusOneShouldEqualTwo) { > + UINTN A; > + UINTN B; > + UINTN C; > + > + A = 1; > + B = 1; > + C = A + B; > + > + ASSERT_EQ (C, 2); > +} > +``` > + > +This uses the simplest form of a GoogleTest unit test using `TEST()` that > +declares the test suite name and the unit test name within that test suite. > +The unit test performs actions and typically makes calls to the code under test > +and contains test assertions to verify that the code under test behaves as > +expected for the given inputs. > + > +In this test case, the `ASSERT_EQ` assertion is being used to establish that the business logic has functioned > +correctly. There are several assertion macros, and you are encouraged to use one that matches as closely to your > +intended test criterium as possible, because the logging is specific to the macro and more specific macros have more > +detailed logs. When in doubt, there are always `ASSERT_TRUE` and `ASSERT_FALSE`. Assertion macros that fail their > +test criterium will immediately return from the test case with a failed status and log an error string. > +_Note_ that this early return can have implications for memory leakage. > + > +There is no return status from a GooglTest unit test. If no assertions are > +triggered then the unit test has a passing status. > + > +### GoogleTest - More Complex Cases > + > +To write more advanced tests, take a look at the > +[GoogleTest User's Guide](http://google.github.io/googletest/). > + > ## Development > > ### Iterating on a Single Test > @@ -243,11 +417,11 @@ stuart_ci_build -c .pytool/CISettings.py TOOL_CHAIN_TAG=VS2017 -p MdePkg -t NOOP > > ### Hooking BaseLib > > -Most unit test mocking can be performed by the functions provided in the UnitTestFramework libraries, but since > +Most unit test mocking can be performed by the functions provided in the UnitTestFrameworkPkg libraries, but since > BaseLib is consumed by the Framework itself, it requires different techniques to substitute parts of the > functionality. > > -To solve some of this, the UnitTestFramework consumes a special implementation of BaseLib for host-based tests. > +To solve some of this, the UnitTestFrameworkPkg consumes a special implementation of BaseLib for host-based tests. > This implementation contains a [hook table](https://github.com/tianocore/edk2/blob/e188ecc8b4aed8fdd26b731d43883861f5e5e7b4/MdePkg/Test/UnitTest/Include/Library/UnitTestHostBaseLib.h#L507) > that can be used to substitute test functionality for any of the BaseLib functions. By default, this implementation > will use the underlying BaseLib implementation, so the unit test writer only has to supply minimal code to test a > @@ -255,7 +429,7 @@ particular case. > > ### Debugging the Framework Itself > > -While most of the tests that are produced by the UnitTestFramework are easy to step through in a debugger, the Framework > +While most of the tests that are produced by the UnitTestFrameworkPkg are easy to step through in a debugger, the Framework > itself consumes code (mostly Cmocka) that sets its own build flags. These flags cause parts of the Framework to not > export symbols and captures exceptions, and as such are harder to debug. We have provided a Stuart parameter to force > symbolic debugging to be enabled. > @@ -269,15 +443,17 @@ stuart_ci_build -c .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019 -p MdePkg -t NOOP > ## Building and Running Host-Based Tests > > The EDK2 CI infrastructure provides a convenient way to run all host-based tests -- in the the entire tree or just > -selected packages -- and aggregate all the the reports, including highlighting any failures. This functionality is > -provided through the Stuart build system (published by EDK2-PyTools) and the `NOOPT` build target. > +selected packages -- and aggregate all the reports, including highlighting any failures. This functionality is > +provided through the Stuart build system (published by EDK2-PyTools) and the `NOOPT` build target. The sections that > +follow use Framework examples. Unit tests based on GoogleTest are built and run the same way. The text output and > +JUNIT XML output format have small differences. > > ### Building Locally > > First, to make sure you're working with the latest PyTools, run the following command: > > ```bash > -# Would recommend to run this in a Python venv, but that's out of scope for this doc. > +# Would recommend running this in a Python venv, but that's out of scope for this doc. > python -m pip install --upgrade -r ./pip-requirements.txt > ``` > > @@ -361,7 +537,7 @@ RUNNING TEST SUITE: Int Safe Conversions Test Suite > ``` > > You can also, if you are so inclined, read the output from the exact instance of the test that was run during > -`stuart_ci_build`. The ouput file can be found on a path that looks like: > +`stuart_ci_build`. The output file can be found on a path that looks like: > > `Build//HostTest//...result.xml` > > @@ -389,22 +565,30 @@ c:\_uefi\MdePkg\Test\UnitTest\Library\BaseSafeIntLib\TestBaseSafeIntLib.c:35: er > > ### XML Reporting Mode > > -Since these applications are built using the CMocka framework, they can also use the following env variables to output > -in a structured XML rather than text: > +Unit test applications using Framework are built using Cmocka that requires the > +following environment variables to be set to generate structured XML output > +rather than text: > > -```text > +``` > CMOCKA_MESSAGE_OUTPUT=xml > CMOCKA_XML_FILE= > ``` > > +Unit test applications using GoogleTest require the following environment > +variable to be set to generate structured XML output rather than text: > + > +``` > +GTEST_OUTPUT=xml: > +``` > + > This mode is used by the test running plugin to aggregate the results for CI test status reporting in the web view. > > ### Important Note > > -This works on both Windows and Linux, but is currently limited to x64 architectures. Working on getting others, but we > +This works on both Windows and Linux but is currently limited to x64 architectures. Working on getting others, but we > also welcome contributions. > > -## Known Limitations > +## Framework Known Limitations > > ### PEI, DXE, SMM > > @@ -418,7 +602,7 @@ PEI, DXE, and SMM is forthcoming, but should be considered beta/staging for now. > The host-based test framework is powered internally by the Cmocka framework. As such, it has abilities > that the target-based tests don't (yet). It would be awesome if this meant that it was a super set of > the target-based tests, and it worked just like the target-based tests but with more features. Unfortunately, > -this is not the case. While care has been taken to keep them as close a possible, there are a few known > +this is not the case. While care has been taken to keep them as close as possible, there are a few known > inconsistencies that we're still ironing out. For example, the logging messages in the target-based tests > are cached internally and associated with the running test case. They can be saved later as part of the > reporting lib. This isn't currently possible with host-based. Only the assertion failures are logged. > @@ -441,6 +625,9 @@ Non-Host-Based (PEI/DXE/SMM/Shell) Tests for a Functionality or Feature | Simi > ComponentY/ > ComponentY.inf > ComponentY.c > + GoogleTest/ > + ComponentYHostGoogleTest.inf # Host-Based Test for Driver Module > + ComponentYGoogleTest.cpp > UnitTest/ > ComponentYHostUnitTest.inf # Host-Based Test for Driver Module > ComponentYUnitTest.c > @@ -455,11 +642,23 @@ Non-Host-Based (PEI/DXE/SMM/Shell) Tests for a Functionality or Feature | Simi > SpecificLibDxe/ > SpecificLibDxe.c > SpecificLibDxe.inf > + GoogleTest/ # Host-Based Test for Specific Library Implementation > + SpecificLibDxeHostGoogleTest.cpp > + SpecificLibDxeHostGoogleTest.inf > UnitTest/ # Host-Based Test for Specific Library Implementation > SpecificLibDxeHostUnitTest.c > SpecificLibDxeHostUnitTest.inf > Test/ > HostTest.dsc # Host-Based Test Apps > + GoogleTest/ > + InterfaceX > + InterfaceXHostGoogleTest.inf # Host-Based App (should be in Test/HostTest.dsc) > + InterfaceXUnitTest.cpp # Test Logic > + > + GeneralPurposeLib/ # Host-Based Test for any implementation of GeneralPurposeLib > + GeneralPurposeLibTest.cpp > + GeneralPurposeLibHostUnitTest.inf > + > UnitTest/ > InterfaceX > InterfaceXHostUnitTest.inf # Host-Based App (should be in Test/HostTest.dsc) > diff --git a/UnitTestFrameworkPkg/Test/GoogleTest/Sample/SampleGoogleTest/SampleGoogleTest.cpp b/UnitTestFrameworkPkg/Test/GoogleTest/Sample/SampleGoogleTest/SampleGoogleTest.cpp > new file mode 100644 > index 000000000000..c83e58596a82 > --- /dev/null > +++ b/UnitTestFrameworkPkg/Test/GoogleTest/Sample/SampleGoogleTest/SampleGoogleTest.cpp > @@ -0,0 +1,263 @@ > +/** @file > + This is a sample to demonstrates the use of GoogleTest that supports host > + execution environments. > + > + Copyright (c) 2022, Intel Corporation. All rights reserved.
> + SPDX-License-Identifier: BSD-2-Clause-Patent > + > +**/ > + > +#include > +extern "C" { > + #include > + #include > + #include > +} > + > +/** > + Sample unit test that verifies the expected result of an unsigned integer > + addition operation. > +**/ > +TEST(SimpleMathTests, OnePlusOneShouldEqualTwo) { > + UINTN A; > + UINTN B; > + UINTN C; > + > + A = 1; > + B = 1; > + C = A + B; > + > + ASSERT_EQ (C, (UINTN)2); > +} > + > +/** > + Sample unit test that verifies that a global BOOLEAN is updatable. > +**/ > +class GlobalBooleanVarTests : public ::testing::Test { > + public: > + BOOLEAN SampleGlobalTestBoolean = FALSE; > +}; > + > +TEST_F(GlobalBooleanVarTests, GlobalBooleanShouldBeChangeable) { > + SampleGlobalTestBoolean = TRUE; > + ASSERT_TRUE (SampleGlobalTestBoolean); > + > + SampleGlobalTestBoolean = FALSE; > + ASSERT_FALSE (SampleGlobalTestBoolean); > +} > + > +/** > + Sample unit test that logs a warning message and verifies that a global > + pointer is updatable. > +**/ > +class GlobalVarTests : public ::testing::Test { > + public: > + VOID *SampleGlobalTestPointer = NULL; > + > + protected: > + void SetUp() override { > + ASSERT_EQ ((UINTN)SampleGlobalTestPointer, (UINTN)NULL); > + } > + void TearDown() { > + SampleGlobalTestPointer = NULL; > + } > +}; > + > +TEST_F(GlobalVarTests, GlobalPointerShouldBeChangeable) { > + SampleGlobalTestPointer = (VOID *)-1; > + ASSERT_EQ ((UINTN)SampleGlobalTestPointer, (UINTN)((VOID *)-1)); > +} > + > + > +/** > + Set PcdDebugPropertyMask for each MacroTestsAssertsEnabledDisabled test > +**/ > +class MacroTestsAssertsEnabledDisabled : public testing::TestWithParam { > + void SetUp() { > + PatchPcdSet8 (PcdDebugPropertyMask, GetParam()); > + } > +}; > + > +/** > + Sample unit test using the ASSERT_TRUE() macro. > +**/ > +TEST_P(MacroTestsAssertsEnabledDisabled, MacroAssertTrue) { > + UINT64 Result; > + > + // > + // This test passes because expression always evaluated to TRUE. > + // > + ASSERT_TRUE (TRUE); > + > + // > + // This test passes because expression always evaluates to TRUE. > + // > + Result = LShiftU64 (BIT0, 1); > + ASSERT_TRUE (Result == BIT1); > +} > + > +/** > + Sample unit test using the ASSERT_FALSE() macro. > +**/ > +TEST_P(MacroTestsAssertsEnabledDisabled, MacroAssertFalse) { > + UINT64 Result; > + > + // > + // This test passes because expression always evaluated to FALSE. > + // > + ASSERT_FALSE (FALSE); > + > + // > + // This test passes because expression always evaluates to FALSE. > + // > + Result = LShiftU64 (BIT0, 1); > + ASSERT_FALSE (Result == BIT0); > +} > + > +/** > + Sample unit test using the ASSERT_EQ() macro. > +**/ > +TEST_P(MacroTestsAssertsEnabledDisabled, MacroAssertEqual) { > + UINT64 Result; > + > + // > + // This test passes because both values are always equal. > + // > + ASSERT_EQ (1, 1); > + > + // > + // This test passes because both values are always equal. > + // > + Result = LShiftU64 (BIT0, 1); > + ASSERT_EQ (Result, (UINT64)BIT1); > +} > + > +/** > + Sample unit test using the ASSERT_STREQ() macro. > +**/ > +TEST_P(MacroTestsAssertsEnabledDisabled, MacroAssertMemEqual) { > + CHAR8 *String1; > + CHAR8 *String2; > + > + // > + // This test passes because String1 and String2 are the same. > + // > + String1 = (CHAR8 *)"Hello"; > + String2 = (CHAR8 *)"Hello"; > + ASSERT_STREQ (String1, String2); > +} > + > +/** > + Sample unit test using the ASSERT_NE() macro. > +**/ > +TEST_P(MacroTestsAssertsEnabledDisabled, MacroAssertNotEqual) { > + UINT64 Result; > + > + // > + // This test passes because both values are never equal. > + // > + ASSERT_NE (0, 1); > + > + // > + // This test passes because both values are never equal. > + // > + Result = LShiftU64 (BIT0, 1); > + ASSERT_NE (Result, (UINT64)BIT0); > +} > + > +/** > + Sample unit test using the ASSERT_TRUE() and ASSERT(FALSE) > + and EFI_EFFOR() macros to check status > +**/ > +TEST_P(MacroTestsAssertsEnabledDisabled, MacroAssertNotEfiError) { > + // > + // This test passes because the status is not an EFI error. > + // > + ASSERT_FALSE (EFI_ERROR (EFI_SUCCESS)); > + > + // > + // This test passes because the status is not an EFI error. > + // > + ASSERT_FALSE (EFI_ERROR (EFI_WARN_BUFFER_TOO_SMALL)); > +} > + > +/** > + Sample unit test using the ASSERT_EQ() macro to compare EFI_STATUS values. > +**/ > +TEST_P(MacroTestsAssertsEnabledDisabled, MacroAssertStatusEqual) { > + // > + // This test passes because the status value are always equal. > + // > + ASSERT_EQ (EFI_SUCCESS, EFI_SUCCESS); > +} > + > +/** > + Sample unit test using ASSERT_NE() macro to make sure a pointer is not NULL. > +**/ > +TEST_P(MacroTestsAssertsEnabledDisabled, MacroAssertNotNull) { > + UINT64 Result; > + > + // > + // This test passes because the pointer is never NULL. > + // > + ASSERT_NE (&Result, (UINT64 *)NULL); > +} > + > +/** > + Sample unit test using that should not generate any ASSERTs() > +**/ > +TEST_P(MacroTestsAssertsEnabledDisabled, MacroExpectNoAssertFailure) { > + // > + // This test passes because it never triggers an ASSERT(). > + // > + ASSERT (TRUE); > + > + // > + // This test passes because DecimalToBcd() does not ASSERT() if the > + // value passed in is <= 99. > + // > + DecimalToBcd8 (99); > +} > + > +/** > + Sample unit test using the ASSERT_DEATH() macro to test expected ASSERT()s. > +**/ > +TEST_P(MacroTestsAssertsEnabledDisabled, MacroExpectAssertFailure) { > + // > + // Skip tests that verify an ASSERT() is triggered if ASSERT()s are disabled. > + // > + if ((PcdGet8 (PcdDebugPropertyMask) & BIT0) == 0x00) { > + return; > + } > + > + // > + // This test passes because it directly triggers an ASSERT(). > + // > + ASSERT_DEATH (ASSERT (FALSE), ""); > + > + // > + // This test passes because DecimalToBcd() generates an ASSERT() if the > + // value passed in is >= 100. The expected ASSERT() is caught by the unit > + // test framework and ASSERT_DEATH() returns without an error. > + // > + ASSERT_DEATH (DecimalToBcd8 (101), ""); > +} > + > +INSTANTIATE_TEST_SUITE_P(ValidInput, > + MacroTestsAssertsEnabledDisabled, > + ::testing::Values(PcdGet8 (PcdDebugPropertyMask) | BIT0, PcdGet8 (PcdDebugPropertyMask) & (~BIT0))); > + > +/** > + Sample unit test using the SCOPED_TRACE() macro for trace messages. > +**/ > +TEST(MacroTestsMessages, MacroTraceMessage) { > + // > + // Example of logging. > + // > + SCOPED_TRACE ("SCOPED_TRACE message\n"); > +} > + > +int main(int argc, char* argv[]) { > + testing::InitGoogleTest(&argc, argv); > + return RUN_ALL_TESTS(); > +} > diff --git a/UnitTestFrameworkPkg/Test/GoogleTest/Sample/SampleGoogleTest/SampleGoogleTestHost.inf b/UnitTestFrameworkPkg/Test/GoogleTest/Sample/SampleGoogleTest/SampleGoogleTestHost.inf > new file mode 100644 > index 000000000000..37e7c86910ed > --- /dev/null > +++ b/UnitTestFrameworkPkg/Test/GoogleTest/Sample/SampleGoogleTest/SampleGoogleTestHost.inf > @@ -0,0 +1,35 @@ > +## @file > +# This is a sample to demonstrates the use of GoogleTest that supports host > +# execution environments. > +# > +# Copyright (c) 2022, Intel Corporation. All rights reserved.
> +# SPDX-License-Identifier: BSD-2-Clause-Patent > +## > + > +[Defines] > + INF_VERSION = 0x00010005 > + BASE_NAME = SampleGoogleTestHost > + FILE_GUID = 7D8BBFBB-7977-4AEE-A59F-257BF5C2F87C > + MODULE_TYPE = HOST_APPLICATION > + VERSION_STRING = 1.0 > + > +# > +# The following information is for reference only and not required by the build tools. > +# > +# VALID_ARCHITECTURES = IA32 X64 > +# > + > +[Sources] > + SampleGoogleTest.cpp > + > +[Packages] > + MdePkg/MdePkg.dec > + UnitTestFrameworkPkg/UnitTestFrameworkPkg.dec > + > +[LibraryClasses] > + GoogleTestLib > + BaseLib > + DebugLib > + > +[Pcd] > + gEfiMdePkgTokenSpaceGuid.PcdDebugPropertyMask > diff --git a/UnitTestFrameworkPkg/Test/UnitTestFrameworkPkgHostTest.dsc b/UnitTestFrameworkPkg/Test/UnitTestFrameworkPkgHostTest.dsc > index 184fdec87acf..708ef7f9ab35 100644 > --- a/UnitTestFrameworkPkg/Test/UnitTestFrameworkPkgHostTest.dsc > +++ b/UnitTestFrameworkPkg/Test/UnitTestFrameworkPkgHostTest.dsc > @@ -23,14 +23,16 @@ [PcdsPatchableInModule] > > [Components] > # > - # Build HOST_APPLICATION that tests the SampleUnitTest > + # Build HOST_APPLICATIONs that test the SampleUnitTest > # > UnitTestFrameworkPkg/Test/UnitTest/Sample/SampleUnitTest/SampleUnitTestHost.inf > + UnitTestFrameworkPkg/Test/GoogleTest/Sample/SampleGoogleTest/SampleGoogleTestHost.inf > > # > # Build HOST_APPLICATION Libraries > # > UnitTestFrameworkPkg/Library/CmockaLib/CmockaLib.inf > + UnitTestFrameworkPkg/Library/GoogleTestLib/GoogleTestLib.inf > UnitTestFrameworkPkg/Library/Posix/DebugLibPosix/DebugLibPosix.inf > UnitTestFrameworkPkg/Library/Posix/MemoryAllocationLibPosix/MemoryAllocationLibPosix.inf > UnitTestFrameworkPkg/Library/UnitTestLib/UnitTestLibCmocka.inf > diff --git a/UnitTestFrameworkPkg/UnitTestFrameworkPkg.ci.yaml b/UnitTestFrameworkPkg/UnitTestFrameworkPkg.ci.yaml > index 77d51e13484c..072df6208c92 100644 > --- a/UnitTestFrameworkPkg/UnitTestFrameworkPkg.ci.yaml > +++ b/UnitTestFrameworkPkg/UnitTestFrameworkPkg.ci.yaml > @@ -78,7 +78,8 @@ > "SpellCheck": { > "AuditOnly": False, # Fails test but run in AuditOnly mode to collect log > "IgnoreFiles": [ # use gitignore syntax to ignore errors in matching files > - "Library/CmockaLib/cmocka/**/*.*" # not going to spell check a submodule > + "Library/CmockaLib/cmocka/**/*.*", # not going to spell check a submodule > + "Library/GoogleTestLib/googletest/**/*.*" # not going to spell check a submodule > ], > "ExtendWords": [ # words to extend to the dictionary for this package > "testcase", > @@ -91,6 +92,7 @@ > "NOFAILURE", > "cmockery", > "DHAVE", # build flag for cmocka in the INF > + "gtest", # file name in GoogleTestLib.inf > "corthon", # Contact GitHub account in Readme > "mdkinney", # Contact GitHub account in Readme > "spbrogan" # Contact GitHub account in Readme > diff --git a/UnitTestFrameworkPkg/UnitTestFrameworkPkg.dec b/UnitTestFrameworkPkg/UnitTestFrameworkPkg.dec > index 069289f00969..ed12f32009d8 100644 > --- a/UnitTestFrameworkPkg/UnitTestFrameworkPkg.dec > +++ b/UnitTestFrameworkPkg/UnitTestFrameworkPkg.dec > @@ -16,11 +16,15 @@ [Defines] > PACKAGE_VERSION = 1.00 > > [Includes] > + Include > Library/CmockaLib/cmocka/include > + Library/GoogleTestLib/googletest/googletest/include > + Library/GoogleTestLib/googletest/googlemock/include > > [Includes.Common.Private] > PrivateInclude > Library/CmockaLib/cmocka/include/cmockery > + Library/GoogleTestLib/googletest/googletest > > [LibraryClasses.Common.Private] > ## @libraryclass Allows save and restore unit test internal state > @@ -35,6 +39,10 @@ [LibraryClasses.Common.Private] > # > UnitTestBootLib|PrivateInclude/Library/UnitTestBootLib.h > > + ## @libraryclass GoogleTest infrastructure > + # > + GoogleTestLib|Include/Library/GoogleTestLib.h > + > [Guids] > gUnitTestFrameworkPkgTokenSpaceGuid = { 0x833d3aba, 0x39b4, 0x43a2, { 0xb9, 0x30, 0x7a, 0x34, 0x53, 0x39, 0x31, 0xb3 } } > > diff --git a/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc b/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc > index 9beeaef1ba5e..8009337552cc 100644 > --- a/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc > +++ b/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc > @@ -14,6 +14,7 @@ [LibraryClasses.common.HOST_APPLICATION] > CpuLib|MdePkg/Library/BaseCpuLibNull/BaseCpuLibNull.inf > CacheMaintenanceLib|MdePkg/Library/BaseCacheMaintenanceLibNull/BaseCacheMaintenanceLibNull.inf > CmockaLib|UnitTestFrameworkPkg/Library/CmockaLib/CmockaLib.inf > + GoogleTestLib|UnitTestFrameworkPkg/Library/GoogleTestLib/GoogleTestLib.inf > UnitTestLib|UnitTestFrameworkPkg/Library/UnitTestLib/UnitTestLibCmocka.inf > DebugLib|UnitTestFrameworkPkg/Library/Posix/DebugLibPosix/DebugLibPosix.inf > MemoryAllocationLib|UnitTestFrameworkPkg/Library/Posix/MemoryAllocationLibPosix/MemoryAllocationLibPosix.inf > @@ -31,6 +32,7 @@ [BuildOptions.common.EDKII.HOST_APPLICATION] > # > # MSFT > # > + MSFT:*_*_*_CC_FLAGS = /EHsc > MSFT:*_*_*_DLINK_FLAGS == /out:"$(BIN_DIR)\$(MODULE_NAME_GUID).exe" /pdb:"$(BIN_DIR)\$(MODULE_NAME_GUID).pdb" /IGNORE:4001 /NOLOGO /SUBSYSTEM:CONSOLE /DEBUG /STACK:0x40000,0x40000 /NODEFAULTLIB:libcmt.lib libcmtd.lib > MSFT:*_*_IA32_DLINK_FLAGS = /MACHINE:I386 > MSFT:*_*_X64_DLINK_FLAGS = /MACHINE:AMD64 > @@ -50,7 +52,7 @@ [BuildOptions.common.EDKII.HOST_APPLICATION] > # > GCC:*_*_IA32_DLINK_FLAGS == -o $(BIN_DIR)/$(MODULE_NAME_GUID) -m32 -no-pie > GCC:*_*_X64_DLINK_FLAGS == -o $(BIN_DIR)/$(MODULE_NAME_GUID) -m64 -no-pie > - GCC:*_*_*_DLINK2_FLAGS == -lgcov > + GCC:*_*_*_DLINK2_FLAGS == -lgcov -lpthread -lstdc++ -lm > > # > # Need to do this link via gcc and not ld as the pathing to libraries changes from OS version to OS version > -- > 2.37.1.windows.1 >