From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mga09.intel.com (mga09.intel.com []) by mx.groups.io with SMTP id smtpd.web12.7269.1579831836643147171 for ; Thu, 23 Jan 2020 18:10:37 -0800 Authentication-Results: mx.groups.io; dkim=missing; spf=fail (domain: intel.com, ip: , mailfrom: michael.d.kinney@intel.com) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 23 Jan 2020 18:10:35 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,356,1574150400"; d="scan'208";a="375395294" Received: from unknown (HELO mdkinney-MOBL2.amr.corp.intel.com) ([10.241.98.74]) by orsmga004.jf.intel.com with ESMTP; 23 Jan 2020 18:10:35 -0800 From: "Michael D Kinney" To: devel@edk2.groups.io Cc: Sean Brogan , Bret Barkelew Subject: [Patch 04/11] UnitTestFrameworkPkg: Add public and private interfaces Date: Thu, 23 Jan 2020 18:10:25 -0800 Message-Id: <20200124021032.13808-5-michael.d.kinney@intel.com> X-Mailer: git-send-email 2.21.0.windows.1 In-Reply-To: <20200124021032.13808-1-michael.d.kinney@intel.com> References: <20200124021032.13808-1-michael.d.kinney@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Add public interfaces for use by unit test implementations. * Include path to cmocka library interfaces. * PcdUnitTestLogLevel to set the unit test logging message level to filter log messages. Add private interfaces that are used by UnitTestLib implementations. * [Private] UnitTestBootLib - Set boot next to continue unit tests across a reboot. * [Private] UnitTestPersistenceLib - Save unit test framework state to a persistent storage device. * [Private] UnitTestResultLib - Output unit test results to a console device. * [Private] UnitTestFrameworkTypes.h - Internal structures used by UnitTestLib implementations to keep track if unit test frameworks, unit test suites, and unit tests along with the serialized storage format to save a unit test framework state to persistent storage. Cc: Sean Brogan Cc: Bret Barkelew Signed-off-by: Michael D Kinney --- .../PrivateInclude/Library/UnitTestBootLib.h | 31 +++ .../Library/UnitTestPersistenceLib.h | 76 ++++++ .../Library/UnitTestResultReportLib.h | 27 ++ .../PrivateInclude/UnitTestFrameworkTypes.h | 183 +++++++++++++ UnitTestFrameworkPkg/ReadMe.md | 257 ++++++++++++++++++ UnitTestFrameworkPkg/UnitTestFrameworkPkg.dec | 50 ++++ UnitTestFrameworkPkg/UnitTestFrameworkPkg.uni | 21 ++ 7 files changed, 645 insertions(+) create mode 100644 UnitTestFrameworkPkg/PrivateInclude/Library/UnitTestBootLib.h create mode 100644 UnitTestFrameworkPkg/PrivateInclude/Library/UnitTestPersistenceLib.h create mode 100644 UnitTestFrameworkPkg/PrivateInclude/Library/UnitTestResultReportLib.h create mode 100644 UnitTestFrameworkPkg/PrivateInclude/UnitTestFrameworkTypes.h create mode 100644 UnitTestFrameworkPkg/ReadMe.md create mode 100644 UnitTestFrameworkPkg/UnitTestFrameworkPkg.dec create mode 100644 UnitTestFrameworkPkg/UnitTestFrameworkPkg.uni diff --git a/UnitTestFrameworkPkg/PrivateInclude/Library/UnitTestBootLib.h b/UnitTestFrameworkPkg/PrivateInclude/Library/UnitTestBootLib.h new file mode 100644 index 0000000000..d90bff0e4c --- /dev/null +++ b/UnitTestFrameworkPkg/PrivateInclude/Library/UnitTestBootLib.h @@ -0,0 +1,31 @@ +/** @file + Provides a library function that can be customized to set the platform to boot + from USB on the next boot. This allows the test framework to reboot back to + USB. Since boot managers are not all the same creating a lib to support + platform customization will make porting to new code base/platform easier. + + Copyright (c) Microsoft Corporation.
+ Copyright (c) 2019 - 2020, Intel Corporation. All rights reserved.
+ SPDX-License-Identifier: BSD-2-Clause-Patent + +**/ + +#ifndef __UNIT_TEST_BOOT_LIB_H__ +#define __UNIT_TEST_BOOT_LIB_H__ + +/** + Set the boot manager to boot from a specific device on the next boot. This + should be set only for the next boot and shouldn't require any manual clean up + + @retval EFI_SUCCESS Boot device for next boot was set. + @retval EFI_UNSUPPORTED Setting the boot device for the next boot is not + supportted. + @retval Other Boot device for next boot can not be set. +**/ +EFI_STATUS +EFIAPI +SetBootNextDevice ( + VOID + ); + +#endif diff --git a/UnitTestFrameworkPkg/PrivateInclude/Library/UnitTestPersistenceLib.h b/UnitTestFrameworkPkg/PrivateInclude/Library/UnitTestPersistenceLib.h new file mode 100644 index 0000000000..af19ba8f53 --- /dev/null +++ b/UnitTestFrameworkPkg/PrivateInclude/Library/UnitTestPersistenceLib.h @@ -0,0 +1,76 @@ +/** @file + This header file describes a library that contains functions to save and + restore unit test internal state, in case the test needs to pause and resume + (eg. a reboot-based test). + + Copyright (c) Microsoft Corporation.
+ Copyright (c) 2019 - 2020, Intel Corporation. All rights reserved.
+ SPDX-License-Identifier: BSD-2-Clause-Patent + +**/ + +#ifndef _UNIT_TEST_PERSISTENCE_LIB_H_ +#define _UNIT_TEST_PERSISTENCE_LIB_H_ + +#include + +#define UNIT_TEST_PERSISTENCE_LIB_VERSION 1 + +/** + Determines whether a persistence cache already exists for + the given framework. + + @param[in] FrameworkHandle A pointer to the framework that is being persisted. + + @retval TRUE + @retval FALSE Cache doesn't exist or an error occurred. + +**/ +BOOLEAN +EFIAPI +DoesCacheExist ( + IN UNIT_TEST_FRAMEWORK_HANDLE FrameworkHandle + ); + +/** + Will save the data associated with an internal Unit Test Framework + state in a manner that can persist a Unit Test Application quit or + even a system reboot. + + @param[in] FrameworkHandle A pointer to the framework that is being persisted. + @param[in] SaveData A pointer to the buffer containing the serialized + framework internal state. + + @retval EFI_SUCCESS Data is persisted and the test can be safely quit. + @retval Others Data is not persisted and test cannot be resumed upon exit. + +**/ +EFI_STATUS +EFIAPI +SaveUnitTestCache ( + IN UNIT_TEST_FRAMEWORK_HANDLE FrameworkHandle, + IN UNIT_TEST_SAVE_HEADER *SaveData + ); + +/** + Will retrieve any cached state associated with the given framework. + Will allocate a buffer to hold the loaded data. + + @param[in] FrameworkHandle A pointer to the framework that is being persisted. + @param[in] SaveData A pointer pointer that will be updated with the address + of the loaded data buffer. + + @retval EFI_SUCCESS Data has been loaded successfully and SaveData is updated + with a pointer to the buffer. + @retval Others An error has occurred and no data has been loaded. SaveData + is set to NULL. + +**/ +EFI_STATUS +EFIAPI +LoadUnitTestCache ( + IN UNIT_TEST_FRAMEWORK_HANDLE FrameworkHandle, + OUT UNIT_TEST_SAVE_HEADER **SaveData + ); + +#endif diff --git a/UnitTestFrameworkPkg/PrivateInclude/Library/UnitTestResultReportLib.h b/UnitTestFrameworkPkg/PrivateInclude/Library/UnitTestResultReportLib.h new file mode 100644 index 0000000000..a417f490dc --- /dev/null +++ b/UnitTestFrameworkPkg/PrivateInclude/Library/UnitTestResultReportLib.h @@ -0,0 +1,27 @@ +/** @file + Provides a unit test result report. This allows new result output formats to + be easily customized. + + Copyright (c) Microsoft Corporation.
+ Copyright (c) 2019 - 2020, Intel Corporation. All rights reserved.
+ SPDX-License-Identifier: BSD-2-Clause-Patent + +**/ + +#ifndef __UNIT_TEST_RESULT_REPORT_LIB_H__ +#define __UNIT_TEST_RESULT_REPORT_LIB_H__ + +#include + +/** +Method to produce the Unit Test run results + +@retval Success +**/ +EFI_STATUS +EFIAPI +OutputUnitTestFrameworkReport ( + IN UNIT_TEST_FRAMEWORK_HANDLE FrameworkHandle + ); + +#endif diff --git a/UnitTestFrameworkPkg/PrivateInclude/UnitTestFrameworkTypes.h b/UnitTestFrameworkPkg/PrivateInclude/UnitTestFrameworkTypes.h new file mode 100644 index 0000000000..e58b30093e --- /dev/null +++ b/UnitTestFrameworkPkg/PrivateInclude/UnitTestFrameworkTypes.h @@ -0,0 +1,183 @@ +/** @file + Provides the basic types and common elements of the unit test framework + + Copyright (c) Microsoft Corporation.
+ Copyright (c) 2019 - 2020, Intel Corporation. All rights reserved.
+ SPDX-License-Identifier: BSD-2-Clause-Patent + +**/ + +#ifndef __UNIT_TEST_TYPES_H__ +#define __UNIT_TEST_TYPES_H__ + +#include + +/// +/// The maximum length of a string stored in the unit test framework +/// +#define UNIT_TEST_MAX_STRING_LENGTH (120) + +/// +/// The size of a firngerprint used to save/resume execution of a unit test +/// framework. This is the size of a CRC32 value which is 32-bit value. +/// +/// +#define UNIT_TEST_FINGERPRINT_SIZE (sizeof (UINT32)) + +/// +/// The maximum length of a test failure message stored in the unit test +/// framework +/// +#define UNIT_TEST_TESTFAILUREMSG_LENGTH (120) + +/// +/// FAILURE_TYPE used to record the type of assert that was triggered by a unit +/// test. +/// +typedef UINT32 FAILURE_TYPE; +#define FAILURETYPE_NOFAILURE (0) +#define FAILURETYPE_OTHER (1) +#define FAILURETYPE_ASSERTTRUE (2) +#define FAILURETYPE_ASSERTFALSE (3) +#define FAILURETYPE_ASSERTEQUAL (4) +#define FAILURETYPE_ASSERTNOTEQUAL (5) +#define FAILURETYPE_ASSERTNOTEFIERROR (6) +#define FAILURETYPE_ASSERTSTATUSEQUAL (7) +#define FAILURETYPE_ASSERTNOTNULL (8) + +/// +/// Unit Test context structure tracked by the unit test framework. +/// +typedef struct { + CHAR8 *Description; + CHAR8 *Name; //can't have spaces and should be short + CHAR8 *Log; + FAILURE_TYPE FailureType; + CHAR8 FailureMessage[UNIT_TEST_TESTFAILUREMSG_LENGTH]; + UINT8 Fingerprint[UNIT_TEST_FINGERPRINT_SIZE]; + UNIT_TEST_STATUS Result; + UNIT_TEST_FUNCTION RunTest; + UNIT_TEST_PREREQUISITE Prerequisite; + UNIT_TEST_CLEANUP CleanUp; + UNIT_TEST_CONTEXT Context; + UNIT_TEST_SUITE_HANDLE ParentSuite; +} UNIT_TEST; + +/// +/// Structure used to store the set of unit tests in a unit test suite as a list. +/// +typedef struct { + LIST_ENTRY Entry; + UNIT_TEST UT; +} UNIT_TEST_LIST_ENTRY; + +/// +/// Unit Test Suite context structure tracked by the unit test framework. +/// +typedef struct { + UINTN NumTests; + CHAR8 *Title; + CHAR8 *Name; + UINT8 Fingerprint[UNIT_TEST_FINGERPRINT_SIZE]; + UNIT_TEST_SUITE_SETUP Setup; + UNIT_TEST_SUITE_TEARDOWN Teardown; + LIST_ENTRY TestCaseList; // UNIT_TEST_LIST_ENTRY + UNIT_TEST_FRAMEWORK_HANDLE ParentFramework; +} UNIT_TEST_SUITE; + +/// +/// Structure used to store the set of unit test suites in a unit test framework +/// as a list. +/// +typedef struct { + LIST_ENTRY Entry; + UNIT_TEST_SUITE UTS; +} UNIT_TEST_SUITE_LIST_ENTRY; + +/// +/// Unit Test Framework context structure tracked by the unit test framework. +/// +typedef struct { + CHAR8 *Title; + CHAR8 *ShortTitle; // This title should contain NO spaces or non-filename characters. Is used in reporting and serialization. + CHAR8 *VersionString; + CHAR8 *Log; + UINT8 Fingerprint[UNIT_TEST_FINGERPRINT_SIZE]; + LIST_ENTRY TestSuiteList; // UNIT_TEST_SUITE_LIST_ENTRY + EFI_TIME StartTime; + EFI_TIME EndTime; + UNIT_TEST *CurrentTest; + VOID *SavedState; // This is an instance of UNIT_TEST_SAVE_HEADER*, if present. +} UNIT_TEST_FRAMEWORK; + +/// +/// Serialized version of a unit test +/// +typedef struct { + UINT32 Size; // Size of the UNIT_TEST_SAVE_TEST including Log[] + UINT8 Fingerprint[UNIT_TEST_FINGERPRINT_SIZE]; // Fingerprint of the test itself. + CHAR8 FailureMessage[UNIT_TEST_TESTFAILUREMSG_LENGTH]; + FAILURE_TYPE FailureType; + UNIT_TEST_STATUS Result; + CHAR8 Log[]; +} UNIT_TEST_SAVE_TEST; + +/// +/// Serialized version of a unit test context +/// +typedef struct { + UINT32 Size; // Size of the UNIT_TEST_SAVE_CONTEXT including Data[] + UINT8 Fingerprint[UNIT_TEST_FINGERPRINT_SIZE]; // Fingerprint of the corresponding test. + UINT8 Data[]; // Actual data of the context. +} UNIT_TEST_SAVE_CONTEXT; + +/// +/// Serialized version of unit test framework +/// +typedef struct { + UINT8 Version; + UINT32 SaveStateSize; // Size of the entire serialized buffer. + UINT8 Fingerprint[UNIT_TEST_FINGERPRINT_SIZE]; // Fingerprint of the framework that has been saved. + EFI_TIME StartTime; + UINT32 TestCount; + BOOLEAN HasSavedContext; + // UNIT_TEST_SAVE_TEST Tests[]; // Array of structures starts here. + // UNIT_TEST_SAVE_CONTEXT SavedContext[]; // Saved context for the currently running test. + // CHAR8 Log[]; // NOTE: Not yet implemented!! +} UNIT_TEST_SAVE_HEADER; + +/** + This function is responsible for initializing the log buffer for a single test. It can + be used internally, but may also be consumed by the test framework to add pre-existing + data to a log before it's used. + + @param[in,out] TestHandle A handle to the test being initialized. + @param[in] Buffer [Optional] A pointer to pre-existing log data that should + be used to initialize the log. Should include a NULL terminator. + @param[in] BufferSize [Optional] The size of the pre-existing log data. + +**/ +VOID +EFIAPI +UnitTestLogInit ( + IN OUT UNIT_TEST *Test, + IN UINT8 *Buffer OPTIONAL, + IN UINTN BufferSize OPTIONAL + ); + +/** + Internal helper function to return a handle to the currently executing framework. + This function is generally used for communication within the UnitTest framework, but is + defined here so that it can be consumed by the Assertion and Logging macros. + + There should be no need to consume as a test writer, but it's there if you need it. + + @retval Handle to the currently executing test framework. + +**/ +UNIT_TEST_FRAMEWORK_HANDLE +GetActiveFrameworkHandle ( + VOID + ); + +#endif diff --git a/UnitTestFrameworkPkg/ReadMe.md b/UnitTestFrameworkPkg/ReadMe.md new file mode 100644 index 0000000000..7296f0a45c --- /dev/null +++ b/UnitTestFrameworkPkg/ReadMe.md @@ -0,0 +1,257 @@ +# Unit Test Framework Package + +## About + +This package adds a unit test framework capable of building tests for multiple contexts including +the UEFI shell environment and host-based environments. It allows for unit test development to focus +on the tests and leave error logging, result formatting, context persistance, and test running to the framework. +The unit test framework works well for low level unit tests as well as system level tests and +fits easily in automation frameworks. + +### UnitTestLib + +The main "framework" library. The core of the framework is the Framework object, which can have any number +of test cases and test suites registered with it. The Framework object is also what drives test execution. + +The Framework also provides helper macros and functions for checking test conditions and +reporting errors. Status and error info will be logged into the test context. There are a number +of Assert macros that make the unit test code friendly to view and easy to understand. + +Finally, the Framework also supports logging strings during the test execution. This data is logged +to the test context and will be available in the test reporting phase. This should be used for +logging test details and helpful messages to resolve test failures. + +### UnitTestPersistenceLib + +Persistence lib has the main job of saving and restoring test context to a storage medium so that for tests +that require exiting the active process and then resuming state can be maintained. This is critical +in supporting a system reboot in the middle of a test run. + +### UnitTestResultReportLib + +Library provides function to run at the end of a framework test run and handles formatting the report. +This is a common customization point and allows the unit test framework to fit its output reports into +other test infrastructure. In this package a simple library instances has been supplied to output test +results to the console as plain text. + +## Samples + +There is a sample unit test provided as both an example of how to write a unit test and leverage +many of the features of the framework. This sample can be found in the `Test/UnitTest/Sample/SampleUnitTest` +directory. + +The sample is provided in PEI, SMM, DXE, and UEFI App flavors. It also has a flavor for the HOST_APPLICATION +build type, which can be run on a host system without needing a target. + +## Usage + +This section is built a lot like a "Getting Started". We'll go through some of the components that are needed +when constructing a unit test and some of the decisions that are made by the test writer. We'll also describe +how to check for expected conditions in test cases and a bit of the logging characteristics. + +Most of these examples will refer to the SampleUnitTestUefiShell app found in this package. + +### Requirements - INF + +In our INF file, we'll need to bring in the `UnitTestLib` library. Conveniently, the interface +header for the `UnitTestLib` is located in `MdePkg`, so you shouldn't need to depend on any other +packages. As long as your DSC file knows where to find the lib implementation that you want to use, +you should be good to go. + +See this example in 'SampleUnitTestApp.inf'... + +``` +[Packages] + MdePkg/MdePkg.dec + +[LibraryClasses] + UefiApplicationEntryPoint + BaseLib + DebugLib + UnitTestLib + PrintLib +``` + +### Requirements - Code + +Not to state the obvious, but let's make sure we have the following include before getting too far along... + +```c +#include +``` + +Now that we've got that squared away, let's look at our 'Main()'' routine (or DriverEntryPoint() or whatever). + +### Configuring the Framework + +Everything in the UnitTestPkg framework is built around an object called -- conveniently -- the Framework. +This Framework object will contain all the information about our test, the test suites and test cases associated +with it, the current location within the test pass, and any results that have been recorded so far. + +To get started with a test, we must first create a Framework instance. The function for this is +`InitUnitTestFramework`. It takes in `CHAR8` strings for the long name, short name, and test version. +The long name and version strings are just for user presentation and relatively flexible. The short name +will be used to name any cache files and/or test results, so should be a name that makes sense in that context. +These strings are copied internally to the Framework, so using stack-allocated or literal strings is fine. + +In the 'SampleUnitTestUefiShell' app, the module name is used as the short name, so the init looks like this. + +```c +DEBUG(( DEBUG_INFO, "%a v%a\n", UNIT_TEST_APP_NAME, UNIT_TEST_APP_VERSION )); + +// +// Start setting up the test framework for running the tests. +// +Status = InitUnitTestFramework( &Framework, UNIT_TEST_APP_NAME, gEfiCallerBaseName, UNIT_TEST_APP_VERSION ); +``` + +The `&Framework` returned here is the handle to the Framework. If it's successfully returned, we can start adding +test suites and test cases. + +Test suites exist purely to help organize test cases and to differentiate the results in reports. If you're writing +a small unit test, you can conceivably put all test cases into a single suite. However, if you end up with 20+ test +cases, it may be beneficial to organize them according to purpose. You _must_ have at least one test suite, even if +it's just a catch-all. The function to create a test suite is `CreateUnitTestSuite`. It takes in a handle to +the Framework object, a `CHAR8` string for the suite title and package name, and optional function pointers for +a setup function and a teardown function. + +The suite title is for user presentation. The package name is for xUnit type reporting and uses a '.'-separated +hierarchical format (see 'SampleUnitTestApp' for example). If provided, the setup and teardown functions will be +called once at the start of the suite (before _any_ tests have run) and once at the end of the suite (after _all_ +tests have run), respectively. If either or both of these are unneeded, pass `NULL`. The function prototypes are +`UNIT_TEST_SUITE_SETUP` and `UNIT_TEST_SUITE_TEARDOWN`. + +Looking at 'SampleUnitTestUefiShell' app, you can see that the first test suite is created as below... + +```c +// +// Populate the SimpleMathTests Unit Test Suite. +// +Status = CreateUnitTestSuite( &SimpleMathTests, Fw, "Simple Math Tests", "Sample.Math", NULL, NULL ); +``` + +This test suite has no setup or teardown functions. The `&SimpleMathTests` returned here is a handle to the suite and +will be used when adding test cases. + +Great! Now we've finished some of the cruft, red tape, and busy work. We're ready to add some tests. Adding a test +to a test suite is accomplished with the -- you guessed it -- `AddTestCase` function. It takes in the suite handle; +a `CHAR8` string for the description and class name; a function pointer for the test case itself; additional, optional +function pointers for prerequisite check and cleanup routines; and and optional pointer to a context structure. + +Okay, that's a lot. Let's take it one piece at a time. The description and class name strings are very similar in +usage to the suite title and package name strings in the test suites. The former is for user presentation and the +latter is for xUnit parsing. The test case function pointer is what is actually executed as the "test" and the +prototype should be `UNIT_TEST_FUNCTION`. The last three parameters require a little bit more explaining. + +The prerequisite check function has a prototype of `UNIT_TEST_PREREQUISITE` and -- if provided -- will be called +immediately before the test case. If this function returns any error, the test case will not be run and will be +recorded as `UNIT_TEST_ERROR_PREREQUISITE_NOT_MET`. The cleanup function (prototype `UNIT_TEST_CLEANUP`) will be called +immediately after the test case to provide an opportunity to reset any global state that may have been changed in the +test case. In the event of a prerequisite failure, the cleanup function will also be skipped. If either of these +functions is not needed, pass `NULL`. + +The context pointer is entirely case-specific. It will be passed to the test case upon execution. One of the purposes +of the context pointer is to allow test case reuse with different input data. (Another use is for testing that wraps +around a system reboot, but that's beyond the scope of this guide.) The test case must know how to interpret the context +pointer, so it could be a simple value, or it could be a complex structure. If unneeded, pass `NULL`. + +In 'SampleUnitTestUefiShell' app, the first test case is added using the code below... + +```c +AddTestCase( SimpleMathTests, "Adding 1 to 1 should produce 2", "Addition", OnePlusOneShouldEqualTwo, NULL, NULL, NULL ); +``` + +This test case calls the function `OnePlusOneShouldEqualTwo` and has no prerequisite, cleanup, or context. + +Once all the suites and cases are added, it's time to run the Framework. + +```c +// +// Execute the tests. +// +Status = RunAllTestSuites( Framework ); +``` + +### A Simple Test Case + +We'll take a look at the below test case from 'SampleUnitTestApp'... + +```c +UNIT_TEST_STATUS +EFIAPI +OnePlusOneShouldEqualTwo ( + IN UNIT_TEST_FRAMEWORK_HANDLE Framework, + IN UNIT_TEST_CONTEXT Context + ) +{ + UINTN A, B, C; + + A = 1; + B = 1; + C = A + B; + + UT_ASSERT_EQUAL(C, 2); + return UNIT_TEST_PASSED; +} // OnePlusOneShouldEqualTwo() +``` + +The prototype for this function matches the `UNIT_TEST_FUNCTION` prototype. It takes in a handle to the Framework +itself and the context pointer. The context pointer could be cast and interpreted as anything within this test case, +which is why it's important to configure contexts carefully. The test case returns a value of `UNIT_TEST_STATUS`, which +will be recorded in the Framework and reported at the end of all suites. + +In this test case, the `UT_ASSERT_EQUAL` assertion is being used to establish that the business logic has functioned +correctly. There are several assertion macros, and you are encouraged to use one that matches as closely to your +intended test criterium as possible, because the logging is specific to the macro and more specific macros have more +detailed logs. When in doubt, there are always `UT_ASSERT_TRUE` and `UT_ASSERT_FALSE`. Assertion macros that fail their +test criterium will immediately return from the test case with `UNIT_TEST_ERROR_TEST_FAILED` and log an error string. +_Note_ that this early return can have implications for memory leakage. + +At the end, if all test criteria pass, you should return `UNIT_TEST_PASSED`. + +### More Complex Cases + +To write more advanced tests, first take a look at all the Assertion and Logging macros provided in the framework. + +Beyond that, if you're writing host-based tests and want to take a dependency on the UnitTestFrameworkPkg, you can +leverage the `cmocka.h` interface and write tests with all the features of the Cmocka framework. + +Documentation for Cmocka can be found here: +https://api.cmocka.org/ + +## Development + +When using the EDK2 Pytools for CI testing, the host-based unit tests will be built and run on any build that includes the `NOOPT` build target. + +If you are trying to iterate on a single test, a convenient pattern is to build only that test module. For example, the following command will build only the SafeIntLib host-based test from the MdePkg... + +```bash +stuart_ci_build -c .pytool/CISettings.py TOOL_CHAIN_TAG=VS2017 -p MdePkg -t NOOPT BUILDMODULE=MdePkg/Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLib.inf +``` + +## Known Limitations + +### PEI, DXE, SMM + +While sample tests have been provided for these execution environments, only cursory build validation +has been performed. Care has been taken while designing the frameworks to allow for execution during +boot phases, but only UEFI Shell and host-based tests have been thoroughly evaluated. Full support for +PEI, DXE, and SMM is forthcoming, but should be considered beta/staging for now. + +### Host-Based Support vs Other Tests + +The host-based test framework is powered internally by the Cmocka framework. As such, it has abilities +that the target-based tests don't (yet). It would be awesome if this meant that it was a super set of +the target-based tests, and it worked just like the target-based tests but with more features. Unfortunately, +this is not the case. While care has been taken to keep them as close a possible, there are a few known +inconsistencies that we're still ironing out. For example, the logging messages in the target-based tests +are cached internally and associated with the running test case. They can be saved later as part of the +reporting lib. This isn't currently possible with host-based. Only the assertion failures are logged. + +We will continue trying to make these as similar as possible. + +## Copyright + +Copyright (c) Microsoft Corporation. +SPDX-License-Identifier: BSD-2-Clause-Patent + diff --git a/UnitTestFrameworkPkg/UnitTestFrameworkPkg.dec b/UnitTestFrameworkPkg/UnitTestFrameworkPkg.dec new file mode 100644 index 0000000000..069289f009 --- /dev/null +++ b/UnitTestFrameworkPkg/UnitTestFrameworkPkg.dec @@ -0,0 +1,50 @@ +## @file +# This Package provides all definitions (including functions, MACROs, +# structures library classes, and PCDs) and libraries instances, which are used +# to support unit testing and interface testing. +# +# Copyright (c) Microsoft Corporation.
+# SPDX-License-Identifier: BSD-2-Clause-Patent +# +## + +[Defines] + DEC_SPECIFICATION = 0x00010005 + PACKAGE_NAME = UnitTestFrameworkPkg + PACKAGE_UNI_FILE = UnitTestFrameworkPkg.uni + PACKAGE_GUID = 4A70C4A0-D72C-4D3F-9943-BE7C41C50BA3 + PACKAGE_VERSION = 1.00 + +[Includes] + Library/CmockaLib/cmocka/include + +[Includes.Common.Private] + PrivateInclude + Library/CmockaLib/cmocka/include/cmockery + +[LibraryClasses.Common.Private] + ## @libraryclass Allows save and restore unit test internal state + # + UnitTestPersistenceLib|PrivateInclude/Library/UnitTestPersistenceLib.h + + ## @libraryclass Provides a unit test result report + # + UnitTestResultReportLib|PrivateInclude/Library/UnitTestResultReportLib.h + + ## @libraryclass Provides boot-option routines useful in shell-based tests. + # + UnitTestBootLib|PrivateInclude/Library/UnitTestBootLib.h + +[Guids] + gUnitTestFrameworkPkgTokenSpaceGuid = { 0x833d3aba, 0x39b4, 0x43a2, { 0xb9, 0x30, 0x7a, 0x34, 0x53, 0x39, 0x31, 0xb3 } } + +[PcdsFixedAtBuild] + ## This flag is used to control build time optimization based on unit test + # log level. The default value is 0xFFFFFFFF to enable all unit test log + # messages. + # BIT0 - Error unit test log messages.
+ # BIT1 - Warning unit test log messages.
+ # BIT2 - Informational unit test log messages.
+ # BIT3 - Verbose unit test log messages.
+ # @Prompt Unit Test Log Message Level + gUnitTestFrameworkPkgTokenSpaceGuid.PcdUnitTestLogLevel|0xFFFFFFFF|UINT32|0x00000001 diff --git a/UnitTestFrameworkPkg/UnitTestFrameworkPkg.uni b/UnitTestFrameworkPkg/UnitTestFrameworkPkg.uni new file mode 100644 index 0000000000..180675ae1a --- /dev/null +++ b/UnitTestFrameworkPkg/UnitTestFrameworkPkg.uni @@ -0,0 +1,21 @@ +// /** @file +// This Package provides all definitions (including functions, MACROs, +// structures library classes, and PCDs) and libraries instances, which are used +// to support unit testing and interface testing. +// +// Copyright (c) 2020, Intel Corporation. All rights reserved.
+// SPDX-License-Identifier: BSD-2-Clause-Patent +// +// **/ + +#string STR_PACKAGE_ABSTRACT #language en-US "This Package provides all definitions (including functions, MACROs, structures library classes, and PCDs) and libraries instances, which are used to support unit testing and interface testing." + +#string STR_PACKAGE_DESCRIPTION #language en-US "This Package provides all definitions (including functions, MACROs, structures library classes, and PCDs) and libraries instances, which are used to support unit testing and interface testing." + +#string STR_gUnitTestFrameworkPkgTokenSpaceGuid_PcdUnitTestLogLevel_PROMPT #language en-US "Unit Test Log Message Level" + +#string STR_gUnitTestFrameworkPkgTokenSpaceGuid_PcdUnitTestLogLevel_HELP #language en-US "This flag is used to control build time optimization based on unit test log level. The default value is 0xFFFFFFFF to enable all unit test log messages.

\n" + "BIT0 - Error unit test log messages.
\n" + "BIT1 - Warning unit test log messages.
\n" + "BIT2 - Informational unit test log messages.
\n" + "BIT3 - Verbose unit test log messages.
\n" -- 2.21.0.windows.1