From: "Michael D Kinney" <michael.d.kinney@intel.com>
To: devel@edk2.groups.io
Cc: Sean Brogan <sean.brogan@microsoft.com>,
Bret Barkelew <Bret.Barkelew@microsoft.com>,
Bob Feng <bob.c.feng@intel.com>,
Liming Gao <liming.gao@intel.com>
Subject: [Patch 02/11] BaseTools/Plugin: Add HostBasedUnitTestRunner plugin
Date: Thu, 23 Jan 2020 18:10:23 -0800 [thread overview]
Message-ID: <20200124021032.13808-3-michael.d.kinney@intel.com> (raw)
In-Reply-To: <20200124021032.13808-1-michael.d.kinney@intel.com>
Add plugin to BaseTools to run host based unit tests.
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
---
.../HostBasedUnitTestRunner.py | 115 ++++++++++++++++++
.../HostBasedUnitTestRunner_plug_in.yaml | 12 ++
2 files changed, 127 insertions(+)
create mode 100644 BaseTools/Plugin/HostBasedUnitTestRunner/HostBasedUnitTestRunner.py
create mode 100644 BaseTools/Plugin/HostBasedUnitTestRunner/HostBasedUnitTestRunner_plug_in.yaml
diff --git a/BaseTools/Plugin/HostBasedUnitTestRunner/HostBasedUnitTestRunner.py b/BaseTools/Plugin/HostBasedUnitTestRunner/HostBasedUnitTestRunner.py
new file mode 100644
index 0000000000..92426760ae
--- /dev/null
+++ b/BaseTools/Plugin/HostBasedUnitTestRunner/HostBasedUnitTestRunner.py
@@ -0,0 +1,115 @@
+# @file HostBasedUnitTestRunner.py
+# Plugin to located any host-based unit tests in the output directory and execute them.
+##
+# Copyright (c) Microsoft Corporation.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+#
+##
+import os
+import logging
+import glob
+import xml.etree.ElementTree
+from edk2toolext.environment.plugintypes.uefi_build_plugin import IUefiBuildPlugin
+from edk2toolext import edk2_logging
+import edk2toollib.windows.locate_tools as locate_tools
+from edk2toolext.environment import shell_environment
+from edk2toollib.utility_functions import RunCmd
+
+
+class HostBasedUnitTestRunner(IUefiBuildPlugin):
+
+ def do_pre_build(self, thebuilder):
+ '''
+ Works with the compiler (either the HostBasedCompilerPlugin or an other Builder) to set
+ up the environment that will be needed to build host-based unit tests.
+
+ EXPECTS:
+ - Build Var 'CI_BUILD_TYPE' - If not set to 'host_unit_test', will not do anything.
+
+ UPDATES:
+ - Shell Var (Several) - Updates the shell with all vars listed in interesting_keys.
+ - Shell Path - Updated from QueryVcVariables()
+ - Shell Var 'CMOCKA_MESSAGE_OUTPUT'
+ '''
+ ci_type = thebuilder.env.GetValue('CI_BUILD_TYPE')
+ if ci_type != 'host_unit_test':
+ return 0
+
+ shell_env = shell_environment.GetEnvironment()
+ # Use the tools lib to determine the correct values for the vars that interest us.
+ interesting_keys = ["ExtensionSdkDir", "INCLUDE", "LIB", "LIBPATH", "UniversalCRTSdkDir",
+ "UCRTVersion", "WindowsLibPath", "WindowsSdkBinPath", "WindowsSdkDir", "WindowsSdkVerBinPath",
+ "WindowsSDKVersion", "VCToolsInstallDir"]
+ vs_vars = locate_tools.QueryVcVariables(interesting_keys, "amd64")
+ for (k, v) in vs_vars.items():
+ if k.upper() == "PATH":
+ shell_env.append_path(v)
+ else:
+ shell_env.set_shell_var(k, v)
+
+ # Set up the reporting type for Cmocka.
+ shell_env.set_shell_var('CMOCKA_MESSAGE_OUTPUT', 'xml')
+ return 0
+
+ def do_post_build(self, thebuilder):
+ '''
+ After a build, will automatically locate and run all host-based unit tests. Logs any
+ failures with Warning severity and will return a count of the failures as the return code.
+
+ EXPECTS:
+ - Build Var 'CI_BUILD_TYPE' - If not set to 'host_unit_test', will not do anything.
+
+ UPDATES:
+ - Shell Var 'CMOCKA_XML_FILE'
+ '''
+ ci_type = thebuilder.env.GetValue('CI_BUILD_TYPE')
+ if ci_type != 'host_unit_test':
+ return 0
+
+ shell_env = shell_environment.GetEnvironment()
+ logging.log(edk2_logging.get_section_level(),
+ "Run Host based Unit Tests")
+ path = thebuilder.env.GetValue("BUILD_OUTPUT_BASE")
+
+ failure_count = 0
+
+ for arch in thebuilder.env.GetValue("TARGET_ARCH").split():
+ logging.log(edk2_logging.get_subsection_level(),
+ "Testing for architecture: " + arch)
+ cp = os.path.join(path, arch)
+
+ # If any old results XML files exist, clean them up.
+ for old_result in glob.iglob(os.path.join(cp, "*.result.xml")):
+ os.remove(old_result)
+
+ # Determine whether any tests exist.
+ testList = glob.glob(os.path.join(cp, "*Test*.exe"))
+ for test in testList:
+ # Configure output name.
+ shell_env.set_shell_var(
+ 'CMOCKA_XML_FILE', test + ".%g." + arch + ".result.xml")
+
+ # Run the test.
+ ret = RunCmd('"' + test + '"', "", workingdir=cp)
+ if(ret != 0):
+ logging.error("UnitTest Execution Error: " +
+ os.path.basename(test))
+ else:
+ logging.info("UnitTest Completed: " +
+ os.path.basename(test))
+ file_match_pattern = test + ".*." + arch + ".result.xml"
+ xml_results_list = glob.glob(file_match_pattern)
+ for xml_result_file in xml_results_list:
+ root = xml.etree.ElementTree.parse(
+ xml_result_file).getroot()
+ for suite in root:
+ for case in suite:
+ for result in case:
+ if result.tag == 'failure':
+ logging.warning(
+ "%s Test Failed" % os.path.basename(test))
+ logging.warning(
+ " %s - %s" % (case.attrib['name'], result.text))
+ failure_count += 1
+
+ return failure_count
diff --git a/BaseTools/Plugin/HostBasedUnitTestRunner/HostBasedUnitTestRunner_plug_in.yaml b/BaseTools/Plugin/HostBasedUnitTestRunner/HostBasedUnitTestRunner_plug_in.yaml
new file mode 100644
index 0000000000..d9eb852e97
--- /dev/null
+++ b/BaseTools/Plugin/HostBasedUnitTestRunner/HostBasedUnitTestRunner_plug_in.yaml
@@ -0,0 +1,12 @@
+##
+# IUefiBuildPlugin used to run any unittests that
+# were built on this build.
+#
+# Copyright (c) Microsoft Corporation.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "scope": "host-test-win",
+ "name": "Windows Host-Based Unit Test Runner",
+ "module": "HostBasedUnitTestRunner"
+}
--
2.21.0.windows.1
next prev parent reply other threads:[~2020-01-24 2:10 UTC|newest]
Thread overview: 33+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-01-24 2:10 [Patch 00/11] Add Unit Test Framework Michael D Kinney
2020-01-24 2:10 ` [Patch 01/11] .pytool: Add CI support for host based unit tests with results Michael D Kinney
2020-01-27 23:28 ` [edk2-devel] " brbarkel
2020-01-24 2:10 ` Michael D Kinney [this message]
2020-01-27 23:29 ` [edk2-devel] [Patch 02/11] BaseTools/Plugin: Add HostBasedUnitTestRunner plugin brbarkel
2020-02-07 2:32 ` Bob Feng
2020-01-24 2:10 ` [Patch 03/11] MdePkg/Include/Library: Add UnitTestLib class Michael D Kinney
2020-01-27 23:42 ` [edk2-devel] " brbarkel
2020-02-07 0:49 ` Michael D Kinney
2020-02-07 1:22 ` Wu, Hao A
2020-02-07 5:43 ` Bret Barkelew
2020-01-24 2:10 ` [Patch 04/11] UnitTestFrameworkPkg: Add public and private interfaces Michael D Kinney
2020-01-27 23:42 ` [edk2-devel] " brbarkel
2020-01-24 2:10 ` [Patch 05/11] UnitTestFrameworkPkg/Library: Add library instances Michael D Kinney
2020-01-27 23:43 ` [edk2-devel] " brbarkel
2020-01-24 2:10 ` [Patch 06/11] UnitTestFrameworkPkg/Test: Add unit test samples Michael D Kinney
2020-01-27 23:43 ` [edk2-devel] " brbarkel
2020-01-24 2:10 ` [Patch 07/11] UnitTestFrameworkPkg: Add DSC, DSC INC, and YAML files Michael D Kinney
2020-01-27 23:43 ` [edk2-devel] " brbarkel
2020-01-24 2:10 ` [Patch 08/11] MdePkg/Test: Add SafeIntLib and BaseLib Base64 unit tests Michael D Kinney
2020-01-27 23:43 ` [edk2-devel] " brbarkel
2020-02-07 1:27 ` Wu, Hao A
2020-02-07 7:56 ` Liming Gao
2020-02-07 16:05 ` Michael D Kinney
2020-01-24 2:10 ` [Patch 09/11] MdeModulePkg: Add DxeResetSystemLib unit test Michael D Kinney
2020-01-27 23:43 ` [edk2-devel] " brbarkel
2020-02-07 1:25 ` Wu, Hao A
2020-01-24 2:10 ` [Patch 10/11] .azurepipelines: Enable CI for UnitTestFrameworkPkg and host tests Michael D Kinney
2020-01-27 23:44 ` [edk2-devel] " brbarkel
2020-01-24 2:10 ` [Patch 11/11] Maintainers.txt: Add UnitTestFrameworkPkg Michael D Kinney
2020-01-24 10:22 ` Laszlo Ersek
2020-01-27 23:44 ` [edk2-devel] " brbarkel
2020-01-27 23:28 ` [edk2-devel] [Patch 00/11] Add Unit Test Framework brbarkel
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-list from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200124021032.13808-3-michael.d.kinney@intel.com \
--to=devel@edk2.groups.io \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox