From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from out2-smtp.messagingengine.com (out2-smtp.messagingengine.com [66.111.4.26]) by mx.groups.io with SMTP id smtpd.web10.44922.1681926607563841803 for ; Wed, 19 Apr 2023 10:50:07 -0700 Authentication-Results: mx.groups.io; dkim=fail reason="signature has expired" header.i=@bsdio.com header.s=fm1 header.b=iYHdcbTc; spf=pass (domain: bsdio.com, ip: 66.111.4.26, mailfrom: rebecca@bsdio.com) Received: from compute6.internal (compute6.nyi.internal [10.202.2.47]) by mailout.nyi.internal (Postfix) with ESMTP id D36815C01E1; Wed, 19 Apr 2023 13:50:06 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute6.internal (MEProxy); Wed, 19 Apr 2023 13:50:06 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdio.com; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm1; t= 1681926606; x=1682013006; bh=6/ys+wslTF6EMSYYS0AiEJJty2mZYExP98g px6vc6JQ=; b=iYHdcbTcWLbGYHnQKyjVNU7Y+OWxUVALTEx2263vJylAJJu2p26 AK8yR9uh1L/RPoVnXYMh9Ie5aWV3br7gt3so/fmcJUW99DQO79Wx9u+bjxGUMH+y Hl/YpnFTeoVW8ySml1xAIG4k/Qd/iGECSd4uF+w40uKga+aizT/8FtBVWm9qdjcA 7CQwIw2UAX4sKgTG5BWoOlyEfKTomyi9HBxwWg/mukqSrjA1JAjBBFWG/oxcaoW/ TYU3xL+v0h8Mo3NbpnAfrl/My2fhh//ZwFi3lEJyVmFdDv15XcoKp82LHhIkvxS5 l+jdAgjql6HbVroyAQ8FlEZvEQMp4MzP7nw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t= 1681926606; x=1682013006; bh=6/ys+wslTF6EMSYYS0AiEJJty2mZYExP98g px6vc6JQ=; b=f4UHXiuBuclPd8qd4TmPoyYSPBeWhnyJHYr9zjQrUqbtf8aG7vk b77NIgY63PuIyA0PgzJzIKr0mR07leHjs1ZaCwvCJkTy3m51ly4tCEmkshKm2AqI fPBgrvlxuJoARMrY93GUaTthZFMZDpKlhekROgU7pvio4G7sHb93CPvYSZQcH1np DxQ6+MVfvrZUFxOVPoT1O4+XymA6wC09uywQ5B2hODo5jXiICrVEXNGfK2rcWIcX xy991YUmtYLVHjZu7LzSen5CHlb7vsGDaBz0exWLqVDofEbBmpFyvlKWXHK3/061 TBmH4iN/ry6guxPX9/3cM7EDHUGAW1j5qYQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedrfedttddguddukecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpefkffggfgfuvfevfhfhjggtgfesthejredttdefjeenucfhrhhomheptfgv sggvtggtrgcuvehrrghnuceorhgvsggvtggtrgessghsughiohdrtghomheqnecuggftrf grthhtvghrnhepheelkefgudetudegheegfffghfetfefhhfetteefveehgfdvjeehtdfh ffeiffelnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomh eprhgvsggvtggtrgessghsughiohdrtghomh X-ME-Proxy: Feedback-ID: i5b994698:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 19 Apr 2023 13:50:05 -0400 (EDT) Message-ID: Date: Wed, 19 Apr 2023 11:50:04 -0600 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Subject: Re: [PATCH v1 1/1] BaseTools/Plugin: Clarify code coverage failure message To: mikuback@linux.microsoft.com, devel@edk2.groups.io Cc: Liming Gao , Bob Feng , Yuwei Chen , Michael D Kinney , Sean Brogan References: <20230418161104.595-1-mikuback@linux.microsoft.com> From: "Rebecca Cran" In-Reply-To: <20230418161104.595-1-mikuback@linux.microsoft.com> Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Reviewed-by: Rebecca Cran On 4/18/23 10:11 AM, mikuback@linux.microsoft.com wrote: > From: Michael Kubacki > > HostBasedUnitTestRunner.py is a build plugin responsible for locating > and executing host-based unit tests. > > Recently, commit 6bb00aa introduced support for the plugin to > generate code coverage reports via lcov and OpenCppCoverage. > > The plugin has discovered unit tests by searching for executables > with "Test" in the name for a while. However, the test coverage > change makes assumptions about test presence when crafting the > OpenCppCoverage command that ultimately fails with an ambiguous error > message if no host-based unit tests are discovered (see "ERROR"). > > ``` > SECTION - Run Host based Unit Tests > SUBSECTION - Testing for architecture: X64 > ERROR - UnitTest Coverage: Failed to generate cobertura format xml in > single package. > PROGRESS - --->Test Success: Host Unit Test Compiler Plugin NOOPT > ``` > > This change preempts that message with a check in the plugin to > determine if any host-based tests were discovered. If not, a message > is printed with more guidance about how the developer should proceed > to either (1) fix their tests so code coverage is generated as > expected or (2) prevent the error message. > > New message: > > ``` > SECTION - Run Host based Unit Tests > SUBSECTION - Testing for architecture: X64 > WARNING - UnitTest Coverage: > No unit tests discovered. Test coverage will not be generated. > > Prevent this message by: > 1. Adding host-based unit tests to this package > 2. Ensuring tests have the word "Test" in their name > 3. Disabling HostUnitTestCompilerPlugin in the package CI YAML file > PROGRESS - --->Test Success: Host Unit Test Compiler Plugin NOOPT > ``` > > Cc: Rebecca Cran > Cc: Liming Gao > Cc: Bob Feng > Cc: Yuwei Chen > Cc: Michael D Kinney > Cc: Sean Brogan > Signed-off-by: Michael Kubacki > --- > BaseTools/Plugin/HostBasedUnitTestRunner/HostBasedUnitTestRunner.py | 13 +++++++++++++ > 1 file changed, 13 insertions(+) > > diff --git a/BaseTools/Plugin/HostBasedUnitTestRunner/HostBasedUnitTestRunner.py b/BaseTools/Plugin/HostBasedUnitTestRunner/HostBasedUnitTestRunner.py > index 0e013c5f1a7f..a384b556294c 100644 > --- a/BaseTools/Plugin/HostBasedUnitTestRunner/HostBasedUnitTestRunner.py > +++ b/BaseTools/Plugin/HostBasedUnitTestRunner/HostBasedUnitTestRunner.py > @@ -16,6 +16,7 @@ import edk2toollib.windows.locate_tools as locate_tools > from edk2toolext.environment import shell_environment > from edk2toollib.utility_functions import RunCmd > from edk2toollib.utility_functions import GetHostInfo > +from textwrap import dedent > > > class HostBasedUnitTestRunner(IUefiBuildPlugin): > @@ -84,6 +85,18 @@ class HostBasedUnitTestRunner(IUefiBuildPlugin): > else: > raise NotImplementedError("Unsupported Operating System") > > + if not testList: > + logging.warning(dedent(""" > + UnitTest Coverage: > + No unit tests discovered. Test coverage will not be generated. > + > + Prevent this message by: > + 1. Adding host-based unit tests to this package > + 2. Ensuring tests have the word "Test" in their name > + 3. Disabling HostUnitTestCompilerPlugin in the package CI YAML file > + """).strip()) > + return 0 > + > for test in testList: > # Configure output name if test uses cmocka. > shell_env.set_shell_var(