From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-it0-x235.google.com (mail-it0-x235.google.com [IPv6:2607:f8b0:4001:c0b::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id B0E8D81E46 for ; Sun, 6 Nov 2016 19:01:46 -0800 (PST) Received: by mail-it0-x235.google.com with SMTP id m138so74277896itm.1 for ; Sun, 06 Nov 2016 19:01:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=QwRTStGizsaK4Pf1FKj5yMx0Myo0U3reMmr/7miE9Tk=; b=ZpdaXMuIUco7Ah3+6JRKrkpSvtNv0G02o4iIlMWgImkRrvoRAt9skw599ID/qm8wD5 b3NT43jBB3a4ukOxFFBtzRXYFmubJ23+pLw66gpOr7fWy45DsjTC5r4K7uWPd1UiVQMf Gk02LvfazjOTMoUUfPJTffZ4rUsLJ1myCt0FelPfOxAqIuLfzhQuv+nKioEEkbVLDpKt 93iEplqHr9s0fAow6LLAOPCMfuK/YcFhN5XY6uP5ch2JecJf0PX8e8HPcYlkNKPro3eN bRaL9qNx3xIGH6Z9Ws7JWVZlKGvfK5oKqGe3vIzOx7kOaD5olfT6gg5wf4Eks3Dxcfj9 RFBg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=QwRTStGizsaK4Pf1FKj5yMx0Myo0U3reMmr/7miE9Tk=; b=B99C5wo4IiPDQ+/byAkcF2hPdMbcd0UxtxDwsnWfydS5/lQqz83/8Ig5GagZPLTAlg DIqhCgrR8x8+iJqamOaxIHPQgxjUC+rnohLozXCRaOFJztshAnSIe/CXWsbw0TlYz5zs 4gbFsW33Rh/yV4SArgUEt0G93Z5jTtaqjX85dm7ekAbx8skQgnD66wu2BDDApFAdVIFl Ibh2quULGeiht+YWqSW6cAL+4jXXYe2yy0cWR+psMdzRfqkCe1475Nb4qSO9jxij57d8 NbIewnOHvurzQ+1OE/X5DCOoo5Oii01C1mMpyofYtBUllKKRktbKzF84oEribEt7TpSY cIuA== X-Gm-Message-State: ABUngveqluLgXFaYda2YGmuFBt5XSFy1JWM8z7UGDJkqCGYa7YLtYiFGlIcVPrOYloud2rz1ek9DWlygO43YBA== X-Received: by 10.157.62.130 with SMTP id b2mr2314858otc.159.1478487708320; Sun, 06 Nov 2016 19:01:48 -0800 (PST) MIME-Version: 1.0 Received: by 10.182.130.66 with HTTP; Sun, 6 Nov 2016 19:01:47 -0800 (PST) In-Reply-To: References: From: Matt Lazarowitz Date: Sun, 6 Nov 2016 19:01:47 -0800 Message-ID: To: "Kinney, Michael D" Cc: "Marvin H?user" , "edk2-devel@lists.01.org" X-Content-Filtered-By: Mailman/MimeDel 2.1.21 Subject: Re: Unit tests and the EDK2 X-BeenThere: edk2-devel@lists.01.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: EDK II Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Nov 2016 03:01:46 -0000 Content-Type: text/plain; charset=UTF-8 I decided to give embUnit a try since it offers basic functionality without the C standard libraries. I have my work available on github at https://github.com/mattlazarowitz/UnitTestNoStdLibPkg The embUnitSamples folder may give you an idea of what I am trying. I create a new folder in the package and set up a new component in the package. I'll add the library with a function I want to unit test and create the tests. The inf is set up to build an application so it stands alone and doe not necessarily need to be run on the target platform. Drivers will potentially be more difficult to create unit tests for. Either I need to be able to incorporate the obj files into an application in my unit test package, or a unit test inf and associated C files that live next to the driver would work but make for a messier file structure. An engineer could architect a driver in such a way that most of it is built as a library and pulled into a driver framework as another alternative. If all the test can live in the package it should be somewhat easy for a build automation system to gather and deploy the unit tests automatically. A simulated environment would be great as a place to execute the tests for fast feedback. Hardware level tests will always be needed. Unit testing is just part of the test pyramid which also has system level and integration tests. The unit tests I have in mind are meant to provide rapid feedback to a programmer before sinking time into longer tests. I don't have any test doubles or any way to get inversion of control yet. I hope a unit test package will be able to borrow from SCT as much as possible. A Microsoft authored unit test harness was mentioned earlier in this thread which might be worth investigating as well. I took a quick look and it seems very similar to what I was able to accomplish with embUnit. I'll have to take a longer look to really compare it. One advantage of embUnit is that is is fairly well documented and is stable code even if it doesn't conform the the EDK2 coding standard. I would also like to evaluate the StdLib package and see how well it will work with some of the more advanced embUnit features that require the C standard libraries. I'm glad to hear I'm not the only one thinking about this. Matt On Sun, Nov 6, 2016 at 1:06 PM, Kinney, Michael D < michael.d.kinney@intel.com> wrote: > Marvin, > > UEFI environment. > > Emulated environments are great for some types of testing. > > OVMF is also interesting for something that is a bit closer to > real HW without requiring the real hardware. > > And types of testing always requires a real HW target. > > > Mike > > > -----Original Message----- > > From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of > Marvin H?user > > Sent: Sunday, November 6, 2016 11:58 AM > > To: edk2-devel@lists.01.org > > Cc: Kinney, Michael D > > Subject: Re: [edk2] Unit tests and the EDK2 > > > > Hey Mike, > > > > Is the framework you plan to RFC a framework within the UEFI environment > (UEFI Shell) > > or within the OS? > > Using the OS implementations of UEFI (Nt32 & Emulator) to run Unit Tests > without the > > need for a separate UEFI device or a reboot sounds pretty compelling to > me, to be > > honest. > > > > Thank you very much! > > > > Regards, > > Marvin. > > > > > -----Original Message----- > > > From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of > > > Kinney, Michael D > > > Sent: Sunday, November 6, 2016 8:04 PM > > > To: Blibbet ; edk2-devel@lists.01.org; Kinney, > Michael > > > D > > > Subject: Re: [edk2] Unit tests and the EDK2 > > > > > > Hi, > > > > > > A test framework for EDK2 is one of my highest priorities to complete > before > > > the end of the year. > > > > > > I am evaluating a number of options and hope to put together a complete > > > proposal as an RFC for consideration in the next few weeks. > > > > > > Thanks, > > > > > > Mike > > > > > > > > > > -----Original Message----- > > > > From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf > Of > > > > Blibbet > > > > Sent: Sunday, November 6, 2016 9:26 AM > > > > To: edk2-devel@lists.01.org > > > > Subject: Re: [edk2] Unit tests and the EDK2 > > > > > > > > On 11/06/2016 01:57 AM, Matt Lazarowitz wrote: > > > > > I would like to find out if anyone has experience with an off the > > > > > shelf unit test framework in the EDK2.[...] > > > > > > > > If you haven't looked at it, two months ago Microsoft open-sourced > > > > some EDK2-centric unit tests. > > > > > > > > https://firmwaresecurity.com/2016/09/23/microsoft-uefi-unit-tests/ > > > > > > > > Lee Fisher > > > > > > > > > > > > _______________________________________________ > > > > edk2-devel mailing list > > > > edk2-devel@lists.01.org > > > > https://lists.01.org/mailman/listinfo/edk2-devel > > > _______________________________________________ > > > edk2-devel mailing list > > > edk2-devel@lists.01.org > > > https://lists.01.org/mailman/listinfo/edk2-devel > > _______________________________________________ > > edk2-devel mailing list > > edk2-devel@lists.01.org > > https://lists.01.org/mailman/listinfo/edk2-devel > _______________________________________________ > edk2-devel mailing list > edk2-devel@lists.01.org > https://lists.01.org/mailman/listinfo/edk2-devel >