Re: Unit tests and the EDK2

[Matt Lazarowitz <mlazarowitz@gmail.com>]

I decided to give embUnit a try since it offers basic functionality without
the C standard libraries. I have my work available on github at
https://github.com/mattlazarowitz/UnitTestNoStdLibPkg

The embUnitSamples folder may give you an idea of what I am trying. I
create a new folder in the package and set up a new component in the
package. I'll add the library with a function I want to unit test and
create the tests. The inf is set up to build an application so it stands
alone and doe not necessarily need to be run on the target platform.
Drivers will potentially be more difficult to create unit tests for. Either
I need to be able to incorporate the obj files into an application in my
unit test package, or a unit test inf and associated C files that live next
to the driver would work but make for a messier file structure.
An engineer could architect a driver in such a way that most of it is built
as a library and pulled into a driver framework as another alternative.
If all the test can live in the package it should be somewhat easy for a
build automation system to gather and deploy the unit tests automatically.
A simulated environment would be great as a place to execute the tests for
fast feedback.

Hardware level tests will always be needed. Unit testing is just part of
the test pyramid which also has system level and integration tests. The
unit tests I have in mind are meant to provide rapid feedback to a
programmer before sinking time into longer tests.

I don't have any test doubles or any way to get inversion of control yet. I
hope a unit test package will be able to borrow from SCT as much as
possible.


A Microsoft authored unit test harness was mentioned earlier in this thread
which might be worth investigating as well. I took a quick look and it
seems very similar to what I was able to accomplish with embUnit. I'll have
to take a longer look to really compare it. One advantage of embUnit is
that is is fairly well documented and is stable code even if it doesn't
conform the the EDK2 coding standard.

I would also like to evaluate the StdLib package and see how well it will
work with some of the more advanced embUnit features that require the C
standard libraries.
I'm glad to hear I'm not the only one thinking about this.

Matt

On Sun, Nov 6, 2016 at 1:06 PM, Kinney, Michael D <
michael.d.kinney@intel.com> wrote:

> Marvin,
>
> UEFI environment.
>
> Emulated environments are great for some types of testing.
>
> OVMF is also interesting for something that is a bit closer to
> real HW without requiring the real hardware.
>
> And types of testing always requires a real HW target.
>
>
> Mike
>
> > -----Original Message-----
> > From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of
> Marvin H?user
> > Sent: Sunday, November 6, 2016 11:58 AM
> > To: edk2-devel@lists.01.org
> > Cc: Kinney, Michael D <michael.d.kinney@intel.com>
> > Subject: Re: [edk2] Unit tests and the EDK2
> >
> > Hey Mike,
> >
> > Is the framework you plan to RFC a framework within the UEFI environment
> (UEFI Shell)
> > or within the OS?
> > Using the OS implementations of UEFI (Nt32 & Emulator) to run Unit Tests
> without the
> > need for a separate UEFI device or a reboot sounds pretty compelling to
> me, to be
> > honest.
> >
> > Thank you very much!
> >
> > Regards,
> > Marvin.
> >
> > > -----Original Message-----
> > > From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of
> > > Kinney, Michael D
> > > Sent: Sunday, November 6, 2016 8:04 PM
> > > To: Blibbet <blibbet@gmail.com>; edk2-devel@lists.01.org; Kinney,
> Michael
> > > D <michael.d.kinney@intel.com>
> > > Subject: Re: [edk2] Unit tests and the EDK2
> > >
> > > Hi,
> > >
> > > A test framework for EDK2 is one of my highest priorities to complete
> before
> > > the end of the year.
> > >
> > > I am evaluating a number of options and hope to put together a complete
> > > proposal as an RFC for consideration in the next few weeks.
> > >
> > > Thanks,
> > >
> > > Mike
> > >
> > >
> > > > -----Original Message-----
> > > > From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf
> Of
> > > > Blibbet
> > > > Sent: Sunday, November 6, 2016 9:26 AM
> > > > To: edk2-devel@lists.01.org
> > > > Subject: Re: [edk2] Unit tests and the EDK2
> > > >
> > > > On 11/06/2016 01:57 AM, Matt Lazarowitz wrote:
> > > > > I would like to find out if anyone has experience with an off the
> > > > > shelf unit test framework in the EDK2.[...]
> > > >
> > > > If you haven't looked at it, two months ago Microsoft open-sourced
> > > > some EDK2-centric unit tests.
> > > >
> > > > https://firmwaresecurity.com/2016/09/23/microsoft-uefi-unit-tests/
> > > >
> > > > Lee Fisher
> > > >
> > > >
> > > > _______________________________________________
> > > > edk2-devel mailing list
> > > > edk2-devel@lists.01.org
> > > > https://lists.01.org/mailman/listinfo/edk2-devel
> > > _______________________________________________
> > > edk2-devel mailing list
> > > edk2-devel@lists.01.org
> > > https://lists.01.org/mailman/listinfo/edk2-devel
> > _______________________________________________
> > edk2-devel mailing list
> > edk2-devel@lists.01.org
> > https://lists.01.org/mailman/listinfo/edk2-devel
> _______________________________________________
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel
>