From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (NAM11-CO1-obe.outbound.protection.outlook.com [40.107.220.105]) by mx.groups.io with SMTP id smtpd.web09.8.1576540404683505341 for ; Mon, 16 Dec 2019 15:53:25 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@microsoft.com header.s=selector2 header.b=JKtgz9bs; spf=pass (domain: microsoft.com, ip: 40.107.220.105, mailfrom: bret.barkelew@microsoft.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Ocbc8DHDPrmHk+YdTKYoay87sZAQAJnDwxEStuT0Go7yXNkW4K/DQjm0xNFVrZwIzZlelj/XwQAbiF4hx7ss56uA1EFNll35PNJZXAjALmd+V9eA5qjccAtOetI5z2iGKHyL1FDsZxiBN8a/klP5nzXYHJPiqvGb3mlB2fECWci+8qfUgt0mWc1zH0D70gg9rTYspwDoGWDdZUjiELlljqZMRMlKXAhuFau10szqExNH0yzPF1NWRajXpX1NlIjrNANsfGpt15cyOYd7r1wqYc/N39iA9JJmK/CGZukFdbG+OCnvJvtsynHWW8Fr51aVWsbCVZE+wl5k0FLtz+9bgg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rNGQoJI7beFdc+rsmN98A80jSx5kcZ74J4QbrbN+ZrQ=; b=IES5HPC5qHs81eanclG4g5LsGSplYDzqfqaNJDS+QECGN52CNkE97vK3MV8tU5FD789NhFLmWGf08hHDjBsMjMLEIfhU2ozxLW69z1fBtZS/7TRbYSi2DGWPvfkqxKKFXHntgwgFHmL5ySn2bhq1YG4uaezBVRoMko/UN7SYdq40cO3jK3mZvHbYW9rhpSSP2osKOBXlyRfLhSY7Ozkcjtd3a1/yZyyOR3hwGfgClHlexrYyLiPxKvS18wC9BltUCFmkpzG+Po4i7OpAq3oVDkPIbdnOMaN3Lo9omRmyOj8oxlAgn6Mb21faGZY8HWUOQUHR4nDTthd4sUCf7eJ6aA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=microsoft.com; dmarc=pass action=none header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rNGQoJI7beFdc+rsmN98A80jSx5kcZ74J4QbrbN+ZrQ=; b=JKtgz9bs2dJjLx8zzd93+zsRZHIFHhrGQPAShkbCWhEALC6YHw34f3GY9PJgoL+lNKoG/kdmrv8Lobl6ldKmyADPX7Ddp8FdoCcITgvRS1U9r/g5VS3ATOdjpReI7oymBv7QOzYymsDNj5isfZ4g35kjAlslZ13Zu/GNIer2SUc= Received: from CH2PR21MB1400.namprd21.prod.outlook.com (20.180.10.144) by CH2PR21MB1445.namprd21.prod.outlook.com (20.180.11.216) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2559.12; Mon, 16 Dec 2019 23:53:23 +0000 Received: from CH2PR21MB1400.namprd21.prod.outlook.com ([fe80::a147:c42f:38de:e8bd]) by CH2PR21MB1400.namprd21.prod.outlook.com ([fe80::a147:c42f:38de:e8bd%4]) with mapi id 15.20.2559.012; Mon, 16 Dec 2019 23:53:23 +0000 From: "Bret Barkelew" To: "Kinney, Michael D" , "rfc@edk2.groups.io" , "devel@edk2.groups.io" , Andrew Fish Subject: Re: [edk2-rfc] [EXTERNAL] Re: [edk2-devel] EDK2 Host-Based Unit Test RFC (Now with docs!) Thread-Topic: [edk2-rfc] [EXTERNAL] Re: [edk2-devel] EDK2 Host-Based Unit Test RFC (Now with docs!) Thread-Index: AQHVoQS12ZW6E6MMrkasWCUh3oP5e6enceDwgALivYCAAAlHOoADZyKbgAsoujaAAUMtS4ADTTNQgAAXsgk= Date: Mon, 16 Dec 2019 23:53:23 +0000 Message-ID: References: ,<0F460E19-2A3B-4FEE-8B6F-BE65B9C5F1D4@apple.com>,<15DD3E3A746110DF.27746@groups.io>,, , In-Reply-To: Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Enabled=True;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SiteId=72f988bf-86f1-41af-91ab-2d7cd011db47;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SetDate=2019-12-04T18:23:45.2427147Z;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ContentBits=0;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Method=Privileged authentication-results: spf=none (sender IP is ) smtp.mailfrom=Bret.Barkelew@microsoft.com; x-originating-ip: [2001:4898:80e8:0:3844:1682:9c0d:492b] x-ms-publictraffictype: Email x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: acbd72b2-53d1-4de6-e269-08d7828322c1 x-ms-traffictypediagnostic: CH2PR21MB1445:|CH2PR21MB1445: x-ld-processed: 72f988bf-86f1-41af-91ab-2d7cd011db47,ExtAddr x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:549; x-forefront-prvs: 02530BD3AA x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(136003)(396003)(376002)(346002)(39860400002)(366004)(189003)(174864002)(13464003)(199004)(6506007)(53546011)(110136005)(9686003)(30864003)(966005)(8990500004)(81166006)(18265965003)(186003)(81156014)(55016002)(316002)(7696005)(91956017)(76116006)(10290500003)(66476007)(66556008)(64756008)(8676002)(478600001)(8936002)(66946007)(66446008)(71200400001)(2906002)(5660300002)(33656002)(52536014)(86362001)(579004)(460985005);DIR:OUT;SFP:1102;SCL:1;SRVR:CH2PR21MB1445;H:CH2PR21MB1400.namprd21.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;MX:1;A:1; received-spf: None (protection.outlook.com: microsoft.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: U0S4Ln0kFU5mFq4xPG1WrZnzh84ci8x5yI9Oubc7EGsKf6+6je57dbQUAnNHVbBPPKQXNi0uQxVipKClB7gKuZeS8VLtA1aCTP0kiOEDVuj5VtsJ2C4n3c3BpA+pAiWY5axc9qtOKYJfTuHx2+8sES9ybXBF/fRtF63QwRACgSvvJ973pSH8uX8IQ5kVTec6osGomHyRarXcknmqroLOqd03huA4rY0Xhr0BRn5SW7tZwBA+JDQT8wi+LLzxT6Q1vj6lT3Wc4mPkHeJyt2VNlAAQ0PItpZ5gFAsZJPGwvlRvldd2/T30dV1VzQEVmo8qAVMlbcAcw52qIZXFwXPOoUwmyOv0Wqq8ys5ve33bXn1kfqH8iVrLXSJzZ/tAUpmZZimy9ZOW5E4YE1yrTNLgW/UrLPJQVgr1NgvHWANmazWeFQAwH/EMjz0yzX5ysd8xEGT6dlp0Y6k9LPqcAx8HaS9DUx2muDrgmHuICno5zhzagIZRkDA3LQBRF2egrkRLftfBtr7NK8vLyAmJ4ERackmrYH2DhSjN4BKM35Dw7uO4oLRqASDzuMqden1w2DDc8fb8X4TmYvpE98zmMN6LdZTrwh3NEUzfNcsQjDYOHao= x-ms-exchange-transport-forked: True MIME-Version: 1.0 X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-Network-Message-Id: acbd72b2-53d1-4de6-e269-08d7828322c1 X-MS-Exchange-CrossTenant-originalarrivaltime: 16 Dec 2019 23:53:23.0368 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: cYsyoytOcsFHucvPie676MA/GFs9h/lbTcfsfPGn3cyESvhw/a2BpqNVgBPPYzkGMcYvuw8qICVlcGdyzcI+Vg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR21MB1445 Content-Language: en-US Content-Type: multipart/alternative; boundary="_000_CH2PR21MB1400108196927D7CE1D91B52EF510CH2PR21MB1400namp_" --_000_CH2PR21MB1400108196927D7CE1D91B52EF510CH2PR21MB1400namp_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Did you make sure to stuart_update? It will pull a dependency repo that is = only needed for CI. - Bret ________________________________ From: Kinney, Michael D Sent: Monday, December 16, 2019 2:31:13 PM To: rfc@edk2.groups.io ; Bret Barkelew ; devel@edk2.groups.io ; Andrew Fish ; Kinney, Michael D Subject: RE: [edk2-rfc] [EXTERNAL] Re: [edk2-devel] EDK2 Host-Based Unit T= est RFC (Now with docs!) Hi Bret, I am looking at the latest version of the content on your branch. I am confused by MdePkg/Test/MdePkgTest.dsc. It makes references to lib classes and packages that do not exist. CmockaLib|CmockaHostUnitTestPkg/Library/CmockaLib/CmockaLib.inf OsServiceLib|HostBasedUnitTestPkg/Library/OsServiceLibHost/OsServiceLibH= ost.inf UnitTestAssertLib|CmockaHostUnitTestPkg/Library/UnitTestAssertLibcmocka/= UnitTestAssertLibcmocka.inf UnitTestLib|CmockaHostUnitTestPkg/Library/UnitTestLibcmocka/UnitTestLibc= mocka.inf Am I looking at an old file? I am just trying to do a local build of the unit tests for the SafeIntLib. Thanks, Mike > -----Original Message----- > From: rfc@edk2.groups.io On Behalf > Of Bret Barkelew via Groups.Io > Sent: Saturday, December 14, 2019 12:07 PM > To: devel@edk2.groups.io; Andrew Fish > ; Kinney, Michael D > > Cc: rfc@edk2.groups.io > Subject: Re: [edk2-rfc] [EXTERNAL] Re: [edk2-devel] > EDK2 Host-Based Unit Test RFC (Now with docs!) > > The host-based tests now build on Linux/GCC, but the > final executables don't seem to get created. Don't know > where the disconnect is there. I can see the test get > compiled (along with all the libraries), but it doesn't > seem to link. > > Can you take a look? Thanks! > > - Bret > > From: Bret Barkelew > Sent: Friday, December 13, 2019 4:46 PM > To: devel@edk2.groups.io; > Andrew Fish; Kinney, Michael > D > Cc: rfc@edk2.groups.io > Subject: RE: [EXTERNAL] Re: [edk2-devel] EDK2 Host- > Based Unit Test RFC (Now with docs!) > > Mike, > > I think I've gotten all the feedback here and all the > action items from our call. The current branch should > be good to go, minus the couple of things that Intel > was going to look into. > > Thanks! > > - Bret > > From: Bret Barkelew > Sent: Friday, December 6, 2019 2:21 PM > To: devel@edk2.groups.io; > Andrew Fish; Kinney, Michael > D > Cc: rfc@edk2.groups.io > Subject: RE: [EXTERNAL] Re: [edk2-devel] EDK2 Host- > Based Unit Test RFC (Now with docs!) > > > 1. I see that MdePkg adds a dependency on > UnitTestPkg. This makes UnitTestPkg the root package > when building and running host based unit tests. This > makes sense, but good to highlight that all packages > that use host based tests will introduce a new package > dependency on UnitTestPkg. > * Since the dependency only applies to unit > tests, can we update the DependencyCheck plugin to > support listing dependencies for FW components separate > from dependencies for unit tests? > * Can move. Capability is there, but mistakenly > added to the wrong section. > 2. I see UnitTestPkg declares 6 new lib classes. > Are all 6 classes intended to be used directly from a > unit test case? If some of these are only intended to > be used from the framework inside the UnitTestPkg can > we make them private? > * Because there are different instances in > multiple places (and conceivably more in the future), > we don't see how we could make them private. > 3. In the MdePkg, I see a new top level directory > called 'HostLibrary'. Since these lib instances are > only used from a host based test, can they be moved > into the 'Test' directory? > * Can move. > 4. MdePkg/MdePkgTest.dsc. > * Can this DSC file be moved into the 'Test' > directory? > > > i. Yes > > * I see this DSC file only supports VS today. > How much work is it to add GCC support? > > > i. Don't know. This is an item on our list, but > lower priority and not a blocker. > > 1. MdePkg/HostLibrary/BaseLibHost > * Why are there 2 INFs. One with ASM and one > without ASM? Can we just use the one with ASM and > assume NASM is installed? > * I see the purpose of this lib instance is to > call the > * SetJump()/LongJump(). So these > implementations always work? It looks like it assumes > BASE_LIBRARY_JUMP_BUFFER is identical to the host OS > user mode application setjmp()/longjmp() state. > * Why are DRx and CRx registers emulated? I > would think and code that depends on read/writing these > registers would not be compatible with host based > testing. Can we change to ASSERT (FALSE)? > * PatchInstructionX86() - I suspect this will > not work in a host based environment because it is self > modifying code. Should it be ASSERT (FALSE)? > * Libraries were taken directly from the Intel > work in HBFA. They worked so we kept them. We're just > as interested in the answers to these questions as you > are. > 2. MdePkg/HostLibrary/DebugLibHost > * What is '#ifndef TEST_WITH_KLEE' > * What is the 'PatchFormat()' function? It is > always disabled with if (0). > * Are the PCDs to set the debug message levels > disabled on purpose? (DebugPrintEnabled(), > DebugPrintLevelEnabled(), DebugCodeEnabled()) > * Libraries were taken directly from the Intel > work in HBFA. They worked so we kept them. We're just > as interested in the answers to these questions as you > are. > 3. MdePkg/HostLibrary/BaseMemoryLibHost > * Why can't we use > MdePkg/Library/BaseMemoryLib/BaseMemoryLib/inf instead > and reduce the number of host specific lib instances? > * Libraries were taken directly from the Intel > work in HBFA. They worked so we kept them. We're just > as interested in the answers to these questions as you > are. > 4. MdePkg/HostLibrary/MemoryAllocationLibHost > * Why is are memcpy(), assert(), memset() used > in this lib? I would expect this lib instance to match > the UefiMemoryAllocationLib with the only the use of > malloc() and free() to replace the UEFI Boot Services > calls. > * Libraries were taken directly from the Intel > work in HBFA. They worked so we kept them. We're just > as interested in the answers to these questions as you > are. > 5. Host library instance naming conventions. > * The current naming convention uses the > environment as a prefix(e.g. Pei, Smm, Uefi) and the > services the lib instance uses as a post fix. Would it > make more sense to use 'Host' as a prefix instead of a > postfix in the lib instance names? > * I don't know if there's a 1:1 relationship > with these. For some library classes (that you might > have host versions of), the class itself is the > PeiSomethingLib, and the host version would be the > PeiSomethingLibHost. No? > 6. Unicode vs ASCII strings > * I see InitUnitTestFramework(), > CreateUnitTestSuite(), and AddTestCase() all take > Unicode strings for the labels. I also see extra code > to convert gEfiCallerBaseName from ASCII to Unicode to > use it as a short name of a test framework. I think it > would be simpler if the parameters to these APIs were > ASCII and the framework can convert to Unicode if > needed. > * No strong feelings, but we already have a > bunch of tests written this way. Since the UnitTestLib > is an abstraction that works as well in Shell as in the > Host, going with Unicode strings seemed to match the > art for Shell apps (since the framework was written for > Shell first). > 7. Will InitUnitTestFramework() and > CreateUnitTestSuite() always be called in pairs? If > so, can we combine these to a single API? > * I see the SafeIntLib example create a single > framework and multiple test suites. Perhaps we can > have a single CreateUnitTestSuite() API where Framework > is an IN/OUT and if it is passed in as NULL, the > Framework handle is created. > > > i. It's not always in pairs. You would only have a > single framework init, but could have multiple suites. > > * I see a pattern where the > CreateUnitTestSuite() 'Package' parameter is used as a > prefix to every call to AddTestCase() in the > 'ClassName' parameter. Can we simplify AddTestCase() > so it only need to pass in the name of the unit test > case, and the framework can append Package and the > unit test case name? > > > i. Right now, our tests just coincidentally share > similar names with the packages, but we don't feel this > is axiomatic and don't want to force this naming on > others, who may be trying to bolt into other reporting > structures. > > 1. I see the use of the 'Fw' variable as a shorthand > for 'Framework'. Can we use something other than 'Fw'. > It may be confused with 'Firmware'. > * No real arguments. Wouldn't fight a find- > replace, but it'll just be a bunch of touches as we > commit. > 2. UnitTestPkg/Include/UnitTestTypes.h > * I see several hard coded string lengths. > Since this runs in a host environment and strings can > always be allocated, can the hard coded lengths be > removed? Update the structs to use pointers to strings > instead of string arrays, and the framework can manage > alloc() and free()? > > > i. Given that this framework is designed to be > nimble enough to work in PEI and SMM and other > constrictive environments, we wanted to keep fixed > sizing. > > * How are Fingerprints used? The idea of using > as hash as a unique identifier is a good idea. How is > the hash calculated? What unit test code artifacts are > used so developers know what parameters must be unique? > Can we just decide to use a specific hash > algorithm/size and the structure can be a pointer to an > allocated buffer instead of a fixed size array to make > it easy to change the algorithm/size in the future? > > > i. Fingerprints are unique IDs to make sure that > serialize/unserialized state matches the tests upon re- > entry. I'm not married to MD5, but it needs to be > predictably unique, and carried with the framework. I > would not want to make any requirements on CryptoLib, > since these aren't crypto-strong. > > * Update all the strings to be ASCII? See > Unicode vs ASCII above. > > > i. Ideally not, unless there's a strong use case. > > * UNIT_TEST_SAVE_TEST - The last field is a > variable sized array, so it can be a formal field. > > > i. Not opposed, but don't really want to make the > change myself. Is there a style guide that this is > breaking? > > * UNIT_TEST_SAVE_CONTEXT - - The last field is a > variable sized array, so it can be a formal field. > * UNIT_TEST_SAVE_HEADER - Can the last 3 fields > be changed to pointers and be formal fields? > * Do the structures really need to be packed? > Specially with the changes suggested above? Is the > intent to potentially share data stored on disk between > different host execution environments that may have > different width architectures? > > > i. That's an interesting point. Could you draw up > your suggestion and submit a PR? > > 1. UnitTestPkg - UnitTestLib.h > * Can you provide more context for the APIs > SaveFrameworkState(), SaveFrameworkStateAndQuit(), > SaveFrameworkStateAndReboot(), > SetFrameworkBootNextDevice(). I do not see any Load() > functions, so how would a set of tests be resumed? If > these do not apply to host based tests, should these be > split out to a different lib class? > > > i. I'll improve the documentation around these > functions. I will acknowledge, however, that this is an > interface that I expect to evolve as we figure out what > kinds of tests the community wants support for "out of > the box". If we want to easily enable tests around - > for example - ExitBootServices, we will likely want to > tweak this going forward to its simplest form. The > version we have here was sufficient to enable the test > cases that we've currently written. > > 1. UnitTestPkg/Library/UnitTestLib > * I see an implementation of MD5. We should not > do our own. We should use an approved implementation > such as OpenSSL. > > > i. Happy to discuss another implementation, but > this is not crypto-strong. It's just for uniqueness. A > sufficiently long CRC would probably work, too. > > 1. UnitTestPkg/Library/UnitTestTerminationLibTbd > * Do we really need this lib instance now? > > > i. This is here so that we can figure out what > this should look like for a host. Host obviously > wouldn't reset, but could conceivably quit. Or maybe > that doesn't make any sense for a host. > > > > - Bret > > > From: devel@edk2.groups.io on > behalf of Bret Barkelew via Groups.Io > > Sent: Wednesday, December 4, 2019 10:24:21 AM > To: Andrew Fish ; devel@edk2.groups.io > ; Kinney, Michael D > > Cc: rfc@edk2.groups.io > Subject: Re: [EXTERNAL] Re: [edk2-devel] EDK2 Host- > Based Unit Test RFC (Now with docs!) > > > Andrew, > > I agree with your points. > > > > Mike, > > You've got a lot more there. Let me take a look and > update the draft. I'll ping back ASAP. > > > > - Bret > > > > From: Andrew Fish > Sent: Wednesday, December 4, 2019 9:50 AM > To: devel@edk2.groups.io; > Kinney, Michael D > Cc: Bret Barkelew; > rfc@edk2.groups.io > Subject: [EXTERNAL] Re: [edk2-devel] EDK2 Host-Based > Unit Test RFC (Now with docs!) > > > > Mike, > > > > I like and agree with your comments. > > > > On the UnitTestPkg(s) dependency issue I think it would > make sense to move as much as possible into the > *Pkg/Test/ ( *Pkg/HostLibrary, MdePkg/MdePkgTest.dsc, > etc.) directory structure. It looks like for > implementation specific tests we skip the Test > directory and drop the UnitTest or FuzzTest directly > into modules directory. Maybe we should follow the same > pattern as *Pkg/Test and have a Test directory? This > will help minimize the number of "reserved" directories > we need for managing the testing. Have a standardized > directory structure will make it easier to > differentiate the code from tests while doing `git > grep` or other forms of code spelunking. > > > > A Host-Based Unit Test for a Library makes sense as you > can link directly against the library and test it. A > Host-Based Unit Test for Protocol/PPI/GUID [1] seems a > little more complex. It is easy enough to write tests > for the interfaces but what APIs do you call to get > access to the protocol? > > > > Per our conversation at the Stewards meeting I think > make sense to try to roll out the testing of libraries > as the 1st phase. The mocking required for drivers > could get quite complex and let us not get bogged down > in all that to get something working. > > > > [1] *Pkg/Test/UnitTest/[Library|Protocol|Ppi|Guid] > > > > Thanks, > > > > Andrew Fish > > > > > > On Dec 2, 2019, at 3:12 PM, Michael D Kinney > el.com>> wrote: > > > > Hi Bret, > > > > Thanks for posting this content. Host based unit > testing is a very valuable addition to the CI checks. > > > > I have the following comments: > > > > 1. I see that MdePkg adds a dependency on > UnitTestPkg. This makes UnitTestPkg the root package > when building and running host based unit tests. This > makes sense, but good to highlight that all packages > that use host based tests will introduce a new package > dependency on UnitTestPkg. > > * Since the dependency only applies to unit > tests, can we update the DependencyCheck plugin to > support listing dependencies for FW components separate > from dependencies for unit tests? > > 1. I see UnitTestPkg declares 6 new lib classes. > Are all 6 classes intended to be used directly from a > unit test case? If some of these are only intended to > be used from the framework inside the UnitTestPkg can > we make them private? > 2. In the MdePkg, I see a new top level directory > called 'HostLibrary'. Since these lib instances are > only used from a host based test, can they be moved > into the 'Test' directory? > 3. MdePkg/MdePkgTest.dsc. > > * Can this DSC file be moved into the 'Test' > directory? > * I see this DSC file only supports VS today. > How much work is it to add GCC support? > > 1. MdePkg/HostLibrary/BaseLibHost > > * Why are there 2 INFs. One with ASM and one > without ASM? Can we just use the one with ASM and > assume NASM is installed? > * I see the purpose of this lib instance is to > call the > * SetJump()/LongJump(). So these > implementations always work? It looks like it assumes > BASE_LIBRARY_JUMP_BUFFER is identical to the host OS > user mode applicationsetjmp()/longjmp() state. > * Why are DRx and CRx registers emulated? I > would think and code that depends on read/writing these > registers would not be compatible with host based > testing. Can we change to ASSERT (FALSE)? > * PatchInstructionX86() - I suspect this will > not work in a host based environment because it is self > modifying code. Should it be ASSERT (FALSE)? > > 1. MdePkg/HostLibrary/DebugLibHost > > * What is '#ifndef TEST_WITH_KLEE' > * What is the 'PatchFormat()' function? It is > always disabled with if (0). > * Are the PCDs to set the debug message levels > disabled on purpose? (DebugPrintEnabled(), > DebugPrintLevelEnabled(), DebugCodeEnabled()) > > 1. MdePkg/HostLibrary/BaseMemoryLibHost > > * Why can't we use > MdePkg/Library/BaseMemoryLib/BaseMemoryLib/inf instead > and reduce the number of host specific lib instances? > > 1. MdePkg/HostLibrary/MemoryAllocationLibHost > > * Why is are memcpy(), assert(), memset() used > in this lib? I would expect this lib instance to match > the UefiMemoryAllocationLib with the only the use of > malloc() and free() to replace the UEFI Boot Services > calls. > > 1. Host library instance naming conventions. > > * The current naming convention uses the > environment as a prefix(e.g. Pei, Smm, Uefi) and the > services the lib instance uses as a post fix. Would it > make more sense to use 'Host' as a prefix instead of a > postfix in the lib instance names? > > 1. Unicode vs ASCII strings > > * I see InitUnitTestFramework(), > CreateUnitTestSuite(), and AddTestCase() all take > Unicode strings for the labels. I also see extra code > to convert gEfiCallerBaseName from ASCII to Unicode to > use it as a short name of a test framework. I think it > would be simpler if the parameters to these APIs were > ASCII and the framework can convert to Unicode if > needed. > > 1. Will InitUnitTestFramework() and > CreateUnitTestSuite() always be called in pairs? If > so, can we combine these to a single API? > > * I see the SafeIntLib example create a single > framework and multiple test suites. Perhaps we can > have a single CreateUnitTestSuite() API where Framework > is an IN/OUT and if it is passed in as NULL, the > Framework handle is created. > * I see a pattern where the > CreateUnitTestSuite() 'Package' parameter is used as a > prefix to every call to AddTestCase() in the > 'ClassName' parameter. Can we simplify AddTestCase() > so it only need to pass in the name of the unit test > case, and the framework can append Package and the > unit test case name? > > 1. I see the use of the 'Fw' variable as a shorthand > for 'Framework'. Can we use something other than 'Fw'. > It may be confused with 'Firmware'. > 2. UnitTestPkg/Include/UnitTestTypes.h > > * I see several hard coded string lengths. > Since this runs in a host environment and strings can > always be allocated, can the hard coded lengths be > removed? Update the structs to use pointers to strings > instead of string arrays, and the framework can manage > alloc() and free()? > * How are Fingerprints used? The idea of using > as hash as a unique identifier is a good idea. How is > the hash calculated? What unit test code artifacts are > used so developers know what parameters must be unique? > Can we just decide to use a specific hash > algorithm/size and the structure can be a pointer to an > allocated buffer instead of a fixed size array to make > it easy to change the algorithm/size in the future? > * Update all the strings to be ASCII? See > Unicode vs ASCII above. > * UNIT_TEST_SAVE_TEST - The last field is a > variable sized array, so it can be a formal field. > * UNIT_TEST_SAVE_CONTEXT - - The last field is a > variable sized array, so it can be a formal field. > * UNIT_TEST_SAVE_HEADER - Can the last 3 fields > be changed to pointers and be formal fields? > * Do the structures really need to be packed? > Specially with the changes suggested above? Is the > intent to potentially share data stored on disk between > different host execution environments that may have > different width architectures? > > 1. UnitTestPkg - UnitTestLib.h > > * Can you provide more context for the APIs > SaveFrameworkState(), SaveFrameworkStateAndQuit(), > SaveFrameworkStateAndReboot(), > SetFrameworkBootNextDevice(). I do not see any Load() > functions, so how would a set of tests be resumed? If > these do not apply to host based tests, should these be > split out to a different lib class? > > 1. UnitTestPkg/Library/UnitTestLib > > * I see an implementation of MD5. We should not > do our own. We should use an approved implementation > such as OpenSSL. > > 1. UnitTestPkg/Library/UnitTestTerminationLibTbd > > * Do we really need this lib instance now? > > > > Thanks, > > > > Mike > > > > From: devel@edk2.groups.io > > On > Behalf Of Bret Barkelew via Groups.Io > Sent: Thursday, November 21, 2019 11:39 PM > To: devel@edk2.groups.io > Subject: [edk2-devel] EDK2 Host-Based Unit Test RFC > (Now with docs!) > > > > Now that CI has landed in edk2/master, we'd like to get > the basic framework for unittesting staged as well. > Target intercept date would be immediately after the > 2019/11 stabilization, so we wanted to go ahead and get > comments now. > > The host unittest framework consists of five primary > pieces: > - The test library (Cmocka) > - Support libraries (Found in UnitTestPkg) > - The test support plugins (HostUnitTestComilerPlugin, > HostUnitTestDxeCompleteCheck, HostBasedUnitTestRunner) > - The configuration in the package-based *.ci.yaml file > and package-based Test.dsc > - The tests themselves > > We have a demo branch set up at: > https://nam06.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fgith= ub.com%2Fcorthon%2Fedk2-staging%2Ftree%2Fedk2-host-&data=3D02%7C01%7Cbr= et.barkelew%40microsoft.com%7Caf69d76bb76f4a4e152e08d78277a97e%7C72f988bf86= f141af91ab2d7cd011db47%7C1%7C0%7C637121322769998529&sdata=3D0%2FEUzZG6J= 3F%2FLnCt0sICGwsxpwjXf7cbUMnQQWeSbtw%3D&reserved=3D0 > test_v2 ?url=3Dhttps%3A%2F%2Fgithub.com%2Fcorthon%2Fedk2- > staging%2Ftree%2Fedk2-host- > test_v2&data=3D02%7C01%7Cbret.barkelew%40microsoft.com%7C > 6eac9932f3f640dd65ac08d778e731e3%7C72f988bf86f141af91ab > 2d7cd011db47%7C1%7C0%7C637110806683189828&sdata=3DJQ%2BoW > xlEBOK2sH55cRAPVa3OpAxTsm6eHcdbWSo9CVo%3D&reserved=3D0> > We also have a demo build pipeline at: > https://nam06.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fdev.= azure.com%2Ftianocore%2Fedk2-ci-&data=3D02%7C01%7Cbret.barkelew%40micro= soft.com%7Caf69d76bb76f4a4e152e08d78277a97e%7C72f988bf86f141af91ab2d7cd011d= b47%7C1%7C0%7C637121322770003516&sdata=3D12H%2FC8YVt%2Fn5ud%2BYOp9vAzs6= vk2by46YFTzplhcz1xs%3D&reserved=3D0 > play/_build?definitionId=3D36&_a=3Dsummary felinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fdev.a > zure.com%2Ftianocore%2Fedk2-ci- > play%2F_build%3FdefinitionId%3D36%26_a%3Dsummary&data=3D0 > 2%7C01%7Cbret.barkelew%40microsoft.com%7C6eac9932f3f640 > dd65ac08d778e731e3%7C72f988bf86f141af91ab2d7cd011db47%7 > C1%7C0%7C637110806683189828&sdata=3DwBwn1ehjyTmYNKVSvlEZS > XK5qyeu4EPAL7FzdYntnt4%3D&reserved=3D0> > > The current demo branch contains a single test in > MdePkg for SafeIntLib (module file > MdePkg\Test\UnitTest\Library\BaseSafeIntLib\TestBaseSaf > eIntLib.inf). This test is automatically detected by > the HostUnitTestComilerPlugin and run by the > HostBasedUnitTestRunner as part of the CI process. > > A few notes about the current demo: > 1) The demo currently only works on Windows build > chains, but there's no reason to believe that it can't > work equally well on Linux build chains, and are happy > to work with anyone to get it there. > > 2) The demo currently has four failing conditions that > can be seen towards the end of MdePkg "Build and Test" > log file for this build: > https://nam06.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fdev.= azure.com%2Ftianocore%2Fedk2-ci-&data=3D02%7C01%7Cbret.barkelew%40micro= soft.com%7Caf69d76bb76f4a4e152e08d78277a97e%7C72f988bf86f141af91ab2d7cd011d= b47%7C1%7C0%7C637121322770003516&sdata=3D12H%2FC8YVt%2Fn5ud%2BYOp9vAzs6= vk2by46YFTzplhcz1xs%3D&reserved=3D0 > play/_build/results?buildId=3D2813 s.protection.outlook.com/?url=3Dhttps%3A%2F%2Fdev.azure.c > om%2Ftianocore%2Fedk2-ci- > play%2F_build%2Fresults%3FbuildId%3D2590&data=3D02%7C01%7 > Cbret.barkelew%40microsoft.com%7C6eac9932f3f640dd65ac08 > d778e731e3%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7 > C637110806683199782&sdata=3D1zd2oq8zRZUn3%2FUiGDuJZEZ5M0s > rtc2bqoa0%2BLbZB3s%3D&reserved=3D0> > "WARNING - Test SafeInt16ToChar8 - Status > d:\a\1\s\MdePkg\Test\UnitTest\Library\BaseSafeIntLib\Te > stBaseSafeIntLib.c:302: error: Failure! > WARNING - TestBaseSafeIntLib.exe Test Failed > WARNING - Test SafeInt32ToChar8 - Status > d:\a\1\s\MdePkg\Test\UnitTest\Library\BaseSafeIntLib\Te > stBaseSafeIntLib.c:638: error: Failure! > WARNING - TestBaseSafeIntLib.exe Test Failed > WARNING - Test SafeIntnToChar8 - Status > d:\a\1\s\MdePkg\Test\UnitTest\Library\BaseSafeIntLib\Te > stBaseSafeIntLib.c:1051: error: Failure! > WARNING - TestBaseSafeIntLib.exe Test Failed > WARNING - Test SafeInt64ToChar8 - Status > d:\a\1\s\MdePkg\Test\UnitTest\Library\BaseSafeIntLib\Te > stBaseSafeIntLib.c:1456: error: Failure!" > These failures seem to be legitimate and further work > should be done by the community to decide whether the > test case is correct or the library is correct, but one > of them needs to change. > > 3) Current demo pulls in test collateral from a fork of > the edk2-test repo. This repo is currently *very* heavy > due to the history of the UEFI SCT project and the > number of binaries that it pulls down. It's possible > that we (the community) need to select a better place > for this code to live. Maybe in edk2 primary (though > it's not explicitly firmware code, so it seems > unnecessary). Maybe in a new edk2-test2 repo or > something like that. > > There's an RFC doc here: > https://nam06.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fgith= ub.com%2Fcorthon%2Fedk2-staging%2Fblob%2Fedk2-host-&data=3D02%7C01%7Cbr= et.barkelew%40microsoft.com%7Caf69d76bb76f4a4e152e08d78277a97e%7C72f988bf86= f141af91ab2d7cd011db47%7C1%7C0%7C637121322770003516&sdata=3Dmroz5b5qSrD= o%2FBtl%2BxWqrTDPpaFclIdhgco6yHJFIFo%3D&reserved=3D0 > test_v2/Readme- > RFC.md url=3Dhttps%3A%2F%2Fgithub.com%2Fcorthon%2Fedk2- > staging%2Fblob%2Fedk2-host-test_v2%2FReadme- > RFC.md&data=3D02%7C01%7Cbret.barkelew%40microsoft.com%7C6 > eac9932f3f640dd65ac08d778e731e3%7C72f988bf86f141af91ab2 > d7cd011db47%7C1%7C0%7C637110806683199782&sdata=3D1nq1mHjs > bSZj4IQM5RBwSrvaQxO5cmDlTvf7VYNDV%2FA%3D&reserved=3D0> > > And a usage guide here: > https://nam06.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fgith= ub.com%2Fcorthon%2Fedk2-staging%2Fblob%2Fedk2-host-&data=3D02%7C01%7Cbr= et.barkelew%40microsoft.com%7Caf69d76bb76f4a4e152e08d78277a97e%7C72f988bf86= f141af91ab2d7cd011db47%7C1%7C0%7C637121322770003516&sdata=3Dmroz5b5qSrD= o%2FBtl%2BxWqrTDPpaFclIdhgco6yHJFIFo%3D&reserved=3D0 > test_v2/UnitTestPkg/ReadMe.md rotection.outlook.com/?url=3Dhttps%3A%2F%2Fgithub.com%2Fc > orthon%2Fedk2-staging%2Fblob%2Fedk2-host- > test_v2%2FUnitTestPkg%2FReadMe.md&data=3D02%7C01%7Cbret.b > arkelew%40microsoft.com%7C6eac9932f3f640dd65ac08d778e73 > 1e3%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637110 > 806683209750&sdata=3D0jRQ2Rzr9PWSYJ1YDs7l2aFS3PfAnTbTousY > Ye8IWTw%3D&reserved=3D0> > > Once again, would love to get this into EDK2 master > after stabilization, and most of this has previously > been shopped around in other discussion threads. This > is just where the rubber meets the road and is the > minimal subset of code that needs to land for folks to > start contributing unittests against the core libraries > that can be run as part of any CI pass. > > Thanks! > - Bret > > > > > > > > >=20 --_000_CH2PR21MB1400108196927D7CE1D91B52EF510CH2PR21MB1400namp_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Did you make sure to stuart_update? It will pull a = dependency repo that is only needed for CI.

 

- Bret

 

From: Kinney, Michael D &l= t;michael.d.kinney@intel.com>
Sent: Monday, December 16, 2019 2:31:13 PM
To: rfc@edk2.groups.io <rfc@edk2.groups.io>; Bret Barkelew &l= t;Bret.Barkelew@microsoft.com>; devel@edk2.groups.io <devel@edk2.grou= ps.io>; Andrew Fish <afish@apple.com>; Kinney, Michael D <micha= el.d.kinney@intel.com>
Subject: RE: [edk2-rfc] [EXTERNAL] Re: [edk2-devel] EDK2 Host-Based= Unit Test RFC (Now with docs!)
 
Hi Bret,

I am looking at the latest version of the content on your branch.

I am confused by MdePkg/Test/MdePkgTest.dsc.  It makes references
to lib classes and packages that do not exist.

  CmockaLib|CmockaHostUnitTestPkg/Library/CmockaLib/CmockaLib.inf
  OsServiceLib|HostBasedUnitTestPkg/Library/OsServiceLibHost/OsServic= eLibHost.inf
  UnitTestAssertLib|CmockaHostUnitTestPkg/Library/UnitTestAssertLibcm= ocka/UnitTestAssertLibcmocka.inf
  UnitTestLib|CmockaHostUnitTestPkg/Library/UnitTestLibcmocka/UnitTes= tLibcmocka.inf

Am I looking at an old file?

I am just trying to do a local build of the unit tests for the SafeIntLib.=

Thanks,

Mike


> -----Original Message-----
> From: rfc@edk2.groups.io <rfc@edk2.groups.io> On Behalf
> Of Bret Barkelew via Groups.Io
> Sent: Saturday, December 14, 2019 12:07 PM
> To: devel@edk2.groups.io; Andrew Fish
> <afish@apple.com>; Kinney, Michael D
> <michael.d.kinney@intel.com>
> Cc: rfc@edk2.groups.io
> Subject: Re: [edk2-rfc] [EXTERNAL] Re: [edk2-devel]
> EDK2 Host-Based Unit Test RFC (Now with docs!)
>
> The host-based tests now build on Linux/GCC, but the
> final executables don't seem to get created. Don't know
> where the disconnect is there. I can see the test get
> compiled (along with all the libraries), but it doesn't
> seem to link.
>
> Can you take a look? Thanks!
>
> - Bret
>
> From: Bret Barkelew<mailto:Bret.Barkelew@microsoft.com>
> Sent: Friday, December 13, 2019 4:46 PM
> To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>;
> Andrew Fish<mailto:afish@apple.= com>; Kinney, Michael
> D<mailto:michael.d.k= inney@intel.com>
> Cc: rfc@edk2.groups.io<mailt= o:rfc@edk2.groups.io>
> Subject: RE: [EXTERNAL] Re: [edk2-devel] EDK2 Host-
> Based Unit Test RFC (Now with docs!)
>
> Mike,
>
> I think I've gotten all the feedback here and all the
> action items from our call. The current branch should
> be good to go, minus the couple of things that Intel
> was going to look into.
>
> Thanks!
>
> - Bret
>
> From: Bret Barkelew<mailto:Bret.Barkelew@microsoft.com>
> Sent: Friday, December 6, 2019 2:21 PM
> To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>;
> Andrew Fish<mailto:afish@apple.= com>; Kinney, Michael
> D<mailto:michael.d.k= inney@intel.com>
> Cc: rfc@edk2.groups.io<mailt= o:rfc@edk2.groups.io>
> Subject: RE: [EXTERNAL] Re: [edk2-devel] EDK2 Host-
> Based Unit Test RFC (Now with docs!)
>
>
>   1.  I see that MdePkg adds a dependency on
> UnitTestPkg.  This makes UnitTestPkg the root package
> when building and running host based unit tests.  This
> makes sense, but good to highlight that all packages
> that use host based tests will introduce a new package
> dependency on UnitTestPkg.
>      *   Since the dependency only= applies to unit
> tests, can we update the DependencyCheck plugin to
> support listing dependencies for FW components separate
> from dependencies for unit tests?
>      *   Can move. Capability is t= here, but mistakenly
> added to the wrong section.
>   2.  I see UnitTestPkg declares 6 new lib classes. > Are all 6 classes intended to be used directly from a
> unit test case?  If some of these are only intended to
> be used from the framework inside the UnitTestPkg can
> we make them private?
>      *   Because there are differe= nt instances in
> multiple places (and conceivably more in the future),
> we don't see how we could make them private.
>   3.  In the MdePkg, I see a new top level directory > called 'HostLibrary'. Since these lib instances are
> only used from a host based test, can they be moved
> into  the 'Test' directory?
>      *   Can move.
>   4.  MdePkg/MdePkgTest.dsc.
>      *   Can this DSC file be move= d into the 'Test'
> directory?
>
>
> i.      Yes
>
>      *   I see this DSC file only = supports VS today.
> How much work is it to add GCC support?
>
>
> i.      Don't know. This is an item on our l= ist, but
> lower priority and not a blocker.
>
>   1.  MdePkg/HostLibrary/BaseLibHost
>      *   Why are there 2 INFs.&nbs= p; One with ASM and one
> without ASM?  Can we just use the one with ASM and
> assume NASM is installed?
>      *   I see the purpose of this= lib instance is to
> call the
>      *   SetJump()/LongJump().&nbs= p; So these
> implementations always work?  It looks like it assumes
> BASE_LIBRARY_JUMP_BUFFER is identical to the host OS
> user mode application setjmp()/longjmp() state.
>      *   Why are DRx and CRx regis= ters emulated?  I
> would think and code that depends on read/writing these
> registers would not be compatible with host based
> testing.  Can we change to ASSERT (FALSE)?
>      *   PatchInstructionX86() - I= suspect this will
> not work in a host based environment because it is self
> modifying code.  Should it be ASSERT (FALSE)?
>      *   Libraries were taken dire= ctly from the Intel
> work in HBFA. They worked so we kept them. We're just
> as interested in the answers to these questions as you
> are.
>   2.  MdePkg/HostLibrary/DebugLibHost
>      *   What is '#ifndef TEST_WIT= H_KLEE'
>      *   What is the 'PatchFormat(= )' function?  It is
> always disabled with if (0).
>      *   Are the PCDs to set the d= ebug message levels
> disabled on purpose? (DebugPrintEnabled(),
> DebugPrintLevelEnabled(), DebugCodeEnabled())
>      *   Libraries were taken dire= ctly from the Intel
> work in HBFA. They worked so we kept them. We're just
> as interested in the answers to these questions as you
> are.
>   3.  MdePkg/HostLibrary/BaseMemoryLibHost
>      *   Why can't we use
> MdePkg/Library/BaseMemoryLib/BaseMemoryLib/inf instead
> and reduce the number of host specific lib instances?
>      *   Libraries were taken dire= ctly from the Intel
> work in HBFA. They worked so we kept them. We're just
> as interested in the answers to these questions as you
> are.
>   4.  MdePkg/HostLibrary/MemoryAllocationLibHost
>      *   Why is are memcpy(), asse= rt(), memset() used
> in this lib?  I would expect this lib instance to match
> the UefiMemoryAllocationLib with the only the use of
> malloc() and free() to replace the UEFI Boot Services
> calls.
>      *   Libraries were taken dire= ctly from the Intel
> work in HBFA. They worked so we kept them. We're just
> as interested in the answers to these questions as you
> are.
>   5.  Host library instance naming conventions.
>      *   The current naming conven= tion uses the
> environment as a prefix(e.g. Pei, Smm, Uefi) and the
> services the lib instance uses as a post fix.  Would it
> make more sense to use 'Host' as a prefix instead of a
> postfix in the lib instance names?
>      *   I don't know if there's a= 1:1 relationship
> with these. For some library classes (that you might
> have host versions of), the class itself is the
> PeiSomethingLib, and the host version would be the
> PeiSomethingLibHost. No?
>   6.  Unicode vs ASCII strings
>      *   I see InitUnitTestFramewo= rk(),
> CreateUnitTestSuite(), and AddTestCase() all take
> Unicode strings for the labels.  I also see extra code
> to convert gEfiCallerBaseName from ASCII to Unicode to
> use it as a short name of a test framework.  I think it
> would be simpler if the parameters to these APIs were
> ASCII and the framework can convert to Unicode if
> needed.
>      *   No strong feelings, but w= e already have a
> bunch of tests written this way. Since the UnitTestLib
> is an abstraction that works as well in Shell as in the
> Host, going with Unicode strings seemed to match the
> art for Shell apps (since the framework was written for
> Shell first).
>   7.  Will InitUnitTestFramework() and
> CreateUnitTestSuite() always be called in pairs?  If
> so, can we combine these to a single API?
>      *   I see the SafeIntLib exam= ple create a single
> framework and multiple test suites.  Perhaps we can
> have a single CreateUnitTestSuite() API where Framework
> is an IN/OUT and if it is passed in as NULL, the
> Framework handle is created.
>
>
> i.      It's not always in pairs. You would = only have a
> single framework init, but could have multiple suites.
>
>      *   I see a pattern where the=
> CreateUnitTestSuite() 'Package' parameter is used as a
> prefix to every call to AddTestCase() in the
> 'ClassName' parameter.  Can we simplify AddTestCase()
> so it only need to pass in the name of the unit test
> case, and the framework can append Package  and the
> unit test case name?
>
>
> i.      Right now, our tests just coincident= ally share
> similar names with the packages, but we don't feel this
> is axiomatic and don't want to force this naming on
> others, who may be trying to bolt into other reporting
> structures.
>
>   1.  I see the use of the 'Fw' variable as a shorthan= d
> for 'Framework'.  Can we use something other than 'Fw'.
> It may be confused with 'Firmware'.
>      *   No real arguments. Wouldn= 't fight a find-
> replace, but it'll just be a bunch of touches as we
> commit.
>   2.  UnitTestPkg/Include/UnitTestTypes.h
>      *   I see several hard coded = string lengths.
> Since this runs in a host environment and strings can
> always be allocated, can the hard coded lengths be
> removed?  Update the structs to use pointers to strings
> instead of string arrays, and the framework can manage
> alloc() and free()?
>
>
> i.      Given that this framework is designe= d to be
> nimble enough to work in PEI and SMM and other
> constrictive environments, we wanted to keep fixed
> sizing.
>
>      *   How are Fingerprints used= ?  The idea of using
> as hash as a unique identifier is a good idea.  How is
> the hash calculated?  What unit test code artifacts are
> used so developers know what parameters must be unique?
> Can we just decide to use a specific hash
> algorithm/size and the structure can be a pointer to an
> allocated buffer instead of a fixed size array to make
> it easy to change the algorithm/size in the future?
>
>
> i.      Fingerprints are unique IDs to make = sure that
> serialize/unserialized state matches the tests upon re-
> entry. I'm not married to MD5, but it needs to be
> predictably unique, and carried with the framework. I
> would not want to make any requirements on CryptoLib,
> since these aren't crypto-strong.
>
>      *   Update all the strings to= be ASCII?  See
> Unicode vs ASCII above.
>
>
> i.      Ideally not, unless there's a strong= use case.
>
>      *   UNIT_TEST_SAVE_TEST - The= last field is a
> variable sized array, so it can be a formal field.
>
>
> i.      Not opposed, but don't really want t= o make the
> change myself. Is there a style guide that this is
> breaking?
>
>      *   UNIT_TEST_SAVE_CONTEXT - = - The last field is a
> variable sized array, so it can be a formal field.
>      *   UNIT_TEST_SAVE_HEADER - C= an the last 3 fields
> be changed to pointers and be formal fields?
>      *   Do the structures really = need to be packed?
> Specially with the changes suggested above?  Is the
> intent to potentially share data stored on disk between
> different host execution environments that may have
> different width architectures?
>
>
> i.      That's an interesting point. Could y= ou draw up
> your suggestion and submit a PR?
>
>   1.  UnitTestPkg - UnitTestLib.h
>      *   Can you provide more cont= ext for the APIs
> SaveFrameworkState(), SaveFrameworkStateAndQuit(),
> SaveFrameworkStateAndReboot(),
> SetFrameworkBootNextDevice().  I do not see any Load()
> functions, so how would a set of tests be resumed?  If
> these do not apply to host based tests, should these be
> split out to a different lib class?
>
>
> i.      I'll improve the documentation aroun= d these
> functions. I will acknowledge, however, that this is an
> interface that I expect to evolve as we figure out what
> kinds of tests the community wants support for "out of
> the box". If we want to easily enable tests around -
> for example - ExitBootServices, we will likely want to
> tweak this going forward to its simplest form. The
> version we have here was sufficient to enable the test
> cases that we've currently written.
>
>   1.  UnitTestPkg/Library/UnitTestLib
>      *   I see an implementation o= f MD5.  We should not
> do our own.  We should use an approved implementation
> such as OpenSSL.
>
>
> i.      Happy to discuss another implementat= ion, but
> this is not crypto-strong. It's just for uniqueness. A
> sufficiently long CRC would probably work, too.
>
>   1.  UnitTestPkg/Library/UnitTestTerminationLibTbd >      *   Do we really need this li= b instance now?
>
>
> i.      This is here so that we can figure o= ut what
> this should look like for a host. Host obviously
> wouldn't reset, but could conceivably quit. Or maybe
> that doesn't make any sense for a host.
>
>
>
> - Bret
>
>
> From: devel@edk2.groups.io <devel@edk2.groups.io> on
> behalf of Bret Barkelew via Groups.Io
> <bret.barkelew=3Dmicrosoft.com@groups.io>
> Sent: Wednesday, December 4, 2019 10:24:21 AM
> To: Andrew Fish <afish@apple.com>; devel@edk2.groups.io
> <devel@edk2.groups.io>; Kinney, Michael D
> <michael.d.kinney@intel.com>
> Cc: rfc@edk2.groups.io <rfc@edk2.groups.io>
> Subject: Re: [EXTERNAL] Re: [edk2-devel] EDK2 Host-
> Based Unit Test RFC (Now with docs!)
>
>
> Andrew,
>
> I agree with your points.
>
>
>
> Mike,
>
> You've got a lot more there. Let me take a look and
> update the draft. I'll ping back ASAP.
>
>
>
> - Bret
>
>
>
> From: Andrew Fish<mailto:afish@= apple.com>
> Sent: Wednesday, December 4, 2019 9:50 AM
> To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>;
> Kinney, Michael D<ma= ilto:michael.d.kinney@intel.com>
> Cc: Bret Barkelew<m= ailto:Bret.Barkelew@microsoft.com>;
> rfc@edk2.groups.io<mailto:rf= c@edk2.groups.io>
> Subject: [EXTERNAL] Re: [edk2-devel] EDK2 Host-Based
> Unit Test RFC (Now with docs!)
>
>
>
> Mike,
>
>
>
> I like and agree with your comments.
>
>
>
> On the UnitTestPkg(s) dependency issue I think it would
> make sense to move as much as possible into the
> *Pkg/Test/ ( *Pkg/HostLibrary,  MdePkg/MdePkgTest.dsc,
> etc.) directory structure. It looks like for
> implementation specific tests we skip the Test
> directory and drop the UnitTest or FuzzTest directly
> into modules directory. Maybe we should follow the same
> pattern as *Pkg/Test and have a Test directory? This
> will help minimize the number of "reserved" directories
> we need for managing the testing. Have a standardized
> directory structure will make it easier to
> differentiate the code from tests while doing `git
> grep` or other forms of code spelunking.
>
>
>
> A Host-Based Unit Test for a Library makes sense as you
> can link directly against the library and test it. A
> Host-Based Unit Test for Protocol/PPI/GUID [1] seems a
> little more complex. It is easy enough to write tests
> for the interfaces but what APIs do you call to get
> access to the protocol?
>
>
>
> Per our conversation at the Stewards meeting I think
> make sense to try to roll out the testing of libraries
> as the 1st phase. The mocking required for drivers
> could get quite complex and let us not get bogged down
> in all that to get something working.
>
>
>
> [1] *Pkg/Test/UnitTest/[Library|Protocol|Ppi|Guid]
>
>
>
> Thanks,
>
>
>
> Andrew Fish
>
>
>
>
>
> On Dec 2, 2019, at 3:12 PM, Michael D Kinney
> <michael.d.kinney@intel.com<mailto:michael.d.kinney@int
> el.com>> wrote:
>
>
>
> Hi Bret,
>
>
>
> Thanks for posting this content.  Host based unit
> testing is a very valuable addition to the CI checks.
>
>
>
> I have the following comments:
>
>
>
>   1.  I see that MdePkg adds a dependency on
> UnitTestPkg.  This makes UnitTestPkg the root package
> when building and running host based unit tests.  This
> makes sense, but good to highlight that all packages
> that use host based tests will introduce a new package
> dependency on UnitTestPkg.
>
>      *   Since the dependency only= applies to unit
> tests, can we update the DependencyCheck plugin to
> support listing dependencies for FW components separate
> from dependencies for unit tests?
>
>   1.  I see UnitTestPkg declares 6 new lib classes. > Are all 6 classes intended to be used directly from a
> unit test case?  If some of these are only intended to
> be used from the framework inside the UnitTestPkg can
> we make them private?
>   2.  In the MdePkg, I see a new top level directory > called 'HostLibrary'. Since these lib instances are
> only used from a host based test, can they be moved
> into  the 'Test' directory?
>   3.  MdePkg/MdePkgTest.dsc.
>
>      *   Can this DSC file be move= d into the 'Test'
> directory?
>      *   I see this DSC file only = supports VS today.
> How much work is it to add GCC support?
>
>   1.  MdePkg/HostLibrary/BaseLibHost
>
>      *   Why are there 2 INFs.&nbs= p; One with ASM and one
> without ASM?  Can we just use the one with ASM and
> assume NASM is installed?
>      *   I see the purpose of this= lib instance is to
> call the
>      *   SetJump()/LongJump().&nbs= p; So these
> implementations always work?  It looks like it assumes
> BASE_LIBRARY_JUMP_BUFFER is identical to the host OS
> user mode applicationsetjmp()/longjmp() state.
>      *   Why are DRx and CRx regis= ters emulated?  I
> would think and code that depends on read/writing these
> registers would not be compatible with host based
> testing.  Can we change to ASSERT (FALSE)?
>      *   PatchInstructionX86() - I= suspect this will
> not work in a host based environment because it is self
> modifying code.  Should it be ASSERT (FALSE)?
>
>   1.  MdePkg/HostLibrary/DebugLibHost
>
>      *   What is '#ifndef TEST_WIT= H_KLEE'
>      *   What is the 'PatchFormat(= )' function?  It is
> always disabled with if (0).
>      *   Are the PCDs to set the d= ebug message levels
> disabled on purpose? (DebugPrintEnabled(),
> DebugPrintLevelEnabled(), DebugCodeEnabled())
>
>   1.  MdePkg/HostLibrary/BaseMemoryLibHost
>
>      *   Why can't we use
> MdePkg/Library/BaseMemoryLib/BaseMemoryLib/inf instead
> and reduce the number of host specific lib instances?
>
>   1.  MdePkg/HostLibrary/MemoryAllocationLibHost
>
>      *   Why is are memcpy(), asse= rt(), memset() used
> in this lib?  I would expect this lib instance to match
> the UefiMemoryAllocationLib with the only the use of
> malloc() and free() to replace the UEFI Boot Services
> calls.
>
>   1.  Host library instance naming conventions.
>
>      *   The current naming conven= tion uses the
> environment as a prefix(e.g. Pei, Smm, Uefi) and the
> services the lib instance uses as a post fix.  Would it
> make more sense to use 'Host' as a prefix instead of a
> postfix in the lib instance names?
>
>   1.  Unicode vs ASCII strings
>
>      *   I see InitUnitTestFramewo= rk(),
> CreateUnitTestSuite(), and AddTestCase() all take
> Unicode strings for the labels.  I also see extra code
> to convert gEfiCallerBaseName from ASCII to Unicode to
> use it as a short name of a test framework.  I think it
> would be simpler if the parameters to these APIs were
> ASCII and the framework can convert to Unicode if
> needed.
>
>   1.  Will InitUnitTestFramework() and
> CreateUnitTestSuite() always be called in pairs?  If
> so, can we combine these to a single API?
>
>      *   I see the SafeIntLib exam= ple create a single
> framework and multiple test suites.  Perhaps we can
> have a single CreateUnitTestSuite() API where Framework
> is an IN/OUT and if it is passed in as NULL, the
> Framework handle is created.
>      *   I see a pattern where the=
> CreateUnitTestSuite() 'Package' parameter is used as a
> prefix to every call to AddTestCase() in the
> 'ClassName' parameter.  Can we simplify AddTestCase()
> so it only need to pass in the name of the unit test
> case, and the framework can append Package  and the
> unit test case name?
>
>   1.  I see the use of the 'Fw' variable as a shorthan= d
> for 'Framework'.  Can we use something other than 'Fw'.
> It may be confused with 'Firmware'.
>   2.  UnitTestPkg/Include/UnitTestTypes.h
>
>      *   I see several hard coded = string lengths.
> Since this runs in a host environment and strings can
> always be allocated, can the hard coded lengths be
> removed?  Update the structs to use pointers to strings
> instead of string arrays, and the framework can manage
> alloc() and free()?
>      *   How are Fingerprints used= ?  The idea of using
> as hash as a unique identifier is a good idea.  How is
> the hash calculated?  What unit test code artifacts are
> used so developers know what parameters must be unique?
> Can we just decide to use a specific hash
> algorithm/size and the structure can be a pointer to an
> allocated buffer instead of a fixed size array to make
> it easy to change the algorithm/size in the future?
>      *   Update all the strings to= be ASCII?  See
> Unicode vs ASCII above.
>      *   UNIT_TEST_SAVE_TEST - The= last field is a
> variable sized array, so it can be a formal field.
>      *   UNIT_TEST_SAVE_CONTEXT - = - The last field is a
> variable sized array, so it can be a formal field.
>      *   UNIT_TEST_SAVE_HEADER - C= an the last 3 fields
> be changed to pointers and be formal fields?
>      *   Do the structures really = need to be packed?
> Specially with the changes suggested above?  Is the
> intent to potentially share data stored on disk between
> different host execution environments that may have
> different width architectures?
>
>   1.  UnitTestPkg - UnitTestLib.h
>
>      *   Can you provide more cont= ext for the APIs
> SaveFrameworkState(), SaveFrameworkStateAndQuit(),
> SaveFrameworkStateAndReboot(),
> SetFrameworkBootNextDevice().  I do not see any Load()
> functions, so how would a set of tests be resumed?  If
> these do not apply to host based tests, should these be
> split out to a different lib class?
>
>   1.  UnitTestPkg/Library/UnitTestLib
>
>      *   I see an implementation o= f MD5.  We should not
> do our own.  We should use an approved implementation
> such as OpenSSL.
>
>   1.  UnitTestPkg/Library/UnitTestTerminationLibTbd >
>      *   Do we really need this li= b instance now?
>
>
>
> Thanks,
>
>
>
> Mike
>
>
>
> From: devel@edk2.groups.io<mailto:devel@edk2.groups.io>
> <devel@edk2.groups.io<mailto:devel@edk2.groups.io>> On > Behalf Of Bret Barkelew via Groups.Io
> Sent: Thursday, November 21, 2019 11:39 PM
> To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>
> Subject: [edk2-devel] EDK2 Host-Based Unit Test RFC
> (Now with docs!)
>
>
>
> Now that CI has landed in edk2/master, we'd like to get
> the basic framework for unittesting staged as well.
> Target intercept date would be immediately after the
> 2019/11 stabilization, so we wanted to go ahead and get
> comments now.
>
> The host unittest framework consists of five primary
> pieces:
> - The test library (Cmocka)
> - Support libraries (Found in UnitTestPkg)
> - The test support plugins (HostUnitTestComilerPlugin,
> HostUnitTestDxeCompleteCheck, HostBasedUnitTestRunner)
> - The configuration in the package-based *.ci.yaml file
> and package-based Test.dsc
> - The tests themselves
>
> We have a demo branch set up at:
> https://nam06.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fgithub= .com%2Fcorthon%2Fedk2-staging%2Ftree%2Fedk2-host-&amp;data=3D02%7C01%7C= bret.barkelew%40microsoft.com%7Caf69d76bb76f4a4e152e08d78277a97e%7C72f988bf= 86f141af91ab2d7cd011db47%7C1%7C0%7C637121322769998529&amp;sdata=3D0%2FE= UzZG6J3F%2FLnCt0sICGwsxpwjXf7cbUMnQQWeSbtw%3D&amp;reserved=3D0
> test_v2<https://nam06.safelinks.protection.outloo= k.com/
> ?url=3Dhttps%3A%2F%2Fgithub.com%2Fcorthon%2Fedk2-
> staging%2Ftree%2Fedk2-host-
> test_v2&data=3D02%7C01%7Cbret.barkelew%40microsoft.com%7C
> 6eac9932f3f640dd65ac08d778e731e3%7C72f988bf86f141af91ab
> 2d7cd011db47%7C1%7C0%7C637110806683189828&sdata=3DJQ%2BoW
> xlEBOK2sH55cRAPVa3OpAxTsm6eHcdbWSo9CVo%3D&reserved=3D0>
> We also have a demo build pipeline at:
> https://nam06.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fdev.az= ure.com%2Ftianocore%2Fedk2-ci-&amp;data=3D02%7C01%7Cbret.barkelew%40mic= rosoft.com%7Caf69d76bb76f4a4e152e08d78277a97e%7C72f988bf86f141af91ab2d7cd01= 1db47%7C1%7C0%7C637121322770003516&amp;sdata=3D12H%2FC8YVt%2Fn5ud%2BYOp= 9vAzs6vk2by46YFTzplhcz1xs%3D&amp;reserved=3D0
> play/_build?definitionId=3D36&_a=3Dsummary<https://nam06.safel= inks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fnam06.sa&amp;data=3D02= %7C01%7Cbret.barkelew%40microsoft.com%7Caf69d76bb76f4a4e152e08d78277a97e%7C= 72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637121322770003516&amp;sdata= = =3DTrBwMkL6jCSS0XiZhegHkA4VQheFvj3auBWkOWD5ysg%3D&amp;reserved=3D0
> felinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fdev.a
> zure.com%2Ftianocore%2Fedk2-ci-
> play%2F_build%3FdefinitionId%3D36%26_a%3Dsummary&data=3D0
> 2%7C01%7Cbret.barkelew%40microsoft.com%7C6eac9932f3f640
> dd65ac08d778e731e3%7C72f988bf86f141af91ab2d7cd011db47%7
> C1%7C0%7C637110806683189828&sdata=3DwBwn1ehjyTmYNKVSvlEZS
> XK5qyeu4EPAL7FzdYntnt4%3D&reserved=3D0>
>
> The current demo branch contains a single test in
> MdePkg for SafeIntLib (module file
> MdePkg\Test\UnitTest\Library\BaseSafeIntLib\TestBaseSaf
> eIntLib.inf). This test is automatically detected by
> the HostUnitTestComilerPlugin and run by the
> HostBasedUnitTestRunner as part of the CI process.
>
> A few notes about the current demo:
> 1) The demo currently only works on Windows build
> chains, but there's no reason to believe that it can't
> work equally well on Linux build chains, and are happy
> to work with anyone to get it there.
>
> 2) The demo currently has four failing conditions that
> can be seen towards the end of MdePkg "Build and Test"
> log file for this build:
> https://nam06.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fdev.az= ure.com%2Ftianocore%2Fedk2-ci-&amp;data=3D02%7C01%7Cbret.barkelew%40mic= rosoft.com%7Caf69d76bb76f4a4e152e08d78277a97e%7C72f988bf86f141af91ab2d7cd01= 1db47%7C1%7C0%7C637121322770003516&amp;sdata=3D12H%2FC8YVt%2Fn5ud%2BYOp= 9vAzs6vk2by46YFTzplhcz1xs%3D&amp;reserved=3D0
> play/_build/results?buildId=3D2813<https://nam06.safelinks.protect= ion.outlook.com/?url=3Dhttps%3A%2F%2Fnam06.safelink&amp;data=3D02%7C01%= 7Cbret.barkelew%40microsoft.com%7Caf69d76bb76f4a4e152e08d78277a97e%7C72f988= bf86f141af91ab2d7cd011db47%7C1%7C0%7C637121322770003516&amp;sdata=3DOrh= L0m%2BJC5obLQoU%2BgmFS9StJgvv2eEfSj8jsmrt9hI%3D&amp;reserved=3D0
> s.protection.outlook.com/?url=3Dhttps%3A%2F%2Fdev.azure.c
> om%2Ftianocore%2Fedk2-ci-
> play%2F_build%2Fresults%3FbuildId%3D2590&data=3D02%7C01%7
> Cbret.barkelew%40microsoft.com%7C6eac9932f3f640dd65ac08
> d778e731e3%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7
> C637110806683199782&sdata=3D1zd2oq8zRZUn3%2FUiGDuJZEZ5M0s
> rtc2bqoa0%2BLbZB3s%3D&reserved=3D0>
> "WARNING -   Test SafeInt16ToChar8 - Status
> d:\a\1\s\MdePkg\Test\UnitTest\Library\BaseSafeIntLib\Te
> stBaseSafeIntLib.c:302: error: Failure!
> WARNING - TestBaseSafeIntLib.exe Test Failed
> WARNING -   Test SafeInt32ToChar8 - Status
> d:\a\1\s\MdePkg\Test\UnitTest\Library\BaseSafeIntLib\Te
> stBaseSafeIntLib.c:638: error: Failure!
> WARNING - TestBaseSafeIntLib.exe Test Failed
> WARNING -   Test SafeIntnToChar8 - Status
> d:\a\1\s\MdePkg\Test\UnitTest\Library\BaseSafeIntLib\Te
> stBaseSafeIntLib.c:1051: error: Failure!
> WARNING - TestBaseSafeIntLib.exe Test Failed
> WARNING -   Test SafeInt64ToChar8 - Status
> d:\a\1\s\MdePkg\Test\UnitTest\Library\BaseSafeIntLib\Te
> stBaseSafeIntLib.c:1456: error: Failure!"
> These failures seem to be legitimate and further work
> should be done by the community to decide whether the
> test case is correct or the library is correct, but one
> of them needs to change.
>
> 3) Current demo pulls in test collateral from a fork of
> the edk2-test repo. This repo is currently *very* heavy
> due to the history of the UEFI SCT project and the
> number of binaries that it pulls down. It's possible
> that we (the community) need to select a better place
> for this code to live. Maybe in edk2 primary (though
> it's not explicitly firmware code, so it seems
> unnecessary). Maybe in a new edk2-test2 repo or
> something like that.
>
> There's an RFC doc here:
> https://nam06.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fgithub= .com%2Fcorthon%2Fedk2-staging%2Fblob%2Fedk2-host-&amp;data=3D02%7C01%7C= bret.barkelew%40microsoft.com%7Caf69d76bb76f4a4e152e08d78277a97e%7C72f988bf= 86f141af91ab2d7cd011db47%7C1%7C0%7C637121322770003516&amp;sdata=3Dmroz5= b5qSrDo%2FBtl%2BxWqrTDPpaFclIdhgco6yHJFIFo%3D&amp;reserved=3D0
> test_v2/Readme-
> RFC.md<https://nam06.safelinks.protection.outlook= .com/?
> url=3Dhttps%3A%2F%2Fgithub.com%2Fcorthon%2Fedk2-
> staging%2Fblob%2Fedk2-host-test_v2%2FReadme-
> RFC.md&data=3D02%7C01%7Cbret.barkelew%40microsoft.com%7C6
> eac9932f3f640dd65ac08d778e731e3%7C72f988bf86f141af91ab2
> d7cd011db47%7C1%7C0%7C637110806683199782&sdata=3D1nq1mHjs
> bSZj4IQM5RBwSrvaQxO5cmDlTvf7VYNDV%2FA%3D&reserved=3D0>
>
> And a usage guide here:
> https://nam06.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fgithub= .com%2Fcorthon%2Fedk2-staging%2Fblob%2Fedk2-host-&amp;data=3D02%7C01%7C= bret.barkelew%40microsoft.com%7Caf69d76bb76f4a4e152e08d78277a97e%7C72f988bf= 86f141af91ab2d7cd011db47%7C1%7C0%7C637121322770003516&amp;sdata=3Dmroz5= b5qSrDo%2FBtl%2BxWqrTDPpaFclIdhgco6yHJFIFo%3D&amp;reserved=3D0
> test_v2/UnitTestPkg/ReadMe.md<https://nam06.safelinks.protection.o= utlook.com/?url=3Dhttps%3A%2F%2Fnam06.safelinks.p&amp;data=3D02%7C01%7C= bret.barkelew%40microsoft.com%7Caf69d76bb76f4a4e152e08d78277a97e%7C72f988bf= 86f141af91ab2d7cd011db47%7C1%7C0%7C637121322770003516&amp;sdata=3DhPFhk= 5UWEA4JzSl9KyhX8%2Fj44ls5hx6T4i2bz8H9SB0%3D&amp;reserved=3D0
> rotection.outlook.com/?url=3Dhttps%3A%2F%2Fgithub.com%2Fc
> orthon%2Fedk2-staging%2Fblob%2Fedk2-host-
> test_v2%2FUnitTestPkg%2FReadMe.md&data=3D02%7C01%7Cbret.b
> arkelew%40microsoft.com%7C6eac9932f3f640dd65ac08d778e73
> 1e3%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637110
> 806683209750&sdata=3D0jRQ2Rzr9PWSYJ1YDs7l2aFS3PfAnTbTousY
> Ye8IWTw%3D&reserved=3D0>
>
> Once again, would love to get this into EDK2 master
> after stabilization, and most of this has previously
> been shopped around in other discussion threads. This
> is just where the rubber meets the road and is the
> minimal subset of code that needs to land for folks to
> start contributing unittests against the core libraries
> that can be run as part of any CI pass.
>
> Thanks!
> - Bret
>
>
>
>
>
>
>
>
>

--_000_CH2PR21MB1400108196927D7CE1D91B52EF510CH2PR21MB1400namp_--