*An* answer is:

- There’s a DevOps virtual environment that you can run that will pretend to be a build agent. I played with it years ago, but that was before the CI tooling (and it was years ago) so I don’t know how the capabilities compare.

My real answer is:

- If you run the three commands -- setup, update, and build -- without any parameters other than your TOOL_CHAIN_TAG, it should run all tests on all packages for that particular OS/toolchain.

- Bret

From: Rebecca Cran via groups.io
Sent: Thursday, March 4, 2021 9:45 AM
To: devel@edk2.groups.io; lersek@redhat.com
Cc: Sean Brogan; Bret Barkelew; Kinney, Michael D
Subject: [EXTERNAL] Re: [edk2-devel] generating the test matrix for local CI runs

On 3/4/21 10:04 AM, Laszlo Ersek wrote:
> (Sigh, my address book pulled one over me -- resending to the correct
> list address now. Please ignore the previous posting that you may have
> gotten "in private".)
>
> Hi All,
>
> while I can run specific CI tests locally, using the commands listed at:
>
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2F.pytool&data=04%7C01%7Cbret.barkelew%40microsoft.com%7Ce4e5493578cf45b804c608d8df354b05%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504767278995847%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=l8WblhD%2B7dpkFaCZ8roSfNUmj1imytHi5eUT%2F%2BKrLCo%3D&reserved=0
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2FArmVirtPkg%2FPlatformCI&data=04%7C01%7Cbret.barkelew%40microsoft.com%7Ce4e5493578cf45b804c608d8df354b05%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504767278995847%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=8yG7Mro9Li4XeCvfmPhfReBbipz47nyAjUmMKMGfh%2Bo%3D&reserved=0
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2FOvmfPkg%2FPlatformCI&data=04%7C01%7Cbret.barkelew%40microsoft.com%7Ce4e5493578cf45b804c608d8df354b05%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504767278995847%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=NUhAJ8zkITwPkvCrUqb7ZogSQTJuBXv0eL6Au0sYC2M%3D&reserved=0
>
> I don't know how I can determine and drive the whole "test plan" locally.
>
> When I submit a PR to github, "something" generates (say) 83 test cases.
> How can I run that "something" locally?
>
> Assume I have the following:
> - a Linux builder VM, with up-to-date tools,
> - a Windows builder VM, with up-to-date tools,
> - a topic branch, on top of "master".
>
> How do I generate and run those same 83 tests, locally?
>
> It has happened to me that all my local CI steps succeeded (meaning both
> the Windows and the Linux builder), but github still rejected the PR --
> the reason was that I failed to think of a particular test that github
> would run. Composing 83 "stuart" command lines isn't something I'd like
> to do manually.

FWIW it looks like people have been looking for similar functionality of
testing a pipeline locally, and not found it.

e.g.:
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmicrosoft%2Fazure-pipelines-agent%2Fissues%2F1764&data=04%7C01%7Cbret.barkelew%40microsoft.com%7Ce4e5493578cf45b804c608d8df354b05%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504767278995847%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=XMxZesyKxVN9Wg3SyRpUbnnmKzojevHCqQhzQ1Pc4%2Bg%3D&reserved=0
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdevelopercommunity.visualstudio.com%2Ft%2Fyaml-how-to-test-yaml-locally-before-commit%2F1302759&data=04%7C01%7Cbret.barkelew%40microsoft.com%7Ce4e5493578cf45b804c608d8df354b05%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504767278995847%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=gwGZAQJAIXIS0LODFpglrCf5Ua2JeMnNm3A3auCrhx8%3D&reserved=0

--
Rebecca Cran