* generating the test matrix for local CI runs @ 2021-03-04 17:04 Laszlo Ersek 2021-03-04 17:45 ` [edk2-devel] " Rebecca Cran 0 siblings, 1 reply; 7+ messages in thread From: Laszlo Ersek @ 2021-03-04 17:04 UTC (permalink / raw) To: edk2-devel-groups-io; +Cc: Sean Brogan, Bret Barkelew, Michael Kinney (Sigh, my address book pulled one over me -- resending to the correct list address now. Please ignore the previous posting that you may have gotten "in private".) Hi All, while I can run specific CI tests locally, using the commands listed at: https://github.com/tianocore/edk2/tree/master/.pytool https://github.com/tianocore/edk2/tree/master/ArmVirtPkg/PlatformCI https://github.com/tianocore/edk2/tree/master/OvmfPkg/PlatformCI I don't know how I can determine and drive the whole "test plan" locally. When I submit a PR to github, "something" generates (say) 83 test cases. How can I run that "something" locally? Assume I have the following: - a Linux builder VM, with up-to-date tools, - a Windows builder VM, with up-to-date tools, - a topic branch, on top of "master". How do I generate and run those same 83 tests, locally? It has happened to me that all my local CI steps succeeded (meaning both the Windows and the Linux builder), but github still rejected the PR -- the reason was that I failed to think of a particular test that github would run. Composing 83 "stuart" command lines isn't something I'd like to do manually. Thanks Laszlo ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [edk2-devel] generating the test matrix for local CI runs 2021-03-04 17:04 generating the test matrix for local CI runs Laszlo Ersek @ 2021-03-04 17:45 ` Rebecca Cran 2021-03-04 17:56 ` [EXTERNAL] " Bret Barkelew 0 siblings, 1 reply; 7+ messages in thread From: Rebecca Cran @ 2021-03-04 17:45 UTC (permalink / raw) To: devel, lersek; +Cc: Sean Brogan, Bret Barkelew, Michael Kinney On 3/4/21 10:04 AM, Laszlo Ersek wrote: > (Sigh, my address book pulled one over me -- resending to the correct > list address now. Please ignore the previous posting that you may have > gotten "in private".) > > Hi All, > > while I can run specific CI tests locally, using the commands listed at: > > https://github.com/tianocore/edk2/tree/master/.pytool > https://github.com/tianocore/edk2/tree/master/ArmVirtPkg/PlatformCI > https://github.com/tianocore/edk2/tree/master/OvmfPkg/PlatformCI > > I don't know how I can determine and drive the whole "test plan" locally. > > When I submit a PR to github, "something" generates (say) 83 test cases. > How can I run that "something" locally? > > Assume I have the following: > - a Linux builder VM, with up-to-date tools, > - a Windows builder VM, with up-to-date tools, > - a topic branch, on top of "master". > > How do I generate and run those same 83 tests, locally? > > It has happened to me that all my local CI steps succeeded (meaning both > the Windows and the Linux builder), but github still rejected the PR -- > the reason was that I failed to think of a particular test that github > would run. Composing 83 "stuart" command lines isn't something I'd like > to do manually. FWIW it looks like people have been looking for similar functionality of testing a pipeline locally, and not found it. e.g.: https://github.com/microsoft/azure-pipelines-agent/issues/1764 https://developercommunity.visualstudio.com/t/yaml-how-to-test-yaml-locally-before-commit/1302759 -- Rebecca Cran ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [EXTERNAL] Re: [edk2-devel] generating the test matrix for local CI runs 2021-03-04 17:45 ` [edk2-devel] " Rebecca Cran @ 2021-03-04 17:56 ` Bret Barkelew 2021-03-04 18:20 ` Bret Barkelew 0 siblings, 1 reply; 7+ messages in thread From: Bret Barkelew @ 2021-03-04 17:56 UTC (permalink / raw) To: devel@edk2.groups.io, rebecca@nuviainc.com, lersek@redhat.com Cc: Sean Brogan, Kinney, Michael D [-- Attachment #1: Type: text/plain, Size: 4453 bytes --] *An* answer is: - There’s a DevOps virtual environment that you can run that will pretend to be a build agent. I played with it years ago, but that was before the CI tooling (and it was years ago) so I don’t know how the capabilities compare. My real answer is: - If you run the three commands -- setup, update, and build -- without any parameters other than your TOOL_CHAIN_TAG, it should run all tests on all packages for that particular OS/toolchain. - Bret From: Rebecca Cran via groups.io<mailto:rebecca=nuviainc.com@groups.io> Sent: Thursday, March 4, 2021 9:45 AM To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; lersek@redhat.com<mailto:lersek@redhat.com> Cc: Sean Brogan<mailto:sean.brogan@microsoft.com>; Bret Barkelew<mailto:Bret.Barkelew@microsoft.com>; Kinney, Michael D<mailto:michael.d.kinney@intel.com> Subject: [EXTERNAL] Re: [edk2-devel] generating the test matrix for local CI runs On 3/4/21 10:04 AM, Laszlo Ersek wrote: > (Sigh, my address book pulled one over me -- resending to the correct > list address now. Please ignore the previous posting that you may have > gotten "in private".) > > Hi All, > > while I can run specific CI tests locally, using the commands listed at: > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2F.pytool&data=04%7C01%7Cbret.barkelew%40microsoft.com%7Ce4e5493578cf45b804c608d8df354b05%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504767278995847%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=l8WblhD%2B7dpkFaCZ8roSfNUmj1imytHi5eUT%2F%2BKrLCo%3D&reserved=0 > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2FArmVirtPkg%2FPlatformCI&data=04%7C01%7Cbret.barkelew%40microsoft.com%7Ce4e5493578cf45b804c608d8df354b05%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504767278995847%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=8yG7Mro9Li4XeCvfmPhfReBbipz47nyAjUmMKMGfh%2Bo%3D&reserved=0 > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2FOvmfPkg%2FPlatformCI&data=04%7C01%7Cbret.barkelew%40microsoft.com%7Ce4e5493578cf45b804c608d8df354b05%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504767278995847%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=NUhAJ8zkITwPkvCrUqb7ZogSQTJuBXv0eL6Au0sYC2M%3D&reserved=0 > > I don't know how I can determine and drive the whole "test plan" locally. > > When I submit a PR to github, "something" generates (say) 83 test cases. > How can I run that "something" locally? > > Assume I have the following: > - a Linux builder VM, with up-to-date tools, > - a Windows builder VM, with up-to-date tools, > - a topic branch, on top of "master". > > How do I generate and run those same 83 tests, locally? > > It has happened to me that all my local CI steps succeeded (meaning both > the Windows and the Linux builder), but github still rejected the PR -- > the reason was that I failed to think of a particular test that github > would run. Composing 83 "stuart" command lines isn't something I'd like > to do manually. FWIW it looks like people have been looking for similar functionality of testing a pipeline locally, and not found it. e.g.: https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmicrosoft%2Fazure-pipelines-agent%2Fissues%2F1764&data=04%7C01%7Cbret.barkelew%40microsoft.com%7Ce4e5493578cf45b804c608d8df354b05%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504767278995847%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=XMxZesyKxVN9Wg3SyRpUbnnmKzojevHCqQhzQ1Pc4%2Bg%3D&reserved=0 https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdevelopercommunity.visualstudio.com%2Ft%2Fyaml-how-to-test-yaml-locally-before-commit%2F1302759&data=04%7C01%7Cbret.barkelew%40microsoft.com%7Ce4e5493578cf45b804c608d8df354b05%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504767278995847%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=gwGZAQJAIXIS0LODFpglrCf5Ua2JeMnNm3A3auCrhx8%3D&reserved=0 -- Rebecca Cran [-- Attachment #2: Type: text/html, Size: 9242 bytes --] ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [EXTERNAL] Re: [edk2-devel] generating the test matrix for local CI runs 2021-03-04 17:56 ` [EXTERNAL] " Bret Barkelew @ 2021-03-04 18:20 ` Bret Barkelew 2021-03-04 18:32 ` Bret Barkelew 2021-03-04 18:45 ` Laszlo Ersek 0 siblings, 2 replies; 7+ messages in thread From: Bret Barkelew @ 2021-03-04 18:20 UTC (permalink / raw) To: devel@edk2.groups.io, rebecca@nuviainc.com, lersek@redhat.com Cc: Sean Brogan, Kinney, Michael D [-- Attachment #1.1: Type: text/plain, Size: 7860 bytes --] Thinking further, it will skip some tests if you haven’t installed the prerequisites (e.g. spell check). We could potentially add a single script to install the prerequisites, but then there are the prereqs for the prereqs (e.g. Node). To more clearly address your question, the “tests” that a GitHub CI is broken up into are arbitrary subdivisions of the tests that run locally (when you run “stuart_ci_build” without parameters) so that the workload can be divided amongst the available worker agents. If you were to compare the individual test logs against your local, monolithic test log, you would find that all cases were covered in both (assuming the aforementioned prereqs are installed). - Bret From: Bret Barkelew via groups.io<mailto:bret.barkelew=microsoft.com@groups.io> Sent: Thursday, March 4, 2021 9:57 AM To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; rebecca@nuviainc.com<mailto:rebecca@nuviainc.com>; lersek@redhat.com<mailto:lersek@redhat.com> Cc: Sean Brogan<mailto:sean.brogan@microsoft.com>; Kinney, Michael D<mailto:michael.d.kinney@intel.com> Subject: Re: [EXTERNAL] Re: [edk2-devel] generating the test matrix for local CI runs *An* answer is: - There’s a DevOps virtual environment that you can run that will pretend to be a build agent. I played with it years ago, but that was before the CI tooling (and it was years ago) so I don’t know how the capabilities compare. My real answer is: - If you run the three commands -- setup, update, and build -- without any parameters other than your TOOL_CHAIN_TAG, it should run all tests on all packages for that particular OS/toolchain. - Bret From: Rebecca Cran via groups.io<mailto:rebecca=nuviainc.com@groups.io> Sent: Thursday, March 4, 2021 9:45 AM To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; lersek@redhat.com<mailto:lersek@redhat.com> Cc: Sean Brogan<mailto:sean.brogan@microsoft.com>; Bret Barkelew<mailto:Bret.Barkelew@microsoft.com>; Kinney, Michael D<mailto:michael.d.kinney@intel.com> Subject: [EXTERNAL] Re: [edk2-devel] generating the test matrix for local CI runs On 3/4/21 10:04 AM, Laszlo Ersek wrote: > (Sigh, my address book pulled one over me -- resending to the correct > list address now. Please ignore the previous posting that you may have > gotten "in private".) > > Hi All, > > while I can run specific CI tests locally, using the commands listed at: > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2F.pytool&data=04%7C01%7Cbret.barkelew%40microsoft.com%7Ce4e5493578cf45b804c608d8df354b05%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504767278995847%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=l8WblhD%2B7dpkFaCZ8roSfNUmj1imytHi5eUT%2F%2BKrLCo%3D&reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2F.pytool&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C385eef1d988f4a56155a08d8df36ecce%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504774296518259%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=UPVktVmDdChYhoc6zErhuz57Dllw21gqcwi9crhUdp8%3D&reserved=0> > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2FArmVirtPkg%2FPlatformCI&data=04%7C01%7Cbret.barkelew%40microsoft.com%7Ce4e5493578cf45b804c608d8df354b05%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504767278995847%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=8yG7Mro9Li4XeCvfmPhfReBbipz47nyAjUmMKMGfh%2Bo%3D&reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2FArmVirtPkg%2FPlatformCI&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C385eef1d988f4a56155a08d8df36ecce%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504774296518259%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=mcoSK9x7FSSJzlI86mdvs%2Btp7zdeF42IJn62mJxxozo%3D&reserved=0> > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2FOvmfPkg%2FPlatformCI&data=04%7C01%7Cbret.barkelew%40microsoft.com%7Ce4e5493578cf45b804c608d8df354b05%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504767278995847%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=NUhAJ8zkITwPkvCrUqb7ZogSQTJuBXv0eL6Au0sYC2M%3D&reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2FOvmfPkg%2FPlatformCI&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C385eef1d988f4a56155a08d8df36ecce%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504774296528219%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=zOv0z287ugh6PmcVk9EMAvxx4RE8U8iHUYemWzO1FaY%3D&reserved=0> > > I don't know how I can determine and drive the whole "test plan" locally. > > When I submit a PR to github, "something" generates (say) 83 test cases. > How can I run that "something" locally? > > Assume I have the following: > - a Linux builder VM, with up-to-date tools, > - a Windows builder VM, with up-to-date tools, > - a topic branch, on top of "master". > > How do I generate and run those same 83 tests, locally? > > It has happened to me that all my local CI steps succeeded (meaning both > the Windows and the Linux builder), but github still rejected the PR -- > the reason was that I failed to think of a particular test that github > would run. Composing 83 "stuart" command lines isn't something I'd like > to do manually. FWIW it looks like people have been looking for similar functionality of testing a pipeline locally, and not found it. e.g.: https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmicrosoft%2Fazure-pipelines-agent%2Fissues%2F1764&data=04%7C01%7Cbret.barkelew%40microsoft.com%7Ce4e5493578cf45b804c608d8df354b05%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504767278995847%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=XMxZesyKxVN9Wg3SyRpUbnnmKzojevHCqQhzQ1Pc4%2Bg%3D&reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmicrosoft%2Fazure-pipelines-agent%2Fissues%2F1764&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C385eef1d988f4a56155a08d8df36ecce%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504774296528219%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=uwpRNBUe4wFq7M0tRXBs8U4O1Qn6T0jf76Acr7n0qu8%3D&reserved=0> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdevelopercommunity.visualstudio.com%2Ft%2Fyaml-how-to-test-yaml-locally-before-commit%2F1302759&data=04%7C01%7Cbret.barkelew%40microsoft.com%7Ce4e5493578cf45b804c608d8df354b05%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504767278995847%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=gwGZAQJAIXIS0LODFpglrCf5Ua2JeMnNm3A3auCrhx8%3D&reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdevelopercommunity.visualstudio.com%2Ft%2Fyaml-how-to-test-yaml-locally-before-commit%2F1302759&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C385eef1d988f4a56155a08d8df36ecce%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504774296538176%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=ISHz%2FhaG88Cg6V%2BMwguTJNLrFwABVeNmOn4mq5meKd8%3D&reserved=0> -- Rebecca Cran [-- Attachment #1.2: Type: text/html, Size: 11230 bytes --] [-- Attachment #2: 61A417DF6E904358AC2AFB5E11FC6A23.png --] [-- Type: image/png, Size: 140 bytes --] ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [EXTERNAL] Re: [edk2-devel] generating the test matrix for local CI runs 2021-03-04 18:20 ` Bret Barkelew @ 2021-03-04 18:32 ` Bret Barkelew 2021-03-04 19:07 ` Laszlo Ersek 2021-03-04 18:45 ` Laszlo Ersek 1 sibling, 1 reply; 7+ messages in thread From: Bret Barkelew @ 2021-03-04 18:32 UTC (permalink / raw) To: devel@edk2.groups.io, rebecca@nuviainc.com, lersek@redhat.com Cc: Sean Brogan, Kinney, Michael D [-- Attachment #1.1: Type: text/plain, Size: 9156 bytes --] Digging in even further, it looks like you’re asking about tests across test pipelines, including some of the platform pipelines. I don’t believe there’s a cohesive way to do that, as some of them require platform-specific parameters (such as the --FlashRom parameter for OvmfPkg). Can you share the specific PR you’re failing so that we can talk in concrete terms? My guess (from experience failing this myself) is that you are failing the spell check remotely and not running it locally (because Node, npm, or cspell are not installed). It’s possible that we should investigate documenting cross-pipeline and cross-package CI dependencies in a central place. Right now it’s distributed (e.g. did you follow the steps here: https://github.com/tianocore/edk2/tree/master/.pytool#spell-checking---cspell). - Bret From: Bret Barkelew via groups.io<mailto:bret.barkelew=microsoft.com@groups.io> Sent: Thursday, March 4, 2021 10:22 AM To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; rebecca@nuviainc.com<mailto:rebecca@nuviainc.com>; lersek@redhat.com<mailto:lersek@redhat.com> Cc: Sean Brogan<mailto:sean.brogan@microsoft.com>; Kinney, Michael D<mailto:michael.d.kinney@intel.com> Subject: Re: [EXTERNAL] Re: [edk2-devel] generating the test matrix for local CI runs Thinking further, it will skip some tests if you haven’t installed the prerequisites (e.g. spell check). We could potentially add a single script to install the prerequisites, but then there are the prereqs for the prereqs (e.g. Node). To more clearly address your question, the “tests” that a GitHub CI is broken up into are arbitrary subdivisions of the tests that run locally (when you run “stuart_ci_build” without parameters) so that the workload can be divided amongst the available worker agents. If you were to compare the individual test logs against your local, monolithic test log, you would find that all cases were covered in both (assuming the aforementioned prereqs are installed). - Bret From: Bret Barkelew via groups.io<mailto:bret.barkelew=microsoft.com@groups.io> Sent: Thursday, March 4, 2021 9:57 AM To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; rebecca@nuviainc.com<mailto:rebecca@nuviainc.com>; lersek@redhat.com<mailto:lersek@redhat.com> Cc: Sean Brogan<mailto:sean.brogan@microsoft.com>; Kinney, Michael D<mailto:michael.d.kinney@intel.com> Subject: Re: [EXTERNAL] Re: [edk2-devel] generating the test matrix for local CI runs *An* answer is: - There’s a DevOps virtual environment that you can run that will pretend to be a build agent. I played with it years ago, but that was before the CI tooling (and it was years ago) so I don’t know how the capabilities compare. My real answer is: - If you run the three commands -- setup, update, and build -- without any parameters other than your TOOL_CHAIN_TAG, it should run all tests on all packages for that particular OS/toolchain. - Bret From: Rebecca Cran via groups.io<mailto:rebecca=nuviainc.com@groups.io> Sent: Thursday, March 4, 2021 9:45 AM To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; lersek@redhat.com<mailto:lersek@redhat.com> Cc: Sean Brogan<mailto:sean.brogan@microsoft.com>; Bret Barkelew<mailto:Bret.Barkelew@microsoft.com>; Kinney, Michael D<mailto:michael.d.kinney@intel.com> Subject: [EXTERNAL] Re: [edk2-devel] generating the test matrix for local CI runs On 3/4/21 10:04 AM, Laszlo Ersek wrote: > (Sigh, my address book pulled one over me -- resending to the correct > list address now. Please ignore the previous posting that you may have > gotten "in private".) > > Hi All, > > while I can run specific CI tests locally, using the commands listed at: > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2F.pytool&data=04%7C01%7Cbret.barkelew%40microsoft.com%7Ce4e5493578cf45b804c608d8df354b05%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504767278995847%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=l8WblhD%2B7dpkFaCZ8roSfNUmj1imytHi5eUT%2F%2BKrLCo%3D&reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2F.pytool&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C17527a8762f346729ca508d8df3a3118%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504789795390940%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=nKugKAbKDAjSZASwvAAnvDTvaev1IezlePOxvURGrF8%3D&reserved=0> > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2FArmVirtPkg%2FPlatformCI&data=04%7C01%7Cbret.barkelew%40microsoft.com%7Ce4e5493578cf45b804c608d8df354b05%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504767278995847%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=8yG7Mro9Li4XeCvfmPhfReBbipz47nyAjUmMKMGfh%2Bo%3D&reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2FArmVirtPkg%2FPlatformCI&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C17527a8762f346729ca508d8df3a3118%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504789795400888%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=xtaTu4SE0yBDaWsr%2BuQ7d8qwY9NySzzjsaKIy5OcLR0%3D&reserved=0> > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2FOvmfPkg%2FPlatformCI&data=04%7C01%7Cbret.barkelew%40microsoft.com%7Ce4e5493578cf45b804c608d8df354b05%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504767278995847%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=NUhAJ8zkITwPkvCrUqb7ZogSQTJuBXv0eL6Au0sYC2M%3D&reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2FOvmfPkg%2FPlatformCI&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C17527a8762f346729ca508d8df3a3118%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504789795410853%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=qazk%2BsYoUUCRlhG66Ixv7zaMWW8cSZ9vJ2bGKcYQkMM%3D&reserved=0> > > I don't know how I can determine and drive the whole "test plan" locally. > > When I submit a PR to github, "something" generates (say) 83 test cases. > How can I run that "something" locally? > > Assume I have the following: > - a Linux builder VM, with up-to-date tools, > - a Windows builder VM, with up-to-date tools, > - a topic branch, on top of "master". > > How do I generate and run those same 83 tests, locally? > > It has happened to me that all my local CI steps succeeded (meaning both > the Windows and the Linux builder), but github still rejected the PR -- > the reason was that I failed to think of a particular test that github > would run. Composing 83 "stuart" command lines isn't something I'd like > to do manually. FWIW it looks like people have been looking for similar functionality of testing a pipeline locally, and not found it. e.g.: https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmicrosoft%2Fazure-pipelines-agent%2Fissues%2F1764&data=04%7C01%7Cbret.barkelew%40microsoft.com%7Ce4e5493578cf45b804c608d8df354b05%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504767278995847%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=XMxZesyKxVN9Wg3SyRpUbnnmKzojevHCqQhzQ1Pc4%2Bg%3D&reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmicrosoft%2Fazure-pipelines-agent%2Fissues%2F1764&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C17527a8762f346729ca508d8df3a3118%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504789795410853%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=zGzaxR6bPunlm3EY4vW%2BATWI9vaJIe74J87kL0yHQng%3D&reserved=0> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdevelopercommunity.visualstudio.com%2Ft%2Fyaml-how-to-test-yaml-locally-before-commit%2F1302759&data=04%7C01%7Cbret.barkelew%40microsoft.com%7Ce4e5493578cf45b804c608d8df354b05%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504767278995847%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=gwGZAQJAIXIS0LODFpglrCf5Ua2JeMnNm3A3auCrhx8%3D&reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdevelopercommunity.visualstudio.com%2Ft%2Fyaml-how-to-test-yaml-locally-before-commit%2F1302759&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C17527a8762f346729ca508d8df3a3118%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504789795420809%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=%2Fwrs35d%2FlFuxjoedyRv2tI%2FWF7h0SsicPfbtgawxIKI%3D&reserved=0> -- Rebecca Cran [-- Attachment #1.2: Type: text/html, Size: 13188 bytes --] [-- Attachment #2: 50944657DF1343288AECE7FF723557B6.png --] [-- Type: image/png, Size: 140 bytes --] ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [EXTERNAL] Re: [edk2-devel] generating the test matrix for local CI runs 2021-03-04 18:32 ` Bret Barkelew @ 2021-03-04 19:07 ` Laszlo Ersek 0 siblings, 0 replies; 7+ messages in thread From: Laszlo Ersek @ 2021-03-04 19:07 UTC (permalink / raw) To: Bret Barkelew, devel@edk2.groups.io, rebecca@nuviainc.com Cc: Sean Brogan, Kinney, Michael D On 03/04/21 19:32, Bret Barkelew wrote: > Can you share the specific PR you\x19re failing so that we can talk in > concrete terms? Sure. In January, I posted the following patch series: [edk2-devel] [PATCH 0/8] ShellPkg, ArmVirtPkg, OvmfPkg: shell usability improvements https://edk2.groups.io/g/devel/message/69590 https://www.redhat.com/archives/edk2-devel-archive/2021-January/msg00070.html Message-Id: 20210104154235.31785-1-lersek@redhat.com This series cleared my local CI just fine (both Windows and Linux VM), before I posted it. When review was done, I attempted to merge the series: https://github.com/tianocore/edk2/pull/1336 Although the PR does not show it now (or at least I cannot make it show it), the github CI uncovered a real problem in my patch set, one that my local run missed. Namely, I had added an OrderedCollectionLib class dependency to the (singleton) UefiShellCommandLib instance, but failed to resolve the OrderedCollectionLib class in the EmulatorPkg and UefiPayloadPkg DSC files. Consequently, those DSC files would no longer build. My local CI runs didn't extend to these packages, but -- thankfully -- the github.com-based CI did. In v2, I added one extra patch for each of EmulatorPkg and UefiPayloadPkg DSC: [edk2-devel] [PATCH v2 00/10] multiple packages: shell usability improvements https://edk2.groups.io/g/devel/message/70216 https://www.redhat.com/archives/edk2-devel-archive/2021-January/msg00700.html Message-Id: 20210113085453.10168-1-lersek@redhat.com (What's more, I tracked down the DSC files in edk2-platforms that used to build the shell, and I resolved the OrderedCollectionLib class in them as well -- see the links inside the above-referenced blurbs.) The v2 CI succeded on github (personal, then merge): https://github.com/tianocore/edk2/pull/1337 https://github.com/tianocore/edk2/pull/1370 In summary, the bug (in my local CI usage) was that I didn't cover EmulatorPkg and UefiPayloadPkg. So, I'd like if the local test builds determined the affected set of packages as well -- I'd like to be *unable* to miss EmulatorPkg and UefiPayloadPkg in the above type of situation. Thanks! Laszlo ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [EXTERNAL] Re: [edk2-devel] generating the test matrix for local CI runs 2021-03-04 18:20 ` Bret Barkelew 2021-03-04 18:32 ` Bret Barkelew @ 2021-03-04 18:45 ` Laszlo Ersek 1 sibling, 0 replies; 7+ messages in thread From: Laszlo Ersek @ 2021-03-04 18:45 UTC (permalink / raw) To: Bret Barkelew, devel@edk2.groups.io, rebecca@nuviainc.com Cc: Sean Brogan, Kinney, Michael D On 03/04/21 19:20, Bret Barkelew wrote: > Thinking further, it will skip some tests if you haven’t installed the prerequisites (e.g. spell check). We could potentially add a single script to install the prerequisites, but then there are the prereqs for the prereqs (e.g. Node). > > To more clearly address your question, the “tests” that a GitHub CI is broken up into are arbitrary subdivisions of the tests that run locally (when you run “stuart_ci_build” without parameters) so that the workload can be divided amongst the available worker agents. If you were to compare the individual test logs against your local, monolithic test log, you would find that all cases were covered in both (assuming the aforementioned prereqs are installed). Thank you -- I hope to test this later. (1) A question about deps: if a dep is missing, can I force an error (rather than a silent skip)? Again, I distinctly remember doing what I then considered "due diligence" locally, only to see github reject the PR later. If that was because some deps were missing locally, I'd really like to turn that scenario into a local error. A script for installing all dependencies would help of course, but not as a *replacement* for this kind of deps enforcement -- instead, only for solving the missing dependency problems. I'd still want the local CI to break hard if it couldn't offer the exact coverage as github.com. Another question below: > > - Bret > > From: Bret Barkelew via groups.io<mailto:bret.barkelew=microsoft.com@groups.io> > Sent: Thursday, March 4, 2021 9:57 AM > To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; rebecca@nuviainc.com<mailto:rebecca@nuviainc.com>; lersek@redhat.com<mailto:lersek@redhat.com> > Cc: Sean Brogan<mailto:sean.brogan@microsoft.com>; Kinney, Michael D<mailto:michael.d.kinney@intel.com> > Subject: Re: [EXTERNAL] Re: [edk2-devel] generating the test matrix for local CI runs > > *An* answer is: > - There’s a DevOps virtual environment that you can run that will pretend to be a build agent. I played with it years ago, but that was before the CI tooling (and it was years ago) so I don’t know how the capabilities compare. > > My real answer is: > - If you run the three commands -- setup, update, and build -- without any parameters other than your TOOL_CHAIN_TAG, it should run all tests on all packages for that particular OS/toolchain. (2) You mention "for that particular OS/toolchain". How can I make sure that I'm using the exact same toolchain as github would? I'd like to tell the tooling only the OS (Windows vs. Linux), and then the toolchain should be auto-selected, same way as it is auto-selected on github. For example, if I'm in my Windows 10 VM, then I should be *unable* to request anything else than the latest VS toolchain; similarly, in the Linux VM, I should be *unable* to ask for (e.g.) the GCC49 toolchain; only the GCC50 toolchain should be usable. When I submit a github PR, I'm not asked to pick toolchains for CI; can we automate the toolchain selection locally too? (Sorry if I misunderstood your comment above -- if you are saying that it's permitted, but *not required*, to specify TOOL_CHAIN_TAG, then my question (2) falls away.) Thanks! Laszlo > > - Bret > > From: Rebecca Cran via groups.io<mailto:rebecca=nuviainc.com@groups.io> > Sent: Thursday, March 4, 2021 9:45 AM > To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; lersek@redhat.com<mailto:lersek@redhat.com> > Cc: Sean Brogan<mailto:sean.brogan@microsoft.com>; Bret Barkelew<mailto:Bret.Barkelew@microsoft.com>; Kinney, Michael D<mailto:michael.d.kinney@intel.com> > Subject: [EXTERNAL] Re: [edk2-devel] generating the test matrix for local CI runs > > On 3/4/21 10:04 AM, Laszlo Ersek wrote: >> (Sigh, my address book pulled one over me -- resending to the correct >> list address now. Please ignore the previous posting that you may have >> gotten "in private".) >> >> Hi All, >> >> while I can run specific CI tests locally, using the commands listed at: >> >> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2F.pytool&data=04%7C01%7Cbret.barkelew%40microsoft.com%7Ce4e5493578cf45b804c608d8df354b05%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504767278995847%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=l8WblhD%2B7dpkFaCZ8roSfNUmj1imytHi5eUT%2F%2BKrLCo%3D&reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2F.pytool&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C385eef1d988f4a56155a08d8df36ecce%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504774296518259%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=UPVktVmDdChYhoc6zErhuz57Dllw21gqcwi9crhUdp8%3D&reserved=0> >> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2FArmVirtPkg%2FPlatformCI&data=04%7C01%7Cbret.barkelew%40microsoft.com%7Ce4e5493578cf45b804c608d8df354b05%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504767278995847%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=8yG7Mro9Li4XeCvfmPhfReBbipz47nyAjUmMKMGfh%2Bo%3D&reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2FArmVirtPkg%2FPlatformCI&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C385eef1d988f4a56155a08d8df36ecce%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504774296518259%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=mcoSK9x7FSSJzlI86mdvs%2Btp7zdeF42IJn62mJxxozo%3D&reserved=0> >> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2FOvmfPkg%2FPlatformCI&data=04%7C01%7Cbret.barkelew%40microsoft.com%7Ce4e5493578cf45b804c608d8df354b05%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504767278995847%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=NUhAJ8zkITwPkvCrUqb7ZogSQTJuBXv0eL6Au0sYC2M%3D&reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2FOvmfPkg%2FPlatformCI&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C385eef1d988f4a56155a08d8df36ecce%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504774296528219%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=zOv0z287ugh6PmcVk9EMAvxx4RE8U8iHUYemWzO1FaY%3D&reserved=0> >> >> I don't know how I can determine and drive the whole "test plan" locally. >> >> When I submit a PR to github, "something" generates (say) 83 test cases. >> How can I run that "something" locally? >> >> Assume I have the following: >> - a Linux builder VM, with up-to-date tools, >> - a Windows builder VM, with up-to-date tools, >> - a topic branch, on top of "master". >> >> How do I generate and run those same 83 tests, locally? >> >> It has happened to me that all my local CI steps succeeded (meaning both >> the Windows and the Linux builder), but github still rejected the PR -- >> the reason was that I failed to think of a particular test that github >> would run. Composing 83 "stuart" command lines isn't something I'd like >> to do manually. > > FWIW it looks like people have been looking for similar functionality of > testing a pipeline locally, and not found it. > > e.g.: > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmicrosoft%2Fazure-pipelines-agent%2Fissues%2F1764&data=04%7C01%7Cbret.barkelew%40microsoft.com%7Ce4e5493578cf45b804c608d8df354b05%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504767278995847%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=XMxZesyKxVN9Wg3SyRpUbnnmKzojevHCqQhzQ1Pc4%2Bg%3D&reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmicrosoft%2Fazure-pipelines-agent%2Fissues%2F1764&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C385eef1d988f4a56155a08d8df36ecce%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504774296528219%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=uwpRNBUe4wFq7M0tRXBs8U4O1Qn6T0jf76Acr7n0qu8%3D&reserved=0> > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdevelopercommunity.visualstudio.com%2Ft%2Fyaml-how-to-test-yaml-locally-before-commit%2F1302759&data=04%7C01%7Cbret.barkelew%40microsoft.com%7Ce4e5493578cf45b804c608d8df354b05%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504767278995847%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=gwGZAQJAIXIS0LODFpglrCf5Ua2JeMnNm3A3auCrhx8%3D&reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdevelopercommunity.visualstudio.com%2Ft%2Fyaml-how-to-test-yaml-locally-before-commit%2F1302759&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C385eef1d988f4a56155a08d8df36ecce%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504774296538176%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=ISHz%2FhaG88Cg6V%2BMwguTJNLrFwABVeNmOn4mq5meKd8%3D&reserved=0> > > -- > Rebecca Cran > > > > > > > ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2021-03-04 19:07 UTC | newest] Thread overview: 7+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2021-03-04 17:04 generating the test matrix for local CI runs Laszlo Ersek 2021-03-04 17:45 ` [edk2-devel] " Rebecca Cran 2021-03-04 17:56 ` [EXTERNAL] " Bret Barkelew 2021-03-04 18:20 ` Bret Barkelew 2021-03-04 18:32 ` Bret Barkelew 2021-03-04 19:07 ` Laszlo Ersek 2021-03-04 18:45 ` Laszlo Ersek
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox
![[-- Attachment #2: 61A417DF6E904358AC2AFB5E11FC6A23.png --] [-- Type: image/png, Size: 140 bytes --]](../../MW4PR21MB1907345A3D58A427F10EBB8CEF979@MW4PR21MB1907.namprd21.prod.outlook.com/2-61A417DF6E904358AC2AFB5E11FC6A23.png){kind=link}
![[-- Attachment #2: 50944657DF1343288AECE7FF723557B6.png --] [-- Type: image/png, Size: 140 bytes --]](../../MW4PR21MB1907060C63A0C85D77BFD537EF979@MW4PR21MB1907.namprd21.prod.outlook.com/2-50944657DF1343288AECE7FF723557B6.png){kind=link}