* generating the test matrix for local CI runs
@ 2021-03-04 17:04 Laszlo Ersek
2021-03-04 17:45 ` [edk2-devel] " Rebecca Cran
0 siblings, 1 reply; 7+ messages in thread
From: Laszlo Ersek @ 2021-03-04 17:04 UTC (permalink / raw)
To: edk2-devel-groups-io; +Cc: Sean Brogan, Bret Barkelew, Michael Kinney
(Sigh, my address book pulled one over me -- resending to the correct
list address now. Please ignore the previous posting that you may have
gotten "in private".)
Hi All,
while I can run specific CI tests locally, using the commands listed at:
https://github.com/tianocore/edk2/tree/master/.pytool
https://github.com/tianocore/edk2/tree/master/ArmVirtPkg/PlatformCI
https://github.com/tianocore/edk2/tree/master/OvmfPkg/PlatformCI
I don't know how I can determine and drive the whole "test plan" locally.
When I submit a PR to github, "something" generates (say) 83 test cases.
How can I run that "something" locally?
Assume I have the following:
- a Linux builder VM, with up-to-date tools,
- a Windows builder VM, with up-to-date tools,
- a topic branch, on top of "master".
How do I generate and run those same 83 tests, locally?
It has happened to me that all my local CI steps succeeded (meaning both
the Windows and the Linux builder), but github still rejected the PR --
the reason was that I failed to think of a particular test that github
would run. Composing 83 "stuart" command lines isn't something I'd like
to do manually.
Thanks
Laszlo
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [edk2-devel] generating the test matrix for local CI runs
2021-03-04 17:04 generating the test matrix for local CI runs Laszlo Ersek
@ 2021-03-04 17:45 ` Rebecca Cran
2021-03-04 17:56 ` [EXTERNAL] " Bret Barkelew
0 siblings, 1 reply; 7+ messages in thread
From: Rebecca Cran @ 2021-03-04 17:45 UTC (permalink / raw)
To: devel, lersek; +Cc: Sean Brogan, Bret Barkelew, Michael Kinney
On 3/4/21 10:04 AM, Laszlo Ersek wrote:
> (Sigh, my address book pulled one over me -- resending to the correct
> list address now. Please ignore the previous posting that you may have
> gotten "in private".)
>
> Hi All,
>
> while I can run specific CI tests locally, using the commands listed at:
>
> https://github.com/tianocore/edk2/tree/master/.pytool
> https://github.com/tianocore/edk2/tree/master/ArmVirtPkg/PlatformCI
> https://github.com/tianocore/edk2/tree/master/OvmfPkg/PlatformCI
>
> I don't know how I can determine and drive the whole "test plan" locally.
>
> When I submit a PR to github, "something" generates (say) 83 test cases.
> How can I run that "something" locally?
>
> Assume I have the following:
> - a Linux builder VM, with up-to-date tools,
> - a Windows builder VM, with up-to-date tools,
> - a topic branch, on top of "master".
>
> How do I generate and run those same 83 tests, locally?
>
> It has happened to me that all my local CI steps succeeded (meaning both
> the Windows and the Linux builder), but github still rejected the PR --
> the reason was that I failed to think of a particular test that github
> would run. Composing 83 "stuart" command lines isn't something I'd like
> to do manually.
FWIW it looks like people have been looking for similar functionality of
testing a pipeline locally, and not found it.
e.g.:
https://github.com/microsoft/azure-pipelines-agent/issues/1764
https://developercommunity.visualstudio.com/t/yaml-how-to-test-yaml-locally-before-commit/1302759
--
Rebecca Cran
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [EXTERNAL] Re: [edk2-devel] generating the test matrix for local CI runs
2021-03-04 17:45 ` [edk2-devel] " Rebecca Cran
@ 2021-03-04 17:56 ` Bret Barkelew
2021-03-04 18:20 ` Bret Barkelew
0 siblings, 1 reply; 7+ messages in thread
From: Bret Barkelew @ 2021-03-04 17:56 UTC (permalink / raw)
To: devel@edk2.groups.io, rebecca@nuviainc.com, lersek@redhat.com
Cc: Sean Brogan, Kinney, Michael D
[-- Attachment #1: Type: text/plain, Size: 4453 bytes --]
*An* answer is:
- There’s a DevOps virtual environment that you can run that will pretend to be a build agent. I played with it years ago, but that was before the CI tooling (and it was years ago) so I don’t know how the capabilities compare.
My real answer is:
- If you run the three commands -- setup, update, and build -- without any parameters other than your TOOL_CHAIN_TAG, it should run all tests on all packages for that particular OS/toolchain.
- Bret
From: Rebecca Cran via groups.io<mailto:rebecca=nuviainc.com@groups.io>
Sent: Thursday, March 4, 2021 9:45 AM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; lersek@redhat.com<mailto:lersek@redhat.com>
Cc: Sean Brogan<mailto:sean.brogan@microsoft.com>; Bret Barkelew<mailto:Bret.Barkelew@microsoft.com>; Kinney, Michael D<mailto:michael.d.kinney@intel.com>
Subject: [EXTERNAL] Re: [edk2-devel] generating the test matrix for local CI runs
On 3/4/21 10:04 AM, Laszlo Ersek wrote:
> (Sigh, my address book pulled one over me -- resending to the correct
> list address now. Please ignore the previous posting that you may have
> gotten "in private".)
>
> Hi All,
>
> while I can run specific CI tests locally, using the commands listed at:
>
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2F.pytool&data=04%7C01%7Cbret.barkelew%40microsoft.com%7Ce4e5493578cf45b804c608d8df354b05%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504767278995847%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=l8WblhD%2B7dpkFaCZ8roSfNUmj1imytHi5eUT%2F%2BKrLCo%3D&reserved=0
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2FArmVirtPkg%2FPlatformCI&data=04%7C01%7Cbret.barkelew%40microsoft.com%7Ce4e5493578cf45b804c608d8df354b05%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504767278995847%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=8yG7Mro9Li4XeCvfmPhfReBbipz47nyAjUmMKMGfh%2Bo%3D&reserved=0
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2FOvmfPkg%2FPlatformCI&data=04%7C01%7Cbret.barkelew%40microsoft.com%7Ce4e5493578cf45b804c608d8df354b05%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504767278995847%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=NUhAJ8zkITwPkvCrUqb7ZogSQTJuBXv0eL6Au0sYC2M%3D&reserved=0
>
> I don't know how I can determine and drive the whole "test plan" locally.
>
> When I submit a PR to github, "something" generates (say) 83 test cases.
> How can I run that "something" locally?
>
> Assume I have the following:
> - a Linux builder VM, with up-to-date tools,
> - a Windows builder VM, with up-to-date tools,
> - a topic branch, on top of "master".
>
> How do I generate and run those same 83 tests, locally?
>
> It has happened to me that all my local CI steps succeeded (meaning both
> the Windows and the Linux builder), but github still rejected the PR --
> the reason was that I failed to think of a particular test that github
> would run. Composing 83 "stuart" command lines isn't something I'd like
> to do manually.
FWIW it looks like people have been looking for similar functionality of
testing a pipeline locally, and not found it.
e.g.:
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmicrosoft%2Fazure-pipelines-agent%2Fissues%2F1764&data=04%7C01%7Cbret.barkelew%40microsoft.com%7Ce4e5493578cf45b804c608d8df354b05%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504767278995847%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=XMxZesyKxVN9Wg3SyRpUbnnmKzojevHCqQhzQ1Pc4%2Bg%3D&reserved=0
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdevelopercommunity.visualstudio.com%2Ft%2Fyaml-how-to-test-yaml-locally-before-commit%2F1302759&data=04%7C01%7Cbret.barkelew%40microsoft.com%7Ce4e5493578cf45b804c608d8df354b05%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504767278995847%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=gwGZAQJAIXIS0LODFpglrCf5Ua2JeMnNm3A3auCrhx8%3D&reserved=0
--
Rebecca Cran
[-- Attachment #2: Type: text/html, Size: 9242 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [EXTERNAL] Re: [edk2-devel] generating the test matrix for local CI runs
2021-03-04 17:56 ` [EXTERNAL] " Bret Barkelew
@ 2021-03-04 18:20 ` Bret Barkelew
2021-03-04 18:32 ` Bret Barkelew
2021-03-04 18:45 ` Laszlo Ersek
0 siblings, 2 replies; 7+ messages in thread
From: Bret Barkelew @ 2021-03-04 18:20 UTC (permalink / raw)
To: devel@edk2.groups.io, rebecca@nuviainc.com, lersek@redhat.com
Cc: Sean Brogan, Kinney, Michael D
[-- Attachment #1.1: Type: text/plain, Size: 7860 bytes --]
Thinking further, it will skip some tests if you haven’t installed the prerequisites (e.g. spell check). We could potentially add a single script to install the prerequisites, but then there are the prereqs for the prereqs (e.g. Node).
To more clearly address your question, the “tests” that a GitHub CI is broken up into are arbitrary subdivisions of the tests that run locally (when you run “stuart_ci_build” without parameters) so that the workload can be divided amongst the available worker agents. If you were to compare the individual test logs against your local, monolithic test log, you would find that all cases were covered in both (assuming the aforementioned prereqs are installed).
- Bret
From: Bret Barkelew via groups.io<mailto:bret.barkelew=microsoft.com@groups.io>
Sent: Thursday, March 4, 2021 9:57 AM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; rebecca@nuviainc.com<mailto:rebecca@nuviainc.com>; lersek@redhat.com<mailto:lersek@redhat.com>
Cc: Sean Brogan<mailto:sean.brogan@microsoft.com>; Kinney, Michael D<mailto:michael.d.kinney@intel.com>
Subject: Re: [EXTERNAL] Re: [edk2-devel] generating the test matrix for local CI runs
*An* answer is:
- There’s a DevOps virtual environment that you can run that will pretend to be a build agent. I played with it years ago, but that was before the CI tooling (and it was years ago) so I don’t know how the capabilities compare.
My real answer is:
- If you run the three commands -- setup, update, and build -- without any parameters other than your TOOL_CHAIN_TAG, it should run all tests on all packages for that particular OS/toolchain.
- Bret
From: Rebecca Cran via groups.io<mailto:rebecca=nuviainc.com@groups.io>
Sent: Thursday, March 4, 2021 9:45 AM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; lersek@redhat.com<mailto:lersek@redhat.com>
Cc: Sean Brogan<mailto:sean.brogan@microsoft.com>; Bret Barkelew<mailto:Bret.Barkelew@microsoft.com>; Kinney, Michael D<mailto:michael.d.kinney@intel.com>
Subject: [EXTERNAL] Re: [edk2-devel] generating the test matrix for local CI runs
On 3/4/21 10:04 AM, Laszlo Ersek wrote:
> (Sigh, my address book pulled one over me -- resending to the correct
> list address now. Please ignore the previous posting that you may have
> gotten "in private".)
>
> Hi All,
>
> while I can run specific CI tests locally, using the commands listed at:
>
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2F.pytool&data=04%7C01%7Cbret.barkelew%40microsoft.com%7Ce4e5493578cf45b804c608d8df354b05%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504767278995847%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=l8WblhD%2B7dpkFaCZ8roSfNUmj1imytHi5eUT%2F%2BKrLCo%3D&reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2F.pytool&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C385eef1d988f4a56155a08d8df36ecce%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504774296518259%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=UPVktVmDdChYhoc6zErhuz57Dllw21gqcwi9crhUdp8%3D&reserved=0>
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2FArmVirtPkg%2FPlatformCI&data=04%7C01%7Cbret.barkelew%40microsoft.com%7Ce4e5493578cf45b804c608d8df354b05%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504767278995847%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=8yG7Mro9Li4XeCvfmPhfReBbipz47nyAjUmMKMGfh%2Bo%3D&reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2FArmVirtPkg%2FPlatformCI&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C385eef1d988f4a56155a08d8df36ecce%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504774296518259%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=mcoSK9x7FSSJzlI86mdvs%2Btp7zdeF42IJn62mJxxozo%3D&reserved=0>
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2FOvmfPkg%2FPlatformCI&data=04%7C01%7Cbret.barkelew%40microsoft.com%7Ce4e5493578cf45b804c608d8df354b05%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504767278995847%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=NUhAJ8zkITwPkvCrUqb7ZogSQTJuBXv0eL6Au0sYC2M%3D&reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2FOvmfPkg%2FPlatformCI&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C385eef1d988f4a56155a08d8df36ecce%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504774296528219%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=zOv0z287ugh6PmcVk9EMAvxx4RE8U8iHUYemWzO1FaY%3D&reserved=0>
>
> I don't know how I can determine and drive the whole "test plan" locally.
>
> When I submit a PR to github, "something" generates (say) 83 test cases.
> How can I run that "something" locally?
>
> Assume I have the following:
> - a Linux builder VM, with up-to-date tools,
> - a Windows builder VM, with up-to-date tools,
> - a topic branch, on top of "master".
>
> How do I generate and run those same 83 tests, locally?
>
> It has happened to me that all my local CI steps succeeded (meaning both
> the Windows and the Linux builder), but github still rejected the PR --
> the reason was that I failed to think of a particular test that github
> would run. Composing 83 "stuart" command lines isn't something I'd like
> to do manually.
FWIW it looks like people have been looking for similar functionality of
testing a pipeline locally, and not found it.
e.g.:
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmicrosoft%2Fazure-pipelines-agent%2Fissues%2F1764&data=04%7C01%7Cbret.barkelew%40microsoft.com%7Ce4e5493578cf45b804c608d8df354b05%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504767278995847%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=XMxZesyKxVN9Wg3SyRpUbnnmKzojevHCqQhzQ1Pc4%2Bg%3D&reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmicrosoft%2Fazure-pipelines-agent%2Fissues%2F1764&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C385eef1d988f4a56155a08d8df36ecce%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504774296528219%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=uwpRNBUe4wFq7M0tRXBs8U4O1Qn6T0jf76Acr7n0qu8%3D&reserved=0>
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdevelopercommunity.visualstudio.com%2Ft%2Fyaml-how-to-test-yaml-locally-before-commit%2F1302759&data=04%7C01%7Cbret.barkelew%40microsoft.com%7Ce4e5493578cf45b804c608d8df354b05%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504767278995847%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=gwGZAQJAIXIS0LODFpglrCf5Ua2JeMnNm3A3auCrhx8%3D&reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdevelopercommunity.visualstudio.com%2Ft%2Fyaml-how-to-test-yaml-locally-before-commit%2F1302759&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C385eef1d988f4a56155a08d8df36ecce%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504774296538176%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=ISHz%2FhaG88Cg6V%2BMwguTJNLrFwABVeNmOn4mq5meKd8%3D&reserved=0>
--
Rebecca Cran
[-- Attachment #1.2: Type: text/html, Size: 11230 bytes --]
[-- Attachment #2: 61A417DF6E904358AC2AFB5E11FC6A23.png --]
[-- Type: image/png, Size: 140 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [EXTERNAL] Re: [edk2-devel] generating the test matrix for local CI runs
2021-03-04 18:20 ` Bret Barkelew
@ 2021-03-04 18:32 ` Bret Barkelew
2021-03-04 19:07 ` Laszlo Ersek
2021-03-04 18:45 ` Laszlo Ersek
1 sibling, 1 reply; 7+ messages in thread
From: Bret Barkelew @ 2021-03-04 18:32 UTC (permalink / raw)
To: devel@edk2.groups.io, rebecca@nuviainc.com, lersek@redhat.com
Cc: Sean Brogan, Kinney, Michael D
[-- Attachment #1.1: Type: text/plain, Size: 9156 bytes --]
Digging in even further, it looks like you’re asking about tests across test pipelines, including some of the platform pipelines. I don’t believe there’s a cohesive way to do that, as some of them require platform-specific parameters (such as the --FlashRom parameter for OvmfPkg).
Can you share the specific PR you’re failing so that we can talk in concrete terms?
My guess (from experience failing this myself) is that you are failing the spell check remotely and not running it locally (because Node, npm, or cspell are not installed). It’s possible that we should investigate documenting cross-pipeline and cross-package CI dependencies in a central place. Right now it’s distributed (e.g. did you follow the steps here: https://github.com/tianocore/edk2/tree/master/.pytool#spell-checking---cspell).
- Bret
From: Bret Barkelew via groups.io<mailto:bret.barkelew=microsoft.com@groups.io>
Sent: Thursday, March 4, 2021 10:22 AM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; rebecca@nuviainc.com<mailto:rebecca@nuviainc.com>; lersek@redhat.com<mailto:lersek@redhat.com>
Cc: Sean Brogan<mailto:sean.brogan@microsoft.com>; Kinney, Michael D<mailto:michael.d.kinney@intel.com>
Subject: Re: [EXTERNAL] Re: [edk2-devel] generating the test matrix for local CI runs
Thinking further, it will skip some tests if you haven’t installed the prerequisites (e.g. spell check). We could potentially add a single script to install the prerequisites, but then there are the prereqs for the prereqs (e.g. Node).
To more clearly address your question, the “tests” that a GitHub CI is broken up into are arbitrary subdivisions of the tests that run locally (when you run “stuart_ci_build” without parameters) so that the workload can be divided amongst the available worker agents. If you were to compare the individual test logs against your local, monolithic test log, you would find that all cases were covered in both (assuming the aforementioned prereqs are installed).
- Bret
From: Bret Barkelew via groups.io<mailto:bret.barkelew=microsoft.com@groups.io>
Sent: Thursday, March 4, 2021 9:57 AM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; rebecca@nuviainc.com<mailto:rebecca@nuviainc.com>; lersek@redhat.com<mailto:lersek@redhat.com>
Cc: Sean Brogan<mailto:sean.brogan@microsoft.com>; Kinney, Michael D<mailto:michael.d.kinney@intel.com>
Subject: Re: [EXTERNAL] Re: [edk2-devel] generating the test matrix for local CI runs
*An* answer is:
- There’s a DevOps virtual environment that you can run that will pretend to be a build agent. I played with it years ago, but that was before the CI tooling (and it was years ago) so I don’t know how the capabilities compare.
My real answer is:
- If you run the three commands -- setup, update, and build -- without any parameters other than your TOOL_CHAIN_TAG, it should run all tests on all packages for that particular OS/toolchain.
- Bret
From: Rebecca Cran via groups.io<mailto:rebecca=nuviainc.com@groups.io>
Sent: Thursday, March 4, 2021 9:45 AM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; lersek@redhat.com<mailto:lersek@redhat.com>
Cc: Sean Brogan<mailto:sean.brogan@microsoft.com>; Bret Barkelew<mailto:Bret.Barkelew@microsoft.com>; Kinney, Michael D<mailto:michael.d.kinney@intel.com>
Subject: [EXTERNAL] Re: [edk2-devel] generating the test matrix for local CI runs
On 3/4/21 10:04 AM, Laszlo Ersek wrote:
> (Sigh, my address book pulled one over me -- resending to the correct
> list address now. Please ignore the previous posting that you may have
> gotten "in private".)
>
> Hi All,
>
> while I can run specific CI tests locally, using the commands listed at:
>
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2F.pytool&data=04%7C01%7Cbret.barkelew%40microsoft.com%7Ce4e5493578cf45b804c608d8df354b05%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504767278995847%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=l8WblhD%2B7dpkFaCZ8roSfNUmj1imytHi5eUT%2F%2BKrLCo%3D&reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2F.pytool&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C17527a8762f346729ca508d8df3a3118%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504789795390940%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=nKugKAbKDAjSZASwvAAnvDTvaev1IezlePOxvURGrF8%3D&reserved=0>
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2FArmVirtPkg%2FPlatformCI&data=04%7C01%7Cbret.barkelew%40microsoft.com%7Ce4e5493578cf45b804c608d8df354b05%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504767278995847%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=8yG7Mro9Li4XeCvfmPhfReBbipz47nyAjUmMKMGfh%2Bo%3D&reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2FArmVirtPkg%2FPlatformCI&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C17527a8762f346729ca508d8df3a3118%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504789795400888%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=xtaTu4SE0yBDaWsr%2BuQ7d8qwY9NySzzjsaKIy5OcLR0%3D&reserved=0>
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2FOvmfPkg%2FPlatformCI&data=04%7C01%7Cbret.barkelew%40microsoft.com%7Ce4e5493578cf45b804c608d8df354b05%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504767278995847%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=NUhAJ8zkITwPkvCrUqb7ZogSQTJuBXv0eL6Au0sYC2M%3D&reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2FOvmfPkg%2FPlatformCI&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C17527a8762f346729ca508d8df3a3118%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504789795410853%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=qazk%2BsYoUUCRlhG66Ixv7zaMWW8cSZ9vJ2bGKcYQkMM%3D&reserved=0>
>
> I don't know how I can determine and drive the whole "test plan" locally.
>
> When I submit a PR to github, "something" generates (say) 83 test cases.
> How can I run that "something" locally?
>
> Assume I have the following:
> - a Linux builder VM, with up-to-date tools,
> - a Windows builder VM, with up-to-date tools,
> - a topic branch, on top of "master".
>
> How do I generate and run those same 83 tests, locally?
>
> It has happened to me that all my local CI steps succeeded (meaning both
> the Windows and the Linux builder), but github still rejected the PR --
> the reason was that I failed to think of a particular test that github
> would run. Composing 83 "stuart" command lines isn't something I'd like
> to do manually.
FWIW it looks like people have been looking for similar functionality of
testing a pipeline locally, and not found it.
e.g.:
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmicrosoft%2Fazure-pipelines-agent%2Fissues%2F1764&data=04%7C01%7Cbret.barkelew%40microsoft.com%7Ce4e5493578cf45b804c608d8df354b05%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504767278995847%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=XMxZesyKxVN9Wg3SyRpUbnnmKzojevHCqQhzQ1Pc4%2Bg%3D&reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmicrosoft%2Fazure-pipelines-agent%2Fissues%2F1764&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C17527a8762f346729ca508d8df3a3118%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504789795410853%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=zGzaxR6bPunlm3EY4vW%2BATWI9vaJIe74J87kL0yHQng%3D&reserved=0>
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdevelopercommunity.visualstudio.com%2Ft%2Fyaml-how-to-test-yaml-locally-before-commit%2F1302759&data=04%7C01%7Cbret.barkelew%40microsoft.com%7Ce4e5493578cf45b804c608d8df354b05%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504767278995847%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=gwGZAQJAIXIS0LODFpglrCf5Ua2JeMnNm3A3auCrhx8%3D&reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdevelopercommunity.visualstudio.com%2Ft%2Fyaml-how-to-test-yaml-locally-before-commit%2F1302759&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C17527a8762f346729ca508d8df3a3118%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504789795420809%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=%2Fwrs35d%2FlFuxjoedyRv2tI%2FWF7h0SsicPfbtgawxIKI%3D&reserved=0>
--
Rebecca Cran
[-- Attachment #1.2: Type: text/html, Size: 13188 bytes --]
[-- Attachment #2: 50944657DF1343288AECE7FF723557B6.png --]
[-- Type: image/png, Size: 140 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [EXTERNAL] Re: [edk2-devel] generating the test matrix for local CI runs
2021-03-04 18:20 ` Bret Barkelew
2021-03-04 18:32 ` Bret Barkelew
@ 2021-03-04 18:45 ` Laszlo Ersek
1 sibling, 0 replies; 7+ messages in thread
From: Laszlo Ersek @ 2021-03-04 18:45 UTC (permalink / raw)
To: Bret Barkelew, devel@edk2.groups.io, rebecca@nuviainc.com
Cc: Sean Brogan, Kinney, Michael D
On 03/04/21 19:20, Bret Barkelew wrote:
> Thinking further, it will skip some tests if you haven’t installed the prerequisites (e.g. spell check). We could potentially add a single script to install the prerequisites, but then there are the prereqs for the prereqs (e.g. Node).
>
> To more clearly address your question, the “tests” that a GitHub CI is broken up into are arbitrary subdivisions of the tests that run locally (when you run “stuart_ci_build” without parameters) so that the workload can be divided amongst the available worker agents. If you were to compare the individual test logs against your local, monolithic test log, you would find that all cases were covered in both (assuming the aforementioned prereqs are installed).
Thank you -- I hope to test this later.
(1) A question about deps: if a dep is missing, can I force an error
(rather than a silent skip)?
Again, I distinctly remember doing what I then considered "due
diligence" locally, only to see github reject the PR later. If that was
because some deps were missing locally, I'd really like to turn that
scenario into a local error.
A script for installing all dependencies would help of course, but not
as a *replacement* for this kind of deps enforcement -- instead, only
for solving the missing dependency problems. I'd still want the local CI
to break hard if it couldn't offer the exact coverage as github.com.
Another question below:
>
> - Bret
>
> From: Bret Barkelew via groups.io<mailto:bret.barkelew=microsoft.com@groups.io>
> Sent: Thursday, March 4, 2021 9:57 AM
> To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; rebecca@nuviainc.com<mailto:rebecca@nuviainc.com>; lersek@redhat.com<mailto:lersek@redhat.com>
> Cc: Sean Brogan<mailto:sean.brogan@microsoft.com>; Kinney, Michael D<mailto:michael.d.kinney@intel.com>
> Subject: Re: [EXTERNAL] Re: [edk2-devel] generating the test matrix for local CI runs
>
> *An* answer is:
> - There’s a DevOps virtual environment that you can run that will pretend to be a build agent. I played with it years ago, but that was before the CI tooling (and it was years ago) so I don’t know how the capabilities compare.
>
> My real answer is:
> - If you run the three commands -- setup, update, and build -- without any parameters other than your TOOL_CHAIN_TAG, it should run all tests on all packages for that particular OS/toolchain.
(2) You mention "for that particular OS/toolchain".
How can I make sure that I'm using the exact same toolchain as github would?
I'd like to tell the tooling only the OS (Windows vs. Linux), and then
the toolchain should be auto-selected, same way as it is auto-selected
on github. For example, if I'm in my Windows 10 VM, then I should be
*unable* to request anything else than the latest VS toolchain;
similarly, in the Linux VM, I should be *unable* to ask for (e.g.) the
GCC49 toolchain; only the GCC50 toolchain should be usable.
When I submit a github PR, I'm not asked to pick toolchains for CI; can
we automate the toolchain selection locally too?
(Sorry if I misunderstood your comment above -- if you are saying that
it's permitted, but *not required*, to specify TOOL_CHAIN_TAG, then my
question (2) falls away.)
Thanks!
Laszlo
>
> - Bret
>
> From: Rebecca Cran via groups.io<mailto:rebecca=nuviainc.com@groups.io>
> Sent: Thursday, March 4, 2021 9:45 AM
> To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; lersek@redhat.com<mailto:lersek@redhat.com>
> Cc: Sean Brogan<mailto:sean.brogan@microsoft.com>; Bret Barkelew<mailto:Bret.Barkelew@microsoft.com>; Kinney, Michael D<mailto:michael.d.kinney@intel.com>
> Subject: [EXTERNAL] Re: [edk2-devel] generating the test matrix for local CI runs
>
> On 3/4/21 10:04 AM, Laszlo Ersek wrote:
>> (Sigh, my address book pulled one over me -- resending to the correct
>> list address now. Please ignore the previous posting that you may have
>> gotten "in private".)
>>
>> Hi All,
>>
>> while I can run specific CI tests locally, using the commands listed at:
>>
>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2F.pytool&data=04%7C01%7Cbret.barkelew%40microsoft.com%7Ce4e5493578cf45b804c608d8df354b05%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504767278995847%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=l8WblhD%2B7dpkFaCZ8roSfNUmj1imytHi5eUT%2F%2BKrLCo%3D&reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2F.pytool&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C385eef1d988f4a56155a08d8df36ecce%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504774296518259%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=UPVktVmDdChYhoc6zErhuz57Dllw21gqcwi9crhUdp8%3D&reserved=0>
>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2FArmVirtPkg%2FPlatformCI&data=04%7C01%7Cbret.barkelew%40microsoft.com%7Ce4e5493578cf45b804c608d8df354b05%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504767278995847%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=8yG7Mro9Li4XeCvfmPhfReBbipz47nyAjUmMKMGfh%2Bo%3D&reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2FArmVirtPkg%2FPlatformCI&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C385eef1d988f4a56155a08d8df36ecce%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504774296518259%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=mcoSK9x7FSSJzlI86mdvs%2Btp7zdeF42IJn62mJxxozo%3D&reserved=0>
>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2FOvmfPkg%2FPlatformCI&data=04%7C01%7Cbret.barkelew%40microsoft.com%7Ce4e5493578cf45b804c608d8df354b05%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504767278995847%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=NUhAJ8zkITwPkvCrUqb7ZogSQTJuBXv0eL6Au0sYC2M%3D&reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2FOvmfPkg%2FPlatformCI&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C385eef1d988f4a56155a08d8df36ecce%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504774296528219%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=zOv0z287ugh6PmcVk9EMAvxx4RE8U8iHUYemWzO1FaY%3D&reserved=0>
>>
>> I don't know how I can determine and drive the whole "test plan" locally.
>>
>> When I submit a PR to github, "something" generates (say) 83 test cases.
>> How can I run that "something" locally?
>>
>> Assume I have the following:
>> - a Linux builder VM, with up-to-date tools,
>> - a Windows builder VM, with up-to-date tools,
>> - a topic branch, on top of "master".
>>
>> How do I generate and run those same 83 tests, locally?
>>
>> It has happened to me that all my local CI steps succeeded (meaning both
>> the Windows and the Linux builder), but github still rejected the PR --
>> the reason was that I failed to think of a particular test that github
>> would run. Composing 83 "stuart" command lines isn't something I'd like
>> to do manually.
>
> FWIW it looks like people have been looking for similar functionality of
> testing a pipeline locally, and not found it.
>
> e.g.:
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmicrosoft%2Fazure-pipelines-agent%2Fissues%2F1764&data=04%7C01%7Cbret.barkelew%40microsoft.com%7Ce4e5493578cf45b804c608d8df354b05%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504767278995847%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=XMxZesyKxVN9Wg3SyRpUbnnmKzojevHCqQhzQ1Pc4%2Bg%3D&reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmicrosoft%2Fazure-pipelines-agent%2Fissues%2F1764&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C385eef1d988f4a56155a08d8df36ecce%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504774296528219%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=uwpRNBUe4wFq7M0tRXBs8U4O1Qn6T0jf76Acr7n0qu8%3D&reserved=0>
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdevelopercommunity.visualstudio.com%2Ft%2Fyaml-how-to-test-yaml-locally-before-commit%2F1302759&data=04%7C01%7Cbret.barkelew%40microsoft.com%7Ce4e5493578cf45b804c608d8df354b05%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504767278995847%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=gwGZAQJAIXIS0LODFpglrCf5Ua2JeMnNm3A3auCrhx8%3D&reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdevelopercommunity.visualstudio.com%2Ft%2Fyaml-how-to-test-yaml-locally-before-commit%2F1302759&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C385eef1d988f4a56155a08d8df36ecce%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637504774296538176%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=ISHz%2FhaG88Cg6V%2BMwguTJNLrFwABVeNmOn4mq5meKd8%3D&reserved=0>
>
> --
> Rebecca Cran
>
>
>
>
>
>
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [EXTERNAL] Re: [edk2-devel] generating the test matrix for local CI runs
2021-03-04 18:32 ` Bret Barkelew
@ 2021-03-04 19:07 ` Laszlo Ersek
0 siblings, 0 replies; 7+ messages in thread
From: Laszlo Ersek @ 2021-03-04 19:07 UTC (permalink / raw)
To: Bret Barkelew, devel@edk2.groups.io, rebecca@nuviainc.com
Cc: Sean Brogan, Kinney, Michael D
On 03/04/21 19:32, Bret Barkelew wrote:
> Can you share the specific PR you\x19re failing so that we can talk in
> concrete terms?
Sure.
In January, I posted the following patch series:
[edk2-devel] [PATCH 0/8]
ShellPkg, ArmVirtPkg, OvmfPkg: shell usability improvements
https://edk2.groups.io/g/devel/message/69590
https://www.redhat.com/archives/edk2-devel-archive/2021-January/msg00070.html
Message-Id: 20210104154235.31785-1-lersek@redhat.com
This series cleared my local CI just fine (both Windows and Linux VM),
before I posted it.
When review was done, I attempted to merge the series:
https://github.com/tianocore/edk2/pull/1336
Although the PR does not show it now (or at least I cannot make it show
it), the github CI uncovered a real problem in my patch set, one that my
local run missed.
Namely, I had added an OrderedCollectionLib class dependency to the
(singleton) UefiShellCommandLib instance, but failed to resolve the
OrderedCollectionLib class in the EmulatorPkg and UefiPayloadPkg DSC
files. Consequently, those DSC files would no longer build. My local CI
runs didn't extend to these packages, but -- thankfully -- the
github.com-based CI did.
In v2, I added one extra patch for each of EmulatorPkg and
UefiPayloadPkg DSC:
[edk2-devel] [PATCH v2 00/10]
multiple packages: shell usability improvements
https://edk2.groups.io/g/devel/message/70216
https://www.redhat.com/archives/edk2-devel-archive/2021-January/msg00700.html
Message-Id: 20210113085453.10168-1-lersek@redhat.com
(What's more, I tracked down the DSC files in edk2-platforms that used
to build the shell, and I resolved the OrderedCollectionLib class in
them as well -- see the links inside the above-referenced blurbs.)
The v2 CI succeded on github (personal, then merge):
https://github.com/tianocore/edk2/pull/1337
https://github.com/tianocore/edk2/pull/1370
In summary, the bug (in my local CI usage) was that I didn't cover
EmulatorPkg and UefiPayloadPkg.
So, I'd like if the local test builds determined the affected set of
packages as well -- I'd like to be *unable* to miss EmulatorPkg and
UefiPayloadPkg in the above type of situation.
Thanks!
Laszlo
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2021-03-04 19:07 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2021-03-04 17:04 generating the test matrix for local CI runs Laszlo Ersek
2021-03-04 17:45 ` [edk2-devel] " Rebecca Cran
2021-03-04 17:56 ` [EXTERNAL] " Bret Barkelew
2021-03-04 18:20 ` Bret Barkelew
2021-03-04 18:32 ` Bret Barkelew
2021-03-04 19:07 ` Laszlo Ersek
2021-03-04 18:45 ` Laszlo Ersek
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox