* spurious CI failures @ 2021-12-22 9:43 Ard Biesheuvel 2021-12-22 16:41 ` Michael D Kinney 0 siblings, 1 reply; 6+ messages in thread From: Ard Biesheuvel @ 2021-12-22 9:43 UTC (permalink / raw) To: edk2-devel-groups-io Cc: Leif Lindholm, Michael Kinney, Sean Brogan, Bret Barkelew, Michael Kubacki Hello all, In one out of ~5 attempts at pushing a PR to master, I get a failure at the following step "Install QEMU and Set QEMU on path" of the PlatformCI_OvmfPkg_Windows_VS2019_PR target. Is there any way we can remediate this? Or at least classify some failures as temporary, so it doesn't look like the failure has anything to do with the contents of the PR? Alternatively, a control for maintainers to override bogus CI results would help as well. Thanks, Ard. ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: spurious CI failures 2021-12-22 9:43 spurious CI failures Ard Biesheuvel @ 2021-12-22 16:41 ` Michael D Kinney 2021-12-22 19:22 ` [edk2-devel] " Pedro Falcato 0 siblings, 1 reply; 6+ messages in thread From: Michael D Kinney @ 2021-12-22 16:41 UTC (permalink / raw) To: Ard Biesheuvel, edk2-devel-groups-io, Kinney, Michael D Cc: Leif Lindholm, Sean Brogan, Bret Barkelew, Kubacki, Michael Hi Ard, Can you point me at some PRs that have this in their log? Thanks, Mike > -----Original Message----- > From: Ard Biesheuvel <ardb@kernel.org> > Sent: Wednesday, December 22, 2021 1:43 AM > To: edk2-devel-groups-io <devel@edk2.groups.io> > Cc: Leif Lindholm <leif@nuviainc.com>; Kinney, Michael D <michael.d.kinney@intel.com>; Sean Brogan <sean.brogan@microsoft.com>; > Bret Barkelew <Bret.Barkelew@microsoft.com>; Kubacki, Michael <michael.kubacki@microsoft.com> > Subject: spurious CI failures > > Hello all, > > In one out of ~5 attempts at pushing a PR to master, I get a failure > at the following step > > "Install QEMU and Set QEMU on path" > > of the PlatformCI_OvmfPkg_Windows_VS2019_PR target. > > Is there any way we can remediate this? Or at least classify some > failures as temporary, so it doesn't look like the failure has > anything to do with the contents of the PR? > > Alternatively, a control for maintainers to override bogus CI results > would help as well. > > Thanks, > Ard. ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [edk2-devel] spurious CI failures 2021-12-22 16:41 ` Michael D Kinney @ 2021-12-22 19:22 ` Pedro Falcato 2021-12-22 20:27 ` Michael D Kinney 0 siblings, 1 reply; 6+ messages in thread From: Pedro Falcato @ 2021-12-22 19:22 UTC (permalink / raw) To: edk2-devel-groups-io, Kinney, Michael D Cc: Ard Biesheuvel, Leif Lindholm, Sean Brogan, Bret Barkelew, Kubacki, Michael [-- Attachment #1: Type: text/plain, Size: 1570 bytes --] Hi Michael, I was curious when I read this email this morning and I found a log: https://dev.azure.com/tianocore/edk2-ci/_build/results?buildId=36430&view=logs&j=417cab99-ad63-55d7-94dd-9898cfdbbe0e&t=5e8e03ff-365b-5151-1b77-4c25d18d4964 It looks like Chocolatey is taking way too long to download/install Qemu (it's taking more than 2700 seconds). Best regards, Pedro On Wed, Dec 22, 2021 at 4:42 PM Michael D Kinney <michael.d.kinney@intel.com> wrote: > Hi Ard, > > Can you point me at some PRs that have this in their log? > > Thanks, > > Mike > > > -----Original Message----- > > From: Ard Biesheuvel <ardb@kernel.org> > > Sent: Wednesday, December 22, 2021 1:43 AM > > To: edk2-devel-groups-io <devel@edk2.groups.io> > > Cc: Leif Lindholm <leif@nuviainc.com>; Kinney, Michael D < > michael.d.kinney@intel.com>; Sean Brogan <sean.brogan@microsoft.com>; > > Bret Barkelew <Bret.Barkelew@microsoft.com>; Kubacki, Michael < > michael.kubacki@microsoft.com> > > Subject: spurious CI failures > > > > Hello all, > > > > In one out of ~5 attempts at pushing a PR to master, I get a failure > > at the following step > > > > "Install QEMU and Set QEMU on path" > > > > of the PlatformCI_OvmfPkg_Windows_VS2019_PR target. > > > > Is there any way we can remediate this? Or at least classify some > > failures as temporary, so it doesn't look like the failure has > > anything to do with the contents of the PR? > > > > Alternatively, a control for maintainers to override bogus CI results > > would help as well. > > > > Thanks, > > Ard. > > > > > > -- Pedro Falcato [-- Attachment #2: Type: text/html, Size: 2905 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [edk2-devel] spurious CI failures 2021-12-22 19:22 ` [edk2-devel] " Pedro Falcato @ 2021-12-22 20:27 ` Michael D Kinney 2021-12-22 21:18 ` Pedro Falcato 0 siblings, 1 reply; 6+ messages in thread From: Michael D Kinney @ 2021-12-22 20:27 UTC (permalink / raw) To: devel@edk2.groups.io, pedro.falcato@gmail.com, Kinney, Michael D Cc: Ard Biesheuvel, Leif Lindholm, Sean Brogan, Bret Barkelew, Kubacki, Michael [-- Attachment #1: Type: text/plain, Size: 2743 bytes --] Hi Pedro, Thanks for the link to the log. I agree that log shows the choco install is very slow and timeout out after 45 minutes. choco does support a flag to specify a larger timeout (--execution-timeout). We can try increasing a bit higher as a temp workaround. If choco is going to be this slow, we may have to use the caching features of CI agents to cache these types of downloads so we only need to download when a version is updated. Mike From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Pedro Falcato Sent: Wednesday, December 22, 2021 11:22 AM To: edk2-devel-groups-io <devel@edk2.groups.io>; Kinney, Michael D <michael.d.kinney@intel.com> Cc: Ard Biesheuvel <ardb@kernel.org>; Leif Lindholm <leif@nuviainc.com>; Sean Brogan <sean.brogan@microsoft.com>; Bret Barkelew <Bret.Barkelew@microsoft.com>; Kubacki, Michael <michael.kubacki@microsoft.com> Subject: Re: [edk2-devel] spurious CI failures Hi Michael, I was curious when I read this email this morning and I found a log: https://dev.azure.com/tianocore/edk2-ci/_build/results?buildId=36430&view=logs&j=417cab99-ad63-55d7-94dd-9898cfdbbe0e&t=5e8e03ff-365b-5151-1b77-4c25d18d4964 It looks like Chocolatey is taking way too long to download/install Qemu (it's taking more than 2700 seconds). Best regards, Pedro On Wed, Dec 22, 2021 at 4:42 PM Michael D Kinney <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>> wrote: Hi Ard, Can you point me at some PRs that have this in their log? Thanks, Mike > -----Original Message----- > From: Ard Biesheuvel <ardb@kernel.org<mailto:ardb@kernel.org>> > Sent: Wednesday, December 22, 2021 1:43 AM > To: edk2-devel-groups-io <devel@edk2.groups.io<mailto:devel@edk2.groups.io>> > Cc: Leif Lindholm <leif@nuviainc.com<mailto:leif@nuviainc.com>>; Kinney, Michael D <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>; Sean Brogan <sean.brogan@microsoft.com<mailto:sean.brogan@microsoft.com>>; > Bret Barkelew <Bret.Barkelew@microsoft.com<mailto:Bret.Barkelew@microsoft.com>>; Kubacki, Michael <michael.kubacki@microsoft.com<mailto:michael.kubacki@microsoft.com>> > Subject: spurious CI failures > > Hello all, > > In one out of ~5 attempts at pushing a PR to master, I get a failure > at the following step > > "Install QEMU and Set QEMU on path" > > of the PlatformCI_OvmfPkg_Windows_VS2019_PR target. > > Is there any way we can remediate this? Or at least classify some > failures as temporary, so it doesn't look like the failure has > anything to do with the contents of the PR? > > Alternatively, a control for maintainers to override bogus CI results > would help as well. > > Thanks, > Ard. -- Pedro Falcato [-- Attachment #2: Type: text/html, Size: 44767 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [edk2-devel] spurious CI failures 2021-12-22 20:27 ` Michael D Kinney @ 2021-12-22 21:18 ` Pedro Falcato 2021-12-22 22:45 ` Michael D Kinney 0 siblings, 1 reply; 6+ messages in thread From: Pedro Falcato @ 2021-12-22 21:18 UTC (permalink / raw) To: edk2-devel-groups-io, Kinney, Michael D Cc: Ard Biesheuvel, Leif Lindholm, Sean Brogan, Bret Barkelew, Kubacki, Michael [-- Attachment #1: Type: text/plain, Size: 3199 bytes --] I do find it weird that Chocolatey is taking 45 minutes to install Qemu. Is this normal? I've never used Chocolatey but I've used plenty of package managers and none have ever taken this long to install something. Maybe it's Azure CI's problem? Anyway, is it not possible to cross-build qemu, as detailed in https://wiki.qemu.org/Hosts/W32#Cross_builds_2, in order to avoid any package manager issues? In the case that it's the package manager's problem. On Wed, Dec 22, 2021 at 8:27 PM Michael D Kinney <michael.d.kinney@intel.com> wrote: > Hi Pedro, > > > > Thanks for the link to the log. I agree that log shows the choco install > is very slow and timeout out after 45 minutes. > > > > choco does support a flag to specify a larger timeout > (--execution-timeout). We can try increasing a bit higher as a temp > workaround. > > > > If choco is going to be this slow, we may have to use the caching > features of CI agents to cache these types of downloads > > so we only need to download when a version is updated. > > > > Mike > > > > *From:* devel@edk2.groups.io <devel@edk2.groups.io> *On Behalf Of *Pedro > Falcato > *Sent:* Wednesday, December 22, 2021 11:22 AM > *To:* edk2-devel-groups-io <devel@edk2.groups.io>; Kinney, Michael D < > michael.d.kinney@intel.com> > *Cc:* Ard Biesheuvel <ardb@kernel.org>; Leif Lindholm <leif@nuviainc.com>; > Sean Brogan <sean.brogan@microsoft.com>; Bret Barkelew < > Bret.Barkelew@microsoft.com>; Kubacki, Michael < > michael.kubacki@microsoft.com> > *Subject:* Re: [edk2-devel] spurious CI failures > > > > Hi Michael, > > > > I was curious when I read this email this morning and I found a log: > https://dev.azure.com/tianocore/edk2-ci/_build/results?buildId=36430&view=logs&j=417cab99-ad63-55d7-94dd-9898cfdbbe0e&t=5e8e03ff-365b-5151-1b77-4c25d18d4964 > > It looks like Chocolatey is taking way too long to download/install Qemu > (it's taking more than 2700 seconds). > > > > Best regards, > > Pedro > > > > On Wed, Dec 22, 2021 at 4:42 PM Michael D Kinney < > michael.d.kinney@intel.com> wrote: > > Hi Ard, > > Can you point me at some PRs that have this in their log? > > Thanks, > > Mike > > > -----Original Message----- > > From: Ard Biesheuvel <ardb@kernel.org> > > Sent: Wednesday, December 22, 2021 1:43 AM > > To: edk2-devel-groups-io <devel@edk2.groups.io> > > Cc: Leif Lindholm <leif@nuviainc.com>; Kinney, Michael D < > michael.d.kinney@intel.com>; Sean Brogan <sean.brogan@microsoft.com>; > > Bret Barkelew <Bret.Barkelew@microsoft.com>; Kubacki, Michael < > michael.kubacki@microsoft.com> > > Subject: spurious CI failures > > > > Hello all, > > > > In one out of ~5 attempts at pushing a PR to master, I get a failure > > at the following step > > > > "Install QEMU and Set QEMU on path" > > > > of the PlatformCI_OvmfPkg_Windows_VS2019_PR target. > > > > Is there any way we can remediate this? Or at least classify some > > failures as temporary, so it doesn't look like the failure has > > anything to do with the contents of the PR? > > > > Alternatively, a control for maintainers to override bogus CI results > > would help as well. > > > > Thanks, > > Ard. > > > > > > > > -- > > Pedro Falcato > > > > -- Pedro Falcato [-- Attachment #2: Type: text/html, Size: 7598 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [edk2-devel] spurious CI failures 2021-12-22 21:18 ` Pedro Falcato @ 2021-12-22 22:45 ` Michael D Kinney 0 siblings, 0 replies; 6+ messages in thread From: Michael D Kinney @ 2021-12-22 22:45 UTC (permalink / raw) To: Pedro Falcato, edk2-devel-groups-io, Kinney, Michael D Cc: Ard Biesheuvel, Leif Lindholm, Sean Brogan, Bret Barkelew, Kubacki, Michael [-- Attachment #1: Type: text/plain, Size: 4274 bytes --] I agree that building QEMU from sources is another option. We could also consider adding QEMU to the nuget feed we use for other tools, so we avoid the current server’s perf issues. Mike From: Pedro Falcato <pedro.falcato@gmail.com> Sent: Wednesday, December 22, 2021 1:19 PM To: edk2-devel-groups-io <devel@edk2.groups.io>; Kinney, Michael D <michael.d.kinney@intel.com> Cc: Ard Biesheuvel <ardb@kernel.org>; Leif Lindholm <leif@nuviainc.com>; Sean Brogan <sean.brogan@microsoft.com>; Bret Barkelew <Bret.Barkelew@microsoft.com>; Kubacki, Michael <michael.kubacki@microsoft.com> Subject: Re: [edk2-devel] spurious CI failures I do find it weird that Chocolatey is taking 45 minutes to install Qemu. Is this normal? I've never used Chocolatey but I've used plenty of package managers and none have ever taken this long to install something. Maybe it's Azure CI's problem? Anyway, is it not possible to cross-build qemu, as detailed in https://wiki.qemu.org/Hosts/W32#Cross_builds_2, in order to avoid any package manager issues? In the case that it's the package manager's problem. On Wed, Dec 22, 2021 at 8:27 PM Michael D Kinney <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>> wrote: Hi Pedro, Thanks for the link to the log. I agree that log shows the choco install is very slow and timeout out after 45 minutes. choco does support a flag to specify a larger timeout (--execution-timeout). We can try increasing a bit higher as a temp workaround. If choco is going to be this slow, we may have to use the caching features of CI agents to cache these types of downloads so we only need to download when a version is updated. Mike From: devel@edk2.groups.io<mailto:devel@edk2.groups.io> <devel@edk2.groups.io<mailto:devel@edk2.groups.io>> On Behalf Of Pedro Falcato Sent: Wednesday, December 22, 2021 11:22 AM To: edk2-devel-groups-io <devel@edk2.groups.io<mailto:devel@edk2.groups.io>>; Kinney, Michael D <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>> Cc: Ard Biesheuvel <ardb@kernel.org<mailto:ardb@kernel.org>>; Leif Lindholm <leif@nuviainc.com<mailto:leif@nuviainc.com>>; Sean Brogan <sean.brogan@microsoft.com<mailto:sean.brogan@microsoft.com>>; Bret Barkelew <Bret.Barkelew@microsoft.com<mailto:Bret.Barkelew@microsoft.com>>; Kubacki, Michael <michael.kubacki@microsoft.com<mailto:michael.kubacki@microsoft.com>> Subject: Re: [edk2-devel] spurious CI failures Hi Michael, I was curious when I read this email this morning and I found a log: https://dev.azure.com/tianocore/edk2-ci/_build/results?buildId=36430&view=logs&j=417cab99-ad63-55d7-94dd-9898cfdbbe0e&t=5e8e03ff-365b-5151-1b77-4c25d18d4964 It looks like Chocolatey is taking way too long to download/install Qemu (it's taking more than 2700 seconds). Best regards, Pedro On Wed, Dec 22, 2021 at 4:42 PM Michael D Kinney <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>> wrote: Hi Ard, Can you point me at some PRs that have this in their log? Thanks, Mike > -----Original Message----- > From: Ard Biesheuvel <ardb@kernel.org<mailto:ardb@kernel.org>> > Sent: Wednesday, December 22, 2021 1:43 AM > To: edk2-devel-groups-io <devel@edk2.groups.io<mailto:devel@edk2.groups.io>> > Cc: Leif Lindholm <leif@nuviainc.com<mailto:leif@nuviainc.com>>; Kinney, Michael D <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>; Sean Brogan <sean.brogan@microsoft.com<mailto:sean.brogan@microsoft.com>>; > Bret Barkelew <Bret.Barkelew@microsoft.com<mailto:Bret.Barkelew@microsoft.com>>; Kubacki, Michael <michael.kubacki@microsoft.com<mailto:michael.kubacki@microsoft.com>> > Subject: spurious CI failures > > Hello all, > > In one out of ~5 attempts at pushing a PR to master, I get a failure > at the following step > > "Install QEMU and Set QEMU on path" > > of the PlatformCI_OvmfPkg_Windows_VS2019_PR target. > > Is there any way we can remediate this? Or at least classify some > failures as temporary, so it doesn't look like the failure has > anything to do with the contents of the PR? > > Alternatively, a control for maintainers to override bogus CI results > would help as well. > > Thanks, > Ard. -- Pedro Falcato -- Pedro Falcato [-- Attachment #2: Type: text/html, Size: 49114 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2021-12-22 22:45 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2021-12-22 9:43 spurious CI failures Ard Biesheuvel 2021-12-22 16:41 ` Michael D Kinney 2021-12-22 19:22 ` [edk2-devel] " Pedro Falcato 2021-12-22 20:27 ` Michael D Kinney 2021-12-22 21:18 ` Pedro Falcato 2021-12-22 22:45 ` Michael D Kinney
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox