* spurious CI failures
@ 2021-12-22 9:43 Ard Biesheuvel
2021-12-22 16:41 ` Michael D Kinney
0 siblings, 1 reply; 6+ messages in thread
From: Ard Biesheuvel @ 2021-12-22 9:43 UTC (permalink / raw)
To: edk2-devel-groups-io
Cc: Leif Lindholm, Michael Kinney, Sean Brogan, Bret Barkelew,
Michael Kubacki
Hello all,
In one out of ~5 attempts at pushing a PR to master, I get a failure
at the following step
"Install QEMU and Set QEMU on path"
of the PlatformCI_OvmfPkg_Windows_VS2019_PR target.
Is there any way we can remediate this? Or at least classify some
failures as temporary, so it doesn't look like the failure has
anything to do with the contents of the PR?
Alternatively, a control for maintainers to override bogus CI results
would help as well.
Thanks,
Ard.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: spurious CI failures
2021-12-22 9:43 spurious CI failures Ard Biesheuvel
@ 2021-12-22 16:41 ` Michael D Kinney
2021-12-22 19:22 ` [edk2-devel] " Pedro Falcato
0 siblings, 1 reply; 6+ messages in thread
From: Michael D Kinney @ 2021-12-22 16:41 UTC (permalink / raw)
To: Ard Biesheuvel, edk2-devel-groups-io, Kinney, Michael D
Cc: Leif Lindholm, Sean Brogan, Bret Barkelew, Kubacki, Michael
Hi Ard,
Can you point me at some PRs that have this in their log?
Thanks,
Mike
> -----Original Message-----
> From: Ard Biesheuvel <ardb@kernel.org>
> Sent: Wednesday, December 22, 2021 1:43 AM
> To: edk2-devel-groups-io <devel@edk2.groups.io>
> Cc: Leif Lindholm <leif@nuviainc.com>; Kinney, Michael D <michael.d.kinney@intel.com>; Sean Brogan <sean.brogan@microsoft.com>;
> Bret Barkelew <Bret.Barkelew@microsoft.com>; Kubacki, Michael <michael.kubacki@microsoft.com>
> Subject: spurious CI failures
>
> Hello all,
>
> In one out of ~5 attempts at pushing a PR to master, I get a failure
> at the following step
>
> "Install QEMU and Set QEMU on path"
>
> of the PlatformCI_OvmfPkg_Windows_VS2019_PR target.
>
> Is there any way we can remediate this? Or at least classify some
> failures as temporary, so it doesn't look like the failure has
> anything to do with the contents of the PR?
>
> Alternatively, a control for maintainers to override bogus CI results
> would help as well.
>
> Thanks,
> Ard.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [edk2-devel] spurious CI failures
2021-12-22 16:41 ` Michael D Kinney
@ 2021-12-22 19:22 ` Pedro Falcato
2021-12-22 20:27 ` Michael D Kinney
0 siblings, 1 reply; 6+ messages in thread
From: Pedro Falcato @ 2021-12-22 19:22 UTC (permalink / raw)
To: edk2-devel-groups-io, Kinney, Michael D
Cc: Ard Biesheuvel, Leif Lindholm, Sean Brogan, Bret Barkelew,
Kubacki, Michael
[-- Attachment #1: Type: text/plain, Size: 1570 bytes --]
Hi Michael,
I was curious when I read this email this morning and I found a log:
https://dev.azure.com/tianocore/edk2-ci/_build/results?buildId=36430&view=logs&j=417cab99-ad63-55d7-94dd-9898cfdbbe0e&t=5e8e03ff-365b-5151-1b77-4c25d18d4964
It looks like Chocolatey is taking way too long to download/install Qemu
(it's taking more than 2700 seconds).
Best regards,
Pedro
On Wed, Dec 22, 2021 at 4:42 PM Michael D Kinney <michael.d.kinney@intel.com>
wrote:
> Hi Ard,
>
> Can you point me at some PRs that have this in their log?
>
> Thanks,
>
> Mike
>
> > -----Original Message-----
> > From: Ard Biesheuvel <ardb@kernel.org>
> > Sent: Wednesday, December 22, 2021 1:43 AM
> > To: edk2-devel-groups-io <devel@edk2.groups.io>
> > Cc: Leif Lindholm <leif@nuviainc.com>; Kinney, Michael D <
> michael.d.kinney@intel.com>; Sean Brogan <sean.brogan@microsoft.com>;
> > Bret Barkelew <Bret.Barkelew@microsoft.com>; Kubacki, Michael <
> michael.kubacki@microsoft.com>
> > Subject: spurious CI failures
> >
> > Hello all,
> >
> > In one out of ~5 attempts at pushing a PR to master, I get a failure
> > at the following step
> >
> > "Install QEMU and Set QEMU on path"
> >
> > of the PlatformCI_OvmfPkg_Windows_VS2019_PR target.
> >
> > Is there any way we can remediate this? Or at least classify some
> > failures as temporary, so it doesn't look like the failure has
> > anything to do with the contents of the PR?
> >
> > Alternatively, a control for maintainers to override bogus CI results
> > would help as well.
> >
> > Thanks,
> > Ard.
>
>
>
>
>
>
--
Pedro Falcato
[-- Attachment #2: Type: text/html, Size: 2905 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [edk2-devel] spurious CI failures
2021-12-22 19:22 ` [edk2-devel] " Pedro Falcato
@ 2021-12-22 20:27 ` Michael D Kinney
2021-12-22 21:18 ` Pedro Falcato
0 siblings, 1 reply; 6+ messages in thread
From: Michael D Kinney @ 2021-12-22 20:27 UTC (permalink / raw)
To: devel@edk2.groups.io, pedro.falcato@gmail.com, Kinney, Michael D
Cc: Ard Biesheuvel, Leif Lindholm, Sean Brogan, Bret Barkelew,
Kubacki, Michael
[-- Attachment #1: Type: text/plain, Size: 2743 bytes --]
Hi Pedro,
Thanks for the link to the log. I agree that log shows the choco install is very slow and timeout out after 45 minutes.
choco does support a flag to specify a larger timeout (--execution-timeout). We can try increasing a bit higher as a temp workaround.
If choco is going to be this slow, we may have to use the caching features of CI agents to cache these types of downloads
so we only need to download when a version is updated.
Mike
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Pedro Falcato
Sent: Wednesday, December 22, 2021 11:22 AM
To: edk2-devel-groups-io <devel@edk2.groups.io>; Kinney, Michael D <michael.d.kinney@intel.com>
Cc: Ard Biesheuvel <ardb@kernel.org>; Leif Lindholm <leif@nuviainc.com>; Sean Brogan <sean.brogan@microsoft.com>; Bret Barkelew <Bret.Barkelew@microsoft.com>; Kubacki, Michael <michael.kubacki@microsoft.com>
Subject: Re: [edk2-devel] spurious CI failures
Hi Michael,
I was curious when I read this email this morning and I found a log: https://dev.azure.com/tianocore/edk2-ci/_build/results?buildId=36430&view=logs&j=417cab99-ad63-55d7-94dd-9898cfdbbe0e&t=5e8e03ff-365b-5151-1b77-4c25d18d4964
It looks like Chocolatey is taking way too long to download/install Qemu (it's taking more than 2700 seconds).
Best regards,
Pedro
On Wed, Dec 22, 2021 at 4:42 PM Michael D Kinney <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>> wrote:
Hi Ard,
Can you point me at some PRs that have this in their log?
Thanks,
Mike
> -----Original Message-----
> From: Ard Biesheuvel <ardb@kernel.org<mailto:ardb@kernel.org>>
> Sent: Wednesday, December 22, 2021 1:43 AM
> To: edk2-devel-groups-io <devel@edk2.groups.io<mailto:devel@edk2.groups.io>>
> Cc: Leif Lindholm <leif@nuviainc.com<mailto:leif@nuviainc.com>>; Kinney, Michael D <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>; Sean Brogan <sean.brogan@microsoft.com<mailto:sean.brogan@microsoft.com>>;
> Bret Barkelew <Bret.Barkelew@microsoft.com<mailto:Bret.Barkelew@microsoft.com>>; Kubacki, Michael <michael.kubacki@microsoft.com<mailto:michael.kubacki@microsoft.com>>
> Subject: spurious CI failures
>
> Hello all,
>
> In one out of ~5 attempts at pushing a PR to master, I get a failure
> at the following step
>
> "Install QEMU and Set QEMU on path"
>
> of the PlatformCI_OvmfPkg_Windows_VS2019_PR target.
>
> Is there any way we can remediate this? Or at least classify some
> failures as temporary, so it doesn't look like the failure has
> anything to do with the contents of the PR?
>
> Alternatively, a control for maintainers to override bogus CI results
> would help as well.
>
> Thanks,
> Ard.
--
Pedro Falcato
[-- Attachment #2: Type: text/html, Size: 44767 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [edk2-devel] spurious CI failures
2021-12-22 20:27 ` Michael D Kinney
@ 2021-12-22 21:18 ` Pedro Falcato
2021-12-22 22:45 ` Michael D Kinney
0 siblings, 1 reply; 6+ messages in thread
From: Pedro Falcato @ 2021-12-22 21:18 UTC (permalink / raw)
To: edk2-devel-groups-io, Kinney, Michael D
Cc: Ard Biesheuvel, Leif Lindholm, Sean Brogan, Bret Barkelew,
Kubacki, Michael
[-- Attachment #1: Type: text/plain, Size: 3199 bytes --]
I do find it weird that Chocolatey is taking 45 minutes to install Qemu. Is
this normal? I've never used Chocolatey but I've used plenty of package
managers and none have ever taken this long to install something.
Maybe it's Azure CI's problem?
Anyway, is it not possible to cross-build qemu, as detailed in
https://wiki.qemu.org/Hosts/W32#Cross_builds_2, in order to avoid any
package manager issues? In the case that it's the package manager's
problem.
On Wed, Dec 22, 2021 at 8:27 PM Michael D Kinney <michael.d.kinney@intel.com>
wrote:
> Hi Pedro,
>
>
>
> Thanks for the link to the log. I agree that log shows the choco install
> is very slow and timeout out after 45 minutes.
>
>
>
> choco does support a flag to specify a larger timeout
> (--execution-timeout). We can try increasing a bit higher as a temp
> workaround.
>
>
>
> If choco is going to be this slow, we may have to use the caching
> features of CI agents to cache these types of downloads
>
> so we only need to download when a version is updated.
>
>
>
> Mike
>
>
>
> *From:* devel@edk2.groups.io <devel@edk2.groups.io> *On Behalf Of *Pedro
> Falcato
> *Sent:* Wednesday, December 22, 2021 11:22 AM
> *To:* edk2-devel-groups-io <devel@edk2.groups.io>; Kinney, Michael D <
> michael.d.kinney@intel.com>
> *Cc:* Ard Biesheuvel <ardb@kernel.org>; Leif Lindholm <leif@nuviainc.com>;
> Sean Brogan <sean.brogan@microsoft.com>; Bret Barkelew <
> Bret.Barkelew@microsoft.com>; Kubacki, Michael <
> michael.kubacki@microsoft.com>
> *Subject:* Re: [edk2-devel] spurious CI failures
>
>
>
> Hi Michael,
>
>
>
> I was curious when I read this email this morning and I found a log:
> https://dev.azure.com/tianocore/edk2-ci/_build/results?buildId=36430&view=logs&j=417cab99-ad63-55d7-94dd-9898cfdbbe0e&t=5e8e03ff-365b-5151-1b77-4c25d18d4964
>
> It looks like Chocolatey is taking way too long to download/install Qemu
> (it's taking more than 2700 seconds).
>
>
>
> Best regards,
>
> Pedro
>
>
>
> On Wed, Dec 22, 2021 at 4:42 PM Michael D Kinney <
> michael.d.kinney@intel.com> wrote:
>
> Hi Ard,
>
> Can you point me at some PRs that have this in their log?
>
> Thanks,
>
> Mike
>
> > -----Original Message-----
> > From: Ard Biesheuvel <ardb@kernel.org>
> > Sent: Wednesday, December 22, 2021 1:43 AM
> > To: edk2-devel-groups-io <devel@edk2.groups.io>
> > Cc: Leif Lindholm <leif@nuviainc.com>; Kinney, Michael D <
> michael.d.kinney@intel.com>; Sean Brogan <sean.brogan@microsoft.com>;
> > Bret Barkelew <Bret.Barkelew@microsoft.com>; Kubacki, Michael <
> michael.kubacki@microsoft.com>
> > Subject: spurious CI failures
> >
> > Hello all,
> >
> > In one out of ~5 attempts at pushing a PR to master, I get a failure
> > at the following step
> >
> > "Install QEMU and Set QEMU on path"
> >
> > of the PlatformCI_OvmfPkg_Windows_VS2019_PR target.
> >
> > Is there any way we can remediate this? Or at least classify some
> > failures as temporary, so it doesn't look like the failure has
> > anything to do with the contents of the PR?
> >
> > Alternatively, a control for maintainers to override bogus CI results
> > would help as well.
> >
> > Thanks,
> > Ard.
>
>
>
>
>
>
>
> --
>
> Pedro Falcato
>
>
>
>
--
Pedro Falcato
[-- Attachment #2: Type: text/html, Size: 7598 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [edk2-devel] spurious CI failures
2021-12-22 21:18 ` Pedro Falcato
@ 2021-12-22 22:45 ` Michael D Kinney
0 siblings, 0 replies; 6+ messages in thread
From: Michael D Kinney @ 2021-12-22 22:45 UTC (permalink / raw)
To: Pedro Falcato, edk2-devel-groups-io, Kinney, Michael D
Cc: Ard Biesheuvel, Leif Lindholm, Sean Brogan, Bret Barkelew,
Kubacki, Michael
[-- Attachment #1: Type: text/plain, Size: 4274 bytes --]
I agree that building QEMU from sources is another option.
We could also consider adding QEMU to the nuget feed we use for other tools, so we avoid the current server’s perf issues.
Mike
From: Pedro Falcato <pedro.falcato@gmail.com>
Sent: Wednesday, December 22, 2021 1:19 PM
To: edk2-devel-groups-io <devel@edk2.groups.io>; Kinney, Michael D <michael.d.kinney@intel.com>
Cc: Ard Biesheuvel <ardb@kernel.org>; Leif Lindholm <leif@nuviainc.com>; Sean Brogan <sean.brogan@microsoft.com>; Bret Barkelew <Bret.Barkelew@microsoft.com>; Kubacki, Michael <michael.kubacki@microsoft.com>
Subject: Re: [edk2-devel] spurious CI failures
I do find it weird that Chocolatey is taking 45 minutes to install Qemu. Is this normal? I've never used Chocolatey but I've used plenty of package managers and none have ever taken this long to install something.
Maybe it's Azure CI's problem?
Anyway, is it not possible to cross-build qemu, as detailed in https://wiki.qemu.org/Hosts/W32#Cross_builds_2, in order to avoid any package manager issues? In the case that it's the package manager's problem.
On Wed, Dec 22, 2021 at 8:27 PM Michael D Kinney <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>> wrote:
Hi Pedro,
Thanks for the link to the log. I agree that log shows the choco install is very slow and timeout out after 45 minutes.
choco does support a flag to specify a larger timeout (--execution-timeout). We can try increasing a bit higher as a temp workaround.
If choco is going to be this slow, we may have to use the caching features of CI agents to cache these types of downloads
so we only need to download when a version is updated.
Mike
From: devel@edk2.groups.io<mailto:devel@edk2.groups.io> <devel@edk2.groups.io<mailto:devel@edk2.groups.io>> On Behalf Of Pedro Falcato
Sent: Wednesday, December 22, 2021 11:22 AM
To: edk2-devel-groups-io <devel@edk2.groups.io<mailto:devel@edk2.groups.io>>; Kinney, Michael D <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>
Cc: Ard Biesheuvel <ardb@kernel.org<mailto:ardb@kernel.org>>; Leif Lindholm <leif@nuviainc.com<mailto:leif@nuviainc.com>>; Sean Brogan <sean.brogan@microsoft.com<mailto:sean.brogan@microsoft.com>>; Bret Barkelew <Bret.Barkelew@microsoft.com<mailto:Bret.Barkelew@microsoft.com>>; Kubacki, Michael <michael.kubacki@microsoft.com<mailto:michael.kubacki@microsoft.com>>
Subject: Re: [edk2-devel] spurious CI failures
Hi Michael,
I was curious when I read this email this morning and I found a log: https://dev.azure.com/tianocore/edk2-ci/_build/results?buildId=36430&view=logs&j=417cab99-ad63-55d7-94dd-9898cfdbbe0e&t=5e8e03ff-365b-5151-1b77-4c25d18d4964
It looks like Chocolatey is taking way too long to download/install Qemu (it's taking more than 2700 seconds).
Best regards,
Pedro
On Wed, Dec 22, 2021 at 4:42 PM Michael D Kinney <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>> wrote:
Hi Ard,
Can you point me at some PRs that have this in their log?
Thanks,
Mike
> -----Original Message-----
> From: Ard Biesheuvel <ardb@kernel.org<mailto:ardb@kernel.org>>
> Sent: Wednesday, December 22, 2021 1:43 AM
> To: edk2-devel-groups-io <devel@edk2.groups.io<mailto:devel@edk2.groups.io>>
> Cc: Leif Lindholm <leif@nuviainc.com<mailto:leif@nuviainc.com>>; Kinney, Michael D <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>; Sean Brogan <sean.brogan@microsoft.com<mailto:sean.brogan@microsoft.com>>;
> Bret Barkelew <Bret.Barkelew@microsoft.com<mailto:Bret.Barkelew@microsoft.com>>; Kubacki, Michael <michael.kubacki@microsoft.com<mailto:michael.kubacki@microsoft.com>>
> Subject: spurious CI failures
>
> Hello all,
>
> In one out of ~5 attempts at pushing a PR to master, I get a failure
> at the following step
>
> "Install QEMU and Set QEMU on path"
>
> of the PlatformCI_OvmfPkg_Windows_VS2019_PR target.
>
> Is there any way we can remediate this? Or at least classify some
> failures as temporary, so it doesn't look like the failure has
> anything to do with the contents of the PR?
>
> Alternatively, a control for maintainers to override bogus CI results
> would help as well.
>
> Thanks,
> Ard.
--
Pedro Falcato
--
Pedro Falcato
[-- Attachment #2: Type: text/html, Size: 49114 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2021-12-22 22:45 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2021-12-22 9:43 spurious CI failures Ard Biesheuvel
2021-12-22 16:41 ` Michael D Kinney
2021-12-22 19:22 ` [edk2-devel] " Pedro Falcato
2021-12-22 20:27 ` Michael D Kinney
2021-12-22 21:18 ` Pedro Falcato
2021-12-22 22:45 ` Michael D Kinney
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox