Re: [edk2-devel] **NOTICE** EDK II CI using Azure Pipelines Agents Failing - Update 2-14-2022 Back Up!

Re: [edk2-devel] **NOTICE** EDK II CI using Azure Pipelines Agents Failing - Update 2-14-2022 Back Up!

[Michael D Kinney]

Hello,

The EDK II CI agents are running again.  I have requested all the back logged PRs to
rerun their CI checks and all of those have completed.  I have also observed new PRs
being processed as expected.

We do not have a complete root cause of the issue yet.  We are continuing to collect
data and as soon as we have identified the root cause, we will share that information
along with proposals to help prevent this type of interrupt in EDK II CI services in
the future.

Thank you for your patience.

We should continue with the stable tag release as planned.

Best regards,

Mike

> -----Original Message-----
> From: Sean Brogan <spbrogan@outlook.com>
> Sent: Friday, February 11, 2022 3:27 PM
> To: devel@edk2.groups.io; Kinney, Michael D <michael.d.kinney@intel.com>; Sean Brogan <sean.brogan@microsoft.com>
> Subject: Re: [edk2-devel] **NOTICE** EDK II CI using Azure Pipelines Agents Failing
> 
> Not a great update.
> 
> I have opened a ticket with our system for supporting Azure DevOps
> Pipelines.  There have been a number of projects hit by this in the past
> few days so I hope we can get to mitigation quickly.
> 
> At the moment I don't have a clear answer why but for some reason our
> usage of the build services have gotten us flagged as "abusers".  More
> digging around has indicated that pipeline builds (in general) are
> getting abused for crypto mining and other nefarious uses.  In some
> cases vulnerabilities in published packages are being used to do this
> (point being not always the projects intent).
> 
> I have noticed that some of our builds have taken nearly two hours to
> install NPM cspell package (generally it only takes 44 seconds).  I have
> no idea if that is anything other than an anomaly.
> 
> https://dev.azure.com/tianocore/edk2-ci/_build/results?buildId=40249&view=logs&j=4a97f94c-8e1f-5e76-68ba-
> 50872604dd63&t=7346d0e9-c9fc-5f26-991e-d6f31d36ba1e
> 
> 
> Anyway, I'll update when i have more.
> 
> Thanks
> Sean
> 
> 
> 
> 
> 
> On 2/11/2022 8:16 AM, Michael D Kinney wrote:
> > Hello,
> >
> > The EDK II CI using Azure Pipelines Agents is not working right now.
> >
> > First failure occurred on 2-10-2022.
> >
> > The issue is being worked.
> >
> > We will send daily status updates until the issue is resolved.
> >
> > We are currently in a hard freeze for the next EDK II stable tag.
> > We will monitor status closely and adjust the release date if needed.
> >
> > Thank you for your patience.
> >
> > Mike
> >
> >
> > 
> >
> >