From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (NAM10-DM6-obe.outbound.protection.outlook.com [40.92.41.92]) by mx.groups.io with SMTP id smtpd.web12.3296.1644622022399477810 for ; Fri, 11 Feb 2022 15:27:02 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@outlook.com header.s=selector1 header.b=qiNxM3fZ; spf=pass (domain: outlook.com, ip: 40.92.41.92, mailfrom: spbrogan@outlook.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=NS8f8OZ8mjQCYE7sNoUFS6jFpvfskSg5Cc6Dxxrr4Ggje/QDF6lqO8IOKcBhp3xfzfZgnfXGmzCLZHi+PxYNFGe/KXYlDlvU4W8BXK6wlwiq5ngkiYTj1Iv4+gKUaIaxKdyGR7PMREmcJhAyIwxDjspK8Dc9DW+b0jgBiG87Fj+2V0TXCQrzalySW3al2cn7LbVyf8nLR09zM1ECvEALHjQWNRTa/q75il3anTrYpcncWbC94zSaJgya0MfP8iplOZ7kU0AWfkLP8PYhouc/khBpfiareEF7J2Nfkjri6mL48OISLt1rqOHTsztfM2cwxBMdciURcfhq/M5rx0qy+Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=OacS2tb5LrmapoEULF4hHeKeU526f7HVyBtkMzhw2vI=; b=iMXRQjJ4HdAZoSVxK37UKU62xz+sIhhnaqgznnyGtBkyGWws9ky7PW87YP6DL1GH2au8ZLsVrSzkRVquImzHYw3Y2NiIkHaRyHzQo1B7NwMU91dBvHNza6q/w2dbt4JaDc9YgBADFaBAbzRYEVPP64tgnA/gvGMdbTYB9FIS9ua5RT6+x1pYDLbO/xKpA1aoQ9Dl2HQO1GaV//BcgI2ER08Xa0SsYa9zeEBPQ7iE/td7n3QlPWotS1A8SD88oFa4Z4sPu1ZjSgK5PKH76TxWxOZI9kDVHnYRD4oA4wpybsAQkIc1yl7dd+DZcAFfLcbM5JlYGxvldMO3x8aXRj0VZw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=OacS2tb5LrmapoEULF4hHeKeU526f7HVyBtkMzhw2vI=; b=qiNxM3fZM9QsHGUlHZQX3jfRkWZIeCByDJOJzrTraat5b8nhQRHRZr1E0LKImqNBr/P4x5njxCfhQAACea4ecxvcwB9+wA5qzCb8EB184J/SaNFG1zNVpiFam8h1y8qBOwVqZUz0Mp8MrSC7hTle6UdFZkD1Q7ZGn73LQMJ+rpNbqGrWlkhq0b584TLnPGc5em51I1/rV5I3Q2TRGsloSzNVNCNCjPjEPTe7Wx6Ym7yL+tZTfYM0IMf7vsaxoGDI9ClSaZ19pgZjqK28usAN4HTwPk+QCPLAycAQKJe8rgRX2xBDEV/uXpUafkt+8uud+/xsfDwcywcXssZE4kvuyw== Received: from BY3PR19MB4900.namprd19.prod.outlook.com (2603:10b6:a03:354::11) by DM5PR19MB0074.namprd19.prod.outlook.com (2603:10b6:4:65::37) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4951.18; Fri, 11 Feb 2022 23:26:56 +0000 Received: from BY3PR19MB4900.namprd19.prod.outlook.com ([fe80::51c:dd9e:9397:b17c]) by BY3PR19MB4900.namprd19.prod.outlook.com ([fe80::51c:dd9e:9397:b17c%4]) with mapi id 15.20.4975.014; Fri, 11 Feb 2022 23:26:56 +0000 Message-ID: Date: Fri, 11 Feb 2022 15:26:54 -0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.5.0 Subject: Re: [edk2-devel] **NOTICE** EDK II CI using Azure Pipelines Agents Failing To: devel@edk2.groups.io, michael.d.kinney@intel.com, Sean Brogan References: From: "Sean" In-Reply-To: X-TMN: [MuYOTxli7+zmN9O0Ntl0O3zzo5OCUp/k] X-ClientProxiedBy: MWHPR01CA0031.prod.exchangelabs.com (2603:10b6:300:101::17) To BY3PR19MB4900.namprd19.prod.outlook.com (2603:10b6:a03:354::11) Return-Path: spbrogan@outlook.com X-Microsoft-Original-Message-ID: <4f3ac7d0-0d83-da30-5a0e-35bcc7b8aed0@outlook.com> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: c4cab5d2-867b-442a-4dbc-08d9edb5fe3c X-MS-TrafficTypeDiagnostic: DM5PR19MB0074:EE_ X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: M1kZPCkIcuANKUU0PaghMY9/3FCCEt+gwiNKwjNIauBySXX34ur9bu9CIfvglMTfhgoD6swza3pQCKs3qDXKwuHYENXJeGGFZsSk4oODCxIzqrSmbpmiyPQ96f7Vw9WVgQrnE8JBWpTcSY0QC4SjqP55qAs65YFEY20QOJrXa0DayoYtBr5ro0wlMg/zHDT5hIyXjgyVdQ5HmxDG+jt4c2z99lKIUr+B1yr2p/Tnzrku9+MFcxBXJs4ARm5qHqRTQliWqjE8IqhmIr/OPrN7u8aGGO5JGaoRdbI+CXJXdauZya9+YAmsBTGfUXRxDLyXQy97OVF40FXd8nrEndXcfsjge56qeJnIo9EKIHQMSzhD13A6uT+mVlGoYGPF2qaEtHEJLBgtjZUIlNq9D0zwff/Sdm5BSP6mEDdJuAz8g+Snu3DXTuu+EYJeSWXNBauHkw5BK+hx0jJZ9Ea6PJRQQZbEtJso8PQ5oCk0lRVFJnVhxElM+OJwlfVXuZ+vzRa7/i3UKW94xV5Pvx4GxbzmvkiIKogbrY8/eUrph6OY7PWEFPkYF7ZSLgxx/sEQm2nBMumFeB/Myksc+9WdTAKzpveO68HLTF7iVJplYfN0vnc2Ck1h5arrOxyR+h+srmm5v9gHZGG4CNkOUj09BD3B2vuaSxu13DBzdsJHdfVk9Xk= X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?S0FJaFBoMVltU05udTYxL21sZGpzRytKK3djWW5ESyt4cTRxWWNTcmZDSjFy?= =?utf-8?B?dnJacEgreEkwTzFUeWZuTXJXVlovNExWN0hXQlVrdnBCRkt1SThxVjR2RFg5?= =?utf-8?B?MnhDbG5zN1Z3c1hjVGM3NENwSm42dlEwaE1qMm9aUnB4eFJESHIrZHE5L0No?= =?utf-8?B?T1ZQUmtCOENmbFRRZURUTnBwT2ZYVEE0NGJOVGx4TTREaWVmaEVRalpkTSto?= =?utf-8?B?czY2WG9VbmhIakRLVndXeDhIM1FVK1dFUWVtTHpFcXQrQmdmakk3MTR0NEE1?= =?utf-8?B?UkNVdFFLeC9MUHQzT0xnV2dMTmJxK0FmdnZ6WXV5SnNwRzNET3o1VHpkT0xm?= =?utf-8?B?VUplZHp3OTVxSFRKUlFRcTJBcjh3aG54c3JkUlhQMmxIVXZTMzhaN3ZFQTVQ?= =?utf-8?B?NTRXY3M0T1NObVRpbHJobkI2V0VOdUJJS0MzUDNweXBHbXd6V2VNZWFrb1di?= =?utf-8?B?K0grZ1VqeTNpTUpwalYvSVg2OGFPM05uNFM5R0ZBOStZdlUwVmtUZVd4bldz?= =?utf-8?B?WG5KbWxCUWZEUmhCZDlFUzVQK1diRzhZWEVaZ1hFMTZRMEp6cDh3QmtmYkQ4?= =?utf-8?B?UmJyRjFoclZGMTZlcENrYk01cnMvLzRqZkVLemh6U0p6VFE4V3JsNlRDT2Fw?= =?utf-8?B?NktqVTR0R3I3OG40d1VETjlUSjJnUVFDVDMxRGJXRXlPOHdiSDNWc3l3OUsr?= =?utf-8?B?c2h6eHNZTnJvQkRHY21VY1V2UFR5dFJLc2FTekxpaWhCbzN4SkJXYjU1elBC?= =?utf-8?B?L3ZmMkczTkE3YXdoWEY1RkNBOG5CLzNucmNOL2thSHoxb1o3T1FyTTQ2RVV5?= =?utf-8?B?K2RMbk5CaGI3eStUZENxVHRpWlBDTlM5N20ycUNJZ3g5T2Zvb0ROVmhpWkVN?= =?utf-8?B?N2RGRnhINlB4S2ZBVUp3d05RZklkanBxb1V3OEt1Zml3Ym4xQjljVWZJeFBF?= =?utf-8?B?cXNVREk2TVVGcWJzbEFsL1FWaGR2RmxyUjl3SUxRM3AvQmdLcDduVGpIcFFS?= =?utf-8?B?V2ZLSExRaXZNTzhWM2pGcS91ZHlnR3o1cDNxTW5aWGFVOHhWZjdsaEk3WFVR?= =?utf-8?B?aFF3SE82NjZDWHV3MHZLVnl5Ry80NlZ4Q2JkRHpLbXJ0eDRrWU9WZU90RHlt?= =?utf-8?B?cUtYWnEzVHROUXprVlZQTjYxTXl1aU1YV0JJWUloL2J5WnpPalhlVGtPZzhw?= =?utf-8?B?aVovSWxZb0xvOExZSHdrMFNJUVU0RnQ0cU5XakJxbnp6Vnl4UEs4TWllbG5C?= =?utf-8?B?WUJuaHJqT3VuRDkxMmFvdGlGT0lhakJYTDlMR2RQWjlxWTdXZz09?= X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: c4cab5d2-867b-442a-4dbc-08d9edb5fe3c X-MS-Exchange-CrossTenant-AuthSource: BY3PR19MB4900.namprd19.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Feb 2022 23:26:56.3457 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR19MB0074 Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Not a great update. I have opened a ticket with our system for supporting Azure DevOps Pipelines. There have been a number of projects hit by this in the past few days so I hope we can get to mitigation quickly. At the moment I don't have a clear answer why but for some reason our usage of the build services have gotten us flagged as "abusers". More digging around has indicated that pipeline builds (in general) are getting abused for crypto mining and other nefarious uses. In some cases vulnerabilities in published packages are being used to do this (point being not always the projects intent). I have noticed that some of our builds have taken nearly two hours to install NPM cspell package (generally it only takes 44 seconds). I have no idea if that is anything other than an anomaly. https://dev.azure.com/tianocore/edk2-ci/_build/results?buildId=40249&view=logs&j=4a97f94c-8e1f-5e76-68ba-50872604dd63&t=7346d0e9-c9fc-5f26-991e-d6f31d36ba1e Anyway, I'll update when i have more. Thanks Sean On 2/11/2022 8:16 AM, Michael D Kinney wrote: > Hello, > > The EDK II CI using Azure Pipelines Agents is not working right now. > > First failure occurred on 2-10-2022. > > The issue is being worked. > > We will send daily status updates until the issue is resolved. > > We are currently in a hard freeze for the next EDK II stable tag. > We will monitor status closely and adjust the release date if needed. > > Thank you for your patience. > > Mike > > > > >