From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by mx.groups.io with SMTP id smtpd.web09.191.1572561879248158243 for ; Thu, 31 Oct 2019 15:44:39 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: intel.com, ip: 192.55.52.115, mailfrom: ashley.e.desimone@intel.com) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 31 Oct 2019 15:44:38 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.68,253,1569308400"; d="scan'208";a="203686975" Received: from orsmsx102.amr.corp.intel.com ([10.22.225.129]) by orsmga003.jf.intel.com with ESMTP; 31 Oct 2019 15:44:38 -0700 Received: from orsmsx161.amr.corp.intel.com (10.22.240.84) by ORSMSX102.amr.corp.intel.com (10.22.225.129) with Microsoft SMTP Server (TLS) id 14.3.439.0; Thu, 31 Oct 2019 15:44:38 -0700 Received: from orsmsx116.amr.corp.intel.com ([169.254.7.79]) by ORSMSX161.amr.corp.intel.com ([169.254.4.146]) with mapi id 14.03.0439.000; Thu, 31 Oct 2019 15:44:38 -0700 From: "Desimone, Ashley E" To: "Desimone, Nathaniel L" , "devel@edk2.groups.io" CC: "Pandya, Puja" Subject: Re: [edk2-staging/EdkRepo] [PATCH] EdkRepo: Python 2.x deprecation warning in pip breaks EdkRepoInstaller Thread-Topic: [edk2-staging/EdkRepo] [PATCH] EdkRepo: Python 2.x deprecation warning in pip breaks EdkRepoInstaller Thread-Index: AQHVj3yIVHwOMTCvT0KhzwXK7e4b8Kd1Wlsw Date: Thu, 31 Oct 2019 22:44:37 +0000 Message-ID: <4CF3A9EB60ABDA47BE7821A4DA3A0A3353C93A6A@ORSMSX116.amr.corp.intel.com> References: <20191030234815.4506-1-nathaniel.l.desimone@intel.com> In-Reply-To: <20191030234815.4506-1-nathaniel.l.desimone@intel.com> Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiOGJhZjAxZWItYzgxNi00ZGZmLThmN2UtNGU3NmQ2NzkyNTJhIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoib2Vpck9DVGxMTjF0UGFMVDByU1BqU0Q4K2ZkWjFldHNVcXRVSkYzalhLS3d6Ujc3cEwyWmlZT2U4QTlBZ1hXVSJ9 x-ctpclassification: CTP_NT dlp-product: dlpe-windows dlp-version: 11.2.0.6 dlp-reaction: no-action x-originating-ip: [10.22.254.139] MIME-Version: 1.0 Return-Path: ashley.e.desimone@intel.com Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Reviewed-by: Ashley DeSimone -----Original Message----- From: Desimone, Nathaniel L=20 Sent: Wednesday, October 30, 2019 4:48 PM To: devel@edk2.groups.io Cc: Desimone, Ashley E ; Pandya, Puja Subject: [edk2-staging/EdkRepo] [PATCH] EdkRepo: Python 2.x deprecation war= ning in pip breaks EdkRepoInstaller Newer versions of pip add the following output in addition to the JSON outp= ut when running on Python 2.7: DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020= . Please upgrade your Python as Python 2.7 won't be maintained after that d= ate. A future version of pip will drop support for Python 2.7. This breaks machine parsing of the JSON output from pip. Adding logic to strip the deprecation warning to fix this issue. Signed-off-by: Nate DeSimone Cc: Ashley E Desimone Cc: Puja Pandya --- .../EdkRepoInstaller/PythonOperations.cs | 37 +++++++++++++++++-- 1 file changed, 34 insertions(+), 3 deletions(-) diff --git a/edkrepo_installer/EdkRepoInstaller/PythonOperations.cs b/edkre= po_installer/EdkRepoInstaller/PythonOperations.cs index f96502c..73d5bec 100644 --- a/edkrepo_installer/EdkRepoInstaller/PythonOperations.cs +++ b/edkrepo_installer/EdkRepoInstaller/PythonOperations.cs @@ -600,17 +600,48 @@ namespace TianoCore.EdkRepoInstaller return PythonPackages; } + private static strin= g SanitizePipOutput(string PipOutput)+ {+ StringBuilder S= anitized =3D new StringBuilder();+ IEnumerable PipLines = =3D PipOutput.SplitLines();+ foreach(string line in PipLines)+ = {+ if(line.StartsWith("DEPRECATION:"))+ = {+ continue;+ }+ if (= string.IsNullOrWhiteSpace(line))+ {+ cont= inue;+ }+ Sanitized.Append(line.Trim());+ = Sanitized.Append("\r\n");+ }+ return Sani= tized.ToString().Trim();+ }+ public static List GetInstalledPythonPackages(string PythonPath) { List= PythonPackages =3D new List(); S= ilentProcess.StdoutDataCapture dataCapture =3D new SilentProcess.StdoutData= Capture(); SilentProcess process =3D SilentProcess.StartConsole= ProcessSilently(PythonPath, "-m pip list --format=3D\"json\" --no-index", d= ataCapture.DataReceivedHandler); process.WaitForExit();+ = bool TryLegacy =3D true; if (process.ExitCode =3D=3D 0) = {- PythonPackages =3D ParseJsonPipList(dataCapture= .GetData());+ try+ {+ Pyth= onPackages =3D ParseJsonPipList(SanitizePipOutput(dataCapture.GetData()));+= TryLegacy =3D false;+ }+ = catch(Exception e)+ {+ InstallLogger.Log(= "Error occurred while trying to parse pip JSON:");+ Inst= allLogger.Log(e.ToString());+ InstallLogger.Log("Falling= back to legacy mode");+ } }- else+ = if(TryLegacy) { // //= Older versions of pip don't support the --format flag, parse the legacy fo= rmat@@ -620,7 +651,7 @@ namespace TianoCore.EdkRepoInstaller process.WaitForExit(); if (process.ExitCod= e =3D=3D 0) {- PythonPackages =3D ParseL= egacyPipList(dataCapture.GetData());+ PythonPackages =3D= ParseLegacyPipList(SanitizePipOutput(dataCapture.GetData())); = } } return PythonPackages;--=20 2.23.0.windows.1