From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by mx.groups.io with SMTP id smtpd.web12.7011.1576646597182211520 for ; Tue, 17 Dec 2019 21:23:17 -0800 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: intel.com, ip: 134.134.136.65, mailfrom: bob.c.feng@intel.com) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 17 Dec 2019 21:23:16 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.69,328,1571727600"; d="scan'208";a="205724934" Received: from fmsmsx108.amr.corp.intel.com ([10.18.124.206]) by orsmga007.jf.intel.com with ESMTP; 17 Dec 2019 21:23:16 -0800 Received: from shsmsx152.ccr.corp.intel.com (10.239.6.52) by FMSMSX108.amr.corp.intel.com (10.18.124.206) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 17 Dec 2019 21:23:12 -0800 Received: from shsmsx104.ccr.corp.intel.com ([169.254.5.90]) by SHSMSX152.ccr.corp.intel.com ([169.254.6.222]) with mapi id 14.03.0439.000; Wed, 18 Dec 2019 13:23:10 +0800 From: "Bob Feng" To: "devel@edk2.groups.io" , "Park, Aiden" CC: "Feng, Bob C" Subject: Re: [PATCH] [edk2/BaseTools] edksetup.bat stuck on unicode locale Windows Thread-Topic: [PATCH] [edk2/BaseTools] edksetup.bat stuck on unicode locale Windows Thread-Index: AdWwdDLVp9xBmfjoQ6mo1EN+7xsRzgE3xbnA Date: Wed, 18 Dec 2019 05:23:09 +0000 Message-ID: <08650203BA1BD64D8AD9B6D5D74A85D16158434D@SHSMSX104.ccr.corp.intel.com> References: In-Reply-To: Accept-Language: zh-CN, en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.239.127.40] MIME-Version: 1.0 Return-Path: bob.c.feng@intel.com Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Aiden, If set kwargs["stdout"] =3D sys.stdout, then stdout and stderr messages fr= om sub process will directly write to sys.stdout. I think this patch works because sys.stdout.encoding is the correct encodi= ng. So what do you think if we fix this Non-Ascii issue by Changing the line of message =3D stdout.decode(encoding=3D'utf-8', errors=3D'ignore') #for comp= atibility in python 2 and 3 to=20 message =3D stdout.decode(encoding=3Dsys.stdout.encoding , errors=3D'ignor= e') #for compatibility in python 2 and 3 Otherwise, I think this patch may need to take care of the code after this= line kwargs["stdout"] =3D sys.stdout, I mean p.communicate() will return empty string after your change, and the follow= ing code would be useless.=20 Thanks, Bob -----Original Message----- From: devel@edk2.groups.io [mailto:devel@edk2.groups.io] On Behalf Of Park= , Aiden Sent: Thursday, December 12, 2019 6:43 AM To: devel@edk2.groups.io Subject: [edk2-devel] [PATCH] [edk2/BaseTools] edksetup.bat stuck on unico= de locale Windows This issue happens under two conditions. 1. Unicode language environment in Windows 2. Call 'edksetup.bat forcerebuild' with python subprocess.call() Steps to reproduce C:\edk2>python Python 2.7.12 (v2.7.12:d33e0cf91556, Jun 27 2016, 15:24:40) Type help, copyright, credits or license for more information. >>> import subprocess >>> subprocess.call(['edksetup.bat', 'forcerebuild']) The edksetup.bat stuck at 'nmake cleanall'. One of multi-threads is on deadlock when python handles stdout and stderr = in a subprocess pipe only if the outputs include unicode chars. Only stderr will be handled in the pipe same as a single thread call. Reported in Slim Bootloader. https://github.com/slimbootloader/slimbootloader/issues/478 Local fix has been made in Slim Bootloader. https://github.com/slimbootloader/slimbootloader/pull/490 Signed-off-by: Aiden Park --- BaseTools/Source/C/Makefiles/NmakeSubdirs.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/BaseTools/Source/C/Makefiles/NmakeSubdirs.py b/BaseTools/Sour= ce/C/Makefiles/NmakeSubdirs.py index 356f5ac..c77bfb0 100644 --- a/BaseTools/Source/C/Makefiles/NmakeSubdirs.py +++ b/BaseTools/Source/C/Makefiles/NmakeSubdirs.py @@ -33,7 +33,7 @@ def RunCommand(WorkDir=3DNone, *Args, **kwargs): if "stderr" not in kwargs: kwargs["stderr"] =3D subprocess.STDOUT if "stdout" not in kwargs: - kwargs["stdout"] =3D subprocess.PIPE + kwargs["stdout"] =3D sys.stdout p =3D subprocess.Popen(Args, cwd=3DWorkDir, stderr=3Dkwargs["stderr"]= , stdout=3Dkwargs["stdout"]) stdout, stderr =3D p.communicate() message =3D "" -- 2.10.2.windows.1