public inbox for devel@edk2.groups.io
 help / color / mirror / Atom feed
From: "Bob Feng" <bob.c.feng@intel.com>
To: "devel@edk2.groups.io" <devel@edk2.groups.io>,
	"Park, Aiden" <aiden.park@intel.com>
Cc: "Feng, Bob C" <bob.c.feng@intel.com>
Subject: Re: [PATCH] [edk2/BaseTools] edksetup.bat stuck on unicode locale Windows
Date: Wed, 18 Dec 2019 05:23:09 +0000	[thread overview]
Message-ID: <08650203BA1BD64D8AD9B6D5D74A85D16158434D@SHSMSX104.ccr.corp.intel.com> (raw)
In-Reply-To: <BN8PR11MB3619EFB859031B0C4F994B208E5A0@BN8PR11MB3619.namprd11.prod.outlook.com>

Hi Aiden,

If set kwargs["stdout"] = sys.stdout, then stdout and stderr messages from sub process will directly write to sys.stdout.
I think this patch works because sys.stdout.encoding is the correct encoding. So what do you think if we fix this Non-Ascii issue by
Changing the line of
message = stdout.decode(encoding='utf-8', errors='ignore') #for compatibility in python 2 and 3
to 
message = stdout.decode(encoding=sys.stdout.encoding , errors='ignore') #for compatibility in python 2 and 3

Otherwise, I think this patch may need to take care of the code after this line kwargs["stdout"] = sys.stdout, I mean
p.communicate() will return empty string after your change, and the following code would be useless. 

Thanks,
Bob

-----Original Message-----
From: devel@edk2.groups.io [mailto:devel@edk2.groups.io] On Behalf Of Park, Aiden
Sent: Thursday, December 12, 2019 6:43 AM
To: devel@edk2.groups.io
Subject: [edk2-devel] [PATCH] [edk2/BaseTools] edksetup.bat stuck on unicode locale Windows

This issue happens under two conditions.
  1. Unicode language environment in Windows
  2. Call 'edksetup.bat forcerebuild' with python subprocess.call()

Steps to reproduce
  C:\edk2>python
  Python 2.7.12 (v2.7.12:d33e0cf91556, Jun 27 2016, 15:24:40)
  Type help, copyright, credits or license for more information.
  >>> import subprocess
  >>> subprocess.call(['edksetup.bat', 'forcerebuild'])

  The edksetup.bat stuck at 'nmake cleanall'.

One of multi-threads is on deadlock when python handles stdout and stderr in a subprocess pipe only if the outputs include unicode chars.
Only stderr will be handled in the pipe same as a single thread call.

Reported in Slim Bootloader.
  https://github.com/slimbootloader/slimbootloader/issues/478
Local fix has been made in Slim Bootloader.
  https://github.com/slimbootloader/slimbootloader/pull/490

Signed-off-by: Aiden Park <aiden.park@intel.com>
---
 BaseTools/Source/C/Makefiles/NmakeSubdirs.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/BaseTools/Source/C/Makefiles/NmakeSubdirs.py b/BaseTools/Source/C/Makefiles/NmakeSubdirs.py
index 356f5ac..c77bfb0 100644
--- a/BaseTools/Source/C/Makefiles/NmakeSubdirs.py
+++ b/BaseTools/Source/C/Makefiles/NmakeSubdirs.py
@@ -33,7 +33,7 @@ def RunCommand(WorkDir=None, *Args, **kwargs):
     if "stderr" not in kwargs:
         kwargs["stderr"] = subprocess.STDOUT
     if "stdout" not in kwargs:
-        kwargs["stdout"] = subprocess.PIPE
+        kwargs["stdout"] = sys.stdout
     p = subprocess.Popen(Args, cwd=WorkDir, stderr=kwargs["stderr"], stdout=kwargs["stdout"])
     stdout, stderr = p.communicate()
     message = ""
--
2.10.2.windows.1




  reply	other threads:[~2019-12-18  5:23 UTC|newest]

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-12-11 22:42 [PATCH] [edk2/BaseTools] edksetup.bat stuck on unicode locale Windows Park, Aiden
2019-12-18  5:23 ` Bob Feng [this message]
2019-12-20  0:23   ` aiden.park
2019-12-20  6:12     ` Bob Feng
2019-12-20  7:03       ` Park, Aiden
2019-12-20  8:19         ` Bob Feng
2019-12-20 18:08           ` Park, Aiden

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-list from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=08650203BA1BD64D8AD9B6D5D74A85D16158434D@SHSMSX104.ccr.corp.intel.com \
    --to=devel@edk2.groups.io \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox