From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=209.132.183.28; helo=mx1.redhat.com; envelope-from=lersek@redhat.com; receiver=edk2-devel@lists.01.org Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id B2F2421B02822 for ; Tue, 8 Jan 2019 08:25:34 -0800 (PST) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 2902158E29; Tue, 8 Jan 2019 16:25:34 +0000 (UTC) Received: from lacos-laptop-7.usersys.redhat.com (ovpn-120-197.rdu2.redhat.com [10.10.120.197]) by smtp.corp.redhat.com (Postfix) with ESMTP id 144E95D75D; Tue, 8 Jan 2019 16:25:32 +0000 (UTC) To: "Gao, Liming" Cc: Gary Lin , "Kinney, Michael D" , "edk2-devel@lists.01.org" References: <4A89E2EF3DFEDB4C8BFDE51014F606A14E39130F@SHSMSX104.ccr.corp.intel.com> <20181228103951.GN4206@GaryWorkstation> <4A89E2EF3DFEDB4C8BFDE51014F606A14E39F4E3@SHSMSX104.ccr.corp.intel.com> <3cd9946d-12e4-489e-605d-df7060f958e7@redhat.com> <4A89E2EF3DFEDB4C8BFDE51014F606A14E3A904F@SHSMSX152.ccr.corp.intel.com> From: Laszlo Ersek Message-ID: Date: Tue, 8 Jan 2019 17:25:31 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: <4A89E2EF3DFEDB4C8BFDE51014F606A14E3A904F@SHSMSX152.ccr.corp.intel.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Tue, 08 Jan 2019 16:25:34 +0000 (UTC) Subject: Re: [RFC] Edk2 BaseTools Python3 Migration Update X-BeenThere: edk2-devel@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: EDK II Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jan 2019 16:25:34 -0000 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit Hi Liming, On 01/04/19 04:29, Gao, Liming wrote: > Laszlo: > This issue has been fixed in edk2 master. I just cherry pick those > fixes from edk2 master to my Python3 branch > (https://github.com/lgao4/edk2/tree/Python3). At commit 90d8b4834fd1 ("BaseTools: Reset FdsGlobalVariable", 2019-01-04): (a) My regression tests using python-2.7.5-69.el7_5.x86_64 were all successful (build and boot, plus S3 wherever applicable). They covered IA32, IA32X64, and X64 OVMF, with/without SMM, and with Fedora and some Windows guests. They also covered ArmVirtQemu on aarch64 KVM. (b) For testing the Python3 enablement, I used a "RHEL8 Beta" virtual machine. (The Python situation in RHEL8 is documented in the following blog post: .) My testing with Python3 failed. I did the following, in a clean checkout of your repo/branch, and a clean shell environment: # export PYTHON3_ENABLE=TRUE # source edksetup.sh # nice make -C "$EDK_TOOLS_PATH" -j3 The final output was: > ====================================================================== > ERROR: testRandomDataCycles (TianoCompress.Tests) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/root/liming/BaseTools/Tests/TianoCompress.py", line 66, in testRandomDataCycles > self.compressionTestCycle(data) > File "/root/liming/BaseTools/Tests/TianoCompress.py", line 39, in compressionTestCycle > self.WriteTmpFile('input', data) > File "/root/liming/BaseTools/Tests/TestTools.py", line 154, in WriteTmpFile > f.write(data) > File "/usr/lib64/python3.6/encodings/iso8859_2.py", line 19, in encode > return codecs.charmap_encode(input,self.errors,encoding_table)[0] > UnicodeEncodeError: 'charmap' codec can't encode character '\xc0' in position 27: character maps to > > ---------------------------------------------------------------------- (The backtrace mentions "iso8859_2.py" because I happen to use LC_CTYPE=hu_HU.ISO8859-2 in my locale settings.) Either way, I think the issue is that an object ("data") is being considered a string, rather than a byte array. (c) To re-iterate my earlier request, would it be possible to set PYTHON_COMMAND externally (in addition to PYTHON3_ENABLE=TRUE)? Because, once we package a new version of edk2 for RHEL8, we'd like to set PYTHON_COMMAND to "/usr/libexec/platform-python", and not to the auto-detected "/usr/bin/python3'. (In fact, "/usr/bin/python3" could be missing from the restricted package build environment.) See the blog post I referenced above -- one of its takeaways is, "use platform-python if you are writing system/admin code for RHEL 8". In other words, when users build upstream edk2 on a RHEL8 machine, it's fine if edk2 detects and uses /usr/bin/python3. However, when the edk2 package of the distro itself is built, it should be possible to direct edk2 towards "/usr/libexec/platform-python". Thank you, Laszlo