From: "Feng, Bob C" <bob.c.feng@intel.com>
To: Laszlo Ersek <lersek@redhat.com>
Cc: "edk2-devel@lists.01.org" <edk2-devel@lists.01.org>,
"Gao, Liming" <liming.gao@intel.com>
Subject: Re: [Patch 00/33] BaseTools python3 migration patch set
Date: Mon, 28 Jan 2019 02:33:01 +0000 [thread overview]
Message-ID: <08650203BA1BD64D8AD9B6D5D74A85D160074FDA@SHSMSX101.ccr.corp.intel.com> (raw)
In-Reply-To: <cab4fed6-4c5d-94a9-b29f-da41ad7f320e@redhat.com>
Hi Laszlo,
Thanks for your testing and detailed testing result. I'll check the issue happened in (a2) and provide a patch for it.
Thanks,
Bob
-----Original Message-----
From: Laszlo Ersek [mailto:lersek@redhat.com]
Sent: Saturday, January 26, 2019 2:18 AM
To: Feng, Bob C <bob.c.feng@intel.com>
Cc: edk2-devel@lists.01.org; Gao, Liming <liming.gao@intel.com>
Subject: Re: [edk2] [Patch 00/33] BaseTools python3 migration patch set
On 01/25/19 10:42, Feng, Bob C wrote:
> [...]
>
> -----Original Message-----
> From: Laszlo Ersek [mailto:lersek@redhat.com]
> Sent: Friday, January 25, 2019 4:57 PM
> To: Feng, Bob C <bob.c.feng@intel.com>
> Cc: edk2-devel@lists.01.org; Gao, Liming <liming.gao@intel.com>
> Subject: Re: [edk2] [Patch 00/33] BaseTools python3 migration patch
> set
>
> [...]
>
> On 01/25/19 05:55, Feng, Bob C wrote:
>> BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=55
>>
>> This patch set is to enable python3 on BaseTools. Basetools code will
>> be compatible with both python3 and python2.
>>
>> We will have two envs PYTHON3_ENABLE and PYTHON_COMMAND. The behavior
>> can be combined as the below to support this usage.
>> If user wants the specific python interpreter, he only needs to set
>> PYTHON_COMMAND env.
>> If PYTHON3_ENABLE is set, PYTHON_COMMAND will be set to the found one
>> by edk2 scripts based on PYTHON3_ENABLE value.
>> If PYTHON3_ENABLE is not set, but PYTHON_COMMAND is set, then
>> PYTHON_COMMAND will be used to run python script. No version check
>> here.
>> If PYTHON3_ENABLE is not set, but PYTHON_COMMAND is not set,
>> PYTHON_COMMAND will be set to the high version python installed in
>> OS.
I ran the following tests, at commit 41c0616fc081 ("BaseTools: Eot tool
Python3 adaption", 2019-01-25). Each test was performed in a clean tree (after running "git clean -ffdx") and clean environment (I re-sourced "edksetup.sh" for each test in separation). In addition, the base tools were rebuilt (again from a clean tree) for each test, with the following command [1]:
nice make -C "$EDK_TOOLS_PATH" -j $(getconf _NPROCESSORS_ONLN)
(a) On my RHEL7.5 Workstation laptop, I have both the system-level python packages installed (python-2.7.5-69.el7_5.x86_64), and the extra
python-3.4 stuff from EPEL-7 (python34-3.4.9-1.el7.x86_64).
(a1) Didn't set either PYTHON3_ENABLE or PYTHON_COMMAND. The build utility picked
PYTHON_COMMAND = /usr/bin/python3.4
and I successfully built OvmfPkg for IA32, IA32X64, and X64; also ArmVirtQemu for AARCH64. The built firmware images passed a smoke test too.
(a2) I removed all the python34 packages (and the dependent packages) from my laptop. Didn't set either of PYTHON3_ENABLE and PYTHON_COMMAND.
(This is the configuration what a "normal" RHEL7 environment would
provide.)
In this case, rebuilding the base tools [1] failed, in the testing
phase:
> ======================================================================
> FAIL: test_Ecc_CParser4_CLexer (CheckPythonSyntax.Tests)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File ".../BaseTools/Tests/CheckPythonSyntax.py", line 55, in <lambda>
> newmethod = lambda self: self.SingleFileTest(filename)
> File ".../BaseTools/Tests/CheckPythonSyntax.py", line 33, in SingleFileTest
> self.fail('syntax error: %s, Error is %s' % (filename, str(e)))
> AssertionError: syntax error: .../BaseTools/Source/Python/Ecc/CParser4/CLexer.py, Error is File ".../BaseTools/Source/Python/Ecc/CParser4/CLexer.py", line 590
> def __init__(self, input=None, output:TextIO = sys.stdout):
> ^
> SyntaxError: invalid syntax
>
>
> ======================================================================
> FAIL: test_Ecc_CParser4_CListener (CheckPythonSyntax.Tests)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File ".../BaseTools/Tests/CheckPythonSyntax.py", line 55, in <lambda>
> newmethod = lambda self: self.SingleFileTest(filename)
> File ".../BaseTools/Tests/CheckPythonSyntax.py", line 33, in SingleFileTest
> self.fail('syntax error: %s, Error is %s' % (filename, str(e)))
> AssertionError: syntax error: .../BaseTools/Source/Python/Ecc/CParser4/CListener.py, Error is File ".../BaseTools/Source/Python/Ecc/CParser4/CListener.py", line 35
> def enterTranslation_unit(self, ctx:CParser.Translation_unitContext):
> ^
> SyntaxError: invalid syntax
>
>
> ======================================================================
> FAIL: test_Ecc_CParser4_CParser (CheckPythonSyntax.Tests)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File ".../BaseTools/Tests/CheckPythonSyntax.py", line 55, in <lambda>
> newmethod = lambda self: self.SingleFileTest(filename)
> File ".../BaseTools/Tests/CheckPythonSyntax.py", line 33, in SingleFileTest
> self.fail('syntax error: %s, Error is %s' % (filename, str(e)))
> AssertionError: syntax error: .../BaseTools/Source/Python/Ecc/CParser4/CParser.py, Error is File ".../BaseTools/Source/Python/Ecc/CParser4/CParser.py", line 744
> def __init__(self, input:TokenStream, output:TextIO = sys.stdout):
> ^
> SyntaxError: invalid syntax
>
>
> ======================================================================
> FAIL: test_Eot_CParser4_CLexer (CheckPythonSyntax.Tests)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File ".../BaseTools/Tests/CheckPythonSyntax.py", line 55, in <lambda>
> newmethod = lambda self: self.SingleFileTest(filename)
> File ".../BaseTools/Tests/CheckPythonSyntax.py", line 33, in SingleFileTest
> self.fail('syntax error: %s, Error is %s' % (filename, str(e)))
> AssertionError: syntax error: .../BaseTools/Source/Python/Eot/CParser4/CLexer.py, Error is File ".../BaseTools/Source/Python/Eot/CParser4/CLexer.py", line 591
> def __init__(self, input=None, output:TextIO = sys.stdout):
> ^
> SyntaxError: invalid syntax
>
>
> ======================================================================
> FAIL: test_Eot_CParser4_CListener (CheckPythonSyntax.Tests)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File ".../BaseTools/Tests/CheckPythonSyntax.py", line 55, in <lambda>
> newmethod = lambda self: self.SingleFileTest(filename)
> File ".../BaseTools/Tests/CheckPythonSyntax.py", line 33, in SingleFileTest
> self.fail('syntax error: %s, Error is %s' % (filename, str(e)))
> AssertionError: syntax error: .../BaseTools/Source/Python/Eot/CParser4/CListener.py, Error is File ".../BaseTools/Source/Python/Eot/CParser4/CListener.py", line 35
> def enterTranslation_unit(self, ctx:CParser.Translation_unitContext):
> ^
> SyntaxError: invalid syntax
>
>
> ======================================================================
> FAIL: test_Eot_CParser4_CParser (CheckPythonSyntax.Tests)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File ".../BaseTools/Tests/CheckPythonSyntax.py", line 55, in <lambda>
> newmethod = lambda self: self.SingleFileTest(filename)
> File ".../BaseTools/Tests/CheckPythonSyntax.py", line 33, in SingleFileTest
> self.fail('syntax error: %s, Error is %s' % (filename, str(e)))
> AssertionError: syntax error: .../BaseTools/Source/Python/Eot/CParser4/CParser.py, Error is File ".../BaseTools/Source/Python/Eot/CParser4/CParser.py", line 744
> def __init__(self, input:TokenStream, output:TextIO = sys.stdout):
> ^
> SyntaxError: invalid syntax
>
>
> ----------------------------------------------------------------------
> Ran 270 tests in 0.938s
>
> FAILED (failures=6)
I didn't continue testing on my RHEL7 laptop beyond this point.
(b) RHEL-8 virtual machine, with "/usr/bin/python3.6" from python36-3.6.6-18.el8.x86_64, and "/usr/libexec/platform-python" from platform-python-3.6.8-1.el8.x86_64.
(b1) Didn't set either PYTHON3_ENABLE or PYTHON_COMMAND. The build utility picked
PYTHON_COMMAND = /usr/bin/python3.6
and I successfully built OvmfPkg for IA32, IA32X64, and X64. (I don't have a cross-compiler installed in this environment yet, nor a RHEL8
aarch64 KVM guest, so I couldn't test ArmVirtQemu for now).
(b2) I set PYTHON_COMMAND to "/usr/libexec/platform-python". Didn't set PYTHON3_ENABLE. The same builds as in (b1) succeeded.
Thus, what seems to be missing is a little bit of python2 compatibility, in (a2).
Thanks,
Laszlo
next prev parent reply other threads:[~2019-01-28 2:33 UTC|newest]
Thread overview: 41+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-01-25 4:55 [Patch 00/33] BaseTools python3 migration patch set Feng, Bob C
2019-01-25 4:55 ` [Patch 01/33] BaseTool:Rename xrange() to range() Feng, Bob C
2019-01-25 4:55 ` [Patch 02/33] BaseTools:use iterate list to replace the itertools Feng, Bob C
2019-01-25 4:55 ` [Patch 03/33] BaseTools: Rename iteritems to items Feng, Bob C
2019-01-25 4:55 ` [Patch 04/33] BaseTools: replace get_bytes_le() to bytes_le Feng, Bob C
2019-01-25 4:55 ` [Patch 05/33] BaseTools: use OrderedDict instead of sdict Feng, Bob C
2019-01-25 4:55 ` [Patch 06/33] BaseTools: nametuple not have verbose parameter in python3 Feng, Bob C
2019-01-25 4:56 ` [Patch 07/33] BaseTools: Remove unnecessary super function Feng, Bob C
2019-01-25 4:56 ` [Patch 08/33] BaseTools: replace long by int Feng, Bob C
2019-01-25 4:56 ` [Patch 09/33] BaseTools:Solve the data sorting problem use python3 Feng, Bob C
2019-01-25 4:56 ` [Patch 10/33] BaseTools: Update argparse arguments since it not have version now Feng, Bob C
2019-01-25 4:56 ` [Patch 11/33] BaseTools:Similar to octal data rectification Feng, Bob C
2019-01-25 4:56 ` [Patch 12/33] BaseTools/UPT:merge UPT Tool use Python2 and Python3 Feng, Bob C
2019-01-25 4:56 ` [Patch 13/33] BaseTools: update Test scripts support python3 Feng, Bob C
2019-01-25 4:56 ` [Patch 14/33] BaseTools/Scripts: Porting PackageDocumentTools code to use Python3 Feng, Bob C
2019-01-25 4:56 ` [Patch 15/33] Basetools: It went wrong when use os.linesep Feng, Bob C
2019-01-25 4:56 ` [Patch 16/33] BaseTools:Fv BaseAddress must set If it not set Feng, Bob C
2019-01-25 4:56 ` [Patch 17/33] BaseTools: Make sure AllPcdList valid Feng, Bob C
2019-01-25 4:56 ` [Patch 18/33] BaseTools:TestTools character encoding issue Feng, Bob C
2019-01-25 4:56 ` [Patch 19/33] BaseTools:Double carriage return inserted from Trim.py on Python3 Feng, Bob C
2019-01-25 4:56 ` [Patch 20/33] BaseTools:File open failed for VPD MapFile Feng, Bob C
2019-01-25 4:56 ` [Patch 21/33] BaseTools: change the Division Operator Feng, Bob C
2019-01-25 4:56 ` [Patch 22/33] BaseTools:There is extra blank line in datalog Feng, Bob C
2019-01-25 4:56 ` [Patch 23/33] BaseTools: Similar to octal data rectification Feng, Bob C
2019-01-25 4:56 ` [Patch 24/33] BaseTools: Update windows and linux run scripts file to use Python3 Feng, Bob C
2019-01-25 4:56 ` [Patch 25/33] BaseTools:Update build tool to print python version information Feng, Bob C
2019-01-25 4:56 ` [Patch 26/33] BaseTools:Linux Python highest version check Feng, Bob C
2019-01-25 4:56 ` [Patch 27/33] BaseTools: Update PYTHON env to PYTHON_COMMAND Feng, Bob C
2019-01-25 4:56 ` [Patch 28/33] BaseTools:Fixed Rsa issue and a set define issue Feng, Bob C
2019-01-25 4:56 ` [Patch 29/33] BaseTools:ord() don't match in py2 and py3 Feng, Bob C
2019-01-25 4:56 ` [Patch 30/33] BaseTools: the list and iterator translation Feng, Bob C
2019-01-25 4:56 ` [Patch 31/33] BaseTools: Handle the bytes and str difference Feng, Bob C
2019-01-25 4:56 ` [Patch 32/33] BaseTools: ECC tool Python3 adaption Feng, Bob C
2019-01-25 4:56 ` [Patch 33/33] BaseTools: Eot " Feng, Bob C
2019-01-25 8:56 ` [Patch 00/33] BaseTools python3 migration patch set Laszlo Ersek
2019-01-25 9:42 ` Feng, Bob C
2019-01-25 18:18 ` Laszlo Ersek
2019-01-28 2:33 ` Feng, Bob C [this message]
2019-01-28 10:35 ` Feng, Bob C
2019-01-28 13:48 ` Laszlo Ersek
2019-01-29 2:15 ` Feng, Bob C
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-list from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=08650203BA1BD64D8AD9B6D5D74A85D160074FDA@SHSMSX101.ccr.corp.intel.com \
--to=devel@edk2.groups.io \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox