From: Laszlo Ersek <lersek@redhat.com>
To: "Feng, Bob C" <bob.c.feng@intel.com>
Cc: "edk2-devel@lists.01.org" <edk2-devel@lists.01.org>,
"Gao, Liming" <liming.gao@intel.com>
Subject: Re: [Patch 00/33] BaseTools python3 migration patch set
Date: Mon, 28 Jan 2019 14:48:27 +0100 [thread overview]
Message-ID: <c6f914d3-f3b1-e2c6-5860-24cb92ca4137@redhat.com> (raw)
In-Reply-To: <08650203BA1BD64D8AD9B6D5D74A85D160076553@SHSMSX101.ccr.corp.intel.com>
Hi Bob,
On 01/28/19 11:35, Feng, Bob C wrote:
> Hi Laszlo,
>
> I sent out 2 patches to fix this issue. And I also pushed the patches to the repo https://github.com/BobCF/edk2.git branch py3basetools
I fetched your branch, and I found two new (fast-forward-able) commits:
$ git log --oneline --reverse 41c0616fc081..ec9e63692453
fb566a8835fa BaseTools: Fixed Eot python2 incompatible issue.
ec9e63692453 BaseTools: Fixed Ecc python2 incompatible issue.
However, the patches that you sent out:
[edk2] [Patch v2 32/33] BaseTools: ECC tool Python3 adaption
[edk2] [Patch v2 33/33] BaseTools: Eot tool Python3 adaption
appear different; I think you squashed the above incremental fixes into
the corresponding original (v1) patches. Is that correct?
I prefer to test a series in the same exact state as it would be
committed; otherwise there could be an issue with the rebase / squashing
(possibly further tweaks), and it wouldn't be correct to apply my
Tested-by to such a version.
In addition, you didn't post the updated (v2) patches #32 and #33 in
response to the v1 blurb:
[edk2] [Patch 00/33] BaseTools python3 migration patch set
and as a result they will drift apart on the list, from each other and
also from the containing set.
In addition, it's a good practice to keep branches unchanged once
they've been posted to the list; further iterations should be named
py3basetools_v2, py3basetools_v3 etc. This lets people return to earlier
iterations, compare versions, associate branches with postings, and so on.
Assuming the v2 32/33, v2 33/33 on-list series is what I should test,
please push *precisely* that series to your repo, under branch name
"py3basetools_v2". I'd also suggest resetting branch "py3basetools" to
the original v1 posting status, that is, commit 41c0616fc081.
Thanks
Laszlo
> -----Original Message-----
> From: Feng, Bob C
> Sent: Monday, January 28, 2019 10:33 AM
> To: 'Laszlo Ersek' <lersek@redhat.com>
> Cc: edk2-devel@lists.01.org; Gao, Liming <liming.gao@intel.com>
> Subject: RE: [edk2] [Patch 00/33] BaseTools python3 migration patch set
>
> Hi Laszlo,
>
> Thanks for your testing and detailed testing result. I'll check the issue happened in (a2) and provide a patch for it.
>
> Thanks,
> Bob
>
> -----Original Message-----
> From: Laszlo Ersek [mailto:lersek@redhat.com]
> Sent: Saturday, January 26, 2019 2:18 AM
> To: Feng, Bob C <bob.c.feng@intel.com>
> Cc: edk2-devel@lists.01.org; Gao, Liming <liming.gao@intel.com>
> Subject: Re: [edk2] [Patch 00/33] BaseTools python3 migration patch set
>
> On 01/25/19 10:42, Feng, Bob C wrote:
>> [...]
>>
>> -----Original Message-----
>> From: Laszlo Ersek [mailto:lersek@redhat.com]
>> Sent: Friday, January 25, 2019 4:57 PM
>> To: Feng, Bob C <bob.c.feng@intel.com>
>> Cc: edk2-devel@lists.01.org; Gao, Liming <liming.gao@intel.com>
>> Subject: Re: [edk2] [Patch 00/33] BaseTools python3 migration patch
>> set
>>
>> [...]
>>
>> On 01/25/19 05:55, Feng, Bob C wrote:
>>> BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=55
>>>
>>> This patch set is to enable python3 on BaseTools. Basetools code will
>>> be compatible with both python3 and python2.
>>>
>>> We will have two envs PYTHON3_ENABLE and PYTHON_COMMAND. The behavior
>>> can be combined as the below to support this usage.
>>> If user wants the specific python interpreter, he only needs to set
>>> PYTHON_COMMAND env.
>>> If PYTHON3_ENABLE is set, PYTHON_COMMAND will be set to the found one
>>> by edk2 scripts based on PYTHON3_ENABLE value.
>>> If PYTHON3_ENABLE is not set, but PYTHON_COMMAND is set, then
>>> PYTHON_COMMAND will be used to run python script. No version check
>>> here.
>>> If PYTHON3_ENABLE is not set, but PYTHON_COMMAND is not set,
>>> PYTHON_COMMAND will be set to the high version python installed in
>>> OS.
>
> I ran the following tests, at commit 41c0616fc081 ("BaseTools: Eot tool
> Python3 adaption", 2019-01-25). Each test was performed in a clean tree (after running "git clean -ffdx") and clean environment (I re-sourced "edksetup.sh" for each test in separation). In addition, the base tools were rebuilt (again from a clean tree) for each test, with the following command [1]:
>
> nice make -C "$EDK_TOOLS_PATH" -j $(getconf _NPROCESSORS_ONLN)
>
> (a) On my RHEL7.5 Workstation laptop, I have both the system-level python packages installed (python-2.7.5-69.el7_5.x86_64), and the extra
> python-3.4 stuff from EPEL-7 (python34-3.4.9-1.el7.x86_64).
>
> (a1) Didn't set either PYTHON3_ENABLE or PYTHON_COMMAND. The build utility picked
>
> PYTHON_COMMAND = /usr/bin/python3.4
>
> and I successfully built OvmfPkg for IA32, IA32X64, and X64; also ArmVirtQemu for AARCH64. The built firmware images passed a smoke test too.
>
> (a2) I removed all the python34 packages (and the dependent packages) from my laptop. Didn't set either of PYTHON3_ENABLE and PYTHON_COMMAND.
> (This is the configuration what a "normal" RHEL7 environment would
> provide.)
>
> In this case, rebuilding the base tools [1] failed, in the testing
> phase:
>
>> ======================================================================
>> FAIL: test_Ecc_CParser4_CLexer (CheckPythonSyntax.Tests)
>> ----------------------------------------------------------------------
>> Traceback (most recent call last):
>> File ".../BaseTools/Tests/CheckPythonSyntax.py", line 55, in <lambda>
>> newmethod = lambda self: self.SingleFileTest(filename)
>> File ".../BaseTools/Tests/CheckPythonSyntax.py", line 33, in SingleFileTest
>> self.fail('syntax error: %s, Error is %s' % (filename, str(e)))
>> AssertionError: syntax error: .../BaseTools/Source/Python/Ecc/CParser4/CLexer.py, Error is File ".../BaseTools/Source/Python/Ecc/CParser4/CLexer.py", line 590
>> def __init__(self, input=None, output:TextIO = sys.stdout):
>> ^
>> SyntaxError: invalid syntax
>>
>>
>> ======================================================================
>> FAIL: test_Ecc_CParser4_CListener (CheckPythonSyntax.Tests)
>> ----------------------------------------------------------------------
>> Traceback (most recent call last):
>> File ".../BaseTools/Tests/CheckPythonSyntax.py", line 55, in <lambda>
>> newmethod = lambda self: self.SingleFileTest(filename)
>> File ".../BaseTools/Tests/CheckPythonSyntax.py", line 33, in SingleFileTest
>> self.fail('syntax error: %s, Error is %s' % (filename, str(e)))
>> AssertionError: syntax error: .../BaseTools/Source/Python/Ecc/CParser4/CListener.py, Error is File ".../BaseTools/Source/Python/Ecc/CParser4/CListener.py", line 35
>> def enterTranslation_unit(self, ctx:CParser.Translation_unitContext):
>> ^
>> SyntaxError: invalid syntax
>>
>>
>> ======================================================================
>> FAIL: test_Ecc_CParser4_CParser (CheckPythonSyntax.Tests)
>> ----------------------------------------------------------------------
>> Traceback (most recent call last):
>> File ".../BaseTools/Tests/CheckPythonSyntax.py", line 55, in <lambda>
>> newmethod = lambda self: self.SingleFileTest(filename)
>> File ".../BaseTools/Tests/CheckPythonSyntax.py", line 33, in SingleFileTest
>> self.fail('syntax error: %s, Error is %s' % (filename, str(e)))
>> AssertionError: syntax error: .../BaseTools/Source/Python/Ecc/CParser4/CParser.py, Error is File ".../BaseTools/Source/Python/Ecc/CParser4/CParser.py", line 744
>> def __init__(self, input:TokenStream, output:TextIO = sys.stdout):
>> ^
>> SyntaxError: invalid syntax
>>
>>
>> ======================================================================
>> FAIL: test_Eot_CParser4_CLexer (CheckPythonSyntax.Tests)
>> ----------------------------------------------------------------------
>> Traceback (most recent call last):
>> File ".../BaseTools/Tests/CheckPythonSyntax.py", line 55, in <lambda>
>> newmethod = lambda self: self.SingleFileTest(filename)
>> File ".../BaseTools/Tests/CheckPythonSyntax.py", line 33, in SingleFileTest
>> self.fail('syntax error: %s, Error is %s' % (filename, str(e)))
>> AssertionError: syntax error: .../BaseTools/Source/Python/Eot/CParser4/CLexer.py, Error is File ".../BaseTools/Source/Python/Eot/CParser4/CLexer.py", line 591
>> def __init__(self, input=None, output:TextIO = sys.stdout):
>> ^
>> SyntaxError: invalid syntax
>>
>>
>> ======================================================================
>> FAIL: test_Eot_CParser4_CListener (CheckPythonSyntax.Tests)
>> ----------------------------------------------------------------------
>> Traceback (most recent call last):
>> File ".../BaseTools/Tests/CheckPythonSyntax.py", line 55, in <lambda>
>> newmethod = lambda self: self.SingleFileTest(filename)
>> File ".../BaseTools/Tests/CheckPythonSyntax.py", line 33, in SingleFileTest
>> self.fail('syntax error: %s, Error is %s' % (filename, str(e)))
>> AssertionError: syntax error: .../BaseTools/Source/Python/Eot/CParser4/CListener.py, Error is File ".../BaseTools/Source/Python/Eot/CParser4/CListener.py", line 35
>> def enterTranslation_unit(self, ctx:CParser.Translation_unitContext):
>> ^
>> SyntaxError: invalid syntax
>>
>>
>> ======================================================================
>> FAIL: test_Eot_CParser4_CParser (CheckPythonSyntax.Tests)
>> ----------------------------------------------------------------------
>> Traceback (most recent call last):
>> File ".../BaseTools/Tests/CheckPythonSyntax.py", line 55, in <lambda>
>> newmethod = lambda self: self.SingleFileTest(filename)
>> File ".../BaseTools/Tests/CheckPythonSyntax.py", line 33, in SingleFileTest
>> self.fail('syntax error: %s, Error is %s' % (filename, str(e)))
>> AssertionError: syntax error: .../BaseTools/Source/Python/Eot/CParser4/CParser.py, Error is File ".../BaseTools/Source/Python/Eot/CParser4/CParser.py", line 744
>> def __init__(self, input:TokenStream, output:TextIO = sys.stdout):
>> ^
>> SyntaxError: invalid syntax
>>
>>
>> ----------------------------------------------------------------------
>> Ran 270 tests in 0.938s
>>
>> FAILED (failures=6)
>
> I didn't continue testing on my RHEL7 laptop beyond this point.
>
>
> (b) RHEL-8 virtual machine, with "/usr/bin/python3.6" from python36-3.6.6-18.el8.x86_64, and "/usr/libexec/platform-python" from platform-python-3.6.8-1.el8.x86_64.
>
> (b1) Didn't set either PYTHON3_ENABLE or PYTHON_COMMAND. The build utility picked
>
> PYTHON_COMMAND = /usr/bin/python3.6
>
> and I successfully built OvmfPkg for IA32, IA32X64, and X64. (I don't have a cross-compiler installed in this environment yet, nor a RHEL8
> aarch64 KVM guest, so I couldn't test ArmVirtQemu for now).
>
> (b2) I set PYTHON_COMMAND to "/usr/libexec/platform-python". Didn't set PYTHON3_ENABLE. The same builds as in (b1) succeeded.
>
>
> Thus, what seems to be missing is a little bit of python2 compatibility, in (a2).
>
> Thanks,
> Laszlo
> _______________________________________________
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel
>
next prev parent reply other threads:[~2019-01-28 13:48 UTC|newest]
Thread overview: 41+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-01-25 4:55 [Patch 00/33] BaseTools python3 migration patch set Feng, Bob C
2019-01-25 4:55 ` [Patch 01/33] BaseTool:Rename xrange() to range() Feng, Bob C
2019-01-25 4:55 ` [Patch 02/33] BaseTools:use iterate list to replace the itertools Feng, Bob C
2019-01-25 4:55 ` [Patch 03/33] BaseTools: Rename iteritems to items Feng, Bob C
2019-01-25 4:55 ` [Patch 04/33] BaseTools: replace get_bytes_le() to bytes_le Feng, Bob C
2019-01-25 4:55 ` [Patch 05/33] BaseTools: use OrderedDict instead of sdict Feng, Bob C
2019-01-25 4:55 ` [Patch 06/33] BaseTools: nametuple not have verbose parameter in python3 Feng, Bob C
2019-01-25 4:56 ` [Patch 07/33] BaseTools: Remove unnecessary super function Feng, Bob C
2019-01-25 4:56 ` [Patch 08/33] BaseTools: replace long by int Feng, Bob C
2019-01-25 4:56 ` [Patch 09/33] BaseTools:Solve the data sorting problem use python3 Feng, Bob C
2019-01-25 4:56 ` [Patch 10/33] BaseTools: Update argparse arguments since it not have version now Feng, Bob C
2019-01-25 4:56 ` [Patch 11/33] BaseTools:Similar to octal data rectification Feng, Bob C
2019-01-25 4:56 ` [Patch 12/33] BaseTools/UPT:merge UPT Tool use Python2 and Python3 Feng, Bob C
2019-01-25 4:56 ` [Patch 13/33] BaseTools: update Test scripts support python3 Feng, Bob C
2019-01-25 4:56 ` [Patch 14/33] BaseTools/Scripts: Porting PackageDocumentTools code to use Python3 Feng, Bob C
2019-01-25 4:56 ` [Patch 15/33] Basetools: It went wrong when use os.linesep Feng, Bob C
2019-01-25 4:56 ` [Patch 16/33] BaseTools:Fv BaseAddress must set If it not set Feng, Bob C
2019-01-25 4:56 ` [Patch 17/33] BaseTools: Make sure AllPcdList valid Feng, Bob C
2019-01-25 4:56 ` [Patch 18/33] BaseTools:TestTools character encoding issue Feng, Bob C
2019-01-25 4:56 ` [Patch 19/33] BaseTools:Double carriage return inserted from Trim.py on Python3 Feng, Bob C
2019-01-25 4:56 ` [Patch 20/33] BaseTools:File open failed for VPD MapFile Feng, Bob C
2019-01-25 4:56 ` [Patch 21/33] BaseTools: change the Division Operator Feng, Bob C
2019-01-25 4:56 ` [Patch 22/33] BaseTools:There is extra blank line in datalog Feng, Bob C
2019-01-25 4:56 ` [Patch 23/33] BaseTools: Similar to octal data rectification Feng, Bob C
2019-01-25 4:56 ` [Patch 24/33] BaseTools: Update windows and linux run scripts file to use Python3 Feng, Bob C
2019-01-25 4:56 ` [Patch 25/33] BaseTools:Update build tool to print python version information Feng, Bob C
2019-01-25 4:56 ` [Patch 26/33] BaseTools:Linux Python highest version check Feng, Bob C
2019-01-25 4:56 ` [Patch 27/33] BaseTools: Update PYTHON env to PYTHON_COMMAND Feng, Bob C
2019-01-25 4:56 ` [Patch 28/33] BaseTools:Fixed Rsa issue and a set define issue Feng, Bob C
2019-01-25 4:56 ` [Patch 29/33] BaseTools:ord() don't match in py2 and py3 Feng, Bob C
2019-01-25 4:56 ` [Patch 30/33] BaseTools: the list and iterator translation Feng, Bob C
2019-01-25 4:56 ` [Patch 31/33] BaseTools: Handle the bytes and str difference Feng, Bob C
2019-01-25 4:56 ` [Patch 32/33] BaseTools: ECC tool Python3 adaption Feng, Bob C
2019-01-25 4:56 ` [Patch 33/33] BaseTools: Eot " Feng, Bob C
2019-01-25 8:56 ` [Patch 00/33] BaseTools python3 migration patch set Laszlo Ersek
2019-01-25 9:42 ` Feng, Bob C
2019-01-25 18:18 ` Laszlo Ersek
2019-01-28 2:33 ` Feng, Bob C
2019-01-28 10:35 ` Feng, Bob C
2019-01-28 13:48 ` Laszlo Ersek [this message]
2019-01-29 2:15 ` Feng, Bob C
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-list from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=c6f914d3-f3b1-e2c6-5860-24cb92ca4137@redhat.com \
--to=devel@edk2.groups.io \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox