From: "Feng, Bob C" <bob.c.feng@intel.com>
To: edk2-devel@lists.01.org
Subject: [Patch 00/33] BaseTools python3 migration patch set
Date: Fri, 25 Jan 2019 12:55:53 +0800 [thread overview]
Message-ID: <20190125045626.14700-1-bob.c.feng@intel.com> (raw)
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=55
This patch set is to enable python3 on BaseTools. Basetools code will be
compatible with both python3 and python2.
We will have two envs PYTHON3_ENABLE and PYTHON_COMMAND. The behavior can be
combined as the below to support this usage.
If user wants the specific python interpreter, he only needs to set PYTHON_COMMAND env.
If PYTHON3_ENABLE is set, PYTHON_COMMAND will be set to the found one by edk2 scripts based on PYTHON3_ENABLE value.
If PYTHON3_ENABLE is not set, but PYTHON_COMMAND is set, then PYTHON_COMMAND will be used to run python script. No version check here.
If PYTHON3_ENABLE is not set, but PYTHON_COMMAND is not set, PYTHON_COMMAND will be set to the high version python installed in OS.
This patch set is verified by basic testing on Ovmf, MinKabylake and MinPurley platform with Python3.7.1 and
minimal testing on Ovmf, MinKabylake and MinPurley with Python2.7.15.
After this change, we will focus on the Python3 validation.
You can also review and try the patch set at https://github.com/BobCF/edk2.git branch py3basetools
Feng, Bob C (8):
BaseTools: Make sure AllPcdList valid.
BaseTools:File open failed for VPD MapFile
BaseTools:Fixed Rsa issue and a set define issue.
BaseTools:ord() don't match in py2 and py3
BaseTools: the list and iterator translation
BaseTools: Handle the bytes and str difference
BaseTools: ECC tool Python3 adaption
BaseTools: Eot tool Python3 adaption
Liming Gao (1):
BaseTools: Update PYTHON env to the specific one PYTHON_COMMAND
Yunhua Feng (4):
BaseTools: use OrderedDict instead of sdict
BaseTools: nametuple not have verbose parameter in python3
BaseTools: Remove unnecessary super function
BaseTools: replace long by int
Zhiju Fan (3):
BaseTools:TestTools character encoding issue
BaseTools:Double carriage return inserted from Trim.py on Python3
BaseTools:There is extra blank line in datalog
Zhijux Fan (17):
BaseTool:Rename xrange() to range()
BaseTools:use iterate list to replace the itertools
BaseTools: Rename iteritems to items
BaseTools: replace get_bytes_le() to bytes_le
BaseTools:Solve the data sorting problem use python3
BaseTools: Update argparse arguments since it not have version now
BaseTools:Similar to octal data rectification
BaseTools/UPT:merge UPT Tool use Python2 and Python3
BaseTools: update Test scripts support python3
BaseTools/Scripts: Porting PackageDocumentTools code to use Python3
Basetools: It went wrong when use os.linesep
BaseTools:Fv BaseAddress must set If it not set
BaseTools: change the Division Operator
BaseTools: Similar to octal data rectification
BaseTools: Update windows and linux run scripts file to use Python3
BaseTools:Update build tool to print python version information
BaseTools:Linux Python highest version check.
BaseTools/Bin/CYGWIN_NT-5.1-i686/Ecc | 6 +-
BaseTools/Bin/CYGWIN_NT-5.1-i686/GenDepex | 6 +-
BaseTools/Bin/CYGWIN_NT-5.1-i686/GenFds | 6 +-
BaseTools/Bin/CYGWIN_NT-5.1-i686/TargetTool | 6 +-
BaseTools/Bin/CYGWIN_NT-5.1-i686/Trim | 6 +-
BaseTools/Bin/CYGWIN_NT-5.1-i686/build | 6 +-
BaseTools/BinWrappers/PosixLike/BPDG | 6 +-
BaseTools/BinWrappers/PosixLike/Ecc | 6 +-
BaseTools/BinWrappers/PosixLike/GenDepex | 6 +-
BaseTools/BinWrappers/PosixLike/GenFds | 6 +-
BaseTools/BinWrappers/PosixLike/GenPatchPcdTable | 6 +-
BaseTools/BinWrappers/PosixLike/GenerateCapsule | 6 +-
BaseTools/BinWrappers/PosixLike/PatchPcdValue | 6 +-
BaseTools/BinWrappers/PosixLike/Pkcs7Sign | 6 +-
BaseTools/BinWrappers/PosixLike/Rsa2048Sha256GenerateKeys | 6 +-
BaseTools/BinWrappers/PosixLike/Rsa2048Sha256Sign | 6 +-
BaseTools/BinWrappers/PosixLike/TargetTool | 6 +-
BaseTools/BinWrappers/PosixLike/Trim | 6 +-
BaseTools/BinWrappers/PosixLike/UPT | 6 +-
BaseTools/BinWrappers/PosixLike/build | 6 +-
BaseTools/BinWrappers/WindowsLike/BPDG.bat | 2 +-
BaseTools/BinWrappers/WindowsLike/Ecc.bat | 2 +-
BaseTools/BinWrappers/WindowsLike/GenDepex.bat | 2 +-
BaseTools/BinWrappers/WindowsLike/GenFds.bat | 2 +-
BaseTools/BinWrappers/WindowsLike/GenPatchPcdTable.bat | 2 +-
BaseTools/BinWrappers/WindowsLike/GenerateCapsule.bat | 2 +-
BaseTools/BinWrappers/WindowsLike/PatchPcdValue.bat | 2 +-
BaseTools/BinWrappers/WindowsLike/Pkcs7Sign.bat | 2 +-
BaseTools/BinWrappers/WindowsLike/Rsa2048Sha256GenerateKeys.bat | 2 +-
BaseTools/BinWrappers/WindowsLike/Rsa2048Sha256Sign.bat | 2 +-
BaseTools/BinWrappers/WindowsLike/TargetTool.bat | 2 +-
BaseTools/BinWrappers/WindowsLike/Trim.bat | 2 +-
BaseTools/BinWrappers/WindowsLike/UPT.bat | 2 +-
BaseTools/BinWrappers/WindowsLike/build.bat | 2 +-
BaseTools/Makefile | 12 +-
BaseTools/Scripts/ConvertFceToStructurePcd.py | 2 +-
BaseTools/Scripts/PackageDocumentTools/packagedoc_cli.py | 6 +-
BaseTools/Scripts/PackageDocumentTools/packagedocapp.pyw | 14 +--
BaseTools/Scripts/PackageDocumentTools/plugins/EdkPlugins/basemodel/doxygen.py | 4 +-
BaseTools/Scripts/PackageDocumentTools/plugins/EdkPlugins/edk2/model/baseobject.py | 16 +--
BaseTools/Scripts/PackageDocumentTools/plugins/EdkPlugins/edk2/model/dec.py | 4 +-
BaseTools/Scripts/PackageDocumentTools/plugins/EdkPlugins/edk2/model/doxygengen.py | 12 +-
BaseTools/Scripts/PackageDocumentTools/plugins/EdkPlugins/edk2/model/doxygengen_spec.py | 12 +-
BaseTools/Scripts/PackageDocumentTools/plugins/EdkPlugins/edk2/model/dsc.py | 4 +-
BaseTools/Scripts/PackageDocumentTools/plugins/EdkPlugins/edk2/model/inf.py | 4 +-
BaseTools/Source/C/Makefile | 12 +-
BaseTools/Source/Python/AutoGen/AutoGen.py | 90 +++++++-------
BaseTools/Source/Python/AutoGen/GenC.py | 18 +--
BaseTools/Source/Python/AutoGen/GenMake.py | 40 ++++---
BaseTools/Source/Python/AutoGen/GenPcdDb.py | 55 ++++-----
BaseTools/Source/Python/AutoGen/GenVar.py | 34 +++---
BaseTools/Source/Python/AutoGen/InfSectionParser.py | 2 +-
BaseTools/Source/Python/AutoGen/StrGather.py | 9 +-
BaseTools/Source/Python/AutoGen/UniClassObject.py | 4 +-
BaseTools/Source/Python/AutoGen/ValidCheckingInfoObject.py | 6 +-
BaseTools/Source/Python/BPDG/GenVpd.py | 32 ++---
BaseTools/Source/Python/Common/Expression.py | 9 +-
BaseTools/Source/Python/Common/LongFilePathOs.py | 3 +-
BaseTools/Source/Python/Common/LongFilePathSupport.py | 12 --
BaseTools/Source/Python/Common/Misc.py | 196 +++++++-----------------------
BaseTools/Source/Python/Common/StringUtils.py | 16 +--
BaseTools/Source/Python/Common/VpdInfoFile.py | 10 +-
BaseTools/Source/Python/Ecc/{ => CParser3}/CLexer.py | 0
BaseTools/Source/Python/Ecc/{ => CParser3}/CParser.py | 0
BaseTools/Source/Python/Ecc/CParser3/__init__.py | 0
BaseTools/Source/Python/Ecc/CParser4/C.g4 | 637 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
BaseTools/Source/Python/Ecc/CParser4/CLexer.py | 631 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
BaseTools/Source/Python/Ecc/CParser4/CListener.py | 673 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
BaseTools/Source/Python/Ecc/CParser4/CParser.py | 5948 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
BaseTools/Source/Python/Ecc/CParser4/__init__.py | 0
BaseTools/Source/Python/Ecc/Check.py | 4 +-
BaseTools/Source/Python/Ecc/CodeFragmentCollector.py | 20 ++--
BaseTools/Source/Python/Ecc/Configuration.py | 3 -
BaseTools/Source/Python/Ecc/EccMain.py | 2 +-
BaseTools/Source/Python/Ecc/EccToolError.py | 4 +-
BaseTools/Source/Python/Ecc/FileProfile.py | 2 +-
BaseTools/Source/Python/Ecc/MetaDataParser.py | 2 +-
BaseTools/Source/Python/Ecc/c.py | 6 +-
BaseTools/Source/Python/Ecc/config.ini | 2 -
BaseTools/Source/Python/Eot/{ => CParser3}/CLexer.py | 0
BaseTools/Source/Python/Eot/{ => CParser3}/CParser.py | 0
BaseTools/Source/Python/Eot/CParser3/__init__.py | 0
BaseTools/Source/Python/Eot/CParser4/CLexer.py | 632 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
BaseTools/Source/Python/Eot/CParser4/CListener.py | 672 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
BaseTools/Source/Python/Eot/CParser4/CParser.py | 5948 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
BaseTools/Source/Python/Eot/CParser4/__init__.py | 0
BaseTools/Source/Python/Eot/CodeFragmentCollector.py | 22 ++--
BaseTools/Source/Python/GenFds/AprioriSection.py | 4 +-
BaseTools/Source/Python/GenFds/Capsule.py | 15 ++-
BaseTools/Source/Python/GenFds/CapsuleData.py | 4 +-
BaseTools/Source/Python/GenFds/DataSection.py | 4 +-
BaseTools/Source/Python/GenFds/EfiSection.py | 4 +-
BaseTools/Source/Python/GenFds/Fd.py | 4 +-
BaseTools/Source/Python/GenFds/FdfParser.py | 22 ++--
BaseTools/Source/Python/GenFds/FfsFileStatement.py | 16 +--
BaseTools/Source/Python/GenFds/FfsInfStatement.py | 22 ++--
BaseTools/Source/Python/GenFds/Fv.py | 52 ++++----
BaseTools/Source/Python/GenFds/FvImageSection.py | 22 ++--
BaseTools/Source/Python/GenFds/GenFds.py | 37 +++---
BaseTools/Source/Python/GenFds/GenFdsGlobalVariable.py | 4 +-
BaseTools/Source/Python/GenFds/Region.py | 10 +-
BaseTools/Source/Python/PatchPcdValue/PatchPcdValue.py | 4 +-
BaseTools/Source/Python/Pkcs7Sign/Pkcs7Sign.py | 15 +--
BaseTools/Source/Python/Rsa2048Sha256Sign/Rsa2048Sha256GenerateKeys.py | 17 +--
BaseTools/Source/Python/Rsa2048Sha256Sign/Rsa2048Sha256Sign.py | 20 ++--
BaseTools/Source/Python/Trim/Trim.py | 20 ++--
BaseTools/Source/Python/UPT/Core/IpiDb.py | 4 +-
BaseTools/Source/Python/UPT/GenMetaFile/GenInfFile.py | 6 +-
BaseTools/Source/Python/UPT/Library/CommentGenerating.py | 6 +-
BaseTools/Source/Python/UPT/Library/CommentParsing.py | 10 +-
BaseTools/Source/Python/UPT/Library/Misc.py | 190 +++++------------------------
BaseTools/Source/Python/UPT/Library/ParserValidate.py | 2 +-
BaseTools/Source/Python/UPT/Library/Parsing.py | 2 +-
BaseTools/Source/Python/UPT/Library/StringUtils.py | 40 +++----
BaseTools/Source/Python/UPT/Library/UniClassObject.py | 6 +-
BaseTools/Source/Python/UPT/Library/Xml/XmlRoutines.py | 2 +-
BaseTools/Source/Python/UPT/Logger/StringTable.py | 2 +-
BaseTools/Source/Python/UPT/Parser/DecParser.py | 8 +-
BaseTools/Source/Python/UPT/Parser/DecParserMisc.py | 30 +----
BaseTools/Source/Python/UPT/Parser/InfAsBuiltProcess.py | 4 +-
BaseTools/Source/Python/UPT/Parser/InfParser.py | 4 +-
BaseTools/Source/Python/UPT/Parser/InfSectionParser.py | 4 +-
BaseTools/Source/Python/UPT/PomAdapter/InfPomAlignment.py | 2 +-
BaseTools/Source/Python/UPT/UPT.py | 1 +
BaseTools/Source/Python/UPT/Xml/IniToXml.py | 2 +-
BaseTools/Source/Python/UPT/Xml/XmlParserMisc.py | 2 +-
BaseTools/Source/Python/Workspace/BuildClassObject.py | 6 +-
BaseTools/Source/Python/Workspace/DscBuildData.py | 34 ++++--
BaseTools/Source/Python/Workspace/InfBuildData.py | 4 +-
BaseTools/Source/Python/Workspace/MetaDataTable.py | 2 +-
BaseTools/Source/Python/Workspace/MetaFileParser.py | 4 +-
BaseTools/Source/Python/build/BuildReport.py | 38 ++++--
BaseTools/Source/Python/build/build.py | 71 +++++------
BaseTools/Tests/CToolsTests.py | 2 +-
BaseTools/Tests/CheckUnicodeSourceFiles.py | 6 +-
BaseTools/Tests/GNUmakefile | 2 +-
BaseTools/Tests/PythonTest.py | 15 +++
BaseTools/Tests/TestTools.py | 14 ++-
BaseTools/toolsetup.bat | 84 +++++++++++--
edksetup.sh | 77 +++++++++++-
140 files changed, 16035 insertions(+), 941 deletions(-)
rename BaseTools/Source/Python/Ecc/{ => CParser3}/CLexer.py (100%)
rename BaseTools/Source/Python/Ecc/{ => CParser3}/CParser.py (100%)
create mode 100644 BaseTools/Source/Python/Ecc/CParser3/__init__.py
create mode 100644 BaseTools/Source/Python/Ecc/CParser4/C.g4
create mode 100644 BaseTools/Source/Python/Ecc/CParser4/CLexer.py
create mode 100644 BaseTools/Source/Python/Ecc/CParser4/CListener.py
create mode 100644 BaseTools/Source/Python/Ecc/CParser4/CParser.py
create mode 100644 BaseTools/Source/Python/Ecc/CParser4/__init__.py
rename BaseTools/Source/Python/Eot/{ => CParser3}/CLexer.py (100%)
rename BaseTools/Source/Python/Eot/{ => CParser3}/CParser.py (100%)
create mode 100644 BaseTools/Source/Python/Eot/CParser3/__init__.py
create mode 100644 BaseTools/Source/Python/Eot/CParser4/CLexer.py
create mode 100644 BaseTools/Source/Python/Eot/CParser4/CListener.py
create mode 100644 BaseTools/Source/Python/Eot/CParser4/CParser.py
create mode 100644 BaseTools/Source/Python/Eot/CParser4/__init__.py
create mode 100644 BaseTools/Tests/PythonTest.py
--
2.20.1.windows.1
next reply other threads:[~2019-01-25 5:04 UTC|newest]
Thread overview: 41+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-01-25 4:55 Feng, Bob C [this message]
2019-01-25 4:55 ` [Patch 01/33] BaseTool:Rename xrange() to range() Feng, Bob C
2019-01-25 4:55 ` [Patch 02/33] BaseTools:use iterate list to replace the itertools Feng, Bob C
2019-01-25 4:55 ` [Patch 03/33] BaseTools: Rename iteritems to items Feng, Bob C
2019-01-25 4:55 ` [Patch 04/33] BaseTools: replace get_bytes_le() to bytes_le Feng, Bob C
2019-01-25 4:55 ` [Patch 05/33] BaseTools: use OrderedDict instead of sdict Feng, Bob C
2019-01-25 4:55 ` [Patch 06/33] BaseTools: nametuple not have verbose parameter in python3 Feng, Bob C
2019-01-25 4:56 ` [Patch 07/33] BaseTools: Remove unnecessary super function Feng, Bob C
2019-01-25 4:56 ` [Patch 08/33] BaseTools: replace long by int Feng, Bob C
2019-01-25 4:56 ` [Patch 09/33] BaseTools:Solve the data sorting problem use python3 Feng, Bob C
2019-01-25 4:56 ` [Patch 10/33] BaseTools: Update argparse arguments since it not have version now Feng, Bob C
2019-01-25 4:56 ` [Patch 11/33] BaseTools:Similar to octal data rectification Feng, Bob C
2019-01-25 4:56 ` [Patch 12/33] BaseTools/UPT:merge UPT Tool use Python2 and Python3 Feng, Bob C
2019-01-25 4:56 ` [Patch 13/33] BaseTools: update Test scripts support python3 Feng, Bob C
2019-01-25 4:56 ` [Patch 14/33] BaseTools/Scripts: Porting PackageDocumentTools code to use Python3 Feng, Bob C
2019-01-25 4:56 ` [Patch 15/33] Basetools: It went wrong when use os.linesep Feng, Bob C
2019-01-25 4:56 ` [Patch 16/33] BaseTools:Fv BaseAddress must set If it not set Feng, Bob C
2019-01-25 4:56 ` [Patch 17/33] BaseTools: Make sure AllPcdList valid Feng, Bob C
2019-01-25 4:56 ` [Patch 18/33] BaseTools:TestTools character encoding issue Feng, Bob C
2019-01-25 4:56 ` [Patch 19/33] BaseTools:Double carriage return inserted from Trim.py on Python3 Feng, Bob C
2019-01-25 4:56 ` [Patch 20/33] BaseTools:File open failed for VPD MapFile Feng, Bob C
2019-01-25 4:56 ` [Patch 21/33] BaseTools: change the Division Operator Feng, Bob C
2019-01-25 4:56 ` [Patch 22/33] BaseTools:There is extra blank line in datalog Feng, Bob C
2019-01-25 4:56 ` [Patch 23/33] BaseTools: Similar to octal data rectification Feng, Bob C
2019-01-25 4:56 ` [Patch 24/33] BaseTools: Update windows and linux run scripts file to use Python3 Feng, Bob C
2019-01-25 4:56 ` [Patch 25/33] BaseTools:Update build tool to print python version information Feng, Bob C
2019-01-25 4:56 ` [Patch 26/33] BaseTools:Linux Python highest version check Feng, Bob C
2019-01-25 4:56 ` [Patch 27/33] BaseTools: Update PYTHON env to PYTHON_COMMAND Feng, Bob C
2019-01-25 4:56 ` [Patch 28/33] BaseTools:Fixed Rsa issue and a set define issue Feng, Bob C
2019-01-25 4:56 ` [Patch 29/33] BaseTools:ord() don't match in py2 and py3 Feng, Bob C
2019-01-25 4:56 ` [Patch 30/33] BaseTools: the list and iterator translation Feng, Bob C
2019-01-25 4:56 ` [Patch 31/33] BaseTools: Handle the bytes and str difference Feng, Bob C
2019-01-25 4:56 ` [Patch 32/33] BaseTools: ECC tool Python3 adaption Feng, Bob C
2019-01-25 4:56 ` [Patch 33/33] BaseTools: Eot " Feng, Bob C
2019-01-25 8:56 ` [Patch 00/33] BaseTools python3 migration patch set Laszlo Ersek
2019-01-25 9:42 ` Feng, Bob C
2019-01-25 18:18 ` Laszlo Ersek
2019-01-28 2:33 ` Feng, Bob C
2019-01-28 10:35 ` Feng, Bob C
2019-01-28 13:48 ` Laszlo Ersek
2019-01-29 2:15 ` Feng, Bob C
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-list from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190125045626.14700-1-bob.c.feng@intel.com \
--to=devel@edk2.groups.io \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox