From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=195.135.221.5; helo=smtp.nue.novell.com; envelope-from=glin@suse.com; receiver=edk2-devel@lists.01.org Received: from smtp.nue.novell.com (smtp.nue.novell.com [195.135.221.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 178B2203B99B7 for ; Mon, 25 Jun 2018 03:31:55 -0700 (PDT) Received: from localhost.localdomain (unknown.telstraglobal.net [134.159.103.118]) by smtp.nue.novell.com with ESMTP (NOT encrypted); Mon, 25 Jun 2018 12:31:52 +0200 From: Gary Lin To: edk2-devel@lists.01.org Cc: Yonghong Zhu , Liming Gao Date: Mon, 25 Jun 2018 18:31:23 +0800 Message-Id: <20180625103136.12531-1-glin@suse.com> X-Mailer: git-send-email 2.17.1 Subject: [PATCH v4 00/20] BaseTools: One step toward python3 X-BeenThere: edk2-devel@lists.01.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: EDK II Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jun 2018 10:31:56 -0000 v4 changes: - Remove the range() patch since it needs python-future - Remove the patch to unify long and int since it caused error in windows. - Split the absolute import patches and will introduce them later v3 changes: - Rebase to the current git HEAD (2e1083038d9aa74fcaa2db8158fdee7c8b4af3bb) - Fix a typo in BaseTools/Scripts/PackageDocumentTools/plugins/EdkPlugins/basemodel/ini.py - Remove the patch for reduce() since it's not used anymore v2 changes: - Rebase to the current git HEAD (821807bcefb9a36e598d71a8004fae5aab2052a0) - Apply "futurize -f libfuturize.fixes.fix_absolute_import" and refactor some python scripts to break the circular imports. This patch series is also available in https://github.com/lcp/edk2/tree/python3-futurize-v3 Since python2 will be EOL in 2020, we start to evaluate the impact of the python2 removal. As expected, OMVF building failed the test. It's actually a task noted in the wiki page: https://github.com/tianocore/tianocore.github.io/wiki/Tasks-BaseTools-Python3-Support Maybe it's time to convert the python scripts gradually. This patchset doesn't make the python scripts in BaseTools compatible with python3 immediately. It aims to do the trivial and safe conversion and replacement to make some statements compatible with both python2 and python3, so we can deal with the difficult cases later. With the help of "futurize" from python-future, it's easier to refactor the statements. This patchset is basically equivalent to "futurize -1"(*) plus "StringIO.StringIO => io.BytesIO" and minus the absolute import. The patchset was tested with the following command in openSUSE Tumbleweed: $ ./OvmfPkg/build.sh -D SECURE_BOOT_ENABLE \ -D NETWORK_IP6_ENABLE \ -D HTTP_BOOT_ENABLE \ -D TLS_ENABLE \ -D TPM2_ENABLE The firmware file was built successfully and I didn't notice any error so far. Testing with other platform is welcome. (*) http://python-future.org/automatic_conversion.html#stage-1-safe-fixes Contributed-under: TianoCore Contribution Agreement 1.1 Cc: Yonghong Zhu Cc: Liming Gao Signed-off-by: Gary Lin Gary Lin (13): BaseTools: Fix a typo in ini.py BaseTools: Refactor python except statements BaseTools: Refactor python print statements BaseTools: Remove the old python "not-equal" BaseTools: Remove tuple parameter in python scripts BaseTools: Remove the deprecated hash_key() BaseTools: Replace StandardError with Expression BaseTools: Remove types.TypeType BaseTools: Refactor python raise statement BaseTools: Adjust the spaces around commas and colons BaseTools: Migrate to the new octal literal BaseTools: Fix old python2 idioms BaseTools: Replace StringIO.StringIO with io.BytesIO .../Bin/CYGWIN_NT-5.1-i686/armcc_wrapper.py | 5 +- BaseTools/Scripts/BinToPcd.py | 7 +- BaseTools/Scripts/ConvertUni.py | 5 - BaseTools/Scripts/FormatDosFiles.py | 3 +- BaseTools/Scripts/MemoryProfileSymbolGen.py | 21 +- .../PackageDocumentTools/packagedoc_cli.py | 47 ++-- .../plugins/EdkPlugins/basemodel/doxygen.py | 11 +- .../plugins/EdkPlugins/basemodel/efibinary.py | 29 +- .../plugins/EdkPlugins/basemodel/ini.py | 4 +- .../EdkPlugins/edk2/model/baseobject.py | 6 +- .../EdkPlugins/edk2/model/doxygengen.py | 2 +- .../EdkPlugins/edk2/model/doxygengen_spec.py | 2 +- .../plugins/EdkPlugins/edk2/model/inf.py | 8 +- BaseTools/Scripts/PatchCheck.py | 2 +- BaseTools/Scripts/RunMakefile.py | 2 +- .../Scripts/SmiHandlerProfileSymbolGen.py | 19 +- BaseTools/Scripts/UpdateBuildVersions.py | 18 +- BaseTools/Source/Python/AutoGen/AutoGen.py | 77 +++--- .../Source/Python/AutoGen/BuildEngine.py | 37 +-- BaseTools/Source/Python/AutoGen/GenC.py | 72 ++--- BaseTools/Source/Python/AutoGen/GenDepex.py | 8 +- BaseTools/Source/Python/AutoGen/GenMake.py | 8 +- BaseTools/Source/Python/AutoGen/GenPcdDb.py | 118 ++++---- BaseTools/Source/Python/AutoGen/GenVar.py | 160 +++++------ .../Source/Python/AutoGen/IdfClassObject.py | 1 - BaseTools/Source/Python/AutoGen/StrGather.py | 8 +- .../Source/Python/AutoGen/UniClassObject.py | 17 +- .../Python/AutoGen/ValidCheckingInfoObject.py | 4 +- BaseTools/Source/Python/BPDG/BPDG.py | 3 +- BaseTools/Source/Python/BPDG/GenVpd.py | 18 +- BaseTools/Source/Python/Common/DataType.py | 4 +- BaseTools/Source/Python/Common/Expression.py | 77 +++--- .../Source/Python/Common/LongFilePathOs.py | 2 +- BaseTools/Source/Python/Common/Misc.py | 49 ++-- .../Source/Python/Common/RangeExpression.py | 33 +-- BaseTools/Source/Python/Common/StringUtils.py | 6 +- .../Python/Common/TargetTxtClassObject.py | 7 +- .../Python/Common/ToolDefClassObject.py | 8 +- BaseTools/Source/Python/Common/VpdInfoFile.py | 23 +- BaseTools/Source/Python/Ecc/CParser.py | 175 ++++++------ BaseTools/Source/Python/Ecc/Check.py | 14 +- .../Python/Ecc/CodeFragmentCollector.py | 69 ++--- BaseTools/Source/Python/Ecc/Configuration.py | 5 +- BaseTools/Source/Python/Ecc/Exception.py | 3 +- .../Ecc/MetaFileWorkspace/MetaDataTable.py | 5 +- .../Ecc/MetaFileWorkspace/MetaFileParser.py | 42 +-- .../Source/Python/Ecc/Xml/XmlRoutines.py | 9 +- BaseTools/Source/Python/Ecc/c.py | 15 +- BaseTools/Source/Python/Eot/CParser.py | 175 ++++++------ .../Python/Eot/CodeFragmentCollector.py | 61 +++-- BaseTools/Source/Python/Eot/InfParserLite.py | 7 +- BaseTools/Source/Python/Eot/Parser.py | 2 +- BaseTools/Source/Python/Eot/c.py | 23 +- .../Source/Python/GenFds/AprioriSection.py | 6 +- BaseTools/Source/Python/GenFds/Capsule.py | 10 +- BaseTools/Source/Python/GenFds/CapsuleData.py | 6 +- BaseTools/Source/Python/GenFds/EfiSection.py | 6 +- BaseTools/Source/Python/GenFds/Fd.py | 12 +- BaseTools/Source/Python/GenFds/FdfParser.py | 45 +-- .../Source/Python/GenFds/FfsFileStatement.py | 4 +- .../Source/Python/GenFds/FfsInfStatement.py | 18 +- BaseTools/Source/Python/GenFds/Fv.py | 10 +- .../Source/Python/GenFds/FvImageSection.py | 8 +- BaseTools/Source/Python/GenFds/GenFds.py | 17 +- .../Python/GenFds/GenFdsGlobalVariable.py | 9 +- BaseTools/Source/Python/GenFds/OptionRom.py | 3 - BaseTools/Source/Python/GenFds/Region.py | 11 +- .../GenPatchPcdTable/GenPatchPcdTable.py | 9 +- .../Source/Python/Pkcs7Sign/Pkcs7Sign.py | 31 ++- .../Rsa2048Sha256GenerateKeys.py | 25 +- .../Rsa2048Sha256Sign/Rsa2048Sha256Sign.py | 35 +-- .../Source/Python/TargetTool/TargetTool.py | 39 +-- BaseTools/Source/Python/Trim/Trim.py | 24 +- .../Source/Python/UPT/Core/DependencyRules.py | 12 +- .../UPT/Core/DistributionPackageClass.py | 4 +- BaseTools/Source/Python/UPT/Core/FileHook.py | 2 +- BaseTools/Source/Python/UPT/Core/IpiDb.py | 6 +- .../Source/Python/UPT/Core/PackageFile.py | 12 +- .../Python/UPT/GenMetaFile/GenDecFile.py | 15 +- .../Python/UPT/GenMetaFile/GenInfFile.py | 37 +-- BaseTools/Source/Python/UPT/InstallPkg.py | 2 +- BaseTools/Source/Python/UPT/InventoryWs.py | 2 +- .../Python/UPT/Library/CommentParsing.py | 2 +- .../Python/UPT/Library/ExpressionValidate.py | 11 +- BaseTools/Source/Python/UPT/Library/Misc.py | 6 +- .../Python/UPT/Library/ParserValidate.py | 2 +- .../Source/Python/UPT/Library/StringUtils.py | 4 +- .../Python/UPT/Library/UniClassObject.py | 17 +- .../Python/UPT/Library/Xml/XmlRoutines.py | 4 +- BaseTools/Source/Python/UPT/MkPkg.py | 2 +- .../UPT/Object/Parser/InfBinaryObject.py | 6 +- .../UPT/Object/Parser/InfDefineObject.py | 2 +- .../Python/UPT/Object/Parser/InfGuidObject.py | 4 +- .../Object/Parser/InfLibraryClassesObject.py | 2 +- .../Python/UPT/Object/Parser/InfMisc.py | 4 +- .../UPT/Object/Parser/InfPackagesObject.py | 4 +- .../Python/UPT/Object/Parser/InfPcdObject.py | 4 +- .../Python/UPT/Object/Parser/InfPpiObject.py | 4 +- .../UPT/Object/Parser/InfProtocolObject.py | 2 +- .../UPT/Object/Parser/InfSoucesObject.py | 3 +- .../Object/Parser/InfUserExtensionObject.py | 4 +- .../Python/UPT/PomAdapter/DecPomAlignment.py | 56 ++-- .../Python/UPT/PomAdapter/InfPomAlignment.py | 3 +- .../UPT/PomAdapter/InfPomAlignmentMisc.py | 3 +- BaseTools/Source/Python/UPT/ReplacePkg.py | 2 +- BaseTools/Source/Python/UPT/RmPkg.py | 2 +- BaseTools/Source/Python/UPT/TestInstall.py | 4 +- BaseTools/Source/Python/UPT/UPT.py | 8 +- .../Python/UPT/UnitTest/DecParserTest.py | 5 +- .../UPT/UnitTest/InfBinarySectionTest.py | 9 +- BaseTools/Source/Python/UPT/Xml/CommonXml.py | 2 +- BaseTools/Source/Python/UPT/Xml/XmlParser.py | 24 +- .../Python/Workspace/BuildClassObject.py | 16 +- .../Source/Python/Workspace/DecBuildData.py | 22 +- .../Source/Python/Workspace/DscBuildData.py | 259 +++++++++--------- .../Source/Python/Workspace/InfBuildData.py | 2 +- .../Source/Python/Workspace/MetaFileParser.py | 69 ++--- .../Source/Python/Workspace/MetaFileTable.py | 10 +- .../Python/Workspace/WorkspaceCommon.py | 2 +- BaseTools/Source/Python/build/BuildReport.py | 21 +- BaseTools/Source/Python/build/build.py | 39 +-- BaseTools/Tests/CheckPythonSyntax.py | 2 +- BaseTools/Tests/TestTools.py | 10 +- BaseTools/Tests/TianoCompress.py | 5 +- BaseTools/gcc/mingw-gcc-build.py | 111 ++++---- 125 files changed, 1376 insertions(+), 1353 deletions(-) -- 2.17.1