From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=195.135.221.5; helo=smtp.nue.novell.com; envelope-from=glin@suse.com; receiver=edk2-devel@lists.01.org Received: from smtp.nue.novell.com (smtp.nue.novell.com [195.135.221.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 9DD3B202E5412 for ; Mon, 25 Jun 2018 20:41:03 -0700 (PDT) Received: from emea4-mta.ukb.novell.com ([10.120.13.87]) by smtp.nue.novell.com with ESMTP (TLS encrypted); Tue, 26 Jun 2018 05:41:01 +0200 Received: from GaryWorkstation (nwb-a10-snat.microfocus.com [10.120.13.202]) by emea4-mta.ukb.novell.com with ESMTP (TLS encrypted); Tue, 26 Jun 2018 04:40:51 +0100 Date: Tue, 26 Jun 2018 11:40:46 +0800 From: Gary Lin To: edk2-devel@lists.01.org Cc: Liming Gao Message-ID: <20180626034046.72reewwyog6777jd@GaryWorkstation> References: <20180625103136.12531-1-glin@suse.com> MIME-Version: 1.0 In-Reply-To: <20180625103136.12531-1-glin@suse.com> User-Agent: NeoMutt/20170912 (1.9.0) Subject: Re: [PATCH v4 00/20] BaseTools: One step toward python3 X-BeenThere: edk2-devel@lists.01.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: EDK II Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jun 2018 03:41:05 -0000 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Mon, Jun 25, 2018 at 06:31:23PM +0800, Gary Lin wrote: > v4 changes: v4 is rebased to 3b03b5e990f8bb347dfdb91926d8ef015d0b607e > - Remove the range() patch since it needs python-future > - Remove the patch to unify long and int since it caused error in > windows. > - Split the absolute import patches and will introduce them later > > v3 changes: > - Rebase to the current git HEAD (2e1083038d9aa74fcaa2db8158fdee7c8b4af3bb) > - Fix a typo in BaseTools/Scripts/PackageDocumentTools/plugins/EdkPlugins/basemodel/ini.py > - Remove the patch for reduce() since it's not used anymore > > v2 changes: > - Rebase to the current git HEAD (821807bcefb9a36e598d71a8004fae5aab2052a0) > - Apply "futurize -f libfuturize.fixes.fix_absolute_import" and > refactor some python scripts to break the circular imports. > > This patch series is also available in > https://github.com/lcp/edk2/tree/python3-futurize-v3 I forgot the update the url. It should be https://github.com/lcp/edk2/tree/python3-futurize-v4 Gary Lin > > Since python2 will be EOL in 2020, we start to evaluate the impact of > the python2 removal. As expected, OMVF building failed the test. It's > actually a task noted in the wiki page: > > https://github.com/tianocore/tianocore.github.io/wiki/Tasks-BaseTools-Python3-Support > > Maybe it's time to convert the python scripts gradually. > > This patchset doesn't make the python scripts in BaseTools compatible > with python3 immediately. It aims to do the trivial and safe conversion > and replacement to make some statements compatible with both python2 and > python3, so we can deal with the difficult cases later. > > With the help of "futurize" from python-future, it's easier to refactor > the statements. This patchset is basically equivalent to "futurize -1"(*) > plus "StringIO.StringIO => io.BytesIO" and minus the absolute import. > > The patchset was tested with the following command in openSUSE > Tumbleweed: > > $ ./OvmfPkg/build.sh -D SECURE_BOOT_ENABLE \ > -D NETWORK_IP6_ENABLE \ > -D HTTP_BOOT_ENABLE \ > -D TLS_ENABLE \ > -D TPM2_ENABLE > > The firmware file was built successfully and I didn't notice any error > so far. Testing with other platform is welcome. > > (*) http://python-future.org/automatic_conversion.html#stage-1-safe-fixes > > Contributed-under: TianoCore Contribution Agreement 1.1 > Cc: Yonghong Zhu > Cc: Liming Gao > Signed-off-by: Gary Lin > > Gary Lin (13): > BaseTools: Fix a typo in ini.py > BaseTools: Refactor python except statements > BaseTools: Refactor python print statements > BaseTools: Remove the old python "not-equal" > BaseTools: Remove tuple parameter in python scripts > BaseTools: Remove the deprecated hash_key() > BaseTools: Replace StandardError with Expression > BaseTools: Remove types.TypeType > BaseTools: Refactor python raise statement > BaseTools: Adjust the spaces around commas and colons > BaseTools: Migrate to the new octal literal > BaseTools: Fix old python2 idioms > BaseTools: Replace StringIO.StringIO with io.BytesIO > > .../Bin/CYGWIN_NT-5.1-i686/armcc_wrapper.py | 5 +- > BaseTools/Scripts/BinToPcd.py | 7 +- > BaseTools/Scripts/ConvertUni.py | 5 - > BaseTools/Scripts/FormatDosFiles.py | 3 +- > BaseTools/Scripts/MemoryProfileSymbolGen.py | 21 +- > .../PackageDocumentTools/packagedoc_cli.py | 47 ++-- > .../plugins/EdkPlugins/basemodel/doxygen.py | 11 +- > .../plugins/EdkPlugins/basemodel/efibinary.py | 29 +- > .../plugins/EdkPlugins/basemodel/ini.py | 4 +- > .../EdkPlugins/edk2/model/baseobject.py | 6 +- > .../EdkPlugins/edk2/model/doxygengen.py | 2 +- > .../EdkPlugins/edk2/model/doxygengen_spec.py | 2 +- > .../plugins/EdkPlugins/edk2/model/inf.py | 8 +- > BaseTools/Scripts/PatchCheck.py | 2 +- > BaseTools/Scripts/RunMakefile.py | 2 +- > .../Scripts/SmiHandlerProfileSymbolGen.py | 19 +- > BaseTools/Scripts/UpdateBuildVersions.py | 18 +- > BaseTools/Source/Python/AutoGen/AutoGen.py | 77 +++--- > .../Source/Python/AutoGen/BuildEngine.py | 37 +-- > BaseTools/Source/Python/AutoGen/GenC.py | 72 ++--- > BaseTools/Source/Python/AutoGen/GenDepex.py | 8 +- > BaseTools/Source/Python/AutoGen/GenMake.py | 8 +- > BaseTools/Source/Python/AutoGen/GenPcdDb.py | 118 ++++---- > BaseTools/Source/Python/AutoGen/GenVar.py | 160 +++++------ > .../Source/Python/AutoGen/IdfClassObject.py | 1 - > BaseTools/Source/Python/AutoGen/StrGather.py | 8 +- > .../Source/Python/AutoGen/UniClassObject.py | 17 +- > .../Python/AutoGen/ValidCheckingInfoObject.py | 4 +- > BaseTools/Source/Python/BPDG/BPDG.py | 3 +- > BaseTools/Source/Python/BPDG/GenVpd.py | 18 +- > BaseTools/Source/Python/Common/DataType.py | 4 +- > BaseTools/Source/Python/Common/Expression.py | 77 +++--- > .../Source/Python/Common/LongFilePathOs.py | 2 +- > BaseTools/Source/Python/Common/Misc.py | 49 ++-- > .../Source/Python/Common/RangeExpression.py | 33 +-- > BaseTools/Source/Python/Common/StringUtils.py | 6 +- > .../Python/Common/TargetTxtClassObject.py | 7 +- > .../Python/Common/ToolDefClassObject.py | 8 +- > BaseTools/Source/Python/Common/VpdInfoFile.py | 23 +- > BaseTools/Source/Python/Ecc/CParser.py | 175 ++++++------ > BaseTools/Source/Python/Ecc/Check.py | 14 +- > .../Python/Ecc/CodeFragmentCollector.py | 69 ++--- > BaseTools/Source/Python/Ecc/Configuration.py | 5 +- > BaseTools/Source/Python/Ecc/Exception.py | 3 +- > .../Ecc/MetaFileWorkspace/MetaDataTable.py | 5 +- > .../Ecc/MetaFileWorkspace/MetaFileParser.py | 42 +-- > .../Source/Python/Ecc/Xml/XmlRoutines.py | 9 +- > BaseTools/Source/Python/Ecc/c.py | 15 +- > BaseTools/Source/Python/Eot/CParser.py | 175 ++++++------ > .../Python/Eot/CodeFragmentCollector.py | 61 +++-- > BaseTools/Source/Python/Eot/InfParserLite.py | 7 +- > BaseTools/Source/Python/Eot/Parser.py | 2 +- > BaseTools/Source/Python/Eot/c.py | 23 +- > .../Source/Python/GenFds/AprioriSection.py | 6 +- > BaseTools/Source/Python/GenFds/Capsule.py | 10 +- > BaseTools/Source/Python/GenFds/CapsuleData.py | 6 +- > BaseTools/Source/Python/GenFds/EfiSection.py | 6 +- > BaseTools/Source/Python/GenFds/Fd.py | 12 +- > BaseTools/Source/Python/GenFds/FdfParser.py | 45 +-- > .../Source/Python/GenFds/FfsFileStatement.py | 4 +- > .../Source/Python/GenFds/FfsInfStatement.py | 18 +- > BaseTools/Source/Python/GenFds/Fv.py | 10 +- > .../Source/Python/GenFds/FvImageSection.py | 8 +- > BaseTools/Source/Python/GenFds/GenFds.py | 17 +- > .../Python/GenFds/GenFdsGlobalVariable.py | 9 +- > BaseTools/Source/Python/GenFds/OptionRom.py | 3 - > BaseTools/Source/Python/GenFds/Region.py | 11 +- > .../GenPatchPcdTable/GenPatchPcdTable.py | 9 +- > .../Source/Python/Pkcs7Sign/Pkcs7Sign.py | 31 ++- > .../Rsa2048Sha256GenerateKeys.py | 25 +- > .../Rsa2048Sha256Sign/Rsa2048Sha256Sign.py | 35 +-- > .../Source/Python/TargetTool/TargetTool.py | 39 +-- > BaseTools/Source/Python/Trim/Trim.py | 24 +- > .../Source/Python/UPT/Core/DependencyRules.py | 12 +- > .../UPT/Core/DistributionPackageClass.py | 4 +- > BaseTools/Source/Python/UPT/Core/FileHook.py | 2 +- > BaseTools/Source/Python/UPT/Core/IpiDb.py | 6 +- > .../Source/Python/UPT/Core/PackageFile.py | 12 +- > .../Python/UPT/GenMetaFile/GenDecFile.py | 15 +- > .../Python/UPT/GenMetaFile/GenInfFile.py | 37 +-- > BaseTools/Source/Python/UPT/InstallPkg.py | 2 +- > BaseTools/Source/Python/UPT/InventoryWs.py | 2 +- > .../Python/UPT/Library/CommentParsing.py | 2 +- > .../Python/UPT/Library/ExpressionValidate.py | 11 +- > BaseTools/Source/Python/UPT/Library/Misc.py | 6 +- > .../Python/UPT/Library/ParserValidate.py | 2 +- > .../Source/Python/UPT/Library/StringUtils.py | 4 +- > .../Python/UPT/Library/UniClassObject.py | 17 +- > .../Python/UPT/Library/Xml/XmlRoutines.py | 4 +- > BaseTools/Source/Python/UPT/MkPkg.py | 2 +- > .../UPT/Object/Parser/InfBinaryObject.py | 6 +- > .../UPT/Object/Parser/InfDefineObject.py | 2 +- > .../Python/UPT/Object/Parser/InfGuidObject.py | 4 +- > .../Object/Parser/InfLibraryClassesObject.py | 2 +- > .../Python/UPT/Object/Parser/InfMisc.py | 4 +- > .../UPT/Object/Parser/InfPackagesObject.py | 4 +- > .../Python/UPT/Object/Parser/InfPcdObject.py | 4 +- > .../Python/UPT/Object/Parser/InfPpiObject.py | 4 +- > .../UPT/Object/Parser/InfProtocolObject.py | 2 +- > .../UPT/Object/Parser/InfSoucesObject.py | 3 +- > .../Object/Parser/InfUserExtensionObject.py | 4 +- > .../Python/UPT/PomAdapter/DecPomAlignment.py | 56 ++-- > .../Python/UPT/PomAdapter/InfPomAlignment.py | 3 +- > .../UPT/PomAdapter/InfPomAlignmentMisc.py | 3 +- > BaseTools/Source/Python/UPT/ReplacePkg.py | 2 +- > BaseTools/Source/Python/UPT/RmPkg.py | 2 +- > BaseTools/Source/Python/UPT/TestInstall.py | 4 +- > BaseTools/Source/Python/UPT/UPT.py | 8 +- > .../Python/UPT/UnitTest/DecParserTest.py | 5 +- > .../UPT/UnitTest/InfBinarySectionTest.py | 9 +- > BaseTools/Source/Python/UPT/Xml/CommonXml.py | 2 +- > BaseTools/Source/Python/UPT/Xml/XmlParser.py | 24 +- > .../Python/Workspace/BuildClassObject.py | 16 +- > .../Source/Python/Workspace/DecBuildData.py | 22 +- > .../Source/Python/Workspace/DscBuildData.py | 259 +++++++++--------- > .../Source/Python/Workspace/InfBuildData.py | 2 +- > .../Source/Python/Workspace/MetaFileParser.py | 69 ++--- > .../Source/Python/Workspace/MetaFileTable.py | 10 +- > .../Python/Workspace/WorkspaceCommon.py | 2 +- > BaseTools/Source/Python/build/BuildReport.py | 21 +- > BaseTools/Source/Python/build/build.py | 39 +-- > BaseTools/Tests/CheckPythonSyntax.py | 2 +- > BaseTools/Tests/TestTools.py | 10 +- > BaseTools/Tests/TianoCompress.py | 5 +- > BaseTools/gcc/mingw-gcc-build.py | 111 ++++---- > 125 files changed, 1376 insertions(+), 1353 deletions(-) > > -- > 2.17.1 > > _______________________________________________ > edk2-devel mailing list > edk2-devel@lists.01.org > https://lists.01.org/mailman/listinfo/edk2-devel >