From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=134.134.136.126; helo=mga18.intel.com; envelope-from=yonghong.zhu@intel.com; receiver=edk2-devel@lists.01.org Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 3DA8D211F887E for ; Mon, 25 Jun 2018 21:48:37 -0700 (PDT) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by orsmga106.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 25 Jun 2018 21:48:36 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,273,1526367600"; d="scan'208";a="60174383" Received: from fmsmsx108.amr.corp.intel.com ([10.18.124.206]) by FMSMGA003.fm.intel.com with ESMTP; 25 Jun 2018 21:48:20 -0700 Received: from fmsmsx111.amr.corp.intel.com (10.18.116.5) by FMSMSX108.amr.corp.intel.com (10.18.124.206) with Microsoft SMTP Server (TLS) id 14.3.319.2; Mon, 25 Jun 2018 21:48:20 -0700 Received: from shsmsx101.ccr.corp.intel.com (10.239.4.153) by fmsmsx111.amr.corp.intel.com (10.18.116.5) with Microsoft SMTP Server (TLS) id 14.3.319.2; Mon, 25 Jun 2018 21:48:19 -0700 Received: from shsmsx103.ccr.corp.intel.com ([169.254.4.51]) by SHSMSX101.ccr.corp.intel.com ([169.254.1.82]) with mapi id 14.03.0319.002; Tue, 26 Jun 2018 12:48:18 +0800 From: "Zhu, Yonghong" To: Gary Lin , "edk2-devel@lists.01.org" CC: "Gao, Liming" , "Zhu, Yonghong" Thread-Topic: [PATCH v4 00/20] BaseTools: One step toward python3 Thread-Index: AQHUDG/N+nKbc6OsoEGHLHjF/xWUsKRx+JWQ Date: Tue, 26 Jun 2018 04:48:17 +0000 Message-ID: References: <20180625103136.12531-1-glin@suse.com> In-Reply-To: <20180625103136.12531-1-glin@suse.com> Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.239.127.40] MIME-Version: 1.0 Subject: Re: [PATCH v4 00/20] BaseTools: One step toward python3 X-BeenThere: edk2-devel@lists.01.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: EDK II Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jun 2018 04:48:37 -0000 Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Gary, The V4 patches are good to me. I will push this series in recent days if no= other comment from community. Reviewed-by: Yonghong Zhu =20 Best Regards, Zhu Yonghong -----Original Message----- From: Gary Lin [mailto:glin@suse.com]=20 Sent: Monday, June 25, 2018 6:31 PM To: edk2-devel@lists.01.org Cc: Zhu, Yonghong ; Gao, Liming Subject: [PATCH v4 00/20] BaseTools: One step toward python3 v4 changes: - Remove the range() patch since it needs python-future - Remove the patch to unify long and int since it caused error in windows. - Split the absolute import patches and will introduce them later v3 changes: - Rebase to the current git HEAD (2e1083038d9aa74fcaa2db8158fdee7c8b4af3b= b) - Fix a typo in BaseTools/Scripts/PackageDocumentTools/plugins/EdkPlugins= /basemodel/ini.py - Remove the patch for reduce() since it's not used anymore=20 v2 changes: - Rebase to the current git HEAD (821807bcefb9a36e598d71a8004fae5aab2052a= 0) - Apply "futurize -f libfuturize.fixes.fix_absolute_import" and refactor some python scripts to break the circular imports. This patch series is also available in https://github.com/lcp/edk2/tree/python3-futurize-v3 Since python2 will be EOL in 2020, we start to evaluate the impact of the p= ython2 removal. As expected, OMVF building failed the test. It's actually a= task noted in the wiki page: https://github.com/tianocore/tianocore.github.io/wiki/Tasks-BaseTools-Pytho= n3-Support Maybe it's time to convert the python scripts gradually. This patchset doesn't make the python scripts in BaseTools compatible with = python3 immediately. It aims to do the trivial and safe conversion and repl= acement to make some statements compatible with both python2 and python3, s= o we can deal with the difficult cases later. With the help of "futurize" from python-future, it's easier to refactor the= statements. This patchset is basically equivalent to "futurize -1"(*) plus= "StringIO.StringIO =3D> io.BytesIO" and minus the absolute import. The patchset was tested with the following command in openSUSE Tumbleweed: $ ./OvmfPkg/build.sh -D SECURE_BOOT_ENABLE \ -D NETWORK_IP6_ENABLE \ -D HTTP_BOOT_ENABLE \ -D TLS_ENABLE \ -D TPM2_ENABLE The firmware file was built successfully and I didn't notice any error so f= ar. Testing with other platform is welcome. (*) http://python-future.org/automatic_conversion.html#stage-1-safe-fixes Contributed-under: TianoCore Contribution Agreement 1.1 Cc: Yonghong Zhu Cc: Liming Gao Signed-off-by: Gary Lin Gary Lin (13): BaseTools: Fix a typo in ini.py BaseTools: Refactor python except statements BaseTools: Refactor python print statements BaseTools: Remove the old python "not-equal" BaseTools: Remove tuple parameter in python scripts BaseTools: Remove the deprecated hash_key() BaseTools: Replace StandardError with Expression BaseTools: Remove types.TypeType BaseTools: Refactor python raise statement BaseTools: Adjust the spaces around commas and colons BaseTools: Migrate to the new octal literal BaseTools: Fix old python2 idioms BaseTools: Replace StringIO.StringIO with io.BytesIO .../Bin/CYGWIN_NT-5.1-i686/armcc_wrapper.py | 5 +- BaseTools/Scripts/BinToPcd.py | 7 +- BaseTools/Scripts/ConvertUni.py | 5 - BaseTools/Scripts/FormatDosFiles.py | 3 +- BaseTools/Scripts/MemoryProfileSymbolGen.py | 21 +- .../PackageDocumentTools/packagedoc_cli.py | 47 ++-- .../plugins/EdkPlugins/basemodel/doxygen.py | 11 +- .../plugins/EdkPlugins/basemodel/efibinary.py | 29 +- .../plugins/EdkPlugins/basemodel/ini.py | 4 +- .../EdkPlugins/edk2/model/baseobject.py | 6 +- .../EdkPlugins/edk2/model/doxygengen.py | 2 +- .../EdkPlugins/edk2/model/doxygengen_spec.py | 2 +- .../plugins/EdkPlugins/edk2/model/inf.py | 8 +- BaseTools/Scripts/PatchCheck.py | 2 +- BaseTools/Scripts/RunMakefile.py | 2 +- .../Scripts/SmiHandlerProfileSymbolGen.py | 19 +- BaseTools/Scripts/UpdateBuildVersions.py | 18 +- BaseTools/Source/Python/AutoGen/AutoGen.py | 77 +++--- .../Source/Python/AutoGen/BuildEngine.py | 37 +-- BaseTools/Source/Python/AutoGen/GenC.py | 72 ++--- BaseTools/Source/Python/AutoGen/GenDepex.py | 8 +- BaseTools/Source/Python/AutoGen/GenMake.py | 8 +- BaseTools/Source/Python/AutoGen/GenPcdDb.py | 118 ++++---- BaseTools/Source/Python/AutoGen/GenVar.py | 160 +++++------ .../Source/Python/AutoGen/IdfClassObject.py | 1 - BaseTools/Source/Python/AutoGen/StrGather.py | 8 +- .../Source/Python/AutoGen/UniClassObject.py | 17 +- .../Python/AutoGen/ValidCheckingInfoObject.py | 4 +- BaseTools/Source/Python/BPDG/BPDG.py | 3 +- BaseTools/Source/Python/BPDG/GenVpd.py | 18 +- BaseTools/Source/Python/Common/DataType.py | 4 +- BaseTools/Source/Python/Common/Expression.py | 77 +++--- .../Source/Python/Common/LongFilePathOs.py | 2 +- BaseTools/Source/Python/Common/Misc.py | 49 ++-- .../Source/Python/Common/RangeExpression.py | 33 +-- BaseTools/Source/Python/Common/StringUtils.py | 6 +- .../Python/Common/TargetTxtClassObject.py | 7 +- .../Python/Common/ToolDefClassObject.py | 8 +- BaseTools/Source/Python/Common/VpdInfoFile.py | 23 +- BaseTools/Source/Python/Ecc/CParser.py | 175 ++++++------ BaseTools/Source/Python/Ecc/Check.py | 14 +- .../Python/Ecc/CodeFragmentCollector.py | 69 ++--- BaseTools/Source/Python/Ecc/Configuration.py | 5 +- BaseTools/Source/Python/Ecc/Exception.py | 3 +- .../Ecc/MetaFileWorkspace/MetaDataTable.py | 5 +- .../Ecc/MetaFileWorkspace/MetaFileParser.py | 42 +-- .../Source/Python/Ecc/Xml/XmlRoutines.py | 9 +- BaseTools/Source/Python/Ecc/c.py | 15 +- BaseTools/Source/Python/Eot/CParser.py | 175 ++++++------ .../Python/Eot/CodeFragmentCollector.py | 61 +++-- BaseTools/Source/Python/Eot/InfParserLite.py | 7 +- BaseTools/Source/Python/Eot/Parser.py | 2 +- BaseTools/Source/Python/Eot/c.py | 23 +- .../Source/Python/GenFds/AprioriSection.py | 6 +- BaseTools/Source/Python/GenFds/Capsule.py | 10 +- BaseTools/Source/Python/GenFds/CapsuleData.py | 6 +- BaseTools/Source/Python/GenFds/EfiSection.py | 6 +- BaseTools/Source/Python/GenFds/Fd.py | 12 +- BaseTools/Source/Python/GenFds/FdfParser.py | 45 +-- .../Source/Python/GenFds/FfsFileStatement.py | 4 +- .../Source/Python/GenFds/FfsInfStatement.py | 18 +- BaseTools/Source/Python/GenFds/Fv.py | 10 +- .../Source/Python/GenFds/FvImageSection.py | 8 +- BaseTools/Source/Python/GenFds/GenFds.py | 17 +- .../Python/GenFds/GenFdsGlobalVariable.py | 9 +- BaseTools/Source/Python/GenFds/OptionRom.py | 3 - BaseTools/Source/Python/GenFds/Region.py | 11 +- .../GenPatchPcdTable/GenPatchPcdTable.py | 9 +- .../Source/Python/Pkcs7Sign/Pkcs7Sign.py | 31 ++- .../Rsa2048Sha256GenerateKeys.py | 25 +- .../Rsa2048Sha256Sign/Rsa2048Sha256Sign.py | 35 +-- .../Source/Python/TargetTool/TargetTool.py | 39 +-- BaseTools/Source/Python/Trim/Trim.py | 24 +- .../Source/Python/UPT/Core/DependencyRules.py | 12 +- .../UPT/Core/DistributionPackageClass.py | 4 +- BaseTools/Source/Python/UPT/Core/FileHook.py | 2 +- BaseTools/Source/Python/UPT/Core/IpiDb.py | 6 +- .../Source/Python/UPT/Core/PackageFile.py | 12 +- .../Python/UPT/GenMetaFile/GenDecFile.py | 15 +- .../Python/UPT/GenMetaFile/GenInfFile.py | 37 +-- BaseTools/Source/Python/UPT/InstallPkg.py | 2 +- BaseTools/Source/Python/UPT/InventoryWs.py | 2 +- .../Python/UPT/Library/CommentParsing.py | 2 +- .../Python/UPT/Library/ExpressionValidate.py | 11 +- BaseTools/Source/Python/UPT/Library/Misc.py | 6 +- .../Python/UPT/Library/ParserValidate.py | 2 +- .../Source/Python/UPT/Library/StringUtils.py | 4 +- .../Python/UPT/Library/UniClassObject.py | 17 +- .../Python/UPT/Library/Xml/XmlRoutines.py | 4 +- BaseTools/Source/Python/UPT/MkPkg.py | 2 +- .../UPT/Object/Parser/InfBinaryObject.py | 6 +- .../UPT/Object/Parser/InfDefineObject.py | 2 +- .../Python/UPT/Object/Parser/InfGuidObject.py | 4 +- .../Object/Parser/InfLibraryClassesObject.py | 2 +- .../Python/UPT/Object/Parser/InfMisc.py | 4 +- .../UPT/Object/Parser/InfPackagesObject.py | 4 +- .../Python/UPT/Object/Parser/InfPcdObject.py | 4 +- .../Python/UPT/Object/Parser/InfPpiObject.py | 4 +- .../UPT/Object/Parser/InfProtocolObject.py | 2 +- .../UPT/Object/Parser/InfSoucesObject.py | 3 +- .../Object/Parser/InfUserExtensionObject.py | 4 +- .../Python/UPT/PomAdapter/DecPomAlignment.py | 56 ++-- .../Python/UPT/PomAdapter/InfPomAlignment.py | 3 +- .../UPT/PomAdapter/InfPomAlignmentMisc.py | 3 +- BaseTools/Source/Python/UPT/ReplacePkg.py | 2 +- BaseTools/Source/Python/UPT/RmPkg.py | 2 +- BaseTools/Source/Python/UPT/TestInstall.py | 4 +- BaseTools/Source/Python/UPT/UPT.py | 8 +- .../Python/UPT/UnitTest/DecParserTest.py | 5 +- .../UPT/UnitTest/InfBinarySectionTest.py | 9 +- BaseTools/Source/Python/UPT/Xml/CommonXml.py | 2 +- BaseTools/Source/Python/UPT/Xml/XmlParser.py | 24 +- .../Python/Workspace/BuildClassObject.py | 16 +- .../Source/Python/Workspace/DecBuildData.py | 22 +- .../Source/Python/Workspace/DscBuildData.py | 259 +++++++++--------- .../Source/Python/Workspace/InfBuildData.py | 2 +- .../Source/Python/Workspace/MetaFileParser.py | 69 ++--- .../Source/Pyth= on/Workspace/MetaFileTable.py | 10 +- .../Python/Workspace/WorkspaceCommon.py | 2 +- BaseTools/Source/Python/build/BuildReport.py | 21 +- BaseTools/Source/Python/build/build.py | 39 +-- BaseTools/Tests/CheckPythonSyntax.py | 2 +- BaseTools/Tests/TestTools.py | 10 +- BaseTools/Tests/TianoCompress.py | 5 +- BaseTools/gcc/mingw-gcc-build.py | 111 ++++---- 125 files changed, 1376 insertions(+), 1353 deletions(-) -- 2.17.1