From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <bob.c.feng@intel.com>
Received-SPF: Pass (sender SPF authorized) identity=mailfrom;
 client-ip=134.134.136.20; helo=mga02.intel.com;
 envelope-from=bob.c.feng@intel.com; receiver=edk2-devel@lists.01.org 
Received: from mga02.intel.com (mga02.intel.com [134.134.136.20])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by ml01.01.org (Postfix) with ESMTPS id 372892194D387
 for <edk2-devel@lists.01.org>; Mon, 28 Jan 2019 18:06:40 -0800 (PST)
X-Amp-Result: SKIPPED(no attachment in message)
X-Amp-File-Uploaded: False
Received: from orsmga005.jf.intel.com ([10.7.209.41])
 by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384;
 28 Jan 2019 18:06:40 -0800
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.56,535,1539673200"; d="scan'208";a="295229830"
Received: from shwdepsi1121.ccr.corp.intel.com ([10.239.158.47])
 by orsmga005.jf.intel.com with ESMTP; 28 Jan 2019 18:06:37 -0800
From: "Feng, Bob C" <bob.c.feng@intel.com>
To: edk2-devel@lists.01.org
Cc: Zhijux Fan <zhijux.fan@intel.com>, Bob Feng <bob.c.feng@intel.com>,
 Liming Gao <liming.gao@intel.com>, Yonghong Zhu <yonghong.zhu@intel.com>
Date: Tue, 29 Jan 2019 10:05:51 +0800
Message-Id: <20190129020610.14300-15-bob.c.feng@intel.com>
X-Mailer: git-send-email 2.20.1.windows.1
In-Reply-To: <20190129020610.14300-1-bob.c.feng@intel.com>
References: <20190129020610.14300-1-bob.c.feng@intel.com>
MIME-Version: 1.0
Subject: [Patch 14/33] BaseTools/Scripts: Porting PackageDocumentTools code to use Python3
X-BeenThere: edk2-devel@lists.01.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: EDK II Development  <edk2-devel.lists.01.org>
List-Unsubscribe: <https://lists.01.org/mailman/options/edk2-devel>,
 <mailto:edk2-devel-request@lists.01.org?subject=unsubscribe>
List-Archive: <http://lists.01.org/pipermail/edk2-devel/>
List-Post: <mailto:edk2-devel@lists.01.org>
List-Help: <mailto:edk2-devel-request@lists.01.org?subject=help>
List-Subscribe: <https://lists.01.org/mailman/listinfo/edk2-devel>,
 <mailto:edk2-devel-request@lists.01.org?subject=subscribe>
X-List-Received-Date: Tue, 29 Jan 2019 02:06:40 -0000
Content-Transfer-Encoding: 8bit

From: Zhijux Fan <zhijux.fan@intel.com>

Porting PackageDocumentTools code to support python2 and python3

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
---
 BaseTools/Scripts/ConvertFceToStructurePcd.py                                           |  2 +-
 BaseTools/Scripts/PackageDocumentTools/packagedoc_cli.py                                |  6 +++---
 BaseTools/Scripts/PackageDocumentTools/packagedocapp.pyw                                | 14 +++++++-------
 BaseTools/Scripts/PackageDocumentTools/plugins/EdkPlugins/basemodel/doxygen.py          |  4 ++--
 BaseTools/Scripts/PackageDocumentTools/plugins/EdkPlugins/edk2/model/baseobject.py      | 16 ++++++++--------
 BaseTools/Scripts/PackageDocumentTools/plugins/EdkPlugins/edk2/model/dec.py             |  4 ++--
 BaseTools/Scripts/PackageDocumentTools/plugins/EdkPlugins/edk2/model/doxygengen.py      | 12 ++++++------
 BaseTools/Scripts/PackageDocumentTools/plugins/EdkPlugins/edk2/model/doxygengen_spec.py | 12 ++++++------
 BaseTools/Scripts/PackageDocumentTools/plugins/EdkPlugins/edk2/model/dsc.py             |  4 ++--
 BaseTools/Scripts/PackageDocumentTools/plugins/EdkPlugins/edk2/model/inf.py             |  4 ++--
 10 files changed, 39 insertions(+), 39 deletions(-)

diff --git a/BaseTools/Scripts/ConvertFceToStructurePcd.py b/BaseTools/Scripts/ConvertFceToStructurePcd.py
index 59eec28d5e..1495ac34d6 100644
--- a/BaseTools/Scripts/ConvertFceToStructurePcd.py
+++ b/BaseTools/Scripts/ConvertFceToStructurePcd.py
@@ -133,11 +133,11 @@ class parser_lst(object):
                       offset = int(offset, 10)
                       tmp_name = pcdname2_re.findall(t_name)[0] + '[0]'
                       tmp_dict[offset] = tmp_name
                       pcdname_num = int(pcdname_num_re.findall(t_name)[0],10)
                       uint = int(unit_num.findall(uint)[0],10)
-                      bit = uint / 8
+                      bit = uint // 8
                       for i in range(1, pcdname_num):
                         offset += bit
                         tmp_name = pcdname2_re.findall(t_name)[0] + '[%s]' % i
                         tmp_dict[offset] = tmp_name
                     else:
diff --git a/BaseTools/Scripts/PackageDocumentTools/packagedoc_cli.py b/BaseTools/Scripts/PackageDocumentTools/packagedoc_cli.py
index 4deeee01a5..e404a07cd7 100644
--- a/BaseTools/Scripts/PackageDocumentTools/packagedoc_cli.py
+++ b/BaseTools/Scripts/PackageDocumentTools/packagedoc_cli.py
@@ -14,12 +14,12 @@
 
 from __future__ import print_function
 import os, sys, logging, traceback, subprocess
 from optparse import OptionParser
 
-import plugins.EdkPlugins.edk2.model.baseobject as baseobject
-import plugins.EdkPlugins.edk2.model.doxygengen as doxygengen
+from .plugins.EdkPlugins.edk2.model import baseobject
+from .plugins.EdkPlugins.edk2.model import doxygengen
 
 gArchMarcoDict = {'ALL'      : 'MDE_CPU_IA32 MDE_CPU_X64 MDE_CPU_EBC MDE_CPU_IPF _MSC_EXTENSIONS __GNUC__ __INTEL_COMPILER',
                   'IA32_MSFT': 'MDE_CPU_IA32 _MSC_EXTENSIONS',
                   'IA32_GNU' : 'MDE_CPU_IA32 __GNUC__',
                   'X64_MSFT' : 'MDE_CPU_X64 _MSC_EXTENSIONS  ASM_PFX= OPTIONAL= ',
@@ -36,11 +36,11 @@ def parseCmdArgs():
                       help='Specify the absolute path for package DEC file. For example: c:\\tianocore\\MdePkg\\MdePkg.dec')
     parser.add_option('-x', '--doxygen', action='store', dest='DoxygenPath',
                       help='Specify the absolute path of doxygen tools installation. For example: C:\\Program Files\\doxygen\bin\doxygen.exe')
     parser.add_option('-o', '--output', action='store', dest='OutputPath',
                       help='Specify the document output path. For example: c:\\docoutput')
-    parser.add_option('-a', '--arch', action='store', dest='Arch', choices=gArchMarcoDict.keys(),
+    parser.add_option('-a', '--arch', action='store', dest='Arch', choices=list(gArchMarcoDict.keys()),
                       help='Specify the architecture used in preprocess package\'s source. For example: -a IA32_MSFT')
     parser.add_option('-m', '--mode', action='store', dest='DocumentMode', choices=['CHM', 'HTML'],
                       help='Specify the document mode from : CHM or HTML')
     parser.add_option('-i', '--includeonly', action='store_true', dest='IncludeOnly',
                       help='Only generate document for package\'s public interfaces produced by include folder. ')
diff --git a/BaseTools/Scripts/PackageDocumentTools/packagedocapp.pyw b/BaseTools/Scripts/PackageDocumentTools/packagedocapp.pyw
index 28f6f9bf5c..2998db3915 100644
--- a/BaseTools/Scripts/PackageDocumentTools/packagedocapp.pyw
+++ b/BaseTools/Scripts/PackageDocumentTools/packagedocapp.pyw
@@ -16,12 +16,12 @@
 import os, sys, wx, logging
 
 import wx.stc
 import wx.lib.newevent
 import wx.lib.agw.genericmessagedialog as GMD
-import plugins.EdkPlugins.edk2.model.baseobject as baseobject
-import plugins.EdkPlugins.edk2.model.doxygengen as doxygengen
+from plugins.EdkPlugins.edk2.model import baseobject
+from plugins.EdkPlugins.edk2.model import doxygengen
 
 if hasattr(sys, "frozen"):
     appPath = os.path.abspath(os.path.dirname(sys.executable))
 else:
     appPath = os.path.abspath(os.path.dirname(__file__))
@@ -718,11 +718,11 @@ class ProgressDialog(wx.Dialog):
             lines = []
             f = open (path_html, "r")
             lines = f.readlines()
             f.close()
             bfound = False
-            for index in xrange(len(lines)):
+            for index in range(len(lines)):
                 if lines[index].find('<a class="el" href="files.html" target="basefrm">File List</a>') != -1:
                     lines[index] = "<!-- %s" % lines[index]
                     bfound = True
                     continue
                 if bfound:
@@ -967,11 +967,11 @@ class ProgressDialog(wx.Dialog):
         self.LogMessage('    >>> Fixup .dox postfix for file %s \n' % path)
         try:
             fd = open(path, 'r')
             text = fd.read()
             fd.close()
-        except Exception, e:
+        except Exception as e:
             self.LogMessage ("   <<<Fail to open file %s" % path)
             return
         text = text.replace ('.s.dox', '.s')
         text = text.replace ('.S.dox', '.S')
         text = text.replace ('.asm.dox', '.asm')
@@ -980,33 +980,33 @@ class ProgressDialog(wx.Dialog):
         text = text.replace ('.Uni.dox', '.Uni')
         try:
             fd = open(path, 'w')
             fd.write(text)
             fd.close()
-        except Exception, e:
+        except Exception as e:
             self.LogMessage ("    <<<Fail to fixup file %s" % path)
             return
         self.LogMessage('    >>> Finish to fixup .dox postfix for file %s \n' % path)
 
     def FixDecDoxygenFileLink(self, path, text):
         self.LogMessage('    >>> Fixup .decdoxygen postfix for file %s \n' % path)
         try:
             fd = open(path, 'r')
             lines = fd.readlines()
             fd.close()
-        except Exception, e:
+        except Exception as e:
             self.LogMessage ("   <<<Fail to open file %s" % path)
             return
         for line in lines:
             if line.find('.decdoxygen') != -1:
                 lines.remove(line)
                 break
         try:
             fd = open(path, 'w')
             fd.write("".join(lines))
             fd.close()
-        except Exception, e:
+        except Exception as e:
             self.LogMessage ("    <<<Fail to fixup file %s" % path)
             return
         self.LogMessage('    >>> Finish to fixup .decdoxygen postfix for file %s \n' % path)
 
 import threading
diff --git a/BaseTools/Scripts/PackageDocumentTools/plugins/EdkPlugins/basemodel/doxygen.py b/BaseTools/Scripts/PackageDocumentTools/plugins/EdkPlugins/basemodel/doxygen.py
index d1e21135cf..ae47ff1344 100644
--- a/BaseTools/Scripts/PackageDocumentTools/plugins/EdkPlugins/basemodel/doxygen.py
+++ b/BaseTools/Scripts/PackageDocumentTools/plugins/EdkPlugins/basemodel/doxygen.py
@@ -65,11 +65,11 @@ class Page(BaseDoxygeItem):
         for page in pageArray:
             self.AddPage(page)
 
     def AddSection(self, section):
         self.mSections.append(section)
-        self.mSections.sort(cmp=lambda x, y: cmp(x.mName.lower(), y.mName.lower()))
+        self.mSections.sort(key=lambda x: x.mName.lower())
 
     def Generate(self):
         if self.mIsMainPage:
             self.mText.append('/** \mainpage %s' % self.mName)
             self.mIsSort = False
@@ -78,11 +78,11 @@ class Page(BaseDoxygeItem):
 
         if len(self.mDescription) != 0:
             self.mText.append(self.mDescription)
         endIndex = len(self.mText)
 
-        self.mSections.sort()
+        self.mSections.sort(key=lambda x: x.mName.lower())
         for sect in self.mSections:
             self.mText += sect.Generate()
 
         endIndex = len(self.mText)
 
diff --git a/BaseTools/Scripts/PackageDocumentTools/plugins/EdkPlugins/edk2/model/baseobject.py b/BaseTools/Scripts/PackageDocumentTools/plugins/EdkPlugins/edk2/model/baseobject.py
index b49c87c8bd..0159bd5269 100644
--- a/BaseTools/Scripts/PackageDocumentTools/plugins/EdkPlugins/edk2/model/baseobject.py
+++ b/BaseTools/Scripts/PackageDocumentTools/plugins/EdkPlugins/edk2/model/baseobject.py
@@ -8,16 +8,16 @@
 # http://opensource.org/licenses/bsd-license.php
 #
 # THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS,
 # WITHOUT WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED.
 
-import plugins.EdkPlugins.basemodel.ini as ini
-import plugins.EdkPlugins.edk2.model.dsc as dsc
-import plugins.EdkPlugins.edk2.model.inf as inf
-import plugins.EdkPlugins.edk2.model.dec as dec
+from ...basemodel import ini
+from ...edk2.model import dsc
+from ...edk2.model import inf
+from ...edk2.model import dec
 import os
-from plugins.EdkPlugins.basemodel.message import *
+from ...basemodel.message import *
 
 class SurfaceObject(object):
     _objs = {}
 
     def __new__(cls, *args, **kwargs):
@@ -653,17 +653,17 @@ class Package(SurfaceObject):
 
     def GetPcds(self):
         return self._pcds
 
     def GetPpis(self):
-        return self._ppis.values()
+        return list(self._ppis.values())
 
     def GetProtocols(self):
-        return self._protocols.values()
+        return list(self._protocols.values())
 
     def GetGuids(self):
-        return self._guids.values()
+        return list(self._guids.values())
 
     def Destroy(self):
         for pcd in self._pcds.values():
             if pcd is not None:
                 pcd.Destroy()
diff --git a/BaseTools/Scripts/PackageDocumentTools/plugins/EdkPlugins/edk2/model/dec.py b/BaseTools/Scripts/PackageDocumentTools/plugins/EdkPlugins/edk2/model/dec.py
index 9ff0df3851..3d210f72ac 100644
--- a/BaseTools/Scripts/PackageDocumentTools/plugins/EdkPlugins/edk2/model/dec.py
+++ b/BaseTools/Scripts/PackageDocumentTools/plugins/EdkPlugins/edk2/model/dec.py
@@ -9,13 +9,13 @@
 #
 # THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS,
 # WITHOUT WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED.
 #
 
-import plugins.EdkPlugins.basemodel.ini as ini
+from ...basemodel import ini
 import re, os
-from plugins.EdkPlugins.basemodel.message import *
+from ...basemodel.message import *
 
 class DECFile(ini.BaseINIFile):
 
     def GetSectionInstance(self, parent, name, isCombined=False):
         return DECSection(parent, name, isCombined)
diff --git a/BaseTools/Scripts/PackageDocumentTools/plugins/EdkPlugins/edk2/model/doxygengen.py b/BaseTools/Scripts/PackageDocumentTools/plugins/EdkPlugins/edk2/model/doxygengen.py
index c22d362ff3..9c299fbfc5 100644
--- a/BaseTools/Scripts/PackageDocumentTools/plugins/EdkPlugins/edk2/model/doxygengen.py
+++ b/BaseTools/Scripts/PackageDocumentTools/plugins/EdkPlugins/edk2/model/doxygengen.py
@@ -14,21 +14,21 @@
 # WITHOUT WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED.
 
 """This file produce action class to generate doxygen document for edk2 codebase.
    The action classes are shared by GUI and command line tools.
 """
-import plugins.EdkPlugins.basemodel.doxygen as doxygen
+from ...basemodel import doxygen
 import os
 try:
     import wx
     gInGui = True
 except:
     gInGui = False
 import re
-import plugins.EdkPlugins.edk2.model.inf as inf
-import plugins.EdkPlugins.edk2.model.dec as dec
-from plugins.EdkPlugins.basemodel.message import *
+from ...edk2.model import inf
+from ...edk2.model import dec
+from ...basemodel.message import *
 
 _ignore_dir = ['.svn', '_svn', 'cvs']
 _inf_key_description_mapping_table = {
   'INF_VERSION':'Version of INF file specification',
   #'BASE_NAME':'Module Name',
@@ -384,11 +384,11 @@ class PackageDocumentAction(DoxygenAction):
             return
 
         configFile.AddFile(path)
 
         no = 0
-        for no in xrange(len(lines)):
+        for no in range(len(lines)):
             if len(lines[no].strip()) == 0:
                 continue
             if lines[no].strip()[:2] in ['##', '//', '/*', '*/']:
                 continue
             index = lines[no].lower().find('include')
@@ -998,11 +998,11 @@ class PackageDocumentAction(DoxygenAction):
         newpath = path + '.dox'
         #import core.textfile as textfile
         #file = textfile.TextFile(path)
 
         try:
-            file = open(path, 'rb')
+            file = open(path, 'r')
         except (IOError, OSError) as msg:
             return None
 
         t = file.read()
         file.close()
diff --git a/BaseTools/Scripts/PackageDocumentTools/plugins/EdkPlugins/edk2/model/doxygengen_spec.py b/BaseTools/Scripts/PackageDocumentTools/plugins/EdkPlugins/edk2/model/doxygengen_spec.py
index 4bae6968a9..3a862a92ea 100644
--- a/BaseTools/Scripts/PackageDocumentTools/plugins/EdkPlugins/edk2/model/doxygengen_spec.py
+++ b/BaseTools/Scripts/PackageDocumentTools/plugins/EdkPlugins/edk2/model/doxygengen_spec.py
@@ -11,21 +11,21 @@
 # http://opensource.org/licenses/bsd-license.php
 #
 # THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS,
 # WITHOUT WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED.
 
-import plugins.EdkPlugins.basemodel.doxygen as doxygen
+from ...basemodel import doxygen
 import os
 try:
     import wx
     gInGui = True
 except:
     gInGui = False
 import re
-import plugins.EdkPlugins.edk2.model.inf as inf
-import plugins.EdkPlugins.edk2.model.dec as dec
-from plugins.EdkPlugins.basemodel.message import *
+from ...edk2.model import inf
+from ...edk2.model import dec
+from ...basemodel.message import *
 
 _ignore_dir = ['.svn', '_svn', 'cvs']
 _inf_key_description_mapping_table = {
   'INF_VERSION':'Version of INF file specification',
   #'BASE_NAME':'Module Name',
@@ -386,11 +386,11 @@ class PackageDocumentAction(DoxygenAction):
             return
 
         configFile.AddFile(path)
         return
         no = 0
-        for no in xrange(len(lines)):
+        for no in range(len(lines)):
             if len(lines[no].strip()) == 0:
                 continue
             if lines[no].strip()[:2] in ['##', '//', '/*', '*/']:
                 continue
             index = lines[no].lower().find('include')
@@ -1001,11 +1001,11 @@ class PackageDocumentAction(DoxygenAction):
         newpath = path + '.dox'
         #import core.textfile as textfile
         #file = textfile.TextFile(path)
 
         try:
-            file = open(path, 'rb')
+            file = open(path, 'r')
         except (IOError, OSError) as msg:
             return None
 
         t = file.read()
         file.close()
diff --git a/BaseTools/Scripts/PackageDocumentTools/plugins/EdkPlugins/edk2/model/dsc.py b/BaseTools/Scripts/PackageDocumentTools/plugins/EdkPlugins/edk2/model/dsc.py
index 0628fa7408..6f59e566b8 100644
--- a/BaseTools/Scripts/PackageDocumentTools/plugins/EdkPlugins/edk2/model/dsc.py
+++ b/BaseTools/Scripts/PackageDocumentTools/plugins/EdkPlugins/edk2/model/dsc.py
@@ -9,13 +9,13 @@
 #
 # THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS,
 # WITHOUT WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED.
 #
 
-import plugins.EdkPlugins.basemodel.ini as ini
+from ...basemodel import ini
 import re, os
-from plugins.EdkPlugins.basemodel.message import *
+from ...basemodel.message import *
 
 class DSCFile(ini.BaseINIFile):
     def GetSectionInstance(self, parent, name, isCombined=False):
         return DSCSection(parent, name, isCombined)
 
diff --git a/BaseTools/Scripts/PackageDocumentTools/plugins/EdkPlugins/edk2/model/inf.py b/BaseTools/Scripts/PackageDocumentTools/plugins/EdkPlugins/edk2/model/inf.py
index 793e95efed..cf2e49d3af 100644
--- a/BaseTools/Scripts/PackageDocumentTools/plugins/EdkPlugins/edk2/model/inf.py
+++ b/BaseTools/Scripts/PackageDocumentTools/plugins/EdkPlugins/edk2/model/inf.py
@@ -9,13 +9,13 @@
 #
 # THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS,
 # WITHOUT WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED.
 #
 
-import plugins.EdkPlugins.basemodel.ini as ini
+from ...basemodel import ini
 import re, os
-from plugins.EdkPlugins.basemodel.message import *
+from ...basemodel.message import *
 
 class INFFile(ini.BaseINIFile):
     _libobjs = {}
 
     def GetSectionInstance(self, parent, name, isCombined=False):
-- 
2.20.1.windows.1