Re: [Patch 1/3] BaseTools: Implement splitquoted function in Build tool

["Carsey, Jaben" <jaben.carsey@intel.com>]

Code looks good, but I think the function comment is lacking.  How about describe the function and what it does instead of saying it replaces distutils function (which while correct, didn't really answer my question of "what is this supposed to do").

> -----Original Message-----
> From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of
> Feng, Bob C
> Sent: Saturday, February 02, 2019 9:55 PM
> To: edk2-devel@lists.01.org
> Cc: Gao, Liming <liming.gao@intel.com>
> Subject: [edk2] [Patch 1/3] BaseTools: Implement splitquoted function in
> Build tool
> 
> https://bugzilla.tianocore.org/show_bug.cgi?id=1509
> On some Linux environment, there may be no distutils.util
> library for python3 that will cause build tool crash.
> This patch implement distutils.util.split_quoted
> in BaseTools so that the build tool will be independent with
> distutils.util library.
> 
> Contributed-under: TianoCore Contribution Agreement 1.1
> Signed-off-by: Bob Feng <bob.c.feng@intel.com>
> Cc: Liming Gao <liming.gao@intel.com>
> ---
>  BaseTools/Source/Python/AutoGen/UniClassObject.py | 50
> ++++++++++++++++++++++++++++++++++++++++++++++----
>  1 file changed, 46 insertions(+), 4 deletions(-)
> 
> diff --git a/BaseTools/Source/Python/AutoGen/UniClassObject.py
> b/BaseTools/Source/Python/AutoGen/UniClassObject.py
> index d162387cc5..c4b39d25d4 100644
> --- a/BaseTools/Source/Python/AutoGen/UniClassObject.py
> +++ b/BaseTools/Source/Python/AutoGen/UniClassObject.py
> @@ -16,11 +16,10 @@
>  ##
>  # Import Modules
>  #
>  from __future__ import print_function
>  import Common.LongFilePathOs as os, codecs, re
> -import distutils.util
>  import Common.EdkLogger as EdkLogger
>  from io import BytesIO
>  from Common.BuildToolError import *
>  from Common.StringUtils import GetLineNo
>  from Common.Misc import PathClass
> @@ -41,11 +40,11 @@ NON_BREAKING_CHAR = u'\uFFF2'
>  CR = u'\u000D'
>  LF = u'\u000A'
>  NULL = u'\u0000'
>  TAB = u'\t'
>  BACK_SLASH_PLACEHOLDER = u'\u0006'
> -
> +WHITESPACE = ' \t\n\r\v\f'
>  gIncludePattern = re.compile("^#include +[\"<]+([^\"< >]+)[>\"]+$",
> re.MULTILINE | re.UNICODE)
> 
>  ## Convert a unicode string to a Hex list
>  #
>  # Convert a unicode string to a Hex list
> @@ -216,10 +215,54 @@ def StripComments(Line):
>              CommentPos = Line.find (Comment, CommentPos + 1)
>          else:
>              return Line[:CommentPos].strip()
>      return Line.strip()
> 
> +#
> +# This function implement distutils.util.split_quoted in another way.
> +#
> +def splitquoted(s):
> +    words = []
> +    word = []
> +    InQuoted = False
> +    escaped = False
> +    CurrentQ = ""
> +    s = s.strip()
> +    for ch in s:
> +        if escaped:
> +            # preserve whatever is escaped;
> +            # This char will become part of the current word
> +            word.append(ch)
> +            escaped = False
> +            continue
> +
> +        if ch in WHITESPACE and not InQuoted:
> +            # unescaped, unquoted whitespace: now
> +            # we definitely have a word delimiter
> +            if "".join(word):
> +                words.append("".join(word))
> +            word = []
> +            continue
> +        elif ch == "\\":
> +            escaped = True
> +            if not InQuoted:
> +                continue
> +        else:
> +            # handle singly-quoted string or doubly-quoted string
> +            if ch =='"' or ch == "'":
> +                if not InQuoted:
> +                    InQuoted = True
> +                    CurrentQ = ch
> +                    continue
> +                elif ch == CurrentQ:
> +                    InQuoted = False
> +                    continue
> +        word.append(ch)
> +    if word:
> +        words.append("".join(word))
> +    return words
> +
>  ## UniFileClassObject
>  #
>  # A structure for .uni file definition
>  #
>  class UniFileClassObject(object):
> @@ -232,16 +275,15 @@ class UniFileClassObject(object):
>          self.OrderedStringListByToken = {}      #{ u'LanguageIdentifier' : {Token:
> StringDefClassObject} }
>          self.IsCompatibleMode = IsCompatibleMode
>          self.IncludePathList = IncludePathList
>          if len(self.FileList) > 0:
>              self.LoadUniFiles(FileList)
> -
>      #
>      # Get Language definition
>      #
>      def GetLangDef(self, File, Line):
> -        Lang = distutils.util.split_quoted((Line.split(u"//")[0]))
> +        Lang = splitquoted((Line.split(u"//")[0]))
>          if len(Lang) != 3:
>              try:
>                  FileIn = UniFileClassObject.OpenUniFile(LongFilePath(File.Path))
>              except UnicodeError as X:
>                  EdkLogger.error("build", FILE_READ_FAILURE, "File read failure: %s"
> % str(X), ExtraData=File);
> --
> 2.20.1.windows.1
> 
> _______________________________________________
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel