From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: mx.groups.io; dkim=pass header.i=@linaro.org header.s=google header.b=hQ6OJhTI; spf=pass (domain: linaro.org, ip: 209.85.128.53, mailfrom: leif.lindholm@linaro.org) Received: from mail-wm1-f53.google.com (mail-wm1-f53.google.com [209.85.128.53]) by groups.io with SMTP; Wed, 04 Sep 2019 02:51:55 -0700 Received: by mail-wm1-f53.google.com with SMTP id p13so2895787wmh.1 for ; Wed, 04 Sep 2019 02:51:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:content-transfer-encoding:in-reply-to :user-agent; bh=hz9nDKwVxnlkZu8YlsV+hF4v8oDRprxw4cVpeWWg+lk=; b=hQ6OJhTIY4zicPdXS+vBJh1IzWcAzjRhoDokZfYZCgHcJtw8O0vs5SBjPOR2AfrZ2b tC72EK+8LVgkqZqERBDc9P0/7AuEfnnOCFdb3CiAApKN5/8GbfD8fKtNyDucG/ASb2EJ g03KcGthgi8p6iSamIcOpQ59Y/Ij3JMhJw2n8CiwfdXkT2HxB9FSALxS+VqcjWRNlwhl GGMnsr4lVkts77dYWwiVcjINxQz4CC1T/JthT5u7vd+BWDTdX31Gk0eYKR+c7jnHa/Vl KcAdQ2YRPCLdPvus/4fe8zbnXQgjGKdapuisGaxgJz6eEToWyLamXYlGr+jU7wtdh+nI uNzA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=hz9nDKwVxnlkZu8YlsV+hF4v8oDRprxw4cVpeWWg+lk=; b=g2W35VnP2Wk2WLyfpUYdaIcB5vDXStcSsJTcoMt/FLhiHGO1r2ctyVGxxWOOlic5YD 9PrChg0qDKTyH3Om7ENA1IPlrpbXDfElLlJOiPPkLd9YhkNqtTGSeQkO2LwlQYu3s7KM BuAD5hMA74QHcGCoQLK9GW6qebbrKnUhbTmBifzblpfrI2ulLrLNNnXqRCAh0Ns6l5eS h0mhJ1woDJlXXq6U7C6S1B/2+LruEyBL3jKHdP6VEBAWZDf3AXfIdrp3zNGbQGFqejmw ZmSY7pqkKeGMiwR1gVoX2gcA18gXrt8a5zCR+R9IvpGQiJT9aw1i51RaRSvz1r4AQnZ1 8GUw== X-Gm-Message-State: APjAAAW52EwUOZB5vrdEwyNWsiEaBPRHgyiHZOsnIBXWxjQsYxHnf4+B oWELap85E1lc+jTyAGRfLIrC9g== X-Google-Smtp-Source: APXvYqz+oMSvxZrwi7w4zTwmZB7SJZ9Xidij5RGpx138TnbmujFZ3PowkDOmJL00RmNA2f99LYT3Jw== X-Received: by 2002:a1c:a011:: with SMTP id j17mr3576880wme.97.1567590713701; Wed, 04 Sep 2019 02:51:53 -0700 (PDT) Return-Path: Received: from bivouac.eciton.net (bivouac.eciton.net. [2a00:1098:0:86:1000:23:0:2]) by smtp.gmail.com with ESMTPSA id q15sm2165168wrg.65.2019.09.04.02.51.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Sep 2019 02:51:52 -0700 (PDT) Date: Wed, 4 Sep 2019 10:51:51 +0100 From: "Leif Lindholm" To: "Feng, Bob C" Cc: "devel@edk2.groups.io" , "Gao, Liming" Subject: Re: [edk2-devel] [edk2] [Patch 32/33] BaseTools: ECC tool Python3 adaption Message-ID: <20190904095151.GE29255@bivouac.eciton.net> References: <20190129020610.14300-1-bob.c.feng@intel.com> <20190129020610.14300-33-bob.c.feng@intel.com> <20190902190211.GZ29255@bivouac.eciton.net> <20190902190438.GA29255@bivouac.eciton.net> <08650203BA1BD64D8AD9B6D5D74A85D16152FF60@SHSMSX104.ccr.corp.intel.com> <20190904083810.GD29255@bivouac.eciton.net> <08650203BA1BD64D8AD9B6D5D74A85D161530544@SHSMSX104.ccr.corp.intel.com> MIME-Version: 1.0 In-Reply-To: <08650203BA1BD64D8AD9B6D5D74A85D161530544@SHSMSX104.ccr.corp.intel.com> User-Agent: Mutt/1.10.1 (2018-07-13) Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi Bob, On Wed, Sep 04, 2019 at 09:12:30AM +0000, Feng, Bob C wrote: > The CLexer.py and CParser.py under CParser3 were generated with > antlr3.0.1 > (https://github.com/tianocore/tianocore.github.io/wiki/ECC-tool) . I > think API version error may be due to antlr-python-runtime on > Debian has different version. What's the antlr-python-runtime on > Debian? Running the antlr3 executable, it says "ANTLR Parser Generator Version 3.5.2". I guess it's worth clarifying here that I am seeing the same behaviour with python2 and antlr3. So I am currently unable to run Ecc at all. Best Regards, Leif > -----Original Message----- > From: Leif Lindholm [mailto:leif.lindholm@linaro.org]=20 > Sent: Wednesday, September 4, 2019 4:38 PM > To: Feng, Bob C > Cc: devel@edk2.groups.io; Gao, Liming > Subject: Re: [edk2-devel] [edk2] [Patch 32/33] BaseTools: ECC tool Pytho= n3 adaption >=20 > Hi Bob, >=20 > On Wed, Sep 04, 2019 at 02:10:23AM +0000, Feng, Bob C wrote: > > Hi Leif, > >=20 > > I have no Debian environment. On Debian, can python3 work with antlr3? >=20 > Yes. The below is equivalent to what I have already done. >=20 > Can you please respond to the question I asked about the API version err= or I see when I then try to run it, included in my original email? >=20 > Best Regards, >=20 > Leif >=20 > > I checked the antlr3 python github repository, the source code is stil= l in beta version and has not been updated for 7 years. > >=20 > > But If yes, I think the import statement in ECC can be changed as: > > try: > > import antlr4 as antlr > > from Ecc.CParser4.CLexer import CLexer > > from Ecc.CParser4.CParser import CParser > > except: > > import antlr3 as antlr > > antlr.InputStream =3D antlr.StringStream > > from Ecc.CParser3.CLexer import CLexer > > from Ecc.CParser3.CParser import CParser > >=20 > >=20 > > Thanks, > > Bob > >=20 > >=20 > > -----Original Message----- > > From: devel@edk2.groups.io [mailto:devel@edk2.groups.io] On Behalf Of= =20 > > Leif Lindholm > > Sent: Tuesday, September 3, 2019 3:05 AM > > To: Feng, Bob C > > Cc: devel@edk2.groups.io; Gao, Liming > > Subject: Re: [edk2-devel] [edk2] [Patch 32/33] BaseTools: ECC tool=20 > > Python3 adaption > >=20 > > Argh - forgot about the mailing list move, forwarding to current list. > >=20 > > / > > Leif > >=20 > > On Mon, Sep 02, 2019 at 08:02:11PM +0100, Leif Lindholm wrote: > > > Hi Bob, > > >=20 > > > I was running Ecc today, apparently for the first time since I=20 > > > switched to Python3 by default. > > >=20 > > > I have raised https://bugzilla.tianocore.org/show_bug.cgi?id=3D2148= =20 > > > over the way Python3 hard codes use of antlr4, whereas it seems to= =20 > > > me it should be possible to ue Python3 with antlr3 (but not Python2= =20 > > > with antlr4). > > >=20 > > > However, whilst that issue could be looked at without extreme=20 > > > urgency, I am curious as to what is causing the error I am seeing=20 > > > upon working around the import failure on my Debian installation=20 > > > (which lacks python3-antlr4). > > >=20 > > > The output I get when running > > > $ Ecc -t /work/git/edk2/EmbeddedPkg/Drivers/DtPlatformDxe/ -s > > >=20 > > > is > > >=20 > > > --- > > > /work/git/edk2/BaseTools/Source/Python/Ecc/EccMain.py:409: Deprecati= onWarning: time.clock has been deprecated in Python 3.3 and will be removed= from Python 3.8: use time.perf_counter or time.process_time instead > > > StartTime =3D time.clock() > > > 11:44:43, Sep.02 2019 [00:00] > > >=20 > > > Loading ECC configuration ... done > > > Building database for Meta Data File Done! > > > Parsing > > > //work/git/edk2/EmbeddedPkg/Drivers/DtPlatformDxe/DtPlatformDxe.c > > > Traceback (most recent call last): > > > File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_ma= in > > > "__main__", mod_spec) > > > File "/usr/lib/python3.7/runpy.py", line 85, in _run_code > > > exec(code, run_globals) > > > File "/work/git/edk2/BaseTools/Source/Python/Ecc/EccMain.py", line= 410, in > > > Ecc =3D Ecc() > > > File "/work/git/edk2/BaseTools/Source/Python/Ecc/EccMain.py", line= 94, in __init__ > > > self.DetectOnlyScanDirs() > > > File "/work/git/edk2/BaseTools/Source/Python/Ecc/EccMain.py", line= 130, in DetectOnlyScanDirs > > > self.BuildDatabase() > > > File "/work/git/edk2/BaseTools/Source/Python/Ecc/EccMain.py", line= 150, in BuildDatabase > > > c.CollectSourceCodeDataIntoDB(EccGlobalData.gTarget) > > > File "/work/git/edk2/BaseTools/Source/Python/Ecc/c.py", line 526, = in CollectSourceCodeDataIntoDB > > > collector.ParseFile() > > > File "/work/git/edk2/BaseTools/Source/Python/Ecc/CodeFragmentColle= ctor.py", line 517, in ParseFile > > > lexer =3D CLexer(cStream) > > > File "/work/git/edk2/BaseTools/Source/Python/Ecc/CParser3/CLexer.p= y", line 147, in __init__ > > > Lexer.__init__(self, input) > > > File "/usr/lib/python3/dist-packages/antlr3/recognizers.py", line = 1039, in __init__ > > > BaseRecognizer.__init__(self, state) > > > File "/usr/lib/python3/dist-packages/antlr3/recognizers.py", line = 169, in __init__ > > > .format(self.api_version)) > > > RuntimeError: ANTLR version mismatch: The recognizer has been genera= ted with API V0, but this runtime does not support this. > > > --- > > >=20 > > > Any idea? > > >=20 > > > Best Regards, > > >=20 > > > Leif > > >=20 > > > On Tue, Jan 29, 2019 at 10:06:09AM +0800, Feng, Bob C wrote: > > > > v2: > > > > The python files under CParser4 are generated by antlr4 and for > > > > python3 usage. They have python3 specific syntax, for example the= =20 > > > > data type declaration for the arguments of a function. That is not= = =20 > > > > compitable with python2. this patch is to remove these syntax. > > > >=20 > > > > ECC tool Python3 adaption. > > > >=20 > > > > Contributed-under: TianoCore Contribution Agreement 1.1 > > > > Signed-off-by: Bob Feng > > > > Cc: Liming Gao > > > > --- > > > > BaseTools/Source/Python/Ecc/{ =3D> CParser3}/CLexer.py | 0 > > > > BaseTools/Source/Python/Ecc/{ =3D> CParser3}/CParser.py | 0 > > > > BaseTools/Source/Python/Ecc/CParser3/__init__.py | 0 > > > > BaseTools/Source/Python/Ecc/CParser4/C.g4 | 637 ++++= +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++= ++++++++++++++++ > > > > BaseTools/Source/Python/Ecc/CParser4/CLexer.py | 632 ++++= +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++= ++++++++++++++++ > > > > BaseTools/Source/Python/Ecc/CParser4/CListener.py | 815 ++++= +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++= +++++++++++++++++++++++++++++++++++++++++++ > > > > BaseTools/Source/Python/Ecc/CParser4/CParser.py | 6279 ++++= +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++= +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++= +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++= +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++= +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++= +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++= +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++= +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++= +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++= +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++= +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++= +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++= ++++++++++++++++++++++++++++++++ > > > > + > > > > BaseTools/Source/Python/Ecc/CParser4/__init__.py | 0 > > > > BaseTools/Source/Python/Ecc/Check.py | 4 +- > > > > BaseTools/Source/Python/Ecc/CodeFragmentCollector.py | 20 +-- > > > > BaseTools/Source/Python/Ecc/Configuration.py | 3 - > > > > BaseTools/Source/Python/Ecc/EccMain.py | 2 +- > > > > BaseTools/Source/Python/Ecc/EccToolError.py | 4 +- > > > > BaseTools/Source/Python/Ecc/FileProfile.py | 2 +- > > > > BaseTools/Source/Python/Ecc/MetaDataParser.py | 2 +- > > > > BaseTools/Source/Python/Ecc/c.py | 6 +- > > > > BaseTools/Source/Python/Ecc/config.ini | 2 - > > > > 17 files changed, 8385 insertions(+), 23 deletions(-) > >=20 > >=20 > >=20