public inbox for devel@edk2.groups.io
 help / color / mirror / Atom feed
From: "Bob Feng" <bob.c.feng@intel.com>
To: Leif Lindholm <leif.lindholm@linaro.org>
Cc: "devel@edk2.groups.io" <devel@edk2.groups.io>,
	"Gao, Liming" <liming.gao@intel.com>
Subject: Re: [edk2-devel] [edk2] [Patch 32/33] BaseTools: ECC tool Python3 adaption
Date: Thu, 5 Sep 2019 05:39:05 +0000	[thread overview]
Message-ID: <08650203BA1BD64D8AD9B6D5D74A85D161531023@SHSMSX104.ccr.corp.intel.com> (raw)
In-Reply-To: <20190904095151.GE29255@bivouac.eciton.net>

Hi Leif,

Would you try to install antlr4-python3-runtime on debian.
pip install antlr4-python3-runtime

I think python3 + antlr3 would not be a good combination, since the antlr3 for python3 is still in beta and has not been update for 7 years. And I think there is no ECC test for such combination.

Thanks,
Bob

-----Original Message-----
From: Leif Lindholm [mailto:leif.lindholm@linaro.org] 
Sent: Wednesday, September 4, 2019 5:52 PM
To: Feng, Bob C <bob.c.feng@intel.com>
Cc: devel@edk2.groups.io; Gao, Liming <liming.gao@intel.com>
Subject: Re: [edk2-devel] [edk2] [Patch 32/33] BaseTools: ECC tool Python3 adaption

Hi Bob,

On Wed, Sep 04, 2019 at 09:12:30AM +0000, Feng, Bob C wrote:
> The CLexer.py and CParser.py under CParser3 were generated with
> antlr3.0.1
> (https://github.com/tianocore/tianocore.github.io/wiki/ECC-tool) . I 
> think API version error may be due to antlr-python-runtime  on Debian 
> has different version. What's the antlr-python-runtime on Debian?

Running the antlr3 executable, it says
"ANTLR Parser Generator Version 3.5.2".

I guess it's worth clarifying here that I am seeing the same behaviour with python2 and antlr3.

So I am currently unable to run Ecc at all.

Best Regards,

Leif

> -----Original Message-----
> From: Leif Lindholm [mailto:leif.lindholm@linaro.org]
> Sent: Wednesday, September 4, 2019 4:38 PM
> To: Feng, Bob C <bob.c.feng@intel.com>
> Cc: devel@edk2.groups.io; Gao, Liming <liming.gao@intel.com>
> Subject: Re: [edk2-devel] [edk2] [Patch 32/33] BaseTools: ECC tool 
> Python3 adaption
> 
> Hi Bob,
> 
> On Wed, Sep 04, 2019 at 02:10:23AM +0000, Feng, Bob C wrote:
> > Hi Leif,
> > 
> > I have no Debian environment. On Debian, can python3 work with antlr3?
> 
> Yes. The below is equivalent to what I have already done.
> 
> Can you please respond to the question I asked about the API version error I see when I then try to run it, included in my original email?
> 
> Best Regards,
> 
> Leif
> 
> > I checked the antlr3 python github repository, the source code is still in beta version and has not been updated for 7 years.
> > 
> > But If yes, I think the import statement in ECC can be changed as:
> > try:
> >     import antlr4 as antlr
> >     from Ecc.CParser4.CLexer import CLexer
> >     from Ecc.CParser4.CParser import CParser
> > except:
> >     import antlr3 as antlr
> >     antlr.InputStream = antlr.StringStream
> >     from Ecc.CParser3.CLexer import CLexer
> >     from Ecc.CParser3.CParser import CParser
> > 
> > 
> > Thanks,
> > Bob
> > 
> > 
> > -----Original Message-----
> > From: devel@edk2.groups.io [mailto:devel@edk2.groups.io] On Behalf 
> > Of Leif Lindholm
> > Sent: Tuesday, September 3, 2019 3:05 AM
> > To: Feng, Bob C <bob.c.feng@intel.com>
> > Cc: devel@edk2.groups.io; Gao, Liming <liming.gao@intel.com>
> > Subject: Re: [edk2-devel] [edk2] [Patch 32/33] BaseTools: ECC tool
> > Python3 adaption
> > 
> > Argh - forgot about the mailing list move, forwarding to current list.
> > 
> > /
> >    Leif
> > 
> > On Mon, Sep 02, 2019 at 08:02:11PM +0100, Leif Lindholm wrote:
> > > Hi Bob,
> > > 
> > > I was running Ecc today, apparently for the first time since I 
> > > switched to Python3 by default.
> > > 
> > > I have raised https://bugzilla.tianocore.org/show_bug.cgi?id=2148
> > > over the way Python3 hard codes use of antlr4, whereas it seems to 
> > > me it should be possible to ue Python3 with antlr3 (but not 
> > > Python2 with antlr4).
> > > 
> > > However, whilst that issue could be looked at without extreme 
> > > urgency, I am curious as to what is causing the error I am seeing 
> > > upon working around the import failure on my Debian installation 
> > > (which lacks python3-antlr4).
> > > 
> > > The output I get when running
> > > $ Ecc -t /work/git/edk2/EmbeddedPkg/Drivers/DtPlatformDxe/ -s
> > > 
> > > is
> > > 
> > > ---
> > > /work/git/edk2/BaseTools/Source/Python/Ecc/EccMain.py:409: DeprecationWarning: time.clock has been deprecated in Python 3.3 and will be removed from Python 3.8: use time.perf_counter or time.process_time instead
> > >   StartTime = time.clock()
> > > 11:44:43, Sep.02 2019 [00:00]
> > > 
> > > Loading ECC configuration ... done Building database for Meta Data 
> > > File Done!
> > > Parsing
> > > //work/git/edk2/EmbeddedPkg/Drivers/DtPlatformDxe/DtPlatformDxe.c
> > > Traceback (most recent call last):
> > >   File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
> > >     "__main__", mod_spec)
> > >   File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
> > >     exec(code, run_globals)
> > >   File "/work/git/edk2/BaseTools/Source/Python/Ecc/EccMain.py", line 410, in <module>
> > >     Ecc = Ecc()
> > >   File "/work/git/edk2/BaseTools/Source/Python/Ecc/EccMain.py", line 94, in __init__
> > >     self.DetectOnlyScanDirs()
> > >   File "/work/git/edk2/BaseTools/Source/Python/Ecc/EccMain.py", line 130, in DetectOnlyScanDirs
> > >     self.BuildDatabase()
> > >   File "/work/git/edk2/BaseTools/Source/Python/Ecc/EccMain.py", line 150, in BuildDatabase
> > >     c.CollectSourceCodeDataIntoDB(EccGlobalData.gTarget)
> > >   File "/work/git/edk2/BaseTools/Source/Python/Ecc/c.py", line 526, in CollectSourceCodeDataIntoDB
> > >     collector.ParseFile()
> > >   File "/work/git/edk2/BaseTools/Source/Python/Ecc/CodeFragmentCollector.py", line 517, in ParseFile
> > >     lexer = CLexer(cStream)
> > >   File "/work/git/edk2/BaseTools/Source/Python/Ecc/CParser3/CLexer.py", line 147, in __init__
> > >     Lexer.__init__(self, input)
> > >   File "/usr/lib/python3/dist-packages/antlr3/recognizers.py", line 1039, in __init__
> > >     BaseRecognizer.__init__(self, state)
> > >   File "/usr/lib/python3/dist-packages/antlr3/recognizers.py", line 169, in __init__
> > >     .format(self.api_version))
> > > RuntimeError: ANTLR version mismatch: The recognizer has been generated with API V0, but this runtime does not support this.
> > > ---
> > > 
> > > Any idea?
> > > 
> > > Best Regards,
> > > 
> > > Leif
> > > 
> > > On Tue, Jan 29, 2019 at 10:06:09AM +0800, Feng, Bob C wrote:
> > > > v2:
> > > > The python files under CParser4 are generated by antlr4 and for
> > > > python3 usage. They have python3 specific syntax, for example 
> > > > the data type declaration for the arguments of a function. That 
> > > > is not compitable with python2. this patch is to remove these syntax.
> > > > 
> > > > ECC tool Python3 adaption.
> > > > 
> > > > Contributed-under: TianoCore Contribution Agreement 1.1
> > > > Signed-off-by: Bob Feng <bob.c.feng@intel.com>
> > > > Cc: Liming Gao <liming.gao@intel.com>
> > > > ---
> > > >  BaseTools/Source/Python/Ecc/{ => CParser3}/CLexer.py  |    0
> > > >  BaseTools/Source/Python/Ecc/{ => CParser3}/CParser.py |    0
> > > >  BaseTools/Source/Python/Ecc/CParser3/__init__.py      |    0
> > > >  BaseTools/Source/Python/Ecc/CParser4/C.g4             |  637 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> > > >  BaseTools/Source/Python/Ecc/CParser4/CLexer.py        |  632 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> > > >  BaseTools/Source/Python/Ecc/CParser4/CListener.py     |  815 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> > > >  BaseTools/Source/Python/Ecc/CParser4/CParser.py       | 6279 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> > > >  +
> > > >  BaseTools/Source/Python/Ecc/CParser4/__init__.py      |    0
> > > >  BaseTools/Source/Python/Ecc/Check.py                  |    4 +-
> > > >  BaseTools/Source/Python/Ecc/CodeFragmentCollector.py  |   20 +--
> > > >  BaseTools/Source/Python/Ecc/Configuration.py          |    3 -
> > > >  BaseTools/Source/Python/Ecc/EccMain.py                |    2 +-
> > > >  BaseTools/Source/Python/Ecc/EccToolError.py           |    4 +-
> > > >  BaseTools/Source/Python/Ecc/FileProfile.py            |    2 +-
> > > >  BaseTools/Source/Python/Ecc/MetaDataParser.py         |    2 +-
> > > >  BaseTools/Source/Python/Ecc/c.py                      |    6 +-
> > > >  BaseTools/Source/Python/Ecc/config.ini                |    2 -
> > > >  17 files changed, 8385 insertions(+), 23 deletions(-)
> > 
> > 
> > 

  reply	other threads:[~2019-09-05  5:39 UTC|newest]

Thread overview: 49+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-01-29  2:05 [Patch v2 00/33] BaseTools python3 migration patch set Feng, Bob C
2019-01-29  2:05 ` [Patch 01/33] BaseTool:Rename xrange() to range() Feng, Bob C
2019-01-29  2:05 ` [Patch 02/33] BaseTools:use iterate list to replace the itertools Feng, Bob C
2019-01-29  2:05 ` [Patch 03/33] BaseTools: Rename iteritems to items Feng, Bob C
2019-01-29  2:05 ` [Patch 04/33] BaseTools: replace get_bytes_le() to bytes_le Feng, Bob C
2019-01-29  2:05 ` [Patch 05/33] BaseTools: use OrderedDict instead of sdict Feng, Bob C
2019-01-29  2:05 ` [Patch 06/33] BaseTools: nametuple not have verbose parameter in python3 Feng, Bob C
2019-01-29  2:05 ` [Patch 07/33] BaseTools: Remove unnecessary super function Feng, Bob C
2019-01-29  2:05 ` [Patch 08/33] BaseTools: replace long by int Feng, Bob C
2019-01-29  2:05 ` [Patch 09/33] BaseTools:Solve the data sorting problem use python3 Feng, Bob C
2019-01-29  2:05 ` [Patch 10/33] BaseTools: Update argparse arguments since it not have version now Feng, Bob C
2019-01-29  2:05 ` [Patch 11/33] BaseTools:Similar to octal data rectification Feng, Bob C
2019-01-29  2:05 ` [Patch 12/33] BaseTools/UPT:merge UPT Tool use Python2 and Python3 Feng, Bob C
2019-01-29  2:05 ` [Patch 13/33] BaseTools: update Test scripts support python3 Feng, Bob C
2019-01-29  2:05 ` [Patch 14/33] BaseTools/Scripts: Porting PackageDocumentTools code to use Python3 Feng, Bob C
2019-01-29  2:05 ` [Patch 15/33] Basetools: It went wrong when use os.linesep Feng, Bob C
2019-01-29  2:05 ` [Patch 16/33] BaseTools:Fv BaseAddress must set If it not set Feng, Bob C
2019-01-29  2:05 ` [Patch 17/33] BaseTools: Make sure AllPcdList valid Feng, Bob C
2019-01-29  2:05 ` [Patch 18/33] BaseTools:TestTools character encoding issue Feng, Bob C
2019-01-29  2:05 ` [Patch 19/33] BaseTools:Double carriage return inserted from Trim.py on Python3 Feng, Bob C
2019-01-29  2:05 ` [Patch 20/33] BaseTools:File open failed for VPD MapFile Feng, Bob C
2019-01-29  2:05 ` [Patch 21/33] BaseTools: change the Division Operator Feng, Bob C
2019-01-29  2:05 ` [Patch 22/33] BaseTools:There is extra blank line in datalog Feng, Bob C
2019-01-29  2:06 ` [Patch 23/33] BaseTools: Similar to octal data rectification Feng, Bob C
2019-01-29  2:06 ` [Patch 24/33] BaseTools: Update windows and linux run scripts file to use Python3 Feng, Bob C
2019-01-29  2:06 ` [Patch 25/33] BaseTools:Update build tool to print python version information Feng, Bob C
2019-01-29  2:06 ` [Patch 26/33] BaseTools:Linux Python highest version check Feng, Bob C
2019-01-29  2:06 ` [Patch 27/33] BaseTools: Update PYTHON env to PYTHON_COMMAND Feng, Bob C
2019-01-29  2:06 ` [Patch 28/33] BaseTools:Fixed Rsa issue and a set define issue Feng, Bob C
2019-01-29  2:06 ` [Patch 29/33] BaseTools:ord() don't match in py2 and py3 Feng, Bob C
2019-01-29  2:06 ` [Patch 30/33] BaseTools: the list and iterator translation Feng, Bob C
2019-01-29  2:06 ` [Patch 31/33] BaseTools: Handle the bytes and str difference Feng, Bob C
2019-01-29  2:06 ` [Patch 32/33] BaseTools: ECC tool Python3 adaption Feng, Bob C
     [not found]   ` <20190902190211.GZ29255@bivouac.eciton.net>
2019-09-02 19:04     ` [edk2] " Leif Lindholm
2019-09-04  2:10       ` [edk2-devel] " Bob Feng
2019-09-04  8:38         ` Leif Lindholm
2019-09-04  9:12           ` Bob Feng
2019-09-04  9:51             ` Leif Lindholm
2019-09-05  5:39               ` Bob Feng [this message]
2019-09-05 10:37                 ` Leif Lindholm
2019-09-05 13:53                   ` Laszlo Ersek
2019-01-29  2:06 ` [Patch v2 33/33] BaseTools: Eot " Feng, Bob C
2019-01-29 13:07 ` [Patch v2 00/33] BaseTools python3 migration patch set Laszlo Ersek
2019-01-30  1:52   ` Gao, Liming
2019-01-30  5:25     ` Feng, Bob C
2019-01-31  8:23       ` Gao, Liming
2019-02-01  3:13         ` Feng, Bob C
2019-02-01  8:50           ` Laszlo Ersek
2019-01-30  2:59   ` Feng, Bob C

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-list from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=08650203BA1BD64D8AD9B6D5D74A85D161531023@SHSMSX104.ccr.corp.intel.com \
    --to=devel@edk2.groups.io \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox