From: "Desimone, Nathaniel L" <nathaniel.l.desimone@intel.com>
To: "Gao, Liming" <liming.gao@intel.com>, Aaron Durbin <adurbin@google.com>
Cc: "edk2-devel@lists.01.org" <edk2-devel@lists.01.org>
Subject: Re: EDK2 build issues
Date: Thu, 16 Nov 2017 21:37:44 +0000 [thread overview]
Message-ID: <02A34F284D1DA44BB705E61F7180EF0A97FB2D01@ORSMSX114.amr.corp.intel.com> (raw)
In-Reply-To: <4A89E2EF3DFEDB4C8BFDE51014F606A14E17D6C3@SHSMSX104.ccr.corp.intel.com>
Hi Liming,
I chatted with Aaron on the phone today. The VfrCompiler race condition was discovered using "make -j <n>" (where n > 1). I have filed the bug in Bugzilla:
https://bugzilla.tianocore.org/show_bug.cgi?id=786
For the ARCH environment variable, we brainstormed two possible new names HOST_ARCH or BASETOOLS_ARCH. Should I file the ARCH variable request in Bugzilla as well?
Thanks,
Nate
-----Original Message-----
From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of Gao, Liming
Sent: Monday, November 13, 2017 5:46 PM
To: Aaron Durbin <adurbin@google.com>
Cc: edk2-devel@lists.01.org
Subject: Re: [edk2] EDK2 build issues
I add my comments.
> -----Original Message-----
> From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of
> Aaron Durbin
> Sent: Tuesday, November 14, 2017 1:38 AM
> To: Gao, Liming <liming.gao@intel.com>
> Cc: edk2-devel@lists.01.org
> Subject: Re: [edk2] EDK2 build issues
>
> On Sun, Nov 12, 2017 at 7:15 PM, Gao, Liming <liming.gao@intel.com> wrote:
> > Yes. BaseTools needs some environments. Before build BaseTools, we need to call edkset.sh to setup them.
> >
> > 1. BaseTools Soure tools are compiled in serial instead of parallel.
> > And, VfrCompiler depends on Anltr tool. So, Anltr is required to be
> built first. I agree that this way takes some time to compile
> BaseTools source. I have one proposal to compile C source tools with multiple thread. I can share my draft patch.
>
> If the depedencies are correctly met in the makefiles then why are
> they built serially? It sounds like you understand the dependencies so
> why aren't those explicitly noted so one can build in parallel?
>
Seemly, VfrCompiler Makefile doesn't list those dependency clearly. Could you submit this issue in bugzillar (https://bugzilla.tianocore.org/)?
Besides, do you use make -j option to enable Parallel Execution? I want to know how to verify it.
> > 2. BaseTools C source are compiled to the executable files. They run
> > in host machine. Here ARCH is for the executable files those can
> run in host machine. edk2\BaseTools\Source\C\GNUmakefile auto detects ARCH based on 'uname -m' command.
>
> ARCH != host is my point. Why are those being conflated? Or are you
> saying edk2 tools define ARCH to be what the host is?
>
Yes. Edk2 BaseTools C source uses it as host arch. I agree ARCH name is too generic. In your environment, ARCH may be defined for the different purpose.
> > 3. There are more GCC compiler distribution. We try to use the
> > generic options in GCC tool chain. If you find the option doesn't
> > work
> on your GCC compiler, please report it. And, we also notice
> tools_def.txt is too big to be hard to be maintain. We have one proposal to simplify it. I will send this RFC to edk2 community.
> >
> >>-----Original Message-----
> >>From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf
> >>Of Aaron Durbin
> >>Sent: Saturday, November 11, 2017 6:27 AM
> >>To: edk2-devel@lists.01.org
> >>Subject: [edk2] EDK2 build issues
> >>
> >>Hello,
> >>
> >>Here are some observations I've encountered trying to utilize edk2
> >>for certain builds. Part of the problem seems to be with implicit
> >>assumptions in how edk2 is used. I'm trying to build things using
> >>edk2 from a clean enviroment on an automated builder. i.e. there
> >>isn't a workspace that exists on one persons computer for the
> >>lifetime of development.
> >>
> >>1. BaseTools can't build in parallel. The tests are racey which
> >>result in test failures. Because of this one has to build these in
> >>serial instead of in parallel.
> >>
> >>===========================================================
> >>===========
> >>FAIL: testHelp (TianoCompress.Tests)
> >>--------------------------------------------------------------------
> >>--
> >>Traceback (most recent call last):
> >> File
> >>"/build/zoombini/tmp/portage/sys-boot/fsp-cnl-9999/work/fsp-cnl-
> >>9999/Edk2/BaseTools/Tests/TianoCompress.py",
> >>line 34, in testHelp
> >> self.assertTrue(result == 0)
> >>AssertionError: False is not true
> >>
> >>===========================================================
> >>===========
> >>FAIL: testRandomDataCycles (TianoCompress.Tests)
> >>--------------------------------------------------------------------
> >>--
> >>Traceback (most recent call last):
> >> File
> >>"/build/zoombini/tmp/portage/sys-boot/fsp-cnl-9999/work/fsp-cnl-
> >>9999/Edk2/BaseTools/Tests/TianoCompress.py",
> >>line 65, in testRandomDataCycles
> >> self.compressionTestCycle(data)
> >> File
> >>"/build/zoombini/tmp/portage/sys-boot/fsp-cnl-9999/work/fsp-cnl-
> >>9999/Edk2/BaseTools/Tests/TianoCompress.py",
> >>line 44, in compressionTestCycle
> >> self.assertTrue(result == 0)
> >>AssertionError: False is not true
> >>
> >>In addition, it seems compilation even breaks trying to build a parser:
> >>
> >>VfrSyntax.cpp:53:1: error: expected class-name before '{' token {^M
> >>^
> >>VfrSyntax.cpp: In constructor 'CVfrDLGLexer::CVfrDLGLexer(DLGFileInput*)':
> >>VfrSyntax.cpp:55:36: error: class 'CVfrDLGLexer' does not have any
> >>field named 'VfrLexer'
> >> CVfrDLGLexer (DLGFileInput *F) : VfrLexer (F) {};^M
> >> ^
> >>
> >>This just slows down builds needing to things serially.
> >>
> >>2. It appears the BaseTools uses the ARCH environment variable. I'm
> >>not sure of the origins, but ARCH seems like a complete misnomer for
> >>the *host* you are trying to build tools on -- not the target.
> >>Trying to incorporate edk2 builds into a portage environment
> >>effectively breaks because of this as ARCH refers to target
> >>architecture -- not host builder's ARCH.
> >>
> >>3. This more of an observation, but the tools definition seems to
> >>make quite the leap on how consistent compilers are of a certain version.
> >>e.g. GCC 4.9 can be built with all kinds of default options that
> >>edk2 implicitly assumes are set based on some distribution's default flags?
> >>And in order to extend toolchain support one needs to create a
> >>series of entries associated with a certain family. Barrier to entry
> >>is pretty high in teasing out where to put things in the ~8000 line
> >>tools_def.template.
> >>
> >>Thanks for the consideration.
> >>
> >>-Aaron
> >>_______________________________________________
> >>edk2-devel mailing list
> >>edk2-devel@lists.01.org
> >>https://lists.01.org/mailman/listinfo/edk2-devel
> _______________________________________________
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel
next prev parent reply other threads:[~2017-11-16 21:33 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-11-10 22:26 EDK2 build issues Aaron Durbin
2017-11-13 2:15 ` Gao, Liming
2017-11-13 17:37 ` Aaron Durbin
2017-11-14 1:46 ` Gao, Liming
2017-11-16 21:37 ` Desimone, Nathaniel L [this message]
2017-11-17 8:45 ` Gao, Liming
2017-11-22 23:48 ` Desimone, Nathaniel L
2017-11-23 0:01 ` Andrew Fish
2017-11-23 5:28 ` Gao, Liming
2017-11-27 16:18 ` Andrew Fish
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-list from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=02A34F284D1DA44BB705E61F7180EF0A97FB2D01@ORSMSX114.amr.corp.intel.com \
--to=devel@edk2.groups.io \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox