From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=134.134.136.20; helo=mga02.intel.com; envelope-from=liming.gao@intel.com; receiver=edk2-devel@lists.01.org Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id E677920349DA9 for ; Mon, 13 Nov 2017 17:42:20 -0800 (PST) Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 13 Nov 2017 17:46:27 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.44,392,1505804400"; d="scan'208";a="173182072" Received: from fmsmsx103.amr.corp.intel.com ([10.18.124.201]) by orsmga005.jf.intel.com with ESMTP; 13 Nov 2017 17:46:26 -0800 Received: from shsmsx103.ccr.corp.intel.com (10.239.4.69) by FMSMSX103.amr.corp.intel.com (10.18.124.201) with Microsoft SMTP Server (TLS) id 14.3.319.2; Mon, 13 Nov 2017 17:46:26 -0800 Received: from shsmsx104.ccr.corp.intel.com ([169.254.5.152]) by SHSMSX103.ccr.corp.intel.com ([169.254.4.213]) with mapi id 14.03.0319.002; Tue, 14 Nov 2017 09:46:24 +0800 From: "Gao, Liming" To: Aaron Durbin CC: "edk2-devel@lists.01.org" Thread-Topic: [edk2] EDK2 build issues Thread-Index: AQHTWnMIgiNl8WcTQ0u+gy17uoE/wKMQH5LwgAHxlYCAAQU14A== Date: Tue, 14 Nov 2017 01:46:23 +0000 Message-ID: <4A89E2EF3DFEDB4C8BFDE51014F606A14E17D6C3@SHSMSX104.ccr.corp.intel.com> References: <4A89E2EF3DFEDB4C8BFDE51014F606A14E17CC73@SHSMSX104.ccr.corp.intel.com> In-Reply-To: Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-version: 11.0.0.116 dlp-reaction: no-action x-originating-ip: [10.239.127.40] MIME-Version: 1.0 Subject: Re: EDK2 build issues X-BeenThere: edk2-devel@lists.01.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: EDK II Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Nov 2017 01:42:21 -0000 Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable I add my comments.=20 > -----Original Message----- > From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of Aa= ron Durbin > Sent: Tuesday, November 14, 2017 1:38 AM > To: Gao, Liming > Cc: edk2-devel@lists.01.org > Subject: Re: [edk2] EDK2 build issues >=20 > On Sun, Nov 12, 2017 at 7:15 PM, Gao, Liming wrote= : > > Yes. BaseTools needs some environments. Before build BaseTools, we need= to call edkset.sh to setup them. > > > > 1. BaseTools Soure tools are compiled in serial instead of parallel. An= d, VfrCompiler depends on Anltr tool. So, Anltr is required to be > built first. I agree that this way takes some time to compile BaseTools s= ource. I have one proposal to compile C source tools with > multiple thread. I can share my draft patch. >=20 > If the depedencies are correctly met in the makefiles then why are > they built serially? It sounds like you understand the dependencies so > why aren't those explicitly noted so one can build in parallel? >=20 Seemly, VfrCompiler Makefile doesn't list those dependency clearly. Could y= ou submit this issue in bugzillar (https://bugzilla.tianocore.org/)? Besides, do you use make -j option to enable Parallel Execution? I want to = know how to verify it.=20 > > 2. BaseTools C source are compiled to the executable files. They run in= host machine. Here ARCH is for the executable files those can > run in host machine. edk2\BaseTools\Source\C\GNUmakefile auto detects ARC= H based on 'uname -m' command. >=20 > ARCH !=3D host is my point. Why are those being conflated? Or are you > saying edk2 tools define ARCH to be what the host is? >=20 Yes. Edk2 BaseTools C source uses it as host arch. I agree ARCH name is too= generic. In your environment, ARCH may be defined for the different purpos= e.=20 > > 3. There are more GCC compiler distribution. We try to use the generic = options in GCC tool chain. If you find the option doesn't work > on your GCC compiler, please report it. And, we also notice tools_def.txt= is too big to be hard to be maintain. We have one proposal to > simplify it. I will send this RFC to edk2 community. > > > >>-----Original Message----- > >>From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of > >>Aaron Durbin > >>Sent: Saturday, November 11, 2017 6:27 AM > >>To: edk2-devel@lists.01.org > >>Subject: [edk2] EDK2 build issues > >> > >>Hello, > >> > >>Here are some observations I've encountered trying to utilize edk2 for > >>certain builds. Part of the problem seems to be with implicit > >>assumptions in how edk2 is used. I'm trying to build things using edk2 > >>from a clean enviroment on an automated builder. i.e. there isn't a > >>workspace that exists on one persons computer for the lifetime of > >>development. > >> > >>1. BaseTools can't build in parallel. The tests are racey which result > >>in test failures. Because of this one has to build these in serial > >>instead of in parallel. > >> > >>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > >>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > >>FAIL: testHelp (TianoCompress.Tests) > >>---------------------------------------------------------------------- > >>Traceback (most recent call last): > >> File "/build/zoombini/tmp/portage/sys-boot/fsp-cnl-9999/work/fsp-cnl- > >>9999/Edk2/BaseTools/Tests/TianoCompress.py", > >>line 34, in testHelp > >> self.assertTrue(result =3D=3D 0) > >>AssertionError: False is not true > >> > >>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > >>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > >>FAIL: testRandomDataCycles (TianoCompress.Tests) > >>---------------------------------------------------------------------- > >>Traceback (most recent call last): > >> File "/build/zoombini/tmp/portage/sys-boot/fsp-cnl-9999/work/fsp-cnl- > >>9999/Edk2/BaseTools/Tests/TianoCompress.py", > >>line 65, in testRandomDataCycles > >> self.compressionTestCycle(data) > >> File "/build/zoombini/tmp/portage/sys-boot/fsp-cnl-9999/work/fsp-cnl- > >>9999/Edk2/BaseTools/Tests/TianoCompress.py", > >>line 44, in compressionTestCycle > >> self.assertTrue(result =3D=3D 0) > >>AssertionError: False is not true > >> > >>In addition, it seems compilation even breaks trying to build a parser: > >> > >>VfrSyntax.cpp:53:1: error: expected class-name before '{' token > >> {^M > >> ^ > >>VfrSyntax.cpp: In constructor 'CVfrDLGLexer::CVfrDLGLexer(DLGFileInput*= )': > >>VfrSyntax.cpp:55:36: error: class 'CVfrDLGLexer' does not have any > >>field named 'VfrLexer' > >> CVfrDLGLexer (DLGFileInput *F) : VfrLexer (F) {};^M > >> ^ > >> > >>This just slows down builds needing to things serially. > >> > >>2. It appears the BaseTools uses the ARCH environment variable. I'm > >>not sure of the origins, but ARCH seems like a complete misnomer for > >>the *host* you are trying to build tools on -- not the target. Trying > >>to incorporate edk2 builds into a portage environment effectively > >>breaks because of this as ARCH refers to target architecture -- not > >>host builder's ARCH. > >> > >>3. This more of an observation, but the tools definition seems to make > >>quite the leap on how consistent compilers are of a certain version. > >>e.g. GCC 4.9 can be built with all kinds of default options that edk2 > >>implicitly assumes are set based on some distribution's default flags? > >>And in order to extend toolchain support one needs to create a series > >>of entries associated with a certain family. Barrier to entry is > >>pretty high in teasing out where to put things in the ~8000 line > >>tools_def.template. > >> > >>Thanks for the consideration. > >> > >>-Aaron > >>_______________________________________________ > >>edk2-devel mailing list > >>edk2-devel@lists.01.org > >>https://lists.01.org/mailman/listinfo/edk2-devel > _______________________________________________ > edk2-devel mailing list > edk2-devel@lists.01.org > https://lists.01.org/mailman/listinfo/edk2-devel