From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=192.55.52.120; helo=mga04.intel.com; envelope-from=nathaniel.l.desimone@intel.com; receiver=edk2-devel@lists.01.org Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 63AB7202E5E4B for ; Wed, 22 Nov 2017 15:44:41 -0800 (PST) Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 22 Nov 2017 15:48:57 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.44,438,1505804400"; d="scan'208";a="5367315" Received: from orsmsx102.amr.corp.intel.com ([10.22.225.129]) by orsmga003.jf.intel.com with ESMTP; 22 Nov 2017 15:48:57 -0800 Received: from orsmsx154.amr.corp.intel.com (10.22.226.12) by ORSMSX102.amr.corp.intel.com (10.22.225.129) with Microsoft SMTP Server (TLS) id 14.3.319.2; Wed, 22 Nov 2017 15:48:57 -0800 Received: from orsmsx114.amr.corp.intel.com ([169.254.8.129]) by ORSMSX154.amr.corp.intel.com ([169.254.11.132]) with mapi id 14.03.0319.002; Wed, 22 Nov 2017 15:48:57 -0800 From: "Desimone, Nathaniel L" To: "Gao, Liming" , Aaron Durbin CC: "edk2-devel@lists.01.org" Thread-Topic: [edk2] EDK2 build issues Thread-Index: AQHTWnMGWqy7aTF8D02DGKR41dGc06MSG9uAgAEBhYCAAIiWgIAD41FQgAFIygCACE/7sA== Date: Wed, 22 Nov 2017 23:48:56 +0000 Message-ID: <02A34F284D1DA44BB705E61F7180EF0A97FB753E@ORSMSX114.amr.corp.intel.com> References: <4A89E2EF3DFEDB4C8BFDE51014F606A14E17CC73@SHSMSX104.ccr.corp.intel.com> <4A89E2EF3DFEDB4C8BFDE51014F606A14E17D6C3@SHSMSX104.ccr.corp.intel.com> <02A34F284D1DA44BB705E61F7180EF0A97FB2D01@ORSMSX114.amr.corp.intel.com> <4A89E2EF3DFEDB4C8BFDE51014F606A14E17FDC9@SHSMSX104.ccr.corp.intel.com> In-Reply-To: <4A89E2EF3DFEDB4C8BFDE51014F606A14E17FDC9@SHSMSX104.ccr.corp.intel.com> Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-version: 11.0.0.116 dlp-reaction: no-action x-originating-ip: [10.22.254.138] MIME-Version: 1.0 Subject: Re: EDK2 build issues X-BeenThere: edk2-devel@lists.01.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: EDK II Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Nov 2017 23:44:41 -0000 Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable https://bugzilla.tianocore.org/show_bug.cgi?id=3D793 Thanks, Nate -----Original Message----- From: Gao, Liming=20 Sent: Friday, November 17, 2017 12:46 AM To: Desimone, Nathaniel L ; Aaron Durbin Cc: edk2-devel@lists.01.org Subject: RE: [edk2] EDK2 build issues Nate: OK. Please submit the tracker on ARCH request.=20 Thanks Liming > -----Original Message----- > From: Desimone, Nathaniel L > Sent: Friday, November 17, 2017 5:38 AM > To: Gao, Liming ; Aaron Durbin=20 > > Cc: edk2-devel@lists.01.org > Subject: RE: [edk2] EDK2 build issues >=20 > Hi Liming, >=20 > I chatted with Aaron on the phone today. The VfrCompiler race=20 > condition was discovered using "make -j " (where n > 1). I have filed = the bug in Bugzilla: >=20 > https://bugzilla.tianocore.org/show_bug.cgi?id=3D786 >=20 > For the ARCH environment variable, we brainstormed two possible new=20 > names HOST_ARCH or BASETOOLS_ARCH. Should I file the ARCH variable reques= t in Bugzilla as well? >=20 > Thanks, >=20 > Nate >=20 > -----Original Message----- > From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of=20 > Gao, Liming > Sent: Monday, November 13, 2017 5:46 PM > To: Aaron Durbin > Cc: edk2-devel@lists.01.org > Subject: Re: [edk2] EDK2 build issues >=20 > I add my comments. >=20 > > -----Original Message----- > > From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf=20 > > Of Aaron Durbin > > Sent: Tuesday, November 14, 2017 1:38 AM > > To: Gao, Liming > > Cc: edk2-devel@lists.01.org > > Subject: Re: [edk2] EDK2 build issues > > > > On Sun, Nov 12, 2017 at 7:15 PM, Gao, Liming wro= te: > > > Yes. BaseTools needs some environments. Before build BaseTools, we ne= ed to call edkset.sh to setup them. > > > > > > 1. BaseTools Soure tools are compiled in serial instead of parallel. > > > And, VfrCompiler depends on Anltr tool. So, Anltr is required to=20 > > > be > > built first. I agree that this way takes some time to compile=20 > > BaseTools source. I have one proposal to compile C source tools with mu= ltiple thread. I can share my draft patch. > > > > If the depedencies are correctly met in the makefiles then why are=20 > > they built serially? It sounds like you understand the dependencies=20 > > so why aren't those explicitly noted so one can build in parallel? > > > Seemly, VfrCompiler Makefile doesn't list those dependency clearly.=20 > Could you submit this issue in bugzillar (https://bugzilla.tianocore.org/= )? > Besides, do you use make -j option to enable Parallel Execution? I want t= o know how to verify it. >=20 > > > 2. BaseTools C source are compiled to the executable files. They=20 > > > run in host machine. Here ARCH is for the executable files those=20 > > > can > > run in host machine. edk2\BaseTools\Source\C\GNUmakefile auto detects A= RCH based on 'uname -m' command. > > > > ARCH !=3D host is my point. Why are those being conflated? Or are you=20 > > saying edk2 tools define ARCH to be what the host is? > > > Yes. Edk2 BaseTools C source uses it as host arch. I agree ARCH name=20 > is too generic. In your environment, ARCH may be defined for the differen= t purpose. >=20 > > > 3. There are more GCC compiler distribution. We try to use the=20 > > > generic options in GCC tool chain. If you find the option doesn't=20 > > > work > > on your GCC compiler, please report it. And, we also notice=20 > > tools_def.txt is too big to be hard to be maintain. We have one proposa= l to simplify it. I will send this RFC to edk2 community. > > > > > >>-----Original Message----- > > >>From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On=20 > > >>Behalf Of Aaron Durbin > > >>Sent: Saturday, November 11, 2017 6:27 AM > > >>To: edk2-devel@lists.01.org > > >>Subject: [edk2] EDK2 build issues > > >> > > >>Hello, > > >> > > >>Here are some observations I've encountered trying to utilize edk2=20 > > >>for certain builds. Part of the problem seems to be with implicit=20 > > >>assumptions in how edk2 is used. I'm trying to build things using > > >>edk2 from a clean enviroment on an automated builder. i.e. there=20 > > >>isn't a workspace that exists on one persons computer for the=20 > > >>lifetime of development. > > >> > > >>1. BaseTools can't build in parallel. The tests are racey which=20 > > >>result in test failures. Because of this one has to build these in=20 > > >>serial instead of in parallel. > > >> > > >>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > >>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > >>FAIL: testHelp (TianoCompress.Tests) > > >>------------------------------------------------------------------ > > >>-- > > >>-- > > >>Traceback (most recent call last): > > >> File > > >>"/build/zoombini/tmp/portage/sys-boot/fsp-cnl-9999/work/fsp-cnl- > > >>9999/Edk2/BaseTools/Tests/TianoCompress.py", > > >>line 34, in testHelp > > >> self.assertTrue(result =3D=3D 0) > > >>AssertionError: False is not true > > >> > > >>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > >>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > >>FAIL: testRandomDataCycles (TianoCompress.Tests) > > >>------------------------------------------------------------------ > > >>-- > > >>-- > > >>Traceback (most recent call last): > > >> File > > >>"/build/zoombini/tmp/portage/sys-boot/fsp-cnl-9999/work/fsp-cnl- > > >>9999/Edk2/BaseTools/Tests/TianoCompress.py", > > >>line 65, in testRandomDataCycles > > >> self.compressionTestCycle(data) > > >> File > > >>"/build/zoombini/tmp/portage/sys-boot/fsp-cnl-9999/work/fsp-cnl- > > >>9999/Edk2/BaseTools/Tests/TianoCompress.py", > > >>line 44, in compressionTestCycle > > >> self.assertTrue(result =3D=3D 0) > > >>AssertionError: False is not true > > >> > > >>In addition, it seems compilation even breaks trying to build a parse= r: > > >> > > >>VfrSyntax.cpp:53:1: error: expected class-name before '{' token =20 > > >>{^M ^ > > >>VfrSyntax.cpp: In constructor 'CVfrDLGLexer::CVfrDLGLexer(DLGFileInpu= t*)': > > >>VfrSyntax.cpp:55:36: error: class 'CVfrDLGLexer' does not have any=20 > > >>field named 'VfrLexer' > > >> CVfrDLGLexer (DLGFileInput *F) : VfrLexer (F) {};^M > > >> ^ > > >> > > >>This just slows down builds needing to things serially. > > >> > > >>2. It appears the BaseTools uses the ARCH environment variable.=20 > > >>I'm not sure of the origins, but ARCH seems like a complete=20 > > >>misnomer for the *host* you are trying to build tools on -- not the t= arget. > > >>Trying to incorporate edk2 builds into a portage environment=20 > > >>effectively breaks because of this as ARCH refers to target=20 > > >>architecture -- not host builder's ARCH. > > >> > > >>3. This more of an observation, but the tools definition seems to=20 > > >>make quite the leap on how consistent compilers are of a certain vers= ion. > > >>e.g. GCC 4.9 can be built with all kinds of default options that > > >>edk2 implicitly assumes are set based on some distribution's default = flags? > > >>And in order to extend toolchain support one needs to create a=20 > > >>series of entries associated with a certain family. Barrier to=20 > > >>entry is pretty high in teasing out where to put things in the=20 > > >>~8000 line tools_def.template. > > >> > > >>Thanks for the consideration. > > >> > > >>-Aaron > > >>_______________________________________________ > > >>edk2-devel mailing list > > >>edk2-devel@lists.01.org > > >>https://lists.01.org/mailman/listinfo/edk2-devel > > _______________________________________________ > > edk2-devel mailing list > > edk2-devel@lists.01.org > > https://lists.01.org/mailman/listinfo/edk2-devel > _______________________________________________ > edk2-devel mailing list > edk2-devel@lists.01.org > https://lists.01.org/mailman/listinfo/edk2-devel