From mboxrd@z Thu Jan  1 00:00:00 1970
Authentication-Results: mx.groups.io;
 dkim=missing; spf=pass (domain: intel.com, ip: 192.55.52.88, mailfrom: jordan.l.justen@intel.com)
Received: from mga01.intel.com (mga01.intel.com [192.55.52.88])
 by groups.io with SMTP; Tue, 23 Jul 2019 01:05:27 -0700
X-Amp-Result: SKIPPED(no attachment in message)
X-Amp-File-Uploaded: False
Received: from orsmga004.jf.intel.com ([10.7.209.38])
  by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 23 Jul 2019 01:05:26 -0700
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.64,298,1559545200"; 
   d="scan'208";a="320929540"
Received: from dannylex-mobl.amr.corp.intel.com (HELO localhost) ([10.254.50.219])
  by orsmga004.jf.intel.com with ESMTP; 23 Jul 2019 01:05:25 -0700
MIME-Version: 1.0
In-Reply-To: <9733a473-7dc0-c8e4-18e5-0a1d0f80f3e0@redhat.com>
References: <20190722005816.96146-1-rebecca@bsdio.com> <156377947230.31344.9139528030621143554@jljusten-skl> <5313f05d-cd57-3fd0-9a44-4290b64db5a9@redhat.com> <156383729562.19762.9432602825677327108@jljusten-skl> <db45b3b0-fd89-a0ce-fcd7-6c360e80da44@bsdio.com> <9733a473-7dc0-c8e4-18e5-0a1d0f80f3e0@redhat.com>
To: Ard Biesheuvel <ard.biesheuvel@linaro.org>, Laszlo Ersek <lersek@redhat.com>, Rebecca Cran <rebecca@bsdio.com>, devel@edk2.groups.io
Subject: Re: [edk2-devel] [PATCH] OvmfPkg: enable multiprocessor builds when using build.sh
From: "Jordan Justen" <jordan.l.justen@intel.com>
Message-ID: <156386912527.18964.11550708881413954016@jljusten-skl>
User-Agent: alot/0.8
Date: Tue, 23 Jul 2019 01:05:25 -0700
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable

On 2019-07-23 00:44:06, Laszlo Ersek wrote:
> On 07/23/19 02:00, Rebecca Cran wrote:
> > On 2019-07-22 17:14, Jordan Justen wrote:
> >>
> >> I was suggesting that if they didn't specify -n as a param to
> >> build.sh, then build.sh should not send -n to the edk2 build command.
> >> The effect would be for the edk2 build command to check
> >> Conf/target.txt. By default, I think target.txt will not set
> >> THREADNUMBER, so "0" would still be the result.
> >>
> >> Yet, it would give them the option to set it in Conf/target.txt.
> >> Today, since we always use the -n param, target.txt is always ignored
> >> for this parameter.
> >=20
> >=20
> > On a related topic, I wonder if we should add a "-j" parameter if we
> > build BaseTools for users (e.g. "make -j4 -C BaseTools")? I've found
> > that it can be pretty slow without it: on my system adding -j4 reduces
> > build time from 55 seconds to 15. Going higher doesn't seem to produce
> > much more benefit: -j32 (on a ThreadRipper system) reduces it to 12 sec=
onds.
> >=20
> >=20
>=20
> Passing
>=20
>   -j $(getconf _NPROCESSORS_ONLN)
>=20
> to "make" (for building BaseTools) makes sense, IMO.

I guess the concern might be that we'll be running a bunch of
make invocations in parallel, each trying to spawn a compilation for
each thread. O(n^2) compilations. :)

In the make man-page for -j: "When make invokes a sub-make, all
instances of make will coordinate to run the specified number of jobs
at a time;", but I'm not sure if that's how `build -n` is implemented.
(With make...)

Since python writes the makefiles, it could be used instead of
getconf, right?

What we need is someone to make the ninja-build backend for BaseTools.
:)

-Jordan