public inbox for devel@edk2.groups.io
 help / color / mirror / Atom feed
From: "Laszlo Ersek" <lersek@redhat.com>
To: devel@edk2.groups.io, bob.c.feng@intel.com
Subject: Re: [edk2-devel] [Patch 00/10 V3] Enable multiple process AutoGen
Date: Tue, 23 Jul 2019 13:30:57 +0200	[thread overview]
Message-ID: <4ea3d3fa-2210-3642-2337-db525312d312@redhat.com> (raw)
In-Reply-To: <20190723035818.21112-1-bob.c.feng@intel.com>

On 07/23/19 05:58, Bob Feng wrote:
> BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1875
> 
> In order to improve the build performance, we implemented
> multiple-processes AutoGen. This change will reduce 20% time
> for AutoGen phase.
> 
> The design document can be got from:
> https://edk2.groups.io/g/devel/files/Designs/2019/0627/Multiple-thread-AutoGen.pdf
> 
> This patch serial pass the build of Ovmf, MinKabylake, MinPurley, packages
> under Edk2 repository and intel client and server platforms.
> 
> V3:
> 1. Fixed incremental build issue.
> 2. Set AutoGen worker number to be align with "-n THREADNUMBER"
> 3. Enable block log queue.
> V2: 
> 1. The first version missed autogen related commit 
> from e812a812c1a0800c49e11507cb46222351520cc7. V2 add those commit
> back.
> 2. Move CreateAsBuildInf into AutoGenWorker process
> 3. Save GlobalVar_<platform guid>_<arch>.bin to build folder.
> 4. Regenerate patches based on master bb824f685d

Tested this series on top of commit cf2d8d4978e8.

(1) The GlobalVar*.bin files are now located under the Build directory,
so that's OK.

(2) Rebuilding OVMF (in a standalone edk2 clone, not as a QEMU
submodule) with different -D flags works fine now. The fw binary hash
matches a similar rebuild without the series applied.

(3) The blocking log queue works fine with Ctrl-S / Ctrl-Q.

Please see my note on the queue size (currently: constant 60) under
patch#10. I think the factor 10 that I suggested makes sense as well,
because I tested V3 with 4 threads (meaning room for 60/4=15 messages
per thread), and I felt that the compilation threads got blocked just a
bit too late after I pressed Ctrl-S. Therefore, a factor of 10 (rather
than the effective 15 that I tested now) will make for a more direct
user experience.

(4) I tested "-n1" and "-n2"; they seem to work fine.

(5) Restarting a build (incrementally) after Ctrl-C seems to work fine.

(6) The build seems to notice changes to .c files. OK.

(7) This version works fine as part of QEMU's edk2 submodule, with "make
-j 4 -C roms efi". All six firmware binaries built there are functional.


* For patches 01 through 09:

Tested-by: Laszlo Ersek <lersek@redhat.com>

* For patch 10: I'd like to test v4, with the proportional log queue
size, and give my T-b then.

* If you change other patches in v4 (01-09), then please drop my
Tested-by from those.

* Note that I didn't test:
- python3
- the "--hash" option
- the "--binary-destination" / "--binary-source" options
  (I'm not sure what those options do in the first place)
- the "--re-parse" option
- the "--no-cache" option
- the "--genfds-multi-thread" option

Hopefully others can test those.

Thanks
Laszlo

  parent reply	other threads:[~2019-07-23 11:30 UTC|newest]

Thread overview: 15+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-07-23  3:58 [Patch 00/10 V3] Enable multiple process AutoGen Bob Feng
2019-07-23  3:58 ` [Patch 01/10 V3] BaseTools: Singleton the object to handle build conf file Bob Feng
2019-07-23  3:58 ` [Patch 02/10 V3] BaseTools: Split WorkspaceAutoGen._InitWorker into multiple functions Bob Feng
2019-07-23  3:58 ` [Patch 03/10 V3] BaseTools: Add functions to get platform scope build options Bob Feng
2019-07-23  3:58 ` [Patch 04/10 V3] BaseTools: Decouple AutoGen Objects Bob Feng
2019-07-23  3:58 ` [Patch 05/10 V3] BaseTools: Enable Multiple Process AutoGen Bob Feng
2019-07-23  3:58 ` [Patch 06/10 V3] BaseTools: Add shared data for processes Bob Feng
2019-07-23  3:58 ` [Patch 07/10 V3] BaseTools: Add LogAgent to support multiple process Autogen Bob Feng
2019-07-23  3:58 ` [Patch 08/10 V3] BaseTools: Move BuildOption parser out of build.py Bob Feng
2019-07-23  3:58 ` [Patch 09/10 V3] BaseTools: Add the support for python 2 Bob Feng
2019-07-23  3:58 ` [Patch 10/10 V3] BaseTools: Enable block queue log agent Bob Feng
2019-07-23 10:11   ` [edk2-devel] " Laszlo Ersek
2019-07-23 12:00     ` Bob Feng
2019-07-23 11:30 ` Laszlo Ersek [this message]
2019-07-30 13:38   ` [edk2-devel] [Patch 00/10 V3] Enable multiple process AutoGen Bob Feng

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-list from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=4ea3d3fa-2210-3642-2337-db525312d312@redhat.com \
    --to=devel@edk2.groups.io \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox