* [PATCH] Platform/Intel: Dynamic NUMBER_OF_PROCESSORS in build. @ 2019-06-13 6:14 Chiu, Chasel 2019-06-13 7:39 ` [edk2-devel] " Bob Feng 2019-06-13 8:19 ` Nate DeSimone 0 siblings, 2 replies; 4+ messages in thread From: Chiu, Chasel @ 2019-06-13 6:14 UTC (permalink / raw) To: devel; +Cc: Michael Kubacki, Nate DeSimone, Liming Gao REF: https://bugzilla.tianocore.org/show_bug.cgi?id=1913 To improve build performance by default the maximum processors should be used for building. To support this, NUMBER_OF_PROCESSORS in build.cfg has new definition: 0 = AUTO. Maximum of processors will be used. non-zero = limit or force the number of processors. Test: tried several NUMBER_OF_PROCESSORS settings and the build parameter can be updated accordingly also builds successfully. Cc: Michael Kubacki <michael.a.kubacki@intel.com> Cc: Nate DeSimone <nathaniel.l.desimone@intel.com> Cc: Liming Gao <liming.gao@intel.com> Signed-off-by: Chasel Chiu <chasel.chiu@intel.com> --- Platform/Intel/build.cfg | 2 +- Platform/Intel/build_bios.py | 8 +++++++- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/Platform/Intel/build.cfg b/Platform/Intel/build.cfg index 6c23e5eabc..fc6e4fe824 100644 --- a/Platform/Intel/build.cfg +++ b/Platform/Intel/build.cfg @@ -46,7 +46,7 @@ FSP_TEST_RELEASE = FALSE SECURE_BOOT_ENABLE = FALSE REBUILD_MODE = BUILD_ROM_ONLY = -NUMBER_OF_PROCESSORS = 1 +NUMBER_OF_PROCESSORS = 0 [PLATFORMS] diff --git a/Platform/Intel/build_bios.py b/Platform/Intel/build_bios.py index 09eceddeff..864511ac4d 100644 --- a/Platform/Intel/build_bios.py +++ b/Platform/Intel/build_bios.py @@ -21,6 +21,7 @@ import shutil import argparse import traceback import subprocess +import multiprocessing from importlib import import_module try: @@ -345,7 +346,12 @@ def build(config): print(" BUILD_ROM_ONLY = ", config.get("BUILD_ROM_ONLY")) print("==========================================") - command = ["build", "-n", config["NUMBER_OF_PROCESSORS"]] + NumberOfProcessors = config["NUMBER_OF_PROCESSORS"] + if config["NUMBER_OF_PROCESSORS"] == "0": + # 0 means AUTO, maximum number of processors will be set + NumberOfProcessors = str(multiprocessing.cpu_count()) + print ("Number of processors set to " + NumberOfProcessors) + command = ["build", "-n", NumberOfProcessors] if config["REBUILD_MODE"] and config["REBUILD_MODE"] != "": command.append(config["REBUILD_MODE"]) -- 2.13.3.windows.1 ^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [edk2-devel] [PATCH] Platform/Intel: Dynamic NUMBER_OF_PROCESSORS in build. 2019-06-13 6:14 [PATCH] Platform/Intel: Dynamic NUMBER_OF_PROCESSORS in build Chiu, Chasel @ 2019-06-13 7:39 ` Bob Feng 2019-06-13 8:23 ` Chiu, Chasel 2019-06-13 8:19 ` Nate DeSimone 1 sibling, 1 reply; 4+ messages in thread From: Bob Feng @ 2019-06-13 7:39 UTC (permalink / raw) To: devel@edk2.groups.io, Chiu, Chasel Cc: Kubacki, Michael A, Desimone, Nathaniel L, Gao, Liming Hi Chasel, Patch looks good. For the process number, build.py does the same thing as this patch. If pass -n 0 to build.py, build.py will set the processor number to multiprocessing.cpu_count() Thanks, Bob -----Original Message----- From: devel@edk2.groups.io [mailto:devel@edk2.groups.io] On Behalf Of Chiu, Chasel Sent: Thursday, June 13, 2019 2:15 PM To: devel@edk2.groups.io Cc: Kubacki, Michael A <michael.a.kubacki@intel.com>; Desimone, Nathaniel L <nathaniel.l.desimone@intel.com>; Gao, Liming <liming.gao@intel.com> Subject: [edk2-devel] [PATCH] Platform/Intel: Dynamic NUMBER_OF_PROCESSORS in build. REF: https://bugzilla.tianocore.org/show_bug.cgi?id=1913 To improve build performance by default the maximum processors should be used for building. To support this, NUMBER_OF_PROCESSORS in build.cfg has new definition: 0 = AUTO. Maximum of processors will be used. non-zero = limit or force the number of processors. Test: tried several NUMBER_OF_PROCESSORS settings and the build parameter can be updated accordingly also builds successfully. Cc: Michael Kubacki <michael.a.kubacki@intel.com> Cc: Nate DeSimone <nathaniel.l.desimone@intel.com> Cc: Liming Gao <liming.gao@intel.com> Signed-off-by: Chasel Chiu <chasel.chiu@intel.com> --- Platform/Intel/build.cfg | 2 +- Platform/Intel/build_bios.py | 8 +++++++- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/Platform/Intel/build.cfg b/Platform/Intel/build.cfg index 6c23e5eabc..fc6e4fe824 100644 --- a/Platform/Intel/build.cfg +++ b/Platform/Intel/build.cfg @@ -46,7 +46,7 @@ FSP_TEST_RELEASE = FALSE SECURE_BOOT_ENABLE = FALSE REBUILD_MODE = BUILD_ROM_ONLY = -NUMBER_OF_PROCESSORS = 1 +NUMBER_OF_PROCESSORS = 0 [PLATFORMS] diff --git a/Platform/Intel/build_bios.py b/Platform/Intel/build_bios.py index 09eceddeff..864511ac4d 100644 --- a/Platform/Intel/build_bios.py +++ b/Platform/Intel/build_bios.py @@ -21,6 +21,7 @@ import shutil import argparse import traceback import subprocess +import multiprocessing from importlib import import_module try: @@ -345,7 +346,12 @@ def build(config): print(" BUILD_ROM_ONLY = ", config.get("BUILD_ROM_ONLY")) print("==========================================") - command = ["build", "-n", config["NUMBER_OF_PROCESSORS"]] + NumberOfProcessors = config["NUMBER_OF_PROCESSORS"] + if config["NUMBER_OF_PROCESSORS"] == "0": + # 0 means AUTO, maximum number of processors will be set + NumberOfProcessors = str(multiprocessing.cpu_count()) + print ("Number of processors set to " + NumberOfProcessors) + command = ["build", "-n", NumberOfProcessors] if config["REBUILD_MODE"] and config["REBUILD_MODE"] != "": command.append(config["REBUILD_MODE"]) -- 2.13.3.windows.1 ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [edk2-devel] [PATCH] Platform/Intel: Dynamic NUMBER_OF_PROCESSORS in build. 2019-06-13 7:39 ` [edk2-devel] " Bob Feng @ 2019-06-13 8:23 ` Chiu, Chasel 0 siblings, 0 replies; 4+ messages in thread From: Chiu, Chasel @ 2019-06-13 8:23 UTC (permalink / raw) To: Feng, Bob C, devel@edk2.groups.io Cc: Kubacki, Michael A, Desimone, Nathaniel L, Gao, Liming Thanks Bob! I was not aware that BaseTools already handled this. I just verified and it works so I will update code review which will only touch build.cfg. Thanks! Chasel > -----Original Message----- > From: Feng, Bob C > Sent: Thursday, June 13, 2019 3:39 PM > To: devel@edk2.groups.io; Chiu, Chasel <chasel.chiu@intel.com> > Cc: Kubacki, Michael A <michael.a.kubacki@intel.com>; Desimone, Nathaniel L > <nathaniel.l.desimone@intel.com>; Gao, Liming <liming.gao@intel.com> > Subject: RE: [edk2-devel] [PATCH] Platform/Intel: Dynamic > NUMBER_OF_PROCESSORS in build. > > Hi Chasel, > > Patch looks good. > > For the process number, build.py does the same thing as this patch. If pass -n 0 to > build.py, build.py will set the processor number to multiprocessing.cpu_count() > > Thanks, > Bob > > > -----Original Message----- > From: devel@edk2.groups.io [mailto:devel@edk2.groups.io] On Behalf Of Chiu, > Chasel > Sent: Thursday, June 13, 2019 2:15 PM > To: devel@edk2.groups.io > Cc: Kubacki, Michael A <michael.a.kubacki@intel.com>; Desimone, Nathaniel L > <nathaniel.l.desimone@intel.com>; Gao, Liming <liming.gao@intel.com> > Subject: [edk2-devel] [PATCH] Platform/Intel: Dynamic > NUMBER_OF_PROCESSORS in build. > > REF: https://bugzilla.tianocore.org/show_bug.cgi?id=1913 > > To improve build performance by default the maximum processors should be > used for building. > To support this, NUMBER_OF_PROCESSORS in build.cfg has new definition: > 0 = AUTO. Maximum of processors will be used. > non-zero = limit or force the number of processors. > > Test: tried several NUMBER_OF_PROCESSORS settings and > the build parameter can be updated accordingly > also builds successfully. > > Cc: Michael Kubacki <michael.a.kubacki@intel.com> > Cc: Nate DeSimone <nathaniel.l.desimone@intel.com> > Cc: Liming Gao <liming.gao@intel.com> > Signed-off-by: Chasel Chiu <chasel.chiu@intel.com> > --- > Platform/Intel/build.cfg | 2 +- > Platform/Intel/build_bios.py | 8 +++++++- > 2 files changed, 8 insertions(+), 2 deletions(-) > > diff --git a/Platform/Intel/build.cfg b/Platform/Intel/build.cfg index > 6c23e5eabc..fc6e4fe824 100644 > --- a/Platform/Intel/build.cfg > +++ b/Platform/Intel/build.cfg > @@ -46,7 +46,7 @@ FSP_TEST_RELEASE = FALSE SECURE_BOOT_ENABLE = > FALSE REBUILD_MODE = BUILD_ROM_ONLY = -NUMBER_OF_PROCESSORS = > 1 > +NUMBER_OF_PROCESSORS = 0 > > > [PLATFORMS] > diff --git a/Platform/Intel/build_bios.py b/Platform/Intel/build_bios.py index > 09eceddeff..864511ac4d 100644 > --- a/Platform/Intel/build_bios.py > +++ b/Platform/Intel/build_bios.py > @@ -21,6 +21,7 @@ import shutil > import argparse > import traceback > import subprocess > +import multiprocessing > from importlib import import_module > > try: > @@ -345,7 +346,12 @@ def build(config): > print(" BUILD_ROM_ONLY = ", config.get("BUILD_ROM_ONLY")) > print("==========================================") > > - command = ["build", "-n", config["NUMBER_OF_PROCESSORS"]] > + NumberOfProcessors = config["NUMBER_OF_PROCESSORS"] > + if config["NUMBER_OF_PROCESSORS"] == "0": > + # 0 means AUTO, maximum number of processors will be set > + NumberOfProcessors = str(multiprocessing.cpu_count()) > + print ("Number of processors set to " + NumberOfProcessors) > + command = ["build", "-n", NumberOfProcessors] > > if config["REBUILD_MODE"] and config["REBUILD_MODE"] != "": > command.append(config["REBUILD_MODE"]) > -- > 2.13.3.windows.1 > > > ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] Platform/Intel: Dynamic NUMBER_OF_PROCESSORS in build. 2019-06-13 6:14 [PATCH] Platform/Intel: Dynamic NUMBER_OF_PROCESSORS in build Chiu, Chasel 2019-06-13 7:39 ` [edk2-devel] " Bob Feng @ 2019-06-13 8:19 ` Nate DeSimone 1 sibling, 0 replies; 4+ messages in thread From: Nate DeSimone @ 2019-06-13 8:19 UTC (permalink / raw) To: Chiu, Chasel, devel@edk2.groups.io; +Cc: Kubacki, Michael A, Gao, Liming Reviewed-by: Nate DeSimone <nathaniel.l.desimone@intel.com> On 6/12/19, 11:17 PM, "Chiu, Chasel" <chasel.chiu@intel.com> wrote: REF: https://bugzilla.tianocore.org/show_bug.cgi?id=1913 To improve build performance by default the maximum processors should be used for building. To support this, NUMBER_OF_PROCESSORS in build.cfg has new definition: 0 = AUTO. Maximum of processors will be used. non-zero = limit or force the number of processors. Test: tried several NUMBER_OF_PROCESSORS settings and the build parameter can be updated accordingly also builds successfully. Cc: Michael Kubacki <michael.a.kubacki@intel.com> Cc: Nate DeSimone <nathaniel.l.desimone@intel.com> Cc: Liming Gao <liming.gao@intel.com> Signed-off-by: Chasel Chiu <chasel.chiu@intel.com> --- Platform/Intel/build.cfg | 2 +- Platform/Intel/build_bios.py | 8 +++++++- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/Platform/Intel/build.cfg b/Platform/Intel/build.cfg index 6c23e5eabc..fc6e4fe824 100644 --- a/Platform/Intel/build.cfg +++ b/Platform/Intel/build.cfg @@ -46,7 +46,7 @@ FSP_TEST_RELEASE = FALSE SECURE_BOOT_ENABLE = FALSE REBUILD_MODE = BUILD_ROM_ONLY = -NUMBER_OF_PROCESSORS = 1 +NUMBER_OF_PROCESSORS = 0 [PLATFORMS] diff --git a/Platform/Intel/build_bios.py b/Platform/Intel/build_bios.py index 09eceddeff..864511ac4d 100644 --- a/Platform/Intel/build_bios.py +++ b/Platform/Intel/build_bios.py @@ -21,6 +21,7 @@ import shutil import argparse import traceback import subprocess +import multiprocessing from importlib import import_module try: @@ -345,7 +346,12 @@ def build(config): print(" BUILD_ROM_ONLY = ", config.get("BUILD_ROM_ONLY")) print("==========================================") - command = ["build", "-n", config["NUMBER_OF_PROCESSORS"]] + NumberOfProcessors = config["NUMBER_OF_PROCESSORS"] + if config["NUMBER_OF_PROCESSORS"] == "0": + # 0 means AUTO, maximum number of processors will be set + NumberOfProcessors = str(multiprocessing.cpu_count()) + print ("Number of processors set to " + NumberOfProcessors) + command = ["build", "-n", NumberOfProcessors] if config["REBUILD_MODE"] and config["REBUILD_MODE"] != "": command.append(config["REBUILD_MODE"]) -- 2.13.3.windows.1 ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2019-06-13 8:33 UTC | newest] Thread overview: 4+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2019-06-13 6:14 [PATCH] Platform/Intel: Dynamic NUMBER_OF_PROCESSORS in build Chiu, Chasel 2019-06-13 7:39 ` [edk2-devel] " Bob Feng 2019-06-13 8:23 ` Chiu, Chasel 2019-06-13 8:19 ` Nate DeSimone
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox