* [PATCH v1 1/1] BaseTools: Update Edk2ToolsBuild.py to use multiple threads on Linux
@ 2020-04-02 17:04 Michael Kubacki
2020-04-03 3:26 ` [edk2-devel] " Bob Feng
2020-04-12 15:00 ` Liming Gao
0 siblings, 2 replies; 3+ messages in thread
From: Michael Kubacki @ 2020-04-02 17:04 UTC (permalink / raw)
To: devel; +Cc: Bob C Feng, Liming Gao
From: Sean Brogan <sean.brogan@microsoft.com>
REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2640
Azure Pipelines agents have 2 threads. This commit has been shown to
reduce the build time in half on those agents.
Cc: Bob C Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
---
BaseTools/Edk2ToolsBuild.py | 16 +++++++++++++++-
1 file changed, 15 insertions(+), 1 deletion(-)
diff --git a/BaseTools/Edk2ToolsBuild.py b/BaseTools/Edk2ToolsBuild.py
index 057d2e9e0633..1ea8187de693 100644
--- a/BaseTools/Edk2ToolsBuild.py
+++ b/BaseTools/Edk2ToolsBuild.py
@@ -11,6 +11,7 @@ import os
import sys
import logging
import argparse
+import multiprocessing
from edk2toolext import edk2_logging
from edk2toolext.environment import self_describing_environment
from edk2toolext.base_abstract_invocable import BaseAbstractInvocable
@@ -141,7 +142,8 @@ class Edk2ToolsBuild(BaseAbstractInvocable):
return ret
elif self.tool_chain_tag.lower().startswith("gcc"):
- ret = RunCmd("make", "-C .", workingdir=shell_env.get_shell_var("EDK_TOOLS_PATH"))
+ cpu_count = self.GetCpuThreads()
+ ret = RunCmd("make", f"-C . -j {cpu_count}", workingdir=shell_env.get_shell_var("EDK_TOOLS_PATH"))
if ret != 0:
raise Exception("Failed to build.")
@@ -154,6 +156,18 @@ class Edk2ToolsBuild(BaseAbstractInvocable):
logging.critical("Tool Chain not supported")
return -1
+ def GetCpuThreads(self) -> int:
+ ''' Function to return number of cpus. If error return 1'''
+ cpus = 1
+ try:
+ cpus = multiprocessing.cpu_count()
+ except:
+ # from the internet there are cases where cpu_count is not implemented.
+ # will handle error by just doing single proc build
+ pass
+ return cpus
+
+
def main():
Edk2ToolsBuild().Invoke()
--
2.16.3.windows.1
^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [edk2-devel] [PATCH v1 1/1] BaseTools: Update Edk2ToolsBuild.py to use multiple threads on Linux
2020-04-02 17:04 [PATCH v1 1/1] BaseTools: Update Edk2ToolsBuild.py to use multiple threads on Linux Michael Kubacki
@ 2020-04-03 3:26 ` Bob Feng
2020-04-12 15:00 ` Liming Gao
1 sibling, 0 replies; 3+ messages in thread
From: Bob Feng @ 2020-04-03 3:26 UTC (permalink / raw)
To: devel@edk2.groups.io, michael.kubacki@outlook.com; +Cc: Gao, Liming
This change is good to me.
Reviewed-by: Bob C Feng <bob.c.feng@intel.com>
-----Original Message-----
From: devel@edk2.groups.io [mailto:devel@edk2.groups.io] On Behalf Of Michael Kubacki
Sent: Friday, April 3, 2020 1:05 AM
To: devel@edk2.groups.io
Cc: Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming <liming.gao@intel.com>
Subject: [edk2-devel] [PATCH v1 1/1] BaseTools: Update Edk2ToolsBuild.py to use multiple threads on Linux
From: Sean Brogan <sean.brogan@microsoft.com>
REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2640
Azure Pipelines agents have 2 threads. This commit has been shown to reduce the build time in half on those agents.
Cc: Bob C Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
---
BaseTools/Edk2ToolsBuild.py | 16 +++++++++++++++-
1 file changed, 15 insertions(+), 1 deletion(-)
diff --git a/BaseTools/Edk2ToolsBuild.py b/BaseTools/Edk2ToolsBuild.py index 057d2e9e0633..1ea8187de693 100644
--- a/BaseTools/Edk2ToolsBuild.py
+++ b/BaseTools/Edk2ToolsBuild.py
@@ -11,6 +11,7 @@ import os
import sys
import logging
import argparse
+import multiprocessing
from edk2toolext import edk2_logging
from edk2toolext.environment import self_describing_environment from edk2toolext.base_abstract_invocable import BaseAbstractInvocable @@ -141,7 +142,8 @@ class Edk2ToolsBuild(BaseAbstractInvocable):
return ret
elif self.tool_chain_tag.lower().startswith("gcc"):
- ret = RunCmd("make", "-C .", workingdir=shell_env.get_shell_var("EDK_TOOLS_PATH"))
+ cpu_count = self.GetCpuThreads()
+ ret = RunCmd("make", f"-C . -j {cpu_count}",
+ workingdir=shell_env.get_shell_var("EDK_TOOLS_PATH"))
if ret != 0:
raise Exception("Failed to build.")
@@ -154,6 +156,18 @@ class Edk2ToolsBuild(BaseAbstractInvocable):
logging.critical("Tool Chain not supported")
return -1
+ def GetCpuThreads(self) -> int:
+ ''' Function to return number of cpus. If error return 1'''
+ cpus = 1
+ try:
+ cpus = multiprocessing.cpu_count()
+ except:
+ # from the internet there are cases where cpu_count is not implemented.
+ # will handle error by just doing single proc build
+ pass
+ return cpus
+
+
def main():
Edk2ToolsBuild().Invoke()
--
2.16.3.windows.1
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [edk2-devel] [PATCH v1 1/1] BaseTools: Update Edk2ToolsBuild.py to use multiple threads on Linux
2020-04-02 17:04 [PATCH v1 1/1] BaseTools: Update Edk2ToolsBuild.py to use multiple threads on Linux Michael Kubacki
2020-04-03 3:26 ` [edk2-devel] " Bob Feng
@ 2020-04-12 15:00 ` Liming Gao
1 sibling, 0 replies; 3+ messages in thread
From: Liming Gao @ 2020-04-12 15:00 UTC (permalink / raw)
To: devel@edk2.groups.io, michael.kubacki@outlook.com; +Cc: Feng, Bob C
Reviewed-by: Liming Gao <liming.gao@intel.com>
> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Michael Kubacki
> Sent: Friday, April 3, 2020 1:05 AM
> To: devel@edk2.groups.io
> Cc: Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming <liming.gao@intel.com>
> Subject: [edk2-devel] [PATCH v1 1/1] BaseTools: Update Edk2ToolsBuild.py to use multiple threads on Linux
>
> From: Sean Brogan <sean.brogan@microsoft.com>
>
> REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2640
>
> Azure Pipelines agents have 2 threads. This commit has been shown to
> reduce the build time in half on those agents.
>
> Cc: Bob C Feng <bob.c.feng@intel.com>
> Cc: Liming Gao <liming.gao@intel.com>
> Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
> ---
> BaseTools/Edk2ToolsBuild.py | 16 +++++++++++++++-
> 1 file changed, 15 insertions(+), 1 deletion(-)
>
> diff --git a/BaseTools/Edk2ToolsBuild.py b/BaseTools/Edk2ToolsBuild.py
> index 057d2e9e0633..1ea8187de693 100644
> --- a/BaseTools/Edk2ToolsBuild.py
> +++ b/BaseTools/Edk2ToolsBuild.py
> @@ -11,6 +11,7 @@ import os
> import sys
> import logging
> import argparse
> +import multiprocessing
> from edk2toolext import edk2_logging
> from edk2toolext.environment import self_describing_environment
> from edk2toolext.base_abstract_invocable import BaseAbstractInvocable
> @@ -141,7 +142,8 @@ class Edk2ToolsBuild(BaseAbstractInvocable):
> return ret
>
> elif self.tool_chain_tag.lower().startswith("gcc"):
> - ret = RunCmd("make", "-C .", workingdir=shell_env.get_shell_var("EDK_TOOLS_PATH"))
> + cpu_count = self.GetCpuThreads()
> + ret = RunCmd("make", f"-C . -j {cpu_count}", workingdir=shell_env.get_shell_var("EDK_TOOLS_PATH"))
> if ret != 0:
> raise Exception("Failed to build.")
>
> @@ -154,6 +156,18 @@ class Edk2ToolsBuild(BaseAbstractInvocable):
> logging.critical("Tool Chain not supported")
> return -1
>
> + def GetCpuThreads(self) -> int:
> + ''' Function to return number of cpus. If error return 1'''
> + cpus = 1
> + try:
> + cpus = multiprocessing.cpu_count()
> + except:
> + # from the internet there are cases where cpu_count is not implemented.
> + # will handle error by just doing single proc build
> + pass
> + return cpus
> +
> +
>
> def main():
> Edk2ToolsBuild().Invoke()
> --
> 2.16.3.windows.1
>
>
>
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2020-04-12 15:00 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2020-04-02 17:04 [PATCH v1 1/1] BaseTools: Update Edk2ToolsBuild.py to use multiple threads on Linux Michael Kubacki
2020-04-03 3:26 ` [edk2-devel] " Bob Feng
2020-04-12 15:00 ` Liming Gao
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox