From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (NAM10-DM6-obe.outbound.protection.outlook.com [40.92.41.90]) by mx.groups.io with SMTP id smtpd.web12.42.1585847129055118690 for ; Thu, 02 Apr 2020 10:05:29 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@outlook.com header.s=selector1 header.b=S2OFPjo+; spf=pass (domain: outlook.com, ip: 40.92.41.90, mailfrom: michael.kubacki@outlook.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=JsQqmWZulkX+aKRBiwE32UWfQmlsJWHBOcRVDzWyWZlpiqNE2dqnM2LiAcLN+MbUEEyQjjd/eKbu1t8OoMwrm0WWBlekOiPd1H0/tP5hmoq4d8jKp6NYRx3TpGq4Sfjf5l7Ci0N5Y8BLlEXXtuST2bjlWPfIx2LeZXPGSG+fo9pcMCgtg7N8qT8VW9KHOziUAgVN8pI5gObjHhYbN7hBt00uKCCqGB69z7Q+BXJr04Gnaa/JXNk3qWwi+GPKwl8xWk+dc8mU/66PjtXFtdZqG/bejFxD3pi5Zkp9m29LEzUlQhMIiE+EhGjKKLwrOqcmhj4CYHCYgVbXu+/67ldSfg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Y3QOmzGcoESX9rROP4g/fZHUXcl+tkxzBGolncyEZvw=; b=cR2j2kcb7qrwGUUmzjpMRec06TQ4OPeAORK3sARJXzRByQBZqD+PB3+tr9giUMgqQ8EDagsFEV3JHS/86T7wYkvjEy02P7IXE4faZenEiQ+f+Z5+IUmv7fs8/Xsdl0cq/OUwT15o6lTPwXOe17+m1rM3TAIW2TWejEB3BTRLpPEQ/HATyAEhg7+5W9io2SOdpf1npnpWeTWMGwv/kwzpsf5DguNvURaTKuoawlRk7cSwsX598eGmrNlkb/FIaLB3VgD993qk/dPY9tZjpa1vt6qXUZW/UZkOEN2kqHlFt1LUjG2oB6u0tGHZO0eWGqJTF3SF5YRHrPZhS2sJoTS9WQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=outlook.com; dmarc=pass action=none header.from=outlook.com; dkim=pass header.d=outlook.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Y3QOmzGcoESX9rROP4g/fZHUXcl+tkxzBGolncyEZvw=; b=S2OFPjo+ENlyGTujSk80ZEWqqsvuBYjNQSAh9VY3F6Vz3Y2f0BGjvACM+rkWGSj/mXjmmAgBW7SG8NXJ8E3w5XrbAWix+nMyEnJUNkmFzN3KmOsrBo91wy8Nfmm4ebDIs0tMIsRwp6YonRuAXAM4Uw4mIKgo8he1n5lz2jLHjfAcmMLTLfaKOcQ1E5ar1LaY6g7ZO18G5L5tu8B8IPDvunB0nG/E7DCdQh3HTTpdWrUuNQ4j4Jpj9NTX6Erefp7D2HwtxdedoeN1cSpNEMiVW2CHkfI4UAOhdE35V02YwklKijihj8bzEGZofzkQ5o5QxBXhxnV4LKR7QgWpNJ+xYw== Received: from BN7NAM10FT056.eop-nam10.prod.protection.outlook.com (2a01:111:e400:7e8f::53) by BN7NAM10HT098.eop-nam10.prod.protection.outlook.com (2a01:111:e400:7e8f::413) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2878.15; Thu, 2 Apr 2020 17:05:27 +0000 Received: from MWHPR07MB3440.namprd07.prod.outlook.com (2a01:111:e400:7e8f::53) by BN7NAM10FT056.mail.protection.outlook.com (2a01:111:e400:7e8f::149) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2878.15 via Frontend Transport; Thu, 2 Apr 2020 17:05:27 +0000 X-IncomingTopHeaderMarker: OriginalChecksum:9853B63EB848FBF854808F274C5580DD007E80411876BB9A396B43E784367AA5;UpperCasedChecksum:326D87879CFB92334E9B3F77C544440087E7C5ADC9345A4D08088BEC1B3F0583;SizeAsReceived:7609;Count:47 Received: from MWHPR07MB3440.namprd07.prod.outlook.com ([fe80::f5a7:e51b:e22a:959f]) by MWHPR07MB3440.namprd07.prod.outlook.com ([fe80::f5a7:e51b:e22a:959f%7]) with mapi id 15.20.2878.014; Thu, 2 Apr 2020 17:05:27 +0000 From: "Michael Kubacki" To: devel@edk2.groups.io Cc: Bob C Feng , Liming Gao Subject: [PATCH v1 1/1] BaseTools: Update Edk2ToolsBuild.py to use multiple threads on Linux Date: Thu, 2 Apr 2020 10:04:52 -0700 Message-ID: X-Mailer: git-send-email 2.16.3.windows.1 X-ClientProxiedBy: MWHPR04CA0043.namprd04.prod.outlook.com (2603:10b6:300:ee::29) To MWHPR07MB3440.namprd07.prod.outlook.com (2603:10b6:301:69::28) Return-Path: michael.kubacki@outlook.com X-Microsoft-Original-Message-ID: <20200402170452.34420-1-michael.kubacki@outlook.com> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from localhost.localdomain (2001:4898:80e8:a:1195:d8d7:bb83:acad) by MWHPR04CA0043.namprd04.prod.outlook.com (2603:10b6:300:ee::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2878.15 via Frontend Transport; Thu, 2 Apr 2020 17:05:25 +0000 X-Mailer: git-send-email 2.16.3.windows.1 X-Microsoft-Original-Message-ID: <20200402170452.34420-1-michael.kubacki@outlook.com> X-TMN: [/w3T6vSTW+1HOYbZwe3IM/i+xacYRVZ14h2aeH/uM/iA9VavS04n+Pvs589bWDZ5] X-MS-PublicTrafficType: Email X-IncomingHeaderCount: 47 X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-Correlation-Id: a7a042af-5fc0-46d7-eb8a-08d7d7280992 X-MS-TrafficTypeDiagnostic: BN7NAM10HT098: X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: uc3JehMWVpJfB6yJ0r3CnXvGQ9aXlhIuXItyFcvKclRmnOfRIGVRAE2I3gJpKDo723b+689ZhAmplao4CpBuMOuQArBzd3E+QlXk89xs/mc7ZnyQ4ibge6gd57yj1gSnfwsaVxFXNROtwrisqqYHtwYN5S+X2VjyUM8nZFI4IZL+h8aq25xj9SDWCvnBBVekA5K9X1GGBIfXq3i5TQ2cbdZ1k6YaoLnR3zS1iAM32TA= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:0;SRV:;IPV:NLI;SFV:NSPM;H:MWHPR07MB3440.namprd07.prod.outlook.com;PTR:;CAT:NONE;SFTY:;SFS:;DIR:OUT;SFP:1901; X-MS-Exchange-AntiSpam-MessageData: T7AFI0DFrxOrzi1Jrzxe1V0WQ2dR+vrq2Ny9i4YhYzuE5FxbxRUhEw8bBaLJsvcK1csnmP0AEU//4/bnI+m2OtRmdKjowqXhHPFF5EUtFrQF1DZJWW77wXXMGpmGHWMU6yEbvMEtYBGJJukky0n90BUxCgAblsxM5HDEkrez/BY6t7WFUkpB3UQvaMpgaPFCqO8sQ2c5VLUV9PU9LTKJOA== X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: a7a042af-5fc0-46d7-eb8a-08d7d7280992 X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Apr 2020 17:05:27.8239 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-FromEntityHeader: Internet X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN7NAM10HT098 Content-Type: text/plain From: Sean Brogan REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2640 Azure Pipelines agents have 2 threads. This commit has been shown to reduce the build time in half on those agents. Cc: Bob C Feng Cc: Liming Gao Signed-off-by: Michael Kubacki --- BaseTools/Edk2ToolsBuild.py | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/BaseTools/Edk2ToolsBuild.py b/BaseTools/Edk2ToolsBuild.py index 057d2e9e0633..1ea8187de693 100644 --- a/BaseTools/Edk2ToolsBuild.py +++ b/BaseTools/Edk2ToolsBuild.py @@ -11,6 +11,7 @@ import os import sys import logging import argparse +import multiprocessing from edk2toolext import edk2_logging from edk2toolext.environment import self_describing_environment from edk2toolext.base_abstract_invocable import BaseAbstractInvocable @@ -141,7 +142,8 @@ class Edk2ToolsBuild(BaseAbstractInvocable): return ret elif self.tool_chain_tag.lower().startswith("gcc"): - ret = RunCmd("make", "-C .", workingdir=shell_env.get_shell_var("EDK_TOOLS_PATH")) + cpu_count = self.GetCpuThreads() + ret = RunCmd("make", f"-C . -j {cpu_count}", workingdir=shell_env.get_shell_var("EDK_TOOLS_PATH")) if ret != 0: raise Exception("Failed to build.") @@ -154,6 +156,18 @@ class Edk2ToolsBuild(BaseAbstractInvocable): logging.critical("Tool Chain not supported") return -1 + def GetCpuThreads(self) -> int: + ''' Function to return number of cpus. If error return 1''' + cpus = 1 + try: + cpus = multiprocessing.cpu_count() + except: + # from the internet there are cases where cpu_count is not implemented. + # will handle error by just doing single proc build + pass + return cpus + + def main(): Edk2ToolsBuild().Invoke() -- 2.16.3.windows.1