From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: intel.com, ip: 134.134.136.65, mailfrom: bob.c.feng@intel.com) Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by groups.io with SMTP; Wed, 17 Jul 2019 23:14:34 -0700 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 17 Jul 2019 23:14:33 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.64,276,1559545200"; d="scan'208";a="319543860" Received: from shwdepsi1121.ccr.corp.intel.com ([10.239.158.47]) by orsmga004.jf.intel.com with ESMTP; 17 Jul 2019 23:14:32 -0700 From: "Bob Feng" To: devel@edk2.groups.io Subject: [Patch 0/9] Enable multiple process AutoGen Date: Thu, 18 Jul 2019 14:14:14 +0800 Message-Id: <20190718061423.30612-1-bob.c.feng@intel.com> X-Mailer: git-send-email 2.20.1.windows.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1875 In order to improve the build performance, we implemented multiple-processes AutoGen. This change will reduce 20% time for AutoGen phase. The design document can be got from: https://edk2.groups.io/g/devel/files/Designs/2019/0627/Multiple-thread-AutoGen.pdf This patch serial pass the build of Ovmf, MinKabylake, MinPurley, packages under Edk2 repository and intel client and server platforms. Feng, Bob C (9): BaseTools: Singleton the object to handle build conf file BaseTools: Split WorkspaceAutoGen._InitWorker into multiple functions BaseTools: Add functions to get platform scope build options BaseTools: Decouple AutoGen Objects BaseTools: Enable Multiple Process AutoGen BaseTools: Add shared data for processes BaseTools: Add LogAgent to support multiple process Autogen BaseTools: Move BuildOption parser out of build.py BaseTools: Add the support for python 2 BaseTools/Source/Python/AutoGen/AutoGen.py | 4227 +---------------- .../Source/Python/AutoGen/AutoGenWorker.py | 220 + .../Source/Python/AutoGen/BuildEngine.py | 22 + BaseTools/Source/Python/AutoGen/DataPipe.py | 153 + BaseTools/Source/Python/AutoGen/GenC.py | 6 +- .../Source/Python/AutoGen/ModuleAutoGen.py | 1883 ++++++++ .../Python/AutoGen/ModuleAutoGenHelper.py | 616 +++ .../Source/Python/AutoGen/PlatformAutoGen.py | 1483 ++++++ .../Source/Python/AutoGen/WorkspaceAutoGen.py | 902 ++++ BaseTools/Source/Python/Common/EdkLogger.py | 117 +- BaseTools/Source/Python/Common/Misc.py | 1 - .../Python/Common/TargetTxtClassObject.py | 28 +- .../Python/Common/ToolDefClassObject.py | 6 +- BaseTools/Source/Python/GenFds/GenFds.py | 4 +- .../Python/GenFds/GenFdsGlobalVariable.py | 54 +- .../Python/PatchPcdValue/PatchPcdValue.py | 1 - .../Source/Python/Workspace/DscBuildData.py | 38 +- .../Source/Python/Workspace/InfBuildData.py | 39 + .../Python/Workspace/WorkspaceCommon.py | 4 + .../Python/Workspace/WorkspaceDatabase.py | 3 + BaseTools/Source/Python/build/BuildReport.py | 4 +- BaseTools/Source/Python/build/build.py | 322 +- BaseTools/Source/Python/build/buildoptions.py | 92 + 23 files changed, 5789 insertions(+), 4436 deletions(-) create mode 100644 BaseTools/Source/Python/AutoGen/AutoGenWorker.py create mode 100644 BaseTools/Source/Python/AutoGen/DataPipe.py create mode 100644 BaseTools/Source/Python/AutoGen/ModuleAutoGen.py create mode 100644 BaseTools/Source/Python/AutoGen/ModuleAutoGenHelper.py create mode 100644 BaseTools/Source/Python/AutoGen/PlatformAutoGen.py create mode 100644 BaseTools/Source/Python/AutoGen/WorkspaceAutoGen.py create mode 100644 BaseTools/Source/Python/build/buildoptions.py -- 2.20.1.windows.1