From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: redhat.com, ip: 209.132.183.28, mailfrom: lersek@redhat.com) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by groups.io with SMTP; Thu, 08 Aug 2019 13:43:42 -0700 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 976828553D; Thu, 8 Aug 2019 20:43:41 +0000 (UTC) Received: from lacos-laptop-7.usersys.redhat.com (ovpn-117-122.ams2.redhat.com [10.36.117.122]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0D5EA194BB; Thu, 8 Aug 2019 20:43:39 +0000 (UTC) Subject: Re: [edk2-devel] [PATCH 0/4] Build cache enhancement To: devel@edk2.groups.io, steven.shi@intel.com Cc: liming.gao@intel.com, bob.c.feng@intel.com, christian.rodriguez@intel.com, michael.johnson@intel.com References: <20190808122519.25516-1-steven.shi@intel.com> From: "Laszlo Ersek" Message-ID: <2667645b-e838-b570-18b2-2a3d9aa58c0e@redhat.com> Date: Thu, 8 Aug 2019 22:43:39 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: <20190808122519.25516-1-steven.shi@intel.com> X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Thu, 08 Aug 2019 20:43:41 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit On 08/08/19 14:25, Steven Shi wrote: > From: "Shi, Steven" > > Enhance the edk2 build cache with below patches: > Patch 01/04: Improve the cache hit rate through new cache checkpoint and hash algorithm > Patch 02/04: Print more info to explain why a module build cache miss > Patch 03/04: Fix the unsafe [self.Arch][self.Name] key usage in build cache > Patch 04/04 Add the GenFds multi-thread support in build cache > > This patch set is based on patch set of [Patch 00/10 V8] Enable multiple process AutoGen > https://edk2.groups.io/g/devel/topic/patch_00_10_v8_enable/32779325?p=,,,20,0,0,0::recentpostdate%2Fsticky,,,20,2,140,32779325 I've now updated BZ#1927 to depend on BZ#1875. > You can directly try this patch set in the branch: > https://github.com/shijunjing/edk2/tree/build_cache_improve_v1 > > > Shi, Steven (4): > BaseTools: Improve the cache hit in the edk2 build cache > BaseTools: Print first cache missing file for build cachle > BaseTools: Change the [Arch][Name] module key in Build cache > BaseTools: Add GenFds multi-thread support in build cache > > .../Source/Python/AutoGen/AutoGenWorker.py | 23 + > BaseTools/Source/Python/AutoGen/CacheIR.py | 28 + > BaseTools/Source/Python/AutoGen/DataPipe.py | 8 + > BaseTools/Source/Python/AutoGen/GenMake.py | 229 +++--- > .../Source/Python/AutoGen/ModuleAutoGen.py | 742 ++++++++++++++++-- > BaseTools/Source/Python/Common/GlobalData.py | 9 + > BaseTools/Source/Python/build/build.py | 171 ++-- > 7 files changed, 979 insertions(+), 231 deletions(-) > mode change 100644 => 100755 BaseTools/Source/Python/AutoGen/AutoGenWorker.py > create mode 100755 BaseTools/Source/Python/AutoGen/CacheIR.py > mode change 100644 => 100755 BaseTools/Source/Python/AutoGen/DataPipe.py > mode change 100644 => 100755 BaseTools/Source/Python/AutoGen/GenMake.py > mode change 100644 => 100755 BaseTools/Source/Python/AutoGen/ModuleAutoGen.py > mode change 100644 => 100755 BaseTools/Source/Python/Common/GlobalData.py > mode change 100644 => 100755 BaseTools/Source/Python/build/build.py > Please change the status of BZ#1927 to IN_PROGRESS, and link this thread (from the mailing list archive) into a comment in BZ#1927, at the same time. Thanks Laszlo