From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=209.132.183.28; helo=mx1.redhat.com; envelope-from=philmd@redhat.com; receiver=edk2-devel@lists.01.org Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id EF20B211B1111 for ; Wed, 6 Feb 2019 04:03:54 -0800 (PST) Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 3A866A7E1; Wed, 6 Feb 2019 12:03:53 +0000 (UTC) Received: from x1w.redhat.com (unknown [10.40.205.222]) by smtp.corp.redhat.com (Postfix) with ESMTPS id C9BCA62F94; Wed, 6 Feb 2019 12:03:47 +0000 (UTC) From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= To: Michael Kinney , edk2-devel@lists.01.org, Carsey Jaben Cc: Laszlo Ersek , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Leif Lindholm Date: Wed, 6 Feb 2019 13:03:44 +0100 Message-Id: <20190206120344.15681-1-philmd@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Wed, 06 Feb 2019 12:03:53 +0000 (UTC) Subject: [PATCH v2] BaseTools: Fix build failure when specifying multiple BUILDTARGET X-BeenThere: edk2-devel@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: EDK II Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Feb 2019 12:03:55 -0000 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit With Python3, the dict.value() method returns an iterator. If a dictionary is updated while an iterator on its keys is used, a RuntimeError is generated. Converting the iterator to a list() forces a copy of the mutable keys in an immutable list which can be safely iterated. Commit f8d11e5a4aaa converted various uses but missed one: When specifying multiple BUILDTARGET, the first target builds successfully, but then the PGen.BuildDatabase._CACHE_ dictionary is updated, and accessing the next target triggers a RuntimeError. Convert this iterator to an immutable list, to solve this build error: $ build -a IA32 -t GCC5 -b RELEASE -b NOOPT -p OvmfPkg/OvmfPkgIa32.dsc [...] Processing meta-data ... build.py... : error C0DE: Unknown fatal error when processing [OvmfPkg/OvmfPkgIa32.dsc] (Please send email to edk2-devel@lists.01.org for help, attaching following call stack trace!) (Python 3.5.3 on linux) Traceback (most recent call last): File "BaseTools/BinWrappers/PosixLike/../../Source/Python/build/build.py", line 2387, in Main MyBuild.Launch() File "BaseTools/BinWrappers/PosixLike/../../Source/Python/build/build.py", line 2141, in Launch self._MultiThreadBuildPlatform() File "BaseTools/BinWrappers/PosixLike/../../Source/Python/build/build.py", line 1921, in _MultiThreadBuildPlatform self.Progress File "BaseTools/Source/Python/AutoGen/AutoGen.py", line 304, in __init__ self._InitWorker(Workspace, MetaFile, Target, Toolchain, Arch, *args, **kwargs) File "BaseTools/Source/Python/AutoGen/AutoGen.py", line 477, in _InitWorker for BuildData in PGen.BuildDatabase._CACHE_.values(): RuntimeError: dictionary changed size during iteration Note: The culprit commit (f8d11e5a4aaa) can not be found with bisection. In 9c2d68c0a299 the build tools default to the python version provided by the ${PYTHON} environment variable, however the Python3 transition is not functional before d943b0c339fe. f8d11e5a4aaa falls between the previous two. Reported-by: Leif Lindholm Fixes: f8d11e5a4aaa90bf63b4789f3993dd6d16c60787 Contributed-under: TianoCore Contribution Agreement 1.1 Signed-off-by: Philippe Mathieu-Daude Tested-by: Leif Lindholm Acked-by: Laszlo Ersek --- v2: - fixed English errors (Laszlo) - the paragraph about bisection not working is not relevant to the fix, keep it as background info but move it after (Laszlo) Signed-off-by: Philippe Mathieu-Daudé --- BaseTools/Source/Python/AutoGen/AutoGen.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/BaseTools/Source/Python/AutoGen/AutoGen.py b/BaseTools/Source/Python/AutoGen/AutoGen.py index a95d2c710e..12592a2a46 100644 --- a/BaseTools/Source/Python/AutoGen/AutoGen.py +++ b/BaseTools/Source/Python/AutoGen/AutoGen.py @@ -474,7 +474,7 @@ class WorkspaceAutoGen(AutoGen): # generate the SourcePcdDict and BinaryPcdDict PGen = PlatformAutoGen(self, self.MetaFile, Target, Toolchain, Arch) - for BuildData in PGen.BuildDatabase._CACHE_.values(): + for BuildData in list(PGen.BuildDatabase._CACHE_.values()): if BuildData.Arch != Arch: continue if BuildData.MetaFile.Ext == '.inf': -- 2.20.1