From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=209.132.183.28; helo=mx1.redhat.com; envelope-from=lersek@redhat.com; receiver=edk2-devel@lists.01.org Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id BB769211CAC89 for ; Wed, 6 Feb 2019 08:26:17 -0800 (PST) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 0EE1940F10; Wed, 6 Feb 2019 16:26:17 +0000 (UTC) Received: from lacos-laptop-7.usersys.redhat.com (ovpn-120-4.rdu2.redhat.com [10.10.120.4]) by smtp.corp.redhat.com (Postfix) with ESMTP id 15F201048127; Wed, 6 Feb 2019 16:26:11 +0000 (UTC) To: =?UTF-8?Q?Philippe_Mathieu-Daud=c3=a9?= , Michael Kinney , edk2-devel@lists.01.org, Carsey Jaben References: <20190206120344.15681-1-philmd@redhat.com> From: Laszlo Ersek Message-ID: <5e7b288c-32ad-124f-b9cd-1d916e6281af@redhat.com> Date: Wed, 6 Feb 2019 17:26:11 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: <20190206120344.15681-1-philmd@redhat.com> X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Wed, 06 Feb 2019 16:26:17 +0000 (UTC) Subject: Re: [PATCH v2] BaseTools: Fix build failure when specifying multiple BUILDTARGET X-BeenThere: edk2-devel@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: EDK II Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Feb 2019 16:26:18 -0000 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit On 02/06/19 13:03, Philippe Mathieu-Daudé wrote: > With Python3, the dict.value() method returns an iterator. > If a dictionary is updated while an iterator on its keys is used, > a RuntimeError is generated. > Converting the iterator to a list() forces a copy of the mutable > keys in an immutable list which can be safely iterated. > > Commit f8d11e5a4aaa converted various uses but missed one: > When specifying multiple BUILDTARGET, the first target builds > successfully, but then the PGen.BuildDatabase._CACHE_ dictionary is > updated, and accessing the next target triggers a RuntimeError. > > Convert this iterator to an immutable list, to solve this build error: > > $ build -a IA32 -t GCC5 -b RELEASE -b NOOPT -p OvmfPkg/OvmfPkgIa32.dsc > [...] > Processing meta-data ... > build.py... > : error C0DE: Unknown fatal error when processing [OvmfPkg/OvmfPkgIa32.dsc] > > (Please send email to edk2-devel@lists.01.org for help, attaching following call stack trace!) > > (Python 3.5.3 on linux) Traceback (most recent call last): > File "BaseTools/BinWrappers/PosixLike/../../Source/Python/build/build.py", line 2387, in Main > MyBuild.Launch() > File "BaseTools/BinWrappers/PosixLike/../../Source/Python/build/build.py", line 2141, in Launch > self._MultiThreadBuildPlatform() > File "BaseTools/BinWrappers/PosixLike/../../Source/Python/build/build.py", line 1921, in _MultiThreadBuildPlatform > self.Progress > File "BaseTools/Source/Python/AutoGen/AutoGen.py", line 304, in __init__ > self._InitWorker(Workspace, MetaFile, Target, Toolchain, Arch, *args, **kwargs) > File "BaseTools/Source/Python/AutoGen/AutoGen.py", line 477, in _InitWorker > for BuildData in PGen.BuildDatabase._CACHE_.values(): > RuntimeError: dictionary changed size during iteration > > Note: The culprit commit (f8d11e5a4aaa) can not be found with bisection. > In 9c2d68c0a299 the build tools default to the python version provided > by the ${PYTHON} environment variable, however the Python3 transition is > not functional before d943b0c339fe. f8d11e5a4aaa falls between the > previous two. > > Reported-by: Leif Lindholm > Fixes: f8d11e5a4aaa90bf63b4789f3993dd6d16c60787 > Contributed-under: TianoCore Contribution Agreement 1.1 > Signed-off-by: Philippe Mathieu-Daude > Tested-by: Leif Lindholm > Acked-by: Laszlo Ersek > --- > v2: > - fixed English errors (Laszlo) > - the paragraph about bisection not working is not relevant to > the fix, keep it as background info but move it after (Laszlo) > Signed-off-by: Philippe Mathieu-Daudé > --- > BaseTools/Source/Python/AutoGen/AutoGen.py | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/BaseTools/Source/Python/AutoGen/AutoGen.py b/BaseTools/Source/Python/AutoGen/AutoGen.py > index a95d2c710e..12592a2a46 100644 > --- a/BaseTools/Source/Python/AutoGen/AutoGen.py > +++ b/BaseTools/Source/Python/AutoGen/AutoGen.py > @@ -474,7 +474,7 @@ class WorkspaceAutoGen(AutoGen): > > # generate the SourcePcdDict and BinaryPcdDict > PGen = PlatformAutoGen(self, self.MetaFile, Target, Toolchain, Arch) > - for BuildData in PGen.BuildDatabase._CACHE_.values(): > + for BuildData in list(PGen.BuildDatabase._CACHE_.values()): > if BuildData.Arch != Arch: > continue > if BuildData.MetaFile.Ext == '.inf': > Looks nice, thanks! My A-b stands. Laszlo