From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=2a00:1450:4864:20::344; helo=mail-wm1-x344.google.com; envelope-from=leif.lindholm@linaro.org; receiver=edk2-devel@lists.01.org Received: from mail-wm1-x344.google.com (mail-wm1-x344.google.com [IPv6:2a00:1450:4864:20::344]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 8C3DA21962301 for ; Tue, 5 Feb 2019 03:47:05 -0800 (PST) Received: by mail-wm1-x344.google.com with SMTP id y139so3241959wmc.5 for ; Tue, 05 Feb 2019 03:47:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:content-transfer-encoding:in-reply-to :user-agent; bh=yjg+Cq5JGNjf5R2YcC8HycUVWiniD8UGLpSgO2UbcbI=; b=tlUT3MggFS25ldAstmQbo8WZtg7JKxBwfMmB1M65FxLqDCrwa58gOTAcymjTX3N6xf c6rW/iXKw9dDLlx9+EMhkxXZJe4QP573S31esibdv7xhWZoUhtNZbRyOcmbkHHdxLA+p k4jvd+KVzmcvH9DCCr34/gJ1XDLyj1e7z8k7NGwA3YKRt7p1vyemMPygMNf/oiOzKOh7 vpck8P02cTRbFJNZbKwI1npN/aK24xwaw8LSdq+O2x243W7OYnxF863q2PKhP5yO1ZKA Vo4MrZp4BT47+RA+8hJcN8jDjgg+dtMqaX+iabgLmkdMtJWs4+BL7K2hK617AyhQvmkf 7xEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=yjg+Cq5JGNjf5R2YcC8HycUVWiniD8UGLpSgO2UbcbI=; b=SdbQMPk092VIq7wsnZcmnCoT3L9tz6Rpiyh4g3WQ0Y/y5pT+uC/neJbftcFYKoSDBa Hbf9+OAw1oF7ZVw5a3fvxDaT5VJpTZRAwLbxd9LdC6iO2gEw6wqW03eI+42T820jZAEy HA428x9+DJ0TJH96LD76xG6KzTLxhxWEhXfDtpCnDN4VRuJdMpI+jjrdGUzavQVxv02U xccHLgfCUudYhzSDbSaVN883sBo9ilTMzv/1T5rb1H4tr7EUhNR1mALIhpxMEk6zrrci PmvhA5yNUbn9nykVVyVCjLQcm7lnjfm7ctVHxEKjaae5Cw1B4bsEQd2YnBXTphcbgFnJ v2Gw== X-Gm-Message-State: AHQUAuYr9ptmKFlY0xU24qLfBgTFYDcMLaD7S8+PSSaRjfkkXlfU/jJK JVSZM9+VWFUFg4pLGn0peQv20A== X-Google-Smtp-Source: AHgI3IbqnUArrlCVDAXkkYXo/KBlHnWET+7BUaJ0blqY/pZTNN9GoBPxkS0LtMCUU3AcszWuR/IK6w== X-Received: by 2002:a1c:bc82:: with SMTP id m124mr3334522wmf.77.1549367223663; Tue, 05 Feb 2019 03:47:03 -0800 (PST) Received: from bivouac.eciton.net (bivouac.eciton.net. [2a00:1098:0:86:1000:23:0:2]) by smtp.gmail.com with ESMTPSA id h133sm9894569wmd.8.2019.02.05.03.47.02 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 05 Feb 2019 03:47:02 -0800 (PST) Date: Tue, 5 Feb 2019 11:47:01 +0000 From: Leif Lindholm To: Laszlo Ersek Cc: Philippe =?iso-8859-1?Q?Mathieu-Daud=E9?= , edk2-devel@lists.01.org, Liming Gao , Bob Feng , "Zhiju . Fan" , Yonghong Zhu , Ard Biesheuvel Message-ID: <20190205114701.rzhrwb75uecwswjl@bivouac.eciton.net> References: <20190205012356.31376-1-philmd@redhat.com> MIME-Version: 1.0 In-Reply-To: User-Agent: NeoMutt/20170113 (1.7.2) Subject: Re: [PATCH] BaseTools: Fix build failure when specifying multiple BUILDTARGET X-BeenThere: edk2-devel@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: EDK II Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Feb 2019 11:47:06 -0000 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit On Tue, Feb 05, 2019 at 10:03:21AM +0100, Laszlo Ersek wrote: > On 02/05/19 02:23, Philippe Mathieu-Daudé wrote: > > Since 9c2d68c0a299 the build tools default to the python version > > provided by the ${PYTHON} environment variable. > > However the Python3 transition is not effective before d943b0c339fe. > > (1) Do you mean "functional" rather than "effective"? > > (2) Why is this information relevant for this commit? I see that commit > f8d11e5a4aaa, referenced below, falls between the above two, but I'm > unsure if that has any special relevance. > > If the above paragraph is just background info, that's OK with me, of > course. > > > With Python3, the dict.value() method returns an iterator. > > If a dictionary is updated while an iterator on his keys is used, > > (3) s/his/its/ > > > a RuntimeError is generated. > > Converting the iterator to a list() forces a copy of the mutable > > keys in an immutable list which can be safely iterated. > > > > Commit f8d11e5a4aaa converted various uses but missed one: > > When specifying multiple BUILDTARGET, the first target builds > > successfully, but then the PGen.BuildDatabase._CACHE_ dictionary is > > updated, and the next target accessing it triggers a RuntimeError. > > (4) Can we clarify this please; I think it's not the "next target" that > accesses the dictionary, instead the code accesses the next target in > the dictionary. How about > > s/the next target accessing it/accessing the next target/ > > ? > > > > > Convert this iterator to an immutable list, to solve this build error: > > > > $ build -a IA32 -t GCC5 -b RELEASE -b NOOPT -p OvmfPkg/OvmfPkgIa32.dsc > > [...] > > Processing meta-data ... > > build.py... > > : error C0DE: Unknown fatal error when processing [OvmfPkg/OvmfPkgIa32.dsc] > > > > (Please send email to edk2-devel@lists.01.org for help, attaching following call stack trace!) > > > > (Python 3.5.3 on linux) Traceback (most recent call last): > > File "BaseTools/BinWrappers/PosixLike/../../Source/Python/build/build.py", line 2387, in Main > > MyBuild.Launch() > > File "BaseTools/BinWrappers/PosixLike/../../Source/Python/build/build.py", line 2141, in Launch > > self._MultiThreadBuildPlatform() > > File "BaseTools/BinWrappers/PosixLike/../../Source/Python/build/build.py", line 1921, in _MultiThreadBuildPlatform > > self.Progress > > File "BaseTools/Source/Python/AutoGen/AutoGen.py", line 304, in __init__ > > self._InitWorker(Workspace, MetaFile, Target, Toolchain, Arch, *args, **kwargs) > > File "BaseTools/Source/Python/AutoGen/AutoGen.py", line 477, in _InitWorker > > for BuildData in PGen.BuildDatabase._CACHE_.values(): > > RuntimeError: dictionary changed size during iteration > > > > Reported-by: Leif Lindholm > > Fixes: f8d11e5a4aaa90bf63b4789f3993dd6d16c60787 > > Contributed-under: TianoCore Contribution Agreement 1.1 > > Signed-off-by: Philippe Mathieu-Daude > > Tested-by: Leif Lindholm > > --- > > BaseTools/Source/Python/AutoGen/AutoGen.py | 2 +- > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > diff --git a/BaseTools/Source/Python/AutoGen/AutoGen.py b/BaseTools/Source/Python/AutoGen/AutoGen.py > > index a95d2c710e..12592a2a46 100644 > > --- a/BaseTools/Source/Python/AutoGen/AutoGen.py > > +++ b/BaseTools/Source/Python/AutoGen/AutoGen.py > > @@ -474,7 +474,7 @@ class WorkspaceAutoGen(AutoGen): > > > > # generate the SourcePcdDict and BinaryPcdDict > > PGen = PlatformAutoGen(self, self.MetaFile, Target, Toolchain, Arch) > > - for BuildData in PGen.BuildDatabase._CACHE_.values(): > > + for BuildData in list(PGen.BuildDatabase._CACHE_.values()): > > if BuildData.Arch != Arch: > > continue > > if BuildData.MetaFile.Ext == '.inf': > > > > LGTM :) > > With the commit message updated (as you prefer): > > Acked-by: Laszlo Ersek I would be surprised if we hear back from the BaseTools maintainers this week (Chinese New Year). For me, I have both a simple workaround and this patch, so I'm OK with waiting. But if anyone reports issues with CI environments or similar, I would say either you or I could push this patch in their absence. / Leif