Re: BaseTools: build failure when specifying multiple BUILDTARGET

[Leif Lindholm <leif.lindholm@linaro.org>]

On Mon, Feb 04, 2019 at 01:22:03PM +0100, Philippe Mathieu-Daudé wrote:
> On 2/4/19 12:54 PM, Philippe Mathieu-Daudé wrote:
> > On 2/4/19 10:58 AM, Leif Lindholm wrote:
> >> Hi Bob, Liming,
> >>
> >> With the latest BaseTools (current HEAD, 6c61ec4c62), building
> >> multiple targets from a single command line crashes.
> >>
> >> To reproduce:
> >> build -a IA32 -t GCC5 -b RELEASE -b NOOPT -p OvmfPkg/OvmfPkgIa32.dsc
> >> (I first built with -n32, but dropped that to see if it would make a
> >> difference - it does not.)
> >>
> >> The first target specified builds successfully. When starting on the
> >> second, the output is as below, and build exits.
> >>
> >> /
> >>     Leif
> >>
> >> Architecture(s)  = IA32
> >> Build target     = NOOPT
> >> Toolchain        = GCC5
> >>
> >> Active Platform          = /work/git/edk2/OvmfPkg/OvmfPkgIa32.dsc
> >> Flash Image Definition   = /work/git/edk2/OvmfPkg/OvmfPkgIa32.fdf
> >>
> >> Processing meta-data ...
> >>
> >>
> >> build.py...
> >>  : error C0DE: Unknown fatal error when processing [/work/git/edk2/OvmfPkg/OvmfPkgIa32.dsc]
> >>
> >> (Please send email to edk2-devel@lists.01.org for help, attaching following call stack trace!)
> >>
> >> (Python 3.5.3 on linux) Traceback (most recent call last):
> >>   File "/work/git/edk2/BaseTools/BinWrappers/PosixLike/../../Source/Python/build/build.py", line 2387, in Main
> >>     MyBuild.Launch()
> >>   File "/work/git/edk2/BaseTools/BinWrappers/PosixLike/../../Source/Python/build/build.py", line 2141, in Launch
> >>     self._MultiThreadBuildPlatform()
> >>   File "/work/git/edk2/BaseTools/BinWrappers/PosixLike/../../Source/Python/build/build.py", line 1921, in _MultiThreadBuildPlatform
> >>     self.Progress
> >>   File "/work/git/edk2/BaseTools/Source/Python/AutoGen/AutoGen.py", line 304, in __init__
> >>     self._InitWorker(Workspace, MetaFile, Target, Toolchain, Arch, *args, **kwargs)
> >>   File "/work/git/edk2/BaseTools/Source/Python/AutoGen/AutoGen.py", line 477, in _InitWorker
> >>     for BuildData in PGen.BuildDatabase._CACHE_.values():
> >> RuntimeError: dictionary changed size during iteration
> > 
> > I believe the culprit is f8d11e5a4aaa.
> 
> With Python3 the dict.value() method returns an iterator.
> Using list() to force a copy works for me.
> 
> Since I'm not a Python expert (and less with joys of py2/py3
> conversion), I'll start sharing a snippet rather than a formal patch :)

I can confirm this change *appears* to fix the problem with both
python3.5 and python2.7. So from my end:
Tested-by: Leif Lindholm <leif.lindholm@linaro.org>

Thanks!

> 
> -- >8 --
> --- a/BaseTools/Source/Python/AutoGen/AutoGen.py
> +++ b/BaseTools/Source/Python/AutoGen/AutoGen.py
> @@ -475,5 +475,5 @@ class WorkspaceAutoGen(AutoGen):
>              # generate the SourcePcdDict and BinaryPcdDict
>              PGen = PlatformAutoGen(self, self.MetaFile, Target,
> Toolchain, Arch)
> -            for BuildData in PGen.BuildDatabase._CACHE_.values():
> +            for BuildData in list(PGen.BuildDatabase._CACHE_.values()):
>                  if BuildData.Arch != Arch:
>                      continue
> ---
>