Re: BaseTools/ build broken due to unreviewed patch - PLEASE FIX

[Ard Biesheuvel <ard.biesheuvel@linaro.org>]

On 13 October 2018 at 11:23, Gao, Liming <liming.gao@intel.com> wrote:
> Ard:
>   I know the failure reason. . edksetup.sh must be called in edk2 before enter BaseTools with make. edksetup.sh is used to detect the python3 and set PYTHON3 env. This env is used in GNUmakefile.
>
>   I don't aware this change before. Because we usually call edksetup.sh, then make BaseTools. We will try to find the way to handle it.
>

Thanks for diagnosing that. As Leif points out, this deviates from the
instructions on the wiki. It also deviates from the build script I use
in my CI environment, hence the breakage.

Unfortunately, our CI environment is based on Debian Stretch, which
provides Python 3.5.3 only not Python 3.6 or later. I will work with
our infrastructure team to find out if we can support Python 3.6 as
well, but it will probably take time.

What is the reason we need 3.6 or later?


>> -----Original Message-----
>> From: Ard Biesheuvel [mailto:ard.biesheuvel@linaro.org]
>> Sent: Saturday, October 13, 2018 4:46 PM
>> To: Gao, Liming <liming.gao@intel.com>
>> Cc: edk2-devel-01 <edk2-devel@lists.01.org>; Laszlo Ersek <lersek@redhat.com>; Leif Lindholm <leif.lindholm@linaro.org>; Kinney,
>> Michael D <michael.d.kinney@intel.com>; Zhu, Yonghong <yonghong.zhu@intel.com>; stephano.cetola@linux.intel.com
>> Subject: Re: [edk2] BaseTools/ build broken due to unreviewed patch - PLEASE FIX
>>
>> On 13 October 2018 at 10:44, Gao, Liming <liming.gao@intel.com> wrote:
>> > Ard:
>> >   I pull the latest edk2 code. I can find RunTests.py in BaseTools/Tests. The below error message shows RunTests.py is not found. But,
>> this file is in BaseTools/Tests. I also run Make command in BaseTools directory. It can work in my Ubuntu machine. I install Python37. Do
>> you install Python36 or Python37 in your machine?
>> >
>> >   Yonghong has sent the mail to notify BaseTools Python3 migration is done. Python3.6 or the above is required.
>> >
>>
>> I have python3 on my machine, yes. But that does not mean it is the
>> default for everyone.
>>
>>
>> >> -----Original Message-----
>> >> From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of Ard Biesheuvel
>> >> Sent: Saturday, October 13, 2018 4:18 PM
>> >> To: edk2-devel-01 <edk2-devel@lists.01.org>; Gao, Liming <liming.gao@intel.com>; Laszlo Ersek <lersek@redhat.com>; Leif
>> Lindholm
>> >> <leif.lindholm@linaro.org>; Kinney, Michael D <michael.d.kinney@intel.com>; Zhu, Yonghong <yonghong.zhu@intel.com>;
>> >> stephano.cetola@linux.intel.com
>> >> Subject: [edk2] BaseTools/ build broken due to unreviewed patch - PLEASE FIX
>> >>
>> >> Commit ea787b6b55499e8f931201f8f49daaaeb2e4409f
>> >>
>> >>   "BaseTools: update Test scripts support python3"
>> >>
>> >> [which has never been sent out for review to the mailing list, as far
>> >> as I can tell] is breaking the BaseTools build
>> >>
>> >> make -C Tests
>> >> make[1]: Entering directory '/home/ard/build/edk2/BaseTools/Tests'
>> >> /bin/sh: 1: RunTests.py: not found
>> >> GNUmakefile:17: recipe for target 'test' failed
>> >> make[1]: *** [test] Error 127
>> >> make[1]: Leaving directory '/home/ard/build/edk2/BaseTools/Tests'
>> >> GNUmakefile:25: recipe for target 'Tests' failed
>> >> make: *** [Tests] Error 2
>> >> make: Leaving directory '/home/ard/build/edk2/BaseTools'
>> >>
>> >> I tried reverting that patch alone, but that gives me
>> >>
>> >> make -C Tests
>> >> make[1]: Entering directory '/home/ard/build/edk2/BaseTools/Tests'
>> >> Traceback (most recent call last):
>> >>   File "RunTests.py", line 36, in <module>
>> >>     allTests = GetAllTestsSuite()
>> >>   File "RunTests.py", line 33, in GetAllTestsSuite
>> >>     return unittest.TestSuite([GetCTestSuite(), GetPythonTestSuite()])
>> >>   File "RunTests.py", line 30, in GetPythonTestSuite
>> >>     return PythonToolsTests.TheTestSuite()
>> >>   File "/home/ard/build/edk2/BaseTools/Tests/PythonToolsTests.py",
>> >> line 27, in TheTestSuite
>> >>     import CheckUnicodeSourceFiles
>> >>   File "/home/ard/build/edk2/BaseTools/Tests/CheckUnicodeSourceFiles.py",
>> >> line 25, in <module>
>> >>     from Common.Misc import PathClass
>> >>   File "/home/ard/build/edk2/BaseTools/Source/Python/Common/Misc.py",
>> >> line 27, in <module>
>> >>     from collections import UserDict as IterableUserDict
>> >> ImportError: cannot import name UserDict
>> >> GNUmakefile:17: recipe for target 'test' failed
>> >> make[1]: *** [test] Error 1
>> >> make[1]: Leaving directory '/home/ard/build/edk2/BaseTools/Tests'
>> >> GNUmakefile:25: recipe for target 'Tests' failed
>> >> make: *** [Tests] Error 2
>> >> make: Leaving directory '/home/ard/build/edk2/BaseTools'
>> >>
>> >> Breaking the build for everyone with an unreviewed patch 2 days before
>> >> the plugfest is NOT OK.
>> >>
>> >> Please fix or revert ASAP
>> >>
>> >> (I suppose that the failing test is a false negative in this case, but
>> >> that is not really the point - my CI jobs are broken due to this and
>> >> so are many others, I imagine)
>> >> _______________________________________________
>> >> edk2-devel mailing list
>> >> edk2-devel@lists.01.org
>> >> https://lists.01.org/mailman/listinfo/edk2-devel