From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=17.171.2.72; helo=ma1-aaemail-dr-lapp03.apple.com; envelope-from=afish@apple.com; receiver=edk2-devel@lists.01.org Received: from ma1-aaemail-dr-lapp03.apple.com (ma1-aaemail-dr-lapp03.apple.com [17.171.2.72]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id E2D2021173C74 for ; Sat, 13 Oct 2018 08:12:40 -0700 (PDT) Received: from pps.filterd (ma1-aaemail-dr-lapp03.apple.com [127.0.0.1]) by ma1-aaemail-dr-lapp03.apple.com (8.16.0.22/8.16.0.22) with SMTP id w9DFCKKP058903; Sat, 13 Oct 2018 08:12:37 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=apple.com; h=mime-version : content-transfer-encoding : content-type : sender : subject : from : in-reply-to : date : cc : message-id : references : to; s=20180706; bh=5wbhtz/HScWV8Oy42cZTs9Ji4x35jgRXdEPBi43yD0M=; b=SnygynX+1QUMOBqzNWA9SWsKxqI+xbZxDjALdShvU37nw3BCdOiyO7vutUr7aDo8BTG+ 6A0ramcWvc1HWRp/HzZZS1g/xF0goSPth0lWAdzOBUbJLiXKs0gX4c3577j1Cdag2gR4 +oSFrVcbxsUt2x/U6WR7P2IbILADcz704rghU31W8pEsf+3cCQicP0Art6/p1Bn4T7SC yBuP0b1oHd0J7J2EVCKEXPNyVmMjwY8DauBUvchhCqnDaGjmLrDo9DLnO8FCg/lWaeem mPzJSiaIU38FGmgBLN4a4CY0azqDBkL7I4qEsFoXuWF3HAy8NoHkxht7uyzf+SVJue8l xg== Received: from mr2-mtap-s02.rno.apple.com (mr2-mtap-s02.rno.apple.com [17.179.226.134]) by ma1-aaemail-dr-lapp03.apple.com with ESMTP id 2n3fpwckm7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Sat, 13 Oct 2018 08:12:37 -0700 MIME-version: 1.0 Received: from ma1-mmpp-sz11.apple.com (ma1-mmpp-sz11.apple.com [17.171.128.33]) by mr2-mtap-s02.rno.apple.com (Oracle Communications Messaging Server 8.0.2.3.20180614 64bit (built Jun 14 2018)) with ESMTPS id <0PGJ00DBNM90ZK70@mr2-mtap-s02.rno.apple.com>; Sat, 13 Oct 2018 08:12:36 -0700 (PDT) Received: from process_viserion-daemon.ma1-mmpp-sz11.apple.com by ma1-mmpp-sz11.apple.com (Oracle Communications Messaging Server 8.0.2.3.20180614 64bit (built Jun 14 2018)) id <0PGJ00F00LOXFZ00@ma1-mmpp-sz11.apple.com>; Sat, 13 Oct 2018 08:12:36 -0700 (PDT) X-Va-A: X-Va-T-CD: 6a8fe94fde202cff83873853a5e2c49c X-Va-E-CD: c67f9a47b66919c2dbbcf320102fb561 X-Va-R-CD: 9924314482fe6662191ab30225ba009d X-Va-CD: 0 X-Va-ID: 85b9644a-58ec-4924-bd7a-fcf6603eed9c X-V-A: X-V-T-CD: 9b3859669d90162ac484de95f7f3dbd2 X-V-E-CD: c67f9a47b66919c2dbbcf320102fb561 X-V-R-CD: 9924314482fe6662191ab30225ba009d X-V-CD: 0 X-V-ID: 973c4df1-2277-4949-af8a-3e341cac1442 Received: from process_milters-daemon.ma1-mmpp-sz11.apple.com by ma1-mmpp-sz11.apple.com (Oracle Communications Messaging Server 8.0.2.3.20180614 64bit (built Jun 14 2018)) id <0PGJ00F00LONFS00@ma1-mmpp-sz11.apple.com>; Sat, 13 Oct 2018 08:12:35 -0700 (PDT) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2018-10-13_10:,, signatures=0 X-Proofpoint-Scanner-Instance: ma-grpmailp-qapp23.corp.apple.com-10000_instance1 Received: from [17.234.241.37] (unknown [17.234.241.37]) by ma1-mmpp-sz11.apple.com (Oracle Communications Messaging Server 8.0.2.3.20180614 64bit (built Jun 14 2018)) with ESMTPSA id <0PGJ001WRM8W7Y20@ma1-mmpp-sz11.apple.com>; Sat, 13 Oct 2018 08:12:35 -0700 (PDT) Sender: afish@apple.com From: Andrew Fish In-reply-to: Date: Sat, 13 Oct 2018 08:12:16 -0700 Cc: "Gao, Liming" , edk2-devel-01 , Mike Kinney , Laszlo Ersek Message-id: <0A85438F-73CF-4ACC-9710-6157A27B18F5@apple.com> References: <4A89E2EF3DFEDB4C8BFDE51014F606A14E339880@SHSMSX104.ccr.corp.intel.com> <4A89E2EF3DFEDB4C8BFDE51014F606A14E339900@SHSMSX104.ccr.corp.intel.com> To: Ard Biesheuvel X-Mailer: Apple Mail (2.3445.6.18) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2018-10-13_10:, , signatures=0 Subject: Re: BaseTools/ build broken due to unreviewed patch - PLEASE FIX X-BeenThere: edk2-devel@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: EDK II Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Oct 2018 15:12:41 -0000 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII > On Oct 13, 2018, at 5:18 AM, Ard Biesheuvel wrote: > > On 13 October 2018 at 11:23, Gao, Liming wrote: >> Ard: >> I know the failure reason. . edksetup.sh must be called in edk2 before enter BaseTools with make. edksetup.sh is used to detect the python3 and set PYTHON3 env. This env is used in GNUmakefile. >> >> I don't aware this change before. Because we usually call edksetup.sh, then make BaseTools. We will try to find the way to handle it. >> > > Thanks for diagnosing that. As Leif points out, this deviates from the > instructions on the wiki. It also deviates from the build script I use > in my CI environment, hence the breakage. > > Unfortunately, our CI environment is based on Debian Stretch, which > provides Python 3.5.3 only not Python 3.6 or later. I will work with > our infrastructure team to find out if we can support Python 3.6 as > well, but it will probably take time. > > What is the reason we need 3.6 or later? > All the macOS based builds are broken too. Python 2.7.* comes with macOS. The instructions are broken for macOS as you never had to install Python before. I'm not able to install an alternate version of Python in our internal CI infrastructure (You build the OS with tools from the OS). This will likely force me to have to check in a version of Python 3.6 into the edk2 tree to keep the build working, or fork off an older version of the build system that works with Python 2.7.*. What problem are we solving by forcing everyone to be on Python 3.6 or later? I understand on Widows you have to install Python, but a lot of the Unix installs you can get an older version of Python installed for free. Thanks, Andrew Fish > >>> -----Original Message----- >>> From: Ard Biesheuvel [mailto:ard.biesheuvel@linaro.org] >>> Sent: Saturday, October 13, 2018 4:46 PM >>> To: Gao, Liming >>> Cc: edk2-devel-01 ; Laszlo Ersek ; Leif Lindholm ; Kinney, >>> Michael D ; Zhu, Yonghong ; stephano.cetola@linux.intel.com >>> Subject: Re: [edk2] BaseTools/ build broken due to unreviewed patch - PLEASE FIX >>> >>> On 13 October 2018 at 10:44, Gao, Liming wrote: >>>> Ard: >>>> I pull the latest edk2 code. I can find RunTests.py in BaseTools/Tests. The below error message shows RunTests.py is not found. But, >>> this file is in BaseTools/Tests. I also run Make command in BaseTools directory. It can work in my Ubuntu machine. I install Python37. Do >>> you install Python36 or Python37 in your machine? >>>> >>>> Yonghong has sent the mail to notify BaseTools Python3 migration is done. Python3.6 or the above is required. >>>> >>> >>> I have python3 on my machine, yes. But that does not mean it is the >>> default for everyone. >>> >>> >>>>> -----Original Message----- >>>>> From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of Ard Biesheuvel >>>>> Sent: Saturday, October 13, 2018 4:18 PM >>>>> To: edk2-devel-01 ; Gao, Liming ; Laszlo Ersek ; Leif >>> Lindholm >>>>> ; Kinney, Michael D ; Zhu, Yonghong ; >>>>> stephano.cetola@linux.intel.com >>>>> Subject: [edk2] BaseTools/ build broken due to unreviewed patch - PLEASE FIX >>>>> >>>>> Commit ea787b6b55499e8f931201f8f49daaaeb2e4409f >>>>> >>>>> "BaseTools: update Test scripts support python3" >>>>> >>>>> [which has never been sent out for review to the mailing list, as far >>>>> as I can tell] is breaking the BaseTools build >>>>> >>>>> make -C Tests >>>>> make[1]: Entering directory '/home/ard/build/edk2/BaseTools/Tests' >>>>> /bin/sh: 1: RunTests.py: not found >>>>> GNUmakefile:17: recipe for target 'test' failed >>>>> make[1]: *** [test] Error 127 >>>>> make[1]: Leaving directory '/home/ard/build/edk2/BaseTools/Tests' >>>>> GNUmakefile:25: recipe for target 'Tests' failed >>>>> make: *** [Tests] Error 2 >>>>> make: Leaving directory '/home/ard/build/edk2/BaseTools' >>>>> >>>>> I tried reverting that patch alone, but that gives me >>>>> >>>>> make -C Tests >>>>> make[1]: Entering directory '/home/ard/build/edk2/BaseTools/Tests' >>>>> Traceback (most recent call last): >>>>> File "RunTests.py", line 36, in >>>>> allTests = GetAllTestsSuite() >>>>> File "RunTests.py", line 33, in GetAllTestsSuite >>>>> return unittest.TestSuite([GetCTestSuite(), GetPythonTestSuite()]) >>>>> File "RunTests.py", line 30, in GetPythonTestSuite >>>>> return PythonToolsTests.TheTestSuite() >>>>> File "/home/ard/build/edk2/BaseTools/Tests/PythonToolsTests.py", >>>>> line 27, in TheTestSuite >>>>> import CheckUnicodeSourceFiles >>>>> File "/home/ard/build/edk2/BaseTools/Tests/CheckUnicodeSourceFiles.py", >>>>> line 25, in >>>>> from Common.Misc import PathClass >>>>> File "/home/ard/build/edk2/BaseTools/Source/Python/Common/Misc.py", >>>>> line 27, in >>>>> from collections import UserDict as IterableUserDict >>>>> ImportError: cannot import name UserDict >>>>> GNUmakefile:17: recipe for target 'test' failed >>>>> make[1]: *** [test] Error 1 >>>>> make[1]: Leaving directory '/home/ard/build/edk2/BaseTools/Tests' >>>>> GNUmakefile:25: recipe for target 'Tests' failed >>>>> make: *** [Tests] Error 2 >>>>> make: Leaving directory '/home/ard/build/edk2/BaseTools' >>>>> >>>>> Breaking the build for everyone with an unreviewed patch 2 days before >>>>> the plugfest is NOT OK. >>>>> >>>>> Please fix or revert ASAP >>>>> >>>>> (I suppose that the failing test is a false negative in this case, but >>>>> that is not really the point - my CI jobs are broken due to this and >>>>> so are many others, I imagine) >>>>> _______________________________________________ >>>>> edk2-devel mailing list >>>>> edk2-devel@lists.01.org >>>>> https://lists.01.org/mailman/listinfo/edk2-devel > _______________________________________________ > edk2-devel mailing list > edk2-devel@lists.01.org > https://lists.01.org/mailman/listinfo/edk2-devel