From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from NAM03-DM3-obe.outbound.protection.outlook.com (NAM03-DM3-obe.outbound.protection.outlook.com [40.107.80.110]) by mx.groups.io with SMTP id smtpd.web11.560.1575395519068496579 for ; Tue, 03 Dec 2019 09:51:59 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@microsoft.com header.s=selector2 header.b=bIPXd5HV; spf=pass (domain: microsoft.com, ip: 40.107.80.110, mailfrom: jerecox@microsoft.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=NwQWsGXevNq08xRXcoTJWYSKVzGDw9fl1MQwf1c5fodUlFD8ZIc9Uv67D1VYgv2agAX5ucNhRdpCz8fksa5Ah9JJ90NKJNonIDcR+MDaA6JvSVj5Xm/8/oQHZJdXg2qHRZeQ3NXcNrtNtPunf0dXU0UgTXUUce+aVJfgbzt+f9hlS1B4oDEWOY3HiV8hL6MuishDHGVOzxbeoqroSMvtFBa6osIfUNHnlDl3ojrrDRWA3HoB+NwmWWl08rR69beu4yWQvxswdBe++5fsuvGpZZrPYsjS1ebe3YZ+leH9dXTdGvQ+gV9mpzf6X9gsbqgyfDva23860jk+CjLJcFWgPg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+8p+X4ZKKxDwiFjasnQh1LDIbX8le9gvZRYdbY9cAF0=; b=Ub34bvmdm0IHlh5MvLiUR6WH93mP6EE8HtHJStbvAfYdCZmxs8oSU+Z4CAarOj+ESW0qfBImr4T43hl82wxlDWKAXLf8uizLe6ftYj11ks0hx7PahRkay9E31akvlpD+0mNMIeyRiARt0X6oM7LpaPdlus3lQO2OYaQpPC30eWVt0ZKnAkpJrEmQZwVKy0PazhjzNqz8hOMHNu5dAWDDU9yWIGWZnQtakl0qbDTDaUDdDZOIhhge9nYrTphfpCjVZB/xyT7l3bjXxoHAwjlb0M5b6U6XSe6jwGTF5zVJ9dSv3yt6jXbKuEY93zBvagLltoFlVxBgCninTTsYrCVujA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=microsoft.com; dmarc=pass action=none header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+8p+X4ZKKxDwiFjasnQh1LDIbX8le9gvZRYdbY9cAF0=; b=bIPXd5HVnouNFG2wnne8/pPoTDHyIERUdf+qTIr3hUg2WtGG4bGqa8kcmPK30wUAuX9aeXa+HuIhYltunLD916TS+/miXbxKx6uAnICrw5qy2lSwv+geRSRbzCHor5BsYaCP8UL3LVUOLMwxLZea66WShRgW1Yq0pL91Dvswp24= Received: from BYAPR21MB1334.namprd21.prod.outlook.com (20.179.60.208) by BYAPR21MB1336.namprd21.prod.outlook.com (20.179.60.210) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2538.3; Tue, 3 Dec 2019 17:51:56 +0000 Received: from BYAPR21MB1334.namprd21.prod.outlook.com ([fe80::8c9e:1d39:6bc2:b538]) by BYAPR21MB1334.namprd21.prod.outlook.com ([fe80::8c9e:1d39:6bc2:b538%3]) with mapi id 15.20.2538.000; Tue, 3 Dec 2019 17:51:56 +0000 From: "Jeremiah Cox" To: Matthew Carlson , "Feng, Bob C" , "devel@edk2.groups.io" , "lersek@redhat.com" , Sean Brogan , Bret Barkelew , Matthew Carlson Subject: Re: [EXTERNAL] RE: [edk2-devel] Seeking feedback: example of building OvmfPkg with Pytools Thread-Topic: [EXTERNAL] RE: [edk2-devel] Seeking feedback: example of building OvmfPkg with Pytools Thread-Index: AQHVpl/KFBBiYbPDAEm2gEDSyJUJ2aenl76SgAAFKI2AAHITgIAAmZAxgAAJv4I= Date: Tue, 3 Dec 2019 17:51:56 +0000 Message-ID: References: ,<08650203BA1BD64D8AD9B6D5D74A85D16156DED1@SHSMSX104.ccr.corp.intel.com>, ,<08650203BA1BD64D8AD9B6D5D74A85D16157A27C@SHSMSX104.ccr.corp.intel.com>, In-Reply-To: Accept-Language: en-US X-Mentions: macarl@microsoft.com X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Enabled=True;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SiteId=72f988bf-86f1-41af-91ab-2d7cd011db47;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SetDate=2019-12-03T16:56:44.2912792Z;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ContentBits=0;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Method=Privileged authentication-results: spf=none (sender IP is ) smtp.mailfrom=jerecox@microsoft.com; x-originating-ip: [172.58.45.43] x-ms-publictraffictype: Email x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: 43fbef79-ceb7-4ed1-5b22-08d778197d3e x-ms-traffictypediagnostic: BYAPR21MB1336:|BYAPR21MB1336: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:7691; x-forefront-prvs: 02408926C4 x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(39860400002)(376002)(346002)(136003)(366004)(396003)(199004)(189003)(174874002)(13464003)(9686003)(229853002)(236005)(54896002)(8676002)(186003)(5660300002)(52536014)(22452003)(316002)(7736002)(66556008)(26005)(2501003)(91956017)(76116006)(14444005)(110136005)(66946007)(99286004)(6436002)(66446008)(64756008)(66476007)(10090500001)(33656002)(8990500004)(19627235002)(11346002)(76176011)(74316002)(478600001)(6506007)(6116002)(256004)(8936002)(3846002)(6636002)(53546011)(6246003)(606006)(966005)(81156014)(86362001)(81166006)(55016002)(2201001)(6306002)(25786009)(2906002)(71200400001)(14454004)(10290500003)(7696005)(102836004)(71190400001)(446003)(30864003)(579004)(460985005)(2480315003);DIR:OUT;SFP:1102;SCL:1;SRVR:BYAPR21MB1336;H:BYAPR21MB1334.namprd21.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; received-spf: None (protection.outlook.com: microsoft.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: pj6GcDbQN3F9HhHQ9sBFrDdSAPjV4QCyxrI/92YGB6fDSfg5Sa54oioYBaBAPLD68DiM4Jb/ND8AaI1oWqrpNeXEcIw6do/HgX2RKq6kYl1tvcRfUOb1JBv/1mhY1V4QLTqpZXUgysrfcUv9w10N8Z8hT3qC/97yGaI4bLr8IHWnMLCNBDvhKo9wA01BVwDOLZVNQsnIh3qnv0EuIY2Rke5mNeDfMqUBE6LXEdII/mgwAR+0Wgpj8Qc+wO58AO1V+I86e4+fL4C8BrmyS9rZzSTIOJZMGEYXYD7QthuXLQF4LTPswOo3bWOL4KxuVYbaoSQffbvQG/cxL7p/TYxwvBaWhgfcK50kpAEDrpfniXf/4rCDqUXSpUTBigWoAQymDa5PIgTYsTAO2VrNmG/sWpD0Om6SVcNKEcjP2m5dp5WoWAFaRpHjMoefv7eRwGhyNYMPBrppxiRZu55TdI7eYRNzMouLB/ptOJuDn3XIP2Q= MIME-Version: 1.0 X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-Network-Message-Id: 43fbef79-ceb7-4ed1-5b22-08d778197d3e X-MS-Exchange-CrossTenant-originalarrivaltime: 03 Dec 2019 17:51:56.5146 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 26h8V3zViQHN+AmdoWeEzzYmofudPH0aHWKcDjp1A/xKY67l/G6bqE/Tiyjb3qHMhqE+93PIggVBJHWq7I/i/ubJhZASzFGcEZDnrCxsd84= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR21MB1336 Content-Language: en-US Content-Type: multipart/alternative; boundary="_000_BYAPR21MB13343FC1C92B4805CFD8F395AD420BYAPR21MB1334namp_" --_000_BYAPR21MB13343FC1C92B4805CFD8F395AD420BYAPR21MB1334namp_ Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable Hi Bob, The 5.2 failure is expected as we do not support VS2015 yet. If suggest a= dding a feature request here : https://github.com/tianocore/= edk2-pytool-extensions/issues Regarding 2. you may need to provide a BLD_ prefix to pass build parameter= s to the platform builder, refer to my README for an example. In Matt's example for 3, processor family is not x86, but rather a group o= f products that share common architectural components, for example IceLake = generation products might share many components. This is somewhat similar t= o KabylakeOpenBoardPackage. @Matthew Carlson, perhaps you could look at t= he second log in 5.2 above, perhaps we are missing a dependency in our requ= irements.txt? System.Net.Http? ________________________________ From: Matthew Carlson Sent: Tuesday, December 3, 2019 9:04:08 AM To: Feng, Bob C ; Jeremiah Cox ; devel@edk2.groups.io ; lersek@redhat.com ; Sean Brogan ; Bret Barkelew ; Matthew Carlson Subject: RE: [EXTERNAL] RE: [edk2-devel] Seeking feedback: example of buil= ding OvmfPkg with Pytools Hello Bob, 1. Right. We actually have three documents forthcoming on how to move p= latforms onto edk2 pytools, one on OVMF, one on Raspberry Pi, and another o= n KabylakeOpenBoardPackage. Jeremiah sent some links to his platform. We=92= ll have more example platforms and tutorial in the future 2. Yes. All the remaining arguments get passed into the UefiBuilder its= elf, which parses it and hands it to build.py 3. Where ever makes sense. Since it=92s a python file, it can be very f= lexible. For our platforms using pytools, it generally goes PROCESSOR FAMIILY | -- ProcessorFamilyPkg | -- ProductNamePkg | | -- PlatformBuild.py 4. Glad to help ?? - Matthew Carlson From: Feng, Bob C Sent: Monday, December 2, 2019 11:47 PM To: Matthew Carlson; Jeremiah Cox; devel@edk2.groups.io; lers= ek@redhat.com; Sean Brogan; Bret Barkelew Subject: RE: [EXTERNAL] RE: [edk2-devel] Seeking feedback: example of buil= ding OvmfPkg with Pytools Hi Matthew and Jeremiah, Thanks for your answers. 1. Cool. So platformBuild.py is intent to handle the additional pre-bui= ld and post-build tasks. I think it would be great if there is a document r= ecord how to create a PlatformBuild.py. 2. So the command-line arguments for build.py can also work for Platfor= mBuild.py. right? 3. Where would FamilyBuild.py be located? 4. Cool. Thanks. For 5.2 the error message in BUILD.txt is: PROGRESS - Running Pre Build INFO - Cmd to run is: e:\pythonvenv1\lib\site-packages\edk2toollib\bin\vsw= here.exe -latest -nologo -all -property installationPath -products * INFO - ------------------------------------------------ INFO - --------------Cmd Output Starting--------------- INFO - ------------------------------------------------ INFO - C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional INFO - ------------------------------------------------ INFO - --------------Cmd Output Finished--------------- INFO - --------- Running Time (mm:ss): 00:00 ---------- INFO - ------------------------------------------------ DEBUG - Calling 'C:\Program Files (x86)\Microsoft Visual Studio\2017\Profe= ssional\VC\Auxiliary\Build\vcvarsall.bat amd64' DEBUG - Getting host info for host: uname_result(system=3D'Windows', node= =3D'sh1gapp1005', release=3D'10', version=3D'10.0.14393', machine=3D'AMD64= ', processor=3D'Intel64 Family 6 Model 85 Stepping 4, GenuineIntel') DEBUG - Getting host info for host: uname_result(system=3D'Windows', node= =3D'sh1gapp1005', release=3D'10', version=3D'10.0.14393', machine=3D'AMD64= ', processor=3D'Intel64 Family 6 Model 85 Stepping 4, GenuineIntel') DEBUG - Getting host info for host: uname_result(system=3D'Windows', node= =3D'sh1gapp1005', release=3D'10', version=3D'10.0.14393', machine=3D'AMD64= ', processor=3D'Intel64 Family 6 Model 85 Stepping 4, GenuineIntel') DEBUG - Plugin Success: Windows RC Path Support DEBUG - Plugin Success: Windows Visual Studio Tool Chain Support INFO - Writing BuildToolsReports to E:\BobFeng\edk2\Build\Ovmf3264\DEBUG_V= S2015x86\BUILD_TOOLS_REPORT DEBUG - Plugin Success: Build Tools Report Generator PROGRESS - Running Build DEBUG DEBUG - Getting all build keys for build type DEBUG INFO - Cmd to run is: build -p OvmfPkg/OvmfPkgIa32X64.dsc -b DEBUG -t VS20= 15x86 -a IA32 -a X64 INFO - ------------------------------------------------ INFO - --------------Cmd Output Starting--------------- INFO - ------------------------------------------------ INFO - Build environment: Windows-10-10.0.14393-SP0 INFO - Build start time: 14:32:35, Dec.03 2019 INFO - INFO - WORKSPACE =3D e:\bobfeng\edk2 INFO - EDK_TOOLS_PATH =3D e:\bobfeng\edk2\basetools INFO - CONF_PATH =3D E:\BobFeng\edk2\Conf INFO - PYTHON_COMMAND =3D e:\pythonvenv1\scripts\python.exe INFO - INFO - INFO - Processing meta-data . INFO - Architecture(s) =3D IA32 X64 INFO - Build target =3D DEBUG INFO - Toolchain =3D VS2015x86 INFO - INFO - Active Platform =3D e:\bobfeng\edk2\OvmfPkg\OvmfPkgIa32X64= .dsc INFO - ........ done! INFO - Building ... e:\bobfeng\edk2\MdePkg\Library\UefiLib\UefiLib.inf [X6= 4] INFO - The system cannot find the path specified. INFO - INFO - INFO - build.py... INFO - : error 7000: Failed to execute command INFO - Vc\bin\nmake.exe /nologo tbuild [e:\bobfeng\edk2\Build\Ovmf3264\D= EBUG_VS2015x86\X64\MdePkg\Library\UefiLib\UefiLib] INFO - INFO - INFO - build.py... INFO - : error F002: Failed to build module INFO - e:\bobfeng\edk2\MdePkg\Library\UefiLib\UefiLib.inf [X64, VS2015x8= 6, DEBUG] INFO - INFO - - Failed - INFO - Build end time: 14:32:46, Dec.03 2019 INFO - Build total time: 00:00:11 INFO - INFO - ------------------------------------------------ INFO - --------------Cmd Output Finished--------------- INFO - --------- Running Time (mm:ss): 00:11 ---------- INFO - ------------------------------------------------ ERROR - Compiler #7000 from : Failed to execute command ERROR - EDK2 #002 from : Failed to build module CRITICAL - Build failed PROGRESS - End time: 2019-12-03 14:32:47.018063 Total time Elaps= ed: 0:00:15 SECTION - Log file is located at: E:\BobFeng\edk2\Build\BUILDLOG_OvmfPkg.t= xt SECTION - Summary PROGRESS - Error For 5.3, the error message in UPDATE.txt is: INFO - Cmd to run is: mono /home/bobfeng/.local/lib/python3.6/site-package= s/edk2toolext/bin/NuGet.exe locals global-packages -list INFO - ------------------------------------------------ INFO - --------------Cmd Output Starting--------------- INFO - ------------------------------------------------ INFO - Could not load file or assembly or one of its dependencies. INFO - Could not load file or assembly or one of its dependencies. INFO - Could not load file or assembly or one of its dependencies. INFO - Could not load file or assembly or one of its dependencies. INFO - Could not load file or assembly or one of its dependencies. INFO - System.AggregateException: One or more errors occurred. ---> System= .IO.FileNotFoundException: Could not load file or assembly or one of its de= pendencies. INFO - --- End of inner exception stack trace --- INFO - at NuGet.CommandLine.Program.Initialize (NuGet.IFileSystem fileSy= stem, NuGet.CommandLine.IConsole console) [0x000ce] in :0 INFO - at NuGet.CommandLine.Program.MainCore (System.String workingDirec= tory, System.String[] args) [0x000d8] in = :0 INFO - ---> (Inner Exception #0) System.IO.FileNotFoundException: Could no= t load file or assembly or one of its dependencies. INFO - File name: 'System.Net.Http, Version=3D4.0.0.0, Culture=3Dneutral, = PublicKeyToken=3Db03f5f7f11d50a3a'<--- INFO - INFO - ---> (Inner Exception #1) System.IO.FileNotFoundException: Could no= t load file or assembly or one of its dependencies. INFO - File name: 'System.Net.Http, Version=3D4.0.0.0, Culture=3Dneutral, = PublicKeyToken=3Db03f5f7f11d50a3a'<--- INFO - INFO - ---> (Inner Exception #2) System.IO.FileNotFoundException: Could no= t load file or assembly or one of its dependencies. INFO - File name: 'System.Net.Http, Version=3D4.0.0.0, Culture=3Dneutral, = PublicKeyToken=3Db03f5f7f11d50a3a'<--- INFO - INFO - ---> (Inner Exception #3) System.IO.FileNotFoundException: Could no= t load file or assembly or one of its dependencies. INFO - File name: 'System.Net.Http, Version=3D4.0.0.0, Culture=3Dneutral, = PublicKeyToken=3Db03f5f7f11d50a3a'<--- INFO - INFO - ---> (Inner Exception #4) System.IO.FileNotFoundException: Could no= t load file or assembly or one of its dependencies. INFO - File name: 'System.Net.Http, Version=3D4.0.0.0, Culture=3Dneutral, = PublicKeyToken=3Db03f5f7f11d50a3a'<--- INFO - INFO - ------------------------------------------------ INFO - --------------Cmd Output Finished--------------- INFO - --------- Running Time (mm:ss): 00:00 ---------- INFO - ------------------------------------------------ Thanks, Bob From: Matthew Carlson [mailto:macarl@microsoft.com] Sent: Tuesday, December 3, 2019 12:56 PM To: Jeremiah Cox ; Feng, Bob C ; devel@edk2.groups.io; lersek@redhat.com; Sean Brogan ; Bret Barkelew Subject: RE: [EXTERNAL] RE: [edk2-devel] Seeking feedback: example of buil= ding OvmfPkg with Pytools Hello Bob, 4. We currently support CLANG and GCC on windows through WSL. I=92ve playe= d around with trying to get CLANG on windows native compiling, but we haven= = =92t pushed it into the pytool tools_def and other configuration. You=92re= welcome to bring your own toolsdef and place it in the CONF folder or spec= ify your own special CONF folder rather than the default one. Pytool techni= cally supports any toolchain but right now on Windows it only has the confi= guration for VS2015-2019. We=92re working on a guide to use WSL, which is a= lready in pull request. Hopefully that answers your question. Matthew Carlson Core UEFI From: Jeremiah Cox Sent: Monday, December 2, 2019 4:40 PM To: Feng, Bob C; devel@edk2.groups.io; lersek@redhat.com; Sean Br= ogan; Matthew Carlson; Bret Barkelew Subject: RE: [EXTERNAL] RE: [edk2-devel] Seeking feedback: example of buil= ding OvmfPkg with Pytools Hi Bob, Responses to your questions: 1. Yes, PlatformBuild.py could be seen as a template for a relatively-s= imple, Pytools platform builder. I am also working on a more complex examp= le for the Edk2-Platforms Kabylake RVP to demonstrate FSP and FIT related p= re and post build steps. The filename PlatformBuild.py is not required, ju= st a convention that we follow. Any Python that overrides UefiBuilder, Buil= dSettingsManager, UpdateSettingsManager, & SetupSettingsManager would work.= You might view PlatformBuild.py as an extension of the EDK2 build system = that provides features helpful to build a platform. Observe that both Ovm= fPkg and EmulatorPkg include build.sh scripts that provide some pre and pos= t build convenience features. Regarding MdeModulePkg, the Pytools Platform= or UefiBuilder could build it, but might not provide compelling value. In= stead, we are leveraging our Pytools CI features to build & test MdeModuleP= kg and more: * https://github.com/tianocore/edk2/tree/master/.pytool * https://github.com/tianocore/edk2/blob/master/MdeModulePkg/MdeMod= ulePkg.ci.yaml 2. For the build environment, I believe we start with the current shell= environment, add Conf/target.txt, override with environment set in Platfor= mBuild.py, then the command-line. @Sean Brogan & @Matthew Carlson for more details. 3. I am working on an example of building KabylakeOpenBoardPackage http= s://github.com/out0xb2/edk2-platforms/pull/2 = . Custom pre and post build steps for a platform can be handled in a Platf= ormBuild.py, for a family they could be in a FamilyBuild.py that is !includ= ed in multiple PlatformBuild.py, or common functionality could be pushed in= to plug-ins. I demonstrate the plug-in model in the Intel Silicon Tools. 4. Hmm, I would ask @Sean Brogan & @M= atthew Carlson 5. We do not currently support VS 2015, only 2017 and 2019. I=92m hear= ing an ask that we should document our matrix of supported OS/toolchains. * We have not added support for VS 2015 * We would need more details to debug this one, perhaps provide Bui= ld\BUILDLOG.txt ? * Again, more details are needed, likely available in Build\UPDATE_= LOG.txt ? i. We = have GCC working for Ubuntu in our automation: https://dev.azure.com/tiano= core/edk2-ci/_build?definitionId=3D31&_a=3Dsummary If there is a good time early morning or later afternoon for a Skype call = and screen sharing, we might be able to close on these issues faster. Let = me know if you have suggestions. Thanks, Jeremiah From: Feng, Bob C Sent: Thursday, November 28, 2019 18:50 To: devel@edk2.groups.io; lersek@redhat.com; Jeremiah Cox Subject: [EXTERNAL] RE: [edk2-devel] Seeking feedback: example of building= OvmfPkg with Pytools Hi Jeremiah, I have some questions for this change. 1. Will there be a PlatformBuild.py in any platform as your design? For ex= ample there will be a PlatformBuild.py in MdeModulePkg, EmulatorPkg and etc= . And this PlatformBuild.py can be seen as a template? 2. The original build options can be set either from PlatformBuild.py Comm= onPlatform class or set from command line, is it right? 3. There is another build scripts edk2-platforms\Platform\Intel\build_bios= .py for build Min-Platforms. Can PlatformBuild.py work together with it? 4. Dose the pytools support clang on windows? 5. I tried this patch on my local machines, but build failed with pytools = and your change. 1) One windows machine only installed VS2015, vswhere can't detect VS2015= . 2) Another windows machine installed VS2015 and VS2017, vswhere can find V= S2017, but build failed with the messages build.py... INFO - : error 7000: Failed to execute command INFO - Vc\bin\nmake.exe 3) I have a Ubuntu 18.04 machine, build failed at the step of "stuart_upda= te -c OvmfPkg/PlatformBuild.py" with the error messages: File "/home/bobfeng/PythonVenv1/lib/python3.6/site-packages/edk2toolex= t/environment/extdeptypes/nuget_dependency.py", line 180, in fetch if self._fetch_from_cache(package_name): File "/home/bobfeng/PythonVenv1/lib/python3.6/site-packages/edk2toolext/= environment/extdeptypes/nuget_dependency.py", line 113, in _fetch_from_cach= e if not os.path.isdir(NugetDependency.global_cache_path): File "/usr/lib/python3.6/genericpath.py", line 42, in isdir st =3D os.stat(s) Thanks, Bob -----Original Message----- From: devel@edk2.groups.io [mailto:devel@edk2= .groups.io] On Behalf Of Laszlo Ersek Sent: Wednesday, November 27, 2019 5:29 PM To: devel@edk2.groups.io; jerecox@microsoft.c= om Subject: Re: [edk2-devel] Seeking feedback: example of building OvmfPkg wi= th Pytools Hi Jeremiah, On 11/27/19 01:03, Jeremiah Cox via Groups.Io wrote: > All, > I created an example of how to build OvmfPkg using the Pytools build env= ironment ( https://nam06.safelinks.protection.outlook.com/?url=3Dhttps%3A%2= F%2Fgithub.com%2Ftianocore%2Fedk2-pytool-extensions&data=3D02%7C01%7Cje= recox%40microsoft.com%7Cb73016ada4ec4ab091c208d77476eb6c%7C72f988bf86f141af= 91ab2d7cd011db47%7C1%7C0%7C637105926421631531&sdata=3DcEITDGTog5kBjBA8H= QDRt%2FyVem9xpqtbnC%2FitE8eyTE%3D&reserved=3D0 ). > It is available for your feedback on GitHub: > https://nam06.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fgith= ub.com%2Fout0xb2%2Fedk2%2Fpull%2F3%2Ffiles&data=3D02%7C01%7Cjerecox%40m= icrosoft.com%7Cb73016ada4ec4ab091c208d77476eb6c%7C72f988bf86f141af91ab2d7cd= 011db47%7C1%7C0%7C637105926421631531&sdata=3DSHlSuJMKDf8GOWwH8KrNIf6MtC= %2FnjhRRZ1htdQT8%2Fvc%3D&reserved=3D0 > > Looking for your thoughts on integrating this example into EDK2. This patch set seems to target the edk2 repository, and at least in part O= vmfPkg. Please submit the patch set to the list for review. Thanks Laszlo --_000_BYAPR21MB13343FC1C92B4805CFD8F395AD420BYAPR21MB1334namp_ Content-Type: text/html; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable
Hi Bob,
The 5.2 failure is expected as we do not support VS2015 yet.  If sugg= est adding a feature request here :
Regarding 2. you may need to provide a BLD_ prefix to pass build parameter= s to the platform builder, refer to my README for an example.

In Matt's example for 3, processor family is not x86, but rather a group o= f products that share common architectural components, for example IceLake = generation products might share many components. This is somewhat similar t= o KabylakeOpenBoardPackage.

@Matthew Carlson, perhaps you= could look at the second log in 5.2 above, perhaps we are missing a dependency in our requirements.txt?  System.Net.Htt= p?

From: Matthew Carlson <= macarl@microsoft.com>
Sent: Tuesday, December 3, 2019 9:04:08 AM
To: Feng, Bob C <bob.c.feng@intel.com>; Jeremiah Cox <jere= cox@microsoft.com>; devel@edk2.groups.io <devel@edk2.groups.io>; l= ersek@redhat.com <lersek@redhat.com>; Sean Brogan <sean.brogan@mic= rosoft.com>; Bret Barkelew <Bret.Barkelew@microsoft.com>; Matthew Carlson <macarl@microsoft.com>
Subject: RE: [EXTERNAL] RE: [edk2-devel] Seeking feedback: example = of building OvmfPkg with Pytools
 

Hello Bob,

 

  1. Right. We actua= lly have three documents forthcoming on how to move platforms onto edk2 pyt= ools, one on OVMF, one on Raspberry Pi, and another on KabylakeOpenBoardPac= kage. Jeremiah sent some links to his platform. We=92ll have more example platforms and tutorial in the future<= /li>
  2. Yes. All the= remaining arguments get passed into the UefiBuilder itself, which parses i= t and hands it to build.py
  3. Where ever makes sense. Since it=92s a python file, it can b= e very flexible. For our platforms using pytools, it generally goes
    PROCESSOR FAMIILY
    |  -- ProcessorFamilyPkg
    |  -- ProductNamePkg
    |     |  -- PlatformBuild.py
  4. Glad to help 😊

 

- Matthew Carlson

 

From: <= a href=3D"mailto:bob.c.feng@intel.com">Feng, Bob C
Sent: Monday, December 2, 2019 11:47 PM
To: Matthew Carlson; Jeremiah Cox; devel@edk2.group= s.io; lersek@redhat.com; Sean Brogan; Bret Barke= lew
Subject: RE: [EXTERNAL] RE: [edk2-devel] Seeking feedback: example = of building OvmfPkg with Pytools

 

Hi Matthew and Jere= miah,

 

Thanks for your ans= wers.

 

  1. Cool. So platform= Build.py is intent to handle the additional pre-build and post-build tasks.= I think it would be great if there is a document record how to create a Pl= atformBuild.py.
  2. So the command-line arguments for build.py can also work for PlatformBuil= d.py. right?
  3. W= here would FamilyBuild.py be located?
  4. Cool. Thanks= .

 

For 5.2 the error m= essage in BUILD.txt is:

 

PROGRESS - Running Pre Build

INFO - Cmd to run is: e:\pythonvenv1\lib\site-packages\edk2toollib\= bin\vswhere.exe -latest -nologo -all -property installationPath -products *=

INFO - ------------------------------------------------

INFO - --------------Cmd Output Starting---------------

INFO - ------------------------------------------------

INFO - C:\Program Files (x86)\Microsoft Visual Studio\2017\Professi= onal

INFO - ------------------------------------------------

INFO - --------------Cmd Output Finished---------------

INFO - --------- Running Time (mm:ss): 00:00 ----------

INFO - ------------------------------------------------

DEBUG - Calling 'C:\Program Files (x86)\Microsoft Visual Studio\201= 7\Professional\VC\Auxiliary\Build\vcvarsall.bat amd64'

DEBUG - Getting host info for host: uname_result(system=3D'Windows'= , node=3D'sh1gapp1005', release=3D'10', version=3D'10.0.14393', machine=3D'= AMD64', processor=3D'Intel64 Family 6 Model 85 Stepping 4, GenuineIntel')

DEBUG - Getting host info for host: uname_result(system=3D'Windows'= , node=3D'sh1gapp1005', release=3D'10', version=3D'10.0.14393', machine=3D'= AMD64', processor=3D'Intel64 Family 6 Model 85 Stepping 4, GenuineIntel')

DEBUG - Getting host info for host: uname_result(system=3D'Windows'= , node=3D'sh1gapp1005', release=3D'10', version=3D'10.0.14393', machine=3D'= AMD64', processor=3D'Intel64 Family 6 Model 85 Stepping 4, GenuineIntel')

DEBUG - Plugin Success: Windows RC Path Support

DEBUG - Plugin Success: Windows Visual Studio Tool Chain Support

INFO - Writing BuildToolsReports to E:\BobFeng\edk2\Build\Ovmf3264\= DEBUG_VS2015x86\BUILD_TOOLS_REPORT

DEBUG - Plugin Success: Build Tools Report Generator

PROGRESS - Running Build DEBUG

DEBUG - Getting all build keys for build type DEBUG

INFO - Cmd to run is: build -p OvmfPkg/OvmfPkgIa32X64.dsc -b DEBUG = -t VS2015x86 -a IA32 -a X64

INFO - ------------------------------------------------

INFO - --------------Cmd Output Starting---------------

INFO - ------------------------------------------------

INFO - Build environment: Windows-10-10.0.14393-SP0

INFO - Build start time: 14:32:35, Dec.03 2019

INFO -

INFO - WORKSPACE        =3D e:\b= obfeng\edk2

INFO - EDK_TOOLS_PATH   =3D e:\bobfeng\edk2\basetools

INFO - CONF_PATH        =3D E:\B= obFeng\edk2\Conf

INFO - PYTHON_COMMAND   =3D e:\pythonvenv1\scripts\python= .exe

INFO -

INFO -

INFO - Processing meta-data .

INFO - Architecture(s)  =3D IA32 X64

INFO - Build target     =3D DEBUG

INFO - Toolchain        =3D VS20= 15x86

INFO -

INFO - Active Platform       &nb= sp;  =3D e:\bobfeng\edk2\OvmfPkg\OvmfPkgIa32X64.dsc

INFO - ........ done!

INFO - Building ... e:\bobfeng\edk2\MdePkg\Library\UefiLib\UefiLib.= inf [X64]

INFO - The system cannot find the path specified.

INFO -

INFO -

INFO - build.py...

INFO -  : error 7000: Failed to execute command

INFO -   Vc\bin\nmake.exe /nologo tbuild [e:\bobfeng\edk2\Buil= d\Ovmf3264\DEBUG_VS2015x86\X64\MdePkg\Library\UefiLib\UefiLib]

INFO -

INFO -

INFO - build.py...

INFO -  : error F002: Failed to build module

INFO -   e:\bobfeng\edk2\MdePkg\Library\UefiLib\UefiLib.inf [X= 64, VS2015x86, DEBUG]

INFO -

INFO - - Failed -

INFO - Build end time: 14:32:46, Dec.03 2019

INFO - Build total time: 00:00:11

INFO -

INFO - ------------------------------------------------

INFO - --------------Cmd Output Finished---------------

INFO - --------- Running Time (mm:ss): 00:11 ----------

INFO - ------------------------------------------------

ERROR - Compiler #7000 from :   Failed to execute command=

ERROR - EDK2 #002 from :   Failed to build module<= /p>

CRITICAL - Build failed

PROGRESS - End time: 2019-12-03 14:32:47.018063   &n= bsp;       Total time Elapsed: 0:00:15=

SECTION - Log file is located at: E:\BobFeng\edk2\Build\BUILDLOG_Ov= mfPkg.txt

SECTION - Summary

PROGRESS - Error

 

For 5.3, the error = message in UPDATE.txt is:

 

INFO - Cmd to run is: mono /home/bobfeng/.local/lib/python3.6/site-= packages/edk2toolext/bin/NuGet.exe locals global-packages -list

INFO - ------------------------------------------------

INFO - --------------Cmd Output Starting---------------

INFO - ------------------------------------------------

INFO - Could not load file or assembly or one of its dependencies.<= /span>

INFO -   Could not load file or assembly or one of its de= pendencies.

INFO -   Could not load file or assembly or one of its de= pendencies.

INFO -   Could not load file or assembly or one of its de= pendencies.

INFO -   Could not load file or assembly or one of its de= pendencies.

INFO - System.AggregateException: One or more errors occurred. ---&= gt; System.IO.FileNotFoundException: Could not load file or assembly or one= of its dependencies.

INFO -    --- End of inner exception stack trace ---=

INFO -   at NuGet.CommandLine.Program.Initialize (NuGet.I= FileSystem fileSystem, NuGet.CommandLine.IConsole console) [0x000ce] in <= ;a4491e56495b4f4098ef5ac3d5c9f065>:0

INFO -   at NuGet.CommandLine.Program.MainCore (System.St= ring workingDirectory, System.String[] args) [0x000d8] in <a4491e56495b4= f4098ef5ac3d5c9f065>:0

INFO - ---> (Inner Exception #0) System.IO.FileNotFoundException= : Could not load file or assembly or one of its dependencies.

INFO - File name: 'System.Net.Http, Version=3D4.0.0.0, Culture=3Dne= utral, PublicKeyToken=3Db03f5f7f11d50a3a'<---

INFO -

INFO - ---> (Inner Exception #1) System.IO.FileNotFoundException= : Could not load file or assembly or one of its dependencies.

INFO - File name: 'System.Net.Http, Version=3D4.0.0.0, Culture=3Dne= utral, PublicKeyToken=3Db03f5f7f11d50a3a'<---

INFO -

INFO - ---> (Inner Exception #2) System.IO.FileNotFoundException= : Could not load file or assembly or one of its dependencies.

INFO - File name: 'System.Net.Http, Version=3D4.0.0.0, Culture=3Dne= utral, PublicKeyToken=3Db03f5f7f11d50a3a'<---

INFO -

INFO - ---> (Inner Exception #3) System.IO.FileNotFoundException= : Could not load file or assembly or one of its dependencies.

INFO - File name: 'System.Net.Http, Version=3D4.0.0.0, Culture=3Dne= utral, PublicKeyToken=3Db03f5f7f11d50a3a'<---

INFO -

INFO - ---> (Inner Exception #4) System.IO.FileNotFoundException= : Could not load file or assembly or one of its dependencies.

INFO - File name: 'System.Net.Http, Version=3D4.0.0.0, Culture=3Dne= utral, PublicKeyToken=3Db03f5f7f11d50a3a'<---

INFO -

INFO - ------------------------------------------------

INFO - --------------Cmd Output Finished---------------

INFO - --------- Running Time (mm:ss): 00:00 ----------

INFO - ------------------------------------------------

 

Thanks,

Bob

From: Matthew Carlson [mailto:macarl@micro= soft.com]
Sent: Tuesday, December 3, 2019 12:56 PM
To: Jeremiah Cox <jerecox@microsoft.com>; Feng, Bob C <bob= .c.feng@intel.com>; devel@edk2.groups.io; lersek@redhat.com; Sean Brogan= <sean.brogan@microsoft.com>; Bret Barkelew <Bret.Barkelew@microso= ft.com>
Subject: RE: [EXTERNAL] RE: [edk2-devel] Seeking feedback: example = of building OvmfPkg with Pytools

 

Hello Bob,

 

4. We currently support CLANG and GCC on windows = through WSL. I=92ve played around with trying to get CLANG on windows nativ= e compiling, but we haven=92t pushed it into the pytool tools_def and other= configuration. You=92re welcome to bring your own toolsdef and place it in the CONF folder or specify your own spe= cial CONF folder rather than the default one. Pytool technically supports a= ny toolchain but right now on Windows it only has the configuration for VS2= 015-2019. We=92re working on a guide to use WSL, which is already in pull request.

 

Hopefully that answers your question.

 

Matthew Carlson

Core UEFI

 

From: Jeremiah Cox
Sent: Monday, December 2, 2019 4:40 PM
To: Feng, Bob C; devel@edk2.groups.io; lersek@redh= at.com; Sean Brogan; Matthew Carlson; Bret B= arkelew
Subject: RE: [EXTERNAL] RE: [edk2-devel] Seeking feedback: example = of building OvmfPkg with Pytools

 

Hi Bob,

Responses to your questions:

  1. Yes, PlatformB= uild.py could be seen as a template for a relatively-simple, Pytools platfo= rm builder.  I am also working on a more complex example for the Edk2-= Platforms Kabylake RVP to demonstrate FSP and FIT related pre and post build steps.  The filename PlatformBuil= d.py is not required, just a convention that we follow. Any Python that ove= rrides UefiBuilder, BuildSettingsManager, UpdateSettingsManager, & Setu= pSettingsManager would work.  You might view PlatformBuild.py as an extension of the EDK2 build system that provides f= eatures helpful to build a platform.   Observe that both OvmfPkg = and EmulatorPkg include build.sh scripts that provide some pre and post bui= ld convenience features.  Regarding MdeModulePkg, the Pytools Platform or UefiBuilder could build it, but might not provide= compelling value.  Instead, we are leveraging our Pytools CI features= to build & test MdeModulePkg and more: 
    1. https://github.com/tianocore/edk2/tree/master/.pytool
    2. https://gi= thub.com/tianocore/edk2/blob/master/MdeModulePkg/MdeModulePkg.ci.yaml
  2. For the build = environment, I believe we start with the current shell environment, add Con= f/target.txt, override with environment set in PlatformBuild.py, then the c= ommand-line.  @Sean Brogan & @Matthew Carlson for more details.
  3. I am working on an example of bui= lding KabylakeOpenBoardPackage https://github.com/out0xb2/edk2-platforms/pull/2 .  Custom pre an= d post build steps for a platform can be handled in a PlatformBuild.py, for= a family they could be in a FamilyBuild.py that is !included in multiple P= latformBuild.py, or common functionality could be pushed into plug-ins.  I demonstrate the plug-in model in t= he Intel Silicon Tools.
  4. Hmm, I would ask @Sean Brogan & @Matthew Carlson
  5. We do not currently support VS 2015, only 2017 and = 2019.  I=92m hearing an ask that we should document our matrix of supp= orted OS/toolchains.  
    1. We have not ad= ded support for VS 2015
    2. We would need more details to debug this one, perhaps provide = Build\BUILDLOG.txt ?
    3. Again, more details are needed, likely available in Build\UPDATE_= LOG.txt ? 

          &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  i.      We have GCC working for Ubuntu in our automation:  https://dev.azure.com/tianocore/edk2-ci/_build?definitionId=3D31&_a=3D= summary

 

If there is a good time early morning or later a= fternoon for a Skype call and screen sharing, we might be able to close on = these issues faster.  Let me know if you have suggestions.

 

Thanks,

Jeremiah

 

From: Feng, Bob C
Sent: Thursday, November 28, 2019 18:50
To: devel@edk2.groups.io; lersek@redhat.com; Jeremiah C= ox
Subject: [EXTERNAL] RE: [edk2-devel] Seeking feedback: example of b= uilding OvmfPkg with Pytools

 

Hi Jeremiah,

I have some questions for this change.
1. Will there be a PlatformBuild.py in any platform as your design? For ex= ample there will be a PlatformBuild.py in MdeModulePkg, EmulatorPkg and etc= . And this PlatformBuild.py can be seen as a template?
2. The original build options can be set either from PlatformBuild.py Comm= onPlatform class or set from command line, is it right?
3. There is another build scripts edk2-platforms\Platform\Intel\build_bios= .py for build Min-Platforms. Can PlatformBuild.py work together with it?
4. Dose the pytools support clang on windows?
5. I tried this patch on my local machines, but build failed with pytools = and your change.
1) One windows machine only installed VS2015,  vswhere can't detect V= S2015.
2) Another windows machine installed VS2015 and VS2017, vswhere can find V= S2017, but build failed with the messages
    build.py...
INFO -  : error 7000: Failed to execute command
INFO -  Vc\bin\nmake.exe

3) I have a Ubuntu 18.04 machine, build failed at the step of "stuart= _update -c OvmfPkg/PlatformBuild.py" with the error messages:
    File "/home/bobfeng/PythonVenv1/lib/python3.6/site= -packages/edk2toolext/environment/extdeptypes/nuget_dependency.py", li= ne 180, in fetch
    if self._fetch_from_cache(package_name):
  File "/home/bobfeng/PythonVenv1/lib/python3.6/site-packages/ed= k2toolext/environment/extdeptypes/nuget_dependency.py", line 113, in _= fetch_from_cache
    if not os.path.isdir(NugetDependency.global_cache_path)= :
  File "/usr/lib/python3.6/genericpath.py", line 42, in isd= ir
    st =3D os.stat(s)

Thanks,
Bob

-----Original Message-----
From: devel@edk2.groups.io [mailto:devel@edk2.groups.io] On B= ehalf Of Laszlo Ersek
Sent: Wednesday, November 27, 2019 5:29 PM
To: devel@edk2.groups.io; jerecox@microsoft.com
Subject: Re: [edk2-devel] Seeking feedback: example of building OvmfPkg wi= th Pytools

Hi Jeremiah,

On 11/27/19 01:03, Jeremiah Cox via Groups.Io wrote:
> All,
> I created an example of how to build OvmfPkg using the Pytools build = environment ( https://nam06.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fgithub= .com%2Ftianocore%2Fedk2-pytool-extensions&amp;data=3D02%7C01%7Cjerecox%= 40microsoft.com%7Cb73016ada4ec4ab091c208d77476eb6c%7C72f988bf86f141af91ab2d= 7cd011db47%7C1%7C0%7C637105926421631531&amp;sdata=3DcEITDGTog5kBjBA8HQD= Rt%2FyVem9xpqtbnC%2FitE8eyTE%3D&amp;reserved=3D0 ).
> It is available for your feedback on GitHub:
> https://nam06.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fgithub= .com%2Fout0xb2%2Fedk2%2Fpull%2F3%2Ffiles&amp;data=3D02%7C01%7Cjerecox%4= 0microsoft.com%7Cb73016ada4ec4ab091c208d77476eb6c%7C72f988bf86f141af91ab2d7= cd011db47%7C1%7C0%7C637105926421631531&amp;sdata=3DSHlSuJMKDf8GOWwH8KrN= If6MtC%2FnjhRRZ1htdQT8%2Fvc%3D&amp;reserved=3D0
>
> Looking for your thoughts on integrating this example into EDK2.

This patch set seems to target the edk2 repository, and at least in part O= vmfPkg. Please submit the patch set to the list for review.

Thanks
Laszlo


 

 

--_000_BYAPR21MB13343FC1C92B4805CFD8F395AD420BYAPR21MB1334namp_--