Re: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?

["Rebecca Cran" <rebecca@bsdio.com>]

[-- Attachment #1: Type: text/plain, Size: 16292 bytes --]

That means you need to install the various mono assemblies, such as the 
libmono-system-net-http4.0-cil package in Ubuntu.

I spent some time chasing down the dependencies last year which got very 
frustrating and tedious, but have forgotten which were needed. All I 
remember is there were quite a lot.


-- 
Rebecca Cran


On 7/1/21 7:36 PM, Steven Shi wrote:
>
> I already installed mono, but NuGet.exe still fails to work in my 
> side. I’m behind a proxy but don’t know how to configure the NuGet.exe 
> proxy in .pytool. Below is my verbose error message.
>
> (unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ stuart_update 
> -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5 -v
>
> … …
>
> SECTION - Start Invocable Tool
>
> SECTION - Initial update of environment
>
> DEBUG - --- self_describing_environment.update_extdeps()
>
> DEBUG - Verify 'mu_nasm' returning 'False'.
>
> DEBUG - Verify 'edk2-acpica-iasl' returning 'False'.
>
> DEBUG - Creating 5 threads for the SDE update
>
> UpdatingDEBUG - Verify 'gcc_aarch64_linux' returning 'True'.
>
> DEBUG - Verify 'gcc_riscv64_unknown' returning 'True'.
>
> DEBUG - Verify 'mu_nasm' returning 'False'.
>
> DEBUG - Verify 'gcc_arm_linux' returning 'True'.
>
> DEBUG - Verify 'edk2-acpica-iasl' returning 'False'.
>
> DEBUG - Cleaning dependency directory for 'mu_nasm'...
>
> DEBUG - Cleaning dependency directory for 'edk2-acpica-iasl'...
>
> INFO - Cmd to run is: mono 
> /home/jshi19/wksp_efi/unit_test_env/lib/python3.8/site-packages/edk2toolext/bin/NuGet.exe 
> locals global-packages -list
>
> INFO - Cmd to run is: mono 
> /home/jshi19/wksp_efi/unit_test_env/lib/python3.8/site-packages/edk2toolext/bin/NuGet.exe 
> locals global-packages -list
>
> INFO - ------------------------------------------------
>
> INFO - ------------------------------------------------
>
> INFO - --------------Cmd Output Starting---------------
>
> INFO - --------------Cmd Output Starting---------------
>
> INFO - ------------------------------------------------
>
> INFO - ------------------------------------------------
>
> ...INFO - Could not load file or assembly 'System.Net.Http, 
> Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or 
> one of its dependencies.
>
> INFO -   Could not load file or assembly 'System.Net.Http, 
> Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or 
> one of its dependencies.
>
> INFO -   Could not load file or assembly 'System.Net.Http, 
> Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or 
> one of its dependencies.
>
> INFO -   Could not load file or assembly 'System.Net.Http, 
> Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or 
> one of its dependencies.
>
> INFO -   Could not load file or assembly 'System.Net.Http, 
> Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or 
> one of its dependencies.
>
> INFO - System.AggregateException: One or more errors occurred. (Could 
> not load file or assembly 'System.Net.Http, Version=4.0.0.0, 
> Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its 
> dependencies.) (Could not load file or assembly 'System.Net.Http, 
> Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or 
> one of its dependencies.) (Could not load file or assembly 
> 'System.Net.Http, Version=4.0.0.0, Culture=neutral, 
> PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could 
> not load file or assembly 'System.Net.Http, Version=4.0.0.0, 
> Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its 
> dependencies.) (Could not load file or assembly 'System.Net.Http, 
> Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or 
> one of its dependencies.) ---> System.IO.FileNotFoundException: Could 
> not load file or assembly 'System.Net.Http, Version=4.0.0.0, 
> Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its 
> dependencies.
>
> INFO -    --- End of inner exception stack trace ---
>
> … …
>
> If I run the Cmd manually, I get below error:
>
> (unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ mono 
> /home/jshi19/wksp_efi/unit_test_env/lib/python3.8/site-packages/edk2toolext/bin/NuGet.exe 
> locals global-packages -list
>
> Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, 
> Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its 
> dependencies.
>
>   Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, 
> Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its 
> dependencies.
>
>   Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, 
> Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its 
> dependencies.
>
>   Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, 
> Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its 
> dependencies.
>
>   Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, 
> Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its 
> dependencies.
>
> System.AggregateException: One or more errors occurred. (Could not 
> load file or assembly 'System.Net.Http, Version=4.0.0.0, 
> Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its 
> dependencies.) (Could not load file or assembly 'System.Net.Http, 
> Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or 
> one of its dependencies.) (Could not load file or assembly 
> 'System.Net.Http, Version=4.0.0.0, Culture=neutral, 
> PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could 
> not load file or assembly 'System.Net.Http, Version=4.0.0.0, 
> Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its 
> dependencies.) (Could not load file or assembly 'System.Net.Http, 
> Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or 
> one of its dependencies.) ---> System.IO.FileNotFoundException: Could 
> not load file or assembly 'System.Net.Http, Version=4.0.0.0, 
> Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its 
> dependencies.
>
>    --- End of inner exception stack trace ---
>
>   at NuGet.CommandLine.Program.Initialize (NuGet.IFileSystem 
> fileSystem, NuGet.CommandLine.IConsole console) [0x000ce] in 
> <ff502667b3fe4a85827635c6fe6b0d0d>:0
>
>   at NuGet.CommandLine.Program.MainCore (System.String 
> workingDirectory, System.String[] args) [0x000d8] in 
> <ff502667b3fe4a85827635c6fe6b0d0d>:0
>
> ---> (Inner Exception #0) System.IO.FileNotFoundException: Could not 
> load file or assembly 'System.Net.Http, Version=4.0.0.0, 
> Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its 
> dependencies.
>
> File name: 'System.Net.Http, Version=4.0.0.0, Culture=neutral, 
> PublicKeyToken=b03f5f7f11d50a3a'<---
>
> ---> (Inner Exception #1) System.IO.FileNotFoundException: Could not 
> load file or assembly 'System.Net.Http, Version=4.0.0.0, 
> Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its 
> dependencies.
>
> File name: 'System.Net.Http, Version=4.0.0.0, Culture=neutral, 
> PublicKeyToken=b03f5f7f11d50a3a'<---
>
> ---> (Inner Exception #2) System.IO.FileNotFoundException: Could not 
> load file or assembly 'System.Net.Http, Version=4.0.0.0, 
> Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its 
> dependencies.
>
> File name: 'System.Net.Http, Version=4.0.0.0, Culture=neutral, 
> PublicKeyToken=b03f5f7f11d50a3a'<---
>
> ---> (Inner Exception #3) System.IO.FileNotFoundException: Could not 
> load file or assembly 'System.Net.Http, Version=4.0.0.0, 
> Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its 
> dependencies.
>
> File name: 'System.Net.Http, Version=4.0.0.0, Culture=neutral, 
> PublicKeyToken=b03f5f7f11d50a3a'<---
>
> ---> (Inner Exception #4) System.IO.FileNotFoundException: Could not 
> load file or assembly 'System.Net.Http, Version=4.0.0.0, 
> Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its 
> dependencies.
>
> File name: 'System.Net.Http, Version=4.0.0.0, Culture=neutral, 
> PublicKeyToken=b03f5f7f11d50a3a'<---
>
> Thanks
>
> *Steven Shi***
>
> **
>
> *From:* Bret Barkelew <Bret.Barkelew@microsoft.com>
> *Sent:* Friday, July 2, 2021 9:04 AM
> *To:* Shi, Steven <steven.shi@intel.com>; devel@edk2.groups.io; 
> rebecca@nuviainc.com
> *Subject:* RE: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests 
> work in Linux?
>
> I believe the Azure DevOps build agents that are used by the EDK2 CI 
> pipeline already have mono installed.
>
> - Bret
>
> *From: *Shi, Steven <mailto:steven.shi@intel.com>
> *Sent: *Thursday, July 1, 2021 5:54 PM
> *To: *devel@edk2.groups.io <mailto:devel@edk2.groups.io>; Bret 
> Barkelew <mailto:Bret.Barkelew@microsoft.com>; rebecca@nuviainc.com 
> <mailto:rebecca@nuviainc.com>
> *Subject: *RE: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests 
> work in Linux?
>
> Hi Bret, Rebecca,
>
> Thanks for the info. Does edk2 CI run the .pytool unit tests in Linux? 
> If yes, how does edk2 CI solve the nuget dependency problem in Linux 
> right now?
>
> Thanks
>
> *Steven Shi*
>
> **
>
> *From:* devel@edk2.groups.io <mailto:devel@edk2.groups.io> 
> <devel@edk2.groups.io <mailto:devel@edk2.groups.io>> *On Behalf Of 
> *Bret Barkelew via groups.io
> *Sent:* Wednesday, June 30, 2021 1:02 AM
> *To:* devel@edk2.groups.io <mailto:devel@edk2.groups.io>; 
> rebecca@nuviainc.com <mailto:rebecca@nuviainc.com>; Shi, Steven 
> <steven.shi@intel.com <mailto:steven.shi@intel.com>>
> *Subject:* Re: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests 
> work in Linux?
>
> Hmmm. If that works well, we might be able to investigate carrying 
> both and picking based on the host OS.
>
> - Bret
>
> *From:*devel@edk2.groups.io <mailto:devel@edk2.groups.io> 
> <devel@edk2.groups.io <mailto:devel@edk2.groups.io>> on behalf of 
> Rebecca Cran via groups.io <rebecca=nuviainc.com@groups.io 
> <mailto:rebecca=nuviainc.com@groups.io>>
> *Sent:* Tuesday, June 29, 2021 9:42:41 AM
> *To:* devel@edk2.groups.io <mailto:devel@edk2.groups.io> 
> <devel@edk2.groups.io <mailto:devel@edk2.groups.io>>; 
> steven.shi@intel.com <mailto:steven.shi@intel.com> 
> <steven.shi@intel.com <mailto:steven.shi@intel.com>>
> *Subject:* [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work 
> in Linux?
>
> Your Linux distribution should have a nuget package available (e.g. 
> https://packages.ubuntu.com/hirsute/nuget 
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpackages.ubuntu.com%2Fhirsute%2Fnuget&data=04%7C01%7Cbret.barkelew%40microsoft.com%7C28bf7b813a9a4d6e365508d93cf3fd8a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637607840913328150%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Z7vCN9JUmx1kQks%2FuWKjcnMRo7koWoyxOET0bnSbN6U%3D&reserved=0>) 
> but the EDK2 CI is designed to use the copy of nuget installed with 
> the edk2toolext package.
>
> -- 
> Rebecca Cran
>
> On 6/29/21 9:44 AM, Steven Shi wrote:
>
>     Cran, Thanks. I’m curious whether the .pytool has native Linux
>     executable binary to run? It looks the edk2 CI framework has more
>     dependency in Linux than Windows.
>
>     Thanks
>
>     *Steven Shi*
>
>     *Intel\IAGS\SFP\FIA (Firmware Infrastructure Automation)*
>
>     **
>
>     *From:* devel@edk2.groups.io <mailto:devel@edk2.groups.io>
>     <devel@edk2.groups.io> <mailto:devel@edk2.groups.io> *On Behalf Of
>     * Rebecca Cran
>     *Sent:* Tuesday, June 29, 2021 8:13 PM
>     *To:* devel@edk2.groups.io <mailto:devel@edk2.groups.io>; Shi,
>     Steven <steven.shi@intel.com> <mailto:steven.shi@intel.com>
>     *Subject:* Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
>
>     NuGet.exe is a .NET assembly/executable, so on Linux you'll need
>     to install Mono (https://www.mono-project.com/
>     <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.mono-project.com%2F&data=04%7C01%7Cbret.barkelew%40microsoft.com%7C28bf7b813a9a4d6e365508d93cf3fd8a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637607840913338106%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=uPkpInnWMx%2FP35GUXlbffQwxbLya9DDDegyLJvtuScU%3D&reserved=0>).
>
>     -- 
>
>     Rebecca Cran
>
>     On 6/28/21 7:45 PM, Steven Shi wrote:
>
>         Hello,
>
>         I’m interested in the edk2 CI unit
>         tests(https://github.com/tianocore/edk2/tree/master/.pytool
>         <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2F.pytool&data=04%7C01%7Cbret.barkelew%40microsoft.com%7C28bf7b813a9a4d6e365508d93cf3fd8a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637607840913338106%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=v%2FqnYohY9bDCIAC8ZmqPkV3RvhaIuyQEXfkY9JNyIqs%3D&reserved=0>).
>         I tested it in my local. It works well in Windows but does not
>         work in Linux. It looks the NuGet in pytool only has windows
>         version
>         (C:\steven\unit_test_env\Lib\site-packages\edk2toolext\bin\NuGet.exe)
>         but not support the Linux. Below is my detail test steps. Do
>         the edk2 CI unit tests work in Linux?
>
>          1. Windows :
>
>         c:\steven>py -m venv unit_test_env
>
>         c:\steven>cd  c:\steven\edk2
>
>         c:\steven\edk2>c:\steven\unit_test_env\Scripts\activate.bat
>
>         (unit_test_env) c:\steven\edk2>pip install --upgrade -r
>         pip-requirements.txt
>
>         (unit_test_env) c:\steven\edk2>stuart_setup -c
>         .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019
>
>         (unit_test_env) c:\steven\edk2>stuart_update -c
>         .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019
>
>         (unit_test_env) c:\steven\edk2>edksetup.bat
>
>         (unit_test_env) c:\steven\edk2>edksetup.bat Rebuild
>
>         (unit_test_env) c:\steven\edk2>stuart_ci_build -c
>         .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019 -a X64,IA32 -t
>         DEBUG --verbose
>
>         …
>
>         PROGRESS - Overall Build Status: Success
>
>         SECTION - Summary
>
>         PROGRESS - Success
>
>          2. Linux:
>
>         jshi19@ub2-uefi-b01:~/wksp_efi$ python3 -m venv unit_test_env
>
>         jshi19@ub2-uefi-b01:~/wksp_efi$ source unit_test_env/bin/activate
>
>         (unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ pip
>         install --upgrade -r pip-requirements.txt
>
>         (unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ make -C
>         BaseTools/
>
>         (unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$
>         stuart_setup -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5
>
>         (unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$
>         stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5
>
>         SECTION - Init SDE
>
>         WARNING - Using Pip Tools based BaseTools
>
>         SECTION - Loading Plugins
>
>         SECTION - Start Invocable Tool
>
>         SECTION - Initial update of environment
>
>         UpdatingWARNING - [SDE] Failed to fetch NugetDependecy:
>         edk2-acpica-iasl@20200717.0.0
>         <mailto:edk2-acpica-iasl@20200717.0.0>: [Nuget] We failed to
>         install this version 20200717.0.0 of edk2-acpica-iasl
>
>         WARNING - [SDE] Failed to fetch NugetDependecy:
>         mu_nasm@2.15.05 <mailto:mu_nasm@2.15.05>: [Nuget] We failed to
>         install this version 2.15.05 of mu_nasm
>
>         . Done
>
>         SECTION -       Updated/Verified 3 dependencies
>
>         SECTION - Second pass update of environment
>
>         UpdatingWARNING - [SDE] Failed to fetch NugetDependecy:
>         edk2-acpica-iasl@20200717.0.0
>         <mailto:edk2-acpica-iasl@20200717.0.0>: [Nuget] We failed to
>         install this version 20200717.0.0 of edk2-acpica-iasl
>
>         WARNING - [SDE] Failed to fetch NugetDependecy:
>         mu_nasm@2.15.05 <mailto:mu_nasm@2.15.05>: [Nuget] We failed to
>         install this version 2.15.05 of mu_nasm
>
>         . Done
>
>         SECTION -       Updated/Verified 3 dependencies
>
>         ERROR - We were unable to successfully update 2 dependencies
>         in environment
>
>         SECTION - Summary
>
>         ERROR - Error
>
>         Thanks
>
>         *Steven Shi*
>
>         **
>
>         **
>
> 

[-- Attachment #2.1: Type: text/html, Size: 35353 bytes --]

[-- Attachment #2.2: image005.png --]
[-- Type: image/png, Size: 195 bytes --]