* Do the edk2 CI unit tests work in Linux?
@ 2021-06-29 1:45 Steven Shi
2021-06-29 12:12 ` [edk2-devel] " Rebecca Cran
0 siblings, 1 reply; 17+ messages in thread
From: Steven Shi @ 2021-06-29 1:45 UTC (permalink / raw)
To: devel@edk2.groups.io
[-- Attachment #1: Type: text/plain, Size: 2881 bytes --]
Hello,
I'm interested in the edk2 CI unit tests(https://github.com/tianocore/edk2/tree/master/.pytool). I tested it in my local. It works well in Windows but does not work in Linux. It looks the NuGet in pytool only has windows version (C:\steven\unit_test_env\Lib\site-packages\edk2toolext\bin\NuGet.exe) but not support the Linux. Below is my detail test steps. Do the edk2 CI unit tests work in Linux?
* Windows :
c:\steven>py -m venv unit_test_env
c:\steven>cd c:\steven\edk2
c:\steven\edk2>c:\steven\unit_test_env\Scripts\activate.bat
(unit_test_env) c:\steven\edk2>pip install --upgrade -r pip-requirements.txt
(unit_test_env) c:\steven\edk2>stuart_setup -c .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019
(unit_test_env) c:\steven\edk2>stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019
(unit_test_env) c:\steven\edk2>edksetup.bat
(unit_test_env) c:\steven\edk2>edksetup.bat Rebuild
(unit_test_env) c:\steven\edk2>stuart_ci_build -c .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019 -a X64,IA32 -t DEBUG --verbose
...
PROGRESS - Overall Build Status: Success
SECTION - Summary
PROGRESS - Success
* Linux:
jshi19@ub2-uefi-b01:~/wksp_efi$ python3 -m venv unit_test_env
jshi19@ub2-uefi-b01:~/wksp_efi$ source unit_test_env/bin/activate
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ pip install --upgrade -r pip-requirements.txt
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ make -C BaseTools/
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ stuart_setup -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5
SECTION - Init SDE
WARNING - Using Pip Tools based BaseTools
SECTION - Loading Plugins
SECTION - Start Invocable Tool
SECTION - Initial update of environment
UpdatingWARNING - [SDE] Failed to fetch NugetDependecy: edk2-acpica-iasl@20200717.0.0<mailto:edk2-acpica-iasl@20200717.0.0>: [Nuget] We failed to install this version 20200717.0.0 of edk2-acpica-iasl
WARNING - [SDE] Failed to fetch NugetDependecy: mu_nasm@2.15.05<mailto:mu_nasm@2.15.05>: [Nuget] We failed to install this version 2.15.05 of mu_nasm
. Done
SECTION - Updated/Verified 3 dependencies
SECTION - Second pass update of environment
UpdatingWARNING - [SDE] Failed to fetch NugetDependecy: edk2-acpica-iasl@20200717.0.0<mailto:edk2-acpica-iasl@20200717.0.0>: [Nuget] We failed to install this version 20200717.0.0 of edk2-acpica-iasl
WARNING - [SDE] Failed to fetch NugetDependecy: mu_nasm@2.15.05<mailto:mu_nasm@2.15.05>: [Nuget] We failed to install this version 2.15.05 of mu_nasm
. Done
SECTION - Updated/Verified 3 dependencies
ERROR - We were unable to successfully update 2 dependencies in environment
SECTION - Summary
ERROR - Error
Thanks
Steven Shi
[-- Attachment #2: Type: text/html, Size: 9571 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
2021-06-29 1:45 Do the edk2 CI unit tests work in Linux? Steven Shi
@ 2021-06-29 12:12 ` Rebecca Cran
2021-06-29 15:44 ` Steven Shi
2021-06-29 15:56 ` Bret Barkelew
0 siblings, 2 replies; 17+ messages in thread
From: Rebecca Cran @ 2021-06-29 12:12 UTC (permalink / raw)
To: devel, steven.shi
[-- Attachment #1: Type: text/plain, Size: 3279 bytes --]
NuGet.exe is a .NET assembly/executable, so on Linux you'll need to
install Mono (https://www.mono-project.com/).
--
Rebecca Cran
On 6/28/21 7:45 PM, Steven Shi wrote:
>
> Hello,
>
> I’m interested in the edk2 CI unit
> tests(https://github.com/tianocore/edk2/tree/master/.pytool
> <https://github.com/tianocore/edk2/tree/master/.pytool>). I tested it
> in my local. It works well in Windows but does not work in Linux. It
> looks the NuGet in pytool only has windows version
> (C:\steven\unit_test_env\Lib\site-packages\edk2toolext\bin\NuGet.exe)
> but not support the Linux. Below is my detail test steps. Do the edk2
> CI unit tests work in Linux?
>
> * Windows :
>
> c:\steven>py -m venv unit_test_env
>
> c:\steven>cd c:\steven\edk2
>
> c:\steven\edk2>c:\steven\unit_test_env\Scripts\activate.bat
>
> (unit_test_env) c:\steven\edk2>pip install --upgrade -r
> pip-requirements.txt
>
> (unit_test_env) c:\steven\edk2>stuart_setup -c .pytool/CISettings.py
> TOOL_CHAIN_TAG=VS2019
>
> (unit_test_env) c:\steven\edk2>stuart_update -c .pytool/CISettings.py
> TOOL_CHAIN_TAG=VS2019
>
> (unit_test_env) c:\steven\edk2>edksetup.bat
>
> (unit_test_env) c:\steven\edk2>edksetup.bat Rebuild
>
> (unit_test_env) c:\steven\edk2>stuart_ci_build -c
> .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019 -a X64,IA32 -t DEBUG --verbose
>
> …
>
> PROGRESS - Overall Build Status: Success
>
> SECTION - Summary
>
> PROGRESS - Success
>
> * Linux:
>
> jshi19@ub2-uefi-b01:~/wksp_efi$ python3 -m venv unit_test_env
>
> jshi19@ub2-uefi-b01:~/wksp_efi$ source unit_test_env/bin/activate
>
> (unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ pip install
> --upgrade -r pip-requirements.txt
>
> (unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ make -C BaseTools/
>
> (unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ stuart_setup -c
> .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5
>
> (unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ stuart_update
> -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5
>
> SECTION - Init SDE
>
> WARNING - Using Pip Tools based BaseTools
>
> SECTION - Loading Plugins
>
> SECTION - Start Invocable Tool
>
> SECTION - Initial update of environment
>
> UpdatingWARNING - [SDE] Failed to fetch NugetDependecy:
> edk2-acpica-iasl@20200717.0.0 <mailto:edk2-acpica-iasl@20200717.0.0>:
> [Nuget] We failed to install this version 20200717.0.0 of edk2-acpica-iasl
>
> WARNING - [SDE] Failed to fetch NugetDependecy: mu_nasm@2.15.05
> <mailto:mu_nasm@2.15.05>: [Nuget] We failed to install this version
> 2.15.05 of mu_nasm
>
> . Done
>
> SECTION - Updated/Verified 3 dependencies
>
> SECTION - Second pass update of environment
>
> UpdatingWARNING - [SDE] Failed to fetch NugetDependecy:
> edk2-acpica-iasl@20200717.0.0 <mailto:edk2-acpica-iasl@20200717.0.0>:
> [Nuget] We failed to install this version 20200717.0.0 of edk2-acpica-iasl
>
> WARNING - [SDE] Failed to fetch NugetDependecy: mu_nasm@2.15.05
> <mailto:mu_nasm@2.15.05>: [Nuget] We failed to install this version
> 2.15.05 of mu_nasm
>
> . Done
>
> SECTION - Updated/Verified 3 dependencies
>
> ERROR - We were unable to successfully update 2 dependencies in
> environment
>
> SECTION - Summary
>
> ERROR - Error
>
> Thanks
>
> *Steven Shi***
>
> **
>
> **
>
>
[-- Attachment #2: Type: text/html, Size: 8844 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
2021-06-29 12:12 ` [edk2-devel] " Rebecca Cran
@ 2021-06-29 15:44 ` Steven Shi
2021-06-29 15:59 ` Bret Barkelew
2021-06-29 16:42 ` Rebecca Cran
2021-06-29 15:56 ` Bret Barkelew
1 sibling, 2 replies; 17+ messages in thread
From: Steven Shi @ 2021-06-29 15:44 UTC (permalink / raw)
To: devel@edk2.groups.io, rebecca@nuviainc.com
[-- Attachment #1: Type: text/plain, Size: 3571 bytes --]
Cran, Thanks. I'm curious whether the .pytool has native Linux executable binary to run? It looks the edk2 CI framework has more dependency in Linux than Windows.
Thanks
Steven Shi
Intel\IAGS\SFP\FIA (Firmware Infrastructure Automation)
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Rebecca Cran
Sent: Tuesday, June 29, 2021 8:13 PM
To: devel@edk2.groups.io; Shi, Steven <steven.shi@intel.com>
Subject: Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
NuGet.exe is a .NET assembly/executable, so on Linux you'll need to install Mono (https://www.mono-project.com/).
--
Rebecca Cran
On 6/28/21 7:45 PM, Steven Shi wrote:
Hello,
I'm interested in the edk2 CI unit tests(https://github.com/tianocore/edk2/tree/master/.pytool). I tested it in my local. It works well in Windows but does not work in Linux. It looks the NuGet in pytool only has windows version (C:\steven\unit_test_env\Lib\site-packages\edk2toolext\bin\NuGet.exe) but not support the Linux. Below is my detail test steps. Do the edk2 CI unit tests work in Linux?
1. Windows :
c:\steven>py -m venv unit_test_env
c:\steven>cd c:\steven\edk2
c:\steven\edk2>c:\steven\unit_test_env\Scripts\activate.bat
(unit_test_env) c:\steven\edk2>pip install --upgrade -r pip-requirements.txt
(unit_test_env) c:\steven\edk2>stuart_setup -c .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019
(unit_test_env) c:\steven\edk2>stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019
(unit_test_env) c:\steven\edk2>edksetup.bat
(unit_test_env) c:\steven\edk2>edksetup.bat Rebuild
(unit_test_env) c:\steven\edk2>stuart_ci_build -c .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019 -a X64,IA32 -t DEBUG --verbose
...
PROGRESS - Overall Build Status: Success
SECTION - Summary
PROGRESS - Success
1. Linux:
jshi19@ub2-uefi-b01:~/wksp_efi$ python3 -m venv unit_test_env
jshi19@ub2-uefi-b01:~/wksp_efi$ source unit_test_env/bin/activate
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ pip install --upgrade -r pip-requirements.txt
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ make -C BaseTools/
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ stuart_setup -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5
SECTION - Init SDE
WARNING - Using Pip Tools based BaseTools
SECTION - Loading Plugins
SECTION - Start Invocable Tool
SECTION - Initial update of environment
UpdatingWARNING - [SDE] Failed to fetch NugetDependecy: edk2-acpica-iasl@20200717.0.0<mailto:edk2-acpica-iasl@20200717.0.0>: [Nuget] We failed to install this version 20200717.0.0 of edk2-acpica-iasl
WARNING - [SDE] Failed to fetch NugetDependecy: mu_nasm@2.15.05<mailto:mu_nasm@2.15.05>: [Nuget] We failed to install this version 2.15.05 of mu_nasm
. Done
SECTION - Updated/Verified 3 dependencies
SECTION - Second pass update of environment
UpdatingWARNING - [SDE] Failed to fetch NugetDependecy: edk2-acpica-iasl@20200717.0.0<mailto:edk2-acpica-iasl@20200717.0.0>: [Nuget] We failed to install this version 20200717.0.0 of edk2-acpica-iasl
WARNING - [SDE] Failed to fetch NugetDependecy: mu_nasm@2.15.05<mailto:mu_nasm@2.15.05>: [Nuget] We failed to install this version 2.15.05 of mu_nasm
. Done
SECTION - Updated/Verified 3 dependencies
ERROR - We were unable to successfully update 2 dependencies in environment
SECTION - Summary
ERROR - Error
Thanks
Steven Shi
[-- Attachment #2: Type: text/html, Size: 14806 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
2021-06-29 12:12 ` [edk2-devel] " Rebecca Cran
2021-06-29 15:44 ` Steven Shi
@ 2021-06-29 15:56 ` Bret Barkelew
1 sibling, 0 replies; 17+ messages in thread
From: Bret Barkelew @ 2021-06-29 15:56 UTC (permalink / raw)
To: devel@edk2.groups.io, rebecca@nuviainc.com, steven.shi@intel.com
[-- Attachment #1.1: Type: text/plain, Size: 4592 bytes --]
Here’s some of the documentation, but I believe the standard apt mono-devel package is now up-to-date enough that the extra apt source is not necessary.
edk2-pytool-extensions/using_extdep.md at master · tianocore/edk2-pytool-extensions (github.com)<https://github.com/tianocore/edk2-pytool-extensions/blob/master/docs/usability/using_extdep.md#a-note-on-nuget-on-linux>
- Bret
From: Rebecca Cran via groups.io<mailto:rebecca=nuviainc.com@groups.io>
Sent: Tuesday, June 29, 2021 5:12 AM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; steven.shi@intel.com<mailto:steven.shi@intel.com>
Subject: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
NuGet.exe is a .NET assembly/executable, so on Linux you'll need to install Mono (https://www.mono-project.com/<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.mono-project.com%2F&data=04%7C01%7CBret.Barkelew%40microsoft.com%7Cfe60fdcf94a242b39a6608d93af73376%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637605655671112304%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=%2BGrt1sTixwcPtDw%2FjFGtLM7ehm2RJff2R%2BcuErU5EDQ%3D&reserved=0>).
--
Rebecca Cran
On 6/28/21 7:45 PM, Steven Shi wrote:
Hello,
I’m interested in the edk2 CI unit tests(https://github.com/tianocore/edk2/tree/master/.pytool<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2F.pytool&data=04%7C01%7CBret.Barkelew%40microsoft.com%7Cfe60fdcf94a242b39a6608d93af73376%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637605655671122239%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=W6PWzgoBh5Gyz8mwEo0ZgH%2F2DfbxQZVSeTPkIJBRafo%3D&reserved=0>). I tested it in my local. It works well in Windows but does not work in Linux. It looks the NuGet in pytool only has windows version (C:\steven\unit_test_env\Lib\site-packages\edk2toolext\bin\NuGet.exe) but not support the Linux. Below is my detail test steps. Do the edk2 CI unit tests work in Linux?
1. Windows :
c:\steven>py -m venv unit_test_env
c:\steven>cd c:\steven\edk2
c:\steven\edk2>c:\steven\unit_test_env\Scripts\activate.bat
(unit_test_env) c:\steven\edk2>pip install --upgrade -r pip-requirements.txt
(unit_test_env) c:\steven\edk2>stuart_setup -c .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019
(unit_test_env) c:\steven\edk2>stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019
(unit_test_env) c:\steven\edk2>edksetup.bat
(unit_test_env) c:\steven\edk2>edksetup.bat Rebuild
(unit_test_env) c:\steven\edk2>stuart_ci_build -c .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019 -a X64,IA32 -t DEBUG --verbose
…
PROGRESS - Overall Build Status: Success
SECTION - Summary
PROGRESS - Success
1. Linux:
jshi19@ub2-uefi-b01:~/wksp_efi$ python3 -m venv unit_test_env
jshi19@ub2-uefi-b01:~/wksp_efi$ source unit_test_env/bin/activate
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ pip install --upgrade -r pip-requirements.txt
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ make -C BaseTools/
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ stuart_setup -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5
SECTION - Init SDE
WARNING - Using Pip Tools based BaseTools
SECTION - Loading Plugins
SECTION - Start Invocable Tool
SECTION - Initial update of environment
UpdatingWARNING - [SDE] Failed to fetch NugetDependecy: edk2-acpica-iasl@20200717.0.0<mailto:edk2-acpica-iasl@20200717.0.0>: [Nuget] We failed to install this version 20200717.0.0 of edk2-acpica-iasl
WARNING - [SDE] Failed to fetch NugetDependecy: mu_nasm@2.15.05<mailto:mu_nasm@2.15.05>: [Nuget] We failed to install this version 2.15.05 of mu_nasm
. Done
SECTION - Updated/Verified 3 dependencies
SECTION - Second pass update of environment
UpdatingWARNING - [SDE] Failed to fetch NugetDependecy: edk2-acpica-iasl@20200717.0.0<mailto:edk2-acpica-iasl@20200717.0.0>: [Nuget] We failed to install this version 20200717.0.0 of edk2-acpica-iasl
WARNING - [SDE] Failed to fetch NugetDependecy: mu_nasm@2.15.05<mailto:mu_nasm@2.15.05>: [Nuget] We failed to install this version 2.15.05 of mu_nasm
. Done
SECTION - Updated/Verified 3 dependencies
ERROR - We were unable to successfully update 2 dependencies in environment
SECTION - Summary
ERROR - Error
Thanks
Steven Shi
[-- Attachment #1.2: Type: text/html, Size: 10947 bytes --]
[-- Attachment #2: ABCB5F07C9BA4761B35D8231E951D573.png --]
[-- Type: image/png, Size: 140 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
2021-06-29 15:44 ` Steven Shi
@ 2021-06-29 15:59 ` Bret Barkelew
2021-06-29 16:37 ` Rebecca Cran
2021-06-29 16:42 ` Rebecca Cran
1 sibling, 1 reply; 17+ messages in thread
From: Bret Barkelew @ 2021-06-29 15:59 UTC (permalink / raw)
To: devel@edk2.groups.io, steven.shi@intel.com, rebecca@nuviainc.com
[-- Attachment #1.1: Type: text/plain, Size: 4883 bytes --]
Which executable are you referring to?
Also, yes, it is entirely possible that Linux has more dependencies since the dependency list has conditionals based on your OS.
- Bret
From: Steven Shi via groups.io<mailto:steven.shi=intel.com@groups.io>
Sent: Tuesday, June 29, 2021 8:44 AM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; rebecca@nuviainc.com<mailto:rebecca@nuviainc.com>
Subject: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
Cran, Thanks. I’m curious whether the .pytool has native Linux executable binary to run? It looks the edk2 CI framework has more dependency in Linux than Windows.
Thanks
Steven Shi
Intel\IAGS\SFP\FIA (Firmware Infrastructure Automation)
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Rebecca Cran
Sent: Tuesday, June 29, 2021 8:13 PM
To: devel@edk2.groups.io; Shi, Steven <steven.shi@intel.com>
Subject: Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
NuGet.exe is a .NET assembly/executable, so on Linux you'll need to install Mono (https://www.mono-project.com/<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.mono-project.com%2F&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C2d053d9e90c54c1367aa08d93b14c604%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637605782683188184%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=cInus0qs0MpcW0xMuRImJH4f%2FJ15gBAXZtaUEKf%2BMmc%3D&reserved=0>).
--
Rebecca Cran
On 6/28/21 7:45 PM, Steven Shi wrote:
Hello,
I’m interested in the edk2 CI unit tests(https://github.com/tianocore/edk2/tree/master/.pytool<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2F.pytool&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C2d053d9e90c54c1367aa08d93b14c604%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637605782683198117%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=SqPYBI8L%2BoM1lqCdY4vOptqNcfdP1ohMGKgD3gojpP0%3D&reserved=0>). I tested it in my local. It works well in Windows but does not work in Linux. It looks the NuGet in pytool only has windows version (C:\steven\unit_test_env\Lib\site-packages\edk2toolext\bin\NuGet.exe) but not support the Linux. Below is my detail test steps. Do the edk2 CI unit tests work in Linux?
1. Windows :
c:\steven>py -m venv unit_test_env
c:\steven>cd c:\steven\edk2
c:\steven\edk2>c:\steven\unit_test_env\Scripts\activate.bat
(unit_test_env) c:\steven\edk2>pip install --upgrade -r pip-requirements.txt
(unit_test_env) c:\steven\edk2>stuart_setup -c .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019
(unit_test_env) c:\steven\edk2>stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019
(unit_test_env) c:\steven\edk2>edksetup.bat
(unit_test_env) c:\steven\edk2>edksetup.bat Rebuild
(unit_test_env) c:\steven\edk2>stuart_ci_build -c .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019 -a X64,IA32 -t DEBUG --verbose
…
PROGRESS - Overall Build Status: Success
SECTION - Summary
PROGRESS - Success
1. Linux:
jshi19@ub2-uefi-b01:~/wksp_efi$ python3 -m venv unit_test_env
jshi19@ub2-uefi-b01:~/wksp_efi$ source unit_test_env/bin/activate
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ pip install --upgrade -r pip-requirements.txt
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ make -C BaseTools/
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ stuart_setup -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5
SECTION - Init SDE
WARNING - Using Pip Tools based BaseTools
SECTION - Loading Plugins
SECTION - Start Invocable Tool
SECTION - Initial update of environment
UpdatingWARNING - [SDE] Failed to fetch NugetDependecy: edk2-acpica-iasl@20200717.0.0<mailto:edk2-acpica-iasl@20200717.0.0>: [Nuget] We failed to install this version 20200717.0.0 of edk2-acpica-iasl
WARNING - [SDE] Failed to fetch NugetDependecy: mu_nasm@2.15.05<mailto:mu_nasm@2.15.05>: [Nuget] We failed to install this version 2.15.05 of mu_nasm
. Done
SECTION - Updated/Verified 3 dependencies
SECTION - Second pass update of environment
UpdatingWARNING - [SDE] Failed to fetch NugetDependecy: edk2-acpica-iasl@20200717.0.0<mailto:edk2-acpica-iasl@20200717.0.0>: [Nuget] We failed to install this version 20200717.0.0 of edk2-acpica-iasl
WARNING - [SDE] Failed to fetch NugetDependecy: mu_nasm@2.15.05<mailto:mu_nasm@2.15.05>: [Nuget] We failed to install this version 2.15.05 of mu_nasm
. Done
SECTION - Updated/Verified 3 dependencies
ERROR - We were unable to successfully update 2 dependencies in environment
SECTION - Summary
ERROR - Error
Thanks
Steven Shi
[-- Attachment #1.2: Type: text/html, Size: 12193 bytes --]
[-- Attachment #2: 3DD9830017BC4F93AACE34599E80F4A7.png --]
[-- Type: image/png, Size: 140 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
2021-06-29 15:59 ` Bret Barkelew
@ 2021-06-29 16:37 ` Rebecca Cran
0 siblings, 0 replies; 17+ messages in thread
From: Rebecca Cran @ 2021-06-29 16:37 UTC (permalink / raw)
To: Bret Barkelew, devel@edk2.groups.io, steven.shi@intel.com
[-- Attachment #1: Type: text/plain, Size: 5565 bytes --]
I think he's referring to NuGet.exe, which unless you're familiar with
.NET appears to be a Windows executable.
--
Rebecca Cran
On 6/29/21 9:59 AM, Bret Barkelew wrote:
>
> Which executable are you referring to?
>
> Also, yes, it is entirely possible that Linux has more dependencies
> since the dependency list has conditionals based on your OS.
>
> - Bret
>
> *From: *Steven Shi via groups.io <mailto:steven.shi=intel.com@groups.io>
> *Sent: *Tuesday, June 29, 2021 8:44 AM
> *To: *devel@edk2.groups.io <mailto:devel@edk2.groups.io>;
> rebecca@nuviainc.com <mailto:rebecca@nuviainc.com>
> *Subject: *[EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work
> in Linux?
>
> Cran, Thanks. I’m curious whether the .pytool has native Linux
> executable binary to run? It looks the edk2 CI framework has more
> dependency in Linux than Windows.
>
> Thanks
>
> *Steven Shi*
>
> *Intel\IAGS\SFP\FIA (Firmware Infrastructure Automation)***
>
> **
>
> *From:* devel@edk2.groups.io <devel@edk2.groups.io> *On Behalf Of
> *Rebecca Cran
> *Sent:* Tuesday, June 29, 2021 8:13 PM
> *To:* devel@edk2.groups.io; Shi, Steven <steven.shi@intel.com>
> *Subject:* Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
>
> NuGet.exe is a .NET assembly/executable, so on Linux you'll need to
> install Mono (https://www.mono-project.com/
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.mono-project.com%2F&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C2d053d9e90c54c1367aa08d93b14c604%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637605782683188184%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=cInus0qs0MpcW0xMuRImJH4f%2FJ15gBAXZtaUEKf%2BMmc%3D&reserved=0>).
>
> --
>
> Rebecca Cran
>
> On 6/28/21 7:45 PM, Steven Shi wrote:
>
> Hello,
>
> I’m interested in the edk2 CI unit
> tests(https://github.com/tianocore/edk2/tree/master/.pytool
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2F.pytool&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C2d053d9e90c54c1367aa08d93b14c604%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637605782683198117%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=SqPYBI8L%2BoM1lqCdY4vOptqNcfdP1ohMGKgD3gojpP0%3D&reserved=0>).
> I tested it in my local. It works well in Windows but does not
> work in Linux. It looks the NuGet in pytool only has windows
> version
> (C:\steven\unit_test_env\Lib\site-packages\edk2toolext\bin\NuGet.exe)
> but not support the Linux. Below is my detail test steps. Do the
> edk2 CI unit tests work in Linux?
>
> 1. Windows :
>
> c:\steven>py -m venv unit_test_env
>
> c:\steven>cd c:\steven\edk2
>
> c:\steven\edk2>c:\steven\unit_test_env\Scripts\activate.bat
>
> (unit_test_env) c:\steven\edk2>pip install --upgrade -r
> pip-requirements.txt
>
> (unit_test_env) c:\steven\edk2>stuart_setup -c
> .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019
>
> (unit_test_env) c:\steven\edk2>stuart_update -c
> .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019
>
> (unit_test_env) c:\steven\edk2>edksetup.bat
>
> (unit_test_env) c:\steven\edk2>edksetup.bat Rebuild
>
> (unit_test_env) c:\steven\edk2>stuart_ci_build -c
> .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019 -a X64,IA32 -t DEBUG
> --verbose
>
> …
>
> PROGRESS - Overall Build Status: Success
>
> SECTION - Summary
>
> PROGRESS - Success
>
> 2. Linux:
>
> jshi19@ub2-uefi-b01:~/wksp_efi$ python3 -m venv unit_test_env
>
> jshi19@ub2-uefi-b01:~/wksp_efi$ source unit_test_env/bin/activate
>
> (unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ pip install
> --upgrade -r pip-requirements.txt
>
> (unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ make -C
> BaseTools/
>
> (unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$
> stuart_setup -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5
>
> (unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$
> stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5
>
> SECTION - Init SDE
>
> WARNING - Using Pip Tools based BaseTools
>
> SECTION - Loading Plugins
>
> SECTION - Start Invocable Tool
>
> SECTION - Initial update of environment
>
> UpdatingWARNING - [SDE] Failed to fetch NugetDependecy:
> edk2-acpica-iasl@20200717.0.0
> <mailto:edk2-acpica-iasl@20200717.0.0>: [Nuget] We failed to
> install this version 20200717.0.0 of edk2-acpica-iasl
>
> WARNING - [SDE] Failed to fetch NugetDependecy: mu_nasm@2.15.05
> <mailto:mu_nasm@2.15.05>: [Nuget] We failed to install this
> version 2.15.05 of mu_nasm
>
> . Done
>
> SECTION - Updated/Verified 3 dependencies
>
> SECTION - Second pass update of environment
>
> UpdatingWARNING - [SDE] Failed to fetch NugetDependecy:
> edk2-acpica-iasl@20200717.0.0
> <mailto:edk2-acpica-iasl@20200717.0.0>: [Nuget] We failed to
> install this version 20200717.0.0 of edk2-acpica-iasl
>
> WARNING - [SDE] Failed to fetch NugetDependecy: mu_nasm@2.15.05
> <mailto:mu_nasm@2.15.05>: [Nuget] We failed to install this
> version 2.15.05 of mu_nasm
>
> . Done
>
> SECTION - Updated/Verified 3 dependencies
>
> ERROR - We were unable to successfully update 2 dependencies in
> environment
>
> SECTION - Summary
>
> ERROR - Error
>
> Thanks
>
> *Steven Shi*
>
> **
>
> **
>
>
>
[-- Attachment #2.1: Type: text/html, Size: 13689 bytes --]
[-- Attachment #2.2: 3DD9830017BC4F93AACE34599E80F4A7.png --]
[-- Type: image/png, Size: 140 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
2021-06-29 15:44 ` Steven Shi
2021-06-29 15:59 ` Bret Barkelew
@ 2021-06-29 16:42 ` Rebecca Cran
2021-06-29 17:02 ` [EXTERNAL] " Bret Barkelew
1 sibling, 1 reply; 17+ messages in thread
From: Rebecca Cran @ 2021-06-29 16:42 UTC (permalink / raw)
To: devel, steven.shi
[-- Attachment #1: Type: text/plain, Size: 4396 bytes --]
Your Linux distribution should have a nuget package available (e.g.
https://packages.ubuntu.com/hirsute/nuget) but the EDK2 CI is designed
to use the copy of nuget installed with the edk2toolext package.
--
Rebecca Cran
On 6/29/21 9:44 AM, Steven Shi wrote:
>
> Cran, Thanks. I’m curious whether the .pytool has native Linux
> executable binary to run? It looks the edk2 CI framework has more
> dependency in Linux than Windows.
>
> Thanks
>
> *Steven Shi***
>
> *Intel\IAGS\SFP\FIA (Firmware Infrastructure Automation)***
>
> **
>
> *From:* devel@edk2.groups.io <devel@edk2.groups.io> *On Behalf Of
> *Rebecca Cran
> *Sent:* Tuesday, June 29, 2021 8:13 PM
> *To:* devel@edk2.groups.io; Shi, Steven <steven.shi@intel.com>
> *Subject:* Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
>
> NuGet.exe is a .NET assembly/executable, so on Linux you'll need to
> install Mono (https://www.mono-project.com/
> <https://www.mono-project.com/>).
>
> --
>
> Rebecca Cran
>
> On 6/28/21 7:45 PM, Steven Shi wrote:
>
> Hello,
>
> I’m interested in the edk2 CI unit
> tests(https://github.com/tianocore/edk2/tree/master/.pytool
> <https://github.com/tianocore/edk2/tree/master/.pytool>). I tested
> it in my local. It works well in Windows but does not work in
> Linux. It looks the NuGet in pytool only has windows version
> (C:\steven\unit_test_env\Lib\site-packages\edk2toolext\bin\NuGet.exe)
> but not support the Linux. Below is my detail test steps. Do the
> edk2 CI unit tests work in Linux?
>
> 1. Windows :
>
> c:\steven>py -m venv unit_test_env
>
> c:\steven>cd c:\steven\edk2
>
> c:\steven\edk2>c:\steven\unit_test_env\Scripts\activate.bat
>
> (unit_test_env) c:\steven\edk2>pip install --upgrade -r
> pip-requirements.txt
>
> (unit_test_env) c:\steven\edk2>stuart_setup -c
> .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019
>
> (unit_test_env) c:\steven\edk2>stuart_update -c
> .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019
>
> (unit_test_env) c:\steven\edk2>edksetup.bat
>
> (unit_test_env) c:\steven\edk2>edksetup.bat Rebuild
>
> (unit_test_env) c:\steven\edk2>stuart_ci_build -c
> .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019 -a X64,IA32 -t DEBUG
> --verbose
>
> …
>
> PROGRESS - Overall Build Status: Success
>
> SECTION - Summary
>
> PROGRESS - Success
>
> 2. Linux:
>
> jshi19@ub2-uefi-b01:~/wksp_efi$ python3 -m venv unit_test_env
>
> jshi19@ub2-uefi-b01:~/wksp_efi$ source unit_test_env/bin/activate
>
> (unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ pip install
> --upgrade -r pip-requirements.txt
>
> (unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ make -C
> BaseTools/
>
> (unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$
> stuart_setup -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5
>
> (unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$
> stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5
>
> SECTION - Init SDE
>
> WARNING - Using Pip Tools based BaseTools
>
> SECTION - Loading Plugins
>
> SECTION - Start Invocable Tool
>
> SECTION - Initial update of environment
>
> UpdatingWARNING - [SDE] Failed to fetch NugetDependecy:
> edk2-acpica-iasl@20200717.0.0
> <mailto:edk2-acpica-iasl@20200717.0.0>: [Nuget] We failed to
> install this version 20200717.0.0 of edk2-acpica-iasl
>
> WARNING - [SDE] Failed to fetch NugetDependecy: mu_nasm@2.15.05
> <mailto:mu_nasm@2.15.05>: [Nuget] We failed to install this
> version 2.15.05 of mu_nasm
>
> . Done
>
> SECTION - Updated/Verified 3 dependencies
>
> SECTION - Second pass update of environment
>
> UpdatingWARNING - [SDE] Failed to fetch NugetDependecy:
> edk2-acpica-iasl@20200717.0.0
> <mailto:edk2-acpica-iasl@20200717.0.0>: [Nuget] We failed to
> install this version 20200717.0.0 of edk2-acpica-iasl
>
> WARNING - [SDE] Failed to fetch NugetDependecy: mu_nasm@2.15.05
> <mailto:mu_nasm@2.15.05>: [Nuget] We failed to install this
> version 2.15.05 of mu_nasm
>
> . Done
>
> SECTION - Updated/Verified 3 dependencies
>
> ERROR - We were unable to successfully update 2 dependencies in
> environment
>
> SECTION - Summary
>
> ERROR - Error
>
> Thanks
>
> *Steven Shi*
>
> **
>
> **
>
>
[-- Attachment #2: Type: text/html, Size: 12140 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
2021-06-29 16:42 ` Rebecca Cran
@ 2021-06-29 17:02 ` Bret Barkelew
2021-07-02 0:54 ` Steven Shi
0 siblings, 1 reply; 17+ messages in thread
From: Bret Barkelew @ 2021-06-29 17:02 UTC (permalink / raw)
To: devel@edk2.groups.io, rebecca@nuviainc.com, steven.shi@intel.com
[-- Attachment #1: Type: text/plain, Size: 5797 bytes --]
Hmmm. If that works well, we might be able to investigate carrying both and picking based on the host OS.
- Bret
________________________________
From: devel@edk2.groups.io <devel@edk2.groups.io> on behalf of Rebecca Cran via groups.io <rebecca=nuviainc.com@groups.io>
Sent: Tuesday, June 29, 2021 9:42:41 AM
To: devel@edk2.groups.io <devel@edk2.groups.io>; steven.shi@intel.com <steven.shi@intel.com>
Subject: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
Your Linux distribution should have a nuget package available (e.g. https://packages.ubuntu.com/hirsute/nuget<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpackages.ubuntu.com%2Fhirsute%2Fnuget&data=04%7C01%7Cbret.barkelew%40microsoft.com%7C07424c8c951c4a7e7eed08d93b1cebe0%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637605817680494682%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=jvpYhiv59XPQ%2FhFkrFTbM4Ug257PPdT3HUZhVHZgZhA%3D&reserved=0>) but the EDK2 CI is designed to use the copy of nuget installed with the edk2toolext package.
--
Rebecca Cran
On 6/29/21 9:44 AM, Steven Shi wrote:
Cran, Thanks. I’m curious whether the .pytool has native Linux executable binary to run? It looks the edk2 CI framework has more dependency in Linux than Windows.
Thanks
Steven Shi
Intel\IAGS\SFP\FIA (Firmware Infrastructure Automation)
From: devel@edk2.groups.io<mailto:devel@edk2.groups.io> <devel@edk2.groups.io><mailto:devel@edk2.groups.io> On Behalf Of Rebecca Cran
Sent: Tuesday, June 29, 2021 8:13 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Shi, Steven <steven.shi@intel.com><mailto:steven.shi@intel.com>
Subject: Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
NuGet.exe is a .NET assembly/executable, so on Linux you'll need to install Mono (https://www.mono-project.com/<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.mono-project.com%2F&data=04%7C01%7Cbret.barkelew%40microsoft.com%7C07424c8c951c4a7e7eed08d93b1cebe0%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637605817680504642%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=KtiO4lxs%2BA8nhxpZHutQr7Y3dP0qxXPG6ToToEsPRrk%3D&reserved=0>).
--
Rebecca Cran
On 6/28/21 7:45 PM, Steven Shi wrote:
Hello,
I’m interested in the edk2 CI unit tests(https://github.com/tianocore/edk2/tree/master/.pytool<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2F.pytool&data=04%7C01%7Cbret.barkelew%40microsoft.com%7C07424c8c951c4a7e7eed08d93b1cebe0%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637605817680504642%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=49RkG1%2F%2BR%2BKWvFZvlVfgCoylFd%2FvfuLBCWl8HeEEIZk%3D&reserved=0>). I tested it in my local. It works well in Windows but does not work in Linux. It looks the NuGet in pytool only has windows version (C:\steven\unit_test_env\Lib\site-packages\edk2toolext\bin\NuGet.exe) but not support the Linux. Below is my detail test steps. Do the edk2 CI unit tests work in Linux?
1. Windows :
c:\steven>py -m venv unit_test_env
c:\steven>cd c:\steven\edk2
c:\steven\edk2>c:\steven\unit_test_env\Scripts\activate.bat
(unit_test_env) c:\steven\edk2>pip install --upgrade -r pip-requirements.txt
(unit_test_env) c:\steven\edk2>stuart_setup -c .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019
(unit_test_env) c:\steven\edk2>stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019
(unit_test_env) c:\steven\edk2>edksetup.bat
(unit_test_env) c:\steven\edk2>edksetup.bat Rebuild
(unit_test_env) c:\steven\edk2>stuart_ci_build -c .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019 -a X64,IA32 -t DEBUG --verbose
…
PROGRESS - Overall Build Status: Success
SECTION - Summary
PROGRESS - Success
1. Linux:
jshi19@ub2-uefi-b01:~/wksp_efi$ python3 -m venv unit_test_env
jshi19@ub2-uefi-b01:~/wksp_efi$ source unit_test_env/bin/activate
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ pip install --upgrade -r pip-requirements.txt
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ make -C BaseTools/
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ stuart_setup -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5
SECTION - Init SDE
WARNING - Using Pip Tools based BaseTools
SECTION - Loading Plugins
SECTION - Start Invocable Tool
SECTION - Initial update of environment
UpdatingWARNING - [SDE] Failed to fetch NugetDependecy: edk2-acpica-iasl@20200717.0.0<mailto:edk2-acpica-iasl@20200717.0.0>: [Nuget] We failed to install this version 20200717.0.0 of edk2-acpica-iasl
WARNING - [SDE] Failed to fetch NugetDependecy: mu_nasm@2.15.05<mailto:mu_nasm@2.15.05>: [Nuget] We failed to install this version 2.15.05 of mu_nasm
. Done
SECTION - Updated/Verified 3 dependencies
SECTION - Second pass update of environment
UpdatingWARNING - [SDE] Failed to fetch NugetDependecy: edk2-acpica-iasl@20200717.0.0<mailto:edk2-acpica-iasl@20200717.0.0>: [Nuget] We failed to install this version 20200717.0.0 of edk2-acpica-iasl
WARNING - [SDE] Failed to fetch NugetDependecy: mu_nasm@2.15.05<mailto:mu_nasm@2.15.05>: [Nuget] We failed to install this version 2.15.05 of mu_nasm
. Done
SECTION - Updated/Verified 3 dependencies
ERROR - We were unable to successfully update 2 dependencies in environment
SECTION - Summary
ERROR - Error
Thanks
Steven Shi
[-- Attachment #2: Type: text/html, Size: 11719 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
2021-06-29 17:02 ` [EXTERNAL] " Bret Barkelew
@ 2021-07-02 0:54 ` Steven Shi
2021-07-02 1:03 ` Bret Barkelew
0 siblings, 1 reply; 17+ messages in thread
From: Steven Shi @ 2021-07-02 0:54 UTC (permalink / raw)
To: devel@edk2.groups.io, bret.barkelew@microsoft.com,
rebecca@nuviainc.com
[-- Attachment #1: Type: text/plain, Size: 6514 bytes --]
Hi Bret, Rebecca,
Thanks for the info. Does edk2 CI run the .pytool unit tests in Linux? If yes, how does edk2 CI solve the nuget dependency problem in Linux right now?
Thanks
Steven Shi
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Bret Barkelew via groups.io
Sent: Wednesday, June 30, 2021 1:02 AM
To: devel@edk2.groups.io; rebecca@nuviainc.com; Shi, Steven <steven.shi@intel.com>
Subject: Re: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
Hmmm. If that works well, we might be able to investigate carrying both and picking based on the host OS.
- Bret
________________________________
From: devel@edk2.groups.io<mailto:devel@edk2.groups.io> <devel@edk2.groups.io<mailto:devel@edk2.groups.io>> on behalf of Rebecca Cran via groups.io <rebecca=nuviainc.com@groups.io<mailto:rebecca=nuviainc.com@groups.io>>
Sent: Tuesday, June 29, 2021 9:42:41 AM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io> <devel@edk2.groups.io<mailto:devel@edk2.groups.io>>; steven.shi@intel.com<mailto:steven.shi@intel.com> <steven.shi@intel.com<mailto:steven.shi@intel.com>>
Subject: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
Your Linux distribution should have a nuget package available (e.g. https://packages.ubuntu.com/hirsute/nuget<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpackages.ubuntu.com%2Fhirsute%2Fnuget&data=04%7C01%7Cbret.barkelew%40microsoft.com%7C07424c8c951c4a7e7eed08d93b1cebe0%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637605817680494682%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=jvpYhiv59XPQ%2FhFkrFTbM4Ug257PPdT3HUZhVHZgZhA%3D&reserved=0>) but the EDK2 CI is designed to use the copy of nuget installed with the edk2toolext package.
--
Rebecca Cran
On 6/29/21 9:44 AM, Steven Shi wrote:
Cran, Thanks. I'm curious whether the .pytool has native Linux executable binary to run? It looks the edk2 CI framework has more dependency in Linux than Windows.
Thanks
Steven Shi
Intel\IAGS\SFP\FIA (Firmware Infrastructure Automation)
From: devel@edk2.groups.io<mailto:devel@edk2.groups.io> <devel@edk2.groups.io><mailto:devel@edk2.groups.io> On Behalf Of Rebecca Cran
Sent: Tuesday, June 29, 2021 8:13 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Shi, Steven <steven.shi@intel.com><mailto:steven.shi@intel.com>
Subject: Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
NuGet.exe is a .NET assembly/executable, so on Linux you'll need to install Mono (https://www.mono-project.com/<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.mono-project.com%2F&data=04%7C01%7Cbret.barkelew%40microsoft.com%7C07424c8c951c4a7e7eed08d93b1cebe0%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637605817680504642%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=KtiO4lxs%2BA8nhxpZHutQr7Y3dP0qxXPG6ToToEsPRrk%3D&reserved=0>).
--
Rebecca Cran
On 6/28/21 7:45 PM, Steven Shi wrote:
Hello,
I'm interested in the edk2 CI unit tests(https://github.com/tianocore/edk2/tree/master/.pytool<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2F.pytool&data=04%7C01%7Cbret.barkelew%40microsoft.com%7C07424c8c951c4a7e7eed08d93b1cebe0%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637605817680504642%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=49RkG1%2F%2BR%2BKWvFZvlVfgCoylFd%2FvfuLBCWl8HeEEIZk%3D&reserved=0>). I tested it in my local. It works well in Windows but does not work in Linux. It looks the NuGet in pytool only has windows version (C:\steven\unit_test_env\Lib\site-packages\edk2toolext\bin\NuGet.exe) but not support the Linux. Below is my detail test steps. Do the edk2 CI unit tests work in Linux?
1. Windows :
c:\steven>py -m venv unit_test_env
c:\steven>cd c:\steven\edk2
c:\steven\edk2>c:\steven\unit_test_env\Scripts\activate.bat
(unit_test_env) c:\steven\edk2>pip install --upgrade -r pip-requirements.txt
(unit_test_env) c:\steven\edk2>stuart_setup -c .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019
(unit_test_env) c:\steven\edk2>stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019
(unit_test_env) c:\steven\edk2>edksetup.bat
(unit_test_env) c:\steven\edk2>edksetup.bat Rebuild
(unit_test_env) c:\steven\edk2>stuart_ci_build -c .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019 -a X64,IA32 -t DEBUG --verbose
...
PROGRESS - Overall Build Status: Success
SECTION - Summary
PROGRESS - Success
1. Linux:
jshi19@ub2-uefi-b01:~/wksp_efi$ python3 -m venv unit_test_env
jshi19@ub2-uefi-b01:~/wksp_efi$ source unit_test_env/bin/activate
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ pip install --upgrade -r pip-requirements.txt
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ make -C BaseTools/
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ stuart_setup -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5
SECTION - Init SDE
WARNING - Using Pip Tools based BaseTools
SECTION - Loading Plugins
SECTION - Start Invocable Tool
SECTION - Initial update of environment
UpdatingWARNING - [SDE] Failed to fetch NugetDependecy: edk2-acpica-iasl@20200717.0.0<mailto:edk2-acpica-iasl@20200717.0.0>: [Nuget] We failed to install this version 20200717.0.0 of edk2-acpica-iasl
WARNING - [SDE] Failed to fetch NugetDependecy: mu_nasm@2.15.05<mailto:mu_nasm@2.15.05>: [Nuget] We failed to install this version 2.15.05 of mu_nasm
. Done
SECTION - Updated/Verified 3 dependencies
SECTION - Second pass update of environment
UpdatingWARNING - [SDE] Failed to fetch NugetDependecy: edk2-acpica-iasl@20200717.0.0<mailto:edk2-acpica-iasl@20200717.0.0>: [Nuget] We failed to install this version 20200717.0.0 of edk2-acpica-iasl
WARNING - [SDE] Failed to fetch NugetDependecy: mu_nasm@2.15.05<mailto:mu_nasm@2.15.05>: [Nuget] We failed to install this version 2.15.05 of mu_nasm
. Done
SECTION - Updated/Verified 3 dependencies
ERROR - We were unable to successfully update 2 dependencies in environment
SECTION - Summary
ERROR - Error
Thanks
Steven Shi
[-- Attachment #2: Type: text/html, Size: 14979 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
2021-07-02 0:54 ` Steven Shi
@ 2021-07-02 1:03 ` Bret Barkelew
2021-07-02 1:36 ` Steven Shi
0 siblings, 1 reply; 17+ messages in thread
From: Bret Barkelew @ 2021-07-02 1:03 UTC (permalink / raw)
To: Shi, Steven, devel@edk2.groups.io, rebecca@nuviainc.com
[-- Attachment #1.1: Type: text/plain, Size: 6924 bytes --]
I believe the Azure DevOps build agents that are used by the EDK2 CI pipeline already have mono installed.
- Bret
From: Shi, Steven<mailto:steven.shi@intel.com>
Sent: Thursday, July 1, 2021 5:54 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Bret Barkelew<mailto:Bret.Barkelew@microsoft.com>; rebecca@nuviainc.com<mailto:rebecca@nuviainc.com>
Subject: RE: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
Hi Bret, Rebecca,
Thanks for the info. Does edk2 CI run the .pytool unit tests in Linux? If yes, how does edk2 CI solve the nuget dependency problem in Linux right now?
Thanks
Steven Shi
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Bret Barkelew via groups.io
Sent: Wednesday, June 30, 2021 1:02 AM
To: devel@edk2.groups.io; rebecca@nuviainc.com; Shi, Steven <steven.shi@intel.com>
Subject: Re: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
Hmmm. If that works well, we might be able to investigate carrying both and picking based on the host OS.
- Bret
From: devel@edk2.groups.io<mailto:devel@edk2.groups.io> <devel@edk2.groups.io<mailto:devel@edk2.groups.io>> on behalf of Rebecca Cran via groups.io <rebecca=nuviainc.com@groups.io<mailto:rebecca=nuviainc.com@groups.io>>
Sent: Tuesday, June 29, 2021 9:42:41 AM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io> <devel@edk2.groups.io<mailto:devel@edk2.groups.io>>; steven.shi@intel.com<mailto:steven.shi@intel.com> <steven.shi@intel.com<mailto:steven.shi@intel.com>>
Subject: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
Your Linux distribution should have a nuget package available (e.g. https://packages.ubuntu.com/hirsute/nuget<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpackages.ubuntu.com%2Fhirsute%2Fnuget&data=04%7C01%7Cbret.barkelew%40microsoft.com%7C28bf7b813a9a4d6e365508d93cf3fd8a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637607840913328150%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Z7vCN9JUmx1kQks%2FuWKjcnMRo7koWoyxOET0bnSbN6U%3D&reserved=0>) but the EDK2 CI is designed to use the copy of nuget installed with the edk2toolext package.
--
Rebecca Cran
On 6/29/21 9:44 AM, Steven Shi wrote:
Cran, Thanks. I’m curious whether the .pytool has native Linux executable binary to run? It looks the edk2 CI framework has more dependency in Linux than Windows.
Thanks
Steven Shi
Intel\IAGS\SFP\FIA (Firmware Infrastructure Automation)
From: devel@edk2.groups.io<mailto:devel@edk2.groups.io> <devel@edk2.groups.io><mailto:devel@edk2.groups.io> On Behalf Of Rebecca Cran
Sent: Tuesday, June 29, 2021 8:13 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Shi, Steven <steven.shi@intel.com><mailto:steven.shi@intel.com>
Subject: Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
NuGet.exe is a .NET assembly/executable, so on Linux you'll need to install Mono (https://www.mono-project.com/<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.mono-project.com%2F&data=04%7C01%7Cbret.barkelew%40microsoft.com%7C28bf7b813a9a4d6e365508d93cf3fd8a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637607840913338106%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=uPkpInnWMx%2FP35GUXlbffQwxbLya9DDDegyLJvtuScU%3D&reserved=0>).
--
Rebecca Cran
On 6/28/21 7:45 PM, Steven Shi wrote:
Hello,
I’m interested in the edk2 CI unit tests(https://github.com/tianocore/edk2/tree/master/.pytool<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2F.pytool&data=04%7C01%7Cbret.barkelew%40microsoft.com%7C28bf7b813a9a4d6e365508d93cf3fd8a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637607840913338106%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=v%2FqnYohY9bDCIAC8ZmqPkV3RvhaIuyQEXfkY9JNyIqs%3D&reserved=0>). I tested it in my local. It works well in Windows but does not work in Linux. It looks the NuGet in pytool only has windows version (C:\steven\unit_test_env\Lib\site-packages\edk2toolext\bin\NuGet.exe) but not support the Linux. Below is my detail test steps. Do the edk2 CI unit tests work in Linux?
1. Windows :
c:\steven>py -m venv unit_test_env
c:\steven>cd c:\steven\edk2
c:\steven\edk2>c:\steven\unit_test_env\Scripts\activate.bat
(unit_test_env) c:\steven\edk2>pip install --upgrade -r pip-requirements.txt
(unit_test_env) c:\steven\edk2>stuart_setup -c .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019
(unit_test_env) c:\steven\edk2>stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019
(unit_test_env) c:\steven\edk2>edksetup.bat
(unit_test_env) c:\steven\edk2>edksetup.bat Rebuild
(unit_test_env) c:\steven\edk2>stuart_ci_build -c .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019 -a X64,IA32 -t DEBUG --verbose
…
PROGRESS - Overall Build Status: Success
SECTION - Summary
PROGRESS - Success
1. Linux:
jshi19@ub2-uefi-b01:~/wksp_efi$ python3 -m venv unit_test_env
jshi19@ub2-uefi-b01:~/wksp_efi$ source unit_test_env/bin/activate
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ pip install --upgrade -r pip-requirements.txt
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ make -C BaseTools/
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ stuart_setup -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5
SECTION - Init SDE
WARNING - Using Pip Tools based BaseTools
SECTION - Loading Plugins
SECTION - Start Invocable Tool
SECTION - Initial update of environment
UpdatingWARNING - [SDE] Failed to fetch NugetDependecy: edk2-acpica-iasl@20200717.0.0<mailto:edk2-acpica-iasl@20200717.0.0>: [Nuget] We failed to install this version 20200717.0.0 of edk2-acpica-iasl
WARNING - [SDE] Failed to fetch NugetDependecy: mu_nasm@2.15.05<mailto:mu_nasm@2.15.05>: [Nuget] We failed to install this version 2.15.05 of mu_nasm
. Done
SECTION - Updated/Verified 3 dependencies
SECTION - Second pass update of environment
UpdatingWARNING - [SDE] Failed to fetch NugetDependecy: edk2-acpica-iasl@20200717.0.0<mailto:edk2-acpica-iasl@20200717.0.0>: [Nuget] We failed to install this version 20200717.0.0 of edk2-acpica-iasl
WARNING - [SDE] Failed to fetch NugetDependecy: mu_nasm@2.15.05<mailto:mu_nasm@2.15.05>: [Nuget] We failed to install this version 2.15.05 of mu_nasm
. Done
SECTION - Updated/Verified 3 dependencies
ERROR - We were unable to successfully update 2 dependencies in environment
SECTION - Summary
ERROR - Error
Thanks
Steven Shi
[-- Attachment #1.2: Type: text/html, Size: 16803 bytes --]
[-- Attachment #2: 29DC0FAC8D8241CFBD35A97A7D76B3DB.png --]
[-- Type: image/png, Size: 152 bytes --]
[-- Attachment #3: 88F1C65C0FB2433097A8A5A3AF92382E.png --]
[-- Type: image/png, Size: 151 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
2021-07-02 1:03 ` Bret Barkelew
@ 2021-07-02 1:36 ` Steven Shi
2021-07-02 1:53 ` Bret Barkelew
2021-07-02 2:00 ` Rebecca Cran
0 siblings, 2 replies; 17+ messages in thread
From: Steven Shi @ 2021-07-02 1:36 UTC (permalink / raw)
To: Bret Barkelew, devel@edk2.groups.io, rebecca@nuviainc.com
[-- Attachment #1.1: Type: text/plain, Size: 14634 bytes --]
I already installed mono, but NuGet.exe still fails to work in my side. I'm behind a proxy but don't know how to configure the NuGet.exe proxy in .pytool. Below is my verbose error message.
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5 -v
... ...
SECTION - Start Invocable Tool
SECTION - Initial update of environment
DEBUG - --- self_describing_environment.update_extdeps()
DEBUG - Verify 'mu_nasm' returning 'False'.
DEBUG - Verify 'edk2-acpica-iasl' returning 'False'.
DEBUG - Creating 5 threads for the SDE update
UpdatingDEBUG - Verify 'gcc_aarch64_linux' returning 'True'.
DEBUG - Verify 'gcc_riscv64_unknown' returning 'True'.
DEBUG - Verify 'mu_nasm' returning 'False'.
DEBUG - Verify 'gcc_arm_linux' returning 'True'.
DEBUG - Verify 'edk2-acpica-iasl' returning 'False'.
DEBUG - Cleaning dependency directory for 'mu_nasm'...
DEBUG - Cleaning dependency directory for 'edk2-acpica-iasl'...
INFO - Cmd to run is: mono /home/jshi19/wksp_efi/unit_test_env/lib/python3.8/site-packages/edk2toolext/bin/NuGet.exe locals global-packages -list
INFO - Cmd to run is: mono /home/jshi19/wksp_efi/unit_test_env/lib/python3.8/site-packages/edk2toolext/bin/NuGet.exe locals global-packages -list
INFO - ------------------------------------------------
INFO - ------------------------------------------------
INFO - --------------Cmd Output Starting---------------
INFO - --------------Cmd Output Starting---------------
INFO - ------------------------------------------------
INFO - ------------------------------------------------
...INFO - Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
INFO - Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
INFO - Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
INFO - Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
INFO - Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
INFO - System.AggregateException: One or more errors occurred. (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) ---> System.IO.FileNotFoundException: Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
INFO - --- End of inner exception stack trace ---
... ...
If I run the Cmd manually, I get below error:
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ mono /home/jshi19/wksp_efi/unit_test_env/lib/python3.8/site-packages/edk2toolext/bin/NuGet.exe locals global-packages -list
Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
System.AggregateException: One or more errors occurred. (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) ---> System.IO.FileNotFoundException: Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
--- End of inner exception stack trace ---
at NuGet.CommandLine.Program.Initialize (NuGet.IFileSystem fileSystem, NuGet.CommandLine.IConsole console) [0x000ce] in <ff502667b3fe4a85827635c6fe6b0d0d>:0
at NuGet.CommandLine.Program.MainCore (System.String workingDirectory, System.String[] args) [0x000d8] in <ff502667b3fe4a85827635c6fe6b0d0d>:0
---> (Inner Exception #0) System.IO.FileNotFoundException: Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
File name: 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'<---
---> (Inner Exception #1) System.IO.FileNotFoundException: Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
File name: 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'<---
---> (Inner Exception #2) System.IO.FileNotFoundException: Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
File name: 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'<---
---> (Inner Exception #3) System.IO.FileNotFoundException: Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
File name: 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'<---
---> (Inner Exception #4) System.IO.FileNotFoundException: Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
File name: 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'<---
Thanks
Steven Shi
From: Bret Barkelew <Bret.Barkelew@microsoft.com>
Sent: Friday, July 2, 2021 9:04 AM
To: Shi, Steven <steven.shi@intel.com>; devel@edk2.groups.io; rebecca@nuviainc.com
Subject: RE: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
I believe the Azure DevOps build agents that are used by the EDK2 CI pipeline already have mono installed.
- Bret
From: Shi, Steven<mailto:steven.shi@intel.com>
Sent: Thursday, July 1, 2021 5:54 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Bret Barkelew<mailto:Bret.Barkelew@microsoft.com>; rebecca@nuviainc.com<mailto:rebecca@nuviainc.com>
Subject: RE: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
Hi Bret, Rebecca,
Thanks for the info. Does edk2 CI run the .pytool unit tests in Linux? If yes, how does edk2 CI solve the nuget dependency problem in Linux right now?
Thanks
Steven Shi
From: devel@edk2.groups.io<mailto:devel@edk2.groups.io> <devel@edk2.groups.io<mailto:devel@edk2.groups.io>> On Behalf Of Bret Barkelew via groups.io
Sent: Wednesday, June 30, 2021 1:02 AM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; rebecca@nuviainc.com<mailto:rebecca@nuviainc.com>; Shi, Steven <steven.shi@intel.com<mailto:steven.shi@intel.com>>
Subject: Re: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
Hmmm. If that works well, we might be able to investigate carrying both and picking based on the host OS.
- Bret
From: devel@edk2.groups.io<mailto:devel@edk2.groups.io> <devel@edk2.groups.io<mailto:devel@edk2.groups.io>> on behalf of Rebecca Cran via groups.io <rebecca=nuviainc.com@groups.io<mailto:rebecca=nuviainc.com@groups.io>>
Sent: Tuesday, June 29, 2021 9:42:41 AM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io> <devel@edk2.groups.io<mailto:devel@edk2.groups.io>>; steven.shi@intel.com<mailto:steven.shi@intel.com> <steven.shi@intel.com<mailto:steven.shi@intel.com>>
Subject: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
Your Linux distribution should have a nuget package available (e.g. https://packages.ubuntu.com/hirsute/nuget<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpackages.ubuntu.com%2Fhirsute%2Fnuget&data=04%7C01%7Cbret.barkelew%40microsoft.com%7C28bf7b813a9a4d6e365508d93cf3fd8a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637607840913328150%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Z7vCN9JUmx1kQks%2FuWKjcnMRo7koWoyxOET0bnSbN6U%3D&reserved=0>) but the EDK2 CI is designed to use the copy of nuget installed with the edk2toolext package.
--
Rebecca Cran
On 6/29/21 9:44 AM, Steven Shi wrote:
Cran, Thanks. I'm curious whether the .pytool has native Linux executable binary to run? It looks the edk2 CI framework has more dependency in Linux than Windows.
Thanks
Steven Shi
Intel\IAGS\SFP\FIA (Firmware Infrastructure Automation)
From: devel@edk2.groups.io<mailto:devel@edk2.groups.io> <devel@edk2.groups.io><mailto:devel@edk2.groups.io> On Behalf Of Rebecca Cran
Sent: Tuesday, June 29, 2021 8:13 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Shi, Steven <steven.shi@intel.com><mailto:steven.shi@intel.com>
Subject: Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
NuGet.exe is a .NET assembly/executable, so on Linux you'll need to install Mono (https://www.mono-project.com/<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.mono-project.com%2F&data=04%7C01%7Cbret.barkelew%40microsoft.com%7C28bf7b813a9a4d6e365508d93cf3fd8a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637607840913338106%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=uPkpInnWMx%2FP35GUXlbffQwxbLya9DDDegyLJvtuScU%3D&reserved=0>).
--
Rebecca Cran
On 6/28/21 7:45 PM, Steven Shi wrote:
Hello,
I'm interested in the edk2 CI unit tests(https://github.com/tianocore/edk2/tree/master/.pytool<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2F.pytool&data=04%7C01%7Cbret.barkelew%40microsoft.com%7C28bf7b813a9a4d6e365508d93cf3fd8a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637607840913338106%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=v%2FqnYohY9bDCIAC8ZmqPkV3RvhaIuyQEXfkY9JNyIqs%3D&reserved=0>). I tested it in my local. It works well in Windows but does not work in Linux. It looks the NuGet in pytool only has windows version (C:\steven\unit_test_env\Lib\site-packages\edk2toolext\bin\NuGet.exe) but not support the Linux. Below is my detail test steps. Do the edk2 CI unit tests work in Linux?
1. Windows :
c:\steven>py -m venv unit_test_env
c:\steven>cd c:\steven\edk2
c:\steven\edk2>c:\steven\unit_test_env\Scripts\activate.bat
(unit_test_env) c:\steven\edk2>pip install --upgrade -r pip-requirements.txt
(unit_test_env) c:\steven\edk2>stuart_setup -c .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019
(unit_test_env) c:\steven\edk2>stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019
(unit_test_env) c:\steven\edk2>edksetup.bat
(unit_test_env) c:\steven\edk2>edksetup.bat Rebuild
(unit_test_env) c:\steven\edk2>stuart_ci_build -c .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019 -a X64,IA32 -t DEBUG --verbose
...
PROGRESS - Overall Build Status: Success
SECTION - Summary
PROGRESS - Success
1. Linux:
jshi19@ub2-uefi-b01:~/wksp_efi$ python3 -m venv unit_test_env
jshi19@ub2-uefi-b01:~/wksp_efi$ source unit_test_env/bin/activate
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ pip install --upgrade -r pip-requirements.txt
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ make -C BaseTools/
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ stuart_setup -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5
SECTION - Init SDE
WARNING - Using Pip Tools based BaseTools
SECTION - Loading Plugins
SECTION - Start Invocable Tool
SECTION - Initial update of environment
UpdatingWARNING - [SDE] Failed to fetch NugetDependecy: edk2-acpica-iasl@20200717.0.0<mailto:edk2-acpica-iasl@20200717.0.0>: [Nuget] We failed to install this version 20200717.0.0 of edk2-acpica-iasl
WARNING - [SDE] Failed to fetch NugetDependecy: mu_nasm@2.15.05<mailto:mu_nasm@2.15.05>: [Nuget] We failed to install this version 2.15.05 of mu_nasm
. Done
SECTION - Updated/Verified 3 dependencies
SECTION - Second pass update of environment
UpdatingWARNING - [SDE] Failed to fetch NugetDependecy: edk2-acpica-iasl@20200717.0.0<mailto:edk2-acpica-iasl@20200717.0.0>: [Nuget] We failed to install this version 20200717.0.0 of edk2-acpica-iasl
WARNING - [SDE] Failed to fetch NugetDependecy: mu_nasm@2.15.05<mailto:mu_nasm@2.15.05>: [Nuget] We failed to install this version 2.15.05 of mu_nasm
. Done
SECTION - Updated/Verified 3 dependencies
ERROR - We were unable to successfully update 2 dependencies in environment
SECTION - Summary
ERROR - Error
Thanks
Steven Shi
[-- Attachment #1.2: Type: text/html, Size: 29522 bytes --]
[-- Attachment #2: image005.png --]
[-- Type: image/png, Size: 195 bytes --]
[-- Attachment #3: image006.png --]
[-- Type: image/png, Size: 195 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
2021-07-02 1:36 ` Steven Shi
@ 2021-07-02 1:53 ` Bret Barkelew
2021-07-02 2:00 ` Rebecca Cran
1 sibling, 0 replies; 17+ messages in thread
From: Bret Barkelew @ 2021-07-02 1:53 UTC (permalink / raw)
To: Shi, Steven, devel@edk2.groups.io, rebecca@nuviainc.com
[-- Attachment #1.1: Type: text/plain, Size: 15271 bytes --]
Good question! I don’t have much experience with that, but let me see what my Googling skills can turn up.
- Bret
________________________________
From: Shi, Steven <steven.shi@intel.com>
Sent: Thursday, July 1, 2021 6:36:31 PM
To: Bret Barkelew <Bret.Barkelew@microsoft.com>; devel@edk2.groups.io <devel@edk2.groups.io>; rebecca@nuviainc.com <rebecca@nuviainc.com>
Subject: RE: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
I already installed mono, but NuGet.exe still fails to work in my side. I’m behind a proxy but don’t know how to configure the NuGet.exe proxy in .pytool. Below is my verbose error message.
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5 -v
… …
SECTION - Start Invocable Tool
SECTION - Initial update of environment
DEBUG - --- self_describing_environment.update_extdeps()
DEBUG - Verify 'mu_nasm' returning 'False'.
DEBUG - Verify 'edk2-acpica-iasl' returning 'False'.
DEBUG - Creating 5 threads for the SDE update
UpdatingDEBUG - Verify 'gcc_aarch64_linux' returning 'True'.
DEBUG - Verify 'gcc_riscv64_unknown' returning 'True'.
DEBUG - Verify 'mu_nasm' returning 'False'.
DEBUG - Verify 'gcc_arm_linux' returning 'True'.
DEBUG - Verify 'edk2-acpica-iasl' returning 'False'.
DEBUG - Cleaning dependency directory for 'mu_nasm'...
DEBUG - Cleaning dependency directory for 'edk2-acpica-iasl'...
INFO - Cmd to run is: mono /home/jshi19/wksp_efi/unit_test_env/lib/python3.8/site-packages/edk2toolext/bin/NuGet.exe locals global-packages -list
INFO - Cmd to run is: mono /home/jshi19/wksp_efi/unit_test_env/lib/python3.8/site-packages/edk2toolext/bin/NuGet.exe locals global-packages -list
INFO - ------------------------------------------------
INFO - ------------------------------------------------
INFO - --------------Cmd Output Starting---------------
INFO - --------------Cmd Output Starting---------------
INFO - ------------------------------------------------
INFO - ------------------------------------------------
...INFO - Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
INFO - Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
INFO - Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
INFO - Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
INFO - Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
INFO - System.AggregateException: One or more errors occurred. (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) ---> System.IO.FileNotFoundException: Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
INFO - --- End of inner exception stack trace ---
… …
If I run the Cmd manually, I get below error:
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ mono /home/jshi19/wksp_efi/unit_test_env/lib/python3.8/site-packages/edk2toolext/bin/NuGet.exe locals global-packages -list
Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
System.AggregateException: One or more errors occurred. (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) ---> System.IO.FileNotFoundException: Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
--- End of inner exception stack trace ---
at NuGet.CommandLine.Program.Initialize (NuGet.IFileSystem fileSystem, NuGet.CommandLine.IConsole console) [0x000ce] in <ff502667b3fe4a85827635c6fe6b0d0d>:0
at NuGet.CommandLine.Program.MainCore (System.String workingDirectory, System.String[] args) [0x000d8] in <ff502667b3fe4a85827635c6fe6b0d0d>:0
---> (Inner Exception #0) System.IO.FileNotFoundException: Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
File name: 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'<---
---> (Inner Exception #1) System.IO.FileNotFoundException: Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
File name: 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'<---
---> (Inner Exception #2) System.IO.FileNotFoundException: Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
File name: 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'<---
---> (Inner Exception #3) System.IO.FileNotFoundException: Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
File name: 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'<---
---> (Inner Exception #4) System.IO.FileNotFoundException: Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
File name: 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'<---
Thanks
Steven Shi
From: Bret Barkelew <Bret.Barkelew@microsoft.com>
Sent: Friday, July 2, 2021 9:04 AM
To: Shi, Steven <steven.shi@intel.com>; devel@edk2.groups.io; rebecca@nuviainc.com
Subject: RE: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
I believe the Azure DevOps build agents that are used by the EDK2 CI pipeline already have mono installed.
- Bret
From: Shi, Steven<mailto:steven.shi@intel.com>
Sent: Thursday, July 1, 2021 5:54 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Bret Barkelew<mailto:Bret.Barkelew@microsoft.com>; rebecca@nuviainc.com<mailto:rebecca@nuviainc.com>
Subject: RE: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
Hi Bret, Rebecca,
Thanks for the info. Does edk2 CI run the .pytool unit tests in Linux? If yes, how does edk2 CI solve the nuget dependency problem in Linux right now?
Thanks
Steven Shi
From: devel@edk2.groups.io<mailto:devel@edk2.groups.io> <devel@edk2.groups.io<mailto:devel@edk2.groups.io>> On Behalf Of Bret Barkelew via groups.io
Sent: Wednesday, June 30, 2021 1:02 AM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; rebecca@nuviainc.com<mailto:rebecca@nuviainc.com>; Shi, Steven <steven.shi@intel.com<mailto:steven.shi@intel.com>>
Subject: Re: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
Hmmm. If that works well, we might be able to investigate carrying both and picking based on the host OS.
- Bret
From: devel@edk2.groups.io<mailto:devel@edk2.groups.io> <devel@edk2.groups.io<mailto:devel@edk2.groups.io>> on behalf of Rebecca Cran via groups.io <rebecca=nuviainc.com@groups.io<mailto:rebecca=nuviainc.com@groups.io>>
Sent: Tuesday, June 29, 2021 9:42:41 AM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io> <devel@edk2.groups.io<mailto:devel@edk2.groups.io>>; steven.shi@intel.com<mailto:steven.shi@intel.com> <steven.shi@intel.com<mailto:steven.shi@intel.com>>
Subject: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
Your Linux distribution should have a nuget package available (e.g. https://packages.ubuntu.com/hirsute/nuget<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpackages.ubuntu.com%2Fhirsute%2Fnuget&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C7fe13d52d6584572cfda08d93cf9d4e7%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637607865993996088%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=ll8vcIM5CwzfExgCUiRIKMdApfQWPe60stkbBfAViVQ%3D&reserved=0>) but the EDK2 CI is designed to use the copy of nuget installed with the edk2toolext package.
--
Rebecca Cran
On 6/29/21 9:44 AM, Steven Shi wrote:
Cran, Thanks. I’m curious whether the .pytool has native Linux executable binary to run? It looks the edk2 CI framework has more dependency in Linux than Windows.
Thanks
Steven Shi
Intel\IAGS\SFP\FIA (Firmware Infrastructure Automation)
From: devel@edk2.groups.io<mailto:devel@edk2.groups.io> <devel@edk2.groups.io><mailto:devel@edk2.groups.io> On Behalf Of Rebecca Cran
Sent: Tuesday, June 29, 2021 8:13 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Shi, Steven <steven.shi@intel.com><mailto:steven.shi@intel.com>
Subject: Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
NuGet.exe is a .NET assembly/executable, so on Linux you'll need to install Mono (https://www.mono-project.com/<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.mono-project.com%2F&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C7fe13d52d6584572cfda08d93cf9d4e7%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637607865993996088%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=DdwZwXAIpiiHYgREfxmB3Vnx7LHV0iQx6bsnBX%2F5s78%3D&reserved=0>).
--
Rebecca Cran
On 6/28/21 7:45 PM, Steven Shi wrote:
Hello,
I’m interested in the edk2 CI unit tests(https://github.com/tianocore/edk2/tree/master/.pytool<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2F.pytool&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C7fe13d52d6584572cfda08d93cf9d4e7%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637607865994006045%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=7JB9U3dR2OAMu4Ec%2FGs5oksRYAc5BemInssLZUyw3HY%3D&reserved=0>). I tested it in my local. It works well in Windows but does not work in Linux. It looks the NuGet in pytool only has windows version (C:\steven\unit_test_env\Lib\site-packages\edk2toolext\bin\NuGet.exe) but not support the Linux. Below is my detail test steps. Do the edk2 CI unit tests work in Linux?
1. Windows :
c:\steven>py -m venv unit_test_env
c:\steven>cd c:\steven\edk2
c:\steven\edk2>c:\steven\unit_test_env\Scripts\activate.bat
(unit_test_env) c:\steven\edk2>pip install --upgrade -r pip-requirements.txt
(unit_test_env) c:\steven\edk2>stuart_setup -c .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019
(unit_test_env) c:\steven\edk2>stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019
(unit_test_env) c:\steven\edk2>edksetup.bat
(unit_test_env) c:\steven\edk2>edksetup.bat Rebuild
(unit_test_env) c:\steven\edk2>stuart_ci_build -c .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019 -a X64,IA32 -t DEBUG --verbose
…
PROGRESS - Overall Build Status: Success
SECTION - Summary
PROGRESS - Success
1. Linux:
jshi19@ub2-uefi-b01:~/wksp_efi$ python3 -m venv unit_test_env
jshi19@ub2-uefi-b01:~/wksp_efi$ source unit_test_env/bin/activate
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ pip install --upgrade -r pip-requirements.txt
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ make -C BaseTools/
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ stuart_setup -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5
SECTION - Init SDE
WARNING - Using Pip Tools based BaseTools
SECTION - Loading Plugins
SECTION - Start Invocable Tool
SECTION - Initial update of environment
UpdatingWARNING - [SDE] Failed to fetch NugetDependecy: edk2-acpica-iasl@20200717.0.0<mailto:edk2-acpica-iasl@20200717.0.0>: [Nuget] We failed to install this version 20200717.0.0 of edk2-acpica-iasl
WARNING - [SDE] Failed to fetch NugetDependecy: mu_nasm@2.15.05<mailto:mu_nasm@2.15.05>: [Nuget] We failed to install this version 2.15.05 of mu_nasm
. Done
SECTION - Updated/Verified 3 dependencies
SECTION - Second pass update of environment
UpdatingWARNING - [SDE] Failed to fetch NugetDependecy: edk2-acpica-iasl@20200717.0.0<mailto:edk2-acpica-iasl@20200717.0.0>: [Nuget] We failed to install this version 20200717.0.0 of edk2-acpica-iasl
WARNING - [SDE] Failed to fetch NugetDependecy: mu_nasm@2.15.05<mailto:mu_nasm@2.15.05>: [Nuget] We failed to install this version 2.15.05 of mu_nasm
. Done
SECTION - Updated/Verified 3 dependencies
ERROR - We were unable to successfully update 2 dependencies in environment
SECTION - Summary
ERROR - Error
Thanks
Steven Shi
[-- Attachment #1.2: Type: text/html, Size: 26107 bytes --]
[-- Attachment #2: image005.png --]
[-- Type: image/png, Size: 195 bytes --]
[-- Attachment #3: image006.png --]
[-- Type: image/png, Size: 195 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
2021-07-02 1:36 ` Steven Shi
2021-07-02 1:53 ` Bret Barkelew
@ 2021-07-02 2:00 ` Rebecca Cran
2021-07-02 2:43 ` Steven Shi
2021-07-02 2:45 ` Bret Barkelew
1 sibling, 2 replies; 17+ messages in thread
From: Rebecca Cran @ 2021-07-02 2:00 UTC (permalink / raw)
To: devel, steven.shi, Bret Barkelew, rebecca@nuviainc.com
[-- Attachment #1: Type: text/plain, Size: 16292 bytes --]
That means you need to install the various mono assemblies, such as the
libmono-system-net-http4.0-cil package in Ubuntu.
I spent some time chasing down the dependencies last year which got very
frustrating and tedious, but have forgotten which were needed. All I
remember is there were quite a lot.
--
Rebecca Cran
On 7/1/21 7:36 PM, Steven Shi wrote:
>
> I already installed mono, but NuGet.exe still fails to work in my
> side. I’m behind a proxy but don’t know how to configure the NuGet.exe
> proxy in .pytool. Below is my verbose error message.
>
> (unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ stuart_update
> -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5 -v
>
> … …
>
> SECTION - Start Invocable Tool
>
> SECTION - Initial update of environment
>
> DEBUG - --- self_describing_environment.update_extdeps()
>
> DEBUG - Verify 'mu_nasm' returning 'False'.
>
> DEBUG - Verify 'edk2-acpica-iasl' returning 'False'.
>
> DEBUG - Creating 5 threads for the SDE update
>
> UpdatingDEBUG - Verify 'gcc_aarch64_linux' returning 'True'.
>
> DEBUG - Verify 'gcc_riscv64_unknown' returning 'True'.
>
> DEBUG - Verify 'mu_nasm' returning 'False'.
>
> DEBUG - Verify 'gcc_arm_linux' returning 'True'.
>
> DEBUG - Verify 'edk2-acpica-iasl' returning 'False'.
>
> DEBUG - Cleaning dependency directory for 'mu_nasm'...
>
> DEBUG - Cleaning dependency directory for 'edk2-acpica-iasl'...
>
> INFO - Cmd to run is: mono
> /home/jshi19/wksp_efi/unit_test_env/lib/python3.8/site-packages/edk2toolext/bin/NuGet.exe
> locals global-packages -list
>
> INFO - Cmd to run is: mono
> /home/jshi19/wksp_efi/unit_test_env/lib/python3.8/site-packages/edk2toolext/bin/NuGet.exe
> locals global-packages -list
>
> INFO - ------------------------------------------------
>
> INFO - ------------------------------------------------
>
> INFO - --------------Cmd Output Starting---------------
>
> INFO - --------------Cmd Output Starting---------------
>
> INFO - ------------------------------------------------
>
> INFO - ------------------------------------------------
>
> ...INFO - Could not load file or assembly 'System.Net.Http,
> Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or
> one of its dependencies.
>
> INFO - Could not load file or assembly 'System.Net.Http,
> Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or
> one of its dependencies.
>
> INFO - Could not load file or assembly 'System.Net.Http,
> Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or
> one of its dependencies.
>
> INFO - Could not load file or assembly 'System.Net.Http,
> Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or
> one of its dependencies.
>
> INFO - Could not load file or assembly 'System.Net.Http,
> Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or
> one of its dependencies.
>
> INFO - System.AggregateException: One or more errors occurred. (Could
> not load file or assembly 'System.Net.Http, Version=4.0.0.0,
> Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its
> dependencies.) (Could not load file or assembly 'System.Net.Http,
> Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or
> one of its dependencies.) (Could not load file or assembly
> 'System.Net.Http, Version=4.0.0.0, Culture=neutral,
> PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could
> not load file or assembly 'System.Net.Http, Version=4.0.0.0,
> Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its
> dependencies.) (Could not load file or assembly 'System.Net.Http,
> Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or
> one of its dependencies.) ---> System.IO.FileNotFoundException: Could
> not load file or assembly 'System.Net.Http, Version=4.0.0.0,
> Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its
> dependencies.
>
> INFO - --- End of inner exception stack trace ---
>
> … …
>
> If I run the Cmd manually, I get below error:
>
> (unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ mono
> /home/jshi19/wksp_efi/unit_test_env/lib/python3.8/site-packages/edk2toolext/bin/NuGet.exe
> locals global-packages -list
>
> Could not load file or assembly 'System.Net.Http, Version=4.0.0.0,
> Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its
> dependencies.
>
> Could not load file or assembly 'System.Net.Http, Version=4.0.0.0,
> Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its
> dependencies.
>
> Could not load file or assembly 'System.Net.Http, Version=4.0.0.0,
> Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its
> dependencies.
>
> Could not load file or assembly 'System.Net.Http, Version=4.0.0.0,
> Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its
> dependencies.
>
> Could not load file or assembly 'System.Net.Http, Version=4.0.0.0,
> Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its
> dependencies.
>
> System.AggregateException: One or more errors occurred. (Could not
> load file or assembly 'System.Net.Http, Version=4.0.0.0,
> Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its
> dependencies.) (Could not load file or assembly 'System.Net.Http,
> Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or
> one of its dependencies.) (Could not load file or assembly
> 'System.Net.Http, Version=4.0.0.0, Culture=neutral,
> PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could
> not load file or assembly 'System.Net.Http, Version=4.0.0.0,
> Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its
> dependencies.) (Could not load file or assembly 'System.Net.Http,
> Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or
> one of its dependencies.) ---> System.IO.FileNotFoundException: Could
> not load file or assembly 'System.Net.Http, Version=4.0.0.0,
> Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its
> dependencies.
>
> --- End of inner exception stack trace ---
>
> at NuGet.CommandLine.Program.Initialize (NuGet.IFileSystem
> fileSystem, NuGet.CommandLine.IConsole console) [0x000ce] in
> <ff502667b3fe4a85827635c6fe6b0d0d>:0
>
> at NuGet.CommandLine.Program.MainCore (System.String
> workingDirectory, System.String[] args) [0x000d8] in
> <ff502667b3fe4a85827635c6fe6b0d0d>:0
>
> ---> (Inner Exception #0) System.IO.FileNotFoundException: Could not
> load file or assembly 'System.Net.Http, Version=4.0.0.0,
> Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its
> dependencies.
>
> File name: 'System.Net.Http, Version=4.0.0.0, Culture=neutral,
> PublicKeyToken=b03f5f7f11d50a3a'<---
>
> ---> (Inner Exception #1) System.IO.FileNotFoundException: Could not
> load file or assembly 'System.Net.Http, Version=4.0.0.0,
> Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its
> dependencies.
>
> File name: 'System.Net.Http, Version=4.0.0.0, Culture=neutral,
> PublicKeyToken=b03f5f7f11d50a3a'<---
>
> ---> (Inner Exception #2) System.IO.FileNotFoundException: Could not
> load file or assembly 'System.Net.Http, Version=4.0.0.0,
> Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its
> dependencies.
>
> File name: 'System.Net.Http, Version=4.0.0.0, Culture=neutral,
> PublicKeyToken=b03f5f7f11d50a3a'<---
>
> ---> (Inner Exception #3) System.IO.FileNotFoundException: Could not
> load file or assembly 'System.Net.Http, Version=4.0.0.0,
> Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its
> dependencies.
>
> File name: 'System.Net.Http, Version=4.0.0.0, Culture=neutral,
> PublicKeyToken=b03f5f7f11d50a3a'<---
>
> ---> (Inner Exception #4) System.IO.FileNotFoundException: Could not
> load file or assembly 'System.Net.Http, Version=4.0.0.0,
> Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its
> dependencies.
>
> File name: 'System.Net.Http, Version=4.0.0.0, Culture=neutral,
> PublicKeyToken=b03f5f7f11d50a3a'<---
>
> Thanks
>
> *Steven Shi***
>
> **
>
> *From:* Bret Barkelew <Bret.Barkelew@microsoft.com>
> *Sent:* Friday, July 2, 2021 9:04 AM
> *To:* Shi, Steven <steven.shi@intel.com>; devel@edk2.groups.io;
> rebecca@nuviainc.com
> *Subject:* RE: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests
> work in Linux?
>
> I believe the Azure DevOps build agents that are used by the EDK2 CI
> pipeline already have mono installed.
>
> - Bret
>
> *From: *Shi, Steven <mailto:steven.shi@intel.com>
> *Sent: *Thursday, July 1, 2021 5:54 PM
> *To: *devel@edk2.groups.io <mailto:devel@edk2.groups.io>; Bret
> Barkelew <mailto:Bret.Barkelew@microsoft.com>; rebecca@nuviainc.com
> <mailto:rebecca@nuviainc.com>
> *Subject: *RE: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests
> work in Linux?
>
> Hi Bret, Rebecca,
>
> Thanks for the info. Does edk2 CI run the .pytool unit tests in Linux?
> If yes, how does edk2 CI solve the nuget dependency problem in Linux
> right now?
>
> Thanks
>
> *Steven Shi*
>
> **
>
> *From:* devel@edk2.groups.io <mailto:devel@edk2.groups.io>
> <devel@edk2.groups.io <mailto:devel@edk2.groups.io>> *On Behalf Of
> *Bret Barkelew via groups.io
> *Sent:* Wednesday, June 30, 2021 1:02 AM
> *To:* devel@edk2.groups.io <mailto:devel@edk2.groups.io>;
> rebecca@nuviainc.com <mailto:rebecca@nuviainc.com>; Shi, Steven
> <steven.shi@intel.com <mailto:steven.shi@intel.com>>
> *Subject:* Re: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests
> work in Linux?
>
> Hmmm. If that works well, we might be able to investigate carrying
> both and picking based on the host OS.
>
> - Bret
>
> *From:*devel@edk2.groups.io <mailto:devel@edk2.groups.io>
> <devel@edk2.groups.io <mailto:devel@edk2.groups.io>> on behalf of
> Rebecca Cran via groups.io <rebecca=nuviainc.com@groups.io
> <mailto:rebecca=nuviainc.com@groups.io>>
> *Sent:* Tuesday, June 29, 2021 9:42:41 AM
> *To:* devel@edk2.groups.io <mailto:devel@edk2.groups.io>
> <devel@edk2.groups.io <mailto:devel@edk2.groups.io>>;
> steven.shi@intel.com <mailto:steven.shi@intel.com>
> <steven.shi@intel.com <mailto:steven.shi@intel.com>>
> *Subject:* [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work
> in Linux?
>
> Your Linux distribution should have a nuget package available (e.g.
> https://packages.ubuntu.com/hirsute/nuget
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpackages.ubuntu.com%2Fhirsute%2Fnuget&data=04%7C01%7Cbret.barkelew%40microsoft.com%7C28bf7b813a9a4d6e365508d93cf3fd8a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637607840913328150%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Z7vCN9JUmx1kQks%2FuWKjcnMRo7koWoyxOET0bnSbN6U%3D&reserved=0>)
> but the EDK2 CI is designed to use the copy of nuget installed with
> the edk2toolext package.
>
> --
> Rebecca Cran
>
> On 6/29/21 9:44 AM, Steven Shi wrote:
>
> Cran, Thanks. I’m curious whether the .pytool has native Linux
> executable binary to run? It looks the edk2 CI framework has more
> dependency in Linux than Windows.
>
> Thanks
>
> *Steven Shi*
>
> *Intel\IAGS\SFP\FIA (Firmware Infrastructure Automation)*
>
> **
>
> *From:* devel@edk2.groups.io <mailto:devel@edk2.groups.io>
> <devel@edk2.groups.io> <mailto:devel@edk2.groups.io> *On Behalf Of
> * Rebecca Cran
> *Sent:* Tuesday, June 29, 2021 8:13 PM
> *To:* devel@edk2.groups.io <mailto:devel@edk2.groups.io>; Shi,
> Steven <steven.shi@intel.com> <mailto:steven.shi@intel.com>
> *Subject:* Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
>
> NuGet.exe is a .NET assembly/executable, so on Linux you'll need
> to install Mono (https://www.mono-project.com/
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.mono-project.com%2F&data=04%7C01%7Cbret.barkelew%40microsoft.com%7C28bf7b813a9a4d6e365508d93cf3fd8a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637607840913338106%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=uPkpInnWMx%2FP35GUXlbffQwxbLya9DDDegyLJvtuScU%3D&reserved=0>).
>
> --
>
> Rebecca Cran
>
> On 6/28/21 7:45 PM, Steven Shi wrote:
>
> Hello,
>
> I’m interested in the edk2 CI unit
> tests(https://github.com/tianocore/edk2/tree/master/.pytool
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2F.pytool&data=04%7C01%7Cbret.barkelew%40microsoft.com%7C28bf7b813a9a4d6e365508d93cf3fd8a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637607840913338106%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=v%2FqnYohY9bDCIAC8ZmqPkV3RvhaIuyQEXfkY9JNyIqs%3D&reserved=0>).
> I tested it in my local. It works well in Windows but does not
> work in Linux. It looks the NuGet in pytool only has windows
> version
> (C:\steven\unit_test_env\Lib\site-packages\edk2toolext\bin\NuGet.exe)
> but not support the Linux. Below is my detail test steps. Do
> the edk2 CI unit tests work in Linux?
>
> 1. Windows :
>
> c:\steven>py -m venv unit_test_env
>
> c:\steven>cd c:\steven\edk2
>
> c:\steven\edk2>c:\steven\unit_test_env\Scripts\activate.bat
>
> (unit_test_env) c:\steven\edk2>pip install --upgrade -r
> pip-requirements.txt
>
> (unit_test_env) c:\steven\edk2>stuart_setup -c
> .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019
>
> (unit_test_env) c:\steven\edk2>stuart_update -c
> .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019
>
> (unit_test_env) c:\steven\edk2>edksetup.bat
>
> (unit_test_env) c:\steven\edk2>edksetup.bat Rebuild
>
> (unit_test_env) c:\steven\edk2>stuart_ci_build -c
> .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019 -a X64,IA32 -t
> DEBUG --verbose
>
> …
>
> PROGRESS - Overall Build Status: Success
>
> SECTION - Summary
>
> PROGRESS - Success
>
> 2. Linux:
>
> jshi19@ub2-uefi-b01:~/wksp_efi$ python3 -m venv unit_test_env
>
> jshi19@ub2-uefi-b01:~/wksp_efi$ source unit_test_env/bin/activate
>
> (unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ pip
> install --upgrade -r pip-requirements.txt
>
> (unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ make -C
> BaseTools/
>
> (unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$
> stuart_setup -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5
>
> (unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$
> stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5
>
> SECTION - Init SDE
>
> WARNING - Using Pip Tools based BaseTools
>
> SECTION - Loading Plugins
>
> SECTION - Start Invocable Tool
>
> SECTION - Initial update of environment
>
> UpdatingWARNING - [SDE] Failed to fetch NugetDependecy:
> edk2-acpica-iasl@20200717.0.0
> <mailto:edk2-acpica-iasl@20200717.0.0>: [Nuget] We failed to
> install this version 20200717.0.0 of edk2-acpica-iasl
>
> WARNING - [SDE] Failed to fetch NugetDependecy:
> mu_nasm@2.15.05 <mailto:mu_nasm@2.15.05>: [Nuget] We failed to
> install this version 2.15.05 of mu_nasm
>
> . Done
>
> SECTION - Updated/Verified 3 dependencies
>
> SECTION - Second pass update of environment
>
> UpdatingWARNING - [SDE] Failed to fetch NugetDependecy:
> edk2-acpica-iasl@20200717.0.0
> <mailto:edk2-acpica-iasl@20200717.0.0>: [Nuget] We failed to
> install this version 20200717.0.0 of edk2-acpica-iasl
>
> WARNING - [SDE] Failed to fetch NugetDependecy:
> mu_nasm@2.15.05 <mailto:mu_nasm@2.15.05>: [Nuget] We failed to
> install this version 2.15.05 of mu_nasm
>
> . Done
>
> SECTION - Updated/Verified 3 dependencies
>
> ERROR - We were unable to successfully update 2 dependencies
> in environment
>
> SECTION - Summary
>
> ERROR - Error
>
> Thanks
>
> *Steven Shi*
>
> **
>
> **
>
>
[-- Attachment #2.1: Type: text/html, Size: 35353 bytes --]
[-- Attachment #2.2: image005.png --]
[-- Type: image/png, Size: 195 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
2021-07-02 2:00 ` Rebecca Cran
@ 2021-07-02 2:43 ` Steven Shi
2021-07-02 7:23 ` Steven Shi
2021-07-02 2:45 ` Bret Barkelew
1 sibling, 1 reply; 17+ messages in thread
From: Steven Shi @ 2021-07-02 2:43 UTC (permalink / raw)
To: Rebecca Cran, devel@edk2.groups.io, Bret Barkelew,
rebecca@nuviainc.com
[-- Attachment #1.1: Type: text/plain, Size: 20392 bytes --]
Thank you to Rebecca. After install the libmono-system-net-http4.0-cil and mono-devel pkg as below, the 'mono /home/jshi19/wksp_efi/unit_test_env/lib/python3.8/site-packages/edk2toolext/bin/NuGet.exe locals global-packages -list' pass.
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ sudo apt-get install libmono-system-net-http4.0-cil
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ sudo apt-get install mono-devel
But now there are new failures show up when install edk2-acpica-iasl:
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5 -v
... ...
INFO - Cmd to run is: mono /home/jshi19/wksp_efi/unit_test_env/lib/python3.8/site-packages/edk2toolext/bin/NuGet.exe locals global-packages -list
INFO - Cmd to run is: mono /home/jshi19/wksp_efi/unit_test_env/lib/python3.8/site-packages/edk2toolext/bin/NuGet.exe locals global-packages -list
INFO - ------------------------------------------------
INFO - ------------------------------------------------
INFO - --------------Cmd Output Starting---------------
INFO - --------------Cmd Output Starting---------------
INFO - ------------------------------------------------
INFO - ------------------------------------------------
...INFO - global-packages: /home/jshi19/.nuget/packages/
INFO - global-packages: /home/jshi19/.nuget/packages/
INFO - ------------------------------------------------
INFO - --------------Cmd Output Finished---------------
INFO - ------------------------------------------------
INFO - --------- Running Time (mm:ss): 00:00 ----------
INFO - --------------Cmd Output Finished---------------
INFO - ----------- Return Code: 0x00000000 ------------
INFO - --------- Running Time (mm:ss): 00:00 ----------
INFO - ------------------------------------------------
INFO - ----------- Return Code: 0x00000000 ------------
INFO - Could not determine Nuget global packages cache location.
INFO - ------------------------------------------------
INFO - Cmd to run is: mono /home/jshi19/wksp_efi/unit_test_env/lib/python3.8/site-packages/edk2toolext/bin/NuGet.exe install edk2-acpica-iasl -Source https://pkgs.dev.azure.com/projectmu/acpica/_packaging/mu_iasl/nuget/v3/index.json -ExcludeVersion -NonInteractive -Version 20200717.0.0 -Verbosity detailed -OutputDirectory "/home/jshi19/wksp_efi/edk2-2/BaseTools/Bin/edk2-acpica-iasl_extdep_temp"
INFO - Could not determine Nuget global packages cache location.
INFO - ------------------------------------------------
INFO - --------------Cmd Output Starting---------------
INFO - Cmd to run is: mono /home/jshi19/wksp_efi/unit_test_env/lib/python3.8/site-packages/edk2toolext/bin/NuGet.exe install mu_nasm -Source https://api.nuget.org/v3/index.json -ExcludeVersion -NonInteractive -Version 2.15.05 -Verbosity detailed -OutputDirectory "/home/jshi19/wksp_efi/edk2-2/BaseTools/Bin/mu_nasm_extdep_temp"
INFO - ------------------------------------------------
INFO - ------------------------------------------------
INFO - --------------Cmd Output Starting---------------
INFO - ------------------------------------------------
INFO - NuGet Version: 5.3.1.6268
INFO - NuGet Version: 5.3.1.6268
INFO - Feeds used:
INFO - https://api.nuget.org/v3/index.json
INFO -
INFO -
INFO -
INFO - Feeds used:
INFO - https://pkgs.dev.azure.com/projectmu/acpica/_packaging/mu_iasl/nuget/v3/index.json
INFO -
INFO - Attempting to gather dependency information for package 'mu_nasm.2.15.5' with respect to project '/home/jshi19/wksp_efi/edk2-2/BaseTools/Bin/mu_nasm_extdep_temp', targeting 'Any,Version=v0.0'
INFO -
INFO -
INFO - Attempting to gather dependency information for package 'edk2-acpica-iasl.20200717.0.0' with respect to project '/home/jshi19/wksp_efi/edk2-2/BaseTools/Bin/edk2-acpica-iasl_extdep_temp', targeting 'Any,Version=v0.0'
INFO - Unable to load the service index for source https://pkgs.dev.azure.com/projectmu/acpica/_packaging/mu_iasl/nuget/v3/index.json.
INFO - GSSAPI operation failed with error - An invalid name was supplied (Configuration file does not specify default realm).
INFO - System.InvalidOperationException: Unable to load the service index for source https://pkgs.dev.azure.com/projectmu/acpica/_packaging/mu_iasl/nuget/v3/index.json. ---> NuGet.Protocol.Core.Types.FatalProtocolException: Unable to load the service index for source https://pkgs.dev.azure.com/projectmu/acpica/_packaging/mu_iasl/nuget/v3/index.json. ---> System.ComponentModel.Win32Exception: GSSAPI operation failed with error - An invalid name was supplied (Configuration file does not specify default realm).
INFO - at System.Net.Security.NegotiateStreamPal.AcquireCredentialsHandle (System.String package, System.Boolean isServer, System.Net.NetworkCredential credential) [0x000a3] in <a85c1a570f9a4f9f9c3d2cfa5504e34f>:0
... ...
Thanks
Steven Shi
Intel\IAGS\SFP\FIA (Firmware Infrastructure Automation)
From: Rebecca Cran <rebecca@bsdio.com>
Sent: Friday, July 2, 2021 10:00 AM
To: devel@edk2.groups.io; Shi, Steven <steven.shi@intel.com>; Bret Barkelew <Bret.Barkelew@microsoft.com>; rebecca@nuviainc.com
Subject: Re: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
That means you need to install the various mono assemblies, such as the libmono-system-net-http4.0-cil package in Ubuntu.
I spent some time chasing down the dependencies last year which got very frustrating and tedious, but have forgotten which were needed. All I remember is there were quite a lot.
--
Rebecca Cran
On 7/1/21 7:36 PM, Steven Shi wrote:
I already installed mono, but NuGet.exe still fails to work in my side. I'm behind a proxy but don't know how to configure the NuGet.exe proxy in .pytool. Below is my verbose error message.
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5 -v
... ...
SECTION - Start Invocable Tool
SECTION - Initial update of environment
DEBUG - --- self_describing_environment.update_extdeps()
DEBUG - Verify 'mu_nasm' returning 'False'.
DEBUG - Verify 'edk2-acpica-iasl' returning 'False'.
DEBUG - Creating 5 threads for the SDE update
UpdatingDEBUG - Verify 'gcc_aarch64_linux' returning 'True'.
DEBUG - Verify 'gcc_riscv64_unknown' returning 'True'.
DEBUG - Verify 'mu_nasm' returning 'False'.
DEBUG - Verify 'gcc_arm_linux' returning 'True'.
DEBUG - Verify 'edk2-acpica-iasl' returning 'False'.
DEBUG - Cleaning dependency directory for 'mu_nasm'...
DEBUG - Cleaning dependency directory for 'edk2-acpica-iasl'...
INFO - Cmd to run is: mono /home/jshi19/wksp_efi/unit_test_env/lib/python3.8/site-packages/edk2toolext/bin/NuGet.exe locals global-packages -list
INFO - Cmd to run is: mono /home/jshi19/wksp_efi/unit_test_env/lib/python3.8/site-packages/edk2toolext/bin/NuGet.exe locals global-packages -list
INFO - ------------------------------------------------
INFO - ------------------------------------------------
INFO - --------------Cmd Output Starting---------------
INFO - --------------Cmd Output Starting---------------
INFO - ------------------------------------------------
INFO - ------------------------------------------------
...INFO - Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
INFO - Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
INFO - Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
INFO - Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
INFO - Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
INFO - System.AggregateException: One or more errors occurred. (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) ---> System.IO.FileNotFoundException: Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
INFO - --- End of inner exception stack trace ---
... ...
If I run the Cmd manually, I get below error:
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ mono /home/jshi19/wksp_efi/unit_test_env/lib/python3.8/site-packages/edk2toolext/bin/NuGet.exe locals global-packages -list
Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
System.AggregateException: One or more errors occurred. (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) ---> System.IO.FileNotFoundException: Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
--- End of inner exception stack trace ---
at NuGet.CommandLine.Program.Initialize (NuGet.IFileSystem fileSystem, NuGet.CommandLine.IConsole console) [0x000ce] in <ff502667b3fe4a85827635c6fe6b0d0d>:0
at NuGet.CommandLine.Program.MainCore (System.String workingDirectory, System.String[] args) [0x000d8] in <ff502667b3fe4a85827635c6fe6b0d0d>:0
---> (Inner Exception #0) System.IO.FileNotFoundException: Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
File name: 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'<---
---> (Inner Exception #1) System.IO.FileNotFoundException: Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
File name: 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'<---
---> (Inner Exception #2) System.IO.FileNotFoundException: Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
File name: 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'<---
---> (Inner Exception #3) System.IO.FileNotFoundException: Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
File name: 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'<---
---> (Inner Exception #4) System.IO.FileNotFoundException: Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
File name: 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'<---
Thanks
Steven Shi
From: Bret Barkelew <Bret.Barkelew@microsoft.com><mailto:Bret.Barkelew@microsoft.com>
Sent: Friday, July 2, 2021 9:04 AM
To: Shi, Steven <steven.shi@intel.com><mailto:steven.shi@intel.com>; devel@edk2.groups.io<mailto:devel@edk2.groups.io>; rebecca@nuviainc.com<mailto:rebecca@nuviainc.com>
Subject: RE: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
I believe the Azure DevOps build agents that are used by the EDK2 CI pipeline already have mono installed.
- Bret
From: Shi, Steven<mailto:steven.shi@intel.com>
Sent: Thursday, July 1, 2021 5:54 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Bret Barkelew<mailto:Bret.Barkelew@microsoft.com>; rebecca@nuviainc.com<mailto:rebecca@nuviainc.com>
Subject: RE: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
Hi Bret, Rebecca,
Thanks for the info. Does edk2 CI run the .pytool unit tests in Linux? If yes, how does edk2 CI solve the nuget dependency problem in Linux right now?
Thanks
Steven Shi
From: devel@edk2.groups.io<mailto:devel@edk2.groups.io> <devel@edk2.groups.io<mailto:devel@edk2.groups.io>> On Behalf Of Bret Barkelew via groups.io
Sent: Wednesday, June 30, 2021 1:02 AM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; rebecca@nuviainc.com<mailto:rebecca@nuviainc.com>; Shi, Steven <steven.shi@intel.com<mailto:steven.shi@intel.com>>
Subject: Re: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
Hmmm. If that works well, we might be able to investigate carrying both and picking based on the host OS.
- Bret
From: devel@edk2.groups.io<mailto:devel@edk2.groups.io> <devel@edk2.groups.io<mailto:devel@edk2.groups.io>> on behalf of Rebecca Cran via groups.io <rebecca=nuviainc.com@groups.io<mailto:rebecca=nuviainc.com@groups.io>>
Sent: Tuesday, June 29, 2021 9:42:41 AM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io> <devel@edk2.groups.io<mailto:devel@edk2.groups.io>>; steven.shi@intel.com<mailto:steven.shi@intel.com> <steven.shi@intel.com<mailto:steven.shi@intel.com>>
Subject: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
Your Linux distribution should have a nuget package available (e.g. https://packages.ubuntu.com/hirsute/nuget<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpackages.ubuntu.com%2Fhirsute%2Fnuget&data=04%7C01%7Cbret.barkelew%40microsoft.com%7C28bf7b813a9a4d6e365508d93cf3fd8a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637607840913328150%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Z7vCN9JUmx1kQks%2FuWKjcnMRo7koWoyxOET0bnSbN6U%3D&reserved=0>) but the EDK2 CI is designed to use the copy of nuget installed with the edk2toolext package.
--
Rebecca Cran
On 6/29/21 9:44 AM, Steven Shi wrote:
Cran, Thanks. I'm curious whether the .pytool has native Linux executable binary to run? It looks the edk2 CI framework has more dependency in Linux than Windows.
Thanks
Steven Shi
Intel\IAGS\SFP\FIA (Firmware Infrastructure Automation)
From: devel@edk2.groups.io<mailto:devel@edk2.groups.io> <devel@edk2.groups.io><mailto:devel@edk2.groups.io> On Behalf Of Rebecca Cran
Sent: Tuesday, June 29, 2021 8:13 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Shi, Steven <steven.shi@intel.com><mailto:steven.shi@intel.com>
Subject: Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
NuGet.exe is a .NET assembly/executable, so on Linux you'll need to install Mono (https://www.mono-project.com/<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.mono-project.com%2F&data=04%7C01%7Cbret.barkelew%40microsoft.com%7C28bf7b813a9a4d6e365508d93cf3fd8a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637607840913338106%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=uPkpInnWMx%2FP35GUXlbffQwxbLya9DDDegyLJvtuScU%3D&reserved=0>).
--
Rebecca Cran
On 6/28/21 7:45 PM, Steven Shi wrote:
Hello,
I'm interested in the edk2 CI unit tests(https://github.com/tianocore/edk2/tree/master/.pytool<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2F.pytool&data=04%7C01%7Cbret.barkelew%40microsoft.com%7C28bf7b813a9a4d6e365508d93cf3fd8a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637607840913338106%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=v%2FqnYohY9bDCIAC8ZmqPkV3RvhaIuyQEXfkY9JNyIqs%3D&reserved=0>). I tested it in my local. It works well in Windows but does not work in Linux. It looks the NuGet in pytool only has windows version (C:\steven\unit_test_env\Lib\site-packages\edk2toolext\bin\NuGet.exe) but not support the Linux. Below is my detail test steps. Do the edk2 CI unit tests work in Linux?
1. Windows :
c:\steven>py -m venv unit_test_env
c:\steven>cd c:\steven\edk2
c:\steven\edk2>c:\steven\unit_test_env\Scripts\activate.bat
(unit_test_env) c:\steven\edk2>pip install --upgrade -r pip-requirements.txt
(unit_test_env) c:\steven\edk2>stuart_setup -c .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019
(unit_test_env) c:\steven\edk2>stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019
(unit_test_env) c:\steven\edk2>edksetup.bat
(unit_test_env) c:\steven\edk2>edksetup.bat Rebuild
(unit_test_env) c:\steven\edk2>stuart_ci_build -c .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019 -a X64,IA32 -t DEBUG --verbose
...
PROGRESS - Overall Build Status: Success
SECTION - Summary
PROGRESS - Success
1. Linux:
jshi19@ub2-uefi-b01:~/wksp_efi$ python3 -m venv unit_test_env
jshi19@ub2-uefi-b01:~/wksp_efi$ source unit_test_env/bin/activate
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ pip install --upgrade -r pip-requirements.txt
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ make -C BaseTools/
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ stuart_setup -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5
SECTION - Init SDE
WARNING - Using Pip Tools based BaseTools
SECTION - Loading Plugins
SECTION - Start Invocable Tool
SECTION - Initial update of environment
UpdatingWARNING - [SDE] Failed to fetch NugetDependecy: edk2-acpica-iasl@20200717.0.0<mailto:edk2-acpica-iasl@20200717.0.0>: [Nuget] We failed to install this version 20200717.0.0 of edk2-acpica-iasl
WARNING - [SDE] Failed to fetch NugetDependecy: mu_nasm@2.15.05<mailto:mu_nasm@2.15.05>: [Nuget] We failed to install this version 2.15.05 of mu_nasm
. Done
SECTION - Updated/Verified 3 dependencies
SECTION - Second pass update of environment
UpdatingWARNING - [SDE] Failed to fetch NugetDependecy: edk2-acpica-iasl@20200717.0.0<mailto:edk2-acpica-iasl@20200717.0.0>: [Nuget] We failed to install this version 20200717.0.0 of edk2-acpica-iasl
WARNING - [SDE] Failed to fetch NugetDependecy: mu_nasm@2.15.05<mailto:mu_nasm@2.15.05>: [Nuget] We failed to install this version 2.15.05 of mu_nasm
. Done
SECTION - Updated/Verified 3 dependencies
ERROR - We were unable to successfully update 2 dependencies in environment
SECTION - Summary
ERROR - Error
Thanks
Steven Shi
[-- Attachment #1.2: Type: text/html, Size: 39372 bytes --]
[-- Attachment #2: image001.png --]
[-- Type: image/png, Size: 195 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
2021-07-02 2:00 ` Rebecca Cran
2021-07-02 2:43 ` Steven Shi
@ 2021-07-02 2:45 ` Bret Barkelew
1 sibling, 0 replies; 17+ messages in thread
From: Bret Barkelew @ 2021-07-02 2:45 UTC (permalink / raw)
To: Rebecca Cran, devel@edk2.groups.io, steven.shi@intel.com,
rebecca@nuviainc.com
[-- Attachment #1.1: Type: text/plain, Size: 15595 bytes --]
I swear I got mine working with just an ‘apt install mono-devel’. But YMMV.
- Bret
From: Rebecca Cran<mailto:rebecca@bsdio.com>
Sent: Thursday, July 1, 2021 7:00 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; steven.shi@intel.com<mailto:steven.shi@intel.com>; Bret Barkelew<mailto:Bret.Barkelew@microsoft.com>; rebecca@nuviainc.com<mailto:rebecca@nuviainc.com>
Subject: Re: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
That means you need to install the various mono assemblies, such as the libmono-system-net-http4.0-cil package in Ubuntu.
I spent some time chasing down the dependencies last year which got very frustrating and tedious, but have forgotten which were needed. All I remember is there were quite a lot.
--
Rebecca Cran
On 7/1/21 7:36 PM, Steven Shi wrote:
I already installed mono, but NuGet.exe still fails to work in my side. I’m behind a proxy but don’t know how to configure the NuGet.exe proxy in .pytool. Below is my verbose error message.
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5 -v
… …
SECTION - Start Invocable Tool
SECTION - Initial update of environment
DEBUG - --- self_describing_environment.update_extdeps()
DEBUG - Verify 'mu_nasm' returning 'False'.
DEBUG - Verify 'edk2-acpica-iasl' returning 'False'.
DEBUG - Creating 5 threads for the SDE update
UpdatingDEBUG - Verify 'gcc_aarch64_linux' returning 'True'.
DEBUG - Verify 'gcc_riscv64_unknown' returning 'True'.
DEBUG - Verify 'mu_nasm' returning 'False'.
DEBUG - Verify 'gcc_arm_linux' returning 'True'.
DEBUG - Verify 'edk2-acpica-iasl' returning 'False'.
DEBUG - Cleaning dependency directory for 'mu_nasm'...
DEBUG - Cleaning dependency directory for 'edk2-acpica-iasl'...
INFO - Cmd to run is: mono /home/jshi19/wksp_efi/unit_test_env/lib/python3.8/site-packages/edk2toolext/bin/NuGet.exe locals global-packages -list
INFO - Cmd to run is: mono /home/jshi19/wksp_efi/unit_test_env/lib/python3.8/site-packages/edk2toolext/bin/NuGet.exe locals global-packages -list
INFO - ------------------------------------------------
INFO - ------------------------------------------------
INFO - --------------Cmd Output Starting---------------
INFO - --------------Cmd Output Starting---------------
INFO - ------------------------------------------------
INFO - ------------------------------------------------
...INFO - Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
INFO - Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
INFO - Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
INFO - Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
INFO - Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
INFO - System.AggregateException: One or more errors occurred. (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) ---> System.IO.FileNotFoundException: Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
INFO - --- End of inner exception stack trace ---
… …
If I run the Cmd manually, I get below error:
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ mono /home/jshi19/wksp_efi/unit_test_env/lib/python3.8/site-packages/edk2toolext/bin/NuGet.exe locals global-packages -list
Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
System.AggregateException: One or more errors occurred. (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) ---> System.IO.FileNotFoundException: Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
--- End of inner exception stack trace ---
at NuGet.CommandLine.Program.Initialize (NuGet.IFileSystem fileSystem, NuGet.CommandLine.IConsole console) [0x000ce] in <ff502667b3fe4a85827635c6fe6b0d0d>:0
at NuGet.CommandLine.Program.MainCore (System.String workingDirectory, System.String[] args) [0x000d8] in <ff502667b3fe4a85827635c6fe6b0d0d>:0
---> (Inner Exception #0) System.IO.FileNotFoundException: Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
File name: 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'<---
---> (Inner Exception #1) System.IO.FileNotFoundException: Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
File name: 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'<---
---> (Inner Exception #2) System.IO.FileNotFoundException: Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
File name: 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'<---
---> (Inner Exception #3) System.IO.FileNotFoundException: Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
File name: 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'<---
---> (Inner Exception #4) System.IO.FileNotFoundException: Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
File name: 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'<---
Thanks
Steven Shi
From: Bret Barkelew <Bret.Barkelew@microsoft.com><mailto:Bret.Barkelew@microsoft.com>
Sent: Friday, July 2, 2021 9:04 AM
To: Shi, Steven <steven.shi@intel.com><mailto:steven.shi@intel.com>; devel@edk2.groups.io<mailto:devel@edk2.groups.io>; rebecca@nuviainc.com<mailto:rebecca@nuviainc.com>
Subject: RE: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
I believe the Azure DevOps build agents that are used by the EDK2 CI pipeline already have mono installed.
- Bret
From: Shi, Steven<mailto:steven.shi@intel.com>
Sent: Thursday, July 1, 2021 5:54 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Bret Barkelew<mailto:Bret.Barkelew@microsoft.com>; rebecca@nuviainc.com<mailto:rebecca@nuviainc.com>
Subject: RE: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
Hi Bret, Rebecca,
Thanks for the info. Does edk2 CI run the .pytool unit tests in Linux? If yes, how does edk2 CI solve the nuget dependency problem in Linux right now?
Thanks
Steven Shi
From: devel@edk2.groups.io<mailto:devel@edk2.groups.io> <devel@edk2.groups.io<mailto:devel@edk2.groups.io>> On Behalf Of Bret Barkelew via groups.io
Sent: Wednesday, June 30, 2021 1:02 AM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; rebecca@nuviainc.com<mailto:rebecca@nuviainc.com>; Shi, Steven <steven.shi@intel.com<mailto:steven.shi@intel.com>>
Subject: Re: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
Hmmm. If that works well, we might be able to investigate carrying both and picking based on the host OS.
- Bret
From: devel@edk2.groups.io<mailto:devel@edk2.groups.io> <devel@edk2.groups.io<mailto:devel@edk2.groups.io>> on behalf of Rebecca Cran via groups.io <rebecca=nuviainc.com@groups.io<mailto:rebecca=nuviainc.com@groups.io>>
Sent: Tuesday, June 29, 2021 9:42:41 AM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io> <devel@edk2.groups.io<mailto:devel@edk2.groups.io>>; steven.shi@intel.com<mailto:steven.shi@intel.com> <steven.shi@intel.com<mailto:steven.shi@intel.com>>
Subject: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
Your Linux distribution should have a nuget package available (e.g. https://packages.ubuntu.com/hirsute/nuget<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpackages.ubuntu.com%2Fhirsute%2Fnuget&data=04%7C01%7CBret.Barkelew%40microsoft.com%7Ce39af464a2924dd237d808d93cfd208f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637607880161863629%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=YarnXT5ifhQuNdwrczr1JvrEGhAUAotvIITeXGsR7SQ%3D&reserved=0>) but the EDK2 CI is designed to use the copy of nuget installed with the edk2toolext package.
--
Rebecca Cran
On 6/29/21 9:44 AM, Steven Shi wrote:
Cran, Thanks. I’m curious whether the .pytool has native Linux executable binary to run? It looks the edk2 CI framework has more dependency in Linux than Windows.
Thanks
Steven Shi
Intel\IAGS\SFP\FIA (Firmware Infrastructure Automation)
From: devel@edk2.groups.io<mailto:devel@edk2.groups.io> <devel@edk2.groups.io><mailto:devel@edk2.groups.io> On Behalf Of Rebecca Cran
Sent: Tuesday, June 29, 2021 8:13 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Shi, Steven <steven.shi@intel.com><mailto:steven.shi@intel.com>
Subject: Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
NuGet.exe is a .NET assembly/executable, so on Linux you'll need to install Mono (https://www.mono-project.com/<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.mono-project.com%2F&data=04%7C01%7CBret.Barkelew%40microsoft.com%7Ce39af464a2924dd237d808d93cfd208f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637607880161863629%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=GPZm1%2FZo8wSIyFCHttTonQR9M1L3FegwL88zqs%2BMVBE%3D&reserved=0>).
--
Rebecca Cran
On 6/28/21 7:45 PM, Steven Shi wrote:
Hello,
I’m interested in the edk2 CI unit tests(https://github.com/tianocore/edk2/tree/master/.pytool<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2F.pytool&data=04%7C01%7CBret.Barkelew%40microsoft.com%7Ce39af464a2924dd237d808d93cfd208f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637607880161873589%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=7oljD5dqqWpKtT0EBeLafVYp0E%2BpNIY7RwyMTISXLtY%3D&reserved=0>). I tested it in my local. It works well in Windows but does not work in Linux. It looks the NuGet in pytool only has windows version (C:\steven\unit_test_env\Lib\site-packages\edk2toolext\bin\NuGet.exe) but not support the Linux. Below is my detail test steps. Do the edk2 CI unit tests work in Linux?
1. Windows :
c:\steven>py -m venv unit_test_env
c:\steven>cd c:\steven\edk2
c:\steven\edk2>c:\steven\unit_test_env\Scripts\activate.bat
(unit_test_env) c:\steven\edk2>pip install --upgrade -r pip-requirements.txt
(unit_test_env) c:\steven\edk2>stuart_setup -c .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019
(unit_test_env) c:\steven\edk2>stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019
(unit_test_env) c:\steven\edk2>edksetup.bat
(unit_test_env) c:\steven\edk2>edksetup.bat Rebuild
(unit_test_env) c:\steven\edk2>stuart_ci_build -c .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019 -a X64,IA32 -t DEBUG --verbose
…
PROGRESS - Overall Build Status: Success
SECTION - Summary
PROGRESS - Success
1. Linux:
jshi19@ub2-uefi-b01:~/wksp_efi$ python3 -m venv unit_test_env
jshi19@ub2-uefi-b01:~/wksp_efi$ source unit_test_env/bin/activate
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ pip install --upgrade -r pip-requirements.txt
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ make -C BaseTools/
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ stuart_setup -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5
SECTION - Init SDE
WARNING - Using Pip Tools based BaseTools
SECTION - Loading Plugins
SECTION - Start Invocable Tool
SECTION - Initial update of environment
UpdatingWARNING - [SDE] Failed to fetch NugetDependecy: edk2-acpica-iasl@20200717.0.0<mailto:edk2-acpica-iasl@20200717.0.0>: [Nuget] We failed to install this version 20200717.0.0 of edk2-acpica-iasl
WARNING - [SDE] Failed to fetch NugetDependecy: mu_nasm@2.15.05<mailto:mu_nasm@2.15.05>: [Nuget] We failed to install this version 2.15.05 of mu_nasm
. Done
SECTION - Updated/Verified 3 dependencies
SECTION - Second pass update of environment
UpdatingWARNING - [SDE] Failed to fetch NugetDependecy: edk2-acpica-iasl@20200717.0.0<mailto:edk2-acpica-iasl@20200717.0.0>: [Nuget] We failed to install this version 20200717.0.0 of edk2-acpica-iasl
WARNING - [SDE] Failed to fetch NugetDependecy: mu_nasm@2.15.05<mailto:mu_nasm@2.15.05>: [Nuget] We failed to install this version 2.15.05 of mu_nasm
. Done
SECTION - Updated/Verified 3 dependencies
ERROR - We were unable to successfully update 2 dependencies in environment
SECTION - Summary
ERROR - Error
Thanks
Steven Shi
[-- Attachment #1.2: Type: text/html, Size: 30938 bytes --]
[-- Attachment #2: image005.png --]
[-- Type: image/png, Size: 195 bytes --]
[-- Attachment #3: 7E242D8960184D46811DF9225220A3C9.png --]
[-- Type: image/png, Size: 139 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
2021-07-02 2:43 ` Steven Shi
@ 2021-07-02 7:23 ` Steven Shi
2021-07-07 18:54 ` Sean
0 siblings, 1 reply; 17+ messages in thread
From: Steven Shi @ 2021-07-02 7:23 UTC (permalink / raw)
To: devel@edk2.groups.io, Shi, Steven, Rebecca Cran, Bret Barkelew,
rebecca@nuviainc.com
[-- Attachment #1.1: Type: text/plain, Size: 23682 bytes --]
Bret,
For my failure of "Unable to load the service index for source...", I see it might be the Nuget proxy setting issue: https://stackoverflow.com/questions/41185443/nuget-connection-attempt-failed-unable-to-load-the-service-index-for-source. I tried to set the Nuget https_proxy and http_proxy as below, but it still does not work. Do you know how to set the Nuget Proxy in Pytool?
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ mono /home/jshi19/wksp_efi/unit_test_env/lib/python3.8/site-packages/edk2toolext/bin/NuGet.exe config -set https_proxy=http://xxx:xxxx@xxxx.intel.com:xxx
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ mono /home/jshi19/wksp_efi/unit_test_env/lib/python3.8/site-packages/edk2toolext/bin/NuGet.exe config -set http_proxy=http://xxx:xxxx@xxxx.intel.com:xxx
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ mono /home/jshi19/wksp_efi/unit_test_env/lib/python3.8/site-packages/edk2toolext/bin/NuGet.exe install edk2-acpica-iasl -Source https://pkgs.dev.azure.com/projectmu/acpica/_packaging/mu_iasl/nuget/v3/index.json -ExcludeVersion -NonInteractive -Version 20200717.0.0 -Verbosity detailed -OutputDirectory "/home/jshi19/wksp_efi/edk2-2/BaseTools/Bin/edk2-acpica-iasl_extdep_temp"
NuGet Version: 5.3.1.6268
Feeds used:
https://pkgs.dev.azure.com/projectmu/acpica/_packaging/mu_iasl/nuget/v3/index.json
Attempting to gather dependency information for package 'edk2-acpica-iasl.20200717.0.0' with respect to project '/home/jshi19/wksp_efi/edk2-2/BaseTools/Bin/edk2-acpica-iasl_extdep_temp', targeting 'Any,Version=v0.0'
Unable to load the service index for source https://pkgs.dev.azure.com/projectmu/acpica/_packaging/mu_iasl/nuget/v3/index.json.
GSSAPI operation failed with error - Unspecified GSS failure. Minor code may provide more information (SPNEGO cannot find mechanisms to negotiate).
System.InvalidOperationException: Unable to load the service index for source https://pkgs.dev.azure.com/projectmu/acpica/_packaging/mu_iasl/nuget/v3/index.json. ---> NuGet.Protocol.Core.Types.FatalProtocolException: Unable to load the service index for source https://pkgs.dev.azure.com/projectmu/acpica/_packaging/mu_iasl/nuget/v3/index.json. ---> System.ComponentModel.Win32Exception: GSSAPI operation failed with error - Unspecified GSS failure. Minor code may provide more information (SPNEGO cannot find mechanisms to negotiate).
at System.Net.NTAuthentication.GetOutgoingBlob (System.Byte[] incomingBlob, System.Boolean throwOnError, System.Net.SecurityStatusPal& statusCode) [0x00258] in <a85c1a570f9a4f9f9c3d2cfa5504e34f>:0
at System.Net.NTAuthentication.GetOutgoingBlob (System.String incomingBlob) [0x00045] in <a85c1a570f9a4f9f9c3d2cfa5504e34f>:0
Thanks
Steven Shi
Intel\IAGS\SFP\FIA (Firmware Infrastructure Automation)
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Steven Shi
Sent: Friday, July 2, 2021 10:44 AM
To: Rebecca Cran <rebecca@bsdio.com>; devel@edk2.groups.io; Bret Barkelew <Bret.Barkelew@microsoft.com>; rebecca@nuviainc.com
Subject: Re: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
Thank you to Rebecca. After install the libmono-system-net-http4.0-cil and mono-devel pkg as below, the 'mono /home/jshi19/wksp_efi/unit_test_env/lib/python3.8/site-packages/edk2toolext/bin/NuGet.exe locals global-packages -list' pass.
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ sudo apt-get install libmono-system-net-http4.0-cil
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ sudo apt-get install mono-devel
But now there are new failures show up when install edk2-acpica-iasl:
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5 -v
... ...
INFO - Cmd to run is: mono /home/jshi19/wksp_efi/unit_test_env/lib/python3.8/site-packages/edk2toolext/bin/NuGet.exe locals global-packages -list
INFO - Cmd to run is: mono /home/jshi19/wksp_efi/unit_test_env/lib/python3.8/site-packages/edk2toolext/bin/NuGet.exe locals global-packages -list
INFO - ------------------------------------------------
INFO - ------------------------------------------------
INFO - --------------Cmd Output Starting---------------
INFO - --------------Cmd Output Starting---------------
INFO - ------------------------------------------------
INFO - ------------------------------------------------
...INFO - global-packages: /home/jshi19/.nuget/packages/
INFO - global-packages: /home/jshi19/.nuget/packages/
INFO - ------------------------------------------------
INFO - --------------Cmd Output Finished---------------
INFO - ------------------------------------------------
INFO - --------- Running Time (mm:ss): 00:00 ----------
INFO - --------------Cmd Output Finished---------------
INFO - ----------- Return Code: 0x00000000 ------------
INFO - --------- Running Time (mm:ss): 00:00 ----------
INFO - ------------------------------------------------
INFO - ----------- Return Code: 0x00000000 ------------
INFO - Could not determine Nuget global packages cache location.
INFO - ------------------------------------------------
INFO - Cmd to run is: mono /home/jshi19/wksp_efi/unit_test_env/lib/python3.8/site-packages/edk2toolext/bin/NuGet.exe install edk2-acpica-iasl -Source https://pkgs.dev.azure.com/projectmu/acpica/_packaging/mu_iasl/nuget/v3/index.json -ExcludeVersion -NonInteractive -Version 20200717.0.0 -Verbosity detailed -OutputDirectory "/home/jshi19/wksp_efi/edk2-2/BaseTools/Bin/edk2-acpica-iasl_extdep_temp"
INFO - Could not determine Nuget global packages cache location.
INFO - ------------------------------------------------
INFO - --------------Cmd Output Starting---------------
INFO - Cmd to run is: mono /home/jshi19/wksp_efi/unit_test_env/lib/python3.8/site-packages/edk2toolext/bin/NuGet.exe install mu_nasm -Source https://api.nuget.org/v3/index.json -ExcludeVersion -NonInteractive -Version 2.15.05 -Verbosity detailed -OutputDirectory "/home/jshi19/wksp_efi/edk2-2/BaseTools/Bin/mu_nasm_extdep_temp"
INFO - ------------------------------------------------
INFO - ------------------------------------------------
INFO - --------------Cmd Output Starting---------------
INFO - ------------------------------------------------
INFO - NuGet Version: 5.3.1.6268
INFO - NuGet Version: 5.3.1.6268
INFO - Feeds used:
INFO - https://api.nuget.org/v3/index.json
INFO -
INFO -
INFO -
INFO - Feeds used:
INFO - https://pkgs.dev.azure.com/projectmu/acpica/_packaging/mu_iasl/nuget/v3/index.json
INFO -
INFO - Attempting to gather dependency information for package 'mu_nasm.2.15.5' with respect to project '/home/jshi19/wksp_efi/edk2-2/BaseTools/Bin/mu_nasm_extdep_temp', targeting 'Any,Version=v0.0'
INFO -
INFO -
INFO - Attempting to gather dependency information for package 'edk2-acpica-iasl.20200717.0.0' with respect to project '/home/jshi19/wksp_efi/edk2-2/BaseTools/Bin/edk2-acpica-iasl_extdep_temp', targeting 'Any,Version=v0.0'
INFO - Unable to load the service index for source https://pkgs.dev.azure.com/projectmu/acpica/_packaging/mu_iasl/nuget/v3/index.json.
INFO - GSSAPI operation failed with error - An invalid name was supplied (Configuration file does not specify default realm).
INFO - System.InvalidOperationException: Unable to load the service index for source https://pkgs.dev.azure.com/projectmu/acpica/_packaging/mu_iasl/nuget/v3/index.json. ---> NuGet.Protocol.Core.Types.FatalProtocolException: Unable to load the service index for source https://pkgs.dev.azure.com/projectmu/acpica/_packaging/mu_iasl/nuget/v3/index.json. ---> System.ComponentModel.Win32Exception: GSSAPI operation failed with error - An invalid name was supplied (Configuration file does not specify default realm).
INFO - at System.Net.Security.NegotiateStreamPal.AcquireCredentialsHandle (System.String package, System.Boolean isServer, System.Net.NetworkCredential credential) [0x000a3] in <a85c1a570f9a4f9f9c3d2cfa5504e34f>:0
... ...
Thanks
Steven Shi
Intel\IAGS\SFP\FIA (Firmware Infrastructure Automation)
From: Rebecca Cran <rebecca@bsdio.com<mailto:rebecca@bsdio.com>>
Sent: Friday, July 2, 2021 10:00 AM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Shi, Steven <steven.shi@intel.com<mailto:steven.shi@intel.com>>; Bret Barkelew <Bret.Barkelew@microsoft.com<mailto:Bret.Barkelew@microsoft.com>>; rebecca@nuviainc.com<mailto:rebecca@nuviainc.com>
Subject: Re: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
That means you need to install the various mono assemblies, such as the libmono-system-net-http4.0-cil package in Ubuntu.
I spent some time chasing down the dependencies last year which got very frustrating and tedious, but have forgotten which were needed. All I remember is there were quite a lot.
--
Rebecca Cran
On 7/1/21 7:36 PM, Steven Shi wrote:
I already installed mono, but NuGet.exe still fails to work in my side. I'm behind a proxy but don't know how to configure the NuGet.exe proxy in .pytool. Below is my verbose error message.
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5 -v
... ...
SECTION - Start Invocable Tool
SECTION - Initial update of environment
DEBUG - --- self_describing_environment.update_extdeps()
DEBUG - Verify 'mu_nasm' returning 'False'.
DEBUG - Verify 'edk2-acpica-iasl' returning 'False'.
DEBUG - Creating 5 threads for the SDE update
UpdatingDEBUG - Verify 'gcc_aarch64_linux' returning 'True'.
DEBUG - Verify 'gcc_riscv64_unknown' returning 'True'.
DEBUG - Verify 'mu_nasm' returning 'False'.
DEBUG - Verify 'gcc_arm_linux' returning 'True'.
DEBUG - Verify 'edk2-acpica-iasl' returning 'False'.
DEBUG - Cleaning dependency directory for 'mu_nasm'...
DEBUG - Cleaning dependency directory for 'edk2-acpica-iasl'...
INFO - Cmd to run is: mono /home/jshi19/wksp_efi/unit_test_env/lib/python3.8/site-packages/edk2toolext/bin/NuGet.exe locals global-packages -list
INFO - Cmd to run is: mono /home/jshi19/wksp_efi/unit_test_env/lib/python3.8/site-packages/edk2toolext/bin/NuGet.exe locals global-packages -list
INFO - ------------------------------------------------
INFO - ------------------------------------------------
INFO - --------------Cmd Output Starting---------------
INFO - --------------Cmd Output Starting---------------
INFO - ------------------------------------------------
INFO - ------------------------------------------------
...INFO - Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
INFO - Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
INFO - Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
INFO - Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
INFO - Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
INFO - System.AggregateException: One or more errors occurred. (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) ---> System.IO.FileNotFoundException: Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
INFO - --- End of inner exception stack trace ---
... ...
If I run the Cmd manually, I get below error:
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ mono /home/jshi19/wksp_efi/unit_test_env/lib/python3.8/site-packages/edk2toolext/bin/NuGet.exe locals global-packages -list
Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
System.AggregateException: One or more errors occurred. (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) ---> System.IO.FileNotFoundException: Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
--- End of inner exception stack trace ---
at NuGet.CommandLine.Program.Initialize (NuGet.IFileSystem fileSystem, NuGet.CommandLine.IConsole console) [0x000ce] in <ff502667b3fe4a85827635c6fe6b0d0d>:0
at NuGet.CommandLine.Program.MainCore (System.String workingDirectory, System.String[] args) [0x000d8] in <ff502667b3fe4a85827635c6fe6b0d0d>:0
---> (Inner Exception #0) System.IO.FileNotFoundException: Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
File name: 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'<---
---> (Inner Exception #1) System.IO.FileNotFoundException: Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
File name: 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'<---
---> (Inner Exception #2) System.IO.FileNotFoundException: Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
File name: 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'<---
---> (Inner Exception #3) System.IO.FileNotFoundException: Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
File name: 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'<---
---> (Inner Exception #4) System.IO.FileNotFoundException: Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
File name: 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'<---
Thanks
Steven Shi
From: Bret Barkelew <Bret.Barkelew@microsoft.com><mailto:Bret.Barkelew@microsoft.com>
Sent: Friday, July 2, 2021 9:04 AM
To: Shi, Steven <steven.shi@intel.com><mailto:steven.shi@intel.com>; devel@edk2.groups.io<mailto:devel@edk2.groups.io>; rebecca@nuviainc.com<mailto:rebecca@nuviainc.com>
Subject: RE: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
I believe the Azure DevOps build agents that are used by the EDK2 CI pipeline already have mono installed.
- Bret
From: Shi, Steven<mailto:steven.shi@intel.com>
Sent: Thursday, July 1, 2021 5:54 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Bret Barkelew<mailto:Bret.Barkelew@microsoft.com>; rebecca@nuviainc.com<mailto:rebecca@nuviainc.com>
Subject: RE: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
Hi Bret, Rebecca,
Thanks for the info. Does edk2 CI run the .pytool unit tests in Linux? If yes, how does edk2 CI solve the nuget dependency problem in Linux right now?
Thanks
Steven Shi
From: devel@edk2.groups.io<mailto:devel@edk2.groups.io> <devel@edk2.groups.io<mailto:devel@edk2.groups.io>> On Behalf Of Bret Barkelew via groups.io
Sent: Wednesday, June 30, 2021 1:02 AM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; rebecca@nuviainc.com<mailto:rebecca@nuviainc.com>; Shi, Steven <steven.shi@intel.com<mailto:steven.shi@intel.com>>
Subject: Re: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
Hmmm. If that works well, we might be able to investigate carrying both and picking based on the host OS.
- Bret
From: devel@edk2.groups.io<mailto:devel@edk2.groups.io> <devel@edk2.groups.io<mailto:devel@edk2.groups.io>> on behalf of Rebecca Cran via groups.io <rebecca=nuviainc.com@groups.io<mailto:rebecca=nuviainc.com@groups.io>>
Sent: Tuesday, June 29, 2021 9:42:41 AM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io> <devel@edk2.groups.io<mailto:devel@edk2.groups.io>>; steven.shi@intel.com<mailto:steven.shi@intel.com> <steven.shi@intel.com<mailto:steven.shi@intel.com>>
Subject: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
Your Linux distribution should have a nuget package available (e.g. https://packages.ubuntu.com/hirsute/nuget<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpackages.ubuntu.com%2Fhirsute%2Fnuget&data=04%7C01%7Cbret.barkelew%40microsoft.com%7C28bf7b813a9a4d6e365508d93cf3fd8a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637607840913328150%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Z7vCN9JUmx1kQks%2FuWKjcnMRo7koWoyxOET0bnSbN6U%3D&reserved=0>) but the EDK2 CI is designed to use the copy of nuget installed with the edk2toolext package.
--
Rebecca Cran
On 6/29/21 9:44 AM, Steven Shi wrote:
Cran, Thanks. I'm curious whether the .pytool has native Linux executable binary to run? It looks the edk2 CI framework has more dependency in Linux than Windows.
Thanks
Steven Shi
Intel\IAGS\SFP\FIA (Firmware Infrastructure Automation)
From: devel@edk2.groups.io<mailto:devel@edk2.groups.io> <devel@edk2.groups.io><mailto:devel@edk2.groups.io> On Behalf Of Rebecca Cran
Sent: Tuesday, June 29, 2021 8:13 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Shi, Steven <steven.shi@intel.com><mailto:steven.shi@intel.com>
Subject: Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
NuGet.exe is a .NET assembly/executable, so on Linux you'll need to install Mono (https://www.mono-project.com/<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.mono-project.com%2F&data=04%7C01%7Cbret.barkelew%40microsoft.com%7C28bf7b813a9a4d6e365508d93cf3fd8a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637607840913338106%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=uPkpInnWMx%2FP35GUXlbffQwxbLya9DDDegyLJvtuScU%3D&reserved=0>).
--
Rebecca Cran
On 6/28/21 7:45 PM, Steven Shi wrote:
Hello,
I'm interested in the edk2 CI unit tests(https://github.com/tianocore/edk2/tree/master/.pytool<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2F.pytool&data=04%7C01%7Cbret.barkelew%40microsoft.com%7C28bf7b813a9a4d6e365508d93cf3fd8a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637607840913338106%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=v%2FqnYohY9bDCIAC8ZmqPkV3RvhaIuyQEXfkY9JNyIqs%3D&reserved=0>). I tested it in my local. It works well in Windows but does not work in Linux. It looks the NuGet in pytool only has windows version (C:\steven\unit_test_env\Lib\site-packages\edk2toolext\bin\NuGet.exe) but not support the Linux. Below is my detail test steps. Do the edk2 CI unit tests work in Linux?
1. Windows :
c:\steven>py -m venv unit_test_env
c:\steven>cd c:\steven\edk2
c:\steven\edk2>c:\steven\unit_test_env\Scripts\activate.bat
(unit_test_env) c:\steven\edk2>pip install --upgrade -r pip-requirements.txt
(unit_test_env) c:\steven\edk2>stuart_setup -c .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019
(unit_test_env) c:\steven\edk2>stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019
(unit_test_env) c:\steven\edk2>edksetup.bat
(unit_test_env) c:\steven\edk2>edksetup.bat Rebuild
(unit_test_env) c:\steven\edk2>stuart_ci_build -c .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019 -a X64,IA32 -t DEBUG --verbose
...
PROGRESS - Overall Build Status: Success
SECTION - Summary
PROGRESS - Success
1. Linux:
jshi19@ub2-uefi-b01:~/wksp_efi$ python3 -m venv unit_test_env
jshi19@ub2-uefi-b01:~/wksp_efi$ source unit_test_env/bin/activate
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ pip install --upgrade -r pip-requirements.txt
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ make -C BaseTools/
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ stuart_setup -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5
(unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5
SECTION - Init SDE
WARNING - Using Pip Tools based BaseTools
SECTION - Loading Plugins
SECTION - Start Invocable Tool
SECTION - Initial update of environment
UpdatingWARNING - [SDE] Failed to fetch NugetDependecy: edk2-acpica-iasl@20200717.0.0<mailto:edk2-acpica-iasl@20200717.0.0>: [Nuget] We failed to install this version 20200717.0.0 of edk2-acpica-iasl
WARNING - [SDE] Failed to fetch NugetDependecy: mu_nasm@2.15.05<mailto:mu_nasm@2.15.05>: [Nuget] We failed to install this version 2.15.05 of mu_nasm
. Done
SECTION - Updated/Verified 3 dependencies
SECTION - Second pass update of environment
UpdatingWARNING - [SDE] Failed to fetch NugetDependecy: edk2-acpica-iasl@20200717.0.0<mailto:edk2-acpica-iasl@20200717.0.0>: [Nuget] We failed to install this version 20200717.0.0 of edk2-acpica-iasl
WARNING - [SDE] Failed to fetch NugetDependecy: mu_nasm@2.15.05<mailto:mu_nasm@2.15.05>: [Nuget] We failed to install this version 2.15.05 of mu_nasm
. Done
SECTION - Updated/Verified 3 dependencies
ERROR - We were unable to successfully update 2 dependencies in environment
SECTION - Summary
ERROR - Error
Thanks
Steven Shi
[-- Attachment #1.2: Type: text/html, Size: 45088 bytes --]
[-- Attachment #2: image001.png --]
[-- Type: image/png, Size: 195 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
2021-07-02 7:23 ` Steven Shi
@ 2021-07-07 18:54 ` Sean
0 siblings, 0 replies; 17+ messages in thread
From: Sean @ 2021-07-07 18:54 UTC (permalink / raw)
To: devel, steven.shi, Rebecca Cran, Bret Barkelew,
rebecca@nuviainc.com
Steven,
Yes, Pytools is cross platform and is used regularly on Linux.
Generally mono has been the problem on Linux. Ideally we could move to
a native Linux package manager to make this have fewer dependencies but
for now can you try following the directions here on getting mono setup.
https://github.com/tianocore/edk2-pytool-extensions/blob/master/docs/usability/using_extdep.md#a-note-on-nuget-on-linux
That said I have not heard of users using nuget proxy features so that
could be an untested path. Definitely want to help get this resolved
for you.
Thanks
Sean
On 7/2/2021 12:23 AM, Steven Shi wrote:
> Bret,
> For my failure of "Unable to load the service index for source...", I see it might be the Nuget proxy setting issue: https://stackoverflow.com/questions/41185443/nuget-connection-attempt-failed-unable-to-load-the-service-index-for-source. I tried to set the Nuget https_proxy and http_proxy as below, but it still does not work. Do you know how to set the Nuget Proxy in Pytool?
>
> (unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ mono /home/jshi19/wksp_efi/unit_test_env/lib/python3.8/site-packages/edk2toolext/bin/NuGet.exe config -set https_proxy=http://xxx:xxxx@xxxx.intel.com:xxx
> (unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ mono /home/jshi19/wksp_efi/unit_test_env/lib/python3.8/site-packages/edk2toolext/bin/NuGet.exe config -set http_proxy=http://xxx:xxxx@xxxx.intel.com:xxx
> (unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ mono /home/jshi19/wksp_efi/unit_test_env/lib/python3.8/site-packages/edk2toolext/bin/NuGet.exe install edk2-acpica-iasl -Source https://pkgs.dev.azure.com/projectmu/acpica/_packaging/mu_iasl/nuget/v3/index.json -ExcludeVersion -NonInteractive -Version 20200717.0.0 -Verbosity detailed -OutputDirectory "/home/jshi19/wksp_efi/edk2-2/BaseTools/Bin/edk2-acpica-iasl_extdep_temp"
> NuGet Version: 5.3.1.6268
> Feeds used:
> https://pkgs.dev.azure.com/projectmu/acpica/_packaging/mu_iasl/nuget/v3/index.json
> Attempting to gather dependency information for package 'edk2-acpica-iasl.20200717.0.0' with respect to project '/home/jshi19/wksp_efi/edk2-2/BaseTools/Bin/edk2-acpica-iasl_extdep_temp', targeting 'Any,Version=v0.0'
> Unable to load the service index for source https://pkgs.dev.azure.com/projectmu/acpica/_packaging/mu_iasl/nuget/v3/index.json.
> GSSAPI operation failed with error - Unspecified GSS failure. Minor code may provide more information (SPNEGO cannot find mechanisms to negotiate).
> System.InvalidOperationException: Unable to load the service index for source https://pkgs.dev.azure.com/projectmu/acpica/_packaging/mu_iasl/nuget/v3/index.json. ---> NuGet.Protocol.Core.Types.FatalProtocolException: Unable to load the service index for source https://pkgs.dev.azure.com/projectmu/acpica/_packaging/mu_iasl/nuget/v3/index.json. ---> System.ComponentModel.Win32Exception: GSSAPI operation failed with error - Unspecified GSS failure. Minor code may provide more information (SPNEGO cannot find mechanisms to negotiate).
> at System.Net.NTAuthentication.GetOutgoingBlob (System.Byte[] incomingBlob, System.Boolean throwOnError, System.Net.SecurityStatusPal& statusCode) [0x00258] in <a85c1a570f9a4f9f9c3d2cfa5504e34f>:0
> at System.Net.NTAuthentication.GetOutgoingBlob (System.String incomingBlob) [0x00045] in <a85c1a570f9a4f9f9c3d2cfa5504e34f>:0
>
> Thanks
>
> Steven Shi
> Intel\IAGS\SFP\FIA (Firmware Infrastructure Automation)
>
>
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Steven Shi
> Sent: Friday, July 2, 2021 10:44 AM
> To: Rebecca Cran <rebecca@bsdio.com>; devel@edk2.groups.io; Bret Barkelew <Bret.Barkelew@microsoft.com>; rebecca@nuviainc.com
> Subject: Re: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
>
> Thank you to Rebecca. After install the libmono-system-net-http4.0-cil and mono-devel pkg as below, the 'mono /home/jshi19/wksp_efi/unit_test_env/lib/python3.8/site-packages/edk2toolext/bin/NuGet.exe locals global-packages -list' pass.
> (unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ sudo apt-get install libmono-system-net-http4.0-cil
> (unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ sudo apt-get install mono-devel
>
> But now there are new failures show up when install edk2-acpica-iasl:
> (unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5 -v
> ... ...
> INFO - Cmd to run is: mono /home/jshi19/wksp_efi/unit_test_env/lib/python3.8/site-packages/edk2toolext/bin/NuGet.exe locals global-packages -list
> INFO - Cmd to run is: mono /home/jshi19/wksp_efi/unit_test_env/lib/python3.8/site-packages/edk2toolext/bin/NuGet.exe locals global-packages -list
> INFO - ------------------------------------------------
> INFO - ------------------------------------------------
> INFO - --------------Cmd Output Starting---------------
> INFO - --------------Cmd Output Starting---------------
> INFO - ------------------------------------------------
> INFO - ------------------------------------------------
> ...INFO - global-packages: /home/jshi19/.nuget/packages/
> INFO - global-packages: /home/jshi19/.nuget/packages/
> INFO - ------------------------------------------------
> INFO - --------------Cmd Output Finished---------------
> INFO - ------------------------------------------------
> INFO - --------- Running Time (mm:ss): 00:00 ----------
> INFO - --------------Cmd Output Finished---------------
> INFO - ----------- Return Code: 0x00000000 ------------
> INFO - --------- Running Time (mm:ss): 00:00 ----------
> INFO - ------------------------------------------------
> INFO - ----------- Return Code: 0x00000000 ------------
> INFO - Could not determine Nuget global packages cache location.
> INFO - ------------------------------------------------
> INFO - Cmd to run is: mono /home/jshi19/wksp_efi/unit_test_env/lib/python3.8/site-packages/edk2toolext/bin/NuGet.exe install edk2-acpica-iasl -Source https://pkgs.dev.azure.com/projectmu/acpica/_packaging/mu_iasl/nuget/v3/index.json -ExcludeVersion -NonInteractive -Version 20200717.0.0 -Verbosity detailed -OutputDirectory "/home/jshi19/wksp_efi/edk2-2/BaseTools/Bin/edk2-acpica-iasl_extdep_temp"
> INFO - Could not determine Nuget global packages cache location.
> INFO - ------------------------------------------------
> INFO - --------------Cmd Output Starting---------------
> INFO - Cmd to run is: mono /home/jshi19/wksp_efi/unit_test_env/lib/python3.8/site-packages/edk2toolext/bin/NuGet.exe install mu_nasm -Source https://api.nuget.org/v3/index.json -ExcludeVersion -NonInteractive -Version 2.15.05 -Verbosity detailed -OutputDirectory "/home/jshi19/wksp_efi/edk2-2/BaseTools/Bin/mu_nasm_extdep_temp"
> INFO - ------------------------------------------------
> INFO - ------------------------------------------------
> INFO - --------------Cmd Output Starting---------------
> INFO - ------------------------------------------------
> INFO - NuGet Version: 5.3.1.6268
> INFO - NuGet Version: 5.3.1.6268
> INFO - Feeds used:
> INFO - https://api.nuget.org/v3/index.json
> INFO -
> INFO -
> INFO -
> INFO - Feeds used:
> INFO - https://pkgs.dev.azure.com/projectmu/acpica/_packaging/mu_iasl/nuget/v3/index.json
> INFO -
> INFO - Attempting to gather dependency information for package 'mu_nasm.2.15.5' with respect to project '/home/jshi19/wksp_efi/edk2-2/BaseTools/Bin/mu_nasm_extdep_temp', targeting 'Any,Version=v0.0'
> INFO -
> INFO -
> INFO - Attempting to gather dependency information for package 'edk2-acpica-iasl.20200717.0.0' with respect to project '/home/jshi19/wksp_efi/edk2-2/BaseTools/Bin/edk2-acpica-iasl_extdep_temp', targeting 'Any,Version=v0.0'
> INFO - Unable to load the service index for source https://pkgs.dev.azure.com/projectmu/acpica/_packaging/mu_iasl/nuget/v3/index.json.
> INFO - GSSAPI operation failed with error - An invalid name was supplied (Configuration file does not specify default realm).
> INFO - System.InvalidOperationException: Unable to load the service index for source https://pkgs.dev.azure.com/projectmu/acpica/_packaging/mu_iasl/nuget/v3/index.json. ---> NuGet.Protocol.Core.Types.FatalProtocolException: Unable to load the service index for source https://pkgs.dev.azure.com/projectmu/acpica/_packaging/mu_iasl/nuget/v3/index.json. ---> System.ComponentModel.Win32Exception: GSSAPI operation failed with error - An invalid name was supplied (Configuration file does not specify default realm).
> INFO - at System.Net.Security.NegotiateStreamPal.AcquireCredentialsHandle (System.String package, System.Boolean isServer, System.Net.NetworkCredential credential) [0x000a3] in <a85c1a570f9a4f9f9c3d2cfa5504e34f>:0
> ... ...
>
> Thanks
>
> Steven Shi
> Intel\IAGS\SFP\FIA (Firmware Infrastructure Automation)
>
>
> From: Rebecca Cran <rebecca@bsdio.com<mailto:rebecca@bsdio.com>>
> Sent: Friday, July 2, 2021 10:00 AM
> To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Shi, Steven <steven.shi@intel.com<mailto:steven.shi@intel.com>>; Bret Barkelew <Bret.Barkelew@microsoft.com<mailto:Bret.Barkelew@microsoft.com>>; rebecca@nuviainc.com<mailto:rebecca@nuviainc.com>
> Subject: Re: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
>
>
> That means you need to install the various mono assemblies, such as the libmono-system-net-http4.0-cil package in Ubuntu.
>
> I spent some time chasing down the dependencies last year which got very frustrating and tedious, but have forgotten which were needed. All I remember is there were quite a lot.
>
>
>
> --
> Rebecca Cran
>
>
> On 7/1/21 7:36 PM, Steven Shi wrote:
> I already installed mono, but NuGet.exe still fails to work in my side. I'm behind a proxy but don't know how to configure the NuGet.exe proxy in .pytool. Below is my verbose error message.
>
> (unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5 -v
> ... ...
> SECTION - Start Invocable Tool
> SECTION - Initial update of environment
> DEBUG - --- self_describing_environment.update_extdeps()
> DEBUG - Verify 'mu_nasm' returning 'False'.
> DEBUG - Verify 'edk2-acpica-iasl' returning 'False'.
> DEBUG - Creating 5 threads for the SDE update
> UpdatingDEBUG - Verify 'gcc_aarch64_linux' returning 'True'.
> DEBUG - Verify 'gcc_riscv64_unknown' returning 'True'.
> DEBUG - Verify 'mu_nasm' returning 'False'.
> DEBUG - Verify 'gcc_arm_linux' returning 'True'.
> DEBUG - Verify 'edk2-acpica-iasl' returning 'False'.
> DEBUG - Cleaning dependency directory for 'mu_nasm'...
> DEBUG - Cleaning dependency directory for 'edk2-acpica-iasl'...
> INFO - Cmd to run is: mono /home/jshi19/wksp_efi/unit_test_env/lib/python3.8/site-packages/edk2toolext/bin/NuGet.exe locals global-packages -list
> INFO - Cmd to run is: mono /home/jshi19/wksp_efi/unit_test_env/lib/python3.8/site-packages/edk2toolext/bin/NuGet.exe locals global-packages -list
> INFO - ------------------------------------------------
> INFO - ------------------------------------------------
> INFO - --------------Cmd Output Starting---------------
> INFO - --------------Cmd Output Starting---------------
> INFO - ------------------------------------------------
> INFO - ------------------------------------------------
> ...INFO - Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
> INFO - Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
> INFO - Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
> INFO - Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
> INFO - Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
> INFO - System.AggregateException: One or more errors occurred. (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) ---> System.IO.FileNotFoundException: Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
> INFO - --- End of inner exception stack trace ---
> ... ...
>
>
> If I run the Cmd manually, I get below error:
> (unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ mono /home/jshi19/wksp_efi/unit_test_env/lib/python3.8/site-packages/edk2toolext/bin/NuGet.exe locals global-packages -list
> Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
> Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
> Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
> Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
> Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
> System.AggregateException: One or more errors occurred. (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) (Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.) ---> System.IO.FileNotFoundException: Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
> --- End of inner exception stack trace ---
> at NuGet.CommandLine.Program.Initialize (NuGet.IFileSystem fileSystem, NuGet.CommandLine.IConsole console) [0x000ce] in <ff502667b3fe4a85827635c6fe6b0d0d>:0
> at NuGet.CommandLine.Program.MainCore (System.String workingDirectory, System.String[] args) [0x000d8] in <ff502667b3fe4a85827635c6fe6b0d0d>:0
> ---> (Inner Exception #0) System.IO.FileNotFoundException: Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
> File name: 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'<---
>
> ---> (Inner Exception #1) System.IO.FileNotFoundException: Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
> File name: 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'<---
>
> ---> (Inner Exception #2) System.IO.FileNotFoundException: Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
> File name: 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'<---
>
> ---> (Inner Exception #3) System.IO.FileNotFoundException: Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
> File name: 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'<---
>
> ---> (Inner Exception #4) System.IO.FileNotFoundException: Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
> File name: 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'<---
>
>
> Thanks
>
> Steven Shi
>
>
> From: Bret Barkelew <Bret.Barkelew@microsoft.com><mailto:Bret.Barkelew@microsoft.com>
> Sent: Friday, July 2, 2021 9:04 AM
> To: Shi, Steven <steven.shi@intel.com><mailto:steven.shi@intel.com>; devel@edk2.groups.io<mailto:devel@edk2.groups.io>; rebecca@nuviainc.com<mailto:rebecca@nuviainc.com>
> Subject: RE: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
>
> I believe the Azure DevOps build agents that are used by the EDK2 CI pipeline already have mono installed.
>
> - Bret
>
> From: Shi, Steven<mailto:steven.shi@intel.com>
> Sent: Thursday, July 1, 2021 5:54 PM
> To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Bret Barkelew<mailto:Bret.Barkelew@microsoft.com>; rebecca@nuviainc.com<mailto:rebecca@nuviainc.com>
> Subject: RE: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
>
> Hi Bret, Rebecca,
> Thanks for the info. Does edk2 CI run the .pytool unit tests in Linux? If yes, how does edk2 CI solve the nuget dependency problem in Linux right now?
>
>
> Thanks
>
> Steven Shi
>
>
> From: devel@edk2.groups.io<mailto:devel@edk2.groups.io> <devel@edk2.groups.io<mailto:devel@edk2.groups.io>> On Behalf Of Bret Barkelew via groups.io
> Sent: Wednesday, June 30, 2021 1:02 AM
> To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; rebecca@nuviainc.com<mailto:rebecca@nuviainc.com>; Shi, Steven <steven.shi@intel.com<mailto:steven.shi@intel.com>>
> Subject: Re: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
>
> Hmmm. If that works well, we might be able to investigate carrying both and picking based on the host OS.
>
> - Bret
>
> From: devel@edk2.groups.io<mailto:devel@edk2.groups.io> <devel@edk2.groups.io<mailto:devel@edk2.groups.io>> on behalf of Rebecca Cran via groups.io <rebecca=nuviainc.com@groups.io<mailto:rebecca=nuviainc.com@groups.io>>
> Sent: Tuesday, June 29, 2021 9:42:41 AM
> To: devel@edk2.groups.io<mailto:devel@edk2.groups.io> <devel@edk2.groups.io<mailto:devel@edk2.groups.io>>; steven.shi@intel.com<mailto:steven.shi@intel.com> <steven.shi@intel.com<mailto:steven.shi@intel.com>>
> Subject: [EXTERNAL] Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
>
>
> Your Linux distribution should have a nuget package available (e.g. https://packages.ubuntu.com/hirsute/nuget<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpackages.ubuntu.com%2Fhirsute%2Fnuget&data=04%7C01%7Cbret.barkelew%40microsoft.com%7C28bf7b813a9a4d6e365508d93cf3fd8a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637607840913328150%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Z7vCN9JUmx1kQks%2FuWKjcnMRo7koWoyxOET0bnSbN6U%3D&reserved=0>) but the EDK2 CI is designed to use the copy of nuget installed with the edk2toolext package.
>
>
>
> --
> Rebecca Cran
>
>
> On 6/29/21 9:44 AM, Steven Shi wrote:
>
> Cran, Thanks. I'm curious whether the .pytool has native Linux executable binary to run? It looks the edk2 CI framework has more dependency in Linux than Windows.
>
>
>
>
>
> Thanks
>
>
>
> Steven Shi
>
> Intel\IAGS\SFP\FIA (Firmware Infrastructure Automation)
>
>
>
>
>
> From: devel@edk2.groups.io<mailto:devel@edk2.groups.io> <devel@edk2.groups.io><mailto:devel@edk2.groups.io> On Behalf Of Rebecca Cran
> Sent: Tuesday, June 29, 2021 8:13 PM
> To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Shi, Steven <steven.shi@intel.com><mailto:steven.shi@intel.com>
> Subject: Re: [edk2-devel] Do the edk2 CI unit tests work in Linux?
>
>
>
> NuGet.exe is a .NET assembly/executable, so on Linux you'll need to install Mono (https://www.mono-project.com/<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.mono-project.com%2F&data=04%7C01%7Cbret.barkelew%40microsoft.com%7C28bf7b813a9a4d6e365508d93cf3fd8a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637607840913338106%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=uPkpInnWMx%2FP35GUXlbffQwxbLya9DDDegyLJvtuScU%3D&reserved=0>).
>
>
>
> --
>
> Rebecca Cran
>
>
>
>
>
> On 6/28/21 7:45 PM, Steven Shi wrote:
>
> Hello,
>
> I'm interested in the edk2 CI unit tests(https://github.com/tianocore/edk2/tree/master/.pytool<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2F.pytool&data=04%7C01%7Cbret.barkelew%40microsoft.com%7C28bf7b813a9a4d6e365508d93cf3fd8a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637607840913338106%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=v%2FqnYohY9bDCIAC8ZmqPkV3RvhaIuyQEXfkY9JNyIqs%3D&reserved=0>). I tested it in my local. It works well in Windows but does not work in Linux. It looks the NuGet in pytool only has windows version (C:\steven\unit_test_env\Lib\site-packages\edk2toolext\bin\NuGet.exe) but not support the Linux. Below is my detail test steps. Do the edk2 CI unit tests work in Linux?
>
>
>
> 1. Windows :
>
> c:\steven>py -m venv unit_test_env
>
> c:\steven>cd c:\steven\edk2
>
> c:\steven\edk2>c:\steven\unit_test_env\Scripts\activate.bat
>
> (unit_test_env) c:\steven\edk2>pip install --upgrade -r pip-requirements.txt
>
> (unit_test_env) c:\steven\edk2>stuart_setup -c .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019
>
> (unit_test_env) c:\steven\edk2>stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019
>
> (unit_test_env) c:\steven\edk2>edksetup.bat
>
> (unit_test_env) c:\steven\edk2>edksetup.bat Rebuild
>
> (unit_test_env) c:\steven\edk2>stuart_ci_build -c .pytool/CISettings.py TOOL_CHAIN_TAG=VS2019 -a X64,IA32 -t DEBUG --verbose
>
> ...
>
> PROGRESS - Overall Build Status: Success
>
> SECTION - Summary
>
> PROGRESS - Success
>
>
>
> 1. Linux:
>
> jshi19@ub2-uefi-b01:~/wksp_efi$ python3 -m venv unit_test_env
>
> jshi19@ub2-uefi-b01:~/wksp_efi$ source unit_test_env/bin/activate
>
> (unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ pip install --upgrade -r pip-requirements.txt
>
> (unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ make -C BaseTools/
>
> (unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ stuart_setup -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5
>
> (unit_test_env) jshi19@ub2-uefi-b01:~/wksp_efi/edk2-2$ stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5
>
> SECTION - Init SDE
>
> WARNING - Using Pip Tools based BaseTools
>
> SECTION - Loading Plugins
>
> SECTION - Start Invocable Tool
>
> SECTION - Initial update of environment
>
> UpdatingWARNING - [SDE] Failed to fetch NugetDependecy: edk2-acpica-iasl@20200717.0.0<mailto:edk2-acpica-iasl@20200717.0.0>: [Nuget] We failed to install this version 20200717.0.0 of edk2-acpica-iasl
>
> WARNING - [SDE] Failed to fetch NugetDependecy: mu_nasm@2.15.05<mailto:mu_nasm@2.15.05>: [Nuget] We failed to install this version 2.15.05 of mu_nasm
>
> . Done
>
> SECTION - Updated/Verified 3 dependencies
>
> SECTION - Second pass update of environment
>
> UpdatingWARNING - [SDE] Failed to fetch NugetDependecy: edk2-acpica-iasl@20200717.0.0<mailto:edk2-acpica-iasl@20200717.0.0>: [Nuget] We failed to install this version 20200717.0.0 of edk2-acpica-iasl
>
> WARNING - [SDE] Failed to fetch NugetDependecy: mu_nasm@2.15.05<mailto:mu_nasm@2.15.05>: [Nuget] We failed to install this version 2.15.05 of mu_nasm
>
> . Done
>
> SECTION - Updated/Verified 3 dependencies
>
> ERROR - We were unable to successfully update 2 dependencies in environment
>
> SECTION - Summary
>
> ERROR - Error
>
>
>
>
>
>
>
>
>
> Thanks
>
>
>
> Steven Shi
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
^ permalink raw reply [flat|nested] 17+ messages in thread
end of thread, other threads:[~2021-07-07 18:54 UTC | newest]
Thread overview: 17+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2021-06-29 1:45 Do the edk2 CI unit tests work in Linux? Steven Shi
2021-06-29 12:12 ` [edk2-devel] " Rebecca Cran
2021-06-29 15:44 ` Steven Shi
2021-06-29 15:59 ` Bret Barkelew
2021-06-29 16:37 ` Rebecca Cran
2021-06-29 16:42 ` Rebecca Cran
2021-06-29 17:02 ` [EXTERNAL] " Bret Barkelew
2021-07-02 0:54 ` Steven Shi
2021-07-02 1:03 ` Bret Barkelew
2021-07-02 1:36 ` Steven Shi
2021-07-02 1:53 ` Bret Barkelew
2021-07-02 2:00 ` Rebecca Cran
2021-07-02 2:43 ` Steven Shi
2021-07-02 7:23 ` Steven Shi
2021-07-07 18:54 ` Sean
2021-07-02 2:45 ` Bret Barkelew
2021-06-29 15:56 ` Bret Barkelew
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox