* running CI locally @ 2020-08-19 12:27 Laszlo Ersek 2020-08-19 15:29 ` [edk2-devel] " Michael D Kinney 0 siblings, 1 reply; 9+ messages in thread From: Laszlo Ersek @ 2020-08-19 12:27 UTC (permalink / raw) To: edk2-devel-groups-io, Sean Brogan, Bret Barkelew Cc: Vladimir Olovyannikov, Rebecca Cran, Tom Lendacky Hi! I'd like to test CI locally. I'm going through ".pytool/Readme.md" with the tree checked out at 7e6f150b6902 (= current HEAD). I'm doing this in a RHEL8 VM, with a python3 virtual environment set up / entered. * My first note is that the command pip install --upgrade pip-requirements.txt under "Prerequisets", has a small typo; it should be pip install --upgrade -r pip-requirements.txt (the "-r" option is missing). ( After adding "-r", the following components are now installed in my virtual env: - edk2-pytool-library: 0.10.12 - edk2-pytool-extensions: 0.13.9 - antlr4-python3-runtime: 4.7.1 - pyyaml: 5.3.1 Stating this because it might matter for the rest of my email. ) * Second, when I run the following command: stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5 I get the following warnings: > WARNING - [SDE] Failed to fetch NugetDependecy: mu_nasm@2.14.02: [Nuget] We failed to install this version 2.14.02 of mu_nasm > WARNING - [SDE] Failed to fetch NugetDependecy: iasl@20190215.0.0: [Nuget] We failed to install this version 20190215.0.0 of iasl (repeated one more time:) > WARNING - [SDE] Failed to fetch NugetDependecy: mu_nasm@2.14.02: [Nuget] We failed to install this version 2.14.02 of mu_nasm > WARNING - [SDE] Failed to fetch NugetDependecy: iasl@20190215.0.0: [Nuget] We failed to install this version 20190215.0.0 of iasl and then finally: > ERROR - We were unable to successfully update 2 dependencies in environment > ERROR - Error The virtual machine has NASM installed (2.13.03-2.el8) and IASL too (acpica-tools-20180629-3.el8). Where do the NASM and IASL version requirements (2.14.02 and 20190215.0.0, respectively) come from? Hm... After a git-grep for those version numbers, I find: - BaseTools/Bin/nasm_ext_dep.yaml - BaseTools/Bin/iasl_ext_dep.yaml I was about to say that these version requirements are too strict: for example, "BaseTools/Conf/tools_def.template" requires "NASM 2.10 or later for use with the GCC toolchain family". What I have installed satisfies that, and so CI shouldn't require anything more recent. *However*, both of the above YAML files have very helpful comments, so I understand these high versions are downloaded afresh, and only for the CI run. And so my question becomes: why do the "nuget" downloads fail for me (because, presumably, they work fine in the central CI env on github / Azure); and how can I fix the issue if it pops up again? I've checked "nuget.org" in my browser, and it has: - https://www.nuget.org/packages/mu_nasm/ --> 2.14.2 - https://www.nuget.org/packages/iasl/ --> 20190215.0.0 ... On a hunch, I've attempted adding the "--verbose" option to the "stuart_update" command; this is the output (excerpt): > DEBUG - Verify 'mu_nasm' returning 'False'. > DEBUG - Verify 'iasl' returning 'False'. > DEBUG - Creating 4 threads for the SDE update > UpdatingDEBUG - Verify 'mu_nasm' returning 'False'. > DEBUG - Verify 'gcc_aarch64_linux' returning 'True'. > DEBUG - Cleaning dependency directory for 'mu_nasm'... > DEBUG - Verify 'gcc_arm_linux' returning 'True'. > INFO - Cmd to run is: mono /root/py3venv/lib/python3.6/site-packages/edk2toolext/bin/NuGet.exe locals global-packages -list > DEBUG - Verify 'iasl' returning 'False'. > INFO - ------------------------------------------------ > DEBUG - Cleaning dependency directory for 'iasl'... > DEBUG - Verify 'gcc_riscv64_unknown' returning 'True'. > INFO - --------------Cmd Output Starting--------------- > INFO - Cmd to run is: mono /root/py3venv/lib/python3.6/site-packages/edk2toolext/bin/NuGet.exe locals global-packages -list > INFO - ------------------------------------------------ > INFO - ------------------------------------------------ > INFO - --------------Cmd Output Starting--------------- > INFO - ------------------------------------------------ > INFO - /bin/sh: mono: command not found > INFO - /bin/sh: mono: command not found > INFO - ------------------------------------------------ > INFO - --------------Cmd Output Finished--------------- > INFO - --------- Running Time (mm:ss): 00:00 ---------- > INFO - ----------- Return Code: 0x0000007f ------------ > INFO - ------------------------------------------------ > INFO - Nuget was unable to provide global packages cache location. > INFO - Cmd to run is: mono /root/py3venv/lib/python3.6/site-packages/edk2toolext/bin/NuGet.exe install mu_nasm -Source https://api.nuget.org/v3/index.json -ExcludeVersion -NonInteractive -Version 2.14.02 -Verbosity detailed -OutputDirectory "/root/src/rhel8/edk2/BaseTools/Bin/mu_nasm_extdep_temp" > INFO - ------------------------------------------------ > INFO - --------------Cmd Output Starting--------------- > INFO - ------------------------------------------------ > INFO - ------------------------------------------------ > INFO - --------------Cmd Output Finished--------------- > INFO - --------- Running Time (mm:ss): 00:00 ---------- > INFO - ----------- Return Code: 0x0000007f ------------ > INFO - ------------------------------------------------ > INFO - Nuget was unable to provide global packages cache location. > INFO - Cmd to run is: mono /root/py3venv/lib/python3.6/site-packages/edk2toolext/bin/NuGet.exe install iasl -Source https://api.nuget.org/v3/index.json -ExcludeVersion -NonInteractive -Version 20190215.0.0 -Verbosity detailed -OutputDirectory "/root/src/rhel8/edk2/BaseTools/Bin/iasl_extdep_temp" > INFO - ------------------------------------------------ > INFO - --------------Cmd Output Starting--------------- > INFO - /bin/sh: mono: command not found > INFO - ------------------------------------------------ > INFO - ------------------------------------------------ > INFO - --------------Cmd Output Finished--------------- > INFO - --------- Running Time (mm:ss): 00:00 ---------- > INFO - ----------- Return Code: 0x0000007f ------------ > INFO - ------------------------------------------------ > WARNING - [SDE] Failed to fetch NugetDependecy: mu_nasm@2.14.02: [Nuget] We failed to install this version 2.14.02 of mu_nasm Aha! So "mono" is another requirement. After installing the "mono-complete" package from EPEL8 (together with its dependencies), the "stuart_update" command completes fine for me. Can we spell out the "mono" dependency in ".pytool/Readme.md" somewhere, or can we perhaps check for mono automatically? * Third, attempting stuart_ci_build -c .pytool/CISettings.py -p OvmfPkg -a IA32,X64 -t DEBUG TOOL_CHAIN_TAG=GCC5 doesn't seem to do anything: > SECTION - Init SDE > SECTION - Loading Plugins > SECTION - Start Invocable Tool > SECTION - Getting Environment > SECTION - Loading plugins > SECTION - Building OvmfPkg Package > PROGRESS - --Running OvmfPkg: Compiler Plugin DEBUG -- > WARNING - --->Test Skipped: in plugin! Compiler Plugin DEBUG > PROGRESS - Overall Build Status: Success > SECTION - Summary > PROGRESS - Success If I remove the "-t DEBUG" option, a bit more happens (but it also fails): > SECTION - Init SDE > SECTION - Loading Plugins > SECTION - Start Invocable Tool > SECTION - Getting Environment > SECTION - Loading plugins > SECTION - Building OvmfPkg Package > PROGRESS - --Running OvmfPkg: Char Encoding Check Test NO-TARGET -- > PROGRESS - --->Test Success: Char Encoding Check Test NO-TARGET > PROGRESS - --Running OvmfPkg: Compiler Plugin DEBUG -- > WARNING - --->Test Skipped: in plugin! Compiler Plugin DEBUG > PROGRESS - --Running OvmfPkg: Compiler Plugin RELEASE -- > WARNING - --->Test Skipped: in plugin! Compiler Plugin RELEASE > PROGRESS - --Running OvmfPkg: Dependency Check Test NO-TARGET -- > PROGRESS - --->Test Success: Dependency Check Test NO-TARGET > PROGRESS - --Running OvmfPkg: Dsc Complete Check Test NO-TARGET -- > PROGRESS - --->Test Success: Dsc Complete Check Test NO-TARGET > PROGRESS - --Running OvmfPkg: Guid Check Test NO-TARGET -- > PROGRESS - --->Test Success: Guid Check Test NO-TARGET > PROGRESS - --Running OvmfPkg: Library Class Check Test NO-TARGET -- > PROGRESS - --->Test Success: Library Class Check Test NO-TARGET > PROGRESS - --Running OvmfPkg: Spell Check Test NO-TARGET -- > WARNING - NodeJs not installed. Test can't run > WARNING - --->Test Skipped: in plugin! Spell Check Test NO-TARGET > PROGRESS - --Running OvmfPkg: EccCheck Test NO-TARGET -- > PROGRESS - --->Test Success: EccCheck Test NO-TARGET > PROGRESS - --Running OvmfPkg: License Check Test NO-TARGET -- > PROGRESS - --->Test Success: License Check Test NO-TARGET > PROGRESS - --Running OvmfPkg: Host Unit Test Compiler Plugin NOOPT -- > WARNING - --->Test Skipped: in plugin! Host Unit Test Compiler Plugin NOOPT > PROGRESS - --Running OvmfPkg: Host Unit Test Dsc Complete Check Test NO-TARGET -- > PROGRESS - --->Test Success: Host Unit Test Dsc Complete Check Test NO-TARGET > PROGRESS - Overall Build Status: Success > SECTION - Summary > PROGRESS - Success I'm not sure about the "NodeJs not installed. Test can't run" message -- I guess it only relates to the "Spell Check Test". But what's missing for the actual build? I've noticed the following warning: > Test Skipped: in plugin! Compiler Plugin DEBUG But even if I add "--verbose", the only lines around that warning are: > INFO - Running on Package: OvmfPkg > PROGRESS - --Running OvmfPkg: Compiler Plugin DEBUG -- > WARNING - --->Test Skipped: in plugin! Compiler Plugin DEBUG > PROGRESS - Overall Build Status: Success Why is the build skipped? --*-- Summary: - the typo in "pip install --upgrade -r pip-requirements.txt" should be fixed - "mono" should be documented, or auto-installed, as a dependency - I don't know why "stuart_ci_build" skips the actual build. Thanks! Laszlo ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [edk2-devel] running CI locally 2020-08-19 12:27 running CI locally Laszlo Ersek @ 2020-08-19 15:29 ` Michael D Kinney 2020-08-19 16:22 ` Sean 2020-08-19 17:56 ` Laszlo Ersek 0 siblings, 2 replies; 9+ messages in thread From: Michael D Kinney @ 2020-08-19 15:29 UTC (permalink / raw) To: devel@edk2.groups.io, lersek@redhat.com, Sean Brogan, Bret Barkelew, Kinney, Michael D Cc: Vladimir Olovyannikov, Rebecca Cran, Tom Lendacky Hi Laszlo, Thank you for the feedback. I agree there are some documentation updates required. The spell check requirements are documented here: https://github.com/tianocore/edk2/blob/master/.pytool/Readme.md#spell-checking---cspell You do need to install nodejs and cspell. * Install nodejs from https://nodejs.org/en/ * Install cspell 1. Open cmd prompt with access to node and npm 2. Run `npm install -g cspell` I agree that the verbosity of the output is very brief. There is a more complete log that is always produced in the build output directory in Build/CI_BUILDLOG.txt. Can you look at that file and see if it provide a more detailed reason for the failure? Thanks, Mike > -----Original Message----- > From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Laszlo Ersek > Sent: Wednesday, August 19, 2020 5:27 AM > To: edk2-devel-groups-io <devel@edk2.groups.io>; Sean Brogan <sean.brogan@microsoft.com>; Bret Barkelew > <Bret.Barkelew@microsoft.com> > Cc: Vladimir Olovyannikov <vladimir.olovyannikov@broadcom.com>; Rebecca Cran <rebecca@bsdio.com>; Tom Lendacky > <thomas.lendacky@amd.com> > Subject: [edk2-devel] running CI locally > > Hi! > > I'd like to test CI locally. I'm going through ".pytool/Readme.md" with > the tree checked out at 7e6f150b6902 (= current HEAD). I'm doing this in > a RHEL8 VM, with a python3 virtual environment set up / entered. > > > * My first note is that the command > > pip install --upgrade pip-requirements.txt > > under "Prerequisets", has a small typo; it should be > > pip install --upgrade -r pip-requirements.txt > > (the "-r" option is missing). > > ( > > After adding "-r", the following components are now installed in my > virtual env: > > - edk2-pytool-library: 0.10.12 > - edk2-pytool-extensions: 0.13.9 > - antlr4-python3-runtime: 4.7.1 > - pyyaml: 5.3.1 > > Stating this because it might matter for the rest of my email. > > ) > > > * Second, when I run the following command: > > stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5 > > I get the following warnings: > > > WARNING - [SDE] Failed to fetch NugetDependecy: mu_nasm@2.14.02: [Nuget] We failed to install this version 2.14.02 of mu_nasm > > WARNING - [SDE] Failed to fetch NugetDependecy: iasl@20190215.0.0: [Nuget] We failed to install this version 20190215.0.0 of iasl > > (repeated one more time:) > > > WARNING - [SDE] Failed to fetch NugetDependecy: mu_nasm@2.14.02: [Nuget] We failed to install this version 2.14.02 of mu_nasm > > WARNING - [SDE] Failed to fetch NugetDependecy: iasl@20190215.0.0: [Nuget] We failed to install this version 20190215.0.0 of iasl > > and then finally: > > > ERROR - We were unable to successfully update 2 dependencies in environment > > ERROR - Error > > The virtual machine has NASM installed (2.13.03-2.el8) and IASL too > (acpica-tools-20180629-3.el8). > > Where do the NASM and IASL version requirements (2.14.02 and > 20190215.0.0, respectively) come from? > > Hm... After a git-grep for those version numbers, I find: > > - BaseTools/Bin/nasm_ext_dep.yaml > - BaseTools/Bin/iasl_ext_dep.yaml > > I was about to say that these version requirements are too strict: for > example, "BaseTools/Conf/tools_def.template" requires "NASM 2.10 or > later for use with the GCC toolchain family". What I have installed > satisfies that, and so CI shouldn't require anything more recent. > *However*, both of the above YAML files have very helpful comments, so I > understand these high versions are downloaded afresh, and only for the > CI run. > > And so my question becomes: why do the "nuget" downloads fail for me > (because, presumably, they work fine in the central CI env on github / > Azure); and how can I fix the issue if it pops up again? > > I've checked "nuget.org" in my browser, and it has: > > - https://www.nuget.org/packages/mu_nasm/ --> 2.14.2 > - https://www.nuget.org/packages/iasl/ --> 20190215.0.0 > > ... On a hunch, I've attempted adding the "--verbose" option to the > "stuart_update" command; this is the output (excerpt): > > > DEBUG - Verify 'mu_nasm' returning 'False'. > > DEBUG - Verify 'iasl' returning 'False'. > > DEBUG - Creating 4 threads for the SDE update > > UpdatingDEBUG - Verify 'mu_nasm' returning 'False'. > > DEBUG - Verify 'gcc_aarch64_linux' returning 'True'. > > DEBUG - Cleaning dependency directory for 'mu_nasm'... > > DEBUG - Verify 'gcc_arm_linux' returning 'True'. > > INFO - Cmd to run is: mono /root/py3venv/lib/python3.6/site-packages/edk2toolext/bin/NuGet.exe locals global-packages -list > > DEBUG - Verify 'iasl' returning 'False'. > > INFO - ------------------------------------------------ > > DEBUG - Cleaning dependency directory for 'iasl'... > > DEBUG - Verify 'gcc_riscv64_unknown' returning 'True'. > > INFO - --------------Cmd Output Starting--------------- > > INFO - Cmd to run is: mono /root/py3venv/lib/python3.6/site-packages/edk2toolext/bin/NuGet.exe locals global-packages -list > > INFO - ------------------------------------------------ > > INFO - ------------------------------------------------ > > INFO - --------------Cmd Output Starting--------------- > > INFO - ------------------------------------------------ > > INFO - /bin/sh: mono: command not found > > INFO - /bin/sh: mono: command not found > > INFO - ------------------------------------------------ > > INFO - --------------Cmd Output Finished--------------- > > INFO - --------- Running Time (mm:ss): 00:00 ---------- > > INFO - ----------- Return Code: 0x0000007f ------------ > > INFO - ------------------------------------------------ > > INFO - Nuget was unable to provide global packages cache location. > > INFO - Cmd to run is: mono /root/py3venv/lib/python3.6/site-packages/edk2toolext/bin/NuGet.exe install mu_nasm -Source > https://api.nuget.org/v3/index.json -ExcludeVersion -NonInteractive -Version 2.14.02 -Verbosity detailed -OutputDirectory > "/root/src/rhel8/edk2/BaseTools/Bin/mu_nasm_extdep_temp" > > INFO - ------------------------------------------------ > > INFO - --------------Cmd Output Starting--------------- > > INFO - ------------------------------------------------ > > INFO - ------------------------------------------------ > > INFO - --------------Cmd Output Finished--------------- > > INFO - --------- Running Time (mm:ss): 00:00 ---------- > > INFO - ----------- Return Code: 0x0000007f ------------ > > INFO - ------------------------------------------------ > > INFO - Nuget was unable to provide global packages cache location. > > INFO - Cmd to run is: mono /root/py3venv/lib/python3.6/site-packages/edk2toolext/bin/NuGet.exe install iasl -Source > https://api.nuget.org/v3/index.json -ExcludeVersion -NonInteractive -Version 20190215.0.0 -Verbosity detailed -OutputDirectory > "/root/src/rhel8/edk2/BaseTools/Bin/iasl_extdep_temp" > > INFO - ------------------------------------------------ > > INFO - --------------Cmd Output Starting--------------- > > INFO - /bin/sh: mono: command not found > > INFO - ------------------------------------------------ > > INFO - ------------------------------------------------ > > INFO - --------------Cmd Output Finished--------------- > > INFO - --------- Running Time (mm:ss): 00:00 ---------- > > INFO - ----------- Return Code: 0x0000007f ------------ > > INFO - ------------------------------------------------ > > WARNING - [SDE] Failed to fetch NugetDependecy: mu_nasm@2.14.02: [Nuget] We failed to install this version 2.14.02 of mu_nasm > > Aha! So "mono" is another requirement. > > After installing the "mono-complete" package from EPEL8 (together with > its dependencies), the "stuart_update" command completes fine for me. > > Can we spell out the "mono" dependency in ".pytool/Readme.md" somewhere, > or can we perhaps check for mono automatically? > > > * Third, attempting > > stuart_ci_build -c .pytool/CISettings.py -p OvmfPkg -a IA32,X64 -t DEBUG TOOL_CHAIN_TAG=GCC5 > > doesn't seem to do anything: > > > SECTION - Init SDE > > SECTION - Loading Plugins > > SECTION - Start Invocable Tool > > SECTION - Getting Environment > > SECTION - Loading plugins > > SECTION - Building OvmfPkg Package > > PROGRESS - --Running OvmfPkg: Compiler Plugin DEBUG -- > > WARNING - --->Test Skipped: in plugin! Compiler Plugin DEBUG > > PROGRESS - Overall Build Status: Success > > SECTION - Summary > > PROGRESS - Success > > If I remove the "-t DEBUG" option, a bit more happens (but it also > fails): > > > SECTION - Init SDE > > SECTION - Loading Plugins > > SECTION - Start Invocable Tool > > SECTION - Getting Environment > > SECTION - Loading plugins > > SECTION - Building OvmfPkg Package > > PROGRESS - --Running OvmfPkg: Char Encoding Check Test NO-TARGET -- > > PROGRESS - --->Test Success: Char Encoding Check Test NO-TARGET > > PROGRESS - --Running OvmfPkg: Compiler Plugin DEBUG -- > > WARNING - --->Test Skipped: in plugin! Compiler Plugin DEBUG > > PROGRESS - --Running OvmfPkg: Compiler Plugin RELEASE -- > > WARNING - --->Test Skipped: in plugin! Compiler Plugin RELEASE > > PROGRESS - --Running OvmfPkg: Dependency Check Test NO-TARGET -- > > PROGRESS - --->Test Success: Dependency Check Test NO-TARGET > > PROGRESS - --Running OvmfPkg: Dsc Complete Check Test NO-TARGET -- > > PROGRESS - --->Test Success: Dsc Complete Check Test NO-TARGET > > PROGRESS - --Running OvmfPkg: Guid Check Test NO-TARGET -- > > PROGRESS - --->Test Success: Guid Check Test NO-TARGET > > PROGRESS - --Running OvmfPkg: Library Class Check Test NO-TARGET -- > > PROGRESS - --->Test Success: Library Class Check Test NO-TARGET > > PROGRESS - --Running OvmfPkg: Spell Check Test NO-TARGET -- > > WARNING - NodeJs not installed. Test can't run > > WARNING - --->Test Skipped: in plugin! Spell Check Test NO-TARGET > > PROGRESS - --Running OvmfPkg: EccCheck Test NO-TARGET -- > > PROGRESS - --->Test Success: EccCheck Test NO-TARGET > > PROGRESS - --Running OvmfPkg: License Check Test NO-TARGET -- > > PROGRESS - --->Test Success: License Check Test NO-TARGET > > PROGRESS - --Running OvmfPkg: Host Unit Test Compiler Plugin NOOPT -- > > WARNING - --->Test Skipped: in plugin! Host Unit Test Compiler Plugin NOOPT > > PROGRESS - --Running OvmfPkg: Host Unit Test Dsc Complete Check Test NO-TARGET -- > > PROGRESS - --->Test Success: Host Unit Test Dsc Complete Check Test NO-TARGET > > PROGRESS - Overall Build Status: Success > > SECTION - Summary > > PROGRESS - Success > > I'm not sure about the "NodeJs not installed. Test can't run" message -- > I guess it only relates to the "Spell Check Test". > > But what's missing for the actual build? I've noticed the following > warning: > > > Test Skipped: in plugin! Compiler Plugin DEBUG > > But even if I add "--verbose", the only lines around that warning are: > > > INFO - Running on Package: OvmfPkg > > PROGRESS - --Running OvmfPkg: Compiler Plugin DEBUG -- > > WARNING - --->Test Skipped: in plugin! Compiler Plugin DEBUG > > PROGRESS - Overall Build Status: Success > > Why is the build skipped? > > --*-- > > Summary: > > - the typo in "pip install --upgrade -r pip-requirements.txt" should be fixed > > - "mono" should be documented, or auto-installed, as a dependency > > - I don't know why "stuart_ci_build" skips the actual build. > > Thanks! > Laszlo > > > ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [edk2-devel] running CI locally 2020-08-19 15:29 ` [edk2-devel] " Michael D Kinney @ 2020-08-19 16:22 ` Sean 2020-08-19 17:59 ` Laszlo Ersek 2020-08-19 17:56 ` Laszlo Ersek 1 sibling, 1 reply; 9+ messages in thread From: Sean @ 2020-08-19 16:22 UTC (permalink / raw) To: devel, michael.d.kinney, lersek@redhat.com, Sean Brogan, Bret Barkelew Cc: Vladimir Olovyannikov, Rebecca Cran, Tom Lendacky Laszlo/Mike, This is the joy of distributed repositories. Remember edk2 ci is actually using edk2-pytool-extensions and edk2-pytool-library. Documentation is in those projects. https://github.com/tianocore/edk2-pytool-extensions/tree/master/docs https://github.com/tianocore/edk2-pytool-library/tree/master/docs I won't say they are great and I hope someday relatively soon we can talk about an edk2 static site generator that can include docs from multiple repositories (much like https://microsoft.github.io/mu/) as I think documentation on edk2 is a weak spot. Regarding Mono and nuget. It is one of those things we wish was different as we have found it very inconsistent across different distributions of linux. But we do have docs here. https://github.com/tianocore/edk2-pytool-extensions/blob/master/docs/usability/using_linux.md And more specifically here: https://github.com/tianocore/edk2-pytool-extensions/blob/master/docs/usability/using_extdep.md#a-note-on-nuget-on-linux Finally getting to why you don't run the compile. Core CI (stuart_ci_build) is a plugin runner. Compile test is just one of those tests. OvmfPkg is a platform and thus didn't opt into core ci compile testing. In the table here i tried to document that OvmfPkg didn't compile using core ci and that a user should look at the readme. https://github.com/tianocore/edk2/blob/master/.pytool/Readme.md#basic-status So for OvmfPkg we enabled what we call platform ci (stuart_build). I think the write up here is pretty complete (although i see it has no mention of mono either). https://github.com/tianocore/edk2/tree/master/OvmfPkg/PlatformCI One final note. Yes the logging to console is by default very brief. This is by design as it is easier to quickly look and see what test failed and then use the log file (as mike mentioned) to find the root cause. Hope that helps and it is great to see people using it. Feedback is much appreciated. Thanks Sean On 8/19/2020 8:29 AM, Michael D Kinney wrote: > Hi Laszlo, > > Thank you for the feedback. I agree there are some documentation updates required. > > The spell check requirements are documented here: > > https://github.com/tianocore/edk2/blob/master/.pytool/Readme.md#spell-checking---cspell > > You do need to install nodejs and cspell. > > * Install nodejs from https://nodejs.org/en/ > * Install cspell > 1. Open cmd prompt with access to node and npm > 2. Run `npm install -g cspell` > > I agree that the verbosity of the output is very brief. There is a more complete > log that is always produced in the build output directory in Build/CI_BUILDLOG.txt. > Can you look at that file and see if it provide a more detailed reason for the > failure? > > Thanks, > > Mike > > >> -----Original Message----- >> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Laszlo Ersek >> Sent: Wednesday, August 19, 2020 5:27 AM >> To: edk2-devel-groups-io <devel@edk2.groups.io>; Sean Brogan <sean.brogan@microsoft.com>; Bret Barkelew >> <Bret.Barkelew@microsoft.com> >> Cc: Vladimir Olovyannikov <vladimir.olovyannikov@broadcom.com>; Rebecca Cran <rebecca@bsdio.com>; Tom Lendacky >> <thomas.lendacky@amd.com> >> Subject: [edk2-devel] running CI locally >> >> Hi! >> >> I'd like to test CI locally. I'm going through ".pytool/Readme.md" with >> the tree checked out at 7e6f150b6902 (= current HEAD). I'm doing this in >> a RHEL8 VM, with a python3 virtual environment set up / entered. >> >> >> * My first note is that the command >> >> pip install --upgrade pip-requirements.txt >> >> under "Prerequisets", has a small typo; it should be >> >> pip install --upgrade -r pip-requirements.txt >> >> (the "-r" option is missing). >> >> ( >> >> After adding "-r", the following components are now installed in my >> virtual env: >> >> - edk2-pytool-library: 0.10.12 >> - edk2-pytool-extensions: 0.13.9 >> - antlr4-python3-runtime: 4.7.1 >> - pyyaml: 5.3.1 >> >> Stating this because it might matter for the rest of my email. >> >> ) >> >> >> * Second, when I run the following command: >> >> stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5 >> >> I get the following warnings: >> >>> WARNING - [SDE] Failed to fetch NugetDependecy: mu_nasm@2.14.02: [Nuget] We failed to install this version 2.14.02 of mu_nasm >>> WARNING - [SDE] Failed to fetch NugetDependecy: iasl@20190215.0.0: [Nuget] We failed to install this version 20190215.0.0 of iasl >> >> (repeated one more time:) >> >>> WARNING - [SDE] Failed to fetch NugetDependecy: mu_nasm@2.14.02: [Nuget] We failed to install this version 2.14.02 of mu_nasm >>> WARNING - [SDE] Failed to fetch NugetDependecy: iasl@20190215.0.0: [Nuget] We failed to install this version 20190215.0.0 of iasl >> >> and then finally: >> >>> ERROR - We were unable to successfully update 2 dependencies in environment >>> ERROR - Error >> >> The virtual machine has NASM installed (2.13.03-2.el8) and IASL too >> (acpica-tools-20180629-3.el8). >> >> Where do the NASM and IASL version requirements (2.14.02 and >> 20190215.0.0, respectively) come from? >> >> Hm... After a git-grep for those version numbers, I find: >> >> - BaseTools/Bin/nasm_ext_dep.yaml >> - BaseTools/Bin/iasl_ext_dep.yaml >> >> I was about to say that these version requirements are too strict: for >> example, "BaseTools/Conf/tools_def.template" requires "NASM 2.10 or >> later for use with the GCC toolchain family". What I have installed >> satisfies that, and so CI shouldn't require anything more recent. >> *However*, both of the above YAML files have very helpful comments, so I >> understand these high versions are downloaded afresh, and only for the >> CI run. >> >> And so my question becomes: why do the "nuget" downloads fail for me >> (because, presumably, they work fine in the central CI env on github / >> Azure); and how can I fix the issue if it pops up again? >> >> I've checked "nuget.org" in my browser, and it has: >> >> - https://www.nuget.org/packages/mu_nasm/ --> 2.14.2 >> - https://www.nuget.org/packages/iasl/ --> 20190215.0.0 >> >> ... On a hunch, I've attempted adding the "--verbose" option to the >> "stuart_update" command; this is the output (excerpt): >> >>> DEBUG - Verify 'mu_nasm' returning 'False'. >>> DEBUG - Verify 'iasl' returning 'False'. >>> DEBUG - Creating 4 threads for the SDE update >>> UpdatingDEBUG - Verify 'mu_nasm' returning 'False'. >>> DEBUG - Verify 'gcc_aarch64_linux' returning 'True'. >>> DEBUG - Cleaning dependency directory for 'mu_nasm'... >>> DEBUG - Verify 'gcc_arm_linux' returning 'True'. >>> INFO - Cmd to run is: mono /root/py3venv/lib/python3.6/site-packages/edk2toolext/bin/NuGet.exe locals global-packages -list >>> DEBUG - Verify 'iasl' returning 'False'. >>> INFO - ------------------------------------------------ >>> DEBUG - Cleaning dependency directory for 'iasl'... >>> DEBUG - Verify 'gcc_riscv64_unknown' returning 'True'. >>> INFO - --------------Cmd Output Starting--------------- >>> INFO - Cmd to run is: mono /root/py3venv/lib/python3.6/site-packages/edk2toolext/bin/NuGet.exe locals global-packages -list >>> INFO - ------------------------------------------------ >>> INFO - ------------------------------------------------ >>> INFO - --------------Cmd Output Starting--------------- >>> INFO - ------------------------------------------------ >>> INFO - /bin/sh: mono: command not found >>> INFO - /bin/sh: mono: command not found >>> INFO - ------------------------------------------------ >>> INFO - --------------Cmd Output Finished--------------- >>> INFO - --------- Running Time (mm:ss): 00:00 ---------- >>> INFO - ----------- Return Code: 0x0000007f ------------ >>> INFO - ------------------------------------------------ >>> INFO - Nuget was unable to provide global packages cache location. >>> INFO - Cmd to run is: mono /root/py3venv/lib/python3.6/site-packages/edk2toolext/bin/NuGet.exe install mu_nasm -Source >> https://api.nuget.org/v3/index.json -ExcludeVersion -NonInteractive -Version 2.14.02 -Verbosity detailed -OutputDirectory >> "/root/src/rhel8/edk2/BaseTools/Bin/mu_nasm_extdep_temp" >>> INFO - ------------------------------------------------ >>> INFO - --------------Cmd Output Starting--------------- >>> INFO - ------------------------------------------------ >>> INFO - ------------------------------------------------ >>> INFO - --------------Cmd Output Finished--------------- >>> INFO - --------- Running Time (mm:ss): 00:00 ---------- >>> INFO - ----------- Return Code: 0x0000007f ------------ >>> INFO - ------------------------------------------------ >>> INFO - Nuget was unable to provide global packages cache location. >>> INFO - Cmd to run is: mono /root/py3venv/lib/python3.6/site-packages/edk2toolext/bin/NuGet.exe install iasl -Source >> https://api.nuget.org/v3/index.json -ExcludeVersion -NonInteractive -Version 20190215.0.0 -Verbosity detailed -OutputDirectory >> "/root/src/rhel8/edk2/BaseTools/Bin/iasl_extdep_temp" >>> INFO - ------------------------------------------------ >>> INFO - --------------Cmd Output Starting--------------- >>> INFO - /bin/sh: mono: command not found >>> INFO - ------------------------------------------------ >>> INFO - ------------------------------------------------ >>> INFO - --------------Cmd Output Finished--------------- >>> INFO - --------- Running Time (mm:ss): 00:00 ---------- >>> INFO - ----------- Return Code: 0x0000007f ------------ >>> INFO - ------------------------------------------------ >>> WARNING - [SDE] Failed to fetch NugetDependecy: mu_nasm@2.14.02: [Nuget] We failed to install this version 2.14.02 of mu_nasm >> >> Aha! So "mono" is another requirement. >> >> After installing the "mono-complete" package from EPEL8 (together with >> its dependencies), the "stuart_update" command completes fine for me. >> >> Can we spell out the "mono" dependency in ".pytool/Readme.md" somewhere, >> or can we perhaps check for mono automatically? >> >> >> * Third, attempting >> >> stuart_ci_build -c .pytool/CISettings.py -p OvmfPkg -a IA32,X64 -t DEBUG TOOL_CHAIN_TAG=GCC5 >> >> doesn't seem to do anything: >> >>> SECTION - Init SDE >>> SECTION - Loading Plugins >>> SECTION - Start Invocable Tool >>> SECTION - Getting Environment >>> SECTION - Loading plugins >>> SECTION - Building OvmfPkg Package >>> PROGRESS - --Running OvmfPkg: Compiler Plugin DEBUG -- >>> WARNING - --->Test Skipped: in plugin! Compiler Plugin DEBUG >>> PROGRESS - Overall Build Status: Success >>> SECTION - Summary >>> PROGRESS - Success >> >> If I remove the "-t DEBUG" option, a bit more happens (but it also >> fails): >> >>> SECTION - Init SDE >>> SECTION - Loading Plugins >>> SECTION - Start Invocable Tool >>> SECTION - Getting Environment >>> SECTION - Loading plugins >>> SECTION - Building OvmfPkg Package >>> PROGRESS - --Running OvmfPkg: Char Encoding Check Test NO-TARGET -- >>> PROGRESS - --->Test Success: Char Encoding Check Test NO-TARGET >>> PROGRESS - --Running OvmfPkg: Compiler Plugin DEBUG -- >>> WARNING - --->Test Skipped: in plugin! Compiler Plugin DEBUG >>> PROGRESS - --Running OvmfPkg: Compiler Plugin RELEASE -- >>> WARNING - --->Test Skipped: in plugin! Compiler Plugin RELEASE >>> PROGRESS - --Running OvmfPkg: Dependency Check Test NO-TARGET -- >>> PROGRESS - --->Test Success: Dependency Check Test NO-TARGET >>> PROGRESS - --Running OvmfPkg: Dsc Complete Check Test NO-TARGET -- >>> PROGRESS - --->Test Success: Dsc Complete Check Test NO-TARGET >>> PROGRESS - --Running OvmfPkg: Guid Check Test NO-TARGET -- >>> PROGRESS - --->Test Success: Guid Check Test NO-TARGET >>> PROGRESS - --Running OvmfPkg: Library Class Check Test NO-TARGET -- >>> PROGRESS - --->Test Success: Library Class Check Test NO-TARGET >>> PROGRESS - --Running OvmfPkg: Spell Check Test NO-TARGET -- >>> WARNING - NodeJs not installed. Test can't run >>> WARNING - --->Test Skipped: in plugin! Spell Check Test NO-TARGET >>> PROGRESS - --Running OvmfPkg: EccCheck Test NO-TARGET -- >>> PROGRESS - --->Test Success: EccCheck Test NO-TARGET >>> PROGRESS - --Running OvmfPkg: License Check Test NO-TARGET -- >>> PROGRESS - --->Test Success: License Check Test NO-TARGET >>> PROGRESS - --Running OvmfPkg: Host Unit Test Compiler Plugin NOOPT -- >>> WARNING - --->Test Skipped: in plugin! Host Unit Test Compiler Plugin NOOPT >>> PROGRESS - --Running OvmfPkg: Host Unit Test Dsc Complete Check Test NO-TARGET -- >>> PROGRESS - --->Test Success: Host Unit Test Dsc Complete Check Test NO-TARGET >>> PROGRESS - Overall Build Status: Success >>> SECTION - Summary >>> PROGRESS - Success >> >> I'm not sure about the "NodeJs not installed. Test can't run" message -- >> I guess it only relates to the "Spell Check Test". >> >> But what's missing for the actual build? I've noticed the following >> warning: >> >>> Test Skipped: in plugin! Compiler Plugin DEBUG >> >> But even if I add "--verbose", the only lines around that warning are: >> >>> INFO - Running on Package: OvmfPkg >>> PROGRESS - --Running OvmfPkg: Compiler Plugin DEBUG -- >>> WARNING - --->Test Skipped: in plugin! Compiler Plugin DEBUG >>> PROGRESS - Overall Build Status: Success >> >> Why is the build skipped? >> >> --*-- >> >> Summary: >> >> - the typo in "pip install --upgrade -r pip-requirements.txt" should be fixed >> >> - "mono" should be documented, or auto-installed, as a dependency >> >> - I don't know why "stuart_ci_build" skips the actual build. >> >> Thanks! >> Laszlo >> >> >> > > > > ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [edk2-devel] running CI locally 2020-08-19 16:22 ` Sean @ 2020-08-19 17:59 ` Laszlo Ersek 2020-08-21 7:23 ` Laszlo Ersek 0 siblings, 1 reply; 9+ messages in thread From: Laszlo Ersek @ 2020-08-19 17:59 UTC (permalink / raw) To: Sean Brogan, devel, michael.d.kinney, Sean Brogan, Bret Barkelew Cc: Vladimir Olovyannikov, Rebecca Cran, Tom Lendacky On 08/19/20 18:22, Sean Brogan wrote: > Laszlo/Mike, > > This is the joy of distributed repositories. > Remember edk2 ci is actually using edk2-pytool-extensions and > edk2-pytool-library. Documentation is in those projects. > > https://github.com/tianocore/edk2-pytool-extensions/tree/master/docs > https://github.com/tianocore/edk2-pytool-library/tree/master/docs > > I won't say they are great and I hope someday relatively soon we can > talk about an edk2 static site generator that can include docs from > multiple repositories (much like https://microsoft.github.io/mu/) as I > think documentation on edk2 is a weak spot. > > Regarding Mono and nuget. It is one of those things we wish was > different as we have found it very inconsistent across different > distributions of linux. But we do have docs here. > https://github.com/tianocore/edk2-pytool-extensions/blob/master/docs/usability/using_linux.md > > > And more specifically here: > https://github.com/tianocore/edk2-pytool-extensions/blob/master/docs/usability/using_extdep.md#a-note-on-nuget-on-linux > > > > Finally getting to why you don't run the compile. > Core CI (stuart_ci_build) is a plugin runner. Compile test is just one > of those tests. OvmfPkg is a platform and thus didn't opt into core ci > compile testing. In the table here i tried to document that OvmfPkg > didn't compile using core ci and that a user should look at the readme. > https://github.com/tianocore/edk2/blob/master/.pytool/Readme.md#basic-status > > > So for OvmfPkg we enabled what we call platform ci (stuart_build). I > think the write up here is pretty complete (although i see it has no > mention of mono either). > https://github.com/tianocore/edk2/tree/master/OvmfPkg/PlatformCI > > One final note. Yes the logging to console is by default very brief. > This is by design as it is easier to quickly look and see what test > failed and then use the log file (as mike mentioned) to find the root > cause. > > Hope that helps and it is great to see people using it. Feedback is > much appreciated. The documentation looks *awesome*, and it's entirely my fault that I couldn't find it. I apologize. I'll report back with more results. Thanks! Laszlo ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [edk2-devel] running CI locally 2020-08-19 17:59 ` Laszlo Ersek @ 2020-08-21 7:23 ` Laszlo Ersek 2020-08-21 21:36 ` Sean [not found] ` <162D662A3371B4DC.7353@groups.io> 0 siblings, 2 replies; 9+ messages in thread From: Laszlo Ersek @ 2020-08-21 7:23 UTC (permalink / raw) To: Sean Brogan, devel, michael.d.kinney, Sean Brogan, Bret Barkelew Cc: Vladimir Olovyannikov, Rebecca Cran, Tom Lendacky Hi Mike, Sean; On 08/19/20 19:59, Laszlo Ersek wrote: > I'll report back with more results. I installed a Fedora 32 Server VM. Installed the "mono-complete" package (+its deps). Installed the "nodejs" package. Installed "cspell" (+deps) with "npm" (with the latter coming from the nodejs package). Edk2 checked out at current HEAD (5a6d764e1d073d28e8f398289ccb5592bf9a72ba). * Platform CI results: - <https://github.com/tianocore/edk2/tree/master/OvmfPkg/PlatformCI>: Built successfully for IA32, X64, and IA32X64 (toolchain: GCC5). - <https://github.com/tianocore/edk2/tree/master/ArmVirtPkg/PlatformCI>: Built successfully for ARM and AARCH64 (toolchain: GCC5). * Core CI results: - <https://github.com/tianocore/edk2/blob/master/.pytool/Readme.md> Complete run successful (toolchain: GCC5); no package / arch / target / test restrictions. I got the following WARNINGs: - ArmVirtPkg, EmulatorPkg, and OvmfPkg are covered by Platform CI, not Core CI -- these are expected: > PROGRESS - --Running ArmVirtPkg: Compiler Plugin DEBUG -- > WARNING - --->Test Skipped: in plugin! Compiler Plugin DEBUG > PROGRESS - --Running ArmVirtPkg: Compiler Plugin RELEASE -- > WARNING - --->Test Skipped: in plugin! Compiler Plugin RELEASE > PROGRESS - --Running EmulatorPkg: Compiler Plugin DEBUG -- > WARNING - --->Test Skipped: in plugin! Compiler Plugin DEBUG > PROGRESS - --Running EmulatorPkg: Compiler Plugin RELEASE -- > WARNING - --->Test Skipped: in plugin! Compiler Plugin RELEASE > PROGRESS - --Running OvmfPkg: Compiler Plugin DEBUG -- > WARNING - --->Test Skipped: in plugin! Compiler Plugin DEBUG > PROGRESS - --Running OvmfPkg: Compiler Plugin RELEASE -- > WARNING - --->Test Skipped: in plugin! Compiler Plugin RELEASE - ArmVirtPkg, DynamicTablesPkg, CryptoPkg, EmulatorPkg, FatPkg, NetworkPkg, OvmfPkg, PcAtChipsetPkg, SecurityPkg and ShellPkg are not covered by host-based unit tests -- I think also expected: > PROGRESS - --Running ArmVirtPkg: Host Unit Test Compiler Plugin NOOPT -- > WARNING - --->Test Skipped: in plugin! Host Unit Test Compiler Plugin NOOPT > PROGRESS - --Running DynamicTablesPkg: Host Unit Test Compiler Plugin NOOPT -- > WARNING - --->Test Skipped: in plugin! Host Unit Test Compiler Plugin NOOPT > PROGRESS - --Running CryptoPkg: Host Unit Test Compiler Plugin NOOPT -- > WARNING - --->Test Skipped: in plugin! Host Unit Test Compiler Plugin NOOPT > PROGRESS - --Running EmulatorPkg: Host Unit Test Compiler Plugin NOOPT -- > WARNING - --->Test Skipped: in plugin! Host Unit Test Compiler Plugin NOOPT > PROGRESS - --Running FatPkg: Host Unit Test Compiler Plugin NOOPT -- > WARNING - --->Test Skipped: in plugin! Host Unit Test Compiler Plugin NOOPT > PROGRESS - --Running NetworkPkg: Host Unit Test Compiler Plugin NOOPT -- > WARNING - --->Test Skipped: in plugin! Host Unit Test Compiler Plugin NOOPT > PROGRESS - --Running OvmfPkg: Host Unit Test Compiler Plugin NOOPT -- > WARNING - --->Test Skipped: in plugin! Host Unit Test Compiler Plugin NOOPT > PROGRESS - --Running PcAtChipsetPkg: Host Unit Test Compiler Plugin NOOPT -- > WARNING - --->Test Skipped: in plugin! Host Unit Test Compiler Plugin NOOPT > PROGRESS - --Running SecurityPkg: Host Unit Test Compiler Plugin NOOPT -- > WARNING - --->Test Skipped: in plugin! Host Unit Test Compiler Plugin NOOPT > PROGRESS - --Running ShellPkg: Host Unit Test Compiler Plugin NOOPT -- > WARNING - --->Test Skipped: in plugin! Host Unit Test Compiler Plugin NOOPT - I'm not really sure about the following warnings. They were emitted for a subset of the above packages. I read the docs at <https://github.com/tianocore/edk2/blob/master/.pytool/Readme.md#host-module-inclusion-test---hostunittestdsccompletecheck> but I still don't understand :) > PROGRESS - --Running CryptoPkg: Host Unit Test Dsc Complete Check Test NO-TARGET -- > WARNING - --->Test Skipped: in plugin! Host Unit Test Dsc Complete Check Test NO-TARGET > PROGRESS - --Running FatPkg: Host Unit Test Dsc Complete Check Test NO-TARGET -- > WARNING - --->Test Skipped: in plugin! Host Unit Test Dsc Complete Check Test NO-TARGET > PROGRESS - --Running NetworkPkg: Host Unit Test Dsc Complete Check Test NO-TARGET -- > WARNING - --->Test Skipped: in plugin! Host Unit Test Dsc Complete Check Test NO-TARGET > PROGRESS - --Running PcAtChipsetPkg: Host Unit Test Dsc Complete Check Test NO-TARGET -- > WARNING - --->Test Skipped: in plugin! Host Unit Test Dsc Complete Check Test NO-TARGET > PROGRESS - --Running SecurityPkg: Host Unit Test Dsc Complete Check Test NO-TARGET -- > WARNING - --->Test Skipped: in plugin! Host Unit Test Dsc Complete Check Test NO-TARGET > PROGRESS - --Running ShellPkg: Host Unit Test Dsc Complete Check Test NO-TARGET -- > WARNING - --->Test Skipped: in plugin! Host Unit Test Dsc Complete Check Test NO-TARGET - Still related to host-based unit tests, I believe the following warnings, for FmpDevicePkg, MdeModulePkg, MdePkg, UefiCpuPkg, and UnitTestFrameworkPkg, report that running the tests on the build host (understandably) prevents the tests from covering all edk2 architectures: > PROGRESS - --Running FmpDevicePkg: Host Unit Test Compiler Plugin NOOPT -- > WARNING - Allowing Override for key TARGET_ARCH > PROGRESS - --Running MdeModulePkg: Host Unit Test Compiler Plugin NOOPT -- > WARNING - Allowing Override for key TARGET_ARCH > PROGRESS - --Running MdePkg: Host Unit Test Compiler Plugin NOOPT -- > WARNING - Allowing Override for key TARGET_ARCH > PROGRESS - --Running UefiCpuPkg: Host Unit Test Compiler Plugin NOOPT -- > WARNING - Allowing Override for key TARGET_ARCH > PROGRESS - --Running UnitTestFrameworkPkg: Host Unit Test Compiler Plugin NOOPT -- > WARNING - Allowing Override for key TARGET_ARCH - The below seems to be related to the "spell checking in audit mode" known issues at <https://github.com/tianocore/edk2/blob/master/.pytool/Readme.md>. What does "audit mode" mean? FWIW, the packages listed below (EmulatorPkg, MdeModulePkg, MdePkg, NetworkPkg, OvmfPkg, ShellPkg, UefiCpuPkg) is a proper subset of the packges noted in ".pytool/Readme.md" -- the latter mentions CryptoPkg and SecurityPkg in addition. > PROGRESS - --Running EmulatorPkg: Spell Check Test NO-TARGET -- > WARNING - --->Test Skipped: in plugin! Spell Check Test NO-TARGET > PROGRESS - --Running MdeModulePkg: Spell Check Test NO-TARGET -- > WARNING - --->Test Skipped: in plugin! Spell Check Test NO-TARGET > PROGRESS - --Running MdePkg: Spell Check Test NO-TARGET -- > WARNING - --->Test Skipped: in plugin! Spell Check Test NO-TARGET > PROGRESS - --Running NetworkPkg: Spell Check Test NO-TARGET -- > WARNING - --->Test Skipped: in plugin! Spell Check Test NO-TARGET > PROGRESS - --Running OvmfPkg: Spell Check Test NO-TARGET -- > WARNING - --->Test Skipped: in plugin! Spell Check Test NO-TARGET > PROGRESS - --Running ShellPkg: Spell Check Test NO-TARGET -- > WARNING - --->Test Skipped: in plugin! Spell Check Test NO-TARGET > PROGRESS - --Running UefiCpuPkg: Spell Check Test NO-TARGET -- > WARNING - --->Test Skipped: in plugin! Spell Check Test NO-TARGET - Not sure about the last one: > PROGRESS - --Running FatPkg: Library Class Check Test NO-TARGET -- > WARNING - --->Test Skipped: in plugin! Library Class Check Test NO-TARGET * One suggestion (in addition to the others made in this thread): - Please add "BaseToolsBuild" to "BaseTools/.gitignore". * A question: - On github, there is some logic that restricts the full Core CI build to packages that may be affected by a patch series. Is this available locally? According to the Core CI documentation, the "stuart_ci_build" command can be limited with "-p" / "-a" / "-t" flags; given a commit range, how do we calculate the "tightest" flag values? Thank you very much! Laszlo ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [edk2-devel] running CI locally 2020-08-21 7:23 ` Laszlo Ersek @ 2020-08-21 21:36 ` Sean 2020-08-24 12:35 ` Laszlo Ersek [not found] ` <162D662A3371B4DC.7353@groups.io> 1 sibling, 1 reply; 9+ messages in thread From: Sean @ 2020-08-21 21:36 UTC (permalink / raw) To: Laszlo Ersek, devel, michael.d.kinney, Sean Brogan, Bret Barkelew Cc: Vladimir Olovyannikov, Rebecca Cran, Tom Lendacky Laszlo, Idea being if the package supports host based unit tests it will scan package to make sure all host based unit tests and libraries are listed in the DSC. As to why you see a different set of packages...that is interesting. If you can include your full build logs i can look a little closer. I know on the Azure CI servers we don't run the On 8/21/2020 12:23 AM, Laszlo Ersek wrote: > Hi Mike, Sean; > > On 08/19/20 19:59, Laszlo Ersek wrote: > >> I'll report back with more results. > > I installed a Fedora 32 Server VM. > > Installed the "mono-complete" package (+its deps). > > Installed the "nodejs" package. > > Installed "cspell" (+deps) with "npm" (with the latter coming from the > nodejs package). > > Edk2 checked out at current HEAD > (5a6d764e1d073d28e8f398289ccb5592bf9a72ba). > > > * Platform CI results: > > - <https://github.com/tianocore/edk2/tree/master/OvmfPkg/PlatformCI>: > > Built successfully for IA32, X64, and IA32X64 (toolchain: GCC5). > > - <https://github.com/tianocore/edk2/tree/master/ArmVirtPkg/PlatformCI>: > > Built successfully for ARM and AARCH64 (toolchain: GCC5). > > > * Core CI results: > > - <https://github.com/tianocore/edk2/blob/master/.pytool/Readme.md> > > Complete run successful (toolchain: GCC5); no package / arch / target > / test restrictions. > > I got the following WARNINGs: > > - ArmVirtPkg, EmulatorPkg, and OvmfPkg are covered by Platform CI, not > Core CI -- these are expected: > >> PROGRESS - --Running ArmVirtPkg: Compiler Plugin DEBUG -- >> WARNING - --->Test Skipped: in plugin! Compiler Plugin DEBUG >> PROGRESS - --Running ArmVirtPkg: Compiler Plugin RELEASE -- >> WARNING - --->Test Skipped: in plugin! Compiler Plugin RELEASE > >> PROGRESS - --Running EmulatorPkg: Compiler Plugin DEBUG -- >> WARNING - --->Test Skipped: in plugin! Compiler Plugin DEBUG >> PROGRESS - --Running EmulatorPkg: Compiler Plugin RELEASE -- >> WARNING - --->Test Skipped: in plugin! Compiler Plugin RELEASE > >> PROGRESS - --Running OvmfPkg: Compiler Plugin DEBUG -- >> WARNING - --->Test Skipped: in plugin! Compiler Plugin DEBUG >> PROGRESS - --Running OvmfPkg: Compiler Plugin RELEASE -- >> WARNING - --->Test Skipped: in plugin! Compiler Plugin RELEASE > > - ArmVirtPkg, DynamicTablesPkg, CryptoPkg, EmulatorPkg, FatPkg, > NetworkPkg, OvmfPkg, PcAtChipsetPkg, SecurityPkg and ShellPkg are > not covered by host-based unit tests -- I think also expected: > >> PROGRESS - --Running ArmVirtPkg: Host Unit Test Compiler Plugin NOOPT -- >> WARNING - --->Test Skipped: in plugin! Host Unit Test Compiler Plugin NOOPT > >> PROGRESS - --Running DynamicTablesPkg: Host Unit Test Compiler Plugin NOOPT -- >> WARNING - --->Test Skipped: in plugin! Host Unit Test Compiler Plugin NOOPT > >> PROGRESS - --Running CryptoPkg: Host Unit Test Compiler Plugin NOOPT -- >> WARNING - --->Test Skipped: in plugin! Host Unit Test Compiler Plugin NOOPT > >> PROGRESS - --Running EmulatorPkg: Host Unit Test Compiler Plugin NOOPT -- >> WARNING - --->Test Skipped: in plugin! Host Unit Test Compiler Plugin NOOPT > >> PROGRESS - --Running FatPkg: Host Unit Test Compiler Plugin NOOPT -- >> WARNING - --->Test Skipped: in plugin! Host Unit Test Compiler Plugin NOOPT > >> PROGRESS - --Running NetworkPkg: Host Unit Test Compiler Plugin NOOPT -- >> WARNING - --->Test Skipped: in plugin! Host Unit Test Compiler Plugin NOOPT > >> PROGRESS - --Running OvmfPkg: Host Unit Test Compiler Plugin NOOPT -- >> WARNING - --->Test Skipped: in plugin! Host Unit Test Compiler Plugin NOOPT > >> PROGRESS - --Running PcAtChipsetPkg: Host Unit Test Compiler Plugin NOOPT -- >> WARNING - --->Test Skipped: in plugin! Host Unit Test Compiler Plugin NOOPT > >> PROGRESS - --Running SecurityPkg: Host Unit Test Compiler Plugin NOOPT -- >> WARNING - --->Test Skipped: in plugin! Host Unit Test Compiler Plugin NOOPT > >> PROGRESS - --Running ShellPkg: Host Unit Test Compiler Plugin NOOPT -- >> WARNING - --->Test Skipped: in plugin! Host Unit Test Compiler Plugin NOOPT > > - I'm not really sure about the following warnings. They were emitted > for a subset of the above packages. I read the docs at > <https://github.com/tianocore/edk2/blob/master/.pytool/Readme.md#host-module-inclusion-test---hostunittestdsccompletecheck> > but I still don't understand :) Each plugin/test has a readme as well. Not sure if this helps explain the HostUnitTestDscCompleteCheck more. https://github.com/tianocore/edk2/tree/master/.pytool/Plugin/HostUnitTestDscCompleteCheck As to why you don't see it skipped for ArmVirtPkg or a few others: I think this is a bug. In Azure CI i see it as test passed. PROGRESS - --Running ArmVirtPkg: Host Unit Test Dsc Complete Check Test NO-TARGET -- PROGRESS - --->Test Success: Host Unit Test Dsc Complete Check Test NO-TARGET When i look at the code https://github.com/tianocore/edk2/blob/master/.pytool/Plugin/HostUnitTestDscCompleteCheck/HostUnitTestDscCompleteCheck.py#L59 I see that how those packages *.ci.yaml have it configured is different than those that show skipped below. It avoids the skip conditions but since those packages don't have any host unit test libraries or modules there is no failure. This should probably be a bugzilla on the HostUnitTestDscCompleteCheck.py as the reporting is not correct. Test should show up as skipped. You can see them here showing up as success https://dev.azure.com/tianocore/edk2-ci/_build/results?buildId=11309&view=logs&j=780eaa17-fc2d-5131-b9d2-21b19bc9bec5&t=0791c90f-e7d8-5372-c9c5-edbe30bf6aeb&l=32 > >> PROGRESS - --Running CryptoPkg: Host Unit Test Dsc Complete Check Test NO-TARGET -- >> WARNING - --->Test Skipped: in plugin! Host Unit Test Dsc Complete Check Test NO-TARGET > >> PROGRESS - --Running FatPkg: Host Unit Test Dsc Complete Check Test NO-TARGET -- >> WARNING - --->Test Skipped: in plugin! Host Unit Test Dsc Complete Check Test NO-TARGET > >> PROGRESS - --Running NetworkPkg: Host Unit Test Dsc Complete Check Test NO-TARGET -- >> WARNING - --->Test Skipped: in plugin! Host Unit Test Dsc Complete Check Test NO-TARGET > >> PROGRESS - --Running PcAtChipsetPkg: Host Unit Test Dsc Complete Check Test NO-TARGET -- >> WARNING - --->Test Skipped: in plugin! Host Unit Test Dsc Complete Check Test NO-TARGET > >> PROGRESS - --Running SecurityPkg: Host Unit Test Dsc Complete Check Test NO-TARGET -- >> WARNING - --->Test Skipped: in plugin! Host Unit Test Dsc Complete Check Test NO-TARGET > >> PROGRESS - --Running ShellPkg: Host Unit Test Dsc Complete Check Test NO-TARGET -- >> WARNING - --->Test Skipped: in plugin! Host Unit Test Dsc Complete Check Test NO-TARGET > > - Still related to host-based unit tests, I believe the following > warnings, for FmpDevicePkg, MdeModulePkg, MdePkg, UefiCpuPkg, and > UnitTestFrameworkPkg, report that running the tests on the build > host (understandably) prevents the tests from covering all edk2 > architectures: Yes this message is because we override the arch for NOOPT to match that of the build system since we are going to run these executables in the host environment. > >> PROGRESS - --Running FmpDevicePkg: Host Unit Test Compiler Plugin NOOPT -- >> WARNING - Allowing Override for key TARGET_ARCH > >> PROGRESS - --Running MdeModulePkg: Host Unit Test Compiler Plugin NOOPT -- >> WARNING - Allowing Override for key TARGET_ARCH > >> PROGRESS - --Running MdePkg: Host Unit Test Compiler Plugin NOOPT -- >> WARNING - Allowing Override for key TARGET_ARCH > >> PROGRESS - --Running UefiCpuPkg: Host Unit Test Compiler Plugin NOOPT -- >> WARNING - Allowing Override for key TARGET_ARCH > >> PROGRESS - --Running UnitTestFrameworkPkg: Host Unit Test Compiler Plugin NOOPT -- >> WARNING - Allowing Override for key TARGET_ARCH > > - The below seems to be related to the "spell checking in audit > mode" known issues at > <https://github.com/tianocore/edk2/blob/master/.pytool/Readme.md>. > > What does "audit mode" mean? see readme here. https://github.com/tianocore/edk2/tree/master/.pytool/Plugin/SpellCheck#configuration Basically runs the tests and reports all the errors as warnings and then reports the test as skipped. It was hopefully to encourage package maintainers to fix their spelling issues but not breaking the build on day 1. > > FWIW, the packages listed below (EmulatorPkg, MdeModulePkg, > MdePkg, NetworkPkg, OvmfPkg, ShellPkg, UefiCpuPkg) is a proper > subset of the packges noted in ".pytool/Readme.md" -- the latter > mentions CryptoPkg and SecurityPkg in addition. > >> PROGRESS - --Running EmulatorPkg: Spell Check Test NO-TARGET -- >> WARNING - --->Test Skipped: in plugin! Spell Check Test NO-TARGET > >> PROGRESS - --Running MdeModulePkg: Spell Check Test NO-TARGET -- >> WARNING - --->Test Skipped: in plugin! Spell Check Test NO-TARGET > >> PROGRESS - --Running MdePkg: Spell Check Test NO-TARGET -- >> WARNING - --->Test Skipped: in plugin! Spell Check Test NO-TARGET > >> PROGRESS - --Running NetworkPkg: Spell Check Test NO-TARGET -- >> WARNING - --->Test Skipped: in plugin! Spell Check Test NO-TARGET > >> PROGRESS - --Running OvmfPkg: Spell Check Test NO-TARGET -- >> WARNING - --->Test Skipped: in plugin! Spell Check Test NO-TARGET > >> PROGRESS - --Running ShellPkg: Spell Check Test NO-TARGET -- >> WARNING - --->Test Skipped: in plugin! Spell Check Test NO-TARGET > >> PROGRESS - --Running UefiCpuPkg: Spell Check Test NO-TARGET -- >> WARNING - --->Test Skipped: in plugin! Spell Check Test NO-TARGET > > - Not sure about the last one: > >> PROGRESS - --Running FatPkg: Library Class Check Test NO-TARGET -- >> WARNING - --->Test Skipped: in plugin! Library Class Check Test NO-TARGET > > > * One suggestion (in addition to the others made in this thread): > > - Please add "BaseToolsBuild" to "BaseTools/.gitignore". There is a bugzilla for this already. https://bugzilla.tianocore.org/show_bug.cgi?id=2581 > > > * A question: > > - On github, there is some logic that restricts the full Core CI build > to packages that may be affected by a patch series. Is this available > locally? > > According to the Core CI documentation, the "stuart_ci_build" command > can be limited with "-p" / "-a" / "-t" flags; given a commit range, > how do we calculate the "tightest" flag values? A couple things here. You can always look at the azurepipelines files to get a better understanding of the CI process. Hopefully there is no magic (besides being able to read yaml). You can see here: https://github.com/tianocore/edk2/blob/master/.azurepipelines/templates/pr-gate-steps.yml#L35 It is actually another stuart tool. stuart_pr_eval. We have bug to document it. https://github.com/tianocore/edk2-pytool-extensions/issues/89 But the python file is pretty well documented. There are 4 policies that can trigger the need for a change to build a package. See here https://github.com/tianocore/edk2-pytool-extensions/blob/master/edk2toolext/invocables/edk2_pr_eval.py#L117 > > Thank you very much! > Laszlo > ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [edk2-devel] running CI locally 2020-08-21 21:36 ` Sean @ 2020-08-24 12:35 ` Laszlo Ersek 0 siblings, 0 replies; 9+ messages in thread From: Laszlo Ersek @ 2020-08-24 12:35 UTC (permalink / raw) To: Sean Brogan, devel, michael.d.kinney, Sean Brogan, Bret Barkelew Cc: Vladimir Olovyannikov, Rebecca Cran, Tom Lendacky On 08/21/20 23:36, Sean Brogan wrote: > On 8/21/2020 12:23 AM, Laszlo Ersek wrote: >> - ArmVirtPkg, DynamicTablesPkg, CryptoPkg, EmulatorPkg, FatPkg, >> NetworkPkg, OvmfPkg, PcAtChipsetPkg, SecurityPkg and ShellPkg >> are not covered by host-based unit tests -- I think also >> expected: >> >>> [...] >> >> - I'm not really sure about the following warnings. They were >> emitted for a subset of the above packages. I read the docs at >> >> <https://github.com/tianocore/edk2/blob/master/.pytool/Readme.md#host-module-inclusion-test---hostunittestdsccompletecheck> >> >> but I still don't understand :) > > Each plugin/test has a readme as well. Not sure if this helps explain > the HostUnitTestDscCompleteCheck more. > > https://github.com/tianocore/edk2/tree/master/.pytool/Plugin/HostUnitTestDscCompleteCheck I've carefully re-read the explanation now, and (apparently) I do understand it now :/ Before, I didn't realize that the expression "those related to host based unit tests" stood for the set of INF files where each INF was MODULE_TYPE=HOST_APPLICATION or LIBRARY_CLASS=FoobarLib|HOST_APPLICATION. > As to why you don't see it skipped for ArmVirtPkg or a few others: I > think this is a bug. > > In Azure CI i see it as test passed. > > PROGRESS - --Running ArmVirtPkg: Host Unit Test Dsc Complete Check > Test NO-TARGET -- > PROGRESS - --->Test Success: Host Unit Test Dsc Complete Check Test > NO-TARGET > > When i look at the code > https://github.com/tianocore/edk2/blob/master/.pytool/Plugin/HostUnitTestDscCompleteCheck/HostUnitTestDscCompleteCheck.py#L59 > > > I see that how those packages *.ci.yaml have it configured is > different than those that show skipped below. It avoids the skip > conditions but since those packages don't have any host unit test > libraries or modules there is no failure. "OvmfPkg/OvmfPkg.ci.yaml" (as an example) sets both "HostUnitTestCompilerPlugin.DscPath" and "HostUnitTestDscCompleteCheck.DscPath" to the empty string. "ShellPkg/ShellPkg.ci.yaml" (an an example) sets neither attribute at all. Both HostUnitTestCompilerPlugin and HostUnitTestDscCompleteCheck check for the existence of the DscPath attribute, as first step, so both plugins warn about, and skip, ShellPkg. OvmfPkg passes the same, first -- i.e., DscPath attribute existence -- check in both plugins. This suggests OvmfPkg is ruled out by *further* checks in one plugin, and not ruled out by any further checks in the other plugin. The next (2nd) check in both plugins seems to be, whether we can actually find the DSC file pointed-to by the DscPath attribute. Furthermore, in HostUnitTestDscCompleteCheck, this check is the last one (no other checks). I think that this 2nd check *too* should fail for OvmfPkg in *both* plugins. Because the DscPath attribute, while it exists, is empty. I don't see why the "DSC path lookup" would fail in HostUnitTestCompilerPlugin (and so we'd get a warning about OvmfPkg), but not in HostUnitTestDscCompleteCheck (and so we'd get no warning about OvmfPkg). ... Is it possible that the 2nd check actually fires in HostUnitTestDscCompleteCheck too, but we suppress the warning by returning 0, and not -1? Compare the return values after the tc.SetSkipped() calls: [https://github.com/tianocore/edk2/blob/master/.pytool/Plugin/HostUnitTestCompilerPlugin/HostUnitTestCompilerPlugin.py#L103] 100 if AP is None or AP_Path is None or not os.path.isfile(APDSC): 101 tc.SetSkipped() 102 tc.LogStdError("Package HostBasedUnitTest Dsc not found.") 103 return -1 ^^ versus [https://github.com/tianocore/edk2/blob/master/.pytool/Plugin/HostUnitTestDscCompleteCheck/HostUnitTestDscCompleteCheck.py#L73] 70 if abs_dsc_path is None or wsr_dsc_path == "" or not os.path.isfile(abs_dsc_path): 71 tc.SetSkipped() 72 tc.LogStdError("Package Host Unit Test Dsc not found") 73 return 0 ^ I guess I could verify if the branch was taken at all, from the standard error... Ah, "Build/TestSuites.xml" seems relevant. ... Yes, I'm fairly sure the zero value after the "return" statement, on line 73, is a typo. Because this is what I have in "Build/TestSuites.xml" (excerpt re-wrapped here for readability): > <testsuites> > <testsuite > id="14" > name="OvmfPkg" > package="Edk2CiBuild.Edk2.OvmfPkg" > errors="0" > tests="12" > failures="0" > skipped="6"> > > <testcase > classname="Edk2CiBuild.Edk2.OvmfPkg.HostUnitTestCompiler.X64" > name="Compile and Run Host-Based UnitTests for Edk2CiBuild.Edk2.OvmfPkg on arch X64" > time="0.0002715587615966797"> > <skipped type="skipped"/> > <system-out/> > <system-err>Package HostBasedUnitTest Dsc not found.</system-err> > </testcase> > > <testcase > classname="Edk2CiBuild.Edk2.OvmfPkg.HostUnitTestDscCompleteCheck" > name="Check the Edk2CiBuild.Edk2.OvmfPkg Host Unit Test DSC for a being complete" > time="0.00020742416381835938"> > <skipped type="skipped"/> > <system-out/> > <system-err>Package Host Unit Test Dsc not found</system-err> > </testcase> > > </testsuite> > </testsuites> Note that both quoted test cases (both plugins) report having been skipped. And from the <system-err> element, we can confirm that both plugins catch OvmfPkg's DscPath attribute being the empty string. It's just that "HostUnitTestDscCompleteCheck" returns zero on that branch. > This should probably be a bugzilla on the > HostUnitTestDscCompleteCheck.py as the reporting is not correct. Test > should show up as skipped. I've now filed: https://bugzilla.tianocore.org/show_bug.cgi?id=2924 For now I've assigned it to you, and set BaseTools as the Package needing the fix. Please update the metadata as necessary :) >> - The below seems to be related to the "spell checking in audit >> mode" known issues at >> <https://github.com/tianocore/edk2/blob/master/.pytool/Readme.md>. >> >> What does "audit mode" mean? > > see readme here. > https://github.com/tianocore/edk2/tree/master/.pytool/Plugin/SpellCheck#configuration > > > Basically runs the tests and reports all the errors as warnings and > then reports the test as skipped. It was hopefully to encourage > package maintainers to fix their spelling issues but not breaking the > build on day 1. Oh. In OvmfPkg, the CI YAML file says "Fails right now with over 270 errors" :/ >> * One suggestion (in addition to the others made in this thread): >> >> - Please add "BaseToolsBuild" to "BaseTools/.gitignore". > > There is a bugzilla for this already. > https://bugzilla.tianocore.org/show_bug.cgi?id=2581 Thanks! >> * A question: >> >> - On github, there is some logic that restricts the full Core CI >> build to packages that may be affected by a patch series. Is this >> available locally? >> >> According to the Core CI documentation, the "stuart_ci_build" >> command can be limited with "-p" / "-a" / "-t" flags; given a >> commit range, how do we calculate the "tightest" flag values? > > A couple things here. > You can always look at the azurepipelines files to get a better > understanding of the CI process. Hopefully there is no magic (besides > being able to read yaml). You can see here: > https://github.com/tianocore/edk2/blob/master/.azurepipelines/templates/pr-gate-steps.yml#L35 > > > It is actually another stuart tool. > stuart_pr_eval. > We have bug to document it. > https://github.com/tianocore/edk2-pytool-extensions/issues/89 > But the python file is pretty well documented. There are 4 policies > that can trigger the need for a change to build a package. > See here > https://github.com/tianocore/edk2-pytool-extensions/blob/master/edk2toolext/invocables/edk2_pr_eval.py#L117 Cool, so I can run something like stuart_pr_eval -c .pytool/CISettings.py -p CANDIDATE-PACKAGE-LIST --pr-target TOPIC-BRANCH \ [output formatting options] I'll try that next time! Thank you! Laszlo ^ permalink raw reply [flat|nested] 9+ messages in thread
[parent not found: <162D662A3371B4DC.7353@groups.io>]
* Re: [edk2-devel] running CI locally [not found] ` <162D662A3371B4DC.7353@groups.io> @ 2020-08-21 21:50 ` Sean 0 siblings, 0 replies; 9+ messages in thread From: Sean @ 2020-08-21 21:50 UTC (permalink / raw) To: Laszlo Ersek, devel, michael.d.kinney, Sean Brogan, Bret Barkelew Cc: Vladimir Olovyannikov, Rebecca Cran, Tom Lendacky sorry. In my last email I started typing at top and then moved inline and didn't delete my initial comments. Please ignore the comments on top and look inline for details. :) Thanks Sean On 8/21/2020 2:36 PM, Sean wrote: > Laszlo, > > > > Idea being if the package supports host based unit tests it will scan > package to make sure all host based unit tests and libraries are listed > in the DSC. > > As to why you see a different set of packages...that is interesting. If > you can include your full build logs i can look a little closer. I know > on the Azure CI servers we don't run the > > > On 8/21/2020 12:23 AM, Laszlo Ersek wrote: >> Hi Mike, Sean; >> >> On 08/19/20 19:59, Laszlo Ersek wrote: >> >>> I'll report back with more results. >> >> I installed a Fedora 32 Server VM. >> >> Installed the "mono-complete" package (+its deps). >> >> Installed the "nodejs" package. >> >> Installed "cspell" (+deps) with "npm" (with the latter coming from the >> nodejs package). >> >> Edk2 checked out at current HEAD >> (5a6d764e1d073d28e8f398289ccb5592bf9a72ba). >> >> >> * Platform CI results: >> >> - <https://github.com/tianocore/edk2/tree/master/OvmfPkg/PlatformCI>: >> >> Built successfully for IA32, X64, and IA32X64 (toolchain: GCC5). >> >> - <https://github.com/tianocore/edk2/tree/master/ArmVirtPkg/PlatformCI>: >> >> Built successfully for ARM and AARCH64 (toolchain: GCC5). >> >> >> * Core CI results: >> >> - <https://github.com/tianocore/edk2/blob/master/.pytool/Readme.md> >> >> Complete run successful (toolchain: GCC5); no package / arch / target >> / test restrictions. >> >> I got the following WARNINGs: >> >> - ArmVirtPkg, EmulatorPkg, and OvmfPkg are covered by Platform CI, not >> Core CI -- these are expected: >> >>> PROGRESS - --Running ArmVirtPkg: Compiler Plugin DEBUG -- >>> WARNING - --->Test Skipped: in plugin! Compiler Plugin DEBUG >>> PROGRESS - --Running ArmVirtPkg: Compiler Plugin RELEASE -- >>> WARNING - --->Test Skipped: in plugin! Compiler Plugin RELEASE >> >>> PROGRESS - --Running EmulatorPkg: Compiler Plugin DEBUG -- >>> WARNING - --->Test Skipped: in plugin! Compiler Plugin DEBUG >>> PROGRESS - --Running EmulatorPkg: Compiler Plugin RELEASE -- >>> WARNING - --->Test Skipped: in plugin! Compiler Plugin RELEASE >> >>> PROGRESS - --Running OvmfPkg: Compiler Plugin DEBUG -- >>> WARNING - --->Test Skipped: in plugin! Compiler Plugin DEBUG >>> PROGRESS - --Running OvmfPkg: Compiler Plugin RELEASE -- >>> WARNING - --->Test Skipped: in plugin! Compiler Plugin RELEASE >> >> - ArmVirtPkg, DynamicTablesPkg, CryptoPkg, EmulatorPkg, FatPkg, >> NetworkPkg, OvmfPkg, PcAtChipsetPkg, SecurityPkg and ShellPkg are >> not covered by host-based unit tests -- I think also expected: >> >>> PROGRESS - --Running ArmVirtPkg: Host Unit Test Compiler Plugin NOOPT -- >>> WARNING - --->Test Skipped: in plugin! Host Unit Test Compiler Plugin >>> NOOPT >> >>> PROGRESS - --Running DynamicTablesPkg: Host Unit Test Compiler Plugin >>> NOOPT -- >>> WARNING - --->Test Skipped: in plugin! Host Unit Test Compiler Plugin >>> NOOPT >> >>> PROGRESS - --Running CryptoPkg: Host Unit Test Compiler Plugin NOOPT -- >>> WARNING - --->Test Skipped: in plugin! Host Unit Test Compiler Plugin >>> NOOPT >> >>> PROGRESS - --Running EmulatorPkg: Host Unit Test Compiler Plugin >>> NOOPT -- >>> WARNING - --->Test Skipped: in plugin! Host Unit Test Compiler Plugin >>> NOOPT >> >>> PROGRESS - --Running FatPkg: Host Unit Test Compiler Plugin NOOPT -- >>> WARNING - --->Test Skipped: in plugin! Host Unit Test Compiler Plugin >>> NOOPT >> >>> PROGRESS - --Running NetworkPkg: Host Unit Test Compiler Plugin NOOPT -- >>> WARNING - --->Test Skipped: in plugin! Host Unit Test Compiler Plugin >>> NOOPT >> >>> PROGRESS - --Running OvmfPkg: Host Unit Test Compiler Plugin NOOPT -- >>> WARNING - --->Test Skipped: in plugin! Host Unit Test Compiler Plugin >>> NOOPT >> >>> PROGRESS - --Running PcAtChipsetPkg: Host Unit Test Compiler Plugin >>> NOOPT -- >>> WARNING - --->Test Skipped: in plugin! Host Unit Test Compiler Plugin >>> NOOPT >> >>> PROGRESS - --Running SecurityPkg: Host Unit Test Compiler Plugin >>> NOOPT -- >>> WARNING - --->Test Skipped: in plugin! Host Unit Test Compiler Plugin >>> NOOPT >> >>> PROGRESS - --Running ShellPkg: Host Unit Test Compiler Plugin NOOPT -- >>> WARNING - --->Test Skipped: in plugin! Host Unit Test Compiler Plugin >>> NOOPT >> >> - I'm not really sure about the following warnings. They were emitted >> for a subset of the above packages. I read the docs at >> >> <https://github.com/tianocore/edk2/blob/master/.pytool/Readme.md#host-module-inclusion-test---hostunittestdsccompletecheck> >> >> but I still don't understand :) > > Each plugin/test has a readme as well. Not sure if this helps explain > the HostUnitTestDscCompleteCheck more. > > https://github.com/tianocore/edk2/tree/master/.pytool/Plugin/HostUnitTestDscCompleteCheck > > > As to why you don't see it skipped for ArmVirtPkg or a few others: I > think this is a bug. > > In Azure CI i see it as test passed. > > PROGRESS - --Running ArmVirtPkg: Host Unit Test Dsc Complete Check Test > NO-TARGET -- > PROGRESS - --->Test Success: Host Unit Test Dsc Complete Check Test > NO-TARGET > > When i look at the code > https://github.com/tianocore/edk2/blob/master/.pytool/Plugin/HostUnitTestDscCompleteCheck/HostUnitTestDscCompleteCheck.py#L59 > > > I see that how those packages *.ci.yaml have it configured is different > than those that show skipped below. It avoids the skip conditions but > since those packages don't have any host unit test libraries or modules > there is no failure. > > This should probably be a bugzilla on the > HostUnitTestDscCompleteCheck.py as the reporting is not correct. Test > should show up as skipped. > > > You can see them here showing up as success > https://dev.azure.com/tianocore/edk2-ci/_build/results?buildId=11309&view=logs&j=780eaa17-fc2d-5131-b9d2-21b19bc9bec5&t=0791c90f-e7d8-5372-c9c5-edbe30bf6aeb&l=32 > > >> >>> PROGRESS - --Running CryptoPkg: Host Unit Test Dsc Complete Check >>> Test NO-TARGET -- >>> WARNING - --->Test Skipped: in plugin! Host Unit Test Dsc Complete >>> Check Test NO-TARGET >> >>> PROGRESS - --Running FatPkg: Host Unit Test Dsc Complete Check Test >>> NO-TARGET -- >>> WARNING - --->Test Skipped: in plugin! Host Unit Test Dsc Complete >>> Check Test NO-TARGET >> >>> PROGRESS - --Running NetworkPkg: Host Unit Test Dsc Complete Check >>> Test NO-TARGET -- >>> WARNING - --->Test Skipped: in plugin! Host Unit Test Dsc Complete >>> Check Test NO-TARGET >> >>> PROGRESS - --Running PcAtChipsetPkg: Host Unit Test Dsc Complete >>> Check Test NO-TARGET -- >>> WARNING - --->Test Skipped: in plugin! Host Unit Test Dsc Complete >>> Check Test NO-TARGET >> >>> PROGRESS - --Running SecurityPkg: Host Unit Test Dsc Complete Check >>> Test NO-TARGET -- >>> WARNING - --->Test Skipped: in plugin! Host Unit Test Dsc Complete >>> Check Test NO-TARGET >> >>> PROGRESS - --Running ShellPkg: Host Unit Test Dsc Complete Check Test >>> NO-TARGET -- >>> WARNING - --->Test Skipped: in plugin! Host Unit Test Dsc Complete >>> Check Test NO-TARGET >> >> - Still related to host-based unit tests, I believe the following >> warnings, for FmpDevicePkg, MdeModulePkg, MdePkg, UefiCpuPkg, and >> UnitTestFrameworkPkg, report that running the tests on the build >> host (understandably) prevents the tests from covering all edk2 >> architectures: > > Yes this message is because we override the arch for NOOPT to match that > of the build system since we are going to run these executables in the > host environment. > >> >>> PROGRESS - --Running FmpDevicePkg: Host Unit Test Compiler Plugin >>> NOOPT -- >>> WARNING - Allowing Override for key TARGET_ARCH >> >>> PROGRESS - --Running MdeModulePkg: Host Unit Test Compiler Plugin >>> NOOPT -- >>> WARNING - Allowing Override for key TARGET_ARCH >> >>> PROGRESS - --Running MdePkg: Host Unit Test Compiler Plugin NOOPT -- >>> WARNING - Allowing Override for key TARGET_ARCH >> >>> PROGRESS - --Running UefiCpuPkg: Host Unit Test Compiler Plugin NOOPT -- >>> WARNING - Allowing Override for key TARGET_ARCH >> >>> PROGRESS - --Running UnitTestFrameworkPkg: Host Unit Test Compiler >>> Plugin NOOPT -- >>> WARNING - Allowing Override for key TARGET_ARCH >> >> - The below seems to be related to the "spell checking in audit >> mode" known issues at >> <https://github.com/tianocore/edk2/blob/master/.pytool/Readme.md>. >> >> What does "audit mode" mean? > > see readme here. > https://github.com/tianocore/edk2/tree/master/.pytool/Plugin/SpellCheck#configuration > > > Basically runs the tests and reports all the errors as warnings and then > reports the test as skipped. > It was hopefully to encourage package maintainers to fix their spelling > issues but not breaking the build on day 1. > > >> >> FWIW, the packages listed below (EmulatorPkg, MdeModulePkg, >> MdePkg, NetworkPkg, OvmfPkg, ShellPkg, UefiCpuPkg) is a proper >> subset of the packges noted in ".pytool/Readme.md" -- the latter >> mentions CryptoPkg and SecurityPkg in addition. >> >>> PROGRESS - --Running EmulatorPkg: Spell Check Test NO-TARGET -- >>> WARNING - --->Test Skipped: in plugin! Spell Check Test NO-TARGET >> >>> PROGRESS - --Running MdeModulePkg: Spell Check Test NO-TARGET -- >>> WARNING - --->Test Skipped: in plugin! Spell Check Test NO-TARGET >> >>> PROGRESS - --Running MdePkg: Spell Check Test NO-TARGET -- >>> WARNING - --->Test Skipped: in plugin! Spell Check Test NO-TARGET >> >>> PROGRESS - --Running NetworkPkg: Spell Check Test NO-TARGET -- >>> WARNING - --->Test Skipped: in plugin! Spell Check Test NO-TARGET >> >>> PROGRESS - --Running OvmfPkg: Spell Check Test NO-TARGET -- >>> WARNING - --->Test Skipped: in plugin! Spell Check Test NO-TARGET >> >>> PROGRESS - --Running ShellPkg: Spell Check Test NO-TARGET -- >>> WARNING - --->Test Skipped: in plugin! Spell Check Test NO-TARGET >> >>> PROGRESS - --Running UefiCpuPkg: Spell Check Test NO-TARGET -- >>> WARNING - --->Test Skipped: in plugin! Spell Check Test NO-TARGET >> >> - Not sure about the last one: >> >>> PROGRESS - --Running FatPkg: Library Class Check Test NO-TARGET -- >>> WARNING - --->Test Skipped: in plugin! Library Class Check Test >>> NO-TARGET >> >> >> * One suggestion (in addition to the others made in this thread): >> >> - Please add "BaseToolsBuild" to "BaseTools/.gitignore". > > There is a bugzilla for this already. > https://bugzilla.tianocore.org/show_bug.cgi?id=2581 > > >> >> >> * A question: >> >> - On github, there is some logic that restricts the full Core CI build >> to packages that may be affected by a patch series. Is this available >> locally? >> >> According to the Core CI documentation, the "stuart_ci_build" command >> can be limited with "-p" / "-a" / "-t" flags; given a commit range, >> how do we calculate the "tightest" flag values? > > A couple things here. > You can always look at the azurepipelines files to get a better > understanding of the CI process. Hopefully there is no magic (besides > being able to read yaml). You can see here: > https://github.com/tianocore/edk2/blob/master/.azurepipelines/templates/pr-gate-steps.yml#L35 > > > It is actually another stuart tool. > stuart_pr_eval. > We have bug to document it. > https://github.com/tianocore/edk2-pytool-extensions/issues/89 > But the python file is pretty well documented. There are 4 policies that > can trigger the need for a change to build a package. > See here > https://github.com/tianocore/edk2-pytool-extensions/blob/master/edk2toolext/invocables/edk2_pr_eval.py#L117 > > > > >> >> Thank you very much! >> Laszlo >> > > > ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [edk2-devel] running CI locally 2020-08-19 15:29 ` [edk2-devel] " Michael D Kinney 2020-08-19 16:22 ` Sean @ 2020-08-19 17:56 ` Laszlo Ersek 1 sibling, 0 replies; 9+ messages in thread From: Laszlo Ersek @ 2020-08-19 17:56 UTC (permalink / raw) To: Kinney, Michael D, devel@edk2.groups.io, Sean Brogan, Bret Barkelew Cc: Vladimir Olovyannikov, Rebecca Cran, Tom Lendacky Hi Mike, On 08/19/20 17:29, Kinney, Michael D wrote: > Hi Laszlo, > > Thank you for the feedback. I agree there are some documentation updates required. > > The spell check requirements are documented here: > > https://github.com/tianocore/edk2/blob/master/.pytool/Readme.md#spell-checking---cspell > > You do need to install nodejs and cspell. > > * Install nodejs from https://nodejs.org/en/ > * Install cspell > 1. Open cmd prompt with access to node and npm > 2. Run `npm install -g cspell` npm is a security hole one can drive a truck through :) Additionally, I just realized that "nuget" downloads native iasl and nasm binaries ("BaseTools/Bin/iasl_extdep/Linux-x86/iasl", "BaseTools/Bin/mu_nasm_extdep/Linux-x86-64/nasm"). This basically requires me to trust two package repositories (npmjs.org, nuget.org) that, well, I don't trust. This is *not* criticism of the CI system; it only means that running it locally on my laptop needs more work from me. In particular, it requires spinning up a VM that I don't use for anything else. Given the typing / clicking / scripting necessary for this, and the network-originated updates for the "in VM" CI system, I wonder if running CI locally actually saves me any time. One thing it would certainly give me is "confidentiality", as I wouldn't have to push my branch to github (for a personal CI build PR) before I were ready to submit the set for real. I think I'll set up a new libvirt/QEMU/KVM domain for this, and report back with more results once I have them. > I agree that the verbosity of the output is very brief. There is a more complete > log that is always produced in the build output directory in Build/CI_BUILDLOG.txt. > Can you look at that file and see if it provide a more detailed reason for the > failure? Yes, I'll check that. Thanks! Laszlo ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2020-08-24 12:35 UTC | newest] Thread overview: 9+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2020-08-19 12:27 running CI locally Laszlo Ersek 2020-08-19 15:29 ` [edk2-devel] " Michael D Kinney 2020-08-19 16:22 ` Sean 2020-08-19 17:59 ` Laszlo Ersek 2020-08-21 7:23 ` Laszlo Ersek 2020-08-21 21:36 ` Sean 2020-08-24 12:35 ` Laszlo Ersek [not found] ` <162D662A3371B4DC.7353@groups.io> 2020-08-21 21:50 ` Sean 2020-08-19 17:56 ` Laszlo Ersek
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox