* [Patch v4 00/22] Enable Phase 1 of EDK II CI
@ 2019-11-07 1:13 Michael D Kinney
2019-11-07 1:13 ` [Patch v4 01/22] Maintainers.txt: Add continuous integration(CI) directories Michael D Kinney
` (22 more replies)
0 siblings, 23 replies; 51+ messages in thread
From: Michael D Kinney @ 2019-11-07 1:13 UTC (permalink / raw)
To: devel
Cc: Sean Brogan, Bret Barkelew, Liming Gao, Bob Feng, Andrew Fish,
Laszlo Ersek, Leif Lindholm, Jian J Wang, Xiaoyu Lu, Ray Ni,
Hao A Wu, Jiaxin Wu, Siyuan Fu, Jiewen Yao, Chao Zhang,
Zhichao Gao, Eric Dong
https://bugzilla.tianocore.org/show_bug.cgi?id=2315
RFC Proposals:
* https://edk2.groups.io/g/rfc/message/93
* https://edk2.groups.io/g/devel/message/46607
Wiki Page:
* https://github.com/tianocore/tianocore.github.io/wiki/EDK-II-Continuous-Integration
Branch for review:
* https://github.com/tianocore/edk2-staging/tree/edk2-ci_V4
Changes in V4
* Only copy BaseTools build logs if BaseTools are built
* Add FINISHED and FAILED jobs to prevent multiple email notifications from
Mergify for PR close and PR comment actions
* Reformat .pytools/Readme.md to 80 columns
* Set WINSDK_PATH_FOR_RC_EXE in BaseTools/set_vsprefix_env.bat for VS2017/VS2019
* Update .gitignore to alphabetic ordering
* Update Maintainers.txt to alphabetic ordering
* Add file header comment block to requirements.txt
* Remove license badge and formatting changes from Readme.md
Previous versions of branches for reference:
* https://github.com/tianocore/edk2-staging/tree/edk2-ci_V3
* https://github.com/tianocore/edk2-staging/tree/edk2-ci_V2
* https://github.com/tianocore/edk2-staging/tree/edk2-ci_V1
Active branch for testing/evaluation:
* https://github.com/tianocore/edk2-staging/tree/edk2-ci
* To test, fork edk2-staging repo, create a branch with a change, and submit
a pull request targeting edk2-staging/edk2-ci. NOTE: the default branch for
the edk2-staging is 'about'. You must select the 'edk2-ci' branch when
a pull request is opened. Set the 'push' label to require commit if all
checks pass.
Pull request history on active branch for testing/evaluation:
* https://github.com/tianocore/edk2-staging/pulls?q=is%3Apr+is%3Aclosed
This patch series enables pre-commit and post-commit checks on edk2/master The
goal is to improve quality of code submissions by requiring all pre-commit
checks to pass before any change is committed to edk2/master. In order to
minimize the impact to the developer process, when an EDK II Maintainer is ready
to commit changes to edk2/master, the EDK II Maintainer pushes a branch to their
personal fork of the edk2 repository and opens a pull request targeting
edk2/master and sets the 'push' label. If all checks pass, then the pull
request is automatically committed.
A developer that wants to know if a patch series passes all checks can either
run the checks locally or open a pull request without setting the 'push' label.
The post-commit checks build all packages and provide status badges and links to
reports in the Readme.md file at the root of edk2/master.
A combination of GitHub, Azure Pipelines, Mergify, and edk2-pytool features are
used to implement this feature. GitHub and Azure Pipelines configuration steps
are required to activate this feature on edk2/master.
Once this feature is active and shown to be stable for 1-2 weeks, write access
to the edk2 repository will be removed for all EDK II Maintainers. Only admins
will retain write access, and admins should avoid use of direct push to resolve
an issue unless there is no method to resolve the issue using the pull request
method.
The following checks are made available by this patch series. Not all checks
are enabled for all packages. Enabling all checks on all packages can be done
incrementally as well as adding more checks over time.
* PatchCheck
* CharEncodingCheck
* CompilerPlugin
* DependencyCheck
* DscCompleteCheck
* GuidCheck
* LibraryClassCheck
* SpellCheck
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Andrew Fish <afish@apple.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Leif Lindholm <leif.lindholm@linaro.org>
Cc: Jian J Wang <jian.j.wang@intel.com>
Cc: Xiaoyu Lu <xiaoyux.lu@intel.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Hao A Wu <hao.a.wu@intel.com>
Cc: Jiaxin Wu <jiaxin.wu@intel.com>
Cc: Siyuan Fu <siyuan.fu@intel.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Chao Zhang <chao.b.zhang@intel.com>
Cc: Zhichao Gao <zhichao.gao@intel.com>
Cc: Eric Dong <eric.dong@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Michael D Kinney (14):
Maintainers.txt: Add continuous integration(CI) directories
CryptoPkg: Add YAML file for CI builds
FatPkg: Add YAML file for CI builds
FmpDevicePkg: Add YAML file for CI builds
MdeModulePkg: Add YAML file for CI builds
MdePkg: Add YAML file for CI builds
NetworkPkg: Add YAML file for CI builds
PcAtChipsetPkg: Add YAML files for CI builds
SecurityPkg: Add YAML files for CI builds
ShellPkg: Add YAML file for CI builds
UefiCpuPkg: Add YAML file for CI builds
SignedCapsulePkg: Use BaseCryptLibNull to reduce package CI time
.mergify: Add Mergify YML pull request rules configuration file
Readme.md: Add CI build status badges
Sean Brogan (8):
.gitignore: Ignore python compiled files, extdeps, and vscode
requirements.txt: Add python pip requirements file
BaseTools: Add RC_PATH define for VS2017/2019
BaseTools: Add YAML files with path env and tool extdeps
BaseTools: Add BaseTools plugins to support CI
.pytool/Plugin: Add CI plugins
.pytool: Add CISettings.py and Readme.md
.azurepipelines: Add Azure Pipelines YML configuration files
.azurepipelines/Ubuntu-GCC5.yml | 18 ++
.azurepipelines/Ubuntu-PatchCheck.yml | 32 +++
.azurepipelines/Windows-VS2019.yml | 18 ++
.../templates/basetools-build-steps.yml | 36 +++
.../templates/pr-gate-build-job.yml | 79 ++++++
.azurepipelines/templates/pr-gate-steps.yml | 129 +++++++++
.../templates/spell-check-prereq-steps.yml | 21 ++
.gitignore | 6 +-
.mergify/config.yml | 97 +++++++
.pytool/CISettings.py | 173 ++++++++++++
.../CharEncodingCheck/CharEncodingCheck.py | 118 ++++++++
.../CharEncodingCheck_plug_in.yaml | 11 +
.pytool/Plugin/CharEncodingCheck/Readme.md | 13 +
.../Plugin/CompilerPlugin/CompilerPlugin.py | 102 +++++++
.../CompilerPlugin/Compiler_plug_in.yaml | 11 +
.../Plugin/DependencyCheck/DependencyCheck.py | 120 +++++++++
.../DependencyCheck_plug_in.yaml | 13 +
.../DscCompleteCheck/DscCompleteCheck.py | 118 ++++++++
.../DscCompleteCheck_plug_in.yaml | 12 +
.pytool/Plugin/DscCompleteCheck/readme.md | 22 ++
.pytool/Plugin/GuidCheck/GuidCheck.py | 251 ++++++++++++++++++
.../Plugin/GuidCheck/GuidCheck_plug_in.yaml | 11 +
.pytool/Plugin/GuidCheck/Readme.md | 60 +++++
.../LibraryClassCheck/LibraryClassCheck.py | 153 +++++++++++
.../LibraryClassCheck_plug_in.yaml | 11 +
.pytool/Plugin/LibraryClassCheck/readme.md | 22 ++
.pytool/Plugin/SpellCheck/Readme.md | 100 +++++++
.pytool/Plugin/SpellCheck/SpellCheck.py | 216 +++++++++++++++
.../Plugin/SpellCheck/SpellCheck_plug_in.yaml | 11 +
.pytool/Plugin/SpellCheck/cspell.base.yaml | 165 ++++++++++++
.pytool/Readme.md | 191 +++++++++++++
BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml | 21 ++
BaseTools/Bin/gcc_arm_linux_ext_dep.yaml | 21 ++
BaseTools/Bin/iasl_ext_dep.yaml | 21 ++
BaseTools/Bin/nasm_ext_dep.yaml | 18 ++
.../BinWrappers/PosixLike/posix_path_env.yaml | 10 +
.../WindowsLike/win_build_tools_path_env.yaml | 10 +
BaseTools/Conf/tools_def.template | 18 +-
BaseTools/Edk2ToolsBuild.py | 163 ++++++++++++
.../BuildToolsReportGenerator.py | 69 +++++
.../BuildToolsReportGenerator_plug_in.yaml | 12 +
.../BuildToolsReport_Template.html | 126 +++++++++
.../LinuxGcc5ToolChain/LinuxGcc5ToolChain.py | 85 ++++++
.../LinuxGcc5ToolChain_plug_in.yaml | 12 +
.../WindowsResourceCompiler/WinRcPath.py | 29 ++
.../WinRcPath_plug_in.yaml | 13 +
.../WindowsVsToolChain/WindowsVsToolChain.py | 126 +++++++++
.../WindowsVsToolChain_plug_in.yaml | 11 +
.../basetool_tiano_python_path_env.yaml | 11 +
BaseTools/basetools_calling_path_env.yaml | 11 +
BaseTools/basetools_path_env.yaml | 11 +
BaseTools/set_vsprefix_envs.bat | 10 +
CryptoPkg/CryptoPkg.ci.yaml | 48 ++++
FatPkg/FatPkg.ci.yaml | 50 ++++
FmpDevicePkg/FmpDevicePkg.ci.yaml | 43 +++
FmpDevicePkg/FmpDevicePkg.dsc | 6 +-
Maintainers.txt | 23 ++
MdeModulePkg/MdeModulePkg.ci.yaml | 81 ++++++
MdePkg/MdePkg.ci.yaml | 90 +++++++
NetworkPkg/NetworkPkg.ci.yaml | 62 +++++
NetworkPkg/NetworkPkg.dsc | 5 +
PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml | 46 ++++
Readme.md | 11 +
SecurityPkg/SecurityPkg.ci.yaml | 80 ++++++
SecurityPkg/SecurityPkg.dsc | 32 ++-
ShellPkg/ShellPkg.ci.yaml | 55 ++++
SignedCapsulePkg/SignedCapsulePkg.dsc | 38 ++-
UefiCpuPkg/UefiCpuPkg.ci.yaml | 51 ++++
requirements.txt | 17 ++
69 files changed, 3878 insertions(+), 8 deletions(-)
create mode 100644 .azurepipelines/Ubuntu-GCC5.yml
create mode 100644 .azurepipelines/Ubuntu-PatchCheck.yml
create mode 100644 .azurepipelines/Windows-VS2019.yml
create mode 100644 .azurepipelines/templates/basetools-build-steps.yml
create mode 100644 .azurepipelines/templates/pr-gate-build-job.yml
create mode 100644 .azurepipelines/templates/pr-gate-steps.yml
create mode 100644 .azurepipelines/templates/spell-check-prereq-steps.yml
create mode 100644 .mergify/config.yml
create mode 100644 .pytool/CISettings.py
create mode 100644 .pytool/Plugin/CharEncodingCheck/CharEncodingCheck.py
create mode 100644 .pytool/Plugin/CharEncodingCheck/CharEncodingCheck_plug_in.yaml
create mode 100644 .pytool/Plugin/CharEncodingCheck/Readme.md
create mode 100644 .pytool/Plugin/CompilerPlugin/CompilerPlugin.py
create mode 100644 .pytool/Plugin/CompilerPlugin/Compiler_plug_in.yaml
create mode 100644 .pytool/Plugin/DependencyCheck/DependencyCheck.py
create mode 100644 .pytool/Plugin/DependencyCheck/DependencyCheck_plug_in.yaml
create mode 100644 .pytool/Plugin/DscCompleteCheck/DscCompleteCheck.py
create mode 100644 .pytool/Plugin/DscCompleteCheck/DscCompleteCheck_plug_in.yaml
create mode 100644 .pytool/Plugin/DscCompleteCheck/readme.md
create mode 100644 .pytool/Plugin/GuidCheck/GuidCheck.py
create mode 100644 .pytool/Plugin/GuidCheck/GuidCheck_plug_in.yaml
create mode 100644 .pytool/Plugin/GuidCheck/Readme.md
create mode 100644 .pytool/Plugin/LibraryClassCheck/LibraryClassCheck.py
create mode 100644 .pytool/Plugin/LibraryClassCheck/LibraryClassCheck_plug_in.yaml
create mode 100644 .pytool/Plugin/LibraryClassCheck/readme.md
create mode 100644 .pytool/Plugin/SpellCheck/Readme.md
create mode 100644 .pytool/Plugin/SpellCheck/SpellCheck.py
create mode 100644 .pytool/Plugin/SpellCheck/SpellCheck_plug_in.yaml
create mode 100644 .pytool/Plugin/SpellCheck/cspell.base.yaml
create mode 100644 .pytool/Readme.md
create mode 100644 BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml
create mode 100644 BaseTools/Bin/gcc_arm_linux_ext_dep.yaml
create mode 100644 BaseTools/Bin/iasl_ext_dep.yaml
create mode 100644 BaseTools/Bin/nasm_ext_dep.yaml
create mode 100644 BaseTools/BinWrappers/PosixLike/posix_path_env.yaml
create mode 100644 BaseTools/BinWrappers/WindowsLike/win_build_tools_path_env.yaml
create mode 100644 BaseTools/Edk2ToolsBuild.py
create mode 100644 BaseTools/Plugin/BuildToolsReport/BuildToolsReportGenerator.py
create mode 100644 BaseTools/Plugin/BuildToolsReport/BuildToolsReportGenerator_plug_in.yaml
create mode 100644 BaseTools/Plugin/BuildToolsReport/BuildToolsReport_Template.html
create mode 100644 BaseTools/Plugin/LinuxGcc5ToolChain/LinuxGcc5ToolChain.py
create mode 100644 BaseTools/Plugin/LinuxGcc5ToolChain/LinuxGcc5ToolChain_plug_in.yaml
create mode 100644 BaseTools/Plugin/WindowsResourceCompiler/WinRcPath.py
create mode 100644 BaseTools/Plugin/WindowsResourceCompiler/WinRcPath_plug_in.yaml
create mode 100644 BaseTools/Plugin/WindowsVsToolChain/WindowsVsToolChain.py
create mode 100644 BaseTools/Plugin/WindowsVsToolChain/WindowsVsToolChain_plug_in.yaml
create mode 100644 BaseTools/Source/Python/basetool_tiano_python_path_env.yaml
create mode 100644 BaseTools/basetools_calling_path_env.yaml
create mode 100644 BaseTools/basetools_path_env.yaml
create mode 100644 CryptoPkg/CryptoPkg.ci.yaml
create mode 100644 FatPkg/FatPkg.ci.yaml
create mode 100644 FmpDevicePkg/FmpDevicePkg.ci.yaml
create mode 100644 MdeModulePkg/MdeModulePkg.ci.yaml
create mode 100644 MdePkg/MdePkg.ci.yaml
create mode 100644 NetworkPkg/NetworkPkg.ci.yaml
create mode 100644 PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml
create mode 100644 SecurityPkg/SecurityPkg.ci.yaml
create mode 100644 ShellPkg/ShellPkg.ci.yaml
create mode 100644 UefiCpuPkg/UefiCpuPkg.ci.yaml
create mode 100644 requirements.txt
--
2.21.0.windows.1
^ permalink raw reply [flat|nested] 51+ messages in thread
* [Patch v4 01/22] Maintainers.txt: Add continuous integration(CI) directories
2019-11-07 1:13 [Patch v4 00/22] Enable Phase 1 of EDK II CI Michael D Kinney
@ 2019-11-07 1:13 ` Michael D Kinney
2019-11-07 1:13 ` [Patch v4 02/22] .gitignore: Ignore python compiled files, extdeps, and vscode Michael D Kinney
` (21 subsequent siblings)
22 siblings, 0 replies; 51+ messages in thread
From: Michael D Kinney @ 2019-11-07 1:13 UTC (permalink / raw)
To: devel; +Cc: Andrew Fish, Laszlo Ersek, Leif Lindholm
Add maintainers and reviewers for the directories associated
with continuous integration steps.
* .azurepipelines
* .mergify
* .pytool
Cc: Andrew Fish <afish@apple.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Leif Lindholm <leif.lindholm@linaro.org>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Leif Lindholm <leif.lindholm@linaro.org>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
---
Maintainers.txt | 23 +++++++++++++++++++++++
1 file changed, 23 insertions(+)
diff --git a/Maintainers.txt b/Maintainers.txt
index 922cc664dd..180ad35d09 100644
--- a/Maintainers.txt
+++ b/Maintainers.txt
@@ -98,6 +98,29 @@ F: */Arm/
M: Leif Lindholm <leif.lindholm@linaro.org>
M: Ard Biesheuvel <ard.biesheuvel@linaro.org>
+EDK II Continuous Integration:
+------------------------------
+.azurepipelines/
+F: .azurepipelines/
+M: Sean Brogan <sean.brogan@microsoft.com>
+M: Bret Barkelew <Bret.Barkelew@microsoft.com>
+R: Michael D Kinney <michael.d.kinney@intel.com>
+R: Liming Gao <liming.gao@intel.com>
+
+.mergify/
+F: .mergify/
+M: Michael D Kinney <michael.d.kinney@intel.com>
+M: Liming Gao <liming.gao@intel.com>
+R: Sean Brogan <sean.brogan@microsoft.com>
+R: Bret Barkelew <Bret.Barkelew@microsoft.com>
+
+.pytool/
+F: .pytool/
+M: Sean Brogan <sean.brogan@microsoft.com>
+M: Bret Barkelew <Bret.Barkelew@microsoft.com>
+R: Michael D Kinney <michael.d.kinney@intel.com>
+R: Liming Gao <liming.gao@intel.com>
+
EDK II Packages:
----------------
ArmPkg
--
2.21.0.windows.1
^ permalink raw reply related [flat|nested] 51+ messages in thread
* [Patch v4 02/22] .gitignore: Ignore python compiled files, extdeps, and vscode
2019-11-07 1:13 [Patch v4 00/22] Enable Phase 1 of EDK II CI Michael D Kinney
2019-11-07 1:13 ` [Patch v4 01/22] Maintainers.txt: Add continuous integration(CI) directories Michael D Kinney
@ 2019-11-07 1:13 ` Michael D Kinney
2019-11-07 10:26 ` Laszlo Ersek
2019-11-07 1:13 ` [Patch v4 03/22] requirements.txt: Add python pip requirements file Michael D Kinney
` (20 subsequent siblings)
22 siblings, 1 reply; 51+ messages in thread
From: Michael D Kinney @ 2019-11-07 1:13 UTC (permalink / raw)
To: devel
Cc: Sean Brogan, Andrew Fish, Laszlo Ersek, Leif Lindholm, Kinney,
Philippe Mathieu-Daude
From: Sean Brogan <sean.brogan@microsoft.com>
Update .gitignore to ignore .pyc files and __pycache__
directories. Python based plugins can be added to any
package or platform, so these files and directories may
be present outside of BaseTools.
Ignore _extdep directories that are generated by the
pytool external dependency feature.
Ignore .vscode directories generated by the VS Code
editor.
Cc: Andrew Fish <afish@apple.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Leif Lindholm <leif.lindholm@linaro.org>
Signed-off-by: Kinney <michael.d.kinney@intel.com>
Reviewed-by: Leif Lindholm <leif.lindholm@linaro.org>
Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
---
.gitignore | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/.gitignore b/.gitignore
index 97f22c348c..1dd30c1410 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,3 +1,7 @@
Build/
-tags/
.DS_Store
+*_extdep/
+*.pyc
+__pycache__/
+tags/
+.vscode/
--
2.21.0.windows.1
^ permalink raw reply related [flat|nested] 51+ messages in thread
* [Patch v4 03/22] requirements.txt: Add python pip requirements file
2019-11-07 1:13 [Patch v4 00/22] Enable Phase 1 of EDK II CI Michael D Kinney
2019-11-07 1:13 ` [Patch v4 01/22] Maintainers.txt: Add continuous integration(CI) directories Michael D Kinney
2019-11-07 1:13 ` [Patch v4 02/22] .gitignore: Ignore python compiled files, extdeps, and vscode Michael D Kinney
@ 2019-11-07 1:13 ` Michael D Kinney
2019-11-07 10:39 ` Laszlo Ersek
2019-11-07 10:49 ` Laszlo Ersek
2019-11-07 1:13 ` [Patch v4 04/22] BaseTools: Add RC_PATH define for VS2017/2019 Michael D Kinney
` (19 subsequent siblings)
22 siblings, 2 replies; 51+ messages in thread
From: Michael D Kinney @ 2019-11-07 1:13 UTC (permalink / raw)
To: devel; +Cc: Sean Brogan, Andrew Fish, Laszlo Ersek, Leif Lindholm
From: Sean Brogan <sean.brogan@microsoft.com>
Add pip requirements file that is used to install the
python pip modules build from the edk2-pytool-library and
edk2-pytool-extensions repositories.
These python modules provide the extensions required to
perform EDK II Continuous Integration(CI) builds.
Cc: Andrew Fish <afish@apple.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Leif Lindholm <leif.lindholm@linaro.org>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
---
requirements.txt | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
create mode 100644 requirements.txt
diff --git a/requirements.txt b/requirements.txt
new file mode 100644
index 0000000000..4ad72cfc98
--- /dev/null
+++ b/requirements.txt
@@ -0,0 +1,17 @@
+## @file
+# EDK II Python PIP requirements file
+#
+# This file provides the list of python components to install using PIP.
+#
+# Copyright (c) Microsoft Corporation. All rights reserved.<BR>
+#
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+#
+# https://pypi.org/project/pip/
+# https://pip.pypa.io/en/stable/user_guide/#requirements-files
+# https://pip.pypa.io/en/stable/reference/pip_install/#requirements-file-format
+#
+##
+
+edk2-pytool-library==0.10.*
+edk2-pytool-extensions==0.12.*
--
2.21.0.windows.1
^ permalink raw reply related [flat|nested] 51+ messages in thread
* [Patch v4 04/22] BaseTools: Add RC_PATH define for VS2017/2019
2019-11-07 1:13 [Patch v4 00/22] Enable Phase 1 of EDK II CI Michael D Kinney
` (2 preceding siblings ...)
2019-11-07 1:13 ` [Patch v4 03/22] requirements.txt: Add python pip requirements file Michael D Kinney
@ 2019-11-07 1:13 ` Michael D Kinney
2019-11-07 1:13 ` [Patch v4 05/22] BaseTools: Add YAML files with path env and tool extdeps Michael D Kinney
` (18 subsequent siblings)
22 siblings, 0 replies; 51+ messages in thread
From: Michael D Kinney @ 2019-11-07 1:13 UTC (permalink / raw)
To: devel; +Cc: Sean Brogan, Bob Feng, Liming Gao
From: Sean Brogan <sean.brogan@microsoft.com>
Add use of RC_PATH define that provides the path to the resource
compiler that is typically provided in a Windows SDK. The path
changes with different Windows SDK releases. This define is set
to the WINSDK_PATH_FOR_RC_EXE environment variable. This
environment variable must be set to the path to the currently
installed resource compiler (rc.exe).
Update set_vsprefix_envs.bat to set WINSDK_PATH_FOR_RC_EXE
if a Windows SDK is detected.
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
---
BaseTools/Conf/tools_def.template | 18 ++++++++++++++++--
BaseTools/set_vsprefix_envs.bat | 10 ++++++++++
2 files changed, 26 insertions(+), 2 deletions(-)
diff --git a/BaseTools/Conf/tools_def.template b/BaseTools/Conf/tools_def.template
index dce6c5875a..afa13a7cae 100755
--- a/BaseTools/Conf/tools_def.template
+++ b/BaseTools/Conf/tools_def.template
@@ -4,9 +4,18 @@
# Portions copyright (c) 2011 - 2014, ARM Ltd. All rights reserved.<BR>
# Copyright (c) 2015, Hewlett-Packard Development Company, L.P.<BR>
# (C) Copyright 2016 Hewlett Packard Enterprise Development LP<BR>
+# Copyright (c) Microsoft Corporation
#
# SPDX-License-Identifier: BSD-2-Clause-Patent
#
+#
+# Increase this version tag any time you want user to get warning about updating this
+# file in the Conf dir. By default it does not do update existing conf dirs.
+#
+# 2.00 - Initial version with changes for CI
+# - Change RC path to use plugin
+#
+#!VERSION=2.00
IDENTIFIER = Default TOOL_CHAIN_CONF
@@ -71,6 +80,11 @@ DEFINE VS2019_BIN_X64 = DEF(VS2019_BIN)\HostDEF(VS_HOST)\x64
DEFINE VS2019_BIN_ARM = DEF(VS2019_BIN)\HostDEF(VS_HOST)\arm
DEFINE VS2019_BIN_AARCH64 = DEF(VS2019_BIN)\HostDEF(VS_HOST)\arm64
+#
+# Resource compiler
+#
+DEFINE RC_PATH = ENV(WINSDK_PATH_FOR_RC_EXE)\rc.exe
+
DEFINE WINSDK_BIN = ENV(WINSDK_PREFIX)
DEFINE WINSDKx86_BIN = ENV(WINSDKx86_PREFIX)
@@ -1566,7 +1580,7 @@ NOOPT_VS2015x86_X64_DLINK_FLAGS = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /OPT:REF
*_VS2017_*_MAKE_PATH = DEF(VS2017_BIN_HOST)\nmake.exe
*_VS2017_*_MAKE_FLAG = /nologo
-*_VS2017_*_RC_PATH = DEF(WINSDK10_BIN)\rc.exe
+*_VS2017_*_RC_PATH = DEF(RC_PATH)
*_VS2017_*_MAKE_FLAGS = /nologo
*_VS2017_*_SLINK_FLAGS = /NOLOGO /LTCG
@@ -1729,7 +1743,7 @@ NOOPT_VS2017_AARCH64_DLINK_FLAGS = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /OPT:REF
*_VS2019_*_MAKE_PATH = DEF(VS2019_BIN_HOST)\nmake.exe
*_VS2019_*_MAKE_FLAG = /nologo
-*_VS2019_*_RC_PATH = DEF(WINSDK10_BIN)\rc.exe
+*_VS2019_*_RC_PATH = DEF(RC_PATH)
*_VS2019_*_MAKE_FLAGS = /nologo
*_VS2019_*_SLINK_FLAGS = /NOLOGO /LTCG
diff --git a/BaseTools/set_vsprefix_envs.bat b/BaseTools/set_vsprefix_envs.bat
index 6dff25d930..9966a5a4ed 100644
--- a/BaseTools/set_vsprefix_envs.bat
+++ b/BaseTools/set_vsprefix_envs.bat
@@ -158,6 +158,11 @@ if defined VCToolsInstallDir (
)
)
)
+if not defined WINSDK_PATH_FOR_RC_EXE (
+ if defined WINSDK10_PREFIX (
+ set "WINSDK_PATH_FOR_RC_EXE=%WINSDK10_PREFIX%x86"
+ )
+)
:SetVS2017
if not defined VS150COMNTOOLS (
@@ -209,6 +214,11 @@ if defined VCToolsInstallDir (
)
)
)
+if not defined WINSDK_PATH_FOR_RC_EXE (
+ if defined WINSDK10_PREFIX (
+ set "WINSDK_PATH_FOR_RC_EXE=%WINSDK10_PREFIX%x86"
+ )
+)
:SetWinDDK
if not defined WINDDK3790_PREFIX (
--
2.21.0.windows.1
^ permalink raw reply related [flat|nested] 51+ messages in thread
* [Patch v4 05/22] BaseTools: Add YAML files with path env and tool extdeps
2019-11-07 1:13 [Patch v4 00/22] Enable Phase 1 of EDK II CI Michael D Kinney
` (3 preceding siblings ...)
2019-11-07 1:13 ` [Patch v4 04/22] BaseTools: Add RC_PATH define for VS2017/2019 Michael D Kinney
@ 2019-11-07 1:13 ` Michael D Kinney
2019-11-07 1:13 ` [Patch v4 06/22] BaseTools: Add BaseTools plugins to support CI Michael D Kinney
` (17 subsequent siblings)
22 siblings, 0 replies; 51+ messages in thread
From: Michael D Kinney @ 2019-11-07 1:13 UTC (permalink / raw)
To: devel; +Cc: Sean Brogan, Bob Feng, Liming Gao
From: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
---
BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml | 21 +++
BaseTools/Bin/gcc_arm_linux_ext_dep.yaml | 21 +++
BaseTools/Bin/iasl_ext_dep.yaml | 21 +++
BaseTools/Bin/nasm_ext_dep.yaml | 18 ++
.../BinWrappers/PosixLike/posix_path_env.yaml | 10 ++
.../WindowsLike/win_build_tools_path_env.yaml | 10 ++
BaseTools/Edk2ToolsBuild.py | 163 ++++++++++++++++++
.../basetool_tiano_python_path_env.yaml | 11 ++
BaseTools/basetools_calling_path_env.yaml | 11 ++
BaseTools/basetools_path_env.yaml | 11 ++
10 files changed, 297 insertions(+)
create mode 100644 BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml
create mode 100644 BaseTools/Bin/gcc_arm_linux_ext_dep.yaml
create mode 100644 BaseTools/Bin/iasl_ext_dep.yaml
create mode 100644 BaseTools/Bin/nasm_ext_dep.yaml
create mode 100644 BaseTools/BinWrappers/PosixLike/posix_path_env.yaml
create mode 100644 BaseTools/BinWrappers/WindowsLike/win_build_tools_path_env.yaml
create mode 100644 BaseTools/Edk2ToolsBuild.py
create mode 100644 BaseTools/Source/Python/basetool_tiano_python_path_env.yaml
create mode 100644 BaseTools/basetools_calling_path_env.yaml
create mode 100644 BaseTools/basetools_path_env.yaml
diff --git a/BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml b/BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml
new file mode 100644
index 0000000000..69648c89d8
--- /dev/null
+++ b/BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml
@@ -0,0 +1,21 @@
+##
+# Download GCC AARCH64 compiler from Linaro's release site
+# Set shell variable GCC5_AARCH64_INSTALL to this folder
+#
+# This is only downloaded when a build activates scope gcc_aarch64_linux
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "scope": "gcc_aarch64_linux",
+ "type": "web",
+ "name": "gcc_aarch64_linux",
+ "source": "http://releases.linaro.org/components/toolchain/binaries/7.4-2019.02/aarch64-linux-gnu/gcc-linaro-7.4.1-2019.02-x86_64_aarch64-linux-gnu.tar.xz",
+ "version": "7.4.1",
+ "sha256": "27f1dc2c491ed61ae8f0d4b0c11de59cd2f7dd9c94761ee7153006fcac1bf9ab",
+ "compression_type": "tar",
+ "internal_path": "/gcc-linaro-7.4.1-2019.02-x86_64_aarch64-linux-gnu/",
+ "flags": ["set_shell_var", ],
+ "var_name": "GCC5_AARCH64_INSTALL"
+}
diff --git a/BaseTools/Bin/gcc_arm_linux_ext_dep.yaml b/BaseTools/Bin/gcc_arm_linux_ext_dep.yaml
new file mode 100644
index 0000000000..e108aaad2a
--- /dev/null
+++ b/BaseTools/Bin/gcc_arm_linux_ext_dep.yaml
@@ -0,0 +1,21 @@
+##
+# Download GCC ARM compiler from Linaro's release site
+# Set shell variable GCC5_ARM_INSTALL to this folder
+#
+# This is only downloaded when a build activates scope gcc_arm_linux
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "scope": "gcc_arm_linux",
+ "type": "web",
+ "name": "gcc_arm_linux",
+ "source": "https://releases.linaro.org/components/toolchain/binaries/7.4-2019.02/arm-linux-gnueabihf/gcc-linaro-7.4.1-2019.02-x86_64_arm-linux-gnueabihf.tar.xz",
+ "version": "7.4.1",
+ "sha256": "3C951CF1941D0FA06D64CC0D5E88612B209D8123B273FA26C16D70BD7BC6B163",
+ "compression_type": "tar",
+ "internal_path": "/gcc-linaro-7.4.1-2019.02-x86_64_arm-linux-gnueabihf/",
+ "flags": ["set_shell_var", ],
+ "var_name": "GCC5_ARM_INSTALL"
+}
diff --git a/BaseTools/Bin/iasl_ext_dep.yaml b/BaseTools/Bin/iasl_ext_dep.yaml
new file mode 100644
index 0000000000..1c63921367
--- /dev/null
+++ b/BaseTools/Bin/iasl_ext_dep.yaml
@@ -0,0 +1,21 @@
+##
+# Download iasl executable tool from a nuget.org package
+# - package contains different binaries based on host
+# Add the folder with the tool to the path
+#
+# This is only downloaded for scope cibuild thus
+# should have no impact on the asl compiler used by any
+# given platform to build.
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "id": "iasl-ci-1",
+ "scope": "cibuild",
+ "type": "nuget",
+ "name": "iasl",
+ "source": "https://api.nuget.org/v3/index.json",
+ "version": "20190215.0.0",
+ "flags": ["set_path", "host_specific"]
+}
diff --git a/BaseTools/Bin/nasm_ext_dep.yaml b/BaseTools/Bin/nasm_ext_dep.yaml
new file mode 100644
index 0000000000..8894516b5a
--- /dev/null
+++ b/BaseTools/Bin/nasm_ext_dep.yaml
@@ -0,0 +1,18 @@
+##
+# Download nasm x86 assembler executable tool from a nuget.org package
+# - package contains different binaries based on host
+# Put on the tool on the path
+#
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "id": "nasm-1",
+ "scope": "edk2-build",
+ "type": "nuget",
+ "name": "mu_nasm",
+ "source": "https://api.nuget.org/v3/index.json",
+ "version": "2.14.02",
+ "flags": ["set_path", "host_specific"]
+}
diff --git a/BaseTools/BinWrappers/PosixLike/posix_path_env.yaml b/BaseTools/BinWrappers/PosixLike/posix_path_env.yaml
new file mode 100644
index 0000000000..eec65ed7db
--- /dev/null
+++ b/BaseTools/BinWrappers/PosixLike/posix_path_env.yaml
@@ -0,0 +1,10 @@
+##
+# Set this folder on the path for all linux builds
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "scope": "global-nix",
+ "flags": ["set_path"]
+}
diff --git a/BaseTools/BinWrappers/WindowsLike/win_build_tools_path_env.yaml b/BaseTools/BinWrappers/WindowsLike/win_build_tools_path_env.yaml
new file mode 100644
index 0000000000..340724f39f
--- /dev/null
+++ b/BaseTools/BinWrappers/WindowsLike/win_build_tools_path_env.yaml
@@ -0,0 +1,10 @@
+##
+# Add this folder to the path on Windows
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "scope": "global-win",
+ "flags": ["set_path"]
+}
diff --git a/BaseTools/Edk2ToolsBuild.py b/BaseTools/Edk2ToolsBuild.py
new file mode 100644
index 0000000000..057d2e9e06
--- /dev/null
+++ b/BaseTools/Edk2ToolsBuild.py
@@ -0,0 +1,163 @@
+# @file Edk2ToolsBuild.py
+# Invocable class that builds the basetool c files.
+#
+# Supports VS2017, VS2019, and GCC5
+##
+# Copyright (c) Microsoft Corporation
+#
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+import os
+import sys
+import logging
+import argparse
+from edk2toolext import edk2_logging
+from edk2toolext.environment import self_describing_environment
+from edk2toolext.base_abstract_invocable import BaseAbstractInvocable
+from edk2toollib.utility_functions import RunCmd
+from edk2toollib.windows.locate_tools import QueryVcVariables
+
+
+class Edk2ToolsBuild(BaseAbstractInvocable):
+
+ def ParseCommandLineOptions(self):
+ ''' parse arguments '''
+ ParserObj = argparse.ArgumentParser()
+ ParserObj.add_argument("-t", "--tool_chain_tag", dest="tct", default="VS2017",
+ help="Set the toolchain used to compile the build tools")
+ args = ParserObj.parse_args()
+ self.tool_chain_tag = args.tct
+
+ def GetWorkspaceRoot(self):
+ ''' Return the workspace root for initializing the SDE '''
+
+ # this is the bastools dir...not the traditional EDK2 workspace root
+ return os.path.dirname(os.path.abspath(__file__))
+
+ def GetActiveScopes(self):
+ ''' return tuple containing scopes that should be active for this process '''
+
+ # for now don't use scopes
+ return ('global',)
+
+ def GetLoggingLevel(self, loggerType):
+ ''' Get the logging level for a given type (return Logging.Level)
+ base == lowest logging level supported
+ con == Screen logging
+ txt == plain text file logging
+ md == markdown file logging
+ '''
+ if(loggerType == "con"):
+ return logging.ERROR
+ else:
+ return logging.DEBUG
+
+ def GetLoggingFolderRelativeToRoot(self):
+ ''' Return a path to folder for log files '''
+ return "BaseToolsBuild"
+
+ def GetVerifyCheckRequired(self):
+ ''' Will call self_describing_environment.VerifyEnvironment if this returns True '''
+ return True
+
+ def GetLoggingFileName(self, loggerType):
+ ''' Get the logging file name for the type.
+ Return None if the logger shouldn't be created
+
+ base == lowest logging level supported
+ con == Screen logging
+ txt == plain text file logging
+ md == markdown file logging
+ '''
+ return "BASETOOLS_BUILD"
+
+ def WritePathEnvFile(self, OutputDir):
+ ''' Write a PyTool path env file for future PyTool based edk2 builds'''
+ content = '''##
+# Set shell variable EDK_TOOLS_BIN to this folder
+#
+# Autogenerated by Edk2ToolsBuild.py
+#
+# Copyright (c), Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "id": "You-Built-BaseTools",
+ "scope": "edk2-build",
+ "flags": ["set_shell_var", "set_path"],
+ "var_name": "EDK_TOOLS_BIN"
+}
+'''
+ with open(os.path.join(OutputDir, "basetoolsbin_path_env.yaml"), "w") as f:
+ f.write(content)
+
+ def Go(self):
+ logging.info("Running Python version: " + str(sys.version_info))
+
+ (build_env, shell_env) = self_describing_environment.BootstrapEnvironment(
+ self.GetWorkspaceRoot(), self.GetActiveScopes())
+
+ # # Bind our current execution environment into the shell vars.
+ ph = os.path.dirname(sys.executable)
+ if " " in ph:
+ ph = '"' + ph + '"'
+ shell_env.set_shell_var("PYTHON_HOME", ph)
+ # PYTHON_COMMAND is required to be set for using edk2 python builds.
+ pc = sys.executable
+ if " " in pc:
+ pc = '"' + pc + '"'
+ shell_env.set_shell_var("PYTHON_COMMAND", pc)
+
+ if self.tool_chain_tag.lower().startswith("vs"):
+
+ # # Update environment with required VC vars.
+ interesting_keys = ["ExtensionSdkDir", "INCLUDE", "LIB"]
+ interesting_keys.extend(
+ ["LIBPATH", "Path", "UniversalCRTSdkDir", "UCRTVersion", "WindowsLibPath", "WindowsSdkBinPath"])
+ interesting_keys.extend(
+ ["WindowsSdkDir", "WindowsSdkVerBinPath", "WindowsSDKVersion", "VCToolsInstallDir"])
+ vc_vars = QueryVcVariables(
+ interesting_keys, 'x86', vs_version=self.tool_chain_tag.lower())
+ for key in vc_vars.keys():
+ logging.debug(f"Var - {key} = {vc_vars[key]}")
+ if key.lower() == 'path':
+ shell_env.insert_path(vc_vars[key])
+ else:
+ shell_env.set_shell_var(key, vc_vars[key])
+
+ self.OutputDir = os.path.join(
+ shell_env.get_shell_var("EDK_TOOLS_PATH"), "Bin", "Win32")
+
+ # compiled tools need to be added to path because antlr is referenced
+ shell_env.insert_path(self.OutputDir)
+
+ # Actually build the tools.
+ ret = RunCmd('nmake.exe', None,
+ workingdir=shell_env.get_shell_var("EDK_TOOLS_PATH"))
+ if ret != 0:
+ raise Exception("Failed to build.")
+
+ self.WritePathEnvFile(self.OutputDir)
+ return ret
+
+ elif self.tool_chain_tag.lower().startswith("gcc"):
+ ret = RunCmd("make", "-C .", workingdir=shell_env.get_shell_var("EDK_TOOLS_PATH"))
+ if ret != 0:
+ raise Exception("Failed to build.")
+
+ self.OutputDir = os.path.join(
+ shell_env.get_shell_var("EDK_TOOLS_PATH"), "Source", "C", "bin")
+
+ self.WritePathEnvFile(self.OutputDir)
+ return ret
+
+ logging.critical("Tool Chain not supported")
+ return -1
+
+
+def main():
+ Edk2ToolsBuild().Invoke()
+
+
+if __name__ == "__main__":
+ main()
diff --git a/BaseTools/Source/Python/basetool_tiano_python_path_env.yaml b/BaseTools/Source/Python/basetool_tiano_python_path_env.yaml
new file mode 100644
index 0000000000..1edc8810ce
--- /dev/null
+++ b/BaseTools/Source/Python/basetool_tiano_python_path_env.yaml
@@ -0,0 +1,11 @@
+##
+# Add this folder to the pypath so modules can be easily
+# loaded
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "scope": "global",
+ "flags": ["set_pypath"]
+}
diff --git a/BaseTools/basetools_calling_path_env.yaml b/BaseTools/basetools_calling_path_env.yaml
new file mode 100644
index 0000000000..cfdbcea104
--- /dev/null
+++ b/BaseTools/basetools_calling_path_env.yaml
@@ -0,0 +1,11 @@
+##
+# Set shell variable BASE_TOOL_PATH to this folder
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "scope": "global",
+ "flags": ["set_shell_var"],
+ "var_name": "BASE_TOOLS_PATH"
+}
diff --git a/BaseTools/basetools_path_env.yaml b/BaseTools/basetools_path_env.yaml
new file mode 100644
index 0000000000..f64a3b8d45
--- /dev/null
+++ b/BaseTools/basetools_path_env.yaml
@@ -0,0 +1,11 @@
+##
+# Set shell variable EDK_TOOL_PATH to this folder
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "scope": "global",
+ "flags": ["set_shell_var", "set_build_var"],
+ "var_name": "EDK_TOOLS_PATH"
+}
--
2.21.0.windows.1
^ permalink raw reply related [flat|nested] 51+ messages in thread
* [Patch v4 06/22] BaseTools: Add BaseTools plugins to support CI
2019-11-07 1:13 [Patch v4 00/22] Enable Phase 1 of EDK II CI Michael D Kinney
` (4 preceding siblings ...)
2019-11-07 1:13 ` [Patch v4 05/22] BaseTools: Add YAML files with path env and tool extdeps Michael D Kinney
@ 2019-11-07 1:13 ` Michael D Kinney
2019-11-07 1:13 ` [Patch v4 07/22] .pytool/Plugin: Add CI plugins Michael D Kinney
` (16 subsequent siblings)
22 siblings, 0 replies; 51+ messages in thread
From: Michael D Kinney @ 2019-11-07 1:13 UTC (permalink / raw)
To: devel; +Cc: Sean Brogan, Bob Feng, Liming Gao
From: Sean Brogan <sean.brogan@microsoft.com>
Add the following plugins that are required to support
EDK II Continuous Integration (CI) builds. These plugins
are added to BaseTools because that support EDK II BaseTools
features.
* BuildToolsReportGenerator
* LinuxGcc5ToolChain
* WindowsResourceCompiler
* WindowsVsToolChain
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
---
.../BuildToolsReportGenerator.py | 69 ++++++++++
.../BuildToolsReportGenerator_plug_in.yaml | 12 ++
.../BuildToolsReport_Template.html | 126 ++++++++++++++++++
.../LinuxGcc5ToolChain/LinuxGcc5ToolChain.py | 85 ++++++++++++
.../LinuxGcc5ToolChain_plug_in.yaml | 12 ++
.../WindowsResourceCompiler/WinRcPath.py | 29 ++++
.../WinRcPath_plug_in.yaml | 13 ++
.../WindowsVsToolChain/WindowsVsToolChain.py | 126 ++++++++++++++++++
.../WindowsVsToolChain_plug_in.yaml | 11 ++
9 files changed, 483 insertions(+)
create mode 100644 BaseTools/Plugin/BuildToolsReport/BuildToolsReportGenerator.py
create mode 100644 BaseTools/Plugin/BuildToolsReport/BuildToolsReportGenerator_plug_in.yaml
create mode 100644 BaseTools/Plugin/BuildToolsReport/BuildToolsReport_Template.html
create mode 100644 BaseTools/Plugin/LinuxGcc5ToolChain/LinuxGcc5ToolChain.py
create mode 100644 BaseTools/Plugin/LinuxGcc5ToolChain/LinuxGcc5ToolChain_plug_in.yaml
create mode 100644 BaseTools/Plugin/WindowsResourceCompiler/WinRcPath.py
create mode 100644 BaseTools/Plugin/WindowsResourceCompiler/WinRcPath_plug_in.yaml
create mode 100644 BaseTools/Plugin/WindowsVsToolChain/WindowsVsToolChain.py
create mode 100644 BaseTools/Plugin/WindowsVsToolChain/WindowsVsToolChain_plug_in.yaml
diff --git a/BaseTools/Plugin/BuildToolsReport/BuildToolsReportGenerator.py b/BaseTools/Plugin/BuildToolsReport/BuildToolsReportGenerator.py
new file mode 100644
index 0000000000..d5df9f2f9b
--- /dev/null
+++ b/BaseTools/Plugin/BuildToolsReport/BuildToolsReportGenerator.py
@@ -0,0 +1,69 @@
+##
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+import os
+import logging
+import json
+
+try:
+ from edk2toolext.environment.plugintypes.uefi_build_plugin import IUefiBuildPlugin
+
+ class BuildToolsReportGenerator(IUefiBuildPlugin):
+ def do_report(self, thebuilder):
+ try:
+ from edk2toolext.environment import version_aggregator
+ except ImportError:
+ logging.critical("Loading BuildToolsReportGenerator failed, please update your Edk2-PyTool-Extensions")
+ return 0
+
+ OutputReport = os.path.join(thebuilder.env.GetValue("BUILD_OUTPUT_BASE"), "BUILD_TOOLS_REPORT")
+ OutputReport = os.path.normpath(OutputReport)
+ if not os.path.isdir(os.path.dirname(OutputReport)):
+ os.makedirs(os.path.dirname(OutputReport))
+
+ Report = BuildToolsReport()
+ Report.MakeReport(version_aggregator.GetVersionAggregator().GetAggregatedVersionInformation(), OutputReport=OutputReport)
+
+ def do_pre_build(self, thebuilder):
+ self.do_report(thebuilder)
+ return 0
+
+ def do_post_build(self, thebuilder):
+ self.do_report(thebuilder)
+ return 0
+
+except ImportError:
+ pass
+
+
+class BuildToolsReport(object):
+ MY_FOLDER = os.path.dirname(os.path.realpath(__file__))
+ VERSION = "1.00"
+
+ def __init__(self):
+ pass
+
+ def MakeReport(self, BuildTools, OutputReport="BuildToolsReport"):
+ logging.info("Writing BuildToolsReports to {0}".format(OutputReport))
+ versions_list = []
+ for key, value in BuildTools.items():
+ versions_list.append(value)
+ versions_list = sorted(versions_list, key=lambda k: k['type'])
+ json_dict = {"modules": versions_list,
+ "PluginVersion": BuildToolsReport.VERSION}
+
+ htmlfile = open(OutputReport + ".html", "w")
+ jsonfile = open(OutputReport + ".json", "w")
+ template = open(os.path.join(BuildToolsReport.MY_FOLDER, "BuildToolsReport_Template.html"), "r")
+
+ for line in template.readlines():
+ if "%TO_BE_FILLED_IN_BY_PYTHON_SCRIPT%" in line:
+ line = line.replace("%TO_BE_FILLED_IN_BY_PYTHON_SCRIPT%", json.dumps(json_dict))
+ htmlfile.write(line)
+
+ jsonfile.write(json.dumps(versions_list, indent=4))
+
+ jsonfile.close()
+ template.close()
+ htmlfile.close()
diff --git a/BaseTools/Plugin/BuildToolsReport/BuildToolsReportGenerator_plug_in.yaml b/BaseTools/Plugin/BuildToolsReport/BuildToolsReportGenerator_plug_in.yaml
new file mode 100644
index 0000000000..50afd53926
--- /dev/null
+++ b/BaseTools/Plugin/BuildToolsReport/BuildToolsReportGenerator_plug_in.yaml
@@ -0,0 +1,12 @@
+##
+# Build Plugin used to output html report of all versions collected
+# during the build
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "scope": "global",
+ "name": "Build Tools Report Generator",
+ "module": "BuildToolsReportGenerator"
+}
diff --git a/BaseTools/Plugin/BuildToolsReport/BuildToolsReport_Template.html b/BaseTools/Plugin/BuildToolsReport/BuildToolsReport_Template.html
new file mode 100644
index 0000000000..26350ab425
--- /dev/null
+++ b/BaseTools/Plugin/BuildToolsReport/BuildToolsReport_Template.html
@@ -0,0 +1,126 @@
+<!doctype html>
+<html lang="en">
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible">
+ <title>Build Tools Report</title>
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <link rel="stylesheet" type="text/css" href="https://ajax.aspnetcdn.com/ajax/bootstrap/3.3.7/css/bootstrap.min.css" />
+ <link rel="stylesheet" type="text/css" href="https://cdn.datatables.net/1.10.15/css/dataTables.bootstrap.min.css" />
+ <style>
+ div.attribution {
+ border: 1px solid #ddd;
+ background-color: #bbb;
+ padding-left: 20px;
+ }
+ </style>
+</head>
+<body>
+ <div class="container-fluid">
+ <h1>Build Tools Report</h1>
+ <ul class="nav nav-tabs">
+ <li class="active"><a data-toggle="tab" href="#tabs-1">Tools</a></li>
+ <li><a data-toggle="tab" href="#tabs-2">About</a></li>
+ </ul>
+ <div class="tab-content">
+ <div id="tabs-1" class="tab-pane fade in active">
+ <table id="modinfo" class="table table-striped table-bordered table-hover" cellspacing="0">
+ <thead>
+ <tr>
+ <th>Key</th>
+ <th>Value</th>
+ <th>Type</th>
+ </tr>
+ </thead>
+ <tbody></tbody>
+ </table>
+ </div>
+ <div id="tabs-2" class="tab-pane">
+ <div class="row">
+ <div class="col-xs-7">
+ <p></p>
+ <p>
+ Build Tools Report Template Version: <span id="ReportTemplateVersion">1.00</span><br />
+ Build Tools Report Plugin Version: <span id='ReportToolVersion'></span><br />
+ </p>
+ <h3>License</h3>
+ <hr />
+ <div id="ToolLicenseContent">
+ <p>
+ <span class="copyright">Copyright (c) Microsoft Corporation. All rights reserved.</span><br />
+ <span class="license">
+ SPDX-License-Identifier: BSD-2-Clause-Patent
+ </span>
+ </p>
+ </div>
+ </div>
+ <div id="AttributionListWrapper" class="col-xs-5">
+ <h3>External Licenses</h3>
+ </div>
+ </div>
+ </div>
+ </div>
+ </div>
+
+ <!-- Javascript libraries -->
+ <script type="text/javascript" charset="utf8" src="https://ajax.aspnetcdn.com/ajax/jQuery/jquery-3.2.1.min.js"></script>
+ <script type="text/javascript" charset="utf8" src="https://cdn.datatables.net/1.10.15/js/jquery.dataTables.min.js"></script>
+ <script type="text/javascript" charset="utf8" src="https://ajax.aspnetcdn.com/ajax/bootstrap/3.3.7/bootstrap.min.js"></script>
+ <script type="text/javascript" charset="utf8" src="https://cdn.datatables.net/1.10.15/js/dataTables.bootstrap.min.js"></script>
+
+ <script>
+ var EmbeddedJd = %TO_BE_FILLED_IN_BY_PYTHON_SCRIPT%;
+ </script>
+ <!-- Add javascript here -->
+ <script>
+ var MODULE_TABLE_OFFSET = 350; //Space needed for other stuff besides the Table
+ $(document).ready(function () {
+ $('span#ReportToolVersion').text(EmbeddedJd.PluginVersion);
+ //To support tabs and correct column width we need this change
+ $('a[data-toggle="tab"][href="#tabs-1"]').on('shown.bs.tab', function (e) {
+ $.fn.dataTable.tables({ visible: true, api: true }).columns.adjust();
+ });
+ //table for modules
+ var mTable = $('table#modinfo').dataTable({
+ "aaData": EmbeddedJd.modules,
+ "paginate": false,
+ "autoWidth": false,
+ "scrollY": ($(window).height() - MODULE_TABLE_OFFSET) + "px",
+ "aaSorting": [[2, "asc"]],
+ "aoColumnDefs": [
+ {
+ "mData": "name",
+ "aTargets": [0]
+ },
+
+ {
+ "mData": "version",
+ "aTargets": [1]
+ },
+ {
+ "mData": "type",
+ "aTargets": [2],
+ }
+ ] //end of column def
+ }); //end of modules table
+
+ //
+ // Create Attribution List for all external libraries used
+ //
+ [
+ { Title: "JQuery", Copyright: "Copyright 2017 The jQuery Foundation", Version: $.fn.jquery, LicenseType: "MIT", LicenseLink: "https://jquery.org/license/" },
+ { Title: "DataTables", Copyright: "DataTables designed and created by SpryMedia Ltd Copyright 2007-2017", Version: $.fn.dataTable.version, LicenseType: "MIT", LicenseLink: "https://datatables.net/license/mit" },
+ { Title: "BootStrap", Copyright: "Code and documentation copyright 2011-2017 the Bootstrap Authors and Twitter, Inc.", Version: "3.3.7", LicenseType: "MIT", LicenseLink: "https://github.com/twbs/bootstrap/blob/master/LICENSE" }
+ ].forEach(function (element) {
+ $("<div class='attribution'><h4>" + element.Title + "</h4><p>Version: <span class='version'>" + element.Version + "</span><br /><span class='copyright'>" +
+ element.Copyright + "</span><br />License: <a class='license' href='" + element.LicenseLink + "'>" + element.LicenseType + "</a></p></div>").appendTo("div#AttributionListWrapper");
+ });
+ });
+ $(window).resize(function() {
+ $.fn.dataTable.tables({ visible: true, api: true }).columns.adjust();
+ });
+
+
+ </script>
+</body>
+</html>
diff --git a/BaseTools/Plugin/LinuxGcc5ToolChain/LinuxGcc5ToolChain.py b/BaseTools/Plugin/LinuxGcc5ToolChain/LinuxGcc5ToolChain.py
new file mode 100644
index 0000000000..c31641e931
--- /dev/null
+++ b/BaseTools/Plugin/LinuxGcc5ToolChain/LinuxGcc5ToolChain.py
@@ -0,0 +1,85 @@
+# @file LinuxGcc5ToolChain.py
+# Plugin to configures paths for GCC5 ARM/AARCH64 Toolchain
+##
+# This plugin works in conjuncture with the tools_def
+#
+# Copyright (c) Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+import os
+import logging
+from edk2toolext.environment.plugintypes.uefi_build_plugin import IUefiBuildPlugin
+from edk2toolext.environment import shell_environment
+
+
+class LinuxGcc5ToolChain(IUefiBuildPlugin):
+
+ def do_post_build(self, thebuilder):
+ return 0
+
+ def do_pre_build(self, thebuilder):
+ self.Logger = logging.getLogger("LinuxGcc5ToolChain")
+
+ #
+ # GCC5 - The ARM and AARCH64 compilers need their paths set if available
+ if thebuilder.env.GetValue("TOOL_CHAIN_TAG") == "GCC5":
+
+ # Start with AARACH64 compiler
+ ret = self._check_aarch64()
+ if ret != 0:
+ self.Logger.critical("Failed in check aarch64")
+ return ret
+
+ # Check arm compiler
+ ret = self._check_arm()
+ if ret != 0:
+ self.Logger.critical("Failed in check arm")
+ return ret
+
+ return 0
+
+ def _check_arm(self):
+ # check to see if full path already configured
+ if shell_environment.GetEnvironment().get_shell_var("GCC5_ARM_PREFIX") is not None:
+ self.Logger.info("GCC5_ARM_PREFIX is already set.")
+
+ else:
+ # now check for install dir. If set then set the Prefix
+ install_path = shell_environment.GetEnvironment().get_shell_var("GCC5_ARM_INSTALL")
+ if install_path is None:
+ return 0
+
+ # make GCC5_ARM_PREFIX to align with tools_def.txt
+ prefix = os.path.join(install_path, "bin", "arm-linux-gnueabihf-")
+ shell_environment.GetEnvironment().set_shell_var("GCC5_ARM_PREFIX", prefix)
+
+ # now confirm it exists
+ if not os.path.exists(shell_environment.GetEnvironment().get_shell_var("GCC5_ARM_PREFIX") + "gcc"):
+ self.Logger.error("Path for GCC5_ARM_PREFIX toolchain is invalid")
+ return -2
+
+ return 0
+
+ def _check_aarch64(self):
+ # check to see if full path already configured
+ if shell_environment.GetEnvironment().get_shell_var("GCC5_AARCH64_PREFIX") is not None:
+ self.Logger.info("GCC5_AARCH64_PREFIX is already set.")
+
+ else:
+ # now check for install dir. If set then set the Prefix
+ install_path = shell_environment.GetEnvironment(
+ ).get_shell_var("GCC5_AARCH64_INSTALL")
+ if install_path is None:
+ return 0
+
+ # make GCC5_AARCH64_PREFIX to align with tools_def.txt
+ prefix = os.path.join(install_path, "bin", "aarch64-linux-gnu-")
+ shell_environment.GetEnvironment().set_shell_var("GCC5_AARCH64_PREFIX", prefix)
+
+ # now confirm it exists
+ if not os.path.exists(shell_environment.GetEnvironment().get_shell_var("GCC5_AARCH64_PREFIX") + "gcc"):
+ self.Logger.error(
+ "Path for GCC5_AARCH64_PREFIX toolchain is invalid")
+ return -2
+
+ return 0
diff --git a/BaseTools/Plugin/LinuxGcc5ToolChain/LinuxGcc5ToolChain_plug_in.yaml b/BaseTools/Plugin/LinuxGcc5ToolChain/LinuxGcc5ToolChain_plug_in.yaml
new file mode 100644
index 0000000000..91cfcfbb28
--- /dev/null
+++ b/BaseTools/Plugin/LinuxGcc5ToolChain/LinuxGcc5ToolChain_plug_in.yaml
@@ -0,0 +1,12 @@
+##
+# Build Plugin used to set the path
+# for the GCC5 ARM/AARCH64 downloaded compilers
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "scope": "global-nix",
+ "name": "Linux GCC5 Tool Chain Support",
+ "module": "LinuxGcc5ToolChain"
+}
diff --git a/BaseTools/Plugin/WindowsResourceCompiler/WinRcPath.py b/BaseTools/Plugin/WindowsResourceCompiler/WinRcPath.py
new file mode 100644
index 0000000000..ec2f2d1298
--- /dev/null
+++ b/BaseTools/Plugin/WindowsResourceCompiler/WinRcPath.py
@@ -0,0 +1,29 @@
+## @file WinRcPath.py
+# Plugin to find Windows SDK Resource Compiler rc.exe
+##
+# This plugin works in conjuncture with the tools_def to support rc.exe
+#
+# Copyright (c) Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+import os
+from edk2toolext.environment.plugintypes.uefi_build_plugin import IUefiBuildPlugin
+import edk2toollib.windows.locate_tools as locate_tools
+from edk2toolext.environment import shell_environment
+from edk2toolext.environment import version_aggregator
+
+class WinRcPath(IUefiBuildPlugin):
+
+ def do_post_build(self, thebuilder):
+ return 0
+
+ def do_pre_build(self, thebuilder):
+ #get the locate tools module
+ path = locate_tools.FindToolInWinSdk("rc.exe")
+ if path is None:
+ thebuilder.logging.warning("Failed to find rc.exe")
+ else:
+ p = os.path.abspath(os.path.dirname(path))
+ shell_environment.GetEnvironment().set_shell_var("WINSDK_PATH_FOR_RC_EXE", p)
+ version_aggregator.GetVersionAggregator().ReportVersion("WINSDK_PATH_FOR_RC_EXE", p, version_aggregator.VersionTypes.INFO)
+ return 0
diff --git a/BaseTools/Plugin/WindowsResourceCompiler/WinRcPath_plug_in.yaml b/BaseTools/Plugin/WindowsResourceCompiler/WinRcPath_plug_in.yaml
new file mode 100644
index 0000000000..030ebe776c
--- /dev/null
+++ b/BaseTools/Plugin/WindowsResourceCompiler/WinRcPath_plug_in.yaml
@@ -0,0 +1,13 @@
+##
+# Build Plugin used to set the path to rc.exe on windows.
+# The plugin is able to use python to locate the tool as to avoid
+# hard-coding the path
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "scope": "global-win",
+ "name": "Windows RC Path Support",
+ "module": "WinRcPath"
+}
diff --git a/BaseTools/Plugin/WindowsVsToolChain/WindowsVsToolChain.py b/BaseTools/Plugin/WindowsVsToolChain/WindowsVsToolChain.py
new file mode 100644
index 0000000000..a8202e5992
--- /dev/null
+++ b/BaseTools/Plugin/WindowsVsToolChain/WindowsVsToolChain.py
@@ -0,0 +1,126 @@
+## @file WindowsVsToolChain.py
+# Plugin to configures paths for the VS2017 and VS2019 tool chain
+##
+# This plugin works in conjuncture with the tools_def
+#
+# Copyright (c) Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+import os
+import logging
+from edk2toolext.environment.plugintypes.uefi_build_plugin import IUefiBuildPlugin
+import edk2toollib.windows.locate_tools as locate_tools
+from edk2toollib.windows.locate_tools import FindWithVsWhere
+from edk2toolext.environment import shell_environment
+from edk2toolext.environment import version_aggregator
+
+class WindowsVsToolChain(IUefiBuildPlugin):
+
+ def do_post_build(self, thebuilder):
+ return 0
+
+ def do_pre_build(self, thebuilder):
+ self.Logger = logging.getLogger("WindowsVsToolChain")
+
+#
+ # VS2017 - Follow VS2017 where there is potential for many versions of the tools.
+ # If a specific version is required then the user must set both env variables:
+ ## VS150INSTALLPATH: base install path on system to VC install dir. Here you will find the VC folder, etc
+ ## VS150TOOLVER: version number for the VC compiler tools
+ ## VS2017_PREFIX: path to MSVC compiler folder with trailing slash (can be used instead of two vars above)
+ if thebuilder.env.GetValue("TOOL_CHAIN_TAG") == "VS2017":
+
+ # check to see if full path already configured
+ if shell_environment.GetEnvironment().get_shell_var("VS2017_PREFIX") != None:
+ self.Logger.info("VS2017_PREFIX is already set.")
+
+ else:
+ install_path = self._get_vs_install_path("VS2017".lower(), "VS150INSTALLPATH")
+ vc_ver = self._get_vc_version(install_path, "VS150TOOLVER")
+
+ if install_path is None or vc_ver is None:
+ self.Logger.error("Failed to configure environment for VS2017")
+ return -1
+
+ version_aggregator.GetVersionAggregator().ReportVersion(
+ "Visual Studio Install Path", install_path, version_aggregator.VersionTypes.INFO)
+ version_aggregator.GetVersionAggregator().ReportVersion(
+ "VC Version", vc_ver, version_aggregator.VersionTypes.TOOL)
+
+ #make VS2017_PREFIX to align with tools_def.txt
+ prefix = os.path.join(install_path, "VC", "Tools", "MSVC", vc_ver)
+ prefix = prefix + os.path.sep
+ shell_environment.GetEnvironment().set_shell_var("VS2017_PREFIX", prefix)
+
+ # now confirm it exists
+ if not os.path.exists(shell_environment.GetEnvironment().get_shell_var("VS2017_PREFIX")):
+ self.Logger.error("Path for VS2017 toolchain is invalid")
+ return -2
+
+ #
+ # VS2019 - Follow VS2019 where there is potential for many versions of the tools.
+ # If a specific version is required then the user must set both env variables:
+ ## VS160INSTALLPATH: base install path on system to VC install dir. Here you will find the VC folder, etc
+ ## VS160TOOLVER: version number for the VC compiler tools
+ ## VS2019_PREFIX: path to MSVC compiler folder with trailing slash (can be used instead of two vars above)
+ elif thebuilder.env.GetValue("TOOL_CHAIN_TAG") == "VS2019":
+
+ # check to see if full path already configured
+ if shell_environment.GetEnvironment().get_shell_var("VS2019_PREFIX") != None:
+ self.Logger.info("VS2019_PREFIX is already set.")
+
+ else:
+ install_path = self._get_vs_install_path("VS2019".lower(), "VS160INSTALLPATH")
+ vc_ver = self._get_vc_version(install_path, "VS160TOOLVER")
+
+ if install_path is None or vc_ver is None:
+ self.Logger.error("Failed to configure environment for VS2019")
+ return -1
+
+ version_aggregator.GetVersionAggregator().ReportVersion(
+ "Visual Studio Install Path", install_path, version_aggregator.VersionTypes.INFO)
+ version_aggregator.GetVersionAggregator().ReportVersion(
+ "VC Version", vc_ver, version_aggregator.VersionTypes.TOOL)
+
+ #make VS2019_PREFIX to align with tools_def.txt
+ prefix = os.path.join(install_path, "VC", "Tools", "MSVC", vc_ver)
+ prefix = prefix + os.path.sep
+ shell_environment.GetEnvironment().set_shell_var("VS2019_PREFIX", prefix)
+
+ # now confirm it exists
+ if not os.path.exists(shell_environment.GetEnvironment().get_shell_var("VS2019_PREFIX")):
+ self.Logger.error("Path for VS2019 toolchain is invalid")
+ return -2
+
+ return 0
+
+ def _get_vs_install_path(self, vs_version, varname):
+ # check if already specified
+ path = shell_environment.GetEnvironment().get_shell_var(varname)
+ if(path is None):
+ # Not specified...find latest
+ (rc, path) = FindWithVsWhere(vs_version=vs_version)
+ if rc == 0 and path is not None and os.path.exists(path):
+ self.Logger.debug("Found VS instance for %s", vs_version)
+ else:
+ self.Logger.error("Failed to find VS instance with VsWhere (%d)" % rc)
+ return path
+
+ def _get_vc_version(self, path, varname):
+ # check if already specified
+ vc_ver = shell_environment.GetEnvironment().get_shell_var(varname)
+ if (path is None):
+ self.Logger.critical("Failed to find Visual Studio tools. Might need to check for VS install")
+ return vc_ver
+ if(vc_ver is None):
+ # Not specified...find latest
+ p2 = os.path.join(path, "VC", "Tools", "MSVC")
+ if not os.path.isdir(p2):
+ self.Logger.critical(
+ "Failed to find VC tools. Might need to check for VS install")
+ return vc_ver
+ vc_ver = os.listdir(p2)[-1].strip() # get last in list
+ self.Logger.debug("Found VC Tool version is %s" % vc_ver)
+ return vc_ver
+
+
diff --git a/BaseTools/Plugin/WindowsVsToolChain/WindowsVsToolChain_plug_in.yaml b/BaseTools/Plugin/WindowsVsToolChain/WindowsVsToolChain_plug_in.yaml
new file mode 100644
index 0000000000..0d7acc2e6d
--- /dev/null
+++ b/BaseTools/Plugin/WindowsVsToolChain/WindowsVsToolChain_plug_in.yaml
@@ -0,0 +1,11 @@
+##
+# Build Plugin used to set the path to the visual studio tools chain
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "scope": "global-win",
+ "name": "Windows Visual Studio Tool Chain Support",
+ "module": "WindowsVsToolChain"
+}
--
2.21.0.windows.1
^ permalink raw reply related [flat|nested] 51+ messages in thread
* [Patch v4 07/22] .pytool/Plugin: Add CI plugins
2019-11-07 1:13 [Patch v4 00/22] Enable Phase 1 of EDK II CI Michael D Kinney
` (5 preceding siblings ...)
2019-11-07 1:13 ` [Patch v4 06/22] BaseTools: Add BaseTools plugins to support CI Michael D Kinney
@ 2019-11-07 1:13 ` Michael D Kinney
2019-11-07 6:58 ` Liming Gao
2019-11-07 1:13 ` [Patch v4 08/22] CryptoPkg: Add YAML file for CI builds Michael D Kinney
` (15 subsequent siblings)
22 siblings, 1 reply; 51+ messages in thread
From: Michael D Kinney @ 2019-11-07 1:13 UTC (permalink / raw)
To: devel; +Cc: Sean Brogan, Bret Barkelew, Liming Gao
From: Sean Brogan <sean.brogan@microsoft.com>
Add .pytool directory to the edk2 repository with the
following plugins. These plugins are in a top level
directory because that can be used with all packages
and platforms.
* CharEncodingCheck
* CompilerPlugin
* DependencyCheck
* DscCompleteCheck
* GuidCheck
* LibraryClassCheck
* SpellCheck
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
---
.../CharEncodingCheck/CharEncodingCheck.py | 118 ++++++++
.../CharEncodingCheck_plug_in.yaml | 11 +
.pytool/Plugin/CharEncodingCheck/Readme.md | 13 +
.../Plugin/CompilerPlugin/CompilerPlugin.py | 102 +++++++
.../CompilerPlugin/Compiler_plug_in.yaml | 11 +
.../Plugin/DependencyCheck/DependencyCheck.py | 120 +++++++++
.../DependencyCheck_plug_in.yaml | 13 +
.../DscCompleteCheck/DscCompleteCheck.py | 118 ++++++++
.../DscCompleteCheck_plug_in.yaml | 12 +
.pytool/Plugin/DscCompleteCheck/readme.md | 22 ++
.pytool/Plugin/GuidCheck/GuidCheck.py | 251 ++++++++++++++++++
.../Plugin/GuidCheck/GuidCheck_plug_in.yaml | 11 +
.pytool/Plugin/GuidCheck/Readme.md | 60 +++++
.../LibraryClassCheck/LibraryClassCheck.py | 153 +++++++++++
.../LibraryClassCheck_plug_in.yaml | 11 +
.pytool/Plugin/LibraryClassCheck/readme.md | 22 ++
.pytool/Plugin/SpellCheck/Readme.md | 100 +++++++
.pytool/Plugin/SpellCheck/SpellCheck.py | 216 +++++++++++++++
.../Plugin/SpellCheck/SpellCheck_plug_in.yaml | 11 +
.pytool/Plugin/SpellCheck/cspell.base.yaml | 165 ++++++++++++
20 files changed, 1540 insertions(+)
create mode 100644 .pytool/Plugin/CharEncodingCheck/CharEncodingCheck.py
create mode 100644 .pytool/Plugin/CharEncodingCheck/CharEncodingCheck_plug_in.yaml
create mode 100644 .pytool/Plugin/CharEncodingCheck/Readme.md
create mode 100644 .pytool/Plugin/CompilerPlugin/CompilerPlugin.py
create mode 100644 .pytool/Plugin/CompilerPlugin/Compiler_plug_in.yaml
create mode 100644 .pytool/Plugin/DependencyCheck/DependencyCheck.py
create mode 100644 .pytool/Plugin/DependencyCheck/DependencyCheck_plug_in.yaml
create mode 100644 .pytool/Plugin/DscCompleteCheck/DscCompleteCheck.py
create mode 100644 .pytool/Plugin/DscCompleteCheck/DscCompleteCheck_plug_in.yaml
create mode 100644 .pytool/Plugin/DscCompleteCheck/readme.md
create mode 100644 .pytool/Plugin/GuidCheck/GuidCheck.py
create mode 100644 .pytool/Plugin/GuidCheck/GuidCheck_plug_in.yaml
create mode 100644 .pytool/Plugin/GuidCheck/Readme.md
create mode 100644 .pytool/Plugin/LibraryClassCheck/LibraryClassCheck.py
create mode 100644 .pytool/Plugin/LibraryClassCheck/LibraryClassCheck_plug_in.yaml
create mode 100644 .pytool/Plugin/LibraryClassCheck/readme.md
create mode 100644 .pytool/Plugin/SpellCheck/Readme.md
create mode 100644 .pytool/Plugin/SpellCheck/SpellCheck.py
create mode 100644 .pytool/Plugin/SpellCheck/SpellCheck_plug_in.yaml
create mode 100644 .pytool/Plugin/SpellCheck/cspell.base.yaml
diff --git a/.pytool/Plugin/CharEncodingCheck/CharEncodingCheck.py b/.pytool/Plugin/CharEncodingCheck/CharEncodingCheck.py
new file mode 100644
index 0000000000..54a2424875
--- /dev/null
+++ b/.pytool/Plugin/CharEncodingCheck/CharEncodingCheck.py
@@ -0,0 +1,118 @@
+# @file CharEncodingCheck.py
+#
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+
+
+import os
+import logging
+from edk2toolext.environment.plugintypes.ci_build_plugin import ICiBuildPlugin
+from edk2toolext.environment.var_dict import VarDict
+
+##
+# map
+##
+EcodingMap = {
+ ".md": 'utf-8',
+ ".dsc": 'utf-8',
+ ".dec": 'utf-8',
+ ".c": 'utf-8',
+ ".h": 'utf-8',
+ ".asm": 'utf-8',
+ ".masm": 'utf-8',
+ ".nasm": 'utf-8',
+ ".s": 'utf-8',
+ ".inf": 'utf-8',
+ ".asl": 'utf-8',
+ ".uni": 'utf-8',
+ ".py": 'utf-8'
+}
+
+
+class CharEncodingCheck(ICiBuildPlugin):
+ """
+ A CiBuildPlugin that scans each file in the code tree and confirms the encoding is correct.
+
+ Configuration options:
+ "CharEncodingCheck": {
+ "IgnoreFiles": []
+ }
+ """
+
+ def GetTestName(self, packagename: str, environment: VarDict) -> tuple:
+ """ Provide the testcase name and classname for use in reporting
+ testclassname: a descriptive string for the testcase can include whitespace
+ classname: should be patterned <packagename>.<plugin>.<optionally any unique condition>
+
+ Args:
+ packagename: string containing name of package to build
+ environment: The VarDict for the test to run in
+ Returns:
+ a tuple containing the testcase name and the classname
+ (testcasename, classname)
+ """
+ return ("Check for valid file encoding for " + packagename, packagename + ".CharEncodingCheck")
+
+ ##
+ # External function of plugin. This function is used to perform the task of the ci_build_plugin Plugin
+ #
+ # - package is the edk2 path to package. This means workspace/packagepath relative.
+ # - edk2path object configured with workspace and packages path
+ # - PkgConfig Object (dict) for the pkg
+ # - EnvConfig Object
+ # - Plugin Manager Instance
+ # - Plugin Helper Obj Instance
+ # - Junit Logger
+ # - output_stream the StringIO output stream from this plugin via logging
+ def RunBuildPlugin(self, packagename, Edk2pathObj, pkgconfig, environment, PLM, PLMHelper, tc, output_stream=None):
+ overall_status = 0
+ files_tested = 0
+
+ abs_pkg_path = Edk2pathObj.GetAbsolutePathOnThisSytemFromEdk2RelativePath(packagename)
+
+ if abs_pkg_path is None:
+ tc.SetSkipped()
+ tc.LogStdError("No Package folder {0}".format(abs_pkg_path))
+ return 0
+
+ for (ext, enc) in EcodingMap.items():
+ files = self.WalkDirectoryForExtension([ext], abs_pkg_path)
+ files = [Edk2pathObj.GetEdk2RelativePathFromAbsolutePath(x) for x in files] # make edk2relative path so can process ignores
+
+ if "IgnoreFiles" in pkgconfig:
+ for a in pkgconfig["IgnoreFiles"]:
+ a = a.replace(os.sep, "/")
+ try:
+ tc.LogStdOut("Ignoring File {0}".format(a))
+ files.remove(a)
+ except:
+ tc.LogStdError("CharEncodingCheck.IgnoreInf -> {0} not found in filesystem. Invalid ignore file".format(a))
+ logging.info("CharEncodingCheck.IgnoreInf -> {0} not found in filesystem. Invalid ignore file".format(a))
+
+ files = [Edk2pathObj.GetAbsolutePathOnThisSytemFromEdk2RelativePath(x) for x in files]
+ for a in files:
+ files_tested += 1
+ if(self.TestEncodingOk(a, enc)):
+ logging.debug("File {0} Passed Encoding Check {1}".format(a, enc))
+ else:
+ tc.LogStdError("Encoding Failure in {0}. Not {1}".format(a, enc))
+ overall_status += 1
+
+ tc.LogStdOut("Tested Encoding on {0} files".format(files_tested))
+ if overall_status is not 0:
+ tc.SetFailed("CharEncoding {0} Failed. Errors {1}".format(packagename, overall_status), "CHAR_ENCODING_CHECK_FAILED")
+ else:
+ tc.SetSuccess()
+ return overall_status
+
+ def TestEncodingOk(self, apath, encodingValue):
+ try:
+ with open(apath, "rb") as fobj:
+ fobj.read().decode(encodingValue)
+ except Exception as exp:
+ logging.error("Encoding failure: file: {0} type: {1}".format(apath, encodingValue))
+ logging.debug("EXCEPTION: while processing {1} - {0}".format(exp, apath))
+ return False
+
+ return True
diff --git a/.pytool/Plugin/CharEncodingCheck/CharEncodingCheck_plug_in.yaml b/.pytool/Plugin/CharEncodingCheck/CharEncodingCheck_plug_in.yaml
new file mode 100644
index 0000000000..915d3f4fdb
--- /dev/null
+++ b/.pytool/Plugin/CharEncodingCheck/CharEncodingCheck_plug_in.yaml
@@ -0,0 +1,11 @@
+##
+# CiBuildPlugin used to check char encoding
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "scope": "cibuild",
+ "name": "Char Encoding Check Test",
+ "module": "CharEncodingCheck"
+}
diff --git a/.pytool/Plugin/CharEncodingCheck/Readme.md b/.pytool/Plugin/CharEncodingCheck/Readme.md
new file mode 100644
index 0000000000..8350542966
--- /dev/null
+++ b/.pytool/Plugin/CharEncodingCheck/Readme.md
@@ -0,0 +1,13 @@
+# Character Encoding Check Plugin
+
+This CiBuildPlugin scans all the files in a package to make sure each file is correctly encoded and all characters can be read. Improper encoding causes tools to fail in some situations especially in different locals.
+
+## Configuration
+
+The plugin can be configured to ignore certain files.
+
+``` yaml
+"CharEncodingCheck": {
+ "IgnoreFiles": [] # optional - list of files to ignore
+}
+```
diff --git a/.pytool/Plugin/CompilerPlugin/CompilerPlugin.py b/.pytool/Plugin/CompilerPlugin/CompilerPlugin.py
new file mode 100644
index 0000000000..0a357309a4
--- /dev/null
+++ b/.pytool/Plugin/CompilerPlugin/CompilerPlugin.py
@@ -0,0 +1,102 @@
+# @file HostUnitTestCompiler_plugin.py
+##
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+
+import logging
+import os
+import re
+from edk2toollib.uefi.edk2.parsers.dsc_parser import DscParser
+from edk2toolext.environment.plugintypes.ci_build_plugin import ICiBuildPlugin
+from edk2toolext.environment.uefi_build import UefiBuilder
+from edk2toolext import edk2_logging
+from edk2toolext.environment.var_dict import VarDict
+
+
+class CompilerPlugin(ICiBuildPlugin):
+ """
+ A CiBuildPlugin that compiles the package dsc
+ from the package being tested.
+
+ Configuration options:
+ "CompilerPlugin": {
+ "DscPath": "<path to dsc from root of pkg>"
+ }
+ """
+
+ def GetTestName(self, packagename: str, environment: VarDict) -> tuple:
+ """ Provide the testcase name and classname for use in reporting
+
+ Args:
+ packagename: string containing name of package to build
+ environment: The VarDict for the test to run in
+ Returns:
+ a tuple containing the testcase name and the classname
+ (testcasename, classname)
+ """
+ target = environment.GetValue("TARGET")
+ return ("Compile " + packagename + " " + target, packagename + ".Compiler." + target)
+
+ def RunsOnTargetList(self):
+ return ["DEBUG", "RELEASE"]
+
+ ##
+ # External function of plugin. This function is used to perform the task of the MuBuild Plugin
+ #
+ # - package is the edk2 path to package. This means workspace/packagepath relative.
+ # - edk2path object configured with workspace and packages path
+ # - PkgConfig Object (dict) for the pkg
+ # - EnvConfig Object
+ # - Plugin Manager Instance
+ # - Plugin Helper Obj Instance
+ # - Junit Logger
+ # - output_stream the StringIO output stream from this plugin via logging
+ def RunBuildPlugin(self, packagename, Edk2pathObj, pkgconfig, environment, PLM, PLMHelper, tc, output_stream=None):
+ self._env = environment
+
+ # Parse the config for required DscPath element
+ if "DscPath" not in pkgconfig:
+ tc.SetSkipped()
+ tc.LogStdError("DscPath not found in config file. Nothing to compile.")
+ return -1
+
+ AP = Edk2pathObj.GetAbsolutePathOnThisSytemFromEdk2RelativePath(packagename)
+
+ APDSC = os.path.join(AP, pkgconfig["DscPath"].strip())
+ AP_Path = Edk2pathObj.GetEdk2RelativePathFromAbsolutePath(APDSC)
+ if AP is None or AP_Path is None or not os.path.isfile(APDSC):
+ tc.SetSkipped()
+ tc.LogStdError("Package Dsc not found.")
+ return -1
+
+ logging.info("Building {0}".format(AP_Path))
+ self._env.SetValue("ACTIVE_PLATFORM", AP_Path, "Set in Compiler Plugin")
+
+ # Parse DSC to check for SUPPORTED_ARCHITECTURES
+ dp = DscParser()
+ dp.SetBaseAbsPath(Edk2pathObj.WorkspacePath)
+ dp.SetPackagePaths(Edk2pathObj.PackagePathList)
+ dp.ParseFile(AP_Path)
+ if "SUPPORTED_ARCHITECTURES" in dp.LocalVars:
+ SUPPORTED_ARCHITECTURES = dp.LocalVars["SUPPORTED_ARCHITECTURES"].split('|')
+ TARGET_ARCHITECTURES = environment.GetValue("TARGET_ARCH").split(' ')
+
+ # Skip if there is no intersection between SUPPORTED_ARCHITECTURES and TARGET_ARCHITECTURES
+ if len(set(SUPPORTED_ARCHITECTURES) & set(TARGET_ARCHITECTURES)) == 0:
+ tc.SetSkipped()
+ tc.LogStdError("No supported architecutres to build")
+ return -1
+
+ uefiBuilder = UefiBuilder()
+ # do all the steps
+ # WorkSpace, PackagesPath, PInHelper, PInManager
+ ret = uefiBuilder.Go(Edk2pathObj.WorkspacePath, os.pathsep.join(Edk2pathObj.PackagePathList), PLMHelper, PLM)
+ if ret != 0: # failure:
+ tc.SetFailed("Compile failed for {0}".format(packagename), "Compile_FAILED")
+ tc.LogStdError("{0} Compile failed with error code {1} ".format(AP_Path, ret))
+ return 1
+
+ else:
+ tc.SetSuccess()
+ return 0
diff --git a/.pytool/Plugin/CompilerPlugin/Compiler_plug_in.yaml b/.pytool/Plugin/CompilerPlugin/Compiler_plug_in.yaml
new file mode 100644
index 0000000000..4f9b3d3113
--- /dev/null
+++ b/.pytool/Plugin/CompilerPlugin/Compiler_plug_in.yaml
@@ -0,0 +1,11 @@
+##
+# CiBuildPlugin used to compile each package
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "scope": "cibuild",
+ "name": "Compiler Plugin",
+ "module": "CompilerPlugin"
+}
diff --git a/.pytool/Plugin/DependencyCheck/DependencyCheck.py b/.pytool/Plugin/DependencyCheck/DependencyCheck.py
new file mode 100644
index 0000000000..497914cf3a
--- /dev/null
+++ b/.pytool/Plugin/DependencyCheck/DependencyCheck.py
@@ -0,0 +1,120 @@
+# @file dependency_check.py
+#
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+
+import logging
+import os
+from edk2toolext.environment.plugintypes.ci_build_plugin import ICiBuildPlugin
+from edk2toollib.uefi.edk2.parsers.inf_parser import InfParser
+from edk2toolext.environment.var_dict import VarDict
+
+
+class DependencyCheck(ICiBuildPlugin):
+ """
+ A CiBuildPlugin that finds all modules (inf files) in a package and reviews the packages used
+ to confirm they are acceptable. This is to help enforce layering and identify improper
+ dependencies between packages.
+
+ Configuration options:
+ "DependencyCheck": {
+ "AcceptableDependencies": [], # Package dec files that are allowed in all INFs. Example: MdePkg/MdePkg.dec
+ "AcceptableDependencies-<MODULE_TYPE>": [], # OPTIONAL Package dependencies for INFs that are HOST_APPLICATION
+ "AcceptableDependencies-HOST_APPLICATION": [], # EXAMPLE Package dependencies for INFs that are HOST_APPLICATION
+ "IgnoreInf": [] # Ignore INF if found in filesystem
+ }
+ """
+
+ def GetTestName(self, packagename: str, environment: VarDict) -> tuple:
+ """ Provide the testcase name and classname for use in reporting
+
+ Args:
+ packagename: string containing name of package to build
+ environment: The VarDict for the test to run in
+ Returns:
+ a tuple containing the testcase name and the classname
+ (testcasename, classname)
+ testclassname: a descriptive string for the testcase can include whitespace
+ classname: should be patterned <packagename>.<plugin>.<optionally any unique condition>
+ """
+ return ("Test Package Dependencies for modules in " + packagename, packagename + ".DependencyCheck")
+
+ ##
+ # External function of plugin. This function is used to perform the task of the MuBuild Plugin
+ #
+ # - package is the edk2 path to package. This means workspace/packagepath relative.
+ # - edk2path object configured with workspace and packages path
+ # - PkgConfig Object (dict) for the pkg
+ # - EnvConfig Object
+ # - Plugin Manager Instance
+ # - Plugin Helper Obj Instance
+ # - Junit Logger
+ # - output_stream the StringIO output stream from this plugin via logging
+ def RunBuildPlugin(self, packagename, Edk2pathObj, pkgconfig, environment, PLM, PLMHelper, tc, output_stream=None):
+ overall_status = 0
+
+ # Get current platform
+ abs_pkg_path = Edk2pathObj.GetAbsolutePathOnThisSytemFromEdk2RelativePath(packagename)
+
+ # Get INF Files
+ INFFiles = self.WalkDirectoryForExtension([".inf"], abs_pkg_path)
+ INFFiles = [Edk2pathObj.GetEdk2RelativePathFromAbsolutePath(x) for x in INFFiles] # make edk2relative path so can compare with Ignore List
+
+ # Remove ignored INFs
+ if "IgnoreInf" in pkgconfig:
+ for a in pkgconfig["IgnoreInf"]:
+ a = a.replace(os.sep, "/") ## convert path sep in case ignore list is bad. Can't change case
+ try:
+ INFFiles.remove(a)
+ tc.LogStdOut("IgnoreInf {0}".format(a))
+ except:
+ logging.info("DependencyConfig.IgnoreInf -> {0} not found in filesystem. Invalid ignore file".format(a))
+ tc.LogStdError("DependencyConfig.IgnoreInf -> {0} not found in filesystem. Invalid ignore file".format(a))
+
+
+ # Get the AccpetableDependencies list
+ if "AcceptableDependencies" not in pkgconfig:
+ logging.info("DependencyCheck Skipped. No Acceptable Dependencies defined.")
+ tc.LogStdOut("DependencyCheck Skipped. No Acceptable Dependencies defined.")
+ tc.SetSkipped()
+ return -1
+
+ # Log dependencies
+ for k in pkgconfig.keys():
+ if k.startswith("AcceptableDependencies"):
+ pkgstring = "\n".join(pkgconfig[k])
+ if ("-" in k):
+ _, _, mod_type = k.partition("-")
+ tc.LogStdOut(f"Additional dependencies for MODULE_TYPE {mod_type}:\n {pkgstring}")
+ else:
+ tc.LogStdOut(f"Acceptable Dependencies:\n {pkgstring}")
+
+ # For each INF file
+ for file in INFFiles:
+ ip = InfParser()
+ logging.debug("Parsing " + file)
+ ip.SetBaseAbsPath(Edk2pathObj.WorkspacePath).SetPackagePaths(Edk2pathObj.PackagePathList).ParseFile(file)
+
+ if("MODULE_TYPE" not in ip.Dict):
+ tc.LogStdOut("Ignoring INF. Missing key for MODULE_TYPE {0}".format(file))
+ continue
+
+ mod_type = ip.Dict["MODULE_TYPE"].upper()
+ for p in ip.PackagesUsed:
+ if p not in pkgconfig["AcceptableDependencies"]:
+ # If not in the main acceptable dependencies list then check module specific
+ mod_specific_key = "AcceptableDependencies-" + mod_type
+ if mod_specific_key in pkgconfig and p in pkgconfig[mod_specific_key]:
+ continue
+
+ logging.error("Dependency Check: Invalid Dependency INF: {0} depends on pkg {1}".format(file, p))
+ tc.LogStdError("Dependency Check: Invalid Dependency INF: {0} depends on pkg {1}".format(file, p))
+ overall_status += 1
+
+ # If XML object exists, add results
+ if overall_status is not 0:
+ tc.SetFailed("Failed with {0} errors".format(overall_status), "DEPENDENCYCHECK_FAILED")
+ else:
+ tc.SetSuccess()
+ return overall_status
diff --git a/.pytool/Plugin/DependencyCheck/DependencyCheck_plug_in.yaml b/.pytool/Plugin/DependencyCheck/DependencyCheck_plug_in.yaml
new file mode 100644
index 0000000000..fdb96d625b
--- /dev/null
+++ b/.pytool/Plugin/DependencyCheck/DependencyCheck_plug_in.yaml
@@ -0,0 +1,13 @@
+##
+# CiBuildPlugin used to check all infs within a package
+# to confirm the packagesdependency are on the configured list of acceptable
+# dependencies.
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "scope": "cibuild",
+ "name": "Dependency Check Test",
+ "module": "DependencyCheck"
+}
diff --git a/.pytool/Plugin/DscCompleteCheck/DscCompleteCheck.py b/.pytool/Plugin/DscCompleteCheck/DscCompleteCheck.py
new file mode 100644
index 0000000000..dcd8946ca6
--- /dev/null
+++ b/.pytool/Plugin/DscCompleteCheck/DscCompleteCheck.py
@@ -0,0 +1,118 @@
+# @file DscCompleteCheck.py
+#
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+import logging
+import os
+from edk2toolext.environment.plugintypes.ci_build_plugin import ICiBuildPlugin
+from edk2toollib.uefi.edk2.parsers.dsc_parser import DscParser
+from edk2toollib.uefi.edk2.parsers.inf_parser import InfParser
+from edk2toolext.environment.var_dict import VarDict
+
+
+class DscCompleteCheck(ICiBuildPlugin):
+ """
+ A CiBuildPlugin that scans the package dsc file and confirms all modules (inf files) are
+ listed in the components sections.
+
+ Configuration options:
+ "DscCompleteCheck": {
+ "DscPath": "<path to dsc from root of pkg>"
+ "IgnoreInf": [] # Ignore INF if found in filesystem by not dsc
+ }
+ """
+
+ def GetTestName(self, packagename: str, environment: VarDict) -> tuple:
+ """ Provide the testcase name and classname for use in reporting
+
+ Args:
+ packagename: string containing name of package to build
+ environment: The VarDict for the test to run in
+ Returns:
+ a tuple containing the testcase name and the classname
+ (testcasename, classname)
+ testclassname: a descriptive string for the testcase can include whitespace
+ classname: should be patterned <packagename>.<plugin>.<optionally any unique condition>
+ """
+ return ("Check the " + packagename + " DSC for a being complete", packagename + ".DscCompleteCheck")
+
+ ##
+ # External function of plugin. This function is used to perform the task of the MuBuild Plugin
+ #
+ # - package is the edk2 path to package. This means workspace/packagepath relative.
+ # - edk2path object configured with workspace and packages path
+ # - PkgConfig Object (dict) for the pkg
+ # - VarDict containing the shell environment Build Vars
+ # - Plugin Manager Instance
+ # - Plugin Helper Obj Instance
+ # - Junit Logger
+ # - output_stream the StringIO output stream from this plugin via logging
+ def RunBuildPlugin(self, packagename, Edk2pathObj, pkgconfig, environment, PLM, PLMHelper, tc, output_stream=None):
+ overall_status = 0
+
+ # Parse the config for required DscPath element
+ if "DscPath" not in pkgconfig:
+ tc.SetSkipped()
+ tc.LogStdError("DscPath not found in config file. Nothing to check.")
+ return -1
+
+ abs_pkg_path = Edk2pathObj.GetAbsolutePathOnThisSytemFromEdk2RelativePath(packagename)
+ abs_dsc_path = os.path.join(abs_pkg_path, pkgconfig["DscPath"].strip())
+ wsr_dsc_path = Edk2pathObj.GetEdk2RelativePathFromAbsolutePath(abs_dsc_path)
+
+ if abs_dsc_path is None or wsr_dsc_path is "" or not os.path.isfile(abs_dsc_path):
+ tc.SetSkipped()
+ tc.LogStdError("Package Dsc not found")
+ return 0
+
+ # Get INF Files
+ INFFiles = self.WalkDirectoryForExtension([".inf"], abs_pkg_path)
+ INFFiles = [Edk2pathObj.GetEdk2RelativePathFromAbsolutePath(x) for x in INFFiles] # make edk2relative path so can compare with DSC
+
+ # remove ignores
+
+ if "IgnoreInf" in pkgconfig:
+ for a in pkgconfig["IgnoreInf"]:
+ a = a.replace(os.sep, "/")
+ try:
+ tc.LogStdOut("Ignoring INF {0}".format(a))
+ INFFiles.remove(a)
+ except:
+ tc.LogStdError("DscCompleteCheck.IgnoreInf -> {0} not found in filesystem. Invalid ignore file".format(a))
+ logging.info("DscCompleteCheck.IgnoreInf -> {0} not found in filesystem. Invalid ignore file".format(a))
+
+ # DSC Parser
+ dp = DscParser()
+ dp.SetBaseAbsPath(Edk2pathObj.WorkspacePath)
+ dp.SetPackagePaths(Edk2pathObj.PackagePathList)
+ dp.SetInputVars(environment.GetAllBuildKeyValues())
+ dp.ParseFile(wsr_dsc_path)
+
+ # Check if INF in component section
+ for INF in INFFiles:
+ if not any(INF.strip() in x for x in dp.ThreeMods) and \
+ not any(INF.strip() in x for x in dp.SixMods) and \
+ not any(INF.strip() in x for x in dp.OtherMods):
+
+ infp = InfParser().SetBaseAbsPath(Edk2pathObj.WorkspacePath)
+ infp.SetPackagePaths(Edk2pathObj.PackagePathList)
+ infp.ParseFile(INF)
+ if("MODULE_TYPE" not in infp.Dict):
+ tc.LogStdOut("Ignoring INF. Missing key for MODULE_TYPE {0}".format(INF))
+ continue
+
+ if(infp.Dict["MODULE_TYPE"] == "HOST_APPLICATION"):
+ tc.LogStdOut("Ignoring INF. Module type is HOST_APPLICATION {0}".format(INF))
+ continue
+
+ logging.critical(INF + " not in " + wsr_dsc_path)
+ tc.LogStdError("{0} not in {1}".format(INF, wsr_dsc_path))
+ overall_status = overall_status + 1
+
+ # If XML object exists, add result
+ if overall_status is not 0:
+ tc.SetFailed("DscCompleteCheck {0} Failed. Errors {1}".format(wsr_dsc_path, overall_status), "CHECK_FAILED")
+ else:
+ tc.SetSuccess()
+ return overall_status
diff --git a/.pytool/Plugin/DscCompleteCheck/DscCompleteCheck_plug_in.yaml b/.pytool/Plugin/DscCompleteCheck/DscCompleteCheck_plug_in.yaml
new file mode 100644
index 0000000000..d84d57d973
--- /dev/null
+++ b/.pytool/Plugin/DscCompleteCheck/DscCompleteCheck_plug_in.yaml
@@ -0,0 +1,12 @@
+##
+# CiBuildPlugin used to confirm all INFs are listed in
+# the components section of package dsc
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "scope": "cibuild",
+ "name": "Dsc Complete Check Test",
+ "module": "DscCompleteCheck"
+}
diff --git a/.pytool/Plugin/DscCompleteCheck/readme.md b/.pytool/Plugin/DscCompleteCheck/readme.md
new file mode 100644
index 0000000000..17e542b8d6
--- /dev/null
+++ b/.pytool/Plugin/DscCompleteCheck/readme.md
@@ -0,0 +1,22 @@
+# Dsc Complete Check Plugin
+
+This CiBuildPlugin scans all INF files from a package and confirms they are listed in the package level DSC file. The test considers it an error if any INF does not appear in the `Components` section of the package-level DSC (indicating that it would not be built if the package were built). This is critical because much of the CI infrastructure assumes that all modules will be listed in the DSC and compiled.
+
+## Configuration
+
+The plugin has a few configuration options to support the UEFI codebase.
+
+``` yaml
+"DscCompleteCheck": {
+ "DscPath": "", # Path to dsc from root of package
+ "IgnoreInf": [] # Ignore INF if found in filesystem by not dsc
+ }
+```
+
+### DscPath
+
+Path to DSC to consider platform dsc
+
+### IgnoreInf
+
+Ignore error if Inf file is not listed in DSC file
diff --git a/.pytool/Plugin/GuidCheck/GuidCheck.py b/.pytool/Plugin/GuidCheck/GuidCheck.py
new file mode 100644
index 0000000000..467e17f3e8
--- /dev/null
+++ b/.pytool/Plugin/GuidCheck/GuidCheck.py
@@ -0,0 +1,251 @@
+# @file GuidCheck.py
+#
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+import logging
+from edk2toolext.environment.plugintypes.ci_build_plugin import ICiBuildPlugin
+from edk2toollib.uefi.edk2.guid_list import GuidList
+from edk2toolext.environment.var_dict import VarDict
+
+
+class GuidCheck(ICiBuildPlugin):
+ """
+ A CiBuildPlugin that scans the code tree and looks for duplicate guids
+ from the package being tested.
+
+ Configuration options:
+ "GuidCheck": {
+ "IgnoreGuidName": [], # provide in format guidname=guidvalue or just guidname
+ "IgnoreGuidValue": [],
+ "IgnoreFoldersAndFiles": [],
+ "IgnoreDuplicates": [] # Provide in format guidname=guidname=guidname...
+ }
+ """
+
+ def GetTestName(self, packagename: str, environment: VarDict) -> tuple:
+ """ Provide the testcase name and classname for use in reporting
+
+ Args:
+ packagename: string containing name of package to build
+ environment: The VarDict for the test to run in
+ Returns:
+ a tuple containing the testcase name and the classname
+ (testcasename, classname)
+ testclassname: a descriptive string for the testcase can include whitespace
+ classname: should be patterned <packagename>.<plugin>.<optionally any unique condition>
+ """
+ return ("Confirm GUIDs are unique in " + packagename, packagename + ".GuidCheck")
+
+ def _FindConflictingGuidValues(self, guidlist: list) -> list:
+ """ Find all duplicate guids by guid value and report them as errors
+ """
+ # Sort the list by guid
+ guidsorted = sorted(
+ guidlist, key=lambda x: x.guid.upper(), reverse=True)
+
+ previous = None # Store previous entry for comparison
+ error = None
+ errors = []
+ for index in range(len(guidsorted)):
+ i = guidsorted[index]
+ if(previous is not None):
+ if i.guid == previous.guid: # Error
+ if(error is None):
+ # Catch errors with more than 1 conflict
+ error = ErrorEntry("guid")
+ error.entries.append(previous)
+ errors.append(error)
+ error.entries.append(i)
+ else:
+ # no match. clear error
+ error = None
+ previous = i
+ return errors
+
+ def _FindConflictingGuidNames(self, guidlist: list) -> list:
+ """ Find all duplicate guids by name and if they are not all
+ from inf files report them as errors. It is ok to have
+ BASE_NAME duplication.
+
+ Is this useful? It would catch two same named guids in dec file
+ that resolve to different values.
+ """
+ # Sort the list by guid
+ namesorted = sorted(guidlist, key=lambda x: x.name.upper())
+
+ previous = None # Store previous entry for comparison
+ error = None
+ errors = []
+ for index in range(len(namesorted)):
+ i = namesorted[index]
+ if(previous is not None):
+ # If name matches
+ if i.name == previous.name:
+ if(error is None):
+ # Catch errors with more than 1 conflict
+ error = ErrorEntry("name")
+ error.entries.append(previous)
+ errors.append(error)
+ error.entries.append(i)
+ else:
+ # no match. clear error
+ error = None
+ previous = i
+
+ # Loop thru and remove any errors where all files are infs as it is ok if
+ # they have the same inf base name.
+ for e in errors[:]:
+ if len( [en for en in e.entries if not en.absfilepath.lower().endswith(".inf")]) == 0:
+ errors.remove(e)
+
+ return errors
+
+ ##
+ # External function of plugin. This function is used to perform the task of the MuBuild Plugin
+ #
+ # - package is the edk2 path to package. This means workspace/packagepath relative.
+ # - edk2path object configured with workspace and packages path
+ # - PkgConfig Object (dict) for the pkg
+ # - EnvConfig Object
+ # - Plugin Manager Instance
+ # - Plugin Helper Obj Instance
+ # - Junit Logger
+ # - output_stream the StringIO output stream from this plugin via logging
+
+ def RunBuildPlugin(self, packagename, Edk2pathObj, pkgconfig, environment, PLM, PLMHelper, tc, output_stream=None):
+ Errors = []
+
+ abs_pkg_path = Edk2pathObj.GetAbsolutePathOnThisSytemFromEdk2RelativePath(
+ packagename)
+
+ if abs_pkg_path is None:
+ tc.SetSkipped()
+ tc.LogStdError("No package {0}".format(packagename))
+ return -1
+
+ All_Ignores = ["/Build", "/Conf"]
+ # Parse the config for other ignores
+ if "IgnoreFoldersAndFiles" in pkgconfig:
+ All_Ignores.extend(pkgconfig["IgnoreFoldersAndFiles"])
+
+ # Parse the workspace for all GUIDs
+ gs = GuidList.guidlist_from_filesystem(
+ Edk2pathObj.WorkspacePath, ignore_lines=All_Ignores)
+
+ # Remove ignored guidvalue
+ if "IgnoreGuidValue" in pkgconfig:
+ for a in pkgconfig["IgnoreGuidValue"]:
+ try:
+ tc.LogStdOut("Ignoring Guid {0}".format(a.upper()))
+ for b in gs[:]:
+ if b.guid == a.upper():
+ gs.remove(b)
+ except:
+ tc.LogStdError("GuidCheck.IgnoreGuid -> {0} not found. Invalid ignore guid".format(a.upper()))
+ logging.info("GuidCheck.IgnoreGuid -> {0} not found. Invalid ignore guid".format(a.upper()))
+
+ # Remove ignored guidname
+ if "IgnoreGuidName" in pkgconfig:
+ for a in pkgconfig["IgnoreGuidName"]:
+ entry = a.split("=")
+ if(len(entry) > 2):
+ tc.LogStdError("GuidCheck.IgnoreGuidName -> {0} Invalid Format.".format(a))
+ logging.info("GuidCheck.IgnoreGuidName -> {0} Invalid Format.".format(a))
+ continue
+ try:
+ tc.LogStdOut("Ignoring Guid {0}".format(a))
+ for b in gs[:]:
+ if b.name == entry[0]:
+ if(len(entry) == 1):
+ gs.remove(b)
+ elif(len(entry) == 2 and b.guid.upper() == entry[1].upper()):
+ gs.remove(b)
+ else:
+ c.LogStdError("GuidCheck.IgnoreGuidName -> {0} incomplete match. Invalid ignore guid".format(a))
+
+ except:
+ tc.LogStdError("GuidCheck.IgnoreGuidName -> {0} not found. Invalid ignore name".format(a))
+ logging.info("GuidCheck.IgnoreGuidName -> {0} not found. Invalid ignore name".format(a))
+
+ # Find conflicting Guid Values
+ Errors.extend(self._FindConflictingGuidValues(gs))
+
+ # Check if there are expected duplicates and remove it from the error list
+ if "IgnoreDuplicates" in pkgconfig:
+ for a in pkgconfig["IgnoreDuplicates"]:
+ names = a.split("=")
+ if len(names) < 2:
+ tc.LogStdError("GuidCheck.IgnoreDuplicates -> {0} invalid format".format(a))
+ logging.info("GuidCheck.IgnoreDuplicates -> {0} invalid format".format(a))
+ continue
+
+ for b in Errors[:]:
+ if b.type != "guid":
+ continue
+ ## Make a list of the names that are not in the names list. If there
+ ## are any in the list then this error should not be ignored.
+ t = [x for x in b.entries if x.name not in names]
+ if(len(t) == len(b.entries)):
+ ## did not apply to any entry
+ continue
+ elif(len(t) == 0):
+ ## full match - ignore duplicate
+ tc.LogStdOut("GuidCheck.IgnoreDuplicates -> {0}".format(a))
+ Errors.remove(b)
+ elif(len(t) < len(b.entries)):
+ ## partial match
+ tc.LogStdOut("GuidCheck.IgnoreDuplicates -> {0} incomplete match".format(a))
+ logging.info("GuidCheck.IgnoreDuplicates -> {0} incomplete match".format(a))
+ else:
+ tc.LogStdOut("GuidCheck.IgnoreDuplicates -> {0} unknown error.".format(a))
+ logging.info("GuidCheck.IgnoreDuplicates -> {0} unknown error".format(a))
+
+
+
+ # Find conflicting Guid Names
+ Errors.extend(self._FindConflictingGuidNames(gs))
+
+ # Log errors for anything within the package under test
+ for er in Errors[:]:
+ InMyPackage = False
+ for a in er.entries:
+ if abs_pkg_path in a.absfilepath:
+ InMyPackage = True
+ break
+ if(not InMyPackage):
+ Errors.remove(er)
+ else:
+ logging.error(str(er))
+ tc.LogStdError(str(er))
+
+ # add result to test case
+ overall_status = len(Errors)
+ if overall_status is not 0:
+ tc.SetFailed("GuidCheck {0} Failed. Errors {1}".format(
+ packagename, overall_status), "CHECK_FAILED")
+ else:
+ tc.SetSuccess()
+ return overall_status
+
+
+class ErrorEntry():
+ """ Custom/private class for reporting errors in the GuidList
+ """
+
+ def __init__(self, errortype):
+ self.type = errortype # 'guid' or 'name' depending on error type
+ self.entries = [] # GuidListEntry that are in error condition
+
+ def __str__(self):
+ a = f"Error Duplicate {self.type}: "
+ if(self.type == "guid"):
+ a += f" {self.entries[0].guid}"
+ elif(self.type == "name"):
+ a += f" {self.entries[0].name}"
+
+ a += f" ({len(self.entries)})\n"
+
+ for e in self.entries:
+ a += "\t" + str(e) + "\n"
+ return a
diff --git a/.pytool/Plugin/GuidCheck/GuidCheck_plug_in.yaml b/.pytool/Plugin/GuidCheck/GuidCheck_plug_in.yaml
new file mode 100644
index 0000000000..531efb7885
--- /dev/null
+++ b/.pytool/Plugin/GuidCheck/GuidCheck_plug_in.yaml
@@ -0,0 +1,11 @@
+##
+# CiBuildPlugin used to check guid uniqueness
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "scope": "cibuild",
+ "name": "Guid Check Test",
+ "module": "GuidCheck"
+}
diff --git a/.pytool/Plugin/GuidCheck/Readme.md b/.pytool/Plugin/GuidCheck/Readme.md
new file mode 100644
index 0000000000..c1bf3d728e
--- /dev/null
+++ b/.pytool/Plugin/GuidCheck/Readme.md
@@ -0,0 +1,60 @@
+# Guid Check Plugin
+
+This CiBuildPlugin scans all the files in a code tree to find all the GUID definitions. After collection it will then look for duplication in the package under test. Uniqueness of all GUIDs are critical within the UEFI environment. Duplication can cause numerous issues including locating the wrong data structure, calling the wrong function, or decoding the wrong data members.
+
+Currently Scanned:
+
+* INF files are scanned for there Module guid
+* DEC files are scanned for all of their Protocols, PPIs, and Guids as well as the one package GUID.
+
+Any GUID value being equal to two names or even just defined in two files is considered an error unless in the ignore list.
+
+Any GUID name that is found more than once is an error unless all occurrences are Module GUIDs. Since the Module GUID is assigned to the Module name it is common to have numerous versions of the same module named the same.
+
+## Configuration
+
+The plugin has numerous configuration options to support the UEFI codebase.
+
+``` yaml
+"GuidCheck": {
+ "IgnoreGuidName": [],
+ "IgnoreGuidValue": [],
+ "IgnoreFoldersAndFiles": [],
+ "IgnoreDuplicates": []
+ }
+```
+
+### IgnoreGuidName
+
+This list allows strings in two formats.
+
+* _GuidName_
+ * This will remove any entry with this GuidName from the list of GUIDs therefore ignoring any error associated with this name.
+* _GuidName=GuidValue_
+ * This will also ignore the GUID by name but only if the value equals the GuidValue.
+ * GuidValue should be in registry format.
+ * This is the suggested format to use as it will limit the ignore to only the defined case.
+
+### IgnoreGuidValue
+
+This list allows strings in guid registry format _GuidValue_.
+
+* This will remove any entry with this GuidValue from the list of GUIDs therefore ignoring any error associated with this value.
+* GuidValue must be in registry format xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
+
+### IgnoreFoldersAndFiles
+
+This supports .gitignore file and folder matching strings including wildcards
+
+* Any folder or file ignored will not be parsed and therefore any GUID defined will be ignored.
+* The plugin will always ignores the following ["/Build", "/Conf"]
+
+### IgnoreDuplicates
+
+This supports strings in the format of _GuidName_=_GuidName_=_GuidName_
+
+* For the error with the GuidNames to be ignored the list must match completely with what is found during the code scan.
+ * For example if there are two GUIDs that are by design equal within the code tree then it should be _GuidName_=_GuidName_
+ * If instead there are three GUIDs then it must be _GuidName_=_GuidName_=_GuidName_
+* This is the best ignore list to use because it is the most strict and will catch new problems when new conflicts are introduced.
+* There are numerous places in the UEFI specification in which two GUID names are assigned the same value. These names should be set in this ignore list so that they don't cause an error but any additional duplication would still be caught.
diff --git a/.pytool/Plugin/LibraryClassCheck/LibraryClassCheck.py b/.pytool/Plugin/LibraryClassCheck/LibraryClassCheck.py
new file mode 100644
index 0000000000..33745dff11
--- /dev/null
+++ b/.pytool/Plugin/LibraryClassCheck/LibraryClassCheck.py
@@ -0,0 +1,153 @@
+# @file LibraryClassCheck.py
+#
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+import logging
+import os
+from edk2toolext.environment.plugintypes.ci_build_plugin import ICiBuildPlugin
+from edk2toollib.uefi.edk2.parsers.dec_parser import DecParser
+from edk2toollib.uefi.edk2.parsers.inf_parser import InfParser
+from edk2toolext.environment.var_dict import VarDict
+
+
+class LibraryClassCheck(ICiBuildPlugin):
+ """
+ A CiBuildPlugin that scans the code tree and library classes for undeclared
+ files
+
+ Configuration options:
+ "LibraryClassCheck": {
+ IgnoreHeaderFile: [], # Ignore a file found on disk
+ IgnoreLibraryClass: [] # Ignore a declaration found in dec file
+ }
+ """
+
+ def GetTestName(self, packagename: str, environment: VarDict) -> tuple:
+ """ Provide the testcase name and classname for use in reporting
+ testclassname: a descriptive string for the testcase can include whitespace
+ classname: should be patterned <packagename>.<plugin>.<optionally any unique condition>
+
+ Args:
+ packagename: string containing name of package to build
+ environment: The VarDict for the test to run in
+ Returns:
+ a tuple containing the testcase name and the classname
+ (testcasename, classname)
+ """
+ return ("Check library class declarations in " + packagename, packagename + ".LibraryClassCheck")
+
+ def __GetPkgDec(self, rootpath):
+ try:
+ allEntries = os.listdir(rootpath)
+ for entry in allEntries:
+ if entry.lower().endswith(".dec"):
+ return(os.path.join(rootpath, entry))
+ except Exception:
+ logging.error("Unable to find DEC for package:{0}".format(rootpath))
+
+ return None
+
+ ##
+ # External function of plugin. This function is used to perform the task of the MuBuild Plugin
+ #
+ # - package is the edk2 path to package. This means workspace/packagepath relative.
+ # - edk2path object configured with workspace and packages path
+ # - PkgConfig Object (dict) for the pkg
+ # - EnvConfig Object
+ # - Plugin Manager Instance
+ # - Plugin Helper Obj Instance
+ # - Junit Logger
+ # - output_stream the StringIO output stream from this plugin via logging
+ def RunBuildPlugin(self, packagename, Edk2pathObj, pkgconfig, environment, PLM, PLMHelper, tc, output_stream=None):
+ overall_status = 0
+ LibraryClassIgnore = []
+
+ abs_pkg_path = Edk2pathObj.GetAbsolutePathOnThisSytemFromEdk2RelativePath(packagename)
+ abs_dec_path = self.__GetPkgDec(abs_pkg_path)
+ wsr_dec_path = Edk2pathObj.GetEdk2RelativePathFromAbsolutePath(abs_dec_path)
+
+ if abs_dec_path is None or wsr_dec_path is "" or not os.path.isfile(abs_dec_path):
+ tc.SetSkipped()
+ tc.LogStdError("No DEC file {0} in package {1}".format(abs_dec_path, abs_pkg_path))
+ return -1
+
+ # Get all include folders
+ dec = DecParser()
+ dec.SetBaseAbsPath(Edk2pathObj.WorkspacePath).SetPackagePaths(Edk2pathObj.PackagePathList)
+ dec.ParseFile(wsr_dec_path)
+
+ AllHeaderFiles = []
+
+ for includepath in dec.IncludePaths:
+ ## Get all header files in the library folder
+ AbsLibraryIncludePath = os.path.join(abs_pkg_path, includepath, "Library")
+ if(not os.path.isdir(AbsLibraryIncludePath)):
+ continue
+
+ hfiles = self.WalkDirectoryForExtension([".h"], AbsLibraryIncludePath)
+ hfiles = [os.path.relpath(x,abs_pkg_path) for x in hfiles] # make package root relative path
+ hfiles = [x.replace("\\", "/") for x in hfiles] # make package relative path
+
+ AllHeaderFiles.extend(hfiles)
+
+ if len(AllHeaderFiles) == 0:
+ tc.SetSkipped()
+ tc.LogStdError(f"No Library include folder in any Include path")
+ return -1
+
+ # Remove ignored paths
+ if "IgnoreHeaderFile" in pkgconfig:
+ for a in pkgconfig["IgnoreHeaderFile"]:
+ try:
+ tc.LogStdOut("Ignoring Library Header File {0}".format(a))
+ AllHeaderFiles.remove(a)
+ except:
+ tc.LogStdError("LibraryClassCheck.IgnoreHeaderFile -> {0} not found. Invalid Header File".format(a))
+ logging.info("LibraryClassCheck.IgnoreHeaderFile -> {0} not found. Invalid Header File".format(a))
+
+ if "IgnoreLibraryClass" in pkgconfig:
+ LibraryClassIgnore = pkgconfig["IgnoreLibraryClass"]
+
+
+ ## Attempt to find library classes
+ for lcd in dec.LibraryClasses:
+ ## Check for correct file path separator
+ if "\\" in lcd.path:
+ tc.LogStdError("LibraryClassCheck.DecFilePathSeparator -> {0} invalid.".format(lcd.path))
+ logging.error("LibraryClassCheck.DecFilePathSeparator -> {0} invalid.".format(lcd.path))
+ overall_status += 1
+ continue
+
+ if lcd.name in LibraryClassIgnore:
+ tc.LogStdOut("Ignoring Library Class Name {0}".format(lcd.name))
+ LibraryClassIgnore.remove(lcd.name)
+ continue
+
+ logging.debug(f"Looking for Library Class {lcd.path}")
+ try:
+ AllHeaderFiles.remove(lcd.path)
+
+ except ValueError:
+ tc.LogStdError(f"Library {lcd.name} with path {lcd.path} not found in package filesystem")
+ logging.error(f"Library {lcd.name} with path {lcd.path} not found in package filesystem")
+ overall_status += 1
+
+ ## any remaining AllHeaderFiles are not described in DEC
+ for h in AllHeaderFiles:
+ tc.LogStdError(f"Library Header File {h} not declared in package DEC {wsr_dec_path}")
+ logging.error(f"Library Header File {h} not declared in package DEC {wsr_dec_path}")
+ overall_status += 1
+
+ ## Warn about any invalid library class names in the ignore list
+ for r in LibraryClassIgnore:
+ tc.LogStdError("LibraryClassCheck.IgnoreLibraryClass -> {0} not found. Library Class not found".format(r))
+ logging.info("LibraryClassCheck.IgnoreLibraryClass -> {0} not found. Library Class not found".format(r))
+
+
+ # If XML object exists, add result
+ if overall_status is not 0:
+ tc.SetFailed("LibraryClassCheck {0} Failed. Errors {1}".format(wsr_dec_path, overall_status), "CHECK_FAILED")
+ else:
+ tc.SetSuccess()
+ return overall_status
diff --git a/.pytool/Plugin/LibraryClassCheck/LibraryClassCheck_plug_in.yaml b/.pytool/Plugin/LibraryClassCheck/LibraryClassCheck_plug_in.yaml
new file mode 100644
index 0000000000..9174453a86
--- /dev/null
+++ b/.pytool/Plugin/LibraryClassCheck/LibraryClassCheck_plug_in.yaml
@@ -0,0 +1,11 @@
+##
+# CiBuildPlugin used to check that all library classes are declared correctly in dec file
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "scope": "cibuild",
+ "name": "Library Class Check Test",
+ "module": "LibraryClassCheck"
+}
diff --git a/.pytool/Plugin/LibraryClassCheck/readme.md b/.pytool/Plugin/LibraryClassCheck/readme.md
new file mode 100644
index 0000000000..dedee16988
--- /dev/null
+++ b/.pytool/Plugin/LibraryClassCheck/readme.md
@@ -0,0 +1,22 @@
+# Library Class Check Plugin
+
+This CiBuildPlugin scans at all library header files found in the `Library` folders in all of the package's declared include directories and ensures that all files have a matching LibraryClass declaration in the DEC file for the package. Any missing declarations will cause a failure.
+
+## Configuration
+
+The plugin has a few configuration options to support the UEFI codebase.
+
+``` yaml
+"LibraryClassCheck": {
+ IgnoreHeaderFile: [], # Ignore a file found on disk
+ IgnoreLibraryClass: [] # Ignore a declaration found in dec file
+}
+```
+
+### IgnoreHeaderFile
+
+Ignore a file found on disk
+
+### IgnoreLibraryClass
+
+Ignore a declaration found in dec file
diff --git a/.pytool/Plugin/SpellCheck/Readme.md b/.pytool/Plugin/SpellCheck/Readme.md
new file mode 100644
index 0000000000..e0ac835191
--- /dev/null
+++ b/.pytool/Plugin/SpellCheck/Readme.md
@@ -0,0 +1,100 @@
+# Spell Check Plugin
+
+This CiBuildPlugin scans all the files in a given package and checks for spelling errors.
+
+This plugin requires NodeJs and cspell. If the plugin doesn't find its required tools then it will mark the test as skipped.
+
+* NodeJS: https://nodejs.org/en/
+* cspell: https://www.npmjs.com/package/cspell
+ * Src and doc available: https://github.com/streetsidesoftware/cspell
+
+## Configuration
+
+The plugin has a few configuration options to support the UEFI codebase.
+
+``` yaml
+ "SpellCheck": {
+ "AuditOnly": False, # If True, log all errors and then mark as skipped
+ "IgnoreFiles": [], # use gitignore syntax to ignore errors in matching files
+ "ExtendWords": [], # words to extend to the dictionary for this package
+ "IgnoreStandardPaths": [], # Standard Plugin defined paths that should be ignore
+ "AdditionalIncludePaths": [] # Additional paths to spell check (wildcards supported)
+ }
+```
+
+### AuditOnly
+
+Boolean - Default is False.
+If True run the test in an Audit only mode which will log all errors but instead of failing the build it will set the test as skipped. This allows visibility into the failures without breaking the build.
+
+### IgnoreFiles
+
+This supports .gitignore file and folder matching strings including wildcards
+
+* All files will be parsed regardless but then any spelling errors found within ignored files will not be reported as an error.
+* Errors in ignored files will still be output to the test results as informational comments.
+
+### ExtendWords
+
+This list allows words to be added to the dictionary for the spell checker when this package is tested. These follow the rules of the cspell config words field.
+
+### IgnoreStandardPaths
+
+This plugin by default will check the below standard paths. If the package would like to ignore any of them list that here.
+
+```python
+[
+# C source
+"*.c",
+"*.h",
+
+# Assembly files
+"*.nasm",
+"*.asm",
+"*.masm",
+"*.s",
+
+# ACPI source language
+"*.asl",
+
+# Edk2 build files
+"*.dsc", "*.dec", "*.fdf", "*.inf",
+
+# Documentation files
+"*.md", "*.txt"
+]
+```
+
+### AdditionalIncludePaths
+
+If the package would to add additional path patterns to be included in spellchecking they can be defined here.
+
+## Other configuration
+
+In the cspell.base.json there are numerous other settings configured. There is no support to override these on a per package basis but future features could make this available. One interesting configuration option is `minWordLength`. Currently it is set to _5_ which means all 2,3, and 4 letter words will be ignored. This helps minimize the number of technical acronyms, register names, and other UEFI specific values that must be ignored .
+
+## False positives
+
+The cspell dictionary is not perfect and there are cases where technical words or acronyms are not found in the dictionary. There are three ways to resolve false positives and the choice for which method should be based on how broadly the word should be accepted.
+
+### CSpell Base Config file
+
+If the change should apply to all UEFI code and documentation then it should be added to the base config file `words` section. The base config file is adjacent to this file and titled `cspell.base.json`. This is a list of accepted words for all spell checking operations on all packages.
+
+### Package Config
+
+In the package `*.ci.yaml` file there is a `SpellCheck` config section. This section allows files to be ignored as well as words that should be considered valid for all files within this package. Add the desired words to the "ExtendedWords" member.
+
+### In-line File
+
+CSpell supports numerous methods to annotate your files to ignore words, sections, etc. This can be found in CSpell documentation. Suggestion here is to use a c-style comment at the top of the file to add words that should be ignored just for this file. Obviously this has the highest maintenance cost so it should only be used for file unique words.
+
+``` c
+// spell-checker:ignore unenroll, word2, word3
+```
+
+or
+
+```ini
+# spell-checker:ignore unenroll, word2, word3
+```
diff --git a/.pytool/Plugin/SpellCheck/SpellCheck.py b/.pytool/Plugin/SpellCheck/SpellCheck.py
new file mode 100644
index 0000000000..94ca4cd071
--- /dev/null
+++ b/.pytool/Plugin/SpellCheck/SpellCheck.py
@@ -0,0 +1,216 @@
+# @file SpellCheck.py
+#
+# An edk2-pytool based plugin wrapper for cspell
+#
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+import logging
+import json
+import yaml
+from io import StringIO
+import os
+from edk2toolext.environment.plugintypes.ci_build_plugin import ICiBuildPlugin
+from edk2toollib.utility_functions import RunCmd
+from edk2toolext.environment.var_dict import VarDict
+from edk2toollib.gitignore_parser import parse_gitignore_lines
+from edk2toolext.environment import version_aggregator
+
+
+class SpellCheck(ICiBuildPlugin):
+ """
+ A CiBuildPlugin that uses the cspell node module to scan the files
+ from the package being tested for spelling errors. The plugin contains
+ the base cspell.json file then thru the configuration options other settings
+ can be changed or extended.
+
+ Configuration options:
+ "SpellCheck": {
+ "AuditOnly": False, # Don't fail the build if there are errors. Just log them
+ "IgnoreFiles": [], # use gitignore syntax to ignore errors in matching files
+ "ExtendWords": [], # words to extend to the dictionary for this package
+ "IgnoreStandardPaths": [], # Standard Plugin defined paths that should be ignore
+ "AdditionalIncludePaths": [] # Additional paths to spell check (wildcards supported)
+ }
+ """
+
+ #
+ # A package can remove any of these using IgnoreStandardPaths
+ #
+ STANDARD_PLUGIN_DEFINED_PATHS = ["*.c", "*.h",
+ "*.nasm", "*.asm", "*.masm", "*.s",
+ "*.asl",
+ "*.dsc", "*.dec", "*.fdf", "*.inf",
+ "*.md", "*.txt"
+ ]
+
+ def GetTestName(self, packagename: str, environment: VarDict) -> tuple:
+ """ Provide the testcase name and classname for use in reporting
+
+ Args:
+ packagename: string containing name of package to build
+ environment: The VarDict for the test to run in
+ Returns:
+ a tuple containing the testcase name and the classname
+ (testcasename, classname)
+ testclassname: a descriptive string for the testcase can include whitespace
+ classname: should be patterned <packagename>.<plugin>.<optionally any unique condition>
+ """
+ return ("Spell check files in " + packagename, packagename + ".SpellCheck")
+
+ ##
+ # External function of plugin. This function is used to perform the task of the CiBuild Plugin
+ #
+ # - package is the edk2 path to package. This means workspace/packagepath relative.
+ # - edk2path object configured with workspace and packages path
+ # - PkgConfig Object (dict) for the pkg
+ # - EnvConfig Object
+ # - Plugin Manager Instance
+ # - Plugin Helper Obj Instance
+ # - Junit Logger
+ # - output_stream the StringIO output stream from this plugin via logging
+
+ def RunBuildPlugin(self, packagename, Edk2pathObj, pkgconfig, environment, PLM, PLMHelper, tc, output_stream=None):
+ Errors = []
+
+ abs_pkg_path = Edk2pathObj.GetAbsolutePathOnThisSytemFromEdk2RelativePath(
+ packagename)
+
+ if abs_pkg_path is None:
+ tc.SetSkipped()
+ tc.LogStdError("No package {0}".format(packagename))
+ return -1
+
+ # check for node
+ return_buffer = StringIO()
+ ret = RunCmd("node", "--version", outstream=return_buffer)
+ if (ret != 0):
+ tc.SetSkipped()
+ tc.LogStdError("NodeJs not installed. Test can't run")
+ logging.warning("NodeJs not installed. Test can't run")
+ return -1
+ node_version = return_buffer.getvalue().strip() # format vXX.XX.XX
+ tc.LogStdOut(f"Node version: {node_version}")
+ version_aggregator.GetVersionAggregator().ReportVersion(
+ "NodeJs", node_version, version_aggregator.VersionTypes.INFO)
+
+ # Check for cspell
+ return_buffer = StringIO()
+ ret = RunCmd("cspell", "--version", outstream=return_buffer)
+ if (ret != 0):
+ tc.SetSkipped()
+ tc.LogStdError("cspell not installed. Test can't run")
+ logging.warning("cspell not installed. Test can't run")
+ return -1
+ cspell_version = return_buffer.getvalue().strip() # format XX.XX.XX
+ tc.LogStdOut(f"CSpell version: {cspell_version}")
+ version_aggregator.GetVersionAggregator().ReportVersion(
+ "CSpell", cspell_version, version_aggregator.VersionTypes.INFO)
+
+ package_relative_paths_to_spell_check = SpellCheck.STANDARD_PLUGIN_DEFINED_PATHS
+
+ #
+ # Allow the ci.yaml to remove any of the above standard paths
+ #
+ if("IgnoreStandardPaths" in pkgconfig):
+ for a in pkgconfig["IgnoreStandardPaths"]:
+ if(a in package_relative_paths_to_spell_check):
+ tc.LogStdOut(
+ f"ignoring standard path due to ci.yaml ignore: {a}")
+ package_relative_paths_to_spell_check.remove(a)
+ else:
+ tc.LogStdOut(f"Invalid IgnoreStandardPaths value: {a}")
+
+ #
+ # check for any additional include paths defined by package config
+ #
+ if("AdditionalIncludePaths" in pkgconfig):
+ package_relative_paths_to_spell_check.extend(
+ pkgconfig["AdditionalIncludePaths"])
+
+ #
+ # Make the path string for cspell to check
+ #
+ relpath = os.path.relpath(abs_pkg_path)
+ cpsell_paths = " ".join(
+ [f"{relpath}/**/{x}" for x in package_relative_paths_to_spell_check])
+
+ # Make the config file
+ config_file_path = os.path.join(
+ Edk2pathObj.WorkspacePath, "Build", packagename, "cspell_actual_config.json")
+ mydir = os.path.dirname(os.path.abspath(__file__))
+ # load as yaml so it can have comments
+ base = os.path.join(mydir, "cspell.base.yaml")
+ with open(base, "r") as i:
+ config = yaml.safe_load(i)
+
+ if("ExtendWords" in pkgconfig):
+ config["words"].extend(pkgconfig["ExtendWords"])
+ with open(config_file_path, "w") as o:
+ json.dump(config, o) # output as json so compat with cspell
+
+ All_Ignores = []
+ # Parse the config for other ignores
+ if "IgnoreFiles" in pkgconfig:
+ All_Ignores.extend(pkgconfig["IgnoreFiles"])
+
+ # spell check all the files
+ ignore = parse_gitignore_lines(All_Ignores, os.path.join(
+ abs_pkg_path, "nofile.txt"), abs_pkg_path)
+
+ # result is a list of strings like this
+ # C:\src\sp-edk2\edk2\FmpDevicePkg\FmpDevicePkg.dec:53:9 - Unknown word (Capule)
+ EasyFix = []
+ results = self._check_spelling(cpsell_paths, config_file_path)
+ for r in results:
+ path, _, word = r.partition(" - Unknown word ")
+ if len(word) == 0:
+ # didn't find pattern
+ continue
+
+ pathinfo = path.rsplit(":", 2) # remove the line no info
+ if(ignore(pathinfo[0])): # check against ignore list
+ tc.LogStdOut(f"ignoring error due to ci.yaml ignore: {r}")
+ continue
+
+ # real error
+ EasyFix.append(word.strip().strip("()"))
+ Errors.append(r)
+
+ # Log all errors tc StdError
+ for l in Errors:
+ tc.LogStdError(l.strip())
+
+ # Helper - Log the syntax needed to add these words to dictionary
+ if len(EasyFix) > 0:
+ EasyFix = sorted(set(a.lower() for a in EasyFix))
+ tc.LogStdOut("\n Easy fix:")
+ OneString = "If these are not errors add this to your ci.yaml file.\n"
+ OneString += '"SpellCheck": {\n "ExtendWords": ['
+ for a in EasyFix:
+ tc.LogStdOut(f'\n"{a}",')
+ OneString += f'\n "{a}",'
+ logging.info(OneString.rstrip(",") + '\n ]\n}')
+
+ # add result to test case
+ overall_status = len(Errors)
+ if overall_status != 0:
+ if "AuditOnly" in pkgconfig and pkgconfig["AuditOnly"]:
+ # set as skipped if AuditOnly
+ tc.SetSkipped()
+ return -1
+ else:
+ tc.SetFailed("SpellCheck {0} Failed. Errors {1}".format(
+ packagename, overall_status), "CHECK_FAILED")
+ else:
+ tc.SetSuccess()
+ return overall_status
+
+ def _check_spelling(self, abs_file_to_check: str, abs_config_file_to_use: str) -> []:
+ output = StringIO()
+ ret = RunCmd(
+ "cspell", f"--config {abs_config_file_to_use} {abs_file_to_check}", outstream=output)
+ if ret == 0:
+ return []
+ else:
+ return output.getvalue().strip().splitlines()
diff --git a/.pytool/Plugin/SpellCheck/SpellCheck_plug_in.yaml b/.pytool/Plugin/SpellCheck/SpellCheck_plug_in.yaml
new file mode 100644
index 0000000000..161045e19e
--- /dev/null
+++ b/.pytool/Plugin/SpellCheck/SpellCheck_plug_in.yaml
@@ -0,0 +1,11 @@
+##
+# CiBuildPlugin used to check spelling
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "scope": "cibuild",
+ "name": "Spell Check Test",
+ "module": "SpellCheck"
+}
diff --git a/.pytool/Plugin/SpellCheck/cspell.base.yaml b/.pytool/Plugin/SpellCheck/cspell.base.yaml
new file mode 100644
index 0000000000..da6c5e5da7
--- /dev/null
+++ b/.pytool/Plugin/SpellCheck/cspell.base.yaml
@@ -0,0 +1,165 @@
+##
+# CSpell configuration
+#
+# Copyright (c) Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "version": "0.1",
+ "language": "en",
+ "dictionaries": [
+ "companies ",
+ "softwareTerms",
+ "python",
+ "cpp"
+ ],
+ "ignorePaths": [
+ "*.pdb",
+ "**/*_extdep/**",
+ "*.pdf",
+ "*.exe",
+ "*.jpg"
+ ],
+ "minWordLength": 5,
+ "allowCompoundWords": false,
+ "ignoreWords": [
+ "muchange"
+ ],
+ "words": [
+ "MTRRs",
+ "Microarchitecture",
+ "Goldmont",
+ "cpuid",
+ "mwait",
+ "cstate",
+ "smram",
+ "scrtm",
+ "smbus",
+ "selftest",
+ "socket",
+ "MMRAM",
+ "qword",
+ "ENDBR",
+ "SMBASE",
+ "FXSAVE",
+ "FXRSTOR",
+ "RDRAND",
+ "IOAPIC",
+ "ATAPI",
+ "movsb",
+ "iretw",
+ "XENSTORE",
+ "cdrom",
+ "oprom",
+ "oproms",
+ "varstore",
+ "EKU",
+ "ascii",
+ "nmake",
+ "NVDIMM",
+ "nasmb",
+ "Mtftp",
+ "Hypercall",
+ "hypercalls",
+ "IOMMU",
+ "QEMU",
+ "qemus",
+ "OVMF",
+ "tiano",
+ "tianocore",
+ "edkii",
+ "coreboot",
+ "uefipayload",
+ "bootloader",
+ "bootloaders",
+ "mdepkg",
+ "skuid",
+ "dxefv",
+ "toolchain",
+ "libraryclass",
+ "preboot",
+ "pythonpath",
+ "cygpath",
+ "nuget",
+ "basetools",
+ "prepi",
+ "OPTEE",
+ "stringid",
+ "peims",
+ "memmap",
+ "guids",
+ "uuids",
+ "smbios",
+ "certdb",
+ "certdbv",
+ "EfiSigList",
+ "depex",
+ "IHANDLE",
+ "Virtio",
+ "Mbytes",
+ "Citrix",
+ "initrd",
+ "semihost",
+ "Semihosting",
+ "Trustzone",
+ "Fastboot",
+ "framebuffer",
+ "genfw",
+ "TTYTERM",
+ "miniport",
+ "LFENCE",
+ "PCANSI",
+ "submodule",
+ "submodules",
+ "brotli",
+ "PCCTS",
+ "softfloat",
+ "whitepaper",
+ "ACPICA",
+ "plugfest",
+ "bringup",
+ "formset", #VFR
+ "ideqvallist",
+ "numberof",
+ "oneof",
+ "endformset",
+ "endnumeric",
+ "endoneof",
+ "disableif",
+ "guidid",
+ "classguid",
+ "efivarstore",
+ "formsetguid",
+ "formid",
+ "suppressif",
+ "grayoutif",
+ "ideqval",
+ "endform",
+ "endcheckbox",
+ "questionid",
+ "questionref",
+ "enddate",
+ "endstring",
+ "guidop",
+ "endguidop",
+ "langdef",
+ "dynamicex",
+ "tokenspace",
+ "tokenguid",
+ "pcd's", #seems like cspell bug
+ "peim's",
+ "autogen",
+ "Disasm",
+ "Torito",
+ "SRIOV",
+ "MRIOV",
+ "UARTs",
+ "Consplitter", # common module in UEFI
+ "FIFOs",
+ "ACPINVS",
+ "Endof", # due to of not being uppercase
+ "bootability",
+ "Sdhci",
+ "inmodule",
+ ]
+}
--
2.21.0.windows.1
^ permalink raw reply related [flat|nested] 51+ messages in thread
* [Patch v4 08/22] CryptoPkg: Add YAML file for CI builds
2019-11-07 1:13 [Patch v4 00/22] Enable Phase 1 of EDK II CI Michael D Kinney
` (6 preceding siblings ...)
2019-11-07 1:13 ` [Patch v4 07/22] .pytool/Plugin: Add CI plugins Michael D Kinney
@ 2019-11-07 1:13 ` Michael D Kinney
2019-11-07 5:06 ` Wang, Jian J
2019-11-07 1:13 ` [Patch v4 09/22] FatPkg: " Michael D Kinney
` (14 subsequent siblings)
22 siblings, 1 reply; 51+ messages in thread
From: Michael D Kinney @ 2019-11-07 1:13 UTC (permalink / raw)
To: devel; +Cc: Jian J Wang, Xiaoyu Lu, Kinney
Add YAML file to the package directory with the
configuration of the checks to perform during a
CI build.
Cc: Jian J Wang <jian.j.wang@intel.com>
Cc: Xiaoyu Lu <xiaoyux.lu@intel.com>
Signed-off-by: Kinney <michael.d.kinney@intel.com>
---
CryptoPkg/CryptoPkg.ci.yaml | 48 +++++++++++++++++++++++++++++++++++++
1 file changed, 48 insertions(+)
create mode 100644 CryptoPkg/CryptoPkg.ci.yaml
diff --git a/CryptoPkg/CryptoPkg.ci.yaml b/CryptoPkg/CryptoPkg.ci.yaml
new file mode 100644
index 0000000000..a20ac79d0e
--- /dev/null
+++ b/CryptoPkg/CryptoPkg.ci.yaml
@@ -0,0 +1,48 @@
+##
+# CI configuration for CryptoPkg
+#
+# Copyright (c) Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "CompilerPlugin": {
+ "DscPath": "CryptoPkg.dsc"
+ },
+ "CharEncodingCheck": {
+ "IgnoreFiles": []
+ },
+ "DependencyCheck": {
+ "AcceptableDependencies": [
+ "MdePkg/MdePkg.dec",
+ "MdeModulePkg/MdeModulePkg.dec",
+ "CryptoPkg/CryptoPkg.dec",
+ #"SecurityPkg/SecurityPkg.dec"
+ ],
+ # For host based unit tests
+ "AcceptableDependencies-HOST_APPLICATION":[],
+ # For UEFI shell based apps
+ "AcceptableDependencies-UEFI_APPLICATION":[],
+ "IgnoreInf": []
+ },
+ "DscCompleteCheck": {
+ "DscPath": "CryptoPkg.dsc",
+ "IgnoreInf": []
+ },
+ "GuidCheck": {
+ "IgnoreGuidName": [],
+ "IgnoreGuidValue": [],
+ "IgnoreFoldersAndFiles": []
+ },
+ "LibraryClassCheck": {
+ "IgnoreHeaderFile": []
+ },
+
+ ## options defined ci/Plugin/SpellCheck
+ "SpellCheck": {
+ "skip": True,
+ "IgnoreFiles": [], # use gitignore syntax to ignore errors in matching files
+ "ExtendWords": [], # words to extend to the dictionary for this package
+ "IgnoreStandardPaths": [], # Standard Plugin defined paths that should be ignore
+ "AdditionalIncludePaths": [] # Additional paths to spell check (wildcards supported)
+ }
+}
--
2.21.0.windows.1
^ permalink raw reply related [flat|nested] 51+ messages in thread
* [Patch v4 09/22] FatPkg: Add YAML file for CI builds
2019-11-07 1:13 [Patch v4 00/22] Enable Phase 1 of EDK II CI Michael D Kinney
` (7 preceding siblings ...)
2019-11-07 1:13 ` [Patch v4 08/22] CryptoPkg: Add YAML file for CI builds Michael D Kinney
@ 2019-11-07 1:13 ` Michael D Kinney
2019-11-07 2:12 ` Ni, Ray
2019-11-07 1:13 ` [Patch v4 10/22] FmpDevicePkg: " Michael D Kinney
` (13 subsequent siblings)
22 siblings, 1 reply; 51+ messages in thread
From: Michael D Kinney @ 2019-11-07 1:13 UTC (permalink / raw)
To: devel; +Cc: Ray Ni
Add YAML file to the package directory with the
configuration of the checks to perform during a
CI build.
Cc: Ray Ni <ray.ni@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
---
FatPkg/FatPkg.ci.yaml | 50 +++++++++++++++++++++++++++++++++++++++++++
1 file changed, 50 insertions(+)
create mode 100644 FatPkg/FatPkg.ci.yaml
diff --git a/FatPkg/FatPkg.ci.yaml b/FatPkg/FatPkg.ci.yaml
new file mode 100644
index 0000000000..d584eea0a7
--- /dev/null
+++ b/FatPkg/FatPkg.ci.yaml
@@ -0,0 +1,50 @@
+##
+# CI configuration for FatPkg
+#
+# Copyright (c) Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "CompilerPlugin": {
+ "DscPath": "FatPkg.dsc"
+ },
+ "CharEncodingCheck": {
+ "IgnoreFiles": []
+ },
+ "DependencyCheck": {
+ "AcceptableDependencies": [
+ "MdePkg/MdePkg.dec",
+ "MdeModulePkg/MdeModulePkg.dec",
+ ],
+ # For host based unit tests
+ "AcceptableDependencies-HOST_APPLICATION":[],
+ # For UEFI shell based apps
+ "AcceptableDependencies-UEFI_APPLICATION":[],
+ "IgnoreInf": []
+ },
+ "DscCompleteCheck": {
+ "IgnoreInf": [],
+ "DscPath": "FatPkg.dsc"
+ },
+ "GuidCheck": {
+ "IgnoreGuidName": [],
+ "IgnoreGuidValue": [],
+ "IgnoreFoldersAndFiles": []
+ },
+ "LibraryClassCheck": {
+ "IgnoreHeaderFile": []
+ },
+ "SpellCheck": {
+ "ExtendWords": [
+ "ELTORITO",
+ "FHAND",
+ "IFILE",
+ "OFILE",
+ "FDISKed",
+ "Lfnbuffer",
+ "FFFFFFFFL",
+ "CDVOL",
+ "DMDEPKG"
+ ]
+ }
+}
--
2.21.0.windows.1
^ permalink raw reply related [flat|nested] 51+ messages in thread
* [Patch v4 10/22] FmpDevicePkg: Add YAML file for CI builds
2019-11-07 1:13 [Patch v4 00/22] Enable Phase 1 of EDK II CI Michael D Kinney
` (8 preceding siblings ...)
2019-11-07 1:13 ` [Patch v4 09/22] FatPkg: " Michael D Kinney
@ 2019-11-07 1:13 ` Michael D Kinney
2019-11-07 1:13 ` [Patch v4 11/22] MdeModulePkg: " Michael D Kinney
` (12 subsequent siblings)
22 siblings, 0 replies; 51+ messages in thread
From: Michael D Kinney @ 2019-11-07 1:13 UTC (permalink / raw)
To: devel; +Cc: Liming Gao
Add YAML file to the package directory with the
configuration of the checks to perform during a
CI build.
Use BaseCryptLibNull for package CI builds to reduce package
build times. Enabled with CONTINUOUS_INTEGRATION in YAML
files. By default CONTINUOUS_INTEGRATION is not defined,
and the original lib mappings are preserved.
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
---
FmpDevicePkg/FmpDevicePkg.ci.yaml | 43 +++++++++++++++++++++++++++++++
FmpDevicePkg/FmpDevicePkg.dsc | 6 ++++-
2 files changed, 48 insertions(+), 1 deletion(-)
create mode 100644 FmpDevicePkg/FmpDevicePkg.ci.yaml
diff --git a/FmpDevicePkg/FmpDevicePkg.ci.yaml b/FmpDevicePkg/FmpDevicePkg.ci.yaml
new file mode 100644
index 0000000000..f089932556
--- /dev/null
+++ b/FmpDevicePkg/FmpDevicePkg.ci.yaml
@@ -0,0 +1,43 @@
+##
+# CI configuration for FmpDevicePkg
+#
+# Copyright (c) Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "CompilerPlugin": {
+ "DscPath": "FmpDevicePkg.dsc"
+ },
+ "CharEncodingCheck": {
+ "IgnoreFiles": []
+ },
+ "DependencyCheck": {
+ "AcceptableDependencies": [
+ "MdePkg/MdePkg.dec",
+ "MdeModulePkg/MdeModulePkg.dec",
+ "FmpDevicePkg/FmpDevicePkg.dec",
+ "CryptoPkg/CryptoPkg.dec"
+ ],
+ "IgnoreInf": []
+ },
+ "DscCompleteCheck": {
+ "DscPath": "FmpDevicePkg.dsc",
+ "IgnoreInf": []
+ },
+ "GuidCheck": {
+ "IgnoreGuidName": [],
+ "IgnoreGuidValue": [],
+ "IgnoreFoldersAndFiles": []
+ },
+ "LibraryClassCheck": {
+ "IgnoreHeaderFile": []
+ },
+ "SpellCheck": {
+ "ExtendWords": [
+ "FMPSTATE",
+ ]
+ },
+ "Defines": {
+ "BLD_*_CONTINUOUS_INTEGRATION": "TRUE",
+ }
+}
diff --git a/FmpDevicePkg/FmpDevicePkg.dsc b/FmpDevicePkg/FmpDevicePkg.dsc
index 67b5aa7a32..f4093d3837 100644
--- a/FmpDevicePkg/FmpDevicePkg.dsc
+++ b/FmpDevicePkg/FmpDevicePkg.dsc
@@ -48,9 +48,13 @@ [LibraryClasses]
DebugLib|MdePkg/Library/UefiDebugLibStdErr/UefiDebugLibStdErr.inf
DebugPrintErrorLevelLib|MdePkg/Library/BaseDebugPrintErrorLevelLib/BaseDebugPrintErrorLevelLib.inf
PcdLib|MdePkg/Library/BasePcdLibNull/BasePcdLibNull.inf
- OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
+!ifdef CONTINUOUS_INTEGRATION
+ BaseCryptLib|CryptoPkg/Library/BaseCryptLibNull/BaseCryptLibNull.inf
+!else
IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf
+ OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
BaseCryptLib|CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf
+!endif
FmpAuthenticationLib|SecurityPkg/Library/FmpAuthenticationLibPkcs7/FmpAuthenticationLibPkcs7.inf
CapsuleUpdatePolicyLib|FmpDevicePkg/Library/CapsuleUpdatePolicyLibNull/CapsuleUpdatePolicyLibNull.inf
FmpPayloadHeaderLib|FmpDevicePkg/Library/FmpPayloadHeaderLibV1/FmpPayloadHeaderLibV1.inf
--
2.21.0.windows.1
^ permalink raw reply related [flat|nested] 51+ messages in thread
* [Patch v4 11/22] MdeModulePkg: Add YAML file for CI builds
2019-11-07 1:13 [Patch v4 00/22] Enable Phase 1 of EDK II CI Michael D Kinney
` (9 preceding siblings ...)
2019-11-07 1:13 ` [Patch v4 10/22] FmpDevicePkg: " Michael D Kinney
@ 2019-11-07 1:13 ` Michael D Kinney
2019-11-07 3:03 ` Wu, Hao A
2019-11-07 1:13 ` [Patch v4 12/22] MdePkg: " Michael D Kinney
` (11 subsequent siblings)
22 siblings, 1 reply; 51+ messages in thread
From: Michael D Kinney @ 2019-11-07 1:13 UTC (permalink / raw)
To: devel; +Cc: Jian J Wang, Hao A Wu
Add YAML file to the package directory with the
configuration of the checks to perform during a
CI build.
Cc: Jian J Wang <jian.j.wang@intel.com>
Cc: Hao A Wu <hao.a.wu@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
---
MdeModulePkg/MdeModulePkg.ci.yaml | 81 +++++++++++++++++++++++++++++++
1 file changed, 81 insertions(+)
create mode 100644 MdeModulePkg/MdeModulePkg.ci.yaml
diff --git a/MdeModulePkg/MdeModulePkg.ci.yaml b/MdeModulePkg/MdeModulePkg.ci.yaml
new file mode 100644
index 0000000000..6ae58d5379
--- /dev/null
+++ b/MdeModulePkg/MdeModulePkg.ci.yaml
@@ -0,0 +1,81 @@
+##
+# CI configuration for MdeModulePkg
+#
+# Copyright (c) Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ ## options defined ci/Plugin/CompilerPlugin
+ "CompilerPlugin": {
+ "DscPath": "MdeModulePkg.dsc"
+ },
+
+ ## options defined ci/Plugin/CharEncodingCheck
+ "CharEncodingCheck": {
+ "IgnoreFiles": []
+ },
+
+ ## options defined ci/Plugin/DependencyCheck
+ "DependencyCheck": {
+ "AcceptableDependencies": [
+ "MdePkg/MdePkg.dec",
+ "MdeModulePkg/MdeModulePkg.dec",
+ "StandaloneMmPkg/StandaloneMmPkg.dec",
+ "ArmPkg/ArmPkg.dec" # this should be fixed by promoting an abstraction
+ ],
+ # For host based unit tests
+ "AcceptableDependencies-HOST_APPLICATION":[],
+ # For UEFI shell based apps
+ "AcceptableDependencies-UEFI_APPLICATION":[],
+ "IgnoreInf": []
+ },
+
+ ## options defined ci/Plugin/DscCompleteCheck
+ "DscCompleteCheck": {
+ "IgnoreInf": [],
+ "DscPath": "MdeModulePkg.dsc"
+ },
+
+ ## options defined ci/Plugin/GuidCheck
+ "GuidCheck": {
+ "IgnoreGuidName": [],
+ "IgnoreGuidValue": ["00000000-0000-0000-0000-000000000000"],
+ "IgnoreFoldersAndFiles": [],
+ "IgnoreDuplicates": [
+ "gEdkiiFormBrowserExProtocolGuid=gEfiFormBrowserExProtocolGuid",
+ "gEfiPeiMmAccessPpiGuid=gPeiSmmAccessPpiGuid",
+ "gPeiSmmControlPpiGuid=gEfiPeiMmControlPpiGuid",
+ "gEdkiiSerialPortLibVendorGuid=SerialDxe" # Is this a bug????
+ ]
+ },
+
+ ## options defined ci/Plugin/LibraryClassCheck
+ "LibraryClassCheck": {
+ "IgnoreHeaderFile": []
+ },
+
+ ## options defined ci/Plugin/SpellCheck
+ "SpellCheck": {
+ "AuditOnly": True, # Fails test but run in AuditOnly mode to collect log
+ "IgnoreStandardPaths": [ # Standard Plugin defined paths that should be ignore
+ "*.c", "*.asm", "*.h", "*.nasm", "*.s", "*.asl", "*.inf"
+ ],
+ "IgnoreFiles": [ # use gitignore syntax to ignore errors in matching files
+ "Library/LzmaCustomDecompressLib/Sdk/DOC/*"
+ ],
+ "ExtendWords": [ # words to extend to the dictionary for this package
+ "LIGHTGRAY",
+ "DARKGRAY",
+ "LIGHTBLUE",
+ "LIGHTGREEN",
+ "LIGHTCYAN",
+ "LIGHTRED",
+ "LIGHTMAGENTA",
+ "FVMAIN",
+ "VARCHECKPCD",
+ "Getxx",
+ "lzturbo"
+ ],
+ "AdditionalIncludePaths": [] # Additional paths to spell check relative to package root (wildcards supported)
+ }
+}
--
2.21.0.windows.1
^ permalink raw reply related [flat|nested] 51+ messages in thread
* [Patch v4 12/22] MdePkg: Add YAML file for CI builds
2019-11-07 1:13 [Patch v4 00/22] Enable Phase 1 of EDK II CI Michael D Kinney
` (10 preceding siblings ...)
2019-11-07 1:13 ` [Patch v4 11/22] MdeModulePkg: " Michael D Kinney
@ 2019-11-07 1:13 ` Michael D Kinney
2019-11-07 1:13 ` [Patch v4 13/22] NetworkPkg: " Michael D Kinney
` (10 subsequent siblings)
22 siblings, 0 replies; 51+ messages in thread
From: Michael D Kinney @ 2019-11-07 1:13 UTC (permalink / raw)
To: devel; +Cc: Liming Gao
Add YAML file to the package directory with the
configuration of the checks to perform during a
CI build.
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
---
MdePkg/MdePkg.ci.yaml | 90 +++++++++++++++++++++++++++++++++++++++++++
1 file changed, 90 insertions(+)
create mode 100644 MdePkg/MdePkg.ci.yaml
diff --git a/MdePkg/MdePkg.ci.yaml b/MdePkg/MdePkg.ci.yaml
new file mode 100644
index 0000000000..4015c07f16
--- /dev/null
+++ b/MdePkg/MdePkg.ci.yaml
@@ -0,0 +1,90 @@
+##
+# CI configuration for MdePkg
+#
+# Copyright (c) Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ ## options defined ci/Plugin/CompilerPlugin
+ "CompilerPlugin": {
+ "DscPath": "MdePkg.dsc"
+ },
+
+ ## options defined ci/Plugin/CharEncodingCheck
+"CharEncodingCheck": {
+ "IgnoreFiles": []
+ },
+
+ ## options defined ci/Plugin/DependencyCheck
+ "DependencyCheck": {
+ "AcceptableDependencies": [
+ "MdePkg/MdePkg.dec"
+ ],
+ # For host based unit tests
+ "AcceptableDependencies-HOST_APPLICATION":[],
+ # For UEFI shell based apps
+ "AcceptableDependencies-UEFI_APPLICATION":[],
+ "IgnoreInf": []
+ },
+
+ ## options defined ci/Plugin/DscCompleteCheck
+ "DscCompleteCheck": {
+ "IgnoreInf": [],
+ "DscPath": "MdePkg.dsc"
+ },
+
+ ## options defined ci/Plugin/GuidCheck
+ "GuidCheck": {
+ "IgnoreGuidName": [
+ "gEfiFirmwareVolumeTopFileGuid" # sec modules must be set to this guid
+ ],
+ "IgnoreGuidValue": ["00000000-0000-0000-0000-000000000000"],
+ "IgnoreFoldersAndFiles": [],
+ "IgnoreDuplicates": [
+ "gEfiFirmwareVolumeBlockProtocolGuid=gEfiFirmwareVolumeBlock2ProtocolGuid",
+ "gEfiMmAccessProtocolGuid=gEfiSmmAccess2ProtocolGuid",
+ "gEfiSmmCommunicationProtocolGuid=gEfiMmCommunicationProtocolGuid",
+ "gEfiMmPciRootBridgeIoProtocolGuid=gEfiSmmPciRootBridgeIoProtocolGuid",
+ "gEfiTrEEProtocolGuid=gEfiTcg2ProtocolGuid",
+ "gEfiSmmPowerButtonDispatch2ProtocolGuid=gEfiMmPowerButtonDispatchProtocolGuid",
+ "gEfiSmmBase2ProtocolGuid=gEfiMmBaseProtocolGuid",
+ "gEfiSmmUsbDispatch2ProtocolGuid=gEfiMmUsbDispatchProtocolGuid",
+ "gEfiSmmCpuProtocolGuid=gEfiMmCpuProtocolGuid",
+ "gEfiAcpiTableGuid=gEfiAcpi20TableGuid",
+ "gEfiSmmControl2ProtocolGuid=gEfiMmControlProtocolGuid",
+ "gEfiSmmStandbyButtonDispatch2ProtocolGuid=gEfiMmStandbyButtonDispatchProtocolGuid",
+ "gEfiSmmStatusCodeProtocolGuid=gEfiMmStatusCodeProtocolGuid",
+ "gEfiDxeSmmReadyToLockProtocolGuid=gEfiDxeMmReadyToLockProtocolGuid",
+ "gEfiSmmIoTrapDispatch2ProtocolGuid=gEfiMmIoTrapDispatchProtocolGuid",
+ "gEfiSmmReadyToLockProtocolGuid=gEfiMmReadyToLockProtocolGuid",
+ "gEfiSmmSxDispatch2ProtocolGuid=gEfiMmSxDispatchProtocolGuid",
+ "gEfiPeiCapsulePpiGuid=gPeiCapsulePpiGuid",
+ "gEfiSmmCpuIo2ProtocolGuid=gEfiMmCpuIoProtocolGuid",
+ "gEfiSmmRscHandlerProtocolGuid=gEfiMmRscHandlerProtocolGuid",
+ "gEfiSmmConfigurationProtocolGuid=gEfiMmConfigurationProtocolGuid",
+ "gEfiSmmGpiDispatch2ProtocolGuid=gEfiMmGpiDispatchProtocolGuid",
+ "gEfiSmmEndOfDxeProtocolGuid=gEfiMmEndOfDxeProtocolGuid",
+ "gEfiSmmSwDispatch2ProtocolGuid=gEfiMmSwDispatchProtocolGuid",
+ "gEfiDebugPortProtocolGuid=gEfiDebugPortVariableGuid=gEfiDebugPortDevicePathGuid", ## is this a bug
+ "gEfiProcessorSpecificErrorSectionGuid=gEfiIa32X64ProcessorErrorSectionGuid", ## is this a bug
+ "gEfiSmmPeriodicTimerDispatch2ProtocolGuid=gEfiMmPeriodicTimerDispatchProtocolGuid",
+ "gEdkiiFormBrowserExProtocolGuid=gEfiFormBrowserExProtocolGuid",
+ "gEfiPeiMmAccessPpiGuid=gPeiSmmAccessPpiGuid",
+ "gPeiSmmControlPpiGuid=gEfiPeiMmControlPpiGuid"
+ ]
+ },
+
+ ## options defined ci/Plugin/LibraryClassCheck
+ "LibraryClassCheck": {
+ "IgnoreHeaderFile": []
+ },
+
+ ## options defined ci/Plugin/SpellCheck
+ "SpellCheck": {
+ "AuditOnly": True, # Fails test but run in AuditOnly mode to collect log
+ "IgnoreFiles": [], # use gitignore syntax to ignore errors in matching files
+ "ExtendWords": [], # words to extend to the dictionary for this package
+ "IgnoreStandardPaths": [], # Standard Plugin defined paths that should be ignore
+ "AdditionalIncludePaths": [] # Additional paths to spell check (wildcards supported)
+ }
+}
--
2.21.0.windows.1
^ permalink raw reply related [flat|nested] 51+ messages in thread
* [Patch v4 13/22] NetworkPkg: Add YAML file for CI builds
2019-11-07 1:13 [Patch v4 00/22] Enable Phase 1 of EDK II CI Michael D Kinney
` (11 preceding siblings ...)
2019-11-07 1:13 ` [Patch v4 12/22] MdePkg: " Michael D Kinney
@ 2019-11-07 1:13 ` Michael D Kinney
2019-11-07 1:13 ` [Patch v4 14/22] PcAtChipsetPkg: Add YAML files " Michael D Kinney
` (9 subsequent siblings)
22 siblings, 0 replies; 51+ messages in thread
From: Michael D Kinney @ 2019-11-07 1:13 UTC (permalink / raw)
To: devel; +Cc: Jiaxin Wu, Siyuan Fu
Add YAML file to the package directory with the
configuration of the checks to perform during a
CI build.
Use BaseCryptLibNull and TlsLibNull for package CI
builds to reduce package build times. Enabled with
CONTINUOUS_INTEGRATION in YAML files. By default
CONTINUOUS_INTEGRATION is not defined, and the
original lib mappings are preserved.
Cc: Jiaxin Wu <jiaxin.wu@intel.com>
Cc: Siyuan Fu <siyuan.fu@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
---
NetworkPkg/NetworkPkg.ci.yaml | 62 +++++++++++++++++++++++++++++++++++
NetworkPkg/NetworkPkg.dsc | 5 +++
2 files changed, 67 insertions(+)
create mode 100644 NetworkPkg/NetworkPkg.ci.yaml
diff --git a/NetworkPkg/NetworkPkg.ci.yaml b/NetworkPkg/NetworkPkg.ci.yaml
new file mode 100644
index 0000000000..ea41c9db66
--- /dev/null
+++ b/NetworkPkg/NetworkPkg.ci.yaml
@@ -0,0 +1,62 @@
+##
+# CI configuration for NetworkPkg
+#
+# Copyright (c) Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "CompilerPlugin": {
+ "DscPath": "NetworkPkg.dsc"
+ },
+ "CharEncodingCheck": {
+ "IgnoreFiles": []
+ },
+ "DependencyCheck": {
+ "AcceptableDependencies": [
+ "MdePkg/MdePkg.dec",
+ "MdeModulePkg/MdeModulePkg.dec",
+ "NetworkPkg/NetworkPkg.dec",
+ "CryptoPkg/CryptoPkg.dec"
+ ],
+ # For host based unit tests
+ "AcceptableDependencies-HOST_APPLICATION":[],
+ # For UEFI shell based apps
+ "AcceptableDependencies-UEFI_APPLICATION":[
+ "ShellPkg/ShellPkg.dec"
+ ],
+ "IgnoreInf": []
+ },
+ "DscCompleteCheck": {
+ "DscPath": "NetworkPkg.dsc",
+ "IgnoreInf": []
+ },
+ "GuidCheck": {
+ "IgnoreGuidName": [],
+ "IgnoreGuidValue": [],
+ "IgnoreFoldersAndFiles": []
+ },
+ "LibraryClassCheck": {
+ "IgnoreHeaderFile": []
+ },
+
+ ## options defined ci/Plugin/SpellCheck
+ "SpellCheck": {
+ "AuditOnly": True, # Fails test but run in AuditOnly mode to collect log
+ "IgnoreFiles": [], # use gitignore syntax to ignore errors in matching files
+ "ExtendWords": [], # words to extend to the dictionary for this package
+ "IgnoreStandardPaths": [], # Standard Plugin defined paths that should be ignore
+ "AdditionalIncludePaths": [] # Additional paths to spell check (wildcards supported)
+ },
+
+ "Defines": {
+ "BLD_*_CONTINUOUS_INTEGRATION": "TRUE",
+ "BLD_*_NETWORK_ENABLE": "TRUE",
+ "BLD_*_NETWORK_SNP_ENABLE": "TRUE",
+ "BLD_*_NETWORK_VLAN_ENABLE": "TRUE",
+ "BLD_*_NETWORK_IP4_ENABLE": "TRUE",
+ "BLD_*_NETWORK_IP6_ENABLE": "TRUE",
+ "BLD_*_NETWORK_TLS_ENABLE": "TRUE",
+ "BLD_*_NETWORK_HTTP_BOOT_ENABLE": "TRUE",
+ "BLD_*_NETWORK_ISCSI_ENABLE": "TRUE",
+ }
+}
diff --git a/NetworkPkg/NetworkPkg.dsc b/NetworkPkg/NetworkPkg.dsc
index 11a2981261..b149453d26 100644
--- a/NetworkPkg/NetworkPkg.dsc
+++ b/NetworkPkg/NetworkPkg.dsc
@@ -41,10 +41,15 @@ [LibraryClasses]
DxeServicesTableLib|MdePkg/Library/DxeServicesTableLib/DxeServicesTableLib.inf
SafeIntLib|MdePkg/Library/BaseSafeIntLib/BaseSafeIntLib.inf
+!ifdef CONTINUOUS_INTEGRATION
+ BaseCryptLib|CryptoPkg/Library/BaseCryptLibNull/BaseCryptLibNull.inf
+ TlsLib|CryptoPkg/Library/TlsLibNull/TlsLibNull.inf
+!else
BaseCryptLib|CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf
OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf
TlsLib|CryptoPkg/Library/TlsLib/TlsLib.inf
+!endif
DebugPrintErrorLevelLib|MdePkg/Library/BaseDebugPrintErrorLevelLib/BaseDebugPrintErrorLevelLib.inf
FileHandleLib|MdePkg/Library/UefiFileHandleLib/UefiFileHandleLib.inf
FileExplorerLib|MdeModulePkg/Library/FileExplorerLib/FileExplorerLib.inf
--
2.21.0.windows.1
^ permalink raw reply related [flat|nested] 51+ messages in thread
* [Patch v4 14/22] PcAtChipsetPkg: Add YAML files for CI builds
2019-11-07 1:13 [Patch v4 00/22] Enable Phase 1 of EDK II CI Michael D Kinney
` (12 preceding siblings ...)
2019-11-07 1:13 ` [Patch v4 13/22] NetworkPkg: " Michael D Kinney
@ 2019-11-07 1:13 ` Michael D Kinney
2019-11-07 2:12 ` Ni, Ray
2019-11-07 1:13 ` [Patch v4 15/22] SecurityPkg: " Michael D Kinney
` (8 subsequent siblings)
22 siblings, 1 reply; 51+ messages in thread
From: Michael D Kinney @ 2019-11-07 1:13 UTC (permalink / raw)
To: devel; +Cc: Ray Ni
Add YAML file to the package directory with the
configuration of the checks to perform during a
CI build.
Cc: Ray Ni <ray.ni@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
---
PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml | 46 +++++++++++++++++++++++++++
1 file changed, 46 insertions(+)
create mode 100644 PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml
diff --git a/PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml b/PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml
new file mode 100644
index 0000000000..6ab9f11761
--- /dev/null
+++ b/PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml
@@ -0,0 +1,46 @@
+##
+# CI configuration for PcAtChipsetPkg
+#
+# Copyright (c) Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "CompilerPlugin": {
+ "DscPath": "PcAtChipsetPkg.dsc"
+ },
+ "CharEncodingCheck": {
+ "IgnoreFiles": []
+ },
+ "DependencyCheck": {
+ "AcceptableDependencies": [
+ "MdePkg/MdePkg.dec",
+ "PcAtChipsetPkg/PcAtChipsetPkg.dec",
+ "UefiCpuPkg/UefiCpuPkg.dec"
+ ],
+ # For host based unit tests
+ "AcceptableDependencies-HOST_APPLICATION":[],
+ # For UEFI shell based apps
+ "AcceptableDependencies-UEFI_APPLICATION":[],
+ "IgnoreInf": []
+ },
+ "DscCompleteCheck": {
+ "DscPath": "PcAtChipsetPkg.dsc",
+ "IgnoreInf": []
+ },
+ "GuidCheck": {
+ "IgnoreGuidName": [],
+ "IgnoreGuidValue": [],
+ "IgnoreFoldersAndFiles": [],
+ "IgnoreDuplicates": []
+ },
+ "LibraryClassCheck": {
+ "IgnoreHeaderFile": []
+ },
+ "SpellCheck": {
+ "ExtendWords": [
+ "ENUMER", # this is part of an IDE enum
+ "PCATCHIPSET",
+ "TXRDY"
+ ]
+ }
+}
--
2.21.0.windows.1
^ permalink raw reply related [flat|nested] 51+ messages in thread
* [Patch v4 15/22] SecurityPkg: Add YAML files for CI builds
2019-11-07 1:13 [Patch v4 00/22] Enable Phase 1 of EDK II CI Michael D Kinney
` (13 preceding siblings ...)
2019-11-07 1:13 ` [Patch v4 14/22] PcAtChipsetPkg: Add YAML files " Michael D Kinney
@ 2019-11-07 1:13 ` Michael D Kinney
2019-11-07 5:08 ` Wang, Jian J
2019-11-07 1:13 ` [Patch v4 16/22] ShellPkg: Add YAML file " Michael D Kinney
` (7 subsequent siblings)
22 siblings, 1 reply; 51+ messages in thread
From: Michael D Kinney @ 2019-11-07 1:13 UTC (permalink / raw)
To: devel; +Cc: Jiewen Yao, Jian J Wang, Chao Zhang
Add YAML file to the package directory with the
configuration of the checks to perform during a
CI build.
Use BaseCryptLibNull for package CI builds to reduce package
build times. Enabled with CONTINUOUS_INTEGRATION in YAML
files. By default CONTINUOUS_INTEGRATION is not defined,
and the original lib mappings are preserved.
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Jian J Wang <jian.j.wang@intel.com>
Cc: Chao Zhang <chao.b.zhang@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
---
SecurityPkg/SecurityPkg.ci.yaml | 80 +++++++++++++++++++++++++++++++++
SecurityPkg/SecurityPkg.dsc | 32 ++++++++++++-
2 files changed, 110 insertions(+), 2 deletions(-)
create mode 100644 SecurityPkg/SecurityPkg.ci.yaml
diff --git a/SecurityPkg/SecurityPkg.ci.yaml b/SecurityPkg/SecurityPkg.ci.yaml
new file mode 100644
index 0000000000..c7e309fcda
--- /dev/null
+++ b/SecurityPkg/SecurityPkg.ci.yaml
@@ -0,0 +1,80 @@
+##
+# CI configuration for SecurityPkg
+#
+# Copyright (c) Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "CompilerPlugin": {
+ "DscPath": "SecurityPkg.dsc"
+ },
+ "CharEncodingCheck": {
+ "IgnoreFiles": []
+ },
+ "DependencyCheck": {
+ "AcceptableDependencies": [
+ "MdePkg/MdePkg.dec",
+ "MdeModulePkg/MdeModulePkg.dec",
+ "SecurityPkg/SecurityPkg.dec",
+ "CryptoPkg/CryptoPkg.dec"
+ ],
+ # For host based unit tests
+ "AcceptableDependencies-HOST_APPLICATION":[],
+ # For UEFI shell based apps
+ "AcceptableDependencies-UEFI_APPLICATION":[],
+ "IgnoreInf": []
+ },
+ "DscCompleteCheck": {
+ "DscPath": "SecurityPkg.dsc",
+ "IgnoreInf": []
+ },
+ "GuidCheck": {
+ "IgnoreGuidName": [],
+ "IgnoreGuidValue": ["00000000-0000-0000-0000-000000000000"],
+ "IgnoreFoldersAndFiles": [],
+ "IgnoreDuplicates": [
+ "Tpm2InstanceLibDTpm=gEfiTpmDeviceInstanceTpm20DtpmGuid", # by design
+ ]
+ },
+ "LibraryClassCheck": {
+ "IgnoreHeaderFile": []
+ },
+
+ ## options defined ci/Plugin/SpellCheck
+ "SpellCheck": {
+ "AuditOnly": True, # Fails test but run in AuditOnly mode to collect log
+ "ExtendWords": [ # words to extend to the dictionary for this package
+ "shortformed", # tpm acpi
+ "autodetect",
+ "blocksid",
+ "comid",
+ "cpinsidpin", #OpalSScV2
+ "ecdsa", # TPM
+ "ecschnorr", # TPM
+ "eisaid", # ACPI
+ "harddisk",
+ "hashall",
+ "hashto",
+ "kek's",
+ "lfanew", # PE/COFF
+ "pcrindex",
+ "pkglength",
+ "ppuser",
+ "preos",
+ "stclear",
+ "toctou",
+ "tpm's",
+ "tpmcmdbuflength",
+ "tpmcommlib",
+ "tpmnvvaluelength",
+ "wrlocked",
+ "xored"
+ ],
+ "IgnoreStandardPaths": [], # Standard Plugin defined paths that should be ignore
+ "AdditionalIncludePaths": [] # Additional paths to spell check (wildcards supported)
+ },
+
+ "Defines": {
+ "BLD_*_CONTINUOUS_INTEGRATION": "TRUE",
+ }
+}
diff --git a/SecurityPkg/SecurityPkg.dsc b/SecurityPkg/SecurityPkg.dsc
index 9a254087a3..a2eeadda7a 100644
--- a/SecurityPkg/SecurityPkg.dsc
+++ b/SecurityPkg/SecurityPkg.dsc
@@ -44,8 +44,6 @@ [LibraryClasses]
HiiLib|MdeModulePkg/Library/UefiHiiLib/UefiHiiLib.inf
UefiHiiServicesLib|MdeModulePkg/Library/UefiHiiServicesLib/UefiHiiServicesLib.inf
PcdLib|MdePkg/Library/BasePcdLibNull/BasePcdLibNull.inf
- IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf
- OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
IoLib|MdePkg/Library/BaseIoLibIntrinsic/BaseIoLibIntrinsic.inf
TpmCommLib|SecurityPkg/Library/TpmCommLib/TpmCommLib.inf
PlatformSecureLib|SecurityPkg/Library/PlatformSecureLibNull/PlatformSecureLibNull.inf
@@ -84,7 +82,13 @@ [LibraryClasses.common.PEIM]
PeiServicesTablePointerLib|MdePkg/Library/PeiServicesTablePointerLib/PeiServicesTablePointerLib.inf
HobLib|MdePkg/Library/PeiHobLib/PeiHobLib.inf
MemoryAllocationLib|MdePkg/Library/PeiMemoryAllocationLib/PeiMemoryAllocationLib.inf
+!ifdef CONTINUOUS_INTEGRATION
+ BaseCryptLib|CryptoPkg/Library/BaseCryptLibNull/BaseCryptLibNull.inf
+!else
+ IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf
+ OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
BaseCryptLib|CryptoPkg/Library/BaseCryptLib/PeiCryptLib.inf
+!endif
HashLib|SecurityPkg/Library/HashLibBaseCryptoRouter/HashLibBaseCryptoRouterPei.inf
ReportStatusCodeLib|MdeModulePkg/Library/PeiReportStatusCodeLib/PeiReportStatusCodeLib.inf
Tpm12DeviceLib|SecurityPkg/Library/Tpm12DeviceLibDTpm/Tpm12DeviceLibDTpm.inf
@@ -95,7 +99,13 @@ [LibraryClasses.common.PEIM]
[LibraryClasses.common.DXE_DRIVER]
HobLib|MdePkg/Library/DxeHobLib/DxeHobLib.inf
ReportStatusCodeLib|MdeModulePkg/Library/DxeReportStatusCodeLib/DxeReportStatusCodeLib.inf
+!ifdef CONTINUOUS_INTEGRATION
+ BaseCryptLib|CryptoPkg/Library/BaseCryptLibNull/BaseCryptLibNull.inf
+!else
+ IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf
+ OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
BaseCryptLib|CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf
+!endif
HashLib|SecurityPkg/Library/HashLibBaseCryptoRouter/HashLibBaseCryptoRouterDxe.inf
Tpm12DeviceLib|SecurityPkg/Library/Tpm12DeviceLibTcg/Tpm12DeviceLibTcg.inf
Tpm2DeviceLib|SecurityPkg/Library/Tpm2DeviceLibTcg2/Tpm2DeviceLibTcg2.inf
@@ -110,13 +120,25 @@ [LibraryClasses.common.UEFI_DRIVER, LibraryClasses.common.DXE_RUNTIME_DRIVER, Li
[LibraryClasses.common.DXE_RUNTIME_DRIVER]
ReportStatusCodeLib|MdeModulePkg/Library/RuntimeDxeReportStatusCodeLib/RuntimeDxeReportStatusCodeLib.inf
+!ifdef CONTINUOUS_INTEGRATION
+ BaseCryptLib|CryptoPkg/Library/BaseCryptLibNull/BaseCryptLibNull.inf
+!else
+ IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf
+ OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
BaseCryptLib|CryptoPkg/Library/BaseCryptLib/RuntimeCryptLib.inf
+!endif
HashLib|SecurityPkg/Library/HashLibBaseCryptoRouter/HashLibBaseCryptoRouterDxe.inf
Tpm12DeviceLib|SecurityPkg/Library/Tpm12DeviceLibTcg/Tpm12DeviceLibTcg.inf
Tpm2DeviceLib|SecurityPkg/Library/Tpm2DeviceLibTcg2/Tpm2DeviceLibTcg2.inf
[LibraryClasses.common.UEFI_DRIVER, LibraryClasses.common.UEFI_APPLICATION]
+!ifdef CONTINUOUS_INTEGRATION
+ BaseCryptLib|CryptoPkg/Library/BaseCryptLibNull/BaseCryptLibNull.inf
+!else
+ IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf
+ OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
BaseCryptLib|CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf
+!endif
HashLib|SecurityPkg/Library/HashLibBaseCryptoRouter/HashLibBaseCryptoRouterDxe.inf
Tpm12DeviceLib|SecurityPkg/Library/Tpm12DeviceLibTcg/Tpm12DeviceLibTcg.inf
Tpm2DeviceLib|SecurityPkg/Library/Tpm2DeviceLibTcg2/Tpm2DeviceLibTcg2.inf
@@ -127,7 +149,13 @@ [LibraryClasses.common.DXE_SMM_DRIVER]
MemoryAllocationLib|MdePkg/Library/SmmMemoryAllocationLib/SmmMemoryAllocationLib.inf
ReportStatusCodeLib|MdeModulePkg/Library/SmmReportStatusCodeLib/SmmReportStatusCodeLib.inf
SmmMemLib|MdePkg/Library/SmmMemLib/SmmMemLib.inf
+!ifdef CONTINUOUS_INTEGRATION
+ BaseCryptLib|CryptoPkg/Library/BaseCryptLibNull/BaseCryptLibNull.inf
+!else
+ IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf
+ OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
BaseCryptLib|CryptoPkg/Library/BaseCryptLib/SmmCryptLib.inf
+!endif
Tpm12DeviceLib|SecurityPkg/Library/Tpm12DeviceLibTcg/Tpm12DeviceLibTcg.inf
Tpm2DeviceLib|SecurityPkg/Library/Tpm2DeviceLibTcg2/Tpm2DeviceLibTcg2.inf
Tcg2PhysicalPresenceLib|SecurityPkg/Library/SmmTcg2PhysicalPresenceLib/SmmTcg2PhysicalPresenceLib.inf
--
2.21.0.windows.1
^ permalink raw reply related [flat|nested] 51+ messages in thread
* [Patch v4 16/22] ShellPkg: Add YAML file for CI builds
2019-11-07 1:13 [Patch v4 00/22] Enable Phase 1 of EDK II CI Michael D Kinney
` (14 preceding siblings ...)
2019-11-07 1:13 ` [Patch v4 15/22] SecurityPkg: " Michael D Kinney
@ 2019-11-07 1:13 ` Michael D Kinney
2019-11-07 2:12 ` Ni, Ray
2019-11-07 1:13 ` [Patch v4 17/22] UefiCpuPkg: " Michael D Kinney
` (6 subsequent siblings)
22 siblings, 1 reply; 51+ messages in thread
From: Michael D Kinney @ 2019-11-07 1:13 UTC (permalink / raw)
To: devel; +Cc: Ray Ni, Zhichao Gao
Add YAML file to the package directory with the
configuration of the checks to perform during a
CI build.
Cc: Ray Ni <ray.ni@intel.com>
Cc: Zhichao Gao <zhichao.gao@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Acked-by: Zhichao Gao <zhichao.gao@intel.com>
---
ShellPkg/ShellPkg.ci.yaml | 55 +++++++++++++++++++++++++++++++++++++++
1 file changed, 55 insertions(+)
create mode 100644 ShellPkg/ShellPkg.ci.yaml
diff --git a/ShellPkg/ShellPkg.ci.yaml b/ShellPkg/ShellPkg.ci.yaml
new file mode 100644
index 0000000000..5bf7330c1a
--- /dev/null
+++ b/ShellPkg/ShellPkg.ci.yaml
@@ -0,0 +1,55 @@
+##
+# CI configuration for ShellPkg
+#
+# Copyright (c) Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "CompilerPlugin": {
+ "DscPath": "ShellPkg.dsc"
+ },
+ "CharEncodingCheck": {
+ "IgnoreFiles": []
+ },
+ "DependencyCheck": {
+ "AcceptableDependencies": [
+ "MdePkg/MdePkg.dec",
+ "MdeModulePkg/MdeModulePkg.dec",
+ "ShellPkg/ShellPkg.dec",
+ "NetworkPkg/NetworkPkg.dec"
+ ],
+ # For host based unit tests
+ "AcceptableDependencies-HOST_APPLICATION":[],
+ # For UEFI shell based apps
+ "AcceptableDependencies-UEFI_APPLICATION":[],
+ "IgnoreInf": []
+ },
+ "DscCompleteCheck": {
+ "DscPath": "ShellPkg.dsc",
+ "IgnoreInf": [
+ "ShellPkg/Application/ShellCTestApp/ShellCTestApp.inf",
+ "ShellPkg/Application/ShellExecTestApp/SA.inf",
+ "ShellPkg/Application/ShellSortTestApp/ShellSortTestApp.inf"
+ ]
+ },
+ "GuidCheck": {
+ "IgnoreGuidName": [],
+ "IgnoreGuidValue": [],
+ "IgnoreFoldersAndFiles": [],
+ "IgnoreDuplicates": [
+ "Shell=gUefiShellFileGuid", # by design
+ ]
+ },
+ "LibraryClassCheck": {
+ "IgnoreHeaderFile": []
+ },
+
+ ## options defined ci/Plugin/SpellCheck
+ "SpellCheck": {
+ "AuditOnly": True, # Fails test but run in AuditOnly mode to collect log
+ "IgnoreFiles": [], # use gitignore syntax to ignore errors in matching files
+ "ExtendWords": [], # words to extend to the dictionary for this package
+ "IgnoreStandardPaths": [], # Standard Plugin defined paths that should be ignore
+ "AdditionalIncludePaths": [] # Additional paths to spell check (wildcards supported)
+ }
+}
--
2.21.0.windows.1
^ permalink raw reply related [flat|nested] 51+ messages in thread
* [Patch v4 17/22] UefiCpuPkg: Add YAML file for CI builds
2019-11-07 1:13 [Patch v4 00/22] Enable Phase 1 of EDK II CI Michael D Kinney
` (15 preceding siblings ...)
2019-11-07 1:13 ` [Patch v4 16/22] ShellPkg: Add YAML file " Michael D Kinney
@ 2019-11-07 1:13 ` Michael D Kinney
2019-11-07 2:12 ` Ni, Ray
2019-11-07 10:42 ` Laszlo Ersek
2019-11-07 1:13 ` [Patch v4 18/22] SignedCapsulePkg: Use BaseCryptLibNull to reduce package CI time Michael D Kinney
` (5 subsequent siblings)
22 siblings, 2 replies; 51+ messages in thread
From: Michael D Kinney @ 2019-11-07 1:13 UTC (permalink / raw)
To: devel; +Cc: Eric Dong, Ray Ni, Laszlo Ersek
Add YAML file to the package directory with the
configuration of the checks to perform during a
CI build.
Cc: Eric Dong <eric.dong@intel.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Acked-by: Laszlo Ersek <lersek@redhat.com>
---
UefiCpuPkg/UefiCpuPkg.ci.yaml | 51 +++++++++++++++++++++++++++++++++++
1 file changed, 51 insertions(+)
create mode 100644 UefiCpuPkg/UefiCpuPkg.ci.yaml
diff --git a/UefiCpuPkg/UefiCpuPkg.ci.yaml b/UefiCpuPkg/UefiCpuPkg.ci.yaml
new file mode 100644
index 0000000000..81da29b878
--- /dev/null
+++ b/UefiCpuPkg/UefiCpuPkg.ci.yaml
@@ -0,0 +1,51 @@
+##
+# CI configuration for UefiCpuPkg
+#
+# Copyright (c) Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "CompilerPlugin": {
+ "DscPath": "UefiCpuPkg.dsc"
+ },
+ "CharEncodingCheck": {
+ "IgnoreFiles": []
+ },
+ "DependencyCheck": {
+ "AcceptableDependencies": [
+ "MdePkg/MdePkg.dec",
+ "MdeModulePkg/MdeModulePkg.dec",
+ "UefiCpuPkg/UefiCpuPkg.dec"
+ ],
+ # For host based unit tests
+ "AcceptableDependencies-HOST_APPLICATION":[],
+ # For UEFI shell based apps
+ "AcceptableDependencies-UEFI_APPLICATION":[],
+ "IgnoreInf": []
+ },
+ "DscCompleteCheck": {
+ "DscPath": "UefiCpuPkg.dsc",
+ "IgnoreInf": [
+ "UefiCpuPkg/ResetVector/FixupVtf/Vtf.inf",
+ "UefiCpuPkg/ResetVector/Vtf0/Vtf0.inf"
+ ]
+ },
+ "GuidCheck": {
+ "IgnoreGuidName": ["SecCore", "ResetVector"], # Expected duplication for gEfiFirmwareVolumeTopFileGuid
+ "IgnoreGuidValue": [],
+ "IgnoreFoldersAndFiles": [],
+ "IgnoreDuplicates": []
+ },
+ "LibraryClassCheck": {
+ "IgnoreHeaderFile": []
+ },
+
+ ## options defined ci/Plugin/SpellCheck
+ "SpellCheck": {
+ "AuditOnly": True, # Fails test but run in AuditOnly mode to collect log
+ "IgnoreFiles": [], # use gitignore syntax to ignore errors in matching files
+ "ExtendWords": [], # words to extend to the dictionary for this package
+ "IgnoreStandardPaths": [], # Standard Plugin defined paths that should be ignore
+ "AdditionalIncludePaths": [] # Additional paths to spell check (wildcards supported)
+ }
+}
--
2.21.0.windows.1
^ permalink raw reply related [flat|nested] 51+ messages in thread
* [Patch v4 18/22] SignedCapsulePkg: Use BaseCryptLibNull to reduce package CI time
2019-11-07 1:13 [Patch v4 00/22] Enable Phase 1 of EDK II CI Michael D Kinney
` (16 preceding siblings ...)
2019-11-07 1:13 ` [Patch v4 17/22] UefiCpuPkg: " Michael D Kinney
@ 2019-11-07 1:13 ` Michael D Kinney
2019-11-07 1:13 ` [Patch v4 19/22] .pytool: Add CISettings.py and Readme.md Michael D Kinney
` (4 subsequent siblings)
22 siblings, 0 replies; 51+ messages in thread
From: Michael D Kinney @ 2019-11-07 1:13 UTC (permalink / raw)
To: devel; +Cc: Jiewen Yao, Chao Zhang
Use BaseCryptLibNull for package CI builds to reduce package
build times. Enabled with PYTOOL_CONTINUOUS_INTEGRATION in YAML
files. By default PYTOOL_CONTINUOUS_INTEGRATION is not defined,
and the original lib mappings are preserved.
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Chao Zhang <chao.b.zhang@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
---
SignedCapsulePkg/SignedCapsulePkg.dsc | 38 +++++++++++++++++++++++++--
1 file changed, 36 insertions(+), 2 deletions(-)
diff --git a/SignedCapsulePkg/SignedCapsulePkg.dsc b/SignedCapsulePkg/SignedCapsulePkg.dsc
index 03f714f9a7..c5080ec1dd 100644
--- a/SignedCapsulePkg/SignedCapsulePkg.dsc
+++ b/SignedCapsulePkg/SignedCapsulePkg.dsc
@@ -87,8 +87,6 @@ [LibraryClasses]
AuthVariableLib|MdeModulePkg/Library/AuthVariableLibNull/AuthVariableLibNull.inf
VarCheckLib|MdeModulePkg/Library/VarCheckLib/VarCheckLib.inf
FileExplorerLib|MdeModulePkg/Library/FileExplorerLib/FileExplorerLib.inf
- IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf
- OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
EdkiiSystemCapsuleLib|SignedCapsulePkg/Library/EdkiiSystemCapsuleLib/EdkiiSystemCapsuleLib.inf
IniParsingLib|SignedCapsulePkg/Library/IniParsingLib/IniParsingLib.inf
@@ -116,7 +114,13 @@ [LibraryClasses.common.PEIM]
HobLib|MdePkg/Library/PeiHobLib/PeiHobLib.inf
MemoryAllocationLib|MdePkg/Library/PeiMemoryAllocationLib/PeiMemoryAllocationLib.inf
LockBoxLib|MdeModulePkg/Library/SmmLockBoxLib/SmmLockBoxPeiLib.inf
+!ifdef CONTINUOUS_INTEGRATION
+ BaseCryptLib|CryptoPkg/Library/BaseCryptLibNull/BaseCryptLibNull.inf
+!else
+ IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf
+ OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
BaseCryptLib|CryptoPkg/Library/BaseCryptLib/PeiCryptLib.inf
+!endif
[LibraryClasses.common.DXE_CORE]
HobLib|MdePkg/Library/DxeCoreHobLib/DxeCoreHobLib.inf
@@ -125,7 +129,13 @@ [LibraryClasses.common.DXE_CORE]
[LibraryClasses.common.DXE_DRIVER]
HobLib|MdePkg/Library/DxeHobLib/DxeHobLib.inf
MemoryAllocationLib|MdePkg/Library/UefiMemoryAllocationLib/UefiMemoryAllocationLib.inf
+!ifdef CONTINUOUS_INTEGRATION
+ BaseCryptLib|CryptoPkg/Library/BaseCryptLibNull/BaseCryptLibNull.inf
+!else
+ IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf
+ OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
BaseCryptLib|CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf
+!endif
CapsuleLib|MdeModulePkg/Library/DxeCapsuleLibFmp/DxeCapsuleLib.inf
@@ -133,7 +143,13 @@ [LibraryClasses.common.DXE_RUNTIME_DRIVER]
HobLib|MdePkg/Library/DxeHobLib/DxeHobLib.inf
MemoryAllocationLib|MdePkg/Library/UefiMemoryAllocationLib/UefiMemoryAllocationLib.inf
DebugLib|MdePkg/Library/UefiDebugLibConOut/UefiDebugLibConOut.inf
+!ifdef CONTINUOUS_INTEGRATION
+ BaseCryptLib|CryptoPkg/Library/BaseCryptLibNull/BaseCryptLibNull.inf
+!else
+ IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf
+ OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
BaseCryptLib|CryptoPkg/Library/BaseCryptLib/RuntimeCryptLib.inf
+!endif
CapsuleLib|MdeModulePkg/Library/DxeCapsuleLibFmp/DxeRuntimeCapsuleLib.inf
@@ -150,19 +166,37 @@ [LibraryClasses.common.DXE_SMM_DRIVER]
MemoryAllocationLib|MdePkg/Library/SmmMemoryAllocationLib/SmmMemoryAllocationLib.inf
SmmServicesTableLib|MdePkg/Library/SmmServicesTableLib/SmmServicesTableLib.inf
SmmMemLib|MdePkg/Library/SmmMemLib/SmmMemLib.inf
+!ifdef CONTINUOUS_INTEGRATION
+ BaseCryptLib|CryptoPkg/Library/BaseCryptLibNull/BaseCryptLibNull.inf
+!else
+ IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf
+ OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
BaseCryptLib|CryptoPkg/Library/BaseCryptLib/SmmCryptLib.inf
+!endif
[LibraryClasses.common.UEFI_DRIVER]
HobLib|MdePkg/Library/DxeHobLib/DxeHobLib.inf
MemoryAllocationLib|MdePkg/Library/UefiMemoryAllocationLib/UefiMemoryAllocationLib.inf
DebugLib|MdePkg/Library/UefiDebugLibConOut/UefiDebugLibConOut.inf
+!ifdef CONTINUOUS_INTEGRATION
+ BaseCryptLib|CryptoPkg/Library/BaseCryptLibNull/BaseCryptLibNull.inf
+!else
+ IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf
+ OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
BaseCryptLib|CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf
+!endif
[LibraryClasses.common.UEFI_APPLICATION]
HobLib|MdePkg/Library/DxeHobLib/DxeHobLib.inf
MemoryAllocationLib|MdePkg/Library/UefiMemoryAllocationLib/UefiMemoryAllocationLib.inf
DebugLib|MdePkg/Library/UefiDebugLibStdErr/UefiDebugLibStdErr.inf
+!ifdef CONTINUOUS_INTEGRATION
+ BaseCryptLib|CryptoPkg/Library/BaseCryptLibNull/BaseCryptLibNull.inf
+!else
+ IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf
+ OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
BaseCryptLib|CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf
+!endif
[PcdsFixedAtBuild]
gEfiMdePkgTokenSpaceGuid.PcdDebugPropertyMask|0x0f
--
2.21.0.windows.1
^ permalink raw reply related [flat|nested] 51+ messages in thread
* [Patch v4 19/22] .pytool: Add CISettings.py and Readme.md
2019-11-07 1:13 [Patch v4 00/22] Enable Phase 1 of EDK II CI Michael D Kinney
` (17 preceding siblings ...)
2019-11-07 1:13 ` [Patch v4 18/22] SignedCapsulePkg: Use BaseCryptLibNull to reduce package CI time Michael D Kinney
@ 2019-11-07 1:13 ` Michael D Kinney
2019-11-07 16:16 ` [edk2-devel] " rebecca
2019-11-07 1:13 ` [Patch v4 20/22] .azurepipelines: Add Azure Pipelines YML configuration files Michael D Kinney
` (3 subsequent siblings)
22 siblings, 1 reply; 51+ messages in thread
From: Michael D Kinney @ 2019-11-07 1:13 UTC (permalink / raw)
To: devel; +Cc: Sean Brogan, Bret Barkelew, Liming Gao
From: Sean Brogan <sean.brogan@microsoft.com>
Add main python script for EDK II Continuous Integration (CI)
builds along with a Readme.md that provides a summary of the
packages, platforms, and checks performs during a CI build.
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
---
.pytool/CISettings.py | 173 ++++++++++++++++++++++++++++++++++++++
.pytool/Readme.md | 191 ++++++++++++++++++++++++++++++++++++++++++
2 files changed, 364 insertions(+)
create mode 100644 .pytool/CISettings.py
create mode 100644 .pytool/Readme.md
diff --git a/.pytool/CISettings.py b/.pytool/CISettings.py
new file mode 100644
index 0000000000..a78e8b974c
--- /dev/null
+++ b/.pytool/CISettings.py
@@ -0,0 +1,173 @@
+# @file
+#
+# Copyright (c) 2018, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+import os
+import logging
+from edk2toolext.environment import shell_environment
+from edk2toolext.invocables.edk2_ci_build import CiBuildSettingsManager
+from edk2toolext.invocables.edk2_setup import SetupSettingsManager, RequiredSubmodule
+from edk2toolext.invocables.edk2_update import UpdateSettingsManager
+from edk2toolext.invocables.edk2_pr_eval import PrEvalSettingsManager
+from edk2toollib.utility_functions import GetHostInfo
+
+
+class Settings(CiBuildSettingsManager, UpdateSettingsManager, SetupSettingsManager, PrEvalSettingsManager):
+
+ def __init__(self):
+ self.ActualPackages = []
+ self.ActualTargets = []
+ self.ActualArchitectures = []
+ self.ActualToolChainTag = ""
+
+ # ####################################################################################### #
+ # Extra CmdLine configuration #
+ # ####################################################################################### #
+
+ def AddCommandLineOptions(self, parserObj):
+ pass
+
+ def RetrieveCommandLineOptions(self, args):
+ pass
+
+ # ####################################################################################### #
+ # Default Support for this Ci Build #
+ # ####################################################################################### #
+
+ def GetPackagesSupported(self):
+ ''' return iterable of edk2 packages supported by this build.
+ These should be edk2 workspace relative paths '''
+
+ return ("MdePkg",
+ "MdeModulePkg",
+ "NetworkPkg",
+ "PcAtChipsetPkg",
+ "SecurityPkg",
+ "UefiCpuPkg",
+ "FmpDevicePkg",
+ "ShellPkg",
+ "FatPkg",
+ "CryptoPkg"
+ )
+
+ def GetArchitecturesSupported(self):
+ ''' return iterable of edk2 architectures supported by this build '''
+ return ("IA32",
+ "X64",
+ "ARM",
+ "AARCH64")
+
+ def GetTargetsSupported(self):
+ ''' return iterable of edk2 target tags supported by this build '''
+ return ("DEBUG", "RELEASE", "NO-TARGET", "NOOPT")
+
+ # ####################################################################################### #
+ # Verify and Save requested Ci Build Config #
+ # ####################################################################################### #
+
+ def SetPackages(self, list_of_requested_packages):
+ ''' Confirm the requested package list is valid and configure SettingsManager
+ to build the requested packages.
+
+ Raise UnsupportedException if a requested_package is not supported
+ '''
+ unsupported = set(list_of_requested_packages) - \
+ set(self.GetPackagesSupported())
+ if(len(unsupported) > 0):
+ logging.critical(
+ "Unsupported Package Requested: " + " ".join(unsupported))
+ raise Exception("Unsupported Package Requested: " +
+ " ".join(unsupported))
+ self.ActualPackages = list_of_requested_packages
+
+ def SetArchitectures(self, list_of_requested_architectures):
+ ''' Confirm the requests architecture list is valid and configure SettingsManager
+ to run only the requested architectures.
+
+ Raise Exception if a list_of_requested_architectures is not supported
+ '''
+ unsupported = set(list_of_requested_architectures) - \
+ set(self.GetArchitecturesSupported())
+ if(len(unsupported) > 0):
+ logging.critical(
+ "Unsupported Architecture Requested: " + " ".join(unsupported))
+ raise Exception(
+ "Unsupported Architecture Requested: " + " ".join(unsupported))
+ self.ActualArchitectures = list_of_requested_architectures
+
+ def SetTargets(self, list_of_requested_target):
+ ''' Confirm the request target list is valid and configure SettingsManager
+ to run only the requested targets.
+
+ Raise UnsupportedException if a requested_target is not supported
+ '''
+ unsupported = set(list_of_requested_target) - \
+ set(self.GetTargetsSupported())
+ if(len(unsupported) > 0):
+ logging.critical(
+ "Unsupported Targets Requested: " + " ".join(unsupported))
+ raise Exception("Unsupported Targets Requested: " +
+ " ".join(unsupported))
+ self.ActualTargets = list_of_requested_target
+
+ # ####################################################################################### #
+ # Actual Configuration for Ci Build #
+ # ####################################################################################### #
+
+ def GetActiveScopes(self):
+ ''' return tuple containing scopes that should be active for this process '''
+ scopes = ("cibuild","edk2-build")
+
+ self.ActualToolChainTag = shell_environment.GetBuildVars().GetValue("TOOL_CHAIN_TAG", "")
+
+ if GetHostInfo().os.upper() == "LINUX" and self.ActualToolChainTag.upper().startswith("GCC"):
+ if "AARCH64" in self.ActualArchitectures:
+ scopes += ("gcc_aarch64_linux",)
+ if "ARM" in self.ActualArchitectures:
+ scopes += ("gcc_arm_linux",)
+
+ return scopes
+
+ def GetRequiredSubmodules(self):
+ ''' return iterable containing RequiredSubmodule objects.
+ If no RequiredSubmodules return an empty iterable
+ '''
+ rs=[]
+ rs.append(RequiredSubmodule(
+ "ArmPkg/Library/ArmSoftFloatLib/berkeley-softfloat-3", False))
+ rs.append(RequiredSubmodule(
+ "CryptoPkg/Library/OpensslLib/openssl", False))
+ return rs
+
+ def GetName(self):
+ return "Edk2"
+
+ def GetDependencies(self):
+ return []
+
+ def GetPackagesPath(self):
+ return ()
+
+ def GetWorkspaceRoot(self):
+ ''' get WorkspacePath '''
+ return os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+
+ def FilterPackagesToTest(self, changedFilesList: list, potentialPackagesList: list) -> list:
+ ''' Filter potential packages to test based on changed files. '''
+ build_these_packages=[]
+ possible_packages=potentialPackagesList.copy()
+ for f in changedFilesList:
+ nodes=f.split("/") # split each part of path for comparison later
+
+ # python file change in .pytool folder causes building all
+ if f.endswith(".py") and ".pytool" in nodes:
+ build_these_packages = possible_packages
+ break
+
+ # BaseTools files that might change the build
+ if "BaseTools" in nodes:
+ if os.path.splitext(f) not in [".txt", ".md"]:
+ build_these_packages = possible_packages
+ break
+ return build_these_packages
diff --git a/.pytool/Readme.md b/.pytool/Readme.md
new file mode 100644
index 0000000000..fa10628fcf
--- /dev/null
+++ b/.pytool/Readme.md
@@ -0,0 +1,191 @@
+# Edk2 Continuous Integration
+
+## Basic Status
+
+| Package | Windows VS2019 | Ubuntu GCC | Known Issues |
+| :---- | :----- | :---- | :--- |
+| ArmPkg |
+| ArmPlatformPkg |
+| ArmVirtPkg |
+| CryptoPkg | :heavy_check_mark: | :heavy_check_mark: | New changes for NULL basecryptlib and tlslib, add IntrinsicLib.inf to dsc
+| DynamicTablesPkg |
+| EmbeddedPkg |
+| EmulatorPkg |
+| FatPkg | :heavy_check_mark: | :heavy_check_mark: |
+| FmpDevicePkg | :heavy_check_mark: | :heavy_check_mark: |
+| IntelFsp2Pkg |
+| IntelFsp2WrapperPkg |
+| MdeModulePkg | :heavy_check_mark: | :heavy_check_mark: | DxeIpl dependency on ArmPkg, Missing Visual Studio AARCH64/ARM support for EBC
+| MdePkg | :heavy_check_mark: | :heavy_check_mark: | Update DSC to add UefiFileHandleLib, Update BaseIoLibIntrinsic to support MSFT ARM/AARCH64
+| NetworkPkg | :heavy_check_mark: | :heavy_check_mark: | Libraries missing from components section for package dsc
+| OvmfPkg |
+| PcAtChipsetPkg | :heavy_check_mark: | :heavy_check_mark: |
+| SecurityPkg | :heavy_check_mark: | :heavy_check_mark: | Incorrect libraryclass in dec
+| ShellPkg | :heavy_check_mark: | :heavy_check_mark: |
+| SignedCapsulePkg |
+| SourceLevelDebugPkg |
+| StandaloneMmPkg |
+| UefiCpuPkg | :heavy_check_mark: | :heavy_check_mark: |
+| UefiPayloadPkg |
+
+For more detailed status look at the test results of the latest CI run on the
+repo readme.
+
+## Background
+
+While a number of CI solutions exist, this proposal will focus on the usage of
+Azure Dev Ops and Build Pipelines. For demonstration, a sample
+[TianoCore repo](https://github.com/tianocore/edk2-staging.git) (branch master)
+and [Dev Ops Pipeline](https://dev.azure.com/tianocore/edk2-ci-play/_build?definitionId=14)
+have been set up.
+
+Furthermore, this proposal will leverage the TianoCore python tools PIP modules:
+[library](https://pypi.org/project/edk2-pytool-library/) and
+[extensions](https://pypi.org/project/edk2-pytool-extensions/) (with repos
+located [here](https://github.com/tianocore/edk2-pytool-library) and
+[here](https://github.com/tianocore/edk2-pytool-extensions)).
+
+The primary execution flows can be found in the
+`ci/AzurePipelines/Windows-VS2019.yml` and `ci/AzurePipelines/Ubuntu-GCC5.yml`
+files. These YAML files are consumed by the Azure Dev Ops Build Pipeline and
+dictate what server resources should be used, how they should be configured, and
+what processes should be run on them. An overview of this schema can be found
+[here](https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=schema).
+
+Inspection of these files reveals the EDKII Tools commands that make up the
+primary processes for the CI build: 'stuart_setup', 'stuart_update', and
+'stuart_ci_build'. These commands come from the EDKII Tools PIP modules and are
+configured as described below. More documentation on the stuart tools can be
+found [here](https://github.com/tianocore/edk2-pytool-extensions/blob/master/docs/using.md)
+and [here](https://github.com/tianocore/edk2-pytool-extensions/blob/master/docs/features/feature_invocables.md).
+
+## Configuration
+
+Configuration of the CI process consists of (in order of precedence):
+
+* command-line arguments passed in via the Pipeline YAML
+* a per-package configuration file (e.g. `<package-name>.ci.yaml`) that is
+ detected by the CI system in EDKII Tools.
+* a global configuration Python module (e.g. `CISetting.py`) passed in via the
+ command-line
+
+The global configuration file is described in
+[this readme](https://github.com/tianocore/edk2-pytool-extensions/blob/master/docs/usability/using_settings_manager.md)
+from the EDKII Tools documentation. This configuration is written as a Python
+module so that decisions can be made dynamically based on command line
+parameters and codebase state.
+
+The per-package configuration file can override most settings in the global
+configuration file, but is not dynamic. This file can be used to skip or
+customize tests that may be incompatible with a specific package. By default,
+the global configuration will try to run all tests on all packages.
+
+## Current PyTool Test Capabilities
+
+All CI tests are instances of EDKII Tools plugins. Documentation on the plugin
+system can be found [here](https://github.com/tianocore/edk2-pytool-extensions/blob/master/docs/usability/using_plugin_manager.md)
+and [here](https://github.com/tianocore/edk2-pytool-extensions/blob/master/docs/features/feature_plugin_manager.md).
+Upon invocation, each plugin will be passed the path to the current package
+under test and a dictionary containing its targeted configuration, as assembled
+from the command line, per-package configuration, and global configuration.
+
+Note: CI plugins are considered unique from build plugins and helper plugins,
+even though some CI plugins may execute steps of a build.
+
+In the example, these plugins live alongside the code under test (in the
+`ci/Plugin` directory), but may be moved to the 'edk2-test' repo if that
+location makes more sense for the community.
+
+### Module Inclusion Test - DscCompleteCheck
+
+This test scans all available modules (via INF files) and compares them to the
+package-level DSC file for the package each module is contained within. The test
+considers it an error if any module does not appear in the `Components` section
+of at least one package-level DSC (indicating that it would not be built if the
+package were built).
+
+### Code Compilation Test - CompilerPlugin
+
+Once the Module Inclusion Test has verified that all modules would be built if
+all package-level DSCs were built, the Code Compilation Test simply runs through
+and builds every package-level DSC on every toolchain and for every architecture
+that is supported. Any module that fails to build is considered an error.
+
+### GUID Uniqueness Test - GuidCheck
+
+This test works on the collection of all packages rather than an individual
+package. It looks at all FILE_GUIDs and GUIDs declared in DEC files and ensures
+that they are unique for the codebase. This prevents, for example, accidental
+duplication of GUIDs when using an existing INF as a template for a new module.
+
+### Cross-Package Dependency Test - DependencyCheck
+
+This test compares the list of all packages used in INFs files for a given
+package against a list of "allowed dependencies" in plugin configuration for
+that package. Any module that depends on a disallowed package will cause a test
+failure.
+
+### Library Declaration Test - LibraryClassCheck
+
+This test scans at all library header files found in the `Library` folders in
+all of the package's declared include directories and ensures that all files
+have a matching LibraryClass declaration in the DEC file for the package. Any
+missing declarations will cause a failure.
+
+### Invalid Character Test - CharEncodingCheck
+
+This test scans all files in a package to make sure that there are no invalid
+Unicode characters that may cause build errors in some character
+sets/localizations.
+
+### Spell Checking - cspell
+
+This test runs a spell checker on all files within the package. This is done
+using the NodeJs cspell tool. For details check `ci/Plugin/SpellCheck`. For
+this plugin to run during ci you must install nodejs and cspell and have both
+available to the command line when running your CI.
+
+Install
+
+* Install nodejs from https://nodejs.org/en/
+* Install cspell
+ 1. Open cmd prompt with access to node and npm
+ 2. Run `npm install -g cspell`
+
+ More cspell info: https://github.com/streetsidesoftware/cspell
+
+## Current Azure Pipeline Tests
+
+When adding a test it can be added as either a *PyTool* test or just added to
+the CI build process. This should be a deliberate choice. Any change added as
+a pipeline test is not as easily run on a private/local workspace. But there
+are times where this is still the preferred method.
+
+## PyTool Scopes
+
+Scopes are how the PyTool ext_dep, path_env, and plugins are activated. Meaning
+that if an invocable process has a scope active then those ext_dep and path_env
+will be active. To allow easy integration of PyTools capabilities there are a
+few standard scopes.
+
+| Scope | Invocable | Description |
+| :---- | :----- | :---- |
+| global | edk2_invocable++ - should be base_abstract_invocable | Running an invocables |
+| global-win | edk2_invocable++ | Running on Microsoft Windows |
+| global-nix | edk2_invocable++ | Running on Linux based OS |
+| edk2-build | | This indicates that an invocable is building EDK2 based UEFI code |
+| cibuild | set in .pytool/CISettings.py | Suggested target for edk2 continuous integration builds. Tools used for CiBuilds can use this scope. Example: asl compiler |
+
+## Future investments
+
+* PatchCheck tests as plugins
+* MacOS/xcode support
+* Clang/LLVM support
+* Visual Studio AARCH64 and ARM support
+* BaseTools C tools CI/PR and binary release process
+* BaseTools Python tools CI/PR process
+* Host based unit testing
+* Extensible private/closed source platform reporting
+* Platform builds, validation
+* UEFI SCTs
+* Other automation
--
2.21.0.windows.1
^ permalink raw reply related [flat|nested] 51+ messages in thread
* [Patch v4 20/22] .azurepipelines: Add Azure Pipelines YML configuration files
2019-11-07 1:13 [Patch v4 00/22] Enable Phase 1 of EDK II CI Michael D Kinney
` (18 preceding siblings ...)
2019-11-07 1:13 ` [Patch v4 19/22] .pytool: Add CISettings.py and Readme.md Michael D Kinney
@ 2019-11-07 1:13 ` Michael D Kinney
2019-11-07 1:13 ` [Patch v4 21/22] .mergify: Add Mergify YML pull request rules configuration file Michael D Kinney
` (2 subsequent siblings)
22 siblings, 0 replies; 51+ messages in thread
From: Michael D Kinney @ 2019-11-07 1:13 UTC (permalink / raw)
To: devel; +Cc: Sean Brogan, Bret Barkelew, Liming Gao
From: Sean Brogan <sean.brogan@microsoft.com>
Add YML configuration files used to run the EDK II Continuous
Integration (CI) checks on Azure Pipelines agents.
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
---
.azurepipelines/Ubuntu-GCC5.yml | 18 +++
.azurepipelines/Ubuntu-PatchCheck.yml | 32 +++++
.azurepipelines/Windows-VS2019.yml | 18 +++
.../templates/basetools-build-steps.yml | 36 +++++
.../templates/pr-gate-build-job.yml | 79 +++++++++++
.azurepipelines/templates/pr-gate-steps.yml | 129 ++++++++++++++++++
.../templates/spell-check-prereq-steps.yml | 21 +++
7 files changed, 333 insertions(+)
create mode 100644 .azurepipelines/Ubuntu-GCC5.yml
create mode 100644 .azurepipelines/Ubuntu-PatchCheck.yml
create mode 100644 .azurepipelines/Windows-VS2019.yml
create mode 100644 .azurepipelines/templates/basetools-build-steps.yml
create mode 100644 .azurepipelines/templates/pr-gate-build-job.yml
create mode 100644 .azurepipelines/templates/pr-gate-steps.yml
create mode 100644 .azurepipelines/templates/spell-check-prereq-steps.yml
diff --git a/.azurepipelines/Ubuntu-GCC5.yml b/.azurepipelines/Ubuntu-GCC5.yml
new file mode 100644
index 0000000000..191bda8a91
--- /dev/null
+++ b/.azurepipelines/Ubuntu-GCC5.yml
@@ -0,0 +1,18 @@
+##
+# Azure Pipeline build file for a build using ubuntu and GCC5
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+trigger:
+- master
+pr:
+- master
+
+jobs:
+- template: templates/pr-gate-build-job.yml
+ parameters:
+ tool_chain_tag: 'GCC5'
+ vm_image: 'ubuntu-latest'
+ arch_list: "IA32,X64,ARM,AARCH64"
+
diff --git a/.azurepipelines/Ubuntu-PatchCheck.yml b/.azurepipelines/Ubuntu-PatchCheck.yml
new file mode 100644
index 0000000000..0d91d78fb2
--- /dev/null
+++ b/.azurepipelines/Ubuntu-PatchCheck.yml
@@ -0,0 +1,32 @@
+## @file
+# Azure Pipielines YML file that evalues the patch series in a PR using the
+# python script BaseTools/Scripts/PatchCheck.py.
+#
+# Copyright (c) 2019, Intel Corporation. All rights reserved.<BR>
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+#
+# https://github.com/tianocore
+#
+##
+
+trigger: none
+
+pr:
+- master
+
+pool:
+ vmImage: 'ubuntu-latest'
+
+steps:
+- checkout: self
+ clean: true
+
+- task: UsePythonVersion@0
+ inputs:
+ versionSpec: '3.7.x'
+ architecture: 'x64'
+
+- script: |
+ git fetch origin $(System.PullRequest.TargetBranch):$(System.PullRequest.TargetBranch)
+ python BaseTools/Scripts/PatchCheck.py $(System.PullRequest.TargetBranch)..$(System.PullRequest.SourceCommitId)
+ displayName: 'Use PatchCheck.py to verify patch series in pull request'
diff --git a/.azurepipelines/Windows-VS2019.yml b/.azurepipelines/Windows-VS2019.yml
new file mode 100644
index 0000000000..2143162e02
--- /dev/null
+++ b/.azurepipelines/Windows-VS2019.yml
@@ -0,0 +1,18 @@
+##
+# Azure Pipeline build file for a build using Windows and VS2019
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+trigger:
+- master
+
+pr:
+- master
+
+jobs:
+- template: templates/pr-gate-build-job.yml
+ parameters:
+ tool_chain_tag: 'VS2019'
+ vm_image: 'windows-latest'
+ arch_list: "IA32,X64"
diff --git a/.azurepipelines/templates/basetools-build-steps.yml b/.azurepipelines/templates/basetools-build-steps.yml
new file mode 100644
index 0000000000..dad59e2a1f
--- /dev/null
+++ b/.azurepipelines/templates/basetools-build-steps.yml
@@ -0,0 +1,36 @@
+# File templates/basetools-build-job.yml
+#
+# template file to build basetools
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+parameters:
+ tool_chain_tag: ''
+
+steps:
+- ${{ if contains(parameters.tool_chain_tag, 'GCC') }}:
+ - bash: sudo apt-get update
+ displayName: Update apt
+ condition: and(gt(variables.pkg_count, 0), succeeded())
+
+ - bash: sudo apt-get install gcc g++ make uuid-dev
+ displayName: Install required tools
+ condition: and(gt(variables.pkg_count, 0), succeeded())
+
+- task: CmdLine@1
+ displayName: Build Base Tools from source
+ inputs:
+ filename: python
+ arguments: BaseTools/Edk2ToolsBuild.py -t ${{ parameters.tool_chain_tag }}
+ condition: and(gt(variables.pkg_count, 0), succeeded())
+
+- task: CopyFiles@2
+ displayName: "Copy base tools build log"
+ inputs:
+ targetFolder: '$(Build.ArtifactStagingDirectory)'
+ SourceFolder: 'BaseTools/BaseToolsBuild'
+ contents: |
+ BASETOOLS_BUILD*.*
+ flattenFolders: true
+ condition: and(gt(variables.pkg_count, 0), succeededOrFailed())
diff --git a/.azurepipelines/templates/pr-gate-build-job.yml b/.azurepipelines/templates/pr-gate-build-job.yml
new file mode 100644
index 0000000000..523ce77544
--- /dev/null
+++ b/.azurepipelines/templates/pr-gate-build-job.yml
@@ -0,0 +1,79 @@
+# File templates/pr-gate-build-job.yml
+#
+# template file used to build supported packages.
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+
+parameters:
+ tool_chain_tag: ''
+ vm_image: ''
+ arch_list: ''
+
+# Build step
+jobs:
+
+- job: Build_${{ parameters.tool_chain_tag }}
+
+ #Use matrix to speed up the build process
+ strategy:
+ matrix:
+ TARGET_MDE_CPU:
+ Build.Pkgs: 'MdePkg,UefiCpuPkg'
+ Build.Targets: 'DEBUG,RELEASE,NO-TARGET'
+ TARGET_MDEMODULE_DEBUG:
+ Build.Pkgs: 'MdeModulePkg'
+ Build.Targets: 'DEBUG'
+ TARGET_MDEMODULE_RELEASE:
+ Build.Pkgs: 'MdeModulePkg'
+ Build.Targets: 'RELEASE,NO-TARGET'
+ TARGET_NETWORK:
+ Build.Pkgs: 'NetworkPkg'
+ Build.Targets: 'DEBUG,RELEASE,NO-TARGET'
+ TARGET_OTHER:
+ Build.Pkgs: 'PcAtChipsetPkg,ShellPkg'
+ Build.Targets: 'DEBUG,RELEASE,NO-TARGET'
+ TARGET_FMP:
+ Build.Pkgs: 'FmpDevicePkg,FatPkg'
+ Build.Targets: 'DEBUG,RELEASE,NO-TARGET'
+ TARGET_CRYPTO:
+ Build.Pkgs: 'CryptoPkg'
+ Build.Targets: 'DEBUG,RELEASE,NO-TARGET'
+ TARGET_SECURITY:
+ Build.Pkgs: 'SecurityPkg'
+ Build.Targets: 'DEBUG,RELEASE,NO-TARGET'
+
+ workspace:
+ clean: all
+
+ pool:
+ vmImage: ${{ parameters.vm_image }}
+
+ steps:
+ - template: pr-gate-steps.yml
+ parameters:
+ tool_chain_tag: ${{ parameters.tool_chain_tag }}
+ build_pkgs: $(Build.Pkgs)
+ build_targets: $(Build.Targets)
+ build_archs: ${{ parameters.arch_list }}
+
+- job: FINISHED
+ dependsOn: Build_${{ parameters.tool_chain_tag }}
+ condition: succeeded()
+ steps:
+ - checkout: none
+ - script: |
+ echo FINISHED
+ sleep 10
+ displayName: FINISHED
+
+- job: FAILED
+ dependsOn: Build_${{ parameters.tool_chain_tag }}
+ condition: failed()
+ steps:
+ - checkout: none
+ - script: |
+ echo FAILED
+ sleep 10
+ displayName: FAILED
diff --git a/.azurepipelines/templates/pr-gate-steps.yml b/.azurepipelines/templates/pr-gate-steps.yml
new file mode 100644
index 0000000000..d3c7eec4ad
--- /dev/null
+++ b/.azurepipelines/templates/pr-gate-steps.yml
@@ -0,0 +1,129 @@
+# File templates/pr-gate-steps.yml
+#
+# template file containing the steps to build
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+
+parameters:
+ tool_chain_tag: ''
+ build_pkgs: ''
+ build_targets: ''
+ build_archs: ''
+
+steps:
+- checkout: self
+ clean: true
+ fetchDepth: 1
+
+- task: UsePythonVersion@0
+ inputs:
+ versionSpec: '3.7.x'
+ architecture: 'x64'
+
+- script: pip install -r requirements.txt --upgrade
+ displayName: 'Install/Upgrade pip modules'
+
+# Set default
+- bash: |
+ echo "##vso[task.setvariable variable=pkgs_to_build]${{ parameters.build_pkgs }}"
+ echo "##vso[task.setvariable variable=pkg_count]${{ 1 }}"
+
+# trim the package list if this is a PR
+- task: CmdLine@1
+ displayName: Check if ${{ parameters.build_pkgs }} need testing
+ inputs:
+ filename: stuart_pr_eval
+ arguments: -c .pytool/CISettings.py -p ${{ parameters.build_pkgs }} --pr-target origin/$(System.PullRequest.targetBranch) --output-csv-format-string "##vso[task.setvariable variable=pkgs_to_build;isOutpout=true]{pkgcsv}" --output-count-format-string "##vso[task.setvariable variable=pkg_count;isOutpout=true]{pkgcount}"
+ condition: eq(variables['Build.Reason'], 'PullRequest')
+
+# build basetools
+- template: basetools-build-steps.yml
+ parameters:
+ tool_chain_tag: ${{ parameters.tool_chain_tag }}
+
+# install spell check prereqs
+- template: spell-check-prereq-steps.yml
+
+# Build repo
+- task: CmdLine@1
+ displayName: Setup ${{ parameters.build_pkgs }} ${{ parameters.build_archs}}
+ inputs:
+ filename: stuart_setup
+ arguments: -c .pytool/CISettings.py -p $(pkgs_to_build) -t ${{ parameters.build_targets}} -a ${{ parameters.build_archs}} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}}
+ condition: and(gt(variables.pkg_count, 0), succeeded())
+
+- task: CmdLine@1
+ displayName: Update ${{ parameters.build_pkgs }} ${{ parameters.build_archs}}
+ inputs:
+ filename: stuart_update
+ arguments: -c .pytool/CISettings.py -p $(pkgs_to_build) -t ${{ parameters.build_targets}} -a ${{ parameters.build_archs}} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}}
+ condition: and(gt(variables.pkg_count, 0), succeeded())
+
+- task: CmdLine@1
+ displayName: Build and Test ${{ parameters.build_pkgs }} ${{ parameters.build_archs}}
+ inputs:
+ filename: stuart_ci_build
+ arguments: -c .pytool/CISettings.py -p $(pkgs_to_build) -t ${{ parameters.build_targets}} -a ${{ parameters.build_archs}} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}}
+ condition: and(gt(variables.pkg_count, 0), succeeded())
+
+# Publish Test Results to Azure Pipelines/TFS
+- task: PublishTestResults@2
+ displayName: 'Publish junit test results'
+ continueOnError: true
+ condition: and( succeededOrFailed(),gt(variables.pkg_count, 0))
+ inputs:
+ testResultsFormat: 'JUnit' # Options: JUnit, NUnit, VSTest, xUnit
+ testResultsFiles: 'Build/TestSuites.xml'
+ #searchFolder: '$(System.DefaultWorkingDirectory)' # Optional
+ mergeTestResults: true # Optional
+ testRunTitle: $(System.JobName) # Optional
+ #buildPlatform: # Optional
+ #buildConfiguration: # Optional
+ publishRunAttachments: true # Optional
+
+# Publish Test Results to Azure Pipelines/TFS
+- task: PublishTestResults@2
+ displayName: 'Publish host based test results for $(System.JobName)'
+ continueOnError: true
+ condition: and( succeededOrFailed(), gt(variables.pkg_count, 0))
+ inputs:
+ testResultsFormat: 'JUnit' # Options: JUnit, NUnit, VSTest, xUnit
+ testResultsFiles: 'Build/**/*.result.xml'
+ #searchFolder: '$(System.DefaultWorkingDirectory)' # Optional
+ mergeTestResults: false # Optional
+ testRunTitle: ${{ parameters.build_pkgs }} # Optional
+ #buildPlatform: # Optional
+ #buildConfiguration: # Optional
+ publishRunAttachments: true # Optional
+
+# Copy the build logs to the artifact staging directory
+- task: CopyFiles@2
+ displayName: "Copy build logs"
+ inputs:
+ targetFolder: '$(Build.ArtifactStagingDirectory)'
+ SourceFolder: 'Build'
+ contents: |
+ BUILDLOG_*.txt
+ BUILDLOG_*.md
+ CI_*.txt
+ CI_*.md
+ CISETUP.txt
+ SETUPLOG.txt
+ UPDATE_LOG.txt
+ PREVALLOG.txt
+ TestSuites.xml
+ **/BUILD_TOOLS_REPORT.html
+ **/OVERRIDELOG.TXT
+ flattenFolders: true
+ condition: succeededOrFailed()
+
+# Publish build artifacts to Azure Artifacts/TFS or a file share
+- task: PublishBuildArtifacts@1
+ continueOnError: true
+ displayName: "Publish build logs"
+ inputs:
+ pathtoPublish: '$(Build.ArtifactStagingDirectory)'
+ artifactName: 'Build Logs $(System.JobName)'
+ condition: succeededOrFailed()
diff --git a/.azurepipelines/templates/spell-check-prereq-steps.yml b/.azurepipelines/templates/spell-check-prereq-steps.yml
new file mode 100644
index 0000000000..7d7797ffed
--- /dev/null
+++ b/.azurepipelines/templates/spell-check-prereq-steps.yml
@@ -0,0 +1,21 @@
+# File templates/spell-check-prereq-steps.yml
+#
+# template file used to install spell checking prerequisits
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+
+parameters:
+ none: ''
+
+steps:
+- task: NodeTool@0
+ inputs:
+ versionSpec: '10.x'
+ #checkLatest: false # Optional
+ condition: and(gt(variables.pkg_count, 0), succeeded())
+
+- script: npm install -g cspell
+ displayName: 'Install cspell npm'
+ condition: and(gt(variables.pkg_count, 0), succeeded())
--
2.21.0.windows.1
^ permalink raw reply related [flat|nested] 51+ messages in thread
* [Patch v4 21/22] .mergify: Add Mergify YML pull request rules configuration file
2019-11-07 1:13 [Patch v4 00/22] Enable Phase 1 of EDK II CI Michael D Kinney
` (19 preceding siblings ...)
2019-11-07 1:13 ` [Patch v4 20/22] .azurepipelines: Add Azure Pipelines YML configuration files Michael D Kinney
@ 2019-11-07 1:13 ` Michael D Kinney
2019-11-07 1:13 ` [Patch v4 22/22] Readme.md: Add CI build status badges Michael D Kinney
2019-11-07 15:35 ` [Patch v4 00/22] Enable Phase 1 of EDK II CI Liming Gao
22 siblings, 0 replies; 51+ messages in thread
From: Michael D Kinney @ 2019-11-07 1:13 UTC (permalink / raw)
To: devel; +Cc: Liming Gao, Sean Brogan, Bret Barkelew
Add directory for the Mergify YML configuration files that
provides rules and actions used to process a pull request.
* Auto commit a PR from EDK II Maintainer with 'push' label
set and all CI checks pass
* Auto close a PR from any developers without 'push' label
set and all CI checks pass.
* Auto close a PR from a non EDK II Maintainer that has
the 'push' label set.
* Post a comment to a PR that has a merge conflict.
Submitter can resolved conflicts and reopen the PR.
* Post a comment to a PR that fails PatchCheck.py
Submitter can resolve PatchCheck.py issues and
reopen the PR.
Cc: Liming Gao <liming.gao@intel.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
---
.mergify/config.yml | 97 +++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 97 insertions(+)
create mode 100644 .mergify/config.yml
diff --git a/.mergify/config.yml b/.mergify/config.yml
new file mode 100644
index 0000000000..26583de0f6
--- /dev/null
+++ b/.mergify/config.yml
@@ -0,0 +1,97 @@
+## @file
+# Mergify YML file that automatically merges a GitHub pull request against
+# edk2-ci if all of the GitHub branch protections have passed. It also
+# contains rules to:
+# * auto close branches that are not from an EDK II Maintainer
+# * post a comment on pull requests that have merge conflicts.
+# * post a comment on pull requests that have PatchCheck.py errors.
+#
+# Configuration Notes:
+# * Update the 'base=edk2-ci' statements with the name of the branch to merge
+# pull requests.
+#
+# * Update the 'status-failure' statement with the name of the name of the Azure
+# Pipelines Build that performs the EDK II Maintainer check.
+#
+# * This file must be checked into the 'default' branch of a repo. Copies
+# of this file on other branches of a repo are ignored by Mergify.
+#
+# Copyright (c) 2019, Intel Corporation. All rights reserved.<BR>
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+#
+# https://github.com/apps/mergify
+# https://doc.mergify.io/
+#
+##
+
+pull_request_rules:
+
+ - name: Automatically merge a PR when all required checks pass and 'push' label is present
+ conditions:
+ - base=master
+ - label=push
+ - author=@tianocore/edk-ii-maintainers
+ - status-success=tianocore.PatchCheck
+ - status-success=Ubuntu GCC5 PR
+ - status-success=Windows VS2019 PR
+ actions:
+ merge:
+ strict: true
+ method: rebase
+
+ - name: Automatically close a PR when all required checks pass and 'push' label is not present
+ conditions:
+ - base=master
+ - -label=push
+ - -closed
+ - status-success=tianocore.PatchCheck
+ - status-success=Ubuntu GCC5 PR
+ - status-success=Windows VS2019 PR
+ - status-success=Ubuntu GCC5 PR (FINISHED)
+ - status-success=Windows VS2019 PR (FINISHED)
+ actions:
+ close:
+ message: All checks passed. Auto close personal build.
+
+ - name: Post a comment on a PR that can not be merged due to a merge conflict
+ conditions:
+ - base=master
+ - conflict
+ actions:
+ comment:
+ message: PR can not be merged due to conflict. Please rebase and resubmit
+
+ - name: Automatically close a PR that fails the EDK II Maintainers membership check and 'push' label is present
+ conditions:
+ - base=master
+ - label=push
+ - -author=@tianocore/edk-ii-maintainers
+ actions:
+ close:
+ message: PR submitter is not a member of the Tianocore EDK II Maintainers team
+
+ - name: Post a comment on a PR if PatchCheck fails
+ conditions:
+ - base=master
+ - status-failure=tianocore.PatchCheck
+ actions:
+ comment:
+ message: PR can not be merged due to a PatchCheck failure. Please resolve and resubmit
+
+ - name: Post a comment on a PR if Ubuntu GCC5 fails
+ conditions:
+ - base=master
+ - status-failure=Ubuntu GCC5 PR
+ - status-success=Ubuntu GCC5 PR (FAILED)
+ actions:
+ comment:
+ message: PR can not be merged due to an Ubuntu GCC5 failure. Please resolve and resubmit
+
+ - name: Post a comment on a PR if Windows VS2019 fails
+ conditions:
+ - base=master
+ - status-failure=Windows VS2019 PR
+ - status-success=Windows VS2019 PR (FAILED)
+ actions:
+ comment:
+ message: PR can not be merged due to a Windows VS2019 failure. Please resolve and resubmit
--
2.21.0.windows.1
^ permalink raw reply related [flat|nested] 51+ messages in thread
* [Patch v4 22/22] Readme.md: Add CI build status badges
2019-11-07 1:13 [Patch v4 00/22] Enable Phase 1 of EDK II CI Michael D Kinney
` (20 preceding siblings ...)
2019-11-07 1:13 ` [Patch v4 21/22] .mergify: Add Mergify YML pull request rules configuration file Michael D Kinney
@ 2019-11-07 1:13 ` Michael D Kinney
2019-11-07 10:44 ` Laszlo Ersek
2019-11-07 16:00 ` Leif Lindholm
2019-11-07 15:35 ` [Patch v4 00/22] Enable Phase 1 of EDK II CI Liming Gao
22 siblings, 2 replies; 51+ messages in thread
From: Michael D Kinney @ 2019-11-07 1:13 UTC (permalink / raw)
To: devel; +Cc: Andrew Fish, Laszlo Ersek, Leif Lindholm
Update Readme.md with status badges from the most recent
EDK II Continuous Integration(CI) build.
Cc: Andrew Fish <afish@apple.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Leif Lindholm <leif.lindholm@linaro.org>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
---
Readme.md | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/Readme.md b/Readme.md
index 27e4ce0771..0207923763 100644
--- a/Readme.md
+++ b/Readme.md
@@ -3,6 +3,17 @@
A modern, feature-rich, cross-platform firmware development environment
for the UEFI and PI specifications from www.uefi.org.
+# Build Status
+
+| Host Type | Toolchain | Branch | Build Status | Test Status | Code Coverage |
+| :-------- | :-------- | :------ | :----------- | :---------- | :------------ |
+| Windows | VS2019 | master | [](https://dev.azure.com/tianocore/edk2-ci-play/_build/latest?definitionId=27&branchName=master) | [](https://dev.azure.com/tianocore/edk2-ci-play/_build/latest?definitionId=27&branchName=master) | [](https://dev.azure.com/tianocore/edk2-ci-play/_build/latest?definitionId=27&branchName=master) |
+| Ubuntu | GCC | master | [](https://dev.azure.com/tianocore/edk2-ci-play/_build/latest?definitionId=26&branchName=master) | [](https://dev.azure.com/tianocore/edk2-ci-play/_build/latest?definitionId=26&branchName=master) | [](https://dev.azure.com/tianocore/edk2-ci-play/_build/latest?definitionId=26&branchName=master) |
+
+[More CI Build information](.pytool/Readme.md)
+
+# License Details
+
The majority of the content in the EDK II open source project uses a
[BSD-2-Clause Plus Patent License](License.txt). The EDK II open source project
contains the following components that are covered by additional licenses:
--
2.21.0.windows.1
^ permalink raw reply related [flat|nested] 51+ messages in thread
* Re: [Patch v4 14/22] PcAtChipsetPkg: Add YAML files for CI builds
2019-11-07 1:13 ` [Patch v4 14/22] PcAtChipsetPkg: Add YAML files " Michael D Kinney
@ 2019-11-07 2:12 ` Ni, Ray
0 siblings, 0 replies; 51+ messages in thread
From: Ni, Ray @ 2019-11-07 2:12 UTC (permalink / raw)
To: Kinney, Michael D, devel@edk2.groups.io
Acked-by: Ray Ni <ray.ni@intel.com>
> -----Original Message-----
> From: Kinney, Michael D <michael.d.kinney@intel.com>
> Sent: Thursday, November 7, 2019 9:14 AM
> To: devel@edk2.groups.io
> Cc: Ni, Ray <ray.ni@intel.com>
> Subject: [Patch v4 14/22] PcAtChipsetPkg: Add YAML files for CI builds
>
> Add YAML file to the package directory with the
> configuration of the checks to perform during a
> CI build.
>
> Cc: Ray Ni <ray.ni@intel.com>
> Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
> ---
> PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml | 46 +++++++++++++++++++++++++++
> 1 file changed, 46 insertions(+)
> create mode 100644 PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml
>
> diff --git a/PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml b/PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml
> new file mode 100644
> index 0000000000..6ab9f11761
> --- /dev/null
> +++ b/PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml
> @@ -0,0 +1,46 @@
> +##
> +# CI configuration for PcAtChipsetPkg
> +#
> +# Copyright (c) Microsoft Corporation
> +# SPDX-License-Identifier: BSD-2-Clause-Patent
> +##
> +{
> + "CompilerPlugin": {
> + "DscPath": "PcAtChipsetPkg.dsc"
> + },
> + "CharEncodingCheck": {
> + "IgnoreFiles": []
> + },
> + "DependencyCheck": {
> + "AcceptableDependencies": [
> + "MdePkg/MdePkg.dec",
> + "PcAtChipsetPkg/PcAtChipsetPkg.dec",
> + "UefiCpuPkg/UefiCpuPkg.dec"
> + ],
> + # For host based unit tests
> + "AcceptableDependencies-HOST_APPLICATION":[],
> + # For UEFI shell based apps
> + "AcceptableDependencies-UEFI_APPLICATION":[],
> + "IgnoreInf": []
> + },
> + "DscCompleteCheck": {
> + "DscPath": "PcAtChipsetPkg.dsc",
> + "IgnoreInf": []
> + },
> + "GuidCheck": {
> + "IgnoreGuidName": [],
> + "IgnoreGuidValue": [],
> + "IgnoreFoldersAndFiles": [],
> + "IgnoreDuplicates": []
> + },
> + "LibraryClassCheck": {
> + "IgnoreHeaderFile": []
> + },
> + "SpellCheck": {
> + "ExtendWords": [
> + "ENUMER", # this is part of an IDE enum
> + "PCATCHIPSET",
> + "TXRDY"
> + ]
> + }
> +}
> --
> 2.21.0.windows.1
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [Patch v4 17/22] UefiCpuPkg: Add YAML file for CI builds
2019-11-07 1:13 ` [Patch v4 17/22] UefiCpuPkg: " Michael D Kinney
@ 2019-11-07 2:12 ` Ni, Ray
2019-11-07 10:42 ` Laszlo Ersek
1 sibling, 0 replies; 51+ messages in thread
From: Ni, Ray @ 2019-11-07 2:12 UTC (permalink / raw)
To: Kinney, Michael D, devel@edk2.groups.io; +Cc: Dong, Eric, Laszlo Ersek
Acked-by: Ray Ni <ray.ni@intel.com>
> -----Original Message-----
> From: Kinney, Michael D <michael.d.kinney@intel.com>
> Sent: Thursday, November 7, 2019 9:14 AM
> To: devel@edk2.groups.io
> Cc: Dong, Eric <eric.dong@intel.com>; Ni, Ray <ray.ni@intel.com>; Laszlo Ersek <lersek@redhat.com>
> Subject: [Patch v4 17/22] UefiCpuPkg: Add YAML file for CI builds
>
> Add YAML file to the package directory with the
> configuration of the checks to perform during a
> CI build.
>
> Cc: Eric Dong <eric.dong@intel.com>
> Cc: Ray Ni <ray.ni@intel.com>
> Cc: Laszlo Ersek <lersek@redhat.com>
> Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
> Acked-by: Laszlo Ersek <lersek@redhat.com>
> ---
> UefiCpuPkg/UefiCpuPkg.ci.yaml | 51 +++++++++++++++++++++++++++++++++++
> 1 file changed, 51 insertions(+)
> create mode 100644 UefiCpuPkg/UefiCpuPkg.ci.yaml
>
> diff --git a/UefiCpuPkg/UefiCpuPkg.ci.yaml b/UefiCpuPkg/UefiCpuPkg.ci.yaml
> new file mode 100644
> index 0000000000..81da29b878
> --- /dev/null
> +++ b/UefiCpuPkg/UefiCpuPkg.ci.yaml
> @@ -0,0 +1,51 @@
> +##
> +# CI configuration for UefiCpuPkg
> +#
> +# Copyright (c) Microsoft Corporation
> +# SPDX-License-Identifier: BSD-2-Clause-Patent
> +##
> +{
> + "CompilerPlugin": {
> + "DscPath": "UefiCpuPkg.dsc"
> + },
> + "CharEncodingCheck": {
> + "IgnoreFiles": []
> + },
> + "DependencyCheck": {
> + "AcceptableDependencies": [
> + "MdePkg/MdePkg.dec",
> + "MdeModulePkg/MdeModulePkg.dec",
> + "UefiCpuPkg/UefiCpuPkg.dec"
> + ],
> + # For host based unit tests
> + "AcceptableDependencies-HOST_APPLICATION":[],
> + # For UEFI shell based apps
> + "AcceptableDependencies-UEFI_APPLICATION":[],
> + "IgnoreInf": []
> + },
> + "DscCompleteCheck": {
> + "DscPath": "UefiCpuPkg.dsc",
> + "IgnoreInf": [
> + "UefiCpuPkg/ResetVector/FixupVtf/Vtf.inf",
> + "UefiCpuPkg/ResetVector/Vtf0/Vtf0.inf"
> + ]
> + },
> + "GuidCheck": {
> + "IgnoreGuidName": ["SecCore", "ResetVector"], # Expected duplication for gEfiFirmwareVolumeTopFileGuid
> + "IgnoreGuidValue": [],
> + "IgnoreFoldersAndFiles": [],
> + "IgnoreDuplicates": []
> + },
> + "LibraryClassCheck": {
> + "IgnoreHeaderFile": []
> + },
> +
> + ## options defined ci/Plugin/SpellCheck
> + "SpellCheck": {
> + "AuditOnly": True, # Fails test but run in AuditOnly mode to collect log
> + "IgnoreFiles": [], # use gitignore syntax to ignore errors in matching files
> + "ExtendWords": [], # words to extend to the dictionary for this package
> + "IgnoreStandardPaths": [], # Standard Plugin defined paths that should be ignore
> + "AdditionalIncludePaths": [] # Additional paths to spell check (wildcards supported)
> + }
> +}
> --
> 2.21.0.windows.1
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [Patch v4 16/22] ShellPkg: Add YAML file for CI builds
2019-11-07 1:13 ` [Patch v4 16/22] ShellPkg: Add YAML file " Michael D Kinney
@ 2019-11-07 2:12 ` Ni, Ray
0 siblings, 0 replies; 51+ messages in thread
From: Ni, Ray @ 2019-11-07 2:12 UTC (permalink / raw)
To: Kinney, Michael D, devel@edk2.groups.io; +Cc: Gao, Zhichao
Acked-by: Ray Ni <ray.ni@intel.com>
> -----Original Message-----
> From: Kinney, Michael D <michael.d.kinney@intel.com>
> Sent: Thursday, November 7, 2019 9:14 AM
> To: devel@edk2.groups.io
> Cc: Ni, Ray <ray.ni@intel.com>; Gao, Zhichao <zhichao.gao@intel.com>
> Subject: [Patch v4 16/22] ShellPkg: Add YAML file for CI builds
>
> Add YAML file to the package directory with the
> configuration of the checks to perform during a
> CI build.
>
> Cc: Ray Ni <ray.ni@intel.com>
> Cc: Zhichao Gao <zhichao.gao@intel.com>
> Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
> Acked-by: Zhichao Gao <zhichao.gao@intel.com>
> ---
> ShellPkg/ShellPkg.ci.yaml | 55 +++++++++++++++++++++++++++++++++++++++
> 1 file changed, 55 insertions(+)
> create mode 100644 ShellPkg/ShellPkg.ci.yaml
>
> diff --git a/ShellPkg/ShellPkg.ci.yaml b/ShellPkg/ShellPkg.ci.yaml
> new file mode 100644
> index 0000000000..5bf7330c1a
> --- /dev/null
> +++ b/ShellPkg/ShellPkg.ci.yaml
> @@ -0,0 +1,55 @@
> +##
> +# CI configuration for ShellPkg
> +#
> +# Copyright (c) Microsoft Corporation
> +# SPDX-License-Identifier: BSD-2-Clause-Patent
> +##
> +{
> + "CompilerPlugin": {
> + "DscPath": "ShellPkg.dsc"
> + },
> + "CharEncodingCheck": {
> + "IgnoreFiles": []
> + },
> + "DependencyCheck": {
> + "AcceptableDependencies": [
> + "MdePkg/MdePkg.dec",
> + "MdeModulePkg/MdeModulePkg.dec",
> + "ShellPkg/ShellPkg.dec",
> + "NetworkPkg/NetworkPkg.dec"
> + ],
> + # For host based unit tests
> + "AcceptableDependencies-HOST_APPLICATION":[],
> + # For UEFI shell based apps
> + "AcceptableDependencies-UEFI_APPLICATION":[],
> + "IgnoreInf": []
> + },
> + "DscCompleteCheck": {
> + "DscPath": "ShellPkg.dsc",
> + "IgnoreInf": [
> + "ShellPkg/Application/ShellCTestApp/ShellCTestApp.inf",
> + "ShellPkg/Application/ShellExecTestApp/SA.inf",
> + "ShellPkg/Application/ShellSortTestApp/ShellSortTestApp.inf"
> + ]
> + },
> + "GuidCheck": {
> + "IgnoreGuidName": [],
> + "IgnoreGuidValue": [],
> + "IgnoreFoldersAndFiles": [],
> + "IgnoreDuplicates": [
> + "Shell=gUefiShellFileGuid", # by design
> + ]
> + },
> + "LibraryClassCheck": {
> + "IgnoreHeaderFile": []
> + },
> +
> + ## options defined ci/Plugin/SpellCheck
> + "SpellCheck": {
> + "AuditOnly": True, # Fails test but run in AuditOnly mode to collect log
> + "IgnoreFiles": [], # use gitignore syntax to ignore errors in matching files
> + "ExtendWords": [], # words to extend to the dictionary for this package
> + "IgnoreStandardPaths": [], # Standard Plugin defined paths that should be ignore
> + "AdditionalIncludePaths": [] # Additional paths to spell check (wildcards supported)
> + }
> +}
> --
> 2.21.0.windows.1
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [Patch v4 09/22] FatPkg: Add YAML file for CI builds
2019-11-07 1:13 ` [Patch v4 09/22] FatPkg: " Michael D Kinney
@ 2019-11-07 2:12 ` Ni, Ray
0 siblings, 0 replies; 51+ messages in thread
From: Ni, Ray @ 2019-11-07 2:12 UTC (permalink / raw)
To: Kinney, Michael D, devel@edk2.groups.io
Acked-by: Ray Ni <ray.ni@intel.com>
> -----Original Message-----
> From: Kinney, Michael D <michael.d.kinney@intel.com>
> Sent: Thursday, November 7, 2019 9:14 AM
> To: devel@edk2.groups.io
> Cc: Ni, Ray <ray.ni@intel.com>
> Subject: [Patch v4 09/22] FatPkg: Add YAML file for CI builds
>
> Add YAML file to the package directory with the
> configuration of the checks to perform during a
> CI build.
>
> Cc: Ray Ni <ray.ni@intel.com>
> Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
> ---
> FatPkg/FatPkg.ci.yaml | 50 +++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 50 insertions(+)
> create mode 100644 FatPkg/FatPkg.ci.yaml
>
> diff --git a/FatPkg/FatPkg.ci.yaml b/FatPkg/FatPkg.ci.yaml
> new file mode 100644
> index 0000000000..d584eea0a7
> --- /dev/null
> +++ b/FatPkg/FatPkg.ci.yaml
> @@ -0,0 +1,50 @@
> +##
> +# CI configuration for FatPkg
> +#
> +# Copyright (c) Microsoft Corporation
> +# SPDX-License-Identifier: BSD-2-Clause-Patent
> +##
> +{
> + "CompilerPlugin": {
> + "DscPath": "FatPkg.dsc"
> + },
> + "CharEncodingCheck": {
> + "IgnoreFiles": []
> + },
> + "DependencyCheck": {
> + "AcceptableDependencies": [
> + "MdePkg/MdePkg.dec",
> + "MdeModulePkg/MdeModulePkg.dec",
> + ],
> + # For host based unit tests
> + "AcceptableDependencies-HOST_APPLICATION":[],
> + # For UEFI shell based apps
> + "AcceptableDependencies-UEFI_APPLICATION":[],
> + "IgnoreInf": []
> + },
> + "DscCompleteCheck": {
> + "IgnoreInf": [],
> + "DscPath": "FatPkg.dsc"
> + },
> + "GuidCheck": {
> + "IgnoreGuidName": [],
> + "IgnoreGuidValue": [],
> + "IgnoreFoldersAndFiles": []
> + },
> + "LibraryClassCheck": {
> + "IgnoreHeaderFile": []
> + },
> + "SpellCheck": {
> + "ExtendWords": [
> + "ELTORITO",
> + "FHAND",
> + "IFILE",
> + "OFILE",
> + "FDISKed",
> + "Lfnbuffer",
> + "FFFFFFFFL",
> + "CDVOL",
> + "DMDEPKG"
> + ]
> + }
> +}
> --
> 2.21.0.windows.1
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [Patch v4 11/22] MdeModulePkg: Add YAML file for CI builds
2019-11-07 1:13 ` [Patch v4 11/22] MdeModulePkg: " Michael D Kinney
@ 2019-11-07 3:03 ` Wu, Hao A
2019-11-07 20:02 ` Michael D Kinney
0 siblings, 1 reply; 51+ messages in thread
From: Wu, Hao A @ 2019-11-07 3:03 UTC (permalink / raw)
To: Kinney, Michael D, devel@edk2.groups.io; +Cc: Wang, Jian J
> -----Original Message-----
> From: Kinney, Michael D
> Sent: Thursday, November 07, 2019 9:14 AM
> To: devel@edk2.groups.io
> Cc: Wang, Jian J; Wu, Hao A
> Subject: [Patch v4 11/22] MdeModulePkg: Add YAML file for CI builds
>
> Add YAML file to the package directory with the
> configuration of the checks to perform during a
> CI build.
>
> Cc: Jian J Wang <jian.j.wang@intel.com>
> Cc: Hao A Wu <hao.a.wu@intel.com>
> Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
> ---
> MdeModulePkg/MdeModulePkg.ci.yaml | 81
> +++++++++++++++++++++++++++++++
> 1 file changed, 81 insertions(+)
> create mode 100644 MdeModulePkg/MdeModulePkg.ci.yaml
>
> diff --git a/MdeModulePkg/MdeModulePkg.ci.yaml
> b/MdeModulePkg/MdeModulePkg.ci.yaml
> new file mode 100644
> index 0000000000..6ae58d5379
> --- /dev/null
> +++ b/MdeModulePkg/MdeModulePkg.ci.yaml
> @@ -0,0 +1,81 @@
> +##
> +# CI configuration for MdeModulePkg
> +#
> +# Copyright (c) Microsoft Corporation
> +# SPDX-License-Identifier: BSD-2-Clause-Patent
> +##
> +{
> + ## options defined ci/Plugin/CompilerPlugin
> + "CompilerPlugin": {
> + "DscPath": "MdeModulePkg.dsc"
> + },
> +
> + ## options defined ci/Plugin/CharEncodingCheck
> + "CharEncodingCheck": {
> + "IgnoreFiles": []
> + },
> +
> + ## options defined ci/Plugin/DependencyCheck
> + "DependencyCheck": {
> + "AcceptableDependencies": [
> + "MdePkg/MdePkg.dec",
> + "MdeModulePkg/MdeModulePkg.dec",
> + "StandaloneMmPkg/StandaloneMmPkg.dec",
> + "ArmPkg/ArmPkg.dec" # this should be fixed by promoting an
> abstraction
> + ],
> + # For host based unit tests
> + "AcceptableDependencies-HOST_APPLICATION":[],
> + # For UEFI shell based apps
> + "AcceptableDependencies-UEFI_APPLICATION":[],
> + "IgnoreInf": []
> + },
> +
> + ## options defined ci/Plugin/DscCompleteCheck
> + "DscCompleteCheck": {
> + "IgnoreInf": [],
> + "DscPath": "MdeModulePkg.dsc"
> + },
> +
> + ## options defined ci/Plugin/GuidCheck
> + "GuidCheck": {
> + "IgnoreGuidName": [],
> + "IgnoreGuidValue": ["00000000-0000-0000-0000-000000000000"],
> + "IgnoreFoldersAndFiles": [],
> + "IgnoreDuplicates": [
> +
> "gEdkiiFormBrowserExProtocolGuid=gEfiFormBrowserExProtocolGuid",
> + "gEfiPeiMmAccessPpiGuid=gPeiSmmAccessPpiGuid",
> + "gPeiSmmControlPpiGuid=gEfiPeiMmControlPpiGuid",
> + "gEdkiiSerialPortLibVendorGuid=SerialDxe" # Is this a bug????
Hello Mike,
I think the above GUID duplication (serial-related) has already been addressed
via commits:
SHA-1: 0d85e67714e31e0dbe4241ab2ebb7c423aba174d
* MdeModulePkg/SerialDxe: Update the file Guid in SerialDxe.inf
SHA-1: 9790f62be1aa5ee9460d4c4ec8c720919523bb62
* MdeModulePkg SerialDxe.inf: Fix wrong FILE_GUID format
Could you help to confirm whether the item can be removed?
With it handled,
Reviewed-by: Hao A Wu <hao.a.wu@intel.com>
Best Regards,
Hao Wu
> + ]
> + },
> +
> + ## options defined ci/Plugin/LibraryClassCheck
> + "LibraryClassCheck": {
> + "IgnoreHeaderFile": []
> + },
> +
> + ## options defined ci/Plugin/SpellCheck
> + "SpellCheck": {
> + "AuditOnly": True, # Fails test but run in AuditOnly mode to collect
> log
> + "IgnoreStandardPaths": [ # Standard Plugin defined paths that should
> be ignore
> + "*.c", "*.asm", "*.h", "*.nasm", "*.s", "*.asl", "*.inf"
> + ],
> + "IgnoreFiles": [ # use gitignore syntax to ignore errors in matching
> files
> + "Library/LzmaCustomDecompressLib/Sdk/DOC/*"
> + ],
> + "ExtendWords": [ # words to extend to the dictionary for this
> package
> + "LIGHTGRAY",
> + "DARKGRAY",
> + "LIGHTBLUE",
> + "LIGHTGREEN",
> + "LIGHTCYAN",
> + "LIGHTRED",
> + "LIGHTMAGENTA",
> + "FVMAIN",
> + "VARCHECKPCD",
> + "Getxx",
> + "lzturbo"
> + ],
> + "AdditionalIncludePaths": [] # Additional paths to spell check relative to
> package root (wildcards supported)
> + }
> +}
> --
> 2.21.0.windows.1
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [Patch v4 08/22] CryptoPkg: Add YAML file for CI builds
2019-11-07 1:13 ` [Patch v4 08/22] CryptoPkg: Add YAML file for CI builds Michael D Kinney
@ 2019-11-07 5:06 ` Wang, Jian J
0 siblings, 0 replies; 51+ messages in thread
From: Wang, Jian J @ 2019-11-07 5:06 UTC (permalink / raw)
To: Kinney, Michael D, devel@edk2.groups.io; +Cc: Lu, XiaoyuX
Hi Mike,
> -----Original Message-----
> From: Kinney, Michael D <michael.d.kinney@intel.com>
> Sent: Thursday, November 07, 2019 9:14 AM
> To: devel@edk2.groups.io
> Cc: Wang, Jian J <jian.j.wang@intel.com>; Lu, XiaoyuX <xiaoyux.lu@intel.com>;
> Kinney, Michael D <michael.d.kinney@intel.com>
> Subject: [Patch v4 08/22] CryptoPkg: Add YAML file for CI builds
>
> Add YAML file to the package directory with the
> configuration of the checks to perform during a
> CI build.
>
> Cc: Jian J Wang <jian.j.wang@intel.com>
> Cc: Xiaoyu Lu <xiaoyux.lu@intel.com>
> Signed-off-by: Kinney <michael.d.kinney@intel.com>
> ---
> CryptoPkg/CryptoPkg.ci.yaml | 48
> +++++++++++++++++++++++++++++++++++++
> 1 file changed, 48 insertions(+)
> create mode 100644 CryptoPkg/CryptoPkg.ci.yaml
>
> diff --git a/CryptoPkg/CryptoPkg.ci.yaml b/CryptoPkg/CryptoPkg.ci.yaml
> new file mode 100644
> index 0000000000..a20ac79d0e
> --- /dev/null
> +++ b/CryptoPkg/CryptoPkg.ci.yaml
> @@ -0,0 +1,48 @@
> +##
> +# CI configuration for CryptoPkg
> +#
> +# Copyright (c) Microsoft Corporation
> +# SPDX-License-Identifier: BSD-2-Clause-Patent
> +##
> +{
> + "CompilerPlugin": {
> + "DscPath": "CryptoPkg.dsc"
> + },
> + "CharEncodingCheck": {
> + "IgnoreFiles": []
> + },
> + "DependencyCheck": {
> + "AcceptableDependencies": [
> + "MdePkg/MdePkg.dec",
> + "MdeModulePkg/MdeModulePkg.dec",
> + "CryptoPkg/CryptoPkg.dec",
> + #"SecurityPkg/SecurityPkg.dec"
It'd be better to remove above line instead of commenting out. I don't
think CryptoPkg will depend on SecurityPkg anyway.
With it addressed,
Reviewed-by: Jian J Wang <jian.j.wang@intel.com>
Regards,
Jian
> + ],
> + # For host based unit tests
> + "AcceptableDependencies-HOST_APPLICATION":[],
> + # For UEFI shell based apps
> + "AcceptableDependencies-UEFI_APPLICATION":[],
> + "IgnoreInf": []
> + },
> + "DscCompleteCheck": {
> + "DscPath": "CryptoPkg.dsc",
> + "IgnoreInf": []
> + },
> + "GuidCheck": {
> + "IgnoreGuidName": [],
> + "IgnoreGuidValue": [],
> + "IgnoreFoldersAndFiles": []
> + },
> + "LibraryClassCheck": {
> + "IgnoreHeaderFile": []
> + },
> +
> + ## options defined ci/Plugin/SpellCheck
> + "SpellCheck": {
> + "skip": True,
> + "IgnoreFiles": [], # use gitignore syntax to ignore errors in matching
> files
> + "ExtendWords": [], # words to extend to the dictionary for this
> package
> + "IgnoreStandardPaths": [], # Standard Plugin defined paths that should be
> ignore
> + "AdditionalIncludePaths": [] # Additional paths to spell check (wildcards
> supported)
> + }
> +}
> --
> 2.21.0.windows.1
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [Patch v4 15/22] SecurityPkg: Add YAML files for CI builds
2019-11-07 1:13 ` [Patch v4 15/22] SecurityPkg: " Michael D Kinney
@ 2019-11-07 5:08 ` Wang, Jian J
0 siblings, 0 replies; 51+ messages in thread
From: Wang, Jian J @ 2019-11-07 5:08 UTC (permalink / raw)
To: Kinney, Michael D, devel@edk2.groups.io; +Cc: Yao, Jiewen, Zhang, Chao B
Reviewed-by: Jian J Wang <jian.j.wang@intel.com>
Regards,
Jian
> -----Original Message-----
> From: Kinney, Michael D <michael.d.kinney@intel.com>
> Sent: Thursday, November 07, 2019 9:14 AM
> To: devel@edk2.groups.io
> Cc: Yao, Jiewen <jiewen.yao@intel.com>; Wang, Jian J <jian.j.wang@intel.com>;
> Zhang, Chao B <chao.b.zhang@intel.com>
> Subject: [Patch v4 15/22] SecurityPkg: Add YAML files for CI builds
>
> Add YAML file to the package directory with the
> configuration of the checks to perform during a
> CI build.
>
> Use BaseCryptLibNull for package CI builds to reduce package
> build times. Enabled with CONTINUOUS_INTEGRATION in YAML
> files. By default CONTINUOUS_INTEGRATION is not defined,
> and the original lib mappings are preserved.
>
> Cc: Jiewen Yao <jiewen.yao@intel.com>
> Cc: Jian J Wang <jian.j.wang@intel.com>
> Cc: Chao Zhang <chao.b.zhang@intel.com>
> Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
> ---
> SecurityPkg/SecurityPkg.ci.yaml | 80 +++++++++++++++++++++++++++++++++
> SecurityPkg/SecurityPkg.dsc | 32 ++++++++++++-
> 2 files changed, 110 insertions(+), 2 deletions(-)
> create mode 100644 SecurityPkg/SecurityPkg.ci.yaml
>
> diff --git a/SecurityPkg/SecurityPkg.ci.yaml b/SecurityPkg/SecurityPkg.ci.yaml
> new file mode 100644
> index 0000000000..c7e309fcda
> --- /dev/null
> +++ b/SecurityPkg/SecurityPkg.ci.yaml
> @@ -0,0 +1,80 @@
> +##
> +# CI configuration for SecurityPkg
> +#
> +# Copyright (c) Microsoft Corporation
> +# SPDX-License-Identifier: BSD-2-Clause-Patent
> +##
> +{
> + "CompilerPlugin": {
> + "DscPath": "SecurityPkg.dsc"
> + },
> + "CharEncodingCheck": {
> + "IgnoreFiles": []
> + },
> + "DependencyCheck": {
> + "AcceptableDependencies": [
> + "MdePkg/MdePkg.dec",
> + "MdeModulePkg/MdeModulePkg.dec",
> + "SecurityPkg/SecurityPkg.dec",
> + "CryptoPkg/CryptoPkg.dec"
> + ],
> + # For host based unit tests
> + "AcceptableDependencies-HOST_APPLICATION":[],
> + # For UEFI shell based apps
> + "AcceptableDependencies-UEFI_APPLICATION":[],
> + "IgnoreInf": []
> + },
> + "DscCompleteCheck": {
> + "DscPath": "SecurityPkg.dsc",
> + "IgnoreInf": []
> + },
> + "GuidCheck": {
> + "IgnoreGuidName": [],
> + "IgnoreGuidValue": ["00000000-0000-0000-0000-000000000000"],
> + "IgnoreFoldersAndFiles": [],
> + "IgnoreDuplicates": [
> + "Tpm2InstanceLibDTpm=gEfiTpmDeviceInstanceTpm20DtpmGuid", # by
> design
> + ]
> + },
> + "LibraryClassCheck": {
> + "IgnoreHeaderFile": []
> + },
> +
> + ## options defined ci/Plugin/SpellCheck
> + "SpellCheck": {
> + "AuditOnly": True, # Fails test but run in AuditOnly mode to collect log
> + "ExtendWords": [ # words to extend to the dictionary for this
> package
> + "shortformed", # tpm acpi
> + "autodetect",
> + "blocksid",
> + "comid",
> + "cpinsidpin", #OpalSScV2
> + "ecdsa", # TPM
> + "ecschnorr", # TPM
> + "eisaid", # ACPI
> + "harddisk",
> + "hashall",
> + "hashto",
> + "kek's",
> + "lfanew", # PE/COFF
> + "pcrindex",
> + "pkglength",
> + "ppuser",
> + "preos",
> + "stclear",
> + "toctou",
> + "tpm's",
> + "tpmcmdbuflength",
> + "tpmcommlib",
> + "tpmnvvaluelength",
> + "wrlocked",
> + "xored"
> + ],
> + "IgnoreStandardPaths": [], # Standard Plugin defined paths that should be
> ignore
> + "AdditionalIncludePaths": [] # Additional paths to spell check (wildcards
> supported)
> + },
> +
> + "Defines": {
> + "BLD_*_CONTINUOUS_INTEGRATION": "TRUE",
> + }
> +}
> diff --git a/SecurityPkg/SecurityPkg.dsc b/SecurityPkg/SecurityPkg.dsc
> index 9a254087a3..a2eeadda7a 100644
> --- a/SecurityPkg/SecurityPkg.dsc
> +++ b/SecurityPkg/SecurityPkg.dsc
> @@ -44,8 +44,6 @@ [LibraryClasses]
> HiiLib|MdeModulePkg/Library/UefiHiiLib/UefiHiiLib.inf
>
> UefiHiiServicesLib|MdeModulePkg/Library/UefiHiiServicesLib/UefiHiiServicesLib
> .inf
> PcdLib|MdePkg/Library/BasePcdLibNull/BasePcdLibNull.inf
> - IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf
> - OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
> IoLib|MdePkg/Library/BaseIoLibIntrinsic/BaseIoLibIntrinsic.inf
> TpmCommLib|SecurityPkg/Library/TpmCommLib/TpmCommLib.inf
>
> PlatformSecureLib|SecurityPkg/Library/PlatformSecureLibNull/PlatformSecureLi
> bNull.inf
> @@ -84,7 +82,13 @@ [LibraryClasses.common.PEIM]
>
> PeiServicesTablePointerLib|MdePkg/Library/PeiServicesTablePointerLib/PeiServi
> cesTablePointerLib.inf
> HobLib|MdePkg/Library/PeiHobLib/PeiHobLib.inf
>
> MemoryAllocationLib|MdePkg/Library/PeiMemoryAllocationLib/PeiMemoryAllo
> cationLib.inf
> +!ifdef CONTINUOUS_INTEGRATION
> + BaseCryptLib|CryptoPkg/Library/BaseCryptLibNull/BaseCryptLibNull.inf
> +!else
> + IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf
> + OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
> BaseCryptLib|CryptoPkg/Library/BaseCryptLib/PeiCryptLib.inf
> +!endif
>
> HashLib|SecurityPkg/Library/HashLibBaseCryptoRouter/HashLibBaseCryptoRout
> erPei.inf
>
> ReportStatusCodeLib|MdeModulePkg/Library/PeiReportStatusCodeLib/PeiRepo
> rtStatusCodeLib.inf
>
> Tpm12DeviceLib|SecurityPkg/Library/Tpm12DeviceLibDTpm/Tpm12DeviceLibDT
> pm.inf
> @@ -95,7 +99,13 @@ [LibraryClasses.common.PEIM]
> [LibraryClasses.common.DXE_DRIVER]
> HobLib|MdePkg/Library/DxeHobLib/DxeHobLib.inf
>
> ReportStatusCodeLib|MdeModulePkg/Library/DxeReportStatusCodeLib/DxeRep
> ortStatusCodeLib.inf
> +!ifdef CONTINUOUS_INTEGRATION
> + BaseCryptLib|CryptoPkg/Library/BaseCryptLibNull/BaseCryptLibNull.inf
> +!else
> + IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf
> + OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
> BaseCryptLib|CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf
> +!endif
>
> HashLib|SecurityPkg/Library/HashLibBaseCryptoRouter/HashLibBaseCryptoRout
> erDxe.inf
>
> Tpm12DeviceLib|SecurityPkg/Library/Tpm12DeviceLibTcg/Tpm12DeviceLibTcg.i
> nf
>
> Tpm2DeviceLib|SecurityPkg/Library/Tpm2DeviceLibTcg2/Tpm2DeviceLibTcg2.in
> f
> @@ -110,13 +120,25 @@ [LibraryClasses.common.UEFI_DRIVER,
> LibraryClasses.common.DXE_RUNTIME_DRIVER, Li
>
> [LibraryClasses.common.DXE_RUNTIME_DRIVER]
>
> ReportStatusCodeLib|MdeModulePkg/Library/RuntimeDxeReportStatusCodeLib
> /RuntimeDxeReportStatusCodeLib.inf
> +!ifdef CONTINUOUS_INTEGRATION
> + BaseCryptLib|CryptoPkg/Library/BaseCryptLibNull/BaseCryptLibNull.inf
> +!else
> + IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf
> + OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
> BaseCryptLib|CryptoPkg/Library/BaseCryptLib/RuntimeCryptLib.inf
> +!endif
>
> HashLib|SecurityPkg/Library/HashLibBaseCryptoRouter/HashLibBaseCryptoRout
> erDxe.inf
>
> Tpm12DeviceLib|SecurityPkg/Library/Tpm12DeviceLibTcg/Tpm12DeviceLibTcg.i
> nf
>
> Tpm2DeviceLib|SecurityPkg/Library/Tpm2DeviceLibTcg2/Tpm2DeviceLibTcg2.in
> f
>
> [LibraryClasses.common.UEFI_DRIVER,
> LibraryClasses.common.UEFI_APPLICATION]
> +!ifdef CONTINUOUS_INTEGRATION
> + BaseCryptLib|CryptoPkg/Library/BaseCryptLibNull/BaseCryptLibNull.inf
> +!else
> + IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf
> + OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
> BaseCryptLib|CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf
> +!endif
>
> HashLib|SecurityPkg/Library/HashLibBaseCryptoRouter/HashLibBaseCryptoRout
> erDxe.inf
>
> Tpm12DeviceLib|SecurityPkg/Library/Tpm12DeviceLibTcg/Tpm12DeviceLibTcg.i
> nf
>
> Tpm2DeviceLib|SecurityPkg/Library/Tpm2DeviceLibTcg2/Tpm2DeviceLibTcg2.in
> f
> @@ -127,7 +149,13 @@ [LibraryClasses.common.DXE_SMM_DRIVER]
>
> MemoryAllocationLib|MdePkg/Library/SmmMemoryAllocationLib/SmmMemory
> AllocationLib.inf
>
> ReportStatusCodeLib|MdeModulePkg/Library/SmmReportStatusCodeLib/SmmR
> eportStatusCodeLib.inf
> SmmMemLib|MdePkg/Library/SmmMemLib/SmmMemLib.inf
> +!ifdef CONTINUOUS_INTEGRATION
> + BaseCryptLib|CryptoPkg/Library/BaseCryptLibNull/BaseCryptLibNull.inf
> +!else
> + IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf
> + OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
> BaseCryptLib|CryptoPkg/Library/BaseCryptLib/SmmCryptLib.inf
> +!endif
>
> Tpm12DeviceLib|SecurityPkg/Library/Tpm12DeviceLibTcg/Tpm12DeviceLibTcg.i
> nf
>
> Tpm2DeviceLib|SecurityPkg/Library/Tpm2DeviceLibTcg2/Tpm2DeviceLibTcg2.in
> f
>
> Tcg2PhysicalPresenceLib|SecurityPkg/Library/SmmTcg2PhysicalPresenceLib/Sm
> mTcg2PhysicalPresenceLib.inf
> --
> 2.21.0.windows.1
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [Patch v4 07/22] .pytool/Plugin: Add CI plugins
2019-11-07 1:13 ` [Patch v4 07/22] .pytool/Plugin: Add CI plugins Michael D Kinney
@ 2019-11-07 6:58 ` Liming Gao
0 siblings, 0 replies; 51+ messages in thread
From: Liming Gao @ 2019-11-07 6:58 UTC (permalink / raw)
To: Kinney, Michael D, devel@edk2.groups.io; +Cc: Sean Brogan, Bret Barkelew
Mike and Sean:
I have one minor comment. Please use the same file Readme.md. I see there are two files are named as readme.md.
With this change, Reviewed-by: Liming Gao <liming.gao@intel.com>
Thanks
Liming
>-----Original Message-----
>From: Kinney, Michael D
>Sent: Thursday, November 07, 2019 9:14 AM
>To: devel@edk2.groups.io
>Cc: Sean Brogan <sean.brogan@microsoft.com>; Bret Barkelew
><Bret.Barkelew@microsoft.com>; Gao, Liming <liming.gao@intel.com>
>Subject: [Patch v4 07/22] .pytool/Plugin: Add CI plugins
>
>From: Sean Brogan <sean.brogan@microsoft.com>
>
>Add .pytool directory to the edk2 repository with the
>following plugins. These plugins are in a top level
>directory because that can be used with all packages
>and platforms.
>
>* CharEncodingCheck
>* CompilerPlugin
>* DependencyCheck
>* DscCompleteCheck
>* GuidCheck
>* LibraryClassCheck
>* SpellCheck
>
>Cc: Sean Brogan <sean.brogan@microsoft.com>
>Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
>Cc: Liming Gao <liming.gao@intel.com>
>Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
>---
> .../CharEncodingCheck/CharEncodingCheck.py | 118 ++++++++
> .../CharEncodingCheck_plug_in.yaml | 11 +
> .pytool/Plugin/CharEncodingCheck/Readme.md | 13 +
> .../Plugin/CompilerPlugin/CompilerPlugin.py | 102 +++++++
> .../CompilerPlugin/Compiler_plug_in.yaml | 11 +
> .../Plugin/DependencyCheck/DependencyCheck.py | 120 +++++++++
> .../DependencyCheck_plug_in.yaml | 13 +
> .../DscCompleteCheck/DscCompleteCheck.py | 118 ++++++++
> .../DscCompleteCheck_plug_in.yaml | 12 +
> .pytool/Plugin/DscCompleteCheck/readme.md | 22 ++
> .pytool/Plugin/GuidCheck/GuidCheck.py | 251 ++++++++++++++++++
> .../Plugin/GuidCheck/GuidCheck_plug_in.yaml | 11 +
> .pytool/Plugin/GuidCheck/Readme.md | 60 +++++
> .../LibraryClassCheck/LibraryClassCheck.py | 153 +++++++++++
> .../LibraryClassCheck_plug_in.yaml | 11 +
> .pytool/Plugin/LibraryClassCheck/readme.md | 22 ++
> .pytool/Plugin/SpellCheck/Readme.md | 100 +++++++
> .pytool/Plugin/SpellCheck/SpellCheck.py | 216 +++++++++++++++
> .../Plugin/SpellCheck/SpellCheck_plug_in.yaml | 11 +
> .pytool/Plugin/SpellCheck/cspell.base.yaml | 165 ++++++++++++
> 20 files changed, 1540 insertions(+)
> create mode
>100644 .pytool/Plugin/CharEncodingCheck/CharEncodingCheck.py
> create mode
>100644 .pytool/Plugin/CharEncodingCheck/CharEncodingCheck_plug_in.yaml
> create mode 100644 .pytool/Plugin/CharEncodingCheck/Readme.md
> create mode 100644 .pytool/Plugin/CompilerPlugin/CompilerPlugin.py
> create mode 100644 .pytool/Plugin/CompilerPlugin/Compiler_plug_in.yaml
> create mode 100644 .pytool/Plugin/DependencyCheck/DependencyCheck.py
> create mode
>100644 .pytool/Plugin/DependencyCheck/DependencyCheck_plug_in.yaml
> create mode
>100644 .pytool/Plugin/DscCompleteCheck/DscCompleteCheck.py
> create mode
>100644 .pytool/Plugin/DscCompleteCheck/DscCompleteCheck_plug_in.yaml
> create mode 100644 .pytool/Plugin/DscCompleteCheck/readme.md
> create mode 100644 .pytool/Plugin/GuidCheck/GuidCheck.py
> create mode 100644 .pytool/Plugin/GuidCheck/GuidCheck_plug_in.yaml
> create mode 100644 .pytool/Plugin/GuidCheck/Readme.md
> create mode 100644 .pytool/Plugin/LibraryClassCheck/LibraryClassCheck.py
> create mode
>100644 .pytool/Plugin/LibraryClassCheck/LibraryClassCheck_plug_in.yaml
> create mode 100644 .pytool/Plugin/LibraryClassCheck/readme.md
> create mode 100644 .pytool/Plugin/SpellCheck/Readme.md
> create mode 100644 .pytool/Plugin/SpellCheck/SpellCheck.py
> create mode 100644 .pytool/Plugin/SpellCheck/SpellCheck_plug_in.yaml
> create mode 100644 .pytool/Plugin/SpellCheck/cspell.base.yaml
>
>diff --git a/.pytool/Plugin/CharEncodingCheck/CharEncodingCheck.py
>b/.pytool/Plugin/CharEncodingCheck/CharEncodingCheck.py
>new file mode 100644
>index 0000000000..54a2424875
>--- /dev/null
>+++ b/.pytool/Plugin/CharEncodingCheck/CharEncodingCheck.py
>@@ -0,0 +1,118 @@
>+# @file CharEncodingCheck.py
>+#
>+# Copyright (c) Microsoft Corporation. All rights reserved.
>+# SPDX-License-Identifier: BSD-2-Clause-Patent
>+##
>+
>+
>+import os
>+import logging
>+from edk2toolext.environment.plugintypes.ci_build_plugin import
>ICiBuildPlugin
>+from edk2toolext.environment.var_dict import VarDict
>+
>+##
>+# map
>+##
>+EcodingMap = {
>+ ".md": 'utf-8',
>+ ".dsc": 'utf-8',
>+ ".dec": 'utf-8',
>+ ".c": 'utf-8',
>+ ".h": 'utf-8',
>+ ".asm": 'utf-8',
>+ ".masm": 'utf-8',
>+ ".nasm": 'utf-8',
>+ ".s": 'utf-8',
>+ ".inf": 'utf-8',
>+ ".asl": 'utf-8',
>+ ".uni": 'utf-8',
>+ ".py": 'utf-8'
>+}
>+
>+
>+class CharEncodingCheck(ICiBuildPlugin):
>+ """
>+ A CiBuildPlugin that scans each file in the code tree and confirms the
>encoding is correct.
>+
>+ Configuration options:
>+ "CharEncodingCheck": {
>+ "IgnoreFiles": []
>+ }
>+ """
>+
>+ def GetTestName(self, packagename: str, environment: VarDict) -> tuple:
>+ """ Provide the testcase name and classname for use in reporting
>+ testclassname: a descriptive string for the testcase can include
>whitespace
>+ classname: should be patterned <packagename>.<plugin>.<optionally
>any unique condition>
>+
>+ Args:
>+ packagename: string containing name of package to build
>+ environment: The VarDict for the test to run in
>+ Returns:
>+ a tuple containing the testcase name and the classname
>+ (testcasename, classname)
>+ """
>+ return ("Check for valid file encoding for " + packagename, packagename
>+ ".CharEncodingCheck")
>+
>+ ##
>+ # External function of plugin. This function is used to perform the task of
>the ci_build_plugin Plugin
>+ #
>+ # - package is the edk2 path to package. This means
>workspace/packagepath relative.
>+ # - edk2path object configured with workspace and packages path
>+ # - PkgConfig Object (dict) for the pkg
>+ # - EnvConfig Object
>+ # - Plugin Manager Instance
>+ # - Plugin Helper Obj Instance
>+ # - Junit Logger
>+ # - output_stream the StringIO output stream from this plugin via logging
>+ def RunBuildPlugin(self, packagename, Edk2pathObj, pkgconfig,
>environment, PLM, PLMHelper, tc, output_stream=None):
>+ overall_status = 0
>+ files_tested = 0
>+
>+ abs_pkg_path =
>Edk2pathObj.GetAbsolutePathOnThisSytemFromEdk2RelativePath(packagen
>ame)
>+
>+ if abs_pkg_path is None:
>+ tc.SetSkipped()
>+ tc.LogStdError("No Package folder {0}".format(abs_pkg_path))
>+ return 0
>+
>+ for (ext, enc) in EcodingMap.items():
>+ files = self.WalkDirectoryForExtension([ext], abs_pkg_path)
>+ files = [Edk2pathObj.GetEdk2RelativePathFromAbsolutePath(x) for x
>in files] # make edk2relative path so can process ignores
>+
>+ if "IgnoreFiles" in pkgconfig:
>+ for a in pkgconfig["IgnoreFiles"]:
>+ a = a.replace(os.sep, "/")
>+ try:
>+ tc.LogStdOut("Ignoring File {0}".format(a))
>+ files.remove(a)
>+ except:
>+ tc.LogStdError("CharEncodingCheck.IgnoreInf -> {0} not found in
>filesystem. Invalid ignore file".format(a))
>+ logging.info("CharEncodingCheck.IgnoreInf -> {0} not found in
>filesystem. Invalid ignore file".format(a))
>+
>+ files =
>[Edk2pathObj.GetAbsolutePathOnThisSytemFromEdk2RelativePath(x) for x in
>files]
>+ for a in files:
>+ files_tested += 1
>+ if(self.TestEncodingOk(a, enc)):
>+ logging.debug("File {0} Passed Encoding Check {1}".format(a, enc))
>+ else:
>+ tc.LogStdError("Encoding Failure in {0}. Not {1}".format(a, enc))
>+ overall_status += 1
>+
>+ tc.LogStdOut("Tested Encoding on {0} files".format(files_tested))
>+ if overall_status is not 0:
>+ tc.SetFailed("CharEncoding {0} Failed. Errors {1}".format(packagename,
>overall_status), "CHAR_ENCODING_CHECK_FAILED")
>+ else:
>+ tc.SetSuccess()
>+ return overall_status
>+
>+ def TestEncodingOk(self, apath, encodingValue):
>+ try:
>+ with open(apath, "rb") as fobj:
>+ fobj.read().decode(encodingValue)
>+ except Exception as exp:
>+ logging.error("Encoding failure: file: {0} type: {1}".format(apath,
>encodingValue))
>+ logging.debug("EXCEPTION: while processing {1} - {0}".format(exp,
>apath))
>+ return False
>+
>+ return True
>diff --git
>a/.pytool/Plugin/CharEncodingCheck/CharEncodingCheck_plug_in.yaml
>b/.pytool/Plugin/CharEncodingCheck/CharEncodingCheck_plug_in.yaml
>new file mode 100644
>index 0000000000..915d3f4fdb
>--- /dev/null
>+++ b/.pytool/Plugin/CharEncodingCheck/CharEncodingCheck_plug_in.yaml
>@@ -0,0 +1,11 @@
>+##
>+# CiBuildPlugin used to check char encoding
>+#
>+# Copyright (c) 2019, Microsoft Corporation
>+# SPDX-License-Identifier: BSD-2-Clause-Patent
>+##
>+{
>+ "scope": "cibuild",
>+ "name": "Char Encoding Check Test",
>+ "module": "CharEncodingCheck"
>+}
>diff --git a/.pytool/Plugin/CharEncodingCheck/Readme.md
>b/.pytool/Plugin/CharEncodingCheck/Readme.md
>new file mode 100644
>index 0000000000..8350542966
>--- /dev/null
>+++ b/.pytool/Plugin/CharEncodingCheck/Readme.md
>@@ -0,0 +1,13 @@
>+# Character Encoding Check Plugin
>+
>+This CiBuildPlugin scans all the files in a package to make sure each file is
>correctly encoded and all characters can be read. Improper encoding causes
>tools to fail in some situations especially in different locals.
>+
>+## Configuration
>+
>+The plugin can be configured to ignore certain files.
>+
>+``` yaml
>+"CharEncodingCheck": {
>+ "IgnoreFiles": [] # optional - list of files to ignore
>+}
>+```
>diff --git a/.pytool/Plugin/CompilerPlugin/CompilerPlugin.py
>b/.pytool/Plugin/CompilerPlugin/CompilerPlugin.py
>new file mode 100644
>index 0000000000..0a357309a4
>--- /dev/null
>+++ b/.pytool/Plugin/CompilerPlugin/CompilerPlugin.py
>@@ -0,0 +1,102 @@
>+# @file HostUnitTestCompiler_plugin.py
>+##
>+# Copyright (c) Microsoft Corporation. All rights reserved.
>+# SPDX-License-Identifier: BSD-2-Clause-Patent
>+##
>+
>+import logging
>+import os
>+import re
>+from edk2toollib.uefi.edk2.parsers.dsc_parser import DscParser
>+from edk2toolext.environment.plugintypes.ci_build_plugin import
>ICiBuildPlugin
>+from edk2toolext.environment.uefi_build import UefiBuilder
>+from edk2toolext import edk2_logging
>+from edk2toolext.environment.var_dict import VarDict
>+
>+
>+class CompilerPlugin(ICiBuildPlugin):
>+ """
>+ A CiBuildPlugin that compiles the package dsc
>+ from the package being tested.
>+
>+ Configuration options:
>+ "CompilerPlugin": {
>+ "DscPath": "<path to dsc from root of pkg>"
>+ }
>+ """
>+
>+ def GetTestName(self, packagename: str, environment: VarDict) -> tuple:
>+ """ Provide the testcase name and classname for use in reporting
>+
>+ Args:
>+ packagename: string containing name of package to build
>+ environment: The VarDict for the test to run in
>+ Returns:
>+ a tuple containing the testcase name and the classname
>+ (testcasename, classname)
>+ """
>+ target = environment.GetValue("TARGET")
>+ return ("Compile " + packagename + " " + target, packagename +
>".Compiler." + target)
>+
>+ def RunsOnTargetList(self):
>+ return ["DEBUG", "RELEASE"]
>+
>+ ##
>+ # External function of plugin. This function is used to perform the task of
>the MuBuild Plugin
>+ #
>+ # - package is the edk2 path to package. This means
>workspace/packagepath relative.
>+ # - edk2path object configured with workspace and packages path
>+ # - PkgConfig Object (dict) for the pkg
>+ # - EnvConfig Object
>+ # - Plugin Manager Instance
>+ # - Plugin Helper Obj Instance
>+ # - Junit Logger
>+ # - output_stream the StringIO output stream from this plugin via logging
>+ def RunBuildPlugin(self, packagename, Edk2pathObj, pkgconfig,
>environment, PLM, PLMHelper, tc, output_stream=None):
>+ self._env = environment
>+
>+ # Parse the config for required DscPath element
>+ if "DscPath" not in pkgconfig:
>+ tc.SetSkipped()
>+ tc.LogStdError("DscPath not found in config file. Nothing to compile.")
>+ return -1
>+
>+ AP =
>Edk2pathObj.GetAbsolutePathOnThisSytemFromEdk2RelativePath(packagen
>ame)
>+
>+ APDSC = os.path.join(AP, pkgconfig["DscPath"].strip())
>+ AP_Path = Edk2pathObj.GetEdk2RelativePathFromAbsolutePath(APDSC)
>+ if AP is None or AP_Path is None or not os.path.isfile(APDSC):
>+ tc.SetSkipped()
>+ tc.LogStdError("Package Dsc not found.")
>+ return -1
>+
>+ logging.info("Building {0}".format(AP_Path))
>+ self._env.SetValue("ACTIVE_PLATFORM", AP_Path, "Set in Compiler
>Plugin")
>+
>+ # Parse DSC to check for SUPPORTED_ARCHITECTURES
>+ dp = DscParser()
>+ dp.SetBaseAbsPath(Edk2pathObj.WorkspacePath)
>+ dp.SetPackagePaths(Edk2pathObj.PackagePathList)
>+ dp.ParseFile(AP_Path)
>+ if "SUPPORTED_ARCHITECTURES" in dp.LocalVars:
>+ SUPPORTED_ARCHITECTURES =
>dp.LocalVars["SUPPORTED_ARCHITECTURES"].split('|')
>+ TARGET_ARCHITECTURES =
>environment.GetValue("TARGET_ARCH").split(' ')
>+
>+ # Skip if there is no intersection between
>SUPPORTED_ARCHITECTURES and TARGET_ARCHITECTURES
>+ if len(set(SUPPORTED_ARCHITECTURES) &
>set(TARGET_ARCHITECTURES)) == 0:
>+ tc.SetSkipped()
>+ tc.LogStdError("No supported architecutres to build")
>+ return -1
>+
>+ uefiBuilder = UefiBuilder()
>+ # do all the steps
>+ # WorkSpace, PackagesPath, PInHelper, PInManager
>+ ret = uefiBuilder.Go(Edk2pathObj.WorkspacePath,
>os.pathsep.join(Edk2pathObj.PackagePathList), PLMHelper, PLM)
>+ if ret != 0: # failure:
>+ tc.SetFailed("Compile failed for {0}".format(packagename),
>"Compile_FAILED")
>+ tc.LogStdError("{0} Compile failed with error code {1}
>".format(AP_Path, ret))
>+ return 1
>+
>+ else:
>+ tc.SetSuccess()
>+ return 0
>diff --git a/.pytool/Plugin/CompilerPlugin/Compiler_plug_in.yaml
>b/.pytool/Plugin/CompilerPlugin/Compiler_plug_in.yaml
>new file mode 100644
>index 0000000000..4f9b3d3113
>--- /dev/null
>+++ b/.pytool/Plugin/CompilerPlugin/Compiler_plug_in.yaml
>@@ -0,0 +1,11 @@
>+##
>+# CiBuildPlugin used to compile each package
>+#
>+# Copyright (c) 2019, Microsoft Corporation
>+# SPDX-License-Identifier: BSD-2-Clause-Patent
>+##
>+{
>+ "scope": "cibuild",
>+ "name": "Compiler Plugin",
>+ "module": "CompilerPlugin"
>+}
>diff --git a/.pytool/Plugin/DependencyCheck/DependencyCheck.py
>b/.pytool/Plugin/DependencyCheck/DependencyCheck.py
>new file mode 100644
>index 0000000000..497914cf3a
>--- /dev/null
>+++ b/.pytool/Plugin/DependencyCheck/DependencyCheck.py
>@@ -0,0 +1,120 @@
>+# @file dependency_check.py
>+#
>+# Copyright (c) Microsoft Corporation. All rights reserved.
>+# SPDX-License-Identifier: BSD-2-Clause-Patent
>+##
>+
>+import logging
>+import os
>+from edk2toolext.environment.plugintypes.ci_build_plugin import
>ICiBuildPlugin
>+from edk2toollib.uefi.edk2.parsers.inf_parser import InfParser
>+from edk2toolext.environment.var_dict import VarDict
>+
>+
>+class DependencyCheck(ICiBuildPlugin):
>+ """
>+ A CiBuildPlugin that finds all modules (inf files) in a package and reviews
>the packages used
>+ to confirm they are acceptable. This is to help enforce layering and identify
>improper
>+ dependencies between packages.
>+
>+ Configuration options:
>+ "DependencyCheck": {
>+ "AcceptableDependencies": [], # Package dec files that are allowed in all
>INFs. Example: MdePkg/MdePkg.dec
>+ "AcceptableDependencies-<MODULE_TYPE>": [], # OPTIONAL Package
>dependencies for INFs that are HOST_APPLICATION
>+ "AcceptableDependencies-HOST_APPLICATION": [], # EXAMPLE Package
>dependencies for INFs that are HOST_APPLICATION
>+ "IgnoreInf": [] # Ignore INF if found in filesystem
>+ }
>+ """
>+
>+ def GetTestName(self, packagename: str, environment: VarDict) -> tuple:
>+ """ Provide the testcase name and classname for use in reporting
>+
>+ Args:
>+ packagename: string containing name of package to build
>+ environment: The VarDict for the test to run in
>+ Returns:
>+ a tuple containing the testcase name and the classname
>+ (testcasename, classname)
>+ testclassname: a descriptive string for the testcase can include
>whitespace
>+ classname: should be patterned
><packagename>.<plugin>.<optionally any unique condition>
>+ """
>+ return ("Test Package Dependencies for modules in " + packagename,
>packagename + ".DependencyCheck")
>+
>+ ##
>+ # External function of plugin. This function is used to perform the task of
>the MuBuild Plugin
>+ #
>+ # - package is the edk2 path to package. This means
>workspace/packagepath relative.
>+ # - edk2path object configured with workspace and packages path
>+ # - PkgConfig Object (dict) for the pkg
>+ # - EnvConfig Object
>+ # - Plugin Manager Instance
>+ # - Plugin Helper Obj Instance
>+ # - Junit Logger
>+ # - output_stream the StringIO output stream from this plugin via logging
>+ def RunBuildPlugin(self, packagename, Edk2pathObj, pkgconfig,
>environment, PLM, PLMHelper, tc, output_stream=None):
>+ overall_status = 0
>+
>+ # Get current platform
>+ abs_pkg_path =
>Edk2pathObj.GetAbsolutePathOnThisSytemFromEdk2RelativePath(packagen
>ame)
>+
>+ # Get INF Files
>+ INFFiles = self.WalkDirectoryForExtension([".inf"], abs_pkg_path)
>+ INFFiles = [Edk2pathObj.GetEdk2RelativePathFromAbsolutePath(x) for x
>in INFFiles] # make edk2relative path so can compare with Ignore List
>+
>+ # Remove ignored INFs
>+ if "IgnoreInf" in pkgconfig:
>+ for a in pkgconfig["IgnoreInf"]:
>+ a = a.replace(os.sep, "/") ## convert path sep in case ignore list is
>bad. Can't change case
>+ try:
>+ INFFiles.remove(a)
>+ tc.LogStdOut("IgnoreInf {0}".format(a))
>+ except:
>+ logging.info("DependencyConfig.IgnoreInf -> {0} not found in
>filesystem. Invalid ignore file".format(a))
>+ tc.LogStdError("DependencyConfig.IgnoreInf -> {0} not found in
>filesystem. Invalid ignore file".format(a))
>+
>+
>+ # Get the AccpetableDependencies list
>+ if "AcceptableDependencies" not in pkgconfig:
>+ logging.info("DependencyCheck Skipped. No Acceptable
>Dependencies defined.")
>+ tc.LogStdOut("DependencyCheck Skipped. No Acceptable
>Dependencies defined.")
>+ tc.SetSkipped()
>+ return -1
>+
>+ # Log dependencies
>+ for k in pkgconfig.keys():
>+ if k.startswith("AcceptableDependencies"):
>+ pkgstring = "\n".join(pkgconfig[k])
>+ if ("-" in k):
>+ _, _, mod_type = k.partition("-")
>+ tc.LogStdOut(f"Additional dependencies for MODULE_TYPE
>{mod_type}:\n {pkgstring}")
>+ else:
>+ tc.LogStdOut(f"Acceptable Dependencies:\n {pkgstring}")
>+
>+ # For each INF file
>+ for file in INFFiles:
>+ ip = InfParser()
>+ logging.debug("Parsing " + file)
>+
>ip.SetBaseAbsPath(Edk2pathObj.WorkspacePath).SetPackagePaths(Edk2path
>Obj.PackagePathList).ParseFile(file)
>+
>+ if("MODULE_TYPE" not in ip.Dict):
>+ tc.LogStdOut("Ignoring INF. Missing key for MODULE_TYPE
>{0}".format(file))
>+ continue
>+
>+ mod_type = ip.Dict["MODULE_TYPE"].upper()
>+ for p in ip.PackagesUsed:
>+ if p not in pkgconfig["AcceptableDependencies"]:
>+ # If not in the main acceptable dependencies list then check
>module specific
>+ mod_specific_key = "AcceptableDependencies-" + mod_type
>+ if mod_specific_key in pkgconfig and p in
>pkgconfig[mod_specific_key]:
>+ continue
>+
>+ logging.error("Dependency Check: Invalid Dependency INF: {0}
>depends on pkg {1}".format(file, p))
>+ tc.LogStdError("Dependency Check: Invalid Dependency INF: {0}
>depends on pkg {1}".format(file, p))
>+ overall_status += 1
>+
>+ # If XML object exists, add results
>+ if overall_status is not 0:
>+ tc.SetFailed("Failed with {0} errors".format(overall_status),
>"DEPENDENCYCHECK_FAILED")
>+ else:
>+ tc.SetSuccess()
>+ return overall_status
>diff --git
>a/.pytool/Plugin/DependencyCheck/DependencyCheck_plug_in.yaml
>b/.pytool/Plugin/DependencyCheck/DependencyCheck_plug_in.yaml
>new file mode 100644
>index 0000000000..fdb96d625b
>--- /dev/null
>+++ b/.pytool/Plugin/DependencyCheck/DependencyCheck_plug_in.yaml
>@@ -0,0 +1,13 @@
>+##
>+# CiBuildPlugin used to check all infs within a package
>+# to confirm the packagesdependency are on the configured list of
>acceptable
>+# dependencies.
>+#
>+# Copyright (c) 2019, Microsoft Corporation
>+# SPDX-License-Identifier: BSD-2-Clause-Patent
>+##
>+{
>+ "scope": "cibuild",
>+ "name": "Dependency Check Test",
>+ "module": "DependencyCheck"
>+}
>diff --git a/.pytool/Plugin/DscCompleteCheck/DscCompleteCheck.py
>b/.pytool/Plugin/DscCompleteCheck/DscCompleteCheck.py
>new file mode 100644
>index 0000000000..dcd8946ca6
>--- /dev/null
>+++ b/.pytool/Plugin/DscCompleteCheck/DscCompleteCheck.py
>@@ -0,0 +1,118 @@
>+# @file DscCompleteCheck.py
>+#
>+# Copyright (c) Microsoft Corporation. All rights reserved.
>+# SPDX-License-Identifier: BSD-2-Clause-Patent
>+##
>+import logging
>+import os
>+from edk2toolext.environment.plugintypes.ci_build_plugin import
>ICiBuildPlugin
>+from edk2toollib.uefi.edk2.parsers.dsc_parser import DscParser
>+from edk2toollib.uefi.edk2.parsers.inf_parser import InfParser
>+from edk2toolext.environment.var_dict import VarDict
>+
>+
>+class DscCompleteCheck(ICiBuildPlugin):
>+ """
>+ A CiBuildPlugin that scans the package dsc file and confirms all modules (inf
>files) are
>+ listed in the components sections.
>+
>+ Configuration options:
>+ "DscCompleteCheck": {
>+ "DscPath": "<path to dsc from root of pkg>"
>+ "IgnoreInf": [] # Ignore INF if found in filesystem by not dsc
>+ }
>+ """
>+
>+ def GetTestName(self, packagename: str, environment: VarDict) -> tuple:
>+ """ Provide the testcase name and classname for use in reporting
>+
>+ Args:
>+ packagename: string containing name of package to build
>+ environment: The VarDict for the test to run in
>+ Returns:
>+ a tuple containing the testcase name and the classname
>+ (testcasename, classname)
>+ testclassname: a descriptive string for the testcase can include
>whitespace
>+ classname: should be patterned
><packagename>.<plugin>.<optionally any unique condition>
>+ """
>+ return ("Check the " + packagename + " DSC for a being complete",
>packagename + ".DscCompleteCheck")
>+
>+ ##
>+ # External function of plugin. This function is used to perform the task of
>the MuBuild Plugin
>+ #
>+ # - package is the edk2 path to package. This means
>workspace/packagepath relative.
>+ # - edk2path object configured with workspace and packages path
>+ # - PkgConfig Object (dict) for the pkg
>+ # - VarDict containing the shell environment Build Vars
>+ # - Plugin Manager Instance
>+ # - Plugin Helper Obj Instance
>+ # - Junit Logger
>+ # - output_stream the StringIO output stream from this plugin via logging
>+ def RunBuildPlugin(self, packagename, Edk2pathObj, pkgconfig,
>environment, PLM, PLMHelper, tc, output_stream=None):
>+ overall_status = 0
>+
>+ # Parse the config for required DscPath element
>+ if "DscPath" not in pkgconfig:
>+ tc.SetSkipped()
>+ tc.LogStdError("DscPath not found in config file. Nothing to check.")
>+ return -1
>+
>+ abs_pkg_path =
>Edk2pathObj.GetAbsolutePathOnThisSytemFromEdk2RelativePath(packagen
>ame)
>+ abs_dsc_path = os.path.join(abs_pkg_path, pkgconfig["DscPath"].strip())
>+ wsr_dsc_path =
>Edk2pathObj.GetEdk2RelativePathFromAbsolutePath(abs_dsc_path)
>+
>+ if abs_dsc_path is None or wsr_dsc_path is "" or not
>os.path.isfile(abs_dsc_path):
>+ tc.SetSkipped()
>+ tc.LogStdError("Package Dsc not found")
>+ return 0
>+
>+ # Get INF Files
>+ INFFiles = self.WalkDirectoryForExtension([".inf"], abs_pkg_path)
>+ INFFiles = [Edk2pathObj.GetEdk2RelativePathFromAbsolutePath(x) for x
>in INFFiles] # make edk2relative path so can compare with DSC
>+
>+ # remove ignores
>+
>+ if "IgnoreInf" in pkgconfig:
>+ for a in pkgconfig["IgnoreInf"]:
>+ a = a.replace(os.sep, "/")
>+ try:
>+ tc.LogStdOut("Ignoring INF {0}".format(a))
>+ INFFiles.remove(a)
>+ except:
>+ tc.LogStdError("DscCompleteCheck.IgnoreInf -> {0} not found in
>filesystem. Invalid ignore file".format(a))
>+ logging.info("DscCompleteCheck.IgnoreInf -> {0} not found in
>filesystem. Invalid ignore file".format(a))
>+
>+ # DSC Parser
>+ dp = DscParser()
>+ dp.SetBaseAbsPath(Edk2pathObj.WorkspacePath)
>+ dp.SetPackagePaths(Edk2pathObj.PackagePathList)
>+ dp.SetInputVars(environment.GetAllBuildKeyValues())
>+ dp.ParseFile(wsr_dsc_path)
>+
>+ # Check if INF in component section
>+ for INF in INFFiles:
>+ if not any(INF.strip() in x for x in dp.ThreeMods) and \
>+ not any(INF.strip() in x for x in dp.SixMods) and \
>+ not any(INF.strip() in x for x in dp.OtherMods):
>+
>+ infp = InfParser().SetBaseAbsPath(Edk2pathObj.WorkspacePath)
>+ infp.SetPackagePaths(Edk2pathObj.PackagePathList)
>+ infp.ParseFile(INF)
>+ if("MODULE_TYPE" not in infp.Dict):
>+ tc.LogStdOut("Ignoring INF. Missing key for MODULE_TYPE
>{0}".format(INF))
>+ continue
>+
>+ if(infp.Dict["MODULE_TYPE"] == "HOST_APPLICATION"):
>+ tc.LogStdOut("Ignoring INF. Module type is HOST_APPLICATION
>{0}".format(INF))
>+ continue
>+
>+ logging.critical(INF + " not in " + wsr_dsc_path)
>+ tc.LogStdError("{0} not in {1}".format(INF, wsr_dsc_path))
>+ overall_status = overall_status + 1
>+
>+ # If XML object exists, add result
>+ if overall_status is not 0:
>+ tc.SetFailed("DscCompleteCheck {0} Failed. Errors
>{1}".format(wsr_dsc_path, overall_status), "CHECK_FAILED")
>+ else:
>+ tc.SetSuccess()
>+ return overall_status
>diff --git
>a/.pytool/Plugin/DscCompleteCheck/DscCompleteCheck_plug_in.yaml
>b/.pytool/Plugin/DscCompleteCheck/DscCompleteCheck_plug_in.yaml
>new file mode 100644
>index 0000000000..d84d57d973
>--- /dev/null
>+++ b/.pytool/Plugin/DscCompleteCheck/DscCompleteCheck_plug_in.yaml
>@@ -0,0 +1,12 @@
>+##
>+# CiBuildPlugin used to confirm all INFs are listed in
>+# the components section of package dsc
>+#
>+# Copyright (c) 2019, Microsoft Corporation
>+# SPDX-License-Identifier: BSD-2-Clause-Patent
>+##
>+{
>+ "scope": "cibuild",
>+ "name": "Dsc Complete Check Test",
>+ "module": "DscCompleteCheck"
>+}
>diff --git a/.pytool/Plugin/DscCompleteCheck/readme.md
>b/.pytool/Plugin/DscCompleteCheck/readme.md
>new file mode 100644
>index 0000000000..17e542b8d6
>--- /dev/null
>+++ b/.pytool/Plugin/DscCompleteCheck/readme.md
>@@ -0,0 +1,22 @@
>+# Dsc Complete Check Plugin
>+
>+This CiBuildPlugin scans all INF files from a package and confirms they are
>listed in the package level DSC file. The test considers it an error if any INF
>does not appear in the `Components` section of the package-level DSC
>(indicating that it would not be built if the package were built). This is critical
>because much of the CI infrastructure assumes that all modules will be listed
>in the DSC and compiled.
>+
>+## Configuration
>+
>+The plugin has a few configuration options to support the UEFI codebase.
>+
>+``` yaml
>+"DscCompleteCheck": {
>+ "DscPath": "", # Path to dsc from root of package
>+ "IgnoreInf": [] # Ignore INF if found in filesystem by not dsc
>+ }
>+```
>+
>+### DscPath
>+
>+Path to DSC to consider platform dsc
>+
>+### IgnoreInf
>+
>+Ignore error if Inf file is not listed in DSC file
>diff --git a/.pytool/Plugin/GuidCheck/GuidCheck.py
>b/.pytool/Plugin/GuidCheck/GuidCheck.py
>new file mode 100644
>index 0000000000..467e17f3e8
>--- /dev/null
>+++ b/.pytool/Plugin/GuidCheck/GuidCheck.py
>@@ -0,0 +1,251 @@
>+# @file GuidCheck.py
>+#
>+# Copyright (c) Microsoft Corporation. All rights reserved.
>+# SPDX-License-Identifier: BSD-2-Clause-Patent
>+##
>+import logging
>+from edk2toolext.environment.plugintypes.ci_build_plugin import
>ICiBuildPlugin
>+from edk2toollib.uefi.edk2.guid_list import GuidList
>+from edk2toolext.environment.var_dict import VarDict
>+
>+
>+class GuidCheck(ICiBuildPlugin):
>+ """
>+ A CiBuildPlugin that scans the code tree and looks for duplicate guids
>+ from the package being tested.
>+
>+ Configuration options:
>+ "GuidCheck": {
>+ "IgnoreGuidName": [], # provide in format guidname=guidvalue or just
>guidname
>+ "IgnoreGuidValue": [],
>+ "IgnoreFoldersAndFiles": [],
>+ "IgnoreDuplicates": [] # Provide in format
>guidname=guidname=guidname...
>+ }
>+ """
>+
>+ def GetTestName(self, packagename: str, environment: VarDict) -> tuple:
>+ """ Provide the testcase name and classname for use in reporting
>+
>+ Args:
>+ packagename: string containing name of package to build
>+ environment: The VarDict for the test to run in
>+ Returns:
>+ a tuple containing the testcase name and the classname
>+ (testcasename, classname)
>+ testclassname: a descriptive string for the testcase can include
>whitespace
>+ classname: should be patterned
><packagename>.<plugin>.<optionally any unique condition>
>+ """
>+ return ("Confirm GUIDs are unique in " + packagename, packagename +
>".GuidCheck")
>+
>+ def _FindConflictingGuidValues(self, guidlist: list) -> list:
>+ """ Find all duplicate guids by guid value and report them as errors
>+ """
>+ # Sort the list by guid
>+ guidsorted = sorted(
>+ guidlist, key=lambda x: x.guid.upper(), reverse=True)
>+
>+ previous = None # Store previous entry for comparison
>+ error = None
>+ errors = []
>+ for index in range(len(guidsorted)):
>+ i = guidsorted[index]
>+ if(previous is not None):
>+ if i.guid == previous.guid: # Error
>+ if(error is None):
>+ # Catch errors with more than 1 conflict
>+ error = ErrorEntry("guid")
>+ error.entries.append(previous)
>+ errors.append(error)
>+ error.entries.append(i)
>+ else:
>+ # no match. clear error
>+ error = None
>+ previous = i
>+ return errors
>+
>+ def _FindConflictingGuidNames(self, guidlist: list) -> list:
>+ """ Find all duplicate guids by name and if they are not all
>+ from inf files report them as errors. It is ok to have
>+ BASE_NAME duplication.
>+
>+ Is this useful? It would catch two same named guids in dec file
>+ that resolve to different values.
>+ """
>+ # Sort the list by guid
>+ namesorted = sorted(guidlist, key=lambda x: x.name.upper())
>+
>+ previous = None # Store previous entry for comparison
>+ error = None
>+ errors = []
>+ for index in range(len(namesorted)):
>+ i = namesorted[index]
>+ if(previous is not None):
>+ # If name matches
>+ if i.name == previous.name:
>+ if(error is None):
>+ # Catch errors with more than 1 conflict
>+ error = ErrorEntry("name")
>+ error.entries.append(previous)
>+ errors.append(error)
>+ error.entries.append(i)
>+ else:
>+ # no match. clear error
>+ error = None
>+ previous = i
>+
>+ # Loop thru and remove any errors where all files are infs as it is ok if
>+ # they have the same inf base name.
>+ for e in errors[:]:
>+ if len( [en for en in e.entries if not
>en.absfilepath.lower().endswith(".inf")]) == 0:
>+ errors.remove(e)
>+
>+ return errors
>+
>+ ##
>+ # External function of plugin. This function is used to perform the task of
>the MuBuild Plugin
>+ #
>+ # - package is the edk2 path to package. This means
>workspace/packagepath relative.
>+ # - edk2path object configured with workspace and packages path
>+ # - PkgConfig Object (dict) for the pkg
>+ # - EnvConfig Object
>+ # - Plugin Manager Instance
>+ # - Plugin Helper Obj Instance
>+ # - Junit Logger
>+ # - output_stream the StringIO output stream from this plugin via logging
>+
>+ def RunBuildPlugin(self, packagename, Edk2pathObj, pkgconfig,
>environment, PLM, PLMHelper, tc, output_stream=None):
>+ Errors = []
>+
>+ abs_pkg_path =
>Edk2pathObj.GetAbsolutePathOnThisSytemFromEdk2RelativePath(
>+ packagename)
>+
>+ if abs_pkg_path is None:
>+ tc.SetSkipped()
>+ tc.LogStdError("No package {0}".format(packagename))
>+ return -1
>+
>+ All_Ignores = ["/Build", "/Conf"]
>+ # Parse the config for other ignores
>+ if "IgnoreFoldersAndFiles" in pkgconfig:
>+ All_Ignores.extend(pkgconfig["IgnoreFoldersAndFiles"])
>+
>+ # Parse the workspace for all GUIDs
>+ gs = GuidList.guidlist_from_filesystem(
>+ Edk2pathObj.WorkspacePath, ignore_lines=All_Ignores)
>+
>+ # Remove ignored guidvalue
>+ if "IgnoreGuidValue" in pkgconfig:
>+ for a in pkgconfig["IgnoreGuidValue"]:
>+ try:
>+ tc.LogStdOut("Ignoring Guid {0}".format(a.upper()))
>+ for b in gs[:]:
>+ if b.guid == a.upper():
>+ gs.remove(b)
>+ except:
>+ tc.LogStdError("GuidCheck.IgnoreGuid -> {0} not found. Invalid
>ignore guid".format(a.upper()))
>+ logging.info("GuidCheck.IgnoreGuid -> {0} not found. Invalid
>ignore guid".format(a.upper()))
>+
>+ # Remove ignored guidname
>+ if "IgnoreGuidName" in pkgconfig:
>+ for a in pkgconfig["IgnoreGuidName"]:
>+ entry = a.split("=")
>+ if(len(entry) > 2):
>+ tc.LogStdError("GuidCheck.IgnoreGuidName -> {0} Invalid
>Format.".format(a))
>+ logging.info("GuidCheck.IgnoreGuidName -> {0} Invalid
>Format.".format(a))
>+ continue
>+ try:
>+ tc.LogStdOut("Ignoring Guid {0}".format(a))
>+ for b in gs[:]:
>+ if b.name == entry[0]:
>+ if(len(entry) == 1):
>+ gs.remove(b)
>+ elif(len(entry) == 2 and b.guid.upper() == entry[1].upper()):
>+ gs.remove(b)
>+ else:
>+ c.LogStdError("GuidCheck.IgnoreGuidName -> {0}
>incomplete match. Invalid ignore guid".format(a))
>+
>+ except:
>+ tc.LogStdError("GuidCheck.IgnoreGuidName -> {0} not found.
>Invalid ignore name".format(a))
>+ logging.info("GuidCheck.IgnoreGuidName -> {0} not found. Invalid
>ignore name".format(a))
>+
>+ # Find conflicting Guid Values
>+ Errors.extend(self._FindConflictingGuidValues(gs))
>+
>+ # Check if there are expected duplicates and remove it from the error list
>+ if "IgnoreDuplicates" in pkgconfig:
>+ for a in pkgconfig["IgnoreDuplicates"]:
>+ names = a.split("=")
>+ if len(names) < 2:
>+ tc.LogStdError("GuidCheck.IgnoreDuplicates -> {0} invalid
>format".format(a))
>+ logging.info("GuidCheck.IgnoreDuplicates -> {0} invalid
>format".format(a))
>+ continue
>+
>+ for b in Errors[:]:
>+ if b.type != "guid":
>+ continue
>+ ## Make a list of the names that are not in the names list. If there
>+ ## are any in the list then this error should not be ignored.
>+ t = [x for x in b.entries if x.name not in names]
>+ if(len(t) == len(b.entries)):
>+ ## did not apply to any entry
>+ continue
>+ elif(len(t) == 0):
>+ ## full match - ignore duplicate
>+ tc.LogStdOut("GuidCheck.IgnoreDuplicates -> {0}".format(a))
>+ Errors.remove(b)
>+ elif(len(t) < len(b.entries)):
>+ ## partial match
>+ tc.LogStdOut("GuidCheck.IgnoreDuplicates -> {0} incomplete
>match".format(a))
>+ logging.info("GuidCheck.IgnoreDuplicates -> {0} incomplete
>match".format(a))
>+ else:
>+ tc.LogStdOut("GuidCheck.IgnoreDuplicates -> {0} unknown
>error.".format(a))
>+ logging.info("GuidCheck.IgnoreDuplicates -> {0} unknown
>error".format(a))
>+
>+
>+
>+ # Find conflicting Guid Names
>+ Errors.extend(self._FindConflictingGuidNames(gs))
>+
>+ # Log errors for anything within the package under test
>+ for er in Errors[:]:
>+ InMyPackage = False
>+ for a in er.entries:
>+ if abs_pkg_path in a.absfilepath:
>+ InMyPackage = True
>+ break
>+ if(not InMyPackage):
>+ Errors.remove(er)
>+ else:
>+ logging.error(str(er))
>+ tc.LogStdError(str(er))
>+
>+ # add result to test case
>+ overall_status = len(Errors)
>+ if overall_status is not 0:
>+ tc.SetFailed("GuidCheck {0} Failed. Errors {1}".format(
>+ packagename, overall_status), "CHECK_FAILED")
>+ else:
>+ tc.SetSuccess()
>+ return overall_status
>+
>+
>+class ErrorEntry():
>+ """ Custom/private class for reporting errors in the GuidList
>+ """
>+
>+ def __init__(self, errortype):
>+ self.type = errortype # 'guid' or 'name' depending on error type
>+ self.entries = [] # GuidListEntry that are in error condition
>+
>+ def __str__(self):
>+ a = f"Error Duplicate {self.type}: "
>+ if(self.type == "guid"):
>+ a += f" {self.entries[0].guid}"
>+ elif(self.type == "name"):
>+ a += f" {self.entries[0].name}"
>+
>+ a += f" ({len(self.entries)})\n"
>+
>+ for e in self.entries:
>+ a += "\t" + str(e) + "\n"
>+ return a
>diff --git a/.pytool/Plugin/GuidCheck/GuidCheck_plug_in.yaml
>b/.pytool/Plugin/GuidCheck/GuidCheck_plug_in.yaml
>new file mode 100644
>index 0000000000..531efb7885
>--- /dev/null
>+++ b/.pytool/Plugin/GuidCheck/GuidCheck_plug_in.yaml
>@@ -0,0 +1,11 @@
>+##
>+# CiBuildPlugin used to check guid uniqueness
>+#
>+# Copyright (c) 2019, Microsoft Corporation
>+# SPDX-License-Identifier: BSD-2-Clause-Patent
>+##
>+{
>+ "scope": "cibuild",
>+ "name": "Guid Check Test",
>+ "module": "GuidCheck"
>+}
>diff --git a/.pytool/Plugin/GuidCheck/Readme.md
>b/.pytool/Plugin/GuidCheck/Readme.md
>new file mode 100644
>index 0000000000..c1bf3d728e
>--- /dev/null
>+++ b/.pytool/Plugin/GuidCheck/Readme.md
>@@ -0,0 +1,60 @@
>+# Guid Check Plugin
>+
>+This CiBuildPlugin scans all the files in a code tree to find all the GUID
>definitions. After collection it will then look for duplication in the package
>under test. Uniqueness of all GUIDs are critical within the UEFI environment.
>Duplication can cause numerous issues including locating the wrong data
>structure, calling the wrong function, or decoding the wrong data members.
>+
>+Currently Scanned:
>+
>+* INF files are scanned for there Module guid
>+* DEC files are scanned for all of their Protocols, PPIs, and Guids as well as
>the one package GUID.
>+
>+Any GUID value being equal to two names or even just defined in two files is
>considered an error unless in the ignore list.
>+
>+Any GUID name that is found more than once is an error unless all
>occurrences are Module GUIDs. Since the Module GUID is assigned to the
>Module name it is common to have numerous versions of the same module
>named the same.
>+
>+## Configuration
>+
>+The plugin has numerous configuration options to support the UEFI
>codebase.
>+
>+``` yaml
>+"GuidCheck": {
>+ "IgnoreGuidName": [],
>+ "IgnoreGuidValue": [],
>+ "IgnoreFoldersAndFiles": [],
>+ "IgnoreDuplicates": []
>+ }
>+```
>+
>+### IgnoreGuidName
>+
>+This list allows strings in two formats.
>+
>+* _GuidName_
>+ * This will remove any entry with this GuidName from the list of GUIDs
>therefore ignoring any error associated with this name.
>+* _GuidName=GuidValue_
>+ * This will also ignore the GUID by name but only if the value equals the
>GuidValue.
>+ * GuidValue should be in registry format.
>+ * This is the suggested format to use as it will limit the ignore to only the
>defined case.
>+
>+### IgnoreGuidValue
>+
>+This list allows strings in guid registry format _GuidValue_.
>+
>+* This will remove any entry with this GuidValue from the list of GUIDs
>therefore ignoring any error associated with this value.
>+* GuidValue must be in registry format xxxxxxxx-xxxx-xxxx-xxxx-
>xxxxxxxxxxxx
>+
>+### IgnoreFoldersAndFiles
>+
>+This supports .gitignore file and folder matching strings including wildcards
>+
>+* Any folder or file ignored will not be parsed and therefore any GUID
>defined will be ignored.
>+* The plugin will always ignores the following ["/Build", "/Conf"]
>+
>+### IgnoreDuplicates
>+
>+This supports strings in the format of
>_GuidName_=_GuidName_=_GuidName_
>+
>+* For the error with the GuidNames to be ignored the list must match
>completely with what is found during the code scan.
>+ * For example if there are two GUIDs that are by design equal within the
>code tree then it should be _GuidName_=_GuidName_
>+ * If instead there are three GUIDs then it must be
>_GuidName_=_GuidName_=_GuidName_
>+* This is the best ignore list to use because it is the most strict and will catch
>new problems when new conflicts are introduced.
>+* There are numerous places in the UEFI specification in which two GUID
>names are assigned the same value. These names should be set in this ignore
>list so that they don't cause an error but any additional duplication would still
>be caught.
>diff --git a/.pytool/Plugin/LibraryClassCheck/LibraryClassCheck.py
>b/.pytool/Plugin/LibraryClassCheck/LibraryClassCheck.py
>new file mode 100644
>index 0000000000..33745dff11
>--- /dev/null
>+++ b/.pytool/Plugin/LibraryClassCheck/LibraryClassCheck.py
>@@ -0,0 +1,153 @@
>+# @file LibraryClassCheck.py
>+#
>+# Copyright (c) Microsoft Corporation. All rights reserved.
>+# SPDX-License-Identifier: BSD-2-Clause-Patent
>+##
>+import logging
>+import os
>+from edk2toolext.environment.plugintypes.ci_build_plugin import
>ICiBuildPlugin
>+from edk2toollib.uefi.edk2.parsers.dec_parser import DecParser
>+from edk2toollib.uefi.edk2.parsers.inf_parser import InfParser
>+from edk2toolext.environment.var_dict import VarDict
>+
>+
>+class LibraryClassCheck(ICiBuildPlugin):
>+ """
>+ A CiBuildPlugin that scans the code tree and library classes for undeclared
>+ files
>+
>+ Configuration options:
>+ "LibraryClassCheck": {
>+ IgnoreHeaderFile: [], # Ignore a file found on disk
>+ IgnoreLibraryClass: [] # Ignore a declaration found in dec file
>+ }
>+ """
>+
>+ def GetTestName(self, packagename: str, environment: VarDict) -> tuple:
>+ """ Provide the testcase name and classname for use in reporting
>+ testclassname: a descriptive string for the testcase can include
>whitespace
>+ classname: should be patterned <packagename>.<plugin>.<optionally
>any unique condition>
>+
>+ Args:
>+ packagename: string containing name of package to build
>+ environment: The VarDict for the test to run in
>+ Returns:
>+ a tuple containing the testcase name and the classname
>+ (testcasename, classname)
>+ """
>+ return ("Check library class declarations in " + packagename,
>packagename + ".LibraryClassCheck")
>+
>+ def __GetPkgDec(self, rootpath):
>+ try:
>+ allEntries = os.listdir(rootpath)
>+ for entry in allEntries:
>+ if entry.lower().endswith(".dec"):
>+ return(os.path.join(rootpath, entry))
>+ except Exception:
>+ logging.error("Unable to find DEC for package:{0}".format(rootpath))
>+
>+ return None
>+
>+ ##
>+ # External function of plugin. This function is used to perform the task of
>the MuBuild Plugin
>+ #
>+ # - package is the edk2 path to package. This means
>workspace/packagepath relative.
>+ # - edk2path object configured with workspace and packages path
>+ # - PkgConfig Object (dict) for the pkg
>+ # - EnvConfig Object
>+ # - Plugin Manager Instance
>+ # - Plugin Helper Obj Instance
>+ # - Junit Logger
>+ # - output_stream the StringIO output stream from this plugin via logging
>+ def RunBuildPlugin(self, packagename, Edk2pathObj, pkgconfig,
>environment, PLM, PLMHelper, tc, output_stream=None):
>+ overall_status = 0
>+ LibraryClassIgnore = []
>+
>+ abs_pkg_path =
>Edk2pathObj.GetAbsolutePathOnThisSytemFromEdk2RelativePath(packagen
>ame)
>+ abs_dec_path = self.__GetPkgDec(abs_pkg_path)
>+ wsr_dec_path =
>Edk2pathObj.GetEdk2RelativePathFromAbsolutePath(abs_dec_path)
>+
>+ if abs_dec_path is None or wsr_dec_path is "" or not
>os.path.isfile(abs_dec_path):
>+ tc.SetSkipped()
>+ tc.LogStdError("No DEC file {0} in package {1}".format(abs_dec_path,
>abs_pkg_path))
>+ return -1
>+
>+ # Get all include folders
>+ dec = DecParser()
>+
>dec.SetBaseAbsPath(Edk2pathObj.WorkspacePath).SetPackagePaths(Edk2pa
>thObj.PackagePathList)
>+ dec.ParseFile(wsr_dec_path)
>+
>+ AllHeaderFiles = []
>+
>+ for includepath in dec.IncludePaths:
>+ ## Get all header files in the library folder
>+ AbsLibraryIncludePath = os.path.join(abs_pkg_path, includepath,
>"Library")
>+ if(not os.path.isdir(AbsLibraryIncludePath)):
>+ continue
>+
>+ hfiles = self.WalkDirectoryForExtension([".h"], AbsLibraryIncludePath)
>+ hfiles = [os.path.relpath(x,abs_pkg_path) for x in hfiles] # make
>package root relative path
>+ hfiles = [x.replace("\\", "/") for x in hfiles] # make package relative
>path
>+
>+ AllHeaderFiles.extend(hfiles)
>+
>+ if len(AllHeaderFiles) == 0:
>+ tc.SetSkipped()
>+ tc.LogStdError(f"No Library include folder in any Include path")
>+ return -1
>+
>+ # Remove ignored paths
>+ if "IgnoreHeaderFile" in pkgconfig:
>+ for a in pkgconfig["IgnoreHeaderFile"]:
>+ try:
>+ tc.LogStdOut("Ignoring Library Header File {0}".format(a))
>+ AllHeaderFiles.remove(a)
>+ except:
>+ tc.LogStdError("LibraryClassCheck.IgnoreHeaderFile -> {0} not
>found. Invalid Header File".format(a))
>+ logging.info("LibraryClassCheck.IgnoreHeaderFile -> {0} not found.
>Invalid Header File".format(a))
>+
>+ if "IgnoreLibraryClass" in pkgconfig:
>+ LibraryClassIgnore = pkgconfig["IgnoreLibraryClass"]
>+
>+
>+ ## Attempt to find library classes
>+ for lcd in dec.LibraryClasses:
>+ ## Check for correct file path separator
>+ if "\\" in lcd.path:
>+ tc.LogStdError("LibraryClassCheck.DecFilePathSeparator -> {0}
>invalid.".format(lcd.path))
>+ logging.error("LibraryClassCheck.DecFilePathSeparator -> {0}
>invalid.".format(lcd.path))
>+ overall_status += 1
>+ continue
>+
>+ if lcd.name in LibraryClassIgnore:
>+ tc.LogStdOut("Ignoring Library Class Name {0}".format(lcd.name))
>+ LibraryClassIgnore.remove(lcd.name)
>+ continue
>+
>+ logging.debug(f"Looking for Library Class {lcd.path}")
>+ try:
>+ AllHeaderFiles.remove(lcd.path)
>+
>+ except ValueError:
>+ tc.LogStdError(f"Library {lcd.name} with path {lcd.path} not found in
>package filesystem")
>+ logging.error(f"Library {lcd.name} with path {lcd.path} not found in
>package filesystem")
>+ overall_status += 1
>+
>+ ## any remaining AllHeaderFiles are not described in DEC
>+ for h in AllHeaderFiles:
>+ tc.LogStdError(f"Library Header File {h} not declared in package DEC
>{wsr_dec_path}")
>+ logging.error(f"Library Header File {h} not declared in package DEC
>{wsr_dec_path}")
>+ overall_status += 1
>+
>+ ## Warn about any invalid library class names in the ignore list
>+ for r in LibraryClassIgnore:
>+ tc.LogStdError("LibraryClassCheck.IgnoreLibraryClass -> {0} not found.
>Library Class not found".format(r))
>+ logging.info("LibraryClassCheck.IgnoreLibraryClass -> {0} not found.
>Library Class not found".format(r))
>+
>+
>+ # If XML object exists, add result
>+ if overall_status is not 0:
>+ tc.SetFailed("LibraryClassCheck {0} Failed. Errors
>{1}".format(wsr_dec_path, overall_status), "CHECK_FAILED")
>+ else:
>+ tc.SetSuccess()
>+ return overall_status
>diff --git a/.pytool/Plugin/LibraryClassCheck/LibraryClassCheck_plug_in.yaml
>b/.pytool/Plugin/LibraryClassCheck/LibraryClassCheck_plug_in.yaml
>new file mode 100644
>index 0000000000..9174453a86
>--- /dev/null
>+++ b/.pytool/Plugin/LibraryClassCheck/LibraryClassCheck_plug_in.yaml
>@@ -0,0 +1,11 @@
>+##
>+# CiBuildPlugin used to check that all library classes are declared correctly in
>dec file
>+#
>+# Copyright (c) 2019, Microsoft Corporation
>+# SPDX-License-Identifier: BSD-2-Clause-Patent
>+##
>+{
>+ "scope": "cibuild",
>+ "name": "Library Class Check Test",
>+ "module": "LibraryClassCheck"
>+}
>diff --git a/.pytool/Plugin/LibraryClassCheck/readme.md
>b/.pytool/Plugin/LibraryClassCheck/readme.md
>new file mode 100644
>index 0000000000..dedee16988
>--- /dev/null
>+++ b/.pytool/Plugin/LibraryClassCheck/readme.md
>@@ -0,0 +1,22 @@
>+# Library Class Check Plugin
>+
>+This CiBuildPlugin scans at all library header files found in the `Library` folders
>in all of the package's declared include directories and ensures that all files
>have a matching LibraryClass declaration in the DEC file for the package. Any
>missing declarations will cause a failure.
>+
>+## Configuration
>+
>+The plugin has a few configuration options to support the UEFI codebase.
>+
>+``` yaml
>+"LibraryClassCheck": {
>+ IgnoreHeaderFile: [], # Ignore a file found on disk
>+ IgnoreLibraryClass: [] # Ignore a declaration found in dec file
>+}
>+```
>+
>+### IgnoreHeaderFile
>+
>+Ignore a file found on disk
>+
>+### IgnoreLibraryClass
>+
>+Ignore a declaration found in dec file
>diff --git a/.pytool/Plugin/SpellCheck/Readme.md
>b/.pytool/Plugin/SpellCheck/Readme.md
>new file mode 100644
>index 0000000000..e0ac835191
>--- /dev/null
>+++ b/.pytool/Plugin/SpellCheck/Readme.md
>@@ -0,0 +1,100 @@
>+# Spell Check Plugin
>+
>+This CiBuildPlugin scans all the files in a given package and checks for spelling
>errors.
>+
>+This plugin requires NodeJs and cspell. If the plugin doesn't find its required
>tools then it will mark the test as skipped.
>+
>+* NodeJS: https://nodejs.org/en/
>+* cspell: https://www.npmjs.com/package/cspell
>+ * Src and doc available: https://github.com/streetsidesoftware/cspell
>+
>+## Configuration
>+
>+The plugin has a few configuration options to support the UEFI codebase.
>+
>+``` yaml
>+ "SpellCheck": {
>+ "AuditOnly": False, # If True, log all errors and then mark as skipped
>+ "IgnoreFiles": [], # use gitignore syntax to ignore errors in matching
>files
>+ "ExtendWords": [], # words to extend to the dictionary for this
>package
>+ "IgnoreStandardPaths": [], # Standard Plugin defined paths that should
>be ignore
>+ "AdditionalIncludePaths": [] # Additional paths to spell check (wildcards
>supported)
>+ }
>+```
>+
>+### AuditOnly
>+
>+Boolean - Default is False.
>+If True run the test in an Audit only mode which will log all errors but instead
>of failing the build it will set the test as skipped. This allows visibility into the
>failures without breaking the build.
>+
>+### IgnoreFiles
>+
>+This supports .gitignore file and folder matching strings including wildcards
>+
>+* All files will be parsed regardless but then any spelling errors found within
>ignored files will not be reported as an error.
>+* Errors in ignored files will still be output to the test results as informational
>comments.
>+
>+### ExtendWords
>+
>+This list allows words to be added to the dictionary for the spell checker
>when this package is tested. These follow the rules of the cspell config words
>field.
>+
>+### IgnoreStandardPaths
>+
>+This plugin by default will check the below standard paths. If the package
>would like to ignore any of them list that here.
>+
>+```python
>+[
>+# C source
>+"*.c",
>+"*.h",
>+
>+# Assembly files
>+"*.nasm",
>+"*.asm",
>+"*.masm",
>+"*.s",
>+
>+# ACPI source language
>+"*.asl",
>+
>+# Edk2 build files
>+"*.dsc", "*.dec", "*.fdf", "*.inf",
>+
>+# Documentation files
>+"*.md", "*.txt"
>+]
>+```
>+
>+### AdditionalIncludePaths
>+
>+If the package would to add additional path patterns to be included in
>spellchecking they can be defined here.
>+
>+## Other configuration
>+
>+In the cspell.base.json there are numerous other settings configured. There
>is no support to override these on a per package basis but future features
>could make this available. One interesting configuration option is
>`minWordLength`. Currently it is set to _5_ which means all 2,3, and 4 letter
>words will be ignored. This helps minimize the number of technical acronyms,
>register names, and other UEFI specific values that must be ignored .
>+
>+## False positives
>+
>+The cspell dictionary is not perfect and there are cases where technical
>words or acronyms are not found in the dictionary. There are three ways to
>resolve false positives and the choice for which method should be based on
>how broadly the word should be accepted.
>+
>+### CSpell Base Config file
>+
>+If the change should apply to all UEFI code and documentation then it should
>be added to the base config file `words` section. The base config file is
>adjacent to this file and titled `cspell.base.json`. This is a list of accepted
>words for all spell checking operations on all packages.
>+
>+### Package Config
>+
>+In the package `*.ci.yaml` file there is a `SpellCheck` config section. This
>section allows files to be ignored as well as words that should be considered
>valid for all files within this package. Add the desired words to the
>"ExtendedWords" member.
>+
>+### In-line File
>+
>+CSpell supports numerous methods to annotate your files to ignore words,
>sections, etc. This can be found in CSpell documentation. Suggestion here is
>to use a c-style comment at the top of the file to add words that should be
>ignored just for this file. Obviously this has the highest maintenance cost so it
>should only be used for file unique words.
>+
>+``` c
>+// spell-checker:ignore unenroll, word2, word3
>+```
>+
>+or
>+
>+```ini
>+# spell-checker:ignore unenroll, word2, word3
>+```
>diff --git a/.pytool/Plugin/SpellCheck/SpellCheck.py
>b/.pytool/Plugin/SpellCheck/SpellCheck.py
>new file mode 100644
>index 0000000000..94ca4cd071
>--- /dev/null
>+++ b/.pytool/Plugin/SpellCheck/SpellCheck.py
>@@ -0,0 +1,216 @@
>+# @file SpellCheck.py
>+#
>+# An edk2-pytool based plugin wrapper for cspell
>+#
>+# Copyright (c) Microsoft Corporation. All rights reserved.
>+# SPDX-License-Identifier: BSD-2-Clause-Patent
>+##
>+import logging
>+import json
>+import yaml
>+from io import StringIO
>+import os
>+from edk2toolext.environment.plugintypes.ci_build_plugin import
>ICiBuildPlugin
>+from edk2toollib.utility_functions import RunCmd
>+from edk2toolext.environment.var_dict import VarDict
>+from edk2toollib.gitignore_parser import parse_gitignore_lines
>+from edk2toolext.environment import version_aggregator
>+
>+
>+class SpellCheck(ICiBuildPlugin):
>+ """
>+ A CiBuildPlugin that uses the cspell node module to scan the files
>+ from the package being tested for spelling errors. The plugin contains
>+ the base cspell.json file then thru the configuration options other settings
>+ can be changed or extended.
>+
>+ Configuration options:
>+ "SpellCheck": {
>+ "AuditOnly": False, # Don't fail the build if there are errors. Just log
>them
>+ "IgnoreFiles": [], # use gitignore syntax to ignore errors in matching
>files
>+ "ExtendWords": [], # words to extend to the dictionary for this
>package
>+ "IgnoreStandardPaths": [], # Standard Plugin defined paths that should
>be ignore
>+ "AdditionalIncludePaths": [] # Additional paths to spell check (wildcards
>supported)
>+ }
>+ """
>+
>+ #
>+ # A package can remove any of these using IgnoreStandardPaths
>+ #
>+ STANDARD_PLUGIN_DEFINED_PATHS = ["*.c", "*.h",
>+ "*.nasm", "*.asm", "*.masm", "*.s",
>+ "*.asl",
>+ "*.dsc", "*.dec", "*.fdf", "*.inf",
>+ "*.md", "*.txt"
>+ ]
>+
>+ def GetTestName(self, packagename: str, environment: VarDict) -> tuple:
>+ """ Provide the testcase name and classname for use in reporting
>+
>+ Args:
>+ packagename: string containing name of package to build
>+ environment: The VarDict for the test to run in
>+ Returns:
>+ a tuple containing the testcase name and the classname
>+ (testcasename, classname)
>+ testclassname: a descriptive string for the testcase can include
>whitespace
>+ classname: should be patterned
><packagename>.<plugin>.<optionally any unique condition>
>+ """
>+ return ("Spell check files in " + packagename, packagename +
>".SpellCheck")
>+
>+ ##
>+ # External function of plugin. This function is used to perform the task of
>the CiBuild Plugin
>+ #
>+ # - package is the edk2 path to package. This means
>workspace/packagepath relative.
>+ # - edk2path object configured with workspace and packages path
>+ # - PkgConfig Object (dict) for the pkg
>+ # - EnvConfig Object
>+ # - Plugin Manager Instance
>+ # - Plugin Helper Obj Instance
>+ # - Junit Logger
>+ # - output_stream the StringIO output stream from this plugin via logging
>+
>+ def RunBuildPlugin(self, packagename, Edk2pathObj, pkgconfig,
>environment, PLM, PLMHelper, tc, output_stream=None):
>+ Errors = []
>+
>+ abs_pkg_path =
>Edk2pathObj.GetAbsolutePathOnThisSytemFromEdk2RelativePath(
>+ packagename)
>+
>+ if abs_pkg_path is None:
>+ tc.SetSkipped()
>+ tc.LogStdError("No package {0}".format(packagename))
>+ return -1
>+
>+ # check for node
>+ return_buffer = StringIO()
>+ ret = RunCmd("node", "--version", outstream=return_buffer)
>+ if (ret != 0):
>+ tc.SetSkipped()
>+ tc.LogStdError("NodeJs not installed. Test can't run")
>+ logging.warning("NodeJs not installed. Test can't run")
>+ return -1
>+ node_version = return_buffer.getvalue().strip() # format vXX.XX.XX
>+ tc.LogStdOut(f"Node version: {node_version}")
>+ version_aggregator.GetVersionAggregator().ReportVersion(
>+ "NodeJs", node_version, version_aggregator.VersionTypes.INFO)
>+
>+ # Check for cspell
>+ return_buffer = StringIO()
>+ ret = RunCmd("cspell", "--version", outstream=return_buffer)
>+ if (ret != 0):
>+ tc.SetSkipped()
>+ tc.LogStdError("cspell not installed. Test can't run")
>+ logging.warning("cspell not installed. Test can't run")
>+ return -1
>+ cspell_version = return_buffer.getvalue().strip() # format XX.XX.XX
>+ tc.LogStdOut(f"CSpell version: {cspell_version}")
>+ version_aggregator.GetVersionAggregator().ReportVersion(
>+ "CSpell", cspell_version, version_aggregator.VersionTypes.INFO)
>+
>+ package_relative_paths_to_spell_check =
>SpellCheck.STANDARD_PLUGIN_DEFINED_PATHS
>+
>+ #
>+ # Allow the ci.yaml to remove any of the above standard paths
>+ #
>+ if("IgnoreStandardPaths" in pkgconfig):
>+ for a in pkgconfig["IgnoreStandardPaths"]:
>+ if(a in package_relative_paths_to_spell_check):
>+ tc.LogStdOut(
>+ f"ignoring standard path due to ci.yaml ignore: {a}")
>+ package_relative_paths_to_spell_check.remove(a)
>+ else:
>+ tc.LogStdOut(f"Invalid IgnoreStandardPaths value: {a}")
>+
>+ #
>+ # check for any additional include paths defined by package config
>+ #
>+ if("AdditionalIncludePaths" in pkgconfig):
>+ package_relative_paths_to_spell_check.extend(
>+ pkgconfig["AdditionalIncludePaths"])
>+
>+ #
>+ # Make the path string for cspell to check
>+ #
>+ relpath = os.path.relpath(abs_pkg_path)
>+ cpsell_paths = " ".join(
>+ [f"{relpath}/**/{x}" for x in package_relative_paths_to_spell_check])
>+
>+ # Make the config file
>+ config_file_path = os.path.join(
>+ Edk2pathObj.WorkspacePath, "Build", packagename,
>"cspell_actual_config.json")
>+ mydir = os.path.dirname(os.path.abspath(__file__))
>+ # load as yaml so it can have comments
>+ base = os.path.join(mydir, "cspell.base.yaml")
>+ with open(base, "r") as i:
>+ config = yaml.safe_load(i)
>+
>+ if("ExtendWords" in pkgconfig):
>+ config["words"].extend(pkgconfig["ExtendWords"])
>+ with open(config_file_path, "w") as o:
>+ json.dump(config, o) # output as json so compat with cspell
>+
>+ All_Ignores = []
>+ # Parse the config for other ignores
>+ if "IgnoreFiles" in pkgconfig:
>+ All_Ignores.extend(pkgconfig["IgnoreFiles"])
>+
>+ # spell check all the files
>+ ignore = parse_gitignore_lines(All_Ignores, os.path.join(
>+ abs_pkg_path, "nofile.txt"), abs_pkg_path)
>+
>+ # result is a list of strings like this
>+ # C:\src\sp-edk2\edk2\FmpDevicePkg\FmpDevicePkg.dec:53:9 -
>Unknown word (Capule)
>+ EasyFix = []
>+ results = self._check_spelling(cpsell_paths, config_file_path)
>+ for r in results:
>+ path, _, word = r.partition(" - Unknown word ")
>+ if len(word) == 0:
>+ # didn't find pattern
>+ continue
>+
>+ pathinfo = path.rsplit(":", 2) # remove the line no info
>+ if(ignore(pathinfo[0])): # check against ignore list
>+ tc.LogStdOut(f"ignoring error due to ci.yaml ignore: {r}")
>+ continue
>+
>+ # real error
>+ EasyFix.append(word.strip().strip("()"))
>+ Errors.append(r)
>+
>+ # Log all errors tc StdError
>+ for l in Errors:
>+ tc.LogStdError(l.strip())
>+
>+ # Helper - Log the syntax needed to add these words to dictionary
>+ if len(EasyFix) > 0:
>+ EasyFix = sorted(set(a.lower() for a in EasyFix))
>+ tc.LogStdOut("\n Easy fix:")
>+ OneString = "If these are not errors add this to your ci.yaml file.\n"
>+ OneString += '"SpellCheck": {\n "ExtendWords": ['
>+ for a in EasyFix:
>+ tc.LogStdOut(f'\n"{a}",')
>+ OneString += f'\n "{a}",'
>+ logging.info(OneString.rstrip(",") + '\n ]\n}')
>+
>+ # add result to test case
>+ overall_status = len(Errors)
>+ if overall_status != 0:
>+ if "AuditOnly" in pkgconfig and pkgconfig["AuditOnly"]:
>+ # set as skipped if AuditOnly
>+ tc.SetSkipped()
>+ return -1
>+ else:
>+ tc.SetFailed("SpellCheck {0} Failed. Errors {1}".format(
>+ packagename, overall_status), "CHECK_FAILED")
>+ else:
>+ tc.SetSuccess()
>+ return overall_status
>+
>+ def _check_spelling(self, abs_file_to_check: str, abs_config_file_to_use:
>str) -> []:
>+ output = StringIO()
>+ ret = RunCmd(
>+ "cspell", f"--config {abs_config_file_to_use} {abs_file_to_check}",
>outstream=output)
>+ if ret == 0:
>+ return []
>+ else:
>+ return output.getvalue().strip().splitlines()
>diff --git a/.pytool/Plugin/SpellCheck/SpellCheck_plug_in.yaml
>b/.pytool/Plugin/SpellCheck/SpellCheck_plug_in.yaml
>new file mode 100644
>index 0000000000..161045e19e
>--- /dev/null
>+++ b/.pytool/Plugin/SpellCheck/SpellCheck_plug_in.yaml
>@@ -0,0 +1,11 @@
>+##
>+# CiBuildPlugin used to check spelling
>+#
>+# Copyright (c) 2019, Microsoft Corporation
>+# SPDX-License-Identifier: BSD-2-Clause-Patent
>+##
>+{
>+ "scope": "cibuild",
>+ "name": "Spell Check Test",
>+ "module": "SpellCheck"
>+}
>diff --git a/.pytool/Plugin/SpellCheck/cspell.base.yaml
>b/.pytool/Plugin/SpellCheck/cspell.base.yaml
>new file mode 100644
>index 0000000000..da6c5e5da7
>--- /dev/null
>+++ b/.pytool/Plugin/SpellCheck/cspell.base.yaml
>@@ -0,0 +1,165 @@
>+##
>+# CSpell configuration
>+#
>+# Copyright (c) Microsoft Corporation
>+# SPDX-License-Identifier: BSD-2-Clause-Patent
>+##
>+{
>+ "version": "0.1",
>+ "language": "en",
>+ "dictionaries": [
>+ "companies ",
>+ "softwareTerms",
>+ "python",
>+ "cpp"
>+ ],
>+ "ignorePaths": [
>+ "*.pdb",
>+ "**/*_extdep/**",
>+ "*.pdf",
>+ "*.exe",
>+ "*.jpg"
>+ ],
>+ "minWordLength": 5,
>+ "allowCompoundWords": false,
>+ "ignoreWords": [
>+ "muchange"
>+ ],
>+ "words": [
>+ "MTRRs",
>+ "Microarchitecture",
>+ "Goldmont",
>+ "cpuid",
>+ "mwait",
>+ "cstate",
>+ "smram",
>+ "scrtm",
>+ "smbus",
>+ "selftest",
>+ "socket",
>+ "MMRAM",
>+ "qword",
>+ "ENDBR",
>+ "SMBASE",
>+ "FXSAVE",
>+ "FXRSTOR",
>+ "RDRAND",
>+ "IOAPIC",
>+ "ATAPI",
>+ "movsb",
>+ "iretw",
>+ "XENSTORE",
>+ "cdrom",
>+ "oprom",
>+ "oproms",
>+ "varstore",
>+ "EKU",
>+ "ascii",
>+ "nmake",
>+ "NVDIMM",
>+ "nasmb",
>+ "Mtftp",
>+ "Hypercall",
>+ "hypercalls",
>+ "IOMMU",
>+ "QEMU",
>+ "qemus",
>+ "OVMF",
>+ "tiano",
>+ "tianocore",
>+ "edkii",
>+ "coreboot",
>+ "uefipayload",
>+ "bootloader",
>+ "bootloaders",
>+ "mdepkg",
>+ "skuid",
>+ "dxefv",
>+ "toolchain",
>+ "libraryclass",
>+ "preboot",
>+ "pythonpath",
>+ "cygpath",
>+ "nuget",
>+ "basetools",
>+ "prepi",
>+ "OPTEE",
>+ "stringid",
>+ "peims",
>+ "memmap",
>+ "guids",
>+ "uuids",
>+ "smbios",
>+ "certdb",
>+ "certdbv",
>+ "EfiSigList",
>+ "depex",
>+ "IHANDLE",
>+ "Virtio",
>+ "Mbytes",
>+ "Citrix",
>+ "initrd",
>+ "semihost",
>+ "Semihosting",
>+ "Trustzone",
>+ "Fastboot",
>+ "framebuffer",
>+ "genfw",
>+ "TTYTERM",
>+ "miniport",
>+ "LFENCE",
>+ "PCANSI",
>+ "submodule",
>+ "submodules",
>+ "brotli",
>+ "PCCTS",
>+ "softfloat",
>+ "whitepaper",
>+ "ACPICA",
>+ "plugfest",
>+ "bringup",
>+ "formset", #VFR
>+ "ideqvallist",
>+ "numberof",
>+ "oneof",
>+ "endformset",
>+ "endnumeric",
>+ "endoneof",
>+ "disableif",
>+ "guidid",
>+ "classguid",
>+ "efivarstore",
>+ "formsetguid",
>+ "formid",
>+ "suppressif",
>+ "grayoutif",
>+ "ideqval",
>+ "endform",
>+ "endcheckbox",
>+ "questionid",
>+ "questionref",
>+ "enddate",
>+ "endstring",
>+ "guidop",
>+ "endguidop",
>+ "langdef",
>+ "dynamicex",
>+ "tokenspace",
>+ "tokenguid",
>+ "pcd's", #seems like cspell bug
>+ "peim's",
>+ "autogen",
>+ "Disasm",
>+ "Torito",
>+ "SRIOV",
>+ "MRIOV",
>+ "UARTs",
>+ "Consplitter", # common module in UEFI
>+ "FIFOs",
>+ "ACPINVS",
>+ "Endof", # due to of not being uppercase
>+ "bootability",
>+ "Sdhci",
>+ "inmodule",
>+ ]
>+}
>--
>2.21.0.windows.1
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [Patch v4 02/22] .gitignore: Ignore python compiled files, extdeps, and vscode
2019-11-07 1:13 ` [Patch v4 02/22] .gitignore: Ignore python compiled files, extdeps, and vscode Michael D Kinney
@ 2019-11-07 10:26 ` Laszlo Ersek
0 siblings, 0 replies; 51+ messages in thread
From: Laszlo Ersek @ 2019-11-07 10:26 UTC (permalink / raw)
To: Michael D Kinney, devel
Cc: Sean Brogan, Andrew Fish, Leif Lindholm, Philippe Mathieu-Daude
On 11/07/19 02:13, Michael D Kinney wrote:
> From: Sean Brogan <sean.brogan@microsoft.com>
>
> Update .gitignore to ignore .pyc files and __pycache__
> directories. Python based plugins can be added to any
> package or platform, so these files and directories may
> be present outside of BaseTools.
>
> Ignore _extdep directories that are generated by the
> pytool external dependency feature.
>
> Ignore .vscode directories generated by the VS Code
> editor.
>
> Cc: Andrew Fish <afish@apple.com>
> Cc: Laszlo Ersek <lersek@redhat.com>
> Cc: Leif Lindholm <leif.lindholm@linaro.org>
> Signed-off-by: Kinney <michael.d.kinney@intel.com>
> Reviewed-by: Leif Lindholm <leif.lindholm@linaro.org>
> Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
> ---
> .gitignore | 6 +++++-
> 1 file changed, 5 insertions(+), 1 deletion(-)
>
> diff --git a/.gitignore b/.gitignore
> index 97f22c348c..1dd30c1410 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -1,3 +1,7 @@
> Build/
> -tags/
> .DS_Store
> +*_extdep/
> +*.pyc
> +__pycache__/
> +tags/
> +.vscode/
>
This seems to be sorted only if we ignore the various prefixes ("*",
"_", "."). I tend to call such a list "sorted" if it filters through the
"sort" utility (using the POSIX locale) without changes.
But, this is certainly not important enough to justify another repost.
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Thanks!
Laszlo
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [Patch v4 03/22] requirements.txt: Add python pip requirements file
2019-11-07 1:13 ` [Patch v4 03/22] requirements.txt: Add python pip requirements file Michael D Kinney
@ 2019-11-07 10:39 ` Laszlo Ersek
2019-11-07 15:43 ` Leif Lindholm
2019-11-07 17:44 ` Michael D Kinney
2019-11-07 10:49 ` Laszlo Ersek
1 sibling, 2 replies; 51+ messages in thread
From: Laszlo Ersek @ 2019-11-07 10:39 UTC (permalink / raw)
To: Michael D Kinney, devel; +Cc: Sean Brogan, Andrew Fish, Leif Lindholm
On 11/07/19 02:13, Michael D Kinney wrote:
> From: Sean Brogan <sean.brogan@microsoft.com>
>
> Add pip requirements file that is used to install the
> python pip modules build from the edk2-pytool-library and
> edk2-pytool-extensions repositories.
>
> These python modules provide the extensions required to
> perform EDK II Continuous Integration(CI) builds.
>
> Cc: Andrew Fish <afish@apple.com>
> Cc: Laszlo Ersek <lersek@redhat.com>
> Cc: Leif Lindholm <leif.lindholm@linaro.org>
> Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
> ---
> requirements.txt | 17 +++++++++++++++++
> 1 file changed, 17 insertions(+)
> create mode 100644 requirements.txt
>
> diff --git a/requirements.txt b/requirements.txt
> new file mode 100644
> index 0000000000..4ad72cfc98
> --- /dev/null
> +++ b/requirements.txt
> @@ -0,0 +1,17 @@
> +## @file
> +# EDK II Python PIP requirements file
> +#
> +# This file provides the list of python components to install using PIP.
> +#
> +# Copyright (c) Microsoft Corporation. All rights reserved.<BR>
> +#
> +# SPDX-License-Identifier: BSD-2-Clause-Patent
> +#
> +# https://pypi.org/project/pip/
> +# https://pip.pypa.io/en/stable/user_guide/#requirements-files
> +# https://pip.pypa.io/en/stable/reference/pip_install/#requirements-file-format
> +#
> +##
> +
> +edk2-pytool-library==0.10.*
> +edk2-pytool-extensions==0.12.*
>
This is better, but I still find the plain "requirements.txt" filename
in the root directory of the project very confusing. What component
*exactly* insists on this file path and file name?
I've checked
https://pip.pypa.io/en/stable/user_guide/#requirements-files
and it seems like "pip" can take any pathname as an argument to option "-r".
There must be a component in the CI environment that invokes "pip". Can
we file a feature request for that component, that it try
"pip-requirements.txt" first?
Hmmm... I just googled "github pip requirements.txt", assuming that
"github" was the component calling "pip". In the result list, I've found:
https://github.com/ClearingHouse/clearinghoused/blob/master/pip-requirements.txt
Is it possible that github already knows to look for
"pip-requirements.txt"? (Honestly I only suggested
"pip-requirements.txt" above because it seemed sensible.) If that's the
case, we should use it.
Furthermore, my understanding is that "the list of python components to
install using PIP" is only there for CI purposes. Can we please state
that explicitly in the comment block? (The commit message already does
that, and that's great.)
Basically when someone clones edk2 and runs "ls -l" for the first time,
there's a good chance "requirements.txt" will be among the few files
they open right after.
Thanks!
Laszlo
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [Patch v4 17/22] UefiCpuPkg: Add YAML file for CI builds
2019-11-07 1:13 ` [Patch v4 17/22] UefiCpuPkg: " Michael D Kinney
2019-11-07 2:12 ` Ni, Ray
@ 2019-11-07 10:42 ` Laszlo Ersek
2019-11-07 10:48 ` Laszlo Ersek
1 sibling, 1 reply; 51+ messages in thread
From: Laszlo Ersek @ 2019-11-07 10:42 UTC (permalink / raw)
To: Michael D Kinney, devel; +Cc: Eric Dong, Ray Ni
On 11/07/19 02:13, Michael D Kinney wrote:
> Add YAML file to the package directory with the
> configuration of the checks to perform during a
> CI build.
>
> Cc: Eric Dong <eric.dong@intel.com>
> Cc: Ray Ni <ray.ni@intel.com>
> Cc: Laszlo Ersek <lersek@redhat.com>
> Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
> Acked-by: Laszlo Ersek <lersek@redhat.com>
> ---
> UefiCpuPkg/UefiCpuPkg.ci.yaml | 51 +++++++++++++++++++++++++++++++++++
> 1 file changed, 51 insertions(+)
> create mode 100644 UefiCpuPkg/UefiCpuPkg.ci.yaml
>
> diff --git a/UefiCpuPkg/UefiCpuPkg.ci.yaml b/UefiCpuPkg/UefiCpuPkg.ci.yaml
> new file mode 100644
> index 0000000000..81da29b878
> --- /dev/null
> +++ b/UefiCpuPkg/UefiCpuPkg.ci.yaml
> @@ -0,0 +1,51 @@
> +##
> +# CI configuration for UefiCpuPkg
> +#
> +# Copyright (c) Microsoft Corporation
> +# SPDX-License-Identifier: BSD-2-Clause-Patent
> +##
> +{
> + "CompilerPlugin": {
> + "DscPath": "UefiCpuPkg.dsc"
> + },
> + "CharEncodingCheck": {
> + "IgnoreFiles": []
> + },
> + "DependencyCheck": {
> + "AcceptableDependencies": [
> + "MdePkg/MdePkg.dec",
> + "MdeModulePkg/MdeModulePkg.dec",
> + "UefiCpuPkg/UefiCpuPkg.dec"
> + ],
> + # For host based unit tests
> + "AcceptableDependencies-HOST_APPLICATION":[],
> + # For UEFI shell based apps
> + "AcceptableDependencies-UEFI_APPLICATION":[],
> + "IgnoreInf": []
> + },
> + "DscCompleteCheck": {
> + "DscPath": "UefiCpuPkg.dsc",
> + "IgnoreInf": [
> + "UefiCpuPkg/ResetVector/FixupVtf/Vtf.inf",
> + "UefiCpuPkg/ResetVector/Vtf0/Vtf0.inf"
> + ]
> + },
> + "GuidCheck": {
> + "IgnoreGuidName": ["SecCore", "ResetVector"], # Expected duplication for gEfiFirmwareVolumeTopFileGuid
> + "IgnoreGuidValue": [],
> + "IgnoreFoldersAndFiles": [],
> + "IgnoreDuplicates": []
> + },
> + "LibraryClassCheck": {
> + "IgnoreHeaderFile": []
> + },
> +
> + ## options defined ci/Plugin/SpellCheck
> + "SpellCheck": {
> + "AuditOnly": True, # Fails test but run in AuditOnly mode to collect log
> + "IgnoreFiles": [], # use gitignore syntax to ignore errors in matching files
> + "ExtendWords": [], # words to extend to the dictionary for this package
> + "IgnoreStandardPaths": [], # Standard Plugin defined paths that should be ignore
> + "AdditionalIncludePaths": [] # Additional paths to spell check (wildcards supported)
> + }
> +}
>
This is a very nice patch: good commit message (subjet and body both),
and good file comment.
One suggestion: I'd recommend adding "@file" to the comment block at the
top.
With that:
Acked-by: Laszlo Ersek <lersek@redhat.com>
(Of course, should "@file" confuse the YAML parser, then I won't insist,
and my ACK still applies.)
Thanks!
Laszlo
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [Patch v4 22/22] Readme.md: Add CI build status badges
2019-11-07 1:13 ` [Patch v4 22/22] Readme.md: Add CI build status badges Michael D Kinney
@ 2019-11-07 10:44 ` Laszlo Ersek
2019-11-07 16:00 ` Leif Lindholm
1 sibling, 0 replies; 51+ messages in thread
From: Laszlo Ersek @ 2019-11-07 10:44 UTC (permalink / raw)
To: Michael D Kinney, devel; +Cc: Andrew Fish, Leif Lindholm
On 11/07/19 02:13, Michael D Kinney wrote:
> Update Readme.md with status badges from the most recent
> EDK II Continuous Integration(CI) build.
>
> Cc: Andrew Fish <afish@apple.com>
> Cc: Laszlo Ersek <lersek@redhat.com>
> Cc: Leif Lindholm <leif.lindholm@linaro.org>
> Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
> ---
> Readme.md | 11 +++++++++++
> 1 file changed, 11 insertions(+)
>
> diff --git a/Readme.md b/Readme.md
> index 27e4ce0771..0207923763 100644
> --- a/Readme.md
> +++ b/Readme.md
> @@ -3,6 +3,17 @@
> A modern, feature-rich, cross-platform firmware development environment
> for the UEFI and PI specifications from www.uefi.org.
>
> +# Build Status
> +
> +| Host Type | Toolchain | Branch | Build Status | Test Status | Code Coverage |
> +| :-------- | :-------- | :------ | :----------- | :---------- | :------------ |
> +| Windows | VS2019 | master | [](https://dev.azure.com/tianocore/edk2-ci-play/_build/latest?definitionId=27&branchName=master) | [](https://dev.azure.com/tianocore/edk2-ci-play/_build/latest?definitionId=27&branchName=master) | [](https://dev.azure.com/tianocore/edk2-ci-play/_build/latest?definitionId=27&branchName=master) |
> +| Ubuntu | GCC | master | [](https://dev.azure.com/tianocore/edk2-ci-play/_build/latest?definitionId=26&branchName=master) | [](https://dev.azure.com/tianocore/edk2-ci-play/_build/latest?definitionId=26&branchName=master) | [](https://dev.azure.com/tianocore/edk2-ci-play/_build/latest?definitionId=26&branchName=master) |
> +
> +[More CI Build information](.pytool/Readme.md)
> +
> +# License Details
> +
> The majority of the content in the EDK II open source project uses a
> [BSD-2-Clause Plus Patent License](License.txt). The EDK II open source project
> contains the following components that are covered by additional licenses:
>
I'll defer to Leif on the source-level review of this change. :)
I did check out the "live" version, at:
https://github.com/tianocore/edk2-staging/tree/edk2-ci_V4#build-status
It looks nice to me.
Acked-by: Laszlo Ersek <lersek@redhat.com>
Thanks!
Laszlo
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [Patch v4 17/22] UefiCpuPkg: Add YAML file for CI builds
2019-11-07 10:42 ` Laszlo Ersek
@ 2019-11-07 10:48 ` Laszlo Ersek
2019-11-07 19:23 ` [edk2-devel] " Michael D Kinney
0 siblings, 1 reply; 51+ messages in thread
From: Laszlo Ersek @ 2019-11-07 10:48 UTC (permalink / raw)
To: Michael D Kinney, devel; +Cc: Eric Dong, Ray Ni
On 11/07/19 11:42, Laszlo Ersek wrote:
> On 11/07/19 02:13, Michael D Kinney wrote:
>> Add YAML file to the package directory with the
>> configuration of the checks to perform during a
>> CI build.
>>
>> Cc: Eric Dong <eric.dong@intel.com>
>> Cc: Ray Ni <ray.ni@intel.com>
>> Cc: Laszlo Ersek <lersek@redhat.com>
>> Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
>> Acked-by: Laszlo Ersek <lersek@redhat.com>
>> ---
>> UefiCpuPkg/UefiCpuPkg.ci.yaml | 51 +++++++++++++++++++++++++++++++++++
>> 1 file changed, 51 insertions(+)
>> create mode 100644 UefiCpuPkg/UefiCpuPkg.ci.yaml
>>
>> diff --git a/UefiCpuPkg/UefiCpuPkg.ci.yaml b/UefiCpuPkg/UefiCpuPkg.ci.yaml
>> new file mode 100644
>> index 0000000000..81da29b878
>> --- /dev/null
>> +++ b/UefiCpuPkg/UefiCpuPkg.ci.yaml
>> @@ -0,0 +1,51 @@
>> +##
>> +# CI configuration for UefiCpuPkg
>> +#
>> +# Copyright (c) Microsoft Corporation
>> +# SPDX-License-Identifier: BSD-2-Clause-Patent
>> +##
>> +{
>> + "CompilerPlugin": {
>> + "DscPath": "UefiCpuPkg.dsc"
>> + },
>> + "CharEncodingCheck": {
>> + "IgnoreFiles": []
>> + },
>> + "DependencyCheck": {
>> + "AcceptableDependencies": [
>> + "MdePkg/MdePkg.dec",
>> + "MdeModulePkg/MdeModulePkg.dec",
>> + "UefiCpuPkg/UefiCpuPkg.dec"
>> + ],
>> + # For host based unit tests
>> + "AcceptableDependencies-HOST_APPLICATION":[],
>> + # For UEFI shell based apps
>> + "AcceptableDependencies-UEFI_APPLICATION":[],
>> + "IgnoreInf": []
>> + },
>> + "DscCompleteCheck": {
>> + "DscPath": "UefiCpuPkg.dsc",
>> + "IgnoreInf": [
>> + "UefiCpuPkg/ResetVector/FixupVtf/Vtf.inf",
>> + "UefiCpuPkg/ResetVector/Vtf0/Vtf0.inf"
>> + ]
>> + },
>> + "GuidCheck": {
>> + "IgnoreGuidName": ["SecCore", "ResetVector"], # Expected duplication for gEfiFirmwareVolumeTopFileGuid
>> + "IgnoreGuidValue": [],
>> + "IgnoreFoldersAndFiles": [],
>> + "IgnoreDuplicates": []
>> + },
>> + "LibraryClassCheck": {
>> + "IgnoreHeaderFile": []
>> + },
>> +
>> + ## options defined ci/Plugin/SpellCheck
>> + "SpellCheck": {
>> + "AuditOnly": True, # Fails test but run in AuditOnly mode to collect log
>> + "IgnoreFiles": [], # use gitignore syntax to ignore errors in matching files
>> + "ExtendWords": [], # words to extend to the dictionary for this package
>> + "IgnoreStandardPaths": [], # Standard Plugin defined paths that should be ignore
>> + "AdditionalIncludePaths": [] # Additional paths to spell check (wildcards supported)
>> + }
>> +}
>>
>
> This is a very nice patch: good commit message (subjet and body both),
> and good file comment.
>
> One suggestion: I'd recommend adding "@file" to the comment block at the
> top.
>
> With that:
>
> Acked-by: Laszlo Ersek <lersek@redhat.com>
>
> (Of course, should "@file" confuse the YAML parser, then I won't insist,
> and my ACK still applies.)
Ah, another remark: I believe the copyright notice should state a year.
(Most likely "2019".)
Thanks
Laszlo
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [Patch v4 03/22] requirements.txt: Add python pip requirements file
2019-11-07 1:13 ` [Patch v4 03/22] requirements.txt: Add python pip requirements file Michael D Kinney
2019-11-07 10:39 ` Laszlo Ersek
@ 2019-11-07 10:49 ` Laszlo Ersek
1 sibling, 0 replies; 51+ messages in thread
From: Laszlo Ersek @ 2019-11-07 10:49 UTC (permalink / raw)
To: Michael D Kinney, devel; +Cc: Sean Brogan, Andrew Fish, Leif Lindholm
On 11/07/19 02:13, Michael D Kinney wrote:
> From: Sean Brogan <sean.brogan@microsoft.com>
>
> Add pip requirements file that is used to install the
> python pip modules build from the edk2-pytool-library and
> edk2-pytool-extensions repositories.
>
> These python modules provide the extensions required to
> perform EDK II Continuous Integration(CI) builds.
>
> Cc: Andrew Fish <afish@apple.com>
> Cc: Laszlo Ersek <lersek@redhat.com>
> Cc: Leif Lindholm <leif.lindholm@linaro.org>
> Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
> ---
> requirements.txt | 17 +++++++++++++++++
> 1 file changed, 17 insertions(+)
> create mode 100644 requirements.txt
>
> diff --git a/requirements.txt b/requirements.txt
> new file mode 100644
> index 0000000000..4ad72cfc98
> --- /dev/null
> +++ b/requirements.txt
> @@ -0,0 +1,17 @@
> +## @file
> +# EDK II Python PIP requirements file
> +#
> +# This file provides the list of python components to install using PIP.
> +#
> +# Copyright (c) Microsoft Corporation. All rights reserved.<BR>
The year seems to be missing from this copyright notice too.
Thanks
Laszlo
> +#
> +# SPDX-License-Identifier: BSD-2-Clause-Patent
> +#
> +# https://pypi.org/project/pip/
> +# https://pip.pypa.io/en/stable/user_guide/#requirements-files
> +# https://pip.pypa.io/en/stable/reference/pip_install/#requirements-file-format
> +#
> +##
> +
> +edk2-pytool-library==0.10.*
> +edk2-pytool-extensions==0.12.*
>
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [Patch v4 00/22] Enable Phase 1 of EDK II CI
2019-11-07 1:13 [Patch v4 00/22] Enable Phase 1 of EDK II CI Michael D Kinney
` (21 preceding siblings ...)
2019-11-07 1:13 ` [Patch v4 22/22] Readme.md: Add CI build status badges Michael D Kinney
@ 2019-11-07 15:35 ` Liming Gao
22 siblings, 0 replies; 51+ messages in thread
From: Liming Gao @ 2019-11-07 15:35 UTC (permalink / raw)
To: Kinney, Michael D, devel@edk2.groups.io
Cc: Sean Brogan, Bret Barkelew, Feng, Bob C, Andrew Fish,
Laszlo Ersek, Leif Lindholm, Wang, Jian J, Lu, XiaoyuX, Ni, Ray,
Wu, Hao A, Wu, Jiaxin, Fu, Siyuan, Yao, Jiewen, Zhang, Chao B,
Gao, Zhichao, Dong, Eric
Mike:
Thanks for your update. I only have one minor comment in Patch7. The full patch set is OK to me. Reviewed-by: Liming Gao <liming.gao@intel.com>.
Thanks
Liming
> -----Original Message-----
> From: Kinney, Michael D <michael.d.kinney@intel.com>
> Sent: Thursday, November 7, 2019 9:13 AM
> To: devel@edk2.groups.io
> Cc: Sean Brogan <sean.brogan@microsoft.com>; Bret Barkelew <Bret.Barkelew@microsoft.com>; Gao, Liming <liming.gao@intel.com>;
> Feng, Bob C <bob.c.feng@intel.com>; Andrew Fish <afish@apple.com>; Laszlo Ersek <lersek@redhat.com>; Leif Lindholm
> <leif.lindholm@linaro.org>; Wang, Jian J <jian.j.wang@intel.com>; Lu, XiaoyuX <xiaoyux.lu@intel.com>; Ni, Ray <ray.ni@intel.com>; Wu,
> Hao A <hao.a.wu@intel.com>; Wu, Jiaxin <jiaxin.wu@intel.com>; Fu, Siyuan <siyuan.fu@intel.com>; Yao, Jiewen
> <jiewen.yao@intel.com>; Zhang, Chao B <chao.b.zhang@intel.com>; Gao, Zhichao <zhichao.gao@intel.com>; Dong, Eric
> <eric.dong@intel.com>
> Subject: [Patch v4 00/22] Enable Phase 1 of EDK II CI
>
> https://bugzilla.tianocore.org/show_bug.cgi?id=2315
>
> RFC Proposals:
> * https://edk2.groups.io/g/rfc/message/93
> * https://edk2.groups.io/g/devel/message/46607
>
> Wiki Page:
> * https://github.com/tianocore/tianocore.github.io/wiki/EDK-II-Continuous-Integration
>
> Branch for review:
> * https://github.com/tianocore/edk2-staging/tree/edk2-ci_V4
>
> Changes in V4
> * Only copy BaseTools build logs if BaseTools are built
> * Add FINISHED and FAILED jobs to prevent multiple email notifications from
> Mergify for PR close and PR comment actions
> * Reformat .pytools/Readme.md to 80 columns
> * Set WINSDK_PATH_FOR_RC_EXE in BaseTools/set_vsprefix_env.bat for VS2017/VS2019
> * Update .gitignore to alphabetic ordering
> * Update Maintainers.txt to alphabetic ordering
> * Add file header comment block to requirements.txt
> * Remove license badge and formatting changes from Readme.md
>
> Previous versions of branches for reference:
> * https://github.com/tianocore/edk2-staging/tree/edk2-ci_V3
> * https://github.com/tianocore/edk2-staging/tree/edk2-ci_V2
> * https://github.com/tianocore/edk2-staging/tree/edk2-ci_V1
>
> Active branch for testing/evaluation:
> * https://github.com/tianocore/edk2-staging/tree/edk2-ci
> * To test, fork edk2-staging repo, create a branch with a change, and submit
> a pull request targeting edk2-staging/edk2-ci. NOTE: the default branch for
> the edk2-staging is 'about'. You must select the 'edk2-ci' branch when
> a pull request is opened. Set the 'push' label to require commit if all
> checks pass.
>
> Pull request history on active branch for testing/evaluation:
> * https://github.com/tianocore/edk2-staging/pulls?q=is%3Apr+is%3Aclosed
>
> This patch series enables pre-commit and post-commit checks on edk2/master The
> goal is to improve quality of code submissions by requiring all pre-commit
> checks to pass before any change is committed to edk2/master. In order to
> minimize the impact to the developer process, when an EDK II Maintainer is ready
> to commit changes to edk2/master, the EDK II Maintainer pushes a branch to their
> personal fork of the edk2 repository and opens a pull request targeting
> edk2/master and sets the 'push' label. If all checks pass, then the pull
> request is automatically committed.
>
> A developer that wants to know if a patch series passes all checks can either
> run the checks locally or open a pull request without setting the 'push' label.
>
> The post-commit checks build all packages and provide status badges and links to
> reports in the Readme.md file at the root of edk2/master.
>
> A combination of GitHub, Azure Pipelines, Mergify, and edk2-pytool features are
> used to implement this feature. GitHub and Azure Pipelines configuration steps
> are required to activate this feature on edk2/master.
>
> Once this feature is active and shown to be stable for 1-2 weeks, write access
> to the edk2 repository will be removed for all EDK II Maintainers. Only admins
> will retain write access, and admins should avoid use of direct push to resolve
> an issue unless there is no method to resolve the issue using the pull request
> method.
>
> The following checks are made available by this patch series. Not all checks
> are enabled for all packages. Enabling all checks on all packages can be done
> incrementally as well as adding more checks over time.
> * PatchCheck
> * CharEncodingCheck
> * CompilerPlugin
> * DependencyCheck
> * DscCompleteCheck
> * GuidCheck
> * LibraryClassCheck
> * SpellCheck
>
> Cc: Sean Brogan <sean.brogan@microsoft.com>
> Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
> Cc: Liming Gao <liming.gao@intel.com>
> Cc: Bob Feng <bob.c.feng@intel.com>
> Cc: Andrew Fish <afish@apple.com>
> Cc: Laszlo Ersek <lersek@redhat.com>
> Cc: Leif Lindholm <leif.lindholm@linaro.org>
> Cc: Jian J Wang <jian.j.wang@intel.com>
> Cc: Xiaoyu Lu <xiaoyux.lu@intel.com>
> Cc: Ray Ni <ray.ni@intel.com>
> Cc: Hao A Wu <hao.a.wu@intel.com>
> Cc: Jiaxin Wu <jiaxin.wu@intel.com>
> Cc: Siyuan Fu <siyuan.fu@intel.com>
> Cc: Jiewen Yao <jiewen.yao@intel.com>
> Cc: Chao Zhang <chao.b.zhang@intel.com>
> Cc: Zhichao Gao <zhichao.gao@intel.com>
> Cc: Eric Dong <eric.dong@intel.com>
> Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
>
> Michael D Kinney (14):
> Maintainers.txt: Add continuous integration(CI) directories
> CryptoPkg: Add YAML file for CI builds
> FatPkg: Add YAML file for CI builds
> FmpDevicePkg: Add YAML file for CI builds
> MdeModulePkg: Add YAML file for CI builds
> MdePkg: Add YAML file for CI builds
> NetworkPkg: Add YAML file for CI builds
> PcAtChipsetPkg: Add YAML files for CI builds
> SecurityPkg: Add YAML files for CI builds
> ShellPkg: Add YAML file for CI builds
> UefiCpuPkg: Add YAML file for CI builds
> SignedCapsulePkg: Use BaseCryptLibNull to reduce package CI time
> .mergify: Add Mergify YML pull request rules configuration file
> Readme.md: Add CI build status badges
>
> Sean Brogan (8):
> .gitignore: Ignore python compiled files, extdeps, and vscode
> requirements.txt: Add python pip requirements file
> BaseTools: Add RC_PATH define for VS2017/2019
> BaseTools: Add YAML files with path env and tool extdeps
> BaseTools: Add BaseTools plugins to support CI
> .pytool/Plugin: Add CI plugins
> .pytool: Add CISettings.py and Readme.md
> .azurepipelines: Add Azure Pipelines YML configuration files
>
> .azurepipelines/Ubuntu-GCC5.yml | 18 ++
> .azurepipelines/Ubuntu-PatchCheck.yml | 32 +++
> .azurepipelines/Windows-VS2019.yml | 18 ++
> .../templates/basetools-build-steps.yml | 36 +++
> .../templates/pr-gate-build-job.yml | 79 ++++++
> .azurepipelines/templates/pr-gate-steps.yml | 129 +++++++++
> .../templates/spell-check-prereq-steps.yml | 21 ++
> .gitignore | 6 +-
> .mergify/config.yml | 97 +++++++
> .pytool/CISettings.py | 173 ++++++++++++
> .../CharEncodingCheck/CharEncodingCheck.py | 118 ++++++++
> .../CharEncodingCheck_plug_in.yaml | 11 +
> .pytool/Plugin/CharEncodingCheck/Readme.md | 13 +
> .../Plugin/CompilerPlugin/CompilerPlugin.py | 102 +++++++
> .../CompilerPlugin/Compiler_plug_in.yaml | 11 +
> .../Plugin/DependencyCheck/DependencyCheck.py | 120 +++++++++
> .../DependencyCheck_plug_in.yaml | 13 +
> .../DscCompleteCheck/DscCompleteCheck.py | 118 ++++++++
> .../DscCompleteCheck_plug_in.yaml | 12 +
> .pytool/Plugin/DscCompleteCheck/readme.md | 22 ++
> .pytool/Plugin/GuidCheck/GuidCheck.py | 251 ++++++++++++++++++
> .../Plugin/GuidCheck/GuidCheck_plug_in.yaml | 11 +
> .pytool/Plugin/GuidCheck/Readme.md | 60 +++++
> .../LibraryClassCheck/LibraryClassCheck.py | 153 +++++++++++
> .../LibraryClassCheck_plug_in.yaml | 11 +
> .pytool/Plugin/LibraryClassCheck/readme.md | 22 ++
> .pytool/Plugin/SpellCheck/Readme.md | 100 +++++++
> .pytool/Plugin/SpellCheck/SpellCheck.py | 216 +++++++++++++++
> .../Plugin/SpellCheck/SpellCheck_plug_in.yaml | 11 +
> .pytool/Plugin/SpellCheck/cspell.base.yaml | 165 ++++++++++++
> .pytool/Readme.md | 191 +++++++++++++
> BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml | 21 ++
> BaseTools/Bin/gcc_arm_linux_ext_dep.yaml | 21 ++
> BaseTools/Bin/iasl_ext_dep.yaml | 21 ++
> BaseTools/Bin/nasm_ext_dep.yaml | 18 ++
> .../BinWrappers/PosixLike/posix_path_env.yaml | 10 +
> .../WindowsLike/win_build_tools_path_env.yaml | 10 +
> BaseTools/Conf/tools_def.template | 18 +-
> BaseTools/Edk2ToolsBuild.py | 163 ++++++++++++
> .../BuildToolsReportGenerator.py | 69 +++++
> .../BuildToolsReportGenerator_plug_in.yaml | 12 +
> .../BuildToolsReport_Template.html | 126 +++++++++
> .../LinuxGcc5ToolChain/LinuxGcc5ToolChain.py | 85 ++++++
> .../LinuxGcc5ToolChain_plug_in.yaml | 12 +
> .../WindowsResourceCompiler/WinRcPath.py | 29 ++
> .../WinRcPath_plug_in.yaml | 13 +
> .../WindowsVsToolChain/WindowsVsToolChain.py | 126 +++++++++
> .../WindowsVsToolChain_plug_in.yaml | 11 +
> .../basetool_tiano_python_path_env.yaml | 11 +
> BaseTools/basetools_calling_path_env.yaml | 11 +
> BaseTools/basetools_path_env.yaml | 11 +
> BaseTools/set_vsprefix_envs.bat | 10 +
> CryptoPkg/CryptoPkg.ci.yaml | 48 ++++
> FatPkg/FatPkg.ci.yaml | 50 ++++
> FmpDevicePkg/FmpDevicePkg.ci.yaml | 43 +++
> FmpDevicePkg/FmpDevicePkg.dsc | 6 +-
> Maintainers.txt | 23 ++
> MdeModulePkg/MdeModulePkg.ci.yaml | 81 ++++++
> MdePkg/MdePkg.ci.yaml | 90 +++++++
> NetworkPkg/NetworkPkg.ci.yaml | 62 +++++
> NetworkPkg/NetworkPkg.dsc | 5 +
> PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml | 46 ++++
> Readme.md | 11 +
> SecurityPkg/SecurityPkg.ci.yaml | 80 ++++++
> SecurityPkg/SecurityPkg.dsc | 32 ++-
> ShellPkg/ShellPkg.ci.yaml | 55 ++++
> SignedCapsulePkg/SignedCapsulePkg.dsc | 38 ++-
> UefiCpuPkg/UefiCpuPkg.ci.yaml | 51 ++++
> requirements.txt | 17 ++
> 69 files changed, 3878 insertions(+), 8 deletions(-)
> create mode 100644 .azurepipelines/Ubuntu-GCC5.yml
> create mode 100644 .azurepipelines/Ubuntu-PatchCheck.yml
> create mode 100644 .azurepipelines/Windows-VS2019.yml
> create mode 100644 .azurepipelines/templates/basetools-build-steps.yml
> create mode 100644 .azurepipelines/templates/pr-gate-build-job.yml
> create mode 100644 .azurepipelines/templates/pr-gate-steps.yml
> create mode 100644 .azurepipelines/templates/spell-check-prereq-steps.yml
> create mode 100644 .mergify/config.yml
> create mode 100644 .pytool/CISettings.py
> create mode 100644 .pytool/Plugin/CharEncodingCheck/CharEncodingCheck.py
> create mode 100644 .pytool/Plugin/CharEncodingCheck/CharEncodingCheck_plug_in.yaml
> create mode 100644 .pytool/Plugin/CharEncodingCheck/Readme.md
> create mode 100644 .pytool/Plugin/CompilerPlugin/CompilerPlugin.py
> create mode 100644 .pytool/Plugin/CompilerPlugin/Compiler_plug_in.yaml
> create mode 100644 .pytool/Plugin/DependencyCheck/DependencyCheck.py
> create mode 100644 .pytool/Plugin/DependencyCheck/DependencyCheck_plug_in.yaml
> create mode 100644 .pytool/Plugin/DscCompleteCheck/DscCompleteCheck.py
> create mode 100644 .pytool/Plugin/DscCompleteCheck/DscCompleteCheck_plug_in.yaml
> create mode 100644 .pytool/Plugin/DscCompleteCheck/readme.md
> create mode 100644 .pytool/Plugin/GuidCheck/GuidCheck.py
> create mode 100644 .pytool/Plugin/GuidCheck/GuidCheck_plug_in.yaml
> create mode 100644 .pytool/Plugin/GuidCheck/Readme.md
> create mode 100644 .pytool/Plugin/LibraryClassCheck/LibraryClassCheck.py
> create mode 100644 .pytool/Plugin/LibraryClassCheck/LibraryClassCheck_plug_in.yaml
> create mode 100644 .pytool/Plugin/LibraryClassCheck/readme.md
> create mode 100644 .pytool/Plugin/SpellCheck/Readme.md
> create mode 100644 .pytool/Plugin/SpellCheck/SpellCheck.py
> create mode 100644 .pytool/Plugin/SpellCheck/SpellCheck_plug_in.yaml
> create mode 100644 .pytool/Plugin/SpellCheck/cspell.base.yaml
> create mode 100644 .pytool/Readme.md
> create mode 100644 BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml
> create mode 100644 BaseTools/Bin/gcc_arm_linux_ext_dep.yaml
> create mode 100644 BaseTools/Bin/iasl_ext_dep.yaml
> create mode 100644 BaseTools/Bin/nasm_ext_dep.yaml
> create mode 100644 BaseTools/BinWrappers/PosixLike/posix_path_env.yaml
> create mode 100644 BaseTools/BinWrappers/WindowsLike/win_build_tools_path_env.yaml
> create mode 100644 BaseTools/Edk2ToolsBuild.py
> create mode 100644 BaseTools/Plugin/BuildToolsReport/BuildToolsReportGenerator.py
> create mode 100644 BaseTools/Plugin/BuildToolsReport/BuildToolsReportGenerator_plug_in.yaml
> create mode 100644 BaseTools/Plugin/BuildToolsReport/BuildToolsReport_Template.html
> create mode 100644 BaseTools/Plugin/LinuxGcc5ToolChain/LinuxGcc5ToolChain.py
> create mode 100644 BaseTools/Plugin/LinuxGcc5ToolChain/LinuxGcc5ToolChain_plug_in.yaml
> create mode 100644 BaseTools/Plugin/WindowsResourceCompiler/WinRcPath.py
> create mode 100644 BaseTools/Plugin/WindowsResourceCompiler/WinRcPath_plug_in.yaml
> create mode 100644 BaseTools/Plugin/WindowsVsToolChain/WindowsVsToolChain.py
> create mode 100644 BaseTools/Plugin/WindowsVsToolChain/WindowsVsToolChain_plug_in.yaml
> create mode 100644 BaseTools/Source/Python/basetool_tiano_python_path_env.yaml
> create mode 100644 BaseTools/basetools_calling_path_env.yaml
> create mode 100644 BaseTools/basetools_path_env.yaml
> create mode 100644 CryptoPkg/CryptoPkg.ci.yaml
> create mode 100644 FatPkg/FatPkg.ci.yaml
> create mode 100644 FmpDevicePkg/FmpDevicePkg.ci.yaml
> create mode 100644 MdeModulePkg/MdeModulePkg.ci.yaml
> create mode 100644 MdePkg/MdePkg.ci.yaml
> create mode 100644 NetworkPkg/NetworkPkg.ci.yaml
> create mode 100644 PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml
> create mode 100644 SecurityPkg/SecurityPkg.ci.yaml
> create mode 100644 ShellPkg/ShellPkg.ci.yaml
> create mode 100644 UefiCpuPkg/UefiCpuPkg.ci.yaml
> create mode 100644 requirements.txt
>
> --
> 2.21.0.windows.1
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [Patch v4 03/22] requirements.txt: Add python pip requirements file
2019-11-07 10:39 ` Laszlo Ersek
@ 2019-11-07 15:43 ` Leif Lindholm
2019-11-07 17:44 ` Michael D Kinney
1 sibling, 0 replies; 51+ messages in thread
From: Leif Lindholm @ 2019-11-07 15:43 UTC (permalink / raw)
To: Laszlo Ersek; +Cc: Michael D Kinney, devel, Sean Brogan, Andrew Fish
On Thu, Nov 07, 2019 at 11:39:37AM +0100, Laszlo Ersek wrote:
> > +## @file
> > +# EDK II Python PIP requirements file
> > +#
> > +# This file provides the list of python components to install using PIP.
> > +#
> > +# Copyright (c) Microsoft Corporation. All rights reserved.<BR>
> > +#
> > +# SPDX-License-Identifier: BSD-2-Clause-Patent
> > +#
> > +# https://pypi.org/project/pip/
> > +# https://pip.pypa.io/en/stable/user_guide/#requirements-files
> > +# https://pip.pypa.io/en/stable/reference/pip_install/#requirements-file-format
> > +#
> > +##
> > +
> > +edk2-pytool-library==0.10.*
> > +edk2-pytool-extensions==0.12.*
> >
>
> This is better, but I still find the plain "requirements.txt" filename
> in the root directory of the project very confusing. What component
> *exactly* insists on this file path and file name?
>
> I've checked
>
> https://pip.pypa.io/en/stable/user_guide/#requirements-files
>
> and it seems like "pip" can take any pathname as an argument to option "-r".
>
> There must be a component in the CI environment that invokes "pip". Can
> we file a feature request for that component, that it try
> "pip-requirements.txt" first?
>
> Hmmm... I just googled "github pip requirements.txt", assuming that
> "github" was the component calling "pip". In the result list, I've found:
>
> https://github.com/ClearingHouse/clearinghoused/blob/master/pip-requirements.txt
>
> Is it possible that github already knows to look for
> "pip-requirements.txt"? (Honestly I only suggested
> "pip-requirements.txt" above because it seemed sensible.) If that's the
> case, we should use it.
I agree it would be sensible to investigate this approach first.
> Furthermore, my understanding is that "the list of python components to
> install using PIP" is only there for CI purposes. Can we please state
> that explicitly in the comment block? (The commit message already does
> that, and that's great.)
Good point.
> Basically when someone clones edk2 and runs "ls -l" for the first time,
> there's a good chance "requirements.txt" will be among the few files
> they open right after.
Yeah. For a specific project to use that as a default config filename
is ... pretty arrogant.
/
Leif
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [Patch v4 22/22] Readme.md: Add CI build status badges
2019-11-07 1:13 ` [Patch v4 22/22] Readme.md: Add CI build status badges Michael D Kinney
2019-11-07 10:44 ` Laszlo Ersek
@ 2019-11-07 16:00 ` Leif Lindholm
2019-11-07 19:42 ` Michael D Kinney
1 sibling, 1 reply; 51+ messages in thread
From: Leif Lindholm @ 2019-11-07 16:00 UTC (permalink / raw)
To: Michael D Kinney; +Cc: devel, Andrew Fish, Laszlo Ersek
On Wed, Nov 06, 2019 at 05:13:49PM -0800, Michael D Kinney wrote:
> Update Readme.md with status badges from the most recent
> EDK II Continuous Integration(CI) build.
>
> Cc: Andrew Fish <afish@apple.com>
> Cc: Laszlo Ersek <lersek@redhat.com>
> Cc: Leif Lindholm <leif.lindholm@linaro.org>
> Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
> ---
> Readme.md | 11 +++++++++++
> 1 file changed, 11 insertions(+)
>
> diff --git a/Readme.md b/Readme.md
> index 27e4ce0771..0207923763 100644
> --- a/Readme.md
> +++ b/Readme.md
> @@ -3,6 +3,17 @@
> A modern, feature-rich, cross-platform firmware development environment
> for the UEFI and PI specifications from www.uefi.org.
>
> +# Build Status
> +
> +| Host Type | Toolchain | Branch | Build Status | Test Status | Code Coverage |
> +| :-------- | :-------- | :------ | :----------- | :---------- | :------------ |
> +| Windows | VS2019 | master | [](https://dev.azure.com/tianocore/edk2-ci-play/_build/latest?definitionId=27&branchName=master) | [](https://dev.azure.com/tianocore/edk2-ci-play/_build/latest?definitionId=27&branchName=master) | [](https://dev.azure.com/tianocore/edk2-ci-play/_build/latest?definitionId=27&branchName=master) |
> +| Ubuntu | GCC | master | [](https://dev.azure.com/tianocore/edk2-ci-play/_build/latest?definitionId=26&branchName=master) | [](https://dev.azure.com/tianocore/edk2-ci-play/_build/latest?definitionId=26&branchName=master) | [](https://dev.azure.com/tianocore/edk2-ci-play/_build/latest?definitionId=26&branchName=master) |
I'm still not super happy about this.
If there is no hacky way of embedding this from another file, could we
consider breaking it out into a separate BuildStatus.md and putting a
link to that at the top of this file?
/
Leif.
> +
> +[More CI Build information](.pytool/Readme.md)
> +
> +# License Details
> +
> The majority of the content in the EDK II open source project uses a
> [BSD-2-Clause Plus Patent License](License.txt). The EDK II open source project
> contains the following components that are covered by additional licenses:
> --
> 2.21.0.windows.1
>
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [edk2-devel] [Patch v4 19/22] .pytool: Add CISettings.py and Readme.md
2019-11-07 1:13 ` [Patch v4 19/22] .pytool: Add CISettings.py and Readme.md Michael D Kinney
@ 2019-11-07 16:16 ` rebecca
0 siblings, 0 replies; 51+ messages in thread
From: rebecca @ 2019-11-07 16:16 UTC (permalink / raw)
To: devel, michael.d.kinney; +Cc: Sean Brogan, Bret Barkelew, Liming Gao
On 11/6/19 6:13 PM, Michael D Kinney wrote:
> +# Copyright (c) 2018, Microsoft Corporation
Should probably be 2019.
> + # ####################################################################################### #
> + # Extra CmdLine configuration #
> + # ####################################################################################### #
> +
> + def AddCommandLineOptions(self, parserObj):
> + pass
The comment blocks don't seem very Pythonic.
> +## Background
> +
> +While a number of CI solutions exist, this proposal will focus on the usage of
> +Azure Dev Ops and Build Pipelines. For demonstration, a sample
> +[TianoCore repo](https://github.com/tianocore/edk2-staging.git) (branch master)
> +and [Dev Ops Pipeline](https://dev.azure.com/tianocore/edk2-ci-play/_build?definitionId=14)
> +have been set up.
Since this is no longer a proposal, should we update this text?
> +Inspection of these files reveals the EDKII Tools commands that make up the
> +primary processes for the CI build: 'stuart_setup', 'stuart_update', and
> +'stuart_ci_build'. These commands come from the EDKII Tools PIP modules and are
> +configured as described below. More documentation on the stuart tools can be
> +found [here](https://github.com/tianocore/edk2-pytool-extensions/blob/master/docs/using.md)
> +and [here](https://github.com/tianocore/edk2-pytool-extensions/blob/master/docs/features/feature_invocables.md).
What's the relevance of "stuart"?
--
Rebecca Cran
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [Patch v4 03/22] requirements.txt: Add python pip requirements file
2019-11-07 10:39 ` Laszlo Ersek
2019-11-07 15:43 ` Leif Lindholm
@ 2019-11-07 17:44 ` Michael D Kinney
2019-11-08 13:12 ` Laszlo Ersek
1 sibling, 1 reply; 51+ messages in thread
From: Michael D Kinney @ 2019-11-07 17:44 UTC (permalink / raw)
To: Laszlo Ersek, devel@edk2.groups.io, Kinney, Michael D
Cc: Sean Brogan, Andrew Fish, Leif Lindholm
Hi Laszlo,
requirements.txt is not only for CI.
If a developer wants to run the same tests that CI runs
locally that also need to install using pip and need
this file.
Sean responded to this feedback earlier and pointed to
some features that may depend on this specific filename
or a specific filename pattern.
https://edk2.groups.io/g/devel/message/49620
I agree that the pip command supports using a different
filename. I considered several options:
1) Keep current requirements.txt in root
2) Change to pip_requirements.txt in root
3) Change to requirements.txt in the .pytool directory
I set (3) aside because the use of the python extensions
installed using pip are not limited to content in the
.pytool directory. There is new content in BaseTools as
well that depends on the pip install components. The
root directory is the only common parent directory.
Given the feedback that there may be some services that
look for requirements.txt, I thought it would be better
to leave the filename alone and add the file header
comment block with a clear description of the file.
With this additional context, if there is still feedback
that the filename must be changed, then I would recommend
a filename change that also follows camel case in the root.
PipRequirements.txt
Best regards,
Mike
> -----Original Message-----
> From: Laszlo Ersek <lersek@redhat.com>
> Sent: Thursday, November 7, 2019 2:40 AM
> To: Kinney, Michael D <michael.d.kinney@intel.com>;
> devel@edk2.groups.io
> Cc: Sean Brogan <sean.brogan@microsoft.com>; Andrew Fish
> <afish@apple.com>; Leif Lindholm
> <leif.lindholm@linaro.org>
> Subject: Re: [Patch v4 03/22] requirements.txt: Add
> python pip requirements file
>
> On 11/07/19 02:13, Michael D Kinney wrote:
> > From: Sean Brogan <sean.brogan@microsoft.com>
> >
> > Add pip requirements file that is used to install the
> python pip
> > modules build from the edk2-pytool-library and edk2-
> pytool-extensions
> > repositories.
> >
> > These python modules provide the extensions required
> to perform EDK II
> > Continuous Integration(CI) builds.
> >
> > Cc: Andrew Fish <afish@apple.com>
> > Cc: Laszlo Ersek <lersek@redhat.com>
> > Cc: Leif Lindholm <leif.lindholm@linaro.org>
> > Signed-off-by: Michael D Kinney
> <michael.d.kinney@intel.com>
> > ---
> > requirements.txt | 17 +++++++++++++++++
> > 1 file changed, 17 insertions(+)
> > create mode 100644 requirements.txt
> >
> > diff --git a/requirements.txt b/requirements.txt new
> file mode 100644
> > index 0000000000..4ad72cfc98
> > --- /dev/null
> > +++ b/requirements.txt
> > @@ -0,0 +1,17 @@
> > +## @file
> > +# EDK II Python PIP requirements file # # This file
> provides the list
> > +of python components to install using PIP.
> > +#
> > +# Copyright (c) Microsoft Corporation. All rights
> reserved.<BR> # #
> > +SPDX-License-Identifier: BSD-2-Clause-Patent # #
> > +https://pypi.org/project/pip/ #
> >
> +https://pip.pypa.io/en/stable/user_guide/#requirements-
> files
> > +#
> >
> +https://pip.pypa.io/en/stable/reference/pip_install/#re
> quirements-fil
> > +e-format
> > +#
> > +##
> > +
> > +edk2-pytool-library==0.10.*
> > +edk2-pytool-extensions==0.12.*
> >
>
> This is better, but I still find the plain
> "requirements.txt" filename in the root directory of the
> project very confusing. What component
> *exactly* insists on this file path and file name?
>
> I've checked
>
>
> https://pip.pypa.io/en/stable/user_guide/#requirements-
> files
>
> and it seems like "pip" can take any pathname as an
> argument to option "-r".
>
> There must be a component in the CI environment that
> invokes "pip". Can we file a feature request for that
> component, that it try "pip-requirements.txt" first?
>
> Hmmm... I just googled "github pip requirements.txt",
> assuming that "github" was the component calling "pip".
> In the result list, I've found:
>
> https://github.com/ClearingHouse/clearinghoused/blob/mas
> ter/pip-requirements.txt
>
> Is it possible that github already knows to look for
> "pip-requirements.txt"? (Honestly I only suggested "pip-
> requirements.txt" above because it seemed sensible.) If
> that's the case, we should use it.
>
>
> Furthermore, my understanding is that "the list of
> python components to install using PIP" is only there
> for CI purposes. Can we please state that explicitly in
> the comment block? (The commit message already does
> that, and that's great.)
>
> Basically when someone clones edk2 and runs "ls -l" for
> the first time, there's a good chance "requirements.txt"
> will be among the few files they open right after.
>
> Thanks!
> Laszlo
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [edk2-devel] [Patch v4 17/22] UefiCpuPkg: Add YAML file for CI builds
2019-11-07 10:48 ` Laszlo Ersek
@ 2019-11-07 19:23 ` Michael D Kinney
2019-11-07 19:33 ` Sean
0 siblings, 1 reply; 51+ messages in thread
From: Michael D Kinney @ 2019-11-07 19:23 UTC (permalink / raw)
To: devel@edk2.groups.io, lersek@redhat.com, Sean Brogan; +Cc: Dong, Eric, Ni, Ray
Hi Laszlo,
I will let Sean comment on the copyright line contents.
Mike
> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On
> Behalf Of Laszlo Ersek
> Sent: Thursday, November 7, 2019 2:48 AM
> To: Kinney, Michael D <michael.d.kinney@intel.com>;
> devel@edk2.groups.io
> Cc: Dong, Eric <eric.dong@intel.com>; Ni, Ray
> <ray.ni@intel.com>
> Subject: Re: [edk2-devel] [Patch v4 17/22] UefiCpuPkg:
> Add YAML file for CI builds
>
> On 11/07/19 11:42, Laszlo Ersek wrote:
> > On 11/07/19 02:13, Michael D Kinney wrote:
> >> Add YAML file to the package directory with the
> configuration of the
> >> checks to perform during a CI build.
> >>
> >> Cc: Eric Dong <eric.dong@intel.com>
> >> Cc: Ray Ni <ray.ni@intel.com>
> >> Cc: Laszlo Ersek <lersek@redhat.com>
> >> Signed-off-by: Michael D Kinney
> <michael.d.kinney@intel.com>
> >> Acked-by: Laszlo Ersek <lersek@redhat.com>
> >> ---
> >> UefiCpuPkg/UefiCpuPkg.ci.yaml | 51
> >> +++++++++++++++++++++++++++++++++++
> >> 1 file changed, 51 insertions(+)
> >> create mode 100644 UefiCpuPkg/UefiCpuPkg.ci.yaml
> >>
> >> diff --git a/UefiCpuPkg/UefiCpuPkg.ci.yaml
> >> b/UefiCpuPkg/UefiCpuPkg.ci.yaml new file mode 100644
> index
> >> 0000000000..81da29b878
> >> --- /dev/null
> >> +++ b/UefiCpuPkg/UefiCpuPkg.ci.yaml
> >> @@ -0,0 +1,51 @@
> >> +##
> >> +# CI configuration for UefiCpuPkg
> >> +#
> >> +# Copyright (c) Microsoft Corporation # SPDX-
> License-Identifier:
> >> +BSD-2-Clause-Patent ## {
> >> + "CompilerPlugin": {
> >> + "DscPath": "UefiCpuPkg.dsc"
> >> + },
> >> + "CharEncodingCheck": {
> >> + "IgnoreFiles": []
> >> + },
> >> + "DependencyCheck": {
> >> + "AcceptableDependencies": [
> >> + "MdePkg/MdePkg.dec",
> >> + "MdeModulePkg/MdeModulePkg.dec",
> >> + "UefiCpuPkg/UefiCpuPkg.dec"
> >> + ],
> >> + # For host based unit tests
> >> + "AcceptableDependencies-
> HOST_APPLICATION":[],
> >> + # For UEFI shell based apps
> >> + "AcceptableDependencies-
> UEFI_APPLICATION":[],
> >> + "IgnoreInf": []
> >> + },
> >> + "DscCompleteCheck": {
> >> + "DscPath": "UefiCpuPkg.dsc",
> >> + "IgnoreInf": [
> >> +
> "UefiCpuPkg/ResetVector/FixupVtf/Vtf.inf",
> >> + "UefiCpuPkg/ResetVector/Vtf0/Vtf0.inf"
> >> + ]
> >> + },
> >> + "GuidCheck": {
> >> + "IgnoreGuidName": ["SecCore",
> "ResetVector"], # Expected duplication for
> gEfiFirmwareVolumeTopFileGuid
> >> + "IgnoreGuidValue": [],
> >> + "IgnoreFoldersAndFiles": [],
> >> + "IgnoreDuplicates": []
> >> + },
> >> + "LibraryClassCheck": {
> >> + "IgnoreHeaderFile": []
> >> + },
> >> +
> >> + ## options defined ci/Plugin/SpellCheck
> >> + "SpellCheck": {
> >> + "AuditOnly": True, # Fails test
> but run in AuditOnly mode to collect log
> >> + "IgnoreFiles": [], # use gitignore
> syntax to ignore errors in matching files
> >> + "ExtendWords": [], # words to
> extend to the dictionary for this package
> >> + "IgnoreStandardPaths": [], # Standard
> Plugin defined paths that should be ignore
> >> + "AdditionalIncludePaths": [] # Additional
> paths to spell check (wildcards supported)
> >> + }
> >> +}
> >>
> >
> > This is a very nice patch: good commit message (subjet
> and body both),
> > and good file comment.
> >
> > One suggestion: I'd recommend adding "@file" to the
> comment block at
> > the top.
> >
> > With that:
> >
> > Acked-by: Laszlo Ersek <lersek@redhat.com>
> >
> > (Of course, should "@file" confuse the YAML parser,
> then I won't
> > insist, and my ACK still applies.)
>
> Ah, another remark: I believe the copyright notice
> should state a year.
> (Most likely "2019".)
>
> Thanks
> Laszlo
>
>
>
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [edk2-devel] [Patch v4 17/22] UefiCpuPkg: Add YAML file for CI builds
2019-11-07 19:23 ` [edk2-devel] " Michael D Kinney
@ 2019-11-07 19:33 ` Sean
2019-11-08 14:43 ` Laszlo Ersek
0 siblings, 1 reply; 51+ messages in thread
From: Sean @ 2019-11-07 19:33 UTC (permalink / raw)
To: Kinney, Michael D, devel@edk2.groups.io, lersek@redhat.com
Cc: Dong, Eric, Ni, Ray
Laszlo,
Our legal team has requested that we don't include the copyright year.
Thanks
Sean
-----Original Message-----
From: Kinney, Michael D <michael.d.kinney@intel.com>
Sent: Thursday, November 7, 2019 11:23 AM
To: devel@edk2.groups.io; lersek@redhat.com; Sean Brogan <sean.brogan@microsoft.com>
Cc: Dong, Eric <eric.dong@intel.com>; Ni, Ray <ray.ni@intel.com>
Subject: RE: [edk2-devel] [Patch v4 17/22] UefiCpuPkg: Add YAML file for CI builds
Hi Laszlo,
I will let Sean comment on the copyright line contents.
Mike
> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Laszlo
> Ersek
> Sent: Thursday, November 7, 2019 2:48 AM
> To: Kinney, Michael D <michael.d.kinney@intel.com>;
> devel@edk2.groups.io
> Cc: Dong, Eric <eric.dong@intel.com>; Ni, Ray <ray.ni@intel.com>
> Subject: Re: [edk2-devel] [Patch v4 17/22] UefiCpuPkg:
> Add YAML file for CI builds
>
> On 11/07/19 11:42, Laszlo Ersek wrote:
> > On 11/07/19 02:13, Michael D Kinney wrote:
> >> Add YAML file to the package directory with the
> configuration of the
> >> checks to perform during a CI build.
> >>
> >> Cc: Eric Dong <eric.dong@intel.com>
> >> Cc: Ray Ni <ray.ni@intel.com>
> >> Cc: Laszlo Ersek <lersek@redhat.com>
> >> Signed-off-by: Michael D Kinney
> <michael.d.kinney@intel.com>
> >> Acked-by: Laszlo Ersek <lersek@redhat.com>
> >> ---
> >> UefiCpuPkg/UefiCpuPkg.ci.yaml | 51
> >> +++++++++++++++++++++++++++++++++++
> >> 1 file changed, 51 insertions(+)
> >> create mode 100644 UefiCpuPkg/UefiCpuPkg.ci.yaml
> >>
> >> diff --git a/UefiCpuPkg/UefiCpuPkg.ci.yaml
> >> b/UefiCpuPkg/UefiCpuPkg.ci.yaml new file mode 100644
> index
> >> 0000000000..81da29b878
> >> --- /dev/null
> >> +++ b/UefiCpuPkg/UefiCpuPkg.ci.yaml
> >> @@ -0,0 +1,51 @@
> >> +##
> >> +# CI configuration for UefiCpuPkg
> >> +#
> >> +# Copyright (c) Microsoft Corporation # SPDX-
> License-Identifier:
> >> +BSD-2-Clause-Patent ## {
> >> + "CompilerPlugin": {
> >> + "DscPath": "UefiCpuPkg.dsc"
> >> + },
> >> + "CharEncodingCheck": {
> >> + "IgnoreFiles": []
> >> + },
> >> + "DependencyCheck": {
> >> + "AcceptableDependencies": [
> >> + "MdePkg/MdePkg.dec",
> >> + "MdeModulePkg/MdeModulePkg.dec",
> >> + "UefiCpuPkg/UefiCpuPkg.dec"
> >> + ],
> >> + # For host based unit tests
> >> + "AcceptableDependencies-
> HOST_APPLICATION":[],
> >> + # For UEFI shell based apps
> >> + "AcceptableDependencies-
> UEFI_APPLICATION":[],
> >> + "IgnoreInf": []
> >> + },
> >> + "DscCompleteCheck": {
> >> + "DscPath": "UefiCpuPkg.dsc",
> >> + "IgnoreInf": [
> >> +
> "UefiCpuPkg/ResetVector/FixupVtf/Vtf.inf",
> >> + "UefiCpuPkg/ResetVector/Vtf0/Vtf0.inf"
> >> + ]
> >> + },
> >> + "GuidCheck": {
> >> + "IgnoreGuidName": ["SecCore",
> "ResetVector"], # Expected duplication for
> gEfiFirmwareVolumeTopFileGuid
> >> + "IgnoreGuidValue": [],
> >> + "IgnoreFoldersAndFiles": [],
> >> + "IgnoreDuplicates": []
> >> + },
> >> + "LibraryClassCheck": {
> >> + "IgnoreHeaderFile": []
> >> + },
> >> +
> >> + ## options defined ci/Plugin/SpellCheck
> >> + "SpellCheck": {
> >> + "AuditOnly": True, # Fails test
> but run in AuditOnly mode to collect log
> >> + "IgnoreFiles": [], # use gitignore
> syntax to ignore errors in matching files
> >> + "ExtendWords": [], # words to
> extend to the dictionary for this package
> >> + "IgnoreStandardPaths": [], # Standard
> Plugin defined paths that should be ignore
> >> + "AdditionalIncludePaths": [] # Additional
> paths to spell check (wildcards supported)
> >> + }
> >> +}
> >>
> >
> > This is a very nice patch: good commit message (subjet
> and body both),
> > and good file comment.
> >
> > One suggestion: I'd recommend adding "@file" to the
> comment block at
> > the top.
> >
> > With that:
> >
> > Acked-by: Laszlo Ersek <lersek@redhat.com>
> >
> > (Of course, should "@file" confuse the YAML parser,
> then I won't
> > insist, and my ACK still applies.)
>
> Ah, another remark: I believe the copyright notice should state a
> year.
> (Most likely "2019".)
>
> Thanks
> Laszlo
>
>
>
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [Patch v4 22/22] Readme.md: Add CI build status badges
2019-11-07 16:00 ` Leif Lindholm
@ 2019-11-07 19:42 ` Michael D Kinney
2019-11-07 23:16 ` Leif Lindholm
2019-11-08 9:24 ` Leif Lindholm
0 siblings, 2 replies; 51+ messages in thread
From: Michael D Kinney @ 2019-11-07 19:42 UTC (permalink / raw)
To: Leif Lindholm, Kinney, Michael D, Sean Brogan
Cc: devel@edk2.groups.io, Andrew Fish, Laszlo Ersek
Hi Leif,
I really like the build status information in Readme.md
so it is visible whenever anyone navigates to the edk2
repo. This matches what many other open source projects
provide.
GitHub markdown does support HTML, so one idea is to
put this content into an HTML table and that would
allow us to format the Readme.md file with each hyperlink
on its own line.
If we remove the content from a markdown table, then there
are some other pure markdown methods to put each hyperlink
on its own line.
If we can get down to one hyperlink per line in Readme.md,
does that address your concerns?
Thanks,
Mike
> -----Original Message-----
> From: Leif Lindholm <leif.lindholm@linaro.org>
> Sent: Thursday, November 7, 2019 8:00 AM
> To: Kinney, Michael D <michael.d.kinney@intel.com>
> Cc: devel@edk2.groups.io; Andrew Fish <afish@apple.com>;
> Laszlo Ersek <lersek@redhat.com>
> Subject: Re: [Patch v4 22/22] Readme.md: Add CI build
> status badges
>
> On Wed, Nov 06, 2019 at 05:13:49PM -0800, Michael D
> Kinney wrote:
> > Update Readme.md with status badges from the most
> recent EDK II
> > Continuous Integration(CI) build.
> >
> > Cc: Andrew Fish <afish@apple.com>
> > Cc: Laszlo Ersek <lersek@redhat.com>
> > Cc: Leif Lindholm <leif.lindholm@linaro.org>
> > Signed-off-by: Michael D Kinney
> <michael.d.kinney@intel.com>
> > ---
> > Readme.md | 11 +++++++++++
> > 1 file changed, 11 insertions(+)
> >
> > diff --git a/Readme.md b/Readme.md
> > index 27e4ce0771..0207923763 100644
> > --- a/Readme.md
> > +++ b/Readme.md
> > @@ -3,6 +3,17 @@
> > A modern, feature-rich, cross-platform firmware
> development
> > environment for the UEFI and PI specifications from
> www.uefi.org.
> >
> > +# Build Status
> > +
> > +| Host Type | Toolchain | Branch | Build Status |
> Test Status | Code
> > +| Coverage |
> > +| :-------- | :-------- | :------ | :----------- | :-
> --------- | :------------ |
> > +| Windows | VS2019 | master | [](https:
> //dev.azure.com/tianocore/edk2-ci-
> play/_build/latest?definitionId=27&branchName=master) |
> [](https://dev.azure.com/tianocore/edk2-ci-
> play/_build/latest?definitionId=27&branchName=master) |
> [](https://dev.azure.com/tianocore/edk2-
> ci-play/_build/latest?definitionId=27&branchName=master)
> |
> > +| Ubuntu | GCC | master | [](https://d
> ev.azure.com/tianocore/edk2-ci-
> play/_build/latest?definitionId=26&branchName=master)
> | [](https://dev.azure.com/tianocore/edk2-ci-
> play/_build/latest?definitionId=26&branchName=master) |
> [](https://dev.azure.com/tianocore/edk2-
> ci-play/_build/latest?definitionId=26&branchName=master)
> |
>
> I'm still not super happy about this.
> If there is no hacky way of embedding this from another
> file, could we consider breaking it out into a separate
> BuildStatus.md and putting a link to that at the top of
> this file?
>
> /
> Leif.
>
> > +
> > +[More CI Build information](.pytool/Readme.md)
> > +
> > +# License Details
> > +
> > The majority of the content in the EDK II open source
> project uses a
> > [BSD-2-Clause Plus Patent License](License.txt). The
> EDK II open
> > source project contains the following components that
> are covered by additional licenses:
> > --
> > 2.21.0.windows.1
> >
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [Patch v4 11/22] MdeModulePkg: Add YAML file for CI builds
2019-11-07 3:03 ` Wu, Hao A
@ 2019-11-07 20:02 ` Michael D Kinney
0 siblings, 0 replies; 51+ messages in thread
From: Michael D Kinney @ 2019-11-07 20:02 UTC (permalink / raw)
To: Wu, Hao A, devel@edk2.groups.io, Kinney, Michael D; +Cc: Wang, Jian J
Hao Wu,
I agree that this duplicate GUID exception can be removed.
I tried it out with a PR against edk2-staging/edk2-ci and
it passed all checks.
https://github.com/tianocore/edk2-staging/pull/180
Mike
> -----Original Message-----
> From: Wu, Hao A <hao.a.wu@intel.com>
> Sent: Wednesday, November 6, 2019 7:03 PM
> To: Kinney, Michael D <michael.d.kinney@intel.com>;
> devel@edk2.groups.io
> Cc: Wang, Jian J <jian.j.wang@intel.com>
> Subject: RE: [Patch v4 11/22] MdeModulePkg: Add YAML
> file for CI builds
>
> > -----Original Message-----
> > From: Kinney, Michael D
> > Sent: Thursday, November 07, 2019 9:14 AM
> > To: devel@edk2.groups.io
> > Cc: Wang, Jian J; Wu, Hao A
> > Subject: [Patch v4 11/22] MdeModulePkg: Add YAML file
> for CI builds
> >
> > Add YAML file to the package directory with the
> configuration of the
> > checks to perform during a CI build.
> >
> > Cc: Jian J Wang <jian.j.wang@intel.com>
> > Cc: Hao A Wu <hao.a.wu@intel.com>
> > Signed-off-by: Michael D Kinney
> <michael.d.kinney@intel.com>
> > ---
> > MdeModulePkg/MdeModulePkg.ci.yaml | 81
> > +++++++++++++++++++++++++++++++
> > 1 file changed, 81 insertions(+)
> > create mode 100644 MdeModulePkg/MdeModulePkg.ci.yaml
> >
> > diff --git a/MdeModulePkg/MdeModulePkg.ci.yaml
> > b/MdeModulePkg/MdeModulePkg.ci.yaml
> > new file mode 100644
> > index 0000000000..6ae58d5379
> > --- /dev/null
> > +++ b/MdeModulePkg/MdeModulePkg.ci.yaml
> > @@ -0,0 +1,81 @@
> > +##
> > +# CI configuration for MdeModulePkg
> > +#
> > +# Copyright (c) Microsoft Corporation # SPDX-License-
> Identifier:
> > +BSD-2-Clause-Patent ## {
> > + ## options defined ci/Plugin/CompilerPlugin
> > + "CompilerPlugin": {
> > + "DscPath": "MdeModulePkg.dsc"
> > + },
> > +
> > + ## options defined ci/Plugin/CharEncodingCheck
> > + "CharEncodingCheck": {
> > + "IgnoreFiles": []
> > + },
> > +
> > + ## options defined ci/Plugin/DependencyCheck
> > + "DependencyCheck": {
> > + "AcceptableDependencies": [
> > + "MdePkg/MdePkg.dec",
> > + "MdeModulePkg/MdeModulePkg.dec",
> > + "StandaloneMmPkg/StandaloneMmPkg.dec",
> > + "ArmPkg/ArmPkg.dec" # this should be
> fixed by promoting
> > + an
> > abstraction
> > + ],
> > + # For host based unit tests
> > + "AcceptableDependencies-HOST_APPLICATION":[],
> > + # For UEFI shell based apps
> > + "AcceptableDependencies-UEFI_APPLICATION":[],
> > + "IgnoreInf": []
> > + },
> > +
> > + ## options defined ci/Plugin/DscCompleteCheck
> > + "DscCompleteCheck": {
> > + "IgnoreInf": [],
> > + "DscPath": "MdeModulePkg.dsc"
> > + },
> > +
> > + ## options defined ci/Plugin/GuidCheck
> > + "GuidCheck": {
> > + "IgnoreGuidName": [],
> > + "IgnoreGuidValue": ["00000000-0000-0000-0000-
> 000000000000"],
> > + "IgnoreFoldersAndFiles": [],
> > + "IgnoreDuplicates": [
> > +
> >
> "gEdkiiFormBrowserExProtocolGuid=gEfiFormBrowserExProtoc
> olGuid",
> > +
> "gEfiPeiMmAccessPpiGuid=gPeiSmmAccessPpiGuid",
> > +
> "gPeiSmmControlPpiGuid=gEfiPeiMmControlPpiGuid",
> > + "gEdkiiSerialPortLibVendorGuid=SerialDxe"
> # Is this a bug????
>
>
> Hello Mike,
>
> I think the above GUID duplication (serial-related) has
> already been addressed via commits:
>
> SHA-1: 0d85e67714e31e0dbe4241ab2ebb7c423aba174d
> * MdeModulePkg/SerialDxe: Update the file Guid in
> SerialDxe.inf
>
> SHA-1: 9790f62be1aa5ee9460d4c4ec8c720919523bb62
> * MdeModulePkg SerialDxe.inf: Fix wrong FILE_GUID format
>
> Could you help to confirm whether the item can be
> removed?
>
> With it handled,
> Reviewed-by: Hao A Wu <hao.a.wu@intel.com>
>
> Best Regards,
> Hao Wu
>
>
> > + ]
> > + },
> > +
> > + ## options defined ci/Plugin/LibraryClassCheck
> > + "LibraryClassCheck": {
> > + "IgnoreHeaderFile": []
> > + },
> > +
> > + ## options defined ci/Plugin/SpellCheck
> > + "SpellCheck": {
> > + "AuditOnly": True, # Fails test but
> run in AuditOnly mode to collect
> > log
> > + "IgnoreStandardPaths": [ # Standard
> Plugin defined paths that should
> > be ignore
> > + "*.c", "*.asm", "*.h", "*.nasm", "*.s",
> "*.asl", "*.inf"
> > + ],
> > + "IgnoreFiles": [ # use gitignore
> syntax to ignore errors in matching
> > files
> > +
> "Library/LzmaCustomDecompressLib/Sdk/DOC/*"
> > + ],
> > + "ExtendWords": [ # words to extend
> to the dictionary for this
> > package
> > + "LIGHTGRAY",
> > + "DARKGRAY",
> > + "LIGHTBLUE",
> > + "LIGHTGREEN",
> > + "LIGHTCYAN",
> > + "LIGHTRED",
> > + "LIGHTMAGENTA",
> > + "FVMAIN",
> > + "VARCHECKPCD",
> > + "Getxx",
> > + "lzturbo"
> > + ],
> > + "AdditionalIncludePaths": [] # Additional
> paths to spell
> > + check relative to
> > package root (wildcards supported)
> > + }
> > +}
> > --
> > 2.21.0.windows.1
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [Patch v4 22/22] Readme.md: Add CI build status badges
2019-11-07 19:42 ` Michael D Kinney
@ 2019-11-07 23:16 ` Leif Lindholm
2019-11-08 9:24 ` Leif Lindholm
1 sibling, 0 replies; 51+ messages in thread
From: Leif Lindholm @ 2019-11-07 23:16 UTC (permalink / raw)
To: Kinney, Michael D
Cc: Sean Brogan, devel@edk2.groups.io, Andrew Fish, Laszlo Ersek
On Thu, Nov 07, 2019 at 07:42:27PM +0000, Kinney, Michael D wrote:
> Hi Leif,
>
> I really like the build status information in Readme.md
> so it is visible whenever anyone navigates to the edk2
> repo.
As do I. Just not at any cost (including nullifying the entire point
of markdown).
> This matches what many other open source projects
> provide.
>
> GitHub markdown does support HTML, so one idea is to
> put this content into an HTML table and that would
> allow us to format the Readme.md file with each hyperlink
> on its own line.
I think that might work.
> If we remove the content from a markdown table, then there
> are some other pure markdown methods to put each hyperlink
> on its own line.
>
> If we can get down to one hyperlink per line in Readme.md,
> does that address your concerns?
My main problem is that the current version basically looks like a
transfer error. If we can make it look like a compartmentalised
section of the file that provides real value when viewed through
another tool, I'm quite OK with that.
Regards,
Leif
> Thanks,
>
> Mike
>
> > -----Original Message-----
> > From: Leif Lindholm <leif.lindholm@linaro.org>
> > Sent: Thursday, November 7, 2019 8:00 AM
> > To: Kinney, Michael D <michael.d.kinney@intel.com>
> > Cc: devel@edk2.groups.io; Andrew Fish <afish@apple.com>;
> > Laszlo Ersek <lersek@redhat.com>
> > Subject: Re: [Patch v4 22/22] Readme.md: Add CI build
> > status badges
> >
> > On Wed, Nov 06, 2019 at 05:13:49PM -0800, Michael D
> > Kinney wrote:
> > > Update Readme.md with status badges from the most
> > recent EDK II
> > > Continuous Integration(CI) build.
> > >
> > > Cc: Andrew Fish <afish@apple.com>
> > > Cc: Laszlo Ersek <lersek@redhat.com>
> > > Cc: Leif Lindholm <leif.lindholm@linaro.org>
> > > Signed-off-by: Michael D Kinney
> > <michael.d.kinney@intel.com>
> > > ---
> > > Readme.md | 11 +++++++++++
> > > 1 file changed, 11 insertions(+)
> > >
> > > diff --git a/Readme.md b/Readme.md
> > > index 27e4ce0771..0207923763 100644
> > > --- a/Readme.md
> > > +++ b/Readme.md
> > > @@ -3,6 +3,17 @@
> > > A modern, feature-rich, cross-platform firmware
> > development
> > > environment for the UEFI and PI specifications from
> > www.uefi.org.
> > >
> > > +# Build Status
> > > +
> > > +| Host Type | Toolchain | Branch | Build Status |
> > Test Status | Code
> > > +| Coverage |
> > > +| :-------- | :-------- | :------ | :----------- | :-
> > --------- | :------------ |
> > > +| Windows | VS2019 | master | [](https:
> > //dev.azure.com/tianocore/edk2-ci-
> > play/_build/latest?definitionId=27&branchName=master) |
> > [](https://dev.azure.com/tianocore/edk2-ci-
> > play/_build/latest?definitionId=27&branchName=master) |
> > [](https://dev.azure.com/tianocore/edk2-
> > ci-play/_build/latest?definitionId=27&branchName=master)
> > |
> > > +| Ubuntu | GCC | master | [](https://d
> > ev.azure.com/tianocore/edk2-ci-
> > play/_build/latest?definitionId=26&branchName=master)
> > | [](https://dev.azure.com/tianocore/edk2-ci-
> > play/_build/latest?definitionId=26&branchName=master) |
> > [](https://dev.azure.com/tianocore/edk2-
> > ci-play/_build/latest?definitionId=26&branchName=master)
> > |
> >
> > I'm still not super happy about this.
> > If there is no hacky way of embedding this from another
> > file, could we consider breaking it out into a separate
> > BuildStatus.md and putting a link to that at the top of
> > this file?
> >
> > /
> > Leif.
> >
> > > +
> > > +[More CI Build information](.pytool/Readme.md)
> > > +
> > > +# License Details
> > > +
> > > The majority of the content in the EDK II open source
> > project uses a
> > > [BSD-2-Clause Plus Patent License](License.txt). The
> > EDK II open
> > > source project contains the following components that
> > are covered by additional licenses:
> > > --
> > > 2.21.0.windows.1
> > >
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [Patch v4 22/22] Readme.md: Add CI build status badges
2019-11-07 19:42 ` Michael D Kinney
2019-11-07 23:16 ` Leif Lindholm
@ 2019-11-08 9:24 ` Leif Lindholm
1 sibling, 0 replies; 51+ messages in thread
From: Leif Lindholm @ 2019-11-08 9:24 UTC (permalink / raw)
To: Kinney, Michael D
Cc: Sean Brogan, devel@edk2.groups.io, Andrew Fish, Laszlo Ersek
Hi Mike,
On Thu, Nov 07, 2019 at 07:42:27PM +0000, Kinney, Michael D wrote:
> Hi Leif,
>
> I really like the build status information in Readme.md
> so it is visible whenever anyone navigates to the edk2
> repo. This matches what many other open source projects
> provide.
>
> GitHub markdown does support HTML, so one idea is to
> put this content into an HTML table and that would
> allow us to format the Readme.md file with each hyperlink
> on its own line.
>
> If we remove the content from a markdown table, then there
> are some other pure markdown methods to put each hyperlink
> on its own line.
>
> If we can get down to one hyperlink per line in Readme.md,
> does that address your concerns?
I think that would be a lot better - the crazy line lengths is the
biggest issue.
Best Regards,
Leif
> Thanks,
>
> Mike
>
> > -----Original Message-----
> > From: Leif Lindholm <leif.lindholm@linaro.org>
> > Sent: Thursday, November 7, 2019 8:00 AM
> > To: Kinney, Michael D <michael.d.kinney@intel.com>
> > Cc: devel@edk2.groups.io; Andrew Fish <afish@apple.com>;
> > Laszlo Ersek <lersek@redhat.com>
> > Subject: Re: [Patch v4 22/22] Readme.md: Add CI build
> > status badges
> >
> > On Wed, Nov 06, 2019 at 05:13:49PM -0800, Michael D
> > Kinney wrote:
> > > Update Readme.md with status badges from the most
> > recent EDK II
> > > Continuous Integration(CI) build.
> > >
> > > Cc: Andrew Fish <afish@apple.com>
> > > Cc: Laszlo Ersek <lersek@redhat.com>
> > > Cc: Leif Lindholm <leif.lindholm@linaro.org>
> > > Signed-off-by: Michael D Kinney
> > <michael.d.kinney@intel.com>
> > > ---
> > > Readme.md | 11 +++++++++++
> > > 1 file changed, 11 insertions(+)
> > >
> > > diff --git a/Readme.md b/Readme.md
> > > index 27e4ce0771..0207923763 100644
> > > --- a/Readme.md
> > > +++ b/Readme.md
> > > @@ -3,6 +3,17 @@
> > > A modern, feature-rich, cross-platform firmware
> > development
> > > environment for the UEFI and PI specifications from
> > www.uefi.org.
> > >
> > > +# Build Status
> > > +
> > > +| Host Type | Toolchain | Branch | Build Status |
> > Test Status | Code
> > > +| Coverage |
> > > +| :-------- | :-------- | :------ | :----------- | :-
> > --------- | :------------ |
> > > +| Windows | VS2019 | master | [](https:
> > //dev.azure.com/tianocore/edk2-ci-
> > play/_build/latest?definitionId=27&branchName=master) |
> > [](https://dev.azure.com/tianocore/edk2-ci-
> > play/_build/latest?definitionId=27&branchName=master) |
> > [](https://dev.azure.com/tianocore/edk2-
> > ci-play/_build/latest?definitionId=27&branchName=master)
> > |
> > > +| Ubuntu | GCC | master | [](https://d
> > ev.azure.com/tianocore/edk2-ci-
> > play/_build/latest?definitionId=26&branchName=master)
> > | [](https://dev.azure.com/tianocore/edk2-ci-
> > play/_build/latest?definitionId=26&branchName=master) |
> > [](https://dev.azure.com/tianocore/edk2-
> > ci-play/_build/latest?definitionId=26&branchName=master)
> > |
> >
> > I'm still not super happy about this.
> > If there is no hacky way of embedding this from another
> > file, could we consider breaking it out into a separate
> > BuildStatus.md and putting a link to that at the top of
> > this file?
> >
> > /
> > Leif.
> >
> > > +
> > > +[More CI Build information](.pytool/Readme.md)
> > > +
> > > +# License Details
> > > +
> > > The majority of the content in the EDK II open source
> > project uses a
> > > [BSD-2-Clause Plus Patent License](License.txt). The
> > EDK II open
> > > source project contains the following components that
> > are covered by additional licenses:
> > > --
> > > 2.21.0.windows.1
> > >
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [Patch v4 03/22] requirements.txt: Add python pip requirements file
2019-11-07 17:44 ` Michael D Kinney
@ 2019-11-08 13:12 ` Laszlo Ersek
2019-11-08 16:58 ` Michael D Kinney
0 siblings, 1 reply; 51+ messages in thread
From: Laszlo Ersek @ 2019-11-08 13:12 UTC (permalink / raw)
To: Kinney, Michael D, devel@edk2.groups.io
Cc: Sean Brogan, Andrew Fish, Leif Lindholm
[-- Attachment #1: Type: text/plain, Size: 6853 bytes --]
Hi Mike,
On 11/07/19 18:44, Kinney, Michael D wrote:
> Hi Laszlo,
>
> requirements.txt is not only for CI.
>
> If a developer wants to run the same tests that CI runs locally that
> also need to install using pip and need this file.
>
> Sean responded to this feedback earlier and pointed to some features
> that may depend on this specific filename or a specific filename
> pattern.
>
> https://edk2.groups.io/g/devel/message/49620
>
> I agree that the pip command supports using a different filename. I
> considered several options:
>
> 1) Keep current requirements.txt in root
> 2) Change to pip_requirements.txt in root
> 3) Change to requirements.txt in the .pytool directory
>
> I set (3) aside because the use of the python extensions installed
> using pip are not limited to content in the .pytool directory. There
> is new content in BaseTools as well that depends on the pip install
> components. The root directory is the only common parent directory.
>
> Given the feedback that there may be some services that look for
> requirements.txt, I thought it would be better to leave the filename
> alone and add the file header comment block with a clear description
> of the file.
>
> With this additional context, if there is still feedback that the
> filename must be changed, then I would recommend a filename change
> that also follows camel case in the root.
>
> PipRequirements.txt
Thanks for all this information.
It looks like "requirements.txt" is a *github.com* feature. It is
something that the (closed source, server-side) <github.com> software
looks at, to establish inter-repo (inter-project) dependencies. With
these dependencies parsed automatically, <github.com> can offer various
features. Such as:
[1] https://github.blog/2018-07-12-security-vulnerability-alerts-for-python/
[2] https://help.github.com/en/github/visualizing-repository-data-with-graphs/listing-the-packages-that-a-repository-depends-on
(This is at least what I gather from the links inside Sean's message
that you reference above.)
Therefore, in my opinion, we should look for a solution (filename) that
satisfies both goals below:
- keep the github.com integration happy and functional (with regard to
the above-linked features),
- use a filename that does not imply "requirements" for the strictly
defined edk2 project itself.
While I agree "PipRequirements.txt" looks native to edk2, I think that
would break the first goal -- github.com would likely not recognize it.
However, there are signs that "pip-requirements.txt" is recognized by
github. We can test this theory as follows:
- with google or another search engine, look for some repositories --
any repositories really -- on <github.com> that use
"pip-requirements.txt" rather than "requirements.txt",
- check whether the <github.com> feature marked with [2] above *works*
for those projects.
Now, here are three -- basically randomly chosen -- repositories on
<github.com> that contain "pip-requirements.txt" and *no*
"requirements.txt":
- https://github.com/datagovuk/ckanext-dgu @ cb17b9e
- https://github.com/ClearingHouse/clearinghoused @ f85881f
- https://github.com/jalajthanaki/NLPython @ 47e6861
In order to subject them to the
does <github.com> recognize "pip-requirements.txt"?
test, we should substitute their organization names and project names in
the following URL pattern, taken from Sean's email:
https://github.com/tianocore/edk2-pytool-extensions/network/dependencies
^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^
ORG_NAME PROJECT_NAME
That makes for:
- https://github.com/datagovuk/ckanext-dgu/network/dependencies
- https://github.com/ClearingHouse/clearinghoused/network/dependencies
- https://github.com/jalajthanaki/NLPython/network/dependencies
All three links work; and in each page, there is a section called
Dependencies defined in pip-requirements.txt
Therefore, I claim that <github.com> recognizes "pip-requirements.txt"
too, not just "requirements.txt".
----*----
So how does that apply to us? It seems like the "dependencies" insight,
using the URL format
https://github.com/ORG_NAME/PROJECT_NAME/network/dependencies
*only* considers the master branch (more precisely, the "default"
branch) of a repository.
Therefore, in order to test the viability of "pip-requirements.txt":
- We need to create two new forks of the edk2-staging project on
<github.com>.
- In one of the forks (let's call it F1), we need to push the current
"edk2-staging/edk2-ci" branch as F1's "master" (or maybe "about")
branch, containing "requirements.txt".
- In the other fork (let's call it F2), we need to push the current
"edk2-staging/edk2-ci" branch as F2's "master" (or perhaps "about")
branch, containing "pip-requirements.txt".
- Finally, we must compare (visually) the following two links:
https://github.com/ORG_NAME/F1/network/dependencies
https://github.com/ORG_NAME/F2/network/dependencies
and ascertain the dependencies that are parsed (from "requirements.txt"
vs. "pip-requirements.txt") are identical.
Now... Obviously, I wanted to perform this test myself. For creating
"F1" and "F2", I wanted to fork edk2-staging twice more, under my
account.
Unfortunately, in its *infinite wisdom*, <github.com> does not allow me
to fork the edk2-staging project at all, at this point (not even under
different names), because I already have a fork of edk2-staging. Sigh.
So here's what I managed to do. In my current edk2-staging fork, I
force-pushed the "edk2-ci" branch *as* both the "master" and "about"
branches, at commit 5f901d3f96e5 ("Readme.md: Fix link to pytool
Readme.md", 2019-11-02). Then I went to
https://github.com/lersek/edk2-staging/network/dependencies
and took a screenshot.
Then, I renamed "requirements.txt" to "pip-requirements.txt":
> commit a55d524ab200593f2a907662dce2260df86810fa (HEAD -> req-txt)
> Author: Laszlo Ersek <lersek@redhat.com>
> Date: Fri Nov 8 13:50:20 2019 +0100
>
> rename "requirements.txt" to "pip-requirements.txt"
>
> Signed-off-by: Laszlo Ersek <lersek@redhat.com>
>
> diff --git a/requirements.txt b/pip-requirements.txt
> similarity index 100%
> rename from requirements.txt
> rename to pip-requirements.txt
pushed the change (again to both the "master" branch and the "about"
branch), and took another screenshot, after reloading the URL
https://github.com/lersek/edk2-staging/network/dependencies
Please find both screenshots attached.
Therefore I (again) claim that <github.com> honors
"pip-requirements.txt" equally, and therefore we should use the
"pip-requirements.txt" filename.
Thanks!
Laszlo
[-- Attachment #2: s1.png --]
[-- Type: image/png, Size: 55806 bytes --]
[-- Attachment #3: s2.png --]
[-- Type: image/png, Size: 56133 bytes --]
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [edk2-devel] [Patch v4 17/22] UefiCpuPkg: Add YAML file for CI builds
2019-11-07 19:33 ` Sean
@ 2019-11-08 14:43 ` Laszlo Ersek
0 siblings, 0 replies; 51+ messages in thread
From: Laszlo Ersek @ 2019-11-08 14:43 UTC (permalink / raw)
To: Sean Brogan, Kinney, Michael D, devel@edk2.groups.io; +Cc: Dong, Eric, Ni, Ray
On 11/07/19 20:33, Sean Brogan wrote:
> Laszlo,
>
> Our legal team has requested that we don't include the copyright year.
This sounds unusual, per
https://en.wikipedia.org/wiki/Copyright_notice#Technical_requirements
and also per
https://edk2-docs.gitbooks.io/edk-ii-c-coding-standards-specification/5_source_files/54_code_file_structure.html
(note "Copyright (c) 20XX, Acme Corporation").
But, obviously, I'm not a lawyer, and your legal team consists of
lawyers :) So if they prefer this format, my only remaining request is
that this preference please be pointed out in the commit message:
"the copyright notice(s) in the new file(s) do not contain a copyright
year on purpose"
The reason I'm asking for this is that "no copyright year" is unusual
enough in edk2 that it deserves a comment ("it's not an oversight").
Thanks!
Laszlo
> -----Original Message-----
> From: Kinney, Michael D <michael.d.kinney@intel.com>
> Sent: Thursday, November 7, 2019 11:23 AM
> To: devel@edk2.groups.io; lersek@redhat.com; Sean Brogan <sean.brogan@microsoft.com>
> Cc: Dong, Eric <eric.dong@intel.com>; Ni, Ray <ray.ni@intel.com>
> Subject: RE: [edk2-devel] [Patch v4 17/22] UefiCpuPkg: Add YAML file for CI builds
>
> Hi Laszlo,
>
> I will let Sean comment on the copyright line contents.
>
> Mike
>
>> -----Original Message-----
>> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Laszlo
>> Ersek
>> Sent: Thursday, November 7, 2019 2:48 AM
>> To: Kinney, Michael D <michael.d.kinney@intel.com>;
>> devel@edk2.groups.io
>> Cc: Dong, Eric <eric.dong@intel.com>; Ni, Ray <ray.ni@intel.com>
>> Subject: Re: [edk2-devel] [Patch v4 17/22] UefiCpuPkg:
>> Add YAML file for CI builds
>>
>> On 11/07/19 11:42, Laszlo Ersek wrote:
>>> On 11/07/19 02:13, Michael D Kinney wrote:
>>>> Add YAML file to the package directory with the
>> configuration of the
>>>> checks to perform during a CI build.
>>>>
>>>> Cc: Eric Dong <eric.dong@intel.com>
>>>> Cc: Ray Ni <ray.ni@intel.com>
>>>> Cc: Laszlo Ersek <lersek@redhat.com>
>>>> Signed-off-by: Michael D Kinney
>> <michael.d.kinney@intel.com>
>>>> Acked-by: Laszlo Ersek <lersek@redhat.com>
>>>> ---
>>>> UefiCpuPkg/UefiCpuPkg.ci.yaml | 51
>>>> +++++++++++++++++++++++++++++++++++
>>>> 1 file changed, 51 insertions(+)
>>>> create mode 100644 UefiCpuPkg/UefiCpuPkg.ci.yaml
>>>>
>>>> diff --git a/UefiCpuPkg/UefiCpuPkg.ci.yaml
>>>> b/UefiCpuPkg/UefiCpuPkg.ci.yaml new file mode 100644
>> index
>>>> 0000000000..81da29b878
>>>> --- /dev/null
>>>> +++ b/UefiCpuPkg/UefiCpuPkg.ci.yaml
>>>> @@ -0,0 +1,51 @@
>>>> +##
>>>> +# CI configuration for UefiCpuPkg
>>>> +#
>>>> +# Copyright (c) Microsoft Corporation # SPDX-
>> License-Identifier:
>>>> +BSD-2-Clause-Patent ## {
>>>> + "CompilerPlugin": {
>>>> + "DscPath": "UefiCpuPkg.dsc"
>>>> + },
>>>> + "CharEncodingCheck": {
>>>> + "IgnoreFiles": []
>>>> + },
>>>> + "DependencyCheck": {
>>>> + "AcceptableDependencies": [
>>>> + "MdePkg/MdePkg.dec",
>>>> + "MdeModulePkg/MdeModulePkg.dec",
>>>> + "UefiCpuPkg/UefiCpuPkg.dec"
>>>> + ],
>>>> + # For host based unit tests
>>>> + "AcceptableDependencies-
>> HOST_APPLICATION":[],
>>>> + # For UEFI shell based apps
>>>> + "AcceptableDependencies-
>> UEFI_APPLICATION":[],
>>>> + "IgnoreInf": []
>>>> + },
>>>> + "DscCompleteCheck": {
>>>> + "DscPath": "UefiCpuPkg.dsc",
>>>> + "IgnoreInf": [
>>>> +
>> "UefiCpuPkg/ResetVector/FixupVtf/Vtf.inf",
>>>> + "UefiCpuPkg/ResetVector/Vtf0/Vtf0.inf"
>>>> + ]
>>>> + },
>>>> + "GuidCheck": {
>>>> + "IgnoreGuidName": ["SecCore",
>> "ResetVector"], # Expected duplication for
>> gEfiFirmwareVolumeTopFileGuid
>>>> + "IgnoreGuidValue": [],
>>>> + "IgnoreFoldersAndFiles": [],
>>>> + "IgnoreDuplicates": []
>>>> + },
>>>> + "LibraryClassCheck": {
>>>> + "IgnoreHeaderFile": []
>>>> + },
>>>> +
>>>> + ## options defined ci/Plugin/SpellCheck
>>>> + "SpellCheck": {
>>>> + "AuditOnly": True, # Fails test
>> but run in AuditOnly mode to collect log
>>>> + "IgnoreFiles": [], # use gitignore
>> syntax to ignore errors in matching files
>>>> + "ExtendWords": [], # words to
>> extend to the dictionary for this package
>>>> + "IgnoreStandardPaths": [], # Standard
>> Plugin defined paths that should be ignore
>>>> + "AdditionalIncludePaths": [] # Additional
>> paths to spell check (wildcards supported)
>>>> + }
>>>> +}
>>>>
>>>
>>> This is a very nice patch: good commit message (subjet
>> and body both),
>>> and good file comment.
>>>
>>> One suggestion: I'd recommend adding "@file" to the
>> comment block at
>>> the top.
>>>
>>> With that:
>>>
>>> Acked-by: Laszlo Ersek <lersek@redhat.com>
>>>
>>> (Of course, should "@file" confuse the YAML parser,
>> then I won't
>>> insist, and my ACK still applies.)
>>
>> Ah, another remark: I believe the copyright notice should state a
>> year.
>> (Most likely "2019".)
>>
>> Thanks
>> Laszlo
>>
>>
>>
>
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [Patch v4 03/22] requirements.txt: Add python pip requirements file
2019-11-08 13:12 ` Laszlo Ersek
@ 2019-11-08 16:58 ` Michael D Kinney
0 siblings, 0 replies; 51+ messages in thread
From: Michael D Kinney @ 2019-11-08 16:58 UTC (permalink / raw)
To: Laszlo Ersek, devel@edk2.groups.io, Kinney, Michael D
Cc: Sean Brogan, Andrew Fish, Leif Lindholm
Laszlo,
Thank you for the extra effort on this evaluation.
I will change to pip-requirements.txt
Mike
> -----Original Message-----
> From: Laszlo Ersek <lersek@redhat.com>
> Sent: Friday, November 8, 2019 5:13 AM
> To: Kinney, Michael D <michael.d.kinney@intel.com>;
> devel@edk2.groups.io
> Cc: Sean Brogan <sean.brogan@microsoft.com>; Andrew Fish
> <afish@apple.com>; Leif Lindholm
> <leif.lindholm@linaro.org>
> Subject: Re: [Patch v4 03/22] requirements.txt: Add
> python pip requirements file
>
> Hi Mike,
>
> On 11/07/19 18:44, Kinney, Michael D wrote:
> > Hi Laszlo,
> >
> > requirements.txt is not only for CI.
> >
> > If a developer wants to run the same tests that CI
> runs locally that
> > also need to install using pip and need this file.
> >
> > Sean responded to this feedback earlier and pointed to
> some features
> > that may depend on this specific filename or a
> specific filename
> > pattern.
> >
> > https://edk2.groups.io/g/devel/message/49620
> >
> > I agree that the pip command supports using a
> different filename. I
> > considered several options:
> >
> > 1) Keep current requirements.txt in root
> > 2) Change to pip_requirements.txt in root
> > 3) Change to requirements.txt in the .pytool directory
> >
> > I set (3) aside because the use of the python
> extensions installed
> > using pip are not limited to content in the .pytool
> directory. There
> > is new content in BaseTools as well that depends on
> the pip install
> > components. The root directory is the only common
> parent directory.
> >
> > Given the feedback that there may be some services
> that look for
> > requirements.txt, I thought it would be better to
> leave the filename
> > alone and add the file header comment block with a
> clear description
> > of the file.
> >
> > With this additional context, if there is still
> feedback that the
> > filename must be changed, then I would recommend a
> filename change
> > that also follows camel case in the root.
> >
> > PipRequirements.txt
>
> Thanks for all this information.
>
> It looks like "requirements.txt" is a *github.com*
> feature. It is something that the (closed source,
> server-side) <github.com> software looks at, to
> establish inter-repo (inter-project) dependencies. With
> these dependencies parsed automatically, <github.com>
> can offer various features. Such as:
>
> [1] https://github.blog/2018-07-12-security-
> vulnerability-alerts-for-python/
> [2] https://help.github.com/en/github/visualizing-
> repository-data-with-graphs/listing-the-packages-that-a-
> repository-depends-on
>
> (This is at least what I gather from the links inside
> Sean's message that you reference above.)
>
> Therefore, in my opinion, we should look for a solution
> (filename) that satisfies both goals below:
>
> - keep the github.com integration happy and functional
> (with regard to
> the above-linked features),
>
> - use a filename that does not imply "requirements" for
> the strictly
> defined edk2 project itself.
>
> While I agree "PipRequirements.txt" looks native to
> edk2, I think that would break the first goal --
> github.com would likely not recognize it.
>
> However, there are signs that "pip-requirements.txt" is
> recognized by github. We can test this theory as
> follows:
>
> - with google or another search engine, look for some
> repositories --
> any repositories really -- on <github.com> that use
> "pip-requirements.txt" rather than "requirements.txt",
>
> - check whether the <github.com> feature marked with [2]
> above *works*
> for those projects.
>
> Now, here are three -- basically randomly chosen --
> repositories on <github.com> that contain "pip-
> requirements.txt" and *no*
> "requirements.txt":
>
> - https://github.com/datagovuk/ckanext-dgu @
> cb17b9e
> - https://github.com/ClearingHouse/clearinghoused @
> f85881f
> - https://github.com/jalajthanaki/NLPython @
> 47e6861
>
> In order to subject them to the
>
> does <github.com> recognize "pip-requirements.txt"?
>
> test, we should substitute their organization names and
> project names in the following URL pattern, taken from
> Sean's email:
>
> https://github.com/tianocore/edk2-pytool-
> extensions/network/dependencies
> ^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^
> ORG_NAME PROJECT_NAME
>
> That makes for:
>
> - https://github.com/datagovuk/ckanext-
> dgu/network/dependencies
> -
> https://github.com/ClearingHouse/clearinghoused/network/
> dependencies
> -
> https://github.com/jalajthanaki/NLPython/network/depende
> ncies
>
> All three links work; and in each page, there is a
> section called
>
> Dependencies defined in pip-requirements.txt
>
> Therefore, I claim that <github.com> recognizes "pip-
> requirements.txt"
> too, not just "requirements.txt".
>
> ----*----
>
> So how does that apply to us? It seems like the
> "dependencies" insight, using the URL format
>
>
> https://github.com/ORG_NAME/PROJECT_NAME/network/depende
> ncies
>
> *only* considers the master branch (more precisely, the
> "default"
> branch) of a repository.
>
> Therefore, in order to test the viability of "pip-
> requirements.txt":
>
> - We need to create two new forks of the edk2-staging
> project on
> <github.com>.
>
> - In one of the forks (let's call it F1), we need to
> push the current
> "edk2-staging/edk2-ci" branch as F1's "master" (or
> maybe "about")
> branch, containing "requirements.txt".
>
> - In the other fork (let's call it F2), we need to push
> the current
> "edk2-staging/edk2-ci" branch as F2's "master" (or
> perhaps "about")
> branch, containing "pip-requirements.txt".
>
> - Finally, we must compare (visually) the following two
> links:
>
> https://github.com/ORG_NAME/F1/network/dependencies
> https://github.com/ORG_NAME/F2/network/dependencies
>
> and ascertain the dependencies that are parsed (from
> "requirements.txt"
> vs. "pip-requirements.txt") are identical.
>
> Now... Obviously, I wanted to perform this test myself.
> For creating "F1" and "F2", I wanted to fork edk2-
> staging twice more, under my account.
>
> Unfortunately, in its *infinite wisdom*, <github.com>
> does not allow me to fork the edk2-staging project at
> all, at this point (not even under different names),
> because I already have a fork of edk2-staging. Sigh.
>
> So here's what I managed to do. In my current edk2-
> staging fork, I force-pushed the "edk2-ci" branch *as*
> both the "master" and "about"
> branches, at commit 5f901d3f96e5 ("Readme.md: Fix link
> to pytool Readme.md", 2019-11-02). Then I went to
>
> https://github.com/lersek/edk2-
> staging/network/dependencies
>
> and took a screenshot.
>
> Then, I renamed "requirements.txt" to "pip-
> requirements.txt":
>
> > commit a55d524ab200593f2a907662dce2260df86810fa (HEAD
> -> req-txt)
> > Author: Laszlo Ersek <lersek@redhat.com>
> > Date: Fri Nov 8 13:50:20 2019 +0100
> >
> > rename "requirements.txt" to "pip-
> requirements.txt"
> >
> > Signed-off-by: Laszlo Ersek <lersek@redhat.com>
> >
> > diff --git a/requirements.txt b/pip-requirements.txt
> similarity index
> > 100% rename from requirements.txt rename to pip-
> requirements.txt
>
> pushed the change (again to both the "master" branch and
> the "about"
> branch), and took another screenshot, after reloading
> the URL
>
> https://github.com/lersek/edk2-
> staging/network/dependencies
>
> Please find both screenshots attached.
>
> Therefore I (again) claim that <github.com> honors "pip-
> requirements.txt" equally, and therefore we should use
> the "pip-requirements.txt" filename.
>
> Thanks!
> Laszlo
^ permalink raw reply [flat|nested] 51+ messages in thread
end of thread, other threads:[~2019-11-08 16:58 UTC | newest]
Thread overview: 51+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2019-11-07 1:13 [Patch v4 00/22] Enable Phase 1 of EDK II CI Michael D Kinney
2019-11-07 1:13 ` [Patch v4 01/22] Maintainers.txt: Add continuous integration(CI) directories Michael D Kinney
2019-11-07 1:13 ` [Patch v4 02/22] .gitignore: Ignore python compiled files, extdeps, and vscode Michael D Kinney
2019-11-07 10:26 ` Laszlo Ersek
2019-11-07 1:13 ` [Patch v4 03/22] requirements.txt: Add python pip requirements file Michael D Kinney
2019-11-07 10:39 ` Laszlo Ersek
2019-11-07 15:43 ` Leif Lindholm
2019-11-07 17:44 ` Michael D Kinney
2019-11-08 13:12 ` Laszlo Ersek
2019-11-08 16:58 ` Michael D Kinney
2019-11-07 10:49 ` Laszlo Ersek
2019-11-07 1:13 ` [Patch v4 04/22] BaseTools: Add RC_PATH define for VS2017/2019 Michael D Kinney
2019-11-07 1:13 ` [Patch v4 05/22] BaseTools: Add YAML files with path env and tool extdeps Michael D Kinney
2019-11-07 1:13 ` [Patch v4 06/22] BaseTools: Add BaseTools plugins to support CI Michael D Kinney
2019-11-07 1:13 ` [Patch v4 07/22] .pytool/Plugin: Add CI plugins Michael D Kinney
2019-11-07 6:58 ` Liming Gao
2019-11-07 1:13 ` [Patch v4 08/22] CryptoPkg: Add YAML file for CI builds Michael D Kinney
2019-11-07 5:06 ` Wang, Jian J
2019-11-07 1:13 ` [Patch v4 09/22] FatPkg: " Michael D Kinney
2019-11-07 2:12 ` Ni, Ray
2019-11-07 1:13 ` [Patch v4 10/22] FmpDevicePkg: " Michael D Kinney
2019-11-07 1:13 ` [Patch v4 11/22] MdeModulePkg: " Michael D Kinney
2019-11-07 3:03 ` Wu, Hao A
2019-11-07 20:02 ` Michael D Kinney
2019-11-07 1:13 ` [Patch v4 12/22] MdePkg: " Michael D Kinney
2019-11-07 1:13 ` [Patch v4 13/22] NetworkPkg: " Michael D Kinney
2019-11-07 1:13 ` [Patch v4 14/22] PcAtChipsetPkg: Add YAML files " Michael D Kinney
2019-11-07 2:12 ` Ni, Ray
2019-11-07 1:13 ` [Patch v4 15/22] SecurityPkg: " Michael D Kinney
2019-11-07 5:08 ` Wang, Jian J
2019-11-07 1:13 ` [Patch v4 16/22] ShellPkg: Add YAML file " Michael D Kinney
2019-11-07 2:12 ` Ni, Ray
2019-11-07 1:13 ` [Patch v4 17/22] UefiCpuPkg: " Michael D Kinney
2019-11-07 2:12 ` Ni, Ray
2019-11-07 10:42 ` Laszlo Ersek
2019-11-07 10:48 ` Laszlo Ersek
2019-11-07 19:23 ` [edk2-devel] " Michael D Kinney
2019-11-07 19:33 ` Sean
2019-11-08 14:43 ` Laszlo Ersek
2019-11-07 1:13 ` [Patch v4 18/22] SignedCapsulePkg: Use BaseCryptLibNull to reduce package CI time Michael D Kinney
2019-11-07 1:13 ` [Patch v4 19/22] .pytool: Add CISettings.py and Readme.md Michael D Kinney
2019-11-07 16:16 ` [edk2-devel] " rebecca
2019-11-07 1:13 ` [Patch v4 20/22] .azurepipelines: Add Azure Pipelines YML configuration files Michael D Kinney
2019-11-07 1:13 ` [Patch v4 21/22] .mergify: Add Mergify YML pull request rules configuration file Michael D Kinney
2019-11-07 1:13 ` [Patch v4 22/22] Readme.md: Add CI build status badges Michael D Kinney
2019-11-07 10:44 ` Laszlo Ersek
2019-11-07 16:00 ` Leif Lindholm
2019-11-07 19:42 ` Michael D Kinney
2019-11-07 23:16 ` Leif Lindholm
2019-11-08 9:24 ` Leif Lindholm
2019-11-07 15:35 ` [Patch v4 00/22] Enable Phase 1 of EDK II CI Liming Gao
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox