public inbox for devel@edk2.groups.io
 help / color / mirror / Atom feed
From: "Michael D Kinney" <michael.d.kinney@intel.com>
To: devel@edk2.groups.io
Cc: Sean Brogan <sean.brogan@microsoft.com>,
	Bret Barkelew <Bret.Barkelew@microsoft.com>,
	Liming Gao <liming.gao@intel.com>,
	Bob Feng <bob.c.feng@intel.com>, Andrew Fish <afish@apple.com>,
	Laszlo Ersek <lersek@redhat.com>,
	Leif Lindholm <leif.lindholm@linaro.org>,
	Jian J Wang <jian.j.wang@intel.com>,
	Xiaoyu Lu <xiaoyux.lu@intel.com>, Ray Ni <ray.ni@intel.com>,
	Hao A Wu <hao.a.wu@intel.com>, Jiaxin Wu <jiaxin.wu@intel.com>,
	Siyuan Fu <siyuan.fu@intel.com>,
	Jiewen Yao <jiewen.yao@intel.com>,
	Chao Zhang <chao.b.zhang@intel.com>,
	Zhichao Gao <zhichao.gao@intel.com>,
	Eric Dong <eric.dong@intel.com>
Subject: [Patch v5 00/22] Enable Phase 1 of EDK II CI
Date: Mon, 11 Nov 2019 00:40:45 -0800	[thread overview]
Message-ID: <20191111084107.5368-1-michael.d.kinney@intel.com> (raw)

https://bugzilla.tianocore.org/show_bug.cgi?id=2315

RFC Proposals:
* https://edk2.groups.io/g/rfc/message/93
* https://edk2.groups.io/g/devel/message/46607

Wiki Page:
* https://github.com/tianocore/tianocore.github.io/wiki/EDK-II-Continuous-Integration

Branch for review:
* https://github.com/tianocore/edk2-staging/tree/edk2-ci_V5

Changes in V5
* Rename requirements.txt -> pip_requirements.txt
* Use HTML syntax for Build Status in Readme.md in root
* Add @file to YML and YAML file headers
* Format Readme.md files to 80 columns
* Add missing Readme.md in .pytool CompilerPlugin
* Add missing Readme.md in .pytool DependencyCheck
* Rename readme.md -> Readme.md in .pytool DscCompleteCheck
* Rename readme.md -> Readme.md in .pytool LibraryClassCheck
* Remove commented out SecurityPkg dependency from CryptoPkg YAML
* Remove SerialPortLib IgnoreDuplicates from MdeModulePkg YAML

Changes in V4
* Only copy BaseTools build logs if BaseTools are built
* Add FINISHED and FAILED jobs to prevent multiple email notifications from
  Mergify for PR close and PR comment actions
* Reformat .pytools/Readme.md to 80 columns
* Set WINSDK_PATH_FOR_RC_EXE in BaseTools/set_vsprefix_env.bat for VS2017/VS2019
* Update .gitignore to alphabetic ordering
* Update Maintainers.txt to alphabetic ordering
* Add file header comment block to requirements.txt
* Remove license badge and formatting changes from Readme.md

Previous versions of branches for reference:
* https://github.com/tianocore/edk2-staging/tree/edk2-ci_V4
* https://github.com/tianocore/edk2-staging/tree/edk2-ci_V3
* https://github.com/tianocore/edk2-staging/tree/edk2-ci_V2
* https://github.com/tianocore/edk2-staging/tree/edk2-ci_V1

Active branch for testing/evaluation:
* https://github.com/tianocore/edk2-staging/tree/edk2-ci
* To test, fork edk2-staging repo, create a branch with a change, and submit
  a pull request targeting edk2-staging/edk2-ci.  NOTE: the default branch for
  the edk2-staging is 'about'.  You must select the 'edk2-ci' branch when
  a pull request is opened.  Set the 'push' label to require commit if all
  checks pass.

Pull request history on active branch for testing/evaluation:
* https://github.com/tianocore/edk2-staging/pulls?q=is%3Apr+is%3Aclosed

This patch series enables pre-commit and post-commit checks on edk2/master The
goal is to improve quality of code submissions by requiring all pre-commit
checks to pass before any change is committed to edk2/master. In order to
minimize the impact to the developer process, when an EDK II Maintainer is ready
to commit changes to edk2/master, the EDK II Maintainer pushes a branch to their
personal fork of the edk2 repository and opens a pull request targeting
edk2/master and sets the 'push' label.  If all checks pass, then the pull
request is automatically committed.

A developer that wants to know if a patch series passes all checks can either
run the checks locally or open a pull request without setting the 'push' label.

The post-commit checks build all packages and provide status badges and links to
reports in the Readme.md file at the root of edk2/master.

A combination of GitHub, Azure Pipelines, Mergify, and edk2-pytool features are
used to implement this feature.  GitHub and Azure Pipelines configuration steps
are required to activate this feature on edk2/master.

Once this feature is active and shown to be stable for 1-2 weeks, write access
to the edk2 repository will be removed for all EDK II Maintainers.  Only admins
will retain write access, and admins should avoid use of direct push to resolve
an issue unless there is no method to resolve the issue using the pull request
method.

The following checks are made available by this patch series.  Not all checks
are enabled for all packages.  Enabling all checks on all packages can be done
incrementally as well as adding more checks over time.
* PatchCheck
* CharEncodingCheck
* CompilerPlugin
* DependencyCheck
* DscCompleteCheck
* GuidCheck
* LibraryClassCheck
* SpellCheck

Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Andrew Fish <afish@apple.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Leif Lindholm <leif.lindholm@linaro.org>
Cc: Jian J Wang <jian.j.wang@intel.com>
Cc: Xiaoyu Lu <xiaoyux.lu@intel.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Hao A Wu <hao.a.wu@intel.com>
Cc: Jiaxin Wu <jiaxin.wu@intel.com>
Cc: Siyuan Fu <siyuan.fu@intel.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Chao Zhang <chao.b.zhang@intel.com>
Cc: Zhichao Gao <zhichao.gao@intel.com>
Cc: Eric Dong <eric.dong@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>

Michael D Kinney (14):
  Maintainers.txt: Add continuous integration(CI) directories
  CryptoPkg: Add YAML file for CI builds
  FatPkg: Add YAML file for CI builds
  FmpDevicePkg: Add YAML file for CI builds
  MdeModulePkg: Add YAML file for CI builds
  MdePkg: Add YAML file for CI builds
  NetworkPkg: Add YAML file for CI builds
  PcAtChipsetPkg: Add YAML files for CI builds
  SecurityPkg: Add YAML files for CI builds
  ShellPkg: Add YAML file for CI builds
  UefiCpuPkg: Add YAML file for CI builds
  SignedCapsulePkg: Use BaseCryptLibNull to reduce package CI time
  .mergify: Add Mergify YML pull request rules configuration file
  Readme.md: Add CI build status badges

Sean Brogan (8):
  .gitignore: Ignore python compiled files, extdeps, and vscode
  pip_requirements.txt: Add python pip requirements file
  BaseTools: Add RC_PATH define for VS2017/2019
  BaseTools: Add YAML files with path env and tool extdeps
  BaseTools: Add BaseTools plugins to support CI
  .pytool/Plugin: Add CI plugins
  .pytool: Add CISettings.py and Readme.md
  .azurepipelines: Add Azure Pipelines YML configuration files

 .azurepipelines/Ubuntu-GCC5.yml               |  18 ++
 .azurepipelines/Ubuntu-PatchCheck.yml         |  35 +++
 .azurepipelines/Windows-VS2019.yml            |  18 ++
 .../templates/basetools-build-steps.yml       |  37 +++
 .../templates/pr-gate-build-job.yml           |  80 ++++++
 .azurepipelines/templates/pr-gate-steps.yml   | 130 +++++++++
 .../templates/spell-check-prereq-steps.yml    |  22 ++
 .gitignore                                    |   6 +-
 .mergify/config.yml                           |  97 +++++++
 .pytool/CISettings.py                         | 173 ++++++++++++
 .../CharEncodingCheck/CharEncodingCheck.py    | 118 ++++++++
 .../CharEncodingCheck_plug_in.yaml            |  11 +
 .pytool/Plugin/CharEncodingCheck/Readme.md    |  18 ++
 .../Plugin/CompilerPlugin/CompilerPlugin.py   | 102 +++++++
 .../CompilerPlugin/Compiler_plug_in.yaml      |  11 +
 .pytool/Plugin/CompilerPlugin/Readme.md       |  17 ++
 .../Plugin/DependencyCheck/DependencyCheck.py | 120 +++++++++
 .../DependencyCheck_plug_in.yaml              |  13 +
 .pytool/Plugin/DependencyCheck/Readme.md      |  31 +++
 .../DscCompleteCheck/DscCompleteCheck.py      | 118 ++++++++
 .../DscCompleteCheck_plug_in.yaml             |  12 +
 .pytool/Plugin/DscCompleteCheck/Readme.md     |  27 ++
 .pytool/Plugin/GuidCheck/GuidCheck.py         | 251 ++++++++++++++++++
 .../Plugin/GuidCheck/GuidCheck_plug_in.yaml   |  11 +
 .pytool/Plugin/GuidCheck/Readme.md            |  80 ++++++
 .../LibraryClassCheck/LibraryClassCheck.py    | 153 +++++++++++
 .../LibraryClassCheck_plug_in.yaml            |  11 +
 .pytool/Plugin/LibraryClassCheck/Readme.md    |  25 ++
 .pytool/Plugin/SpellCheck/Readme.md           | 127 +++++++++
 .pytool/Plugin/SpellCheck/SpellCheck.py       | 216 +++++++++++++++
 .../Plugin/SpellCheck/SpellCheck_plug_in.yaml |  11 +
 .pytool/Plugin/SpellCheck/cspell.base.yaml    | 165 ++++++++++++
 .pytool/Readme.md                             | 223 ++++++++++++++++
 BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml  |  21 ++
 BaseTools/Bin/gcc_arm_linux_ext_dep.yaml      |  21 ++
 BaseTools/Bin/iasl_ext_dep.yaml               |  21 ++
 BaseTools/Bin/nasm_ext_dep.yaml               |  18 ++
 .../BinWrappers/PosixLike/posix_path_env.yaml |  10 +
 .../WindowsLike/win_build_tools_path_env.yaml |  10 +
 BaseTools/Conf/tools_def.template             |  18 +-
 BaseTools/Edk2ToolsBuild.py                   | 163 ++++++++++++
 .../BuildToolsReportGenerator.py              |  69 +++++
 .../BuildToolsReportGenerator_plug_in.yaml    |  12 +
 .../BuildToolsReport_Template.html            | 126 +++++++++
 .../LinuxGcc5ToolChain/LinuxGcc5ToolChain.py  |  85 ++++++
 .../LinuxGcc5ToolChain_plug_in.yaml           |  12 +
 .../WindowsResourceCompiler/WinRcPath.py      |  29 ++
 .../WinRcPath_plug_in.yaml                    |  13 +
 .../WindowsVsToolChain/WindowsVsToolChain.py  | 126 +++++++++
 .../WindowsVsToolChain_plug_in.yaml           |  11 +
 .../basetool_tiano_python_path_env.yaml       |  11 +
 BaseTools/basetools_calling_path_env.yaml     |  11 +
 BaseTools/basetools_path_env.yaml             |  11 +
 BaseTools/set_vsprefix_envs.bat               |  10 +
 CryptoPkg/CryptoPkg.ci.yaml                   |  47 ++++
 FatPkg/FatPkg.ci.yaml                         |  50 ++++
 FmpDevicePkg/FmpDevicePkg.ci.yaml             |  43 +++
 FmpDevicePkg/FmpDevicePkg.dsc                 |   6 +-
 Maintainers.txt                               |  23 ++
 MdeModulePkg/MdeModulePkg.ci.yaml             |  80 ++++++
 MdePkg/MdePkg.ci.yaml                         |  90 +++++++
 NetworkPkg/NetworkPkg.ci.yaml                 |  62 +++++
 NetworkPkg/NetworkPkg.dsc                     |   5 +
 PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml         |  46 ++++
 Readme.md                                     |  51 ++++
 SecurityPkg/SecurityPkg.ci.yaml               |  80 ++++++
 SecurityPkg/SecurityPkg.dsc                   |  32 ++-
 ShellPkg/ShellPkg.ci.yaml                     |  55 ++++
 SignedCapsulePkg/SignedCapsulePkg.dsc         |  38 ++-
 UefiCpuPkg/UefiCpuPkg.ci.yaml                 |  51 ++++
 pip_requirements.txt                          |  17 ++
 71 files changed, 4063 insertions(+), 8 deletions(-)
 create mode 100644 .azurepipelines/Ubuntu-GCC5.yml
 create mode 100644 .azurepipelines/Ubuntu-PatchCheck.yml
 create mode 100644 .azurepipelines/Windows-VS2019.yml
 create mode 100644 .azurepipelines/templates/basetools-build-steps.yml
 create mode 100644 .azurepipelines/templates/pr-gate-build-job.yml
 create mode 100644 .azurepipelines/templates/pr-gate-steps.yml
 create mode 100644 .azurepipelines/templates/spell-check-prereq-steps.yml
 create mode 100644 .mergify/config.yml
 create mode 100644 .pytool/CISettings.py
 create mode 100644 .pytool/Plugin/CharEncodingCheck/CharEncodingCheck.py
 create mode 100644 .pytool/Plugin/CharEncodingCheck/CharEncodingCheck_plug_in.yaml
 create mode 100644 .pytool/Plugin/CharEncodingCheck/Readme.md
 create mode 100644 .pytool/Plugin/CompilerPlugin/CompilerPlugin.py
 create mode 100644 .pytool/Plugin/CompilerPlugin/Compiler_plug_in.yaml
 create mode 100644 .pytool/Plugin/CompilerPlugin/Readme.md
 create mode 100644 .pytool/Plugin/DependencyCheck/DependencyCheck.py
 create mode 100644 .pytool/Plugin/DependencyCheck/DependencyCheck_plug_in.yaml
 create mode 100644 .pytool/Plugin/DependencyCheck/Readme.md
 create mode 100644 .pytool/Plugin/DscCompleteCheck/DscCompleteCheck.py
 create mode 100644 .pytool/Plugin/DscCompleteCheck/DscCompleteCheck_plug_in.yaml
 create mode 100644 .pytool/Plugin/DscCompleteCheck/Readme.md
 create mode 100644 .pytool/Plugin/GuidCheck/GuidCheck.py
 create mode 100644 .pytool/Plugin/GuidCheck/GuidCheck_plug_in.yaml
 create mode 100644 .pytool/Plugin/GuidCheck/Readme.md
 create mode 100644 .pytool/Plugin/LibraryClassCheck/LibraryClassCheck.py
 create mode 100644 .pytool/Plugin/LibraryClassCheck/LibraryClassCheck_plug_in.yaml
 create mode 100644 .pytool/Plugin/LibraryClassCheck/Readme.md
 create mode 100644 .pytool/Plugin/SpellCheck/Readme.md
 create mode 100644 .pytool/Plugin/SpellCheck/SpellCheck.py
 create mode 100644 .pytool/Plugin/SpellCheck/SpellCheck_plug_in.yaml
 create mode 100644 .pytool/Plugin/SpellCheck/cspell.base.yaml
 create mode 100644 .pytool/Readme.md
 create mode 100644 BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml
 create mode 100644 BaseTools/Bin/gcc_arm_linux_ext_dep.yaml
 create mode 100644 BaseTools/Bin/iasl_ext_dep.yaml
 create mode 100644 BaseTools/Bin/nasm_ext_dep.yaml
 create mode 100644 BaseTools/BinWrappers/PosixLike/posix_path_env.yaml
 create mode 100644 BaseTools/BinWrappers/WindowsLike/win_build_tools_path_env.yaml
 create mode 100644 BaseTools/Edk2ToolsBuild.py
 create mode 100644 BaseTools/Plugin/BuildToolsReport/BuildToolsReportGenerator.py
 create mode 100644 BaseTools/Plugin/BuildToolsReport/BuildToolsReportGenerator_plug_in.yaml
 create mode 100644 BaseTools/Plugin/BuildToolsReport/BuildToolsReport_Template.html
 create mode 100644 BaseTools/Plugin/LinuxGcc5ToolChain/LinuxGcc5ToolChain.py
 create mode 100644 BaseTools/Plugin/LinuxGcc5ToolChain/LinuxGcc5ToolChain_plug_in.yaml
 create mode 100644 BaseTools/Plugin/WindowsResourceCompiler/WinRcPath.py
 create mode 100644 BaseTools/Plugin/WindowsResourceCompiler/WinRcPath_plug_in.yaml
 create mode 100644 BaseTools/Plugin/WindowsVsToolChain/WindowsVsToolChain.py
 create mode 100644 BaseTools/Plugin/WindowsVsToolChain/WindowsVsToolChain_plug_in.yaml
 create mode 100644 BaseTools/Source/Python/basetool_tiano_python_path_env.yaml
 create mode 100644 BaseTools/basetools_calling_path_env.yaml
 create mode 100644 BaseTools/basetools_path_env.yaml
 create mode 100644 CryptoPkg/CryptoPkg.ci.yaml
 create mode 100644 FatPkg/FatPkg.ci.yaml
 create mode 100644 FmpDevicePkg/FmpDevicePkg.ci.yaml
 create mode 100644 MdeModulePkg/MdeModulePkg.ci.yaml
 create mode 100644 MdePkg/MdePkg.ci.yaml
 create mode 100644 NetworkPkg/NetworkPkg.ci.yaml
 create mode 100644 PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml
 create mode 100644 SecurityPkg/SecurityPkg.ci.yaml
 create mode 100644 ShellPkg/ShellPkg.ci.yaml
 create mode 100644 UefiCpuPkg/UefiCpuPkg.ci.yaml
 create mode 100644 pip_requirements.txt

-- 
2.21.0.windows.1


             reply	other threads:[~2019-11-11  8:41 UTC|newest]

Thread overview: 30+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-11-11  8:40 Michael D Kinney [this message]
2019-11-11  8:40 ` [Patch v5 01/22] Maintainers.txt: Add continuous integration(CI) directories Michael D Kinney
2019-11-11  8:40 ` [Patch v5 02/22] .gitignore: Ignore python compiled files, extdeps, and vscode Michael D Kinney
2019-11-11  8:40 ` [Patch v5 03/22] pip_requirements.txt: Add python pip requirements file Michael D Kinney
2019-11-11  9:58   ` Laszlo Ersek
2019-11-11 16:35     ` Michael D Kinney
2019-11-11 16:41       ` Laszlo Ersek
2019-11-11  8:40 ` [Patch v5 04/22] BaseTools: Add RC_PATH define for VS2017/2019 Michael D Kinney
2019-11-11  8:40 ` [Patch v5 05/22] BaseTools: Add YAML files with path env and tool extdeps Michael D Kinney
2019-11-11  8:40 ` [Patch v5 06/22] BaseTools: Add BaseTools plugins to support CI Michael D Kinney
2019-11-11  8:40 ` [Patch v5 07/22] .pytool/Plugin: Add CI plugins Michael D Kinney
2019-11-11  8:40 ` [Patch v5 08/22] CryptoPkg: Add YAML file for CI builds Michael D Kinney
2019-11-11  8:40 ` [Patch v5 09/22] FatPkg: " Michael D Kinney
2019-11-11  8:40 ` [Patch v5 10/22] FmpDevicePkg: " Michael D Kinney
2019-11-11  8:40 ` [Patch v5 11/22] MdeModulePkg: " Michael D Kinney
2019-11-11  8:40 ` [Patch v5 12/22] MdePkg: " Michael D Kinney
2019-11-11  8:40 ` [Patch v5 13/22] NetworkPkg: " Michael D Kinney
2019-11-11  8:40 ` [Patch v5 14/22] PcAtChipsetPkg: Add YAML files " Michael D Kinney
2019-11-11  8:41 ` [Patch v5 15/22] SecurityPkg: " Michael D Kinney
2019-11-11  8:41 ` [Patch v5 16/22] ShellPkg: Add YAML file " Michael D Kinney
2019-11-11  8:41 ` [Patch v5 17/22] UefiCpuPkg: " Michael D Kinney
2019-11-11  8:41 ` [Patch v5 18/22] SignedCapsulePkg: Use BaseCryptLibNull to reduce package CI time Michael D Kinney
2019-11-11  8:41 ` [Patch v5 19/22] .pytool: Add CISettings.py and Readme.md Michael D Kinney
2019-11-11  8:41 ` [Patch v5 20/22] .azurepipelines: Add Azure Pipelines YML configuration files Michael D Kinney
2019-11-11  8:41 ` [Patch v5 21/22] .mergify: Add Mergify YML pull request rules configuration file Michael D Kinney
2020-01-17 19:44   ` [edk2-devel] " Leif Lindholm
2020-01-17 20:22     ` Michael D Kinney
2020-01-17 20:29       ` Leif Lindholm
2019-11-11  8:41 ` [Patch v5 22/22] Readme.md: Add CI build status badges Michael D Kinney
2019-11-11 17:14   ` Leif Lindholm

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-list from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20191111084107.5368-1-michael.d.kinney@intel.com \
    --to=devel@edk2.groups.io \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox