From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by mx.groups.io with SMTP id smtpd.web11.1463.1573089233798338813 for ; Wed, 06 Nov 2019 17:13:54 -0800 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: intel.com, ip: 134.134.136.24, mailfrom: michael.d.kinney@intel.com) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 06 Nov 2019 17:13:53 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.68,276,1569308400"; d="scan'208";a="205520568" Received: from unknown (HELO mdkinney-MOBL2.amr.corp.intel.com) ([10.241.98.74]) by orsmga003.jf.intel.com with ESMTP; 06 Nov 2019 17:13:53 -0800 From: "Michael D Kinney" To: devel@edk2.groups.io Cc: Sean Brogan , Bret Barkelew , Liming Gao , Bob Feng , Andrew Fish , Laszlo Ersek , Leif Lindholm , Jian J Wang , Xiaoyu Lu , Ray Ni , Hao A Wu , Jiaxin Wu , Siyuan Fu , Jiewen Yao , Chao Zhang , Zhichao Gao , Eric Dong Subject: [Patch v4 00/22] Enable Phase 1 of EDK II CI Date: Wed, 6 Nov 2019 17:13:27 -0800 Message-Id: <20191107011349.16524-1-michael.d.kinney@intel.com> X-Mailer: git-send-email 2.21.0.windows.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit https://bugzilla.tianocore.org/show_bug.cgi?id=2315 RFC Proposals: * https://edk2.groups.io/g/rfc/message/93 * https://edk2.groups.io/g/devel/message/46607 Wiki Page: * https://github.com/tianocore/tianocore.github.io/wiki/EDK-II-Continuous-Integration Branch for review: * https://github.com/tianocore/edk2-staging/tree/edk2-ci_V4 Changes in V4 * Only copy BaseTools build logs if BaseTools are built * Add FINISHED and FAILED jobs to prevent multiple email notifications from Mergify for PR close and PR comment actions * Reformat .pytools/Readme.md to 80 columns * Set WINSDK_PATH_FOR_RC_EXE in BaseTools/set_vsprefix_env.bat for VS2017/VS2019 * Update .gitignore to alphabetic ordering * Update Maintainers.txt to alphabetic ordering * Add file header comment block to requirements.txt * Remove license badge and formatting changes from Readme.md Previous versions of branches for reference: * https://github.com/tianocore/edk2-staging/tree/edk2-ci_V3 * https://github.com/tianocore/edk2-staging/tree/edk2-ci_V2 * https://github.com/tianocore/edk2-staging/tree/edk2-ci_V1 Active branch for testing/evaluation: * https://github.com/tianocore/edk2-staging/tree/edk2-ci * To test, fork edk2-staging repo, create a branch with a change, and submit a pull request targeting edk2-staging/edk2-ci. NOTE: the default branch for the edk2-staging is 'about'. You must select the 'edk2-ci' branch when a pull request is opened. Set the 'push' label to require commit if all checks pass. Pull request history on active branch for testing/evaluation: * https://github.com/tianocore/edk2-staging/pulls?q=is%3Apr+is%3Aclosed This patch series enables pre-commit and post-commit checks on edk2/master The goal is to improve quality of code submissions by requiring all pre-commit checks to pass before any change is committed to edk2/master. In order to minimize the impact to the developer process, when an EDK II Maintainer is ready to commit changes to edk2/master, the EDK II Maintainer pushes a branch to their personal fork of the edk2 repository and opens a pull request targeting edk2/master and sets the 'push' label. If all checks pass, then the pull request is automatically committed. A developer that wants to know if a patch series passes all checks can either run the checks locally or open a pull request without setting the 'push' label. The post-commit checks build all packages and provide status badges and links to reports in the Readme.md file at the root of edk2/master. A combination of GitHub, Azure Pipelines, Mergify, and edk2-pytool features are used to implement this feature. GitHub and Azure Pipelines configuration steps are required to activate this feature on edk2/master. Once this feature is active and shown to be stable for 1-2 weeks, write access to the edk2 repository will be removed for all EDK II Maintainers. Only admins will retain write access, and admins should avoid use of direct push to resolve an issue unless there is no method to resolve the issue using the pull request method. The following checks are made available by this patch series. Not all checks are enabled for all packages. Enabling all checks on all packages can be done incrementally as well as adding more checks over time. * PatchCheck * CharEncodingCheck * CompilerPlugin * DependencyCheck * DscCompleteCheck * GuidCheck * LibraryClassCheck * SpellCheck Cc: Sean Brogan Cc: Bret Barkelew Cc: Liming Gao Cc: Bob Feng Cc: Andrew Fish Cc: Laszlo Ersek Cc: Leif Lindholm Cc: Jian J Wang Cc: Xiaoyu Lu Cc: Ray Ni Cc: Hao A Wu Cc: Jiaxin Wu Cc: Siyuan Fu Cc: Jiewen Yao Cc: Chao Zhang Cc: Zhichao Gao Cc: Eric Dong Signed-off-by: Michael D Kinney Michael D Kinney (14): Maintainers.txt: Add continuous integration(CI) directories CryptoPkg: Add YAML file for CI builds FatPkg: Add YAML file for CI builds FmpDevicePkg: Add YAML file for CI builds MdeModulePkg: Add YAML file for CI builds MdePkg: Add YAML file for CI builds NetworkPkg: Add YAML file for CI builds PcAtChipsetPkg: Add YAML files for CI builds SecurityPkg: Add YAML files for CI builds ShellPkg: Add YAML file for CI builds UefiCpuPkg: Add YAML file for CI builds SignedCapsulePkg: Use BaseCryptLibNull to reduce package CI time .mergify: Add Mergify YML pull request rules configuration file Readme.md: Add CI build status badges Sean Brogan (8): .gitignore: Ignore python compiled files, extdeps, and vscode requirements.txt: Add python pip requirements file BaseTools: Add RC_PATH define for VS2017/2019 BaseTools: Add YAML files with path env and tool extdeps BaseTools: Add BaseTools plugins to support CI .pytool/Plugin: Add CI plugins .pytool: Add CISettings.py and Readme.md .azurepipelines: Add Azure Pipelines YML configuration files .azurepipelines/Ubuntu-GCC5.yml | 18 ++ .azurepipelines/Ubuntu-PatchCheck.yml | 32 +++ .azurepipelines/Windows-VS2019.yml | 18 ++ .../templates/basetools-build-steps.yml | 36 +++ .../templates/pr-gate-build-job.yml | 79 ++++++ .azurepipelines/templates/pr-gate-steps.yml | 129 +++++++++ .../templates/spell-check-prereq-steps.yml | 21 ++ .gitignore | 6 +- .mergify/config.yml | 97 +++++++ .pytool/CISettings.py | 173 ++++++++++++ .../CharEncodingCheck/CharEncodingCheck.py | 118 ++++++++ .../CharEncodingCheck_plug_in.yaml | 11 + .pytool/Plugin/CharEncodingCheck/Readme.md | 13 + .../Plugin/CompilerPlugin/CompilerPlugin.py | 102 +++++++ .../CompilerPlugin/Compiler_plug_in.yaml | 11 + .../Plugin/DependencyCheck/DependencyCheck.py | 120 +++++++++ .../DependencyCheck_plug_in.yaml | 13 + .../DscCompleteCheck/DscCompleteCheck.py | 118 ++++++++ .../DscCompleteCheck_plug_in.yaml | 12 + .pytool/Plugin/DscCompleteCheck/readme.md | 22 ++ .pytool/Plugin/GuidCheck/GuidCheck.py | 251 ++++++++++++++++++ .../Plugin/GuidCheck/GuidCheck_plug_in.yaml | 11 + .pytool/Plugin/GuidCheck/Readme.md | 60 +++++ .../LibraryClassCheck/LibraryClassCheck.py | 153 +++++++++++ .../LibraryClassCheck_plug_in.yaml | 11 + .pytool/Plugin/LibraryClassCheck/readme.md | 22 ++ .pytool/Plugin/SpellCheck/Readme.md | 100 +++++++ .pytool/Plugin/SpellCheck/SpellCheck.py | 216 +++++++++++++++ .../Plugin/SpellCheck/SpellCheck_plug_in.yaml | 11 + .pytool/Plugin/SpellCheck/cspell.base.yaml | 165 ++++++++++++ .pytool/Readme.md | 191 +++++++++++++ BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml | 21 ++ BaseTools/Bin/gcc_arm_linux_ext_dep.yaml | 21 ++ BaseTools/Bin/iasl_ext_dep.yaml | 21 ++ BaseTools/Bin/nasm_ext_dep.yaml | 18 ++ .../BinWrappers/PosixLike/posix_path_env.yaml | 10 + .../WindowsLike/win_build_tools_path_env.yaml | 10 + BaseTools/Conf/tools_def.template | 18 +- BaseTools/Edk2ToolsBuild.py | 163 ++++++++++++ .../BuildToolsReportGenerator.py | 69 +++++ .../BuildToolsReportGenerator_plug_in.yaml | 12 + .../BuildToolsReport_Template.html | 126 +++++++++ .../LinuxGcc5ToolChain/LinuxGcc5ToolChain.py | 85 ++++++ .../LinuxGcc5ToolChain_plug_in.yaml | 12 + .../WindowsResourceCompiler/WinRcPath.py | 29 ++ .../WinRcPath_plug_in.yaml | 13 + .../WindowsVsToolChain/WindowsVsToolChain.py | 126 +++++++++ .../WindowsVsToolChain_plug_in.yaml | 11 + .../basetool_tiano_python_path_env.yaml | 11 + BaseTools/basetools_calling_path_env.yaml | 11 + BaseTools/basetools_path_env.yaml | 11 + BaseTools/set_vsprefix_envs.bat | 10 + CryptoPkg/CryptoPkg.ci.yaml | 48 ++++ FatPkg/FatPkg.ci.yaml | 50 ++++ FmpDevicePkg/FmpDevicePkg.ci.yaml | 43 +++ FmpDevicePkg/FmpDevicePkg.dsc | 6 +- Maintainers.txt | 23 ++ MdeModulePkg/MdeModulePkg.ci.yaml | 81 ++++++ MdePkg/MdePkg.ci.yaml | 90 +++++++ NetworkPkg/NetworkPkg.ci.yaml | 62 +++++ NetworkPkg/NetworkPkg.dsc | 5 + PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml | 46 ++++ Readme.md | 11 + SecurityPkg/SecurityPkg.ci.yaml | 80 ++++++ SecurityPkg/SecurityPkg.dsc | 32 ++- ShellPkg/ShellPkg.ci.yaml | 55 ++++ SignedCapsulePkg/SignedCapsulePkg.dsc | 38 ++- UefiCpuPkg/UefiCpuPkg.ci.yaml | 51 ++++ requirements.txt | 17 ++ 69 files changed, 3878 insertions(+), 8 deletions(-) create mode 100644 .azurepipelines/Ubuntu-GCC5.yml create mode 100644 .azurepipelines/Ubuntu-PatchCheck.yml create mode 100644 .azurepipelines/Windows-VS2019.yml create mode 100644 .azurepipelines/templates/basetools-build-steps.yml create mode 100644 .azurepipelines/templates/pr-gate-build-job.yml create mode 100644 .azurepipelines/templates/pr-gate-steps.yml create mode 100644 .azurepipelines/templates/spell-check-prereq-steps.yml create mode 100644 .mergify/config.yml create mode 100644 .pytool/CISettings.py create mode 100644 .pytool/Plugin/CharEncodingCheck/CharEncodingCheck.py create mode 100644 .pytool/Plugin/CharEncodingCheck/CharEncodingCheck_plug_in.yaml create mode 100644 .pytool/Plugin/CharEncodingCheck/Readme.md create mode 100644 .pytool/Plugin/CompilerPlugin/CompilerPlugin.py create mode 100644 .pytool/Plugin/CompilerPlugin/Compiler_plug_in.yaml create mode 100644 .pytool/Plugin/DependencyCheck/DependencyCheck.py create mode 100644 .pytool/Plugin/DependencyCheck/DependencyCheck_plug_in.yaml create mode 100644 .pytool/Plugin/DscCompleteCheck/DscCompleteCheck.py create mode 100644 .pytool/Plugin/DscCompleteCheck/DscCompleteCheck_plug_in.yaml create mode 100644 .pytool/Plugin/DscCompleteCheck/readme.md create mode 100644 .pytool/Plugin/GuidCheck/GuidCheck.py create mode 100644 .pytool/Plugin/GuidCheck/GuidCheck_plug_in.yaml create mode 100644 .pytool/Plugin/GuidCheck/Readme.md create mode 100644 .pytool/Plugin/LibraryClassCheck/LibraryClassCheck.py create mode 100644 .pytool/Plugin/LibraryClassCheck/LibraryClassCheck_plug_in.yaml create mode 100644 .pytool/Plugin/LibraryClassCheck/readme.md create mode 100644 .pytool/Plugin/SpellCheck/Readme.md create mode 100644 .pytool/Plugin/SpellCheck/SpellCheck.py create mode 100644 .pytool/Plugin/SpellCheck/SpellCheck_plug_in.yaml create mode 100644 .pytool/Plugin/SpellCheck/cspell.base.yaml create mode 100644 .pytool/Readme.md create mode 100644 BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml create mode 100644 BaseTools/Bin/gcc_arm_linux_ext_dep.yaml create mode 100644 BaseTools/Bin/iasl_ext_dep.yaml create mode 100644 BaseTools/Bin/nasm_ext_dep.yaml create mode 100644 BaseTools/BinWrappers/PosixLike/posix_path_env.yaml create mode 100644 BaseTools/BinWrappers/WindowsLike/win_build_tools_path_env.yaml create mode 100644 BaseTools/Edk2ToolsBuild.py create mode 100644 BaseTools/Plugin/BuildToolsReport/BuildToolsReportGenerator.py create mode 100644 BaseTools/Plugin/BuildToolsReport/BuildToolsReportGenerator_plug_in.yaml create mode 100644 BaseTools/Plugin/BuildToolsReport/BuildToolsReport_Template.html create mode 100644 BaseTools/Plugin/LinuxGcc5ToolChain/LinuxGcc5ToolChain.py create mode 100644 BaseTools/Plugin/LinuxGcc5ToolChain/LinuxGcc5ToolChain_plug_in.yaml create mode 100644 BaseTools/Plugin/WindowsResourceCompiler/WinRcPath.py create mode 100644 BaseTools/Plugin/WindowsResourceCompiler/WinRcPath_plug_in.yaml create mode 100644 BaseTools/Plugin/WindowsVsToolChain/WindowsVsToolChain.py create mode 100644 BaseTools/Plugin/WindowsVsToolChain/WindowsVsToolChain_plug_in.yaml create mode 100644 BaseTools/Source/Python/basetool_tiano_python_path_env.yaml create mode 100644 BaseTools/basetools_calling_path_env.yaml create mode 100644 BaseTools/basetools_path_env.yaml create mode 100644 CryptoPkg/CryptoPkg.ci.yaml create mode 100644 FatPkg/FatPkg.ci.yaml create mode 100644 FmpDevicePkg/FmpDevicePkg.ci.yaml create mode 100644 MdeModulePkg/MdeModulePkg.ci.yaml create mode 100644 MdePkg/MdePkg.ci.yaml create mode 100644 NetworkPkg/NetworkPkg.ci.yaml create mode 100644 PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml create mode 100644 SecurityPkg/SecurityPkg.ci.yaml create mode 100644 ShellPkg/ShellPkg.ci.yaml create mode 100644 UefiCpuPkg/UefiCpuPkg.ci.yaml create mode 100644 requirements.txt -- 2.21.0.windows.1