public inbox for devel@edk2.groups.io
 help / color / mirror / Atom feed
* [PATCH edk2-platforms v2 0/6] Enable edk2-platforms CI for JunoPkg
@ 2021-07-27 16:49 PierreGondois
  2021-07-27 16:49 ` [PATCH edk2-platforms v2 1/6] edk2-platforms: add edk2 repository as a submodule PierreGondois
                   ` (5 more replies)
  0 siblings, 6 replies; 8+ messages in thread
From: PierreGondois @ 2021-07-27 16:49 UTC (permalink / raw)
  To: devel, Sean Brogan, Bret Barkelew, Michael D Kinney, Liming Gao,
	Sami Mujawar

From: Pierre Gondois <Pierre.Gondois@arm.com>

v2:
 - Use tianocore repository instead of personal repository
   for edk2 submodule [Pierre]
 - Bugzilla associated to the topic:
   https://bugzilla.tianocore.org/show_bug.cgi?id=3509

This patch-set is dependent on the following patch-set:
edk2-platforms:
  [PATCH v1 0/2] Fix duplicated GUID
  https://edk2.groups.io/g/devel/message/76910

It provides the configuration files necessary to run an upstream CI
similar to the one currently used for the main edk2 repository. The
configuration is mostly similar aswell.
Enabling the CI requires administrator rights on the edk2-platforms
repository. This configuration was tested on a private repository,
but will require additional configuration from the administrator.

The changes can be seen at: https://github.com/PierreARM/edk2-platforms/tree/1628_Enable_edk2_platforms_ci_for_JunoPkg_v2

Pierre Gondois (6):
  edk2-platforms: add edk2 repository as a submodule
  pip-requirements.txt: Add python pip requirements file
  .pytool/Plugin: Add CI plugins
  .azurepipelines: Add Azure Pipelines YML configuration files
  .mergify: Add Mergify YML pull request rules configuration file
  Platform/ARM: Juno: Add JunoPkg.ci.yaml for CI support

 .azurepipelines/ReadMe.md                     |  50 ++++
 .azurepipelines/Ubuntu-GCC5.yml               |  21 ++
 .azurepipelines/Ubuntu-PatchCheck.yml         |  36 +++
 .azurepipelines/Windows-VS2019.yml            |  20 ++
 .azurepipelines/templates/ReadMe.md           |  59 ++++
 .../templates/basetools-build-steps.yml       |  37 +++
 .../templates/platform-build-run-steps.yml    | 144 ++++++++++
 .../templates/pr-gate-build-job.yml           |  63 ++++
 .azurepipelines/templates/pr-gate-steps.yml   | 142 +++++++++
 .../templates/spell-check-prereq-steps.yml    |  22 ++
 .gitmodules                                   |   3 +
 .mergify/config.yml                           |  98 +++++++
 .pytool/CISettings.py                         | 185 ++++++++++++
 .pytool/Readme.md                             | 271 ++++++++++++++++++
 Platform/ARM/JunoPkg/JunoPkg.ci.yaml          | 104 +++++++
 edk2                                          |   1 +
 pip-requirements.txt                          |  18 ++
 17 files changed, 1274 insertions(+)
 create mode 100644 .azurepipelines/ReadMe.md
 create mode 100644 .azurepipelines/Ubuntu-GCC5.yml
 create mode 100644 .azurepipelines/Ubuntu-PatchCheck.yml
 create mode 100644 .azurepipelines/Windows-VS2019.yml
 create mode 100644 .azurepipelines/templates/ReadMe.md
 create mode 100644 .azurepipelines/templates/basetools-build-steps.yml
 create mode 100644 .azurepipelines/templates/platform-build-run-steps.yml
 create mode 100644 .azurepipelines/templates/pr-gate-build-job.yml
 create mode 100644 .azurepipelines/templates/pr-gate-steps.yml
 create mode 100644 .azurepipelines/templates/spell-check-prereq-steps.yml
 create mode 100644 .mergify/config.yml
 create mode 100644 .pytool/CISettings.py
 create mode 100644 .pytool/Readme.md
 create mode 100644 Platform/ARM/JunoPkg/JunoPkg.ci.yaml
 create mode 160000 edk2
 create mode 100644 pip-requirements.txt

-- 
2.17.1


^ permalink raw reply	[flat|nested] 8+ messages in thread

* [PATCH edk2-platforms v2 1/6] edk2-platforms: add edk2 repository as a submodule
  2021-07-27 16:49 [PATCH edk2-platforms v2 0/6] Enable edk2-platforms CI for JunoPkg PierreGondois
@ 2021-07-27 16:49 ` PierreGondois
  2021-07-27 16:50 ` [PATCH edk2-platforms v2 2/6] pip-requirements.txt: Add python pip requirements file PierreGondois
                   ` (4 subsequent siblings)
  5 siblings, 0 replies; 8+ messages in thread
From: PierreGondois @ 2021-07-27 16:49 UTC (permalink / raw)
  To: devel, Sean Brogan, Bret Barkelew, Michael D Kinney, Liming Gao,
	Sami Mujawar

From: Pierre Gondois <Pierre.Gondois@arm.com>

Add the edk2 repository as a submodule:
https://github.com/tianocore/edk2

Platforms in edk2-platforms often relies on modules available
in the edk2 repository. In order to enable an upstream CI
for edk2-platforms, adding edk2 as a submodule is a convenient
way to advertise this dependency.

Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3509

Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Sami Mujawar <sami.mujawar@arm.com>
Signed-off-by: Sami Mujawar <sami.mujawar@arm.com>
Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
---

Notes:
    v2:
     - Use tianocore repository instead of personal repository [Pierre]

 .gitmodules | 3 +++
 edk2        | 1 +
 2 files changed, 4 insertions(+)
 create mode 160000 edk2

diff --git a/.gitmodules b/.gitmodules
index 88aafaf15820..ed4b2d436cdb 100644
--- a/.gitmodules
+++ b/.gitmodules
@@ -1,3 +1,6 @@
 [submodule "Silicon/RISC-V/ProcessorPkg/Library/RiscVOpensbiLib/opensbi"]
   path = Silicon/RISC-V/ProcessorPkg/Library/RiscVOpensbiLib/opensbi
   url = https://github.com/riscv/opensbi
+[submodule "edk2"]
+	path = edk2
+	url = https://github.com/tianocore/edk2/
diff --git a/edk2 b/edk2
new file mode 160000
index 000000000000..3c81382742fd
--- /dev/null
+++ b/edk2
@@ -0,0 +1 @@
+Subproject commit 3c81382742fdde028b4c23e822f6a6b11f2ab586
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 8+ messages in thread

* [PATCH edk2-platforms v2 2/6] pip-requirements.txt: Add python pip requirements file
  2021-07-27 16:49 [PATCH edk2-platforms v2 0/6] Enable edk2-platforms CI for JunoPkg PierreGondois
  2021-07-27 16:49 ` [PATCH edk2-platforms v2 1/6] edk2-platforms: add edk2 repository as a submodule PierreGondois
@ 2021-07-27 16:50 ` PierreGondois
  2021-07-27 16:50 ` [PATCH edk2-platforms v2 3/6] .pytool/Plugin: Add CI plugins PierreGondois
                   ` (3 subsequent siblings)
  5 siblings, 0 replies; 8+ messages in thread
From: PierreGondois @ 2021-07-27 16:50 UTC (permalink / raw)
  To: devel, Sean Brogan, Bret Barkelew, Michael D Kinney, Liming Gao,
	Sami Mujawar

From: Pierre Gondois <Pierre.Gondois@arm.com>

To enable CI support of the tianocore/edk2-platforms repository,
add pip requirements file to install the python modules
required to perform EDK II Continuous Integration (CI) builds.

This file is a copy of the file from the tianocore/edk2
repository. Any modification to the tianocore/edk2 file must be
reflected on the tianocore/edk2-platforms copy.

Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3509

Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Sami Mujawar <sami.mujawar@arm.com>
Signed-off-by: Sami Mujawar <sami.mujawar@arm.com>
Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
---
 pip-requirements.txt | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)
 create mode 100644 pip-requirements.txt

diff --git a/pip-requirements.txt b/pip-requirements.txt
new file mode 100644
index 000000000000..aea2e6ece431
--- /dev/null
+++ b/pip-requirements.txt
@@ -0,0 +1,18 @@
+## @file
+# EDK II Python PIP requirements file
+#
+# This file provides the list of python components to install using PIP.
+#
+# Copyright (c) Microsoft Corporation.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+#
+# https://pypi.org/project/pip/
+# https://pip.pypa.io/en/stable/user_guide/#requirements-files
+# https://pip.pypa.io/en/stable/reference/pip_install/#requirements-file-format
+# https://www.python.org/dev/peps/pep-0440/#version-specifiers
+##
+
+edk2-pytool-library==0.10.*
+edk2-pytool-extensions~=0.13.3
+edk2-basetools==0.1.2
+antlr4-python3-runtime==4.7.1
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 8+ messages in thread

* [PATCH edk2-platforms v2 3/6] .pytool/Plugin: Add CI plugins
  2021-07-27 16:49 [PATCH edk2-platforms v2 0/6] Enable edk2-platforms CI for JunoPkg PierreGondois
  2021-07-27 16:49 ` [PATCH edk2-platforms v2 1/6] edk2-platforms: add edk2 repository as a submodule PierreGondois
  2021-07-27 16:50 ` [PATCH edk2-platforms v2 2/6] pip-requirements.txt: Add python pip requirements file PierreGondois
@ 2021-07-27 16:50 ` PierreGondois
  2021-07-27 16:50 ` [PATCH edk2-platforms v2 4/6] .azurepipelines: Add Azure Pipelines YML configuration files PierreGondois
                   ` (2 subsequent siblings)
  5 siblings, 0 replies; 8+ messages in thread
From: PierreGondois @ 2021-07-27 16:50 UTC (permalink / raw)
  To: devel, Sean Brogan, Bret Barkelew, Michael D Kinney, Liming Gao,
	Sami Mujawar

From: Pierre Gondois <Pierre.Gondois@arm.com>

To enable CI support of the tianocore/edk2-platforms repository,
add a .pytool directory containing the following files:
 - .pytool/CISettings.py
 - .pytool/Readme.md

These files are largely inspired from the same files available in
the edk2 repository. The .pytool/Plugin/* files containing the
CI tests to run are not copied. edk2-platforms will rely on the
edk2basetools python package and on the edk2 python files, as
edk2 is imported as a submodule of edk2-platforms.

Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3509

Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Sami Mujawar <sami.mujawar@arm.com>
Signed-off-by: Sami Mujawar <sami.mujawar@arm.com>
Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
---
 .pytool/CISettings.py | 185 ++++++++++++++++++++++++++++
 .pytool/Readme.md     | 271 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 456 insertions(+)
 create mode 100644 .pytool/CISettings.py
 create mode 100644 .pytool/Readme.md

diff --git a/.pytool/CISettings.py b/.pytool/CISettings.py
new file mode 100644
index 000000000000..18604646030f
--- /dev/null
+++ b/.pytool/CISettings.py
@@ -0,0 +1,185 @@
+# @file
+#
+# Copyright (c) Microsoft Corporation.
+# Copyright (c) 2020, Hewlett Packard Enterprise Development LP. All rights reserved.<BR>
+# Copyright (c) 2020 - 2021, ARM Limited. All rights reserved.<BR>
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+import os
+import logging
+import edk2basetools
+
+from edk2toolext.environment import shell_environment
+from edk2toolext.invocables.edk2_ci_build import CiBuildSettingsManager
+from edk2toolext.invocables.edk2_setup import SetupSettingsManager, RequiredSubmodule
+from edk2toolext.invocables.edk2_update import UpdateSettingsManager
+from edk2toolext.invocables.edk2_pr_eval import PrEvalSettingsManager
+from edk2toollib.utility_functions import GetHostInfo
+
+
+class Settings(CiBuildSettingsManager, UpdateSettingsManager, SetupSettingsManager, PrEvalSettingsManager):
+
+    def __init__(self):
+        self.ActualPackages = []
+        self.ActualTargets = []
+        self.ActualArchitectures = []
+        self.ActualToolChainTag = ""
+        self.ActualScopes = None
+
+    # ####################################################################################### #
+    #                             Extra CmdLine configuration                                 #
+    # ####################################################################################### #
+
+    def AddCommandLineOptions(self, parserObj):
+        pass
+    def RetrieveCommandLineOptions(self, args):
+        pass
+
+    # ####################################################################################### #
+    #                        Default Support for this Ci Build                                #
+    # ####################################################################################### #
+
+    def GetPackagesSupported(self):
+        ''' return iterable of edk2 packages supported by this build.
+        These should be edk2 workspace relative paths '''
+        return (
+                "JunoPkg",
+                "VExpressPkg"
+                )
+
+    def GetArchitecturesSupported(self):
+        ''' return iterable of edk2 architectures supported by this build '''
+        return (
+                "IA32",
+                "X64",
+                "ARM",
+                "AARCH64",
+                "RISCV64")
+
+    def GetTargetsSupported(self):
+        ''' return iterable of edk2 target tags supported by this build '''
+        return ("DEBUG", "RELEASE", "NO-TARGET", "NOOPT")
+
+    # ####################################################################################### #
+    #                     Verify and Save requested Ci Build Config                           #
+    # ####################################################################################### #
+
+    def SetPackages(self, list_of_requested_packages):
+        ''' Confirm the requested package list is valid and configure SettingsManager
+        to build the requested packages.
+
+        Raise UnsupportedException if a requested_package is not supported
+        '''
+        unsupported = set(list_of_requested_packages) - \
+            set(self.GetPackagesSupported())
+        if(len(unsupported) > 0):
+            logging.critical(
+                "Unsupported Package Requested: " + " ".join(unsupported))
+            raise Exception("Unsupported Package Requested: " +
+                            " ".join(unsupported))
+        self.ActualPackages = list_of_requested_packages
+
+    def SetArchitectures(self, list_of_requested_architectures):
+        ''' Confirm the requests architecture list is valid and configure SettingsManager
+        to run only the requested architectures.
+
+        Raise Exception if a list_of_requested_architectures is not supported
+        '''
+        unsupported = set(list_of_requested_architectures) - \
+            set(self.GetArchitecturesSupported())
+        if(len(unsupported) > 0):
+            logging.critical(
+                "Unsupported Architecture Requested: " + " ".join(unsupported))
+            raise Exception(
+                "Unsupported Architecture Requested: " + " ".join(unsupported))
+        self.ActualArchitectures = list_of_requested_architectures
+
+    def SetTargets(self, list_of_requested_target):
+        ''' Confirm the request target list is valid and configure SettingsManager
+        to run only the requested targets.
+
+        Raise UnsupportedException if a requested_target is not supported
+        '''
+        unsupported = set(list_of_requested_target) - \
+            set(self.GetTargetsSupported())
+        if(len(unsupported) > 0):
+            logging.critical(
+                "Unsupported Targets Requested: " + " ".join(unsupported))
+            raise Exception("Unsupported Targets Requested: " +
+                            " ".join(unsupported))
+        self.ActualTargets = list_of_requested_target
+
+    # ####################################################################################### #
+    #                         Actual Configuration for Ci Build                               #
+    # ####################################################################################### #
+
+    def GetActiveScopes(self):
+        ''' return tuple containing scopes that should be active for this process '''
+        if self.ActualScopes is None:
+            scopes = ("cibuild", "edk2-build", "host-based-test")
+
+            self.ActualToolChainTag = shell_environment.GetBuildVars().GetValue("TOOL_CHAIN_TAG", "")
+
+            is_linux = GetHostInfo().os.upper() == "LINUX"
+            scopes += ('pipbuild-unix',) if is_linux else ('pipbuild-win',)
+
+            if is_linux and self.ActualToolChainTag.upper().startswith("GCC"):
+                if "AARCH64" in self.ActualArchitectures:
+                    scopes += ("gcc_aarch64_linux",)
+                if "ARM" in self.ActualArchitectures:
+                    scopes += ("gcc_arm_linux",)
+                if "RISCV64" in self.ActualArchitectures:
+                    scopes += ("gcc_riscv64_unknown",)
+            self.ActualScopes = scopes
+        return self.ActualScopes
+
+    def GetRequiredSubmodules(self):
+        ''' return iterable containing RequiredSubmodule objects.
+        If no RequiredSubmodules return an empty iterable
+        '''
+        rs = []
+        rs.append(RequiredSubmodule(
+            "edk2", True))
+        rs.append(RequiredSubmodule(
+            "Silicon/RISC-V/ProcessorPkg/Library/RiscVOpensbiLib/opensbi", False))
+        return rs
+
+    def GetName(self):
+        return "Edk2-platforms"
+
+    def GetDependencies(self):
+        return [
+        ]
+
+    def GetPackagesPath(self):
+        ''' Return a list of workspace relative paths that should be mapped as edk2 PackagesPath '''
+        edk2_platforms_path = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+        return [
+            edk2_platforms_path,
+            os.path.join(edk2_platforms_path, "Platform", "ARM"),
+            os.path.join(edk2_platforms_path, "edk2")
+            ]
+
+    def GetWorkspaceRoot(self):
+        ''' get WorkspacePath '''
+        return os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+
+    def FilterPackagesToTest(self, changedFilesList: list, potentialPackagesList: list) -> list:
+        ''' Filter potential packages to test based on changed files. '''
+        build_these_packages = []
+        possible_packages = potentialPackagesList.copy()
+        for f in changedFilesList:
+            # split each part of path for comparison later
+            nodes = f.split("/")
+
+            # python file change in .pytool folder causes building all
+            if f.endswith(".py") and ".pytool" in nodes:
+                build_these_packages = possible_packages
+                break
+
+            # BaseTools files that might change the build
+            if "BaseTools" in nodes:
+                if os.path.splitext(f) not in [".txt", ".md"]:
+                    build_these_packages = possible_packages
+                    break
+        return build_these_packages
diff --git a/.pytool/Readme.md b/.pytool/Readme.md
new file mode 100644
index 000000000000..5b3d9679d7f2
--- /dev/null
+++ b/.pytool/Readme.md
@@ -0,0 +1,271 @@
+# Edk2-platforms Continuous Integration
+
+## Basic Status
+
+| Package                      | Windows VS2019 (IA32/X64)| Ubuntu GCC (IA32/X64/ARM/AARCH64) | Known Issues |
+| :----                        | :-----                   | :----                             | :---         |
+| Platfrom/ARM/JunoPkg         |                          | :heavy_check_mark:                | Spell checking in audit mode. CompilerCheck disabled (need a PlatformCI).
+
+For more detailed status look at the test results of the latest CI run on the
+repo readme.
+
+## Background
+
+This Continuous integration and testing infrastructure leverages the TianoCore EDKII Tools PIP modules:
+[library](https://pypi.org/project/edk2-pytool-library/) and
+[extensions](https://pypi.org/project/edk2-pytool-extensions/) (with repos
+located [here](https://github.com/tianocore/edk2-pytool-library) and
+[here](https://github.com/tianocore/edk2-pytool-extensions)).
+
+The primary execution flows can be found in the
+`.azurepipelines/Windows-VS2019.yml` and `.azurepipelines/Ubuntu-GCC5.yml`
+files. These YAML files are consumed by the Azure Dev Ops Build Pipeline and
+dictate what server resources should be used, how they should be configured, and
+what processes should be run on them. An overview of this schema can be found
+[here](https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=schema).
+
+Inspection of these files reveals the EDKII Tools commands that make up the
+primary processes for the CI build: 'stuart_setup', 'stuart_update', and
+'stuart_ci_build'. These commands come from the EDKII Tools PIP modules and are
+configured as described below. More documentation on the tools can be
+found [here](https://github.com/tianocore/edk2-pytool-extensions/blob/master/docs/using.md)
+and [here](https://github.com/tianocore/edk2-pytool-extensions/blob/master/docs/features/feature_invocables.md).
+
+## Configuration
+
+Configuration of the CI process consists of (in order of precedence):
+
+* command-line arguments passed in via the Pipeline YAML
+* a per-package configuration file (e.g. `<package-name>.ci.yaml`) that is
+  detected by the CI system in EDKII Tools.
+* a global configuration Python module (e.g. `CISetting.py`) passed in via the
+  command-line
+
+The global configuration file is described in
+[this readme](https://github.com/tianocore/edk2-pytool-extensions/blob/master/docs/usability/using_settings_manager.md)
+from the EDKII Tools documentation. This configuration is written as a Python
+module so that decisions can be made dynamically based on command line
+parameters and codebase state.
+
+The per-package configuration file can override most settings in the global
+configuration file, but is not dynamic. This file can be used to skip or
+customize tests that may be incompatible with a specific package.  Each test generally requires
+per package configuration which comes from this file.
+
+## Running CI locally
+
+The EDKII Tools environment (and by extension the ci) is designed to support
+easily and consistently running locally and in a cloud ci environment.  To do
+that a few steps should be followed.  Details of EDKII Tools can be found in the
+[docs folder here](https://github.com/tianocore/edk2-pytool-extensions/tree/master/docs)
+
+### Prerequisets
+
+1. A supported toolchain (others might work but this is what is tested and validated)
+   * Windows 10:
+     * VS 2017 or VS 2019
+     * Windows SDK (for rc)
+     * Windows WDK (for capsules)
+   * Ubuntu 18.04 or Fedora
+     * GCC5
+   * Easy to add more but this is the current state
+2. Python 3.7.x or newer on path
+3. git on path
+4. Recommended to setup and activate a python virtual environment
+5. Install the requirements `pip install --upgrade pip-requirements.txt`
+
+### Running CI
+
+1. clone your edk2-platforms repo
+2. Activate your python virtual environment in cmd window
+3. Get code dependencies (done only when submodules change)
+   * `stuart_setup -c .pytool/CISettings.py TOOL_CHAIN_TAG=<your tag here>`
+4. Update other dependencies (done more often)
+   * `stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=<your tag here>`
+5. Run CI build (--help will give you options)
+   * `stuart_ci_build -c .pytool/CISettings.py TOOL_CHAIN_TAG=<your tag here>`
+   * -p <pkg1,pkg2,pkg3> : To build only certain packages use a CSV list
+   * -a <arch1,arch2,arch3>: To run only certain architectures use a CSV list
+   * -t <target1,target2>: To run only tests related to certain targets use a
+     CSV list
+   * By default all tests are opted in.  Then given a package.ci.yaml file those
+     tests can be configured for a package. Finally setting the check to the
+     value `skip` will skip that plugin.  Examples:
+     * `CompilerPlugin=skip` skip the build test
+     * `GuidCheck=skip` skip the Guid check
+     * `SpellCheck=skip` skip the spell checker
+     * etc
+6. Detailed reports and logs per package are captured in the `Build` directory
+
+## Current PyTool Test Capabilities
+
+All CI tests are instances of EDKII Tools plugins. Documentation on the plugin
+system can be found [here](https://github.com/tianocore/edk2-pytool-extensions/blob/master/docs/usability/using_plugin_manager.md)
+and [here](https://github.com/tianocore/edk2-pytool-extensions/blob/master/docs/features/feature_plugin_manager.md).
+Upon invocation, each plugin will be passed the path to the current package
+under test and a dictionary containing its targeted configuration, as assembled
+from the command line, per-package configuration, and global configuration.
+
+Note: CI plugins are considered unique from build plugins and helper plugins,
+even though some CI plugins may execute steps of a build.
+
+In the example, these plugins live alongside the code under test (in the
+`.pytool/Plugin` directory), but may be moved to the 'edk2-test' repo if that
+location makes more sense for the community.
+
+### Module Inclusion Test - DscCompleteCheck
+
+This scans all INF files from a package and confirms they are
+listed in the package level DSC file. The test considers it an error if any INF
+does not appear in the `Components` section of the package-level DSC (indicating
+that it would not be built if the package were built). This is critical because
+much of the CI infrastructure assumes that all modules will be listed in the DSC
+and compiled.
+
+This test will ignore INFs in the following cases:
+
+1. When `MODULE_TYPE` = `HOST_APPLICATION`
+2. When a Library instance **only** supports the `HOST_APPLICATION` environment
+
+### Host Module Inclusion Test - HostUnitTestDscCompleteCheck
+
+This test scans all INF files from a package for those related to host
+based unit tests and confirms they are listed in the unit test DSC file for the package.
+The test considers it an error if any INF meeting the requirements does not appear
+in the `Components` section of the unit test DSC. This is critical because
+much of the CI infrastructure assumes that  modules will be listed in the DSC
+and compiled.
+
+This test will only require INFs in the following cases:
+
+1. When `MODULE_TYPE` = `HOST_APPLICATION`
+2. When a Library instance explicitly supports the `HOST_APPLICATION` environment
+
+### Code Compilation Test - CompilerPlugin
+
+Once the Module Inclusion Test has verified that all modules would be built if
+all package-level DSCs were built, the Code Compilation Test simply runs through
+and builds every package-level DSC on every toolchain and for every architecture
+that is supported. Any module that fails to build is considered an error.
+
+### Host Unit Test Compilation and Run Test - HostUnitTestCompilerPlugin
+
+A test that compiles the dsc for host based unit test apps.
+On Windows this will also enable a build plugin to execute that will run the unit tests and verify the results.
+
+These tools will be invoked on any CI
+pass that includes the NOOPT target. In order for these tools to do their job,
+the package and tests must be configured in a particular way...
+
+#### Including Host-Based Tests in the Package YAML
+
+For example, looking at the `MdeModulePkg.ci.yaml` config file, there are two
+config options that control HostBased test behavior:
+
+```json
+    ## options defined .pytool/Plugin/HostUnitTestCompilerPlugin
+    "HostUnitTestCompilerPlugin": {
+        "DscPath": "Test/MdeModulePkgHostTest.dsc"
+    },
+```
+
+This option tell the test builder to run. The test builder needs to know which
+modules in this package are host-based tests, so that DSC path is provided.
+
+#### Configuring the HostBased DSC
+
+The HostBased DSC for `MdeModulePkg` is located at
+`MdeModulePkg/Test/MdeModulePkgHostTest.dsc`.
+
+To add automated host-based unit test building to a new package, create a
+similar DSC. The new DSC should make sure to have the `NOOPT` BUILD_TARGET
+and should include the line:
+
+```
+!include UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc
+```
+
+All of the modules that are included in the `Components` section of this
+DSC should be of type HOST_APPLICATION.
+
+### GUID Uniqueness Test - GuidCheck
+
+This test works on the collection of all packages rather than an individual
+package. It looks at all FILE_GUIDs and GUIDs declared in DEC files and ensures
+that they are unique for the codebase. This prevents, for example, accidental
+duplication of GUIDs when using an existing INF as a template for a new module.
+
+### Cross-Package Dependency Test - DependencyCheck
+
+This test compares the list of all packages used in INFs files for a given
+package against a list of "allowed dependencies" in plugin configuration for
+that package. Any module that depends on a disallowed package will cause a test
+failure.
+
+### Library Declaration Test - LibraryClassCheck
+
+This test scans at all library header files found in the `Library` folders in
+all of the package's declared include directories and ensures that all files
+have a matching LibraryClass declaration in the DEC file for the package. Any
+missing declarations will cause a failure.
+
+### Invalid Character Test - CharEncodingCheck
+
+This test scans all files in a package to make sure that there are no invalid
+Unicode characters that may cause build errors in some character
+sets/localizations.
+
+### Spell Checking - cspell
+
+This test runs a spell checker on all files within the package.  This is done
+using the NodeJs cspell tool.  For details check `.pytool/Plugin/SpellCheck`.
+For this plugin to run during ci you must install nodejs and cspell and have
+both available to the command line when running your CI.
+
+Install
+
+* Install nodejs from https://nodejs.org/en/
+* Install cspell
+  1. Open cmd prompt with access to node and npm
+  2. Run `npm install -g cspell`
+
+  More cspell info: https://github.com/streetsidesoftware/cspell
+
+### License Checking - LicenseCheck
+
+Scans all new added files in a package to make sure code is contributed under
+BSD-2-Clause-Patent.
+
+### Ecc tool - EccCheck
+
+Run the Ecc tool on the package. The Ecc tool is available in the BaseTools
+package. It checks that the code complies to the EDKII coding standard.
+
+## PyTool Scopes
+
+Scopes are how the PyTool ext_dep, path_env, and plugins are activated.  Meaning
+that if an invocable process has a scope active then those ext_dep and path_env
+will be active. To allow easy integration of PyTools capabilities there are a
+few standard scopes.
+
+| Scope      | Invocable                                            | Description    |
+| :----      | :-----                                               | :----          |
+| global     | edk2_invocable++ - should be base_abstract_invocable | Running an invocables |
+| global-win | edk2_invocable++                                     | Running on Microsoft Windows |
+| global-nix | edk2_invocable++                                     | Running on Linux based OS    |
+| edk2-build |                                                      | This indicates that an invocable is building EDK2 based UEFI code |
+| cibuild    | set in .pytool/CISettings.py                         | Suggested target for edk2 continuous integration builds.  Tools used for CiBuilds can use this scope.  Example: asl compiler |
+| host-based-test | set in .pytool/CISettings.py                    | Turns on the host based tests and plugin |
+| host-test-win | set in .pytool/CISettings.py                      | Enables the host based test runner for Windows |
+
+## Future investments
+
+* PatchCheck tests as plugins
+* MacOS/xcode support
+* Clang/LLVM support
+* Visual Studio AARCH64 and ARM support
+* BaseTools C tools CI/PR and binary release process
+* BaseTools Python tools CI/PR process
+* Extensible private/closed source platform reporting
+* UEFI SCTs
+* Other automation
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 8+ messages in thread

* [PATCH edk2-platforms v2 4/6] .azurepipelines: Add Azure Pipelines YML configuration files
  2021-07-27 16:49 [PATCH edk2-platforms v2 0/6] Enable edk2-platforms CI for JunoPkg PierreGondois
                   ` (2 preceding siblings ...)
  2021-07-27 16:50 ` [PATCH edk2-platforms v2 3/6] .pytool/Plugin: Add CI plugins PierreGondois
@ 2021-07-27 16:50 ` PierreGondois
  2021-07-27 16:50 ` [PATCH edk2-platforms v2 5/6] .mergify: Add Mergify YML pull request rules configuration file PierreGondois
  2021-07-27 16:50 ` [PATCH edk2-platforms v2 6/6] Platform/ARM: Juno: Add JunoPkg.ci.yaml for CI support PierreGondois
  5 siblings, 0 replies; 8+ messages in thread
From: PierreGondois @ 2021-07-27 16:50 UTC (permalink / raw)
  To: devel, Sean Brogan, Bret Barkelew, Michael D Kinney, Liming Gao,
	Sami Mujawar

From: Pierre Gondois <Pierre.Gondois@arm.com>

To enable CI support of the tianocore/edk2-platforms repository,
add YML configuration files used to run Continuous Integration (CI)
checks on Azure Pipelines agents.

These files are copies of the files from the tianocore/edk2
repository. Any modification to the tianocore/edk2 files must be
reflected on the tianocore/edk2-platforms copies.

The following files have been modified:
 - .azurepipelines/templates/platform-build-run-steps.yml
 - .azurepipelines/templates/pr-gate-build-job.yml
 - .azurepipelines/templates/pr-gate-steps.yml

The sections modified are marked with the following comments:
 -EDK2_PLATFORMS_MODIF_START
 -EDK2_PLATFORMS_MODIF_END

Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3509

Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Sami Mujawar <sami.mujawar@arm.com>
Signed-off-by: Sami Mujawar <sami.mujawar@arm.com>
Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
---
 .azurepipelines/ReadMe.md                     |  50 ++++++
 .azurepipelines/Ubuntu-GCC5.yml               |  21 +++
 .azurepipelines/Ubuntu-PatchCheck.yml         |  36 +++++
 .azurepipelines/Windows-VS2019.yml            |  20 +++
 .azurepipelines/templates/ReadMe.md           |  59 +++++++
 .../templates/basetools-build-steps.yml       |  37 +++++
 .../templates/platform-build-run-steps.yml    | 144 ++++++++++++++++++
 .../templates/pr-gate-build-job.yml           |  63 ++++++++
 .azurepipelines/templates/pr-gate-steps.yml   | 142 +++++++++++++++++
 .../templates/spell-check-prereq-steps.yml    |  22 +++
 10 files changed, 594 insertions(+)
 create mode 100644 .azurepipelines/ReadMe.md
 create mode 100644 .azurepipelines/Ubuntu-GCC5.yml
 create mode 100644 .azurepipelines/Ubuntu-PatchCheck.yml
 create mode 100644 .azurepipelines/Windows-VS2019.yml
 create mode 100644 .azurepipelines/templates/ReadMe.md
 create mode 100644 .azurepipelines/templates/basetools-build-steps.yml
 create mode 100644 .azurepipelines/templates/platform-build-run-steps.yml
 create mode 100644 .azurepipelines/templates/pr-gate-build-job.yml
 create mode 100644 .azurepipelines/templates/pr-gate-steps.yml
 create mode 100644 .azurepipelines/templates/spell-check-prereq-steps.yml

diff --git a/.azurepipelines/ReadMe.md b/.azurepipelines/ReadMe.md
new file mode 100644
index 000000000000..cf57d282c197
--- /dev/null
+++ b/.azurepipelines/ReadMe.md
@@ -0,0 +1,50 @@
+# Azure DevOps Pipelines
+
+These yml files are used to provide CI builds using the Azure DevOps Pipeline Service.
+Most of the CI leverages edk2-pytools to support cross platform building and execution.
+
+## Core CI
+
+Focused on building and testing all packages in Edk2 without an actual target platform.
+
+See `.pytools/ReadMe.py` for more details
+
+## Platform CI
+
+Focused on building a single target platform and confirming functionality on that platform.
+
+## Conventions
+
+* Files extension should be *.yml.  *.yaml is also supported but in Edk2 we use those for our package configuration.
+* Platform CI files should be in the `<PlatformPkg>/.azurepipelines` folder.
+* Core CI files are in the root folder.
+* Shared templates are in the `templates` folder.
+* Top level CI files should be named `<host os>-<tool_chain_tag>.yml`
+
+## Links
+
+* Basic Azure Landing Site - https://docs.microsoft.com/en-us/azure/devops/pipelines/?view=azure-devops
+* Pipeline jobs - https://docs.microsoft.com/en-us/azure/devops/pipelines/process/phases?view=azure-devops&tabs=yaml
+* Pipeline yml scheme - https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=schema%2Cparameter-schema
+* Pipeline expression - https://docs.microsoft.com/en-us/azure/devops/pipelines/process/expressions?view=azure-devops
+* PyTools - https://github.com/tianocore/edk2-pytool-extensions and https://github.com/tianocore/edk2-pytool-library
+
+## Lessons Learned
+
+### Templates and parameters
+
+They are great but evil.  If they are used as part of determining the steps of a build they must resolve before the build starts.  They can not use variables set in a yml or determined as part of a matrix.  If they are used in a step then they can be bound late.
+
+### File matching patterns
+
+On Linux this can hang if there are too many files in the search list.
+
+### Templates and file splitting
+
+Suggestion is to do one big yaml file that does what you want for one of your targets.  Then do the second one and find the deltas.  From that you can start to figure out the right split of files, steps, jobs.
+
+### Conditional steps
+
+If you want the step to show up in the log but not run, use a step conditional. This is great when a platform doesn't currently support a feature but you want the builders to know that the features exists and maybe someday it will.
+
+If you want the step to not show up use a template step conditional wrapper.  Beware this will be evaluated early (at build start).  This can hide things not needed on a given OS for example.
diff --git a/.azurepipelines/Ubuntu-GCC5.yml b/.azurepipelines/Ubuntu-GCC5.yml
new file mode 100644
index 000000000000..3760c6efe105
--- /dev/null
+++ b/.azurepipelines/Ubuntu-GCC5.yml
@@ -0,0 +1,21 @@
+## @file
+# Azure Pipeline build file for a build using ubuntu and GCC5
+#
+# Copyright (c) Microsoft Corporation.
+# Copyright (c) 2020, Hewlett Packard Enterprise Development LP. All rights reserved.<BR>
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+trigger:
+- master
+- stable/*
+pr:
+- master
+- stable/*
+
+jobs:
+- template: templates/pr-gate-build-job.yml
+  parameters:
+    tool_chain_tag: 'GCC5'
+    vm_image: 'ubuntu-latest'
+    arch_list: "IA32,X64,ARM,AARCH64,RISCV64"
+
diff --git a/.azurepipelines/Ubuntu-PatchCheck.yml b/.azurepipelines/Ubuntu-PatchCheck.yml
new file mode 100644
index 000000000000..4de453bf9db8
--- /dev/null
+++ b/.azurepipelines/Ubuntu-PatchCheck.yml
@@ -0,0 +1,36 @@
+## @file
+# Azure Pipielines YML file that evalues the patch series in a PR using the
+# python script BaseTools/Scripts/PatchCheck.py.
+#
+# NOTE: This example monitors pull requests against the edk2-ci branch.  Most
+# environments would replace 'edk2-ci' with 'master'.
+#
+# Copyright (c) 2019 - 2020, Intel Corporation. All rights reserved.<BR>
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+#
+# https://github.com/tianocore
+#
+##
+
+trigger: none
+
+pr:
+- master
+- stable/*
+
+pool:
+  vmImage: 'ubuntu-latest'
+
+steps:
+- checkout: self
+  clean: true
+
+- task: UsePythonVersion@0
+  inputs:
+    versionSpec: '3.7.x'
+    architecture: 'x64'
+
+- script: |
+    git fetch origin $(System.PullRequest.TargetBranch):$(System.PullRequest.TargetBranch)
+    python BaseTools/Scripts/PatchCheck.py $(System.PullRequest.TargetBranch)..$(System.PullRequest.SourceCommitId)
+  displayName: 'Use PatchCheck.py to verify patch series in pull request'
diff --git a/.azurepipelines/Windows-VS2019.yml b/.azurepipelines/Windows-VS2019.yml
new file mode 100644
index 000000000000..22f2d88c2c6a
--- /dev/null
+++ b/.azurepipelines/Windows-VS2019.yml
@@ -0,0 +1,20 @@
+## @file
+# Azure Pipeline build file for a build using Windows and VS2019
+#
+# Copyright (c) Microsoft Corporation.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+trigger:
+- master
+- stable/*
+
+pr:
+- master
+- stable/*
+
+jobs:
+- template: templates/pr-gate-build-job.yml
+  parameters:
+    tool_chain_tag: 'VS2019'
+    vm_image: 'windows-latest'
+    arch_list: "IA32,X64"
diff --git a/.azurepipelines/templates/ReadMe.md b/.azurepipelines/templates/ReadMe.md
new file mode 100644
index 000000000000..fa433e3ef597
--- /dev/null
+++ b/.azurepipelines/templates/ReadMe.md
@@ -0,0 +1,59 @@
+# CI Templates
+
+This folder contains azure pipeline yml templates for "Core" and "Platform" Continuous Integration and PR validation.
+
+## Common CI templates
+
+### basetools-build-steps.yml
+
+This template compiles the Edk2 basetools from source.  The steps in this template are
+conditional and will only run if variable `pkg_count` is greater than 0.
+
+It also has two conditional steps only used when the toolchain contains GCC. These two steps
+use `apt` to update the system packages and add those necessary for Edk2 builds.
+
+## Core CI templates
+
+### pr-gate-build-job.yml
+
+This templates contains the jobs and most importantly the matrix of which packages and
+targets to run for Core CI.
+
+### pr-gate-steps.yml
+
+This template is the main Core CI template.  It controls all the steps run and is responsible for most functionality of the Core CI process.  This template sets
+the `pkg_count` variable using the `stuart_pr_eval` tool when the
+build type is "pull request"
+
+### spell-check-prereq-steps.yml
+
+This template installs the node based tools used by the spell checker plugin. The steps
+in this template are conditional and will only run if variable `pkg_count` is greater than 0.
+
+## Platform CI templates
+
+### platform-build-run-steps.yml
+
+This template makes heavy use of pytools to build and run a platform in the Edk2 repo
+
+Also uses basetools-build-steps.yml to compile basetools
+
+#### Special Notes
+
+* For a build type of pull request it will conditionally build if the patches change files that impact the platform.
+  * uses `stuart_pr_eval` to determine impact
+* For manual builds or CI builds it will always build the platform
+* It compiles basetools from source
+* Will use `stuart_build --FlashOnly` to attempt to run the built image if the `Run` parameter is set.
+* See the parameters block for expected configuration options
+* Parameter `extra_install_step` allows the caller to insert extra steps.  This is useful if additional dependencies, tools, or other things need to be installed.  Here is an example of installing qemu on Windows.
+
+    ``` yaml
+    steps:
+    - template: ../../.azurepipelines/templates/build-run-steps.yml
+      parameters:
+        extra_install_step:
+        - powershell: choco install qemu; Write-Host "##vso[task.prependpath]c:\Program Files\qemu"
+          displayName: Install QEMU and Set QEMU on path # friendly name displayed in the UI
+          condition: and(gt(variables.pkg_count, 0), succeeded())
+    ```
diff --git a/.azurepipelines/templates/basetools-build-steps.yml b/.azurepipelines/templates/basetools-build-steps.yml
new file mode 100644
index 000000000000..d8c108c6e212
--- /dev/null
+++ b/.azurepipelines/templates/basetools-build-steps.yml
@@ -0,0 +1,37 @@
+## @file
+# File templates/basetools-build-job.yml
+#
+# template file to build basetools
+#
+# Copyright (c) Microsoft Corporation.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+parameters:
+  tool_chain_tag: ''
+
+steps:
+- ${{ if contains(parameters.tool_chain_tag, 'GCC') }}:
+  - bash: sudo apt-get update
+    displayName: Update apt
+    condition: and(gt(variables.pkg_count, 0), succeeded())
+
+  - bash: sudo apt-get install gcc g++ make uuid-dev
+    displayName: Install required tools
+    condition: and(gt(variables.pkg_count, 0), succeeded())
+
+- task: CmdLine@1
+  displayName: Build Base Tools from source
+  inputs:
+    filename: python
+    arguments: BaseTools/Edk2ToolsBuild.py -t ${{ parameters.tool_chain_tag }}
+  condition: and(gt(variables.pkg_count, 0), succeeded())
+
+- task: CopyFiles@2
+  displayName: "Copy base tools build log"
+  inputs:
+    targetFolder: '$(Build.ArtifactStagingDirectory)'
+    SourceFolder: 'BaseTools/BaseToolsBuild'
+    contents: |
+      BASETOOLS_BUILD*.*
+    flattenFolders: true
+  condition: and(gt(variables.pkg_count, 0), succeededOrFailed())
diff --git a/.azurepipelines/templates/platform-build-run-steps.yml b/.azurepipelines/templates/platform-build-run-steps.yml
new file mode 100644
index 000000000000..2a07e387754b
--- /dev/null
+++ b/.azurepipelines/templates/platform-build-run-steps.yml
@@ -0,0 +1,144 @@
+
+## @file
+# File steps.yml
+#
+# template file containing the steps to build
+#
+# Copyright (c) 2021, Arm Limited. All rights reserved.<BR>
+# Copyright (c) Microsoft Corporation.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+
+parameters:
+- name: tool_chain_tag
+  type: string
+  default: ''
+- name: build_pkg
+  type: string
+  default: ''
+- name: build_target
+  type: string
+  default: ''
+- name: build_arch
+  type: string
+  default: ''
+- name: build_file
+  type: string
+  default: ''
+- name: build_flags
+  type: string
+  default: ''
+- name: run_flags
+  type: string
+  default: ''
+
+- name: extra_install_step
+  type: stepList
+  default: []
+
+steps:
+- checkout: self
+  clean: true
+  fetchDepth: 1
+
+- task: UsePythonVersion@0
+  inputs:
+    versionSpec: "3.8.x"
+    architecture: "x64"
+
+- script: pip install -r pip-requirements.txt --upgrade
+  displayName: 'Install/Upgrade pip modules'
+
+# Set default
+- bash: echo "##vso[task.setvariable variable=pkg_count]${{ 1 }}"
+
+# Fetch the target branch so that pr_eval can diff them.
+# Seems like azure pipelines/github changed checkout process in nov 2020.
+- script: git fetch origin $(System.PullRequest.targetBranch)
+  displayName: fetch target branch
+  condition: eq(variables['Build.Reason'], 'PullRequest')
+
+# trim the package list if this is a PR
+- task: CmdLine@1
+  displayName: Check if ${{ parameters.build_pkg }} need testing
+  inputs:
+    filename: stuart_pr_eval
+    arguments: -c ${{ parameters.build_file }} -t ${{ parameters.build_target}} -a ${{ parameters.build_arch}} --pr-target origin/$(System.PullRequest.targetBranch) --output-count-format-string "##vso[task.setvariable variable=pkg_count;isOutpout=true]{pkgcount}"
+  condition: eq(variables['Build.Reason'], 'PullRequest')
+
+ # Setup repo
+- task: CmdLine@1
+  displayName: Setup
+  inputs:
+    filename: stuart_setup
+    arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}} -t ${{ parameters.build_target}} -a ${{ parameters.build_arch}} ${{ parameters.build_flags}}
+  condition: and(gt(variables.pkg_count, 0), succeeded())
+
+# Stuart Update
+- task: CmdLine@1
+  displayName: Update
+  inputs:
+    filename: stuart_update
+    arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}} -t ${{ parameters.build_target}} -a ${{ parameters.build_arch}} ${{ parameters.build_flags}}
+  condition: and(gt(variables.pkg_count, 0), succeeded())
+
+# EDK2_PLATFORMS_MODIF_START:
+# The base tools are imported in .pytool/CISettings.py via the 'edk2basetools' python module.
+# # build basetools
+# #   do this after setup and update so that code base dependencies
+# #   are all resolved.
+# - template: basetools-build-steps.yml
+#   parameters:
+#     tool_chain_tag: ${{ parameters.tool_chain_tag }}
+# EDK2_PLATFORMS_MODIF_END
+
+# Potential Extra steps
+- ${{ parameters.extra_install_step }}
+
+# Build
+- task: CmdLine@1
+  displayName: Build
+  inputs:
+    filename: stuart_build
+    arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}} TARGET=${{ parameters.build_target}} -a ${{ parameters.build_arch}} ${{ parameters.build_flags}}
+  condition: and(gt(variables.pkg_count, 0), succeeded())
+
+# Run
+- task: CmdLine@1
+  displayName: Run to shell
+  inputs:
+    filename: stuart_build
+    arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}} TARGET=${{ parameters.build_target}} -a ${{ parameters.build_arch}} ${{ parameters.build_flags}} ${{ parameters.run_flags }} --FlashOnly
+  condition: and(and(gt(variables.pkg_count, 0), succeeded()), eq(variables['Run'], true))
+  timeoutInMinutes: 1
+
+# Copy the build logs to the artifact staging directory
+- task: CopyFiles@2
+  displayName: "Copy build logs"
+  inputs:
+    targetFolder: "$(Build.ArtifactStagingDirectory)"
+    SourceFolder: "Build"
+    contents: |
+      BUILDLOG_*.txt
+      BUILDLOG_*.md
+      CI_*.txt
+      CI_*.md
+      CISETUP.txt
+      SETUPLOG.txt
+      UPDATE_LOG.txt
+      PREVALLOG.txt
+      TestSuites.xml
+      **/BUILD_TOOLS_REPORT.html
+      **/OVERRIDELOG.TXT
+      BASETOOLS_BUILD*.*
+    flattenFolders: true
+  condition: succeededOrFailed()
+
+# Publish build artifacts to Azure Artifacts/TFS or a file share
+- task: PublishBuildArtifacts@1
+  continueOnError: true
+  displayName: "Publish build logs"
+  inputs:
+    pathtoPublish: "$(Build.ArtifactStagingDirectory)"
+    artifactName: "Build Logs $(System.JobName)"
+  condition: succeededOrFailed()
diff --git a/.azurepipelines/templates/pr-gate-build-job.yml b/.azurepipelines/templates/pr-gate-build-job.yml
new file mode 100644
index 000000000000..e436618a91f2
--- /dev/null
+++ b/.azurepipelines/templates/pr-gate-build-job.yml
@@ -0,0 +1,63 @@
+## @file
+# File templates/pr-gate-build-job.yml
+#
+# template file used to build supported packages.
+#
+# Copyright (c) Microsoft Corporation.
+# Copyright (c) 2020 - 2021, ARM Limited. All rights reserved.<BR>
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+
+parameters:
+  tool_chain_tag: ''
+  vm_image: ''
+  arch_list: ''
+
+# Build step
+jobs:
+
+- job: Build_${{ parameters.tool_chain_tag }}
+
+# EDK2_PLATFORMS_MODIF_START:
+# Build edk2-platforms packages
+  #Use matrix to speed up the build process
+  strategy:
+    matrix:
+      TARGET_ARM_PLATFORMS:
+        Build.Pkgs: 'JunoPkg'
+        Build.Targets: 'DEBUG,RELEASE,NO-TARGET,NOOPT'
+# EDK2_PLATFORMS_MODIF_END
+
+  workspace:
+    clean: all
+
+  pool:
+    vmImage: ${{ parameters.vm_image }}
+
+  steps:
+  - template: pr-gate-steps.yml
+    parameters:
+      tool_chain_tag: ${{ parameters.tool_chain_tag }}
+      build_pkgs: $(Build.Pkgs)
+      build_targets: $(Build.Targets)
+      build_archs: ${{ parameters.arch_list }}
+
+- job: FINISHED
+  dependsOn: Build_${{ parameters.tool_chain_tag }}
+  condition: succeeded()
+  steps:
+  - checkout: none
+  - script: |
+      echo FINISHED
+      sleep 10
+    displayName: FINISHED
+
+- job: FAILED
+  dependsOn: Build_${{ parameters.tool_chain_tag }}
+  condition: failed()
+  steps:
+  - checkout: none
+  - script: |
+      echo FAILED
+      sleep 10
+    displayName: FAILED
diff --git a/.azurepipelines/templates/pr-gate-steps.yml b/.azurepipelines/templates/pr-gate-steps.yml
new file mode 100644
index 000000000000..0a8a0e1e09a3
--- /dev/null
+++ b/.azurepipelines/templates/pr-gate-steps.yml
@@ -0,0 +1,142 @@
+## @file
+# File templates/pr-gate-steps.yml
+#
+# template file containing the steps to build
+#
+# Copyright (c) 2021, Arm Limited. All rights reserved.<BR>
+# Copyright (c) Microsoft Corporation.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+
+parameters:
+  tool_chain_tag: ''
+  build_pkgs: ''
+  build_targets: ''
+  build_archs: ''
+
+steps:
+- checkout: self
+  clean: true
+  fetchDepth: 1
+
+- task: UsePythonVersion@0
+  inputs:
+    versionSpec: '3.8.x'
+    architecture: 'x64'
+
+- script: pip install -r pip-requirements.txt --upgrade
+  displayName: 'Install/Upgrade pip modules'
+
+# Set default
+- bash: |
+    echo "##vso[task.setvariable variable=pkgs_to_build]${{ parameters.build_pkgs }}"
+    echo "##vso[task.setvariable variable=pkg_count]${{ 1 }}"
+
+# Fetch the target branch so that pr_eval can diff them.
+# Seems like azure pipelines/github changed checkout process in nov 2020.
+- script: git fetch origin $(System.PullRequest.targetBranch)
+  displayName: fetch target branch
+  condition: eq(variables['Build.Reason'], 'PullRequest')
+
+# trim the package list if this is a PR
+- task: CmdLine@1
+  displayName: Check if ${{ parameters.build_pkgs }} need testing
+  inputs:
+    filename: stuart_pr_eval
+    arguments: -c .pytool/CISettings.py -p ${{ parameters.build_pkgs }} --pr-target origin/$(System.PullRequest.targetBranch) --output-csv-format-string "##vso[task.setvariable variable=pkgs_to_build;isOutpout=true]{pkgcsv}" --output-count-format-string "##vso[task.setvariable variable=pkg_count;isOutpout=true]{pkgcount}"
+  condition: eq(variables['Build.Reason'], 'PullRequest')
+
+# install spell check prereqs
+- template: spell-check-prereq-steps.yml
+
+# Build repo
+- task: CmdLine@1
+  displayName: Setup ${{ parameters.build_pkgs }} ${{ parameters.build_archs}}
+  inputs:
+    filename: stuart_setup
+    arguments: -c .pytool/CISettings.py -p $(pkgs_to_build) -t ${{ parameters.build_targets}} -a ${{ parameters.build_archs}} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}}
+  condition: and(gt(variables.pkg_count, 0), succeeded())
+
+- task: CmdLine@1
+  displayName: Update ${{ parameters.build_pkgs }} ${{ parameters.build_archs}}
+  inputs:
+    filename: stuart_update
+    arguments: -c .pytool/CISettings.py -p $(pkgs_to_build) -t ${{ parameters.build_targets}} -a ${{ parameters.build_archs}} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}}
+  condition: and(gt(variables.pkg_count, 0), succeeded())
+
+# build basetools
+#   do this after setup and update so that code base dependencies
+#   are all resolved.
+# EDK2_PLATFORMS_MODIF_START:
+# The base tools are imported in .pytool/CISettings.py via the 'edk2basetools' python module.
+# - template: basetools-build-steps.yml
+#   parameters:
+#     tool_chain_tag: ${{ parameters.tool_chain_tag }}
+# EDK2_PLATFORMS_MODIF_END
+
+- task: CmdLine@1
+  displayName: Build and Test ${{ parameters.build_pkgs }} ${{ parameters.build_archs}}
+  inputs:
+    filename: stuart_ci_build
+    arguments: -c .pytool/CISettings.py -p $(pkgs_to_build) -t ${{ parameters.build_targets}} -a ${{ parameters.build_archs}} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}}
+  condition: and(gt(variables.pkg_count, 0), succeeded())
+
+# Publish Test Results to Azure Pipelines/TFS
+- task: PublishTestResults@2
+  displayName: 'Publish junit test results'
+  continueOnError: true
+  condition: and( succeededOrFailed(),gt(variables.pkg_count, 0))
+  inputs:
+    testResultsFormat: 'JUnit' # Options: JUnit, NUnit, VSTest, xUnit
+    testResultsFiles: 'Build/TestSuites.xml'
+    #searchFolder: '$(System.DefaultWorkingDirectory)' # Optional
+    mergeTestResults: true # Optional
+    testRunTitle: $(System.JobName) # Optional
+    #buildPlatform: # Optional
+    #buildConfiguration: # Optional
+    publishRunAttachments: true # Optional
+
+# Publish Test Results to Azure Pipelines/TFS
+- task: PublishTestResults@2
+  displayName: 'Publish host based test results for $(System.JobName)'
+  continueOnError: true
+  condition: and( succeededOrFailed(), gt(variables.pkg_count, 0))
+  inputs:
+    testResultsFormat: 'JUnit' # Options: JUnit, NUnit, VSTest, xUnit
+    testResultsFiles: 'Build/**/*.result.xml'
+    #searchFolder: '$(System.DefaultWorkingDirectory)' # Optional
+    mergeTestResults: false # Optional
+    testRunTitle: ${{ parameters.build_pkgs }} # Optional
+    #buildPlatform: # Optional
+    #buildConfiguration: # Optional
+    publishRunAttachments: true # Optional
+
+# Copy the build logs to the artifact staging directory
+- task: CopyFiles@2
+  displayName: "Copy build logs"
+  inputs:
+    targetFolder: '$(Build.ArtifactStagingDirectory)'
+    SourceFolder: 'Build'
+    contents: |
+      BUILDLOG_*.txt
+      BUILDLOG_*.md
+      CI_*.txt
+      CI_*.md
+      CISETUP.txt
+      SETUPLOG.txt
+      UPDATE_LOG.txt
+      PREVALLOG.txt
+      TestSuites.xml
+      **/BUILD_TOOLS_REPORT.html
+      **/OVERRIDELOG.TXT
+    flattenFolders: true
+  condition: succeededOrFailed()
+
+# Publish build artifacts to Azure Artifacts/TFS or a file share
+- task: PublishBuildArtifacts@1
+  continueOnError: true
+  displayName: "Publish build logs"
+  inputs:
+    pathtoPublish: '$(Build.ArtifactStagingDirectory)'
+    artifactName: 'Build Logs $(System.JobName)'
+  condition: succeededOrFailed()
diff --git a/.azurepipelines/templates/spell-check-prereq-steps.yml b/.azurepipelines/templates/spell-check-prereq-steps.yml
new file mode 100644
index 000000000000..e1570d4f2aac
--- /dev/null
+++ b/.azurepipelines/templates/spell-check-prereq-steps.yml
@@ -0,0 +1,22 @@
+## @file
+# File templates/spell-check-prereq-steps.yml
+#
+# template file used to install spell checking prerequisits
+#
+# Copyright (c) Microsoft Corporation.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+
+parameters:
+  none: ''
+
+steps:
+- task: NodeTool@0
+  inputs:
+    versionSpec: '10.x'
+    #checkLatest: false # Optional
+  condition: and(gt(variables.pkg_count, 0), succeeded())
+
+- script: npm install -g cspell
+  displayName: 'Install cspell npm'
+  condition: and(gt(variables.pkg_count, 0), succeeded())
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 8+ messages in thread

* [PATCH edk2-platforms v2 5/6] .mergify: Add Mergify YML pull request rules configuration file
  2021-07-27 16:49 [PATCH edk2-platforms v2 0/6] Enable edk2-platforms CI for JunoPkg PierreGondois
                   ` (3 preceding siblings ...)
  2021-07-27 16:50 ` [PATCH edk2-platforms v2 4/6] .azurepipelines: Add Azure Pipelines YML configuration files PierreGondois
@ 2021-07-27 16:50 ` PierreGondois
  2021-07-28 20:25   ` [edk2-devel] " Michael D Kinney
  2021-07-27 16:50 ` [PATCH edk2-platforms v2 6/6] Platform/ARM: Juno: Add JunoPkg.ci.yaml for CI support PierreGondois
  5 siblings, 1 reply; 8+ messages in thread
From: PierreGondois @ 2021-07-27 16:50 UTC (permalink / raw)
  To: devel, Sean Brogan, Bret Barkelew, Michael D Kinney, Liming Gao,
	Sami Mujawar

From: Pierre Gondois <Pierre.Gondois@arm.com>

These files are copies of the files from the tianocore/edk2
repository. Any modification to the tianocore/edk2 files must be
reflected on the tianocore/edk2-platforms copies.

Initial commid-id in the edk2 repository: ab060128768b
Initial message:

Add directory for the Mergify YML configuration files that
provides rules and actions used to process a pull request.

* Auto commit a PR from EDK II Maintainer with 'push' label
  set and all CI checks pass
* Auto close a PR from any developers without 'push' label
  set and all CI checks pass.
* Auto close a PR from a non EDK II Maintainer that has
  the 'push' label set.
* Post a comment to a PR that has a merge conflict.
  Submitter can resolved conflicts and reopen the PR.
* Post a comment to a PR that fails PatchCheck.py
  Submitter can resolve PatchCheck.py issues and
  reopen the PR.

Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3509

Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Sami Mujawar <sami.mujawar@arm.com>
Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
---
 .mergify/config.yml | 98 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 98 insertions(+)
 create mode 100644 .mergify/config.yml

diff --git a/.mergify/config.yml b/.mergify/config.yml
new file mode 100644
index 000000000000..ee27a4152a75
--- /dev/null
+++ b/.mergify/config.yml
@@ -0,0 +1,98 @@
+## @file
+# Mergify YML file that automatically merges a GitHub pull request against
+# edk2-ci if all of the GitHub branch protections have passed.  It also
+# contains rules to:
+# * auto close branches that are not from an EDK II Maintainer
+# * post a comment on pull requests that have merge conflicts.
+# * post a comment on pull requests that have PatchCheck.py errors.
+#
+# Configuration Notes:
+# * Update the 'base=edk2-ci' statements with the name of the branch to merge
+#   pull requests.
+#
+# * Update the 'status-failure' statement with the name of the name of the Azure
+#   Pipelines Build that performs the EDK II Maintainer check.
+#
+# * This file must be checked into the 'default' branch of a repo.  Copies
+#   of this file on other branches of a repo are ignored by Mergify.
+#
+# Copyright (c) 2021, Arm Ltd. All rights reserved.<BR>
+# Copyright (c) 2019 - 2020, Intel Corporation. All rights reserved.<BR>
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+#
+# https://github.com/apps/mergify
+# https://doc.mergify.io/
+#
+##
+
+pull_request_rules:
+
+  - name: Automatically merge a PR when all required checks pass and 'push' label is present
+    conditions:
+      - base~=(^master|^stable/)
+      - label=push
+      - author=@tianocore/edk-ii-maintainers
+      - status-success=tianocore.PatchCheck
+      - status-success=Ubuntu GCC5 PR
+      - status-success=Windows VS2019 PR
+    actions:
+      merge:
+        strict: true
+        method: rebase
+
+  - name: Automatically close a PR when all required checks pass and 'push' label is not present
+    conditions:
+      - base~=(^master|^stable/)
+      - -label=push
+      - -closed
+      - status-success=tianocore.PatchCheck
+      - status-success=Ubuntu GCC5 PR
+      - status-success=Windows VS2019 PR
+      - status-success=Ubuntu GCC5 PR (FINISHED)
+      - status-success=Windows VS2019 PR (FINISHED)
+    actions:
+      close:
+        message: All checks passed. Auto close personal build.
+
+  - name: Post a comment on a PR that can not be merged due to a merge conflict
+    conditions:
+      - base~=(^master|^stable/)
+      - conflict
+    actions:
+      comment:
+        message: PR can not be merged due to conflict.  Please rebase and resubmit
+
+  - name: Automatically close a PR that fails the EDK II Maintainers membership check and 'push' label is present
+    conditions:
+      - base~=(^master|^stable/)
+      - label=push
+      - -author=@tianocore/edk-ii-maintainers
+    actions:
+      close:
+        message: PR submitter is not a member of the Tianocore EDK II Maintainers team
+
+  - name: Post a comment on a PR if PatchCheck fails
+    conditions:
+      - base~=(^master|^stable/)
+      - status-failure=tianocore.PatchCheck
+    actions:
+      comment:
+        message: PR can not be merged due to a PatchCheck failure.  Please resolve and resubmit
+
+  - name: Post a comment on a PR if Ubuntu GCC5 fails
+    conditions:
+      - base~=(^master|^stable/)
+      - status-failure=Ubuntu GCC5 PR
+      - status-success=Ubuntu GCC5 PR (FAILED)
+    actions:
+      comment:
+        message: PR can not be merged due to an Ubuntu GCC5 failure.  Please resolve and resubmit
+
+  - name: Post a comment on a PR if Windows VS2019 fails
+    conditions:
+      - base~=(^master|^stable/)
+      - status-failure=Windows VS2019 PR
+      - status-success=Windows VS2019 PR (FAILED)
+    actions:
+      comment:
+        message: PR can not be merged due to a Windows VS2019 failure.  Please resolve and resubmit
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 8+ messages in thread

* [PATCH edk2-platforms v2 6/6] Platform/ARM: Juno: Add JunoPkg.ci.yaml for CI support
  2021-07-27 16:49 [PATCH edk2-platforms v2 0/6] Enable edk2-platforms CI for JunoPkg PierreGondois
                   ` (4 preceding siblings ...)
  2021-07-27 16:50 ` [PATCH edk2-platforms v2 5/6] .mergify: Add Mergify YML pull request rules configuration file PierreGondois
@ 2021-07-27 16:50 ` PierreGondois
  5 siblings, 0 replies; 8+ messages in thread
From: PierreGondois @ 2021-07-27 16:50 UTC (permalink / raw)
  To: devel, Sean Brogan, Bret Barkelew, Michael D Kinney, Liming Gao,
	Sami Mujawar

From: Pierre Gondois <Pierre.Gondois@arm.com>

Add a JunoPkg.ci.yaml file to enable the CI for the JunoPkg.

Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Sami Mujawar <sami.mujawar@arm.com>
Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
---
 Platform/ARM/JunoPkg/JunoPkg.ci.yaml | 104 +++++++++++++++++++++++++++
 1 file changed, 104 insertions(+)
 create mode 100644 Platform/ARM/JunoPkg/JunoPkg.ci.yaml

diff --git a/Platform/ARM/JunoPkg/JunoPkg.ci.yaml b/Platform/ARM/JunoPkg/JunoPkg.ci.yaml
new file mode 100644
index 000000000000..7e7f201b40ec
--- /dev/null
+++ b/Platform/ARM/JunoPkg/JunoPkg.ci.yaml
@@ -0,0 +1,104 @@
+## @file
+# Core CI configuration for JunoPkg
+#
+# VExpressPkg is part of Platform CI for builds so this is only
+# used for code analysis.
+#
+# Copyright (c) Microsoft Corporation
+# Copyright (c) 2021, Arm Ltd. All rights reserved.<BR>
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+    ## options defined .pytool/Plugin/LicenseCheck
+    "LicenseCheck": {
+        "IgnoreFiles": []
+    },
+    "EccCheck": {
+        ## Exception sample looks like below:
+        ## "ExceptionList": [
+        ##     "<ErrorID>", "<KeyWord>"
+        ## ]
+        "ExceptionList": [
+        ],
+        ## Both file path and directory path are accepted.
+        "IgnoreFiles": [
+        ]
+    },
+    ## options defined .pytool/Plugin/CompilerPlugin
+    "CompilerPlugin": {
+        "DscPath": "" # Don't support this test
+                      # Build the Package using a PlatformCI, similarly to ArmVirtPkg.
+    },
+
+    ## options defined .pytool/Plugin/HostUnitTestCompilerPlugin
+    "HostUnitTestCompilerPlugin": {
+        "DscPath": "" # Don't support this test
+    },
+
+    ## options defined .pytool/Plugin/CharEncodingCheck
+    "CharEncodingCheck": {
+        "IgnoreFiles": []
+    },
+
+    ## options defined .pytool/Plugin/DependencyCheck
+    "DependencyCheck": {
+        "AcceptableDependencies": [
+            "ArmPkg/ArmPkg.dec",
+            "ArmPlatformPkg/ArmPlatformPkg.dec",
+            "DynamicTablesPkg/DynamicTablesPkg.dec",
+            "EmbeddedPkg/EmbeddedPkg.dec",
+            "MdePkg/MdePkg.dec",
+            "MdeModulePkg/MdeModulePkg.dec",
+            "Platform/ARM/ARM.dec",
+            "Platform/ARM/Drivers/FdtPlatformDxe/FdtPlatformDxe.dec",
+            "Platform/ARM/JunoPkg/ArmJuno.dec",
+        ],
+        # For host based unit tests
+        "AcceptableDependencies-HOST_APPLICATION":[
+            "UnitTestFrameworkPkg/UnitTestFrameworkPkg.dec"
+        ],
+        # For UEFI shell based apps
+        "AcceptableDependencies-UEFI_APPLICATION":[
+
+        ],
+        "IgnoreInf": []
+    },
+
+    ## options defined .pytool/Plugin/DscCompleteCheck
+    "DscCompleteCheck": {
+        "IgnoreInf": [""],
+        "DscPath": ""  # Don't support this test
+        # "DscPath": "ArmJuno.dsc"  # Don't support this test
+    },
+
+    ## options defined .pytool/Plugin/HostUnitTestDscCompleteCheck
+    "HostUnitTestDscCompleteCheck": {
+        "IgnoreInf": [""],
+        "DscPath": "" # Don't support this test
+    },
+
+    ## options defined .pytool/Plugin/GuidCheck
+    "GuidCheck": {
+        "IgnoreGuidName": [], # Expected duplication for gEfiFirmwareVolumeTopFileGuid
+        "IgnoreGuidValue": [
+        ],
+        "IgnoreFoldersAndFiles": [],
+        "IgnoreDuplicates": [],
+    },
+
+    ## options defined .pytool/Plugin/LibraryClassCheck
+    "LibraryClassCheck": {
+        "IgnoreHeaderFile": []
+    },
+
+    ## options defined .pytool/Plugin/SpellCheck
+    "SpellCheck": {
+        "AuditOnly": True,
+        "IgnoreFiles": [],           # use gitignore syntax to ignore errors in matching files
+        "ExtendWords": [
+
+        ],           # words to extend to the dictionary for this package
+        "IgnoreStandardPaths": [],   # Standard Plugin defined paths that should be ignore
+        "AdditionalIncludePaths": [] # Additional paths to spell check (wildcards supported)
+    }
+}
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 8+ messages in thread

* Re: [edk2-devel] [PATCH edk2-platforms v2 5/6] .mergify: Add Mergify YML pull request rules configuration file
  2021-07-27 16:50 ` [PATCH edk2-platforms v2 5/6] .mergify: Add Mergify YML pull request rules configuration file PierreGondois
@ 2021-07-28 20:25   ` Michael D Kinney
  0 siblings, 0 replies; 8+ messages in thread
From: Michael D Kinney @ 2021-07-28 20:25 UTC (permalink / raw)
  To: devel@edk2.groups.io, pierre.gondois@arm.com, Sean Brogan,
	Bret Barkelew, Liming Gao, Sami Mujawar, Kinney, Michael D

The Mergify rules in edk2 repo were recently updated.

Please align with the latest that uses the new Mergify queue feature
along with other improvements.

Thanks,

Mike

> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of PierreGondois
> Sent: Tuesday, July 27, 2021 9:50 AM
> To: devel@edk2.groups.io; Sean Brogan <sean.brogan@microsoft.com>; Bret Barkelew <Bret.Barkelew@microsoft.com>; Kinney,
> Michael D <michael.d.kinney@intel.com>; Liming Gao <gaoliming@byosoft.com.cn>; Sami Mujawar <sami.mujawar@arm.com>
> Subject: [edk2-devel] [PATCH edk2-platforms v2 5/6] .mergify: Add Mergify YML pull request rules configuration file
> 
> From: Pierre Gondois <Pierre.Gondois@arm.com>
> 
> These files are copies of the files from the tianocore/edk2
> repository. Any modification to the tianocore/edk2 files must be
> reflected on the tianocore/edk2-platforms copies.
> 
> Initial commid-id in the edk2 repository: ab060128768b
> Initial message:
> 
> Add directory for the Mergify YML configuration files that
> provides rules and actions used to process a pull request.
> 
> * Auto commit a PR from EDK II Maintainer with 'push' label
>   set and all CI checks pass
> * Auto close a PR from any developers without 'push' label
>   set and all CI checks pass.
> * Auto close a PR from a non EDK II Maintainer that has
>   the 'push' label set.
> * Post a comment to a PR that has a merge conflict.
>   Submitter can resolved conflicts and reopen the PR.
> * Post a comment to a PR that fails PatchCheck.py
>   Submitter can resolve PatchCheck.py issues and
>   reopen the PR.
> 
> Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3509
> 
> Cc: Sean Brogan <sean.brogan@microsoft.com>
> Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
> Cc: Michael D Kinney <michael.d.kinney@intel.com>
> Cc: Liming Gao <gaoliming@byosoft.com.cn>
> Cc: Sami Mujawar <sami.mujawar@arm.com>
> Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
> ---
>  .mergify/config.yml | 98 +++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 98 insertions(+)
>  create mode 100644 .mergify/config.yml
> 
> diff --git a/.mergify/config.yml b/.mergify/config.yml
> new file mode 100644
> index 000000000000..ee27a4152a75
> --- /dev/null
> +++ b/.mergify/config.yml
> @@ -0,0 +1,98 @@
> +## @file
> +# Mergify YML file that automatically merges a GitHub pull request against
> +# edk2-ci if all of the GitHub branch protections have passed.  It also
> +# contains rules to:
> +# * auto close branches that are not from an EDK II Maintainer
> +# * post a comment on pull requests that have merge conflicts.
> +# * post a comment on pull requests that have PatchCheck.py errors.
> +#
> +# Configuration Notes:
> +# * Update the 'base=edk2-ci' statements with the name of the branch to merge
> +#   pull requests.
> +#
> +# * Update the 'status-failure' statement with the name of the name of the Azure
> +#   Pipelines Build that performs the EDK II Maintainer check.
> +#
> +# * This file must be checked into the 'default' branch of a repo.  Copies
> +#   of this file on other branches of a repo are ignored by Mergify.
> +#
> +# Copyright (c) 2021, Arm Ltd. All rights reserved.<BR>
> +# Copyright (c) 2019 - 2020, Intel Corporation. All rights reserved.<BR>
> +# SPDX-License-Identifier: BSD-2-Clause-Patent
> +#
> +# https://github.com/apps/mergify
> +# https://doc.mergify.io/
> +#
> +##
> +
> +pull_request_rules:
> +
> +  - name: Automatically merge a PR when all required checks pass and 'push' label is present
> +    conditions:
> +      - base~=(^master|^stable/)
> +      - label=push
> +      - author=@tianocore/edk-ii-maintainers
> +      - status-success=tianocore.PatchCheck
> +      - status-success=Ubuntu GCC5 PR
> +      - status-success=Windows VS2019 PR
> +    actions:
> +      merge:
> +        strict: true
> +        method: rebase
> +
> +  - name: Automatically close a PR when all required checks pass and 'push' label is not present
> +    conditions:
> +      - base~=(^master|^stable/)
> +      - -label=push
> +      - -closed
> +      - status-success=tianocore.PatchCheck
> +      - status-success=Ubuntu GCC5 PR
> +      - status-success=Windows VS2019 PR
> +      - status-success=Ubuntu GCC5 PR (FINISHED)
> +      - status-success=Windows VS2019 PR (FINISHED)
> +    actions:
> +      close:
> +        message: All checks passed. Auto close personal build.
> +
> +  - name: Post a comment on a PR that can not be merged due to a merge conflict
> +    conditions:
> +      - base~=(^master|^stable/)
> +      - conflict
> +    actions:
> +      comment:
> +        message: PR can not be merged due to conflict.  Please rebase and resubmit
> +
> +  - name: Automatically close a PR that fails the EDK II Maintainers membership check and 'push' label is present
> +    conditions:
> +      - base~=(^master|^stable/)
> +      - label=push
> +      - -author=@tianocore/edk-ii-maintainers
> +    actions:
> +      close:
> +        message: PR submitter is not a member of the Tianocore EDK II Maintainers team
> +
> +  - name: Post a comment on a PR if PatchCheck fails
> +    conditions:
> +      - base~=(^master|^stable/)
> +      - status-failure=tianocore.PatchCheck
> +    actions:
> +      comment:
> +        message: PR can not be merged due to a PatchCheck failure.  Please resolve and resubmit
> +
> +  - name: Post a comment on a PR if Ubuntu GCC5 fails
> +    conditions:
> +      - base~=(^master|^stable/)
> +      - status-failure=Ubuntu GCC5 PR
> +      - status-success=Ubuntu GCC5 PR (FAILED)
> +    actions:
> +      comment:
> +        message: PR can not be merged due to an Ubuntu GCC5 failure.  Please resolve and resubmit
> +
> +  - name: Post a comment on a PR if Windows VS2019 fails
> +    conditions:
> +      - base~=(^master|^stable/)
> +      - status-failure=Windows VS2019 PR
> +      - status-success=Windows VS2019 PR (FAILED)
> +    actions:
> +      comment:
> +        message: PR can not be merged due to a Windows VS2019 failure.  Please resolve and resubmit
> --
> 2.17.1
> 
> 
> 
> 
> 


^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2021-07-28 20:25 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2021-07-27 16:49 [PATCH edk2-platforms v2 0/6] Enable edk2-platforms CI for JunoPkg PierreGondois
2021-07-27 16:49 ` [PATCH edk2-platforms v2 1/6] edk2-platforms: add edk2 repository as a submodule PierreGondois
2021-07-27 16:50 ` [PATCH edk2-platforms v2 2/6] pip-requirements.txt: Add python pip requirements file PierreGondois
2021-07-27 16:50 ` [PATCH edk2-platforms v2 3/6] .pytool/Plugin: Add CI plugins PierreGondois
2021-07-27 16:50 ` [PATCH edk2-platforms v2 4/6] .azurepipelines: Add Azure Pipelines YML configuration files PierreGondois
2021-07-27 16:50 ` [PATCH edk2-platforms v2 5/6] .mergify: Add Mergify YML pull request rules configuration file PierreGondois
2021-07-28 20:25   ` [edk2-devel] " Michael D Kinney
2021-07-27 16:50 ` [PATCH edk2-platforms v2 6/6] Platform/ARM: Juno: Add JunoPkg.ci.yaml for CI support PierreGondois

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox