[PATCH edk2-platforms v6 0/6] Enable edk2-platforms CI for JunoPkg

[PATCH edk2-platforms v6 0/6] Enable edk2-platforms CI for JunoPkg

[PierreGondois]

From: Pierre Gondois <Pierre.Gondois@arm.com>

v6:
 - New patch not have the edk2 repository as an external dependency
   instead of a submodule. [Michael]
 - Use EDK2_REPO to use a custom edk2 repository instead of downloading
   it. [Pierre]
 - PatchCheck to install pip-requirements and download edk2. [Pierre]
v5:
 - Remove Signed-off from Sami (I replaced it with Co-authored-by)
   [Leif]
 - Update edk2 submodule to latest master. [Pierre]
v4:
 - Checkout edk2 repository in Ubuntu-PatchCheck.yml to
   get the PatchCheck.py script. [Pierre]
v3:
 - Replace Readme(s)'s by a reference to the initial Readmed(s) in
   tianocore repository. [Sean]
 - Align CI files with the latest version in the edk2 repository.
   [Sean/Michael]
 - Add additional step to checkout edk2's latest master in the CI.
   [Pierre]
v2:
 - Use tianocore repository instead of personal repository
   for edk2 submodule [Pierre]
 - Bugzilla associated to the topic:
   https://bugzilla.tianocore.org/show_bug.cgi?id=3509

It provides the configuration files necessary to run an upstream CI
similar to the one currently used for the main edk2 repository. The
configuration is mostly similar aswell.
Enabling the CI requires administrator rights on the edk2-platforms
repository. This configuration was tested on a private repository,
but will require additional configuration from the administrator.

CI plugins (.pytool/Plugin/) and BaseTools are located in the
Tianocore edk2 repository. To re-use them, add the edk2 repository
as an external dependency.
It is not currently possible to specify a branch name in the "version"
field. Thus, a fixed commit hash (current edk2 master) is written.
It will eventually be replaced by 'master'.
If a platform requires a specific version of edk2, another description
of the edk2 repository in the edk2_ext_dep.yaml can be added. Only the
edk2 repository with the right scope will be installed.
It is possible to use a custom edk2 repository instead. To do so, run
the stuart_[update|build] commands with EDK2_REPO pointing to your
custom edk2 repository.

Compared to edk2's CI, some steps are modified and indicated by
EDK2_PLATFORMS_MODIF_[START|END] statements.

The changes can be seen at:
https://github.com/PierreARM/edk2-platforms/tree/1628_Enable_edk2_platforms_ci_for_JunoPkg_v6

Pierre Gondois (6):
  pip-requirements.txt: Add python pip requirements file
  CI: Add Tianocore edk2 as an external dependency
  .pytool/Plugin: Add CI plugins
  .azurepipelines: Add Azure Pipelines YML configuration files
  .mergify: Add Mergify YML pull request rules configuration file
  Platform/ARM: Juno: Add JunoPkg.ci.yaml for CI support

 .azurepipelines/ReadMe.md                     |   5 +
 .azurepipelines/Ubuntu-GCC5.yml               |  21 ++
 .azurepipelines/Ubuntu-PatchCheck.yml         |  55 ++++++
 .azurepipelines/Windows-VS2019.yml            |  20 ++
 .azurepipelines/templates/ReadMe.md           |   5 +
 .../templates/pr-gate-build-job.yml           |  43 ++++
 .azurepipelines/templates/pr-gate-steps.yml   | 142 +++++++++++++
 .../templates/spell-check-prereq-steps.yml    |  22 +++
 .mergify/config.yml                           |  50 +++++
 .pytool/CISettings.py                         | 186 ++++++++++++++++++
 .pytool/Readme.md                             |  30 +++
 Platform/ARM/JunoPkg/JunoPkg.ci.yaml          | 104 ++++++++++
 edk2_ext_dep.yaml                             |  15 ++
 pip-requirements.txt                          |  18 ++
 14 files changed, 716 insertions(+)
 create mode 100644 .azurepipelines/ReadMe.md
 create mode 100644 .azurepipelines/Ubuntu-GCC5.yml
 create mode 100644 .azurepipelines/Ubuntu-PatchCheck.yml
 create mode 100644 .azurepipelines/Windows-VS2019.yml
 create mode 100644 .azurepipelines/templates/ReadMe.md
 create mode 100644 .azurepipelines/templates/pr-gate-build-job.yml
 create mode 100644 .azurepipelines/templates/pr-gate-steps.yml
 create mode 100644 .azurepipelines/templates/spell-check-prereq-steps.yml
 create mode 100644 .mergify/config.yml
 create mode 100644 .pytool/CISettings.py
 create mode 100644 .pytool/Readme.md
 create mode 100644 Platform/ARM/JunoPkg/JunoPkg.ci.yaml
 create mode 100644 edk2_ext_dep.yaml
 create mode 100644 pip-requirements.txt

-- 
2.17.1

[PATCH edk2-platforms v6 1/6] pip-requirements.txt: Add python pip requirements file

[PierreGondois]

From: Pierre Gondois <Pierre.Gondois@arm.com>

To enable CI support of the tianocore/edk2-platforms repository,
add pip requirements file to install the python modules
required to perform EDK II Continuous Integration (CI) builds.

This file is a copy of the file from the tianocore/edk2
repository. Any modification to the tianocore/edk2 file must be
reflected on the tianocore/edk2-platforms copy.

Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3509

Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Leif Lindholm <leif@nuviainc.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Sami Mujawar <sami.mujawar@arm.com>
Co-authored-by: Sami Mujawar <sami.mujawar@arm.com>
Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
---
 pip-requirements.txt | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)
 create mode 100644 pip-requirements.txt

diff --git a/pip-requirements.txt b/pip-requirements.txt
new file mode 100644
index 000000000000..aea2e6ece431
--- /dev/null
+++ b/pip-requirements.txt
@@ -0,0 +1,18 @@
+## @file
+# EDK II Python PIP requirements file
+#
+# This file provides the list of python components to install using PIP.
+#
+# Copyright (c) Microsoft Corporation.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+#
+# https://pypi.org/project/pip/
+# https://pip.pypa.io/en/stable/user_guide/#requirements-files
+# https://pip.pypa.io/en/stable/reference/pip_install/#requirements-file-format
+# https://www.python.org/dev/peps/pep-0440/#version-specifiers
+##
+
+edk2-pytool-library==0.10.*
+edk2-pytool-extensions~=0.13.3
+edk2-basetools==0.1.2
+antlr4-python3-runtime==4.7.1
-- 
2.17.1

[PATCH edk2-platforms v6 2/6] CI: Add Tianocore edk2 as an external dependency

[PierreGondois]

From: Pierre Gondois <Pierre.Gondois@arm.com>

CI plugins (.pytool/Plugin/) and BaseTools are located in the
Tianocore edk2 repository. To re-use them, add the edk2 repository
as an external dependency.
Once the pip-requirements installed, the stuart_* set of commands
will download the required dependencies to locally run the CI. The
edk2 repository and its dependencies will be downloaded.

It is not currently possible to specify a branch name in the "version"
field. Thus, a fixed commit hash (current edk2 master) is written.
It will eventually be replaced by 'master'.

Ref:https://bugzilla.tianocore.org/show_bug.cgi?id=3509

Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Leif Lindholm <leif@nuviainc.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Sami Mujawar <sami.mujawar@arm.com>
Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
---
 edk2_ext_dep.yaml | 15 +++++++++++++++
 1 file changed, 15 insertions(+)
 create mode 100644 edk2_ext_dep.yaml

diff --git a/edk2_ext_dep.yaml b/edk2_ext_dep.yaml
new file mode 100644
index 000000000000..8a3d681bcc9e
--- /dev/null
+++ b/edk2_ext_dep.yaml
@@ -0,0 +1,15 @@
+## @file
+# Download Tianocore edk2
+#
+# Copyright (c) 2021, Arm Limited. All rights reserved.<BR>
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+    "scope": "edk2-repo",
+    "type": "git",
+    "name": "edk2",
+    "source": "https://github.com/tianocore/edk2",
+    "version": "e1e7306b54147e65cb7347b060e94f336d4a82d2",
+    "internal_path": "/edk2/",
+    "flags": []
+}
\ No newline at end of file
-- 
2.17.1

[PATCH edk2-platforms v6 3/6] .pytool/Plugin: Add CI plugins

[PierreGondois]

From: Pierre Gondois <Pierre.Gondois@arm.com>

To enable CI support of the tianocore/edk2-platforms repository,
add a .pytool directory containing the following files:
 - .pytool/CISettings.py
 - .pytool/Readme.md

These files are largely inspired from the same files available in
the edk2 repository. The .pytool/Plugin/* files containing the
CI tests to run are not copied. edk2-platforms will rely on the
edk2basetools python package and on the edk2 python files, as
edk2 is imported as a submodule of edk2-platforms.

Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3509

Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Leif Lindholm <leif@nuviainc.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Sami Mujawar <sami.mujawar@arm.com>
Co-authored-by: Sami Mujawar <sami.mujawar@arm.com>
Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
---
 .pytool/CISettings.py | 186 ++++++++++++++++++++++++++++++++++++++++++
 .pytool/Readme.md     |  30 +++++++
 2 files changed, 216 insertions(+)
 create mode 100644 .pytool/CISettings.py
 create mode 100644 .pytool/Readme.md

diff --git a/.pytool/CISettings.py b/.pytool/CISettings.py
new file mode 100644
index 000000000000..6435c60a4bc5
--- /dev/null
+++ b/.pytool/CISettings.py
@@ -0,0 +1,186 @@
+# @file
+#
+# Copyright (c) Microsoft Corporation.
+# Copyright (c) 2020, Hewlett Packard Enterprise Development LP. All rights reserved.<BR>
+# Copyright (c) 2020 - 2021, ARM Limited. All rights reserved.<BR>
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+import os
+import logging
+import edk2basetools
+
+from edk2toolext.environment import shell_environment
+from edk2toolext.invocables.edk2_ci_build import CiBuildSettingsManager
+from edk2toolext.invocables.edk2_setup import SetupSettingsManager, RequiredSubmodule
+from edk2toolext.invocables.edk2_update import UpdateSettingsManager
+from edk2toolext.invocables.edk2_pr_eval import PrEvalSettingsManager
+from edk2toollib.utility_functions import GetHostInfo
+
+
+class Settings(CiBuildSettingsManager, UpdateSettingsManager, SetupSettingsManager, PrEvalSettingsManager):
+
+    def __init__(self):
+        self.ActualPackages = []
+        self.ActualTargets = []
+        self.ActualArchitectures = []
+        self.ActualToolChainTag = ""
+        self.ActualScopes = None
+
+    # ####################################################################################### #
+    #                             Extra CmdLine configuration                                 #
+    # ####################################################################################### #
+
+    def AddCommandLineOptions(self, parserObj):
+        pass
+    def RetrieveCommandLineOptions(self, args):
+        pass
+
+    # ####################################################################################### #
+    #                        Default Support for this Ci Build                                #
+    # ####################################################################################### #
+
+    def GetPackagesSupported(self):
+        ''' return iterable of edk2-platforms packages supported by this build.
+        These should be edk2-platforms workspace relative paths '''
+        return (
+                "JunoPkg",
+                "VExpressPkg"
+                )
+
+    def GetArchitecturesSupported(self):
+        ''' return iterable of edk2-platforms architectures supported by this build '''
+        return (
+                "IA32",
+                "X64",
+                "ARM",
+                "AARCH64",
+                "RISCV64")
+
+    def GetTargetsSupported(self):
+        ''' return iterable of edk2-platforms target tags supported by this build '''
+        return ("DEBUG", "RELEASE", "NO-TARGET", "NOOPT")
+
+    # ####################################################################################### #
+    #                     Verify and Save requested Ci Build Config                           #
+    # ####################################################################################### #
+
+    def SetPackages(self, list_of_requested_packages):
+        ''' Confirm the requested package list is valid and configure SettingsManager
+        to build the requested packages.
+
+        Raise UnsupportedException if a requested_package is not supported
+        '''
+        unsupported = set(list_of_requested_packages) - \
+            set(self.GetPackagesSupported())
+        if(len(unsupported) > 0):
+            logging.critical(
+                "Unsupported Package Requested: " + " ".join(unsupported))
+            raise Exception("Unsupported Package Requested: " +
+                            " ".join(unsupported))
+        self.ActualPackages = list_of_requested_packages
+
+    def SetArchitectures(self, list_of_requested_architectures):
+        ''' Confirm the requests architecture list is valid and configure SettingsManager
+        to run only the requested architectures.
+
+        Raise Exception if a list_of_requested_architectures is not supported
+        '''
+        unsupported = set(list_of_requested_architectures) - \
+            set(self.GetArchitecturesSupported())
+        if(len(unsupported) > 0):
+            logging.critical(
+                "Unsupported Architecture Requested: " + " ".join(unsupported))
+            raise Exception(
+                "Unsupported Architecture Requested: " + " ".join(unsupported))
+        self.ActualArchitectures = list_of_requested_architectures
+
+    def SetTargets(self, list_of_requested_target):
+        ''' Confirm the request target list is valid and configure SettingsManager
+        to run only the requested targets.
+
+        Raise UnsupportedException if a requested_target is not supported
+        '''
+        unsupported = set(list_of_requested_target) - \
+            set(self.GetTargetsSupported())
+        if(len(unsupported) > 0):
+            logging.critical(
+                "Unsupported Targets Requested: " + " ".join(unsupported))
+            raise Exception("Unsupported Targets Requested: " +
+                            " ".join(unsupported))
+        self.ActualTargets = list_of_requested_target
+
+    # ####################################################################################### #
+    #                         Actual Configuration for Ci Build                               #
+    # ####################################################################################### #
+
+    def GetActiveScopes(self):
+        ''' return tuple containing scopes that should be active for this process '''
+        if self.ActualScopes is None:
+            scopes = ("cibuild", "edk2-build", "host-based-test")
+
+            self.ActualToolChainTag = shell_environment.GetBuildVars().GetValue("TOOL_CHAIN_TAG", "")
+
+            is_linux = GetHostInfo().os.upper() == "LINUX"
+            scopes += ('pipbuild-unix',) if is_linux else ('pipbuild-win',)
+
+            if is_linux and self.ActualToolChainTag.upper().startswith("GCC"):
+                if "AARCH64" in self.ActualArchitectures:
+                    scopes += ("gcc_aarch64_linux",)
+                if "ARM" in self.ActualArchitectures:
+                    scopes += ("gcc_arm_linux",)
+                if "RISCV64" in self.ActualArchitectures:
+                    scopes += ("gcc_riscv64_unknown",)
+
+            # If EDK2_REPO is not provided, download it.
+            if not shell_environment.GetBuildVars().GetValue("EDK2_REPO", ""):
+                    scopes += ("edk2-repo",)
+
+            self.ActualScopes = scopes
+        return self.ActualScopes
+
+    def GetRequiredSubmodules(self):
+        ''' return iterable containing RequiredSubmodule objects.
+        If no RequiredSubmodules return an empty iterable
+        '''
+        rs = []
+        rs.append(RequiredSubmodule(
+            "Silicon/RISC-V/ProcessorPkg/Library/RiscVOpensbiLib/opensbi", False))
+        return rs
+
+    def GetName(self):
+        return "Edk2-platforms"
+
+    def GetDependencies(self):
+        return [
+        ]
+
+    def GetPackagesPath(self):
+        ''' Return a list of workspace relative paths that should be mapped as edk2-platforms PackagesPath '''
+        packages = []
+
+        edk2_repo = shell_environment.GetBuildVars().GetValue("EDK2_REPO", "")
+        if not edk2_repo:
+            edk2_repo = os.path.join("edk2_extdep", "edk2")
+        packages.append(edk2_repo)
+
+        packages.append(os.path.join("Platform", "ARM"))
+        return packages
+
+    def GetWorkspaceRoot(self):
+        ''' get WorkspacePath '''
+        return os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+
+    def FilterPackagesToTest(self, changedFilesList: list, potentialPackagesList: list) -> list:
+        ''' Filter potential packages to test based on changed files. '''
+        build_these_packages = []
+        possible_packages = potentialPackagesList.copy()
+        for f in changedFilesList:
+            # split each part of path for comparison later
+            nodes = f.split("/")
+
+            # python file change in .pytool folder causes building all
+            if f.endswith(".py") and ".pytool" in nodes:
+                build_these_packages = possible_packages
+                break
+
+        return build_these_packages
diff --git a/.pytool/Readme.md b/.pytool/Readme.md
new file mode 100644
index 000000000000..b084ff642b2f
--- /dev/null
+++ b/.pytool/Readme.md
@@ -0,0 +1,30 @@
+# Edk2-platforms Continuous Integration
+
+## Basic Status
+
+| Package                      | Windows VS2019 (IA32/X64)| Ubuntu GCC (IA32/X64/ARM/AARCH64) | Known Issues |
+| :----                        | :-----                   | :----                             | :---         |
+| Platfrom/ARM/JunoPkg         |                          | :heavy_check_mark:                | Spell checking in audit mode. CompilerCheck disabled (need a PlatformCI).
+
+For more detailed status look at the test results of the latest CI run on the
+repo readme.
+
+## edk2 dependency
+
+It is possible that the edk2-platforms repository relies on new modifications
+in the edk2 repository. The edk2-platforms CI fetches the edk2 repository
+via the edk2_ext_dep.yaml. The edk2 repository is hence treated as an external
+dependency.
+To use a custom edk2 repository:
+- Place it inside the edk2-platforms folder
+- Run the stuart_[update|build] commands with EDK2_REPO pointing to your
+  custom edk2 repository.
+E.g.:
+stuart_update -c .pytool/CISettings.py EDK2_REPO=./my_edk2_repo`
+stuart_ci_build -c .pytool/CISettings.py EDK2_REPO=./my_edk2_repo`
+
+## Readme
+
+As the content of the .pytool folder has been imported from the tianocore repository at:
+https://github.com/tianocore/edk2
+Please use the Readme.md that can be found there.
-- 
2.17.1

[PATCH edk2-platforms v6 4/6] .azurepipelines: Add Azure Pipelines YML configuration files

[PierreGondois]

From: Pierre Gondois <Pierre.Gondois@arm.com>

To enable CI support of the tianocore/edk2-platforms repository,
add YML configuration files used to run Continuous Integration (CI)
checks on Azure Pipelines agents.

These files are copies of the files from the tianocore/edk2
repository. Any modification to the tianocore/edk2 files must be
reflected on the tianocore/edk2-platforms copies.

The following files have been modified:
 - .azurepipelines/templates/platform-build-run-steps.yml
 - .azurepipelines/templates/pr-gate-build-job.yml
 - .azurepipelines/templates/pr-gate-steps.yml

The sections modified are marked with the following comments:
 -EDK2_PLATFORMS_MODIF_START
 -EDK2_PLATFORMS_MODIF_END

Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3509

Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Leif Lindholm <leif@nuviainc.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Sami Mujawar <sami.mujawar@arm.com>
Co-authored-by: Sami Mujawar <sami.mujawar@arm.com>
Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
---
 .azurepipelines/ReadMe.md                     |   5 +
 .azurepipelines/Ubuntu-GCC5.yml               |  21 +++
 .azurepipelines/Ubuntu-PatchCheck.yml         |  55 +++++++
 .azurepipelines/Windows-VS2019.yml            |  20 +++
 .azurepipelines/templates/ReadMe.md           |   5 +
 .../templates/pr-gate-build-job.yml           |  43 ++++++
 .azurepipelines/templates/pr-gate-steps.yml   | 142 ++++++++++++++++++
 .../templates/spell-check-prereq-steps.yml    |  22 +++
 8 files changed, 313 insertions(+)
 create mode 100644 .azurepipelines/ReadMe.md
 create mode 100644 .azurepipelines/Ubuntu-GCC5.yml
 create mode 100644 .azurepipelines/Ubuntu-PatchCheck.yml
 create mode 100644 .azurepipelines/Windows-VS2019.yml
 create mode 100644 .azurepipelines/templates/ReadMe.md
 create mode 100644 .azurepipelines/templates/pr-gate-build-job.yml
 create mode 100644 .azurepipelines/templates/pr-gate-steps.yml
 create mode 100644 .azurepipelines/templates/spell-check-prereq-steps.yml

diff --git a/.azurepipelines/ReadMe.md b/.azurepipelines/ReadMe.md
new file mode 100644
index 000000000000..de69a4ca220d
--- /dev/null
+++ b/.azurepipelines/ReadMe.md
@@ -0,0 +1,5 @@
+EDK2_PLATFORMS_MODIF_START
+As the content of this folder has been imported from the tianocore repository at:
+https://github.com/tianocore/edk2
+Please use the Readme.md that can be found there.
+EDK2_PLATFORMS_MODIF_END
diff --git a/.azurepipelines/Ubuntu-GCC5.yml b/.azurepipelines/Ubuntu-GCC5.yml
new file mode 100644
index 000000000000..3760c6efe105
--- /dev/null
+++ b/.azurepipelines/Ubuntu-GCC5.yml
@@ -0,0 +1,21 @@
+## @file
+# Azure Pipeline build file for a build using ubuntu and GCC5
+#
+# Copyright (c) Microsoft Corporation.
+# Copyright (c) 2020, Hewlett Packard Enterprise Development LP. All rights reserved.<BR>
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+trigger:
+- master
+- stable/*
+pr:
+- master
+- stable/*
+
+jobs:
+- template: templates/pr-gate-build-job.yml
+  parameters:
+    tool_chain_tag: 'GCC5'
+    vm_image: 'ubuntu-latest'
+    arch_list: "IA32,X64,ARM,AARCH64,RISCV64"
+
diff --git a/.azurepipelines/Ubuntu-PatchCheck.yml b/.azurepipelines/Ubuntu-PatchCheck.yml
new file mode 100644
index 000000000000..f15c224dcbca
--- /dev/null
+++ b/.azurepipelines/Ubuntu-PatchCheck.yml
@@ -0,0 +1,55 @@
+## @file
+# Azure Pipielines YML file that evalues the patch series in a PR using the
+# python script BaseTools/Scripts/PatchCheck.py.
+#
+# NOTE: This example monitors pull requests against the edk2-ci branch.  Most
+# environments would replace 'edk2-ci' with 'master'.
+#
+# Copyright (c) 2021, Arm Limited. All rights reserved.<BR>
+# Copyright (c) 2019 - 2020, Intel Corporation. All rights reserved.<BR>
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+#
+# https://github.com/tianocore
+#
+##
+
+trigger: none
+
+pr:
+- master
+- stable/*
+
+pool:
+  vmImage: 'ubuntu-latest'
+
+steps:
+- checkout: self
+  clean: true
+
+- task: UsePythonVersion@0
+  inputs:
+    versionSpec: '3.7.x'
+    architecture: 'x64'
+
+# EDK2_PLATFORMS_MODIF_START:
+# Get dependencies. PatchCheck.py is in edk2, which is downloaded by stuart_update.
+- script: pip install -r pip-requirements.txt --upgrade
+  displayName: 'Install/Upgrade pip modules'
+
+- task: CmdLine@1
+  displayName: Setup
+  inputs:
+    filename: stuart_setup
+    arguments: -c .pytool/CISettings.py
+
+- task: CmdLine@1
+  displayName: Update
+  inputs:
+    filename: stuart_update
+    arguments: -c .pytool/CISettings.py
+
+- script: |
+    git fetch origin $(System.PullRequest.TargetBranch):$(System.PullRequest.TargetBranch)
+    python edk2_extdep/edk2/BaseTools/Scripts/PatchCheck.py $(System.PullRequest.TargetBranch)..$(System.PullRequest.SourceCommitId)
+  displayName: 'Use PatchCheck.py to verify patch series in pull request'
+# EDK2_PLATFORMS_MODIF_END
diff --git a/.azurepipelines/Windows-VS2019.yml b/.azurepipelines/Windows-VS2019.yml
new file mode 100644
index 000000000000..22f2d88c2c6a
--- /dev/null
+++ b/.azurepipelines/Windows-VS2019.yml
@@ -0,0 +1,20 @@
+## @file
+# Azure Pipeline build file for a build using Windows and VS2019
+#
+# Copyright (c) Microsoft Corporation.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+trigger:
+- master
+- stable/*
+
+pr:
+- master
+- stable/*
+
+jobs:
+- template: templates/pr-gate-build-job.yml
+  parameters:
+    tool_chain_tag: 'VS2019'
+    vm_image: 'windows-latest'
+    arch_list: "IA32,X64"
diff --git a/.azurepipelines/templates/ReadMe.md b/.azurepipelines/templates/ReadMe.md
new file mode 100644
index 000000000000..de69a4ca220d
--- /dev/null
+++ b/.azurepipelines/templates/ReadMe.md
@@ -0,0 +1,5 @@
+EDK2_PLATFORMS_MODIF_START
+As the content of this folder has been imported from the tianocore repository at:
+https://github.com/tianocore/edk2
+Please use the Readme.md that can be found there.
+EDK2_PLATFORMS_MODIF_END
diff --git a/.azurepipelines/templates/pr-gate-build-job.yml b/.azurepipelines/templates/pr-gate-build-job.yml
new file mode 100644
index 000000000000..9bb8e2819793
--- /dev/null
+++ b/.azurepipelines/templates/pr-gate-build-job.yml
@@ -0,0 +1,43 @@
+## @file
+# File templates/pr-gate-build-job.yml
+#
+# template file used to build supported packages.
+#
+# Copyright (c) Microsoft Corporation.
+# Copyright (c) 2020 - 2021, ARM Limited. All rights reserved.<BR>
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+
+parameters:
+  tool_chain_tag: ''
+  vm_image: ''
+  arch_list: ''
+
+# Build step
+jobs:
+
+- job: Build_${{ parameters.tool_chain_tag }}
+
+# EDK2_PLATFORMS_MODIF_START:
+# Build edk2-platforms packages
+  #Use matrix to speed up the build process
+  strategy:
+    matrix:
+      TARGET_ARM_PLATFORMS:
+        Build.Pkgs: 'JunoPkg'
+        Build.Targets: 'DEBUG,RELEASE,NO-TARGET,NOOPT'
+# EDK2_PLATFORMS_MODIF_END
+
+  workspace:
+    clean: all
+
+  pool:
+    vmImage: ${{ parameters.vm_image }}
+
+  steps:
+  - template: pr-gate-steps.yml
+    parameters:
+      tool_chain_tag: ${{ parameters.tool_chain_tag }}
+      build_pkgs: $(Build.Pkgs)
+      build_targets: $(Build.Targets)
+      build_archs: ${{ parameters.arch_list }}
diff --git a/.azurepipelines/templates/pr-gate-steps.yml b/.azurepipelines/templates/pr-gate-steps.yml
new file mode 100644
index 000000000000..0a8a0e1e09a3
--- /dev/null
+++ b/.azurepipelines/templates/pr-gate-steps.yml
@@ -0,0 +1,142 @@
+## @file
+# File templates/pr-gate-steps.yml
+#
+# template file containing the steps to build
+#
+# Copyright (c) 2021, Arm Limited. All rights reserved.<BR>
+# Copyright (c) Microsoft Corporation.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+
+parameters:
+  tool_chain_tag: ''
+  build_pkgs: ''
+  build_targets: ''
+  build_archs: ''
+
+steps:
+- checkout: self
+  clean: true
+  fetchDepth: 1
+
+- task: UsePythonVersion@0
+  inputs:
+    versionSpec: '3.8.x'
+    architecture: 'x64'
+
+- script: pip install -r pip-requirements.txt --upgrade
+  displayName: 'Install/Upgrade pip modules'
+
+# Set default
+- bash: |
+    echo "##vso[task.setvariable variable=pkgs_to_build]${{ parameters.build_pkgs }}"
+    echo "##vso[task.setvariable variable=pkg_count]${{ 1 }}"
+
+# Fetch the target branch so that pr_eval can diff them.
+# Seems like azure pipelines/github changed checkout process in nov 2020.
+- script: git fetch origin $(System.PullRequest.targetBranch)
+  displayName: fetch target branch
+  condition: eq(variables['Build.Reason'], 'PullRequest')
+
+# trim the package list if this is a PR
+- task: CmdLine@1
+  displayName: Check if ${{ parameters.build_pkgs }} need testing
+  inputs:
+    filename: stuart_pr_eval
+    arguments: -c .pytool/CISettings.py -p ${{ parameters.build_pkgs }} --pr-target origin/$(System.PullRequest.targetBranch) --output-csv-format-string "##vso[task.setvariable variable=pkgs_to_build;isOutpout=true]{pkgcsv}" --output-count-format-string "##vso[task.setvariable variable=pkg_count;isOutpout=true]{pkgcount}"
+  condition: eq(variables['Build.Reason'], 'PullRequest')
+
+# install spell check prereqs
+- template: spell-check-prereq-steps.yml
+
+# Build repo
+- task: CmdLine@1
+  displayName: Setup ${{ parameters.build_pkgs }} ${{ parameters.build_archs}}
+  inputs:
+    filename: stuart_setup
+    arguments: -c .pytool/CISettings.py -p $(pkgs_to_build) -t ${{ parameters.build_targets}} -a ${{ parameters.build_archs}} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}}
+  condition: and(gt(variables.pkg_count, 0), succeeded())
+
+- task: CmdLine@1
+  displayName: Update ${{ parameters.build_pkgs }} ${{ parameters.build_archs}}
+  inputs:
+    filename: stuart_update
+    arguments: -c .pytool/CISettings.py -p $(pkgs_to_build) -t ${{ parameters.build_targets}} -a ${{ parameters.build_archs}} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}}
+  condition: and(gt(variables.pkg_count, 0), succeeded())
+
+# build basetools
+#   do this after setup and update so that code base dependencies
+#   are all resolved.
+# EDK2_PLATFORMS_MODIF_START:
+# The base tools are imported in .pytool/CISettings.py via the 'edk2basetools' python module.
+# - template: basetools-build-steps.yml
+#   parameters:
+#     tool_chain_tag: ${{ parameters.tool_chain_tag }}
+# EDK2_PLATFORMS_MODIF_END
+
+- task: CmdLine@1
+  displayName: Build and Test ${{ parameters.build_pkgs }} ${{ parameters.build_archs}}
+  inputs:
+    filename: stuart_ci_build
+    arguments: -c .pytool/CISettings.py -p $(pkgs_to_build) -t ${{ parameters.build_targets}} -a ${{ parameters.build_archs}} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}}
+  condition: and(gt(variables.pkg_count, 0), succeeded())
+
+# Publish Test Results to Azure Pipelines/TFS
+- task: PublishTestResults@2
+  displayName: 'Publish junit test results'
+  continueOnError: true
+  condition: and( succeededOrFailed(),gt(variables.pkg_count, 0))
+  inputs:
+    testResultsFormat: 'JUnit' # Options: JUnit, NUnit, VSTest, xUnit
+    testResultsFiles: 'Build/TestSuites.xml'
+    #searchFolder: '$(System.DefaultWorkingDirectory)' # Optional
+    mergeTestResults: true # Optional
+    testRunTitle: $(System.JobName) # Optional
+    #buildPlatform: # Optional
+    #buildConfiguration: # Optional
+    publishRunAttachments: true # Optional
+
+# Publish Test Results to Azure Pipelines/TFS
+- task: PublishTestResults@2
+  displayName: 'Publish host based test results for $(System.JobName)'
+  continueOnError: true
+  condition: and( succeededOrFailed(), gt(variables.pkg_count, 0))
+  inputs:
+    testResultsFormat: 'JUnit' # Options: JUnit, NUnit, VSTest, xUnit
+    testResultsFiles: 'Build/**/*.result.xml'
+    #searchFolder: '$(System.DefaultWorkingDirectory)' # Optional
+    mergeTestResults: false # Optional
+    testRunTitle: ${{ parameters.build_pkgs }} # Optional
+    #buildPlatform: # Optional
+    #buildConfiguration: # Optional
+    publishRunAttachments: true # Optional
+
+# Copy the build logs to the artifact staging directory
+- task: CopyFiles@2
+  displayName: "Copy build logs"
+  inputs:
+    targetFolder: '$(Build.ArtifactStagingDirectory)'
+    SourceFolder: 'Build'
+    contents: |
+      BUILDLOG_*.txt
+      BUILDLOG_*.md
+      CI_*.txt
+      CI_*.md
+      CISETUP.txt
+      SETUPLOG.txt
+      UPDATE_LOG.txt
+      PREVALLOG.txt
+      TestSuites.xml
+      **/BUILD_TOOLS_REPORT.html
+      **/OVERRIDELOG.TXT
+    flattenFolders: true
+  condition: succeededOrFailed()
+
+# Publish build artifacts to Azure Artifacts/TFS or a file share
+- task: PublishBuildArtifacts@1
+  continueOnError: true
+  displayName: "Publish build logs"
+  inputs:
+    pathtoPublish: '$(Build.ArtifactStagingDirectory)'
+    artifactName: 'Build Logs $(System.JobName)'
+  condition: succeededOrFailed()
diff --git a/.azurepipelines/templates/spell-check-prereq-steps.yml b/.azurepipelines/templates/spell-check-prereq-steps.yml
new file mode 100644
index 000000000000..98ee3cfa6bc6
--- /dev/null
+++ b/.azurepipelines/templates/spell-check-prereq-steps.yml
@@ -0,0 +1,22 @@
+## @file
+# File templates/spell-check-prereq-steps.yml
+#
+# template file used to install spell checking prerequisits
+#
+# Copyright (c) Microsoft Corporation.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+
+parameters:
+  none: ''
+
+steps:
+- task: NodeTool@0
+  inputs:
+    versionSpec: '14.x'
+    #checkLatest: false # Optional
+  condition: and(gt(variables.pkg_count, 0), succeeded())
+
+- script: npm install -g cspell
+  displayName: 'Install cspell npm'
+  condition: and(gt(variables.pkg_count, 0), succeeded())
-- 
2.17.1

[PATCH edk2-platforms v6 5/6] .mergify: Add Mergify YML pull request rules configuration file

[PierreGondois]

From: Pierre Gondois <Pierre.Gondois@arm.com>

These files are copies of the files from the tianocore/edk2
repository. Any modification to the tianocore/edk2 files must be
reflected on the tianocore/edk2-platforms copies.

Initial commid-id in the edk2 repository: ab060128768b
Initial message:

Add directory for the Mergify YML configuration files that
provides rules and actions used to process a pull request.

* Auto commit a PR from EDK II Maintainer with 'push' label
  set and all CI checks pass
* Auto close a PR from any developers without 'push' label
  set and all CI checks pass.
* Auto close a PR from a non EDK II Maintainer that has
  the 'push' label set.
* Post a comment to a PR that has a merge conflict.
  Submitter can resolved conflicts and reopen the PR.
* Post a comment to a PR that fails PatchCheck.py
  Submitter can resolve PatchCheck.py issues and
  reopen the PR.

Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3509

Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Leif Lindholm <leif@nuviainc.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Sami Mujawar <sami.mujawar@arm.com>
Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
---
 .mergify/config.yml | 50 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 50 insertions(+)
 create mode 100644 .mergify/config.yml

diff --git a/.mergify/config.yml b/.mergify/config.yml
new file mode 100644
index 000000000000..bd6da4c77937
--- /dev/null
+++ b/.mergify/config.yml
@@ -0,0 +1,50 @@
+## @file
+# Mergify YML file that automatically merges a GitHub pull request against
+# edk2-ci if all of the GitHub branch protections have passed.  It also
+# contains rules to:
+# * auto close branches that are not from an EDK II Maintainer
+# * post a comment on pull requests that have merge conflicts.
+# * post a comment on pull requests that have PatchCheck.py errors.
+#
+# Configuration Notes:
+# * Update the 'base=edk2-ci' statements with the name of the branch to merge
+#   pull requests.
+#
+# * Update the 'status-failure' statement with the name of the name of the Azure
+#   Pipelines Build that performs the EDK II Maintainer check.
+#
+# * This file must be checked into the 'default' branch of a repo.  Copies
+#   of this file on other branches of a repo are ignored by Mergify.
+#
+# Copyright (c) 2019 - 2021, Intel Corporation. All rights reserved.<BR>
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+#
+# https://github.com/apps/mergify
+# https://doc.mergify.io/
+#
+##
+
+queue_rules:
+  - name: default
+    conditions:
+      - base~=(^main|^master|^stable/)
+      - label=push
+
+pull_request_rules:
+  - name: Automatically merge a PR when all required checks pass and 'push' label is present
+    conditions:
+      - base~=(^main|^master|^stable/)
+      - label=push
+    actions:
+      queue:
+        method: rebase
+        rebase_fallback: none
+        name: default
+
+  - name: Post a comment on a PR that can not be merged due to a merge conflict
+    conditions:
+      - base~=(^main|^master|^stable/)
+      - conflict
+    actions:
+      comment:
+        message: PR can not be merged due to conflict.  Please rebase and resubmit
-- 
2.17.1

[PATCH edk2-platforms v6 6/6] Platform/ARM: Juno: Add JunoPkg.ci.yaml for CI support

[PierreGondois]

From: Pierre Gondois <Pierre.Gondois@arm.com>

Add a JunoPkg.ci.yaml file to enable the CI for the JunoPkg.

Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Leif Lindholm <leif@nuviainc.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Sami Mujawar <sami.mujawar@arm.com>
Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
---
 Platform/ARM/JunoPkg/JunoPkg.ci.yaml | 104 +++++++++++++++++++++++++++
 1 file changed, 104 insertions(+)
 create mode 100644 Platform/ARM/JunoPkg/JunoPkg.ci.yaml

diff --git a/Platform/ARM/JunoPkg/JunoPkg.ci.yaml b/Platform/ARM/JunoPkg/JunoPkg.ci.yaml
new file mode 100644
index 000000000000..7e7f201b40ec
--- /dev/null
+++ b/Platform/ARM/JunoPkg/JunoPkg.ci.yaml
@@ -0,0 +1,104 @@
+## @file
+# Core CI configuration for JunoPkg
+#
+# VExpressPkg is part of Platform CI for builds so this is only
+# used for code analysis.
+#
+# Copyright (c) Microsoft Corporation
+# Copyright (c) 2021, Arm Ltd. All rights reserved.<BR>
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+    ## options defined .pytool/Plugin/LicenseCheck
+    "LicenseCheck": {
+        "IgnoreFiles": []
+    },
+    "EccCheck": {
+        ## Exception sample looks like below:
+        ## "ExceptionList": [
+        ##     "<ErrorID>", "<KeyWord>"
+        ## ]
+        "ExceptionList": [
+        ],
+        ## Both file path and directory path are accepted.
+        "IgnoreFiles": [
+        ]
+    },
+    ## options defined .pytool/Plugin/CompilerPlugin
+    "CompilerPlugin": {
+        "DscPath": "" # Don't support this test
+                      # Build the Package using a PlatformCI, similarly to ArmVirtPkg.
+    },
+
+    ## options defined .pytool/Plugin/HostUnitTestCompilerPlugin
+    "HostUnitTestCompilerPlugin": {
+        "DscPath": "" # Don't support this test
+    },
+
+    ## options defined .pytool/Plugin/CharEncodingCheck
+    "CharEncodingCheck": {
+        "IgnoreFiles": []
+    },
+
+    ## options defined .pytool/Plugin/DependencyCheck
+    "DependencyCheck": {
+        "AcceptableDependencies": [
+            "ArmPkg/ArmPkg.dec",
+            "ArmPlatformPkg/ArmPlatformPkg.dec",
+            "DynamicTablesPkg/DynamicTablesPkg.dec",
+            "EmbeddedPkg/EmbeddedPkg.dec",
+            "MdePkg/MdePkg.dec",
+            "MdeModulePkg/MdeModulePkg.dec",
+            "Platform/ARM/ARM.dec",
+            "Platform/ARM/Drivers/FdtPlatformDxe/FdtPlatformDxe.dec",
+            "Platform/ARM/JunoPkg/ArmJuno.dec",
+        ],
+        # For host based unit tests
+        "AcceptableDependencies-HOST_APPLICATION":[
+            "UnitTestFrameworkPkg/UnitTestFrameworkPkg.dec"
+        ],
+        # For UEFI shell based apps
+        "AcceptableDependencies-UEFI_APPLICATION":[
+
+        ],
+        "IgnoreInf": []
+    },
+
+    ## options defined .pytool/Plugin/DscCompleteCheck
+    "DscCompleteCheck": {
+        "IgnoreInf": [""],
+        "DscPath": ""  # Don't support this test
+        # "DscPath": "ArmJuno.dsc"  # Don't support this test
+    },
+
+    ## options defined .pytool/Plugin/HostUnitTestDscCompleteCheck
+    "HostUnitTestDscCompleteCheck": {
+        "IgnoreInf": [""],
+        "DscPath": "" # Don't support this test
+    },
+
+    ## options defined .pytool/Plugin/GuidCheck
+    "GuidCheck": {
+        "IgnoreGuidName": [], # Expected duplication for gEfiFirmwareVolumeTopFileGuid
+        "IgnoreGuidValue": [
+        ],
+        "IgnoreFoldersAndFiles": [],
+        "IgnoreDuplicates": [],
+    },
+
+    ## options defined .pytool/Plugin/LibraryClassCheck
+    "LibraryClassCheck": {
+        "IgnoreHeaderFile": []
+    },
+
+    ## options defined .pytool/Plugin/SpellCheck
+    "SpellCheck": {
+        "AuditOnly": True,
+        "IgnoreFiles": [],           # use gitignore syntax to ignore errors in matching files
+        "ExtendWords": [
+
+        ],           # words to extend to the dictionary for this package
+        "IgnoreStandardPaths": [],   # Standard Plugin defined paths that should be ignore
+        "AdditionalIncludePaths": [] # Additional paths to spell check (wildcards supported)
+    }
+}
-- 
2.17.1