public inbox for devel@edk2.groups.io
 help / color / mirror / Atom feed
From: "Michael D Kinney" <michael.d.kinney@intel.com>
To: devel@edk2.groups.io
Cc: Sean Brogan <sean.brogan@microsoft.com>,
	Bret Barkelew <Bret.Barkelew@microsoft.com>,
	Liming Gao <liming.gao@intel.com>
Subject: [Patch v3 20/22] .azurepipelines: Add Azure Pipelines YML configuration files
Date: Tue, 29 Oct 2019 12:55:15 -0700	[thread overview]
Message-ID: <20191029195517.20028-21-michael.d.kinney@intel.com> (raw)
In-Reply-To: <20191029195517.20028-1-michael.d.kinney@intel.com>

From: Sean Brogan <sean.brogan@microsoft.com>

https://bugzilla.tianocore.org/show_bug.cgi?id=2315

Add YML configuration files used to run the EDK II Continuous
Integration (CI) checks on Azure Pipelines agents.

Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
---
 .azurepipelines/Ubuntu-GCC5.yml               |  18 +++
 .azurepipelines/Ubuntu-PatchCheck.yml         |  35 +++++
 .azurepipelines/Windows-VS2019.yml            |  18 +++
 .../templates/basetools-build-steps.yml       |  36 +++++
 .../templates/pr-gate-build-job.yml           |  60 ++++++++
 .azurepipelines/templates/pr-gate-steps.yml   | 129 ++++++++++++++++++
 .../templates/spell-check-prereq-steps.yml    |  21 +++
 7 files changed, 317 insertions(+)
 create mode 100644 .azurepipelines/Ubuntu-GCC5.yml
 create mode 100644 .azurepipelines/Ubuntu-PatchCheck.yml
 create mode 100644 .azurepipelines/Windows-VS2019.yml
 create mode 100644 .azurepipelines/templates/basetools-build-steps.yml
 create mode 100644 .azurepipelines/templates/pr-gate-build-job.yml
 create mode 100644 .azurepipelines/templates/pr-gate-steps.yml
 create mode 100644 .azurepipelines/templates/spell-check-prereq-steps.yml

diff --git a/.azurepipelines/Ubuntu-GCC5.yml b/.azurepipelines/Ubuntu-GCC5.yml
new file mode 100644
index 0000000000..191bda8a91
--- /dev/null
+++ b/.azurepipelines/Ubuntu-GCC5.yml
@@ -0,0 +1,18 @@
+##
+# Azure Pipeline build file for a build using ubuntu and GCC5
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+trigger:
+- master
+pr:
+- master
+
+jobs:
+- template: templates/pr-gate-build-job.yml
+  parameters:
+    tool_chain_tag: 'GCC5'
+    vm_image: 'ubuntu-latest'
+    arch_list: "IA32,X64,ARM,AARCH64"
+
diff --git a/.azurepipelines/Ubuntu-PatchCheck.yml b/.azurepipelines/Ubuntu-PatchCheck.yml
new file mode 100644
index 0000000000..dff8f579b6
--- /dev/null
+++ b/.azurepipelines/Ubuntu-PatchCheck.yml
@@ -0,0 +1,35 @@
+## @file
+# Azure Pipielines YML file that evalues the patch series in a PR using the
+# python script BaseTools/Scripts/PatchCheck.py.
+#
+# NOTE: This example monitors pull requests against the edk2-ci branch.  Most
+# environments would replace 'edk2-ci' with 'master'.
+#
+# Copyright (c) 2019, Intel Corporation. All rights reserved.<BR>
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+#
+# https://github.com/tianocore
+#
+##
+
+trigger: none
+
+pr:
+- master
+
+pool:
+  vmImage: 'ubuntu-latest'
+
+steps:
+- checkout: self
+  clean: true
+
+- task: UsePythonVersion@0
+  inputs:
+    versionSpec: '3.7.x'
+    architecture: 'x64'
+
+- script: |
+    git fetch origin $(System.PullRequest.TargetBranch):$(System.PullRequest.TargetBranch)
+    python BaseTools/Scripts/PatchCheck.py $(System.PullRequest.TargetBranch)..$(System.PullRequest.SourceCommitId)
+  displayName: 'Use PatchCheck.py to verify patch series in pull request'
diff --git a/.azurepipelines/Windows-VS2019.yml b/.azurepipelines/Windows-VS2019.yml
new file mode 100644
index 0000000000..2143162e02
--- /dev/null
+++ b/.azurepipelines/Windows-VS2019.yml
@@ -0,0 +1,18 @@
+##
+# Azure Pipeline build file for a build using Windows and VS2019
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+trigger:
+- master
+
+pr:
+- master
+
+jobs:
+- template: templates/pr-gate-build-job.yml
+  parameters:
+    tool_chain_tag: 'VS2019'
+    vm_image: 'windows-latest'
+    arch_list: "IA32,X64"
diff --git a/.azurepipelines/templates/basetools-build-steps.yml b/.azurepipelines/templates/basetools-build-steps.yml
new file mode 100644
index 0000000000..dc1fdffd64
--- /dev/null
+++ b/.azurepipelines/templates/basetools-build-steps.yml
@@ -0,0 +1,36 @@
+# File templates/basetools-build-job.yml
+#
+# template file to build basetools
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+parameters:
+  tool_chain_tag: ''
+
+steps:
+- ${{ if contains(parameters.tool_chain_tag, 'GCC') }}:
+  - bash: sudo apt-get update
+    displayName: Update apt
+    condition: and(gt(variables.pkg_count, 0), succeeded())
+
+  - bash: sudo apt-get install gcc g++ make uuid-dev
+    displayName: Install required tools
+    condition: and(gt(variables.pkg_count, 0), succeeded())
+
+- task: CmdLine@1
+  displayName: Build Base Tools from source
+  inputs:
+    filename: python
+    arguments: BaseTools/Edk2ToolsBuild.py -t ${{ parameters.tool_chain_tag }}
+  condition: and(gt(variables.pkg_count, 0), succeeded())
+
+- task: CopyFiles@2
+  displayName: "Copy base tools build log"
+  inputs:
+    targetFolder: '$(Build.ArtifactStagingDirectory)'
+    SourceFolder: 'BaseTools/BaseToolsBuild'
+    contents: |
+      BASETOOLS_BUILD*.*
+    flattenFolders: true
+  condition: succeededOrFailed()
diff --git a/.azurepipelines/templates/pr-gate-build-job.yml b/.azurepipelines/templates/pr-gate-build-job.yml
new file mode 100644
index 0000000000..98e0ee7877
--- /dev/null
+++ b/.azurepipelines/templates/pr-gate-build-job.yml
@@ -0,0 +1,60 @@
+# File templates/pr-gate-build-job.yml
+#
+# template file used to build supported packages.
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+
+parameters:
+  tool_chain_tag: ''
+  vm_image: ''
+  arch_list: ''
+
+# Build step
+jobs:
+
+- job: Build_${{ parameters.tool_chain_tag }}
+
+  #Use matrix to speed up the build process
+  strategy:
+    matrix:
+      TARGET_MDE_CPU:
+        Build.Pkgs: 'MdePkg,UefiCpuPkg'
+        Build.Targets: 'DEBUG,RELEASE,NO-TARGET'
+      TARGET_MDEMODULE_DEBUG:
+        Build.Pkgs: 'MdeModulePkg'
+        Build.Targets: 'DEBUG'
+      TARGET_MDEMODULE_RELEASE:
+        Build.Pkgs: 'MdeModulePkg'
+        Build.Targets: 'RELEASE,NO-TARGET'
+      TARGET_NETWORK:
+        Build.Pkgs: 'NetworkPkg'
+        Build.Targets: 'DEBUG,RELEASE,NO-TARGET'
+      TARGET_OTHER:
+        Build.Pkgs: 'PcAtChipsetPkg,ShellPkg'
+        Build.Targets: 'DEBUG,RELEASE,NO-TARGET'
+      TARGET_FMP:
+        Build.Pkgs: 'FmpDevicePkg,FatPkg'
+        Build.Targets: 'DEBUG,RELEASE,NO-TARGET'
+      TARGET_CRYPTO:
+        Build.Pkgs: 'CryptoPkg'
+        Build.Targets: 'DEBUG,RELEASE,NO-TARGET'
+      TARGET_SECURITY:
+        Build.Pkgs: 'SecurityPkg'
+        Build.Targets: 'DEBUG,RELEASE,NO-TARGET'
+
+  workspace:
+    clean: all
+
+  pool:
+    vmImage: ${{ parameters.vm_image }}
+
+  steps:
+  - template: pr-gate-steps.yml
+    parameters:
+      tool_chain_tag: ${{ parameters.tool_chain_tag }}
+      build_pkgs: $(Build.Pkgs)
+      build_targets: $(Build.Targets)
+      build_archs: ${{ parameters.arch_list }}
+
diff --git a/.azurepipelines/templates/pr-gate-steps.yml b/.azurepipelines/templates/pr-gate-steps.yml
new file mode 100644
index 0000000000..d3c7eec4ad
--- /dev/null
+++ b/.azurepipelines/templates/pr-gate-steps.yml
@@ -0,0 +1,129 @@
+# File templates/pr-gate-steps.yml
+#
+# template file containing the steps to build
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+
+parameters:
+  tool_chain_tag: ''
+  build_pkgs: ''
+  build_targets: ''
+  build_archs: ''
+
+steps:
+- checkout: self
+  clean: true
+  fetchDepth: 1
+
+- task: UsePythonVersion@0
+  inputs:
+    versionSpec: '3.7.x'
+    architecture: 'x64'
+
+- script: pip install -r requirements.txt --upgrade
+  displayName: 'Install/Upgrade pip modules'
+
+# Set default
+- bash: |
+    echo "##vso[task.setvariable variable=pkgs_to_build]${{ parameters.build_pkgs }}"
+    echo "##vso[task.setvariable variable=pkg_count]${{ 1 }}"
+
+# trim the package list if this is a PR
+- task: CmdLine@1
+  displayName: Check if ${{ parameters.build_pkgs }} need testing
+  inputs:
+    filename: stuart_pr_eval
+    arguments: -c .pytool/CISettings.py -p ${{ parameters.build_pkgs }} --pr-target origin/$(System.PullRequest.targetBranch) --output-csv-format-string "##vso[task.setvariable variable=pkgs_to_build;isOutpout=true]{pkgcsv}" --output-count-format-string "##vso[task.setvariable variable=pkg_count;isOutpout=true]{pkgcount}"
+  condition: eq(variables['Build.Reason'], 'PullRequest')
+
+# build basetools
+- template: basetools-build-steps.yml
+  parameters:
+    tool_chain_tag: ${{ parameters.tool_chain_tag }}
+
+# install spell check prereqs
+- template: spell-check-prereq-steps.yml
+
+# Build repo
+- task: CmdLine@1
+  displayName: Setup ${{ parameters.build_pkgs }} ${{ parameters.build_archs}}
+  inputs:
+    filename: stuart_setup
+    arguments: -c .pytool/CISettings.py -p $(pkgs_to_build) -t ${{ parameters.build_targets}} -a ${{ parameters.build_archs}} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}}
+  condition: and(gt(variables.pkg_count, 0), succeeded())
+
+- task: CmdLine@1
+  displayName: Update ${{ parameters.build_pkgs }} ${{ parameters.build_archs}}
+  inputs:
+    filename: stuart_update
+    arguments: -c .pytool/CISettings.py -p $(pkgs_to_build) -t ${{ parameters.build_targets}} -a ${{ parameters.build_archs}} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}}
+  condition: and(gt(variables.pkg_count, 0), succeeded())
+
+- task: CmdLine@1
+  displayName: Build and Test ${{ parameters.build_pkgs }} ${{ parameters.build_archs}}
+  inputs:
+    filename: stuart_ci_build
+    arguments: -c .pytool/CISettings.py -p $(pkgs_to_build) -t ${{ parameters.build_targets}} -a ${{ parameters.build_archs}} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}}
+  condition: and(gt(variables.pkg_count, 0), succeeded())
+
+# Publish Test Results to Azure Pipelines/TFS
+- task: PublishTestResults@2
+  displayName: 'Publish junit test results'
+  continueOnError: true
+  condition: and( succeededOrFailed(),gt(variables.pkg_count, 0))
+  inputs:
+    testResultsFormat: 'JUnit' # Options: JUnit, NUnit, VSTest, xUnit
+    testResultsFiles: 'Build/TestSuites.xml'
+    #searchFolder: '$(System.DefaultWorkingDirectory)' # Optional
+    mergeTestResults: true # Optional
+    testRunTitle: $(System.JobName) # Optional
+    #buildPlatform: # Optional
+    #buildConfiguration: # Optional
+    publishRunAttachments: true # Optional
+
+# Publish Test Results to Azure Pipelines/TFS
+- task: PublishTestResults@2
+  displayName: 'Publish host based test results for $(System.JobName)'
+  continueOnError: true
+  condition: and( succeededOrFailed(), gt(variables.pkg_count, 0))
+  inputs:
+    testResultsFormat: 'JUnit' # Options: JUnit, NUnit, VSTest, xUnit
+    testResultsFiles: 'Build/**/*.result.xml'
+    #searchFolder: '$(System.DefaultWorkingDirectory)' # Optional
+    mergeTestResults: false # Optional
+    testRunTitle: ${{ parameters.build_pkgs }} # Optional
+    #buildPlatform: # Optional
+    #buildConfiguration: # Optional
+    publishRunAttachments: true # Optional
+
+# Copy the build logs to the artifact staging directory
+- task: CopyFiles@2
+  displayName: "Copy build logs"
+  inputs:
+    targetFolder: '$(Build.ArtifactStagingDirectory)'
+    SourceFolder: 'Build'
+    contents: |
+      BUILDLOG_*.txt
+      BUILDLOG_*.md
+      CI_*.txt
+      CI_*.md
+      CISETUP.txt
+      SETUPLOG.txt
+      UPDATE_LOG.txt
+      PREVALLOG.txt
+      TestSuites.xml
+      **/BUILD_TOOLS_REPORT.html
+      **/OVERRIDELOG.TXT
+    flattenFolders: true
+  condition: succeededOrFailed()
+
+# Publish build artifacts to Azure Artifacts/TFS or a file share
+- task: PublishBuildArtifacts@1
+  continueOnError: true
+  displayName: "Publish build logs"
+  inputs:
+    pathtoPublish: '$(Build.ArtifactStagingDirectory)'
+    artifactName: 'Build Logs $(System.JobName)'
+  condition: succeededOrFailed()
diff --git a/.azurepipelines/templates/spell-check-prereq-steps.yml b/.azurepipelines/templates/spell-check-prereq-steps.yml
new file mode 100644
index 0000000000..7d7797ffed
--- /dev/null
+++ b/.azurepipelines/templates/spell-check-prereq-steps.yml
@@ -0,0 +1,21 @@
+# File templates/spell-check-prereq-steps.yml
+#
+# template file used to install spell checking prerequisits
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+
+parameters:
+  none: ''
+
+steps:
+- task: NodeTool@0
+  inputs:
+    versionSpec: '10.x'
+    #checkLatest: false # Optional
+  condition: and(gt(variables.pkg_count, 0), succeeded())
+
+- script: npm install -g cspell
+  displayName: 'Install cspell npm'
+  condition: and(gt(variables.pkg_count, 0), succeeded())
-- 
2.21.0.windows.1


  parent reply	other threads:[~2019-10-29 19:55 UTC|newest]

Thread overview: 56+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-10-29 19:54 [Patch v3 00/22] Enable Phase 1 of EDK II CI Michael D Kinney
2019-10-29 19:54 ` [Patch v3 01/22] Maintainers.txt: Add continuous integration(CI) directories Michael D Kinney
2019-10-30  2:34   ` Leif Lindholm
2019-10-31  9:03     ` Laszlo Ersek
2019-10-29 19:54 ` [Patch v3 02/22] .gitignore: Ignore python compiled files, extdeps, and vscode Michael D Kinney
2019-10-30  2:29   ` Leif Lindholm
2019-10-31  4:43     ` Michael D Kinney
2019-10-31  9:02       ` Laszlo Ersek
2019-11-04 16:02         ` [edk2-devel] " Philippe Mathieu-Daudé
2019-10-29 19:54 ` [Patch v3 03/22] requirements.txt: Add python pip requirements file Michael D Kinney
2019-10-30  2:27   ` Leif Lindholm
2019-10-30  2:52     ` [edk2-devel] " Sean
2019-10-30  3:03       ` Leif Lindholm
2019-10-31  4:39         ` Michael D Kinney
2019-10-31  4:58           ` Leif Lindholm
2019-10-31  7:18             ` Sean
2019-10-31  8:59     ` Laszlo Ersek
2019-10-29 19:54 ` [Patch v3 04/22] BaseTools/tools_def.template: Add RC_PATH define Michael D Kinney
2019-11-05  8:18   ` Liming Gao
2019-11-07  0:32     ` Michael D Kinney
2019-10-29 19:55 ` [Patch v3 05/22] BaseTools: Add YAML files with path env and tool extdeps Michael D Kinney
2019-10-29 19:55 ` [Patch v3 06/22] BaseTools: Add BaseTools plugins to support CI Michael D Kinney
2019-10-29 19:55 ` [Patch v3 07/22] .pytool/Plugin: Add CI plugins Michael D Kinney
2019-10-29 19:55 ` [Patch v3 08/22] CryptoPkg: Add YAML file for CI builds Michael D Kinney
2019-10-29 19:55 ` [Patch v3 09/22] FatPkg: " Michael D Kinney
2019-10-29 19:55 ` [Patch v3 10/22] FmpDevicePkg: " Michael D Kinney
2019-10-29 19:55 ` [Patch v3 11/22] MdeModulePkg: " Michael D Kinney
2019-10-29 19:55 ` [Patch v3 12/22] MdePkg: " Michael D Kinney
2019-11-05  8:22   ` Liming Gao
2019-11-05 17:04     ` Michael D Kinney
2019-11-06  0:20       ` Liming Gao
2019-11-06  0:29         ` Michael D Kinney
2019-10-29 19:55 ` [Patch v3 13/22] NetworkPkg: " Michael D Kinney
2019-10-29 19:55 ` [Patch v3 14/22] PcAtChipsetPkg: Add YAML files " Michael D Kinney
2019-10-29 19:55 ` [Patch v3 15/22] SecurityPkg: " Michael D Kinney
2019-10-29 19:55 ` [Patch v3 16/22] ShellPkg: Add YAML file " Michael D Kinney
2019-11-05  2:19   ` Gao, Zhichao
2019-10-29 19:55 ` [Patch v3 17/22] UefiCpuPkg: " Michael D Kinney
2019-10-31  8:55   ` Laszlo Ersek
2019-10-29 19:55 ` [Patch v3 18/22] SignedCapsulePkg: Use BaseCryptLibNull to reduce package CI time Michael D Kinney
2019-11-05  8:23   ` [edk2-devel] " Liming Gao
2019-11-05 17:05     ` Michael D Kinney
2019-10-29 19:55 ` [Patch v3 19/22] .pytool: Add CISettings.py and Readme.md Michael D Kinney
2019-10-29 19:55 ` Michael D Kinney [this message]
2019-10-29 19:55 ` [Patch v3 21/22] .merify: Add Megify YML pull request rules configuration file Michael D Kinney
2019-10-29 19:55 ` [Patch v3 22/22] Readme.md: Add CI build status badges Michael D Kinney
2019-10-30  2:58   ` Leif Lindholm
2019-10-31  4:42     ` Michael D Kinney
2019-11-06  0:21   ` [edk2-devel] " Liming Gao
2019-11-06  0:34     ` Michael D Kinney
2019-10-31  9:54 ` [edk2-devel] [Patch v3 00/22] Enable Phase 1 of EDK II CI Laszlo Ersek
2019-11-01  5:03   ` Michael D Kinney
2019-11-01 22:39     ` Laszlo Ersek
2019-11-01 23:22       ` Laszlo Ersek
2019-11-02  0:32       ` Michael D Kinney
2019-11-02 10:49         ` Laszlo Ersek

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-list from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20191029195517.20028-21-michael.d.kinney@intel.com \
    --to=devel@edk2.groups.io \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox