From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mga01.intel.com (mga01.intel.com []) by mx.groups.io with SMTP id smtpd.web11.1594.1573461679711800722 for ; Mon, 11 Nov 2019 00:41:21 -0800 Authentication-Results: mx.groups.io; dkim=missing; spf=fail (domain: intel.com, ip: , mailfrom: michael.d.kinney@intel.com) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 11 Nov 2019 00:41:18 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.68,292,1569308400"; d="scan'208";a="234403717" Received: from mdkinney-mobl2.amr.corp.intel.com ([10.255.231.122]) by fmsmga002.fm.intel.com with ESMTP; 11 Nov 2019 00:41:18 -0800 From: "Michael D Kinney" To: devel@edk2.groups.io Cc: Sean Brogan , Bret Barkelew , Liming Gao Subject: [Patch v5 20/22] .azurepipelines: Add Azure Pipelines YML configuration files Date: Mon, 11 Nov 2019 00:41:05 -0800 Message-Id: <20191111084107.5368-21-michael.d.kinney@intel.com> X-Mailer: git-send-email 2.21.0.windows.1 In-Reply-To: <20191111084107.5368-1-michael.d.kinney@intel.com> References: <20191111084107.5368-1-michael.d.kinney@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: Sean Brogan Add YML configuration files used to run the EDK II Continuous Integration (CI) checks on Azure Pipelines agents. Cc: Sean Brogan Cc: Bret Barkelew Cc: Liming Gao Signed-off-by: Michael D Kinney Reviewed-by: Liming Gao --- .azurepipelines/Ubuntu-GCC5.yml | 18 +++ .azurepipelines/Ubuntu-PatchCheck.yml | 35 +++++ .azurepipelines/Windows-VS2019.yml | 18 +++ .../templates/basetools-build-steps.yml | 37 +++++ .../templates/pr-gate-build-job.yml | 80 +++++++++++ .azurepipelines/templates/pr-gate-steps.yml | 130 ++++++++++++++++++ .../templates/spell-check-prereq-steps.yml | 22 +++ 7 files changed, 340 insertions(+) create mode 100644 .azurepipelines/Ubuntu-GCC5.yml create mode 100644 .azurepipelines/Ubuntu-PatchCheck.yml create mode 100644 .azurepipelines/Windows-VS2019.yml create mode 100644 .azurepipelines/templates/basetools-build-steps.yml create mode 100644 .azurepipelines/templates/pr-gate-build-job.yml create mode 100644 .azurepipelines/templates/pr-gate-steps.yml create mode 100644 .azurepipelines/templates/spell-check-prereq-steps.yml diff --git a/.azurepipelines/Ubuntu-GCC5.yml b/.azurepipelines/Ubuntu-GCC5.yml new file mode 100644 index 0000000000..16d4e74158 --- /dev/null +++ b/.azurepipelines/Ubuntu-GCC5.yml @@ -0,0 +1,18 @@ +## @file +# Azure Pipeline build file for a build using ubuntu and GCC5 +# +# Copyright (c) 2019, Microsoft Corporation +# SPDX-License-Identifier: BSD-2-Clause-Patent +## +trigger: +- master +pr: +- master + +jobs: +- template: templates/pr-gate-build-job.yml + parameters: + tool_chain_tag: 'GCC5' + vm_image: 'ubuntu-latest' + arch_list: "IA32,X64,ARM,AARCH64" + diff --git a/.azurepipelines/Ubuntu-PatchCheck.yml b/.azurepipelines/Ubuntu-PatchCheck.yml new file mode 100644 index 0000000000..dff8f579b6 --- /dev/null +++ b/.azurepipelines/Ubuntu-PatchCheck.yml @@ -0,0 +1,35 @@ +## @file +# Azure Pipielines YML file that evalues the patch series in a PR using the +# python script BaseTools/Scripts/PatchCheck.py. +# +# NOTE: This example monitors pull requests against the edk2-ci branch. Most +# environments would replace 'edk2-ci' with 'master'. +# +# Copyright (c) 2019, Intel Corporation. All rights reserved.
+# SPDX-License-Identifier: BSD-2-Clause-Patent +# +# https://github.com/tianocore +# +## + +trigger: none + +pr: +- master + +pool: + vmImage: 'ubuntu-latest' + +steps: +- checkout: self + clean: true + +- task: UsePythonVersion@0 + inputs: + versionSpec: '3.7.x' + architecture: 'x64' + +- script: | + git fetch origin $(System.PullRequest.TargetBranch):$(System.PullRequest.TargetBranch) + python BaseTools/Scripts/PatchCheck.py $(System.PullRequest.TargetBranch)..$(System.PullRequest.SourceCommitId) + displayName: 'Use PatchCheck.py to verify patch series in pull request' diff --git a/.azurepipelines/Windows-VS2019.yml b/.azurepipelines/Windows-VS2019.yml new file mode 100644 index 0000000000..4a5262f3ff --- /dev/null +++ b/.azurepipelines/Windows-VS2019.yml @@ -0,0 +1,18 @@ +## @file +# Azure Pipeline build file for a build using Windows and VS2019 +# +# Copyright (c) 2019, Microsoft Corporation +# SPDX-License-Identifier: BSD-2-Clause-Patent +## +trigger: +- master + +pr: +- master + +jobs: +- template: templates/pr-gate-build-job.yml + parameters: + tool_chain_tag: 'VS2019' + vm_image: 'windows-latest' + arch_list: "IA32,X64" diff --git a/.azurepipelines/templates/basetools-build-steps.yml b/.azurepipelines/templates/basetools-build-steps.yml new file mode 100644 index 0000000000..05e12c9565 --- /dev/null +++ b/.azurepipelines/templates/basetools-build-steps.yml @@ -0,0 +1,37 @@ +## @file +# File templates/basetools-build-job.yml +# +# template file to build basetools +# +# Copyright (c) 2019, Microsoft Corporation +# SPDX-License-Identifier: BSD-2-Clause-Patent +## +parameters: + tool_chain_tag: '' + +steps: +- ${{ if contains(parameters.tool_chain_tag, 'GCC') }}: + - bash: sudo apt-get update + displayName: Update apt + condition: and(gt(variables.pkg_count, 0), succeeded()) + + - bash: sudo apt-get install gcc g++ make uuid-dev + displayName: Install required tools + condition: and(gt(variables.pkg_count, 0), succeeded()) + +- task: CmdLine@1 + displayName: Build Base Tools from source + inputs: + filename: python + arguments: BaseTools/Edk2ToolsBuild.py -t ${{ parameters.tool_chain_tag }} + condition: and(gt(variables.pkg_count, 0), succeeded()) + +- task: CopyFiles@2 + displayName: "Copy base tools build log" + inputs: + targetFolder: '$(Build.ArtifactStagingDirectory)' + SourceFolder: 'BaseTools/BaseToolsBuild' + contents: | + BASETOOLS_BUILD*.* + flattenFolders: true + condition: and(gt(variables.pkg_count, 0), succeededOrFailed()) diff --git a/.azurepipelines/templates/pr-gate-build-job.yml b/.azurepipelines/templates/pr-gate-build-job.yml new file mode 100644 index 0000000000..bc30628796 --- /dev/null +++ b/.azurepipelines/templates/pr-gate-build-job.yml @@ -0,0 +1,80 @@ +## @file +# File templates/pr-gate-build-job.yml +# +# template file used to build supported packages. +# +# Copyright (c) 2019, Microsoft Corporation +# SPDX-License-Identifier: BSD-2-Clause-Patent +## + +parameters: + tool_chain_tag: '' + vm_image: '' + arch_list: '' + +# Build step +jobs: + +- job: Build_${{ parameters.tool_chain_tag }} + + #Use matrix to speed up the build process + strategy: + matrix: + TARGET_MDE_CPU: + Build.Pkgs: 'MdePkg,UefiCpuPkg' + Build.Targets: 'DEBUG,RELEASE,NO-TARGET' + TARGET_MDEMODULE_DEBUG: + Build.Pkgs: 'MdeModulePkg' + Build.Targets: 'DEBUG' + TARGET_MDEMODULE_RELEASE: + Build.Pkgs: 'MdeModulePkg' + Build.Targets: 'RELEASE,NO-TARGET' + TARGET_NETWORK: + Build.Pkgs: 'NetworkPkg' + Build.Targets: 'DEBUG,RELEASE,NO-TARGET' + TARGET_OTHER: + Build.Pkgs: 'PcAtChipsetPkg,ShellPkg' + Build.Targets: 'DEBUG,RELEASE,NO-TARGET' + TARGET_FMP: + Build.Pkgs: 'FmpDevicePkg,FatPkg' + Build.Targets: 'DEBUG,RELEASE,NO-TARGET' + TARGET_CRYPTO: + Build.Pkgs: 'CryptoPkg' + Build.Targets: 'DEBUG,RELEASE,NO-TARGET' + TARGET_SECURITY: + Build.Pkgs: 'SecurityPkg' + Build.Targets: 'DEBUG,RELEASE,NO-TARGET' + + workspace: + clean: all + + pool: + vmImage: ${{ parameters.vm_image }} + + steps: + - template: pr-gate-steps.yml + parameters: + tool_chain_tag: ${{ parameters.tool_chain_tag }} + build_pkgs: $(Build.Pkgs) + build_targets: $(Build.Targets) + build_archs: ${{ parameters.arch_list }} + +- job: FINISHED + dependsOn: Build_${{ parameters.tool_chain_tag }} + condition: succeeded() + steps: + - checkout: none + - script: | + echo FINISHED + sleep 10 + displayName: FINISHED + +- job: FAILED + dependsOn: Build_${{ parameters.tool_chain_tag }} + condition: failed() + steps: + - checkout: none + - script: | + echo FAILED + sleep 10 + displayName: FAILED diff --git a/.azurepipelines/templates/pr-gate-steps.yml b/.azurepipelines/templates/pr-gate-steps.yml new file mode 100644 index 0000000000..945053c337 --- /dev/null +++ b/.azurepipelines/templates/pr-gate-steps.yml @@ -0,0 +1,130 @@ +## @file +# File templates/pr-gate-steps.yml +# +# template file containing the steps to build +# +# Copyright (c) 2019, Microsoft Corporation +# SPDX-License-Identifier: BSD-2-Clause-Patent +## + +parameters: + tool_chain_tag: '' + build_pkgs: '' + build_targets: '' + build_archs: '' + +steps: +- checkout: self + clean: true + fetchDepth: 1 + +- task: UsePythonVersion@0 + inputs: + versionSpec: '3.7.x' + architecture: 'x64' + +- script: pip install -r pip_requirements.txt --upgrade + displayName: 'Install/Upgrade pip modules' + +# Set default +- bash: | + echo "##vso[task.setvariable variable=pkgs_to_build]${{ parameters.build_pkgs }}" + echo "##vso[task.setvariable variable=pkg_count]${{ 1 }}" + +# trim the package list if this is a PR +- task: CmdLine@1 + displayName: Check if ${{ parameters.build_pkgs }} need testing + inputs: + filename: stuart_pr_eval + arguments: -c .pytool/CISettings.py -p ${{ parameters.build_pkgs }} --pr-target origin/$(System.PullRequest.targetBranch) --output-csv-format-string "##vso[task.setvariable variable=pkgs_to_build;isOutpout=true]{pkgcsv}" --output-count-format-string "##vso[task.setvariable variable=pkg_count;isOutpout=true]{pkgcount}" + condition: eq(variables['Build.Reason'], 'PullRequest') + +# build basetools +- template: basetools-build-steps.yml + parameters: + tool_chain_tag: ${{ parameters.tool_chain_tag }} + +# install spell check prereqs +- template: spell-check-prereq-steps.yml + +# Build repo +- task: CmdLine@1 + displayName: Setup ${{ parameters.build_pkgs }} ${{ parameters.build_archs}} + inputs: + filename: stuart_setup + arguments: -c .pytool/CISettings.py -p $(pkgs_to_build) -t ${{ parameters.build_targets}} -a ${{ parameters.build_archs}} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}} + condition: and(gt(variables.pkg_count, 0), succeeded()) + +- task: CmdLine@1 + displayName: Update ${{ parameters.build_pkgs }} ${{ parameters.build_archs}} + inputs: + filename: stuart_update + arguments: -c .pytool/CISettings.py -p $(pkgs_to_build) -t ${{ parameters.build_targets}} -a ${{ parameters.build_archs}} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}} + condition: and(gt(variables.pkg_count, 0), succeeded()) + +- task: CmdLine@1 + displayName: Build and Test ${{ parameters.build_pkgs }} ${{ parameters.build_archs}} + inputs: + filename: stuart_ci_build + arguments: -c .pytool/CISettings.py -p $(pkgs_to_build) -t ${{ parameters.build_targets}} -a ${{ parameters.build_archs}} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}} + condition: and(gt(variables.pkg_count, 0), succeeded()) + +# Publish Test Results to Azure Pipelines/TFS +- task: PublishTestResults@2 + displayName: 'Publish junit test results' + continueOnError: true + condition: and( succeededOrFailed(),gt(variables.pkg_count, 0)) + inputs: + testResultsFormat: 'JUnit' # Options: JUnit, NUnit, VSTest, xUnit + testResultsFiles: 'Build/TestSuites.xml' + #searchFolder: '$(System.DefaultWorkingDirectory)' # Optional + mergeTestResults: true # Optional + testRunTitle: $(System.JobName) # Optional + #buildPlatform: # Optional + #buildConfiguration: # Optional + publishRunAttachments: true # Optional + +# Publish Test Results to Azure Pipelines/TFS +- task: PublishTestResults@2 + displayName: 'Publish host based test results for $(System.JobName)' + continueOnError: true + condition: and( succeededOrFailed(), gt(variables.pkg_count, 0)) + inputs: + testResultsFormat: 'JUnit' # Options: JUnit, NUnit, VSTest, xUnit + testResultsFiles: 'Build/**/*.result.xml' + #searchFolder: '$(System.DefaultWorkingDirectory)' # Optional + mergeTestResults: false # Optional + testRunTitle: ${{ parameters.build_pkgs }} # Optional + #buildPlatform: # Optional + #buildConfiguration: # Optional + publishRunAttachments: true # Optional + +# Copy the build logs to the artifact staging directory +- task: CopyFiles@2 + displayName: "Copy build logs" + inputs: + targetFolder: '$(Build.ArtifactStagingDirectory)' + SourceFolder: 'Build' + contents: | + BUILDLOG_*.txt + BUILDLOG_*.md + CI_*.txt + CI_*.md + CISETUP.txt + SETUPLOG.txt + UPDATE_LOG.txt + PREVALLOG.txt + TestSuites.xml + **/BUILD_TOOLS_REPORT.html + **/OVERRIDELOG.TXT + flattenFolders: true + condition: succeededOrFailed() + +# Publish build artifacts to Azure Artifacts/TFS or a file share +- task: PublishBuildArtifacts@1 + continueOnError: true + displayName: "Publish build logs" + inputs: + pathtoPublish: '$(Build.ArtifactStagingDirectory)' + artifactName: 'Build Logs $(System.JobName)' + condition: succeededOrFailed() diff --git a/.azurepipelines/templates/spell-check-prereq-steps.yml b/.azurepipelines/templates/spell-check-prereq-steps.yml new file mode 100644 index 0000000000..74d5eda2e7 --- /dev/null +++ b/.azurepipelines/templates/spell-check-prereq-steps.yml @@ -0,0 +1,22 @@ +## @file +# File templates/spell-check-prereq-steps.yml +# +# template file used to install spell checking prerequisits +# +# Copyright (c) 2019, Microsoft Corporation +# SPDX-License-Identifier: BSD-2-Clause-Patent +## + +parameters: + none: '' + +steps: +- task: NodeTool@0 + inputs: + versionSpec: '10.x' + #checkLatest: false # Optional + condition: and(gt(variables.pkg_count, 0), succeeded()) + +- script: npm install -g cspell + displayName: 'Install cspell npm' + condition: and(gt(variables.pkg_count, 0), succeeded()) -- 2.21.0.windows.1