From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web12.7372.1637157961959153732 for ; Wed, 17 Nov 2021 06:06:02 -0800 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: pierre.gondois@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9B0F41396; Wed, 17 Nov 2021 06:06:01 -0800 (PST) Received: from e120189.home (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id A3FB53F70D; Wed, 17 Nov 2021 06:05:59 -0800 (PST) From: "PierreGondois" To: devel@edk2.groups.io Cc: Ard Biesheuvel , Leif Lindholm , Sean Brogan , Bret Barkelew , Michael D Kinney , Liming Gao , Sami Mujawar Subject: [PATCH edk2-platforms v6 4/6] .azurepipelines: Add Azure Pipelines YML configuration files Date: Wed, 17 Nov 2021 14:05:26 +0000 Message-Id: <20211117140528.13600-5-Pierre.Gondois@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20211117140528.13600-1-Pierre.Gondois@arm.com> References: <20211117140528.13600-1-Pierre.Gondois@arm.com> From: Pierre Gondois To enable CI support of the tianocore/edk2-platforms repository, add YML configuration files used to run Continuous Integration (CI) checks on Azure Pipelines agents. These files are copies of the files from the tianocore/edk2 repository. Any modification to the tianocore/edk2 files must be reflected on the tianocore/edk2-platforms copies. The following files have been modified: - .azurepipelines/templates/platform-build-run-steps.yml - .azurepipelines/templates/pr-gate-build-job.yml - .azurepipelines/templates/pr-gate-steps.yml The sections modified are marked with the following comments: -EDK2_PLATFORMS_MODIF_START -EDK2_PLATFORMS_MODIF_END Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3509 Cc: Ard Biesheuvel Cc: Leif Lindholm Cc: Sean Brogan Cc: Bret Barkelew Cc: Michael D Kinney Cc: Liming Gao Cc: Sami Mujawar Co-authored-by: Sami Mujawar Signed-off-by: Pierre Gondois --- .azurepipelines/ReadMe.md | 5 + .azurepipelines/Ubuntu-GCC5.yml | 21 +++ .azurepipelines/Ubuntu-PatchCheck.yml | 55 +++++++ .azurepipelines/Windows-VS2019.yml | 20 +++ .azurepipelines/templates/ReadMe.md | 5 + .../templates/pr-gate-build-job.yml | 43 ++++++ .azurepipelines/templates/pr-gate-steps.yml | 142 ++++++++++++++++++ .../templates/spell-check-prereq-steps.yml | 22 +++ 8 files changed, 313 insertions(+) create mode 100644 .azurepipelines/ReadMe.md create mode 100644 .azurepipelines/Ubuntu-GCC5.yml create mode 100644 .azurepipelines/Ubuntu-PatchCheck.yml create mode 100644 .azurepipelines/Windows-VS2019.yml create mode 100644 .azurepipelines/templates/ReadMe.md create mode 100644 .azurepipelines/templates/pr-gate-build-job.yml create mode 100644 .azurepipelines/templates/pr-gate-steps.yml create mode 100644 .azurepipelines/templates/spell-check-prereq-steps.yml diff --git a/.azurepipelines/ReadMe.md b/.azurepipelines/ReadMe.md new file mode 100644 index 000000000000..de69a4ca220d --- /dev/null +++ b/.azurepipelines/ReadMe.md @@ -0,0 +1,5 @@ +EDK2_PLATFORMS_MODIF_START +As the content of this folder has been imported from the tianocore repository at: +https://github.com/tianocore/edk2 +Please use the Readme.md that can be found there. +EDK2_PLATFORMS_MODIF_END diff --git a/.azurepipelines/Ubuntu-GCC5.yml b/.azurepipelines/Ubuntu-GCC5.yml new file mode 100644 index 000000000000..3760c6efe105 --- /dev/null +++ b/.azurepipelines/Ubuntu-GCC5.yml @@ -0,0 +1,21 @@ +## @file +# Azure Pipeline build file for a build using ubuntu and GCC5 +# +# Copyright (c) Microsoft Corporation. +# Copyright (c) 2020, Hewlett Packard Enterprise Development LP. All rights reserved.
+# SPDX-License-Identifier: BSD-2-Clause-Patent +## +trigger: +- master +- stable/* +pr: +- master +- stable/* + +jobs: +- template: templates/pr-gate-build-job.yml + parameters: + tool_chain_tag: 'GCC5' + vm_image: 'ubuntu-latest' + arch_list: "IA32,X64,ARM,AARCH64,RISCV64" + diff --git a/.azurepipelines/Ubuntu-PatchCheck.yml b/.azurepipelines/Ubuntu-PatchCheck.yml new file mode 100644 index 000000000000..f15c224dcbca --- /dev/null +++ b/.azurepipelines/Ubuntu-PatchCheck.yml @@ -0,0 +1,55 @@ +## @file +# Azure Pipielines YML file that evalues the patch series in a PR using the +# python script BaseTools/Scripts/PatchCheck.py. +# +# NOTE: This example monitors pull requests against the edk2-ci branch. Most +# environments would replace 'edk2-ci' with 'master'. +# +# Copyright (c) 2021, Arm Limited. All rights reserved.
+# Copyright (c) 2019 - 2020, Intel Corporation. All rights reserved.
+# SPDX-License-Identifier: BSD-2-Clause-Patent +# +# https://github.com/tianocore +# +## + +trigger: none + +pr: +- master +- stable/* + +pool: + vmImage: 'ubuntu-latest' + +steps: +- checkout: self + clean: true + +- task: UsePythonVersion@0 + inputs: + versionSpec: '3.7.x' + architecture: 'x64' + +# EDK2_PLATFORMS_MODIF_START: +# Get dependencies. PatchCheck.py is in edk2, which is downloaded by stuart_update. +- script: pip install -r pip-requirements.txt --upgrade + displayName: 'Install/Upgrade pip modules' + +- task: CmdLine@1 + displayName: Setup + inputs: + filename: stuart_setup + arguments: -c .pytool/CISettings.py + +- task: CmdLine@1 + displayName: Update + inputs: + filename: stuart_update + arguments: -c .pytool/CISettings.py + +- script: | + git fetch origin $(System.PullRequest.TargetBranch):$(System.PullRequest.TargetBranch) + python edk2_extdep/edk2/BaseTools/Scripts/PatchCheck.py $(System.PullRequest.TargetBranch)..$(System.PullRequest.SourceCommitId) + displayName: 'Use PatchCheck.py to verify patch series in pull request' +# EDK2_PLATFORMS_MODIF_END diff --git a/.azurepipelines/Windows-VS2019.yml b/.azurepipelines/Windows-VS2019.yml new file mode 100644 index 000000000000..22f2d88c2c6a --- /dev/null +++ b/.azurepipelines/Windows-VS2019.yml @@ -0,0 +1,20 @@ +## @file +# Azure Pipeline build file for a build using Windows and VS2019 +# +# Copyright (c) Microsoft Corporation. +# SPDX-License-Identifier: BSD-2-Clause-Patent +## +trigger: +- master +- stable/* + +pr: +- master +- stable/* + +jobs: +- template: templates/pr-gate-build-job.yml + parameters: + tool_chain_tag: 'VS2019' + vm_image: 'windows-latest' + arch_list: "IA32,X64" diff --git a/.azurepipelines/templates/ReadMe.md b/.azurepipelines/templates/ReadMe.md new file mode 100644 index 000000000000..de69a4ca220d --- /dev/null +++ b/.azurepipelines/templates/ReadMe.md @@ -0,0 +1,5 @@ +EDK2_PLATFORMS_MODIF_START +As the content of this folder has been imported from the tianocore repository at: +https://github.com/tianocore/edk2 +Please use the Readme.md that can be found there. +EDK2_PLATFORMS_MODIF_END diff --git a/.azurepipelines/templates/pr-gate-build-job.yml b/.azurepipelines/templates/pr-gate-build-job.yml new file mode 100644 index 000000000000..9bb8e2819793 --- /dev/null +++ b/.azurepipelines/templates/pr-gate-build-job.yml @@ -0,0 +1,43 @@ +## @file +# File templates/pr-gate-build-job.yml +# +# template file used to build supported packages. +# +# Copyright (c) Microsoft Corporation. +# Copyright (c) 2020 - 2021, ARM Limited. All rights reserved.
+# SPDX-License-Identifier: BSD-2-Clause-Patent +## + +parameters: + tool_chain_tag: '' + vm_image: '' + arch_list: '' + +# Build step +jobs: + +- job: Build_${{ parameters.tool_chain_tag }} + +# EDK2_PLATFORMS_MODIF_START: +# Build edk2-platforms packages + #Use matrix to speed up the build process + strategy: + matrix: + TARGET_ARM_PLATFORMS: + Build.Pkgs: 'JunoPkg' + Build.Targets: 'DEBUG,RELEASE,NO-TARGET,NOOPT' +# EDK2_PLATFORMS_MODIF_END + + workspace: + clean: all + + pool: + vmImage: ${{ parameters.vm_image }} + + steps: + - template: pr-gate-steps.yml + parameters: + tool_chain_tag: ${{ parameters.tool_chain_tag }} + build_pkgs: $(Build.Pkgs) + build_targets: $(Build.Targets) + build_archs: ${{ parameters.arch_list }} diff --git a/.azurepipelines/templates/pr-gate-steps.yml b/.azurepipelines/templates/pr-gate-steps.yml new file mode 100644 index 000000000000..0a8a0e1e09a3 --- /dev/null +++ b/.azurepipelines/templates/pr-gate-steps.yml @@ -0,0 +1,142 @@ +## @file +# File templates/pr-gate-steps.yml +# +# template file containing the steps to build +# +# Copyright (c) 2021, Arm Limited. All rights reserved.
+# Copyright (c) Microsoft Corporation. +# SPDX-License-Identifier: BSD-2-Clause-Patent +## + +parameters: + tool_chain_tag: '' + build_pkgs: '' + build_targets: '' + build_archs: '' + +steps: +- checkout: self + clean: true + fetchDepth: 1 + +- task: UsePythonVersion@0 + inputs: + versionSpec: '3.8.x' + architecture: 'x64' + +- script: pip install -r pip-requirements.txt --upgrade + displayName: 'Install/Upgrade pip modules' + +# Set default +- bash: | + echo "##vso[task.setvariable variable=pkgs_to_build]${{ parameters.build_pkgs }}" + echo "##vso[task.setvariable variable=pkg_count]${{ 1 }}" + +# Fetch the target branch so that pr_eval can diff them. +# Seems like azure pipelines/github changed checkout process in nov 2020. +- script: git fetch origin $(System.PullRequest.targetBranch) + displayName: fetch target branch + condition: eq(variables['Build.Reason'], 'PullRequest') + +# trim the package list if this is a PR +- task: CmdLine@1 + displayName: Check if ${{ parameters.build_pkgs }} need testing + inputs: + filename: stuart_pr_eval + arguments: -c .pytool/CISettings.py -p ${{ parameters.build_pkgs }} --pr-target origin/$(System.PullRequest.targetBranch) --output-csv-format-string "##vso[task.setvariable variable=pkgs_to_build;isOutpout=true]{pkgcsv}" --output-count-format-string "##vso[task.setvariable variable=pkg_count;isOutpout=true]{pkgcount}" + condition: eq(variables['Build.Reason'], 'PullRequest') + +# install spell check prereqs +- template: spell-check-prereq-steps.yml + +# Build repo +- task: CmdLine@1 + displayName: Setup ${{ parameters.build_pkgs }} ${{ parameters.build_archs}} + inputs: + filename: stuart_setup + arguments: -c .pytool/CISettings.py -p $(pkgs_to_build) -t ${{ parameters.build_targets}} -a ${{ parameters.build_archs}} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}} + condition: and(gt(variables.pkg_count, 0), succeeded()) + +- task: CmdLine@1 + displayName: Update ${{ parameters.build_pkgs }} ${{ parameters.build_archs}} + inputs: + filename: stuart_update + arguments: -c .pytool/CISettings.py -p $(pkgs_to_build) -t ${{ parameters.build_targets}} -a ${{ parameters.build_archs}} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}} + condition: and(gt(variables.pkg_count, 0), succeeded()) + +# build basetools +# do this after setup and update so that code base dependencies +# are all resolved. +# EDK2_PLATFORMS_MODIF_START: +# The base tools are imported in .pytool/CISettings.py via the 'edk2basetools' python module. +# - template: basetools-build-steps.yml +# parameters: +# tool_chain_tag: ${{ parameters.tool_chain_tag }} +# EDK2_PLATFORMS_MODIF_END + +- task: CmdLine@1 + displayName: Build and Test ${{ parameters.build_pkgs }} ${{ parameters.build_archs}} + inputs: + filename: stuart_ci_build + arguments: -c .pytool/CISettings.py -p $(pkgs_to_build) -t ${{ parameters.build_targets}} -a ${{ parameters.build_archs}} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}} + condition: and(gt(variables.pkg_count, 0), succeeded()) + +# Publish Test Results to Azure Pipelines/TFS +- task: PublishTestResults@2 + displayName: 'Publish junit test results' + continueOnError: true + condition: and( succeededOrFailed(),gt(variables.pkg_count, 0)) + inputs: + testResultsFormat: 'JUnit' # Options: JUnit, NUnit, VSTest, xUnit + testResultsFiles: 'Build/TestSuites.xml' + #searchFolder: '$(System.DefaultWorkingDirectory)' # Optional + mergeTestResults: true # Optional + testRunTitle: $(System.JobName) # Optional + #buildPlatform: # Optional + #buildConfiguration: # Optional + publishRunAttachments: true # Optional + +# Publish Test Results to Azure Pipelines/TFS +- task: PublishTestResults@2 + displayName: 'Publish host based test results for $(System.JobName)' + continueOnError: true + condition: and( succeededOrFailed(), gt(variables.pkg_count, 0)) + inputs: + testResultsFormat: 'JUnit' # Options: JUnit, NUnit, VSTest, xUnit + testResultsFiles: 'Build/**/*.result.xml' + #searchFolder: '$(System.DefaultWorkingDirectory)' # Optional + mergeTestResults: false # Optional + testRunTitle: ${{ parameters.build_pkgs }} # Optional + #buildPlatform: # Optional + #buildConfiguration: # Optional + publishRunAttachments: true # Optional + +# Copy the build logs to the artifact staging directory +- task: CopyFiles@2 + displayName: "Copy build logs" + inputs: + targetFolder: '$(Build.ArtifactStagingDirectory)' + SourceFolder: 'Build' + contents: | + BUILDLOG_*.txt + BUILDLOG_*.md + CI_*.txt + CI_*.md + CISETUP.txt + SETUPLOG.txt + UPDATE_LOG.txt + PREVALLOG.txt + TestSuites.xml + **/BUILD_TOOLS_REPORT.html + **/OVERRIDELOG.TXT + flattenFolders: true + condition: succeededOrFailed() + +# Publish build artifacts to Azure Artifacts/TFS or a file share +- task: PublishBuildArtifacts@1 + continueOnError: true + displayName: "Publish build logs" + inputs: + pathtoPublish: '$(Build.ArtifactStagingDirectory)' + artifactName: 'Build Logs $(System.JobName)' + condition: succeededOrFailed() diff --git a/.azurepipelines/templates/spell-check-prereq-steps.yml b/.azurepipelines/templates/spell-check-prereq-steps.yml new file mode 100644 index 000000000000..98ee3cfa6bc6 --- /dev/null +++ b/.azurepipelines/templates/spell-check-prereq-steps.yml @@ -0,0 +1,22 @@ +## @file +# File templates/spell-check-prereq-steps.yml +# +# template file used to install spell checking prerequisits +# +# Copyright (c) Microsoft Corporation. +# SPDX-License-Identifier: BSD-2-Clause-Patent +## + +parameters: + none: '' + +steps: +- task: NodeTool@0 + inputs: + versionSpec: '14.x' + #checkLatest: false # Optional + condition: and(gt(variables.pkg_count, 0), succeeded()) + +- script: npm install -g cspell + displayName: 'Install cspell npm' + condition: and(gt(variables.pkg_count, 0), succeeded()) -- 2.17.1