* [PATCH v8 01/12] CI: make Python version configurable
[not found] <20221130200037.1098765-1-osteffen@redhat.com>
@ 2022-11-30 20:01 ` Oliver Steffen
2022-11-30 20:01 ` [PATCH v8 02/12] OvmfPkg: CI: use Python version from defaults template Oliver Steffen
` (10 subsequent siblings)
11 siblings, 0 replies; 17+ messages in thread
From: Oliver Steffen @ 2022-11-30 20:01 UTC (permalink / raw)
To: devel
Cc: Andrew Fish, Ard Biesheuvel, Bob Feng, Daniel Schaefer,
Gerd Hoffmann, Jiewen Yao, Jordan Justen, Leif Lindholm,
Liming Gao, Michael D Kinney, Michael Kubacki, Ray Ni,
Sami Mujawar, Sean Brogan, Sunil V L, Yuwei Chen, Zhiguang Liu,
Pawel Polawski, Chris Fernald, Oliver Steffen
Add a new parameter "usePythonVersion" to the CI job templates.
This makes it possible to specify the version of Python to use.
The default value is '', in which case Python will not be downloaded
at runtime and the one provided by the VM/container image will be used.
Additionally, add a template .azurepipelines/templates/defaults.yml,
from which the default Pyhton version string can be obtained.
Signed-off-by: Oliver Steffen <osteffen@redhat.com>
---
.azurepipelines/Windows-VS2019.yml | 4 ++++
.azurepipelines/templates/defaults.yml | 3 +++
.azurepipelines/templates/platform-build-run-steps.yml | 6 +++++-
.azurepipelines/templates/pr-gate-build-job.yml | 2 ++
.azurepipelines/templates/pr-gate-steps.yml | 6 ++++--
5 files changed, 18 insertions(+), 3 deletions(-)
create mode 100644 .azurepipelines/templates/defaults.yml
diff --git a/.azurepipelines/Windows-VS2019.yml
b/.azurepipelines/Windows-VS2019.yml
index e4bd4b1d2283..89bccde82575 100644
--- a/.azurepipelines/Windows-VS2019.yml
+++ b/.azurepipelines/Windows-VS2019.yml
@@ -12,9 +12,13 @@ pr:
- master
- stable/*
+variables:
+ - template: templates/defaults.yml
+
jobs:
- template: templates/pr-gate-build-job.yml
parameters:
tool_chain_tag: 'VS2019'
vm_image: 'windows-2019'
arch_list: "IA32,X64"
+ usePythonVersion: ${{ variables.default_python_version }}
diff --git a/.azurepipelines/templates/defaults.yml
b/.azurepipelines/templates/defaults.yml
new file mode 100644
index 000000000000..314d03e33af8
--- /dev/null
+++ b/.azurepipelines/templates/defaults.yml
@@ -0,0 +1,3 @@
+
+variables:
+ default_python_version: ">=3.10.6"
diff --git a/.azurepipelines/templates/platform-build-run-steps.yml
b/.azurepipelines/templates/platform-build-run-steps.yml
index 40a31a509fc5..8803d80cf51c 100644
--- a/.azurepipelines/templates/platform-build-run-steps.yml
+++ b/.azurepipelines/templates/platform-build-run-steps.yml
@@ -34,6 +34,9 @@ parameters:
- name: extra_install_step
type: stepList
default: []
+- name: usePythonVersion
+ type: string
+ default: ''
steps:
- checkout: self
@@ -42,8 +45,9 @@ steps:
- task: UsePythonVersion@0
inputs:
- versionSpec: ">=3.10.6"
+ versionSpec: ${{ parameters.usePythonVersion }}
architecture: "x64"
+ condition: ne('${{ parameters.usePythonVersion }}', '')
- script: pip install -r pip-requirements.txt --upgrade
displayName: 'Install/Upgrade pip modules'
diff --git a/.azurepipelines/templates/pr-gate-build-job.yml
b/.azurepipelines/templates/pr-gate-build-job.yml
index 7f88b41dc8d3..f0ef1142da49 100644
--- a/.azurepipelines/templates/pr-gate-build-job.yml
+++ b/.azurepipelines/templates/pr-gate-build-job.yml
@@ -12,6 +12,7 @@ parameters:
tool_chain_tag: ''
vm_image: ''
arch_list: ''
+ usePythonVersion: ''
# Build step
jobs:
@@ -77,3 +78,4 @@ jobs:
build_pkgs: $(Build.Pkgs)
build_targets: $(Build.Targets)
build_archs: ${{ parameters.arch_list }}
+ usePythonVersion: ${{ parameters.usePythonVersion }}
diff --git a/.azurepipelines/templates/pr-gate-steps.yml
b/.azurepipelines/templates/pr-gate-steps.yml
index cb431e53fcd1..27a44c06011c 100644
--- a/.azurepipelines/templates/pr-gate-steps.yml
+++ b/.azurepipelines/templates/pr-gate-steps.yml
@@ -12,6 +12,7 @@ parameters:
build_pkgs: ''
build_targets: ''
build_archs: ''
+ usePythonVersion: ''
steps:
- checkout: self
@@ -20,8 +21,9 @@ steps:
- task: UsePythonVersion@0
inputs:
- versionSpec: '>=3.10.6'
- architecture: 'x64'
+ versionSpec: ${{ parameters.usePythonVersion }}
+ architecture: "x64"
+ condition: ne('${{ parameters.usePythonVersion }}', '')
- script: pip install -r pip-requirements.txt --upgrade
displayName: 'Install/Upgrade pip modules'
--
2.38.1
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH v8 02/12] OvmfPkg: CI: use Python version from defaults template
[not found] <20221130200037.1098765-1-osteffen@redhat.com>
2022-11-30 20:01 ` [PATCH v8 01/12] CI: make Python version configurable Oliver Steffen
@ 2022-11-30 20:01 ` Oliver Steffen
2022-11-30 20:01 ` [PATCH v8 03/12] EmulatorPkg: " Oliver Steffen
` (9 subsequent siblings)
11 siblings, 0 replies; 17+ messages in thread
From: Oliver Steffen @ 2022-11-30 20:01 UTC (permalink / raw)
To: devel
Cc: Andrew Fish, Ard Biesheuvel, Bob Feng, Daniel Schaefer,
Gerd Hoffmann, Jiewen Yao, Jordan Justen, Leif Lindholm,
Liming Gao, Michael D Kinney, Michael Kubacki, Ray Ni,
Sami Mujawar, Sean Brogan, Sunil V L, Yuwei Chen, Zhiguang Liu,
Pawel Polawski, Chris Fernald, Oliver Steffen
Use the default Python version from the defaults template
(.azurepipelines/templates/defaults.yml) in the Windows
CI jobs.
Previous changes to the CI job templates make it necessary
to specify a version number, if Python shall be pulled
at CI runtime.
The Linux CI jobs will use Python from the container image,
so these changes are for Windows only.
Signed-off-by: Oliver Steffen <osteffen@redhat.com>
---
OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml
b/OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml
index 7d6344d6383d..7e63f419b26b 100644
--- a/OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml
+++ b/OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml
@@ -14,6 +14,10 @@ trigger:
pr:
- master
- stable/*
+
+variables:
+ - template: ../../../.azurepipelines/templates/defaults.yml
+
jobs:
- job: Platform_CI
variables:
@@ -133,6 +137,7 @@ jobs:
build_file: $(Build.File)
build_flags: $(Build.Flags)
run_flags: $(Run.Flags)
+ usePythonVersion: ${{ variables.default_python_version }}
extra_install_step:
- powershell: choco install qemu --version=2021.5.5;
Write-Host "##vso[task.prependpath]c:\Program Files\qemu"
displayName: Install QEMU and Set QEMU on path # friendly
name displayed in the UI
--
2.38.1
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH v8 03/12] EmulatorPkg: CI: use Python version from defaults template
[not found] <20221130200037.1098765-1-osteffen@redhat.com>
2022-11-30 20:01 ` [PATCH v8 01/12] CI: make Python version configurable Oliver Steffen
2022-11-30 20:01 ` [PATCH v8 02/12] OvmfPkg: CI: use Python version from defaults template Oliver Steffen
@ 2022-11-30 20:01 ` Oliver Steffen
2022-11-30 20:01 ` [PATCH v8 04/12] CI: add ~/.local/bin to PATH (Linux only) Oliver Steffen
` (8 subsequent siblings)
11 siblings, 0 replies; 17+ messages in thread
From: Oliver Steffen @ 2022-11-30 20:01 UTC (permalink / raw)
To: devel
Cc: Andrew Fish, Ard Biesheuvel, Bob Feng, Daniel Schaefer,
Gerd Hoffmann, Jiewen Yao, Jordan Justen, Leif Lindholm,
Liming Gao, Michael D Kinney, Michael Kubacki, Ray Ni,
Sami Mujawar, Sean Brogan, Sunil V L, Yuwei Chen, Zhiguang Liu,
Pawel Polawski, Chris Fernald, Oliver Steffen
Use the default Python version from the defaults template
(.azurepipelines/templates/defaults.yml) in the Windows
CI jobs.
Previous changes to the CI job templates make it necessary
to specify a version number, if Python shall be pulled
at CI runtime.
The Linux CI jobs will use Python from the container image,
so these changes are for Windows only.
Signed-off-by: Oliver Steffen <osteffen@redhat.com>
---
EmulatorPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/EmulatorPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml
b/EmulatorPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml
index e7ead06ae266..09960e7c7a6e 100644
--- a/EmulatorPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml
+++ b/EmulatorPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml
@@ -16,6 +16,9 @@ pr:
- master
- stable/*
+variables:
+ - template: ../../../.azurepipelines/templates/defaults.yml
+
jobs:
- job: Platform_CI
variables:
@@ -128,3 +131,4 @@ jobs:
build_file: $(Build.File)
build_flags: $(Build.Flags)
run_flags: $(Run.Flags)
+ usePythonVersion: ${{ variables.default_python_version }}
--
2.38.1
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH v8 04/12] CI: add ~/.local/bin to PATH (Linux only)
[not found] <20221130200037.1098765-1-osteffen@redhat.com>
` (2 preceding siblings ...)
2022-11-30 20:01 ` [PATCH v8 03/12] EmulatorPkg: " Oliver Steffen
@ 2022-11-30 20:01 ` Oliver Steffen
2022-11-30 20:01 ` [PATCH v8 05/12] CI: Allow running in a container Oliver Steffen
` (7 subsequent siblings)
11 siblings, 0 replies; 17+ messages in thread
From: Oliver Steffen @ 2022-11-30 20:01 UTC (permalink / raw)
To: devel
Cc: Andrew Fish, Ard Biesheuvel, Bob Feng, Daniel Schaefer,
Gerd Hoffmann, Jiewen Yao, Jordan Justen, Leif Lindholm,
Liming Gao, Michael D Kinney, Michael Kubacki, Ray Ni,
Sami Mujawar, Sean Brogan, Sunil V L, Yuwei Chen, Zhiguang Liu,
Pawel Polawski, Chris Fernald, Oliver Steffen
Fix pip install under certain conditions.
Signed-off-by: Oliver Steffen <osteffen@redhat.com>
---
.azurepipelines/templates/platform-build-run-steps.yml | 6 ++++++
.azurepipelines/templates/pr-gate-steps.yml | 6 ++++++
2 files changed, 12 insertions(+)
diff --git a/.azurepipelines/templates/platform-build-run-steps.yml
b/.azurepipelines/templates/platform-build-run-steps.yml
index 8803d80cf51c..087f460d7fec 100644
--- a/.azurepipelines/templates/platform-build-run-steps.yml
+++ b/.azurepipelines/templates/platform-build-run-steps.yml
@@ -39,6 +39,12 @@ parameters:
default: ''
steps:
+- bash: |
+ echo "##vso[task.prependpath]${HOME}/.local/bin"
+ echo "new PATH=${PATH}"
+ displayName: Set PATH
+ condition: eq('${{ parameters.tool_chain_tag }}', 'GCC5')
+
- checkout: self
clean: true
fetchDepth: 1
diff --git a/.azurepipelines/templates/pr-gate-steps.yml
b/.azurepipelines/templates/pr-gate-steps.yml
index 27a44c06011c..99d52cfa0605 100644
--- a/.azurepipelines/templates/pr-gate-steps.yml
+++ b/.azurepipelines/templates/pr-gate-steps.yml
@@ -15,6 +15,12 @@ parameters:
usePythonVersion: ''
steps:
+- bash: |
+ echo "##vso[task.prependpath]${HOME}/.local/bin"
+ echo "new PATH=${PATH}"
+ displayName: Set PATH
+ condition: eq('${{ parameters.tool_chain_tag }}', 'GCC5')
+
- checkout: self
clean: true
fetchDepth: 1
--
2.38.1
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH v8 05/12] CI: Allow running in a container.
[not found] <20221130200037.1098765-1-osteffen@redhat.com>
` (3 preceding siblings ...)
2022-11-30 20:01 ` [PATCH v8 04/12] CI: add ~/.local/bin to PATH (Linux only) Oliver Steffen
@ 2022-11-30 20:01 ` Oliver Steffen
2022-11-30 20:01 ` [PATCH v8 06/12] BaseTools: CI: Use Fedora 35 container (Linux only) Oliver Steffen
` (6 subsequent siblings)
11 siblings, 0 replies; 17+ messages in thread
From: Oliver Steffen @ 2022-11-30 20:01 UTC (permalink / raw)
To: devel
Cc: Andrew Fish, Ard Biesheuvel, Bob Feng, Daniel Schaefer,
Gerd Hoffmann, Jiewen Yao, Jordan Justen, Leif Lindholm,
Liming Gao, Michael D Kinney, Michael Kubacki, Ray Ni,
Sami Mujawar, Sean Brogan, Sunil V L, Yuwei Chen, Zhiguang Liu,
Pawel Polawski, Chris Fernald, Oliver Steffen
Add a parameter of the pr-gate-build-job template to specify a
container image URL. If the value is not '' (default), then the
jobs will be run inside a container based on that image.
Signed-off-by: Oliver Steffen <osteffen@redhat.com>
---
.azurepipelines/templates/pr-gate-build-job.yml | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/.azurepipelines/templates/pr-gate-build-job.yml
b/.azurepipelines/templates/pr-gate-build-job.yml
index f0ef1142da49..5ffcaca7b591 100644
--- a/.azurepipelines/templates/pr-gate-build-job.yml
+++ b/.azurepipelines/templates/pr-gate-build-job.yml
@@ -12,6 +12,7 @@ parameters:
tool_chain_tag: ''
vm_image: ''
arch_list: ''
+ container: ''
usePythonVersion: ''
# Build step
@@ -71,6 +72,9 @@ jobs:
pool:
vmImage: ${{ parameters.vm_image }}
+ ${{ if not(eq(parameters.container, '')) }}:
+ container: ${{ parameters.container }}
+
steps:
- template: pr-gate-steps.yml
parameters:
--
2.38.1
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH v8 06/12] BaseTools: CI: Use Fedora 35 container (Linux only)
[not found] <20221130200037.1098765-1-osteffen@redhat.com>
` (4 preceding siblings ...)
2022-11-30 20:01 ` [PATCH v8 05/12] CI: Allow running in a container Oliver Steffen
@ 2022-11-30 20:01 ` Oliver Steffen
2022-11-30 20:01 ` [PATCH v8 07/12] ArmVirtPkg: " Oliver Steffen
` (5 subsequent siblings)
11 siblings, 0 replies; 17+ messages in thread
From: Oliver Steffen @ 2022-11-30 20:01 UTC (permalink / raw)
To: devel
Cc: Andrew Fish, Ard Biesheuvel, Bob Feng, Daniel Schaefer,
Gerd Hoffmann, Jiewen Yao, Jordan Justen, Leif Lindholm,
Liming Gao, Michael D Kinney, Michael Kubacki, Ray Ni,
Sami Mujawar, Sean Brogan, Sunil V L, Yuwei Chen, Zhiguang Liu,
Pawel Polawski, Chris Fernald, Oliver Steffen
Run all Linux based jobs in a container, using a custom Fedora 35 image
(gcc 11). The image is hosted on ghcr.io and the Dockerfiles are
here: https://github.com/tianocore/containers
The version numbers of gcc, iasl, and nasm are pinned to avoid
unintended upgrades during image rebuild.
Do not run apt-get in CI jobs to install qemu and gcc dependencies.
Assume the container image provides these.
Use Python from the container image, do not download at runtime.
Remove BaseTools/Bin/gcc_*_ext_dep.yaml to stop
downloading gcc from external locations; use the
gcc provided by the container image instead.
The container image sets the variable GCC5_*_PREFIX accordingly.
Signed-off-by: Oliver Steffen <osteffen@redhat.com>
---
.azurepipelines/Ubuntu-GCC5.yml | 2 ++
.../templates/basetools-build-steps.yml | 9 --------
BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml | 21 ------------------
BaseTools/Bin/gcc_arm_linux_ext_dep.yaml | 21 ------------------
...gcc_loongarch64_unknown_linux_ext_dep.yaml | 22 -------------------
.../Bin/gcc_riscv64_unknown_ext_dep.yaml | 22 -------------------
6 files changed, 2 insertions(+), 95 deletions(-)
delete mode 100644 BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml
delete mode 100644 BaseTools/Bin/gcc_arm_linux_ext_dep.yaml
delete mode 100644 BaseTools/Bin/gcc_loongarch64_unknown_linux_ext_dep.yaml
delete mode 100644 BaseTools/Bin/gcc_riscv64_unknown_ext_dep.yaml
diff --git a/.azurepipelines/Ubuntu-GCC5.yml b/.azurepipelines/Ubuntu-GCC5.yml
index 1acd8d2a46a7..0dd9b74de57a 100644
--- a/.azurepipelines/Ubuntu-GCC5.yml
+++ b/.azurepipelines/Ubuntu-GCC5.yml
@@ -18,5 +18,7 @@ jobs:
parameters:
tool_chain_tag: 'GCC5'
vm_image: 'ubuntu-latest'
+ container: 'ghcr.io/tianocore/containers/fedora-35-build:2113a0e'
+ usePythonVersion: '' # use Python from the container image
arch_list: "IA32,X64,ARM,AARCH64,RISCV64,LOONGARCH64"
diff --git a/.azurepipelines/templates/basetools-build-steps.yml
b/.azurepipelines/templates/basetools-build-steps.yml
index d8c108c6e212..a72758bc3395 100644
--- a/.azurepipelines/templates/basetools-build-steps.yml
+++ b/.azurepipelines/templates/basetools-build-steps.yml
@@ -10,15 +10,6 @@ parameters:
tool_chain_tag: ''
steps:
-- ${{ if contains(parameters.tool_chain_tag, 'GCC') }}:
- - bash: sudo apt-get update
- displayName: Update apt
- condition: and(gt(variables.pkg_count, 0), succeeded())
-
- - bash: sudo apt-get install gcc g++ make uuid-dev
- displayName: Install required tools
- condition: and(gt(variables.pkg_count, 0), succeeded())
-
- task: CmdLine@1
displayName: Build Base Tools from source
inputs:
diff --git a/BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml
b/BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml
deleted file mode 100644
index ff8a9e868100..000000000000
--- a/BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml
+++ /dev/null
@@ -1,21 +0,0 @@
-## @file
-# Download GCC AARCH64 compiler from Arm's release site
-# Set shell variable GCC5_AARCH64_INSTALL to this folder
-#
-# This is only downloaded when a build activates scope gcc_aarch64_linux
-#
-# Copyright (c) Microsoft Corporation.
-# SPDX-License-Identifier: BSD-2-Clause-Patent
-##
-{
- "scope": "gcc_aarch64_linux",
- "type": "web",
- "name": "gcc_aarch64_linux",
- "source": "https://developer.arm.com/-/media/Files/downloads/gnu/11.2-2022.02/binrel/gcc-arm-11.2-2022.02-x86_64-aarch64-none-linux-gnu.tar.xz",
- "version": "11.2-2022.02",
- "sha256": "52dbac3eb71dbe0916f60a8c5ab9b7dc9b66b3ce513047baa09fae56234e53f3",
- "compression_type": "tar",
- "internal_path": "/gcc-arm-11.2-2022.02-x86_64-aarch64-none-linux-gnu/",
- "flags": ["set_shell_var", ],
- "var_name": "GCC5_AARCH64_INSTALL"
-}
diff --git a/BaseTools/Bin/gcc_arm_linux_ext_dep.yaml
b/BaseTools/Bin/gcc_arm_linux_ext_dep.yaml
deleted file mode 100644
index 151cbfa4b532..000000000000
--- a/BaseTools/Bin/gcc_arm_linux_ext_dep.yaml
+++ /dev/null
@@ -1,21 +0,0 @@
-## @file
-# Download GCC ARM compiler from Arm's release site
-# Set shell variable GCC5_ARM_INSTALL to this folder
-#
-# This is only downloaded when a build activates scope gcc_arm_linux
-#
-# Copyright (c) Microsoft Corporation.
-# SPDX-License-Identifier: BSD-2-Clause-Patent
-##
-{
- "scope": "gcc_arm_linux",
- "type": "web",
- "name": "gcc_arm_linux",
- "source": "https://developer.arm.com/-/media/Files/downloads/gnu/11.2-2022.02/binrel/gcc-arm-11.2-2022.02-x86_64-arm-none-linux-gnueabihf.tar.xz",
- "version": "11.2-2022.02",
- "sha256": "c254f7199261fe76c32ef42187502839bda7efad0a66646cf739d074eff45fad",
- "compression_type": "tar",
- "internal_path": "/gcc-arm-11.2-2022.02-x86_64-arm-none-linux-gnueabihf/",
- "flags": ["set_shell_var", ],
- "var_name": "GCC5_ARM_INSTALL"
-}
diff --git a/BaseTools/Bin/gcc_loongarch64_unknown_linux_ext_dep.yaml
b/BaseTools/Bin/gcc_loongarch64_unknown_linux_ext_dep.yaml
deleted file mode 100644
index fe9a2448531a..000000000000
--- a/BaseTools/Bin/gcc_loongarch64_unknown_linux_ext_dep.yaml
+++ /dev/null
@@ -1,22 +0,0 @@
-## @file
-# Download GCC LoongArch64 compiler from LoongArch GitHub release site
-# Set shell variable GCC5_LOONGARCH64_INSTALL to this folder
-#
-# This is only downloaded when a build activates scope
gcc_loongarch64_unknown_linux
-#
-# Copyright (c) Microsoft Corporation.
-# Copyright (c) 2022, Loongson Technology Corporation Limited. All
rights reserved.<BR>
-# SPDX-License-Identifier: BSD-2-Clause-Patent
-##
-{
- "scope": "gcc_loongarch64_unknown_linux",
- "type": "web",
- "name": "gcc_loongarch64_unknown_linux",
- "source":"https://github.com/loongson/build-tools/releases/download/2022.09.06/loongarch64-clfs-6.3-cross-tools-gcc-full.tar.xz",
- "version": "13.0.0",
- "sha256":"27a43c5bb127794f091d0e75da0003c4d0eec28a958d8f2cc7cd290a6e6133ab",
- "compression_type": "tar",
- "internal_path": "/cross-tools/",
- "flags": ["set_shell_var", ],
- "var_name": "GCC5_LOONGARCH64_INSTALL"
-}
diff --git a/BaseTools/Bin/gcc_riscv64_unknown_ext_dep.yaml
b/BaseTools/Bin/gcc_riscv64_unknown_ext_dep.yaml
deleted file mode 100644
index 8abbcd7ba040..000000000000
--- a/BaseTools/Bin/gcc_riscv64_unknown_ext_dep.yaml
+++ /dev/null
@@ -1,22 +0,0 @@
-## @file
-# Download GCC RISCV64 compiler from RISC-V Organization release site
-# Set shell variable GCC5_RISCV64_INSTALL to this folder
-#
-# This is only downloaded when a build activates scope gcc_riscv64_unknown
-#
-# Copyright (c) Microsoft Corporation.
-# Copyright (c) 2020, Hewlett Packard Enterprise Development LP. All
rights reserved.<BR>
-# SPDX-License-Identifier: BSD-2-Clause-Patent
-##
-{
- "scope": "gcc_riscv64_unknown",
- "type": "web",
- "name": "gcc_riscv64_unknown",
- "source": "https://raw.githubusercontent.com/riscv/riscv-uefi-edk2-docs/master/gcc-riscv-edk2-ci-toolchain/gcc-riscv-9.2.0-2020.04-x86_64_riscv64-unknown-gnu.tar.xz",
- "version": "9.2.0",
- "compression_type": "tar",
- "sha256": "28373643b69f0ce008273c3dc63f172aa1121952f1b9ae94d7485ac94af7f344",
- "internal_path": "/gcc-riscv-9.2.0-2020.04-x86_64_riscv64-unknown-gnu",
- "flags": ["set_shell_var", ],
- "var_name": "GCC5_RISCV64_INSTALL"
-}
--
2.38.1
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH v8 07/12] ArmVirtPkg: CI: Use Fedora 35 container (Linux only)
[not found] <20221130200037.1098765-1-osteffen@redhat.com>
` (5 preceding siblings ...)
2022-11-30 20:01 ` [PATCH v8 06/12] BaseTools: CI: Use Fedora 35 container (Linux only) Oliver Steffen
@ 2022-11-30 20:01 ` Oliver Steffen
2022-11-30 20:01 ` [PATCH v8 08/12] OvmfPkg: " Oliver Steffen
` (4 subsequent siblings)
11 siblings, 0 replies; 17+ messages in thread
From: Oliver Steffen @ 2022-11-30 20:01 UTC (permalink / raw)
To: devel
Cc: Andrew Fish, Ard Biesheuvel, Bob Feng, Daniel Schaefer,
Gerd Hoffmann, Jiewen Yao, Jordan Justen, Leif Lindholm,
Liming Gao, Michael D Kinney, Michael Kubacki, Ray Ni,
Sami Mujawar, Sean Brogan, Sunil V L, Yuwei Chen, Zhiguang Liu,
Pawel Polawski, Chris Fernald, Oliver Steffen
Run all Linux based jobs in a container, using a custom Fedora 35 image
(gcc 11). The image is hosted on ghcr.io and the Dockerfiles are
here: https://github.com/tianocore/containers
The version numbers of gcc, iasl, and nasm are pinned to avoid
unintended upgrades during image rebuild.
Do not run apt-get in CI jobs to install qemu and gcc dependencies.
Assume the container image provides these.
Use Python from the container image, do not download at runtime.
Signed-off-by: Oliver Steffen <osteffen@redhat.com>
---
ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 7 +++----
1 file changed, 3 insertions(+), 4 deletions(-)
diff --git a/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
b/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
index b07e3199f143..6d75502b5628 100644
--- a/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
+++ b/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
@@ -75,6 +75,8 @@ jobs:
pool:
vmImage: $(vm_image)
+ container: 'ghcr.io/tianocore/containers/fedora-35-test:2113a0e'
+
steps:
- template: ../../../.azurepipelines/templates/platform-build-run-steps.yml
parameters:
@@ -85,7 +87,4 @@ jobs:
build_file: $(Build.File)
build_flags: $(Build.Flags)
run_flags: $(Run.Flags)
- extra_install_step:
- - bash: sudo apt-get install qemu
- displayName: Install qemu
- condition: and(gt(variables.pkg_count, 0), succeeded())
+ usePythonVersion: '' # use Python from the container image
--
2.38.1
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH v8 08/12] OvmfPkg: CI: Use Fedora 35 container (Linux only)
[not found] <20221130200037.1098765-1-osteffen@redhat.com>
` (6 preceding siblings ...)
2022-11-30 20:01 ` [PATCH v8 07/12] ArmVirtPkg: " Oliver Steffen
@ 2022-11-30 20:01 ` Oliver Steffen
2022-11-30 20:01 ` [PATCH v8 09/12] EmulatorPkg: " Oliver Steffen
` (3 subsequent siblings)
11 siblings, 0 replies; 17+ messages in thread
From: Oliver Steffen @ 2022-11-30 20:01 UTC (permalink / raw)
To: devel
Cc: Andrew Fish, Ard Biesheuvel, Bob Feng, Daniel Schaefer,
Gerd Hoffmann, Jiewen Yao, Jordan Justen, Leif Lindholm,
Liming Gao, Michael D Kinney, Michael Kubacki, Ray Ni,
Sami Mujawar, Sean Brogan, Sunil V L, Yuwei Chen, Zhiguang Liu,
Pawel Polawski, Chris Fernald, Oliver Steffen
Run all Linux based jobs in a container, using a custom Fedora 35 image
(gcc 11). The image is hosted on ghcr.io and the Dockerfiles are
here: https://github.com/tianocore/containers
The version numbers of gcc, iasl, and nasm are pinned to avoid
unintended upgrades during image rebuild.
Do not run apt-get in CI jobs to install qemu and gcc dependencies.
Assume the container image provides these.
Use Python from the container image, do not download at runtime.
Signed-off-by: Oliver Steffen <osteffen@redhat.com>
---
OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 7 +++----
1 file changed, 3 insertions(+), 4 deletions(-)
diff --git a/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
b/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
index 7160d95f7e04..984782aa7d19 100644
--- a/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
+++ b/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
@@ -177,6 +177,8 @@ jobs:
pool:
vmImage: $(vm_image)
+ container: 'ghcr.io/tianocore/containers/fedora-35-test:2113a0e'
+
steps:
- template: ../../../.azurepipelines/templates/platform-build-run-steps.yml
parameters:
@@ -187,7 +189,4 @@ jobs:
build_file: $(Build.File)
build_flags: $(Build.Flags)
run_flags: $(Run.Flags)
- extra_install_step:
- - bash: sudo apt-get install qemu
- displayName: Install qemu
- condition: and(gt(variables.pkg_count, 0), succeeded())
+ usePythonVersion: '' # use Python from the container image
--
2.38.1
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH v8 09/12] EmulatorPkg: CI: Use Fedora 35 container (Linux only)
[not found] <20221130200037.1098765-1-osteffen@redhat.com>
` (7 preceding siblings ...)
2022-11-30 20:01 ` [PATCH v8 08/12] OvmfPkg: " Oliver Steffen
@ 2022-11-30 20:01 ` Oliver Steffen
2022-12-01 2:34 ` Ni, Ray
2022-11-30 20:01 ` [PATCH v8 10/12] CI: Don't install cspell Oliver Steffen
` (2 subsequent siblings)
11 siblings, 1 reply; 17+ messages in thread
From: Oliver Steffen @ 2022-11-30 20:01 UTC (permalink / raw)
To: devel
Cc: Andrew Fish, Ard Biesheuvel, Bob Feng, Daniel Schaefer,
Gerd Hoffmann, Jiewen Yao, Jordan Justen, Leif Lindholm,
Liming Gao, Michael D Kinney, Michael Kubacki, Ray Ni,
Sami Mujawar, Sean Brogan, Sunil V L, Yuwei Chen, Zhiguang Liu,
Pawel Polawski, Chris Fernald, Oliver Steffen
Run all Linux based jobs in a container, using a custom Fedora 35 image
(gcc 11). The image is hosted on ghcr.io and the Dockerfiles are
here: https://github.com/tianocore/containers
The version numbers of gcc, iasl, and nasm are pinned to avoid
unintended upgrades during image rebuild.
Use Python from the container image, do not download at runtime.
Signed-off-by: Oliver Steffen <osteffen@redhat.com>
---
EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 3 +++
1 file changed, 3 insertions(+)
diff --git a/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
b/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
index 416c15e70840..e58bb5b55245 100644
--- a/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
+++ b/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
@@ -75,6 +75,8 @@ jobs:
pool:
vmImage: $(vm_image)
+ container: 'ghcr.io/tianocore/containers/fedora-35-test:2113a0e'
+
steps:
- template: ../../../.azurepipelines/templates/platform-build-run-steps.yml
parameters:
@@ -85,3 +87,4 @@ jobs:
build_file: $(Build.File)
build_flags: $(Build.Flags)
run_flags: $(Run.Flags)
+ usePythonVersion: '' # use Python from the container image
--
2.38.1
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH v8 10/12] CI: Don't install cspell
[not found] <20221130200037.1098765-1-osteffen@redhat.com>
` (8 preceding siblings ...)
2022-11-30 20:01 ` [PATCH v8 09/12] EmulatorPkg: " Oliver Steffen
@ 2022-11-30 20:01 ` Oliver Steffen
2022-11-30 20:44 ` [edk2-devel] " Chris Fernald
2022-11-30 20:01 ` [PATCH v8 11/12] .pytool: CISettings.py: don't add scopes for GCC Oliver Steffen
2022-11-30 20:01 ` [PATCH v8 12/12] CI: use latest vm_image (Linux only) Oliver Steffen
11 siblings, 1 reply; 17+ messages in thread
From: Oliver Steffen @ 2022-11-30 20:01 UTC (permalink / raw)
To: devel
Cc: Andrew Fish, Ard Biesheuvel, Bob Feng, Daniel Schaefer,
Gerd Hoffmann, Jiewen Yao, Jordan Justen, Leif Lindholm,
Liming Gao, Michael D Kinney, Michael Kubacki, Ray Ni,
Sami Mujawar, Sean Brogan, Sunil V L, Yuwei Chen, Zhiguang Liu,
Pawel Polawski, Chris Fernald, Oliver Steffen
cspell is already included in the container image.
Signed-off-by: Oliver Steffen <osteffen@redhat.com>
---
.azurepipelines/templates/spell-check-prereq-steps.yml | 4 ----
1 file changed, 4 deletions(-)
diff --git a/.azurepipelines/templates/spell-check-prereq-steps.yml
b/.azurepipelines/templates/spell-check-prereq-steps.yml
index 226909d54c4e..66354ad78b5c 100644
--- a/.azurepipelines/templates/spell-check-prereq-steps.yml
+++ b/.azurepipelines/templates/spell-check-prereq-steps.yml
@@ -16,7 +16,3 @@ steps:
versionSpec: '14.x'
#checkLatest: false # Optional
condition: and(gt(variables.pkg_count, 0), succeeded())
-
-- script: npm install -g cspell@5.20.0
- displayName: 'Install cspell npm'
- condition: and(gt(variables.pkg_count, 0), succeeded())
--
2.38.1
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH v8 11/12] .pytool: CISettings.py: don't add scopes for GCC
[not found] <20221130200037.1098765-1-osteffen@redhat.com>
` (9 preceding siblings ...)
2022-11-30 20:01 ` [PATCH v8 10/12] CI: Don't install cspell Oliver Steffen
@ 2022-11-30 20:01 ` Oliver Steffen
2022-11-30 20:01 ` [PATCH v8 12/12] CI: use latest vm_image (Linux only) Oliver Steffen
11 siblings, 0 replies; 17+ messages in thread
From: Oliver Steffen @ 2022-11-30 20:01 UTC (permalink / raw)
To: devel
Cc: Andrew Fish, Ard Biesheuvel, Bob Feng, Daniel Schaefer,
Gerd Hoffmann, Jiewen Yao, Jordan Justen, Leif Lindholm,
Liming Gao, Michael D Kinney, Michael Kubacki, Ray Ni,
Sami Mujawar, Sean Brogan, Sunil V L, Yuwei Chen, Zhiguang Liu,
Pawel Polawski, Chris Fernald, Oliver Steffen
All ext_dep.yml files for gcc have been removed and gcc
is expected to be installed on the system (GCC5_*_PREFIX
may indicate the location).
No need to adjust the toolchain scopes for Linux builds
anymore.
Signed-off-by: Oliver Steffen <osteffen@redhat.com>
---
.pytool/CISettings.py | 9 ---------
1 file changed, 9 deletions(-)
diff --git a/.pytool/CISettings.py b/.pytool/CISettings.py
index 76ac2b09dba6..d87c8e838e61 100644
--- a/.pytool/CISettings.py
+++ b/.pytool/CISettings.py
@@ -169,15 +169,6 @@ class Settings(CiBuildSettingsManager,
UpdateSettingsManager, SetupSettingsManag
else:
logging.warning("Falling back to using in-tree BaseTools")
- if is_linux and self.ActualToolChainTag.upper().startswith("GCC"):
- if "AARCH64" in self.ActualArchitectures:
- scopes += ("gcc_aarch64_linux",)
- if "ARM" in self.ActualArchitectures:
- scopes += ("gcc_arm_linux",)
- if "RISCV64" in self.ActualArchitectures:
- scopes += ("gcc_riscv64_unknown",)
- if "LOONGARCH64" in self.ActualArchitectures:
- scopes += ("gcc_loongarch64_unknown_linux",)
self.ActualScopes = scopes
return self.ActualScopes
--
2.38.1
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH v8 12/12] CI: use latest vm_image (Linux only)
[not found] <20221130200037.1098765-1-osteffen@redhat.com>
` (10 preceding siblings ...)
2022-11-30 20:01 ` [PATCH v8 11/12] .pytool: CISettings.py: don't add scopes for GCC Oliver Steffen
@ 2022-11-30 20:01 ` Oliver Steffen
2022-11-30 20:38 ` [edk2-devel] " Michael D Kinney
11 siblings, 1 reply; 17+ messages in thread
From: Oliver Steffen @ 2022-11-30 20:01 UTC (permalink / raw)
To: devel
Cc: Andrew Fish, Ard Biesheuvel, Bob Feng, Daniel Schaefer,
Gerd Hoffmann, Jiewen Yao, Jordan Justen, Leif Lindholm,
Liming Gao, Michael D Kinney, Michael Kubacki, Ray Ni,
Sami Mujawar, Sean Brogan, Sunil V L, Yuwei Chen, Zhiguang Liu,
Pawel Polawski, Chris Fernald, Oliver Steffen
Switch over to `ubuntu-latest` as the vm_image for Linux
CI jobs (EmulatorPkg, ArmVirtPkg, OvmfPkg).
The previously used ubuntu-18.04 will not be available
after Dec 1st 2022.
Signed-off-by: Oliver Steffen <osteffen@redhat.com>
---
ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 2 +-
EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 2 +-
OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 2 +-
3 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
b/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
index 6d75502b5628..f7844229e60c 100644
--- a/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
+++ b/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
@@ -19,7 +19,7 @@ jobs:
- job: Platform_CI
variables:
package: 'ArmVirtPkg'
- vm_image: 'ubuntu-18.04'
+ vm_image: 'ubuntu-latest'
should_run: true
run_flags: "MAKE_STARTUP_NSH=TRUE QEMU_HEADLESS=TRUE"
diff --git a/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
b/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
index e58bb5b55245..3124de6634b2 100644
--- a/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
+++ b/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
@@ -19,7 +19,7 @@ jobs:
- job: Platform_CI
variables:
package: 'EmulatorPkg'
- vm_image: 'ubuntu-18.04'
+ vm_image: 'ubuntu-latest'
should_run: false
run_flags: "MAKE_STARTUP_NSH=TRUE"
diff --git a/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
b/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
index 984782aa7d19..831b0eb254b4 100644
--- a/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
+++ b/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
@@ -19,7 +19,7 @@ jobs:
- job: Platform_CI
variables:
package: 'OvmfPkg'
- vm_image: 'ubuntu-18.04'
+ vm_image: 'ubuntu-latest'
should_run: true
run_flags: "MAKE_STARTUP_NSH=TRUE QEMU_HEADLESS=TRUE"
--
2.38.1
^ permalink raw reply related [flat|nested] 17+ messages in thread
* Re: [edk2-devel] [PATCH v8 12/12] CI: use latest vm_image (Linux only)
2022-11-30 20:01 ` [PATCH v8 12/12] CI: use latest vm_image (Linux only) Oliver Steffen
@ 2022-11-30 20:38 ` Michael D Kinney
2022-11-30 21:53 ` Oliver Steffen
0 siblings, 1 reply; 17+ messages in thread
From: Michael D Kinney @ 2022-11-30 20:38 UTC (permalink / raw)
To: devel@edk2.groups.io, osteffen@redhat.com, Kinney, Michael D
Cc: Andrew Fish, Ard Biesheuvel, Feng, Bob C, Daniel Schaefer,
Gerd Hoffmann, Yao, Jiewen, Justen, Jordan L, Leif Lindholm,
Gao, Liming, Michael Kubacki, Ni, Ray, Sami Mujawar, Sean Brogan,
Sunil V L, Chen, Christine, Liu, Zhiguang, Pawel Polawski,
fernald, chris
I know we have run into issues before when using ubuntu-latest when
a change was made to the latest image that broke CI.
I would prefer to always set this to a specific version, so we can
choose to move to a newer one with a PR that can run through CI.
I am ok with selecting the highest version available that matches
current ubuntu-latest version number.
Thanks,
Mike
> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Oliver Steffen
> Sent: Wednesday, November 30, 2022 12:01 PM
> To: devel@edk2.groups.io
> Cc: Andrew Fish <afish@apple.com>; Ard Biesheuvel <ardb+tianocore@kernel.org>; Feng, Bob C <bob.c.feng@intel.com>; Daniel
> Schaefer <git@danielschaefer.me>; Gerd Hoffmann <kraxel@redhat.com>; Yao, Jiewen <jiewen.yao@intel.com>; Justen, Jordan L
> <jordan.l.justen@intel.com>; Leif Lindholm <quic_llindhol@quicinc.com>; Gao, Liming <gaoliming@byosoft.com.cn>; Kinney,
> Michael D <michael.d.kinney@intel.com>; Michael Kubacki <mikuback@linux.microsoft.com>; Ni, Ray <ray.ni@intel.com>; Sami
> Mujawar <sami.mujawar@arm.com>; Sean Brogan <sean.brogan@microsoft.com>; Sunil V L <sunilvl@ventanamicro.com>; Chen,
> Christine <yuwei.chen@intel.com>; Liu, Zhiguang <zhiguang.liu@intel.com>; Pawel Polawski <ppolawsk@redhat.com>; fernald,
> chris <chfernal@microsoft.com>; Oliver Steffen <osteffen@redhat.com>
> Subject: [edk2-devel] [PATCH v8 12/12] CI: use latest vm_image (Linux only)
>
> Switch over to `ubuntu-latest` as the vm_image for Linux
> CI jobs (EmulatorPkg, ArmVirtPkg, OvmfPkg).
> The previously used ubuntu-18.04 will not be available
> after Dec 1st 2022.
>
> Signed-off-by: Oliver Steffen <osteffen@redhat.com>
> ---
> ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 2 +-
> EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 2 +-
> OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 2 +-
> 3 files changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> b/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> index 6d75502b5628..f7844229e60c 100644
> --- a/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> +++ b/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> @@ -19,7 +19,7 @@ jobs:
> - job: Platform_CI
> variables:
> package: 'ArmVirtPkg'
> - vm_image: 'ubuntu-18.04'
> + vm_image: 'ubuntu-latest'
> should_run: true
> run_flags: "MAKE_STARTUP_NSH=TRUE QEMU_HEADLESS=TRUE"
>
> diff --git a/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> b/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> index e58bb5b55245..3124de6634b2 100644
> --- a/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> +++ b/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> @@ -19,7 +19,7 @@ jobs:
> - job: Platform_CI
> variables:
> package: 'EmulatorPkg'
> - vm_image: 'ubuntu-18.04'
> + vm_image: 'ubuntu-latest'
> should_run: false
> run_flags: "MAKE_STARTUP_NSH=TRUE"
>
> diff --git a/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> b/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> index 984782aa7d19..831b0eb254b4 100644
> --- a/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> +++ b/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> @@ -19,7 +19,7 @@ jobs:
> - job: Platform_CI
> variables:
> package: 'OvmfPkg'
> - vm_image: 'ubuntu-18.04'
> + vm_image: 'ubuntu-latest'
> should_run: true
> run_flags: "MAKE_STARTUP_NSH=TRUE QEMU_HEADLESS=TRUE"
>
> --
> 2.38.1
>
>
>
>
>
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [edk2-devel] [PATCH v8 10/12] CI: Don't install cspell
2022-11-30 20:01 ` [PATCH v8 10/12] CI: Don't install cspell Oliver Steffen
@ 2022-11-30 20:44 ` Chris Fernald
2022-11-30 21:51 ` Oliver Steffen
0 siblings, 1 reply; 17+ messages in thread
From: Chris Fernald @ 2022-11-30 20:44 UTC (permalink / raw)
To: devel, osteffen
Cc: Andrew Fish, Ard Biesheuvel, Bob Feng, Daniel Schaefer,
Gerd Hoffmann, Jiewen Yao, Jordan Justen, Leif Lindholm,
Liming Gao, Michael D Kinney, Michael Kubacki, Ray Ni,
Sami Mujawar, Sean Brogan, Sunil V L, Yuwei Chen, Zhiguang Liu,
Pawel Polawski, Chris Fernald
I apologize as I think I mislead you on this. I think we can only remove
this operation for the builds using the container. I believe Windows
builds will still require this step for the time being. The Windows
pipeline on your PR is currently failing to find cspell. Sorry for not
being clear on this originally.
On 11/30/2022 12:01 PM, Oliver Steffen wrote:
> cspell is already included in the container image.
>
> Signed-off-by: Oliver Steffen <osteffen@redhat.com>
> ---
> .azurepipelines/templates/spell-check-prereq-steps.yml | 4 ----
> 1 file changed, 4 deletions(-)
>
> diff --git a/.azurepipelines/templates/spell-check-prereq-steps.yml
> b/.azurepipelines/templates/spell-check-prereq-steps.yml
> index 226909d54c4e..66354ad78b5c 100644
> --- a/.azurepipelines/templates/spell-check-prereq-steps.yml
> +++ b/.azurepipelines/templates/spell-check-prereq-steps.yml
> @@ -16,7 +16,3 @@ steps:
> versionSpec: '14.x'
> #checkLatest: false # Optional
> condition: and(gt(variables.pkg_count, 0), succeeded())
> -
> -- script: npm install -g cspell@5.20.0
> - displayName: 'Install cspell npm'
> - condition: and(gt(variables.pkg_count, 0), succeeded())
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [edk2-devel] [PATCH v8 10/12] CI: Don't install cspell
2022-11-30 20:44 ` [edk2-devel] " Chris Fernald
@ 2022-11-30 21:51 ` Oliver Steffen
0 siblings, 0 replies; 17+ messages in thread
From: Oliver Steffen @ 2022-11-30 21:51 UTC (permalink / raw)
To: Chris Fernald
Cc: devel, Andrew Fish, Ard Biesheuvel, Bob Feng, Daniel Schaefer,
Gerd Hoffmann, Jiewen Yao, Jordan Justen, Leif Lindholm,
Liming Gao, Michael D Kinney, Michael Kubacki, Ray Ni,
Sami Mujawar, Sean Brogan, Sunil V L, Yuwei Chen, Zhiguang Liu,
Pawel Polawski, Chris Fernald
On Wed, Nov 30, 2022 at 9:45 PM Chris Fernald <chris.fernald@outlook.com> wrote:
>
> I apologize as I think I mislead you on this. I think we can only remove
> this operation for the builds using the container. I believe Windows
> builds will still require this step for the time being. The Windows
> pipeline on your PR is currently failing to find cspell. Sorry for not
> being clear on this originally.
No worries. Sorry for misunderstanding.
I'll drop this patch for now. Don't have a simple and good way to
skip this on Linux only. I also don't think this is too critical now,
the install should not do much if cspell is already there.
We can clean this up later, and track this in an issue / BZ.
- Oliver
> On 11/30/2022 12:01 PM, Oliver Steffen wrote:
> > cspell is already included in the container image.
> >
> > Signed-off-by: Oliver Steffen <osteffen@redhat.com>
> > ---
> > .azurepipelines/templates/spell-check-prereq-steps.yml | 4 ----
> > 1 file changed, 4 deletions(-)
> >
> > diff --git a/.azurepipelines/templates/spell-check-prereq-steps.yml
> > b/.azurepipelines/templates/spell-check-prereq-steps.yml
> > index 226909d54c4e..66354ad78b5c 100644
> > --- a/.azurepipelines/templates/spell-check-prereq-steps.yml
> > +++ b/.azurepipelines/templates/spell-check-prereq-steps.yml
> > @@ -16,7 +16,3 @@ steps:
> > versionSpec: '14.x'
> > #checkLatest: false # Optional
> > condition: and(gt(variables.pkg_count, 0), succeeded())
> > -
> > -- script: npm install -g cspell@5.20.0
> > - displayName: 'Install cspell npm'
> > - condition: and(gt(variables.pkg_count, 0), succeeded())
>
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [edk2-devel] [PATCH v8 12/12] CI: use latest vm_image (Linux only)
2022-11-30 20:38 ` [edk2-devel] " Michael D Kinney
@ 2022-11-30 21:53 ` Oliver Steffen
0 siblings, 0 replies; 17+ messages in thread
From: Oliver Steffen @ 2022-11-30 21:53 UTC (permalink / raw)
To: Kinney, Michael D
Cc: devel@edk2.groups.io, Andrew Fish, Ard Biesheuvel, Feng, Bob C,
Daniel Schaefer, Gerd Hoffmann, Yao, Jiewen, Justen, Jordan L,
Leif Lindholm, Gao, Liming, Michael Kubacki, Ni, Ray,
Sami Mujawar, Sean Brogan, Sunil V L, Chen, Christine,
Liu, Zhiguang, Pawel Polawski, fernald, chris
On Wed, Nov 30, 2022 at 9:39 PM Kinney, Michael D
<michael.d.kinney@intel.com> wrote:
>
> I know we have run into issues before when using ubuntu-latest when
> a change was made to the latest image that broke CI.
>
> I would prefer to always set this to a specific version, so we can
> choose to move to a newer one with a PR that can run through CI.
>
> I am ok with selecting the highest version available that matches
> current ubuntu-latest version number.
I'll set it to ubuntu-22.04.
Although the actual VM does not do much anymore except
spawning a container, which I would assume to "always" work.
But better be safe than sorry.
-Oliver
>
> Thanks,
>
> Mike
>
> > -----Original Message-----
> > From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Oliver Steffen
> > Sent: Wednesday, November 30, 2022 12:01 PM
> > To: devel@edk2.groups.io
> > Cc: Andrew Fish <afish@apple.com>; Ard Biesheuvel <ardb+tianocore@kernel.org>; Feng, Bob C <bob.c.feng@intel.com>; Daniel
> > Schaefer <git@danielschaefer.me>; Gerd Hoffmann <kraxel@redhat.com>; Yao, Jiewen <jiewen.yao@intel.com>; Justen, Jordan L
> > <jordan.l.justen@intel.com>; Leif Lindholm <quic_llindhol@quicinc.com>; Gao, Liming <gaoliming@byosoft.com.cn>; Kinney,
> > Michael D <michael.d.kinney@intel.com>; Michael Kubacki <mikuback@linux.microsoft.com>; Ni, Ray <ray.ni@intel.com>; Sami
> > Mujawar <sami.mujawar@arm.com>; Sean Brogan <sean.brogan@microsoft.com>; Sunil V L <sunilvl@ventanamicro.com>; Chen,
> > Christine <yuwei.chen@intel.com>; Liu, Zhiguang <zhiguang.liu@intel.com>; Pawel Polawski <ppolawsk@redhat.com>; fernald,
> > chris <chfernal@microsoft.com>; Oliver Steffen <osteffen@redhat.com>
> > Subject: [edk2-devel] [PATCH v8 12/12] CI: use latest vm_image (Linux only)
> >
> > Switch over to `ubuntu-latest` as the vm_image for Linux
> > CI jobs (EmulatorPkg, ArmVirtPkg, OvmfPkg).
> > The previously used ubuntu-18.04 will not be available
> > after Dec 1st 2022.
> >
> > Signed-off-by: Oliver Steffen <osteffen@redhat.com>
> > ---
> > ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 2 +-
> > EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 2 +-
> > OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 2 +-
> > 3 files changed, 3 insertions(+), 3 deletions(-)
> >
> > diff --git a/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> > b/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> > index 6d75502b5628..f7844229e60c 100644
> > --- a/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> > +++ b/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> > @@ -19,7 +19,7 @@ jobs:
> > - job: Platform_CI
> > variables:
> > package: 'ArmVirtPkg'
> > - vm_image: 'ubuntu-18.04'
> > + vm_image: 'ubuntu-latest'
> > should_run: true
> > run_flags: "MAKE_STARTUP_NSH=TRUE QEMU_HEADLESS=TRUE"
> >
> > diff --git a/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> > b/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> > index e58bb5b55245..3124de6634b2 100644
> > --- a/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> > +++ b/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> > @@ -19,7 +19,7 @@ jobs:
> > - job: Platform_CI
> > variables:
> > package: 'EmulatorPkg'
> > - vm_image: 'ubuntu-18.04'
> > + vm_image: 'ubuntu-latest'
> > should_run: false
> > run_flags: "MAKE_STARTUP_NSH=TRUE"
> >
> > diff --git a/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> > b/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> > index 984782aa7d19..831b0eb254b4 100644
> > --- a/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> > +++ b/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> > @@ -19,7 +19,7 @@ jobs:
> > - job: Platform_CI
> > variables:
> > package: 'OvmfPkg'
> > - vm_image: 'ubuntu-18.04'
> > + vm_image: 'ubuntu-latest'
> > should_run: true
> > run_flags: "MAKE_STARTUP_NSH=TRUE QEMU_HEADLESS=TRUE"
> >
> > --
> > 2.38.1
> >
> >
> >
> >
> >
>
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v8 09/12] EmulatorPkg: CI: Use Fedora 35 container (Linux only)
2022-11-30 20:01 ` [PATCH v8 09/12] EmulatorPkg: " Oliver Steffen
@ 2022-12-01 2:34 ` Ni, Ray
0 siblings, 0 replies; 17+ messages in thread
From: Ni, Ray @ 2022-12-01 2:34 UTC (permalink / raw)
To: Oliver Steffen, devel@edk2.groups.io
Cc: Andrew Fish, Ard Biesheuvel, Feng, Bob C, Daniel Schaefer,
Gerd Hoffmann, Yao, Jiewen, Justen, Jordan L, Leif Lindholm,
Gao, Liming, Kinney, Michael D, Michael Kubacki, Sami Mujawar,
Sean Brogan, Sunil V L, Chen, Christine, Liu, Zhiguang,
Pawel Polawski, fernald, chris
Acked-by: Ray Ni <ray.ni@intel.com>
> -----Original Message-----
> From: Oliver Steffen <osteffen@redhat.com>
> Sent: Thursday, December 1, 2022 4:01 AM
> To: devel@edk2.groups.io
> Cc: Andrew Fish <afish@apple.com>; Ard Biesheuvel
> <ardb+tianocore@kernel.org>; Feng, Bob C <bob.c.feng@intel.com>; Daniel
> Schaefer <git@danielschaefer.me>; Gerd Hoffmann <kraxel@redhat.com>;
> Yao, Jiewen <jiewen.yao@intel.com>; Justen, Jordan L
> <jordan.l.justen@intel.com>; Leif Lindholm <quic_llindhol@quicinc.com>;
> Gao, Liming <gaoliming@byosoft.com.cn>; Kinney, Michael D
> <michael.d.kinney@intel.com>; Michael Kubacki
> <mikuback@linux.microsoft.com>; Ni, Ray <ray.ni@intel.com>; Sami
> Mujawar <sami.mujawar@arm.com>; Sean Brogan
> <sean.brogan@microsoft.com>; Sunil V L <sunilvl@ventanamicro.com>;
> Chen, Christine <yuwei.chen@intel.com>; Liu, Zhiguang
> <zhiguang.liu@intel.com>; Pawel Polawski <ppolawsk@redhat.com>; fernald,
> chris <chfernal@microsoft.com>; Oliver Steffen <osteffen@redhat.com>
> Subject: [PATCH v8 09/12] EmulatorPkg: CI: Use Fedora 35 container (Linux
> only)
>
> Run all Linux based jobs in a container, using a custom Fedora 35 image
> (gcc 11). The image is hosted on ghcr.io and the Dockerfiles are
> here: https://github.com/tianocore/containers
> The version numbers of gcc, iasl, and nasm are pinned to avoid
> unintended upgrades during image rebuild.
>
> Use Python from the container image, do not download at runtime.
>
> Signed-off-by: Oliver Steffen <osteffen@redhat.com>
> ---
> EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 3 +++
> 1 file changed, 3 insertions(+)
>
> diff --git a/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> b/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> index 416c15e70840..e58bb5b55245 100644
> --- a/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> +++ b/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> @@ -75,6 +75,8 @@ jobs:
> pool:
> vmImage: $(vm_image)
>
> + container: 'ghcr.io/tianocore/containers/fedora-35-test:2113a0e'
> +
> steps:
> - template: ../../../.azurepipelines/templates/platform-build-run-
> steps.yml
> parameters:
> @@ -85,3 +87,4 @@ jobs:
> build_file: $(Build.File)
> build_flags: $(Build.Flags)
> run_flags: $(Run.Flags)
> + usePythonVersion: '' # use Python from the container image
> --
> 2.38.1
^ permalink raw reply [flat|nested] 17+ messages in thread
end of thread, other threads:[~2022-12-01 2:34 UTC | newest]
Thread overview: 17+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <20221130200037.1098765-1-osteffen@redhat.com>
2022-11-30 20:01 ` [PATCH v8 01/12] CI: make Python version configurable Oliver Steffen
2022-11-30 20:01 ` [PATCH v8 02/12] OvmfPkg: CI: use Python version from defaults template Oliver Steffen
2022-11-30 20:01 ` [PATCH v8 03/12] EmulatorPkg: " Oliver Steffen
2022-11-30 20:01 ` [PATCH v8 04/12] CI: add ~/.local/bin to PATH (Linux only) Oliver Steffen
2022-11-30 20:01 ` [PATCH v8 05/12] CI: Allow running in a container Oliver Steffen
2022-11-30 20:01 ` [PATCH v8 06/12] BaseTools: CI: Use Fedora 35 container (Linux only) Oliver Steffen
2022-11-30 20:01 ` [PATCH v8 07/12] ArmVirtPkg: " Oliver Steffen
2022-11-30 20:01 ` [PATCH v8 08/12] OvmfPkg: " Oliver Steffen
2022-11-30 20:01 ` [PATCH v8 09/12] EmulatorPkg: " Oliver Steffen
2022-12-01 2:34 ` Ni, Ray
2022-11-30 20:01 ` [PATCH v8 10/12] CI: Don't install cspell Oliver Steffen
2022-11-30 20:44 ` [edk2-devel] " Chris Fernald
2022-11-30 21:51 ` Oliver Steffen
2022-11-30 20:01 ` [PATCH v8 11/12] .pytool: CISettings.py: don't add scopes for GCC Oliver Steffen
2022-11-30 20:01 ` [PATCH v8 12/12] CI: use latest vm_image (Linux only) Oliver Steffen
2022-11-30 20:38 ` [edk2-devel] " Michael D Kinney
2022-11-30 21:53 ` Oliver Steffen
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox