* [PATCH 1/4] CI: don't force python verison (Linux only)
2022-04-19 7:34 [PATCH 0/4] CI: Use Fedora 35 container for Linux jobs Oliver Steffen
@ 2022-04-19 7:34 ` Oliver Steffen
0 siblings, 0 replies; 6+ messages in thread
From: Oliver Steffen @ 2022-04-19 7:34 UTC (permalink / raw)
To: osteffen
Cc: afish, ardb+tianocore, bob.c.feng, Bret.Barkelew, devel,
gaoliming, jiewen.yao, jordan.l.justen, kraxel, michael.d.kinney,
quic_llindhol, ray.ni, sami.mujawar, sean.brogan, yuwei.chen
Use the Python supplied by the container image.
Signed-off-by: Oliver Steffen <osteffen@redhat.com>
---
.azurepipelines/templates/platform-build-run-steps.yml | 1 +
.azurepipelines/templates/pr-gate-steps.yml | 1 +
2 files changed, 2 insertions(+)
diff --git a/.azurepipelines/templates/platform-build-run-steps.yml b/.azurepipelines/templates/platform-build-run-steps.yml
index 97e7faa26682..b0219bc61e67 100644
--- a/.azurepipelines/templates/platform-build-run-steps.yml
+++ b/.azurepipelines/templates/platform-build-run-steps.yml
@@ -44,6 +44,7 @@ steps:
inputs:
versionSpec: "3.8.x"
architecture: "x64"
+ condition: not(eq('${{ parameters.tool_chain_tag }}', 'GCC5'))
- script: pip install -r pip-requirements.txt --upgrade
displayName: 'Install/Upgrade pip modules'
diff --git a/.azurepipelines/templates/pr-gate-steps.yml b/.azurepipelines/templates/pr-gate-steps.yml
index 70c19a462194..f5cf5ff0376f 100644
--- a/.azurepipelines/templates/pr-gate-steps.yml
+++ b/.azurepipelines/templates/pr-gate-steps.yml
@@ -22,6 +22,7 @@ steps:
inputs:
versionSpec: '3.8.x'
architecture: 'x64'
+ condition: not(eq('${{ parameters.tool_chain_tag }}', 'GCC5'))
- script: pip install -r pip-requirements.txt --upgrade
displayName: 'Install/Upgrade pip modules'
--
2.35.1
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH 0/4] CI: Use Fedora 35 container for Linux jobs
@ 2022-04-19 7:36 Oliver Steffen
2022-04-19 7:36 ` [PATCH 1/4] CI: don't force python verison (Linux only) Oliver Steffen
` (3 more replies)
0 siblings, 4 replies; 6+ messages in thread
From: Oliver Steffen @ 2022-04-19 7:36 UTC (permalink / raw)
To: devel
Cc: afish, ardb+tianocore, bob.c.feng, Bret.Barkelew, gaoliming,
jiewen.yao, jordan.l.justen, kraxel, michael.d.kinney,
quic_llindhol, ray.ni, sami.mujawar, sean.brogan, yuwei.chen,
Oliver Steffen
PR: https://github.com/tianocore/edk2/pull/2711
Update CI, run all Linux (aka Ubuntu-GCC5) based jobs in a custom
Fedora 35 container. The image provides gcc 11 from Fedora for all
architectures. The external dependencies for gcc have been removed
so stuart does not download them. iasl and nasm are also
included in the image, but remain as ext-dep for now (CI jobs on
Windows need those).
The Dockerfiles are here:
https://github.com/osteffenrh/edk2-build-images, but they should, of
course, then move to a repo in the Tianocore group, or something like
that. The images are built automatically via GitHub Actions and then
pushed to ghcr.io.
Oliver Steffen (4):
CI: don't force python verison (Linux only)
CI: add ~/.local/bin to PATH (Linux only)
CI: Use Fedora 35 container (Linux only)
BaseTools: Remove ext. gcc dependencies (Linux only)
.../templates/basetools-build-steps.yml | 9 --------
.../templates/platform-build-run-steps.yml | 7 ++++++
.../templates/pr-gate-build-job.yml | 3 +++
.azurepipelines/templates/pr-gate-steps.yml | 7 ++++++
.../.azurepipelines/Ubuntu-GCC5.yml | 6 ++---
BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml | 21 ------------------
BaseTools/Bin/gcc_arm_linux_ext_dep.yaml | 21 ------------------
.../Bin/gcc_riscv64_unknown_ext_dep.yaml | 22 -------------------
.../.azurepipelines/Ubuntu-GCC5.yml | 2 ++
.../.azurepipelines/Ubuntu-GCC5.yml | 6 ++---
10 files changed, 23 insertions(+), 81 deletions(-)
delete mode 100644 BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml
delete mode 100644 BaseTools/Bin/gcc_arm_linux_ext_dep.yaml
delete mode 100644 BaseTools/Bin/gcc_riscv64_unknown_ext_dep.yaml
--
2.35.1
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH 1/4] CI: don't force python verison (Linux only)
2022-04-19 7:36 [PATCH 0/4] CI: Use Fedora 35 container for Linux jobs Oliver Steffen
@ 2022-04-19 7:36 ` Oliver Steffen
2022-04-19 7:36 ` [PATCH 2/4] CI: add ~/.local/bin to PATH " Oliver Steffen
` (2 subsequent siblings)
3 siblings, 0 replies; 6+ messages in thread
From: Oliver Steffen @ 2022-04-19 7:36 UTC (permalink / raw)
To: devel
Cc: afish, ardb+tianocore, bob.c.feng, Bret.Barkelew, gaoliming,
jiewen.yao, jordan.l.justen, kraxel, michael.d.kinney,
quic_llindhol, ray.ni, sami.mujawar, sean.brogan, yuwei.chen,
Oliver Steffen
Use the Python supplied by the container image.
Signed-off-by: Oliver Steffen <osteffen@redhat.com>
---
.azurepipelines/templates/platform-build-run-steps.yml | 1 +
.azurepipelines/templates/pr-gate-steps.yml | 1 +
2 files changed, 2 insertions(+)
diff --git a/.azurepipelines/templates/platform-build-run-steps.yml b/.azurepipelines/templates/platform-build-run-steps.yml
index 97e7faa26682..b0219bc61e67 100644
--- a/.azurepipelines/templates/platform-build-run-steps.yml
+++ b/.azurepipelines/templates/platform-build-run-steps.yml
@@ -44,6 +44,7 @@ steps:
inputs:
versionSpec: "3.8.x"
architecture: "x64"
+ condition: not(eq('${{ parameters.tool_chain_tag }}', 'GCC5'))
- script: pip install -r pip-requirements.txt --upgrade
displayName: 'Install/Upgrade pip modules'
diff --git a/.azurepipelines/templates/pr-gate-steps.yml b/.azurepipelines/templates/pr-gate-steps.yml
index 70c19a462194..f5cf5ff0376f 100644
--- a/.azurepipelines/templates/pr-gate-steps.yml
+++ b/.azurepipelines/templates/pr-gate-steps.yml
@@ -22,6 +22,7 @@ steps:
inputs:
versionSpec: '3.8.x'
architecture: 'x64'
+ condition: not(eq('${{ parameters.tool_chain_tag }}', 'GCC5'))
- script: pip install -r pip-requirements.txt --upgrade
displayName: 'Install/Upgrade pip modules'
--
2.35.1
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH 2/4] CI: add ~/.local/bin to PATH (Linux only)
2022-04-19 7:36 [PATCH 0/4] CI: Use Fedora 35 container for Linux jobs Oliver Steffen
2022-04-19 7:36 ` [PATCH 1/4] CI: don't force python verison (Linux only) Oliver Steffen
@ 2022-04-19 7:36 ` Oliver Steffen
2022-04-19 7:36 ` [PATCH 3/4] CI: Use Fedora 35 container " Oliver Steffen
2022-04-19 7:36 ` [PATCH 4/4] BaseTools: Remove ext. gcc dependencies " Oliver Steffen
3 siblings, 0 replies; 6+ messages in thread
From: Oliver Steffen @ 2022-04-19 7:36 UTC (permalink / raw)
To: devel
Cc: afish, ardb+tianocore, bob.c.feng, Bret.Barkelew, gaoliming,
jiewen.yao, jordan.l.justen, kraxel, michael.d.kinney,
quic_llindhol, ray.ni, sami.mujawar, sean.brogan, yuwei.chen,
Oliver Steffen
Fix pip install.
Signed-off-by: Oliver Steffen <osteffen@redhat.com>
---
.azurepipelines/templates/platform-build-run-steps.yml | 6 ++++++
.azurepipelines/templates/pr-gate-steps.yml | 6 ++++++
2 files changed, 12 insertions(+)
diff --git a/.azurepipelines/templates/platform-build-run-steps.yml b/.azurepipelines/templates/platform-build-run-steps.yml
index b0219bc61e67..9d27f49a2b40 100644
--- a/.azurepipelines/templates/platform-build-run-steps.yml
+++ b/.azurepipelines/templates/platform-build-run-steps.yml
@@ -36,6 +36,12 @@ parameters:
default: []
steps:
+- bash: |
+ echo "##vso[task.prependpath]${HOME}/.local/bin"
+ echo "new PATH=${PATH}"
+ displayName: Set PATH
+ condition: eq('${{ parameters.tool_chain_tag }}', 'GCC5')
+
- checkout: self
clean: true
fetchDepth: 1
diff --git a/.azurepipelines/templates/pr-gate-steps.yml b/.azurepipelines/templates/pr-gate-steps.yml
index f5cf5ff0376f..b78d2c4fff5d 100644
--- a/.azurepipelines/templates/pr-gate-steps.yml
+++ b/.azurepipelines/templates/pr-gate-steps.yml
@@ -14,6 +14,12 @@ parameters:
build_archs: ''
steps:
+- bash: |
+ echo "##vso[task.prependpath]${HOME}/.local/bin"
+ echo "new PATH=${PATH}"
+ displayName: Set PATH
+ condition: eq('${{ parameters.tool_chain_tag }}', 'GCC5')
+
- checkout: self
clean: true
fetchDepth: 1
--
2.35.1
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH 3/4] CI: Use Fedora 35 container (Linux only)
2022-04-19 7:36 [PATCH 0/4] CI: Use Fedora 35 container for Linux jobs Oliver Steffen
2022-04-19 7:36 ` [PATCH 1/4] CI: don't force python verison (Linux only) Oliver Steffen
2022-04-19 7:36 ` [PATCH 2/4] CI: add ~/.local/bin to PATH " Oliver Steffen
@ 2022-04-19 7:36 ` Oliver Steffen
2022-04-19 7:36 ` [PATCH 4/4] BaseTools: Remove ext. gcc dependencies " Oliver Steffen
3 siblings, 0 replies; 6+ messages in thread
From: Oliver Steffen @ 2022-04-19 7:36 UTC (permalink / raw)
To: devel
Cc: afish, ardb+tianocore, bob.c.feng, Bret.Barkelew, gaoliming,
jiewen.yao, jordan.l.justen, kraxel, michael.d.kinney,
quic_llindhol, ray.ni, sami.mujawar, sean.brogan, yuwei.chen,
Oliver Steffen
Run all Linux based jobs in a container, using a custom Fedora 35 image
(gcc 11). The image is hosted on ghcr.io and the container-files are
here: https://github.com/osteffenrh/edk2-build-images.
Do not run apt-get to install qemu and gcc dependencies. Assume the
container image provides these.
Signed-off-by: Oliver Steffen <osteffen@redhat.com>
---
.azurepipelines/templates/basetools-build-steps.yml | 9 ---------
.azurepipelines/templates/pr-gate-build-job.yml | 3 +++
ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 6 ++----
EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 2 ++
OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 6 ++----
5 files changed, 9 insertions(+), 17 deletions(-)
diff --git a/.azurepipelines/templates/basetools-build-steps.yml b/.azurepipelines/templates/basetools-build-steps.yml
index d8c108c6e212..a72758bc3395 100644
--- a/.azurepipelines/templates/basetools-build-steps.yml
+++ b/.azurepipelines/templates/basetools-build-steps.yml
@@ -10,15 +10,6 @@ parameters:
tool_chain_tag: ''
steps:
-- ${{ if contains(parameters.tool_chain_tag, 'GCC') }}:
- - bash: sudo apt-get update
- displayName: Update apt
- condition: and(gt(variables.pkg_count, 0), succeeded())
-
- - bash: sudo apt-get install gcc g++ make uuid-dev
- displayName: Install required tools
- condition: and(gt(variables.pkg_count, 0), succeeded())
-
- task: CmdLine@1
displayName: Build Base Tools from source
inputs:
diff --git a/.azurepipelines/templates/pr-gate-build-job.yml b/.azurepipelines/templates/pr-gate-build-job.yml
index 69e4c50afdbb..716664391957 100644
--- a/.azurepipelines/templates/pr-gate-build-job.yml
+++ b/.azurepipelines/templates/pr-gate-build-job.yml
@@ -63,6 +63,9 @@ jobs:
pool:
vmImage: ${{ parameters.vm_image }}
+ ${{ if contains(parameters.tool_chain_tag, 'GCC') }}
+ container: 'ghcr.io/osteffenrh/edk2-build-images/fedora-35:latest'
+
steps:
- template: pr-gate-steps.yml
parameters:
diff --git a/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml b/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
index b07e3199f143..a0a79b7108c9 100644
--- a/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
+++ b/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
@@ -75,6 +75,8 @@ jobs:
pool:
vmImage: $(vm_image)
+ container: 'ghcr.io/osteffenrh/edk2-build-images/fedora-35:latest'
+
steps:
- template: ../../../.azurepipelines/templates/platform-build-run-steps.yml
parameters:
@@ -85,7 +87,3 @@ jobs:
build_file: $(Build.File)
build_flags: $(Build.Flags)
run_flags: $(Run.Flags)
- extra_install_step:
- - bash: sudo apt-get install qemu
- displayName: Install qemu
- condition: and(gt(variables.pkg_count, 0), succeeded())
diff --git a/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml b/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
index 416c15e70840..589eea7d1985 100644
--- a/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
+++ b/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
@@ -75,6 +75,8 @@ jobs:
pool:
vmImage: $(vm_image)
+ container: 'ghcr.io/osteffenrh/edk2-build-images/fedora-35:latest'
+
steps:
- template: ../../../.azurepipelines/templates/platform-build-run-steps.yml
parameters:
diff --git a/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml b/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
index 6dec90d9fcaf..45a97cd246f7 100644
--- a/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
+++ b/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
@@ -166,6 +166,8 @@ jobs:
pool:
vmImage: $(vm_image)
+ container: 'ghcr.io/osteffenrh/edk2-build-images/fedora-35:latest'
+
steps:
- template: ../../../.azurepipelines/templates/platform-build-run-steps.yml
parameters:
@@ -176,7 +178,3 @@ jobs:
build_file: $(Build.File)
build_flags: $(Build.Flags)
run_flags: $(Run.Flags)
- extra_install_step:
- - bash: sudo apt-get install qemu
- displayName: Install qemu
- condition: and(gt(variables.pkg_count, 0), succeeded())
--
2.35.1
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH 4/4] BaseTools: Remove ext. gcc dependencies (Linux only)
2022-04-19 7:36 [PATCH 0/4] CI: Use Fedora 35 container for Linux jobs Oliver Steffen
` (2 preceding siblings ...)
2022-04-19 7:36 ` [PATCH 3/4] CI: Use Fedora 35 container " Oliver Steffen
@ 2022-04-19 7:36 ` Oliver Steffen
3 siblings, 0 replies; 6+ messages in thread
From: Oliver Steffen @ 2022-04-19 7:36 UTC (permalink / raw)
To: devel
Cc: afish, ardb+tianocore, bob.c.feng, Bret.Barkelew, gaoliming,
jiewen.yao, jordan.l.justen, kraxel, michael.d.kinney,
quic_llindhol, ray.ni, sami.mujawar, sean.brogan, yuwei.chen,
Oliver Steffen
Remove BaseTools/Bin/gcc*_linux_ext_dep.yaml to stop
downloading gcc from external locations; use the
toolchains provided by the container image instead.
The image needs to set the GCC5_*_PREFIX accordingly.
Signed-off-by: Oliver Steffen <osteffen@redhat.com>
---
BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml | 21 ------------------
BaseTools/Bin/gcc_arm_linux_ext_dep.yaml | 21 ------------------
.../Bin/gcc_riscv64_unknown_ext_dep.yaml | 22 -------------------
3 files changed, 64 deletions(-)
delete mode 100644 BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml
delete mode 100644 BaseTools/Bin/gcc_arm_linux_ext_dep.yaml
delete mode 100644 BaseTools/Bin/gcc_riscv64_unknown_ext_dep.yaml
diff --git a/BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml b/BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml
deleted file mode 100644
index ff8a9e868100..000000000000
--- a/BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml
+++ /dev/null
@@ -1,21 +0,0 @@
-## @file
-# Download GCC AARCH64 compiler from Arm's release site
-# Set shell variable GCC5_AARCH64_INSTALL to this folder
-#
-# This is only downloaded when a build activates scope gcc_aarch64_linux
-#
-# Copyright (c) Microsoft Corporation.
-# SPDX-License-Identifier: BSD-2-Clause-Patent
-##
-{
- "scope": "gcc_aarch64_linux",
- "type": "web",
- "name": "gcc_aarch64_linux",
- "source": "https://developer.arm.com/-/media/Files/downloads/gnu/11.2-2022.02/binrel/gcc-arm-11.2-2022.02-x86_64-aarch64-none-linux-gnu.tar.xz",
- "version": "11.2-2022.02",
- "sha256": "52dbac3eb71dbe0916f60a8c5ab9b7dc9b66b3ce513047baa09fae56234e53f3",
- "compression_type": "tar",
- "internal_path": "/gcc-arm-11.2-2022.02-x86_64-aarch64-none-linux-gnu/",
- "flags": ["set_shell_var", ],
- "var_name": "GCC5_AARCH64_INSTALL"
-}
diff --git a/BaseTools/Bin/gcc_arm_linux_ext_dep.yaml b/BaseTools/Bin/gcc_arm_linux_ext_dep.yaml
deleted file mode 100644
index 151cbfa4b532..000000000000
--- a/BaseTools/Bin/gcc_arm_linux_ext_dep.yaml
+++ /dev/null
@@ -1,21 +0,0 @@
-## @file
-# Download GCC ARM compiler from Arm's release site
-# Set shell variable GCC5_ARM_INSTALL to this folder
-#
-# This is only downloaded when a build activates scope gcc_arm_linux
-#
-# Copyright (c) Microsoft Corporation.
-# SPDX-License-Identifier: BSD-2-Clause-Patent
-##
-{
- "scope": "gcc_arm_linux",
- "type": "web",
- "name": "gcc_arm_linux",
- "source": "https://developer.arm.com/-/media/Files/downloads/gnu/11.2-2022.02/binrel/gcc-arm-11.2-2022.02-x86_64-arm-none-linux-gnueabihf.tar.xz",
- "version": "11.2-2022.02",
- "sha256": "c254f7199261fe76c32ef42187502839bda7efad0a66646cf739d074eff45fad",
- "compression_type": "tar",
- "internal_path": "/gcc-arm-11.2-2022.02-x86_64-arm-none-linux-gnueabihf/",
- "flags": ["set_shell_var", ],
- "var_name": "GCC5_ARM_INSTALL"
-}
diff --git a/BaseTools/Bin/gcc_riscv64_unknown_ext_dep.yaml b/BaseTools/Bin/gcc_riscv64_unknown_ext_dep.yaml
deleted file mode 100644
index 8abbcd7ba040..000000000000
--- a/BaseTools/Bin/gcc_riscv64_unknown_ext_dep.yaml
+++ /dev/null
@@ -1,22 +0,0 @@
-## @file
-# Download GCC RISCV64 compiler from RISC-V Organization release site
-# Set shell variable GCC5_RISCV64_INSTALL to this folder
-#
-# This is only downloaded when a build activates scope gcc_riscv64_unknown
-#
-# Copyright (c) Microsoft Corporation.
-# Copyright (c) 2020, Hewlett Packard Enterprise Development LP. All rights reserved.<BR>
-# SPDX-License-Identifier: BSD-2-Clause-Patent
-##
-{
- "scope": "gcc_riscv64_unknown",
- "type": "web",
- "name": "gcc_riscv64_unknown",
- "source": "https://raw.githubusercontent.com/riscv/riscv-uefi-edk2-docs/master/gcc-riscv-edk2-ci-toolchain/gcc-riscv-9.2.0-2020.04-x86_64_riscv64-unknown-gnu.tar.xz",
- "version": "9.2.0",
- "compression_type": "tar",
- "sha256": "28373643b69f0ce008273c3dc63f172aa1121952f1b9ae94d7485ac94af7f344",
- "internal_path": "/gcc-riscv-9.2.0-2020.04-x86_64_riscv64-unknown-gnu",
- "flags": ["set_shell_var", ],
- "var_name": "GCC5_RISCV64_INSTALL"
-}
--
2.35.1
^ permalink raw reply related [flat|nested] 6+ messages in thread
end of thread, other threads:[~2022-04-19 7:36 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2022-04-19 7:36 [PATCH 0/4] CI: Use Fedora 35 container for Linux jobs Oliver Steffen
2022-04-19 7:36 ` [PATCH 1/4] CI: don't force python verison (Linux only) Oliver Steffen
2022-04-19 7:36 ` [PATCH 2/4] CI: add ~/.local/bin to PATH " Oliver Steffen
2022-04-19 7:36 ` [PATCH 3/4] CI: Use Fedora 35 container " Oliver Steffen
2022-04-19 7:36 ` [PATCH 4/4] BaseTools: Remove ext. gcc dependencies " Oliver Steffen
-- strict thread matches above, loose matches on Subject: below --
2022-04-19 7:34 [PATCH 0/4] CI: Use Fedora 35 container for Linux jobs Oliver Steffen
2022-04-19 7:34 ` [PATCH 1/4] CI: don't force python verison (Linux only) Oliver Steffen
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox