* [PATCH v11 01/17] CI: make Python version configurable
2023-01-16 17:40 [PATCH v11 00/17] CI: Use Fedora 35 container for Linux jobs Oliver Steffen
@ 2023-01-16 17:40 ` Oliver Steffen
2023-01-16 17:40 ` [PATCH v11 02/17] ArmVirtPkg: CI: use Python version from defaults template Oliver Steffen
` (16 subsequent siblings)
17 siblings, 0 replies; 32+ messages in thread
From: Oliver Steffen @ 2023-01-16 17:40 UTC (permalink / raw)
To: devel
Cc: Andrew Fish, Ard Biesheuvel, Bob Feng, Dandan Bi, Gerd Hoffmann,
Jian J Wang, Jiewen Yao, Jordan Justen, Leif Lindholm, Liming Gao,
Michael D Kinney, Michael Kubacki, Ray Ni, Sami Mujawar,
Sean Brogan, Yuwei Chen, Pawel Polawski, chris.fernald,
Oliver Steffen, Michael Kubacki
Add a new parameter "usePythonVersion" to the CI job templates.
This makes it possible to specify the version of Python to use.
The default value is '', in which case Python will not be downloaded
at runtime and the one provided by the VM/container image will be used.
Additionally, add a template .azurepipelines/templates/defaults.yml,
from which the default Pyhton version string can be obtained.
Signed-off-by: Oliver Steffen <osteffen@redhat.com>
Reviewed-by: Michael Kubacki <michael.kubacki@microsoft.com>
---
.azurepipelines/Ubuntu-GCC5.yml | 9 +++++----
.azurepipelines/Windows-VS2019.yml | 4 ++++
.azurepipelines/templates/defaults.yml | 11 +++++++++++
.../templates/platform-build-run-steps.yml | 6 +++++-
.azurepipelines/templates/pr-gate-build-job.yml | 2 ++
.azurepipelines/templates/pr-gate-steps.yml | 6 ++++--
6 files changed, 31 insertions(+), 7 deletions(-)
create mode 100644 .azurepipelines/templates/defaults.yml
diff --git a/.azurepipelines/Ubuntu-GCC5.yml b/.azurepipelines/Ubuntu-GCC5.yml
index f83951eeaf86..4ed6cb601b8e 100644
--- a/.azurepipelines/Ubuntu-GCC5.yml
+++ b/.azurepipelines/Ubuntu-GCC5.yml
@@ -13,13 +13,14 @@ pr:
- master
- stable/*
+variables:
+ - template: templates/defaults.yml
+
jobs:
- template: templates/pr-gate-build-job.yml
parameters:
tool_chain_tag: 'GCC5'
vm_image: 'ubuntu-latest'
arch_list: "IA32,X64,ARM,AARCH64,RISCV64,LOONGARCH64"
- extra_install_step:
- - bash: sudo apt-get install -y lcov
- displayName: Install Code Coverage Tools
- condition: and(gt(variables.pkg_count, 0), succeeded())
+ usePythonVersion: ${{ variables.default_python_version }}
+
diff --git a/.azurepipelines/Windows-VS2019.yml b/.azurepipelines/Windows-VS2019.yml
index c07e5bb43424..58bb98d42b28 100644
--- a/.azurepipelines/Windows-VS2019.yml
+++ b/.azurepipelines/Windows-VS2019.yml
@@ -12,12 +12,16 @@ pr:
- master
- stable/*
+variables:
+ - template: templates/defaults.yml
+
jobs:
- template: templates/pr-gate-build-job.yml
parameters:
tool_chain_tag: 'VS2019'
vm_image: 'windows-2019'
arch_list: "IA32,X64"
+ usePythonVersion: ${{ variables.default_python_version }}
extra_install_step:
- powershell: choco install opencppcoverage; Write-Host "##vso[task.prependpath]C:\Program Files\OpenCppCoverage"
displayName: Install Code Coverage Tool
diff --git a/.azurepipelines/templates/defaults.yml b/.azurepipelines/templates/defaults.yml
new file mode 100644
index 000000000000..b4909448bdea
--- /dev/null
+++ b/.azurepipelines/templates/defaults.yml
@@ -0,0 +1,11 @@
+## @file
+# File templates/default.yml
+#
+# template file containing common default values
+#
+# Copyright (c) Red Hat, Inc.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+
+variables:
+ default_python_version: ">=3.10.6"
diff --git a/.azurepipelines/templates/platform-build-run-steps.yml b/.azurepipelines/templates/platform-build-run-steps.yml
index 40a31a509fc5..8803d80cf51c 100644
--- a/.azurepipelines/templates/platform-build-run-steps.yml
+++ b/.azurepipelines/templates/platform-build-run-steps.yml
@@ -34,6 +34,9 @@ parameters:
- name: extra_install_step
type: stepList
default: []
+- name: usePythonVersion
+ type: string
+ default: ''
steps:
- checkout: self
@@ -42,8 +45,9 @@ steps:
- task: UsePythonVersion@0
inputs:
- versionSpec: ">=3.10.6"
+ versionSpec: ${{ parameters.usePythonVersion }}
architecture: "x64"
+ condition: ne('${{ parameters.usePythonVersion }}', '')
- script: pip install -r pip-requirements.txt --upgrade
displayName: 'Install/Upgrade pip modules'
diff --git a/.azurepipelines/templates/pr-gate-build-job.yml b/.azurepipelines/templates/pr-gate-build-job.yml
index fff61a3193b0..b8573b90da1c 100644
--- a/.azurepipelines/templates/pr-gate-build-job.yml
+++ b/.azurepipelines/templates/pr-gate-build-job.yml
@@ -13,6 +13,7 @@ parameters:
vm_image: ''
arch_list: ''
extra_install_step: []
+ usePythonVersion: ''
# Build step
jobs:
@@ -78,6 +79,7 @@ jobs:
build_pkgs: $(Build.Pkgs)
build_targets: $(Build.Targets)
build_archs: ${{ parameters.arch_list }}
+ usePythonVersion: ${{ parameters.usePythonVersion }}
extra_install_step: ${{ parameters.extra_install_step }}
- job: Build_${{ parameters.tool_chain_tag }}_TARGET_CODE_COVERAGE
diff --git a/.azurepipelines/templates/pr-gate-steps.yml b/.azurepipelines/templates/pr-gate-steps.yml
index 080f60aea6ab..ebc1e86c37d4 100644
--- a/.azurepipelines/templates/pr-gate-steps.yml
+++ b/.azurepipelines/templates/pr-gate-steps.yml
@@ -12,6 +12,7 @@ parameters:
build_pkgs: ''
build_targets: ''
build_archs: ''
+ usePythonVersion: ''
extra_install_step: []
steps:
@@ -21,8 +22,9 @@ steps:
- task: UsePythonVersion@0
inputs:
- versionSpec: '>=3.10.6'
- architecture: 'x64'
+ versionSpec: ${{ parameters.usePythonVersion }}
+ architecture: "x64"
+ condition: ne('${{ parameters.usePythonVersion }}', '')
- script: pip install -r pip-requirements.txt --upgrade
displayName: 'Install/Upgrade pip modules'
--
2.39.0
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v11 02/17] ArmVirtPkg: CI: use Python version from defaults template
2023-01-16 17:40 [PATCH v11 00/17] CI: Use Fedora 35 container for Linux jobs Oliver Steffen
2023-01-16 17:40 ` [PATCH v11 01/17] CI: make Python version configurable Oliver Steffen
@ 2023-01-16 17:40 ` Oliver Steffen
2023-01-16 17:40 ` [PATCH v11 03/17] EmulatorPkg: " Oliver Steffen
` (15 subsequent siblings)
17 siblings, 0 replies; 32+ messages in thread
From: Oliver Steffen @ 2023-01-16 17:40 UTC (permalink / raw)
To: devel
Cc: Andrew Fish, Ard Biesheuvel, Bob Feng, Dandan Bi, Gerd Hoffmann,
Jian J Wang, Jiewen Yao, Jordan Justen, Leif Lindholm, Liming Gao,
Michael D Kinney, Michael Kubacki, Ray Ni, Sami Mujawar,
Sean Brogan, Yuwei Chen, Pawel Polawski, chris.fernald,
Oliver Steffen, Michael Kubacki
Use the default Python version from the defaults template
(.azurepipelines/templates/defaults.yml) in the Windows and
Linux CI jobs.
Previous changes to the CI job templates make it necessary
to specify a version number, if Python shall be pulled
at CI runtime.
Signed-off-by: Oliver Steffen <osteffen@redhat.com>
Reviewed-by: Michael Kubacki <michael.kubacki@microsoft.com>
---
ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml b/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
index b07e3199f143..5a0e589ed4a7 100644
--- a/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
+++ b/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
@@ -15,6 +15,9 @@ pr:
- master
- stable/*
+variables:
+ - template: ../../../.azurepipelines/templates/defaults.yml
+
jobs:
- job: Platform_CI
variables:
@@ -85,6 +88,7 @@ jobs:
build_file: $(Build.File)
build_flags: $(Build.Flags)
run_flags: $(Run.Flags)
+ usePythonVersion: ${{ variables.default_python_version }}
extra_install_step:
- bash: sudo apt-get install qemu
displayName: Install qemu
--
2.39.0
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v11 03/17] EmulatorPkg: CI: use Python version from defaults template
2023-01-16 17:40 [PATCH v11 00/17] CI: Use Fedora 35 container for Linux jobs Oliver Steffen
2023-01-16 17:40 ` [PATCH v11 01/17] CI: make Python version configurable Oliver Steffen
2023-01-16 17:40 ` [PATCH v11 02/17] ArmVirtPkg: CI: use Python version from defaults template Oliver Steffen
@ 2023-01-16 17:40 ` Oliver Steffen
2023-01-16 17:40 ` [PATCH v11 04/17] OvmfPkg: " Oliver Steffen
` (14 subsequent siblings)
17 siblings, 0 replies; 32+ messages in thread
From: Oliver Steffen @ 2023-01-16 17:40 UTC (permalink / raw)
To: devel
Cc: Andrew Fish, Ard Biesheuvel, Bob Feng, Dandan Bi, Gerd Hoffmann,
Jian J Wang, Jiewen Yao, Jordan Justen, Leif Lindholm, Liming Gao,
Michael D Kinney, Michael Kubacki, Ray Ni, Sami Mujawar,
Sean Brogan, Yuwei Chen, Pawel Polawski, chris.fernald,
Oliver Steffen, Michael Kubacki
Use the default Python version from the defaults template
(.azurepipelines/templates/defaults.yml) in the Windows and
Linux CI jobs.
Previous changes to the CI job templates make it necessary
to specify a version number, if Python shall be pulled
at CI runtime.
Signed-off-by: Oliver Steffen <osteffen@redhat.com>
Reviewed-by: Michael Kubacki <michael.kubacki@microsoft.com>
---
EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 5 +++++
EmulatorPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml | 4 ++++
2 files changed, 9 insertions(+)
diff --git a/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml b/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
index 416c15e70840..a32c57d4aab4 100644
--- a/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
+++ b/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
@@ -15,6 +15,10 @@ trigger:
pr:
- master
- stable/*
+
+variables:
+ - template: ../../../.azurepipelines/templates/defaults.yml
+
jobs:
- job: Platform_CI
variables:
@@ -85,3 +89,4 @@ jobs:
build_file: $(Build.File)
build_flags: $(Build.Flags)
run_flags: $(Run.Flags)
+ usePythonVersion: ${{ variables.default_python_version }}
diff --git a/EmulatorPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml b/EmulatorPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml
index e7ead06ae266..09960e7c7a6e 100644
--- a/EmulatorPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml
+++ b/EmulatorPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml
@@ -16,6 +16,9 @@ pr:
- master
- stable/*
+variables:
+ - template: ../../../.azurepipelines/templates/defaults.yml
+
jobs:
- job: Platform_CI
variables:
@@ -128,3 +131,4 @@ jobs:
build_file: $(Build.File)
build_flags: $(Build.Flags)
run_flags: $(Run.Flags)
+ usePythonVersion: ${{ variables.default_python_version }}
--
2.39.0
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v11 04/17] OvmfPkg: CI: use Python version from defaults template
2023-01-16 17:40 [PATCH v11 00/17] CI: Use Fedora 35 container for Linux jobs Oliver Steffen
` (2 preceding siblings ...)
2023-01-16 17:40 ` [PATCH v11 03/17] EmulatorPkg: " Oliver Steffen
@ 2023-01-16 17:40 ` Oliver Steffen
2023-01-17 0:14 ` [edk2-devel] " Yao, Jiewen
2023-01-16 17:40 ` [PATCH v11 05/17] CI: add ~/.local/bin to PATH (Linux only) Oliver Steffen
` (13 subsequent siblings)
17 siblings, 1 reply; 32+ messages in thread
From: Oliver Steffen @ 2023-01-16 17:40 UTC (permalink / raw)
To: devel
Cc: Andrew Fish, Ard Biesheuvel, Bob Feng, Dandan Bi, Gerd Hoffmann,
Jian J Wang, Jiewen Yao, Jordan Justen, Leif Lindholm, Liming Gao,
Michael D Kinney, Michael Kubacki, Ray Ni, Sami Mujawar,
Sean Brogan, Yuwei Chen, Pawel Polawski, chris.fernald,
Oliver Steffen, Michael Kubacki
Use the default Python version from the defaults template
(.azurepipelines/templates/defaults.yml) in the Windows and
Linux CI jobs.
Previous changes to the CI job templates make it necessary
to specify a version number, if Python shall be pulled
at CI runtime.
Signed-off-by: Oliver Steffen <osteffen@redhat.com>
Reviewed-by: Michael Kubacki <michael.kubacki@microsoft.com>
---
OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 4 ++++
OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml | 5 +++++
2 files changed, 9 insertions(+)
diff --git a/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml b/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
index 7160d95f7e04..6dd90711ac70 100644
--- a/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
+++ b/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
@@ -15,6 +15,9 @@ pr:
- master
- stable/*
+variables:
+ - template: ../../../.azurepipelines/templates/defaults.yml
+
jobs:
- job: Platform_CI
variables:
@@ -187,6 +190,7 @@ jobs:
build_file: $(Build.File)
build_flags: $(Build.Flags)
run_flags: $(Run.Flags)
+ usePythonVersion: ${{ variables.default_python_version }}
extra_install_step:
- bash: sudo apt-get install qemu
displayName: Install qemu
diff --git a/OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml b/OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml
index 7d6344d6383d..7e63f419b26b 100644
--- a/OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml
+++ b/OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml
@@ -14,6 +14,10 @@ trigger:
pr:
- master
- stable/*
+
+variables:
+ - template: ../../../.azurepipelines/templates/defaults.yml
+
jobs:
- job: Platform_CI
variables:
@@ -133,6 +137,7 @@ jobs:
build_file: $(Build.File)
build_flags: $(Build.Flags)
run_flags: $(Run.Flags)
+ usePythonVersion: ${{ variables.default_python_version }}
extra_install_step:
- powershell: choco install qemu --version=2021.5.5; Write-Host "##vso[task.prependpath]c:\Program Files\qemu"
displayName: Install QEMU and Set QEMU on path # friendly name displayed in the UI
--
2.39.0
^ permalink raw reply related [flat|nested] 32+ messages in thread* Re: [edk2-devel] [PATCH v11 04/17] OvmfPkg: CI: use Python version from defaults template
2023-01-16 17:40 ` [PATCH v11 04/17] OvmfPkg: " Oliver Steffen
@ 2023-01-17 0:14 ` Yao, Jiewen
0 siblings, 0 replies; 32+ messages in thread
From: Yao, Jiewen @ 2023-01-17 0:14 UTC (permalink / raw)
To: devel@edk2.groups.io, osteffen@redhat.com
Cc: Andrew Fish, Ard Biesheuvel, Feng, Bob C, Bi, Dandan,
Gerd Hoffmann, Wang, Jian J, Justen, Jordan L, Leif Lindholm,
Gao, Liming, Kinney, Michael D, Michael Kubacki, Ni, Ray,
Sami Mujawar, Sean Brogan, Chen, Christine, Pawel Polawski,
chris.fernald@outlook.com, Kubacki, Michael
Acked-by: Jiewen Yao <Jiewen.yao@intel.com>
> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Oliver
> Steffen
> Sent: Tuesday, January 17, 2023 1:41 AM
> To: devel@edk2.groups.io
> Cc: Andrew Fish <afish@apple.com>; Ard Biesheuvel
> <ardb+tianocore@kernel.org>; Feng, Bob C <bob.c.feng@intel.com>; Bi,
> Dandan <dandan.bi@intel.com>; Gerd Hoffmann <kraxel@redhat.com>;
> Wang, Jian J <jian.j.wang@intel.com>; Yao, Jiewen <jiewen.yao@intel.com>;
> Justen, Jordan L <jordan.l.justen@intel.com>; Leif Lindholm
> <quic_llindhol@quicinc.com>; Gao, Liming <gaoliming@byosoft.com.cn>;
> Kinney, Michael D <michael.d.kinney@intel.com>; Michael Kubacki
> <mikuback@linux.microsoft.com>; Ni, Ray <ray.ni@intel.com>; Sami
> Mujawar <sami.mujawar@arm.com>; Sean Brogan
> <sean.brogan@microsoft.com>; Chen, Christine <yuwei.chen@intel.com>;
> Pawel Polawski <ppolawsk@redhat.com>; chris.fernald@outlook.com; Oliver
> Steffen <osteffen@redhat.com>; Kubacki, Michael
> <michael.kubacki@microsoft.com>
> Subject: [edk2-devel] [PATCH v11 04/17] OvmfPkg: CI: use Python version
> from defaults template
>
> Use the default Python version from the defaults template
> (.azurepipelines/templates/defaults.yml) in the Windows and
> Linux CI jobs.
>
> Previous changes to the CI job templates make it necessary
> to specify a version number, if Python shall be pulled
> at CI runtime.
>
> Signed-off-by: Oliver Steffen <osteffen@redhat.com>
> Reviewed-by: Michael Kubacki <michael.kubacki@microsoft.com>
> ---
> OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 4 ++++
> OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml | 5 +++++
> 2 files changed, 9 insertions(+)
>
> diff --git a/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> b/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> index 7160d95f7e04..6dd90711ac70 100644
> --- a/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> +++ b/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> @@ -15,6 +15,9 @@ pr:
> - master
> - stable/*
>
> +variables:
> + - template: ../../../.azurepipelines/templates/defaults.yml
> +
> jobs:
> - job: Platform_CI
> variables:
> @@ -187,6 +190,7 @@ jobs:
> build_file: $(Build.File)
> build_flags: $(Build.Flags)
> run_flags: $(Run.Flags)
> + usePythonVersion: ${{ variables.default_python_version }}
> extra_install_step:
> - bash: sudo apt-get install qemu
> displayName: Install qemu
> diff --git a/OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml
> b/OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml
> index 7d6344d6383d..7e63f419b26b 100644
> --- a/OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml
> +++ b/OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml
> @@ -14,6 +14,10 @@ trigger:
> pr:
> - master
> - stable/*
> +
> +variables:
> + - template: ../../../.azurepipelines/templates/defaults.yml
> +
> jobs:
> - job: Platform_CI
> variables:
> @@ -133,6 +137,7 @@ jobs:
> build_file: $(Build.File)
> build_flags: $(Build.Flags)
> run_flags: $(Run.Flags)
> + usePythonVersion: ${{ variables.default_python_version }}
> extra_install_step:
> - powershell: choco install qemu --version=2021.5.5; Write-Host
> "##vso[task.prependpath]c:\Program Files\qemu"
> displayName: Install QEMU and Set QEMU on path # friendly name
> displayed in the UI
> --
> 2.39.0
>
>
>
>
>
^ permalink raw reply [flat|nested] 32+ messages in thread
* [PATCH v11 05/17] CI: add ~/.local/bin to PATH (Linux only)
2023-01-16 17:40 [PATCH v11 00/17] CI: Use Fedora 35 container for Linux jobs Oliver Steffen
` (3 preceding siblings ...)
2023-01-16 17:40 ` [PATCH v11 04/17] OvmfPkg: " Oliver Steffen
@ 2023-01-16 17:40 ` Oliver Steffen
2023-01-16 17:40 ` [PATCH v11 06/17] CI: Allow running in a container Oliver Steffen
` (12 subsequent siblings)
17 siblings, 0 replies; 32+ messages in thread
From: Oliver Steffen @ 2023-01-16 17:40 UTC (permalink / raw)
To: devel
Cc: Andrew Fish, Ard Biesheuvel, Bob Feng, Dandan Bi, Gerd Hoffmann,
Jian J Wang, Jiewen Yao, Jordan Justen, Leif Lindholm, Liming Gao,
Michael D Kinney, Michael Kubacki, Ray Ni, Sami Mujawar,
Sean Brogan, Yuwei Chen, Pawel Polawski, chris.fernald,
Oliver Steffen, Michael Kubacki
Without adding ~/.local/bin to PATH, `pip install` will throw
an error when running inside a container.
Containers will be introduced to the CI in the following commits.
Signed-off-by: Oliver Steffen <osteffen@redhat.com>
Reviewed-by: Michael Kubacki <michael.kubacki@microsoft.com>
---
.azurepipelines/templates/platform-build-run-steps.yml | 6 ++++++
.azurepipelines/templates/pr-gate-steps.yml | 6 ++++++
2 files changed, 12 insertions(+)
diff --git a/.azurepipelines/templates/platform-build-run-steps.yml b/.azurepipelines/templates/platform-build-run-steps.yml
index 8803d80cf51c..087f460d7fec 100644
--- a/.azurepipelines/templates/platform-build-run-steps.yml
+++ b/.azurepipelines/templates/platform-build-run-steps.yml
@@ -39,6 +39,12 @@ parameters:
default: ''
steps:
+- bash: |
+ echo "##vso[task.prependpath]${HOME}/.local/bin"
+ echo "new PATH=${PATH}"
+ displayName: Set PATH
+ condition: eq('${{ parameters.tool_chain_tag }}', 'GCC5')
+
- checkout: self
clean: true
fetchDepth: 1
diff --git a/.azurepipelines/templates/pr-gate-steps.yml b/.azurepipelines/templates/pr-gate-steps.yml
index ebc1e86c37d4..78b2b2c8d934 100644
--- a/.azurepipelines/templates/pr-gate-steps.yml
+++ b/.azurepipelines/templates/pr-gate-steps.yml
@@ -16,6 +16,12 @@ parameters:
extra_install_step: []
steps:
+- bash: |
+ echo "##vso[task.prependpath]${HOME}/.local/bin"
+ echo "new PATH=${PATH}"
+ displayName: Set PATH
+ condition: eq('${{ parameters.tool_chain_tag }}', 'GCC5')
+
- checkout: self
clean: true
fetchDepth: 1
--
2.39.0
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v11 06/17] CI: Allow running in a container.
2023-01-16 17:40 [PATCH v11 00/17] CI: Use Fedora 35 container for Linux jobs Oliver Steffen
` (4 preceding siblings ...)
2023-01-16 17:40 ` [PATCH v11 05/17] CI: add ~/.local/bin to PATH (Linux only) Oliver Steffen
@ 2023-01-16 17:40 ` Oliver Steffen
2023-01-16 17:40 ` [PATCH v11 07/17] CI: Use Fedora 35 container (Linux only) Oliver Steffen
` (11 subsequent siblings)
17 siblings, 0 replies; 32+ messages in thread
From: Oliver Steffen @ 2023-01-16 17:40 UTC (permalink / raw)
To: devel
Cc: Andrew Fish, Ard Biesheuvel, Bob Feng, Dandan Bi, Gerd Hoffmann,
Jian J Wang, Jiewen Yao, Jordan Justen, Leif Lindholm, Liming Gao,
Michael D Kinney, Michael Kubacki, Ray Ni, Sami Mujawar,
Sean Brogan, Yuwei Chen, Pawel Polawski, chris.fernald,
Oliver Steffen, Michael Kubacki
Add a parameter of the pr-gate-build-job template to specify a
container image URL. If the value is not '' (default), then the
jobs will be run inside a container based on that image.
Signed-off-by: Oliver Steffen <osteffen@redhat.com>
Reviewed-by: Michael Kubacki <michael.kubacki@microsoft.com>
---
.azurepipelines/templates/pr-gate-build-job.yml | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/.azurepipelines/templates/pr-gate-build-job.yml b/.azurepipelines/templates/pr-gate-build-job.yml
index b8573b90da1c..3999bb166813 100644
--- a/.azurepipelines/templates/pr-gate-build-job.yml
+++ b/.azurepipelines/templates/pr-gate-build-job.yml
@@ -14,6 +14,7 @@ parameters:
arch_list: ''
extra_install_step: []
usePythonVersion: ''
+ container: ''
# Build step
jobs:
@@ -72,6 +73,9 @@ jobs:
pool:
vmImage: ${{ parameters.vm_image }}
+ ${{ if not(eq(parameters.container, '')) }}:
+ container: ${{ parameters.container }}
+
steps:
- template: pr-gate-steps.yml
parameters:
--
2.39.0
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v11 07/17] CI: Use Fedora 35 container (Linux only)
2023-01-16 17:40 [PATCH v11 00/17] CI: Use Fedora 35 container for Linux jobs Oliver Steffen
` (5 preceding siblings ...)
2023-01-16 17:40 ` [PATCH v11 06/17] CI: Allow running in a container Oliver Steffen
@ 2023-01-16 17:40 ` Oliver Steffen
2023-01-17 17:27 ` Michael Kubacki
2023-01-16 17:40 ` [PATCH v11 08/17] ArmVirtPkg: " Oliver Steffen
` (10 subsequent siblings)
17 siblings, 1 reply; 32+ messages in thread
From: Oliver Steffen @ 2023-01-16 17:40 UTC (permalink / raw)
To: devel
Cc: Andrew Fish, Ard Biesheuvel, Bob Feng, Dandan Bi, Gerd Hoffmann,
Jian J Wang, Jiewen Yao, Jordan Justen, Leif Lindholm, Liming Gao,
Michael D Kinney, Michael Kubacki, Ray Ni, Sami Mujawar,
Sean Brogan, Yuwei Chen, Pawel Polawski, chris.fernald,
Oliver Steffen
Run all Linux based jobs in a container, using a custom Fedora 35 image
(gcc 11). The image URL specified in the defaults.yml template, so that
all CI jobs can use it. The image is hosted on ghcr.io and the
Dockerfiles are here: https://github.com/tianocore/containers The
version numbers of gcc, iasl, and nasm are pinned to avoid unintended
upgrades during image rebuild.
Do not run apt-get in CI jobs to install qemu and gcc dependencies.
Assume the container image provides these.
Use Python from the container image, do not download at runtime.
Signed-off-by: Oliver Steffen <osteffen@redhat.com>
---
.azurepipelines/Ubuntu-GCC5.yml | 3 ++-
.azurepipelines/templates/basetools-build-steps.yml | 9 ---------
.azurepipelines/templates/defaults.yml | 1 +
3 files changed, 3 insertions(+), 10 deletions(-)
diff --git a/.azurepipelines/Ubuntu-GCC5.yml b/.azurepipelines/Ubuntu-GCC5.yml
index 4ed6cb601b8e..246e2d272e20 100644
--- a/.azurepipelines/Ubuntu-GCC5.yml
+++ b/.azurepipelines/Ubuntu-GCC5.yml
@@ -21,6 +21,7 @@ jobs:
parameters:
tool_chain_tag: 'GCC5'
vm_image: 'ubuntu-latest'
+ container: ${{ variables.default_linux_image }}
arch_list: "IA32,X64,ARM,AARCH64,RISCV64,LOONGARCH64"
- usePythonVersion: ${{ variables.default_python_version }}
+ usePythonVersion: '' # use Python from the container image
diff --git a/.azurepipelines/templates/basetools-build-steps.yml b/.azurepipelines/templates/basetools-build-steps.yml
index d8c108c6e212..a72758bc3395 100644
--- a/.azurepipelines/templates/basetools-build-steps.yml
+++ b/.azurepipelines/templates/basetools-build-steps.yml
@@ -10,15 +10,6 @@ parameters:
tool_chain_tag: ''
steps:
-- ${{ if contains(parameters.tool_chain_tag, 'GCC') }}:
- - bash: sudo apt-get update
- displayName: Update apt
- condition: and(gt(variables.pkg_count, 0), succeeded())
-
- - bash: sudo apt-get install gcc g++ make uuid-dev
- displayName: Install required tools
- condition: and(gt(variables.pkg_count, 0), succeeded())
-
- task: CmdLine@1
displayName: Build Base Tools from source
inputs:
diff --git a/.azurepipelines/templates/defaults.yml b/.azurepipelines/templates/defaults.yml
index b4909448bdea..74d6b417839d 100644
--- a/.azurepipelines/templates/defaults.yml
+++ b/.azurepipelines/templates/defaults.yml
@@ -9,3 +9,4 @@
variables:
default_python_version: ">=3.10.6"
+ default_linux_image: "ghcr.io/tianocore/containers/fedora-35-test:47addc9"
--
2.39.0
^ permalink raw reply related [flat|nested] 32+ messages in thread* Re: [PATCH v11 07/17] CI: Use Fedora 35 container (Linux only)
2023-01-16 17:40 ` [PATCH v11 07/17] CI: Use Fedora 35 container (Linux only) Oliver Steffen
@ 2023-01-17 17:27 ` Michael Kubacki
0 siblings, 0 replies; 32+ messages in thread
From: Michael Kubacki @ 2023-01-17 17:27 UTC (permalink / raw)
To: Oliver Steffen, devel
Cc: Andrew Fish, Ard Biesheuvel, Bob Feng, Dandan Bi, Gerd Hoffmann,
Jian J Wang, Jiewen Yao, Jordan Justen, Leif Lindholm, Liming Gao,
Michael D Kinney, Ray Ni, Sami Mujawar, Sean Brogan, Yuwei Chen,
Pawel Polawski, chris.fernald
Reviewed-by: Michael Kubacki <michael.kubacki@microsoft.com>
On 1/16/2023 12:40 PM, Oliver Steffen wrote:
> Run all Linux based jobs in a container, using a custom Fedora 35 image
> (gcc 11). The image URL specified in the defaults.yml template, so that
> all CI jobs can use it. The image is hosted on ghcr.io and the
> Dockerfiles are here: https://github.com/tianocore/containers The
> version numbers of gcc, iasl, and nasm are pinned to avoid unintended
> upgrades during image rebuild.
>
> Do not run apt-get in CI jobs to install qemu and gcc dependencies.
> Assume the container image provides these.
>
> Use Python from the container image, do not download at runtime.
>
> Signed-off-by: Oliver Steffen <osteffen@redhat.com>
> ---
> .azurepipelines/Ubuntu-GCC5.yml | 3 ++-
> .azurepipelines/templates/basetools-build-steps.yml | 9 ---------
> .azurepipelines/templates/defaults.yml | 1 +
> 3 files changed, 3 insertions(+), 10 deletions(-)
>
> diff --git a/.azurepipelines/Ubuntu-GCC5.yml b/.azurepipelines/Ubuntu-GCC5.yml
> index 4ed6cb601b8e..246e2d272e20 100644
> --- a/.azurepipelines/Ubuntu-GCC5.yml
> +++ b/.azurepipelines/Ubuntu-GCC5.yml
> @@ -21,6 +21,7 @@ jobs:
> parameters:
> tool_chain_tag: 'GCC5'
> vm_image: 'ubuntu-latest'
> + container: ${{ variables.default_linux_image }}
> arch_list: "IA32,X64,ARM,AARCH64,RISCV64,LOONGARCH64"
> - usePythonVersion: ${{ variables.default_python_version }}
> + usePythonVersion: '' # use Python from the container image
>
> diff --git a/.azurepipelines/templates/basetools-build-steps.yml b/.azurepipelines/templates/basetools-build-steps.yml
> index d8c108c6e212..a72758bc3395 100644
> --- a/.azurepipelines/templates/basetools-build-steps.yml
> +++ b/.azurepipelines/templates/basetools-build-steps.yml
> @@ -10,15 +10,6 @@ parameters:
> tool_chain_tag: ''
>
> steps:
> -- ${{ if contains(parameters.tool_chain_tag, 'GCC') }}:
> - - bash: sudo apt-get update
> - displayName: Update apt
> - condition: and(gt(variables.pkg_count, 0), succeeded())
> -
> - - bash: sudo apt-get install gcc g++ make uuid-dev
> - displayName: Install required tools
> - condition: and(gt(variables.pkg_count, 0), succeeded())
> -
> - task: CmdLine@1
> displayName: Build Base Tools from source
> inputs:
> diff --git a/.azurepipelines/templates/defaults.yml b/.azurepipelines/templates/defaults.yml
> index b4909448bdea..74d6b417839d 100644
> --- a/.azurepipelines/templates/defaults.yml
> +++ b/.azurepipelines/templates/defaults.yml
> @@ -9,3 +9,4 @@
>
> variables:
> default_python_version: ">=3.10.6"
> + default_linux_image: "ghcr.io/tianocore/containers/fedora-35-test:47addc9"
^ permalink raw reply [flat|nested] 32+ messages in thread
* [PATCH v11 08/17] ArmVirtPkg: CI: Use Fedora 35 container (Linux only)
2023-01-16 17:40 [PATCH v11 00/17] CI: Use Fedora 35 container for Linux jobs Oliver Steffen
` (6 preceding siblings ...)
2023-01-16 17:40 ` [PATCH v11 07/17] CI: Use Fedora 35 container (Linux only) Oliver Steffen
@ 2023-01-16 17:40 ` Oliver Steffen
2023-01-17 17:27 ` Michael Kubacki
2023-01-16 17:40 ` [PATCH v11 09/17] EmulatorPkg: " Oliver Steffen
` (9 subsequent siblings)
17 siblings, 1 reply; 32+ messages in thread
From: Oliver Steffen @ 2023-01-16 17:40 UTC (permalink / raw)
To: devel
Cc: Andrew Fish, Ard Biesheuvel, Bob Feng, Dandan Bi, Gerd Hoffmann,
Jian J Wang, Jiewen Yao, Jordan Justen, Leif Lindholm, Liming Gao,
Michael D Kinney, Michael Kubacki, Ray Ni, Sami Mujawar,
Sean Brogan, Yuwei Chen, Pawel Polawski, chris.fernald,
Oliver Steffen
Run the Linux jobs of the ArmVirtPkg platform CI inside a container,
in the same way the general CI does now. Make use of the default image
specified in the defaults.yml template.
Do not run apt-get in CI jobs to install qemu and gcc dependencies.
Assume the container image provides these.
Use Python from the container image, do not download at runtime.
Signed-off-by: Oliver Steffen <osteffen@redhat.com>
---
ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 8 +++-----
1 file changed, 3 insertions(+), 5 deletions(-)
diff --git a/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml b/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
index 5a0e589ed4a7..c0a2e0f20d4f 100644
--- a/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
+++ b/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
@@ -78,6 +78,8 @@ jobs:
pool:
vmImage: $(vm_image)
+ container: ${{ variables.default_linux_image }}
+
steps:
- template: ../../../.azurepipelines/templates/platform-build-run-steps.yml
parameters:
@@ -88,8 +90,4 @@ jobs:
build_file: $(Build.File)
build_flags: $(Build.Flags)
run_flags: $(Run.Flags)
- usePythonVersion: ${{ variables.default_python_version }}
- extra_install_step:
- - bash: sudo apt-get install qemu
- displayName: Install qemu
- condition: and(gt(variables.pkg_count, 0), succeeded())
+ usePythonVersion: '' # use Python from the container image
--
2.39.0
^ permalink raw reply related [flat|nested] 32+ messages in thread* Re: [PATCH v11 08/17] ArmVirtPkg: CI: Use Fedora 35 container (Linux only)
2023-01-16 17:40 ` [PATCH v11 08/17] ArmVirtPkg: " Oliver Steffen
@ 2023-01-17 17:27 ` Michael Kubacki
0 siblings, 0 replies; 32+ messages in thread
From: Michael Kubacki @ 2023-01-17 17:27 UTC (permalink / raw)
To: Oliver Steffen, devel
Cc: Andrew Fish, Ard Biesheuvel, Bob Feng, Dandan Bi, Gerd Hoffmann,
Jian J Wang, Jiewen Yao, Jordan Justen, Leif Lindholm, Liming Gao,
Michael D Kinney, Ray Ni, Sami Mujawar, Sean Brogan, Yuwei Chen,
Pawel Polawski, chris.fernald
Reviewed-by: Michael Kubacki <michael.kubacki@microsoft.com>
On 1/16/2023 12:40 PM, Oliver Steffen wrote:
> Run the Linux jobs of the ArmVirtPkg platform CI inside a container,
> in the same way the general CI does now. Make use of the default image
> specified in the defaults.yml template.
>
> Do not run apt-get in CI jobs to install qemu and gcc dependencies.
> Assume the container image provides these.
>
> Use Python from the container image, do not download at runtime.
>
> Signed-off-by: Oliver Steffen <osteffen@redhat.com>
> ---
> ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 8 +++-----
> 1 file changed, 3 insertions(+), 5 deletions(-)
>
> diff --git a/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml b/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> index 5a0e589ed4a7..c0a2e0f20d4f 100644
> --- a/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> +++ b/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> @@ -78,6 +78,8 @@ jobs:
> pool:
> vmImage: $(vm_image)
>
> + container: ${{ variables.default_linux_image }}
> +
> steps:
> - template: ../../../.azurepipelines/templates/platform-build-run-steps.yml
> parameters:
> @@ -88,8 +90,4 @@ jobs:
> build_file: $(Build.File)
> build_flags: $(Build.Flags)
> run_flags: $(Run.Flags)
> - usePythonVersion: ${{ variables.default_python_version }}
> - extra_install_step:
> - - bash: sudo apt-get install qemu
> - displayName: Install qemu
> - condition: and(gt(variables.pkg_count, 0), succeeded())
> + usePythonVersion: '' # use Python from the container image
^ permalink raw reply [flat|nested] 32+ messages in thread
* [PATCH v11 09/17] EmulatorPkg: CI: Use Fedora 35 container (Linux only)
2023-01-16 17:40 [PATCH v11 00/17] CI: Use Fedora 35 container for Linux jobs Oliver Steffen
` (7 preceding siblings ...)
2023-01-16 17:40 ` [PATCH v11 08/17] ArmVirtPkg: " Oliver Steffen
@ 2023-01-16 17:40 ` Oliver Steffen
2023-01-17 17:28 ` Michael Kubacki
2023-01-16 17:40 ` [PATCH v11 10/17] OvmfPkg: " Oliver Steffen
` (8 subsequent siblings)
17 siblings, 1 reply; 32+ messages in thread
From: Oliver Steffen @ 2023-01-16 17:40 UTC (permalink / raw)
To: devel
Cc: Andrew Fish, Ard Biesheuvel, Bob Feng, Dandan Bi, Gerd Hoffmann,
Jian J Wang, Jiewen Yao, Jordan Justen, Leif Lindholm, Liming Gao,
Michael D Kinney, Michael Kubacki, Ray Ni, Sami Mujawar,
Sean Brogan, Yuwei Chen, Pawel Polawski, chris.fernald,
Oliver Steffen
Run the Linux jobs of the EmulatorPkg platform CI inside a container,
in the same way the general CI does now. Make use of the default image
specified in the defaults.yml template.
Do not run apt-get in CI jobs to install qemu and gcc dependencies.
Assume the container image provides these.
Use Python from the container image, do not download at runtime.
Signed-off-by: Oliver Steffen <osteffen@redhat.com>
---
EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml b/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
index a32c57d4aab4..3861457ac70a 100644
--- a/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
+++ b/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
@@ -79,6 +79,8 @@ jobs:
pool:
vmImage: $(vm_image)
+ container: ${{ variables.default_linux_image }}
+
steps:
- template: ../../../.azurepipelines/templates/platform-build-run-steps.yml
parameters:
@@ -89,4 +91,4 @@ jobs:
build_file: $(Build.File)
build_flags: $(Build.Flags)
run_flags: $(Run.Flags)
- usePythonVersion: ${{ variables.default_python_version }}
+ usePythonVersion: '' # use Python from the container image
--
2.39.0
^ permalink raw reply related [flat|nested] 32+ messages in thread* Re: [PATCH v11 09/17] EmulatorPkg: CI: Use Fedora 35 container (Linux only)
2023-01-16 17:40 ` [PATCH v11 09/17] EmulatorPkg: " Oliver Steffen
@ 2023-01-17 17:28 ` Michael Kubacki
0 siblings, 0 replies; 32+ messages in thread
From: Michael Kubacki @ 2023-01-17 17:28 UTC (permalink / raw)
To: Oliver Steffen, devel
Cc: Andrew Fish, Ard Biesheuvel, Bob Feng, Dandan Bi, Gerd Hoffmann,
Jian J Wang, Jiewen Yao, Jordan Justen, Leif Lindholm, Liming Gao,
Michael D Kinney, Ray Ni, Sami Mujawar, Sean Brogan, Yuwei Chen,
Pawel Polawski, chris.fernald
Reviewed-by: Michael Kubacki <michael.kubacki@microsoft.com>
On 1/16/2023 12:40 PM, Oliver Steffen wrote:
> Run the Linux jobs of the EmulatorPkg platform CI inside a container,
> in the same way the general CI does now. Make use of the default image
> specified in the defaults.yml template.
>
> Do not run apt-get in CI jobs to install qemu and gcc dependencies.
> Assume the container image provides these.
>
> Use Python from the container image, do not download at runtime.
>
> Signed-off-by: Oliver Steffen <osteffen@redhat.com>
> ---
> EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 4 +++-
> 1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml b/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> index a32c57d4aab4..3861457ac70a 100644
> --- a/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> +++ b/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> @@ -79,6 +79,8 @@ jobs:
> pool:
> vmImage: $(vm_image)
>
> + container: ${{ variables.default_linux_image }}
> +
> steps:
> - template: ../../../.azurepipelines/templates/platform-build-run-steps.yml
> parameters:
> @@ -89,4 +91,4 @@ jobs:
> build_file: $(Build.File)
> build_flags: $(Build.Flags)
> run_flags: $(Run.Flags)
> - usePythonVersion: ${{ variables.default_python_version }}
> + usePythonVersion: '' # use Python from the container image
^ permalink raw reply [flat|nested] 32+ messages in thread
* [PATCH v11 10/17] OvmfPkg: CI: Use Fedora 35 container (Linux only)
2023-01-16 17:40 [PATCH v11 00/17] CI: Use Fedora 35 container for Linux jobs Oliver Steffen
` (8 preceding siblings ...)
2023-01-16 17:40 ` [PATCH v11 09/17] EmulatorPkg: " Oliver Steffen
@ 2023-01-16 17:40 ` Oliver Steffen
2023-01-17 0:14 ` [edk2-devel] " Yao, Jiewen
2023-01-17 17:28 ` Michael Kubacki
2023-01-16 17:40 ` [PATCH v11 11/17] .pytool: CISettings.py: don't add scopes for GCC Oliver Steffen
` (7 subsequent siblings)
17 siblings, 2 replies; 32+ messages in thread
From: Oliver Steffen @ 2023-01-16 17:40 UTC (permalink / raw)
To: devel
Cc: Andrew Fish, Ard Biesheuvel, Bob Feng, Dandan Bi, Gerd Hoffmann,
Jian J Wang, Jiewen Yao, Jordan Justen, Leif Lindholm, Liming Gao,
Michael D Kinney, Michael Kubacki, Ray Ni, Sami Mujawar,
Sean Brogan, Yuwei Chen, Pawel Polawski, chris.fernald,
Oliver Steffen
Run the Linux jobs of the OvmfPkg platform CI inside a container,
in the same way the general CI does now. Make use of the default image
specified in the defaults.yml template.
Do not run apt-get in CI jobs to install qemu and gcc dependencies.
Assume the container image provides these.
Use Python from the container image, do not download at runtime.
Signed-off-by: Oliver Steffen <osteffen@redhat.com>
---
OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 8 +++-----
1 file changed, 3 insertions(+), 5 deletions(-)
diff --git a/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml b/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
index 6dd90711ac70..85be431ca5e7 100644
--- a/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
+++ b/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
@@ -180,6 +180,8 @@ jobs:
pool:
vmImage: $(vm_image)
+ container: ${{ variables.default_linux_image }}
+
steps:
- template: ../../../.azurepipelines/templates/platform-build-run-steps.yml
parameters:
@@ -190,8 +192,4 @@ jobs:
build_file: $(Build.File)
build_flags: $(Build.Flags)
run_flags: $(Run.Flags)
- usePythonVersion: ${{ variables.default_python_version }}
- extra_install_step:
- - bash: sudo apt-get install qemu
- displayName: Install qemu
- condition: and(gt(variables.pkg_count, 0), succeeded())
+ usePythonVersion: '' # use Python from the container image
--
2.39.0
^ permalink raw reply related [flat|nested] 32+ messages in thread* Re: [edk2-devel] [PATCH v11 10/17] OvmfPkg: CI: Use Fedora 35 container (Linux only)
2023-01-16 17:40 ` [PATCH v11 10/17] OvmfPkg: " Oliver Steffen
@ 2023-01-17 0:14 ` Yao, Jiewen
2023-01-17 17:28 ` Michael Kubacki
1 sibling, 0 replies; 32+ messages in thread
From: Yao, Jiewen @ 2023-01-17 0:14 UTC (permalink / raw)
To: devel@edk2.groups.io, osteffen@redhat.com
Cc: Andrew Fish, Ard Biesheuvel, Feng, Bob C, Bi, Dandan,
Gerd Hoffmann, Wang, Jian J, Justen, Jordan L, Leif Lindholm,
Gao, Liming, Kinney, Michael D, Michael Kubacki, Ni, Ray,
Sami Mujawar, Sean Brogan, Chen, Christine, Pawel Polawski,
chris.fernald@outlook.com
Acked-by: Jiewen Yao <Jiewen.yao@intel.com>
> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Oliver
> Steffen
> Sent: Tuesday, January 17, 2023 1:41 AM
> To: devel@edk2.groups.io
> Cc: Andrew Fish <afish@apple.com>; Ard Biesheuvel
> <ardb+tianocore@kernel.org>; Feng, Bob C <bob.c.feng@intel.com>; Bi,
> Dandan <dandan.bi@intel.com>; Gerd Hoffmann <kraxel@redhat.com>;
> Wang, Jian J <jian.j.wang@intel.com>; Yao, Jiewen <jiewen.yao@intel.com>;
> Justen, Jordan L <jordan.l.justen@intel.com>; Leif Lindholm
> <quic_llindhol@quicinc.com>; Gao, Liming <gaoliming@byosoft.com.cn>;
> Kinney, Michael D <michael.d.kinney@intel.com>; Michael Kubacki
> <mikuback@linux.microsoft.com>; Ni, Ray <ray.ni@intel.com>; Sami
> Mujawar <sami.mujawar@arm.com>; Sean Brogan
> <sean.brogan@microsoft.com>; Chen, Christine <yuwei.chen@intel.com>;
> Pawel Polawski <ppolawsk@redhat.com>; chris.fernald@outlook.com; Oliver
> Steffen <osteffen@redhat.com>
> Subject: [edk2-devel] [PATCH v11 10/17] OvmfPkg: CI: Use Fedora 35
> container (Linux only)
>
> Run the Linux jobs of the OvmfPkg platform CI inside a container,
> in the same way the general CI does now. Make use of the default image
> specified in the defaults.yml template.
>
> Do not run apt-get in CI jobs to install qemu and gcc dependencies.
> Assume the container image provides these.
>
> Use Python from the container image, do not download at runtime.
>
> Signed-off-by: Oliver Steffen <osteffen@redhat.com>
> ---
> OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 8 +++-----
> 1 file changed, 3 insertions(+), 5 deletions(-)
>
> diff --git a/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> b/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> index 6dd90711ac70..85be431ca5e7 100644
> --- a/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> +++ b/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> @@ -180,6 +180,8 @@ jobs:
> pool:
> vmImage: $(vm_image)
>
> + container: ${{ variables.default_linux_image }}
> +
> steps:
> - template: ../../../.azurepipelines/templates/platform-build-run-
> steps.yml
> parameters:
> @@ -190,8 +192,4 @@ jobs:
> build_file: $(Build.File)
> build_flags: $(Build.Flags)
> run_flags: $(Run.Flags)
> - usePythonVersion: ${{ variables.default_python_version }}
> - extra_install_step:
> - - bash: sudo apt-get install qemu
> - displayName: Install qemu
> - condition: and(gt(variables.pkg_count, 0), succeeded())
> + usePythonVersion: '' # use Python from the container image
> --
> 2.39.0
>
>
>
>
>
^ permalink raw reply [flat|nested] 32+ messages in thread* Re: [edk2-devel] [PATCH v11 10/17] OvmfPkg: CI: Use Fedora 35 container (Linux only)
2023-01-16 17:40 ` [PATCH v11 10/17] OvmfPkg: " Oliver Steffen
2023-01-17 0:14 ` [edk2-devel] " Yao, Jiewen
@ 2023-01-17 17:28 ` Michael Kubacki
1 sibling, 0 replies; 32+ messages in thread
From: Michael Kubacki @ 2023-01-17 17:28 UTC (permalink / raw)
To: devel, osteffen
Cc: Andrew Fish, Ard Biesheuvel, Bob Feng, Dandan Bi, Gerd Hoffmann,
Jian J Wang, Jiewen Yao, Jordan Justen, Leif Lindholm, Liming Gao,
Michael D Kinney, Ray Ni, Sami Mujawar, Sean Brogan, Yuwei Chen,
Pawel Polawski, chris.fernald
Reviewed-by: Michael Kubacki <michael.kubacki@microsoft.com>
On 1/16/2023 12:40 PM, Oliver Steffen wrote:
> Run the Linux jobs of the OvmfPkg platform CI inside a container,
> in the same way the general CI does now. Make use of the default image
> specified in the defaults.yml template.
>
> Do not run apt-get in CI jobs to install qemu and gcc dependencies.
> Assume the container image provides these.
>
> Use Python from the container image, do not download at runtime.
>
> Signed-off-by: Oliver Steffen <osteffen@redhat.com>
> ---
> OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 8 +++-----
> 1 file changed, 3 insertions(+), 5 deletions(-)
>
> diff --git a/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml b/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> index 6dd90711ac70..85be431ca5e7 100644
> --- a/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> +++ b/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> @@ -180,6 +180,8 @@ jobs:
> pool:
> vmImage: $(vm_image)
>
> + container: ${{ variables.default_linux_image }}
> +
> steps:
> - template: ../../../.azurepipelines/templates/platform-build-run-steps.yml
> parameters:
> @@ -190,8 +192,4 @@ jobs:
> build_file: $(Build.File)
> build_flags: $(Build.Flags)
> run_flags: $(Run.Flags)
> - usePythonVersion: ${{ variables.default_python_version }}
> - extra_install_step:
> - - bash: sudo apt-get install qemu
> - displayName: Install qemu
> - condition: and(gt(variables.pkg_count, 0), succeeded())
> + usePythonVersion: '' # use Python from the container image
^ permalink raw reply [flat|nested] 32+ messages in thread
* [PATCH v11 11/17] .pytool: CISettings.py: don't add scopes for GCC
2023-01-16 17:40 [PATCH v11 00/17] CI: Use Fedora 35 container for Linux jobs Oliver Steffen
` (9 preceding siblings ...)
2023-01-16 17:40 ` [PATCH v11 10/17] OvmfPkg: " Oliver Steffen
@ 2023-01-16 17:40 ` Oliver Steffen
2023-01-16 17:40 ` [PATCH v11 12/17] BaseTools: remove ext_dep files for gcc Oliver Steffen
` (6 subsequent siblings)
17 siblings, 0 replies; 32+ messages in thread
From: Oliver Steffen @ 2023-01-16 17:40 UTC (permalink / raw)
To: devel
Cc: Andrew Fish, Ard Biesheuvel, Bob Feng, Dandan Bi, Gerd Hoffmann,
Jian J Wang, Jiewen Yao, Jordan Justen, Leif Lindholm, Liming Gao,
Michael D Kinney, Michael Kubacki, Ray Ni, Sami Mujawar,
Sean Brogan, Yuwei Chen, Pawel Polawski, chris.fernald,
Oliver Steffen, Michael Kubacki
All ext_dep.yml files for gcc have been removed and gcc
is expected to be installed on the system (GCC5_*_PREFIX
may indicate the location).
No need to adjust the toolchain scopes for Linux builds
anymore.
Signed-off-by: Oliver Steffen <osteffen@redhat.com>
Reviewed-by: Michael Kubacki <michael.kubacki@microsoft.com>
---
.pytool/CISettings.py | 9 ---------
1 file changed, 9 deletions(-)
diff --git a/.pytool/CISettings.py b/.pytool/CISettings.py
index 76ac2b09dba6..d87c8e838e61 100644
--- a/.pytool/CISettings.py
+++ b/.pytool/CISettings.py
@@ -169,15 +169,6 @@ class Settings(CiBuildSettingsManager, UpdateSettingsManager, SetupSettingsManag
else:
logging.warning("Falling back to using in-tree BaseTools")
- if is_linux and self.ActualToolChainTag.upper().startswith("GCC"):
- if "AARCH64" in self.ActualArchitectures:
- scopes += ("gcc_aarch64_linux",)
- if "ARM" in self.ActualArchitectures:
- scopes += ("gcc_arm_linux",)
- if "RISCV64" in self.ActualArchitectures:
- scopes += ("gcc_riscv64_unknown",)
- if "LOONGARCH64" in self.ActualArchitectures:
- scopes += ("gcc_loongarch64_unknown_linux",)
self.ActualScopes = scopes
return self.ActualScopes
--
2.39.0
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v11 12/17] BaseTools: remove ext_dep files for gcc
2023-01-16 17:40 [PATCH v11 00/17] CI: Use Fedora 35 container for Linux jobs Oliver Steffen
` (10 preceding siblings ...)
2023-01-16 17:40 ` [PATCH v11 11/17] .pytool: CISettings.py: don't add scopes for GCC Oliver Steffen
@ 2023-01-16 17:40 ` Oliver Steffen
2023-01-16 17:40 ` [PATCH v11 13/17] ArmVirtPkg: CI: use ubuntu-22.04 vm_image (Linux only) Oliver Steffen
` (5 subsequent siblings)
17 siblings, 0 replies; 32+ messages in thread
From: Oliver Steffen @ 2023-01-16 17:40 UTC (permalink / raw)
To: devel
Cc: Andrew Fish, Ard Biesheuvel, Bob Feng, Dandan Bi, Gerd Hoffmann,
Jian J Wang, Jiewen Yao, Jordan Justen, Leif Lindholm, Liming Gao,
Michael D Kinney, Michael Kubacki, Ray Ni, Sami Mujawar,
Sean Brogan, Yuwei Chen, Pawel Polawski, chris.fernald,
Oliver Steffen, Michael Kubacki
Remove BaseTools/Bin/gcc_*_ext_dep.yaml to stop
downloading gcc from external locations; use the
gcc provided by the container image instead.
The container image sets the variable GCC5_*_PREFIX accordingly.
Signed-off-by: Oliver Steffen <osteffen@redhat.com>
Reviewed-by: Michael Kubacki <michael.kubacki@microsoft.com>
---
BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml | 21 ------------------
BaseTools/Bin/gcc_arm_linux_ext_dep.yaml | 21 ------------------
...gcc_loongarch64_unknown_linux_ext_dep.yaml | 22 -------------------
.../Bin/gcc_riscv64_unknown_ext_dep.yaml | 22 -------------------
4 files changed, 86 deletions(-)
delete mode 100644 BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml
delete mode 100644 BaseTools/Bin/gcc_arm_linux_ext_dep.yaml
delete mode 100644 BaseTools/Bin/gcc_loongarch64_unknown_linux_ext_dep.yaml
delete mode 100644 BaseTools/Bin/gcc_riscv64_unknown_ext_dep.yaml
diff --git a/BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml b/BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml
deleted file mode 100644
index ff8a9e868100..000000000000
--- a/BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml
+++ /dev/null
@@ -1,21 +0,0 @@
-## @file
-# Download GCC AARCH64 compiler from Arm's release site
-# Set shell variable GCC5_AARCH64_INSTALL to this folder
-#
-# This is only downloaded when a build activates scope gcc_aarch64_linux
-#
-# Copyright (c) Microsoft Corporation.
-# SPDX-License-Identifier: BSD-2-Clause-Patent
-##
-{
- "scope": "gcc_aarch64_linux",
- "type": "web",
- "name": "gcc_aarch64_linux",
- "source": "https://developer.arm.com/-/media/Files/downloads/gnu/11.2-2022.02/binrel/gcc-arm-11.2-2022.02-x86_64-aarch64-none-linux-gnu.tar.xz",
- "version": "11.2-2022.02",
- "sha256": "52dbac3eb71dbe0916f60a8c5ab9b7dc9b66b3ce513047baa09fae56234e53f3",
- "compression_type": "tar",
- "internal_path": "/gcc-arm-11.2-2022.02-x86_64-aarch64-none-linux-gnu/",
- "flags": ["set_shell_var", ],
- "var_name": "GCC5_AARCH64_INSTALL"
-}
diff --git a/BaseTools/Bin/gcc_arm_linux_ext_dep.yaml b/BaseTools/Bin/gcc_arm_linux_ext_dep.yaml
deleted file mode 100644
index 151cbfa4b532..000000000000
--- a/BaseTools/Bin/gcc_arm_linux_ext_dep.yaml
+++ /dev/null
@@ -1,21 +0,0 @@
-## @file
-# Download GCC ARM compiler from Arm's release site
-# Set shell variable GCC5_ARM_INSTALL to this folder
-#
-# This is only downloaded when a build activates scope gcc_arm_linux
-#
-# Copyright (c) Microsoft Corporation.
-# SPDX-License-Identifier: BSD-2-Clause-Patent
-##
-{
- "scope": "gcc_arm_linux",
- "type": "web",
- "name": "gcc_arm_linux",
- "source": "https://developer.arm.com/-/media/Files/downloads/gnu/11.2-2022.02/binrel/gcc-arm-11.2-2022.02-x86_64-arm-none-linux-gnueabihf.tar.xz",
- "version": "11.2-2022.02",
- "sha256": "c254f7199261fe76c32ef42187502839bda7efad0a66646cf739d074eff45fad",
- "compression_type": "tar",
- "internal_path": "/gcc-arm-11.2-2022.02-x86_64-arm-none-linux-gnueabihf/",
- "flags": ["set_shell_var", ],
- "var_name": "GCC5_ARM_INSTALL"
-}
diff --git a/BaseTools/Bin/gcc_loongarch64_unknown_linux_ext_dep.yaml b/BaseTools/Bin/gcc_loongarch64_unknown_linux_ext_dep.yaml
deleted file mode 100644
index 6ebc93a07c66..000000000000
--- a/BaseTools/Bin/gcc_loongarch64_unknown_linux_ext_dep.yaml
+++ /dev/null
@@ -1,22 +0,0 @@
-## @file
-# Download GCC LoongArch64 compiler from LoongArch GitHub release site
-# Set shell variable GCC5_LOONGARCH64_INSTALL to this folder
-#
-# This is only downloaded when a build activates scope gcc_loongarch64_unknown_linux
-#
-# Copyright (c) Microsoft Corporation.
-# Copyright (c) 2022, Loongson Technology Corporation Limited. All rights reserved.<BR>
-# SPDX-License-Identifier: BSD-2-Clause-Patent
-##
-{
- "scope": "gcc_loongarch64_unknown_linux",
- "type": "web",
- "name": "gcc_loongarch64_unknown_linux",
- "source":"https://github.com/loongson/build-tools/releases/download/2022.09.06/loongarch64-clfs-6.3-cross-tools-c-only.tar.xz",
- "version": "13.0.0",
- "sha256":"744cbb50c6ef07c96551bddf07a8b85cee3250bb18b6040f6ed051e389514951",
- "compression_type": "tar",
- "internal_path": "/cross-tools/",
- "flags": ["set_shell_var", ],
- "var_name": "GCC5_LOONGARCH64_INSTALL"
-}
diff --git a/BaseTools/Bin/gcc_riscv64_unknown_ext_dep.yaml b/BaseTools/Bin/gcc_riscv64_unknown_ext_dep.yaml
deleted file mode 100644
index 8abbcd7ba040..000000000000
--- a/BaseTools/Bin/gcc_riscv64_unknown_ext_dep.yaml
+++ /dev/null
@@ -1,22 +0,0 @@
-## @file
-# Download GCC RISCV64 compiler from RISC-V Organization release site
-# Set shell variable GCC5_RISCV64_INSTALL to this folder
-#
-# This is only downloaded when a build activates scope gcc_riscv64_unknown
-#
-# Copyright (c) Microsoft Corporation.
-# Copyright (c) 2020, Hewlett Packard Enterprise Development LP. All rights reserved.<BR>
-# SPDX-License-Identifier: BSD-2-Clause-Patent
-##
-{
- "scope": "gcc_riscv64_unknown",
- "type": "web",
- "name": "gcc_riscv64_unknown",
- "source": "https://raw.githubusercontent.com/riscv/riscv-uefi-edk2-docs/master/gcc-riscv-edk2-ci-toolchain/gcc-riscv-9.2.0-2020.04-x86_64_riscv64-unknown-gnu.tar.xz",
- "version": "9.2.0",
- "compression_type": "tar",
- "sha256": "28373643b69f0ce008273c3dc63f172aa1121952f1b9ae94d7485ac94af7f344",
- "internal_path": "/gcc-riscv-9.2.0-2020.04-x86_64_riscv64-unknown-gnu",
- "flags": ["set_shell_var", ],
- "var_name": "GCC5_RISCV64_INSTALL"
-}
--
2.39.0
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v11 13/17] ArmVirtPkg: CI: use ubuntu-22.04 vm_image (Linux only)
2023-01-16 17:40 [PATCH v11 00/17] CI: Use Fedora 35 container for Linux jobs Oliver Steffen
` (11 preceding siblings ...)
2023-01-16 17:40 ` [PATCH v11 12/17] BaseTools: remove ext_dep files for gcc Oliver Steffen
@ 2023-01-16 17:40 ` Oliver Steffen
2023-01-16 17:40 ` [PATCH v11 14/17] EmulatorPgk: " Oliver Steffen
` (4 subsequent siblings)
17 siblings, 0 replies; 32+ messages in thread
From: Oliver Steffen @ 2023-01-16 17:40 UTC (permalink / raw)
To: devel
Cc: Andrew Fish, Ard Biesheuvel, Bob Feng, Dandan Bi, Gerd Hoffmann,
Jian J Wang, Jiewen Yao, Jordan Justen, Leif Lindholm, Liming Gao,
Michael D Kinney, Michael Kubacki, Ray Ni, Sami Mujawar,
Sean Brogan, Yuwei Chen, Pawel Polawski, chris.fernald,
Oliver Steffen, Michael Kubacki
Switch over to ubuntu-22.04 as the vm_image for Linux CI jobs. The
previously used ubuntu-18.04 which is not available anymore since
Dec 1st 2022.
Signed-off-by: Oliver Steffen <osteffen@redhat.com>
Reviewed-by: Michael Kubacki <michael.kubacki@microsoft.com>
---
ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml b/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
index c0a2e0f20d4f..5fa7518d2c5e 100644
--- a/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
+++ b/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
@@ -22,7 +22,7 @@ jobs:
- job: Platform_CI
variables:
package: 'ArmVirtPkg'
- vm_image: 'ubuntu-18.04'
+ vm_image: 'ubuntu-22.04'
should_run: true
run_flags: "MAKE_STARTUP_NSH=TRUE QEMU_HEADLESS=TRUE"
--
2.39.0
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v11 14/17] EmulatorPgk: CI: use ubuntu-22.04 vm_image (Linux only)
2023-01-16 17:40 [PATCH v11 00/17] CI: Use Fedora 35 container for Linux jobs Oliver Steffen
` (12 preceding siblings ...)
2023-01-16 17:40 ` [PATCH v11 13/17] ArmVirtPkg: CI: use ubuntu-22.04 vm_image (Linux only) Oliver Steffen
@ 2023-01-16 17:40 ` Oliver Steffen
2023-01-16 17:40 ` [PATCH v11 15/17] OvmfPkg: " Oliver Steffen
` (3 subsequent siblings)
17 siblings, 0 replies; 32+ messages in thread
From: Oliver Steffen @ 2023-01-16 17:40 UTC (permalink / raw)
To: devel
Cc: Andrew Fish, Ard Biesheuvel, Bob Feng, Dandan Bi, Gerd Hoffmann,
Jian J Wang, Jiewen Yao, Jordan Justen, Leif Lindholm, Liming Gao,
Michael D Kinney, Michael Kubacki, Ray Ni, Sami Mujawar,
Sean Brogan, Yuwei Chen, Pawel Polawski, chris.fernald,
Oliver Steffen, Michael Kubacki
Switch over to ubuntu-22.04 as the vm_image for Linux CI jobs. The
previously used ubuntu-18.04 which is not available anymore since
Dec 1st 2022.
Signed-off-by: Oliver Steffen <osteffen@redhat.com>
Reviewed-by: Michael Kubacki <michael.kubacki@microsoft.com>
---
EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml b/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
index 3861457ac70a..8af4c9c778cb 100644
--- a/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
+++ b/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
@@ -23,7 +23,7 @@ jobs:
- job: Platform_CI
variables:
package: 'EmulatorPkg'
- vm_image: 'ubuntu-18.04'
+ vm_image: 'ubuntu-22.04'
should_run: false
run_flags: "MAKE_STARTUP_NSH=TRUE"
--
2.39.0
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v11 15/17] OvmfPkg: CI: use ubuntu-22.04 vm_image (Linux only)
2023-01-16 17:40 [PATCH v11 00/17] CI: Use Fedora 35 container for Linux jobs Oliver Steffen
` (13 preceding siblings ...)
2023-01-16 17:40 ` [PATCH v11 14/17] EmulatorPgk: " Oliver Steffen
@ 2023-01-16 17:40 ` Oliver Steffen
2023-01-17 0:15 ` [edk2-devel] " Yao, Jiewen
2023-01-16 17:40 ` [PATCH v11 16/17] CI: use ubuntu-22.04 image " Oliver Steffen
` (2 subsequent siblings)
17 siblings, 1 reply; 32+ messages in thread
From: Oliver Steffen @ 2023-01-16 17:40 UTC (permalink / raw)
To: devel
Cc: Andrew Fish, Ard Biesheuvel, Bob Feng, Dandan Bi, Gerd Hoffmann,
Jian J Wang, Jiewen Yao, Jordan Justen, Leif Lindholm, Liming Gao,
Michael D Kinney, Michael Kubacki, Ray Ni, Sami Mujawar,
Sean Brogan, Yuwei Chen, Pawel Polawski, chris.fernald,
Oliver Steffen, Michael Kubacki
Switch over to ubuntu-22.04 as the vm_image for Linux CI jobs. The
previously used ubuntu-18.04 which is not available anymore since
Dec 1st 2022.
Signed-off-by: Oliver Steffen <osteffen@redhat.com>
Reviewed-by: Michael Kubacki <michael.kubacki@microsoft.com>
---
OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml b/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
index 85be431ca5e7..1cf9bdf8b9c2 100644
--- a/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
+++ b/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
@@ -22,7 +22,7 @@ jobs:
- job: Platform_CI
variables:
package: 'OvmfPkg'
- vm_image: 'ubuntu-18.04'
+ vm_image: 'ubuntu-22.04'
should_run: true
run_flags: "MAKE_STARTUP_NSH=TRUE QEMU_HEADLESS=TRUE"
--
2.39.0
^ permalink raw reply related [flat|nested] 32+ messages in thread* Re: [edk2-devel] [PATCH v11 15/17] OvmfPkg: CI: use ubuntu-22.04 vm_image (Linux only)
2023-01-16 17:40 ` [PATCH v11 15/17] OvmfPkg: " Oliver Steffen
@ 2023-01-17 0:15 ` Yao, Jiewen
0 siblings, 0 replies; 32+ messages in thread
From: Yao, Jiewen @ 2023-01-17 0:15 UTC (permalink / raw)
To: devel@edk2.groups.io, osteffen@redhat.com
Cc: Andrew Fish, Ard Biesheuvel, Feng, Bob C, Bi, Dandan,
Gerd Hoffmann, Wang, Jian J, Justen, Jordan L, Leif Lindholm,
Gao, Liming, Kinney, Michael D, Michael Kubacki, Ni, Ray,
Sami Mujawar, Sean Brogan, Chen, Christine, Pawel Polawski,
chris.fernald@outlook.com, Kubacki, Michael
Acked-by: Jiewen Yao <Jiewen.yao@intel.com>
> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Oliver
> Steffen
> Sent: Tuesday, January 17, 2023 1:41 AM
> To: devel@edk2.groups.io
> Cc: Andrew Fish <afish@apple.com>; Ard Biesheuvel
> <ardb+tianocore@kernel.org>; Feng, Bob C <bob.c.feng@intel.com>; Bi,
> Dandan <dandan.bi@intel.com>; Gerd Hoffmann <kraxel@redhat.com>;
> Wang, Jian J <jian.j.wang@intel.com>; Yao, Jiewen <jiewen.yao@intel.com>;
> Justen, Jordan L <jordan.l.justen@intel.com>; Leif Lindholm
> <quic_llindhol@quicinc.com>; Gao, Liming <gaoliming@byosoft.com.cn>;
> Kinney, Michael D <michael.d.kinney@intel.com>; Michael Kubacki
> <mikuback@linux.microsoft.com>; Ni, Ray <ray.ni@intel.com>; Sami
> Mujawar <sami.mujawar@arm.com>; Sean Brogan
> <sean.brogan@microsoft.com>; Chen, Christine <yuwei.chen@intel.com>;
> Pawel Polawski <ppolawsk@redhat.com>; chris.fernald@outlook.com; Oliver
> Steffen <osteffen@redhat.com>; Kubacki, Michael
> <michael.kubacki@microsoft.com>
> Subject: [edk2-devel] [PATCH v11 15/17] OvmfPkg: CI: use ubuntu-22.04
> vm_image (Linux only)
>
> Switch over to ubuntu-22.04 as the vm_image for Linux CI jobs. The
> previously used ubuntu-18.04 which is not available anymore since
> Dec 1st 2022.
>
> Signed-off-by: Oliver Steffen <osteffen@redhat.com>
> Reviewed-by: Michael Kubacki <michael.kubacki@microsoft.com>
> ---
> OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> b/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> index 85be431ca5e7..1cf9bdf8b9c2 100644
> --- a/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> +++ b/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> @@ -22,7 +22,7 @@ jobs:
> - job: Platform_CI
> variables:
> package: 'OvmfPkg'
> - vm_image: 'ubuntu-18.04'
> + vm_image: 'ubuntu-22.04'
> should_run: true
> run_flags: "MAKE_STARTUP_NSH=TRUE QEMU_HEADLESS=TRUE"
>
> --
> 2.39.0
>
>
>
>
>
^ permalink raw reply [flat|nested] 32+ messages in thread
* [PATCH v11 16/17] CI: use ubuntu-22.04 image (Linux only)
2023-01-16 17:40 [PATCH v11 00/17] CI: Use Fedora 35 container for Linux jobs Oliver Steffen
` (14 preceding siblings ...)
2023-01-16 17:40 ` [PATCH v11 15/17] OvmfPkg: " Oliver Steffen
@ 2023-01-16 17:40 ` Oliver Steffen
2023-01-16 17:40 ` [PATCH v11 17/17] OfmvPkg: CI: Qemu tests: set CPU to Broadwell Oliver Steffen
2023-01-17 18:22 ` [PATCH v11 00/17] CI: Use Fedora 35 container for Linux jobs Ard Biesheuvel
17 siblings, 0 replies; 32+ messages in thread
From: Oliver Steffen @ 2023-01-16 17:40 UTC (permalink / raw)
To: devel
Cc: Andrew Fish, Ard Biesheuvel, Bob Feng, Dandan Bi, Gerd Hoffmann,
Jian J Wang, Jiewen Yao, Jordan Justen, Leif Lindholm, Liming Gao,
Michael D Kinney, Michael Kubacki, Ray Ni, Sami Mujawar,
Sean Brogan, Yuwei Chen, Pawel Polawski, chris.fernald,
Oliver Steffen, Michael Kubacki
Use the same vm_image as the other Linux CIs in
ArmVirtPkg, EmulatorPkg, and OvmfPkg.
Switch over to ubuntu-22.04 form ubuntu-latest.
Signed-off-by: Oliver Steffen <osteffen@redhat.com>
Reviewed-by: Michael Kubacki <michael.kubacki@microsoft.com>
---
.azurepipelines/Ubuntu-GCC5.yml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/.azurepipelines/Ubuntu-GCC5.yml b/.azurepipelines/Ubuntu-GCC5.yml
index 246e2d272e20..d93604fadbdb 100644
--- a/.azurepipelines/Ubuntu-GCC5.yml
+++ b/.azurepipelines/Ubuntu-GCC5.yml
@@ -20,7 +20,7 @@ jobs:
- template: templates/pr-gate-build-job.yml
parameters:
tool_chain_tag: 'GCC5'
- vm_image: 'ubuntu-latest'
+ vm_image: 'ubuntu-22.04'
container: ${{ variables.default_linux_image }}
arch_list: "IA32,X64,ARM,AARCH64,RISCV64,LOONGARCH64"
usePythonVersion: '' # use Python from the container image
--
2.39.0
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v11 17/17] OfmvPkg: CI: Qemu tests: set CPU to Broadwell
2023-01-16 17:40 [PATCH v11 00/17] CI: Use Fedora 35 container for Linux jobs Oliver Steffen
` (15 preceding siblings ...)
2023-01-16 17:40 ` [PATCH v11 16/17] CI: use ubuntu-22.04 image " Oliver Steffen
@ 2023-01-16 17:40 ` Oliver Steffen
2023-01-17 0:16 ` Yao, Jiewen
2023-01-17 18:22 ` [PATCH v11 00/17] CI: Use Fedora 35 container for Linux jobs Ard Biesheuvel
17 siblings, 1 reply; 32+ messages in thread
From: Oliver Steffen @ 2023-01-16 17:40 UTC (permalink / raw)
To: devel
Cc: Andrew Fish, Ard Biesheuvel, Bob Feng, Dandan Bi, Gerd Hoffmann,
Jian J Wang, Jiewen Yao, Jordan Justen, Leif Lindholm, Liming Gao,
Michael D Kinney, Michael Kubacki, Ray Ni, Sami Mujawar,
Sean Brogan, Yuwei Chen, Pawel Polawski, chris.fernald,
Oliver Steffen
Work around a bug in Ovmf that causes Ovmf to crash on startup
when running in recent versions of Qemu. Setting the cpu model
to something older, for example "Broadwell" works around the issue.
Signed-off-by: Oliver Steffen <osteffen@redhat.com>
---
OvmfPkg/PlatformCI/PlatformBuildLib.py | 1 +
1 file changed, 1 insertion(+)
diff --git a/OvmfPkg/PlatformCI/PlatformBuildLib.py b/OvmfPkg/PlatformCI/PlatformBuildLib.py
index bfef9849c749..93f7d7aa13d6 100644
--- a/OvmfPkg/PlatformCI/PlatformBuildLib.py
+++ b/OvmfPkg/PlatformCI/PlatformBuildLib.py
@@ -194,6 +194,7 @@ class PlatformBuilder( UefiBuilder, BuildSettingsManager):
cmd = "qemu-system-x86_64"
args = "-debugcon stdio" # write messages to stdio
args += " -global isa-debugcon.iobase=0x402" # debug messages out thru virtual io port
+ args += " -cpu Broadwell" # workaround Qvmf bug
args += " -net none" # turn off network
args += f" -drive file=fat:rw:{VirtualDrive},format=raw,media=disk" # Mount disk with startup.nsh
--
2.39.0
^ permalink raw reply related [flat|nested] 32+ messages in thread* Re: [PATCH v11 17/17] OfmvPkg: CI: Qemu tests: set CPU to Broadwell
2023-01-16 17:40 ` [PATCH v11 17/17] OfmvPkg: CI: Qemu tests: set CPU to Broadwell Oliver Steffen
@ 2023-01-17 0:16 ` Yao, Jiewen
2023-01-17 10:37 ` [edk2-devel] " Oliver Steffen
0 siblings, 1 reply; 32+ messages in thread
From: Yao, Jiewen @ 2023-01-17 0:16 UTC (permalink / raw)
To: Oliver Steffen, devel@edk2.groups.io
Cc: Andrew Fish, Ard Biesheuvel, Feng, Bob C, Bi, Dandan,
Gerd Hoffmann, Wang, Jian J, Justen, Jordan L, Leif Lindholm,
Gao, Liming, Kinney, Michael D, Michael Kubacki, Ni, Ray,
Sami Mujawar, Sean Brogan, Chen, Christine, Pawel Polawski,
chris.fernald@outlook.com
Not sure the detail.
Could you please add more comment, such as, what issue? Why crash? Why changing to Broadwell can resolve it?
any URL?
> -----Original Message-----
> From: Oliver Steffen <osteffen@redhat.com>
> Sent: Tuesday, January 17, 2023 1:41 AM
> To: devel@edk2.groups.io
> Cc: Andrew Fish <afish@apple.com>; Ard Biesheuvel
> <ardb+tianocore@kernel.org>; Feng, Bob C <bob.c.feng@intel.com>; Bi,
> Dandan <dandan.bi@intel.com>; Gerd Hoffmann <kraxel@redhat.com>;
> Wang, Jian J <jian.j.wang@intel.com>; Yao, Jiewen <jiewen.yao@intel.com>;
> Justen, Jordan L <jordan.l.justen@intel.com>; Leif Lindholm
> <quic_llindhol@quicinc.com>; Gao, Liming <gaoliming@byosoft.com.cn>;
> Kinney, Michael D <michael.d.kinney@intel.com>; Michael Kubacki
> <mikuback@linux.microsoft.com>; Ni, Ray <ray.ni@intel.com>; Sami
> Mujawar <sami.mujawar@arm.com>; Sean Brogan
> <sean.brogan@microsoft.com>; Chen, Christine <yuwei.chen@intel.com>;
> Pawel Polawski <ppolawsk@redhat.com>; chris.fernald@outlook.com; Oliver
> Steffen <osteffen@redhat.com>
> Subject: [PATCH v11 17/17] OfmvPkg: CI: Qemu tests: set CPU to Broadwell
>
> Work around a bug in Ovmf that causes Ovmf to crash on startup
> when running in recent versions of Qemu. Setting the cpu model
> to something older, for example "Broadwell" works around the issue.
>
> Signed-off-by: Oliver Steffen <osteffen@redhat.com>
> ---
> OvmfPkg/PlatformCI/PlatformBuildLib.py | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/OvmfPkg/PlatformCI/PlatformBuildLib.py
> b/OvmfPkg/PlatformCI/PlatformBuildLib.py
> index bfef9849c749..93f7d7aa13d6 100644
> --- a/OvmfPkg/PlatformCI/PlatformBuildLib.py
> +++ b/OvmfPkg/PlatformCI/PlatformBuildLib.py
> @@ -194,6 +194,7 @@ class PlatformBuilder( UefiBuilder,
> BuildSettingsManager):
> cmd = "qemu-system-x86_64"
> args = "-debugcon stdio" # write messages to
> stdio
> args += " -global isa-debugcon.iobase=0x402" # debug
> messages out thru virtual io port
> + args += " -cpu Broadwell" # workaround Qvmf
> bug
> args += " -net none" # turn off network
> args += f" -drive file=fat:rw:{VirtualDrive},format=raw,media=disk" #
> Mount disk with startup.nsh
>
> --
> 2.39.0
^ permalink raw reply [flat|nested] 32+ messages in thread* Re: [edk2-devel] [PATCH v11 17/17] OfmvPkg: CI: Qemu tests: set CPU to Broadwell
2023-01-17 0:16 ` Yao, Jiewen
@ 2023-01-17 10:37 ` Oliver Steffen
2023-01-17 12:20 ` Yao, Jiewen
0 siblings, 1 reply; 32+ messages in thread
From: Oliver Steffen @ 2023-01-17 10:37 UTC (permalink / raw)
To: devel, jiewen.yao
Cc: Andrew Fish, Ard Biesheuvel, Feng, Bob C, Bi, Dandan,
Gerd Hoffmann, Wang, Jian J, Justen, Jordan L, Leif Lindholm,
Gao, Liming, Kinney, Michael D, Michael Kubacki, Ni, Ray,
Sami Mujawar, Sean Brogan, Chen, Christine, Pawel Polawski,
chris.fernald@outlook.com
[-- Attachment #1: Type: text/plain, Size: 2573 bytes --]
It is this one:
https://bugzilla.tianocore.org/show_bug.cgi?id=4251
Should have put that into the commit message, sorry.
On Tue, Jan 17, 2023 at 1:16 AM Yao, Jiewen <jiewen.yao@intel.com> wrote:
> Not sure the detail.
>
> Could you please add more comment, such as, what issue? Why crash? Why
> changing to Broadwell can resolve it?
> any URL?
>
>
>
> > -----Original Message-----
> > From: Oliver Steffen <osteffen@redhat.com>
> > Sent: Tuesday, January 17, 2023 1:41 AM
> > To: devel@edk2.groups.io
> > Cc: Andrew Fish <afish@apple.com>; Ard Biesheuvel
> > <ardb+tianocore@kernel.org>; Feng, Bob C <bob.c.feng@intel.com>; Bi,
> > Dandan <dandan.bi@intel.com>; Gerd Hoffmann <kraxel@redhat.com>;
> > Wang, Jian J <jian.j.wang@intel.com>; Yao, Jiewen <jiewen.yao@intel.com
> >;
> > Justen, Jordan L <jordan.l.justen@intel.com>; Leif Lindholm
> > <quic_llindhol@quicinc.com>; Gao, Liming <gaoliming@byosoft.com.cn>;
> > Kinney, Michael D <michael.d.kinney@intel.com>; Michael Kubacki
> > <mikuback@linux.microsoft.com>; Ni, Ray <ray.ni@intel.com>; Sami
> > Mujawar <sami.mujawar@arm.com>; Sean Brogan
> > <sean.brogan@microsoft.com>; Chen, Christine <yuwei.chen@intel.com>;
> > Pawel Polawski <ppolawsk@redhat.com>; chris.fernald@outlook.com; Oliver
> > Steffen <osteffen@redhat.com>
> > Subject: [PATCH v11 17/17] OfmvPkg: CI: Qemu tests: set CPU to Broadwell
> >
> > Work around a bug in Ovmf that causes Ovmf to crash on startup
> > when running in recent versions of Qemu. Setting the cpu model
> > to something older, for example "Broadwell" works around the issue.
> >
> > Signed-off-by: Oliver Steffen <osteffen@redhat.com>
> > ---
> > OvmfPkg/PlatformCI/PlatformBuildLib.py | 1 +
> > 1 file changed, 1 insertion(+)
> >
> > diff --git a/OvmfPkg/PlatformCI/PlatformBuildLib.py
> > b/OvmfPkg/PlatformCI/PlatformBuildLib.py
> > index bfef9849c749..93f7d7aa13d6 100644
> > --- a/OvmfPkg/PlatformCI/PlatformBuildLib.py
> > +++ b/OvmfPkg/PlatformCI/PlatformBuildLib.py
> > @@ -194,6 +194,7 @@ class PlatformBuilder( UefiBuilder,
> > BuildSettingsManager):
> > cmd = "qemu-system-x86_64"
> > args = "-debugcon stdio"
> # write messages to
> > stdio
> > args += " -global isa-debugcon.iobase=0x402"
> # debug
> > messages out thru virtual io port
> > + args += " -cpu Broadwell"
> # workaround Qvmf
> > bug
> > args += " -net none"
> # turn off network
> > args += f" -drive
> file=fat:rw:{VirtualDrive},format=raw,media=disk" #
> > Mount disk with startup.nsh
> >
> > --
> > 2.39.0
>
>
>
>
>
>
>
[-- Attachment #2: Type: text/html, Size: 5234 bytes --]
^ permalink raw reply [flat|nested] 32+ messages in thread* Re: [edk2-devel] [PATCH v11 17/17] OfmvPkg: CI: Qemu tests: set CPU to Broadwell
2023-01-17 10:37 ` [edk2-devel] " Oliver Steffen
@ 2023-01-17 12:20 ` Yao, Jiewen
2023-01-17 12:38 ` Gerd Hoffmann
0 siblings, 1 reply; 32+ messages in thread
From: Yao, Jiewen @ 2023-01-17 12:20 UTC (permalink / raw)
To: devel@edk2.groups.io, osteffen@redhat.com
Cc: Andrew Fish, Ard Biesheuvel, Feng, Bob C, Bi, Dandan,
Gerd Hoffmann, Wang, Jian J, Justen, Jordan L, Leif Lindholm,
Gao, Liming, Kinney, Michael D, Michael Kubacki, Ni, Ray,
Sami Mujawar, Sean Brogan, Chen, Christine, Pawel Polawski,
chris.fernald@outlook.com
[-- Attachment #1: Type: text/plain, Size: 4495 bytes --]
I have seen Gerd sent out V4 patch set for #4251. Is that not enough?
In general, I prefer no work-around, unless there is strong reason.
I would like to have Gerd and Ard to feedback.
Thank you
Yao, Jiewen
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Oliver Steffen
Sent: Tuesday, January 17, 2023 6:37 PM
To: devel@edk2.groups.io; Yao, Jiewen <jiewen.yao@intel.com>
Cc: Andrew Fish <afish@apple.com>; Ard Biesheuvel <ardb+tianocore@kernel.org>; Feng, Bob C <bob.c.feng@intel.com>; Bi, Dandan <dandan.bi@intel.com>; Gerd Hoffmann <kraxel@redhat.com>; Wang, Jian J <jian.j.wang@intel.com>; Justen, Jordan L <jordan.l.justen@intel.com>; Leif Lindholm <quic_llindhol@quicinc.com>; Gao, Liming <gaoliming@byosoft.com.cn>; Kinney, Michael D <michael.d.kinney@intel.com>; Michael Kubacki <mikuback@linux.microsoft.com>; Ni, Ray <ray.ni@intel.com>; Sami Mujawar <sami.mujawar@arm.com>; Sean Brogan <sean.brogan@microsoft.com>; Chen, Christine <yuwei.chen@intel.com>; Pawel Polawski <ppolawsk@redhat.com>; chris.fernald@outlook.com
Subject: Re: [edk2-devel] [PATCH v11 17/17] OfmvPkg: CI: Qemu tests: set CPU to Broadwell
It is this one:
https://bugzilla.tianocore.org/show_bug.cgi?id=4251
Should have put that into the commit message, sorry.
On Tue, Jan 17, 2023 at 1:16 AM Yao, Jiewen <jiewen.yao@intel.com<mailto:jiewen.yao@intel.com>> wrote:
Not sure the detail.
Could you please add more comment, such as, what issue? Why crash? Why changing to Broadwell can resolve it?
any URL?
> -----Original Message-----
> From: Oliver Steffen <osteffen@redhat.com<mailto:osteffen@redhat.com>>
> Sent: Tuesday, January 17, 2023 1:41 AM
> To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>
> Cc: Andrew Fish <afish@apple.com<mailto:afish@apple.com>>; Ard Biesheuvel
> <ardb+tianocore@kernel.org<mailto:ardb%2Btianocore@kernel.org>>; Feng, Bob C <bob.c.feng@intel.com<mailto:bob.c.feng@intel.com>>; Bi,
> Dandan <dandan.bi@intel.com<mailto:dandan.bi@intel.com>>; Gerd Hoffmann <kraxel@redhat.com<mailto:kraxel@redhat.com>>;
> Wang, Jian J <jian.j.wang@intel.com<mailto:jian.j.wang@intel.com>>; Yao, Jiewen <jiewen.yao@intel.com<mailto:jiewen.yao@intel.com>>;
> Justen, Jordan L <jordan.l.justen@intel.com<mailto:jordan.l.justen@intel.com>>; Leif Lindholm
> <quic_llindhol@quicinc.com<mailto:quic_llindhol@quicinc.com>>; Gao, Liming <gaoliming@byosoft.com.cn<mailto:gaoliming@byosoft.com.cn>>;
> Kinney, Michael D <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>; Michael Kubacki
> <mikuback@linux.microsoft.com<mailto:mikuback@linux.microsoft.com>>; Ni, Ray <ray.ni@intel.com<mailto:ray.ni@intel.com>>; Sami
> Mujawar <sami.mujawar@arm.com<mailto:sami.mujawar@arm.com>>; Sean Brogan
> <sean.brogan@microsoft.com<mailto:sean.brogan@microsoft.com>>; Chen, Christine <yuwei.chen@intel.com<mailto:yuwei.chen@intel.com>>;
> Pawel Polawski <ppolawsk@redhat.com<mailto:ppolawsk@redhat.com>>; chris.fernald@outlook.com<mailto:chris.fernald@outlook.com>; Oliver
> Steffen <osteffen@redhat.com<mailto:osteffen@redhat.com>>
> Subject: [PATCH v11 17/17] OfmvPkg: CI: Qemu tests: set CPU to Broadwell
>
> Work around a bug in Ovmf that causes Ovmf to crash on startup
> when running in recent versions of Qemu. Setting the cpu model
> to something older, for example "Broadwell" works around the issue.
>
> Signed-off-by: Oliver Steffen <osteffen@redhat.com<mailto:osteffen@redhat.com>>
> ---
> OvmfPkg/PlatformCI/PlatformBuildLib.py | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/OvmfPkg/PlatformCI/PlatformBuildLib.py
> b/OvmfPkg/PlatformCI/PlatformBuildLib.py
> index bfef9849c749..93f7d7aa13d6 100644
> --- a/OvmfPkg/PlatformCI/PlatformBuildLib.py
> +++ b/OvmfPkg/PlatformCI/PlatformBuildLib.py
> @@ -194,6 +194,7 @@ class PlatformBuilder( UefiBuilder,
> BuildSettingsManager):
> cmd = "qemu-system-x86_64"
> args = "-debugcon stdio" # write messages to
> stdio
> args += " -global isa-debugcon.iobase=0x402" # debug
> messages out thru virtual io port
> + args += " -cpu Broadwell" # workaround Qvmf
> bug
> args += " -net none" # turn off network
> args += f" -drive file=fat:rw:{VirtualDrive},format=raw,media=disk" #
> Mount disk with startup.nsh
>
> --
> 2.39.0
[-- Attachment #2: Type: text/html, Size: 9738 bytes --]
^ permalink raw reply [flat|nested] 32+ messages in thread* Re: [edk2-devel] [PATCH v11 17/17] OfmvPkg: CI: Qemu tests: set CPU to Broadwell
2023-01-17 12:20 ` Yao, Jiewen
@ 2023-01-17 12:38 ` Gerd Hoffmann
2023-01-17 14:49 ` Yao, Jiewen
0 siblings, 1 reply; 32+ messages in thread
From: Gerd Hoffmann @ 2023-01-17 12:38 UTC (permalink / raw)
To: Yao, Jiewen
Cc: devel@edk2.groups.io, osteffen@redhat.com, Andrew Fish,
Ard Biesheuvel, Feng, Bob C, Bi, Dandan, Wang, Jian J,
Justen, Jordan L, Leif Lindholm, Gao, Liming, Kinney, Michael D,
Michael Kubacki, Ni, Ray, Sami Mujawar, Sean Brogan,
Chen, Christine, Pawel Polawski, chris.fernald@outlook.com
On Tue, Jan 17, 2023 at 12:20:16PM +0000, Yao, Jiewen wrote:
> I have seen Gerd sent out V4 patch set for #4251. Is that not enough?
If that is merged first patch 17 should not be needed.
take care,
Gerd
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [edk2-devel] [PATCH v11 17/17] OfmvPkg: CI: Qemu tests: set CPU to Broadwell
2023-01-17 12:38 ` Gerd Hoffmann
@ 2023-01-17 14:49 ` Yao, Jiewen
2023-01-17 14:52 ` Oliver Steffen
0 siblings, 1 reply; 32+ messages in thread
From: Yao, Jiewen @ 2023-01-17 14:49 UTC (permalink / raw)
To: devel@edk2.groups.io, kraxel@redhat.com
Cc: osteffen@redhat.com, Andrew Fish, Ard Biesheuvel, Feng, Bob C,
Bi, Dandan, Wang, Jian J, Justen, Jordan L, Leif Lindholm,
Gao, Liming, Kinney, Michael D, Michael Kubacki, Ni, Ray,
Sami Mujawar, Sean Brogan, Chen, Christine, Pawel Polawski,
chris.fernald@outlook.com
Cool!
Then I suggest to drop patch 17.
> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Gerd
> Hoffmann
> Sent: Tuesday, January 17, 2023 8:39 PM
> To: Yao, Jiewen <jiewen.yao@intel.com>
> Cc: devel@edk2.groups.io; osteffen@redhat.com; Andrew Fish
> <afish@apple.com>; Ard Biesheuvel <ardb+tianocore@kernel.org>; Feng,
> Bob C <bob.c.feng@intel.com>; Bi, Dandan <dandan.bi@intel.com>; Wang,
> Jian J <jian.j.wang@intel.com>; Justen, Jordan L <jordan.l.justen@intel.com>;
> Leif Lindholm <quic_llindhol@quicinc.com>; Gao, Liming
> <gaoliming@byosoft.com.cn>; Kinney, Michael D
> <michael.d.kinney@intel.com>; Michael Kubacki
> <mikuback@linux.microsoft.com>; Ni, Ray <ray.ni@intel.com>; Sami
> Mujawar <sami.mujawar@arm.com>; Sean Brogan
> <sean.brogan@microsoft.com>; Chen, Christine <yuwei.chen@intel.com>;
> Pawel Polawski <ppolawsk@redhat.com>; chris.fernald@outlook.com
> Subject: Re: [edk2-devel] [PATCH v11 17/17] OfmvPkg: CI: Qemu tests: set
> CPU to Broadwell
>
> On Tue, Jan 17, 2023 at 12:20:16PM +0000, Yao, Jiewen wrote:
> > I have seen Gerd sent out V4 patch set for #4251. Is that not enough?
>
> If that is merged first patch 17 should not be needed.
>
> take care,
> Gerd
>
>
>
>
>
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [edk2-devel] [PATCH v11 17/17] OfmvPkg: CI: Qemu tests: set CPU to Broadwell
2023-01-17 14:49 ` Yao, Jiewen
@ 2023-01-17 14:52 ` Oliver Steffen
0 siblings, 0 replies; 32+ messages in thread
From: Oliver Steffen @ 2023-01-17 14:52 UTC (permalink / raw)
To: Yao, Jiewen
Cc: devel@edk2.groups.io, kraxel@redhat.com, Andrew Fish,
Ard Biesheuvel, Feng, Bob C, Bi, Dandan, Wang, Jian J,
Justen, Jordan L, Leif Lindholm, Gao, Liming, Kinney, Michael D,
Michael Kubacki, Ni, Ray, Sami Mujawar, Sean Brogan,
Chen, Christine, Pawel Polawski, chris.fernald@outlook.com
[-- Attachment #1: Type: text/plain, Size: 1408 bytes --]
On Tue, Jan 17, 2023 at 3:50 PM Yao, Jiewen <jiewen.yao@intel.com> wrote:
> Cool!
> Then I suggest to drop patch 17.
>
Perfectly fine with me.
>
> > -----Original Message-----
> > From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Gerd
> > Hoffmann
> > Sent: Tuesday, January 17, 2023 8:39 PM
> > To: Yao, Jiewen <jiewen.yao@intel.com>
> > Cc: devel@edk2.groups.io; osteffen@redhat.com; Andrew Fish
> > <afish@apple.com>; Ard Biesheuvel <ardb+tianocore@kernel.org>; Feng,
> > Bob C <bob.c.feng@intel.com>; Bi, Dandan <dandan.bi@intel.com>; Wang,
> > Jian J <jian.j.wang@intel.com>; Justen, Jordan L <
> jordan.l.justen@intel.com>;
> > Leif Lindholm <quic_llindhol@quicinc.com>; Gao, Liming
> > <gaoliming@byosoft.com.cn>; Kinney, Michael D
> > <michael.d.kinney@intel.com>; Michael Kubacki
> > <mikuback@linux.microsoft.com>; Ni, Ray <ray.ni@intel.com>; Sami
> > Mujawar <sami.mujawar@arm.com>; Sean Brogan
> > <sean.brogan@microsoft.com>; Chen, Christine <yuwei.chen@intel.com>;
> > Pawel Polawski <ppolawsk@redhat.com>; chris.fernald@outlook.com
> > Subject: Re: [edk2-devel] [PATCH v11 17/17] OfmvPkg: CI: Qemu tests: set
> > CPU to Broadwell
> >
> > On Tue, Jan 17, 2023 at 12:20:16PM +0000, Yao, Jiewen wrote:
> > > I have seen Gerd sent out V4 patch set for #4251. Is that not enough?
> >
> > If that is merged first patch 17 should not be needed.
> >
> > take care,
> > Gerd
> >
>
>
[-- Attachment #2: Type: text/html, Size: 3618 bytes --]
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v11 00/17] CI: Use Fedora 35 container for Linux jobs
2023-01-16 17:40 [PATCH v11 00/17] CI: Use Fedora 35 container for Linux jobs Oliver Steffen
` (16 preceding siblings ...)
2023-01-16 17:40 ` [PATCH v11 17/17] OfmvPkg: CI: Qemu tests: set CPU to Broadwell Oliver Steffen
@ 2023-01-17 18:22 ` Ard Biesheuvel
17 siblings, 0 replies; 32+ messages in thread
From: Ard Biesheuvel @ 2023-01-17 18:22 UTC (permalink / raw)
To: Oliver Steffen
Cc: devel, Andrew Fish, Ard Biesheuvel, Bob Feng, Dandan Bi,
Gerd Hoffmann, Jian J Wang, Jiewen Yao, Jordan Justen,
Leif Lindholm, Liming Gao, Michael D Kinney, Michael Kubacki,
Ray Ni, Sami Mujawar, Sean Brogan, Yuwei Chen, Pawel Polawski,
chris.fernald, Chris Fernald
On Mon, 16 Jan 2023 at 18:41, Oliver Steffen <osteffen@redhat.com> wrote:
>
> Update CI, run all Linux (aka Ubuntu-GCC5) based jobs in custom
> containers. This decouples the CI environment from the virtual machine
> images that Azure DevOps provides. The currently used ubuntu-18.04 image
> has been deprecated for a while now and will be removed soon.
>
> The container image provides the required compiler toolchains and Qemu
> for the supported architectures. These are then no longer downloaded at
> runtime, avoiding CI failures due to download errors. This approach also
> makes it easier to switch to other or newer compilers. It makes the CI
> setup independent from the default images that Azure DevOps provides.
> It can also help debugging CI problems, because the CI environment
> can be reproduced on a local machine.
>
> The container images are hosted on ghcr.io and are automatically
> generated using GitHub Actions. The Dockerfiles are maintained in the
> Tianocore "containers" repository:
> https://github.com/tianocore/containers.
>
> The current image is based on Fedora 35, with gcc 11. Fedora was chosen
> because of its fast release cycle which makes it easy to keep the
> toolchains up-to-date.
>
> Some further possible changes not included in this series:
> - The Tianocore/containers repository provides stack of layered images.
> One image for general purpose (build+test) and build-only jobs.
> The build+test image is based on the build-only one and adds Qemu,
> for the testing job that involve Qemu. The work in the image side
> is done, we just need to change the CI setup accordingly.
> This patch set uses the build+test images for all jobs.
> - Further reduce the number of external dependencies that need to be
> downloaded at runtime. Candidates are iasl and nasm, which are already
> included in the image but not used yet.
>
> PR: https://github.com/tianocore/edk2/pull/3904
>
> v11:
> - Rebase to current master branch
> - Use lasted image, includes Qemu 7.2
> - Set container image URL in a central place (the defaults.yml template)
>
> v10:
> - Split commits by package
> - Use ubuntu-22.04 as vm_image (instead of the floating ubuntu-latest)
> - Some fixups around the Python version template argument and defaults.yml
> - Fix some commit messages
> - Add license statement to defaults.yml
>
> v9:
> - Drop the "Don't install cspell" patch
> - Use explicitly use ubuntu-22.04 as vm_image
>
> v8:
> - Use updated container image that contains gcc for LoongArch64.
> - Remove ext_dep files for the LoongArch64 gcc.
> - Don't change the scopes in .pytool/CISettings.py if running Linux CI
> it.
> - Split commits that touch multiple packages.
> - Use the smaller "build" image for jobs that allow it.
> - Add a CI template file as a central place to define the default Python version
> and use it where needed.
>
> v7:
> - Rebase to latest master branch.
> - Use latest Fedora 35 CI image.
> - Stop using the ubuntu-18.04 vm_image since this will no longer be available
> after Dec 1st. Use ubuntu-latest instead.
>
> v6:
> - Include suggestions by Chris Fernald.
> - Added a parameter for the container image to the job template, makes usage
> of containers optional.
> - Added a parameter to configure the Python version to download. Allows
> using Python from the VM/container image also.
> - Restructure the commits (no further functional changes).
>
> v5:
> - Update image
>
> v4:
> - Use the latest image from the tianocode/containers repository which
> - does not include acpica-tools
> - includes Pyhton 3.10
>
> v3:
> - Use the latest image from the tianocode/containers repository which
> pins down version numbers of gcc, iasl, and nasm in the Dockerfile.
>
> v2:
> - Images are now hosted under the Tianocore Organization
> https://github.com/tianocore/containers
>
> v1:
> - Thread: https://edk2.groups.io/g/devel/message/89058
> - Images were hosted at https://github.com/osteffenrh/edk2-build-images
>
> Signed-off-by: Oliver Steffen <osteffen@redhat.com>
> Acked-by: Ard Biesheuvel <ardb@kernel.org>
> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
> Reviewed-by: Chris Fernald <chfernal@microsoft.com>
>
> Oliver Steffen (17):
> CI: make Python version configurable
> ArmVirtPkg: CI: use Python version from defaults template
> EmulatorPkg: CI: use Python version from defaults template
> OvmfPkg: CI: use Python version from defaults template
> CI: add ~/.local/bin to PATH (Linux only)
> CI: Allow running in a container.
> CI: Use Fedora 35 container (Linux only)
> ArmVirtPkg: CI: Use Fedora 35 container (Linux only)
> EmulatorPkg: CI: Use Fedora 35 container (Linux only)
> OvmfPkg: CI: Use Fedora 35 container (Linux only)
> .pytool: CISettings.py: don't add scopes for GCC
> BaseTools: remove ext_dep files for gcc
> ArmVirtPkg: CI: use ubuntu-22.04 vm_image (Linux only)
> EmulatorPgk: CI: use ubuntu-22.04 vm_image (Linux only)
> OvmfPkg: CI: use ubuntu-22.04 vm_image (Linux only)
> CI: use ubuntu-22.04 image (Linux only)
Thanks
I have created a PR here
https://github.com/tianocore/edk2/pull/3913
with all the acks added.
> OfmvPkg: CI: Qemu tests: set CPU to Broadwell
>
... and I omitted this one (as discussed on the thread)
> .azurepipelines/Ubuntu-GCC5.yml | 12 +++++-----
> .azurepipelines/Windows-VS2019.yml | 4 ++++
> .../templates/basetools-build-steps.yml | 9 --------
> .azurepipelines/templates/defaults.yml | 12 ++++++++++
> .../templates/platform-build-run-steps.yml | 12 +++++++++-
> .../templates/pr-gate-build-job.yml | 6 +++++
> .azurepipelines/templates/pr-gate-steps.yml | 12 ++++++++--
> .pytool/CISettings.py | 9 --------
> .../.azurepipelines/Ubuntu-GCC5.yml | 12 +++++-----
> BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml | 21 ------------------
> BaseTools/Bin/gcc_arm_linux_ext_dep.yaml | 21 ------------------
> ...gcc_loongarch64_unknown_linux_ext_dep.yaml | 22 -------------------
> .../Bin/gcc_riscv64_unknown_ext_dep.yaml | 22 -------------------
> .../.azurepipelines/Ubuntu-GCC5.yml | 9 +++++++-
> .../.azurepipelines/Windows-VS2019.yml | 4 ++++
> .../.azurepipelines/Ubuntu-GCC5.yml | 12 +++++-----
> .../.azurepipelines/Windows-VS2019.yml | 5 +++++
> OvmfPkg/PlatformCI/PlatformBuildLib.py | 1 +
> 18 files changed, 82 insertions(+), 123 deletions(-)
> create mode 100644 .azurepipelines/templates/defaults.yml
> delete mode 100644 BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml
> delete mode 100644 BaseTools/Bin/gcc_arm_linux_ext_dep.yaml
> delete mode 100644 BaseTools/Bin/gcc_loongarch64_unknown_linux_ext_dep.yaml
> delete mode 100644 BaseTools/Bin/gcc_riscv64_unknown_ext_dep.yaml
>
> --
> 2.39.0
>
^ permalink raw reply [flat|nested] 32+ messages in thread