[edk2-devel] [PATCH v1 0/4] Use Qemu 8 in CI and enable SMP in tests

[edk2-devel] [PATCH v1 0/4] Use Qemu 8 in CI and enable SMP in tests

[Oliver Steffen]

REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4324
PR:  https://github.com/tianocore/edk2/pull/4711

This series switches the OVMF CI jobs on Windows to a recent Qemu
version and re-enables SMP in the Qemu "Run to shell" CI jobs.
The Qemu tests run a little bit slower in SMP mode and sometimes
hit the time limit of 2 minutes, thus this limit is increased to 4
minutes.

Together with 81cb0371f9db ("CI: Use latest image for Linux jobs (Qemu
8, gcc 12)" this series is sufficient to close Bug 4324.


Gerd Hoffmann (1):
  OvmfPkg/PlatformCI: Boot OVMF in SMP mode.

Oliver Steffen (3):
  OvmfPkg/PlatformCI: Use recent Qemu on Windows
  OvmfPkg/PlatformCI VS2019: Disable workaround for cpuhp bugfix
  CI: Increase time limit of "Run to shell" task to 4 minutes

 .../templates/platform-build-run-steps.yml          |  2 +-
 .../PlatformCI/.azurepipelines/Windows-VS2019.yml   |  4 ++--
 OvmfPkg/PlatformCI/PlatformBuildLib.py              | 13 +------------
 3 files changed, 4 insertions(+), 15 deletions(-)

-- 
2.41.0



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#107534): https://edk2.groups.io/g/devel/message/107534
Mute This Topic: https://groups.io/mt/100524063/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

[edk2-devel] [PATCH v1 1/4] OvmfPkg/PlatformCI: Use recent Qemu on Windows

[Oliver Steffen]

Bump the version of the Qemu chocolatey package up to 2023.7.25.
The Linux CI is already using Qemu 8.

REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4324

Signed-off-by: Oliver Steffen <osteffen@redhat.com>
---
 OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml b/OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml
index b3b91aa84ea0..f7cba193d737 100644
--- a/OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml
+++ b/OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml
@@ -139,7 +139,7 @@ jobs:
         run_flags: $(Run.Flags)
         usePythonVersion: ${{ variables.default_python_version }}
         extra_install_step:
-        - powershell: choco install qemu --version=2021.5.5; Write-Host "##vso[task.prependpath]c:\Program Files\qemu"
+        - powershell: choco install qemu --version=2023.7.25; Write-Host "##vso[task.prependpath]c:\Program Files\qemu"
           displayName: Install QEMU and Set QEMU on path # friendly name displayed in the UI
           condition: and(gt(variables.pkg_count, 0), succeeded())
 
-- 
2.41.0



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#107536): https://edk2.groups.io/g/devel/message/107536
Mute This Topic: https://groups.io/mt/100524066/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

[edk2-devel] [PATCH v1 2/4] OvmfPkg/PlatformCI VS2019: Disable workaround for cpuhp bugfix

[Oliver Steffen]

This reverts commit 3beb8c965455f4c1cc3184e36c627ef1d9bfe5f9.

Both Windows and Linux CI jobs are now using Qemu 8, this workaround is
no longer needed.

REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4324

Signed-off-by: Oliver Steffen <osteffen@redhat.com>
---
 .../PlatformCI/.azurepipelines/Windows-VS2019.yml    |  2 +-
 OvmfPkg/PlatformCI/PlatformBuildLib.py               | 12 ------------
 2 files changed, 1 insertion(+), 13 deletions(-)

diff --git a/OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml b/OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml
index f7cba193d737..cada81023548 100644
--- a/OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml
+++ b/OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml
@@ -24,7 +24,7 @@ jobs:
       package: 'OvmfPkg'
       vm_image: 'windows-2019'
       should_run: true
-      run_flags: "MAKE_STARTUP_NSH=TRUE QEMU_HEADLESS=TRUE QEMU_CPUHP_QUIRK=TRUE"
+      run_flags: "MAKE_STARTUP_NSH=TRUE QEMU_HEADLESS=TRUE"
 
     #Use matrix to speed up the build process
     strategy:
diff --git a/OvmfPkg/PlatformCI/PlatformBuildLib.py b/OvmfPkg/PlatformCI/PlatformBuildLib.py
index 64fca48a7d1d..c8e805c56616 100644
--- a/OvmfPkg/PlatformCI/PlatformBuildLib.py
+++ b/OvmfPkg/PlatformCI/PlatformBuildLib.py
@@ -170,7 +170,6 @@ class PlatformBuilder( UefiBuilder, BuildSettingsManager):
         self.env.SetValue("PRODUCT_NAME", "OVMF", "Platform Hardcoded")
         self.env.SetValue("MAKE_STARTUP_NSH", "FALSE", "Default to false")
         self.env.SetValue("QEMU_HEADLESS", "FALSE", "Default to false")
-        self.env.SetValue("QEMU_CPUHP_QUIRK", "FALSE", "Default to false")
         return 0
 
     def PlatformPreBuild(self):
@@ -211,17 +210,6 @@ class PlatformBuilder( UefiBuilder, BuildSettingsManager):
             args += " -pflash " + os.path.join(OutputPath_FV, "OVMF.fd")    # path to firmware
 
 
-        ###
-        ### NOTE This is a temporary workaround to allow platform CI to cope with
-        ###      a QEMU bug in the CPU hotplug code. Once the CI environment has
-        ###      been updated to carry a fixed version of QEMU, this can be
-        ###      removed again
-        ###
-        ### Bugzilla: https://bugzilla.tianocore.org/show_bug.cgi?id=4250
-        ###
-        if (self.env.GetValue("QEMU_CPUHP_QUIRK").upper() == "TRUE"):
-            args += "  -fw_cfg name=opt/org.tianocore/X-Cpuhp-Bugcheck-Override,string=yes"
-
         if (self.env.GetValue("MAKE_STARTUP_NSH").upper() == "TRUE"):
             f = open(os.path.join(VirtualDrive, "startup.nsh"), "w")
             f.write("BOOT SUCCESS !!! \n")
-- 
2.41.0



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#107535): https://edk2.groups.io/g/devel/message/107535
Mute This Topic: https://groups.io/mt/100524065/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

[edk2-devel] [PATCH v1 3/4] OvmfPkg/PlatformCI: Boot OVMF in SMP mode.

[Oliver Steffen]

From: Gerd Hoffmann <kraxel@redhat.com>

Increase the chance that CI finds bugs in MP changes.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
(cherry picked from commit f92a9dce10281c103b04d6b38283e0ff1d677b91)

Reapplying, since the CI (Windows and Linux) use Qemu 8 now, which works
with SMP again.

REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4324

Signed-off-by: Oliver Steffen <osteffen@redhat.com>
---
 OvmfPkg/PlatformCI/PlatformBuildLib.py | 1 +
 1 file changed, 1 insertion(+)

diff --git a/OvmfPkg/PlatformCI/PlatformBuildLib.py b/OvmfPkg/PlatformCI/PlatformBuildLib.py
index c8e805c56616..bca36d0bd758 100644
--- a/OvmfPkg/PlatformCI/PlatformBuildLib.py
+++ b/OvmfPkg/PlatformCI/PlatformBuildLib.py
@@ -195,6 +195,7 @@ class PlatformBuilder( UefiBuilder, BuildSettingsManager):
         args  = "-debugcon stdio"                                           # write messages to stdio
         args += " -global isa-debugcon.iobase=0x402"                        # debug messages out thru virtual io port
         args += " -net none"                                                # turn off network
+        args += " -smp 4"
         args += f" -drive file=fat:rw:{VirtualDrive},format=raw,media=disk" # Mount disk with startup.nsh
 
         if (self.env.GetValue("QEMU_HEADLESS").upper() == "TRUE"):
-- 
2.41.0



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#107537): https://edk2.groups.io/g/devel/message/107537
Mute This Topic: https://groups.io/mt/100524067/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

[edk2-devel] [PATCH v1 4/4] CI: Increase time limit of "Run to shell" task to 4 minutes

[Oliver Steffen]

The current job time limit of 2 minutes is just a little bit too short
for running Qemu in SMP mode.

Signed-off-by: Oliver Steffen <osteffen@redhat.com>
---
 .azurepipelines/templates/platform-build-run-steps.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.azurepipelines/templates/platform-build-run-steps.yml b/.azurepipelines/templates/platform-build-run-steps.yml
index 8be46cda0e2d..098078ffaca9 100644
--- a/.azurepipelines/templates/platform-build-run-steps.yml
+++ b/.azurepipelines/templates/platform-build-run-steps.yml
@@ -116,7 +116,7 @@ steps:
     filename: stuart_build
     arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}} TARGET=${{ parameters.build_target}} -a ${{ parameters.build_arch}} ${{ parameters.build_flags}} ${{ parameters.run_flags }} --FlashOnly
   condition: and(and(gt(variables.pkg_count, 0), succeeded()), eq(variables['Run'], true))
-  timeoutInMinutes: 2
+  timeoutInMinutes: 4
 
 # Copy the build logs to the artifact staging directory
 - task: CopyFiles@2
-- 
2.41.0



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#107538): https://edk2.groups.io/g/devel/message/107538
Mute This Topic: https://groups.io/mt/100524069/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] [PATCH v1 0/4] Use Qemu 8 in CI and enable SMP in tests

[Ard Biesheuvel]

On Thu, 3 Aug 2023 at 13:44, Oliver Steffen <osteffen@redhat.com> wrote:
>
> REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4324
> PR:  https://github.com/tianocore/edk2/pull/4711
>
> This series switches the OVMF CI jobs on Windows to a recent Qemu
> version and re-enables SMP in the Qemu "Run to shell" CI jobs.
> The Qemu tests run a little bit slower in SMP mode and sometimes
> hit the time limit of 2 minutes, thus this limit is increased to 4
> minutes.
>
> Together with 81cb0371f9db ("CI: Use latest image for Linux jobs (Qemu
> 8, gcc 12)" this series is sufficient to close Bug 4324.
>

Thanks a lot for taking care of this.

If nobody objects to increasing the Ci time limit, I intend to merge
this by end of tomorrow.

(I'd put the CI patch first, though)


>
> Gerd Hoffmann (1):
>   OvmfPkg/PlatformCI: Boot OVMF in SMP mode.
>
> Oliver Steffen (3):
>   OvmfPkg/PlatformCI: Use recent Qemu on Windows
>   OvmfPkg/PlatformCI VS2019: Disable workaround for cpuhp bugfix
>   CI: Increase time limit of "Run to shell" task to 4 minutes
>
>  .../templates/platform-build-run-steps.yml          |  2 +-
>  .../PlatformCI/.azurepipelines/Windows-VS2019.yml   |  4 ++--
>  OvmfPkg/PlatformCI/PlatformBuildLib.py              | 13 +------------
>  3 files changed, 4 insertions(+), 15 deletions(-)
>
> --
> 2.41.0
>


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#107539): https://edk2.groups.io/g/devel/message/107539
Mute This Topic: https://groups.io/mt/100524063/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] [PATCH v1 0/4] Use Qemu 8 in CI and enable SMP in tests

[Laszlo Ersek]

On 8/3/23 15:03, Ard Biesheuvel wrote:
> On Thu, 3 Aug 2023 at 13:44, Oliver Steffen <osteffen@redhat.com> wrote:
>>
>> REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4324
>> PR:  https://github.com/tianocore/edk2/pull/4711
>>
>> This series switches the OVMF CI jobs on Windows to a recent Qemu
>> version and re-enables SMP in the Qemu "Run to shell" CI jobs.
>> The Qemu tests run a little bit slower in SMP mode and sometimes
>> hit the time limit of 2 minutes, thus this limit is increased to 4
>> minutes.
>>
>> Together with 81cb0371f9db ("CI: Use latest image for Linux jobs (Qemu
>> 8, gcc 12)" this series is sufficient to close Bug 4324.
>>
> 
> Thanks a lot for taking care of this.
> 
> If nobody objects to increasing the Ci time limit, I intend to merge
> this by end of tomorrow.
> 
> (I'd put the CI patch first, though)

Yes, that makes sense.

Thank you Oliver for writing this series.

With the patch ordering suggested by Ard:

Reviewed-by: Laszlo Ersek <lersek@redhat.com>

Thanks!
Laszlo

> 
> 
>>
>> Gerd Hoffmann (1):
>>   OvmfPkg/PlatformCI: Boot OVMF in SMP mode.
>>
>> Oliver Steffen (3):
>>   OvmfPkg/PlatformCI: Use recent Qemu on Windows
>>   OvmfPkg/PlatformCI VS2019: Disable workaround for cpuhp bugfix
>>   CI: Increase time limit of "Run to shell" task to 4 minutes
>>
>>  .../templates/platform-build-run-steps.yml          |  2 +-
>>  .../PlatformCI/.azurepipelines/Windows-VS2019.yml   |  4 ++--
>>  OvmfPkg/PlatformCI/PlatformBuildLib.py              | 13 +------------
>>  3 files changed, 4 insertions(+), 15 deletions(-)
>>
>> --
>> 2.41.0
>>
> 



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#107544): https://edk2.groups.io/g/devel/message/107544
Mute This Topic: https://groups.io/mt/100524063/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/leave/12367111/7686176/1913456212/xyzzy [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] [PATCH v1 0/4] Use Qemu 8 in CI and enable SMP in tests

[Oliver Steffen]

On Thu, Aug 3, 2023 at 3:51 PM Laszlo Ersek <lersek@redhat.com> wrote:
>
> On 8/3/23 15:03, Ard Biesheuvel wrote:
> > On Thu, 3 Aug 2023 at 13:44, Oliver Steffen <osteffen@redhat.com> wrote:
> >>
> >> REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4324
> >> PR:  https://github.com/tianocore/edk2/pull/4711
> >>
> >> This series switches the OVMF CI jobs on Windows to a recent Qemu
> >> version and re-enables SMP in the Qemu "Run to shell" CI jobs.
> >> The Qemu tests run a little bit slower in SMP mode and sometimes
> >> hit the time limit of 2 minutes, thus this limit is increased to 4
> >> minutes.
> >>
> >> Together with 81cb0371f9db ("CI: Use latest image for Linux jobs (Qemu
> >> 8, gcc 12)" this series is sufficient to close Bug 4324.
> >>
> >
> > Thanks a lot for taking care of this.
> >
> > If nobody objects to increasing the Ci time limit, I intend to merge
> > this by end of tomorrow.
> >
> > (I'd put the CI patch first, though)
>
> Yes, that makes sense.
Indeed, sorry about that and thanks for taking care of it!

-Oliver

>
> Thank you Oliver for writing this series.
>
> With the patch ordering suggested by Ard:
>
> Reviewed-by: Laszlo Ersek <lersek@redhat.com>
>
> Thanks!
> Laszlo
>
> >
> >
> >>
> >> Gerd Hoffmann (1):
> >>   OvmfPkg/PlatformCI: Boot OVMF in SMP mode.
> >>
> >> Oliver Steffen (3):
> >>   OvmfPkg/PlatformCI: Use recent Qemu on Windows
> >>   OvmfPkg/PlatformCI VS2019: Disable workaround for cpuhp bugfix
> >>   CI: Increase time limit of "Run to shell" task to 4 minutes
> >>
> >>  .../templates/platform-build-run-steps.yml          |  2 +-
> >>  .../PlatformCI/.azurepipelines/Windows-VS2019.yml   |  4 ++--
> >>  OvmfPkg/PlatformCI/PlatformBuildLib.py              | 13 +------------
> >>  3 files changed, 4 insertions(+), 15 deletions(-)
> >>
> >> --
> >> 2.41.0
> >>
> >
>



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#107546): https://edk2.groups.io/g/devel/message/107546
Mute This Topic: https://groups.io/mt/100524063/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] [PATCH v1 0/4] Use Qemu 8 in CI and enable SMP in tests

[Michael Kubacki]

For the series.

Reviewed-by: Michael Kubacki <michael.kubacki@microsoft.com>

On 8/3/2023 7:44 AM, Oliver Steffen wrote:
> REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4324
> PR:  https://github.com/tianocore/edk2/pull/4711
> 
> This series switches the OVMF CI jobs on Windows to a recent Qemu
> version and re-enables SMP in the Qemu "Run to shell" CI jobs.
> The Qemu tests run a little bit slower in SMP mode and sometimes
> hit the time limit of 2 minutes, thus this limit is increased to 4
> minutes.
> 
> Together with 81cb0371f9db ("CI: Use latest image for Linux jobs (Qemu
> 8, gcc 12)" this series is sufficient to close Bug 4324.
> 
> 
> Gerd Hoffmann (1):
>    OvmfPkg/PlatformCI: Boot OVMF in SMP mode.
> 
> Oliver Steffen (3):
>    OvmfPkg/PlatformCI: Use recent Qemu on Windows
>    OvmfPkg/PlatformCI VS2019: Disable workaround for cpuhp bugfix
>    CI: Increase time limit of "Run to shell" task to 4 minutes
> 
>   .../templates/platform-build-run-steps.yml          |  2 +-
>   .../PlatformCI/.azurepipelines/Windows-VS2019.yml   |  4 ++--
>   OvmfPkg/PlatformCI/PlatformBuildLib.py              | 13 +------------
>   3 files changed, 4 insertions(+), 15 deletions(-)
> 


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#107547): https://edk2.groups.io/g/devel/message/107547
Mute This Topic: https://groups.io/mt/100524063/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] [PATCH v1 0/4] Use Qemu 8 in CI and enable SMP in tests

[Ard Biesheuvel]

On Thu, 3 Aug 2023 at 16:37, Michael Kubacki
<mikuback@linux.microsoft.com> wrote:
>
> For the series.
>
> Reviewed-by: Michael Kubacki <michael.kubacki@microsoft.com>
>

Thanks

I've pushed this to the CI but Platform_CI OVMF_IA32X64_FULL_DEBUG
keeps timing out for me

https://github.com/tianocore/edk2/pull/4718

Might it help if we restrict SMP to a subset of the builds? And would
it help if we added "--accel=tcg,thread=multi" ?



> On 8/3/2023 7:44 AM, Oliver Steffen wrote:
> > REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4324
> > PR:  https://github.com/tianocore/edk2/pull/4711
> >
> > This series switches the OVMF CI jobs on Windows to a recent Qemu
> > version and re-enables SMP in the Qemu "Run to shell" CI jobs.
> > The Qemu tests run a little bit slower in SMP mode and sometimes
> > hit the time limit of 2 minutes, thus this limit is increased to 4
> > minutes.
> >
> > Together with 81cb0371f9db ("CI: Use latest image for Linux jobs (Qemu
> > 8, gcc 12)" this series is sufficient to close Bug 4324.
> >
> >
> > Gerd Hoffmann (1):
> >    OvmfPkg/PlatformCI: Boot OVMF in SMP mode.
> >
> > Oliver Steffen (3):
> >    OvmfPkg/PlatformCI: Use recent Qemu on Windows
> >    OvmfPkg/PlatformCI VS2019: Disable workaround for cpuhp bugfix
> >    CI: Increase time limit of "Run to shell" task to 4 minutes
> >
> >   .../templates/platform-build-run-steps.yml          |  2 +-
> >   .../PlatformCI/.azurepipelines/Windows-VS2019.yml   |  4 ++--
> >   OvmfPkg/PlatformCI/PlatformBuildLib.py              | 13 +------------
> >   3 files changed, 4 insertions(+), 15 deletions(-)
> >
>
>
> 
>
>


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#107553): https://edk2.groups.io/g/devel/message/107553
Mute This Topic: https://groups.io/mt/100524063/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] [PATCH v1 0/4] Use Qemu 8 in CI and enable SMP in tests

[Michael D Kinney]

Hi Ard,

This is the email with the analysis and the flag
setting required for SMP to work with SMM enabled for
IA32/X64 QEMU.

https://edk2.groups.io/g/devel/message/102364


I tried latest version of QEMU earlier in the Summer
and did not see any improvements.

Mike


> -----Original Message-----
> From: Ard Biesheuvel <ardb@kernel.org>
> Sent: Thursday, August 3, 2023 9:54 AM
> To: devel@edk2.groups.io; mikuback@linux.microsoft.com
> Cc: osteffen@redhat.com; Feng, Bob C <bob.c.feng@intel.com>; Gerd
> Hoffmann <kraxel@redhat.com>; Yao, Jiewen <jiewen.yao@intel.com>;
> Justen, Jordan L <jordan.l.justen@intel.com>; Gao, Liming
> <gaoliming@byosoft.com.cn>; Kinney, Michael D
> <michael.d.kinney@intel.com>; Rebecca Cran <rebecca@bsdio.com>; Sean
> Brogan <sean.brogan@microsoft.com>; Chen, Christine
> <yuwei.chen@intel.com>; Laszlo Ersek <lersek@redhat.com>
> Subject: Re: [edk2-devel] [PATCH v1 0/4] Use Qemu 8 in CI and enable SMP
> in tests
> 
> On Thu, 3 Aug 2023 at 16:37, Michael Kubacki
> <mikuback@linux.microsoft.com> wrote:
> >
> > For the series.
> >
> > Reviewed-by: Michael Kubacki <michael.kubacki@microsoft.com>
> >
> 
> Thanks
> 
> I've pushed this to the CI but Platform_CI OVMF_IA32X64_FULL_DEBUG
> keeps timing out for me
> 
> https://github.com/tianocore/edk2/pull/4718
> 
> Might it help if we restrict SMP to a subset of the builds? And would
> it help if we added "--accel=tcg,thread=multi" ?
> 
> 
> 
> > On 8/3/2023 7:44 AM, Oliver Steffen wrote:
> > > REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4324
> > > PR:  https://github.com/tianocore/edk2/pull/4711
> > >
> > > This series switches the OVMF CI jobs on Windows to a recent Qemu
> > > version and re-enables SMP in the Qemu "Run to shell" CI jobs.
> > > The Qemu tests run a little bit slower in SMP mode and sometimes
> > > hit the time limit of 2 minutes, thus this limit is increased to 4
> > > minutes.
> > >
> > > Together with 81cb0371f9db ("CI: Use latest image for Linux jobs
> (Qemu
> > > 8, gcc 12)" this series is sufficient to close Bug 4324.
> > >
> > >
> > > Gerd Hoffmann (1):
> > >    OvmfPkg/PlatformCI: Boot OVMF in SMP mode.
> > >
> > > Oliver Steffen (3):
> > >    OvmfPkg/PlatformCI: Use recent Qemu on Windows
> > >    OvmfPkg/PlatformCI VS2019: Disable workaround for cpuhp bugfix
> > >    CI: Increase time limit of "Run to shell" task to 4 minutes
> > >
> > >   .../templates/platform-build-run-steps.yml          |  2 +-
> > >   .../PlatformCI/.azurepipelines/Windows-VS2019.yml   |  4 ++--
> > >   OvmfPkg/PlatformCI/PlatformBuildLib.py              | 13 +--------
> ----
> > >   3 files changed, 4 insertions(+), 15 deletions(-)
> > >
> >
> >
> > 
> >
> >


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#107556): https://edk2.groups.io/g/devel/message/107556
Mute This Topic: https://groups.io/mt/100524063/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/leave/12367111/7686176/1913456212/xyzzy [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] [PATCH v1 0/4] Use Qemu 8 in CI and enable SMP in tests

[Ard Biesheuvel]

On Thu, 3 Aug 2023 at 21:21, Kinney, Michael D
<michael.d.kinney@intel.com> wrote:
>
> Hi Ard,
>
> This is the email with the analysis and the flag
> setting required for SMP to work with SMM enabled for
> IA32/X64 QEMU.
>
> https://edk2.groups.io/g/devel/message/102364
>
>
> I tried latest version of QEMU earlier in the Summer
> and did not see any improvements.
>

Ah thanks, I had forgotten all about that rabbit hole you went down :-)

So let's respin this with -smp 4 --accel=tcg,thread=single. It is
still an improvement to exercise the MP code even if it doesn't
strictly execute concurrently. (It will be very hard to
catch/reproduce true concurrency issues in TCG anyway)

Oliver, mind spinning a v2?

Thanks,



> > -----Original Message-----
> > From: Ard Biesheuvel <ardb@kernel.org>
> > Sent: Thursday, August 3, 2023 9:54 AM
> > To: devel@edk2.groups.io; mikuback@linux.microsoft.com
> > Cc: osteffen@redhat.com; Feng, Bob C <bob.c.feng@intel.com>; Gerd
> > Hoffmann <kraxel@redhat.com>; Yao, Jiewen <jiewen.yao@intel.com>;
> > Justen, Jordan L <jordan.l.justen@intel.com>; Gao, Liming
> > <gaoliming@byosoft.com.cn>; Kinney, Michael D
> > <michael.d.kinney@intel.com>; Rebecca Cran <rebecca@bsdio.com>; Sean
> > Brogan <sean.brogan@microsoft.com>; Chen, Christine
> > <yuwei.chen@intel.com>; Laszlo Ersek <lersek@redhat.com>
> > Subject: Re: [edk2-devel] [PATCH v1 0/4] Use Qemu 8 in CI and enable SMP
> > in tests
> >
> > On Thu, 3 Aug 2023 at 16:37, Michael Kubacki
> > <mikuback@linux.microsoft.com> wrote:
> > >
> > > For the series.
> > >
> > > Reviewed-by: Michael Kubacki <michael.kubacki@microsoft.com>
> > >
> >
> > Thanks
> >
> > I've pushed this to the CI but Platform_CI OVMF_IA32X64_FULL_DEBUG
> > keeps timing out for me
> >
> > https://github.com/tianocore/edk2/pull/4718
> >
> > Might it help if we restrict SMP to a subset of the builds? And would
> > it help if we added "--accel=tcg,thread=multi" ?
> >
> >
> >
> > > On 8/3/2023 7:44 AM, Oliver Steffen wrote:
> > > > REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4324
> > > > PR:  https://github.com/tianocore/edk2/pull/4711
> > > >
> > > > This series switches the OVMF CI jobs on Windows to a recent Qemu
> > > > version and re-enables SMP in the Qemu "Run to shell" CI jobs.
> > > > The Qemu tests run a little bit slower in SMP mode and sometimes
> > > > hit the time limit of 2 minutes, thus this limit is increased to 4
> > > > minutes.
> > > >
> > > > Together with 81cb0371f9db ("CI: Use latest image for Linux jobs
> > (Qemu
> > > > 8, gcc 12)" this series is sufficient to close Bug 4324.
> > > >
> > > >
> > > > Gerd Hoffmann (1):
> > > >    OvmfPkg/PlatformCI: Boot OVMF in SMP mode.
> > > >
> > > > Oliver Steffen (3):
> > > >    OvmfPkg/PlatformCI: Use recent Qemu on Windows
> > > >    OvmfPkg/PlatformCI VS2019: Disable workaround for cpuhp bugfix
> > > >    CI: Increase time limit of "Run to shell" task to 4 minutes
> > > >
> > > >   .../templates/platform-build-run-steps.yml          |  2 +-
> > > >   .../PlatformCI/.azurepipelines/Windows-VS2019.yml   |  4 ++--
> > > >   OvmfPkg/PlatformCI/PlatformBuildLib.py              | 13 +--------
> > ----
> > > >   3 files changed, 4 insertions(+), 15 deletions(-)
> > > >
> > >
> > >
> > > 
> > >
> > >


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#107557): https://edk2.groups.io/g/devel/message/107557
Mute This Topic: https://groups.io/mt/100524063/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] [PATCH v1 0/4] Use Qemu 8 in CI and enable SMP in tests

[Ard Biesheuvel]

On Thu, 3 Aug 2023 at 23:25, Ard Biesheuvel <ardb@kernel.org> wrote:
>
> On Thu, 3 Aug 2023 at 21:21, Kinney, Michael D
> <michael.d.kinney@intel.com> wrote:
> >
> > Hi Ard,
> >
> > This is the email with the analysis and the flag
> > setting required for SMP to work with SMM enabled for
> > IA32/X64 QEMU.
> >
> > https://edk2.groups.io/g/devel/message/102364
> >
> >
> > I tried latest version of QEMU earlier in the Summer
> > and did not see any improvements.
> >
>
> Ah thanks, I had forgotten all about that rabbit hole you went down :-)
>
> So let's respin this with -smp 4 --accel=tcg,thread=single. It is
> still an improvement to exercise the MP code even if it doesn't
> strictly execute concurrently. (It will be very hard to
> catch/reproduce true concurrency issues in TCG anyway)
>

Something like this is what I had in mind:

--- a/OvmfPkg/PlatformCI/PlatformBuildLib.py
+++ b/OvmfPkg/PlatformCI/PlatformBuildLib.py
@@ -195,6 +195,7 @@ class PlatformBuilder( UefiBuilder, BuildSettingsManager):
         args  = "-debugcon stdio"
      # write messages to stdio
         args += " -global isa-debugcon.iobase=0x402"
      # debug messages out thru virtual io port
         args += " -net none"
      # turn off network
+        args += " -smp 4"
         args += f" -drive
file=fat:rw:{VirtualDrive},format=raw,media=disk" # Mount disk with
startup.nsh

         if (self.env.GetValue("QEMU_HEADLESS").upper() == "TRUE"):
@@ -202,6 +203,7 @@ class PlatformBuilder( UefiBuilder, BuildSettingsManager):

         if (self.env.GetBuildValue("SMM_REQUIRE") == "1"):
             args += " -machine q35,smm=on" #,accel=(tcg|kvm)"
+            args += " --accel tcg,thread=single"
             #args += " -m ..."
             args += " -global driver=cfi.pflash01,property=secure,value=on"
             args += " -drive if=pflash,format=raw,unit=0,file=" +
os.path.join(OutputPath_FV, "OVMF_CODE.fd") + ",readonly=on"

I can just fold that in if nobody minds. I suspect the time limit
increase may not be needed either this way.


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#107566): https://edk2.groups.io/g/devel/message/107566
Mute This Topic: https://groups.io/mt/100524063/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] [PATCH v1 0/4] Use Qemu 8 in CI and enable SMP in tests

[Oliver Steffen]

On Fri, Aug 4, 2023 at 3:17 PM Ard Biesheuvel <ardb@kernel.org> wrote:
>
> On Thu, 3 Aug 2023 at 23:25, Ard Biesheuvel <ardb@kernel.org> wrote:
> >
> > On Thu, 3 Aug 2023 at 21:21, Kinney, Michael D
> > <michael.d.kinney@intel.com> wrote:
> > >
> > > Hi Ard,
> > >
> > > This is the email with the analysis and the flag
> > > setting required for SMP to work with SMM enabled for
> > > IA32/X64 QEMU.
> > >
> > > https://edk2.groups.io/g/devel/message/102364
> > >
> > >
> > > I tried latest version of QEMU earlier in the Summer
> > > and did not see any improvements.
> > >
> >
> > Ah thanks, I had forgotten all about that rabbit hole you went down :-)
> >
> > So let's respin this with -smp 4 --accel=tcg,thread=single. It is
> > still an improvement to exercise the MP code even if it doesn't
> > strictly execute concurrently. (It will be very hard to
> > catch/reproduce true concurrency issues in TCG anyway)
> >
>
> Something like this is what I had in mind:
>
> --- a/OvmfPkg/PlatformCI/PlatformBuildLib.py
> +++ b/OvmfPkg/PlatformCI/PlatformBuildLib.py
> @@ -195,6 +195,7 @@ class PlatformBuilder( UefiBuilder, BuildSettingsManager):
>          args  = "-debugcon stdio"
>       # write messages to stdio
>          args += " -global isa-debugcon.iobase=0x402"
>       # debug messages out thru virtual io port
>          args += " -net none"
>       # turn off network
> +        args += " -smp 4"
>          args += f" -drive
> file=fat:rw:{VirtualDrive},format=raw,media=disk" # Mount disk with
> startup.nsh
>
>          if (self.env.GetValue("QEMU_HEADLESS").upper() == "TRUE"):
> @@ -202,6 +203,7 @@ class PlatformBuilder( UefiBuilder, BuildSettingsManager):
>
>          if (self.env.GetBuildValue("SMM_REQUIRE") == "1"):
>              args += " -machine q35,smm=on" #,accel=(tcg|kvm)"
> +            args += " --accel tcg,thread=single"
>              #args += " -m ..."
>              args += " -global driver=cfi.pflash01,property=secure,value=on"
>              args += " -drive if=pflash,format=raw,unit=0,file=" +
> os.path.join(OutputPath_FV, "OVMF_CODE.fd") + ",readonly=on"
>
> I can just fold that in if nobody minds. I suspect the time limit
> increase may not be needed either this way.

Yes, please do. Since you already have the changes ready.
I was just looking into it now.
If we do not need the larger timeout, even better.

-Oliver



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#107567): https://edk2.groups.io/g/devel/message/107567
Mute This Topic: https://groups.io/mt/100524063/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] [PATCH v1 0/4] Use Qemu 8 in CI and enable SMP in tests

[Laszlo Ersek]

On 8/3/23 23:25, Ard Biesheuvel wrote:
> On Thu, 3 Aug 2023 at 21:21, Kinney, Michael D
> <michael.d.kinney@intel.com> wrote:
>>
>> Hi Ard,
>>
>> This is the email with the analysis and the flag
>> setting required for SMP to work with SMM enabled for
>> IA32/X64 QEMU.
>>
>> https://edk2.groups.io/g/devel/message/102364
>>
>>
>> I tried latest version of QEMU earlier in the Summer
>> and did not see any improvements.
>>
> 
> Ah thanks, I had forgotten all about that rabbit hole you went down :-)

Sorry Mike, I had also forgotten about that problem.

> 
> So let's respin this with -smp 4 --accel=tcg,thread=single. It is
> still an improvement to exercise the MP code even if it doesn't
> strictly execute concurrently. (It will be very hard to
> catch/reproduce true concurrency issues in TCG anyway)

I think that's a good workaround.

Thanks
Laszlo

> 
> Oliver, mind spinning a v2?
> 
> Thanks,
> 
> 
> 
>>> -----Original Message-----
>>> From: Ard Biesheuvel <ardb@kernel.org>
>>> Sent: Thursday, August 3, 2023 9:54 AM
>>> To: devel@edk2.groups.io; mikuback@linux.microsoft.com
>>> Cc: osteffen@redhat.com; Feng, Bob C <bob.c.feng@intel.com>; Gerd
>>> Hoffmann <kraxel@redhat.com>; Yao, Jiewen <jiewen.yao@intel.com>;
>>> Justen, Jordan L <jordan.l.justen@intel.com>; Gao, Liming
>>> <gaoliming@byosoft.com.cn>; Kinney, Michael D
>>> <michael.d.kinney@intel.com>; Rebecca Cran <rebecca@bsdio.com>; Sean
>>> Brogan <sean.brogan@microsoft.com>; Chen, Christine
>>> <yuwei.chen@intel.com>; Laszlo Ersek <lersek@redhat.com>
>>> Subject: Re: [edk2-devel] [PATCH v1 0/4] Use Qemu 8 in CI and enable SMP
>>> in tests
>>>
>>> On Thu, 3 Aug 2023 at 16:37, Michael Kubacki
>>> <mikuback@linux.microsoft.com> wrote:
>>>>
>>>> For the series.
>>>>
>>>> Reviewed-by: Michael Kubacki <michael.kubacki@microsoft.com>
>>>>
>>>
>>> Thanks
>>>
>>> I've pushed this to the CI but Platform_CI OVMF_IA32X64_FULL_DEBUG
>>> keeps timing out for me
>>>
>>> https://github.com/tianocore/edk2/pull/4718
>>>
>>> Might it help if we restrict SMP to a subset of the builds? And would
>>> it help if we added "--accel=tcg,thread=multi" ?
>>>
>>>
>>>
>>>> On 8/3/2023 7:44 AM, Oliver Steffen wrote:
>>>>> REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4324
>>>>> PR:  https://github.com/tianocore/edk2/pull/4711
>>>>>
>>>>> This series switches the OVMF CI jobs on Windows to a recent Qemu
>>>>> version and re-enables SMP in the Qemu "Run to shell" CI jobs.
>>>>> The Qemu tests run a little bit slower in SMP mode and sometimes
>>>>> hit the time limit of 2 minutes, thus this limit is increased to 4
>>>>> minutes.
>>>>>
>>>>> Together with 81cb0371f9db ("CI: Use latest image for Linux jobs
>>> (Qemu
>>>>> 8, gcc 12)" this series is sufficient to close Bug 4324.
>>>>>
>>>>>
>>>>> Gerd Hoffmann (1):
>>>>>    OvmfPkg/PlatformCI: Boot OVMF in SMP mode.
>>>>>
>>>>> Oliver Steffen (3):
>>>>>    OvmfPkg/PlatformCI: Use recent Qemu on Windows
>>>>>    OvmfPkg/PlatformCI VS2019: Disable workaround for cpuhp bugfix
>>>>>    CI: Increase time limit of "Run to shell" task to 4 minutes
>>>>>
>>>>>   .../templates/platform-build-run-steps.yml          |  2 +-
>>>>>   .../PlatformCI/.azurepipelines/Windows-VS2019.yml   |  4 ++--
>>>>>   OvmfPkg/PlatformCI/PlatformBuildLib.py              | 13 +--------
>>> ----
>>>>>   3 files changed, 4 insertions(+), 15 deletions(-)
>>>>>
>>>>
>>>>
>>>> 
>>>>
>>>>
> 



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#107615): https://edk2.groups.io/g/devel/message/107615
Mute This Topic: https://groups.io/mt/100524063/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/leave/12367111/7686176/1913456212/xyzzy [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-