Guidance about CI

Guidance about CI

[Boeuf, Sebastien]

Hi folks,

I was wondering if you would be okay with me adding Cloud Hypervisor to the EDK2 CI.
The idea would be to run a quick/simple test that Cloud Hypervisor can properly boot
with the OVMF binary built from source on every pull request.

And if you think that makes sense, any guidance on how to achieve that the right way?

Thanks,
Sebastien
---------------------------------------------------------------------
Intel Corporation SAS (French simplified joint stock company)
Registered headquarters: "Les Montalets"- 2, rue de Paris, 
92196 Meudon Cedex, France
Registration Number:  302 456 199 R.C.S. NANTERRE
Capital: 4,572,000 Euros

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

Re: [edk2-devel] Guidance about CI

[Sean]

can cloud hypervisor boot on any of the free CI providers?
If you look at ArmVirt, Ovmf, and even the emulatorpkg those all do 
similar things.  They are able to run on azure dev-ops cloud agents.

If that is the case following a pattern from defined here would lead to 
the easiest and most consistent path.

https://github.com/tianocore/edk2/tree/master/ArmVirtPkg/PlatformCI
https://github.com/tianocore/edk2/tree/master/EmulatorPkg/PlatformCI
https://github.com/tianocore/edk2/tree/master/OvmfPkg/PlatformCI

Thanks
Sean


On 1/4/2022 3:37 PM, Boeuf, Sebastien wrote:
> ight way?

Re: [edk2-devel] Guidance about CI

[Boeuf, Sebastien]

[-- Attachment #1: Type: text/plain, Size: 2061 bytes --]

Hi Sean,

Cloud Hypervisor can boot on Microsoft Azure VMs as this is what our project
relies on to validate every PR. And it should be able to boot on any other CI
providers.

I've been looking at https://github.com/tianocore/edk2/tree/master/OvmfPkg/PlatformCI
but I don't understand where the QEMU binary is built (I will need to do the same
for Cloud Hypervisor so that it's available on the machine), and I can't find the actual
tests that are being run there.

Thanks,
Sebastien
________________________________
From: devel@edk2.groups.io <devel@edk2.groups.io> on behalf of Sean <spbrogan@outlook.com>
Sent: Wednesday, January 5, 2022 12:40 AM
To: devel@edk2.groups.io <devel@edk2.groups.io>; Boeuf, Sebastien <sebastien.boeuf@intel.com>; Yao, Jiewen <jiewen.yao@intel.com>; kraxel@redhat.com <kraxel@redhat.com>
Subject: Re: [edk2-devel] Guidance about CI

can cloud hypervisor boot on any of the free CI providers?
If you look at ArmVirt, Ovmf, and even the emulatorpkg those all do
similar things.  They are able to run on azure dev-ops cloud agents.

If that is the case following a pattern from defined here would lead to
the easiest and most consistent path.

https://github.com/tianocore/edk2/tree/master/ArmVirtPkg/PlatformCI
https://github.com/tianocore/edk2/tree/master/EmulatorPkg/PlatformCI
https://github.com/tianocore/edk2/tree/master/OvmfPkg/PlatformCI

Thanks
Sean


On 1/4/2022 3:37 PM, Boeuf, Sebastien wrote:
> ight way?





---------------------------------------------------------------------
Intel Corporation SAS (French simplified joint stock company)
Registered headquarters: "Les Montalets"- 2, rue de Paris, 
92196 Meudon Cedex, France
Registration Number:  302 456 199 R.C.S. NANTERRE
Capital: 4,572,000 Euros

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

[-- Attachment #2: Type: text/html, Size: 4762 bytes --]

Re: [edk2-devel] Guidance about CI

[Boeuf, Sebastien]

[-- Attachment #1: Type: text/plain, Size: 3268 bytes --]

Ah nevermind I found out QEMU was installed from packaging.

We don't have packages for Cloud Hypervisor, but we can download
a static binary from a specific release, do you think that would be
acceptable?

Thanks,
Sebastien
________________________________
From: devel@edk2.groups.io <devel@edk2.groups.io> on behalf of Boeuf, Sebastien <sebastien.boeuf@intel.com>
Sent: Wednesday, January 5, 2022 2:32 PM
To: devel@edk2.groups.io <devel@edk2.groups.io>; Yao, Jiewen <jiewen.yao@intel.com>; kraxel@redhat.com <kraxel@redhat.com>; spbrogan@outlook.com <spbrogan@outlook.com>
Subject: Re: [edk2-devel] Guidance about CI

Hi Sean,

Cloud Hypervisor can boot on Microsoft Azure VMs as this is what our project
relies on to validate every PR. And it should be able to boot on any other CI
providers.

I've been looking at https://github.com/tianocore/edk2/tree/master/OvmfPkg/PlatformCI
but I don't understand where the QEMU binary is built (I will need to do the same
for Cloud Hypervisor so that it's available on the machine), and I can't find the actual
tests that are being run there.

Thanks,
Sebastien
________________________________
From: devel@edk2.groups.io <devel@edk2.groups.io> on behalf of Sean <spbrogan@outlook.com>
Sent: Wednesday, January 5, 2022 12:40 AM
To: devel@edk2.groups.io <devel@edk2.groups.io>; Boeuf, Sebastien <sebastien.boeuf@intel.com>; Yao, Jiewen <jiewen.yao@intel.com>; kraxel@redhat.com <kraxel@redhat.com>
Subject: Re: [edk2-devel] Guidance about CI

can cloud hypervisor boot on any of the free CI providers?
If you look at ArmVirt, Ovmf, and even the emulatorpkg those all do
similar things.  They are able to run on azure dev-ops cloud agents.

If that is the case following a pattern from defined here would lead to
the easiest and most consistent path.

https://github.com/tianocore/edk2/tree/master/ArmVirtPkg/PlatformCI
https://github.com/tianocore/edk2/tree/master/EmulatorPkg/PlatformCI
https://github.com/tianocore/edk2/tree/master/OvmfPkg/PlatformCI

Thanks
Sean


On 1/4/2022 3:37 PM, Boeuf, Sebastien wrote:
> ight way?






---------------------------------------------------------------------
Intel Corporation SAS (French simplified joint stock company)
Registered headquarters: "Les Montalets"- 2, rue de Paris,
92196 Meudon Cedex, France
Registration Number:  302 456 199 R.C.S. NANTERRE
Capital: 4,572,000 Euros

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.


---------------------------------------------------------------------
Intel Corporation SAS (French simplified joint stock company)
Registered headquarters: "Les Montalets"- 2, rue de Paris, 
92196 Meudon Cedex, France
Registration Number:  302 456 199 R.C.S. NANTERRE
Capital: 4,572,000 Euros

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

[-- Attachment #2: Type: text/html, Size: 7178 bytes --]

Re: [edk2-devel] Guidance about CI

[Gerd Hoffmann]

On Wed, Jan 05, 2022 at 01:44:01PM +0000, Boeuf, Sebastien wrote:
> Ah nevermind I found out QEMU was installed from packaging.

On ubuntu.

> We don't have packages for Cloud Hypervisor, but we can download
> a static binary from a specific release, do you think that would be
> acceptable?

As far I know the same happens for qemu on windows,
so that should be fine.

take care,
  Gerd

Re: [edk2-devel] Guidance about CI

[Boeuf, Sebastien]

On Wed, 2022-01-05 at 17:55 +0100, kraxel@redhat.com wrote:
> On Wed, Jan 05, 2022 at 01:44:01PM +0000, Boeuf, Sebastien wrote:
> > Ah nevermind I found out QEMU was installed from packaging.
> 
> On ubuntu.
> 
> > We don't have packages for Cloud Hypervisor, but we can download
> > a static binary from a specific release, do you think that would be
> > acceptable?
> 
> As far I know the same happens for qemu on windows,
> so that should be fine.

Cool!

BTW, about microvm, I saw that you're skipping QEMU, so does that mean
you're not *really* testing that OVMF works with microvm, or am I
missing something?

Thanks,
Sebastien

> 
> take care,
>   Gerd
> 

---------------------------------------------------------------------
Intel Corporation SAS (French simplified joint stock company)
Registered headquarters: "Les Montalets"- 2, rue de Paris, 
92196 Meudon Cedex, France
Registration Number:  302 456 199 R.C.S. NANTERRE
Capital: 4,572,000 Euros

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

Re: [edk2-devel] Guidance about CI

[Gerd Hoffmann]

On Thu, Jan 06, 2022 at 02:21:59PM +0000, Boeuf, Sebastien wrote:
> On Wed, 2022-01-05 at 17:55 +0100, kraxel@redhat.com wrote:
> > On Wed, Jan 05, 2022 at 01:44:01PM +0000, Boeuf, Sebastien wrote:
> > > Ah nevermind I found out QEMU was installed from packaging.
> > 
> > On ubuntu.
> > 
> > > We don't have packages for Cloud Hypervisor, but we can download
> > > a static binary from a specific release, do you think that would be
> > > acceptable?
> > 
> > As far I know the same happens for qemu on windows,
> > so that should be fine.
> 
> Cool!
> 
> BTW, about microvm, I saw that you're skipping QEMU, so does that mean
> you're not *really* testing that OVMF works with microvm, or am I
> missing something?

Correct, not tested right now.  The code needs some improvements, it's
not flexible enough, havn't found the time to do that yet.  Also have
to check whenever the qemu version shipped by ubuntu is new enough to
actually have microvm support ...

take care,
  Gerd

Re: [edk2-devel] Guidance about CI

[Boeuf, Sebastien]

On Fri, 2022-01-07 at 13:17 +0100, kraxel@redhat.com wrote:
> On Thu, Jan 06, 2022 at 02:21:59PM +0000, Boeuf, Sebastien wrote:
> > On Wed, 2022-01-05 at 17:55 +0100, kraxel@redhat.com wrote:
> > > On Wed, Jan 05, 2022 at 01:44:01PM +0000, Boeuf, Sebastien wrote:
> > > > Ah nevermind I found out QEMU was installed from packaging.
> > > 
> > > On ubuntu.
> > > 
> > > > We don't have packages for Cloud Hypervisor, but we can
> > > > download
> > > > a static binary from a specific release, do you think that
> > > > would be
> > > > acceptable?
> > > 
> > > As far I know the same happens for qemu on windows,
> > > so that should be fine.
> > 
> > Cool!
> > 
> > BTW, about microvm, I saw that you're skipping QEMU, so does that
> > mean
> > you're not *really* testing that OVMF works with microvm, or am I
> > missing something?
> 
> Correct, not tested right now.  The code needs some improvements,
> it's
> not flexible enough, havn't found the time to do that yet.  Also have
> to check whenever the qemu version shipped by ubuntu is new enough to
> actually have microvm support ...

Okay thanks for your confirmation, that makes sense :)

> 
> take care,
>   Gerd
> 

---------------------------------------------------------------------
Intel Corporation SAS (French simplified joint stock company)
Registered headquarters: "Les Montalets"- 2, rue de Paris, 
92196 Meudon Cedex, France
Registration Number:  302 456 199 R.C.S. NANTERRE
Capital: 4,572,000 Euros

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

Re: [edk2-devel] Guidance about CI

[Boeuf, Sebastien]

Is the CI only relying on QEMU using TCG? I have some patches to run
the CI with Cloud Hypervisor but I got an error about /dev/kvm not
being available :/

Is there a certain type of machine used by the AzurePipelines that
would support nested virtualization?

Thanks,
Sebastien

On Fri, 2022-01-07 at 13:17 +0100, kraxel@redhat.com wrote:
> On Thu, Jan 06, 2022 at 02:21:59PM +0000, Boeuf, Sebastien wrote:
> > On Wed, 2022-01-05 at 17:55 +0100, kraxel@redhat.com wrote:
> > > On Wed, Jan 05, 2022 at 01:44:01PM +0000, Boeuf, Sebastien wrote:
> > > > Ah nevermind I found out QEMU was installed from packaging.
> > > 
> > > On ubuntu.
> > > 
> > > > We don't have packages for Cloud Hypervisor, but we can
> > > > download
> > > > a static binary from a specific release, do you think that
> > > > would be
> > > > acceptable?
> > > 
> > > As far I know the same happens for qemu on windows,
> > > so that should be fine.
> > 
> > Cool!
> > 
> > BTW, about microvm, I saw that you're skipping QEMU, so does that
> > mean
> > you're not *really* testing that OVMF works with microvm, or am I
> > missing something?
> 
> Correct, not tested right now.  The code needs some improvements,
> it's
> not flexible enough, havn't found the time to do that yet.  Also have
> to check whenever the qemu version shipped by ubuntu is new enough to
> actually have microvm support ...
> 
> take care,
>   Gerd
> 

---------------------------------------------------------------------
Intel Corporation SAS (French simplified joint stock company)
Registered headquarters: "Les Montalets"- 2, rue de Paris, 
92196 Meudon Cedex, France
Registration Number:  302 456 199 R.C.S. NANTERRE
Capital: 4,572,000 Euros

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.