Where to put the bhyve code in the edk2 repo: BhyvePkg, or under OvmfPkg?

Where to put the bhyve code in the edk2 repo: BhyvePkg, or under OvmfPkg?

[Rebecca Cran]

During the Community Meeting last night, I was asked to send this email 
starting a discussion about where to put the bhyve code in the edk2 
tree: whether it should be in a new BhyvePkg, or added under OvmfPkg. It 
appears it's already been decided it should be in edk2 along with the 
other virtual platforms and not edk2-platforms, where code for physical 
platforms will reside.


-- 
Rebecca Cran

Re: Where to put the bhyve code in the edk2 repo: BhyvePkg, or under OvmfPkg?

[Laszlo Ersek]

(CC'ing Ard and Jordan.)

On 05/08/20 17:44, Rebecca Cran wrote:
> During the Community Meeting last night, I was asked to send this email
> starting a discussion about where to put the bhyve code in the edk2
> tree: whether it should be in a new BhyvePkg, or added under OvmfPkg.

I prefer a top-level BhyvePkg.

If most edk2 consumers wouldn't like to see a top-level BhyvePkg
directory, I can certainly live with OvmfPkg/Bhyve.

I can also live with OvmfPkg/Bhyve*, OvmfPkg/Library/Bhyve*, etc, modules.

So I guess these would be my choices in decreasing order of preference.
(To be clear, I consider my option#3 still a lot better than not having
bhyve support in upstream edk2 at all.)

In either case, "Maintainers.txt" should get a new section listing the
bhyve-specific modules as being under your and Peter Grehan's
reviewership ("R").

> It
> appears it's already been decided it should be in edk2 along with the
> other virtual platforms and not edk2-platforms, where code for physical
> platforms will reside.

I haven't been aware that this is a done deal, but if it is, it makes me
glad! I've always wanted bhyve stuff to be in edk2 and not in
edk2-platforms.

Thanks!
Laszlo

Re: Where to put the bhyve code in the edk2 repo: BhyvePkg, or under OvmfPkg?

[Ard Biesheuvel]

On 5/11/20 5:55 PM, Laszlo Ersek wrote:
> (CC'ing Ard and Jordan.)
> 
> On 05/08/20 17:44, Rebecca Cran wrote:
>> During the Community Meeting last night, I was asked to send this email
>> starting a discussion about where to put the bhyve code in the edk2
>> tree: whether it should be in a new BhyvePkg, or added under OvmfPkg.
> 
> I prefer a top-level BhyvePkg.
> 
> If most edk2 consumers wouldn't like to see a top-level BhyvePkg
> directory, I can certainly live with OvmfPkg/Bhyve.
> 
> I can also live with OvmfPkg/Bhyve*, OvmfPkg/Library/Bhyve*, etc, modules.
> 
> So I guess these would be my choices in decreasing order of preference.
> (To be clear, I consider my option#3 still a lot better than not having
> bhyve support in upstream edk2 at all.)
> 
> In either case, "Maintainers.txt" should get a new section listing the
> bhyve-specific modules as being under your and Peter Grehan's
> reviewership ("R").
> 
>> It
>> appears it's already been decided it should be in edk2 along with the
>> other virtual platforms and not edk2-platforms, where code for physical
>> platforms will reside.
> 
> I haven't been aware that this is a done deal, but if it is, it makes me
> glad! I've always wanted bhyve stuff to be in edk2 and not in
> edk2-platforms.
> 

I think it is a good thing to have support for virtual platforms in core 
EDK2, given that such a platform is only a download away for anyone who 
wants to try it. I am strongly opposed to the idea that core EDK2 should 
just be a repository of bits and pieces that platforms can incorporate, 
especially because it can make regressions unsolveable once we get 
ourselves into a state where reverting some patch fixes a problem on one 
platform and creates one on another.

However, I don't think every platforms in core EDK2 can be a first class 
citizen. There is simply no way we can expect contributors to make sure 
that their changes don't break under Bhyve, and the same will be true 
once (if) we merge kvmtool guest support, which is under development as 
well (given that it supports virtualization only, and so unlike QEMU, 
which supports emulation as well, it requires a native host)

So I agree that it makes sense to incorporate Bhyve into core EDK2, but 
we have to decide on some rules regarding 'second class' platforms: 
how/when to test them, and how urgently we treat regressions found 
during such testing. We can treat ArmVirtXen the same way, imo, as well 
as KvmTool when it lands.

Whether we create a BhyvePkg depends on our future intent wrt merging 
OVMF with other virtual platforms. I think it would make sense for the 
ArmVirtPkg and OvmfPkg to be merged at some point, at which time it will 
probably make little sense to have a separate BhyvePkg. But I'm not sure 
what Laszlo's take is on this.

In summary, I can live with any of these options, as long as the 
underlying rules and assumptions are clarified.

Re: [edk2-devel] Where to put the bhyve code in the edk2 repo: BhyvePkg, or under OvmfPkg?

[Michael D Kinney]

Laszlo,

I prefer all virtual firmware content to go into 
OvmfPkg.  Subdirectories within OvmfPkg for each
type of virtual firmware makes sense to me.

If a developer does not want virtual firmware source in
their local tree, they can use sparse checkout skipping
OvmfPkg.

Thanks,

Mike

> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On
> Behalf Of Laszlo Ersek
> Sent: Monday, May 11, 2020 8:56 AM
> To: Rebecca Cran <rebecca@bsdio.com>; edk2-devel-
> groups-io <devel@edk2.groups.io>
> Cc: Kinney, Michael D <michael.d.kinney@intel.com>;
> Andrew Fish <afish@apple.com>; Leif Lindholm
> <leif@nuviainc.com>; Ard Biesheuvel (ARM address)
> <ard.biesheuvel@arm.com>; Justen, Jordan L
> <jordan.l.justen@intel.com>
> Subject: Re: [edk2-devel] Where to put the bhyve code
> in the edk2 repo: BhyvePkg, or under OvmfPkg?
> 
> (CC'ing Ard and Jordan.)
> 
> On 05/08/20 17:44, Rebecca Cran wrote:
> > During the Community Meeting last night, I was asked
> to send this email
> > starting a discussion about where to put the bhyve
> code in the edk2
> > tree: whether it should be in a new BhyvePkg, or
> added under OvmfPkg.
> 
> I prefer a top-level BhyvePkg.
> 
> If most edk2 consumers wouldn't like to see a top-level
> BhyvePkg
> directory, I can certainly live with OvmfPkg/Bhyve.
> 
> I can also live with OvmfPkg/Bhyve*,
> OvmfPkg/Library/Bhyve*, etc, modules.
> 
> So I guess these would be my choices in decreasing
> order of preference.
> (To be clear, I consider my option#3 still a lot better
> than not having
> bhyve support in upstream edk2 at all.)
> 
> In either case, "Maintainers.txt" should get a new
> section listing the
> bhyve-specific modules as being under your and Peter
> Grehan's
> reviewership ("R").
> 
> > It
> > appears it's already been decided it should be in
> edk2 along with the
> > other virtual platforms and not edk2-platforms, where
> code for physical
> > platforms will reside.
> 
> I haven't been aware that this is a done deal, but if
> it is, it makes me
> glad! I've always wanted bhyve stuff to be in edk2 and
> not in
> edk2-platforms.
> 
> Thanks!
> Laszlo
> 
> 
> 

Re: Where to put the bhyve code in the edk2 repo: BhyvePkg, or under OvmfPkg?

[Michael D Kinney]

I agree that ArmVirtPkg contents should be added to OvmfPkg.

Mike

> -----Original Message-----
> From: Ard Biesheuvel <ard.biesheuvel@arm.com>
> Sent: Monday, May 11, 2020 9:21 AM
> To: Laszlo Ersek <lersek@redhat.com>; Rebecca Cran
> <rebecca@bsdio.com>; edk2-devel-groups-io
> <devel@edk2.groups.io>
> Cc: Kinney, Michael D <michael.d.kinney@intel.com>;
> Andrew Fish <afish@apple.com>; Leif Lindholm
> <leif@nuviainc.com>; Justen, Jordan L
> <jordan.l.justen@intel.com>
> Subject: Re: Where to put the bhyve code in the edk2
> repo: BhyvePkg, or under OvmfPkg?
> 
> On 5/11/20 5:55 PM, Laszlo Ersek wrote:
> > (CC'ing Ard and Jordan.)
> >
> > On 05/08/20 17:44, Rebecca Cran wrote:
> >> During the Community Meeting last night, I was asked
> to send this email
> >> starting a discussion about where to put the bhyve
> code in the edk2
> >> tree: whether it should be in a new BhyvePkg, or
> added under OvmfPkg.
> >
> > I prefer a top-level BhyvePkg.
> >
> > If most edk2 consumers wouldn't like to see a top-
> level BhyvePkg
> > directory, I can certainly live with OvmfPkg/Bhyve.
> >
> > I can also live with OvmfPkg/Bhyve*,
> OvmfPkg/Library/Bhyve*, etc, modules.
> >
> > So I guess these would be my choices in decreasing
> order of preference.
> > (To be clear, I consider my option#3 still a lot
> better than not having
> > bhyve support in upstream edk2 at all.)
> >
> > In either case, "Maintainers.txt" should get a new
> section listing the
> > bhyve-specific modules as being under your and Peter
> Grehan's
> > reviewership ("R").
> >
> >> It
> >> appears it's already been decided it should be in
> edk2 along with the
> >> other virtual platforms and not edk2-platforms,
> where code for physical
> >> platforms will reside.
> >
> > I haven't been aware that this is a done deal, but if
> it is, it makes me
> > glad! I've always wanted bhyve stuff to be in edk2
> and not in
> > edk2-platforms.
> >
> 
> I think it is a good thing to have support for virtual
> platforms in core
> EDK2, given that such a platform is only a download
> away for anyone who
> wants to try it. I am strongly opposed to the idea that
> core EDK2 should
> just be a repository of bits and pieces that platforms
> can incorporate,
> especially because it can make regressions unsolveable
> once we get
> ourselves into a state where reverting some patch fixes
> a problem on one
> platform and creates one on another.
> 
> However, I don't think every platforms in core EDK2 can
> be a first class
> citizen. There is simply no way we can expect
> contributors to make sure
> that their changes don't break under Bhyve, and the
> same will be true
> once (if) we merge kvmtool guest support, which is
> under development as
> well (given that it supports virtualization only, and
> so unlike QEMU,
> which supports emulation as well, it requires a native
> host)
> 
> So I agree that it makes sense to incorporate Bhyve
> into core EDK2, but
> we have to decide on some rules regarding 'second
> class' platforms:
> how/when to test them, and how urgently we treat
> regressions found
> during such testing. We can treat ArmVirtXen the same
> way, imo, as well
> as KvmTool when it lands.
> 
> Whether we create a BhyvePkg depends on our future
> intent wrt merging
> OVMF with other virtual platforms. I think it would
> make sense for the
> ArmVirtPkg and OvmfPkg to be merged at some point, at
> which time it will
> probably make little sense to have a separate BhyvePkg.
> But I'm not sure
> what Laszlo's take is on this.
> 
> In summary, I can live with any of these options, as
> long as the
> underlying rules and assumptions are clarified.
> 

Re: Where to put the bhyve code in the edk2 repo: BhyvePkg, or under OvmfPkg?

[Andrew Fish]

Crazy question. Should we add a VirtualizationPkg and move everything under that? I'm not sure the disruption to OVMF is worth, but figured I'd ask.

Thanks,

Andrew Fish

> On May 11, 2020, at 9:36 AM, Kinney, Michael D <michael.d.kinney@intel.com> wrote:
> 
> I agree that ArmVirtPkg contents should be added to OvmfPkg.
> 
> Mike
> 
>> -----Original Message-----
>> From: Ard Biesheuvel <ard.biesheuvel@arm.com>
>> Sent: Monday, May 11, 2020 9:21 AM
>> To: Laszlo Ersek <lersek@redhat.com>; Rebecca Cran
>> <rebecca@bsdio.com>; edk2-devel-groups-io
>> <devel@edk2.groups.io>
>> Cc: Kinney, Michael D <michael.d.kinney@intel.com>;
>> Andrew Fish <afish@apple.com>; Leif Lindholm
>> <leif@nuviainc.com>; Justen, Jordan L
>> <jordan.l.justen@intel.com>
>> Subject: Re: Where to put the bhyve code in the edk2
>> repo: BhyvePkg, or under OvmfPkg?
>> 
>> On 5/11/20 5:55 PM, Laszlo Ersek wrote:
>>> (CC'ing Ard and Jordan.)
>>> 
>>> On 05/08/20 17:44, Rebecca Cran wrote:
>>>> During the Community Meeting last night, I was asked
>> to send this email
>>>> starting a discussion about where to put the bhyve
>> code in the edk2
>>>> tree: whether it should be in a new BhyvePkg, or
>> added under OvmfPkg.
>>> 
>>> I prefer a top-level BhyvePkg.
>>> 
>>> If most edk2 consumers wouldn't like to see a top-
>> level BhyvePkg
>>> directory, I can certainly live with OvmfPkg/Bhyve.
>>> 
>>> I can also live with OvmfPkg/Bhyve*,
>> OvmfPkg/Library/Bhyve*, etc, modules.
>>> 
>>> So I guess these would be my choices in decreasing
>> order of preference.
>>> (To be clear, I consider my option#3 still a lot
>> better than not having
>>> bhyve support in upstream edk2 at all.)
>>> 
>>> In either case, "Maintainers.txt" should get a new
>> section listing the
>>> bhyve-specific modules as being under your and Peter
>> Grehan's
>>> reviewership ("R").
>>> 
>>>> It
>>>> appears it's already been decided it should be in
>> edk2 along with the
>>>> other virtual platforms and not edk2-platforms,
>> where code for physical
>>>> platforms will reside.
>>> 
>>> I haven't been aware that this is a done deal, but if
>> it is, it makes me
>>> glad! I've always wanted bhyve stuff to be in edk2
>> and not in
>>> edk2-platforms.
>>> 
>> 
>> I think it is a good thing to have support for virtual
>> platforms in core
>> EDK2, given that such a platform is only a download
>> away for anyone who
>> wants to try it. I am strongly opposed to the idea that
>> core EDK2 should
>> just be a repository of bits and pieces that platforms
>> can incorporate,
>> especially because it can make regressions unsolveable
>> once we get
>> ourselves into a state where reverting some patch fixes
>> a problem on one
>> platform and creates one on another.
>> 
>> However, I don't think every platforms in core EDK2 can
>> be a first class
>> citizen. There is simply no way we can expect
>> contributors to make sure
>> that their changes don't break under Bhyve, and the
>> same will be true
>> once (if) we merge kvmtool guest support, which is
>> under development as
>> well (given that it supports virtualization only, and
>> so unlike QEMU,
>> which supports emulation as well, it requires a native
>> host)
>> 
>> So I agree that it makes sense to incorporate Bhyve
>> into core EDK2, but
>> we have to decide on some rules regarding 'second
>> class' platforms:
>> how/when to test them, and how urgently we treat
>> regressions found
>> during such testing. We can treat ArmVirtXen the same
>> way, imo, as well
>> as KvmTool when it lands.
>> 
>> Whether we create a BhyvePkg depends on our future
>> intent wrt merging
>> OVMF with other virtual platforms. I think it would
>> make sense for the
>> ArmVirtPkg and OvmfPkg to be merged at some point, at
>> which time it will
>> probably make little sense to have a separate BhyvePkg.
>> But I'm not sure
>> what Laszlo's take is on this.
>> 
>> In summary, I can live with any of these options, as
>> long as the
>> underlying rules and assumptions are clarified.
>> 
> 

Re: [edk2-devel] Where to put the bhyve code in the edk2 repo: BhyvePkg, or under OvmfPkg?

[Michael D Kinney]

Andrew,

OvmfPkg README already has a broad definition.


=== OVMF OVERVIEW ===

The Open Virtual Machine Firmware (OVMF) project aims
to support firmware for Virtual Machines using the edk2
code base.  More information can be found at:


Mike

> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On
> Behalf Of Andrew Fish via groups.io
> Sent: Monday, May 11, 2020 9:39 AM
> To: Kinney, Michael D <michael.d.kinney@intel.com>
> Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>; Laszlo
> Ersek <lersek@redhat.com>; Rebecca Cran
> <rebecca@bsdio.com>; edk2-devel-groups-io
> <devel@edk2.groups.io>; Leif Lindholm
> <leif@nuviainc.com>; Justen, Jordan L
> <jordan.l.justen@intel.com>
> Subject: Re: [edk2-devel] Where to put the bhyve code
> in the edk2 repo: BhyvePkg, or under OvmfPkg?
> 
> Crazy question. Should we add a VirtualizationPkg and
> move everything under that? I'm not sure the disruption
> to OVMF is worth, but figured I'd ask.
> 
> Thanks,
> 
> Andrew Fish
> 
> > On May 11, 2020, at 9:36 AM, Kinney, Michael D
> <michael.d.kinney@intel.com> wrote:
> >
> > I agree that ArmVirtPkg contents should be added to
> OvmfPkg.
> >
> > Mike
> >
> >> -----Original Message-----
> >> From: Ard Biesheuvel <ard.biesheuvel@arm.com>
> >> Sent: Monday, May 11, 2020 9:21 AM
> >> To: Laszlo Ersek <lersek@redhat.com>; Rebecca Cran
> >> <rebecca@bsdio.com>; edk2-devel-groups-io
> >> <devel@edk2.groups.io>
> >> Cc: Kinney, Michael D <michael.d.kinney@intel.com>;
> >> Andrew Fish <afish@apple.com>; Leif Lindholm
> >> <leif@nuviainc.com>; Justen, Jordan L
> >> <jordan.l.justen@intel.com>
> >> Subject: Re: Where to put the bhyve code in the edk2
> >> repo: BhyvePkg, or under OvmfPkg?
> >>
> >> On 5/11/20 5:55 PM, Laszlo Ersek wrote:
> >>> (CC'ing Ard and Jordan.)
> >>>
> >>> On 05/08/20 17:44, Rebecca Cran wrote:
> >>>> During the Community Meeting last night, I was
> asked
> >> to send this email
> >>>> starting a discussion about where to put the bhyve
> >> code in the edk2
> >>>> tree: whether it should be in a new BhyvePkg, or
> >> added under OvmfPkg.
> >>>
> >>> I prefer a top-level BhyvePkg.
> >>>
> >>> If most edk2 consumers wouldn't like to see a top-
> >> level BhyvePkg
> >>> directory, I can certainly live with OvmfPkg/Bhyve.
> >>>
> >>> I can also live with OvmfPkg/Bhyve*,
> >> OvmfPkg/Library/Bhyve*, etc, modules.
> >>>
> >>> So I guess these would be my choices in decreasing
> >> order of preference.
> >>> (To be clear, I consider my option#3 still a lot
> >> better than not having
> >>> bhyve support in upstream edk2 at all.)
> >>>
> >>> In either case, "Maintainers.txt" should get a new
> >> section listing the
> >>> bhyve-specific modules as being under your and
> Peter
> >> Grehan's
> >>> reviewership ("R").
> >>>
> >>>> It
> >>>> appears it's already been decided it should be in
> >> edk2 along with the
> >>>> other virtual platforms and not edk2-platforms,
> >> where code for physical
> >>>> platforms will reside.
> >>>
> >>> I haven't been aware that this is a done deal, but
> if
> >> it is, it makes me
> >>> glad! I've always wanted bhyve stuff to be in edk2
> >> and not in
> >>> edk2-platforms.
> >>>
> >>
> >> I think it is a good thing to have support for
> virtual
> >> platforms in core
> >> EDK2, given that such a platform is only a download
> >> away for anyone who
> >> wants to try it. I am strongly opposed to the idea
> that
> >> core EDK2 should
> >> just be a repository of bits and pieces that
> platforms
> >> can incorporate,
> >> especially because it can make regressions
> unsolveable
> >> once we get
> >> ourselves into a state where reverting some patch
> fixes
> >> a problem on one
> >> platform and creates one on another.
> >>
> >> However, I don't think every platforms in core EDK2
> can
> >> be a first class
> >> citizen. There is simply no way we can expect
> >> contributors to make sure
> >> that their changes don't break under Bhyve, and the
> >> same will be true
> >> once (if) we merge kvmtool guest support, which is
> >> under development as
> >> well (given that it supports virtualization only,
> and
> >> so unlike QEMU,
> >> which supports emulation as well, it requires a
> native
> >> host)
> >>
> >> So I agree that it makes sense to incorporate Bhyve
> >> into core EDK2, but
> >> we have to decide on some rules regarding 'second
> >> class' platforms:
> >> how/when to test them, and how urgently we treat
> >> regressions found
> >> during such testing. We can treat ArmVirtXen the
> same
> >> way, imo, as well
> >> as KvmTool when it lands.
> >>
> >> Whether we create a BhyvePkg depends on our future
> >> intent wrt merging
> >> OVMF with other virtual platforms. I think it would
> >> make sense for the
> >> ArmVirtPkg and OvmfPkg to be merged at some point,
> at
> >> which time it will
> >> probably make little sense to have a separate
> BhyvePkg.
> >> But I'm not sure
> >> what Laszlo's take is on this.
> >>
> >> In summary, I can live with any of these options, as
> >> long as the
> >> underlying rules and assumptions are clarified.
> >>
> >
> 
> 
> 

Re: Where to put the bhyve code in the edk2 repo: BhyvePkg, or under OvmfPkg?

[Laszlo Ersek]

On 05/11/20 18:36, Kinney, Michael D wrote:
> I agree that ArmVirtPkg contents should be added to OvmfPkg.

I guess "OvmfPkg/Secondary/Bhyve" would be a compromise.

(I would actually prefer "Staging" to "Secondary", according to the following definition:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/staging/Kconfig

menuconfig STAGING
	bool "Staging drivers"
	---help---
	  This option allows you to select a number of drivers that are
	  not of the "normal" Linux kernel quality level.  These drivers
	  are placed here in order to get a wider audience to make use of
	  them.  Please note that these drivers are under heavy
	  development, may or may not work, and may contain userspace
	  interfaces that most likely will be changed in the near
	  future.

	  Using any of these drivers will taint your kernel which might
	  affect support options from both the community, and various
	  commercial support organizations.

	  If you wish to work on these drivers, to help improve them, or
	  to report problems you have with them, please see the
	  drivers/staging/<driver_name>/TODO file to see what needs to be
	  worked on, and who to contact.

	  If in doubt, say N here.

However, edk2 already uses a separate "staging" repo in (nearly) the same sense, so I figure the "staging" term is already taken. Hence "Secondary", or even "SecondClass".)

Thanks
Laszlo

>> -----Original Message-----
>> From: Ard Biesheuvel <ard.biesheuvel@arm.com>
>> Sent: Monday, May 11, 2020 9:21 AM
>> To: Laszlo Ersek <lersek@redhat.com>; Rebecca Cran
>> <rebecca@bsdio.com>; edk2-devel-groups-io
>> <devel@edk2.groups.io>
>> Cc: Kinney, Michael D <michael.d.kinney@intel.com>;
>> Andrew Fish <afish@apple.com>; Leif Lindholm
>> <leif@nuviainc.com>; Justen, Jordan L
>> <jordan.l.justen@intel.com>
>> Subject: Re: Where to put the bhyve code in the edk2
>> repo: BhyvePkg, or under OvmfPkg?
>>
>> On 5/11/20 5:55 PM, Laszlo Ersek wrote:
>>> (CC'ing Ard and Jordan.)
>>>
>>> On 05/08/20 17:44, Rebecca Cran wrote:
>>>> During the Community Meeting last night, I was asked
>> to send this email
>>>> starting a discussion about where to put the bhyve
>> code in the edk2
>>>> tree: whether it should be in a new BhyvePkg, or
>> added under OvmfPkg.
>>>
>>> I prefer a top-level BhyvePkg.
>>>
>>> If most edk2 consumers wouldn't like to see a top-
>> level BhyvePkg
>>> directory, I can certainly live with OvmfPkg/Bhyve.
>>>
>>> I can also live with OvmfPkg/Bhyve*,
>> OvmfPkg/Library/Bhyve*, etc, modules.
>>>
>>> So I guess these would be my choices in decreasing
>> order of preference.
>>> (To be clear, I consider my option#3 still a lot
>> better than not having
>>> bhyve support in upstream edk2 at all.)
>>>
>>> In either case, "Maintainers.txt" should get a new
>> section listing the
>>> bhyve-specific modules as being under your and Peter
>> Grehan's
>>> reviewership ("R").
>>>
>>>> It
>>>> appears it's already been decided it should be in
>> edk2 along with the
>>>> other virtual platforms and not edk2-platforms,
>> where code for physical
>>>> platforms will reside.
>>>
>>> I haven't been aware that this is a done deal, but if
>> it is, it makes me
>>> glad! I've always wanted bhyve stuff to be in edk2
>> and not in
>>> edk2-platforms.
>>>
>>
>> I think it is a good thing to have support for virtual
>> platforms in core
>> EDK2, given that such a platform is only a download
>> away for anyone who
>> wants to try it. I am strongly opposed to the idea that
>> core EDK2 should
>> just be a repository of bits and pieces that platforms
>> can incorporate,
>> especially because it can make regressions unsolveable
>> once we get
>> ourselves into a state where reverting some patch fixes
>> a problem on one
>> platform and creates one on another.
>>
>> However, I don't think every platforms in core EDK2 can
>> be a first class
>> citizen. There is simply no way we can expect
>> contributors to make sure
>> that their changes don't break under Bhyve, and the
>> same will be true
>> once (if) we merge kvmtool guest support, which is
>> under development as
>> well (given that it supports virtualization only, and
>> so unlike QEMU,
>> which supports emulation as well, it requires a native
>> host)
>>
>> So I agree that it makes sense to incorporate Bhyve
>> into core EDK2, but
>> we have to decide on some rules regarding 'second
>> class' platforms:
>> how/when to test them, and how urgently we treat
>> regressions found
>> during such testing. We can treat ArmVirtXen the same
>> way, imo, as well
>> as KvmTool when it lands.
>>
>> Whether we create a BhyvePkg depends on our future
>> intent wrt merging
>> OVMF with other virtual platforms. I think it would
>> make sense for the
>> ArmVirtPkg and OvmfPkg to be merged at some point, at
>> which time it will
>> probably make little sense to have a separate BhyvePkg.
>> But I'm not sure
>> what Laszlo's take is on this.
>>
>> In summary, I can live with any of these options, as
>> long as the
>> underlying rules and assumptions are clarified.
>>
> 

Re: Where to put the bhyve code in the edk2 repo: BhyvePkg, or under OvmfPkg?

[Michael D Kinney]

Hi Laszlo,

Is there an option for Bhyve to live in an edk2-platforms branch
or in edk2-staging branch until it meets the quality criteria for
OvmfPkg in the edk2 repo?

Mike

> -----Original Message-----
> From: Laszlo Ersek <lersek@redhat.com>
> Sent: Monday, May 11, 2020 2:12 PM
> To: Kinney, Michael D <michael.d.kinney@intel.com>; Ard
> Biesheuvel <ard.biesheuvel@arm.com>; Rebecca Cran
> <rebecca@bsdio.com>; edk2-devel-groups-io
> <devel@edk2.groups.io>
> Cc: Andrew Fish <afish@apple.com>; Leif Lindholm
> <leif@nuviainc.com>; Justen, Jordan L
> <jordan.l.justen@intel.com>
> Subject: Re: Where to put the bhyve code in the edk2
> repo: BhyvePkg, or under OvmfPkg?
> 
> On 05/11/20 18:36, Kinney, Michael D wrote:
> > I agree that ArmVirtPkg contents should be added to
> OvmfPkg.
> 
> I guess "OvmfPkg/Secondary/Bhyve" would be a
> compromise.
> 
> (I would actually prefer "Staging" to "Secondary",
> according to the following definition:
> 
> https://git.kernel.org/pub/scm/linux/kernel/git/torvald
> s/linux.git/tree/drivers/staging/Kconfig
> 
> menuconfig STAGING
> 	bool "Staging drivers"
> 	---help---
> 	  This option allows you to select a number of
> drivers that are
> 	  not of the "normal" Linux kernel quality level.
> These drivers
> 	  are placed here in order to get a wider audience
> to make use of
> 	  them.  Please note that these drivers are under
> heavy
> 	  development, may or may not work, and may contain
> userspace
> 	  interfaces that most likely will be changed in the
> near
> 	  future.
> 
> 	  Using any of these drivers will taint your kernel
> which might
> 	  affect support options from both the community,
> and various
> 	  commercial support organizations.
> 
> 	  If you wish to work on these drivers, to help
> improve them, or
> 	  to report problems you have with them, please see
> the
> 	  drivers/staging/<driver_name>/TODO file to see
> what needs to be
> 	  worked on, and who to contact.
> 
> 	  If in doubt, say N here.
> 
> However, edk2 already uses a separate "staging" repo in
> (nearly) the same sense, so I figure the "staging" term
> is already taken. Hence "Secondary", or even
> "SecondClass".)
> 
> Thanks
> Laszlo
> 
> >> -----Original Message-----
> >> From: Ard Biesheuvel <ard.biesheuvel@arm.com>
> >> Sent: Monday, May 11, 2020 9:21 AM
> >> To: Laszlo Ersek <lersek@redhat.com>; Rebecca Cran
> >> <rebecca@bsdio.com>; edk2-devel-groups-io
> >> <devel@edk2.groups.io>
> >> Cc: Kinney, Michael D <michael.d.kinney@intel.com>;
> >> Andrew Fish <afish@apple.com>; Leif Lindholm
> >> <leif@nuviainc.com>; Justen, Jordan L
> >> <jordan.l.justen@intel.com>
> >> Subject: Re: Where to put the bhyve code in the edk2
> >> repo: BhyvePkg, or under OvmfPkg?
> >>
> >> On 5/11/20 5:55 PM, Laszlo Ersek wrote:
> >>> (CC'ing Ard and Jordan.)
> >>>
> >>> On 05/08/20 17:44, Rebecca Cran wrote:
> >>>> During the Community Meeting last night, I was
> asked
> >> to send this email
> >>>> starting a discussion about where to put the bhyve
> >> code in the edk2
> >>>> tree: whether it should be in a new BhyvePkg, or
> >> added under OvmfPkg.
> >>>
> >>> I prefer a top-level BhyvePkg.
> >>>
> >>> If most edk2 consumers wouldn't like to see a top-
> >> level BhyvePkg
> >>> directory, I can certainly live with OvmfPkg/Bhyve.
> >>>
> >>> I can also live with OvmfPkg/Bhyve*,
> >> OvmfPkg/Library/Bhyve*, etc, modules.
> >>>
> >>> So I guess these would be my choices in decreasing
> >> order of preference.
> >>> (To be clear, I consider my option#3 still a lot
> >> better than not having
> >>> bhyve support in upstream edk2 at all.)
> >>>
> >>> In either case, "Maintainers.txt" should get a new
> >> section listing the
> >>> bhyve-specific modules as being under your and
> Peter
> >> Grehan's
> >>> reviewership ("R").
> >>>
> >>>> It
> >>>> appears it's already been decided it should be in
> >> edk2 along with the
> >>>> other virtual platforms and not edk2-platforms,
> >> where code for physical
> >>>> platforms will reside.
> >>>
> >>> I haven't been aware that this is a done deal, but
> if
> >> it is, it makes me
> >>> glad! I've always wanted bhyve stuff to be in edk2
> >> and not in
> >>> edk2-platforms.
> >>>
> >>
> >> I think it is a good thing to have support for
> virtual
> >> platforms in core
> >> EDK2, given that such a platform is only a download
> >> away for anyone who
> >> wants to try it. I am strongly opposed to the idea
> that
> >> core EDK2 should
> >> just be a repository of bits and pieces that
> platforms
> >> can incorporate,
> >> especially because it can make regressions
> unsolveable
> >> once we get
> >> ourselves into a state where reverting some patch
> fixes
> >> a problem on one
> >> platform and creates one on another.
> >>
> >> However, I don't think every platforms in core EDK2
> can
> >> be a first class
> >> citizen. There is simply no way we can expect
> >> contributors to make sure
> >> that their changes don't break under Bhyve, and the
> >> same will be true
> >> once (if) we merge kvmtool guest support, which is
> >> under development as
> >> well (given that it supports virtualization only,
> and
> >> so unlike QEMU,
> >> which supports emulation as well, it requires a
> native
> >> host)
> >>
> >> So I agree that it makes sense to incorporate Bhyve
> >> into core EDK2, but
> >> we have to decide on some rules regarding 'second
> >> class' platforms:
> >> how/when to test them, and how urgently we treat
> >> regressions found
> >> during such testing. We can treat ArmVirtXen the
> same
> >> way, imo, as well
> >> as KvmTool when it lands.
> >>
> >> Whether we create a BhyvePkg depends on our future
> >> intent wrt merging
> >> OVMF with other virtual platforms. I think it would
> >> make sense for the
> >> ArmVirtPkg and OvmfPkg to be merged at some point,
> at
> >> which time it will
> >> probably make little sense to have a separate
> BhyvePkg.
> >> But I'm not sure
> >> what Laszlo's take is on this.
> >>
> >> In summary, I can live with any of these options, as
> >> long as the
> >> underlying rules and assumptions are clarified.
> >>
> >

Re: [edk2-devel] Where to put the bhyve code in the edk2 repo: BhyvePkg, or under OvmfPkg?

[Rebecca Cran]

On May 11, 2020, at 3:22 PM, Michael D Kinney <michael.d.kinney@intel.com> wrote:
> 
> Hi Laszlo,
> 
> Is there an option for Bhyve to live in an edk2-platforms branch
> or in edk2-staging branch until it meets the quality criteria for
> OvmfPkg in the edk2 repo?

Also, what is the quality criteria? I’d be interested in working to improve any problems it currently has. I’m also planning to work on the Azure agent for FreeBSD and get a set of CI tests running for it.

— 
Rebecca Cran

Re: [edk2-devel] Where to put the bhyve code in the edk2 repo: BhyvePkg, or under OvmfPkg?

[Laszlo Ersek]

On 05/11/20 23:58, Rebecca Cran wrote:
> On May 11, 2020, at 3:22 PM, Michael D Kinney
> <michael.d.kinney@intel.com> wrote:
>>
>> Hi Laszlo,
>>
>> Is there an option for Bhyve to live in an edk2-platforms branch
>> or in edk2-staging branch until it meets the quality criteria for
>> OvmfPkg in the edk2 repo?

I'd like to have Bhyve in edk2 in order for it to share the git history
with the other edk2 modules (OvmfPkg, core, etc).

edk2-platforms being separate is already causing massive amounts of
pain. It's now common that we can't merge an otherwise self-contained
edk2 patch series, implemented, posted, and reviewed as a unit of work
with well defined boundaries, because it would break edk2-platforms, and
there is no way to cover two distinct repositories in a single patch
set.

There have been three separate instances of that concern just in the
last ~5 days:

- https://edk2.groups.io/g/devel/message/58872
  (May 8th)

- https://edk2.groups.io/g/devel/message/58874
  (May 8th)

- https://edk2.groups.io/g/devel/message/59204
  (May 11th)

> Also, what is the quality criteria? I'd be interested in working to
> improve any problems it currently has. I'm also planning to work on
> the Azure agent for FreeBSD and get a set of CI tests running for it.

Hm. Maybe I made a mistake. I segued from Ard's words to the Linux
"staging" definition; that may have been wrong. Ard's words were:

> However, I don't think every platforms in core EDK2 can be a first
> class citizen. There is simply no way we can expect contributors to
> make sure that their changes don't break under Bhyve, and the same
> will be true once (if) we merge kvmtool guest support, which is under
> development as well (given that it supports virtualization only, and
> so unlike QEMU, which supports emulation as well, it requires a native
> host)
>
> So I agree that it makes sense to incorporate Bhyve into core EDK2,
> but we have to decide on some rules regarding 'second class'
> platforms: how/when to test them, and how urgently we treat
> regressions found during such testing. We can treat ArmVirtXen the
> same way, imo, as well as KvmTool when it lands.

So maybe the question is how large the user base of a given
virtualization platform is, how wide-spread and easy-to-use the
virtualization platform is. How much time and presence can be dedicated
to maintaining its firmware.

If it's a niche virt platform, then keeping its firmware-side
counterparts regression-free is more difficult for the community (due to
the disproportionate amount of resources that its testing would
require), and so the quality of *that* firmware code is expected to be
lower.

Importantly, I do think "manpower dedicated" outweighs "platform is
niche". There was a time when UefiCpuPkg changes would break OVMF every
two weeks. We didn't solve that by making QEMU/KVM "less of a niche" for
the UefiCpuPkg owners -- we solved it by me asking (first informally) to
be CC'd on all UefiCpuPkg patches, and then formally to be marked as a
designated reviewer for UefiCpuPkg. And I'd review and regression-test a
whole bunch of UefiCpuPkg patches, just so I could catch regressions
before they made it into the tree.

If the bhyve community can *permanently* provide reviews /
regression-testing for such OVMF contributors that never use bhyve, that
would significantly increase the stability of bhyve firmware code, and
it would outweigh bhyve's user base (likely) being smaller. Xen
regressions were also reduced when the Xen community finally delegated
designated reviewers to edk2.

Reviewing and testing patches you don't really care for, but see as
possibly regressive for the platform you do care about, is a *lot* of
work. So I guess it could boil down to how much work your platform's
user base can contribute to the edk2 project.

Thanks
Laszlo

Re: [edk2-devel] Where to put the bhyve code in the edk2 repo: BhyvePkg, or under OvmfPkg?

[Ard Biesheuvel]

On 5/12/20 11:28 AM, Laszlo Ersek wrote:
> On 05/11/20 23:58, Rebecca Cran wrote:
>> On May 11, 2020, at 3:22 PM, Michael D Kinney
>> <michael.d.kinney@intel.com> wrote:
>>>
>>> Hi Laszlo,
>>>
>>> Is there an option for Bhyve to live in an edk2-platforms branch
>>> or in edk2-staging branch until it meets the quality criteria for
>>> OvmfPkg in the edk2 repo?
> 
> I'd like to have Bhyve in edk2 in order for it to share the git history
> with the other edk2 modules (OvmfPkg, core, etc).
> 
> edk2-platforms being separate is already causing massive amounts of
> pain. It's now common that we can't merge an otherwise self-contained
> edk2 patch series, implemented, posted, and reviewed as a unit of work
> with well defined boundaries, because it would break edk2-platforms, and
> there is no way to cover two distinct repositories in a single patch
> set.
> 
> There have been three separate instances of that concern just in the
> last ~5 days:
> 
> - https://edk2.groups.io/g/devel/message/58872
>    (May 8th)
> 
> - https://edk2.groups.io/g/devel/message/58874
>    (May 8th)
> 
> - https://edk2.groups.io/g/devel/message/59204
>    (May 11th)
> 
>> Also, what is the quality criteria? I'd be interested in working to
>> improve any problems it currently has. I'm also planning to work on
>> the Azure agent for FreeBSD and get a set of CI tests running for it.
> 
> Hm. Maybe I made a mistake. I segued from Ard's words to the Linux
> "staging" definition; that may have been wrong.
Yes. I never suggested that we should merge any code that doesn't meet 
our quality *at all*, nor did I suggest that the Bhyve code fails to 
meet them (and I wouldn't even know since I haven't looked at the code 
that carefully)


> Ard's words were:
> 
>> However, I don't think every platforms in core EDK2 can be a first
>> class citizen. There is simply no way we can expect contributors to
>> make sure that their changes don't break under Bhyve, and the same
>> will be true once (if) we merge kvmtool guest support, which is under
>> development as well (given that it supports virtualization only, and
>> so unlike QEMU, which supports emulation as well, it requires a native
>> host)
>>
>> So I agree that it makes sense to incorporate Bhyve into core EDK2,
>> but we have to decide on some rules regarding 'second class'
>> platforms: how/when to test them, and how urgently we treat
>> regressions found during such testing. We can treat ArmVirtXen the
>> same way, imo, as well as KvmTool when it lands.
> 
> So maybe the question is how large the user base of a given
> virtualization platform is, how wide-spread and easy-to-use the
> virtualization platform is. How much time and presence can be dedicated
> to maintaining its firmware.
> 
> If it's a niche virt platform, then keeping its firmware-side
> counterparts regression-free is more difficult for the community (due to
> the disproportionate amount of resources that its testing would
> require), and so the quality of *that* firmware code is expected to be
> lower.
> 

It is not about code quality. It is about rules and guidelines we set 
for contributors. I think it would be excellent if the azure CI would be 
extended to cover boot of a VM under bhyve. But what if that fails 
without any usable output? Is the burden going to be on me as a 
contributor to go and debug that platform if my code changes break it 
but nothing else?

> Importantly, I do think "manpower dedicated" outweighs "platform is
> niche". There was a time when UefiCpuPkg changes would break OVMF every
> two weeks. We didn't solve that by making QEMU/KVM "less of a niche" for
> the UefiCpuPkg owners -- we solved it by me asking (first informally) to
> be CC'd on all UefiCpuPkg patches, and then formally to be marked as a
> designated reviewer for UefiCpuPkg. And I'd review and regression-test a
> whole bunch of UefiCpuPkg patches, just so I could catch regressions
> before they made it into the tree.
> 
> If the bhyve community can *permanently* provide reviews /
> regression-testing for such OVMF contributors that never use bhyve, that
> would significantly increase the stability of bhyve firmware code, and
> it would outweigh bhyve's user base (likely) being smaller. Xen
> regressions were also reduced when the Xen community finally delegated
> designated reviewers to edk2.
> 
> Reviewing and testing patches you don't really care for, but see as
> possibly regressive for the platform you do care about, is a *lot* of
> work. So I guess it could boil down to how much work your platform's
> user base can contribute to the edk2 project.
> 

That is an important aspect, but not really what I am concerned about 
here. If nobody is invested enough in a platform to notice that is gets 
broken, I don't think we should put the burden on other maintainers to 
take responsibility for that.

Re: [edk2-devel] Where to put the bhyve code in the edk2 repo: BhyvePkg, or under OvmfPkg?

[Laszlo Ersek]

On 05/12/20 11:52, Ard Biesheuvel wrote:

> If nobody is invested enough in a platform to notice that is gets
> broken, I don't think we should put the burden on other maintainers to
> take responsibility for that.

I meant to say the same thing.

(I could try to explain how your words map -- IMO -- 1:1 to my
UefiCpuPkg example, but I guess I won't risk even more confusion. So
I'll just say "I agree".)

Thanks
Laszlo

Re: [edk2-devel] Where to put the bhyve code in the edk2 repo: BhyvePkg, or under OvmfPkg?

[Rebecca Cran]

(cc Peter Grehan)

On 5/12/20 3:28 AM, Laszlo Ersek wrote:

>
> If the bhyve community can *permanently* provide reviews /
> regression-testing for such OVMF contributors that never use bhyve, that
> would significantly increase the stability of bhyve firmware code, and
> it would outweigh bhyve's user base (likely) being smaller. Xen
> regressions were also reduced when the Xen community finally delegated
> designated reviewers to edk2.
>
> Reviewing and testing patches you don't really care for, but see as
> possibly regressive for the platform you do care about, is a *lot* of
> work. So I guess it could boil down to how much work your platform's
> user base can contribute to the edk2 project.

I certainly can't commit to reviewing and manually regression-testing 
all applicable OVMF patches, since I'm doing this on a volunteer basis 
and I know there will be days/weeks when my attention shifts elsewhere.

The best I could do is provide semi-regular testing and integration 
perhaps every month, and make available a permanent FreeBSD machine that 
any contributors/maintainers could remote into in order to do their own 
testing - in addition to providing integration into the CI system once 
.NET and Azure Agent support has been fixed on FreeBSD.


-- 

Rebecca Cran

Re: [edk2-devel] Where to put the bhyve code in the edk2 repo: BhyvePkg, or under OvmfPkg?

[Laszlo Ersek]

On 05/14/20 04:34, Rebecca Cran wrote:
> (cc Peter Grehan)
> 
> On 5/12/20 3:28 AM, Laszlo Ersek wrote:
> 
>>
>> If the bhyve community can *permanently* provide reviews /
>> regression-testing for such OVMF contributors that never use bhyve, that
>> would significantly increase the stability of bhyve firmware code, and
>> it would outweigh bhyve's user base (likely) being smaller. Xen
>> regressions were also reduced when the Xen community finally delegated
>> designated reviewers to edk2.
>>
>> Reviewing and testing patches you don't really care for, but see as
>> possibly regressive for the platform you do care about, is a *lot* of
>> work. So I guess it could boil down to how much work your platform's
>> user base can contribute to the edk2 project.
> 
> I certainly can't commit to reviewing and manually regression-testing
> all applicable OVMF patches, since I'm doing this on a volunteer basis
> and I know there will be days/weeks when my attention shifts elsewhere.
> 
> The best I could do is provide semi-regular testing and integration
> perhaps every month, and make available a permanent FreeBSD machine that
> any contributors/maintainers could remote into in order to do their own
> testing - in addition to providing integration into the CI system once
> .NET and Azure Agent support has been fixed on FreeBSD.

- Adding FreeBSD to CI (which I think was proposed by Ard too) sounds great.

- The community not having any human resources permanently dedicated to
bhyve regressions (testing, review, and post factum fixing) is fine, as
long as the bhyve stakeholders can live with a matching frequency of
regressions.

Thanks,
Laszlo

Re: [edk2-devel] Where to put the bhyve code in the edk2 repo: BhyvePkg, or under OvmfPkg?

[Rebecca Cran]

> On May 14, 2020, at 4:24 AM, Laszlo Ersek <lersek@redhat.com> wrote:
> 
> - The community not having any human resources permanently dedicated to
> bhyve regressions (testing, review, and post factum fixing) is fine, as
> long as the bhyve stakeholders can live with a matching frequency of
> regressions.

Yes, I believe that would be acceptable. 
Has there been a decision on the directory structure yet, or is that likely to be something that will need resolved at the next Stewards Meeting?

— 
Rebecca Cran

Re: [edk2-devel] Where to put the bhyve code in the edk2 repo: BhyvePkg, or under OvmfPkg?

[Sean]

I really don't agree with this direction.

Adding another platform to a core repository that might or might not 
work at any given time is a burden to all core contributors and doesn't 
bring value to the core project.

The direction I would like to see is a new repo created at 
github.com/tianocore/Bhyve_Platform.git

Then this platform can use a submodule for edk2.  It can stay stable 
because the submodule will point to edk2 at the last time the bhyve 
maintainers "integrated" edk2.  If the maintainer wants to be very 
active they can do this daily or at each commit or if they have other 
priorities they can do it at each stable tag.

If we as a community get our stuff together then we can build out CI 
infrastructure for Bhyve that could provide PR feedback to indicates if 
the PR proposed change will break the bhyve platform.  This feedback can 
then be used by reviewers to decide to approve or request more PR changes.

The above workflow then scales nicely to other platforms and gives 
control to the platform maintainers.  This also makes it clear the 
different steps that are needed to integrate an edk2 revision into a 
platform and provides great insight into derivative projects.  Finally, 
this workflow allows a platform to scale up and down it's investment and 
relevancy to the edk2 project without directly changing edk2 source.

I hope offering a different viewpoint isn't viewed as me "continuing to 
stand on the sidelines shouting "YOU'RE DOING IT WRONG"."

:)

Thanks
Sean



On 5/14/2020 9:20 AM, Rebecca Cran wrote:
> 
>> On May 14, 2020, at 4:24 AM, Laszlo Ersek <lersek@redhat.com> wrote:
>>
>> - The community not having any human resources permanently dedicated to
>> bhyve regressions (testing, review, and post factum fixing) is fine, as
>> long as the bhyve stakeholders can live with a matching frequency of
>> regressions.
> 
> Yes, I believe that would be acceptable.
> Has there been a decision on the directory structure yet, or is that likely to be something that will need resolved at the next Stewards Meeting?
> 
> —
> Rebecca Cran
> 
> 
> 
> 

Re: [edk2-devel] Where to put the bhyve code in the edk2 repo: BhyvePkg, or under OvmfPkg?

[Rebecca Cran]

On 5/14/20 11:48 AM, Sean Brogan wrote:

> Adding another platform to a core repository that might or might not 
> work at any given time is a burden to all core contributors and 
> doesn't bring value to the core project.
>
> The direction I would like to see is a new repo created at 
> github.com/tianocore/Bhyve_Platform.git

Given I'm a FreeBSD committer but not a TianoCore maintainer, I think 
I'd prefer to just keep using github.com/freebsd/uefi-edk2.git as the 
location for bhyve sources in that case, since it's already set up and I 
have commit access to it.

At this point I'm keen to move on from discussing where the code should 
live, since I'd like to get my changes committed wherever and start 
creating builds for people who are waiting for them and improve bhyve 
support in edk2.


-- 
Rebecca Cran

Re: [edk2-devel] Where to put the bhyve code in the edk2 repo: BhyvePkg, or under OvmfPkg?

[Sean]

Rebecca,

I think hosting open platforms outside the tianocore org is great too. 
This gives the maintainers even more freedom to run their own project 
and this scenario is actually how nearly all edk2 consumers do it. 
Today, edk2 just doesn't have the infrastructure to allow them to report 
back as a signal for core changes. I am still very interested in 
building and enabling a CI signal from these platforms.

Looks like github.com/freebsd/uefi-edk2.git is a fork of edk2.  Is there 
interest in setting it up to "consume" edk2 as is rather than be a fork? 
  Aligning with Edk2 rather than forking seems like a baseline 
requirement if you want to build against incoming PRs/changes and allow 
simple automation to help keep current.

Thanks
Sean



On 5/14/2020 11:22 AM, Rebecca Cran wrote:
> On 5/14/20 11:48 AM, Sean Brogan wrote:
> 
>> Adding another platform to a core repository that might or might not 
>> work at any given time is a burden to all core contributors and 
>> doesn't bring value to the core project.
>>
>> The direction I would like to see is a new repo created at 
>> github.com/tianocore/Bhyve_Platform.git
> 
> Given I'm a FreeBSD committer but not a TianoCore maintainer, I think 
> I'd prefer to just keep using github.com/freebsd/uefi-edk2.git as the 
> location for bhyve sources in that case, since it's already set up and I 
> have commit access to it.
> 
> At this point I'm keen to move on from discussing where the code should 
> live, since I'd like to get my changes committed wherever and start 
> creating builds for people who are waiting for them and improve bhyve 
> support in edk2.
> 
> 

Re: [edk2-devel] Where to put the bhyve code in the edk2 repo: BhyvePkg, or under OvmfPkg?

[Rebecca Cran]

> On May 14, 2020, at 12:47 PM, Sean <spbrogan@outlook.com> wrote:.
> 
> Is there interest in setting it up to "consume" edk2 as is rather than be a fork? 

I’m not sure if we’ll switch to having edk2 be a submodule, but I intend to keep master be the same as upstream master, and regularly integrate changes into the bhyve/* branches.

— 
Rebecca Cran

Re: [edk2-devel] Where to put the bhyve code in the edk2 repo: BhyvePkg, or under OvmfPkg?

[Laszlo Ersek]

On 05/14/20 19:48, Sean Brogan wrote:

> Adding another platform to a core repository that might or might not
> work at any given time is a burden to all core contributors and doesn't
> bring value to the core project.

Platforms that depend on edk2, but are not inside edk2, are near
guaranteed to break when core edk2 changes occur. Such external
dependencies are impossible to locate with "git grep"; you can't run a
git-grep on "all projects in the universe that consume edk2".

Even if you manage to identify some high-profile out-of-tree platforms
(by sheer diligence, or by those platforms' own CI systems -- doesn't
matter), you are effectively prevented from adjusting them in the *right
patch order*, as you cannot cover multiple repositories in a single
patch series.

This is already causing *massive* and everyday pain with edk2-platforms;
contributions that would normally take the form of a run-of-the-mill
patch series, now have to be split in at least three waves (prepare the
core, update external platforms, switch over the core). With in-tree
platforms, the approach is of course the same, the difference is that
these stages can be managed, and merged, in a unified series of commits.
Addressing external dependencies by splitting work into sub-series slows
development to a crawl.

(BaseTools is no exception. I've agreed to splitting it out because the
demand for that seems to be huge, and with careful maintenance of
"pip-requirements.txt", and with the features offered by pip virtual
environments, it appears technically possible. Conceptually, with these
bits in place, the BaseTools <-> edk2 dependency tracking should not
lose any safety. It *will* lose efficiency.)


Furthermore , I disagree with your "burden to all core contributors" and
"doesn't bring value to the project" statements 100%.

- I *am* a core contributor (feel free to look up the patches I've
contributed, with git-log),
- DynamicTablesPkg, EmulatorPkg, FmpDevicePkg, IntelFsp2Pkg,
IntelFsp2WrapperPkg, SignedCapsulePkg, SourceLevelDebugPkg,
StandaloneMmPkg, UefiPayloadPkg, UnitTestFrameworkPkg carry exactly zero
interest for me,
- but they are also exactly zero burden to me.

So your statement "burden to all core contributors" is false
(EmulatorPkg in particular is a platform which sometimes does break),
and it does not bother me at all.

Then, having bhyve in-tree *does* bring value to the project, because it
simplifies the testing and development of core edk2 modules on a new,
thus far not officially targeted, hypervisor platform. It exposes edk2
to a wider audience; people using FreeBSD as their everyday desktop will
have an easier time running and hopefully developing edk2.

Laszlo

Re: [edk2-devel] Where to put the bhyve code in the edk2 repo: BhyvePkg, or under OvmfPkg?

[Laszlo Ersek]

On 05/14/20 18:20, Rebecca Cran wrote:
> 
>> On May 14, 2020, at 4:24 AM, Laszlo Ersek <lersek@redhat.com> wrote:
>>
>> - The community not having any human resources permanently dedicated to
>> bhyve regressions (testing, review, and post factum fixing) is fine, as
>> long as the bhyve stakeholders can live with a matching frequency of
>> regressions.
> 
> Yes, I believe that would be acceptable. 
> Has there been a decision on the directory structure yet, or is that likely to be something that will need resolved at the next Stewards Meeting?

Based on the discussion thus far, I'd suggest
"OvmfPkg/SecondClass/Bhyve". If you have the time, just go ahead and
submit the series like that, and wait for review.

If you'd first like to be sure that everyone's OK with this pathname,
then please wait for more feedback in this thread.

Thanks
Laszlo

Re: [edk2-devel] Where to put the bhyve code in the edk2 repo: BhyvePkg, or under OvmfPkg?

[Ard Biesheuvel]

On 5/15/20 11:42 AM, Laszlo Ersek wrote:
> On 05/14/20 18:20, Rebecca Cran wrote:
>>
>>> On May 14, 2020, at 4:24 AM, Laszlo Ersek <lersek@redhat.com> wrote:
>>>
>>> - The community not having any human resources permanently dedicated to
>>> bhyve regressions (testing, review, and post factum fixing) is fine, as
>>> long as the bhyve stakeholders can live with a matching frequency of
>>> regressions.
>>
>> Yes, I believe that would be acceptable.
>> Has there been a decision on the directory structure yet, or is that likely to be something that will need resolved at the next Stewards Meeting?
> 
> Based on the discussion thus far, I'd suggest
> "OvmfPkg/SecondClass/Bhyve". If you have the time, just go ahead and
> submit the series like that, and wait for review.
> 
> If you'd first like to be sure that everyone's OK with this pathname,
> then please wait for more feedback in this thread.
> 

Please no. SecondClass/ implies some kind of hall of shame, which is not 
a fair characterization.

I think it would be better to simply host this code under OvmfPkg/Bhyve, 
and put some annotation in Maintainers.txt to document that regressions 
that only affect Bhyve are not treated with the same level of urgency as 
ones that affect OVMF for QEMU.

Re: [edk2-devel] Where to put the bhyve code in the edk2 repo: BhyvePkg, or under OvmfPkg?

[Laszlo Ersek]

On 05/15/20 11:47, Ard Biesheuvel wrote:
> On 5/15/20 11:42 AM, Laszlo Ersek wrote:
>> On 05/14/20 18:20, Rebecca Cran wrote:
>>>
>>>> On May 14, 2020, at 4:24 AM, Laszlo Ersek <lersek@redhat.com> wrote:
>>>>
>>>> - The community not having any human resources permanently dedicated to
>>>> bhyve regressions (testing, review, and post factum fixing) is fine, as
>>>> long as the bhyve stakeholders can live with a matching frequency of
>>>> regressions.
>>>
>>> Yes, I believe that would be acceptable.
>>> Has there been a decision on the directory structure yet, or is that
>>> likely to be something that will need resolved at the next Stewards
>>> Meeting?
>>
>> Based on the discussion thus far, I'd suggest
>> "OvmfPkg/SecondClass/Bhyve". If you have the time, just go ahead and
>> submit the series like that, and wait for review.
>>
>> If you'd first like to be sure that everyone's OK with this pathname,
>> then please wait for more feedback in this thread.
>>
> 
> Please no. SecondClass/ implies some kind of hall of shame, which is not
> a fair characterization.

OK. I didn't mean to put bhyve in a "pillory" (I agree it would be
unfair), I just couldn't find better words for reflecting the separation
you asked for.

> I think it would be better to simply host this code under OvmfPkg/Bhyve,

OK!

> and put some annotation in Maintainers.txt to document that regressions
> that only affect Bhyve are not treated with the same level of urgency as
> ones that affect OVMF for QEMU.

How about "S: Odd Fixes"? From:

  S: Status, one of the following:
     Supported:  Someone is actually paid to look after this.
     Maintained: Someone actually looks after it.
     Odd Fixes:  It has a maintainer but they don't have time to do
                 much other than throw the odd patch in. See below.
     Orphan:     No current maintainer [but maybe you could take the
                 role as you write your new code].
     Obsolete:   Old code. Something tagged obsolete generally means
                 it has been replaced by a better system and you
                 should be using that.

Thanks
Laszlo

Re: [edk2-devel] Where to put the bhyve code in the edk2 repo: BhyvePkg, or under OvmfPkg?

[Leif Lindholm]

On Fri, May 15, 2020 at 14:51:28 +0200, Laszlo Ersek wrote:
> On 05/15/20 11:47, Ard Biesheuvel wrote:
> > On 5/15/20 11:42 AM, Laszlo Ersek wrote:
> >> On 05/14/20 18:20, Rebecca Cran wrote:
> >>>
> >>>> On May 14, 2020, at 4:24 AM, Laszlo Ersek <lersek@redhat.com> wrote:
> >>>>
> >>>> - The community not having any human resources permanently dedicated to
> >>>> bhyve regressions (testing, review, and post factum fixing) is fine, as
> >>>> long as the bhyve stakeholders can live with a matching frequency of
> >>>> regressions.
> >>>
> >>> Yes, I believe that would be acceptable.
> >>> Has there been a decision on the directory structure yet, or is that
> >>> likely to be something that will need resolved at the next Stewards
> >>> Meeting?
> >>
> >> Based on the discussion thus far, I'd suggest
> >> "OvmfPkg/SecondClass/Bhyve". If you have the time, just go ahead and
> >> submit the series like that, and wait for review.
> >>
> >> If you'd first like to be sure that everyone's OK with this pathname,
> >> then please wait for more feedback in this thread.
> >>
> > 
> > Please no. SecondClass/ implies some kind of hall of shame, which is not
> > a fair characterization.
> 
> OK. I didn't mean to put bhyve in a "pillory" (I agree it would be
> unfair), I just couldn't find better words for reflecting the separation
> you asked for.
> 
> > I think it would be better to simply host this code under OvmfPkg/Bhyve,
> 
> OK!
> 
> > and put some annotation in Maintainers.txt to document that regressions
> > that only affect Bhyve are not treated with the same level of urgency as
> > ones that affect OVMF for QEMU.
> 
> How about "S: Odd Fixes"? From:
> 
>   S: Status, one of the following:
>      Supported:  Someone is actually paid to look after this.
>      Maintained: Someone actually looks after it.
>      Odd Fixes:  It has a maintainer but they don't have time to do
>                  much other than throw the odd patch in. See below.
>      Orphan:     No current maintainer [but maybe you could take the
>                  role as you write your new code].
>      Obsolete:   Old code. Something tagged obsolete generally means
>                  it has been replaced by a better system and you
>                  should be using that.

That looks like exactly what it's for.

It *will* (since f355b986068a) mean GetMaintainer.py will print a
warning. If that's an issue, we could discuss changing the level at
which a warning is generated.

/
    Leif