[RFC] Plan to delete ShellBinPkg from edk2/master

[RFC] Plan to delete ShellBinPkg from edk2/master

[Bi, Dandan]

Hi All,

ShellBinPkg is the remaining binary package in Edk2 repo.  We plan to delete ShellBinPkg from edk2/master, and keep source ShellPkg only in edk2 repo.
Before the deletion, I will update the existing consumers in Edk2 and Edk2Platforms to use ShellPkg directly.

If you have any concern please raise here before mid-April . If there is no concern, I will create patches for this task after mid-April.

Bugzilla for this task:
https://bugzilla.tianocore.org/show_bug.cgi?id=1675


Thanks,
Dandan

Re: [RFC] Plan to delete ShellBinPkg from edk2/master

[Laszlo Ersek]

On 04/02/19 07:38, Bi, Dandan wrote:
> Hi All,
> 
> ShellBinPkg is the remaining binary package in Edk2 repo.  We plan to delete ShellBinPkg from edk2/master, and keep source ShellPkg only in edk2 repo.
> Before the deletion, I will update the existing consumers in Edk2 and Edk2Platforms to use ShellPkg directly.
> 
> If you have any concern please raise here before mid-April . If there is no concern, I will create patches for this task after mid-April.
> 
> Bugzilla for this task:
> https://bugzilla.tianocore.org/show_bug.cgi?id=1675

(adding a few CC's)

I think we should not remove ShellBinPkg without a replacement *somehwere*.

A shell binary that is built from a validated edk2 tree, with a set of
library resolutions and PCD settings that are known to keep platform
dependencies *out* of the shell binary, is extremely useful.

IIRC, Andrew suggested earlier that we should treat the shell even as an
"OS", with better compatibility standards than we currently maintain.

I think we should only remove ShellBinPkg if we permanently offer a
separate download location instead, and we rebuild the shell binary from
"ShellPkg/ShellPkg.dsc" at every stable tag.

In that case, removing ShellBinPkg would indeed improve the edk2 tree,
in my opinion.

Thanks,
Laszlo

Re: [RFC] Plan to delete ShellBinPkg from edk2/master

[Leif Lindholm]

Hi Laszlo,

On Tue, Apr 02, 2019 at 10:49:16AM +0200, Laszlo Ersek wrote:
> On 04/02/19 07:38, Bi, Dandan wrote:
> > Hi All,
> > 
> > ShellBinPkg is the remaining binary package in Edk2 repo.  We plan
> > to delete ShellBinPkg from edk2/master, and keep source ShellPkg
> > only in edk2 repo.
> > Before the deletion, I will update the existing consumers in Edk2
> > and Edk2Platforms to use ShellPkg directly.
> > 
> > If you have any concern please raise here before mid-April . If
> > there is no concern, I will create patches for this task after
> > mid-April.
> > 
> > Bugzilla for this task:
> > https://bugzilla.tianocore.org/show_bug.cgi?id=1675
> 
> (adding a few CC's)
> 
> I think we should not remove ShellBinPkg without a replacement *somehwere*.
> 
> A shell binary that is built from a validated edk2 tree, with a set of
> library resolutions and PCD settings that are known to keep platform
> dependencies *out* of the shell binary, is extremely useful.

Agreed. However, I am not sure that accurately describes what we have
today.

> IIRC, Andrew suggested earlier that we should treat the shell even as an
> "OS", with better compatibility standards than we currently maintain.
> 
> I think we should only remove ShellBinPkg if we permanently offer a
> separate download location instead, and we rebuild the shell binary from
> "ShellPkg/ShellPkg.dsc" at every stable tag.

I think this sounds like an exellent improvement. When I saw the
patch, I did immediately think maybe I should start including them in
my Linaro releases. But if we could automate a build of binaries for
all supported architectures, and have a place to publish them, that
would be much better.

Best Regards,

Leif

> In that case, removing ShellBinPkg would indeed improve the edk2 tree,
> in my opinion.
> 
> Thanks,
> Laszlo

Re: [RFC] Plan to delete ShellBinPkg from edk2/master

[Ryszard Knop]

On Tue, 2019-04-02 at 10:12 +0100, Leif Lindholm wrote:
> Hi Laszlo,
> 
> On Tue, Apr 02, 2019 at 10:49:16AM +0200, Laszlo Ersek wrote:
> > On 04/02/19 07:38, Bi, Dandan wrote:
> > > Hi All,
> > > 
> > > ShellBinPkg is the remaining binary package in Edk2 repo.  We
> > > plan
> > > to delete ShellBinPkg from edk2/master, and keep source ShellPkg
> > > only in edk2 repo.
> > > Before the deletion, I will update the existing consumers in Edk2
> > > and Edk2Platforms to use ShellPkg directly.
> > > 
> > > If you have any concern please raise here before mid-April . If
> > > there is no concern, I will create patches for this task after
> > > mid-April.
> > > 
> > > Bugzilla for this task:
> > > https://bugzilla.tianocore.org/show_bug.cgi?id=1675
> > 
> > (adding a few CC's)
> > 
> > I think we should not remove ShellBinPkg without a replacement
> > *somehwere*.
> > 
> > A shell binary that is built from a validated edk2 tree, with a set
> > of
> > library resolutions and PCD settings that are known to keep
> > platform
> > dependencies *out* of the shell binary, is extremely useful.
> 
> Agreed. However, I am not sure that accurately describes what we have
> today.
> 
> > IIRC, Andrew suggested earlier that we should treat the shell even
> > as an
> > "OS", with better compatibility standards than we currently
> > maintain.
> > 
> > I think we should only remove ShellBinPkg if we permanently offer a
> > separate download location instead, and we rebuild the shell binary
> > from
> > "ShellPkg/ShellPkg.dsc" at every stable tag.
> 
> I think this sounds like an exellent improvement. When I saw the
> patch, I did immediately think maybe I should start including them in
> my Linaro releases. But if we could automate a build of binaries for
> all supported architectures, and have a place to publish them, that
> would be much better.

One place to put them might be next to the stable releases on GitHub.
Sources are automatically packaged there, binaries can also be
uploaded, also automatically via CI. (Maybe it could also be used to
keep stable OVMF images? Would be nice for quick testing for people not
involved in UEFI development and not having these binaries available in
their OS repos, or having issues with 3rd party builds.)

> Best Regards,
> 
> Leif
> 
> > In that case, removing ShellBinPkg would indeed improve the edk2
> > tree,
> > in my opinion.
> > 
> > Thanks,
> > Laszlo
> _______________________________________________
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel

Re: [RFC] Plan to delete ShellBinPkg from edk2/master

[Laszlo Ersek]

On 04/02/19 13:29, Ryszard Knop wrote:
> On Tue, 2019-04-02 at 10:12 +0100, Leif Lindholm wrote:
>> Hi Laszlo,
>>
>> On Tue, Apr 02, 2019 at 10:49:16AM +0200, Laszlo Ersek wrote:
>>> On 04/02/19 07:38, Bi, Dandan wrote:
>>>> Hi All,
>>>>
>>>> ShellBinPkg is the remaining binary package in Edk2 repo.  We
>>>> plan
>>>> to delete ShellBinPkg from edk2/master, and keep source ShellPkg
>>>> only in edk2 repo.
>>>> Before the deletion, I will update the existing consumers in Edk2
>>>> and Edk2Platforms to use ShellPkg directly.
>>>>
>>>> If you have any concern please raise here before mid-April . If
>>>> there is no concern, I will create patches for this task after
>>>> mid-April.
>>>>
>>>> Bugzilla for this task:
>>>> https://bugzilla.tianocore.org/show_bug.cgi?id=1675
>>>
>>> (adding a few CC's)
>>>
>>> I think we should not remove ShellBinPkg without a replacement
>>> *somehwere*.
>>>
>>> A shell binary that is built from a validated edk2 tree, with a set
>>> of
>>> library resolutions and PCD settings that are known to keep
>>> platform
>>> dependencies *out* of the shell binary, is extremely useful.
>>
>> Agreed. However, I am not sure that accurately describes what we have
>> today.
>>
>>> IIRC, Andrew suggested earlier that we should treat the shell even
>>> as an
>>> "OS", with better compatibility standards than we currently
>>> maintain.
>>>
>>> I think we should only remove ShellBinPkg if we permanently offer a
>>> separate download location instead, and we rebuild the shell binary
>>> from
>>> "ShellPkg/ShellPkg.dsc" at every stable tag.
>>
>> I think this sounds like an exellent improvement. When I saw the
>> patch, I did immediately think maybe I should start including them in
>> my Linaro releases. But if we could automate a build of binaries for
>> all supported architectures, and have a place to publish them, that
>> would be much better.
> 
> One place to put them might be next to the stable releases on GitHub.
> Sources are automatically packaged there, binaries can also be
> uploaded, also automatically via CI. (Maybe it could also be used to
> keep stable OVMF images? Would be nice for quick testing for people not
> involved in UEFI development and not having these binaries available in
> their OS repos, or having issues with 3rd party builds.)

OVMF and ArmVirtQemu binaries are being bundled with upstream QEMU. The
series should be merged into QEMU 4.1.

The plan is to refresh the firmware binaries for each edk2 stable tag.

(It's also possible that users could benefit from synching QEMU and edk2
releases to an extent, similarly to how SeaBIOS and QEMU tend to be in
sync. (IIRC SeaBIOS tends to cut a new release shortly before QEMU, so
that QEMU can pick up the most recent release while it's fresh.) Anyway
this sub-topic is for the future.)

Thanks
Laszlo

Re: [RFC] Plan to delete ShellBinPkg from edk2/master

[Ryszard Knop]

On Tue, 2019-04-02 at 13:45 +0200, Laszlo Ersek wrote:
> On 04/02/19 13:29, Ryszard Knop wrote:
> > On Tue, 2019-04-02 at 10:12 +0100, Leif Lindholm wrote:
> > > Hi Laszlo,
> > > 
> > > On Tue, Apr 02, 2019 at 10:49:16AM +0200, Laszlo Ersek wrote:
> > > > On 04/02/19 07:38, Bi, Dandan wrote:
> > > > > Hi All,
> > > > > 
> > > > > ShellBinPkg is the remaining binary package in Edk2 repo.  We
> > > > > plan
> > > > > to delete ShellBinPkg from edk2/master, and keep source
> > > > > ShellPkg
> > > > > only in edk2 repo.
> > > > > Before the deletion, I will update the existing consumers in
> > > > > Edk2
> > > > > and Edk2Platforms to use ShellPkg directly.
> > > > > 
> > > > > If you have any concern please raise here before mid-April .
> > > > > If
> > > > > there is no concern, I will create patches for this task
> > > > > after
> > > > > mid-April.
> > > > > 
> > > > > Bugzilla for this task:
> > > > > https://bugzilla.tianocore.org/show_bug.cgi?id=1675
> > > > 
> > > > (adding a few CC's)
> > > > 
> > > > I think we should not remove ShellBinPkg without a replacement
> > > > *somehwere*.
> > > > 
> > > > A shell binary that is built from a validated edk2 tree, with a
> > > > set
> > > > of
> > > > library resolutions and PCD settings that are known to keep
> > > > platform
> > > > dependencies *out* of the shell binary, is extremely useful.
> > > 
> > > Agreed. However, I am not sure that accurately describes what we
> > > have
> > > today.
> > > 
> > > > IIRC, Andrew suggested earlier that we should treat the shell
> > > > even
> > > > as an
> > > > "OS", with better compatibility standards than we currently
> > > > maintain.
> > > > 
> > > > I think we should only remove ShellBinPkg if we permanently
> > > > offer a
> > > > separate download location instead, and we rebuild the shell
> > > > binary
> > > > from
> > > > "ShellPkg/ShellPkg.dsc" at every stable tag.
> > > 
> > > I think this sounds like an exellent improvement. When I saw the
> > > patch, I did immediately think maybe I should start including
> > > them in
> > > my Linaro releases. But if we could automate a build of binaries
> > > for
> > > all supported architectures, and have a place to publish them,
> > > that
> > > would be much better.
> > 
> > One place to put them might be next to the stable releases on
> > GitHub.
> > Sources are automatically packaged there, binaries can also be
> > uploaded, also automatically via CI. (Maybe it could also be used
> > to
> > keep stable OVMF images? Would be nice for quick testing for people
> > not
> > involved in UEFI development and not having these binaries
> > available in
> > their OS repos, or having issues with 3rd party builds.)
> 
> OVMF and ArmVirtQemu binaries are being bundled with upstream QEMU.
> The
> series should be merged into QEMU 4.1.

Right, but for people stuck on older QEMU versions it'd be useful to
find everything conveniently in one place when googling "OVMF". And
there are quite a few of these versions in the wild: 
https://repology.org/project/qemu/versions

> The plan is to refresh the firmware binaries for each edk2 stable
> tag.
> 
> (It's also possible that users could benefit from synching QEMU and
> edk2
> releases to an extent, similarly to how SeaBIOS and QEMU tend to be
> in
> sync. (IIRC SeaBIOS tends to cut a new release shortly before QEMU,
> so
> that QEMU can pick up the most recent release while it's fresh.)
> Anyway
> this sub-topic is for the future.)
> 
> Thanks
> Laszlo

Re: [RFC] Plan to delete ShellBinPkg from edk2/master

[Laszlo Ersek]

On 04/02/19 13:50, Ryszard Knop wrote:
> On Tue, 2019-04-02 at 13:45 +0200, Laszlo Ersek wrote:
>> On 04/02/19 13:29, Ryszard Knop wrote:
>>> On Tue, 2019-04-02 at 10:12 +0100, Leif Lindholm wrote:
>>>> Hi Laszlo,
>>>>
>>>> On Tue, Apr 02, 2019 at 10:49:16AM +0200, Laszlo Ersek wrote:
>>>>> On 04/02/19 07:38, Bi, Dandan wrote:
>>>>>> Hi All,
>>>>>>
>>>>>> ShellBinPkg is the remaining binary package in Edk2 repo.  We
>>>>>> plan
>>>>>> to delete ShellBinPkg from edk2/master, and keep source
>>>>>> ShellPkg
>>>>>> only in edk2 repo.
>>>>>> Before the deletion, I will update the existing consumers in
>>>>>> Edk2
>>>>>> and Edk2Platforms to use ShellPkg directly.
>>>>>>
>>>>>> If you have any concern please raise here before mid-April .
>>>>>> If
>>>>>> there is no concern, I will create patches for this task
>>>>>> after
>>>>>> mid-April.
>>>>>>
>>>>>> Bugzilla for this task:
>>>>>> https://bugzilla.tianocore.org/show_bug.cgi?id=1675
>>>>>
>>>>> (adding a few CC's)
>>>>>
>>>>> I think we should not remove ShellBinPkg without a replacement
>>>>> *somehwere*.
>>>>>
>>>>> A shell binary that is built from a validated edk2 tree, with a
>>>>> set
>>>>> of
>>>>> library resolutions and PCD settings that are known to keep
>>>>> platform
>>>>> dependencies *out* of the shell binary, is extremely useful.
>>>>
>>>> Agreed. However, I am not sure that accurately describes what we
>>>> have
>>>> today.
>>>>
>>>>> IIRC, Andrew suggested earlier that we should treat the shell
>>>>> even
>>>>> as an
>>>>> "OS", with better compatibility standards than we currently
>>>>> maintain.
>>>>>
>>>>> I think we should only remove ShellBinPkg if we permanently
>>>>> offer a
>>>>> separate download location instead, and we rebuild the shell
>>>>> binary
>>>>> from
>>>>> "ShellPkg/ShellPkg.dsc" at every stable tag.
>>>>
>>>> I think this sounds like an exellent improvement. When I saw the
>>>> patch, I did immediately think maybe I should start including
>>>> them in
>>>> my Linaro releases. But if we could automate a build of binaries
>>>> for
>>>> all supported architectures, and have a place to publish them,
>>>> that
>>>> would be much better.
>>>
>>> One place to put them might be next to the stable releases on
>>> GitHub.
>>> Sources are automatically packaged there, binaries can also be
>>> uploaded, also automatically via CI. (Maybe it could also be used
>>> to
>>> keep stable OVMF images? Would be nice for quick testing for people
>>> not
>>> involved in UEFI development and not having these binaries
>>> available in
>>> their OS repos, or having issues with 3rd party builds.)
>>
>> OVMF and ArmVirtQemu binaries are being bundled with upstream QEMU.
>> The
>> series should be merged into QEMU 4.1.
> 
> Right, but for people stuck on older QEMU versions it'd be useful to
> find everything conveniently in one place when googling "OVMF". And
> there are quite a few of these versions in the wild: 
> https://repology.org/project/qemu/versions

This doesn't seem to be a problem with the UEFI shell, but 'everything
conveniently in one place when googling "OVMF"' seems like a ship that
has sailed. To exaggerate a bit, when you google "Linux" or "QEMU", you
don't find everything in one place either. :)

For bleeding edge builds, people can (and do) already consume
<https://www.kraxel.org/repos/>. The point of bundling binaries, built
from the latest edk2-stableYYYYMM tag, with QEMU, is two-fold: (a)
following the tradition of bundling SeaBIOS, iPXE, and other
firmware(-level) binaries with QEMU, (b) allowing QEMU's own "make
check" to run UEFI helper apps in guests, without external dependencies
(both the platform firmware binaries and the UEFI app(s) are to be
tracked in the QEMU git tree).

NB purpose (a) is not universally accepted for QEMU either; there have
been ideas to move all of those firmware thingies out of QEMU. I'm
neutral on that longer-term question; for now, adding ArmVirtQemu and
OVMF binaries to the QEMU tree sticks with the tradition and improves
the availability of those firmware binaries.

That said, I'm not against offering OVMF builds via some kind of
TianoCore-owned CI. We should just be aware that adding more download
locations and calling them "official" will likely not eliminate any
builds we consider "unofficial". I'm undecided whether a new download
location will mitigate the confusion, or contribute to it.

Thanks,
Laszlo

>> The plan is to refresh the firmware binaries for each edk2 stable
>> tag.
>>
>> (It's also possible that users could benefit from synching QEMU and
>> edk2
>> releases to an extent, similarly to how SeaBIOS and QEMU tend to be
>> in
>> sync. (IIRC SeaBIOS tends to cut a new release shortly before QEMU,
>> so
>> that QEMU can pick up the most recent release while it's fresh.)
>> Anyway
>> this sub-topic is for the future.)
>>
>> Thanks
>> Laszlo
> 

Re: [RFC] Plan to delete ShellBinPkg from edk2/master

[Ni, Ray]

> -----Original Message-----
> From: edk2-devel <edk2-devel-bounces@lists.01.org> On Behalf Of Laszlo
> Ersek
> Sent: Tuesday, April 2, 2019 4:49 PM
> To: Bi, Dandan <dandan.bi@intel.com>; edk2-devel@lists.01.org
> Cc: Cetola, Stephano <stephano.cetola@intel.com>; Kinney, Michael D
> <michael.d.kinney@intel.com>; Gao, Liming <liming.gao@intel.com>; Carsey,
> Jaben <jaben.carsey@intel.com>
> Subject: Re: [edk2] [RFC] Plan to delete ShellBinPkg from edk2/master
> 
> On 04/02/19 07:38, Bi, Dandan wrote:
> > Hi All,
> >
> > ShellBinPkg is the remaining binary package in Edk2 repo.  We plan to
> delete ShellBinPkg from edk2/master, and keep source ShellPkg only in edk2
> repo.
> > Before the deletion, I will update the existing consumers in Edk2 and
> Edk2Platforms to use ShellPkg directly.
> >
> > If you have any concern please raise here before mid-April . If there is no
> concern, I will create patches for this task after mid-April.
> >
> > Bugzilla for this task:
> > https://bugzilla.tianocore.org/show_bug.cgi?id=1675
> 
> (adding a few CC's)
> 
> I think we should not remove ShellBinPkg without a replacement
> *somehwere*.
> 
> A shell binary that is built from a validated edk2 tree, with a set of library
> resolutions and PCD settings that are known to keep platform dependencies
> *out* of the shell binary, is extremely useful.

I understand the concern.
Maybe a "Shell.dsc.inc" provided by ShellPkg which lists all library resolutions
, PCD settings and build options can be included by platform DSC to resolve such
dependency issue.

> 
> IIRC, Andrew suggested earlier that we should treat the shell even as an "OS",
> with better compatibility standards than we currently maintain.
> 
> I think we should only remove ShellBinPkg if we permanently offer a
> separate download location instead, and we rebuild the shell binary from
> "ShellPkg/ShellPkg.dsc" at every stable tag.

I do not quite understand. All other modules in edk2 repo are source-included by
OvmfPkg and daily commits directly generates new binaries for OvmfPkg.
I do not think we should have a different "binary-generation" model for
shell.

> 
> In that case, removing ShellBinPkg would indeed improve the edk2 tree, in
> my opinion.
> 
> Thanks,
> Laszlo
> _______________________________________________
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel

Re: [RFC] Plan to delete ShellBinPkg from edk2/master

[Laszlo Ersek]

On 04/03/19 04:17, Ni, Ray wrote:
> 
> 
>> -----Original Message-----
>> From: edk2-devel <edk2-devel-bounces@lists.01.org> On Behalf Of Laszlo
>> Ersek
>> Sent: Tuesday, April 2, 2019 4:49 PM
>> To: Bi, Dandan <dandan.bi@intel.com>; edk2-devel@lists.01.org
>> Cc: Cetola, Stephano <stephano.cetola@intel.com>; Kinney, Michael D
>> <michael.d.kinney@intel.com>; Gao, Liming <liming.gao@intel.com>; Carsey,
>> Jaben <jaben.carsey@intel.com>
>> Subject: Re: [edk2] [RFC] Plan to delete ShellBinPkg from edk2/master
>>
>> On 04/02/19 07:38, Bi, Dandan wrote:
>>> Hi All,
>>>
>>> ShellBinPkg is the remaining binary package in Edk2 repo.  We plan to
>> delete ShellBinPkg from edk2/master, and keep source ShellPkg only in edk2
>> repo.
>>> Before the deletion, I will update the existing consumers in Edk2 and
>> Edk2Platforms to use ShellPkg directly.
>>>
>>> If you have any concern please raise here before mid-April . If there is no
>> concern, I will create patches for this task after mid-April.
>>>
>>> Bugzilla for this task:
>>> https://bugzilla.tianocore.org/show_bug.cgi?id=1675
>>
>> (adding a few CC's)
>>
>> I think we should not remove ShellBinPkg without a replacement
>> *somehwere*.
>>
>> A shell binary that is built from a validated edk2 tree, with a set of library
>> resolutions and PCD settings that are known to keep platform dependencies
>> *out* of the shell binary, is extremely useful.
> 
> I understand the concern.
> Maybe a "Shell.dsc.inc" provided by ShellPkg which lists all library resolutions
> , PCD settings and build options can be included by platform DSC to resolve such
> dependency issue.
> 
>>
>> IIRC, Andrew suggested earlier that we should treat the shell even as an "OS",
>> with better compatibility standards than we currently maintain.
>>
>> I think we should only remove ShellBinPkg if we permanently offer a
>> separate download location instead, and we rebuild the shell binary from
>> "ShellPkg/ShellPkg.dsc" at every stable tag.
> 
> I do not quite understand. All other modules in edk2 repo are source-included by
> OvmfPkg and daily commits directly generates new binaries for OvmfPkg.
> I do not think we should have a different "binary-generation" model for
> shell.

The standalone shell binary would not be offered for OVMF, but for all
possible UEFI platforms (physical and virtual alike).

People frequently turn to the UEFI shell for debugging UEFI issues on
their physical machines. Such users are generally not interested in
building the shell from source, just booting it as easily as possible.

Thanks,
Laszlo


>> In that case, removing ShellBinPkg would indeed improve the edk2 tree, in
>> my opinion.
>>
>> Thanks,
>> Laszlo
>> _______________________________________________
>> edk2-devel mailing list
>> edk2-devel@lists.01.org
>> https://lists.01.org/mailman/listinfo/edk2-devel

Re: [RFC] Plan to delete ShellBinPkg from edk2/master

[Kinney, Michael D]

Laszlo,

I think it makes sense to post validated shell binaries
with the edk2-stable tag releases.  GitHub does support
this when a release tag is made.

However, we would need to make it simple for a platform
to use a binary from that location.  We may need some
enhancements to pull in binary artifacts from different
locations to support a platform build that uses one or
more pre-built binaries.

Mike

> -----Original Message-----
> From: Laszlo Ersek [mailto:lersek@redhat.com]
> Sent: Wednesday, April 3, 2019 3:10 AM
> To: Ni, Ray <ray.ni@intel.com>; Bi, Dandan
> <dandan.bi@intel.com>; edk2-devel@lists.01.org
> Cc: Cetola, Stephano <stephano.cetola@intel.com>;
> Kinney, Michael D <michael.d.kinney@intel.com>; Gao,
> Liming <liming.gao@intel.com>; Carsey, Jaben
> <jaben.carsey@intel.com>
> Subject: Re: [edk2] [RFC] Plan to delete ShellBinPkg
> from edk2/master
> 
> On 04/03/19 04:17, Ni, Ray wrote:
> >
> >
> >> -----Original Message-----
> >> From: edk2-devel <edk2-devel-bounces@lists.01.org>
> On Behalf Of Laszlo
> >> Ersek
> >> Sent: Tuesday, April 2, 2019 4:49 PM
> >> To: Bi, Dandan <dandan.bi@intel.com>; edk2-
> devel@lists.01.org
> >> Cc: Cetola, Stephano <stephano.cetola@intel.com>;
> Kinney, Michael D
> >> <michael.d.kinney@intel.com>; Gao, Liming
> <liming.gao@intel.com>; Carsey,
> >> Jaben <jaben.carsey@intel.com>
> >> Subject: Re: [edk2] [RFC] Plan to delete ShellBinPkg
> from edk2/master
> >>
> >> On 04/02/19 07:38, Bi, Dandan wrote:
> >>> Hi All,
> >>>
> >>> ShellBinPkg is the remaining binary package in Edk2
> repo.  We plan to
> >> delete ShellBinPkg from edk2/master, and keep source
> ShellPkg only in edk2
> >> repo.
> >>> Before the deletion, I will update the existing
> consumers in Edk2 and
> >> Edk2Platforms to use ShellPkg directly.
> >>>
> >>> If you have any concern please raise here before
> mid-April . If there is no
> >> concern, I will create patches for this task after
> mid-April.
> >>>
> >>> Bugzilla for this task:
> >>> https://bugzilla.tianocore.org/show_bug.cgi?id=1675
> >>
> >> (adding a few CC's)
> >>
> >> I think we should not remove ShellBinPkg without a
> replacement
> >> *somehwere*.
> >>
> >> A shell binary that is built from a validated edk2
> tree, with a set of library
> >> resolutions and PCD settings that are known to keep
> platform dependencies
> >> *out* of the shell binary, is extremely useful.
> >
> > I understand the concern.
> > Maybe a "Shell.dsc.inc" provided by ShellPkg which
> lists all library resolutions
> > , PCD settings and build options can be included by
> platform DSC to resolve such
> > dependency issue.
> >
> >>
> >> IIRC, Andrew suggested earlier that we should treat
> the shell even as an "OS",
> >> with better compatibility standards than we
> currently maintain.
> >>
> >> I think we should only remove ShellBinPkg if we
> permanently offer a
> >> separate download location instead, and we rebuild
> the shell binary from
> >> "ShellPkg/ShellPkg.dsc" at every stable tag.
> >
> > I do not quite understand. All other modules in edk2
> repo are source-included by
> > OvmfPkg and daily commits directly generates new
> binaries for OvmfPkg.
> > I do not think we should have a different "binary-
> generation" model for
> > shell.
> 
> The standalone shell binary would not be offered for
> OVMF, but for all
> possible UEFI platforms (physical and virtual alike).
> 
> People frequently turn to the UEFI shell for debugging
> UEFI issues on
> their physical machines. Such users are generally not
> interested in
> building the shell from source, just booting it as
> easily as possible.
> 
> Thanks,
> Laszlo
> 
> 
> >> In that case, removing ShellBinPkg would indeed
> improve the edk2 tree, in
> >> my opinion.
> >>
> >> Thanks,
> >> Laszlo
> >> _______________________________________________
> >> edk2-devel mailing list
> >> edk2-devel@lists.01.org
> >> https://lists.01.org/mailman/listinfo/edk2-devel

Re: [RFC] Plan to delete ShellBinPkg from edk2/master

[Laszlo Ersek]

On 04/03/19 17:49, Kinney, Michael D wrote:
> Laszlo,
> 
> I think it makes sense to post validated shell binaries
> with the edk2-stable tag releases.  GitHub does support
> this when a release tag is made.
> 
> However, we would need to make it simple for a platform
> to use a binary from that location.  We may need some
> enhancements to pull in binary artifacts from different
> locations to support a platform build that uses one or
> more pre-built binaries.

Can PREBUILD scripts (written by platform owners) download the required
binaries?

Thanks,
Laszlo

Re: [edk2] [RFC] Plan to delete ShellBinPkg from edk2/master

[Michael D Kinney]

Laszlo,

A PREBUILD action looks like a good idea.

Release artifacts in GitHub are available as an http
link, so a python script to download into a path
in the build output directory and referenced by DSC/FDF
should work.

We could even post shell binaries for the past couple
edk2-stable tags because release content can be updated.

Mike

> -----Original Message-----
> From: Laszlo Ersek [mailto:lersek@redhat.com]
> Sent: Wednesday, April 3, 2019 9:07 AM
> To: Kinney, Michael D <michael.d.kinney@intel.com>; Ni,
> Ray <ray.ni@intel.com>; Bi, Dandan
> <dandan.bi@intel.com>; edk2-devel@lists.01.org
> Cc: Cetola, Stephano <stephano.cetola@intel.com>; Gao,
> Liming <liming.gao@intel.com>; Carsey, Jaben
> <jaben.carsey@intel.com>
> Subject: Re: [edk2] [RFC] Plan to delete ShellBinPkg
> from edk2/master
> 
> On 04/03/19 17:49, Kinney, Michael D wrote:
> > Laszlo,
> >
> > I think it makes sense to post validated shell
> binaries
> > with the edk2-stable tag releases.  GitHub does
> support
> > this when a release tag is made.
> >
> > However, we would need to make it simple for a
> platform
> > to use a binary from that location.  We may need some
> > enhancements to pull in binary artifacts from
> different
> > locations to support a platform build that uses one
> or
> > more pre-built binaries.
> 
> Can PREBUILD scripts (written by platform owners)
> download the required
> binaries?
> 
> Thanks,
> Laszlo

Re: [edk2] [RFC] Plan to delete ShellBinPkg from edk2/master

[ray.ni]

Mike, Laszlo,
It's a good idea to store the shell binaries into the assets of each stable tag.

If we go in this way, it means "build" requires network connection to download the
shell binary from the assets of a certain release.
Do you think it's acceptable?

Or we can separate the binary download and build into two phases so build phase
can be independent on network connection.

Is there any known practice/solution for such requirement (stable sub-component binaries
needed by a production image generation)?

Thanks,
Ray

> -----Original Message-----
> From: Kinney, Michael D <michael.d.kinney@intel.com>
> Sent: Thursday, April 4, 2019 5:47 AM
> To: Laszlo Ersek <lersek@redhat.com>; Ni, Ray <ray.ni@intel.com>; Bi,
> Dandan <dandan.bi@intel.com>; devel@edk2.groups.io; Kinney, Michael D
> <michael.d.kinney@intel.com>
> Cc: Cetola, Stephano <stephano.cetola@intel.com>; Gao, Liming
> <liming.gao@intel.com>; Carsey, Jaben <jaben.carsey@intel.com>
> Subject: RE: [edk2] [RFC] Plan to delete ShellBinPkg from edk2/master
> 
> Laszlo,
> 
> A PREBUILD action looks like a good idea.
> 
> Release artifacts in GitHub are available as an http link, so a python script to
> download into a path in the build output directory and referenced by
> DSC/FDF should work.
> 
> We could even post shell binaries for the past couple edk2-stable tags
> because release content can be updated.
> 
> Mike
> 
> > -----Original Message-----
> > From: Laszlo Ersek [mailto:lersek@redhat.com]
> > Sent: Wednesday, April 3, 2019 9:07 AM
> > To: Kinney, Michael D <michael.d.kinney@intel.com>; Ni, Ray
> > <ray.ni@intel.com>; Bi, Dandan <dandan.bi@intel.com>;
> > edk2-devel@lists.01.org
> > Cc: Cetola, Stephano <stephano.cetola@intel.com>; Gao, Liming
> > <liming.gao@intel.com>; Carsey, Jaben <jaben.carsey@intel.com>
> > Subject: Re: [edk2] [RFC] Plan to delete ShellBinPkg from edk2/master
> >
> > On 04/03/19 17:49, Kinney, Michael D wrote:
> > > Laszlo,
> > >
> > > I think it makes sense to post validated shell
> > binaries
> > > with the edk2-stable tag releases.  GitHub does
> > support
> > > this when a release tag is made.
> > >
> > > However, we would need to make it simple for a
> > platform
> > > to use a binary from that location.  We may need some enhancements
> > > to pull in binary artifacts from
> > different
> > > locations to support a platform build that uses one
> > or
> > > more pre-built binaries.
> >
> > Can PREBUILD scripts (written by platform owners) download the
> > required binaries?
> >
> > Thanks,
> > Laszlo

Re: [edk2-devel] [edk2] [RFC] Plan to delete ShellBinPkg from edk2/master

[Andrew Fish]

[-- Attachment #1: Type: text/plain, Size: 4130 bytes --]



> On Apr 3, 2019, at 8:42 PM, Ni, Ray <ray.ni@intel.com> wrote:
> 
> Mike, Laszlo,
> It's a good idea to store the shell binaries into the assets of each stable tag.
> 
> If we go in this way, it means "build" requires network connection to download the
> shell binary from the assets of a certain release.
> Do you think it's acceptable?
> 

Ray,

The other option would be to have a configuration step, like installing Python or the C compilers, that copies the binary. You need a network connection to clone the git repo and to stay in sync with it. I guess you could model that as a git submodule, or actually have a script that grabs the binary you want from a remote system, and fall back to the local copy if you don't have a network connection. 


> Or we can separate the binary download and build into two phases so build phase
> can be independent on network connection.
> 
> Is there any known practice/solution for such requirement (stable sub-component binaries
> needed by a production image generation)?
> 

I think to some extent this kind of thing is driven by the customers build rules. Basically what the customer think of as their manifest of parts for software version X.

 It might be possible for a build script to do something like this to just grab a specific version of the EFI Shell. I guess HEAD could really come from the manifest of what version you want, and the initial setup could run this command to make sure a backup version of Shell.efi is present in case the network is not present. 
git archive --remote=ssh://host/pathto/repo.git HEAD Shell.efi

I'm sure an organization building something as complex as an OS probably has custom tools to deal with the manifest, but we can probably do something simple with a script/makefile. 

Thanks,
 
Andrew Fish


> Thanks,
> Ray
> 
>> -----Original Message-----
>> From: Kinney, Michael D <michael.d.kinney@intel.com <mailto:michael.d.kinney@intel.com>>
>> Sent: Thursday, April 4, 2019 5:47 AM
>> To: Laszlo Ersek <lersek@redhat.com <mailto:lersek@redhat.com>>; Ni, Ray <ray.ni@intel.com <mailto:ray.ni@intel.com>>; Bi,
>> Dandan <dandan.bi@intel.com <mailto:dandan.bi@intel.com>>; devel@edk2.groups.io <mailto:devel@edk2.groups.io>; Kinney, Michael D
>> <michael.d.kinney@intel.com <mailto:michael.d.kinney@intel.com>>
>> Cc: Cetola, Stephano <stephano.cetola@intel.com <mailto:stephano.cetola@intel.com>>; Gao, Liming
>> <liming.gao@intel.com <mailto:liming.gao@intel.com>>; Carsey, Jaben <jaben.carsey@intel.com <mailto:jaben.carsey@intel.com>>
>> Subject: RE: [edk2] [RFC] Plan to delete ShellBinPkg from edk2/master
>> 
>> Laszlo,
>> 
>> A PREBUILD action looks like a good idea.
>> 
>> Release artifacts in GitHub are available as an http link, so a python script to
>> download into a path in the build output directory and referenced by
>> DSC/FDF should work.
>> 
>> We could even post shell binaries for the past couple edk2-stable tags
>> because release content can be updated.
>> 
>> Mike
>> 
>>> -----Original Message-----
>>> From: Laszlo Ersek [mailto:lersek@redhat.com]
>>> Sent: Wednesday, April 3, 2019 9:07 AM
>>> To: Kinney, Michael D <michael.d.kinney@intel.com>; Ni, Ray
>>> <ray.ni@intel.com>; Bi, Dandan <dandan.bi@intel.com>;
>>> edk2-devel@lists.01.org
>>> Cc: Cetola, Stephano <stephano.cetola@intel.com>; Gao, Liming
>>> <liming.gao@intel.com>; Carsey, Jaben <jaben.carsey@intel.com>
>>> Subject: Re: [edk2] [RFC] Plan to delete ShellBinPkg from edk2/master
>>> 
>>> On 04/03/19 17:49, Kinney, Michael D wrote:
>>>> Laszlo,
>>>> 
>>>> I think it makes sense to post validated shell
>>> binaries
>>>> with the edk2-stable tag releases.  GitHub does
>>> support
>>>> this when a release tag is made.
>>>> 
>>>> However, we would need to make it simple for a
>>> platform
>>>> to use a binary from that location.  We may need some enhancements
>>>> to pull in binary artifacts from
>>> different
>>>> locations to support a platform build that uses one
>>> or
>>>> more pre-built binaries.
>>> 
>>> Can PREBUILD scripts (written by platform owners) download the
>>> required binaries?
>>> 
>>> Thanks,
>>> Laszlo
> 
> 


[-- Attachment #2: Type: text/html, Size: 17124 bytes --]

Re: [edk2-devel] [edk2] [RFC] Plan to delete ShellBinPkg from edk2/master

[Laszlo Ersek]

On 04/04/19 06:09, Andrew Fish wrote:
> 
> 
>> On Apr 3, 2019, at 8:42 PM, Ni, Ray <ray.ni@intel.com> wrote:
>>
>> Mike, Laszlo,
>> It's a good idea to store the shell binaries into the assets of each stable tag.
>>
>> If we go in this way, it means "build" requires network connection to download the
>> shell binary from the assets of a certain release.
>> Do you think it's acceptable?
>>
> 
> Ray,
> 
> The other option would be to have a configuration step, like installing Python or the C compilers, that copies the binary. You need a network connection to clone the git repo and to stay in sync with it. I guess you could model that as a git submodule, or actually have a script that grabs the binary you want from a remote system, and fall back to the local copy if you don't have a network connection. 
> 
> 
>> Or we can separate the binary download and build into two phases so build phase
>> can be independent on network connection.
>>
>> Is there any known practice/solution for such requirement (stable sub-component binaries
>> needed by a production image generation)?
>>
> 
> I think to some extent this kind of thing is driven by the customers build rules. Basically what the customer think of as their manifest of parts for software version X.

I suggested PREBUILD because I took it as a given, from Mike's problem
statement, that "build" had to ensure, internally, the local
availability of the shell binary.

If that's a not requirement, then IMO it's much better to leave it to
organizations to fetch the prerequisites of their platform builds. I'd
say that's out of scope for upstream edk2 -- if they need the shell
binary to be available off-line, at their build time, they can download
it earlier and cache it locally.

Thanks
Laszlo

Re: [edk2-devel] [edk2] [RFC] Plan to delete ShellBinPkg from edk2/master

[Andrew Fish]

[-- Attachment #1: Type: text/plain, Size: 2764 bytes --]



> On Apr 4, 2019, at 3:45 AM, Laszlo Ersek <lersek@redhat.com> wrote:
> 
> On 04/04/19 06:09, Andrew Fish wrote:
>> 
>> 
>>> On Apr 3, 2019, at 8:42 PM, Ni, Ray <ray.ni@intel.com> wrote:
>>> 
>>> Mike, Laszlo,
>>> It's a good idea to store the shell binaries into the assets of each stable tag.
>>> 
>>> If we go in this way, it means "build" requires network connection to download the
>>> shell binary from the assets of a certain release.
>>> Do you think it's acceptable?
>>> 
>> 
>> Ray,
>> 
>> The other option would be to have a configuration step, like installing Python or the C compilers, that copies the binary. You need a network connection to clone the git repo and to stay in sync with it. I guess you could model that as a git submodule, or actually have a script that grabs the binary you want from a remote system, and fall back to the local copy if you don't have a network connection. 
>> 
>> 
>>> Or we can separate the binary download and build into two phases so build phase
>>> can be independent on network connection.
>>> 
>>> Is there any known practice/solution for such requirement (stable sub-component binaries
>>> needed by a production image generation)?
>>> 
>> 
>> I think to some extent this kind of thing is driven by the customers build rules. Basically what the customer think of as their manifest of parts for software version X.
> 
> I suggested PREBUILD because I took it as a given, from Mike's problem
> statement, that "build" had to ensure, internally, the local
> availability of the shell binary.
> 
> If that's a not requirement, then IMO it's much better to leave it to
> organizations to fetch the prerequisites of their platform builds. I'd
> say that's out of scope for upstream edk2 -- if they need the shell
> binary to be available off-line, at their build time, they can download
> it earlier and cache it locally.
> 

Laszlo,

I guess for edk2 projects the maintainers own the manifest. So the edk2 projects that need the Shell should define how that works. I don't think we need to define a generic solution for 3rd parties as I'd guess Red Hat and Microsoft probably already have tools and strategies to deal with cobbling together software from different packages.  

So I guess we should ask the maintainers of the ekd2 packages does the version of the Shell matter? If no then just pre-install a shell binary as part of the setup. If the version matters then we should look into doing something a little more fancy, and use the pre-installed shell binary as the fallback. 

Is there anyway to tell the Shell version from the Shell PE/COFF? One option could be a build warning if the shell is old and just have the user manually update the shell if needed. 

Thanks,

Andrew Fish


> Thanks
> Laszlo


[-- Attachment #2: Type: text/html, Size: 12856 bytes --]

Re: [edk2-devel] [edk2] [RFC] Plan to delete ShellBinPkg from edk2/master

[Laszlo Ersek]

On 04/04/19 17:10, Andrew Fish wrote:
> 
> 
>> On Apr 4, 2019, at 3:45 AM, Laszlo Ersek <lersek@redhat.com> wrote:
>>
>> On 04/04/19 06:09, Andrew Fish wrote:
>>>
>>>
>>>> On Apr 3, 2019, at 8:42 PM, Ni, Ray <ray.ni@intel.com> wrote:
>>>>
>>>> Mike, Laszlo,
>>>> It's a good idea to store the shell binaries into the assets of each stable tag.
>>>>
>>>> If we go in this way, it means "build" requires network connection to download the
>>>> shell binary from the assets of a certain release.
>>>> Do you think it's acceptable?
>>>>
>>>
>>> Ray,
>>>
>>> The other option would be to have a configuration step, like installing Python or the C compilers, that copies the binary. You need a network connection to clone the git repo and to stay in sync with it. I guess you could model that as a git submodule, or actually have a script that grabs the binary you want from a remote system, and fall back to the local copy if you don't have a network connection. 
>>>
>>>
>>>> Or we can separate the binary download and build into two phases so build phase
>>>> can be independent on network connection.
>>>>
>>>> Is there any known practice/solution for such requirement (stable sub-component binaries
>>>> needed by a production image generation)?
>>>>
>>>
>>> I think to some extent this kind of thing is driven by the customers build rules. Basically what the customer think of as their manifest of parts for software version X.
>>
>> I suggested PREBUILD because I took it as a given, from Mike's problem
>> statement, that "build" had to ensure, internally, the local
>> availability of the shell binary.
>>
>> If that's a not requirement, then IMO it's much better to leave it to
>> organizations to fetch the prerequisites of their platform builds. I'd
>> say that's out of scope for upstream edk2 -- if they need the shell
>> binary to be available off-line, at their build time, they can download
>> it earlier and cache it locally.
>>
> 
> Laszlo,
> 
> I guess for edk2 projects the maintainers own the manifest. So the edk2 projects that need the Shell should define how that works. I don't think we need to define a generic solution for 3rd parties as I'd guess Red Hat and Microsoft probably already have tools and strategies to deal with cobbling together software from different packages.  
> 
> So I guess we should ask the maintainers of the ekd2 packages does the version of the Shell matter? If no then just pre-install a shell binary as part of the setup. If the version matters then we should look into doing something a little more fancy, and use the pre-installed shell binary as the fallback. 
> 
> Is there anyway to tell the Shell version from the Shell PE/COFF? One option could be a build warning if the shell is old and just have the user manually update the shell if needed. 

As a co-maintainer under OvmfPkg and ArmVirtPkg, I prefer to build the
shell from source at all times, namely from the source code that is part
of the entire edk2 tree at a given commit / checkout.

I don't see any possibility or desire for the virtual firmware packages
(RPMs) that I have a say in to consume/ship a pre-built UEFI shell binary.

--*--

The reason I recommend for us (the TianoCore community) to offer the
shell as a prebuilt binary too, somewhere on the web, is because it
would help UEFI users (in the most general sense).

Thanks,
Laszlo

Re: [edk2-devel] [edk2] [RFC] Plan to delete ShellBinPkg from edk2/master

[Andrew Fish]

[-- Attachment #1: Type: text/plain, Size: 4056 bytes --]



> On Apr 4, 2019, at 9:06 AM, Laszlo Ersek <lersek@redhat.com> wrote:
> 
> On 04/04/19 17:10, Andrew Fish wrote:
>> 
>> 
>>> On Apr 4, 2019, at 3:45 AM, Laszlo Ersek <lersek@redhat.com> wrote:
>>> 
>>> On 04/04/19 06:09, Andrew Fish wrote:
>>>> 
>>>> 
>>>>> On Apr 3, 2019, at 8:42 PM, Ni, Ray <ray.ni@intel.com> wrote:
>>>>> 
>>>>> Mike, Laszlo,
>>>>> It's a good idea to store the shell binaries into the assets of each stable tag.
>>>>> 
>>>>> If we go in this way, it means "build" requires network connection to download the
>>>>> shell binary from the assets of a certain release.
>>>>> Do you think it's acceptable?
>>>>> 
>>>> 
>>>> Ray,
>>>> 
>>>> The other option would be to have a configuration step, like installing Python or the C compilers, that copies the binary. You need a network connection to clone the git repo and to stay in sync with it. I guess you could model that as a git submodule, or actually have a script that grabs the binary you want from a remote system, and fall back to the local copy if you don't have a network connection. 
>>>> 
>>>> 
>>>>> Or we can separate the binary download and build into two phases so build phase
>>>>> can be independent on network connection.
>>>>> 
>>>>> Is there any known practice/solution for such requirement (stable sub-component binaries
>>>>> needed by a production image generation)?
>>>>> 
>>>> 
>>>> I think to some extent this kind of thing is driven by the customers build rules. Basically what the customer think of as their manifest of parts for software version X.
>>> 
>>> I suggested PREBUILD because I took it as a given, from Mike's problem
>>> statement, that "build" had to ensure, internally, the local
>>> availability of the shell binary.
>>> 
>>> If that's a not requirement, then IMO it's much better to leave it to
>>> organizations to fetch the prerequisites of their platform builds. I'd
>>> say that's out of scope for upstream edk2 -- if they need the shell
>>> binary to be available off-line, at their build time, they can download
>>> it earlier and cache it locally.
>>> 
>> 
>> Laszlo,
>> 
>> I guess for edk2 projects the maintainers own the manifest. So the edk2 projects that need the Shell should define how that works. I don't think we need to define a generic solution for 3rd parties as I'd guess Red Hat and Microsoft probably already have tools and strategies to deal with cobbling together software from different packages.  
>> 
>> So I guess we should ask the maintainers of the ekd2 packages does the version of the Shell matter? If no then just pre-install a shell binary as part of the setup. If the version matters then we should look into doing something a little more fancy, and use the pre-installed shell binary as the fallback. 
>> 
>> Is there anyway to tell the Shell version from the Shell PE/COFF? One option could be a build warning if the shell is old and just have the user manually update the shell if needed. 
> 
> As a co-maintainer under OvmfPkg and ArmVirtPkg, I prefer to build the
> shell from source at all times, namely from the source code that is part
> of the entire edk2 tree at a given commit / checkout.
> 
> I don't see any possibility or desire for the virtual firmware packages
> (RPMs) that I have a say in to consume/ship a pre-built UEFI shell binary.
> 
> --*--
> 
> The reason I recommend for us (the TianoCore community) to offer the
> shell as a prebuilt binary too, somewhere on the web, is because it
> would help UEFI users (in the most general sense).
> 

Laszlo,

Sorry I might have taken the discussion off the rails. 

1) It seems like that for edk2 folks are happy with build form source. 
2)I agree it would useful to have a known good binary that people can download from the web, and the hash that proves it is the version you think it is. Some instructions on how build a bootable USB key would be useful too. 
3) We don't really need to solve the problem of a build system and binaries if non of the edk2 packages require that feature. 

Thanks,

Andrew Fish

> Thanks,
> Laszlo
> 
> 


[-- Attachment #2: Type: text/html, Size: 16948 bytes --]

Re: [edk2-devel] [edk2] [RFC] Plan to delete ShellBinPkg from edk2/master

[Dandan Bi]

[-- Attachment #1: Type: text/plain, Size: 5050 bytes --]

Thanks Andrew for the summary.

For now there is no objection for platform to use shell source build, so I think I can remove the dependency of ShellBinPkg in all open platforms firstly.
And I will delete ShellBinPkg if there is no need to publish shell binaries or after shell binaries have been uploaded with stable tag.

If all think publish stable shell binaries is useful and need to upload them with stable tag.
Liming,  could you help work out the process for shell binaries publish with each edk2 stable tag?
Then we can follow the process to upload the binaries and remove ShellBinPkg finally.

Thanks,
Dandan

From: afish@apple.com [mailto:afish@apple.com]
Sent: Friday, April 5, 2019 12:36 AM
To: devel@edk2.groups.io; Laszlo Ersek <lersek@redhat.com>
Cc: Ni, Ray <ray.ni@intel.com>; Kinney, Michael D <michael.d.kinney@intel.com>; Bi, Dandan <dandan.bi@intel.com>; Cetola, Stephano <stephano.cetola@intel.com>; Gao, Liming <liming.gao@intel.com>; Carsey, Jaben <jaben.carsey@intel.com>
Subject: Re: [edk2-devel] [edk2] [RFC] Plan to delete ShellBinPkg from edk2/master




On Apr 4, 2019, at 9:06 AM, Laszlo Ersek <lersek@redhat.com<mailto:lersek@redhat.com>> wrote:

On 04/04/19 17:10, Andrew Fish wrote:




On Apr 4, 2019, at 3:45 AM, Laszlo Ersek <lersek@redhat.com<mailto:lersek@redhat.com>> wrote:

On 04/04/19 06:09, Andrew Fish wrote:




On Apr 3, 2019, at 8:42 PM, Ni, Ray <ray.ni@intel.com<mailto:ray.ni@intel.com>> wrote:

Mike, Laszlo,
It's a good idea to store the shell binaries into the assets of each stable tag.

If we go in this way, it means "build" requires network connection to download the
shell binary from the assets of a certain release.
Do you think it's acceptable?

Ray,

The other option would be to have a configuration step, like installing Python or the C compilers, that copies the binary. You need a network connection to clone the git repo and to stay in sync with it. I guess you could model that as a git submodule, or actually have a script that grabs the binary you want from a remote system, and fall back to the local copy if you don't have a network connection.



Or we can separate the binary download and build into two phases so build phase
can be independent on network connection.

Is there any known practice/solution for such requirement (stable sub-component binaries
needed by a production image generation)?

I think to some extent this kind of thing is driven by the customers build rules. Basically what the customer think of as their manifest of parts for software version X.

I suggested PREBUILD because I took it as a given, from Mike's problem
statement, that "build" had to ensure, internally, the local
availability of the shell binary.

If that's a not requirement, then IMO it's much better to leave it to
organizations to fetch the prerequisites of their platform builds. I'd
say that's out of scope for upstream edk2 -- if they need the shell
binary to be available off-line, at their build time, they can download
it earlier and cache it locally.

Laszlo,

I guess for edk2 projects the maintainers own the manifest. So the edk2 projects that need the Shell should define how that works. I don't think we need to define a generic solution for 3rd parties as I'd guess Red Hat and Microsoft probably already have tools and strategies to deal with cobbling together software from different packages.

So I guess we should ask the maintainers of the ekd2 packages does the version of the Shell matter? If no then just pre-install a shell binary as part of the setup. If the version matters then we should look into doing something a little more fancy, and use the pre-installed shell binary as the fallback.

Is there anyway to tell the Shell version from the Shell PE/COFF? One option could be a build warning if the shell is old and just have the user manually update the shell if needed.

As a co-maintainer under OvmfPkg and ArmVirtPkg, I prefer to build the
shell from source at all times, namely from the source code that is part
of the entire edk2 tree at a given commit / checkout.

I don't see any possibility or desire for the virtual firmware packages
(RPMs) that I have a say in to consume/ship a pre-built UEFI shell binary.

--*--

The reason I recommend for us (the TianoCore community) to offer the
shell as a prebuilt binary too, somewhere on the web, is because it
would help UEFI users (in the most general sense).


Laszlo,

Sorry I might have taken the discussion off the rails.

1) It seems like that for edk2 folks are happy with build form source.
2)I agree it would useful to have a known good binary that people can download from the web, and the hash that proves it is the version you think it is. Some instructions on how build a bootable USB key would be useful too.
3) We don't really need to solve the problem of a build system and binaries if non of the edk2 packages require that feature.

Thanks,

Andrew Fish


Thanks,
Laszlo




[-- Attachment #2: Type: text/html, Size: 15110 bytes --]

Re: [edk2-devel] [edk2] [RFC] Plan to delete ShellBinPkg from edk2/master

[Liming Gao]

[-- Attachment #1: Type: text/plain, Size: 5989 bytes --]

Ard:
  I see IA32/X64 Shell binary has been updated for 201903 stable tag. But, ARM/ARCH64 version shell binary is not updated. Could you upload new version binaries? Then, I can upload them into 2019 stable tag Assets.

Thanks
Liming
From: Bi, Dandan
Sent: Monday, April 8, 2019 4:08 PM
To: afish@apple.com; devel@edk2.groups.io; Laszlo Ersek <lersek@redhat.com>
Cc: Ni, Ray <ray.ni@intel.com>; Kinney, Michael D <michael.d.kinney@intel.com>; Cetola, Stephano <stephano.cetola@intel.com>; Gao, Liming <liming.gao@intel.com>; Carsey, Jaben <jaben.carsey@intel.com>; Bi, Dandan <dandan.bi@intel.com>
Subject: RE: [edk2-devel] [edk2] [RFC] Plan to delete ShellBinPkg from edk2/master

Thanks Andrew for the summary.

For now there is no objection for platform to use shell source build, so I think I can remove the dependency of ShellBinPkg in all open platforms firstly.
And I will delete ShellBinPkg if there is no need to publish shell binaries or after shell binaries have been uploaded with stable tag.

If all think publish stable shell binaries is useful and need to upload them with stable tag.
Liming,  could you help work out the process for shell binaries publish with each edk2 stable tag?
Then we can follow the process to upload the binaries and remove ShellBinPkg finally.

Thanks,
Dandan

From: afish@apple.com<mailto:afish@apple.com> [mailto:afish@apple.com]
Sent: Friday, April 5, 2019 12:36 AM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Laszlo Ersek <lersek@redhat.com<mailto:lersek@redhat.com>>
Cc: Ni, Ray <ray.ni@intel.com<mailto:ray.ni@intel.com>>; Kinney, Michael D <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>; Bi, Dandan <dandan.bi@intel.com<mailto:dandan.bi@intel.com>>; Cetola, Stephano <stephano.cetola@intel.com<mailto:stephano.cetola@intel.com>>; Gao, Liming <liming.gao@intel.com<mailto:liming.gao@intel.com>>; Carsey, Jaben <jaben.carsey@intel.com<mailto:jaben.carsey@intel.com>>
Subject: Re: [edk2-devel] [edk2] [RFC] Plan to delete ShellBinPkg from edk2/master



On Apr 4, 2019, at 9:06 AM, Laszlo Ersek <lersek@redhat.com<mailto:lersek@redhat.com>> wrote:

On 04/04/19 17:10, Andrew Fish wrote:


On Apr 4, 2019, at 3:45 AM, Laszlo Ersek <lersek@redhat.com<mailto:lersek@redhat.com>> wrote:

On 04/04/19 06:09, Andrew Fish wrote:


On Apr 3, 2019, at 8:42 PM, Ni, Ray <ray.ni@intel.com<mailto:ray.ni@intel.com>> wrote:

Mike, Laszlo,
It's a good idea to store the shell binaries into the assets of each stable tag.

If we go in this way, it means "build" requires network connection to download the
shell binary from the assets of a certain release.
Do you think it's acceptable?

Ray,

The other option would be to have a configuration step, like installing Python or the C compilers, that copies the binary. You need a network connection to clone the git repo and to stay in sync with it. I guess you could model that as a git submodule, or actually have a script that grabs the binary you want from a remote system, and fall back to the local copy if you don't have a network connection.


Or we can separate the binary download and build into two phases so build phase
can be independent on network connection.

Is there any known practice/solution for such requirement (stable sub-component binaries
needed by a production image generation)?

I think to some extent this kind of thing is driven by the customers build rules. Basically what the customer think of as their manifest of parts for software version X.

I suggested PREBUILD because I took it as a given, from Mike's problem
statement, that "build" had to ensure, internally, the local
availability of the shell binary.

If that's a not requirement, then IMO it's much better to leave it to
organizations to fetch the prerequisites of their platform builds. I'd
say that's out of scope for upstream edk2 -- if they need the shell
binary to be available off-line, at their build time, they can download
it earlier and cache it locally.

Laszlo,

I guess for edk2 projects the maintainers own the manifest. So the edk2 projects that need the Shell should define how that works. I don't think we need to define a generic solution for 3rd parties as I'd guess Red Hat and Microsoft probably already have tools and strategies to deal with cobbling together software from different packages.

So I guess we should ask the maintainers of the ekd2 packages does the version of the Shell matter? If no then just pre-install a shell binary as part of the setup. If the version matters then we should look into doing something a little more fancy, and use the pre-installed shell binary as the fallback.

Is there anyway to tell the Shell version from the Shell PE/COFF? One option could be a build warning if the shell is old and just have the user manually update the shell if needed.

As a co-maintainer under OvmfPkg and ArmVirtPkg, I prefer to build the
shell from source at all times, namely from the source code that is part
of the entire edk2 tree at a given commit / checkout.

I don't see any possibility or desire for the virtual firmware packages
(RPMs) that I have a say in to consume/ship a pre-built UEFI shell binary.

--*--

The reason I recommend for us (the TianoCore community) to offer the
shell as a prebuilt binary too, somewhere on the web, is because it
would help UEFI users (in the most general sense).

Laszlo,

Sorry I might have taken the discussion off the rails.

1) It seems like that for edk2 folks are happy with build form source.
2)I agree it would useful to have a known good binary that people can download from the web, and the hash that proves it is the version you think it is. Some instructions on how build a bootable USB key would be useful too.
3) We don't really need to solve the problem of a build system and binaries if non of the edk2 packages require that feature.

Thanks,

Andrew Fish

Thanks,
Laszlo




[-- Attachment #2: Type: text/html, Size: 17009 bytes --]