[edk2-devel] libspdm Breaking Builds

[edk2-devel] libspdm Breaking Builds

[Michael Kubacki]

Recently, the libspdm submodule was added to SecurityPkg 
(SecurityPkg/DeviceSecurity/SpdmLib/libspdm) [1].

That submodule points to https://github.com/DMTF/libspdm.git.

The libspdm repo contains a submodule for cmocka that is hosted on 
cryptomilk.org [2].

Cloning that submodule is failing today, breaking all builds/CI 
dependent on SecurityPkg.

This has been an issue in the past and it is very disruptive given the 
number of dependencies on edk2/SecurityPkg. Because of its reliability 
issues, edk2 moved to a version of the repo hosted in tianocore [3] in 
2021 [4]

Can you please make a similar change in libspdm and we get edk2 updated 
as soon as possible?

Thanks,
Michael

[1] 
https://github.com/tianocore/edk2/commit/54a4fd9b35cac79e7c54b750d3dddd0fbb43dca1

[2] https://github.com/DMTF/libspdm/blob/main/.gitmodules

[3] https://github.com/tianocore/edk2-cmocka

[4] 
https://github.com/tianocore/edk2/commit/2ad22420a710dc07e3b644f91a5b55c09c39ecf3


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#119117): https://edk2.groups.io/g/devel/message/119117
Mute This Topic: https://groups.io/mt/106250971/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] libspdm Breaking Builds

[Michael D Kinney]

This was discussed and the decision was made to no use recursive clone on libspdm.

Mike

> -----Original Message-----
> From: Michael Kubacki <mikuback@linux.microsoft.com>
> Sent: Wednesday, May 22, 2024 1:40 PM
> To: devel@edk2.groups.io; Yao, Jiewen <jiewen.yao@intel.com>
> Cc: Kinney, Michael D <michael.d.kinney@intel.com>
> Subject: libspdm Breaking Builds
> 
> Recently, the libspdm submodule was added to SecurityPkg
> (SecurityPkg/DeviceSecurity/SpdmLib/libspdm) [1].
> 
> That submodule points to https://github.com/DMTF/libspdm.git.
> 
> The libspdm repo contains a submodule for cmocka that is hosted on
> cryptomilk.org [2].
> 
> Cloning that submodule is failing today, breaking all builds/CI
> dependent on SecurityPkg.
> 
> This has been an issue in the past and it is very disruptive given the
> number of dependencies on edk2/SecurityPkg. Because of its reliability
> issues, edk2 moved to a version of the repo hosted in tianocore [3] in
> 2021 [4]
> 
> Can you please make a similar change in libspdm and we get edk2 updated
> as soon as possible?
> 
> Thanks,
> Michael
> 
> [1]
> https://github.com/tianocore/edk2/commit/54a4fd9b35cac79e7c54b750d3dddd0fbb43
> dca1
> 
> [2] https://github.com/DMTF/libspdm/blob/main/.gitmodules
> 
> [3] https://github.com/tianocore/edk2-cmocka
> 
> [4]
> https://github.com/tianocore/edk2/commit/2ad22420a710dc07e3b644f91a5b55c09c39
> ecf3


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#119119): https://edk2.groups.io/g/devel/message/119119
Mute This Topic: https://groups.io/mt/106250971/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] libspdm Breaking Builds

[Michael Kubacki]

I don't think that's a very good solution given the diversity of 
downstream projects dependent on the repo or even for the libspdm repo 
itself.

Thanks,
Michael

On 5/22/2024 5:57 PM, Michael D Kinney wrote:
> This was discussed and the decision was made to no use recursive clone on libspdm.
> 
> Mike
> 
>> -----Original Message-----
>> From: Michael Kubacki <mikuback@linux.microsoft.com>
>> Sent: Wednesday, May 22, 2024 1:40 PM
>> To: devel@edk2.groups.io; Yao, Jiewen <jiewen.yao@intel.com>
>> Cc: Kinney, Michael D <michael.d.kinney@intel.com>
>> Subject: libspdm Breaking Builds
>>
>> Recently, the libspdm submodule was added to SecurityPkg
>> (SecurityPkg/DeviceSecurity/SpdmLib/libspdm) [1].
>>
>> That submodule points to https://github.com/DMTF/libspdm.git.
>>
>> The libspdm repo contains a submodule for cmocka that is hosted on
>> cryptomilk.org [2].
>>
>> Cloning that submodule is failing today, breaking all builds/CI
>> dependent on SecurityPkg.
>>
>> This has been an issue in the past and it is very disruptive given the
>> number of dependencies on edk2/SecurityPkg. Because of its reliability
>> issues, edk2 moved to a version of the repo hosted in tianocore [3] in
>> 2021 [4]
>>
>> Can you please make a similar change in libspdm and we get edk2 updated
>> as soon as possible?
>>
>> Thanks,
>> Michael
>>
>> [1]
>> https://github.com/tianocore/edk2/commit/54a4fd9b35cac79e7c54b750d3dddd0fbb43
>> dca1
>>
>> [2] https://github.com/DMTF/libspdm/blob/main/.gitmodules
>>
>> [3] https://github.com/tianocore/edk2-cmocka
>>
>> [4]
>> https://github.com/tianocore/edk2/commit/2ad22420a710dc07e3b644f91a5b55c09c39
>> ecf3
> 
> 
> 
> 
> 


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#119120): https://edk2.groups.io/g/devel/message/119120
Mute This Topic: https://groups.io/mt/106250971/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] libspdm Breaking Builds

[Michael D Kinney]

Libspdm also depends on openssl.  We did not want to clone openssl twice.

I though stuart config specifies which submodules to clone.  Can’t we skip
all the submodules within libspdm to fix CI?

Can't devs choose to not use --recursive?

Mike

> -----Original Message-----
> From: Michael Kubacki <mikuback@linux.microsoft.com>
> Sent: Wednesday, May 22, 2024 3:16 PM
> To: devel@edk2.groups.io; Kinney, Michael D <michael.d.kinney@intel.com>;
> Yao, Jiewen <jiewen.yao@intel.com>
> Subject: Re: [edk2-devel] libspdm Breaking Builds
> 
> I don't think that's a very good solution given the diversity of
> downstream projects dependent on the repo or even for the libspdm repo
> itself.
> 
> Thanks,
> Michael
> 
> On 5/22/2024 5:57 PM, Michael D Kinney wrote:
> > This was discussed and the decision was made to no use recursive clone on
> libspdm.
> >
> > Mike
> >
> >> -----Original Message-----
> >> From: Michael Kubacki <mikuback@linux.microsoft.com>
> >> Sent: Wednesday, May 22, 2024 1:40 PM
> >> To: devel@edk2.groups.io; Yao, Jiewen <jiewen.yao@intel.com>
> >> Cc: Kinney, Michael D <michael.d.kinney@intel.com>
> >> Subject: libspdm Breaking Builds
> >>
> >> Recently, the libspdm submodule was added to SecurityPkg
> >> (SecurityPkg/DeviceSecurity/SpdmLib/libspdm) [1].
> >>
> >> That submodule points to https://github.com/DMTF/libspdm.git.
> >>
> >> The libspdm repo contains a submodule for cmocka that is hosted on
> >> cryptomilk.org [2].
> >>
> >> Cloning that submodule is failing today, breaking all builds/CI
> >> dependent on SecurityPkg.
> >>
> >> This has been an issue in the past and it is very disruptive given the
> >> number of dependencies on edk2/SecurityPkg. Because of its reliability
> >> issues, edk2 moved to a version of the repo hosted in tianocore [3] in
> >> 2021 [4]
> >>
> >> Can you please make a similar change in libspdm and we get edk2 updated
> >> as soon as possible?
> >>
> >> Thanks,
> >> Michael
> >>
> >> [1]
> >>
> https://github.com/tianocore/edk2/commit/54a4fd9b35cac79e7c54b750d3dddd0fbb43
> >> dca1
> >>
> >> [2] https://github.com/DMTF/libspdm/blob/main/.gitmodules
> >>
> >> [3] https://github.com/tianocore/edk2-cmocka
> >>
> >> [4]
> >>
> https://github.com/tianocore/edk2/commit/2ad22420a710dc07e3b644f91a5b55c09c39
> >> ecf3
> >
> >
> > 
> >
> >


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#119121): https://edk2.groups.io/g/devel/message/119121
Mute This Topic: https://groups.io/mt/106250971/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] libspdm Breaking Builds

[Michael Kubacki]

We looked at Stuart and it can prevent a recursive submodule update at 
the first level but then it would prevent further updates. Here Repo A 
can prevent a recursive update in edk2 but it would then not be able to 
get libspdm.

   [Repo A] -[1]> [edk2] -[2]> [libspdm] -[3]> [cmocka]

At its root, the issue is that this is broken, not wasteful. Therefore, 
it is disruptive and a regression for existing workflows.

We, and I'm sure many other projects, recursively clone packages in edk2 
with submodules. For years, this has been fine except for a few brief 
exceptions. We pulled the change with the libspdm dependency into our 
codebase 8 days ago and this has been broken all day. The track record 
of cryptomilk.org in the past was also very poor and frequently caused 
problems. There is not an incident response team that I'm aware of at 
cryptomilk.org that provides status updates and proactively addresses 
services issues (i.e. https://www.githubstatus.com/).

Also, libspdm is now a dependency and cloning cmocka there may fail. 
Users should expect that they can clone and work in that repo as part of 
their firmware development process without frequent service disruptions 
in the way.

While I started this thread to raise the issue for users impacted here, 
I filed https://github.com/DMTF/libspdm/issues/2707 to track the request 
in the libspdm repo.

Thanks,
Michael

On 5/22/2024 6:24 PM, Kinney, Michael D wrote:
> Libspdm also depends on openssl.  We did not want to clone openssl twice.
> 
> I though stuart config specifies which submodules to clone.  Can’t we skip
> all the submodules within libspdm to fix CI?
> 
> Can't devs choose to not use --recursive?
> 
> Mike
> 
>> -----Original Message-----
>> From: Michael Kubacki <mikuback@linux.microsoft.com>
>> Sent: Wednesday, May 22, 2024 3:16 PM
>> To: devel@edk2.groups.io; Kinney, Michael D <michael.d.kinney@intel.com>;
>> Yao, Jiewen <jiewen.yao@intel.com>
>> Subject: Re: [edk2-devel] libspdm Breaking Builds
>>
>> I don't think that's a very good solution given the diversity of
>> downstream projects dependent on the repo or even for the libspdm repo
>> itself.
>>
>> Thanks,
>> Michael
>>
>> On 5/22/2024 5:57 PM, Michael D Kinney wrote:
>>> This was discussed and the decision was made to no use recursive clone on
>> libspdm.
>>>
>>> Mike
>>>
>>>> -----Original Message-----
>>>> From: Michael Kubacki <mikuback@linux.microsoft.com>
>>>> Sent: Wednesday, May 22, 2024 1:40 PM
>>>> To: devel@edk2.groups.io; Yao, Jiewen <jiewen.yao@intel.com>
>>>> Cc: Kinney, Michael D <michael.d.kinney@intel.com>
>>>> Subject: libspdm Breaking Builds
>>>>
>>>> Recently, the libspdm submodule was added to SecurityPkg
>>>> (SecurityPkg/DeviceSecurity/SpdmLib/libspdm) [1].
>>>>
>>>> That submodule points to https://github.com/DMTF/libspdm.git.
>>>>
>>>> The libspdm repo contains a submodule for cmocka that is hosted on
>>>> cryptomilk.org [2].
>>>>
>>>> Cloning that submodule is failing today, breaking all builds/CI
>>>> dependent on SecurityPkg.
>>>>
>>>> This has been an issue in the past and it is very disruptive given the
>>>> number of dependencies on edk2/SecurityPkg. Because of its reliability
>>>> issues, edk2 moved to a version of the repo hosted in tianocore [3] in
>>>> 2021 [4]
>>>>
>>>> Can you please make a similar change in libspdm and we get edk2 updated
>>>> as soon as possible?
>>>>
>>>> Thanks,
>>>> Michael
>>>>
>>>> [1]
>>>>
>> https://github.com/tianocore/edk2/commit/54a4fd9b35cac79e7c54b750d3dddd0fbb43
>>>> dca1
>>>>
>>>> [2] https://github.com/DMTF/libspdm/blob/main/.gitmodules
>>>>
>>>> [3] https://github.com/tianocore/edk2-cmocka
>>>>
>>>> [4]
>>>>
>> https://github.com/tianocore/edk2/commit/2ad22420a710dc07e3b644f91a5b55c09c39
>>>> ecf3
>>>
>>>
>>> 
>>>
>>>


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#119122): https://edk2.groups.io/g/devel/message/119122
Mute This Topic: https://groups.io/mt/106250971/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] libspdm Breaking Builds

[Michael D Kinney]

We have a mirror of cmocka in tianocore.

	https://github.com/tianocore/edk2-cmocka

It is out of sync because GitHub keep disabling the workflow.

And the workflow can not run until cmocka repo is back up.

We updated UnitTestFrameworkPkg to use tianocore cmocka mirror long ago
for this exact failure case.

Since we do not have control over libspdm submodule link to cmocka, what
we need is an override or a failover submodule link to tianocore mirror.

Any ideas on how to implement that concept.  Does git have failover or 
override URL for git submodules?

Or do we need more stuart feature to have more fine grain control over 
Submodules?

Mike

> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Michael
> Kubacki
> Sent: Wednesday, May 22, 2024 6:04 PM
> To: Kinney, Michael D <michael.d.kinney@intel.com>; devel@edk2.groups.io;
> Yao, Jiewen <jiewen.yao@intel.com>
> Subject: Re: [edk2-devel] libspdm Breaking Builds
> 
> We looked at Stuart and it can prevent a recursive submodule update at
> the first level but then it would prevent further updates. Here Repo A
> can prevent a recursive update in edk2 but it would then not be able to
> get libspdm.
> 
>    [Repo A] -[1]> [edk2] -[2]> [libspdm] -[3]> [cmocka]
> 
> At its root, the issue is that this is broken, not wasteful. Therefore,
> it is disruptive and a regression for existing workflows.
> 
> We, and I'm sure many other projects, recursively clone packages in edk2
> with submodules. For years, this has been fine except for a few brief
> exceptions. We pulled the change with the libspdm dependency into our
> codebase 8 days ago and this has been broken all day. The track record
> of cryptomilk.org in the past was also very poor and frequently caused
> problems. There is not an incident response team that I'm aware of at
> cryptomilk.org that provides status updates and proactively addresses
> services issues (i.e. https://www.githubstatus.com/).
> 
> Also, libspdm is now a dependency and cloning cmocka there may fail.
> Users should expect that they can clone and work in that repo as part of
> their firmware development process without frequent service disruptions
> in the way.
> 
> While I started this thread to raise the issue for users impacted here,
> I filed https://github.com/DMTF/libspdm/issues/2707 to track the request
> in the libspdm repo.
> 
> Thanks,
> Michael
> 
> On 5/22/2024 6:24 PM, Kinney, Michael D wrote:
> > Libspdm also depends on openssl.  We did not want to clone openssl twice.
> >
> > I though stuart config specifies which submodules to clone.  Can’t we skip
> > all the submodules within libspdm to fix CI?
> >
> > Can't devs choose to not use --recursive?
> >
> > Mike
> >
> >> -----Original Message-----
> >> From: Michael Kubacki <mikuback@linux.microsoft.com>
> >> Sent: Wednesday, May 22, 2024 3:16 PM
> >> To: devel@edk2.groups.io; Kinney, Michael D <michael.d.kinney@intel.com>;
> >> Yao, Jiewen <jiewen.yao@intel.com>
> >> Subject: Re: [edk2-devel] libspdm Breaking Builds
> >>
> >> I don't think that's a very good solution given the diversity of
> >> downstream projects dependent on the repo or even for the libspdm repo
> >> itself.
> >>
> >> Thanks,
> >> Michael
> >>
> >> On 5/22/2024 5:57 PM, Michael D Kinney wrote:
> >>> This was discussed and the decision was made to no use recursive clone on
> >> libspdm.
> >>>
> >>> Mike
> >>>
> >>>> -----Original Message-----
> >>>> From: Michael Kubacki <mikuback@linux.microsoft.com>
> >>>> Sent: Wednesday, May 22, 2024 1:40 PM
> >>>> To: devel@edk2.groups.io; Yao, Jiewen <jiewen.yao@intel.com>
> >>>> Cc: Kinney, Michael D <michael.d.kinney@intel.com>
> >>>> Subject: libspdm Breaking Builds
> >>>>
> >>>> Recently, the libspdm submodule was added to SecurityPkg
> >>>> (SecurityPkg/DeviceSecurity/SpdmLib/libspdm) [1].
> >>>>
> >>>> That submodule points to https://github.com/DMTF/libspdm.git.
> >>>>
> >>>> The libspdm repo contains a submodule for cmocka that is hosted on
> >>>> cryptomilk.org [2].
> >>>>
> >>>> Cloning that submodule is failing today, breaking all builds/CI
> >>>> dependent on SecurityPkg.
> >>>>
> >>>> This has been an issue in the past and it is very disruptive given the
> >>>> number of dependencies on edk2/SecurityPkg. Because of its reliability
> >>>> issues, edk2 moved to a version of the repo hosted in tianocore [3] in
> >>>> 2021 [4]
> >>>>
> >>>> Can you please make a similar change in libspdm and we get edk2 updated
> >>>> as soon as possible?
> >>>>
> >>>> Thanks,
> >>>> Michael
> >>>>
> >>>> [1]
> >>>>
> >>
> https://github.com/tianocore/edk2/commit/54a4fd9b35cac79e7c54b750d3dddd0fbb43
> >>>> dca1
> >>>>
> >>>> [2] https://github.com/DMTF/libspdm/blob/main/.gitmodules
> >>>>
> >>>> [3] https://github.com/tianocore/edk2-cmocka
> >>>>
> >>>> [4]
> >>>>
> >>
> https://github.com/tianocore/edk2/commit/2ad22420a710dc07e3b644f91a5b55c09c39
> >>>> ecf3
> >>>
> >>>
> >>>
> >>>
> >>>
> 
> 
> 
> 



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#119123): https://edk2.groups.io/g/devel/message/119123
Mute This Topic: https://groups.io/mt/106250971/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] libspdm Breaking Builds

[Yao, Jiewen]

Hello
I am sorry to bring the inconvenience.
I think the libspdm maintainers are aware of fact that the breaking of cmoka impacts the CI.

In history of libspdm, we did see this sometimes, but usually it was back online after a while. 
That is the reason libspdm project is still using cmoka github, since it is good at most of time.

I will discuss libspdm/cmoka issue in regular SPDM TF meeting, and update the issue https://github.com/DMTF/libspdm/issues/2707.


It is similar to what I have observed in tianocore project. Tianocore CI breaks sometimes, and works again after a while. But that is NOT a reason to disable it.

Anyway, I think tianocore project has freedom to choose whatever options, independent with libspdm project. And I hope we have a consistent way to handle all projects.

Thank you
Yao, Jiewen


> -----Original Message-----
> From: Kinney, Michael D <michael.d.kinney@intel.com>
> Sent: Thursday, May 23, 2024 9:52 AM
> To: devel@edk2.groups.io; mikuback@linux.microsoft.com; Yao, Jiewen
> <jiewen.yao@intel.com>
> Cc: Kinney, Michael D <michael.d.kinney@intel.com>
> Subject: RE: [edk2-devel] libspdm Breaking Builds
> 
> We have a mirror of cmocka in tianocore.
> 
> 	https://github.com/tianocore/edk2-cmocka
> 
> It is out of sync because GitHub keep disabling the workflow.
> 
> And the workflow can not run until cmocka repo is back up.
> 
> We updated UnitTestFrameworkPkg to use tianocore cmocka mirror long ago
> for this exact failure case.
> 
> Since we do not have control over libspdm submodule link to cmocka, what
> we need is an override or a failover submodule link to tianocore mirror.
> 
> Any ideas on how to implement that concept.  Does git have failover or
> override URL for git submodules?
> 
> Or do we need more stuart feature to have more fine grain control over
> Submodules?
> 
> Mike
> 
> > -----Original Message-----
> > From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Michael
> > Kubacki
> > Sent: Wednesday, May 22, 2024 6:04 PM
> > To: Kinney, Michael D <michael.d.kinney@intel.com>; devel@edk2.groups.io;
> > Yao, Jiewen <jiewen.yao@intel.com>
> > Subject: Re: [edk2-devel] libspdm Breaking Builds
> >
> > We looked at Stuart and it can prevent a recursive submodule update at
> > the first level but then it would prevent further updates. Here Repo A
> > can prevent a recursive update in edk2 but it would then not be able to
> > get libspdm.
> >
> >    [Repo A] -[1]> [edk2] -[2]> [libspdm] -[3]> [cmocka]
> >
> > At its root, the issue is that this is broken, not wasteful. Therefore,
> > it is disruptive and a regression for existing workflows.
> >
> > We, and I'm sure many other projects, recursively clone packages in edk2
> > with submodules. For years, this has been fine except for a few brief
> > exceptions. We pulled the change with the libspdm dependency into our
> > codebase 8 days ago and this has been broken all day. The track record
> > of cryptomilk.org in the past was also very poor and frequently caused
> > problems. There is not an incident response team that I'm aware of at
> > cryptomilk.org that provides status updates and proactively addresses
> > services issues (i.e. https://www.githubstatus.com/).
> >
> > Also, libspdm is now a dependency and cloning cmocka there may fail.
> > Users should expect that they can clone and work in that repo as part of
> > their firmware development process without frequent service disruptions
> > in the way.
> >
> > While I started this thread to raise the issue for users impacted here,
> > I filed https://github.com/DMTF/libspdm/issues/2707 to track the request
> > in the libspdm repo.
> >
> > Thanks,
> > Michael
> >
> > On 5/22/2024 6:24 PM, Kinney, Michael D wrote:
> > > Libspdm also depends on openssl.  We did not want to clone openssl twice.
> > >
> > > I though stuart config specifies which submodules to clone.  Can’t we skip
> > > all the submodules within libspdm to fix CI?
> > >
> > > Can't devs choose to not use --recursive?
> > >
> > > Mike
> > >
> > >> -----Original Message-----
> > >> From: Michael Kubacki <mikuback@linux.microsoft.com>
> > >> Sent: Wednesday, May 22, 2024 3:16 PM
> > >> To: devel@edk2.groups.io; Kinney, Michael D
> <michael.d.kinney@intel.com>;
> > >> Yao, Jiewen <jiewen.yao@intel.com>
> > >> Subject: Re: [edk2-devel] libspdm Breaking Builds
> > >>
> > >> I don't think that's a very good solution given the diversity of
> > >> downstream projects dependent on the repo or even for the libspdm repo
> > >> itself.
> > >>
> > >> Thanks,
> > >> Michael
> > >>
> > >> On 5/22/2024 5:57 PM, Michael D Kinney wrote:
> > >>> This was discussed and the decision was made to no use recursive clone on
> > >> libspdm.
> > >>>
> > >>> Mike
> > >>>
> > >>>> -----Original Message-----
> > >>>> From: Michael Kubacki <mikuback@linux.microsoft.com>
> > >>>> Sent: Wednesday, May 22, 2024 1:40 PM
> > >>>> To: devel@edk2.groups.io; Yao, Jiewen <jiewen.yao@intel.com>
> > >>>> Cc: Kinney, Michael D <michael.d.kinney@intel.com>
> > >>>> Subject: libspdm Breaking Builds
> > >>>>
> > >>>> Recently, the libspdm submodule was added to SecurityPkg
> > >>>> (SecurityPkg/DeviceSecurity/SpdmLib/libspdm) [1].
> > >>>>
> > >>>> That submodule points to https://github.com/DMTF/libspdm.git.
> > >>>>
> > >>>> The libspdm repo contains a submodule for cmocka that is hosted on
> > >>>> cryptomilk.org [2].
> > >>>>
> > >>>> Cloning that submodule is failing today, breaking all builds/CI
> > >>>> dependent on SecurityPkg.
> > >>>>
> > >>>> This has been an issue in the past and it is very disruptive given the
> > >>>> number of dependencies on edk2/SecurityPkg. Because of its reliability
> > >>>> issues, edk2 moved to a version of the repo hosted in tianocore [3] in
> > >>>> 2021 [4]
> > >>>>
> > >>>> Can you please make a similar change in libspdm and we get edk2
> updated
> > >>>> as soon as possible?
> > >>>>
> > >>>> Thanks,
> > >>>> Michael
> > >>>>
> > >>>> [1]
> > >>>>
> > >>
> >
> https://github.com/tianocore/edk2/commit/54a4fd9b35cac79e7c54b750d3dddd
> 0fbb43
> > >>>> dca1
> > >>>>
> > >>>> [2] https://github.com/DMTF/libspdm/blob/main/.gitmodules
> > >>>>
> > >>>> [3] https://github.com/tianocore/edk2-cmocka
> > >>>>
> > >>>> [4]
> > >>>>
> > >>
> >
> https://github.com/tianocore/edk2/commit/2ad22420a710dc07e3b644f91a5b55
> c09c39
> > >>>> ecf3
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> >
> >
> > 
> >



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#119126): https://edk2.groups.io/g/devel/message/119126
Mute This Topic: https://groups.io/mt/106250971/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] libspdm Breaking Builds

[Yao, Jiewen]

Hello
Thanks for Michael Kubacki's effort. The cmocka for libspdm is switched to https://gitlab.com/cmocka/cmocka.git (https://github.com/DMTF/libspdm/pull/2710).

The next libspdm release is planned at the end of June.

Please let us know if the preference for libspdm submodule. (Below options)
1) Keep current libspdm official 3.3.0 release, and update to next release at the beginning of July.
2) Update libspdm immediately with the new cmocka submodule, which is NOT an official release.


Thank you
Yao, Jiewen

> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Yao, Jiewen
> Sent: Thursday, May 23, 2024 10:17 AM
> To: Kinney, Michael D <michael.d.kinney@intel.com>; devel@edk2.groups.io;
> mikuback@linux.microsoft.com
> Subject: Re: [edk2-devel] libspdm Breaking Builds
> 
> Hello
> I am sorry to bring the inconvenience.
> I think the libspdm maintainers are aware of fact that the breaking of cmoka
> impacts the CI.
> 
> In history of libspdm, we did see this sometimes, but usually it was back online
> after a while.
> That is the reason libspdm project is still using cmoka github, since it is good at
> most of time.
> 
> I will discuss libspdm/cmoka issue in regular SPDM TF meeting, and update the
> issue https://github.com/DMTF/libspdm/issues/2707.
> 
> 
> It is similar to what I have observed in tianocore project. Tianocore CI breaks
> sometimes, and works again after a while. But that is NOT a reason to disable it.
> 
> Anyway, I think tianocore project has freedom to choose whatever options,
> independent with libspdm project. And I hope we have a consistent way to handle
> all projects.
> 
> Thank you
> Yao, Jiewen
> 
> 
> > -----Original Message-----
> > From: Kinney, Michael D <michael.d.kinney@intel.com>
> > Sent: Thursday, May 23, 2024 9:52 AM
> > To: devel@edk2.groups.io; mikuback@linux.microsoft.com; Yao, Jiewen
> > <jiewen.yao@intel.com>
> > Cc: Kinney, Michael D <michael.d.kinney@intel.com>
> > Subject: RE: [edk2-devel] libspdm Breaking Builds
> >
> > We have a mirror of cmocka in tianocore.
> >
> > 	https://github.com/tianocore/edk2-cmocka
> >
> > It is out of sync because GitHub keep disabling the workflow.
> >
> > And the workflow can not run until cmocka repo is back up.
> >
> > We updated UnitTestFrameworkPkg to use tianocore cmocka mirror long ago
> > for this exact failure case.
> >
> > Since we do not have control over libspdm submodule link to cmocka, what
> > we need is an override or a failover submodule link to tianocore mirror.
> >
> > Any ideas on how to implement that concept.  Does git have failover or
> > override URL for git submodules?
> >
> > Or do we need more stuart feature to have more fine grain control over
> > Submodules?
> >
> > Mike
> >
> > > -----Original Message-----
> > > From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Michael
> > > Kubacki
> > > Sent: Wednesday, May 22, 2024 6:04 PM
> > > To: Kinney, Michael D <michael.d.kinney@intel.com>; devel@edk2.groups.io;
> > > Yao, Jiewen <jiewen.yao@intel.com>
> > > Subject: Re: [edk2-devel] libspdm Breaking Builds
> > >
> > > We looked at Stuart and it can prevent a recursive submodule update at
> > > the first level but then it would prevent further updates. Here Repo A
> > > can prevent a recursive update in edk2 but it would then not be able to
> > > get libspdm.
> > >
> > >    [Repo A] -[1]> [edk2] -[2]> [libspdm] -[3]> [cmocka]
> > >
> > > At its root, the issue is that this is broken, not wasteful. Therefore,
> > > it is disruptive and a regression for existing workflows.
> > >
> > > We, and I'm sure many other projects, recursively clone packages in edk2
> > > with submodules. For years, this has been fine except for a few brief
> > > exceptions. We pulled the change with the libspdm dependency into our
> > > codebase 8 days ago and this has been broken all day. The track record
> > > of cryptomilk.org in the past was also very poor and frequently caused
> > > problems. There is not an incident response team that I'm aware of at
> > > cryptomilk.org that provides status updates and proactively addresses
> > > services issues (i.e. https://www.githubstatus.com/).
> > >
> > > Also, libspdm is now a dependency and cloning cmocka there may fail.
> > > Users should expect that they can clone and work in that repo as part of
> > > their firmware development process without frequent service disruptions
> > > in the way.
> > >
> > > While I started this thread to raise the issue for users impacted here,
> > > I filed https://github.com/DMTF/libspdm/issues/2707 to track the request
> > > in the libspdm repo.
> > >
> > > Thanks,
> > > Michael
> > >
> > > On 5/22/2024 6:24 PM, Kinney, Michael D wrote:
> > > > Libspdm also depends on openssl.  We did not want to clone openssl twice.
> > > >
> > > > I though stuart config specifies which submodules to clone.  Can't we skip
> > > > all the submodules within libspdm to fix CI?
> > > >
> > > > Can't devs choose to not use --recursive?
> > > >
> > > > Mike
> > > >
> > > >> -----Original Message-----
> > > >> From: Michael Kubacki <mikuback@linux.microsoft.com>
> > > >> Sent: Wednesday, May 22, 2024 3:16 PM
> > > >> To: devel@edk2.groups.io; Kinney, Michael D
> > <michael.d.kinney@intel.com>;
> > > >> Yao, Jiewen <jiewen.yao@intel.com>
> > > >> Subject: Re: [edk2-devel] libspdm Breaking Builds
> > > >>
> > > >> I don't think that's a very good solution given the diversity of
> > > >> downstream projects dependent on the repo or even for the libspdm repo
> > > >> itself.
> > > >>
> > > >> Thanks,
> > > >> Michael
> > > >>
> > > >> On 5/22/2024 5:57 PM, Michael D Kinney wrote:
> > > >>> This was discussed and the decision was made to no use recursive clone
> on
> > > >> libspdm.
> > > >>>
> > > >>> Mike
> > > >>>
> > > >>>> -----Original Message-----
> > > >>>> From: Michael Kubacki <mikuback@linux.microsoft.com>
> > > >>>> Sent: Wednesday, May 22, 2024 1:40 PM
> > > >>>> To: devel@edk2.groups.io; Yao, Jiewen <jiewen.yao@intel.com>
> > > >>>> Cc: Kinney, Michael D <michael.d.kinney@intel.com>
> > > >>>> Subject: libspdm Breaking Builds
> > > >>>>
> > > >>>> Recently, the libspdm submodule was added to SecurityPkg
> > > >>>> (SecurityPkg/DeviceSecurity/SpdmLib/libspdm) [1].
> > > >>>>
> > > >>>> That submodule points to https://github.com/DMTF/libspdm.git.
> > > >>>>
> > > >>>> The libspdm repo contains a submodule for cmocka that is hosted on
> > > >>>> cryptomilk.org [2].
> > > >>>>
> > > >>>> Cloning that submodule is failing today, breaking all builds/CI
> > > >>>> dependent on SecurityPkg.
> > > >>>>
> > > >>>> This has been an issue in the past and it is very disruptive given the
> > > >>>> number of dependencies on edk2/SecurityPkg. Because of its reliability
> > > >>>> issues, edk2 moved to a version of the repo hosted in tianocore [3] in
> > > >>>> 2021 [4]
> > > >>>>
> > > >>>> Can you please make a similar change in libspdm and we get edk2
> > updated
> > > >>>> as soon as possible?
> > > >>>>
> > > >>>> Thanks,
> > > >>>> Michael
> > > >>>>
> > > >>>> [1]
> > > >>>>
> > > >>
> > >
> >
> https://github.com/tianocore/edk2/commit/54a4fd9b35cac79e7c54b750d3dddd
> > 0fbb43
> > > >>>> dca1
> > > >>>>
> > > >>>> [2] https://github.com/DMTF/libspdm/blob/main/.gitmodules
> > > >>>>
> > > >>>> [3] https://github.com/tianocore/edk2-cmocka
> > > >>>>
> > > >>>> [4]
> > > >>>>
> > > >>
> > >
> >
> https://github.com/tianocore/edk2/commit/2ad22420a710dc07e3b644f91a5b55
> > c09c39
> > > >>>> ecf3
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > >
> > >
> > >
> > >
> 
> 
> 
> 
> 



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#119178): https://edk2.groups.io/g/devel/message/119178
Mute This Topic: https://groups.io/mt/106250971/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] libspdm Breaking Builds

[Pete Batard via groups.io]

Hello all,

On 2024.05.24 03:13, Yao, Jiewen via groups.io wrote:
> Please let us know if the preference for libspdm submodule. (Below options)
> 1) Keep current libspdm official 3.3.0 release, and update to next release at the beginning of July.
> 2) Update libspdm immediately with the new cmocka submodule, which is NOT an official release.

Considering that I (and I expect anybody who tries to use EDK2 as a 
submodule in their UEFI build projects with GitHub Actions), I have to 
vote for option 2.

An example of the current issue can be shown on a project that simply 
attempts to build the UEFI Shell from the latest stable EDK2 release, 
using EDK2 as a submodule, can be shown at 
https://github.com/pbatard/UEFI-Shell/actions/runs/9290685065/job/25567879807 
or 
https://github.com/pbatard/UEFI-Shell/actions/runs/9290988138/job/25568511355 
and as you can see, it makes building the project completely impossible 
unless you ditch using EDK2 as a submodule (which isn't a viable option 
IMO, because a build toolchain that cannot be used as a git submodule is 
a very limiting toolchain).

For information, there's only so much fine grained tuning GitHub Actions 
offers on submodules, and no matter how you try to play with the fetch 
depth, the fact that one of the libspdm sub-dependency has essentially 
become M.I.A. is something that should be addressed as a matter of urgency.

So I hope that a commit that updates libspdm to the new cmocka submodule 
can find its way into EDK2 fairly soon, as it is currently halting a 
projects that aims at producing trusted UEFI Shell releases.

Regards,

/Pete


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#119358): https://edk2.groups.io/g/devel/message/119358
Mute This Topic: https://groups.io/mt/106250971/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] libspdm Breaking Builds

[Michael D Kinney]

Hi Pete,

There is another option for developers and CI agents.

Git supports a URL insteadof option to redirect git requests.

    https://git-scm.com/docs/git-config#Documentation/git-config.txt-urlltbasegtinsteadOf

We can use this to redirect a request from cryptomilk cmocka to the 
TianoCore mirror of cryptomilk cmocka.

For developers, this can be a global config setting so it works 
for all edk2 trees on their system.

For a CI agent, this could be ab early step in all CI jobs to perform
a git config action.  Perhaps a feature Stuart could adopt to support
URL redirects.

Mike


> -----Original Message-----
> From: Pete Batard <pete@akeo.ie>
> Sent: Wednesday, May 29, 2024 11:18 AM
> To: devel@edk2.groups.io; Yao, Jiewen <jiewen.yao@intel.com>; Kinney, Michael
> D <michael.d.kinney@intel.com>; mikuback@linux.microsoft.com
> Subject: Re: [edk2-devel] libspdm Breaking Builds
> 
> Hello all,
> 
> On 2024.05.24 03:13, Yao, Jiewen via groups.io wrote:
> > Please let us know if the preference for libspdm submodule. (Below options)
> > 1) Keep current libspdm official 3.3.0 release, and update to next release
> at the beginning of July.
> > 2) Update libspdm immediately with the new cmocka submodule, which is NOT
> an official release.
> 
> Considering that I (and I expect anybody who tries to use EDK2 as a
> submodule in their UEFI build projects with GitHub Actions), I have to
> vote for option 2.
> 
> An example of the current issue can be shown on a project that simply
> attempts to build the UEFI Shell from the latest stable EDK2 release,
> using EDK2 as a submodule, can be shown at
> https://github.com/pbatard/UEFI-Shell/actions/runs/9290685065/job/25567879807
> or
> https://github.com/pbatard/UEFI-Shell/actions/runs/9290988138/job/25568511355
> and as you can see, it makes building the project completely impossible
> unless you ditch using EDK2 as a submodule (which isn't a viable option
> IMO, because a build toolchain that cannot be used as a git submodule is
> a very limiting toolchain).
> 
> For information, there's only so much fine grained tuning GitHub Actions
> offers on submodules, and no matter how you try to play with the fetch
> depth, the fact that one of the libspdm sub-dependency has essentially
> become M.I.A. is something that should be addressed as a matter of urgency.
> 
> So I hope that a commit that updates libspdm to the new cmocka submodule
> can find its way into EDK2 fairly soon, as it is currently halting a
> projects that aims at producing trusted UEFI Shell releases.
> 
> Regards,
> 
> /Pete


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#119361): https://edk2.groups.io/g/devel/message/119361
Mute This Topic: https://groups.io/mt/106250971/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] libspdm Breaking Builds

[Michael D Kinney]

Hi Pete,

I just tested this config command and it works when cryptomilk is down.

    git config --global url."https://github.com/tianocore/edk2-cmocka.git".insteadOf "https://git.cryptomilk.org/projects/cmocka.git"

I updated edk2-cmocka mirror last week.

I tested this with

    git clone https://github.com/tianocore/edk2 --recursive

Best regards,

Mike

> -----Original Message-----
> From: Kinney, Michael D <michael.d.kinney@intel.com>
> Sent: Wednesday, May 29, 2024 11:33 AM
> To: Pete Batard <pete@akeo.ie>; devel@edk2.groups.io; Yao, Jiewen
> <jiewen.yao@intel.com>; mikuback@linux.microsoft.com
> Cc: Kinney, Michael D <michael.d.kinney@intel.com>
> Subject: RE: [edk2-devel] libspdm Breaking Builds
> 
> Hi Pete,
> 
> There is another option for developers and CI agents.
> 
> Git supports a URL insteadof option to redirect git requests.
> 
>     https://git-scm.com/docs/git-config#Documentation/git-config.txt-
> urlltbasegtinsteadOf
> 
> We can use this to redirect a request from cryptomilk cmocka to the
> TianoCore mirror of cryptomilk cmocka.
> 
> For developers, this can be a global config setting so it works
> for all edk2 trees on their system.
> 
> For a CI agent, this could be ab early step in all CI jobs to perform
> a git config action.  Perhaps a feature Stuart could adopt to support
> URL redirects.
> 
> Mike
> 
> 
> > -----Original Message-----
> > From: Pete Batard <pete@akeo.ie>
> > Sent: Wednesday, May 29, 2024 11:18 AM
> > To: devel@edk2.groups.io; Yao, Jiewen <jiewen.yao@intel.com>; Kinney,
> Michael
> > D <michael.d.kinney@intel.com>; mikuback@linux.microsoft.com
> > Subject: Re: [edk2-devel] libspdm Breaking Builds
> >
> > Hello all,
> >
> > On 2024.05.24 03:13, Yao, Jiewen via groups.io wrote:
> > > Please let us know if the preference for libspdm submodule. (Below
> options)
> > > 1) Keep current libspdm official 3.3.0 release, and update to next
> release
> > at the beginning of July.
> > > 2) Update libspdm immediately with the new cmocka submodule, which is NOT
> > an official release.
> >
> > Considering that I (and I expect anybody who tries to use EDK2 as a
> > submodule in their UEFI build projects with GitHub Actions), I have to
> > vote for option 2.
> >
> > An example of the current issue can be shown on a project that simply
> > attempts to build the UEFI Shell from the latest stable EDK2 release,
> > using EDK2 as a submodule, can be shown at
> > https://github.com/pbatard/UEFI-
> Shell/actions/runs/9290685065/job/25567879807
> > or
> > https://github.com/pbatard/UEFI-
> Shell/actions/runs/9290988138/job/25568511355
> > and as you can see, it makes building the project completely impossible
> > unless you ditch using EDK2 as a submodule (which isn't a viable option
> > IMO, because a build toolchain that cannot be used as a git submodule is
> > a very limiting toolchain).
> >
> > For information, there's only so much fine grained tuning GitHub Actions
> > offers on submodules, and no matter how you try to play with the fetch
> > depth, the fact that one of the libspdm sub-dependency has essentially
> > become M.I.A. is something that should be addressed as a matter of urgency.
> >
> > So I hope that a commit that updates libspdm to the new cmocka submodule
> > can find its way into EDK2 fairly soon, as it is currently halting a
> > projects that aims at producing trusted UEFI Shell releases.
> >
> > Regards,
> >
> > /Pete


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#119363): https://edk2.groups.io/g/devel/message/119363
Mute This Topic: https://groups.io/mt/106250971/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] libspdm Breaking Builds

[Michael Kubacki]

Since we already reached agreement in libspdm to move its cmocka 
submodule to the gitlab mirror in 
https://github.com/DMTF/libspdm/issues/2707 and that was merged in 
https://github.com/tianocore/edk2/pull/5697.

I'd like to suggest we go with option 2 as well. This is impacting our 
ability to use the DeviceSecurity modules. We've currently disabled 
those modules entirely in our local copy of SecurityPkg and disabled the 
libspdm submodule until a more stable solution is in place.

Here's a PR for option 2.

https://github.com/tianocore/edk2/pull/5697

Thanks,
Michael

On 5/29/2024 3:40 PM, Kinney, Michael D wrote:
> Hi Pete,
> 
> I just tested this config command and it works when cryptomilk is down.
> 
>      git config --global url."https://github.com/tianocore/edk2-cmocka.git".insteadOf "https://git.cryptomilk.org/projects/cmocka.git"
> 
> I updated edk2-cmocka mirror last week.
> 
> I tested this with
> 
>      git clone https://github.com/tianocore/edk2 --recursive
> 
> Best regards,
> 
> Mike
> 
>> -----Original Message-----
>> From: Kinney, Michael D <michael.d.kinney@intel.com>
>> Sent: Wednesday, May 29, 2024 11:33 AM
>> To: Pete Batard <pete@akeo.ie>; devel@edk2.groups.io; Yao, Jiewen
>> <jiewen.yao@intel.com>; mikuback@linux.microsoft.com
>> Cc: Kinney, Michael D <michael.d.kinney@intel.com>
>> Subject: RE: [edk2-devel] libspdm Breaking Builds
>>
>> Hi Pete,
>>
>> There is another option for developers and CI agents.
>>
>> Git supports a URL insteadof option to redirect git requests.
>>
>>      https://git-scm.com/docs/git-config#Documentation/git-config.txt-
>> urlltbasegtinsteadOf
>>
>> We can use this to redirect a request from cryptomilk cmocka to the
>> TianoCore mirror of cryptomilk cmocka.
>>
>> For developers, this can be a global config setting so it works
>> for all edk2 trees on their system.
>>
>> For a CI agent, this could be ab early step in all CI jobs to perform
>> a git config action.  Perhaps a feature Stuart could adopt to support
>> URL redirects.
>>
>> Mike
>>
>>
>>> -----Original Message-----
>>> From: Pete Batard <pete@akeo.ie>
>>> Sent: Wednesday, May 29, 2024 11:18 AM
>>> To: devel@edk2.groups.io; Yao, Jiewen <jiewen.yao@intel.com>; Kinney,
>> Michael
>>> D <michael.d.kinney@intel.com>; mikuback@linux.microsoft.com
>>> Subject: Re: [edk2-devel] libspdm Breaking Builds
>>>
>>> Hello all,
>>>
>>> On 2024.05.24 03:13, Yao, Jiewen via groups.io wrote:
>>>> Please let us know if the preference for libspdm submodule. (Below
>> options)
>>>> 1) Keep current libspdm official 3.3.0 release, and update to next
>> release
>>> at the beginning of July.
>>>> 2) Update libspdm immediately with the new cmocka submodule, which is NOT
>>> an official release.
>>>
>>> Considering that I (and I expect anybody who tries to use EDK2 as a
>>> submodule in their UEFI build projects with GitHub Actions), I have to
>>> vote for option 2.
>>>
>>> An example of the current issue can be shown on a project that simply
>>> attempts to build the UEFI Shell from the latest stable EDK2 release,
>>> using EDK2 as a submodule, can be shown at
>>> https://github.com/pbatard/UEFI-
>> Shell/actions/runs/9290685065/job/25567879807
>>> or
>>> https://github.com/pbatard/UEFI-
>> Shell/actions/runs/9290988138/job/25568511355
>>> and as you can see, it makes building the project completely impossible
>>> unless you ditch using EDK2 as a submodule (which isn't a viable option
>>> IMO, because a build toolchain that cannot be used as a git submodule is
>>> a very limiting toolchain).
>>>
>>> For information, there's only so much fine grained tuning GitHub Actions
>>> offers on submodules, and no matter how you try to play with the fetch
>>> depth, the fact that one of the libspdm sub-dependency has essentially
>>> become M.I.A. is something that should be addressed as a matter of urgency.
>>>
>>> So I hope that a commit that updates libspdm to the new cmocka submodule
>>> can find its way into EDK2 fairly soon, as it is currently halting a
>>> projects that aims at producing trusted UEFI Shell releases.
>>>
>>> Regards,
>>>
>>> /Pete


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#119364): https://edk2.groups.io/g/devel/message/119364
Mute This Topic: https://groups.io/mt/106250971/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] libspdm Breaking Builds

[Yao, Jiewen]

Thanks Michael Kubacki,

It is merged.

> -----Original Message-----
> From: Michael Kubacki <mikuback@linux.microsoft.com>
> Sent: Thursday, May 30, 2024 3:57 AM
> To: Kinney, Michael D <michael.d.kinney@intel.com>; Pete Batard
> <pete@akeo.ie>; devel@edk2.groups.io; Yao, Jiewen <jiewen.yao@intel.com>
> Subject: Re: [edk2-devel] libspdm Breaking Builds
> 
> Since we already reached agreement in libspdm to move its cmocka
> submodule to the gitlab mirror in
> https://github.com/DMTF/libspdm/issues/2707 and that was merged in
> https://github.com/tianocore/edk2/pull/5697.
> 
> I'd like to suggest we go with option 2 as well. This is impacting our
> ability to use the DeviceSecurity modules. We've currently disabled
> those modules entirely in our local copy of SecurityPkg and disabled the
> libspdm submodule until a more stable solution is in place.
> 
> Here's a PR for option 2.
> 
> https://github.com/tianocore/edk2/pull/5697
> 
> Thanks,
> Michael
> 
> On 5/29/2024 3:40 PM, Kinney, Michael D wrote:
> > Hi Pete,
> >
> > I just tested this config command and it works when cryptomilk is down.
> >
> >      git config --global url."https://github.com/tianocore/edk2-
> cmocka.git".insteadOf "https://git.cryptomilk.org/projects/cmocka.git"
> >
> > I updated edk2-cmocka mirror last week.
> >
> > I tested this with
> >
> >      git clone https://github.com/tianocore/edk2 --recursive
> >
> > Best regards,
> >
> > Mike
> >
> >> -----Original Message-----
> >> From: Kinney, Michael D <michael.d.kinney@intel.com>
> >> Sent: Wednesday, May 29, 2024 11:33 AM
> >> To: Pete Batard <pete@akeo.ie>; devel@edk2.groups.io; Yao, Jiewen
> >> <jiewen.yao@intel.com>; mikuback@linux.microsoft.com
> >> Cc: Kinney, Michael D <michael.d.kinney@intel.com>
> >> Subject: RE: [edk2-devel] libspdm Breaking Builds
> >>
> >> Hi Pete,
> >>
> >> There is another option for developers and CI agents.
> >>
> >> Git supports a URL insteadof option to redirect git requests.
> >>
> >>      https://git-scm.com/docs/git-config#Documentation/git-config.txt-
> >> urlltbasegtinsteadOf
> >>
> >> We can use this to redirect a request from cryptomilk cmocka to the
> >> TianoCore mirror of cryptomilk cmocka.
> >>
> >> For developers, this can be a global config setting so it works
> >> for all edk2 trees on their system.
> >>
> >> For a CI agent, this could be ab early step in all CI jobs to perform
> >> a git config action.  Perhaps a feature Stuart could adopt to support
> >> URL redirects.
> >>
> >> Mike
> >>
> >>
> >>> -----Original Message-----
> >>> From: Pete Batard <pete@akeo.ie>
> >>> Sent: Wednesday, May 29, 2024 11:18 AM
> >>> To: devel@edk2.groups.io; Yao, Jiewen <jiewen.yao@intel.com>; Kinney,
> >> Michael
> >>> D <michael.d.kinney@intel.com>; mikuback@linux.microsoft.com
> >>> Subject: Re: [edk2-devel] libspdm Breaking Builds
> >>>
> >>> Hello all,
> >>>
> >>> On 2024.05.24 03:13, Yao, Jiewen via groups.io wrote:
> >>>> Please let us know if the preference for libspdm submodule. (Below
> >> options)
> >>>> 1) Keep current libspdm official 3.3.0 release, and update to next
> >> release
> >>> at the beginning of July.
> >>>> 2) Update libspdm immediately with the new cmocka submodule, which is
> NOT
> >>> an official release.
> >>>
> >>> Considering that I (and I expect anybody who tries to use EDK2 as a
> >>> submodule in their UEFI build projects with GitHub Actions), I have to
> >>> vote for option 2.
> >>>
> >>> An example of the current issue can be shown on a project that simply
> >>> attempts to build the UEFI Shell from the latest stable EDK2 release,
> >>> using EDK2 as a submodule, can be shown at
> >>> https://github.com/pbatard/UEFI-
> >> Shell/actions/runs/9290685065/job/25567879807
> >>> or
> >>> https://github.com/pbatard/UEFI-
> >> Shell/actions/runs/9290988138/job/25568511355
> >>> and as you can see, it makes building the project completely impossible
> >>> unless you ditch using EDK2 as a submodule (which isn't a viable option
> >>> IMO, because a build toolchain that cannot be used as a git submodule is
> >>> a very limiting toolchain).
> >>>
> >>> For information, there's only so much fine grained tuning GitHub Actions
> >>> offers on submodules, and no matter how you try to play with the fetch
> >>> depth, the fact that one of the libspdm sub-dependency has essentially
> >>> become M.I.A. is something that should be addressed as a matter of urgency.
> >>>
> >>> So I hope that a commit that updates libspdm to the new cmocka submodule
> >>> can find its way into EDK2 fairly soon, as it is currently halting a
> >>> projects that aims at producing trusted UEFI Shell releases.
> >>>
> >>> Regards,
> >>>
> >>> /Pete


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#119370): https://edk2.groups.io/g/devel/message/119370
Mute This Topic: https://groups.io/mt/106250971/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] libspdm Breaking Builds

[Pete Batard via groups.io]

Thanks Michael,

After working around some weird rate-limiting from GitHub, I was finally 
able to apply your suggested workaround, and can validate that it indeed 
works: 
https://github.com/pbatard/UEFI-Shell/actions/runs/9293562444/job/25576978957

Still, I would assert that many people trying to use EDK2 as a submodule 
dependency in automated workflow will want to avoid having juggling with 
workarounds, so the sooner the breaking dependency is changed in EDK2, 
the better.

Regards,

/Pete

On 2024.05.29 20:40, Kinney, Michael D wrote:
> Hi Pete,
> 
> I just tested this config command and it works when cryptomilk is down.
> 
>      git config --global url."https://github.com/tianocore/edk2-cmocka.git".insteadOf "https://git.cryptomilk.org/projects/cmocka.git"
> 
> I updated edk2-cmocka mirror last week.
> 
> I tested this with
> 
>      git clone https://github.com/tianocore/edk2 --recursive
> 
> Best regards,
> 
> Mike
> 
>> -----Original Message-----
>> From: Kinney, Michael D <michael.d.kinney@intel.com>
>> Sent: Wednesday, May 29, 2024 11:33 AM
>> To: Pete Batard <pete@akeo.ie>; devel@edk2.groups.io; Yao, Jiewen
>> <jiewen.yao@intel.com>; mikuback@linux.microsoft.com
>> Cc: Kinney, Michael D <michael.d.kinney@intel.com>
>> Subject: RE: [edk2-devel] libspdm Breaking Builds
>>
>> Hi Pete,
>>
>> There is another option for developers and CI agents.
>>
>> Git supports a URL insteadof option to redirect git requests.
>>
>>      https://git-scm.com/docs/git-config#Documentation/git-config.txt-
>> urlltbasegtinsteadOf
>>
>> We can use this to redirect a request from cryptomilk cmocka to the
>> TianoCore mirror of cryptomilk cmocka.
>>
>> For developers, this can be a global config setting so it works
>> for all edk2 trees on their system.
>>
>> For a CI agent, this could be ab early step in all CI jobs to perform
>> a git config action.  Perhaps a feature Stuart could adopt to support
>> URL redirects.
>>
>> Mike
>>
>>
>>> -----Original Message-----
>>> From: Pete Batard <pete@akeo.ie>
>>> Sent: Wednesday, May 29, 2024 11:18 AM
>>> To: devel@edk2.groups.io; Yao, Jiewen <jiewen.yao@intel.com>; Kinney,
>> Michael
>>> D <michael.d.kinney@intel.com>; mikuback@linux.microsoft.com
>>> Subject: Re: [edk2-devel] libspdm Breaking Builds
>>>
>>> Hello all,
>>>
>>> On 2024.05.24 03:13, Yao, Jiewen via groups.io wrote:
>>>> Please let us know if the preference for libspdm submodule. (Below
>> options)
>>>> 1) Keep current libspdm official 3.3.0 release, and update to next
>> release
>>> at the beginning of July.
>>>> 2) Update libspdm immediately with the new cmocka submodule, which is NOT
>>> an official release.
>>>
>>> Considering that I (and I expect anybody who tries to use EDK2 as a
>>> submodule in their UEFI build projects with GitHub Actions), I have to
>>> vote for option 2.
>>>
>>> An example of the current issue can be shown on a project that simply
>>> attempts to build the UEFI Shell from the latest stable EDK2 release,
>>> using EDK2 as a submodule, can be shown at
>>> https://github.com/pbatard/UEFI-
>> Shell/actions/runs/9290685065/job/25567879807
>>> or
>>> https://github.com/pbatard/UEFI-
>> Shell/actions/runs/9290988138/job/25568511355
>>> and as you can see, it makes building the project completely impossible
>>> unless you ditch using EDK2 as a submodule (which isn't a viable option
>>> IMO, because a build toolchain that cannot be used as a git submodule is
>>> a very limiting toolchain).
>>>
>>> For information, there's only so much fine grained tuning GitHub Actions
>>> offers on submodules, and no matter how you try to play with the fetch
>>> depth, the fact that one of the libspdm sub-dependency has essentially
>>> become M.I.A. is something that should be addressed as a matter of urgency.
>>>
>>> So I hope that a commit that updates libspdm to the new cmocka submodule
>>> can find its way into EDK2 fairly soon, as it is currently halting a
>>> projects that aims at producing trusted UEFI Shell releases.
>>>
>>> Regards,
>>>
>>> /Pete



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#119367): https://edk2.groups.io/g/devel/message/119367
Mute This Topic: https://groups.io/mt/106250971/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] libspdm Breaking Builds

[Michael D Kinney]

Hi Pete,

https://github.com/tianocore/edk2/pull/5697 was merged and should
resolve your issue.

Please let us know if it does not.

Mike

> -----Original Message-----
> From: Pete Batard <pete@akeo.ie>
> Sent: Wednesday, May 29, 2024 3:01 PM
> To: Kinney, Michael D <michael.d.kinney@intel.com>; devel@edk2.groups.io;
> Yao, Jiewen <jiewen.yao@intel.com>; mikuback@linux.microsoft.com
> Subject: Re: [edk2-devel] libspdm Breaking Builds
> 
> Thanks Michael,
> 
> After working around some weird rate-limiting from GitHub, I was finally
> able to apply your suggested workaround, and can validate that it indeed
> works:
> https://github.com/pbatard/UEFI-Shell/actions/runs/9293562444/job/25576978957
> 
> Still, I would assert that many people trying to use EDK2 as a submodule
> dependency in automated workflow will want to avoid having juggling with
> workarounds, so the sooner the breaking dependency is changed in EDK2,
> the better.
> 
> Regards,
> 
> /Pete
> 
> On 2024.05.29 20:40, Kinney, Michael D wrote:
> > Hi Pete,
> >
> > I just tested this config command and it works when cryptomilk is down.
> >
> >      git config --global url."https://github.com/tianocore/edk2-
> cmocka.git".insteadOf "https://git.cryptomilk.org/projects/cmocka.git"
> >
> > I updated edk2-cmocka mirror last week.
> >
> > I tested this with
> >
> >      git clone https://github.com/tianocore/edk2 --recursive
> >
> > Best regards,
> >
> > Mike
> >
> >> -----Original Message-----
> >> From: Kinney, Michael D <michael.d.kinney@intel.com>
> >> Sent: Wednesday, May 29, 2024 11:33 AM
> >> To: Pete Batard <pete@akeo.ie>; devel@edk2.groups.io; Yao, Jiewen
> >> <jiewen.yao@intel.com>; mikuback@linux.microsoft.com
> >> Cc: Kinney, Michael D <michael.d.kinney@intel.com>
> >> Subject: RE: [edk2-devel] libspdm Breaking Builds
> >>
> >> Hi Pete,
> >>
> >> There is another option for developers and CI agents.
> >>
> >> Git supports a URL insteadof option to redirect git requests.
> >>
> >>      https://git-scm.com/docs/git-config#Documentation/git-config.txt-
> >> urlltbasegtinsteadOf
> >>
> >> We can use this to redirect a request from cryptomilk cmocka to the
> >> TianoCore mirror of cryptomilk cmocka.
> >>
> >> For developers, this can be a global config setting so it works
> >> for all edk2 trees on their system.
> >>
> >> For a CI agent, this could be ab early step in all CI jobs to perform
> >> a git config action.  Perhaps a feature Stuart could adopt to support
> >> URL redirects.
> >>
> >> Mike
> >>
> >>
> >>> -----Original Message-----
> >>> From: Pete Batard <pete@akeo.ie>
> >>> Sent: Wednesday, May 29, 2024 11:18 AM
> >>> To: devel@edk2.groups.io; Yao, Jiewen <jiewen.yao@intel.com>; Kinney,
> >> Michael
> >>> D <michael.d.kinney@intel.com>; mikuback@linux.microsoft.com
> >>> Subject: Re: [edk2-devel] libspdm Breaking Builds
> >>>
> >>> Hello all,
> >>>
> >>> On 2024.05.24 03:13, Yao, Jiewen via groups.io wrote:
> >>>> Please let us know if the preference for libspdm submodule. (Below
> >> options)
> >>>> 1) Keep current libspdm official 3.3.0 release, and update to next
> >> release
> >>> at the beginning of July.
> >>>> 2) Update libspdm immediately with the new cmocka submodule, which is
> NOT
> >>> an official release.
> >>>
> >>> Considering that I (and I expect anybody who tries to use EDK2 as a
> >>> submodule in their UEFI build projects with GitHub Actions), I have to
> >>> vote for option 2.
> >>>
> >>> An example of the current issue can be shown on a project that simply
> >>> attempts to build the UEFI Shell from the latest stable EDK2 release,
> >>> using EDK2 as a submodule, can be shown at
> >>> https://github.com/pbatard/UEFI-
> >> Shell/actions/runs/9290685065/job/25567879807
> >>> or
> >>> https://github.com/pbatard/UEFI-
> >> Shell/actions/runs/9290988138/job/25568511355
> >>> and as you can see, it makes building the project completely impossible
> >>> unless you ditch using EDK2 as a submodule (which isn't a viable option
> >>> IMO, because a build toolchain that cannot be used as a git submodule is
> >>> a very limiting toolchain).
> >>>
> >>> For information, there's only so much fine grained tuning GitHub Actions
> >>> offers on submodules, and no matter how you try to play with the fetch
> >>> depth, the fact that one of the libspdm sub-dependency has essentially
> >>> become M.I.A. is something that should be addressed as a matter of
> urgency.
> >>>
> >>> So I hope that a commit that updates libspdm to the new cmocka submodule
> >>> can find its way into EDK2 fairly soon, as it is currently halting a
> >>> projects that aims at producing trusted UEFI Shell releases.
> >>>
> >>> Regards,
> >>>
> >>> /Pete



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#119371): https://edk2.groups.io/g/devel/message/119371
Mute This Topic: https://groups.io/mt/106250971/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-