PCD Migration issue

PCD Migration issue

[Andrew Fish]

We recently hit an issue when updating our UDK version in our common code base. Our recovery update path started failing for older platforms since the PEI was the old UDK version and the DXE from the Capsule was the new UDK version. Basically the version of the PCD changed. It looks like there are some checks, but the seem to be more about making sure the build systems matches the code vs. dealing with an update case like we hit. 

It does not look like the PCD database is designed to deal with this. We are fixing this by extracting the PEI PCD database from the capsule and then having some platform specific code to patch any PCD entries that got set in PEI that are needed by DXE. 

In the future it would be helpful if the PCD database would change in some what of a backward compatible way, and have the headers needed to parse the old and new version. 

It also seems like adding a Dynamic PCD could potentially change the token layout and thus break compatibility? Is there any scheme to keep a previous token layout? 

Thanks,

Andrew Fish

Re: [edk2-devel] PCD Migration issue

[Liming Gao]

Andrew:
  Current implementation requires PEI PcdDB and DXE PcdDB match. This is like a new request for PCD DataBase. It requests PCD DB binary format and Dynamic PCD token assignment be compatible in the different version.

Thanks
Liming
> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Andrew Fish via Groups.Io
> Sent: Tuesday, February 11, 2020 1:39 AM
> To: devel@edk2.groups.io
> Subject: [edk2-devel] PCD Migration issue
> 
> We recently hit an issue when updating our UDK version in our common code base. Our recovery update path started failing for older
> platforms since the PEI was the old UDK version and the DXE from the Capsule was the new UDK version. Basically the version of the PCD
> changed. It looks like there are some checks, but the seem to be more about making sure the build systems matches the code vs. dealing
> with an update case like we hit.
> 
> It does not look like the PCD database is designed to deal with this. We are fixing this by extracting the PEI PCD database from the
> capsule and then having some platform specific code to patch any PCD entries that got set in PEI that are needed by DXE.
> 
> In the future it would be helpful if the PCD database would change in some what of a backward compatible way, and have the headers
> needed to parse the old and new version.
> 
> It also seems like adding a Dynamic PCD could potentially change the token layout and thus break compatibility? Is there any scheme to
> keep a previous token layout?
> 
> Thanks,
> 
> Andrew Fish
> 

Re: [edk2-devel] PCD Migration issue

[Andrew Fish]

Liming,

I agree I don't think this has been thought about. 

Does the edk2 still support booting a split PEI from ROM and DXE from capsule for update? If yes that seems to imply the assumption is that platform would not change any Dynamic PCD layout, or update the PCD database versions. If you have a platform in a common core base, and update the EDK2 code to pick up security fixes it seems the PCD data base implies you have to freeze Dynamic PCD usage in your common core? 

Thanks,

Andrew Fish

> On Feb 11, 2020, at 7:54 PM, Liming Gao <liming.gao@intel.com> wrote:
> 
> Andrew:
>  Current implementation requires PEI PcdDB and DXE PcdDB match. This is like a new request for PCD DataBase. It requests PCD DB binary format and Dynamic PCD token assignment be compatible in the different version.
> 
> Thanks
> Liming
>> -----Original Message-----
>> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Andrew Fish via Groups.Io
>> Sent: Tuesday, February 11, 2020 1:39 AM
>> To: devel@edk2.groups.io
>> Subject: [edk2-devel] PCD Migration issue
>> 
>> We recently hit an issue when updating our UDK version in our common code base. Our recovery update path started failing for older
>> platforms since the PEI was the old UDK version and the DXE from the Capsule was the new UDK version. Basically the version of the PCD
>> changed. It looks like there are some checks, but the seem to be more about making sure the build systems matches the code vs. dealing
>> with an update case like we hit.
>> 
>> It does not look like the PCD database is designed to deal with this. We are fixing this by extracting the PEI PCD database from the
>> capsule and then having some platform specific code to patch any PCD entries that got set in PEI that are needed by DXE.
>> 
>> In the future it would be helpful if the PCD database would change in some what of a backward compatible way, and have the headers
>> needed to parse the old and new version.
>> 
>> It also seems like adding a Dynamic PCD could potentially change the token layout and thus break compatibility? Is there any scheme to
>> keep a previous token layout?
>> 
>> Thanks,
>> 
>> Andrew Fish
>> 
> 
> 
> 
> 

Re: [edk2-devel] PCD Migration issue

[Liming Gao]

Andrew:
  For the work model with old PEI and new DXE, besides Dynamic PCD, there may be Hob or Variable issue. This is a complex topic to make sure the mismatch PEI and DXE work together.

Thanks
Liming
> -----Original Message-----
> From: afish@apple.com <afish@apple.com>
> Sent: Wednesday, February 12, 2020 2:22 PM
> To: devel@edk2.groups.io; Gao, Liming <liming.gao@intel.com>
> Subject: Re: [edk2-devel] PCD Migration issue
> 
> Liming,
> 
> I agree I don't think this has been thought about.
> 
> Does the edk2 still support booting a split PEI from ROM and DXE from capsule for update? If yes that seems to imply the assumption is
> that platform would not change any Dynamic PCD layout, or update the PCD database versions. If you have a platform in a common core
> base, and update the EDK2 code to pick up security fixes it seems the PCD data base implies you have to freeze Dynamic PCD usage in
> your common core?
> 
> Thanks,
> 
> Andrew Fish
> 
> > On Feb 11, 2020, at 7:54 PM, Liming Gao <liming.gao@intel.com> wrote:
> >
> > Andrew:
> >  Current implementation requires PEI PcdDB and DXE PcdDB match. This is like a new request for PCD DataBase. It requests PCD DB
> binary format and Dynamic PCD token assignment be compatible in the different version.
> >
> > Thanks
> > Liming
> >> -----Original Message-----
> >> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Andrew Fish via Groups.Io
> >> Sent: Tuesday, February 11, 2020 1:39 AM
> >> To: devel@edk2.groups.io
> >> Subject: [edk2-devel] PCD Migration issue
> >>
> >> We recently hit an issue when updating our UDK version in our common code base. Our recovery update path started failing for older
> >> platforms since the PEI was the old UDK version and the DXE from the Capsule was the new UDK version. Basically the version of the
> PCD
> >> changed. It looks like there are some checks, but the seem to be more about making sure the build systems matches the code vs.
> dealing
> >> with an update case like we hit.
> >>
> >> It does not look like the PCD database is designed to deal with this. We are fixing this by extracting the PEI PCD database from the
> >> capsule and then having some platform specific code to patch any PCD entries that got set in PEI that are needed by DXE.
> >>
> >> In the future it would be helpful if the PCD database would change in some what of a backward compatible way, and have the
> headers
> >> needed to parse the old and new version.
> >>
> >> It also seems like adding a Dynamic PCD could potentially change the token layout and thus break compatibility? Is there any scheme
> to
> >> keep a previous token layout?
> >>
> >> Thanks,
> >>
> >> Andrew Fish
> >>
> >
> >
> > 
> >