Re: [edk2-devel] [edk2-platforms] [RFC] Compatibility Expectations in edk2-platforms

["Michael Kubacki" <michael.kubacki@outlook.com>]

Hi Nate,

On 10/6/2020 9:19 PM, Desimone, Nathaniel L wrote:
> Hey Michael,
> 
> On 10/5/20, 3:36 PM, Michael Kubacki <michael.kubacki@outlook.com> wrote:
>>
>> 1. Inconsistent maintainer support
>>      * Some packages currently do not build. Some packages are not getting
>> updated often.
>>
>>      * Example: Last week I had to update Vlv2TbltDevicePkg which did not
>> build.
>>      * Example: Many packages only document support for old toolchains.
> 
> 100% agreed here.
> 
>>
>> 2. Inconsistent toolchain support
>>
>> To build these according to instruction, a developer needs to install Visual
>> Studio dating back to 2015 (though it is 2020), and multiple versions of iASL,
>> NASM, a separate host OS for Linux/Windows builds, etc.
> 
> IMHO, the best way forward would be to finish the EDK II native clang port that Liming started some time ago. This would need to include robust support for Windows, Linux, and Mac host systems. From what I remember the last status update was issues with stability of the LLVM linker's support for PE/COFF output.

Aligning on Clang support would be great. It's worth noting that current 
CI support in edk2 does not support Clang so that needs to be enabled - 
https://github.com/tianocore/edk2/blob/master/.pytool/Readme.md.

Is anyone looking at those LLVM issues now?

> 
> Otherwise build reproducibility will always be a problem since the binary generated will be different depending on which OS/compiler the developer was using at the time. The coreboot community handles this issue by only supporting GCC, which I think is inappropriate for TianoCore since Windows ports of GCC differ greatly from Linux versions. While I think the goal of supporting many different C toolchains is admirable and definitely appropriate for edk2 core code... for edk2-platforms I think it would be better to have everyone agree on a single cross-platform toolchain; at least for x86 and ARM since clang supports those architectures well (maybe other architectures depending on the clang's maturity.)
> 
>>
>> 3. Inconsistent build requirements
>>
>> Many builds use the "build" command. Others have script wrappers with
>> unique parameters. Platforms are free to choose what they do and do not
>> support and developers have to figure it out.
> 
> We solved that for MinPlatform based boards with build_bios.py, but agreed that that at an overall project level this is still an issue. I believe Bob and Liming were working on adding extensibility to BaseTools needed for "build" to work everywhere, but I'm not sure what the status of that work is.

I've always liked the simplicity of build_bios.py. There's also a lot of 
effort put into edk2-pytool-extensions. I'm not partial to any 
particular solution as I've had positive experiences with both but 
having more consistency at a repo level would be awesome.

Perhaps a community discussion around leveraging existing tool support 
for open source platforms would help with adoption. Apart from 
consistency across open source platforms, usage could also serve as a 
practical example to closed source consumers on how to better integrate 
such tools into their environments.

> 
>>
>> 4. Lack of build health indicators
>>
>> Basically, there is no public CI across platforms. It is not clear exactly what
>> platform builds are broken, what configurations they are broken against,
>> how long they have been broken, etc.
> 
> Public CI seems like a great idea. Public automated testing would also be awesome. I believe 9elements has been working on building a pool of hardware for automated testing of Open System Firmware, maybe we should check and see if they would be interested in supporting automated testing for TianoCore. We would also need to see what the intersection is between what they have in their pool and what boards are supported in edk2-platforms.

I look forward to hearing from 9elements.

Do you think it would be feasible for Intel to support something like 
KabylakeOpenBoardPkg/GalagoPro3 and/or WhiskeylakeOpenBoardPkg/UpXtreme 
in public CI?

> 
>> Without such support, I believe platforms can only have a dependency on
>> edk2 (not vice versa). Maintainers move their edk2 pointer when they have
>> verified that their platform properly integrates the latest changes. This is
>> relatively common in relationships with package-based dependencies and
>> how this is typically handled outside edk2-platforms. I believe this is
>> reasonable even with public CI in place unless maintainers understand and
>> accept the challenges and additional support that is involved with being on
>> edk2/master.
> 
> Coreboot handles this problem by auditing the how good the maintenance is of boards over time. If a given board becomes stale, then the board becomes a candidate for getting deleted from master and moved to a legacy branch. Sometimes entire technologies are dropped, for example FSP v1.0 is only supported up to coreboot 4.11, for that reason there is a long-lived 4.11_branch in coreboot git to maintain platforms dependent on FSP v1.0 binaries.
> 
> I think we need some sort of deprecation process for edk2-platforms as well because as you note, the Bay Trail Minnow Max is not receiving particularly good maintenance at this point. A similar issue happened with PurleyOpenBoardPkg last year and you actually sent the patch series to delete it.

Deprecation branches sound reasonable to me personally. I'm not aware of 
prior documentation or discussion around platform deprecation in 
edk2-platforms. Is anyone else?

> 
>>
>> I just wanted to give my observation of some recent challenges and see if
>> the community can align on some practices to help simplify edk2-platforms
>> integration and testing.
>>
>> Thanks,
>> Michael
>>
>>
>> 
>>
>