FW: TianoCore Community Design Meeting Minutes - Jan 10, 2020

FW: TianoCore Community Design Meeting Minutes - Jan 10, 2020

[Ni, Ray]

Discussion of the Pytools Build components continues in devel mailing list.

-----Original Message-----
From: Jeremiah Cox <jerecox@microsoft.com> 
Sent: Saturday, January 11, 2020 12:09 PM
To: Ni, Ray <ray.ni@intel.com>; announce@edk2.groups.io
Cc: Sean Brogan <sean.brogan@microsoft.com>; Matthew Carlson <macarl@microsoft.com>
Subject: RE: TianoCore Community Design Meeting Minutes - Jan 10, 2020

Thank you Ray and TianoCore attendees for taking the time to review and discuss Pytools Build components and examples.

It was requested that we summarize the value proposition of PyTools build compared with the classic EDK2 build systems.  Here are some thoughts:

PyTools extend the EDK2 build system, building on top of it with a goal of enabling larger-scale development and maintenance of families of platforms across architectures & generations.   

Goals:
* Modular build plug-ins to share common build operations across applicable platforms
* Custom, modular pre-build and post-build operations
* Workspaces with multiple, independently-owned Git repositories (via Git submodules)
* Download of external binary dependencies from NuGet and the web
* Minimized environment assumptions, for example path & workspace layout, delegating that to a platform configuration file and the modules that it includes
* No hardcoded paths to plug-ins and dependencies (for workspace agility as platforms appear and evolve)
* Independent of build platform, shell (cmd, bash, psh, ...), & shell environment, easing build reproducibility and CI
* Consistent logging throughout the extensions

Most EDK2-based platforms, including OVMF and EmulatorPkg, include scripts, bash or cmd, and/or README's that provide pre and post build steps, beyond EDK2 build, that are  needed to complete building and deployment of the platform.  PyTools build strive to include these abilities in a  platform's one (1 singular) UefiBuilder Python module (which can be leveraged for build/deployment across build platforms, environments, and devices).  If your day-to-day job is modifications to core EDK2 and BaseTools, PyTools build components may have less to offer you, but we believe Pytools CI and libraries have a lot to offer (topics for further discussions).

Thanks for the opportunity to present, and we look forward to more discussions on these topics!

Jeremiah Cox
Core UEFI and Project Mu
Microsoft

-----Original Message-----
From: announce@edk2.groups.io <announce@edk2.groups.io> On Behalf Of Ni, Ray via Groups.Io
Sent: Friday, January 10, 2020 00:55
To: announce@edk2.groups.io
Cc: announce@edk2.groups.io
Subject: [EXTERNAL] [edk2-announce] TianoCore Community Design Meeting Minutes - Jan 10, 2020

1. EDK2 Pytools "Stuart": Platform Creation (Microsoft)
Presenter: Jeremiah Cox
Slides: https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Ffiles%2FDesigns%2F2020%2F0110%2FEnablingPytools-Edk2PlatformsAndOvmf.pdf&amp;data=02%7C01%7Cjerecox%40microsoft.com%7Cf9ba7574e02c458eecac08d795aada32%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637142434090340667&amp;sdata=KpJBnuhgPPM9og4TBPtlyn4rvlevNbnATo7ofxgA6LM%3D&amp;reserved=0

What's "Stuart": page #2, #3
  EDK2 Pytools = EDK2 Pytool Library + EDK2 Pytool Extensions (including "Stuart")
  "Stuart" facilitates building EDK2-based platforms and CI testing of EDK2 packages.
  This topic only covers how "Stuart" builds EDK2-based platforms.

Stuart's CLI tools: page #4
  stuart_setup: update source code
  stuart_update: update external tool dependencies
  stuart_build: perform EDK2 build

@Liming: What are the differences between stuart_setup and stuart_update?
@Sean: stuart_setup is using git focusing on source code. stuart_update is focusing on external dependencies like iasl.
@Liming: What is the order between the two?
@Sean: stuart_setup, then stuart_update.
@Sean: Stuart works on 1-repo-N-submodule usage while EdkRepo tool works on N-repo usage.

Enabling Stuart Build to a Platform: page #5 (abstract), page #10 ~ #13 (details)
  Subclass Stuart classes in platform specific python file
    SetupSettingsManager: declare submodules for stuart_setup
    UpdateSettingsManager: declare external dep scopes for stuart_update
    BuildSettingsManager & UefiBuilder: consumed by stuart_build
    refer to https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-pytool-extensions%2Fblob%2Fmaster%2Fdocs%2Fusability%2Fusing_settings_manager.md&amp;data=02%7C01%7Cjerecox%40microsoft.com%7Cf9ba7574e02c458eecac08d795aada32%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637142434090340667&amp;sdata=i9LNkBwwjkVmyECUQ5JtZjMmYXlNM0dTClb5Fu7csf4%3D&amp;reserved=0

Building OVMF with EDK2 Pytools: page #6 ~ #9
  refer to https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fout0xb2%2Fedk2%2Fblob%2Ffeature%2FpytoolsForOvmf%2FOvmfPkg%2FREADME-pytools.md&amp;data=02%7C01%7Cjerecox%40microsoft.com%7Cf9ba7574e02c458eecac08d795aada32%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637142434090340667&amp;sdata=3zgXYz5aHeMfTeEisV4wJuz0qKV%2BnzZgeG%2B5l66Ulc4%3D&amp;reserved=0
  EDK2 build defines are passed through parameters "BLD_*_<define>" when calling stuart_build. e.g.:
    stuart_build -c OvmfPkg/PlatformBuild.py BLD_*_E1000_ENABLE=1
  @Ray: Why is "BLD_*_<macro> but not "BLD_<macro>"?
  @Sean: "*" can be the name of different build targets like "DEBUG", "RELEASE".
  @Liming: EDK2 build defines are global and cannot be associated with individual build targets.
  @Mike: Need a clear understanding of namespace defined by Stuart.

Building Kabylake Open Board with EDK2 Pytools: page #14 ~ #17
  refer to https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fout0xb2%2Fedk2-platforms%2Fblob%2Ffeature%2Fpy_platforms%2FPlatform%2FIntel%2FKabylakeOpenBoardPkg%2FKabylakeRvp3%2FREADME-Pytools.md&amp;data=02%7C01%7Cjerecox%40microsoft.com%7Cf9ba7574e02c458eecac08d795aada32%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637142434090340667&amp;sdata=oZekZ1DV4NGhQN2El3KXzpFLRukAzdKZe2zccwuLqyY%3D&amp;reserved=0

Answer how Pytools helps on Surface development: page #19 ~ #21

@Mike: Advantages and disadvantages of today's bulid style and MS' stuart build style.
@Jeremiah: Will summarize and reply to the meeting minutes.

Thanks,
Ray