From mboxrd@z Thu Jan 1 00:00:00 1970 Subject: [edk2-devel] RFC for edk2-tools-library v2 previously known as RFC for Edk2-Library To: Michael D Kinney ,devel@edk2.groups.io From: "Sean" X-Originating-IP: 167.220.2.83 User-Agent: GROUPS.IO Web Poster MIME-Version: 1.0 Date: Wed, 29 May 2019 19:15:21 -0700 References: In-Reply-To: Message-ID: <11919.1559182521567877556@groups.io> Content-Type: multipart/alternative; boundary="ovs37IaRycxT7Fs3bIWj" --ovs37IaRycxT7Fs3bIWj Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable RFC=C2=A0 edk2-tools-library creation Create a new tianocore owned repository to host a python library package i= n support of UEFI development.=C2=A0 This package will allow easy sharing o= f python library code to facilitate reuse. =C2=A0Inclusion of this package = and dependency management should be managed using Pip/Pypi.=C2=A0 =C2=A0Thi= s is a supplemental package and is not required to be used for edk2 builds.= To avoid conflicting dependencies on your host platform it is strongly sug= gested to leverage python virtual environments. Examples of content here * Edk2 file type parsing * UEFI structure encode/decode in python * Packaging tools (Capsules generation, signing, INF gen, Cat gen) * TPM support code * No command line tools/interface Maintainers * Sean Brogan * Bret Barkelew * Placeholder for existing maintainer from the basetools License * BSD + Patent (edk2 aligned) Contribution process and issue tracking * Follow Github PR process for contributions and issue tracking * Contributor forks repo in github * Contributor creates branch for work * Contributor updates release notes to indicate change (if necessary) * Contributor submits PR to master branch of tianocore/edk2-tools-library = repo * Review feedback is given in PR * Python Tests are run on the repo (new contributions need unit tests) * Python Style (flake8) must pass * All review feedback must be completed, maintainers approved, and tests r= un successfully before PR is squash merged into master. Since squash merge = is used there is no support for patchsets. The PR should be sized appropria= tely. Documentation * Use Github IO documentation/wiki hosting * Example content * https://microsoft.github.io/mu/dyn/mu_pip_python_library/developing/ ( h= ttps://nam06.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fmicrosof= t.github.io%2Fmu%2Fdyn%2Fmu_pip_python_library%2Fdeveloping%2F&data=3D01%7C= 01%7Csean.brogan%40microsoft.com%7C47c4ca03e19b4fffc8ad08d6d314774a%7C72f98= 8bf86f141af91ab2d7cd011db47%7C1&sdata=3DfKIb1Pfj4AqoGVOWudcyFMxMypJk%2FHTts= 9aMxZ8HukI%3D&reserved=3D0 ) * https://microsoft.github.io/mu/dyn/mu_pip_python_library/publishing/ ( h= ttps://nam06.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fmicrosof= t.github.io%2Fmu%2Fdyn%2Fmu_pip_python_library%2Fpublishing%2F&data=3D01%7C= 01%7Csean.brogan%40microsoft.com%7C47c4ca03e19b4fffc8ad08d6d314774a%7C72f98= 8bf86f141af91ab2d7cd011db47%7C1&sdata=3DQJMUMB1hIusaRVJhgsi9kF9KIbgdhS0WRnI= XVkGeBCM%3D&reserved=3D0 ) * Readme at root of repo * Example: https://github.com/Microsoft/mu_pip_python_library ( https://na= m06.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fgithub.com%2FMicr= osoft%2Fmu_pip_python_library&data=3D01%7C01%7Csean.brogan%40microsoft.com%= 7C47c4ca03e19b4fffc8ad08d6d314774a%7C72f988bf86f141af91ab2d7cd011db47%7C1&s= data=3DDiuPnNMagvYYf0XXxDycSVHqijBBcDT0fXHzVY9U6%2Fw%3D&reserved=3D0 ) * API documentation * Figure out autogenerated pydocs CI Builds * CI build process using dev ops (new TianoCore organization created in Az= ure devops) * Validation is done thru build process * Release publication done thru manual CI Build * Examples from Mu-Python-Library * Windows CI - https://dev.azure.com/projectmu/mu%20pip/_build?definitionI= d=3D13 ( https://nam06.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%= 2Fdev.azure.com%2Fprojectmu%2Fmu%2520pip%2F_build%3FdefinitionId%3D13&data= =3D01%7C01%7Csean.brogan%40microsoft.com%7C47c4ca03e19b4fffc8ad08d6d314774= a%7C72f988bf86f141af91ab2d7cd011db47%7C1&sdata=3DjQCCZo2u8JVisCchOoMLSgKJbG= 3YEk%2FG1JP9fI4g2JY%3D&reserved=3D0 ) * Linux CI - https://dev.azure.com/projectmu/mu%20pip/_build?definitionId= =3D12 ( https://nam06.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%= 2Fdev.azure.com%2Fprojectmu%2Fmu%2520pip%2F_build%3FdefinitionId%3D12&data= =3D01%7C01%7Csean.brogan%40microsoft.com%7C47c4ca03e19b4fffc8ad08d6d314774= a%7C72f988bf86f141af91ab2d7cd011db47%7C1&sdata=3DCILlTdeEGpsi%2BCQNiZdIqd2V= t2RQV6L3qjz2rWEARYE%3D&reserved=3D0 ) * Publishing - https://dev.azure.com/projectmu/mu%20pip/_build?definitionI= d=3D16 ( https://nam06.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%= 2Fdev.azure.com%2Fprojectmu%2Fmu%2520pip%2F_build%3FdefinitionId%3D16&data= =3D01%7C01%7Csean.brogan%40microsoft.com%7C47c4ca03e19b4fffc8ad08d6d314774= a%7C72f988bf86f141af91ab2d7cd011db47%7C1&sdata=3Dhkpzn9P6RBjduYBWjN56Y2qRBW= yOG32mFkP%2BwtY7KBQ%3D&reserved=3D0 ) Release * Release to Pypi as edk2-tools-library for easy usage in product environm= ent * Versioned follows: Aa.bb.cc and is based on tags in git * AA =3D=3D Major version.=C2=A0 Changes don=E2=80=99t need to be backward= compatible * BB =3D=3D Minor version.=C2=A0 Significant new features.=C2=A0 Backward = compatibility maintained * CC =3D=3D Bug fix/patch/small optional feature * Package on Pypi will be owned by Tianocore group * Example for mu-python-library: https://pypi.org/project/mu-python-librar= y/ ( https://nam06.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fpy= pi.org%2Fproject%2Fmu-python-library%2F&data=3D01%7C01%7Csean.brogan%40micr= osoft.com%7C47c4ca03e19b4fffc8ad08d6d314774a%7C72f988bf86f141af91ab2d7cd011= db47%7C1&sdata=3D5nTb93dDnAo%2FsWyfv1zYEIkJF8L58YY5P3BkzKi4Ivc%3D&reserved= =3D0 ) Other Notes * Only support Python 3 (prefer 3.7+) * This was discussed on the edk2 design meetings (4/18) https://edk2.group= s.io/g/devel/files/Designs/2019/0418 ( https://edk2.groups.io/g/devel/files= /Designs/2019/0418 ) * There will be one more RFC for another python package/repo for edk2-tool= s-extensions * Will start version 00.09.00 while getting the repository setup and funct= ioning with 1.0.0 being the first real release --ovs37IaRycxT7Fs3bIWj Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: quoted-printable

RFC  edk2-tools-library creation

  To avoid conflicting depe= ndencies on your host platform it is strongly suggested to leverage python = virtual environments. 

  • Edk2 file type parsing
  • UEFI structure encode/decode in python
  • Packaging tools (Capsules generation, signing, INF g= en, Cat gen)
  • TPM support code
  • No command line tools/interface

  • Sean Brogan
  • Bret Barkelew
  • Placeholder for existing maintainer from the basetoo= ls

  • BSD + Patent (edk2 aligned)

  • Follow Github PR process for contributions and issue tracking
    1. Contributor forks repo in github
    2. Contributor creates branch for work
    3. Contributor updates release notes to indicate change= (if necessary)
    4. Contributor submits PR to master branch of tianocore= /edk2-tools-library repo
    5. Review feedback is given in PR
    6. Python Tests are run on the repo (new contributions = need unit tests)
    7. Python Style (flake8) must pass
    8. All review feedback must be completed, maintainers a= pproved, and tests run successfully before PR is squash merged=  into master. Since squash merge is used there is no support for patch= sets.  The PR should be size= d appropriately.

  • Use Github IO documentation/wiki hosting
    1. Example content
      1. https://microsoft.github.io/mu/dyn/mu_pip_python_library/develo= ping/
      2. = https://microsoft.github.io/mu/dyn/mu_pip_python_library/publishing/=
  • Readme at root of repo
    1. Example: https://github.com/Microsoft/mu_pip_python_library
  • API documentation
    1. Figure o= ut autogenerated pydocs

     

  • CI build process using dev ops (new TianoCore organization created in Azur= e devops)
  • Validation is done thru build process
  • Release publication done thru manual CI Build=
  • Examples from Mu-Python-Library
    1. Windows CI - https://dev.azure.com/projectmu/mu%20pip/_build?d= efinitionId=3D13
    2. Linux CI - https://dev.azure.com/project= mu/mu%20pip/_build?definitionId=3D12
    3. Publishing - https://dev.azure.com/proje= ctmu/mu%20pip/_build?definitionId=3D16

  • Release to Pypi as edk2-tools-library for easy usage in product environmen= t
    1. Versioned follows: Aa.bb.cc and is based on tags in git
    2. AA =3D=3D Major version.  Changes don’t n= eed to be backward compatible
    3. BB =3D=3D Minor version.  Significant new featu= res.  Backward compatibility maintained
    4. CC =3D=3D Bug fix/patch/small optional feature
    5. Package on Pypi will be owned by Tianocore group
    1. Example for mu-python-library: = https://pypi.org/project/mu-python-= library/

  • Only support Python 3 (prefer 3.7+)
  • This was discussed on the edk2 desig= n meetings (4/18) https://edk2.groups.io/g/devel/files/Design= s/2019/0418
  • There will be one more RFC for another python packag= e/repo for edk2-tools-extensions
  • Will start version 00.09.00 while getting the reposi= tory setup and functioning with 1.0.0 being the first real release --ovs37IaRycxT7Fs3bIWj--