public inbox for devel@edk2.groups.io
 help / color / mirror / Atom feed
* RFC for Edk2-ToolEnv
@ 2019-05-14  2:55 Sean
  2019-05-14  4:01 ` [edk2-devel] " rebecca
  0 siblings, 1 reply; 13+ messages in thread
From: Sean @ 2019-05-14  2:55 UTC (permalink / raw)
  To: devel

[-- Attachment #1: Type: text/plain, Size: 6527 bytes --]

RFC  Edk2-ToolEnv creation

Create a new tianocore owned repository to host python code to support an extensible, pluggable, rich environment.  This environment has command line interfaces to support building a product, building CI, running tests, and downloading dependencies. This environment also provides the building blocks for developers to write their own tools to launch in the environment and leverage the capabilities provided by the environment. The unique capabilities provided help support building products with multiple repositories and having each repository contribute/plugin to the build process in a scalable way. The environment will scan the files in the code tree (multiple repos) and discover plugins, dependencies, path adjustments, environment variable settings, etc. This provides easy methods for common repos to share build tools/steps.

Inclusion of this package and dependency management should be managed using Pip/Pypi.   To start this is a supplemental package and is not required to be used for edk2 builds.

More details below but much of this content is coming from combining two existing repos

* https://github.com/microsoft/mu_pip_environment ( https://github.com/microsoft/mu_pip_environment )
* https://github.com/microsoft/mu_pip_build ( https://github.com/microsoft/mu_pip_build )

This RFC is part 2 of content from design review: https://edk2.groups.io/g/devel/files/Designs/2019/0418/2019-04-18%20Microsoft%20-%20Build%20Tools%20-%20Design%20Review%20.pdf ( https://edk2.groups.io/g/devel/files/Designs/2019/0418/2019-04-18%20Microsoft%20-%20Build%20Tools%20-%20Design%20Review%20.pdf )

Examples of content here

* CI build support - https://github.com/microsoft/mu_pip_build/blob/master/MuBuild/MuBuild.py ( https://github.com/microsoft/mu_pip_build/blob/master/MuBuild/MuBuild.py )
* Binary Dependency resolution
* Nuget - https://github.com/microsoft/mu_pip_environment/blob/master/MuEnvironment/ExternalDependencies.py ( https://github.com/microsoft/mu_pip_environment/blob/master/MuEnvironment/ExternalDependencies.py )
* GitHub Releases - WIP
* URL dependency - WIP
* Logging
* File logger
* Markdown logger
* In Memory loggers - For tool parsing
* Console loggers with colors
* PlugIns
* Support pre/post build steps
* Support supplying functions
* UefiBuild - A wrapper around Edk2 build process that a Platform would subclass. https://github.com/microsoft/mu_pip_environment/blob/master/MuEnvironment/UefiBuild.py ( https://github.com/microsoft/mu_pip_environment/blob/master/MuEnvironment/UefiBuild.py )
* VarDict and ShellEnvironment - Manage the shell environment and all the name/value pairs used in the build process (including pre/post).
* Omnicache - Support a super cache of git repos to speed up creating and updating multiple work spaces and minimizing filesystem impact

Maintainers

* Sean Brogan
* Bret Barkelew
* Placeholder for existing maintainer from the basetools

License

* BSD + Patent (edk2 aligned)

Contribution process and issue tracking

* Follow Github PR process for contributions and issue tracking
* Contributor forks repo in github
* Contributor creates branch for work
* Contributor updates release notes to indicate change (if necessary)
* Contributor submits PR to master branch of tianocore/Edk2-ToolEnv repo
* Review feedback is given in PR
* Python Tests are run on the repo (new contributions need unit tests)
* Python Style (flake8) must pass
* All review feedback must be completed, maintainers approved, and tests run successfully before PR is *squash merged* into master

Documentation

* Use Github IO documentation/wiki hosting
* Example content

i. https://microsoft.github.io/mu/dyn/mu_pip_environment/developing/ ( https://microsoft.github.io/mu/dyn/mu_pip_environment/developing/ )

ii. https://microsoft.github.io/mu/dyn/mu_pip_environment/publishing/ ( https://microsoft.github.io/mu/dyn/mu_pip_environment/publishing/ )

* Readme at root of repo
* Example: https://github.com/Microsoft/mu_pip_environment ( https://github.com/Microsoft/mu_pip_environment )

CI Builds

* CI build process using dev ops
* Validation is done thru build process
* Release publication done thru manual CI Build
* Examples from Mu-Environment
* Windows CI - https://dev.azure.com/projectmu/mu%20pip/_build?definitionId=10 ( https://dev.azure.com/projectmu/mu%20pip/_build?definitionId=10 )
* Linux CI - https://dev.azure.com/projectmu/mu%20pip/_build?definitionId=11 ( https://dev.azure.com/projectmu/mu%20pip/_build?definitionId=11 )
* Publishing - https://dev.azure.com/projectmu/mu%20pip/_build?definitionId=17 ( https://dev.azure.com/projectmu/mu%20pip/_build?definitionId=17 )

Release

* Release to Pypi as Edk2-ToolEnv for easy usage in product environment
* Versioned follows: Aa.bb.cc.dd
* AA == Major version.  Changes don’t need to be backward compatible
* BB == Minor version.  Significant new features.  Backward compatibility maintained
* CC == Bug fix/patch/small optional feature
* DD == build/Release version.
* Package on Pypi will be owned by Tianocore group
* Example for mu-environment: https://pypi.org/project/mu-environment/ ( https://pypi.org/project/mu-environment/ )

Other Notes

* Only support Python 3 (prefer 3.7+)
* This was discussed on the edk2 design meetings (4/18) https://edk2.groups.io/g/devel/files/Designs/2019/0418 ( https://edk2.groups.io/g/devel/files/Designs/2019/0418 )
* There is RFC #2. The 1 st RFC is for Edk2-Library which this RFC depends on.
* Simple example of usage on an open platform: https://github.com/ms-iot/MU_PLATFORM_NXP/blob/master/NXP/MCIMX8M_MINI_EVK_2GB/PlatformBuildWorker.py ( https://github.com/ms-iot/MU_PLATFORM_NXP/blob/master/NXP/MCIMX8M_MINI_EVK_2GB/PlatformBuildWorker.py )
* Example of CI of something like Tianocore:
* Windows - Mu_Basecore: https://dev.azure.com/projectmu/mu/_build?definitionId=4&view=buildsHistory ( https://dev.azure.com/projectmu/mu/_build?definitionId=4&view=buildsHistory )
* Linux - Mu_Basecore: https://dev.azure.com/projectmu/mu/_build?definitionId=19&view=buildsHistory ( https://dev.azure.com/projectmu/mu/_build?definitionId=19&view=buildsHistory )
* Windows - Project Mu Plus code: https://dev.azure.com/projectmu/mu/_build?definitionId=6&view=buildsHistory ( https://dev.azure.com/projectmu/mu/_build?definitionId=6&view=buildsHistory )

Timeline

* RFC open for comment thru 5/21/2019 since the same as RFC Edk2-Library

[-- Attachment #2: Type: text/html, Size: 35689 bytes --]

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [edk2-devel] RFC for Edk2-ToolEnv
  2019-05-14  2:55 RFC for Edk2-ToolEnv Sean
@ 2019-05-14  4:01 ` rebecca
  2019-05-14  7:37   ` Sean
  0 siblings, 1 reply; 13+ messages in thread
From: rebecca @ 2019-05-14  4:01 UTC (permalink / raw)
  To: devel, sean.brogan

[-- Attachment #1: Type: text/plain, Size: 1104 bytes --]

On 2019-05-13 20:55, Sean via Groups.Io wrote:
>
> RFC  Edk2-ToolEnv creation
>
>  
>
> Create a new tianocore owned repository to host python code to support
> an extensible, pluggable, rich environment.  This environment has
> command line interfaces to support building a product, building CI,
> running tests, and downloading dependencies.  This environment also
> provides the building blocks for developers to write their own tools
> to launch in the environment and leverage the capabilities provided by
> the environment.  The unique capabilities provided help support
> building products with multiple repositories and having each
> repository contribute/plugin to the build process in a scalable way.
>  The environment will scan the files in the code tree (multiple repos)
> and discover plugins, dependencies, path adjustments, environment
> variable settings, etc.   This provides easy methods for common repos
> to share build tools/steps. 
>

Is there not an existing framework that could be used instead of
creating something new?


-- 
Rebecca Cran


[-- Attachment #2: Type: text/html, Size: 2651 bytes --]

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [edk2-devel] RFC for Edk2-ToolEnv
  2019-05-14  4:01 ` [edk2-devel] " rebecca
@ 2019-05-14  7:37   ` Sean
  2019-05-14 22:24     ` rebecca
  0 siblings, 1 reply; 13+ messages in thread
From: Sean @ 2019-05-14  7:37 UTC (permalink / raw)
  To: Rebecca Cran, devel

[-- Attachment #1: Type: text/plain, Size: 1040 bytes --]

Rebecca,

If you know of anything I am interested as I don't like building and supporting something unnecessary.

This tooling isn't trying to reinvent anything but is really focused on providing reusable/sharable functionality that can then be pieced together by a platform to produce the required output.  Today in edk2 you see shell script files (bash/bat) and a lot of redefinition of variables and assumptions.  This is made much worse in complex closed src code bases, complicated pre and post build requirements, and even then managing the path and locations to tools and scripts is a fragile mess.  In practice this environment has made our build process much more reliable, lower cost to maintain, and immune to necessary churn at all levels of the code tree.  It also has allowed our products to get significant code reuse so we lower the cost of ongoing maintenance and new feature introduction.

Looking forward to gathering more input from the community as we all don't need to build similar things.

Thanks
Sean

[-- Attachment #2: Type: text/html, Size: 1113 bytes --]

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [edk2-devel] RFC for Edk2-ToolEnv
  2019-05-14  7:37   ` Sean
@ 2019-05-14 22:24     ` rebecca
  2019-05-14 23:23       ` Sean
  0 siblings, 1 reply; 13+ messages in thread
From: rebecca @ 2019-05-14 22:24 UTC (permalink / raw)
  To: devel, sean.brogan

On 2019-05-14 01:37, Sean via Groups.Io wrote:
>
> If you know of anything I am interested as I don't like building and
> supporting something unnecessary.  
>
> This tooling isn't trying to reinvent anything but is really focused
> on providing reusable/sharable functionality that can then be pieced
> together by a platform to produce the required output.  Today in edk2
> you see shell script files (bash/bat) and a lot of redefinition of
> variables and assumptions.  This is made much worse in complex closed
> src code bases, complicated pre and post build requirements, and even
> then managing the path and locations to tools and scripts is a fragile
> mess.  In practice this environment has made our build process much
> more reliable, lower cost to maintain, and immune to necessary churn
> at all levels of the code tree.  It also has allowed our products to
> get significant code reuse so we lower the cost of ongoing maintenance
> and new feature introduction.  
>
> Looking forward to gathering more input from the community as we all
> don't need to build similar things.


It sounds like a lot of the functionality _should_ already exist: for
example I know it's not cross-platform, but MSBuild has various logging
features, while I know Jenkins (unfortunately I've not worked with Azure
yet) has lots of plugins for CI and running tests: at my work we have a
relatively small python script that submits a "smoke" job to Jenkins and
collects the results for pre-commit testing, while the same instance
also checks for new changesets in the main repo and runs CI builds
against them.


-- 
Rebecca Cran


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [edk2-devel] RFC for Edk2-ToolEnv
  2019-05-14 22:24     ` rebecca
@ 2019-05-14 23:23       ` Sean
  2019-05-14 23:34         ` rebecca
  0 siblings, 1 reply; 13+ messages in thread
From: Sean @ 2019-05-14 23:23 UTC (permalink / raw)
  To: Rebecca Cran, devel

[-- Attachment #1: Type: text/plain, Size: 395 bytes --]

Take a look at the proposed content and how it is used.  We even have examples of calling from DevOps and i don't think Jenkins would be any different.  I don't think we are trying to duplicate CI functionality.  We are providing the "last mile" so that those CI engines can run EDK specific tests and tools.  Standard CI engines have no concept of packages, DSC, FDF, INFs, firmware, etc.

[-- Attachment #2: Type: text/html, Size: 418 bytes --]

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [edk2-devel] RFC for Edk2-ToolEnv
  2019-05-14 23:23       ` Sean
@ 2019-05-14 23:34         ` rebecca
  2019-05-23  2:39           ` Michael D Kinney
  0 siblings, 1 reply; 13+ messages in thread
From: rebecca @ 2019-05-14 23:34 UTC (permalink / raw)
  To: Sean, devel

On 2019-05-14 17:23, sean.brogan via groups.io wrote:
> Take a look at the proposed content and how it is used.  We even have
> examples of calling from DevOps and i don't think Jenkins would be any
> different.  I don't think we are trying to duplicate CI
> functionality.  We are providing the "last mile" so that those CI
> engines can run EDK specific tests and tools.  Standard CI engines
> have no concept of packages, DSC, FDF, INFs, firmware, etc.   


Okay, that's great. Of course we do also have lots of code running on
the CI server at work, not the client, that does things like packaging
etc., and this proposal will include server-side code too.

Also, I don't think there is anything that'll be as nicely integrated as
this, so I'm happy with it.


-- 
Rebecca Cran


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [edk2-devel] RFC for Edk2-ToolEnv
  2019-05-14 23:34         ` rebecca
@ 2019-05-23  2:39           ` Michael D Kinney
  2019-05-23  6:46             ` Sean
  0 siblings, 1 reply; 13+ messages in thread
From: Michael D Kinney @ 2019-05-23  2:39 UTC (permalink / raw)
  To: devel@edk2.groups.io, rebecca@bluestop.org, Sean

Hi Sean,

Does the PIP module here support both local platform builds and 
CI builds?

I am looking at the name of the repo and trying to align with
the edk2-tools-library repo name so it is obvious the two repos
are related.  Maybe focus on the CI part for the name and we 
reuse the CI features to simplify local builds.

	edk2-tools-ci

Finalizing the name is the only open I am aware of.

Thanks,

Mike

> -----Original Message-----
> From: devel@edk2.groups.io [mailto:devel@edk2.groups.io]
> On Behalf Of rebecca@bluestop.org
> Sent: Tuesday, May 14, 2019 4:34 PM
> To: Sean <sean.brogan@microsoft.com>;
> devel@edk2.groups.io
> Subject: Re: [edk2-devel] RFC for Edk2-ToolEnv
> 
> On 2019-05-14 17:23, sean.brogan via groups.io wrote:
> > Take a look at the proposed content and how it is
> used.  We even have
> > examples of calling from DevOps and i don't think
> Jenkins would be any
> > different.  I don't think we are trying to duplicate CI
> > functionality.  We are providing the "last mile" so
> that those CI
> > engines can run EDK specific tests and tools.  Standard
> CI engines
> > have no concept of packages, DSC, FDF, INFs, firmware,
> etc.
> 
> 
> Okay, that's great. Of course we do also have lots of
> code running on
> the CI server at work, not the client, that does things
> like packaging
> etc., and this proposal will include server-side code
> too.
> 
> Also, I don't think there is anything that'll be as
> nicely integrated as
> this, so I'm happy with it.
> 
> 
> --
> Rebecca Cran
> 
> 
> 


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [edk2-devel] RFC for Edk2-ToolEnv
  2019-05-23  2:39           ` Michael D Kinney
@ 2019-05-23  6:46             ` Sean
  2019-05-23 17:14               ` Michael D Kinney
  0 siblings, 1 reply; 13+ messages in thread
From: Sean @ 2019-05-23  6:46 UTC (permalink / raw)
  To: Kinney, Michael D, devel@edk2.groups.io

Yes the plan would be to support both CI and local builds.  There is actually more features related to support platform builds so I think it would be better to keep ci out of the name.  The reason why Tool-Env was suggested is the modules can be used to run anything within the python environment not just builds.  We have a git submodule update tool, external dependency management tool (package mgmt/binary files), platform build tool, and CI build tool.  

Look at https://github.com/microsoft/mu_pip_environment and https://github.com/microsoft/mu_pip_build to get an idea of the content proposed.  

Thanks
Sean



-----Original Message-----
From: Kinney, Michael D <michael.d.kinney@intel.com> 
Sent: Wednesday, May 22, 2019 7:39 PM
To: devel@edk2.groups.io; rebecca@bluestop.org; Sean Brogan <sean.brogan@microsoft.com>
Subject: RE: [edk2-devel] RFC for Edk2-ToolEnv

Hi Sean,

Does the PIP module here support both local platform builds and CI builds?

I am looking at the name of the repo and trying to align with the edk2-tools-library repo name so it is obvious the two repos are related.  Maybe focus on the CI part for the name and we reuse the CI features to simplify local builds.

	edk2-tools-ci

Finalizing the name is the only open I am aware of.

Thanks,

Mike

> -----Original Message-----
> From: devel@edk2.groups.io [mailto:devel@edk2.groups.io] On Behalf Of 
> rebecca@bluestop.org
> Sent: Tuesday, May 14, 2019 4:34 PM
> To: Sean <sean.brogan@microsoft.com>;
> devel@edk2.groups.io
> Subject: Re: [edk2-devel] RFC for Edk2-ToolEnv
> 
> On 2019-05-14 17:23, sean.brogan via groups.io wrote:
> > Take a look at the proposed content and how it is
> used.  We even have
> > examples of calling from DevOps and i don't think
> Jenkins would be any
> > different.  I don't think we are trying to duplicate CI 
> > functionality.  We are providing the "last mile" so
> that those CI
> > engines can run EDK specific tests and tools.  Standard
> CI engines
> > have no concept of packages, DSC, FDF, INFs, firmware,
> etc.
> 
> 
> Okay, that's great. Of course we do also have lots of code running on 
> the CI server at work, not the client, that does things like packaging 
> etc., and this proposal will include server-side code too.
> 
> Also, I don't think there is anything that'll be as nicely integrated 
> as this, so I'm happy with it.
> 
> 
> --
> Rebecca Cran
> 
> 
> 


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [edk2-devel] RFC for Edk2-ToolEnv
  2019-05-23  6:46             ` Sean
@ 2019-05-23 17:14               ` Michael D Kinney
  2019-06-11 18:55                 ` Purma, Kondal R
  0 siblings, 1 reply; 13+ messages in thread
From: Michael D Kinney @ 2019-05-23 17:14 UTC (permalink / raw)
  To: devel@edk2.groups.io, sean.brogan@microsoft.com,
	Kinney, Michael D

Hi Sean,

Thanks for the clarification that this PIP module has more than
just build and CI.  And over time, it may add more features to
help developers maintain their code and platforms.  How about

  edk2-tool-extensions

And we perhaps remove plural from the library repo

  edk2-tool-library

Do you think that python needs to be in the repo name so it
is obvious these are python components.  Or is the top level
Readme.md sufficient to make this obvious.  Perhaps:

  edk2-pytool-extensions
  edk2-pytool-library

Mike


> -----Original Message-----
> From: devel@edk2.groups.io
> [mailto:devel@edk2.groups.io] On Behalf Of Sean via
> Groups.Io
> Sent: Wednesday, May 22, 2019 11:46 PM
> To: Kinney, Michael D <michael.d.kinney@intel.com>;
> devel@edk2.groups.io
> Subject: Re: [edk2-devel] RFC for Edk2-ToolEnv
> 
> Yes the plan would be to support both CI and local
> builds.  There is actually more features related to
> support platform builds so I think it would be better
> to keep ci out of the name.  The reason why Tool-Env
> was suggested is the modules can be used to run
> anything within the python environment not just builds.
> We have a git submodule update tool, external
> dependency management tool (package mgmt/binary files),
> platform build tool, and CI build tool.
> 
> Look at https://github.com/microsoft/mu_pip_environment
> and https://github.com/microsoft/mu_pip_build to get an
> idea of the content proposed.
> 
> Thanks
> Sean
> 
> 
> 
> -----Original Message-----
> From: Kinney, Michael D <michael.d.kinney@intel.com>
> Sent: Wednesday, May 22, 2019 7:39 PM
> To: devel@edk2.groups.io; rebecca@bluestop.org; Sean
> Brogan <sean.brogan@microsoft.com>
> Subject: RE: [edk2-devel] RFC for Edk2-ToolEnv
> 
> Hi Sean,
> 
> Does the PIP module here support both local platform
> builds and CI builds?
> 
> I am looking at the name of the repo and trying to
> align with the edk2-tools-library repo name so it is
> obvious the two repos are related.  Maybe focus on the
> CI part for the name and we reuse the CI features to
> simplify local builds.
> 
> 	edk2-tools-ci
> 
> Finalizing the name is the only open I am aware of.
> 
> Thanks,
> 
> Mike
> 
> > -----Original Message-----
> > From: devel@edk2.groups.io
> [mailto:devel@edk2.groups.io] On Behalf Of
> > rebecca@bluestop.org
> > Sent: Tuesday, May 14, 2019 4:34 PM
> > To: Sean <sean.brogan@microsoft.com>;
> > devel@edk2.groups.io
> > Subject: Re: [edk2-devel] RFC for Edk2-ToolEnv
> >
> > On 2019-05-14 17:23, sean.brogan via groups.io wrote:
> > > Take a look at the proposed content and how it is
> > used.  We even have
> > > examples of calling from DevOps and i don't think
> > Jenkins would be any
> > > different.  I don't think we are trying to
> duplicate CI
> > > functionality.  We are providing the "last mile" so
> > that those CI
> > > engines can run EDK specific tests and
> tools.  Standard
> > CI engines
> > > have no concept of packages, DSC, FDF, INFs,
> firmware,
> > etc.
> >
> >
> > Okay, that's great. Of course we do also have lots of
> code running on
> > the CI server at work, not the client, that does
> things like packaging
> > etc., and this proposal will include server-side code
> too.
> >
> > Also, I don't think there is anything that'll be as
> nicely integrated
> > as this, so I'm happy with it.
> >
> >
> > --
> > Rebecca Cran
> >
> >
> >
> 
> 
> 


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [edk2-devel] RFC for Edk2-ToolEnv
  2019-05-23 17:14               ` Michael D Kinney
@ 2019-06-11 18:55                 ` Purma, Kondal R
  2019-06-12  6:02                   ` Sean
  0 siblings, 1 reply; 13+ messages in thread
From: Purma, Kondal R @ 2019-06-11 18:55 UTC (permalink / raw)
  To: devel@edk2.groups.io, Kinney, Michael D,
	sean.brogan@microsoft.com

Hi Sean ,

Sorry I replied to wrong subject and it's about ToolEnv.

It's great that all python files must pass flake8 Python Style. I remember flake8 does not show errors for undefined member variables or instances . 

I feel this is one of most common use cases of code failures, due to typing errors and won't be visible unless  test that use case.

Are we planning to use any flake8 plug-ins to cover this or is it good idea to use Pylint (only to cover features not covered by falke8)on top of flake8.

Thanks,
Kondal.

-----Original Message-----
From: devel@edk2.groups.io [mailto:devel@edk2.groups.io] On Behalf Of Michael D Kinney
Sent: Thursday, May 23, 2019 10:14 AM
To: devel@edk2.groups.io; sean.brogan@microsoft.com; Kinney, Michael D <michael.d.kinney@intel.com>
Subject: Re: [edk2-devel] RFC for Edk2-ToolEnv

Hi Sean,

Thanks for the clarification that this PIP module has more than just build and CI.  And over time, it may add more features to help developers maintain their code and platforms.  How about

  edk2-tool-extensions

And we perhaps remove plural from the library repo

  edk2-tool-library

Do you think that python needs to be in the repo name so it is obvious these are python components.  Or is the top level Readme.md sufficient to make this obvious.  Perhaps:

  edk2-pytool-extensions
  edk2-pytool-library

Mike


> -----Original Message-----
> From: devel@edk2.groups.io
> [mailto:devel@edk2.groups.io] On Behalf Of Sean via Groups.Io
> Sent: Wednesday, May 22, 2019 11:46 PM
> To: Kinney, Michael D <michael.d.kinney@intel.com>; 
> devel@edk2.groups.io
> Subject: Re: [edk2-devel] RFC for Edk2-ToolEnv
> 
> Yes the plan would be to support both CI and local builds.  There is 
> actually more features related to support platform builds so I think 
> it would be better to keep ci out of the name.  The reason why 
> Tool-Env was suggested is the modules can be used to run anything 
> within the python environment not just builds.
> We have a git submodule update tool, external dependency management 
> tool (package mgmt/binary files), platform build tool, and CI build 
> tool.
> 
> Look at https://github.com/microsoft/mu_pip_environment
> and https://github.com/microsoft/mu_pip_build to get an idea of the 
> content proposed.
> 
> Thanks
> Sean
> 
> 
> 
> -----Original Message-----
> From: Kinney, Michael D <michael.d.kinney@intel.com>
> Sent: Wednesday, May 22, 2019 7:39 PM
> To: devel@edk2.groups.io; rebecca@bluestop.org; Sean Brogan 
> <sean.brogan@microsoft.com>
> Subject: RE: [edk2-devel] RFC for Edk2-ToolEnv
> 
> Hi Sean,
> 
> Does the PIP module here support both local platform builds and CI 
> builds?
> 
> I am looking at the name of the repo and trying to align with the 
> edk2-tools-library repo name so it is obvious the two repos are 
> related.  Maybe focus on the CI part for the name and we reuse the CI 
> features to simplify local builds.
> 
> 	edk2-tools-ci
> 
> Finalizing the name is the only open I am aware of.
> 
> Thanks,
> 
> Mike
> 
> > -----Original Message-----
> > From: devel@edk2.groups.io
> [mailto:devel@edk2.groups.io] On Behalf Of
> > rebecca@bluestop.org
> > Sent: Tuesday, May 14, 2019 4:34 PM
> > To: Sean <sean.brogan@microsoft.com>; devel@edk2.groups.io
> > Subject: Re: [edk2-devel] RFC for Edk2-ToolEnv
> >
> > On 2019-05-14 17:23, sean.brogan via groups.io wrote:
> > > Take a look at the proposed content and how it is
> > used.  We even have
> > > examples of calling from DevOps and i don't think
> > Jenkins would be any
> > > different.  I don't think we are trying to
> duplicate CI
> > > functionality.  We are providing the "last mile" so
> > that those CI
> > > engines can run EDK specific tests and
> tools.  Standard
> > CI engines
> > > have no concept of packages, DSC, FDF, INFs,
> firmware,
> > etc.
> >
> >
> > Okay, that's great. Of course we do also have lots of
> code running on
> > the CI server at work, not the client, that does
> things like packaging
> > etc., and this proposal will include server-side code
> too.
> >
> > Also, I don't think there is anything that'll be as
> nicely integrated
> > as this, so I'm happy with it.
> >
> >
> > --
> > Rebecca Cran
> >
> >
> >
> 
> 
> 





^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [edk2-devel] RFC for Edk2-ToolEnv
  2019-06-11 18:55                 ` Purma, Kondal R
@ 2019-06-12  6:02                   ` Sean
  2019-06-28 22:13                     ` Sean
  0 siblings, 1 reply; 13+ messages in thread
From: Sean @ 2019-06-12  6:02 UTC (permalink / raw)
  To: Purma, Kondal R, devel

[-- Attachment #1: Type: text/plain, Size: 770 bytes --]

Both of these RFCs listed flake8 because that is what we have today and it should be relatively easy to get parity.  I am super happy to have you add/propose/provide pr with additional test capability like Pylint.  Thats one of the goals of adding this to tianocore.

You can see what we are basing this on here:
https://dev.azure.com/projectmu/mu%20pip/_build?definitionId=10

And you can see how the build works in the Azure DevOps yaml file:
https://dev.azure.com/projectmu/mu%20pip/_apps/hub/ms.vss-build-web.ci-designer-hub?pipelineId=10&nonce=BIDKbQdfN9%2BmJevdri5mWw%3D%3D&branch=master ( https://dev.azure.com/projectmu/mu%20pip/_apps/hub/ms.vss-build-web.ci-designer-hub?pipelineId=10&nonce=BIDKbQdfN9%2BmJevdri5mWw%3D%3D&name=spbrogan&branch=master )

[-- Attachment #2: Type: text/html, Size: 952 bytes --]

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [edk2-devel] RFC for Edk2-ToolEnv
  2019-06-12  6:02                   ` Sean
@ 2019-06-28 22:13                     ` Sean
  2019-06-28 22:22                       ` Michael D Kinney
  0 siblings, 1 reply; 13+ messages in thread
From: Sean @ 2019-06-28 22:13 UTC (permalink / raw)
  To: Sean, devel

[-- Attachment #1: Type: text/plain, Size: 406 bytes --]

Wrapping up all comments on the RFC and requesting the RFC move to the implementation phase.

Changes after community discussion:

* GitHub repository will be called edk2-pytool-extensions
* An additional comment will be added to the contribution process to make it clear that PRs will be squashed and therefore patch-sets are not supported and it is best to create smaller individual pull requests.

[-- Attachment #2: Type: text/html, Size: 484 bytes --]

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [edk2-devel] RFC for Edk2-ToolEnv
  2019-06-28 22:13                     ` Sean
@ 2019-06-28 22:22                       ` Michael D Kinney
  0 siblings, 0 replies; 13+ messages in thread
From: Michael D Kinney @ 2019-06-28 22:22 UTC (permalink / raw)
  To: devel@edk2.groups.io, sean.brogan@microsoft.com,
	Kinney, Michael D

[-- Attachment #1: Type: text/plain, Size: 739 bytes --]

Hi Sean,

The edk2-pytool-extensions repo has been created.

Thanks,

Mike

From: devel@edk2.groups.io [mailto:devel@edk2.groups.io] On Behalf Of Sean via Groups.Io
Sent: Friday, June 28, 2019 3:14 PM
To: Sean <sean.brogan@microsoft.com>; devel@edk2.groups.io
Subject: Re: [edk2-devel] RFC for Edk2-ToolEnv

Wrapping up all comments on the RFC and requesting the RFC move to the implementation phase.

Changes after community discussion:

  1.  GitHub repository will be called edk2-pytool-extensions
  2.  An additional comment will be added to the contribution process to make it clear that PRs will be squashed and therefore patch-sets are not supported and it is best to create smaller individual pull requests.



[-- Attachment #2: Type: text/html, Size: 40657 bytes --]

^ permalink raw reply	[flat|nested] 13+ messages in thread

end of thread, other threads:[~2019-06-28 22:22 UTC | newest]

Thread overview: 13+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2019-05-14  2:55 RFC for Edk2-ToolEnv Sean
2019-05-14  4:01 ` [edk2-devel] " rebecca
2019-05-14  7:37   ` Sean
2019-05-14 22:24     ` rebecca
2019-05-14 23:23       ` Sean
2019-05-14 23:34         ` rebecca
2019-05-23  2:39           ` Michael D Kinney
2019-05-23  6:46             ` Sean
2019-05-23 17:14               ` Michael D Kinney
2019-06-11 18:55                 ` Purma, Kondal R
2019-06-12  6:02                   ` Sean
2019-06-28 22:13                     ` Sean
2019-06-28 22:22                       ` Michael D Kinney

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox