Re: [edk2-devel] [RFC] EDK II Continuous Integration Phase 1

["Liming Gao" <liming.gao@intel.com>]

Mike:

> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Michael D Kinney
> Sent: Tuesday, September 24, 2019 1:44 AM
> To: Sean Brogan <sean.brogan@microsoft.com>; devel@edk2.groups.io; rfc@edk2.groups.io; Kinney, Michael D
> <michael.d.kinney@intel.com>
> Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
> Subject: Re: [edk2-devel] [RFC] EDK II Continuous Integration Phase 1
> 
> Hi Sean,
> 
> For host based tests, I agree that VS2017 or VS2019
> would be a good choice.  Pick the one with the best
> coverage and easiest for developers to get feedback on
> the test results and test coverage.  That may be sufficient
> for automated CI tests.  Enabling other tool chains for
> host based testing will be required to support developers
> that implement unit tests and want to test them locally
> before adding to the automated CI tests.
> 
> For build tests, I think we need VS2015, VS2017, GCC,
> and XCODE5.  We can add VS2019 if there is a request to make
> that one of the fully validated tool chains for EDK II.
VS2019 tool chain has been added for IA32/X64/ARM/AARH64/EBC archs. 
CLANG9 tool chain for IA32/X64 will be added. I request to add CLANG9 build test. 

Thanks
Liming
> For pre-commit gates, we could choose to do build tests of
> only the libs/modules touched by the patch series.  Then
> perform complete package/platform build tests in a
> Daily/Weekly/Release scope.
> 
> We need to update the RFC with the specific tool chains and
> tool versions for both Host Based tests and Code Compilation
> Tests.  They are not the same.
> 
> Thanks,
> 
> Mike
> 
> > -----Original Message-----
> > From: Sean Brogan <sean.brogan@microsoft.com>
> > Sent: Friday, September 20, 2019 2:30 PM
> > To: Kinney, Michael D <michael.d.kinney@intel.com>;
> > devel@edk2.groups.io; rfc@edk2.groups.io; Kinney,
> > Michael D <michael.d.kinney@intel.com>
> > Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
> > Subject: RE: [edk2-devel] [RFC] EDK II Continuous
> > Integration Phase 1
> >
> > Currently it is building on Windows with VS2019.
> > VS2017 would be trivial but not worth it in my
> > perspective given how aligned the two are.   If you
> > really wanted to do a weekly or nightly build it could
> > be added to that but I have been focused on a PR build.
> > I have a pipeline for GCC on linux.  It doesn't'
> > support host based unit tests but I am working to get
> > the rest of the tests running.
> > I do not have a pipeline for Mac or LLVM/Clang.  That
> > would be next and I think your files below should help.
> >
> > To date we use PYTOOLs features to install our extra
> > tools.  This makes it super easy and works for both
> > local and server based builds.  It gives strong
> > versioning and management of that.  It also tracks
> > those versions and you can see them on every build in
> > the Built Tool Report artifact.   It can be downloaded
> > here for html version.
> > https://dev.azure.com/tianocore/edk2-ci-
> > play/_build/results?buildId=803&view=artifacts.
> > BUILD_TOOL_REPORT.html
> >
> > Here it is pasted as plain text below.  You can see the
> > iasl and nasm versions here.
> >
> > Key	Value	Type
> > TOOL_CHAIN_TAG	VS2019	TOOL
> > VC Version	14.22.27905	TOOL
> > d:\a\1\s\Conf\build_rule.txt	1.04	INFO
> > d:\a\1\s\Conf\target.txt	1.03	INFO
> > d:\a\1\s\Conf\tools_def.txt	1.22	INFO
> > iasl	20190215.0.0	INFO
> > Mu-Basetools	2019.03.1	INFO
> > mu_nasm	2.14.02	INFO
> >
> >
> > Hope that helps.
> >
> > I would be happy to queue up this topic for next design
> > meeting and open it up to questions?  I hope others
> > would take a look prior to meeting.
> >
> > Thanks
> > Sean
> >
> >
> >
> > -----Original Message-----
> > From: Kinney, Michael D <michael.d.kinney@intel.com>
> > Sent: Thursday, September 19, 2019 2:56 PM
> > To: devel@edk2.groups.io; Sean Brogan
> > <sean.brogan@microsoft.com>; rfc@edk2.groups.io;
> > Kinney, Michael D <michael.d.kinney@intel.com>
> > Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
> > Subject: RE: [edk2-devel] [RFC] EDK II Continuous
> > Integration Phase 1
> >
> > Hi Sean,
> >
> > Which OS/Compiler configurations are currently enabled
> > for the Code Compilation Test?
> >
> > I have been working on enabling multiple OS/Compiler
> > configurations in Azure Pipelines.  There are some
> > tools that need to be installed for each of these
> > environments.
> > Examples include NASM, iASL, Python.
> >
> > For the work you have done, how are these extra tools
> > installed?  Is it in the YML files or in the Python
> > scripts.
> >
> > One critical task is to identify the tools and their
> > specific versions that the CI system is configured to
> > use.
> > These configurations should be documented in a Wiki
> > page and updated as new tools are released and adopted
> > by EDK II.
> > The inventory of tools used to validate a release
> > should Also be documented in a release notes for a
> > stable tag.
> >
> > Here are the YML files that install the additional
> > tools required to support EDK II builds.  I need the
> > source and versions of these tools to be reviewed and
> > approved.
> >
> > https://nam06.safelinks.protection.outlook.com/?url=htt
> > ps%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2-
> > ci%2Fblob%2Fmaster%2FAzurePipelines%2FWindowsPrerequisi
> > tes.yml&amp;data=02%7C01%7Csean.brogan%40microsoft.com%
> > 7C9e7d685fa1764cf500af08d73d4c1e1b%7C72f988bf86f141af91
> > ab2d7cd011db47%7C1%7C0%7C637045269452466989&amp;sdata=h
> > M4AKoMutISI5Oc%2FeVMevx%2FVRUCjJHuhEwqom0R30Ak%3D&amp;r
> > eserved=0
> > https://nam06.safelinks.protection.outlook.com/?url=htt
> > ps%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2-
> > ci%2Fblob%2Fmaster%2FAzurePipelines%2FUbuntuPrerequisit
> > es.yml&amp;data=02%7C01%7Csean.brogan%40microsoft.com%7
> > C9e7d685fa1764cf500af08d73d4c1e1b%7C72f988bf86f141af91a
> > b2d7cd011db47%7C1%7C0%7C637045269452466989&amp;sdata=KQ
> > 2zas2bJ%2FCjSRWGyMrxq5Rk4cW5lOgXQNR99QJbEKY%3D&amp;rese
> > rved=0
> > https://nam06.safelinks.protection.outlook.com/?url=htt
> > ps%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2-
> > ci%2Fblob%2Fmaster%2FAzurePipelines%2FMacOsPrerequisite
> > s.yml&amp;data=02%7C01%7Csean.brogan%40microsoft.com%7C
> > 9e7d685fa1764cf500af08d73d4c1e1b%7C72f988bf86f141af91ab
> > 2d7cd011db47%7C1%7C0%7C637045269452466989&amp;sdata=OzA
> > mzYkRJ3rQOHEDgKTREz%2BNp7acOWUACh1s%2Fb4UPGk%3D&amp;res
> > erved=0
> >
> > Thanks,
> >
> > Mike
> >
> > > -----Original Message-----
> > > From: devel@edk2.groups.io <devel@edk2.groups.io> On
> > Behalf Of Sean
> > > via Groups.Io
> > > Sent: Thursday, August 29, 2019 7:22 PM
> > > To: rfc@edk2.groups.io; Kinney, Michael D
> > > <michael.d.kinney@intel.com>; devel@edk2.groups.io
> > > Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
> > > Subject: Re: [edk2-devel] [RFC] EDK II Continuous
> > Integration Phase 1
> > >
> > > Mike, as you mentioned we have been working towards
> > enabling a
> > > practical and extensible CI for Edk2 using Azure dev
> > ops and the
> > > recently added edk2-pytool infrastructure.  We have
> > been using similar
> > > CI for Project Mu for the last few years.
> > >
> > > Our approach is a little different in that we focus
> > on validating the
> > > whole code base rather than just the incoming patch.
> > We do this
> > > because we have found unexpected consequences of
> > patches and overall
> > > we want all code to be compliant not just new
> > additions.  We have
> > > found the time to test the whole tree is not much
> > longer than only the
> > > parts impacted by a code change (except maybe when
> > running the entire
> > > compile test on every package).  This obviously comes
> > with an initial
> > > tax of needing to get the codebase into compliant
> > form.
> > > Anyway we have prepared an RFC in addition to yours
> > and would like to
> > > see these two efforts merged together.
> > >
> > > We are still working on making a few optimizations.
> > > Currently if the full set of tests are run we take
> > about 20 minutes.
> > > This is because compiling MdeModulePkg for debug,
> > release, and host
> > > based tests take a while.  Most other packages are in
> > the 10 minute
> > > range.  We do have easy ways to disable or limit
> > certain tests as well
> > > as expand the matrix to leverage more cloud resources
> > (more parallel
> > > builds).
> > >
> > >
> > > Content is best viewed online with links to helpful
> > content but is
> > > also attached below:
> > >
> > https://nam06.safelinks.protection.outlook.com/?url=htt
> > ps%3A%2F%2Fgith
> > > ub.com%2Fspbrogan%2Fedk2-staging%2Fblob%2Fedk2-
> > &amp;data=02%7C01%7Csea
> > >
> > n.brogan%40microsoft.com%7C9e7d685fa1764cf500af08d73d4c
> > 1e1b%7C72f988bf
> > >
> > 86f141af91ab2d7cd011db47%7C1%7C0%7C637045269452466989&a
> > mp;sdata=xOiPHH
> > >
> > VbMhFBsIpj%2F9mhOhrkhdsF2Gbehd6%2Frk0XYgM%3D&amp;reserv
> > ed=0
> > > stuart-ci-latest/Readme-CI-RFC.md
> > >
> > > # CI and PR Gates
> > >
> > > ## Background
> > >
> > > Historically, while the TianoCore maintainers and
> > stewards have done a
> > > fantastic job of keeping contribution policies
> > consistent and
> > > contributions clean and well-documented, there have
> > been few processes
> > > that ran to verify the sanity, cleanliness, and
> > efficacy of the
> > > codebase, and even fewer that publicly published
> > their results for the
> > > community at large. This has caused inconsistancies
> > and issues within
> > > the codebase from time to time.
> > >
> > > Adding continuous integration (and potentially PR
> > > gates) to the checkin process ensures that simple
> > errors like these
> > > are caught and can be fixed on a regular basis.
> > >
> > > ## Strategy
> > >
> > > While a number of CI solutions exist, this proposal
> > will focus on the
> > > usage of Azure Dev Ops and Build Pipelines. For
> > demonstration, a
> > > sample [TianoCore
> > >
> > repo](https://nam06.safelinks.protection.outlook.com/?u
> > rl=https%3A%2F%
> > > 2Fgithub.com%2Fspbrogan%2Fedk2-
> > staging.git&amp;data=02%7C01%7Csean.bro
> > >
> > gan%40microsoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%
> > 7C72f988bf86f14
> > >
> > 1af91ab2d7cd011db47%7C1%7C0%7C637045269452466989&amp;sd
> > ata=QmzQhemLUNB
> > > 7FGJwoGeeewExThrUjUArxnkmtro1b8A%3D&amp;reserved=0)
> > > (branch edk2-stuart-ci-latest) and [Dev Ops
> > >
> > Pipeline](https://nam06.safelinks.protection.outlook.co
> > m/?url=https%3A
> > > %2F%2Fdev.azure.com%2Ftianocore%2Fedk2-ci-
> > &amp;data=02%7C01%7Csean.bro
> > >
> > gan%40microsoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%
> > 7C72f988bf86f14
> > >
> > 1af91ab2d7cd011db47%7C1%7C0%7C637045269452466989&amp;sd
> > ata=rS%2FrjIFkS
> > >
> > uEhI%2FDWyslf34i711%2FbY8Gw%2Byexq%2FwydjU%3D&amp;reser
> > ved=0
> > > play/_build?definitionId=12) have been set up.
> > >
> > > Furthermore, this proposal will leverage the
> > TianoCore python tools
> > > PIP modules:
> > >
> > [library](https://nam06.safelinks.protection.outlook.co
> > m/?url=https%3A
> > > %2F%2Fpypi.org%2Fproject%2Fedk2-pytool-
> > &amp;data=02%7C01%7Csean.brogan
> > >
> > %40microsoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%7C7
> > 2f988bf86f141af
> > >
> > 91ab2d7cd011db47%7C1%7C0%7C637045269452466989&amp;sdata
> > =jzM6MKZFFAEvyR
> > > 4h0%2Bkf5pUcBSsoVzkIHFFi1Bd6Il4%3D&amp;reserved=0
> > > library/) and
> > >
> > [extensions](https://nam06.safelinks.protection.outlook
> > .com/?url=https
> > > %3A%2F%2Fpypi.org%2Fproject%2Fedk2-pytool-
> > &amp;data=02%7C01%7Csean.bro
> > >
> > gan%40microsoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%
> > 7C72f988bf86f14
> > >
> > 1af91ab2d7cd011db47%7C1%7C0%7C637045269452466989&amp;sd
> > ata=jzM6MKZFFAE
> > > vyR4h0%2Bkf5pUcBSsoVzkIHFFi1Bd6Il4%3D&amp;reserved=0
> > > extensions/) (with repos located
> > >
> > [here](https://nam06.safelinks.protection.outlook.com/?
> > url=https%3A%2F
> > > %2Fgithub.com%2Ftianocore%2Fedk2-pytool-
> > &amp;data=02%7C01%7Csean.broga
> > >
> > n%40microsoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%7C
> > 72f988bf86f141a
> > >
> > f91ab2d7cd011db47%7C1%7C0%7C637045269452466989&amp;sdat
> > a=OwT%2BiNTN9Rv
> > > r14Oj67IdDlK8WGJClTIsmEzdyrBt2Ho%3D&amp;reserved=0
> > > library) and
> > >
> > [here](https://nam06.safelinks.protection.outlook.com/?
> > url=https%3A%2F
> > > %2Fgithub.com%2Ftianocore%2Fedk2-
> > &amp;data=02%7C01%7Csean.brogan%40mic
> > >
> > rosoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%7C72f988b
> > f86f141af91ab2d
> > >
> > 7cd011db47%7C1%7C0%7C637045269452466989&amp;sdata=tInVw
> > GwCXXGBgcmHQ%2B
> > > 0UwRaEbtZWRy8hTp5wpRtN0R0%3D&amp;reserved=0
> > > pytool-extensions)).
> > >
> > > The primary execution flows can be found in the
> > `azure-
> > > pipelines-pr-gate.yml` and `azure-pipelines-pr-gate-
> > linux.yml` files.
> > > These YAML files are consumed by the Azure Dev Ops
> > Build Pipeline and
> > > dictate what server resources should be used, how
> > they should be
> > > configured, and what processes should be run on them.
> > > An overview of this schema can be found
> > >
> > [here](https://nam06.safelinks.protection.outlook.com/?
> > url=https%3A%2F
> > > %2Fdocs.microsoft.com%2Fen-
> > &amp;data=02%7C01%7Csean.brogan%40microsoft
> > >
> > .com%7C9e7d685fa1764cf500af08d73d4c1e1b%7C72f988bf86f14
> > 1af91ab2d7cd011
> > >
> > db47%7C1%7C0%7C637045269452466989&amp;sdata=j8nyYG3Vniz
> > wKFZ8l1BWoLcPy%
> > > 2BuaSoVT0jN2Esi7wH8%3D&amp;reserved=0
> > > us/azure/devops/pipelines/yaml-schema?view=azure-
> > > devops&tabs=schema).
> > >
> > > Inspection of these files reveals the EDKII Tools
> > commands that make
> > > up the primary processes for the CI
> > > build: 'stuart_setup', 'stuart_update', and
> > 'stuart_ci_build'. These
> > > commands come from the EDKII Tools PIP modules and
> > are configured as
> > > described below. More documentation on the stuart
> > tools can be found
> > >
> > [here](https://nam06.safelinks.protection.outlook.com/?
> > url=https%3A%2F
> > > %2Fgithub.com%2Ftianocore%2Fedk2-pytool-
> > &amp;data=02%7C01%7Csean.broga
> > >
> > n%40microsoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%7C
> > 72f988bf86f141a
> > >
> > f91ab2d7cd011db47%7C1%7C0%7C637045269452466989&amp;sdat
> > a=OwT%2BiNTN9Rv
> > > r14Oj67IdDlK8WGJClTIsmEzdyrBt2Ho%3D&amp;reserved=0
> > > extensions/blob/master/docs/using.md) and
> > >
> > [here](https://nam06.safelinks.protection.outlook.com/?
> > url=https%3A%2F
> > > %2Fgithub.com%2Ftianocore%2Fedk2-pytool-
> > &amp;data=02%7C01%7Csean.broga
> > >
> > n%40microsoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%7C
> > 72f988bf86f141a
> > >
> > f91ab2d7cd011db47%7C1%7C0%7C637045269452466989&amp;sdat
> > a=OwT%2BiNTN9Rv
> > > r14Oj67IdDlK8WGJClTIsmEzdyrBt2Ho%3D&amp;reserved=0
> > >
> > extensions/blob/master/docs/features/feature_invocables
> > > .md).
> > >
> > > ## Configuration
> > >
> > > Configuration of the CI process consists of (in order
> > of precedence):
> > > * command-line arguments passed in via the Pipeline
> > YAML
> > > * a per-package configuration file (e.g. `<package-
> > > name>.mu.yaml`) that is detected by the CI system in
> > > EDKII Tools.
> > > * a global configuration Python module (e.g.
> > > `CISetting.py`) passed in via the command-line
> > >
> > > The global configuration file is described in [this
> > >
> > readme](https://nam06.safelinks.protection.outlook.com/
> > ?url=https%3A%2
> > > F%2Fgithub.com%2Ftianocore%2Fedk2-pytool-
> > &amp;data=02%7C01%7Csean.brog
> > >
> > an%40microsoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%7
> > C72f988bf86f141
> > >
> > af91ab2d7cd011db47%7C1%7C0%7C637045269452466989&amp;sda
> > ta=OwT%2BiNTN9R
> > > vr14Oj67IdDlK8WGJClTIsmEzdyrBt2Ho%3D&amp;reserved=0
> > >
> > extensions/blob/master/docs/usability/using_settings_ma
> > > nager.md) from the EDKII Tools documentation. This
> > configuration is
> > > written as a Python module so that decisions can be
> > made dynamically
> > > based on command line parameters and codebase state.
> > >
> > > The per-package configuration file can override most
> > settings in the
> > > global configuration file, but is not dynamic. This
> > file can be used
> > > to skip or customize tests that may be incompatible
> > with a specific
> > > package.
> > > By default, the global configuration will try to run
> > all tests on all
> > > packages.
> > >
> > > ## CI Test Types
> > >
> > > All CI tests are instances of EDKII Tools plugins.
> > > Documentation on the plugin system can be found
> > >
> > [here](https://nam06.safelinks.protection.outlook.com/?
> > url=https%3A%2F
> > > %2Fgithub.com%2Ftianocore%2Fedk2-pytool-
> > &amp;data=02%7C01%7Csean.broga
> > >
> > n%40microsoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%7C
> > 72f988bf86f141a
> > >
> > f91ab2d7cd011db47%7C1%7C0%7C637045269452476994&amp;sdat
> > a=1AM4geY%2BEFh
> > > ekun7oQOYyVVt%2Bwn6CAF2hhtkS0sgwPU%3D&amp;reserved=0
> > >
> > extensions/blob/master/docs/usability/using_plugin_mana
> > > ger.md) and
> > >
> > [here](https://nam06.safelinks.protection.outlook.com/?
> > url=https%3A%2F
> > > %2Fgithub.com%2Ftianocore%2Fedk2-
> > &amp;data=02%7C01%7Csean.brogan%40mic
> > >
> > rosoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%7C72f988b
> > f86f141af91ab2d
> > >
> > 7cd011db47%7C1%7C0%7C637045269452476994&amp;sdata=3vyPM
> > vrfG3V4doNM0%2B
> > > cKTpDSHEEEvkfxKAtw3eiJWsY%3D&amp;reserved=0
> > > pytool-
> > >
> > extensions/blob/master/docs/features/feature_plugin_man
> > > ager.md). Upon invocation, each plugin will be passed
> > the path to the
> > > current package under test and a dictionary
> > containing its targeted
> > > configuration, as assembled from the command line,
> > per-package
> > > configuration, and global configuration.
> > >
> > > Note: CI plugins are considered unique from build
> > plugins and helper
> > > plugins, even though some CI plugins may execute
> > steps of a build.
> > >
> > > In the example, these plugins live alongside the code
> > under test (in
> > > the `BaseTools` directory), but may be moved to the
> > 'edk2-test' repo
> > > if that location makes more sense for the community.
> > >
> > > ### Module Inclusion Test - DscCompleteCheck
> > >
> > > This test scans all available modules (via INF files)
> > and compares
> > > them to the package-level DSC file for the package
> > each module is
> > > contained within. The test considers it an error if
> > any module does
> > > not appear in the `Components` section of at least
> > one package-level
> > > DSC (indicating that it would not be built if the
> > package were built).
> > >
> > > ### Code Compilation Test - CompilerPlugin
> > >
> > > Once the Module Inclusion Test has verified that all
> > modules would be
> > > built if all package-level DSCs were built, the Code
> > Compilation Test
> > > simply runs through and builds every package-level
> > DSC on every
> > > toolchain and for every architecture that is
> > supported. Any module
> > > that fails to build is considered an error.
> > >
> > > ### Host-Based UnitTests - HostUnitTestCompilerPlugin
> > and
> > > HostUnitTestDscCompleteCheck
> > >
> > > The [Testing RFC doc](Readme-Testing-RFC.md) has much
> > more detail on
> > > this, but the basic idea is that host- based unit
> > tests can be
> > > compiled against individual modules and libraries and
> > run on the build
> > > agent (in this case, the Dev Ops build server). The
> > successful and
> > > failing test case results are collected and included
> > in the final
> > > build report.
> > >
> > > ### GUID Uniqueness Test - GuidCheck
> > >
> > > This test works on the collection of all packages
> > rather than an
> > > individual package. It looks at all FILE_GUIDs and
> > GUIDs declared in
> > > DEC files and ensures that they are unique for the
> > codebase. This
> > > prevents, for example, accidental duplication of
> > GUIDs when using an
> > > existing INF as a template for a new module.
> > >
> > > ### Cross-Package Dependency Test - DependencyCheck
> > >
> > > This test compares the list of all packages used in
> > INFs files for a
> > > given package against a list of "allowed
> > dependencies" in plugin
> > > configuration for that package. Any module that
> > depends on a
> > > disallowed package will cause a test failure.
> > >
> > > ### Library Declaration Test - LibraryClassCheck
> > >
> > > This test looks at all library header files found in
> > a package's
> > > `Include/Library` directory and ensures that all
> > files have a matching
> > > LibraryClass declaration in the DEC file for the
> > package. Any missing
> > > declarations will cause a failure.
> > >
> > > ### Invalid Character Test - CharEncodingCheck
> > >
> > > This test scans all files in a package to make sure
> > that there are no
> > > invalid Unicode characters that may cause build
> > errors in some
> > > character sets/localizations.
> > >
> > > ## Next Steps
> > >
> > > * Receive community feedback on RFC.
> > > * Determine where this phase makes sense given
> > existing RFCs from
> > > other TianoCore contributors.
> > > * Optimize testing beharior.
> > >   * Only run a subset of tests on PRs or individual
> > commits.
> > >   * Run full testing either once per day or once
> > every several
> > > commits.
> > > * Add more tests/capabilities.
> > > * Continue to improve results formatting.
> > > * Continue to improve CI documentation.
> > >   * Much of this documentation effort is pending
> > community feedback on
> > > which parts are needed and what phases are
> > priorities.
> > >
> > > Thanks
> > >
> > >
> > > -----Original Message-----
> > > From: rfc@edk2.groups.io <rfc@edk2.groups.io> On
> > Behalf Of Michael D
> > > Kinney via Groups.Io
> > > Sent: Thursday, August 29, 2019 1:23 PM
> > > To: devel@edk2.groups.io; rfc@edk2.groups.io
> > > Subject: [edk2-rfc] [RFC] EDK II Continuous
> > Integration Phase 1
> > >
> > > Hello,
> > >
> > > This is a proposal for a first step towards
> > continuous integration for
> > > all TianoCore repositories to help improve to quality
> > of commits and
> > > automate testing and release processes for all EDK II
> > packages and
> > > platforms.
> > >
> > > This is based on work from a number of EDK II
> > community members that
> > > have provide valuable input and evaluations.
> > >
> > > * Rebecca Cran <mailto:rebecca@bsdio.com> Jenkins
> > evaluation
> > > * Laszlo Ersek <mailto:lersek@redhat.com> GitLab
> > evaluation
> > > * Philippe Mathieu-Daudé <mailto:philmd@redhat.com>
> > GitLab evaluation
> > > * Sean Brogan <mailto:sean.brogan@microsoft.com>
> > Azure Pipelines and
> > > HBFA
> > > * Bret Barkelew <mailto:Bret.Barkelew@microsoft.com>
> > > Azure Pipelines and HBFA
> > > * Jiewen Yao <mailto:jiewen.yao@intel.com> HBFA
> > >
> > > The following link is a link to an EDK II WIKI page
> > that contains a
> > > summary of the work to date.  Please provide feedback
> > in the EDK II
> > > mailing lists.  The WIKI pages will be updated with
> > input from the
> > > entire EDK II community.
> > >
> > >
> > >
> > https://nam06.safelinks.protection.outlook.com/?url=htt
> > >
> > ps%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io
> > > %2Fwiki%2FEDK-II-Continuous-
> > >
> > Integration&amp;data=02%7C01%7Csean.brogan%40microsoft.
> > >
> > com%7C6f67f169a6c746b4288608d72cbea7b6%7C72f988bf86f141
> > >
> > af91ab2d7cd011db47%7C1%7C0%7C637027069686644659&amp;sda
> > >
> > ta=GR9wN6gP3mJlCTopAaQ2rlzhby1nuF%2BwDVsfFIQAZjA%3D&amp
> > > ;reserved=0
> > >
> > > Proposal
> > > ========
> > > Phase 1 of adding continuous integration is limited
> > to the
> > > edk2 repository.  Additional repositories will be
> > added later.
> > >
> > > The following changes are proposed:
> > > * Remove EDK II Maintainers write access to edk2
> > repository.
> > >   Only EDK II Administrators will continue to have
> > write
> > >   access, and that should only be used to handle
> > extraordinary
> > >   events.
> > > * EDK II Maintainers use a GitHub Pull Request
> > instead of push
> > >   to commit a patch series to the edk2 repository.
> > > There are
> > >   no other changes to the development and review
> > process.  The
> > >   patch series is prepared in an EDK II maintainer
> > branch with
> > >   all commit message requirements met on each patch
> > in the series.
> > > * The edk2 repository only accepts Pull Requests from
> > members
> > >   of the EDK II Maintainers team.  Pull Requests from
> > anyone else
> > >   are rejected.
> > > * Run pre-commit checks using Azure Pipelines
> > > * If all pre-commit checks pass, then the patch
> > series is auto
> > >   committed.  The result of this commit must match
> > the contents
> > >   and commit history that would have occurred using
> > the previous
> > >   push operation.
> > > * If any pre-commit checks fail, then notify the
> > submitter.
> > >   A typical reason for a failure would be a merge
> > conflict with
> > >   another pull request that was just processed.
> > > * Limit pre-commit checks execution time to 10
> > minutes.
> > > * Provide on-demand builds to EDK II Maintainers that
> > to allow
> > >   EDK II Maintainers to submit a branch through for
> > the same
> > >   set of pre-commit checks without submitting a pull
> > request.
> > >
> > > ## Pre-Commit Checks in Phase 1
> > > * Run and pass PatchCheck.py with no errors
> > >
> > > =====================================================
> > >
> > > The following are some additional pre-commit check
> > ideas that could be
> > > quickly added once the initial version using
> > PatchCheck.py is fully
> > > functional.
> > > Please provide feedback on the ones you like and
> > additional ones you
> > > think may improve the quality of the commits to the
> > edk2 repository.
> > >
> > > ## Proposed Pre-Commit Checks in Phase 2
> > > * Verify Reviewed-by and Acked-by tags are present
> > with
> > >   correct maintainer email addresses
> > > * Verify no non-ASCII characters in modified files
> > > * Verify no binary files in set of modified files
> > > * Verify package dependency rules in modified files
> > >
> > > ## Proposed Pre-Commit Checks in Phase 3
> > > * Run ECC on modified files
> > > * Verify modified modules/libs build
> > > * Run available host based tests (HBFA) against
> > modified
> > >   modules/libs
> > >
> > > Best regards,
> > >
> > > Mike
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> 
> 
>