[edk2-rfc] GitHub Pull Request based Code Review Process

[edk2-rfc] GitHub Pull Request based Code Review Process

[Michael D Kinney]

Hello,

This is a proposal to change from the current email-based code review process to
a GitHub pull request-based code review process for all repositories maintained
in TianoCore.  The current email-based code review process and commit message
requirements are documented in Readme.md or Readme.rst at the root of
repositories along with a few Wiki pages:

* https://github.com/tianocore/edk2/blob/master/ReadMe.rst
* https://github.com/tianocore/tianocore.github.io/wiki/EDK-II-Development-Process
* https://github.com/tianocore/tianocore.github.io/wiki/Laszlo's-unkempt-git-guide-for-edk2-contributors-and-maintainers
* https://github.com/tianocore/tianocore.github.io/wiki/Commit-Message-Format
* https://github.com/tianocore/tianocore.github.io/wiki/Commit-Signature-Format

The goal is to post changes by opening a GitHub pull request and perform all
code review activity using the GitHub web interface.  This proposal does not
change any licenses or commit message requirements.  It does require all
developers, maintainers, and reviewers to have GitHub accounts.

One requirement that was collected from previous discussions on this topic is
the need for an email archive of all patches and code review activities.  The
existing GitHub features to produce an email archive were deemed insufficient.
A proof of concept of a GitHub webhook has been implemented to provide the email
archive service.  This email archive is read-only.  You will not be able to send
emails to this archive or reply to emails in the archive.

The sections below provide more details on the proposed GitHub pull request
based code review process, details on the email archive service, and a set of
remaining tasks make the email archive service production quality.  It does not
make sense to support both the existing email-based code review and the GitHub
pull request-based code review at the same time.  Instead, this proposal is to
switch to the GitHub pull request-based code review and retire the email based
code review process on the same date.

The edk2 repository is using GitHub pull requests today to run automated
CI checks on the code changes and allows a maintainer to set the `push` label to
request the changes to be merged if all CI checks pass.  With this proposal,
once the code review is complete and the commit messages have been updated, the
same pull request can be used to perform a final set of CI checks and merge the
changes into the master branch.

I would like to collect feedback on this proposal and the email archive service
over the next two weeks with close of comments on Friday May 22, 2020.  If all
issues and concerns can be addressed, then I would like to see the community
agree to make this change as soon as all remaining tasks are completed.

# TianoCore Repositories to enable

* [edk2](https://github.com/tianocore/edk2)
* [edk2-platforms](https://github.com/tianocore/edk2-platforms)
* [edk2-non-osi](https://github.com/tianocore/edk2-non-osi)
* [edk2-test](https://github.com/tianocore/edk2-test)
* [edk2-libc](https://github.com/tianocore/edk2-libc)
* [edk2-staging](https://github.com/tianocore/edk2-staging)

# GitHub Pull Request Code Review Process

**NOTE**: All steps below use [edk2](https://github.com/tianocore/edk2) as an
example.  Several repositories are supported.

## Author/Developer Steps
  * Create a personal fork of [edk2](https://github.com/tianocore/edk2)

    https://help.github.com/en/github/getting-started-with-github/fork-a-repo

  * Create a new branch from edk2/master in personal fork of edk2 repository.

  * Add set of commits for new feature or bug fix to new branch.  Make sure to
    follow the commit message format requirements.  The only change with this
    RFC is that the Cc: lines to maintainers/reviewers should **not** be added.
    The Cc: lines are still supported, but they should only be used to add
    reviewers that do not have GitHub IDs or are not members of TianoCore.

  * Push branch with new commits to personal fork
  * Create a pull request against TianoCore edk2/master

    https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/creating-a-pull-request

  * If pull request has more than 1 commit, then fill in the pull request title
    and decryption information for Patch #0.  Do not leave defaults.

  * Do not assign reviewers.  The webhook assigns maintainers and reviewers to
    the pull request and each commit in the pull request.

  * If maintainers/reviewers provide feedback that requires changes, then make
    add commits to the current branch with the requested changes.  Once all
    changes are accepted on the current branch, reformulate the patch series and
    commit comments as needed for perform a forced push to the branch in the
    personal fork of the edk2 repository.  This step may be repeated if multiple
    versions of the patch series are required to address all code review
    feedback.

  **OPEN**: How should minimum review period be set?  Labels?

## TianoCore GitHub Email Archive Webhook Service Steps
  * Receive an event that a new pull request was opened
  * Evaluate the files modified by the entire pull request and each commit in
    the pull request and cross references against `Maintainters.txt` in the root
    of the repository to assign maintainers/reviewers to the pull request and
    each commit in the pull request. Individual commit assignments are performed
    by adding a commit comment of the following form:

    [CodeReview] Review-request @mdkinney

  * Generate and sends git patch review emails to the email archive.  Emails
    are also sent to any Cc: tags in the commit messages.

  * If the author/developer performs a forced push to the branch in their
    personal fork of the edk2 repository, then a new set of patch review emails
    with patch series Vx is sent to the email archive and any Cc: tags in commit
    messages.

  * Receive events associated with all code review activities and generate
    and send emails to the email archive that shows all review comments and
    all responses closely matching the email contents seen in the current email
    based code review process.

  * Generate and send email when pull request is merged or closed.

## Maintainer/Reviewer Steps

  * Make sure GitHub configuration is setup to 'Watch' the repositories that
    you have maintainer ship or review responsibilities and that email
    notifications from GitHub are enabled.  This enables email notifications
    when a maintainer/reviewer is assigned to a pull request and individual
    commits.

    https://help.github.com/en/github/managing-subscriptions-and-notifications-on-github/configuring-notifications

  * Subscribe to the email archive associated with the TianoCore GitHub Email
    Archive Webhook Service.

    https://www.redhat.com/mailman/listinfo/tianocore-code-review-poc

  * Review pull requests and commits assigned by the TianoCore GitHub Email
    Archive Webhook Service and use the GitHub web UI to provide all review
    feedback.

    https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/reviewing-changes-in-pull-requests

  * Wait for Author/Developer to respond to all feedback and add commits with
    code changes as needed to resolve all feedback.  This step may be repeated
    if the developer/author need to produce multiple versions of the patch
    series to address all feedback.

  * Once all feedback is addressed, add Reviewed-by, Acked-by, and Tested-by
    responses on individual commits.  Or add Series-reviewed-by, Series-acked-by,
    or Series-Tested-by responses to the entire pull request.

  * Wait for Developer/Author to add tags to commit messages in the pull request.

  * Perform final review of patches and commit message tags.  If there are not
    issues, set the `push` label to run final set of CI checks and auto merge
    the pull request into master.

# Maintainers.txt Format Changes

Add GitHub IDs of all maintainers and reviewers at the end of M: and R: lines
in [].  For example:

    M: Michael D Kinney <michael.d.kinney@intel.com> [mdkinney]

# TianoCore GitHub Email Archive Webhook Service

Assign reviewers to commits in a GitHub pull request based on assignments
documented in Maintainers.txt and generates an email archive of all pull request
and code review activities.

https://github.com/mdkinney/edk2-email-archive-webhook

# Email Archive Subscription Service

The emails are being delivered to the following RedHat email subscription
service.  Please subscribe to receive the emails and to be able to view the
email archives.

https://www.redhat.com/mailman/listinfo/tianocore-code-review-poc

The email archives are at this link:

https://www.redhat.com/mailman/private/tianocore-code-review-poc/index.html

The following sections show some example pull requests and code reviews to
help review the generated emails, their contents, and threading.

## Email Achieve Thread View

https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/thread.html#00289

## Example patch series with 1 patch

https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/thread.html#00340

## Example patch series with < 10 patches

* https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00289.html
* https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00030.html
* https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00018.html
* https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00008.html

## Example patch series with > 80 patches

* https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00198.html
* https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00116.html
* https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00035.html

# Tasks to Complete

* Create edk2-codereview repository for evaluation of new code review process.
* Add GitHub IDs to Maintainers.txt in edk2-codereview repository
* Update BaseTools/Scripts/GetMaintainer.py to be compatible with GitHub IDs at
  the end of M: and R: statements
* Update webhook to use Rabbit MQ to manage requests and emails
* Determine if webhook requests must be serialized?  Current POC is serialized.
* Make sure webhook has error handling for all unexpected events/states.
* Add logging of all events and emails to webhook
* Add admin interface to webhook
* Deploy webhook on a production server with 24/7 support

# Ideas for Future Enhancements

* Run PatchCheck.py before assigning maintainers/reviewers.
* Add a simple check that fails if a single patch spans more than one package.
* Monitor comments for Reviewed-by, Acked-by, Tested-by, Series-Reviewed-by,
  Series-Acked-by, Series-Tested-by made by assigned maintainers/reviewers.
  Once all commits have required tags, auto update commit messages in the
  branch and wait for maintainer to set the `Push` label to run CI and auto
  merge if all CI checks pass.

Best regards,

Mike

Re: [edk2-rfc] GitHub Pull Request based Code Review Process

[Ni, Ray]

Mike,
It's a huge improvement to me as an Outlook user if pull-request-based review is enabled!

Please help me to understand: The pull-request-based review has been enabled naturally when edk2
was migrated to Github. People don't use it because it's not accepted by community. Your process
tries to meet community's needs of achieving all review comments in mails so pull-request-based
review can be accepted by community. Right?

I just subscribed at https://www.redhat.com/mailman/listinfo/tianocore-code-review-poc with
empty password.
I received the confirmation mail and clicked the link in the mail to confirm.
But I waited for ~15 minutes and didn't receive the additional mail containing the auto-generated password.
I went to https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/thread.html#00289.
However, the page requires me to enter password.
Can you please change the setting so that viewing the mail achieve doesn't need password?

Please advise me what else I can try. I am happy to try as many steps as I can before all the process are enabled.

Thanks,
Ray


> -----Original Message-----
> From: rfc@edk2.groups.io <rfc@edk2.groups.io> On Behalf Of Michael D Kinney
> Sent: Saturday, May 9, 2020 11:00 AM
> To: devel@edk2.groups.io; rfc@edk2.groups.io; Kinney, Michael D <michael.d.kinney@intel.com>
> Subject: [edk2-rfc] GitHub Pull Request based Code Review Process
> 
> Hello,
> 
> This is a proposal to change from the current email-based code review process to
> a GitHub pull request-based code review process for all repositories maintained
> in TianoCore.  The current email-based code review process and commit message
> requirements are documented in Readme.md or Readme.rst at the root of
> repositories along with a few Wiki pages:
> 
> * https://github.com/tianocore/edk2/blob/master/ReadMe.rst
> * https://github.com/tianocore/tianocore.github.io/wiki/EDK-II-Development-Process
> * https://github.com/tianocore/tianocore.github.io/wiki/Laszlo's-unkempt-git-guide-for-edk2-contributors-and-maintainers
> * https://github.com/tianocore/tianocore.github.io/wiki/Commit-Message-Format
> * https://github.com/tianocore/tianocore.github.io/wiki/Commit-Signature-Format
> 
> The goal is to post changes by opening a GitHub pull request and perform all
> code review activity using the GitHub web interface.  This proposal does not
> change any licenses or commit message requirements.  It does require all
> developers, maintainers, and reviewers to have GitHub accounts.
> 
> One requirement that was collected from previous discussions on this topic is
> the need for an email archive of all patches and code review activities.  The
> existing GitHub features to produce an email archive were deemed insufficient.
> A proof of concept of a GitHub webhook has been implemented to provide the email
> archive service.  This email archive is read-only.  You will not be able to send
> emails to this archive or reply to emails in the archive.
> 
> The sections below provide more details on the proposed GitHub pull request
> based code review process, details on the email archive service, and a set of
> remaining tasks make the email archive service production quality.  It does not
> make sense to support both the existing email-based code review and the GitHub
> pull request-based code review at the same time.  Instead, this proposal is to
> switch to the GitHub pull request-based code review and retire the email based
> code review process on the same date.
> 
> The edk2 repository is using GitHub pull requests today to run automated
> CI checks on the code changes and allows a maintainer to set the `push` label to
> request the changes to be merged if all CI checks pass.  With this proposal,
> once the code review is complete and the commit messages have been updated, the
> same pull request can be used to perform a final set of CI checks and merge the
> changes into the master branch.
> 
> I would like to collect feedback on this proposal and the email archive service
> over the next two weeks with close of comments on Friday May 22, 2020.  If all
> issues and concerns can be addressed, then I would like to see the community
> agree to make this change as soon as all remaining tasks are completed.
> 
> # TianoCore Repositories to enable
> 
> * [edk2](https://github.com/tianocore/edk2)
> * [edk2-platforms](https://github.com/tianocore/edk2-platforms)
> * [edk2-non-osi](https://github.com/tianocore/edk2-non-osi)
> * [edk2-test](https://github.com/tianocore/edk2-test)
> * [edk2-libc](https://github.com/tianocore/edk2-libc)
> * [edk2-staging](https://github.com/tianocore/edk2-staging)
> 
> # GitHub Pull Request Code Review Process
> 
> **NOTE**: All steps below use [edk2](https://github.com/tianocore/edk2) as an
> example.  Several repositories are supported.
> 
> ## Author/Developer Steps
>   * Create a personal fork of [edk2](https://github.com/tianocore/edk2)
> 
>     https://help.github.com/en/github/getting-started-with-github/fork-a-repo
> 
>   * Create a new branch from edk2/master in personal fork of edk2 repository.
> 
>   * Add set of commits for new feature or bug fix to new branch.  Make sure to
>     follow the commit message format requirements.  The only change with this
>     RFC is that the Cc: lines to maintainers/reviewers should **not** be added.
>     The Cc: lines are still supported, but they should only be used to add
>     reviewers that do not have GitHub IDs or are not members of TianoCore.
> 
>   * Push branch with new commits to personal fork
>   * Create a pull request against TianoCore edk2/master
> 
>     https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/creating-a-pull-request
> 
>   * If pull request has more than 1 commit, then fill in the pull request title
>     and decryption information for Patch #0.  Do not leave defaults.
> 
>   * Do not assign reviewers.  The webhook assigns maintainers and reviewers to
>     the pull request and each commit in the pull request.
> 
>   * If maintainers/reviewers provide feedback that requires changes, then make
>     add commits to the current branch with the requested changes.  Once all
>     changes are accepted on the current branch, reformulate the patch series and
>     commit comments as needed for perform a forced push to the branch in the
>     personal fork of the edk2 repository.  This step may be repeated if multiple
>     versions of the patch series are required to address all code review
>     feedback.
> 
>   **OPEN**: How should minimum review period be set?  Labels?
> 
> ## TianoCore GitHub Email Archive Webhook Service Steps
>   * Receive an event that a new pull request was opened
>   * Evaluate the files modified by the entire pull request and each commit in
>     the pull request and cross references against `Maintainters.txt` in the root
>     of the repository to assign maintainers/reviewers to the pull request and
>     each commit in the pull request. Individual commit assignments are performed
>     by adding a commit comment of the following form:
> 
>     [CodeReview] Review-request @mdkinney
> 
>   * Generate and sends git patch review emails to the email archive.  Emails
>     are also sent to any Cc: tags in the commit messages.
> 
>   * If the author/developer performs a forced push to the branch in their
>     personal fork of the edk2 repository, then a new set of patch review emails
>     with patch series Vx is sent to the email archive and any Cc: tags in commit
>     messages.
> 
>   * Receive events associated with all code review activities and generate
>     and send emails to the email archive that shows all review comments and
>     all responses closely matching the email contents seen in the current email
>     based code review process.
> 
>   * Generate and send email when pull request is merged or closed.
> 
> ## Maintainer/Reviewer Steps
> 
>   * Make sure GitHub configuration is setup to 'Watch' the repositories that
>     you have maintainer ship or review responsibilities and that email
>     notifications from GitHub are enabled.  This enables email notifications
>     when a maintainer/reviewer is assigned to a pull request and individual
>     commits.
> 
>     https://help.github.com/en/github/managing-subscriptions-and-notifications-on-github/configuring-notifications
> 
>   * Subscribe to the email archive associated with the TianoCore GitHub Email
>     Archive Webhook Service.
> 
>     https://www.redhat.com/mailman/listinfo/tianocore-code-review-poc
> 
>   * Review pull requests and commits assigned by the TianoCore GitHub Email
>     Archive Webhook Service and use the GitHub web UI to provide all review
>     feedback.
> 
>     https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/reviewing-changes-in-pull-requests
> 
>   * Wait for Author/Developer to respond to all feedback and add commits with
>     code changes as needed to resolve all feedback.  This step may be repeated
>     if the developer/author need to produce multiple versions of the patch
>     series to address all feedback.
> 
>   * Once all feedback is addressed, add Reviewed-by, Acked-by, and Tested-by
>     responses on individual commits.  Or add Series-reviewed-by, Series-acked-by,
>     or Series-Tested-by responses to the entire pull request.
> 
>   * Wait for Developer/Author to add tags to commit messages in the pull request.
> 
>   * Perform final review of patches and commit message tags.  If there are not
>     issues, set the `push` label to run final set of CI checks and auto merge
>     the pull request into master.
> 
> # Maintainers.txt Format Changes
> 
> Add GitHub IDs of all maintainers and reviewers at the end of M: and R: lines
> in [].  For example:
> 
>     M: Michael D Kinney <michael.d.kinney@intel.com> [mdkinney]
> 
> # TianoCore GitHub Email Archive Webhook Service
> 
> Assign reviewers to commits in a GitHub pull request based on assignments
> documented in Maintainers.txt and generates an email archive of all pull request
> and code review activities.
> 
> https://github.com/mdkinney/edk2-email-archive-webhook
> 
> # Email Archive Subscription Service
> 
> The emails are being delivered to the following RedHat email subscription
> service.  Please subscribe to receive the emails and to be able to view the
> email archives.
> 
> https://www.redhat.com/mailman/listinfo/tianocore-code-review-poc
> 
> The email archives are at this link:
> 
> https://www.redhat.com/mailman/private/tianocore-code-review-poc/index.html
> 
> The following sections show some example pull requests and code reviews to
> help review the generated emails, their contents, and threading.
> 
> ## Email Achieve Thread View
> 
> https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/thread.html#00289
> 
> ## Example patch series with 1 patch
> 
> https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/thread.html#00340
> 
> ## Example patch series with < 10 patches
> 
> * https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00289.html
> * https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00030.html
> * https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00018.html
> * https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00008.html
> 
> ## Example patch series with > 80 patches
> 
> * https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00198.html
> * https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00116.html
> * https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00035.html
> 
> # Tasks to Complete
> 
> * Create edk2-codereview repository for evaluation of new code review process.
> * Add GitHub IDs to Maintainers.txt in edk2-codereview repository
> * Update BaseTools/Scripts/GetMaintainer.py to be compatible with GitHub IDs at
>   the end of M: and R: statements
> * Update webhook to use Rabbit MQ to manage requests and emails
> * Determine if webhook requests must be serialized?  Current POC is serialized.
> * Make sure webhook has error handling for all unexpected events/states.
> * Add logging of all events and emails to webhook
> * Add admin interface to webhook
> * Deploy webhook on a production server with 24/7 support
> 
> # Ideas for Future Enhancements
> 
> * Run PatchCheck.py before assigning maintainers/reviewers.
> * Add a simple check that fails if a single patch spans more than one package.
> * Monitor comments for Reviewed-by, Acked-by, Tested-by, Series-Reviewed-by,
>   Series-Acked-by, Series-Tested-by made by assigned maintainers/reviewers.
>   Once all commits have required tags, auto update commit messages in the
>   branch and wait for maintainer to set the `Push` label to run CI and auto
>   merge if all CI checks pass.
> 
> Best regards,
> 
> Mike
> 
> 
> 
> 

Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

[Rebecca Cran]

On 5/8/20 8:59 PM, Michael D Kinney wrote:

>    * Perform final review of patches and commit message tags.  If there are not
>      issues, set the `push` label to run final set of CI checks and auto merge
>      the pull request into master.

What's the difference between the CI that runs when a user submits the 
Pull Request, and the final CI checks that run before the request is merged?

Also, I'm wondering why Mergify is being used instead of the maintainer 
hitting the "Merge Pull Request" button, or however it's worded?


-- 
Rebecca Cran

Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

[Michael D Kinney]

Rebecca,

There is no difference between CI checks run during code review
and the final CI checks before merge.  I think it is an interesting
conversation to decide how many times those CI checks should be 
run and if they should run automatically on every change during
review or on demand.

Mergify is more flexible.  We want to make sure the git history
is linear with not git merges and supports both single patches
and patch series without squashing.  GitHub merge button by 
default squashes all commits into a single commit.

Thanks,

Mike

> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On
> Behalf Of Rebecca Cran
> Sent: Saturday, May 9, 2020 11:25 AM
> To: devel@edk2.groups.io; Kinney, Michael D
> <michael.d.kinney@intel.com>; rfc@edk2.groups.io
> Subject: Re: [edk2-devel] [edk2-rfc] GitHub Pull
> Request based Code Review Process
> 
> On 5/8/20 8:59 PM, Michael D Kinney wrote:
> 
> >    * Perform final review of patches and commit
> message tags.  If there are not
> >      issues, set the `push` label to run final set of
> CI checks and auto merge
> >      the pull request into master.
> 
> What's the difference between the CI that runs when a
> user submits the
> Pull Request, and the final CI checks that run before
> the request is merged?
> 
> Also, I'm wondering why Mergify is being used instead
> of the maintainer
> hitting the "Merge Pull Request" button, or however
> it's worded?
> 
> 
> --
> Rebecca Cran
> 
> 
> 
> 

Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

[Rebecca Cran]

Mike,

On 5/10/20 3:29 PM, Michael D Kinney wrote:

> There is no difference between CI checks run during code review
> and the final CI checks before merge.  I think it is an interesting
> conversation to decide how many times those CI checks should be
> run and if they should run automatically on every change during
> review or on demand.

I'd suggest following what other Github projects do, which I think is to 
run the CI checks automatically on every change that's made in a pull 
request - I don't know if it might also be necessary to run them during 
the merge, if master has changed in the meantime. That gives the 
_submitter_ feedback about any changes they need to make, instead of 
having to wait until the maintainer tells them their change has broken 
something: it speeds up the development process.

> Mergify is more flexible.  We want to make sure the git history
> is linear with not git merges and supports both single patches
> and patch series without squashing.  GitHub merge button by
> default squashes all commits into a single commit.

Wouldn't disabling all but "Allow rebase merging" do the same thing 
without the additional potential failure point? Though it sounds like 
we've resolved the problems with Mergify, so it's not important.

https://help.github.com/en/github/administering-a-repository/configuring-commit-squashing-for-pull-requests


-- 
Rebecca Cran

Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

[Michael D Kinney]

Rebecca,

I agree that the first version should rerun CI checks
on every time commits are added to a PR or there is a 
forced push to the PR.

Perhaps we should use Draft Pull Requests as a way
to indicate the content is not ready for code review
or CI checks yet.

https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/about-pull-requests#draft-pull-requests

We also want emails added to the email archive when
the pull request is either abandoned or merged.
merify can add comments to a PR that are picked up
by the webhook.

I agree with reducing the number of services required.
There was feedback from Laszlo related to rebase for
pull requests using the current CI process.  I will
do more investigations of GitHub features, webhook
features, and Mergify features to see if there is 
simpler overall solution.

Mike

> -----Original Message-----
> From: Rebecca Cran <rebecca@bsdio.com>
> Sent: Sunday, May 10, 2020 2:44 PM
> To: devel@edk2.groups.io; Kinney, Michael D
> <michael.d.kinney@intel.com>; rfc@edk2.groups.io
> Subject: Re: [edk2-devel] [edk2-rfc] GitHub Pull
> Request based Code Review Process
> 
> Mike,
> 
> On 5/10/20 3:29 PM, Michael D Kinney wrote:
> 
> > There is no difference between CI checks run during
> code review
> > and the final CI checks before merge.  I think it is
> an interesting
> > conversation to decide how many times those CI checks
> should be
> > run and if they should run automatically on every
> change during
> > review or on demand.
> 
> I'd suggest following what other Github projects do,
> which I think is to
> run the CI checks automatically on every change that's
> made in a pull
> request - I don't know if it might also be necessary to
> run them during
> the merge, if master has changed in the meantime. That
> gives the
> _submitter_ feedback about any changes they need to
> make, instead of
> having to wait until the maintainer tells them their
> change has broken
> something: it speeds up the development process.
> 
> > Mergify is more flexible.  We want to make sure the
> git history
> > is linear with not git merges and supports both
> single patches
> > and patch series without squashing.  GitHub merge
> button by
> > default squashes all commits into a single commit.
> 
> Wouldn't disabling all but "Allow rebase merging" do
> the same thing
> without the additional potential failure point? Though
> it sounds like
> we've resolved the problems with Mergify, so it's not
> important.
> 
> https://help.github.com/en/github/administering-a-
> repository/configuring-commit-squashing-for-pull-
> requests
> 
> 
> --
> Rebecca Cran
> 

Re: [edk2-rfc] GitHub Pull Request based Code Review Process

[Michael D Kinney]

Hello,

I have added the following repository to TianoCore to
support the evaluation of the GitHub pull request based
code review process and the email archive webbook.  This
is a copy of tianocore/edk2 repo as of May 10, 2020. 

	https://github.com/tianocore/edk2-codereview

I have updated Maintainers.txt in this repo to add
GitHub IDs for the maintainers and reviewers.  Please
review these updates to make sure they are correct.

    https://github.com/tianocore/edk2-codereview/blob/master/Maintainers.txt

There are a few maintainers and reviewers that I need
GitHub IDs.  Please send me your GitHub IDs and I will
complete the update of Maintainers.txt.

    M: Chao Zhang <chao.b.zhang@intel.com>
    R: Julien Grall <julien@xen.org>
    R: Marc-André Lureau <marcandre.lureau@redhat.com>
    R: Stefan Berger <stefanb@linux.ibm.com>
    R: Liran Alon <liran.alon@oracle.com>
    R: Nikita Leshenko <nikita.leshchenko@oracle.com>

Thanks,

Mike


> -----Original Message-----
> From: Kinney, Michael D <michael.d.kinney@intel.com>
> Sent: Friday, May 8, 2020 8:00 PM
> To: devel@edk2.groups.io; rfc@edk2.groups.io; Kinney,
> Michael D <michael.d.kinney@intel.com>
> Subject: [edk2-rfc] GitHub Pull Request based Code
> Review Process
> 
> Hello,
> 
> This is a proposal to change from the current email-
> based code review process to
> a GitHub pull request-based code review process for all
> repositories maintained
> in TianoCore.  The current email-based code review
> process and commit message
> requirements are documented in Readme.md or Readme.rst
> at the root of
> repositories along with a few Wiki pages:
> 
> *
> https://github.com/tianocore/edk2/blob/master/ReadMe.rs
> t
> *
> https://github.com/tianocore/tianocore.github.io/wiki/E
> DK-II-Development-Process
> *
> https://github.com/tianocore/tianocore.github.io/wiki/L
> aszlo's-unkempt-git-guide-for-edk2-contributors-and-
> maintainers
> *
> https://github.com/tianocore/tianocore.github.io/wiki/C
> ommit-Message-Format
> *
> https://github.com/tianocore/tianocore.github.io/wiki/C
> ommit-Signature-Format
> 
> The goal is to post changes by opening a GitHub pull
> request and perform all
> code review activity using the GitHub web interface.
> This proposal does not
> change any licenses or commit message requirements.  It
> does require all
> developers, maintainers, and reviewers to have GitHub
> accounts.
> 
> One requirement that was collected from previous
> discussions on this topic is
> the need for an email archive of all patches and code
> review activities.  The
> existing GitHub features to produce an email archive
> were deemed insufficient.
> A proof of concept of a GitHub webhook has been
> implemented to provide the email
> archive service.  This email archive is read-only.  You
> will not be able to send
> emails to this archive or reply to emails in the
> archive.
> 
> The sections below provide more details on the proposed
> GitHub pull request
> based code review process, details on the email archive
> service, and a set of
> remaining tasks make the email archive service
> production quality.  It does not
> make sense to support both the existing email-based
> code review and the GitHub
> pull request-based code review at the same time.
> Instead, this proposal is to
> switch to the GitHub pull request-based code review and
> retire the email based
> code review process on the same date.
> 
> The edk2 repository is using GitHub pull requests today
> to run automated
> CI checks on the code changes and allows a maintainer
> to set the `push` label to
> request the changes to be merged if all CI checks pass.
> With this proposal,
> once the code review is complete and the commit
> messages have been updated, the
> same pull request can be used to perform a final set of
> CI checks and merge the
> changes into the master branch.
> 
> I would like to collect feedback on this proposal and
> the email archive service
> over the next two weeks with close of comments on
> Friday May 22, 2020.  If all
> issues and concerns can be addressed, then I would like
> to see the community
> agree to make this change as soon as all remaining
> tasks are completed.
> 
> # TianoCore Repositories to enable
> 
> * [edk2](https://github.com/tianocore/edk2)
> * [edk2-platforms](https://github.com/tianocore/edk2-
> platforms)
> * [edk2-non-osi](https://github.com/tianocore/edk2-non-
> osi)
> * [edk2-test](https://github.com/tianocore/edk2-test)
> * [edk2-libc](https://github.com/tianocore/edk2-libc)
> * [edk2-staging](https://github.com/tianocore/edk2-
> staging)
> 
> # GitHub Pull Request Code Review Process
> 
> **NOTE**: All steps below use
> [edk2](https://github.com/tianocore/edk2) as an
> example.  Several repositories are supported.
> 
> ## Author/Developer Steps
>   * Create a personal fork of
> [edk2](https://github.com/tianocore/edk2)
> 
>     https://help.github.com/en/github/getting-started-
> with-github/fork-a-repo
> 
>   * Create a new branch from edk2/master in personal
> fork of edk2 repository.
> 
>   * Add set of commits for new feature or bug fix to
> new branch.  Make sure to
>     follow the commit message format requirements.  The
> only change with this
>     RFC is that the Cc: lines to maintainers/reviewers
> should **not** be added.
>     The Cc: lines are still supported, but they should
> only be used to add
>     reviewers that do not have GitHub IDs or are not
> members of TianoCore.
> 
>   * Push branch with new commits to personal fork
>   * Create a pull request against TianoCore edk2/master
> 
>     https://help.github.com/en/github/collaborating-
> with-issues-and-pull-requests/creating-a-pull-request
> 
>   * If pull request has more than 1 commit, then fill
> in the pull request title
>     and decryption information for Patch #0.  Do not
> leave defaults.
> 
>   * Do not assign reviewers.  The webhook assigns
> maintainers and reviewers to
>     the pull request and each commit in the pull
> request.
> 
>   * If maintainers/reviewers provide feedback that
> requires changes, then make
>     add commits to the current branch with the
> requested changes.  Once all
>     changes are accepted on the current branch,
> reformulate the patch series and
>     commit comments as needed for perform a forced push
> to the branch in the
>     personal fork of the edk2 repository.  This step
> may be repeated if multiple
>     versions of the patch series are required to
> address all code review
>     feedback.
> 
>   **OPEN**: How should minimum review period be set?
> Labels?
> 
> ## TianoCore GitHub Email Archive Webhook Service Steps
>   * Receive an event that a new pull request was opened
>   * Evaluate the files modified by the entire pull
> request and each commit in
>     the pull request and cross references against
> `Maintainters.txt` in the root
>     of the repository to assign maintainers/reviewers
> to the pull request and
>     each commit in the pull request. Individual commit
> assignments are performed
>     by adding a commit comment of the following form:
> 
>     [CodeReview] Review-request @mdkinney
> 
>   * Generate and sends git patch review emails to the
> email archive.  Emails
>     are also sent to any Cc: tags in the commit
> messages.
> 
>   * If the author/developer performs a forced push to
> the branch in their
>     personal fork of the edk2 repository, then a new
> set of patch review emails
>     with patch series Vx is sent to the email archive
> and any Cc: tags in commit
>     messages.
> 
>   * Receive events associated with all code review
> activities and generate
>     and send emails to the email archive that shows all
> review comments and
>     all responses closely matching the email contents
> seen in the current email
>     based code review process.
> 
>   * Generate and send email when pull request is merged
> or closed.
> 
> ## Maintainer/Reviewer Steps
> 
>   * Make sure GitHub configuration is setup to 'Watch'
> the repositories that
>     you have maintainer ship or review responsibilities
> and that email
>     notifications from GitHub are enabled.  This
> enables email notifications
>     when a maintainer/reviewer is assigned to a pull
> request and individual
>     commits.
> 
>     https://help.github.com/en/github/managing-
> subscriptions-and-notifications-on-github/configuring-
> notifications
> 
>   * Subscribe to the email archive associated with the
> TianoCore GitHub Email
>     Archive Webhook Service.
> 
>     https://www.redhat.com/mailman/listinfo/tianocore-
> code-review-poc
> 
>   * Review pull requests and commits assigned by the
> TianoCore GitHub Email
>     Archive Webhook Service and use the GitHub web UI
> to provide all review
>     feedback.
> 
>     https://help.github.com/en/github/collaborating-
> with-issues-and-pull-requests/reviewing-changes-in-
> pull-requests
> 
>   * Wait for Author/Developer to respond to all
> feedback and add commits with
>     code changes as needed to resolve all feedback.
> This step may be repeated
>     if the developer/author need to produce multiple
> versions of the patch
>     series to address all feedback.
> 
>   * Once all feedback is addressed, add Reviewed-by,
> Acked-by, and Tested-by
>     responses on individual commits.  Or add Series-
> reviewed-by, Series-acked-by,
>     or Series-Tested-by responses to the entire pull
> request.
> 
>   * Wait for Developer/Author to add tags to commit
> messages in the pull request.
> 
>   * Perform final review of patches and commit message
> tags.  If there are not
>     issues, set the `push` label to run final set of CI
> checks and auto merge
>     the pull request into master.
> 
> # Maintainers.txt Format Changes
> 
> Add GitHub IDs of all maintainers and reviewers at the
> end of M: and R: lines
> in [].  For example:
> 
>     M: Michael D Kinney <michael.d.kinney@intel.com>
> [mdkinney]
> 
> # TianoCore GitHub Email Archive Webhook Service
> 
> Assign reviewers to commits in a GitHub pull request
> based on assignments
> documented in Maintainers.txt and generates an email
> archive of all pull request
> and code review activities.
> 
> https://github.com/mdkinney/edk2-email-archive-webhook
> 
> # Email Archive Subscription Service
> 
> The emails are being delivered to the following RedHat
> email subscription
> service.  Please subscribe to receive the emails and to
> be able to view the
> email archives.
> 
> https://www.redhat.com/mailman/listinfo/tianocore-code-
> review-poc
> 
> The email archives are at this link:
> 
> https://www.redhat.com/mailman/private/tianocore-code-
> review-poc/index.html
> 
> The following sections show some example pull requests
> and code reviews to
> help review the generated emails, their contents, and
> threading.
> 
> ## Email Achieve Thread View
> 
> https://www.redhat.com/mailman/private/tianocore-code-
> review-poc/2020-May/thread.html#00289
> 
> ## Example patch series with 1 patch
> 
> https://www.redhat.com/mailman/private/tianocore-code-
> review-poc/2020-May/thread.html#00340
> 
> ## Example patch series with < 10 patches
> 
> * https://www.redhat.com/mailman/private/tianocore-
> code-review-poc/2020-May/msg00289.html
> * https://www.redhat.com/mailman/private/tianocore-
> code-review-poc/2020-May/msg00030.html
> * https://www.redhat.com/mailman/private/tianocore-
> code-review-poc/2020-May/msg00018.html
> * https://www.redhat.com/mailman/private/tianocore-
> code-review-poc/2020-May/msg00008.html
> 
> ## Example patch series with > 80 patches
> 
> * https://www.redhat.com/mailman/private/tianocore-
> code-review-poc/2020-May/msg00198.html
> * https://www.redhat.com/mailman/private/tianocore-
> code-review-poc/2020-May/msg00116.html
> * https://www.redhat.com/mailman/private/tianocore-
> code-review-poc/2020-May/msg00035.html
> 
> # Tasks to Complete
> 
> * Create edk2-codereview repository for evaluation of
> new code review process.
> * Add GitHub IDs to Maintainers.txt in edk2-codereview
> repository
> * Update BaseTools/Scripts/GetMaintainer.py to be
> compatible with GitHub IDs at
>   the end of M: and R: statements
> * Update webhook to use Rabbit MQ to manage requests
> and emails
> * Determine if webhook requests must be serialized?
> Current POC is serialized.
> * Make sure webhook has error handling for all
> unexpected events/states.
> * Add logging of all events and emails to webhook
> * Add admin interface to webhook
> * Deploy webhook on a production server with 24/7
> support
> 
> # Ideas for Future Enhancements
> 
> * Run PatchCheck.py before assigning
> maintainers/reviewers.
> * Add a simple check that fails if a single patch spans
> more than one package.
> * Monitor comments for Reviewed-by, Acked-by, Tested-
> by, Series-Reviewed-by,
>   Series-Acked-by, Series-Tested-by made by assigned
> maintainers/reviewers.
>   Once all commits have required tags, auto update
> commit messages in the
>   branch and wait for maintainer to set the `Push`
> label to run CI and auto
>   merge if all CI checks pass.
> 
> Best regards,
> 
> Mike
> 

Re: [edk2-rfc] GitHub Pull Request based Code Review Process

[Michael D Kinney]

Hi Ray,

Comments below.

Mike

> -----Original Message-----
> From: Ni, Ray <ray.ni@intel.com>
> Sent: Friday, May 8, 2020 9:23 PM
> To: rfc@edk2.groups.io; Kinney, Michael D
> <michael.d.kinney@intel.com>; devel@edk2.groups.io
> Subject: RE: [edk2-rfc] GitHub Pull Request based Code
> Review Process
> 
> Mike,
> It's a huge improvement to me as an Outlook user if
> pull-request-based review is enabled!
> 
> Please help me to understand: The pull-request-based
> review has been enabled naturally when edk2
> was migrated to Github. People don't use it because
> it's not accepted by community. Your process
> tries to meet community's needs of achieving all review
> comments in mails so pull-request-based
> review can be accepted by community. Right?

Yes.

> 
> I just subscribed at
> https://www.redhat.com/mailman/listinfo/tianocore-code-
> review-poc with
> empty password.
> I received the confirmation mail and clicked the link
> in the mail to confirm.
> But I waited for ~15 minutes and didn't receive the
> additional mail containing the auto-generated password.
> I went to
> https://www.redhat.com/mailman/private/tianocore-code-
> review-poc/2020-May/thread.html#00289.
> However, the page requires me to enter password.
> Can you please change the setting so that viewing the
> mail achieve doesn't need password?

Laszlo has graciously volunteered to help setup this email
subscription service to help evaluate the POC.  I will let 
him comment on the settings available.  He should also be
able to approve your subscription so you can see the archive
and receive any emails generated by the webhook.

> 
> Please advise me what else I can try. I am happy to try
> as many steps as I can before all the process are
> enabled.
> 
> Thanks,
> Ray
> 
> 
> > -----Original Message-----
> > From: rfc@edk2.groups.io <rfc@edk2.groups.io> On
> Behalf Of Michael D Kinney
> > Sent: Saturday, May 9, 2020 11:00 AM
> > To: devel@edk2.groups.io; rfc@edk2.groups.io; Kinney,
> Michael D <michael.d.kinney@intel.com>
> > Subject: [edk2-rfc] GitHub Pull Request based Code
> Review Process
> >
> > Hello,
> >
> > This is a proposal to change from the current email-
> based code review process to
> > a GitHub pull request-based code review process for
> all repositories maintained
> > in TianoCore.  The current email-based code review
> process and commit message
> > requirements are documented in Readme.md or
> Readme.rst at the root of
> > repositories along with a few Wiki pages:
> >
> > *
> https://github.com/tianocore/edk2/blob/master/ReadMe.rs
> t
> > *
> https://github.com/tianocore/tianocore.github.io/wiki/E
> DK-II-Development-Process
> > *
> https://github.com/tianocore/tianocore.github.io/wiki/L
> aszlo's-unkempt-git-guide-for-edk2-contributors-and-
> maintainers
> > *
> https://github.com/tianocore/tianocore.github.io/wiki/C
> ommit-Message-Format
> > *
> https://github.com/tianocore/tianocore.github.io/wiki/C
> ommit-Signature-Format
> >
> > The goal is to post changes by opening a GitHub pull
> request and perform all
> > code review activity using the GitHub web interface.
> This proposal does not
> > change any licenses or commit message requirements.
> It does require all
> > developers, maintainers, and reviewers to have GitHub
> accounts.
> >
> > One requirement that was collected from previous
> discussions on this topic is
> > the need for an email archive of all patches and code
> review activities.  The
> > existing GitHub features to produce an email archive
> were deemed insufficient.
> > A proof of concept of a GitHub webhook has been
> implemented to provide the email
> > archive service.  This email archive is read-only.
> You will not be able to send
> > emails to this archive or reply to emails in the
> archive.
> >
> > The sections below provide more details on the
> proposed GitHub pull request
> > based code review process, details on the email
> archive service, and a set of
> > remaining tasks make the email archive service
> production quality.  It does not
> > make sense to support both the existing email-based
> code review and the GitHub
> > pull request-based code review at the same time.
> Instead, this proposal is to
> > switch to the GitHub pull request-based code review
> and retire the email based
> > code review process on the same date.
> >
> > The edk2 repository is using GitHub pull requests
> today to run automated
> > CI checks on the code changes and allows a maintainer
> to set the `push` label to
> > request the changes to be merged if all CI checks
> pass.  With this proposal,
> > once the code review is complete and the commit
> messages have been updated, the
> > same pull request can be used to perform a final set
> of CI checks and merge the
> > changes into the master branch.
> >
> > I would like to collect feedback on this proposal and
> the email archive service
> > over the next two weeks with close of comments on
> Friday May 22, 2020.  If all
> > issues and concerns can be addressed, then I would
> like to see the community
> > agree to make this change as soon as all remaining
> tasks are completed.
> >
> > # TianoCore Repositories to enable
> >
> > * [edk2](https://github.com/tianocore/edk2)
> > * [edk2-platforms](https://github.com/tianocore/edk2-
> platforms)
> > * [edk2-non-osi](https://github.com/tianocore/edk2-
> non-osi)
> > * [edk2-test](https://github.com/tianocore/edk2-test)
> > * [edk2-libc](https://github.com/tianocore/edk2-libc)
> > * [edk2-staging](https://github.com/tianocore/edk2-
> staging)
> >
> > # GitHub Pull Request Code Review Process
> >
> > **NOTE**: All steps below use
> [edk2](https://github.com/tianocore/edk2) as an
> > example.  Several repositories are supported.
> >
> > ## Author/Developer Steps
> >   * Create a personal fork of
> [edk2](https://github.com/tianocore/edk2)
> >
> >     https://help.github.com/en/github/getting-
> started-with-github/fork-a-repo
> >
> >   * Create a new branch from edk2/master in personal
> fork of edk2 repository.
> >
> >   * Add set of commits for new feature or bug fix to
> new branch.  Make sure to
> >     follow the commit message format requirements.
> The only change with this
> >     RFC is that the Cc: lines to
> maintainers/reviewers should **not** be added.
> >     The Cc: lines are still supported, but they
> should only be used to add
> >     reviewers that do not have GitHub IDs or are not
> members of TianoCore.
> >
> >   * Push branch with new commits to personal fork
> >   * Create a pull request against TianoCore
> edk2/master
> >
> >     https://help.github.com/en/github/collaborating-
> with-issues-and-pull-requests/creating-a-pull-request
> >
> >   * If pull request has more than 1 commit, then fill
> in the pull request title
> >     and decryption information for Patch #0.  Do not
> leave defaults.
> >
> >   * Do not assign reviewers.  The webhook assigns
> maintainers and reviewers to
> >     the pull request and each commit in the pull
> request.
> >
> >   * If maintainers/reviewers provide feedback that
> requires changes, then make
> >     add commits to the current branch with the
> requested changes.  Once all
> >     changes are accepted on the current branch,
> reformulate the patch series and
> >     commit comments as needed for perform a forced
> push to the branch in the
> >     personal fork of the edk2 repository.  This step
> may be repeated if multiple
> >     versions of the patch series are required to
> address all code review
> >     feedback.
> >
> >   **OPEN**: How should minimum review period be set?
> Labels?
> >
> > ## TianoCore GitHub Email Archive Webhook Service
> Steps
> >   * Receive an event that a new pull request was
> opened
> >   * Evaluate the files modified by the entire pull
> request and each commit in
> >     the pull request and cross references against
> `Maintainters.txt` in the root
> >     of the repository to assign maintainers/reviewers
> to the pull request and
> >     each commit in the pull request. Individual
> commit assignments are performed
> >     by adding a commit comment of the following form:
> >
> >     [CodeReview] Review-request @mdkinney
> >
> >   * Generate and sends git patch review emails to the
> email archive.  Emails
> >     are also sent to any Cc: tags in the commit
> messages.
> >
> >   * If the author/developer performs a forced push to
> the branch in their
> >     personal fork of the edk2 repository, then a new
> set of patch review emails
> >     with patch series Vx is sent to the email archive
> and any Cc: tags in commit
> >     messages.
> >
> >   * Receive events associated with all code review
> activities and generate
> >     and send emails to the email archive that shows
> all review comments and
> >     all responses closely matching the email contents
> seen in the current email
> >     based code review process.
> >
> >   * Generate and send email when pull request is
> merged or closed.
> >
> > ## Maintainer/Reviewer Steps
> >
> >   * Make sure GitHub configuration is setup to
> 'Watch' the repositories that
> >     you have maintainer ship or review
> responsibilities and that email
> >     notifications from GitHub are enabled.  This
> enables email notifications
> >     when a maintainer/reviewer is assigned to a pull
> request and individual
> >     commits.
> >
> >     https://help.github.com/en/github/managing-
> subscriptions-and-notifications-on-github/configuring-
> notifications
> >
> >   * Subscribe to the email archive associated with
> the TianoCore GitHub Email
> >     Archive Webhook Service.
> >
> >
> https://www.redhat.com/mailman/listinfo/tianocore-code-
> review-poc
> >
> >   * Review pull requests and commits assigned by the
> TianoCore GitHub Email
> >     Archive Webhook Service and use the GitHub web UI
> to provide all review
> >     feedback.
> >
> >     https://help.github.com/en/github/collaborating-
> with-issues-and-pull-requests/reviewing-changes-in-
> pull-requests
> >
> >   * Wait for Author/Developer to respond to all
> feedback and add commits with
> >     code changes as needed to resolve all feedback.
> This step may be repeated
> >     if the developer/author need to produce multiple
> versions of the patch
> >     series to address all feedback.
> >
> >   * Once all feedback is addressed, add Reviewed-by,
> Acked-by, and Tested-by
> >     responses on individual commits.  Or add Series-
> reviewed-by, Series-acked-by,
> >     or Series-Tested-by responses to the entire pull
> request.
> >
> >   * Wait for Developer/Author to add tags to commit
> messages in the pull request.
> >
> >   * Perform final review of patches and commit
> message tags.  If there are not
> >     issues, set the `push` label to run final set of
> CI checks and auto merge
> >     the pull request into master.
> >
> > # Maintainers.txt Format Changes
> >
> > Add GitHub IDs of all maintainers and reviewers at
> the end of M: and R: lines
> > in [].  For example:
> >
> >     M: Michael D Kinney <michael.d.kinney@intel.com>
> [mdkinney]
> >
> > # TianoCore GitHub Email Archive Webhook Service
> >
> > Assign reviewers to commits in a GitHub pull request
> based on assignments
> > documented in Maintainers.txt and generates an email
> archive of all pull request
> > and code review activities.
> >
> > https://github.com/mdkinney/edk2-email-archive-
> webhook
> >
> > # Email Archive Subscription Service
> >
> > The emails are being delivered to the following
> RedHat email subscription
> > service.  Please subscribe to receive the emails and
> to be able to view the
> > email archives.
> >
> > https://www.redhat.com/mailman/listinfo/tianocore-
> code-review-poc
> >
> > The email archives are at this link:
> >
> > https://www.redhat.com/mailman/private/tianocore-
> code-review-poc/index.html
> >
> > The following sections show some example pull
> requests and code reviews to
> > help review the generated emails, their contents, and
> threading.
> >
> > ## Email Achieve Thread View
> >
> > https://www.redhat.com/mailman/private/tianocore-
> code-review-poc/2020-May/thread.html#00289
> >
> > ## Example patch series with 1 patch
> >
> > https://www.redhat.com/mailman/private/tianocore-
> code-review-poc/2020-May/thread.html#00340
> >
> > ## Example patch series with < 10 patches
> >
> > * https://www.redhat.com/mailman/private/tianocore-
> code-review-poc/2020-May/msg00289.html
> > * https://www.redhat.com/mailman/private/tianocore-
> code-review-poc/2020-May/msg00030.html
> > * https://www.redhat.com/mailman/private/tianocore-
> code-review-poc/2020-May/msg00018.html
> > * https://www.redhat.com/mailman/private/tianocore-
> code-review-poc/2020-May/msg00008.html
> >
> > ## Example patch series with > 80 patches
> >
> > * https://www.redhat.com/mailman/private/tianocore-
> code-review-poc/2020-May/msg00198.html
> > * https://www.redhat.com/mailman/private/tianocore-
> code-review-poc/2020-May/msg00116.html
> > * https://www.redhat.com/mailman/private/tianocore-
> code-review-poc/2020-May/msg00035.html
> >
> > # Tasks to Complete
> >
> > * Create edk2-codereview repository for evaluation of
> new code review process.
> > * Add GitHub IDs to Maintainers.txt in edk2-
> codereview repository
> > * Update BaseTools/Scripts/GetMaintainer.py to be
> compatible with GitHub IDs at
> >   the end of M: and R: statements
> > * Update webhook to use Rabbit MQ to manage requests
> and emails
> > * Determine if webhook requests must be serialized?
> Current POC is serialized.
> > * Make sure webhook has error handling for all
> unexpected events/states.
> > * Add logging of all events and emails to webhook
> > * Add admin interface to webhook
> > * Deploy webhook on a production server with 24/7
> support
> >
> > # Ideas for Future Enhancements
> >
> > * Run PatchCheck.py before assigning
> maintainers/reviewers.
> > * Add a simple check that fails if a single patch
> spans more than one package.
> > * Monitor comments for Reviewed-by, Acked-by, Tested-
> by, Series-Reviewed-by,
> >   Series-Acked-by, Series-Tested-by made by assigned
> maintainers/reviewers.
> >   Once all commits have required tags, auto update
> commit messages in the
> >   branch and wait for maintainer to set the `Push`
> label to run CI and auto
> >   merge if all CI checks pass.
> >
> > Best regards,
> >
> > Mike
> >
> >
> >
> > 
> 

Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

[Laszlo Ersek]

On 05/09/20 04:59, Michael D Kinney wrote:
> Hello,
> 
> This is a proposal to change from the current email-based code review process to
> a GitHub pull request-based code review process for all repositories maintained
> in TianoCore.  The current email-based code review process and commit message
> requirements are documented in Readme.md or Readme.rst at the root of
> repositories along with a few Wiki pages:
> 
> * https://github.com/tianocore/edk2/blob/master/ReadMe.rst
> * https://github.com/tianocore/tianocore.github.io/wiki/EDK-II-Development-Process
> * https://github.com/tianocore/tianocore.github.io/wiki/Laszlo's-unkempt-git-guide-for-edk2-contributors-and-maintainers
> * https://github.com/tianocore/tianocore.github.io/wiki/Commit-Message-Format
> * https://github.com/tianocore/tianocore.github.io/wiki/Commit-Signature-Format
> 
> The goal is to post changes by opening a GitHub pull request and perform all
> code review activity using the GitHub web interface.  This proposal does not
> change any licenses or commit message requirements.  It does require all
> developers, maintainers, and reviewers to have GitHub accounts.
> 
> One requirement that was collected from previous discussions on this topic is
> the need for an email archive of all patches and code review activities.  The
> existing GitHub features to produce an email archive were deemed insufficient.
> A proof of concept of a GitHub webhook has been implemented to provide the email
> archive service.  This email archive is read-only.  You will not be able to send
> emails to this archive or reply to emails in the archive.
> 
> The sections below provide more details on the proposed GitHub pull request
> based code review process, details on the email archive service, and a set of
> remaining tasks make the email archive service production quality.  It does not
> make sense to support both the existing email-based code review and the GitHub
> pull request-based code review at the same time.  Instead, this proposal is to
> switch to the GitHub pull request-based code review and retire the email based
> code review process on the same date.
> 
> The edk2 repository is using GitHub pull requests today to run automated
> CI checks on the code changes and allows a maintainer to set the `push` label to
> request the changes to be merged if all CI checks pass.  With this proposal,
> once the code review is complete and the commit messages have been updated, the
> same pull request can be used to perform a final set of CI checks and merge the
> changes into the master branch.
> 
> I would like to collect feedback on this proposal and the email archive service
> over the next two weeks with close of comments on Friday May 22, 2020.  If all
> issues and concerns can be addressed, then I would like to see the community
> agree to make this change as soon as all remaining tasks are completed.
> 
> # TianoCore Repositories to enable
> 
> * [edk2](https://github.com/tianocore/edk2)
> * [edk2-platforms](https://github.com/tianocore/edk2-platforms)
> * [edk2-non-osi](https://github.com/tianocore/edk2-non-osi)
> * [edk2-test](https://github.com/tianocore/edk2-test)
> * [edk2-libc](https://github.com/tianocore/edk2-libc)
> * [edk2-staging](https://github.com/tianocore/edk2-staging)
> 
> # GitHub Pull Request Code Review Process
> 
> **NOTE**: All steps below use [edk2](https://github.com/tianocore/edk2) as an
> example.  Several repositories are supported.
> 
> ## Author/Developer Steps
>   * Create a personal fork of [edk2](https://github.com/tianocore/edk2)
> 
>     https://help.github.com/en/github/getting-started-with-github/fork-a-repo
> 
>   * Create a new branch from edk2/master in personal fork of edk2 repository.
> 
>   * Add set of commits for new feature or bug fix to new branch.  Make sure to
>     follow the commit message format requirements.  The only change with this
>     RFC is that the Cc: lines to maintainers/reviewers should **not** be added.
>     The Cc: lines are still supported, but they should only be used to add
>     reviewers that do not have GitHub IDs or are not members of TianoCore.
> 
>   * Push branch with new commits to personal fork
>   * Create a pull request against TianoCore edk2/master
> 
>     https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/creating-a-pull-request
> 
>   * If pull request has more than 1 commit, then fill in the pull request title
>     and decryption information for Patch #0.  Do not leave defaults.

s/decryption/description/

(Because I'm assuming this will turn into a wiki article at some point.)

> 
>   * Do not assign reviewers.  The webhook assigns maintainers and reviewers to
>     the pull request and each commit in the pull request.
> 
>   * If maintainers/reviewers provide feedback that requires changes, then make
>     add commits to the current branch with the requested changes.  Once all

s/make add/add/

>     changes are accepted on the current branch, reformulate the patch series and
>     commit comments as needed for perform a forced push to the branch in the
>     personal fork of the edk2 repository.  This step may be repeated if multiple
>     versions of the patch series are required to address all code review
>     feedback.

Do I understand correctly that this recommends the contributor first
push incremental patches on top of the series, then do a rebase
(squashing updates as necessary) and finally do a force-push, for the
next round of review?

To me as a reviewer, that's extra work. I'm used to locally comparing
the v(n) patch set to v(n+1) with git-range-diff, and/or with some
personal scripts. I wouldn't encourage incremental patches appended --
even temporarily -- to the branch, because (a) it's extra review work
(it requires me to review something that has zero chance to get into the
git history as-is), and (b) it superficially resembles the
github.com-specific bad practice called "squash on merge", and (c) it
runs the risk that the maintainer responsible for ultimately merging the
series ends up actually merging the incremental (= "fixup") patches in
isolation (without squashing them).

> 
>   **OPEN**: How should minimum review period be set?  Labels?

Not sure about the best tooling. My recommendation would be to require
reviewers to start providing their feedback within one week.

One thing that I find important is that a maintainer can signal "I got
your work in my queue, but I may need more time". And a special case of
that are automated out-of-office responses. I think they are very
helpful (when a contributor feels they are bottlenecked on review), but
I'm not sure how one can configure that via github. I certainly would
not share my out-of-office times with github. (I set the start/end dates
in my email infrastructure, at the moment, but the out-of-office
messages it sends do not contain the dates either, on purpose.)

> 
> ## TianoCore GitHub Email Archive Webhook Service Steps
>   * Receive an event that a new pull request was opened
>   * Evaluate the files modified by the entire pull request and each commit in
>     the pull request and cross references against `Maintainters.txt` in the root

s/cross references/cross reference them/ ?

>     of the repository to assign maintainers/reviewers to the pull request and
>     each commit in the pull request. Individual commit assignments are performed
>     by adding a commit comment of the following form:
> 
>     [CodeReview] Review-request @mdkinney
> 
>   * Generate and sends git patch review emails to the email archive.  Emails

s/sends/send/

>     are also sent to any Cc: tags in the commit messages.
> 
>   * If the author/developer performs a forced push to the branch in their
>     personal fork of the edk2 repository, then a new set of patch review emails
>     with patch series Vx is sent to the email archive and any Cc: tags in commit
>     messages.
> 
>   * Receive events associated with all code review activities and generate
>     and send emails to the email archive that shows all review comments and
>     all responses closely matching the email contents seen in the current email
>     based code review process.
> 
>   * Generate and send email when pull request is merged or closed.
> 
> ## Maintainer/Reviewer Steps
> 
>   * Make sure GitHub configuration is setup to 'Watch' the repositories that
>     you have maintainer ship or review responsibilities and that email

s/maintainer ship/maintainership/

>     notifications from GitHub are enabled.  This enables email notifications
>     when a maintainer/reviewer is assigned to a pull request and individual
>     commits.
> 
>     https://help.github.com/en/github/managing-subscriptions-and-notifications-on-github/configuring-notifications
> 
>   * Subscribe to the email archive associated with the TianoCore GitHub Email
>     Archive Webhook Service.
> 
>     https://www.redhat.com/mailman/listinfo/tianocore-code-review-poc

Important: as the name says ("-poc"), this is a Proof of Concept list,
for now. Once we're ready to switch over, I'll file an internal ticket
at RH to either rename the list, or (which is probably better) to create
a new list (no "-poc" suffix).

The second option seems more useful because then the webhook development
/ bugfixing (if any) could perhaps occur in parallel to the normal edk2
workflow.

> 
>   * Review pull requests and commits assigned by the TianoCore GitHub Email
>     Archive Webhook Service and use the GitHub web UI to provide all review
>     feedback.
> 
>     https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/reviewing-changes-in-pull-requests
> 
>   * Wait for Author/Developer to respond to all feedback and add commits with
>     code changes as needed to resolve all feedback.  This step may be repeated
>     if the developer/author need to produce multiple versions of the patch
>     series to address all feedback.

(same question about the incremental fixup patches as above)

> 
>   * Once all feedback is addressed, add Reviewed-by, Acked-by, and Tested-by
>     responses on individual commits.  Or add Series-reviewed-by, Series-acked-by,
>     or Series-Tested-by responses to the entire pull request.
> 
>   * Wait for Developer/Author to add tags to commit messages in the pull request.
> 
>   * Perform final review of patches and commit message tags.  If there are not
>     issues, set the `push` label to run final set of CI checks and auto merge
>     the pull request into master.
> 
> # Maintainers.txt Format Changes
> 
> Add GitHub IDs of all maintainers and reviewers at the end of M: and R: lines
> in [].  For example:
> 
>     M: Michael D Kinney <michael.d.kinney@intel.com> [mdkinney]
> 
> # TianoCore GitHub Email Archive Webhook Service
> 
> Assign reviewers to commits in a GitHub pull request based on assignments
> documented in Maintainers.txt and generates an email archive of all pull request
> and code review activities.

s/generates/generate/

(or s/Assign/Assigns/)

> 
> https://github.com/mdkinney/edk2-email-archive-webhook
> 
> # Email Archive Subscription Service
> 
> The emails are being delivered to the following RedHat email subscription
> service.  Please subscribe to receive the emails and to be able to view the
> email archives.
> 
> https://www.redhat.com/mailman/listinfo/tianocore-code-review-poc
> 
> The email archives are at this link:
> 
> https://www.redhat.com/mailman/private/tianocore-code-review-poc/index.html
> 
> The following sections show some example pull requests and code reviews to
> help review the generated emails, their contents, and threading.
> 
> ## Email Achieve Thread View
> 
> https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/thread.html#00289
> 
> ## Example patch series with 1 patch
> 
> https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/thread.html#00340
> 
> ## Example patch series with < 10 patches
> 
> * https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00289.html
> * https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00030.html
> * https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00018.html
> * https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00008.html
> 
> ## Example patch series with > 80 patches
> 
> * https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00198.html
> * https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00116.html
> * https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00035.html
> 
> # Tasks to Complete
> 
> * Create edk2-codereview repository for evaluation of new code review process.
> * Add GitHub IDs to Maintainers.txt in edk2-codereview repository
> * Update BaseTools/Scripts/GetMaintainer.py to be compatible with GitHub IDs at
>   the end of M: and R: statements
> * Update webhook to use Rabbit MQ to manage requests and emails
> * Determine if webhook requests must be serialized?  Current POC is serialized.
> * Make sure webhook has error handling for all unexpected events/states.
> * Add logging of all events and emails to webhook

The logging sounds very useful, thank you.

Whenever a log message relates to an email, please consider logging the
message-id of that email, if possible.

> * Add admin interface to webhook
> * Deploy webhook on a production server with 24/7 support
> 
> # Ideas for Future Enhancements
> 
> * Run PatchCheck.py before assigning maintainers/reviewers.
> * Add a simple check that fails if a single patch spans more than one package.

Hmmm, good idea in general, but there have been valid exceptions to this
rule.

> * Monitor comments for Reviewed-by, Acked-by, Tested-by, Series-Reviewed-by,
>   Series-Acked-by, Series-Tested-by made by assigned maintainers/reviewers.
>   Once all commits have required tags, auto update commit messages in the
>   branch and wait for maintainer to set the `Push` label to run CI and auto
>   merge if all CI checks pass.

Thank you for writing this up (and for implementing the webhook)!
Laszlo

Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

[Laszlo Ersek]

On 05/09/20 06:22, Ni, Ray wrote:
> Mike,
> It's a huge improvement to me as an Outlook user if pull-request-based review is enabled!
> 
> Please help me to understand: The pull-request-based review has been enabled naturally when edk2
> was migrated to Github. People don't use it because it's not accepted by community. Your process
> tries to meet community's needs of achieving all review comments in mails so pull-request-based
> review can be accepted by community. Right?
> 
> I just subscribed at https://www.redhat.com/mailman/listinfo/tianocore-code-review-poc with
> empty password.
> I received the confirmation mail and clicked the link in the mail to confirm.
> But I waited for ~15 minutes and didn't receive the additional mail containing the auto-generated password.

That's because the proof-of-concept list is currently subscriber-only,
and subscription requests have to be manually approved -- by Phil, or by
me. The PoC list contains a bunch of webhook test messages, and while
they are not secret, they are not useful to the grand public (and
arguably shouldn't be indexed by web search engines either).

Once we go live, the intent is that production list be publicly visible.
(Of course spam could become a problem; we'll see.)

Importantly, I totally don't "insist" that the email archive be hosted
on redhat.com (in fact it's extra moderation work for me, which I don't
necessarily welcome); I just offered because Red Hat associates can
request such public-facing mailing lists if they support relevant open
source development efforts.

The traffic should be federated to multiple lists, preferably, and the
redhat.com-hosted list need not be the primary archive address. Wherever
the primary list will live, we can subscribe the mail-archive.com daemon
to it, too.

> I went to https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/thread.html#00289.
> However, the page requires me to enter password.
> Can you please change the setting so that viewing the mail achieve doesn't need password?

The password protection should remain in place for now, I think. I've
approved your subscription request; sorry about the delay. (I avoid
reading work email on the weekend.)

Thanks!
Laszlo

Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

[Laszlo Ersek]

On 05/10/20 23:29, Michael D Kinney wrote:
> Rebecca,
> 
> There is no difference between CI checks run during code review
> and the final CI checks before merge.  I think it is an interesting
> conversation to decide how many times those CI checks should be 
> run and if they should run automatically on every change during
> review or on demand.
> 
> Mergify is more flexible.  We want to make sure the git history
> is linear with not git merges and supports both single patches
> and patch series without squashing.  GitHub merge button by 
> default squashes all commits into a single commit.

(

Wow, "squash-on-merge" is even the *default* now? That's terrible.
Unfortunately, github.com sets a very bad example with this, which is
made worse by github's popularity.

How can we expect developers to think about bisectability and patch
series structuring as first class traits of their contributions if
github.com actively educates them to ignore those aspects? Shaking my head.

)

Laszlo

Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

[Laszlo Ersek]

On 05/10/20 23:43, Rebecca Cran wrote:
> Mike,
> 
> On 5/10/20 3:29 PM, Michael D Kinney wrote:
> 
>> There is no difference between CI checks run during code review
>> and the final CI checks before merge.  I think it is an interesting
>> conversation to decide how many times those CI checks should be
>> run and if they should run automatically on every change during
>> review or on demand.
> 
> I'd suggest following what other Github projects do, which I think is to
> run the CI checks automatically on every change that's made in a pull
> request - I don't know if it might also be necessary to run them during
> the merge, if master has changed in the meantime. That gives the
> _submitter_ feedback about any changes they need to make, instead of
> having to wait until the maintainer tells them their change has broken
> something: it speeds up the development process.

Build-testing at every stage through a patch series is important for
ensuring bisectability.

But there's a critical ingredient to that: based on the assumption that
our build system / build rules are good, the builds mentioned above
should be *incremental*.

That is, if we have a patch set with 10 patches, then then the first
patch in the series should trigger a complete build, and the 9 later
patches should trigger only incremental builds.

(During a bisection, the same commits wouldn't be visited in that same
order of course, but that's where the sanity of the build system / build
rules comes in! Basically, if your builds succeed with a linear
progression through the series, then the build system / build rules
ought to *guarantee* that the same "tree states" will build
incrementally just fine when visited in any particular order. "git
checkout" updates the relevant files, and the build system should be
able to derive the minimum set of necessary actions.

Anyway, digression ends.)

The incremental nature of builds is important for saving energy, and
also for saving developer time. The above 10-part example series should
not take 10 times as long to build as 10 independent patches, submitted
in isolation. Patches#2 through #10 should only rebuild a few modules
each (unless lib class headers, protocol headers and such are modified).


> 
>> Mergify is more flexible.  We want to make sure the git history
>> is linear with not git merges and supports both single patches
>> and patch series without squashing.  GitHub merge button by
>> default squashes all commits into a single commit.
> 
> Wouldn't disabling all but "Allow rebase merging" do the same thing
> without the additional potential failure point? Though it sounds like
> we've resolved the problems with Mergify, so it's not important.
> 
> https://help.github.com/en/github/administering-a-repository/configuring-commit-squashing-for-pull-requests

mergify has been pretty stable for me!

Thanks,
Laszlo

Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

[Laszlo Ersek]

On 05/11/20 03:37, Michael D Kinney wrote:

> There was feedback from Laszlo related to rebase for
> pull requests using the current CI process.

To clarify, I don't think we should allow any github-side automatism to
auto-rebase pull requests. I think such rebases need to occur on
personal developer machines, under human oversight, and then resubmitted
(likely: force-pushed). My request is that the build costs (time,
energy) associated with such force-pushes be reduced somehow.

For example, on a local machine, the following sequence:

$ git checkout master
$ git pull
$ git rebase -i master my_topic_branch
$ build ...

would trigger an incremental build. *.c files not touched by either
operation would not have to be re-built (assuming their dependencies
didn't change either, such as lib class headers, protocol headers, ...)

Thanks,
Laszlo

Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

[Bret Barkelew]

[-- Attachment #1: Type: text/plain, Size: 23706 bytes --]

As a counterpoint: if we force a new branch or force push on every tweak, we lose the “thread” of discussion on what caused the change, what changed as a result, and the easy hook for the original change requester to reply directly to the change as is.

- Bret

From: Laszlo Ersek via groups.io<mailto:lersek=redhat.com@groups.io>
Sent: Monday, May 11, 2020 12:39 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Kinney, Michael D<mailto:michael.d.kinney@intel.com>; rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>
Subject: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

On 05/09/20 04:59, Michael D Kinney wrote:
> Hello,
>
> This is a proposal to change from the current email-based code review process to
> a GitHub pull request-based code review process for all repositories maintained
> in TianoCore.  The current email-based code review process and commit message
> requirements are documented in Readme.md or Readme.rst at the root of
> repositories along with a few Wiki pages:
>
> * https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fblob%2Fmaster%2FReadMe.rst&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=lVjWRLsBC3xJpyRFeDrGjFhMOzAgi2V3vsAPxj7lIDw%3D&amp;reserved=0
> * https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FEDK-II-Development-Process&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=sgAhQxCpyjmzC%2FW%2BFiLLwaF2M8wscBz3k93ne25qUXs%3D&amp;reserved=0
> * https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FLaszlo%27s-unkempt-git-guide-for-edk2-contributors-and-maintainers&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=eHP9fcPMw6yjqTU%2B%2BUZ3FZkq8jZeM1LTU6dGTzmFp4Q%3D&amp;reserved=0
> * https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FCommit-Message-Format&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=uq8G6nGyLpa7m%2F0fD2pwrcM9uixbKs6SLTge8e77M%2FY%3D&amp;reserved=0
> * https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FCommit-Signature-Format&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Mz8dUn2L8dFwJdlo4LbaIKt2JrWE%2Fn4tBtVWenK%2F8Ck%3D&amp;reserved=0
>
> The goal is to post changes by opening a GitHub pull request and perform all
> code review activity using the GitHub web interface.  This proposal does not
> change any licenses or commit message requirements.  It does require all
> developers, maintainers, and reviewers to have GitHub accounts.
>
> One requirement that was collected from previous discussions on this topic is
> the need for an email archive of all patches and code review activities.  The
> existing GitHub features to produce an email archive were deemed insufficient.
> A proof of concept of a GitHub webhook has been implemented to provide the email
> archive service.  This email archive is read-only.  You will not be able to send
> emails to this archive or reply to emails in the archive.
>
> The sections below provide more details on the proposed GitHub pull request
> based code review process, details on the email archive service, and a set of
> remaining tasks make the email archive service production quality.  It does not
> make sense to support both the existing email-based code review and the GitHub
> pull request-based code review at the same time.  Instead, this proposal is to
> switch to the GitHub pull request-based code review and retire the email based
> code review process on the same date.
>
> The edk2 repository is using GitHub pull requests today to run automated
> CI checks on the code changes and allows a maintainer to set the `push` label to
> request the changes to be merged if all CI checks pass.  With this proposal,
> once the code review is complete and the commit messages have been updated, the
> same pull request can be used to perform a final set of CI checks and merge the
> changes into the master branch.
>
> I would like to collect feedback on this proposal and the email archive service
> over the next two weeks with close of comments on Friday May 22, 2020.  If all
> issues and concerns can be addressed, then I would like to see the community
> agree to make this change as soon as all remaining tasks are completed.
>
> # TianoCore Repositories to enable
>
> * [edk2](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0)
> * [edk2-platforms](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-platforms&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=g8mgGL6B%2FRsvm3935OpZMctOTKUoeHGi8jPuCVKQjbI%3D&amp;reserved=0)
> * [edk2-non-osi](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-non-osi&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=9lrEsZWOpc3wqylKs7yF%2FzxYwZsUUamP3oUrWDWcHCc%3D&amp;reserved=0)
> * [edk2-test](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-test&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=8v205MD3HTYg3yLmGJS3SIDA5um9sVJfOa5CXViZjyU%3D&amp;reserved=0)
> * [edk2-libc](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-libc&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Tzt293HJzFnGSkh1mUBew8dAsaZ4axWq2ml8UhQ%2FSTI%3D&amp;reserved=0)
> * [edk2-staging](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-staging&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=bcNbt7Y7KoBrcnW4fAc4jbGgJL%2B4lYUkVLhYNo37OiM%3D&amp;reserved=0)
>
> # GitHub Pull Request Code Review Process
>
> **NOTE**: All steps below use [edk2](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0) as an
> example.  Several repositories are supported.
>
> ## Author/Developer Steps
>   * Create a personal fork of [edk2](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0)
>
>     https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fgetting-started-with-github%2Ffork-a-repo&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=umI3eqOh03qmt9YlPo33ujypu90YwImAvuxh5SlrM%2Bw%3D&amp;reserved=0
>
>   * Create a new branch from edk2/master in personal fork of edk2 repository.
>
>   * Add set of commits for new feature or bug fix to new branch.  Make sure to
>     follow the commit message format requirements.  The only change with this
>     RFC is that the Cc: lines to maintainers/reviewers should **not** be added.
>     The Cc: lines are still supported, but they should only be used to add
>     reviewers that do not have GitHub IDs or are not members of TianoCore.
>
>   * Push branch with new commits to personal fork
>   * Create a pull request against TianoCore edk2/master
>
>     https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fcollaborating-with-issues-and-pull-requests%2Fcreating-a-pull-request&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=2GVrQy0FGwd4CCeGveh99HL3zS1ekRfAAaKhhRiOMpU%3D&amp;reserved=0
>
>   * If pull request has more than 1 commit, then fill in the pull request title
>     and decryption information for Patch #0.  Do not leave defaults.

s/decryption/description/

(Because I'm assuming this will turn into a wiki article at some point.)

>
>   * Do not assign reviewers.  The webhook assigns maintainers and reviewers to
>     the pull request and each commit in the pull request.
>
>   * If maintainers/reviewers provide feedback that requires changes, then make
>     add commits to the current branch with the requested changes.  Once all

s/make add/add/

>     changes are accepted on the current branch, reformulate the patch series and
>     commit comments as needed for perform a forced push to the branch in the
>     personal fork of the edk2 repository.  This step may be repeated if multiple
>     versions of the patch series are required to address all code review
>     feedback.

Do I understand correctly that this recommends the contributor first
push incremental patches on top of the series, then do a rebase
(squashing updates as necessary) and finally do a force-push, for the
next round of review?

To me as a reviewer, that's extra work. I'm used to locally comparing
the v(n) patch set to v(n+1) with git-range-diff, and/or with some
personal scripts. I wouldn't encourage incremental patches appended --
even temporarily -- to the branch, because (a) it's extra review work
(it requires me to review something that has zero chance to get into the
git history as-is), and (b) it superficially resembles the
github.com-specific bad practice called "squash on merge", and (c) it
runs the risk that the maintainer responsible for ultimately merging the
series ends up actually merging the incremental (= "fixup") patches in
isolation (without squashing them).

>
>   **OPEN**: How should minimum review period be set?  Labels?

Not sure about the best tooling. My recommendation would be to require
reviewers to start providing their feedback within one week.

One thing that I find important is that a maintainer can signal "I got
your work in my queue, but I may need more time". And a special case of
that are automated out-of-office responses. I think they are very
helpful (when a contributor feels they are bottlenecked on review), but
I'm not sure how one can configure that via github. I certainly would
not share my out-of-office times with github. (I set the start/end dates
in my email infrastructure, at the moment, but the out-of-office
messages it sends do not contain the dates either, on purpose.)

>
> ## TianoCore GitHub Email Archive Webhook Service Steps
>   * Receive an event that a new pull request was opened
>   * Evaluate the files modified by the entire pull request and each commit in
>     the pull request and cross references against `Maintainters.txt` in the root

s/cross references/cross reference them/ ?

>     of the repository to assign maintainers/reviewers to the pull request and
>     each commit in the pull request. Individual commit assignments are performed
>     by adding a commit comment of the following form:
>
>     [CodeReview] Review-request @mdkinney
>
>   * Generate and sends git patch review emails to the email archive.  Emails

s/sends/send/

>     are also sent to any Cc: tags in the commit messages.
>
>   * If the author/developer performs a forced push to the branch in their
>     personal fork of the edk2 repository, then a new set of patch review emails
>     with patch series Vx is sent to the email archive and any Cc: tags in commit
>     messages.
>
>   * Receive events associated with all code review activities and generate
>     and send emails to the email archive that shows all review comments and
>     all responses closely matching the email contents seen in the current email
>     based code review process.
>
>   * Generate and send email when pull request is merged or closed.
>
> ## Maintainer/Reviewer Steps
>
>   * Make sure GitHub configuration is setup to 'Watch' the repositories that
>     you have maintainer ship or review responsibilities and that email

s/maintainer ship/maintainership/

>     notifications from GitHub are enabled.  This enables email notifications
>     when a maintainer/reviewer is assigned to a pull request and individual
>     commits.
>
>     https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fmanaging-subscriptions-and-notifications-on-github%2Fconfiguring-notifications&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=OlkiyymcQi39P8%2FOJZG4yjh4h%2FHerkHBe5bCSQQFLOU%3D&amp;reserved=0
>
>   * Subscribe to the email archive associated with the TianoCore GitHub Email
>     Archive Webhook Service.
>
>     https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Ftianocore-code-review-poc&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=q0IuvS318pEkJU2td9xX87oIm0LbSlEvOvhpyOOFrE8%3D&amp;reserved=0

Important: as the name says ("-poc"), this is a Proof of Concept list,
for now. Once we're ready to switch over, I'll file an internal ticket
at RH to either rename the list, or (which is probably better) to create
a new list (no "-poc" suffix).

The second option seems more useful because then the webhook development
/ bugfixing (if any) could perhaps occur in parallel to the normal edk2
workflow.

>
>   * Review pull requests and commits assigned by the TianoCore GitHub Email
>     Archive Webhook Service and use the GitHub web UI to provide all review
>     feedback.
>
>     https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fcollaborating-with-issues-and-pull-requests%2Freviewing-changes-in-pull-requests&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=51Ljm3wUbBTWT8hcaBD1ZQznSROvAQqnoTzQmD6K%2FLY%3D&amp;reserved=0
>
>   * Wait for Author/Developer to respond to all feedback and add commits with
>     code changes as needed to resolve all feedback.  This step may be repeated
>     if the developer/author need to produce multiple versions of the patch
>     series to address all feedback.

(same question about the incremental fixup patches as above)

>
>   * Once all feedback is addressed, add Reviewed-by, Acked-by, and Tested-by
>     responses on individual commits.  Or add Series-reviewed-by, Series-acked-by,
>     or Series-Tested-by responses to the entire pull request.
>
>   * Wait for Developer/Author to add tags to commit messages in the pull request.
>
>   * Perform final review of patches and commit message tags.  If there are not
>     issues, set the `push` label to run final set of CI checks and auto merge
>     the pull request into master.
>
> # Maintainers.txt Format Changes
>
> Add GitHub IDs of all maintainers and reviewers at the end of M: and R: lines
> in [].  For example:
>
>     M: Michael D Kinney <michael.d.kinney@intel.com> [mdkinney]
>
> # TianoCore GitHub Email Archive Webhook Service
>
> Assign reviewers to commits in a GitHub pull request based on assignments
> documented in Maintainers.txt and generates an email archive of all pull request
> and code review activities.

s/generates/generate/

(or s/Assign/Assigns/)

>
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2-email-archive-webhook&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=7CJNJMEXrxoynjavmEwjzUyRbfNUIZ3FEG4kDRXvhI4%3D&amp;reserved=0
>
> # Email Archive Subscription Service
>
> The emails are being delivered to the following RedHat email subscription
> service.  Please subscribe to receive the emails and to be able to view the
> email archives.
>
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Ftianocore-code-review-poc&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=q0IuvS318pEkJU2td9xX87oIm0LbSlEvOvhpyOOFrE8%3D&amp;reserved=0
>
> The email archives are at this link:
>
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2Findex.html&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=nedUfkmMmI5T6GtAxQCW4q6xt38%2FezeDYmfq6cpRD0M%3D&amp;reserved=0
>
> The following sections show some example pull requests and code reviews to
> help review the generated emails, their contents, and threading.
>
> ## Email Achieve Thread View
>
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fthread.html%2300289&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=GtrEudehfXiSU6ZwH2zKO35CPPPVk0ctZIzhkpI6DkE%3D&amp;reserved=0
>
> ## Example patch series with 1 patch
>
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fthread.html%2300340&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=ZGpI8%2BzIA9OMFm3pSCc2DQ4F5ZxtDSAXtjdFjD%2BY3NA%3D&amp;reserved=0
>
> ## Example patch series with < 10 patches
>
> * https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00289.html&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=JyaUyvYfZD7b%2F2wN%2BpS%2B68b%2BwyKoZ3Rba4ol%2FyahQVU%3D&amp;reserved=0
> * https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00030.html&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=bQHIJIQq4Pri8iK3vPxMDMWz%2BKtXcyuPdhr8y7gFpXA%3D&amp;reserved=0
> * https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00018.html&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=uMIRGOq%2BVCOSwDzXkG4yueYS4ZJ7BWfsp3Z4%2B9lh6hE%3D&amp;reserved=0
> * https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00008.html&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=3CBkdqDxRt4IxtECpWQdKJL%2Bf4HFqqHCXo4loxNTzAE%3D&amp;reserved=0
>
> ## Example patch series with > 80 patches
>
> * https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00198.html&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=fDfQnifOMzyzLMdP4xH8koKCiSj7ZiuYyrrSZXTf3d4%3D&amp;reserved=0
> * https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00116.html&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=lcxA3tTna%2BdmTpcNMmPlS%2B47llMAcIEjhCEqxV7TDOc%3D&amp;reserved=0
> * https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00035.html&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=CgvZ8e%2B7L4nacvRE35KqEyC%2F1CjDYP6wI10qn%2BoX39Y%3D&amp;reserved=0
>
> # Tasks to Complete
>
> * Create edk2-codereview repository for evaluation of new code review process.
> * Add GitHub IDs to Maintainers.txt in edk2-codereview repository
> * Update BaseTools/Scripts/GetMaintainer.py to be compatible with GitHub IDs at
>   the end of M: and R: statements
> * Update webhook to use Rabbit MQ to manage requests and emails
> * Determine if webhook requests must be serialized?  Current POC is serialized.
> * Make sure webhook has error handling for all unexpected events/states.
> * Add logging of all events and emails to webhook

The logging sounds very useful, thank you.

Whenever a log message relates to an email, please consider logging the
message-id of that email, if possible.

> * Add admin interface to webhook
> * Deploy webhook on a production server with 24/7 support
>
> # Ideas for Future Enhancements
>
> * Run PatchCheck.py before assigning maintainers/reviewers.
> * Add a simple check that fails if a single patch spans more than one package.

Hmmm, good idea in general, but there have been valid exceptions to this
rule.

> * Monitor comments for Reviewed-by, Acked-by, Tested-by, Series-Reviewed-by,
>   Series-Acked-by, Series-Tested-by made by assigned maintainers/reviewers.
>   Once all commits have required tags, auto update commit messages in the
>   branch and wait for maintainer to set the `Push` label to run CI and auto
>   merge if all CI checks pass.

Thank you for writing this up (and for implementing the webhook)!
Laszlo





[-- Attachment #2: Type: text/html, Size: 38049 bytes --]

Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

[Michael D Kinney]

[-- Attachment #1: Type: text/plain, Size: 25161 bytes --]

Hi Bret,

This is a good point.

What I am proposing is the first version of the patch series submitted as a pull request.  Let the community do a complete review of the content.  The submitter can add patches to the end of the pull request addressing feedback and can even add patches that make changes to previous patches until all feedback/conversations are resolved.  This keeps the conversations complete and the conversations will also be archived to the email archive.

At this point, the developer can reformulate the patch series and do forced push of V2.  Reviewers can review the cleaned up patch series and repeat the process if there is more feedback, or move to final approval.

By doing all the work on a single pull request, we minimize the total number of pull requests in the repo.

An alternative approach would be to open a new pull request for each new version of the series.  This would preserve the GitHub conversations for each version of the pull request.  All the earlier ones would be closed/abandoned, and only the final one would be closed/merged.

Best regards,

Mike

From: Bret Barkelew <Bret.Barkelew@microsoft.com>
Sent: Monday, May 11, 2020 1:10 PM
To: devel@edk2.groups.io; lersek@redhat.com; Kinney, Michael D <michael.d.kinney@intel.com>; rfc@edk2.groups.io
Subject: RE: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

As a counterpoint: if we force a new branch or force push on every tweak, we lose the “thread” of discussion on what caused the change, what changed as a result, and the easy hook for the original change requester to reply directly to the change as is.

- Bret

From: Laszlo Ersek via groups.io<mailto:lersek=redhat.com@groups.io>
Sent: Monday, May 11, 2020 12:39 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Kinney, Michael D<mailto:michael.d.kinney@intel.com>; rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>
Subject: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

On 05/09/20 04:59, Michael D Kinney wrote:
> Hello,
>
> This is a proposal to change from the current email-based code review process to
> a GitHub pull request-based code review process for all repositories maintained
> in TianoCore.  The current email-based code review process and commit message
> requirements are documented in Readme.md or Readme.rst at the root of
> repositories along with a few Wiki pages:
>
> * https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fblob%2Fmaster%2FReadMe.rst&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=lVjWRLsBC3xJpyRFeDrGjFhMOzAgi2V3vsAPxj7lIDw%3D&amp;reserved=0
> * https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FEDK-II-Development-Process&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=sgAhQxCpyjmzC%2FW%2BFiLLwaF2M8wscBz3k93ne25qUXs%3D&amp;reserved=0
> * https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FLaszlo%27s-unkempt-git-guide-for-edk2-contributors-and-maintainers&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=eHP9fcPMw6yjqTU%2B%2BUZ3FZkq8jZeM1LTU6dGTzmFp4Q%3D&amp;reserved=0
> * https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FCommit-Message-Format&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=uq8G6nGyLpa7m%2F0fD2pwrcM9uixbKs6SLTge8e77M%2FY%3D&amp;reserved=0
> * https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FCommit-Signature-Format&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Mz8dUn2L8dFwJdlo4LbaIKt2JrWE%2Fn4tBtVWenK%2F8Ck%3D&amp;reserved=0
>
> The goal is to post changes by opening a GitHub pull request and perform all
> code review activity using the GitHub web interface.  This proposal does not
> change any licenses or commit message requirements.  It does require all
> developers, maintainers, and reviewers to have GitHub accounts.
>
> One requirement that was collected from previous discussions on this topic is
> the need for an email archive of all patches and code review activities.  The
> existing GitHub features to produce an email archive were deemed insufficient.
> A proof of concept of a GitHub webhook has been implemented to provide the email
> archive service.  This email archive is read-only.  You will not be able to send
> emails to this archive or reply to emails in the archive.
>
> The sections below provide more details on the proposed GitHub pull request
> based code review process, details on the email archive service, and a set of
> remaining tasks make the email archive service production quality.  It does not
> make sense to support both the existing email-based code review and the GitHub
> pull request-based code review at the same time.  Instead, this proposal is to
> switch to the GitHub pull request-based code review and retire the email based
> code review process on the same date.
>
> The edk2 repository is using GitHub pull requests today to run automated
> CI checks on the code changes and allows a maintainer to set the `push` label to
> request the changes to be merged if all CI checks pass.  With this proposal,
> once the code review is complete and the commit messages have been updated, the
> same pull request can be used to perform a final set of CI checks and merge the
> changes into the master branch.
>
> I would like to collect feedback on this proposal and the email archive service
> over the next two weeks with close of comments on Friday May 22, 2020.  If all
> issues and concerns can be addressed, then I would like to see the community
> agree to make this change as soon as all remaining tasks are completed.
>
> # TianoCore Repositories to enable
>
> * [edk2](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0)
> * [edk2-platforms](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-platforms&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=g8mgGL6B%2FRsvm3935OpZMctOTKUoeHGi8jPuCVKQjbI%3D&amp;reserved=0)
> * [edk2-non-osi](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-non-osi&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=9lrEsZWOpc3wqylKs7yF%2FzxYwZsUUamP3oUrWDWcHCc%3D&amp;reserved=0)
> * [edk2-test](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-test&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=8v205MD3HTYg3yLmGJS3SIDA5um9sVJfOa5CXViZjyU%3D&amp;reserved=0)
> * [edk2-libc](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-libc&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Tzt293HJzFnGSkh1mUBew8dAsaZ4axWq2ml8UhQ%2FSTI%3D&amp;reserved=0)
> * [edk2-staging](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-staging&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=bcNbt7Y7KoBrcnW4fAc4jbGgJL%2B4lYUkVLhYNo37OiM%3D&amp;reserved=0)
>
> # GitHub Pull Request Code Review Process
>
> **NOTE**: All steps below use [edk2](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0) as an
> example.  Several repositories are supported.
>
> ## Author/Developer Steps
>   * Create a personal fork of [edk2](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0)
>
>     https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fgetting-started-with-github%2Ffork-a-repo&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=umI3eqOh03qmt9YlPo33ujypu90YwImAvuxh5SlrM%2Bw%3D&amp;reserved=0
>
>   * Create a new branch from edk2/master in personal fork of edk2 repository.
>
>   * Add set of commits for new feature or bug fix to new branch.  Make sure to
>     follow the commit message format requirements.  The only change with this
>     RFC is that the Cc: lines to maintainers/reviewers should **not** be added.
>     The Cc: lines are still supported, but they should only be used to add
>     reviewers that do not have GitHub IDs or are not members of TianoCore.
>
>   * Push branch with new commits to personal fork
>   * Create a pull request against TianoCore edk2/master
>
>     https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fcollaborating-with-issues-and-pull-requests%2Fcreating-a-pull-request&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=2GVrQy0FGwd4CCeGveh99HL3zS1ekRfAAaKhhRiOMpU%3D&amp;reserved=0
>
>   * If pull request has more than 1 commit, then fill in the pull request title
>     and decryption information for Patch #0.  Do not leave defaults.

s/decryption/description/

(Because I'm assuming this will turn into a wiki article at some point.)

>
>   * Do not assign reviewers.  The webhook assigns maintainers and reviewers to
>     the pull request and each commit in the pull request.
>
>   * If maintainers/reviewers provide feedback that requires changes, then make
>     add commits to the current branch with the requested changes.  Once all

s/make add/add/

>     changes are accepted on the current branch, reformulate the patch series and
>     commit comments as needed for perform a forced push to the branch in the
>     personal fork of the edk2 repository.  This step may be repeated if multiple
>     versions of the patch series are required to address all code review
>     feedback.

Do I understand correctly that this recommends the contributor first
push incremental patches on top of the series, then do a rebase
(squashing updates as necessary) and finally do a force-push, for the
next round of review?

To me as a reviewer, that's extra work. I'm used to locally comparing
the v(n) patch set to v(n+1) with git-range-diff, and/or with some
personal scripts. I wouldn't encourage incremental patches appended --
even temporarily -- to the branch, because (a) it's extra review work
(it requires me to review something that has zero chance to get into the
git history as-is), and (b) it superficially resembles the
github.com-specific bad practice called "squash on merge", and (c) it
runs the risk that the maintainer responsible for ultimately merging the
series ends up actually merging the incremental (= "fixup") patches in
isolation (without squashing them).

>
>   **OPEN**: How should minimum review period be set?  Labels?

Not sure about the best tooling. My recommendation would be to require
reviewers to start providing their feedback within one week.

One thing that I find important is that a maintainer can signal "I got
your work in my queue, but I may need more time". And a special case of
that are automated out-of-office responses. I think they are very
helpful (when a contributor feels they are bottlenecked on review), but
I'm not sure how one can configure that via github. I certainly would
not share my out-of-office times with github. (I set the start/end dates
in my email infrastructure, at the moment, but the out-of-office
messages it sends do not contain the dates either, on purpose.)

>
> ## TianoCore GitHub Email Archive Webhook Service Steps
>   * Receive an event that a new pull request was opened
>   * Evaluate the files modified by the entire pull request and each commit in
>     the pull request and cross references against `Maintainters.txt` in the root

s/cross references/cross reference them/ ?

>     of the repository to assign maintainers/reviewers to the pull request and
>     each commit in the pull request. Individual commit assignments are performed
>     by adding a commit comment of the following form:
>
>     [CodeReview] Review-request @mdkinney
>
>   * Generate and sends git patch review emails to the email archive.  Emails

s/sends/send/

>     are also sent to any Cc: tags in the commit messages.
>
>   * If the author/developer performs a forced push to the branch in their
>     personal fork of the edk2 repository, then a new set of patch review emails
>     with patch series Vx is sent to the email archive and any Cc: tags in commit
>     messages.
>
>   * Receive events associated with all code review activities and generate
>     and send emails to the email archive that shows all review comments and
>     all responses closely matching the email contents seen in the current email
>     based code review process.
>
>   * Generate and send email when pull request is merged or closed.
>
> ## Maintainer/Reviewer Steps
>
>   * Make sure GitHub configuration is setup to 'Watch' the repositories that
>     you have maintainer ship or review responsibilities and that email

s/maintainer ship/maintainership/

>     notifications from GitHub are enabled.  This enables email notifications
>     when a maintainer/reviewer is assigned to a pull request and individual
>     commits.
>
>     https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fmanaging-subscriptions-and-notifications-on-github%2Fconfiguring-notifications&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=OlkiyymcQi39P8%2FOJZG4yjh4h%2FHerkHBe5bCSQQFLOU%3D&amp;reserved=0
>
>   * Subscribe to the email archive associated with the TianoCore GitHub Email
>     Archive Webhook Service.
>
>     https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Ftianocore-code-review-poc&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=q0IuvS318pEkJU2td9xX87oIm0LbSlEvOvhpyOOFrE8%3D&amp;reserved=0

Important: as the name says ("-poc"), this is a Proof of Concept list,
for now. Once we're ready to switch over, I'll file an internal ticket
at RH to either rename the list, or (which is probably better) to create
a new list (no "-poc" suffix).

The second option seems more useful because then the webhook development
/ bugfixing (if any) could perhaps occur in parallel to the normal edk2
workflow.

>
>   * Review pull requests and commits assigned by the TianoCore GitHub Email
>     Archive Webhook Service and use the GitHub web UI to provide all review
>     feedback.
>
>     https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fcollaborating-with-issues-and-pull-requests%2Freviewing-changes-in-pull-requests&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=51Ljm3wUbBTWT8hcaBD1ZQznSROvAQqnoTzQmD6K%2FLY%3D&amp;reserved=0
>
>   * Wait for Author/Developer to respond to all feedback and add commits with
>     code changes as needed to resolve all feedback.  This step may be repeated
>     if the developer/author need to produce multiple versions of the patch
>     series to address all feedback.

(same question about the incremental fixup patches as above)

>
>   * Once all feedback is addressed, add Reviewed-by, Acked-by, and Tested-by
>     responses on individual commits.  Or add Series-reviewed-by, Series-acked-by,
>     or Series-Tested-by responses to the entire pull request.
>
>   * Wait for Developer/Author to add tags to commit messages in the pull request.
>
>   * Perform final review of patches and commit message tags.  If there are not
>     issues, set the `push` label to run final set of CI checks and auto merge
>     the pull request into master.
>
> # Maintainers.txt Format Changes
>
> Add GitHub IDs of all maintainers and reviewers at the end of M: and R: lines
> in [].  For example:
>
>     M: Michael D Kinney <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>> [mdkinney]
>
> # TianoCore GitHub Email Archive Webhook Service
>
> Assign reviewers to commits in a GitHub pull request based on assignments
> documented in Maintainers.txt and generates an email archive of all pull request
> and code review activities.

s/generates/generate/

(or s/Assign/Assigns/)

>
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2-email-archive-webhook&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=7CJNJMEXrxoynjavmEwjzUyRbfNUIZ3FEG4kDRXvhI4%3D&amp;reserved=0
>
> # Email Archive Subscription Service
>
> The emails are being delivered to the following RedHat email subscription
> service.  Please subscribe to receive the emails and to be able to view the
> email archives.
>
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Ftianocore-code-review-poc&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=q0IuvS318pEkJU2td9xX87oIm0LbSlEvOvhpyOOFrE8%3D&amp;reserved=0
>
> The email archives are at this link:
>
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2Findex.html&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=nedUfkmMmI5T6GtAxQCW4q6xt38%2FezeDYmfq6cpRD0M%3D&amp;reserved=0
>
> The following sections show some example pull requests and code reviews to
> help review the generated emails, their contents, and threading.
>
> ## Email Achieve Thread View
>
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fthread.html%2300289&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=GtrEudehfXiSU6ZwH2zKO35CPPPVk0ctZIzhkpI6DkE%3D&amp;reserved=0
>
> ## Example patch series with 1 patch
>
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fthread.html%2300340&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=ZGpI8%2BzIA9OMFm3pSCc2DQ4F5ZxtDSAXtjdFjD%2BY3NA%3D&amp;reserved=0
>
> ## Example patch series with < 10 patches
>
> * https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00289.html&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=JyaUyvYfZD7b%2F2wN%2BpS%2B68b%2BwyKoZ3Rba4ol%2FyahQVU%3D&amp;reserved=0
> * https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00030.html&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=bQHIJIQq4Pri8iK3vPxMDMWz%2BKtXcyuPdhr8y7gFpXA%3D&amp;reserved=0
> * https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00018.html&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=uMIRGOq%2BVCOSwDzXkG4yueYS4ZJ7BWfsp3Z4%2B9lh6hE%3D&amp;reserved=0
> * https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00008.html&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=3CBkdqDxRt4IxtECpWQdKJL%2Bf4HFqqHCXo4loxNTzAE%3D&amp;reserved=0
>
> ## Example patch series with > 80 patches
>
> * https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00198.html&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=fDfQnifOMzyzLMdP4xH8koKCiSj7ZiuYyrrSZXTf3d4%3D&amp;reserved=0
> * https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00116.html&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=lcxA3tTna%2BdmTpcNMmPlS%2B47llMAcIEjhCEqxV7TDOc%3D&amp;reserved=0
> * https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00035.html&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=CgvZ8e%2B7L4nacvRE35KqEyC%2F1CjDYP6wI10qn%2BoX39Y%3D&amp;reserved=0
>
> # Tasks to Complete
>
> * Create edk2-codereview repository for evaluation of new code review process.
> * Add GitHub IDs to Maintainers.txt in edk2-codereview repository
> * Update BaseTools/Scripts/GetMaintainer.py to be compatible with GitHub IDs at
>   the end of M: and R: statements
> * Update webhook to use Rabbit MQ to manage requests and emails
> * Determine if webhook requests must be serialized?  Current POC is serialized.
> * Make sure webhook has error handling for all unexpected events/states.
> * Add logging of all events and emails to webhook

The logging sounds very useful, thank you.

Whenever a log message relates to an email, please consider logging the
message-id of that email, if possible.

> * Add admin interface to webhook
> * Deploy webhook on a production server with 24/7 support
>
> # Ideas for Future Enhancements
>
> * Run PatchCheck.py before assigning maintainers/reviewers.
> * Add a simple check that fails if a single patch spans more than one package.

Hmmm, good idea in general, but there have been valid exceptions to this
rule.

> * Monitor comments for Reviewed-by, Acked-by, Tested-by, Series-Reviewed-by,
>   Series-Acked-by, Series-Tested-by made by assigned maintainers/reviewers.
>   Once all commits have required tags, auto update commit messages in the
>   branch and wait for maintainer to set the `Push` label to run CI and auto
>   merge if all CI checks pass.

Thank you for writing this up (and for implementing the webhook)!
Laszlo





[-- Attachment #2: Type: text/html, Size: 81750 bytes --]

Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

[Laszlo Ersek]

On 05/11/20 22:09, Bret Barkelew wrote:
> As a counterpoint: if we force a new branch or force push on every tweak, we lose the “thread� of discussion on what caused the change,

This is a github.com limitation.

And the email archive mitigates it.

In the current process, when I review v2 of a 10-part series, I have one
Thunderbird window open with the v1 thread, containing both the v1
patches and my (and others') review comments given for them.

(I open the new window by right-clicking the v1 blurb, and then
selecting "open message in new window". Then I navigate between the
messages of the v1 thread with the "f" and "b" hotkeys. The "scope" of
the new window is set to the v1 thread, recursively, when I open the new
window like that, and so "f" and "b" just do the right thing.)

In another window, to the right side, I run "git-range-diff", to
interdiff the v1 patches (patch by patch) with the v2 patches. (An
interdiff is a diff of diffs.) Importantly, the interdiff also
highlights commit message differences.

I verify that all the feedback comments from the v1 thread have been
addressed (per patch), and also that any otherwise "uncalled-for"
changes in v2 are in fact justified. (The contributor may have
justifiedly implemented further changes than what I requested under v1.)
This is also the reason why I meticulously number my feedback comments,
as I'm going to require a complete (one by one) coverage in the next
version of the patch set. (Except for those comments of course that the
contributor successfully refutes.)

When the v2 series has different structure from v1, then git-range-diff
is not as helpful -- in that case, I compare only a subset of the
patches like described above, and the entirely new patches in v2 I have
to review from zero.

The entire process depends on having unfettered access to comments given
for *any* earlier version of the patch set (it's not uncommon that I
refer back to v(n-3) or v(n-2) when reviewing v(n)), with those comments
being tightly bound (for display and for re-reading) to their subject
patches.

The github webui destroys (at least visually) the comments given before
a force-push. I can't fathom how incremental reviews can work on
github.com *at all*, in other projects. Hence my earlier suggestion to
use new pull requests rather than force-pushes.

But the mailing list archive generated by the webhook will solve this
completely -- I will use that list as a primary review support tool (for
v2, v3, ...), not only as an archive.


... After all, I guess I could reformulate like this: it's not my intent
to prevent people from pushing incremental fixups *temporarily*; I'm
only saying I will ignore those patches, and I will review only the next
full version of the branch.


My concern that does persist is this: "it runs the risk that the
maintainer responsible for ultimately merging the series ends up
actually merging the incremental (= "fixup") patches in isolation
(without squashing them)".

The git history should neither be littered with fixup patches, nor
contain huge squashes. The structure of a patch series is a first class
trait; it is an aspect to iterate upon, when a branch is being
contributed. The tooling should support that. (And the list traffic
generated by the webhook does.)

For instance, the last time I've given feedback regarding patch series
structure was just an hour ago, under the series "[PATCH V4 00/27]
Disabling safe string constraint assertions". I requested moving a hunk
from patch#1 to patch#26. Having the hunk in patch#1 does not break
bisection, and it's irrelevant for the end-state after the whole series
is applied (the end-state is the same). But the hunk still doesn't
*belong* in patch#1 -- wherever we add a new bit to a bitmask PCD
(patch#26), the UNI file (= documentation) udpate belongs in the exact
same patch.

> what changed as a result, and the easy hook for the original change requester to reply directly to the change as is.

No matter what I say about an incremental/fixup patch in isolation,
things can easily go wrong when the contributor squashes the fixup into
the more substantial patch that needs the fixup. Not to mention any
commit message updates on the more substantial patch, as necessitated by
the fixup. So I'll have to review the next full version of the topic
branch anyway, with git-range-diff, and compare the interdiff against my
earlier feedback.

Thanks!
Laszlo


> 
> - Bret
> 
> From: Laszlo Ersek via groups.io<mailto:lersek=redhat.com@groups.io>
> Sent: Monday, May 11, 2020 12:39 PM
> To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Kinney, Michael D<mailto:michael.d.kinney@intel.com>; rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>
> Subject: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process
> 
> On 05/09/20 04:59, Michael D Kinney wrote:
>> Hello,
>>
>> This is a proposal to change from the current email-based code review process to
>> a GitHub pull request-based code review process for all repositories maintained
>> in TianoCore.  The current email-based code review process and commit message
>> requirements are documented in Readme.md or Readme.rst at the root of
>> repositories along with a few Wiki pages:
>>
>> * https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fblob%2Fmaster%2FReadMe.rst&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=lVjWRLsBC3xJpyRFeDrGjFhMOzAgi2V3vsAPxj7lIDw%3D&amp;reserved=0
>> * https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FEDK-II-Development-Process&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=sgAhQxCpyjmzC%2FW%2BFiLLwaF2M8wscBz3k93ne25qUXs%3D&amp;reserved=0
>> * https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FLaszlo%27s-unkempt-git-guide-for-edk2-contributors-and-maintainers&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=eHP9fcPMw6yjqTU%2B%2BUZ3FZkq8jZeM1LTU6dGTzmFp4Q%3D&amp;reserved=0
>> * https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FCommit-Message-Format&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=uq8G6nGyLpa7m%2F0fD2pwrcM9uixbKs6SLTge8e77M%2FY%3D&amp;reserved=0
>> * https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FCommit-Signature-Format&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Mz8dUn2L8dFwJdlo4LbaIKt2JrWE%2Fn4tBtVWenK%2F8Ck%3D&amp;reserved=0
>>
>> The goal is to post changes by opening a GitHub pull request and perform all
>> code review activity using the GitHub web interface.  This proposal does not
>> change any licenses or commit message requirements.  It does require all
>> developers, maintainers, and reviewers to have GitHub accounts.
>>
>> One requirement that was collected from previous discussions on this topic is
>> the need for an email archive of all patches and code review activities.  The
>> existing GitHub features to produce an email archive were deemed insufficient.
>> A proof of concept of a GitHub webhook has been implemented to provide the email
>> archive service.  This email archive is read-only.  You will not be able to send
>> emails to this archive or reply to emails in the archive.
>>
>> The sections below provide more details on the proposed GitHub pull request
>> based code review process, details on the email archive service, and a set of
>> remaining tasks make the email archive service production quality.  It does not
>> make sense to support both the existing email-based code review and the GitHub
>> pull request-based code review at the same time.  Instead, this proposal is to
>> switch to the GitHub pull request-based code review and retire the email based
>> code review process on the same date.
>>
>> The edk2 repository is using GitHub pull requests today to run automated
>> CI checks on the code changes and allows a maintainer to set the `push` label to
>> request the changes to be merged if all CI checks pass.  With this proposal,
>> once the code review is complete and the commit messages have been updated, the
>> same pull request can be used to perform a final set of CI checks and merge the
>> changes into the master branch.
>>
>> I would like to collect feedback on this proposal and the email archive service
>> over the next two weeks with close of comments on Friday May 22, 2020.  If all
>> issues and concerns can be addressed, then I would like to see the community
>> agree to make this change as soon as all remaining tasks are completed.
>>
>> # TianoCore Repositories to enable
>>
>> * [edk2](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0)
>> * [edk2-platforms](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-platforms&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=g8mgGL6B%2FRsvm3935OpZMctOTKUoeHGi8jPuCVKQjbI%3D&amp;reserved=0)
>> * [edk2-non-osi](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-non-osi&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=9lrEsZWOpc3wqylKs7yF%2FzxYwZsUUamP3oUrWDWcHCc%3D&amp;reserved=0)
>> * [edk2-test](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-test&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=8v205MD3HTYg3yLmGJS3SIDA5um9sVJfOa5CXViZjyU%3D&amp;reserved=0)
>> * [edk2-libc](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-libc&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Tzt293HJzFnGSkh1mUBew8dAsaZ4axWq2ml8UhQ%2FSTI%3D&amp;reserved=0)
>> * [edk2-staging](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-staging&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=bcNbt7Y7KoBrcnW4fAc4jbGgJL%2B4lYUkVLhYNo37OiM%3D&amp;reserved=0)
>>
>> # GitHub Pull Request Code Review Process
>>
>> **NOTE**: All steps below use [edk2](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0) as an
>> example.  Several repositories are supported.
>>
>> ## Author/Developer Steps
>>   * Create a personal fork of [edk2](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0)
>>
>>     https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fgetting-started-with-github%2Ffork-a-repo&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=umI3eqOh03qmt9YlPo33ujypu90YwImAvuxh5SlrM%2Bw%3D&amp;reserved=0
>>
>>   * Create a new branch from edk2/master in personal fork of edk2 repository.
>>
>>   * Add set of commits for new feature or bug fix to new branch.  Make sure to
>>     follow the commit message format requirements.  The only change with this
>>     RFC is that the Cc: lines to maintainers/reviewers should **not** be added.
>>     The Cc: lines are still supported, but they should only be used to add
>>     reviewers that do not have GitHub IDs or are not members of TianoCore.
>>
>>   * Push branch with new commits to personal fork
>>   * Create a pull request against TianoCore edk2/master
>>
>>     https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fcollaborating-with-issues-and-pull-requests%2Fcreating-a-pull-request&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=2GVrQy0FGwd4CCeGveh99HL3zS1ekRfAAaKhhRiOMpU%3D&amp;reserved=0
>>
>>   * If pull request has more than 1 commit, then fill in the pull request title
>>     and decryption information for Patch #0.  Do not leave defaults.
> 
> s/decryption/description/
> 
> (Because I'm assuming this will turn into a wiki article at some point.)
> 
>>
>>   * Do not assign reviewers.  The webhook assigns maintainers and reviewers to
>>     the pull request and each commit in the pull request.
>>
>>   * If maintainers/reviewers provide feedback that requires changes, then make
>>     add commits to the current branch with the requested changes.  Once all
> 
> s/make add/add/
> 
>>     changes are accepted on the current branch, reformulate the patch series and
>>     commit comments as needed for perform a forced push to the branch in the
>>     personal fork of the edk2 repository.  This step may be repeated if multiple
>>     versions of the patch series are required to address all code review
>>     feedback.
> 
> Do I understand correctly that this recommends the contributor first
> push incremental patches on top of the series, then do a rebase
> (squashing updates as necessary) and finally do a force-push, for the
> next round of review?
> 
> To me as a reviewer, that's extra work. I'm used to locally comparing
> the v(n) patch set to v(n+1) with git-range-diff, and/or with some
> personal scripts. I wouldn't encourage incremental patches appended --
> even temporarily -- to the branch, because (a) it's extra review work
> (it requires me to review something that has zero chance to get into the
> git history as-is), and (b) it superficially resembles the
> github.com-specific bad practice called "squash on merge", and (c) it
> runs the risk that the maintainer responsible for ultimately merging the
> series ends up actually merging the incremental (= "fixup") patches in
> isolation (without squashing them).
> 
>>
>>   **OPEN**: How should minimum review period be set?  Labels?
> 
> Not sure about the best tooling. My recommendation would be to require
> reviewers to start providing their feedback within one week.
> 
> One thing that I find important is that a maintainer can signal "I got
> your work in my queue, but I may need more time". And a special case of
> that are automated out-of-office responses. I think they are very
> helpful (when a contributor feels they are bottlenecked on review), but
> I'm not sure how one can configure that via github. I certainly would
> not share my out-of-office times with github. (I set the start/end dates
> in my email infrastructure, at the moment, but the out-of-office
> messages it sends do not contain the dates either, on purpose.)
> 
>>
>> ## TianoCore GitHub Email Archive Webhook Service Steps
>>   * Receive an event that a new pull request was opened
>>   * Evaluate the files modified by the entire pull request and each commit in
>>     the pull request and cross references against `Maintainters.txt` in the root
> 
> s/cross references/cross reference them/ ?
> 
>>     of the repository to assign maintainers/reviewers to the pull request and
>>     each commit in the pull request. Individual commit assignments are performed
>>     by adding a commit comment of the following form:
>>
>>     [CodeReview] Review-request @mdkinney
>>
>>   * Generate and sends git patch review emails to the email archive.  Emails
> 
> s/sends/send/
> 
>>     are also sent to any Cc: tags in the commit messages.
>>
>>   * If the author/developer performs a forced push to the branch in their
>>     personal fork of the edk2 repository, then a new set of patch review emails
>>     with patch series Vx is sent to the email archive and any Cc: tags in commit
>>     messages.
>>
>>   * Receive events associated with all code review activities and generate
>>     and send emails to the email archive that shows all review comments and
>>     all responses closely matching the email contents seen in the current email
>>     based code review process.
>>
>>   * Generate and send email when pull request is merged or closed.
>>
>> ## Maintainer/Reviewer Steps
>>
>>   * Make sure GitHub configuration is setup to 'Watch' the repositories that
>>     you have maintainer ship or review responsibilities and that email
> 
> s/maintainer ship/maintainership/
> 
>>     notifications from GitHub are enabled.  This enables email notifications
>>     when a maintainer/reviewer is assigned to a pull request and individual
>>     commits.
>>
>>     https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fmanaging-subscriptions-and-notifications-on-github%2Fconfiguring-notifications&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=OlkiyymcQi39P8%2FOJZG4yjh4h%2FHerkHBe5bCSQQFLOU%3D&amp;reserved=0
>>
>>   * Subscribe to the email archive associated with the TianoCore GitHub Email
>>     Archive Webhook Service.
>>
>>     https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Ftianocore-code-review-poc&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=q0IuvS318pEkJU2td9xX87oIm0LbSlEvOvhpyOOFrE8%3D&amp;reserved=0
> 
> Important: as the name says ("-poc"), this is a Proof of Concept list,
> for now. Once we're ready to switch over, I'll file an internal ticket
> at RH to either rename the list, or (which is probably better) to create
> a new list (no "-poc" suffix).
> 
> The second option seems more useful because then the webhook development
> / bugfixing (if any) could perhaps occur in parallel to the normal edk2
> workflow.
> 
>>
>>   * Review pull requests and commits assigned by the TianoCore GitHub Email
>>     Archive Webhook Service and use the GitHub web UI to provide all review
>>     feedback.
>>
>>     https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fcollaborating-with-issues-and-pull-requests%2Freviewing-changes-in-pull-requests&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=51Ljm3wUbBTWT8hcaBD1ZQznSROvAQqnoTzQmD6K%2FLY%3D&amp;reserved=0
>>
>>   * Wait for Author/Developer to respond to all feedback and add commits with
>>     code changes as needed to resolve all feedback.  This step may be repeated
>>     if the developer/author need to produce multiple versions of the patch
>>     series to address all feedback.
> 
> (same question about the incremental fixup patches as above)
> 
>>
>>   * Once all feedback is addressed, add Reviewed-by, Acked-by, and Tested-by
>>     responses on individual commits.  Or add Series-reviewed-by, Series-acked-by,
>>     or Series-Tested-by responses to the entire pull request.
>>
>>   * Wait for Developer/Author to add tags to commit messages in the pull request.
>>
>>   * Perform final review of patches and commit message tags.  If there are not
>>     issues, set the `push` label to run final set of CI checks and auto merge
>>     the pull request into master.
>>
>> # Maintainers.txt Format Changes
>>
>> Add GitHub IDs of all maintainers and reviewers at the end of M: and R: lines
>> in [].  For example:
>>
>>     M: Michael D Kinney <michael.d.kinney@intel.com> [mdkinney]
>>
>> # TianoCore GitHub Email Archive Webhook Service
>>
>> Assign reviewers to commits in a GitHub pull request based on assignments
>> documented in Maintainers.txt and generates an email archive of all pull request
>> and code review activities.
> 
> s/generates/generate/
> 
> (or s/Assign/Assigns/)
> 
>>
>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2-email-archive-webhook&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=7CJNJMEXrxoynjavmEwjzUyRbfNUIZ3FEG4kDRXvhI4%3D&amp;reserved=0
>>
>> # Email Archive Subscription Service
>>
>> The emails are being delivered to the following RedHat email subscription
>> service.  Please subscribe to receive the emails and to be able to view the
>> email archives.
>>
>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Ftianocore-code-review-poc&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=q0IuvS318pEkJU2td9xX87oIm0LbSlEvOvhpyOOFrE8%3D&amp;reserved=0
>>
>> The email archives are at this link:
>>
>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2Findex.html&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=nedUfkmMmI5T6GtAxQCW4q6xt38%2FezeDYmfq6cpRD0M%3D&amp;reserved=0
>>
>> The following sections show some example pull requests and code reviews to
>> help review the generated emails, their contents, and threading.
>>
>> ## Email Achieve Thread View
>>
>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fthread.html%2300289&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=GtrEudehfXiSU6ZwH2zKO35CPPPVk0ctZIzhkpI6DkE%3D&amp;reserved=0
>>
>> ## Example patch series with 1 patch
>>
>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fthread.html%2300340&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=ZGpI8%2BzIA9OMFm3pSCc2DQ4F5ZxtDSAXtjdFjD%2BY3NA%3D&amp;reserved=0
>>
>> ## Example patch series with < 10 patches
>>
>> * https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00289.html&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=JyaUyvYfZD7b%2F2wN%2BpS%2B68b%2BwyKoZ3Rba4ol%2FyahQVU%3D&amp;reserved=0
>> * https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00030.html&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=bQHIJIQq4Pri8iK3vPxMDMWz%2BKtXcyuPdhr8y7gFpXA%3D&amp;reserved=0
>> * https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00018.html&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=uMIRGOq%2BVCOSwDzXkG4yueYS4ZJ7BWfsp3Z4%2B9lh6hE%3D&amp;reserved=0
>> * https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00008.html&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=3CBkdqDxRt4IxtECpWQdKJL%2Bf4HFqqHCXo4loxNTzAE%3D&amp;reserved=0
>>
>> ## Example patch series with > 80 patches
>>
>> * https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00198.html&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=fDfQnifOMzyzLMdP4xH8koKCiSj7ZiuYyrrSZXTf3d4%3D&amp;reserved=0
>> * https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00116.html&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=lcxA3tTna%2BdmTpcNMmPlS%2B47llMAcIEjhCEqxV7TDOc%3D&amp;reserved=0
>> * https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00035.html&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=CgvZ8e%2B7L4nacvRE35KqEyC%2F1CjDYP6wI10qn%2BoX39Y%3D&amp;reserved=0
>>
>> # Tasks to Complete
>>
>> * Create edk2-codereview repository for evaluation of new code review process.
>> * Add GitHub IDs to Maintainers.txt in edk2-codereview repository
>> * Update BaseTools/Scripts/GetMaintainer.py to be compatible with GitHub IDs at
>>   the end of M: and R: statements
>> * Update webhook to use Rabbit MQ to manage requests and emails
>> * Determine if webhook requests must be serialized?  Current POC is serialized.
>> * Make sure webhook has error handling for all unexpected events/states.
>> * Add logging of all events and emails to webhook
> 
> The logging sounds very useful, thank you.
> 
> Whenever a log message relates to an email, please consider logging the
> message-id of that email, if possible.
> 
>> * Add admin interface to webhook
>> * Deploy webhook on a production server with 24/7 support
>>
>> # Ideas for Future Enhancements
>>
>> * Run PatchCheck.py before assigning maintainers/reviewers.
>> * Add a simple check that fails if a single patch spans more than one package.
> 
> Hmmm, good idea in general, but there have been valid exceptions to this
> rule.
> 
>> * Monitor comments for Reviewed-by, Acked-by, Tested-by, Series-Reviewed-by,
>>   Series-Acked-by, Series-Tested-by made by assigned maintainers/reviewers.
>>   Once all commits have required tags, auto update commit messages in the
>>   branch and wait for maintainer to set the `Push` label to run CI and auto
>>   merge if all CI checks pass.
> 
> Thank you for writing this up (and for implementing the webhook)!
> Laszlo
> 
> 
> 
> 
> 

Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

[Bret Barkelew]

[-- Attachment #1: Type: text/plain, Size: 36552 bytes --]

I feel like this process is a good compromise. It’s not perfect (frankly, I’m a fan of enforced squash merges, which can maintain bisectability if managed well), but it allows for rapid iteration, ease of contribution, and approaches the workflow that many who have never used email to maintain a project would be familiar with.

It’s code management for the Instagram generation, and I for one welcome our new insect overlords.

- Bret

From: Kinney, Michael D<mailto:michael.d.kinney@intel.com>
Sent: Monday, May 11, 2020 1:43 PM
To: Bret Barkelew<mailto:Bret.Barkelew@microsoft.com>; devel@edk2.groups.io<mailto:devel@edk2.groups.io>; lersek@redhat.com<mailto:lersek@redhat.com>; rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>; Kinney, Michael D<mailto:michael.d.kinney@intel.com>
Subject: RE: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Hi Bret,

This is a good point.

What I am proposing is the first version of the patch series submitted as a pull request.  Let the community do a complete review of the content.  The submitter can add patches to the end of the pull request addressing feedback and can even add patches that make changes to previous patches until all feedback/conversations are resolved.  This keeps the conversations complete and the conversations will also be archived to the email archive.

At this point, the developer can reformulate the patch series and do forced push of V2.  Reviewers can review the cleaned up patch series and repeat the process if there is more feedback, or move to final approval.

By doing all the work on a single pull request, we minimize the total number of pull requests in the repo.

An alternative approach would be to open a new pull request for each new version of the series.  This would preserve the GitHub conversations for each version of the pull request.  All the earlier ones would be closed/abandoned, and only the final one would be closed/merged.

Best regards,

Mike

From: Bret Barkelew <Bret.Barkelew@microsoft.com>
Sent: Monday, May 11, 2020 1:10 PM
To: devel@edk2.groups.io; lersek@redhat.com; Kinney, Michael D <michael.d.kinney@intel.com>; rfc@edk2.groups.io
Subject: RE: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

As a counterpoint: if we force a new branch or force push on every tweak, we lose the “thread” of discussion on what caused the change, what changed as a result, and the easy hook for the original change requester to reply directly to the change as is.

- Bret

From: Laszlo Ersek via groups.io<mailto:lersek=redhat.com@groups.io>
Sent: Monday, May 11, 2020 12:39 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Kinney, Michael D<mailto:michael.d.kinney@intel.com>; rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>
Subject: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

On 05/09/20 04:59, Michael D Kinney wrote:
> Hello,
>
> This is a proposal to change from the current email-based code review process to
> a GitHub pull request-based code review process for all repositories maintained
> in TianoCore.  The current email-based code review process and commit message
> requirements are documented in Readme.md or Readme.rst at the root of
> repositories along with a few Wiki pages:
>
> * https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fblob%2Fmaster%2FReadMe.rst&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=lVjWRLsBC3xJpyRFeDrGjFhMOzAgi2V3vsAPxj7lIDw%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fblob%2Fmaster%2FReadMe.rst&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951667090&sdata=S%2Fu9iHwOFHKtYL7jeqIVGZLwDRbG%2F8%2BUm6qQxtLpwH0%3D&reserved=0>
> * https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FEDK-II-Development-Process&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=sgAhQxCpyjmzC%2FW%2BFiLLwaF2M8wscBz3k93ne25qUXs%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FEDK-II-Development-Process&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951667090&sdata=l4uZzb0JliBkFXCQ7YfNBXs3Aoky0RYQn5gVT34AlH4%3D&reserved=0>
> * https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FLaszlo%27s-unkempt-git-guide-for-edk2-contributors-and-maintainers&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=eHP9fcPMw6yjqTU%2B%2BUZ3FZkq8jZeM1LTU6dGTzmFp4Q%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FLaszlo%27s-unkempt-git-guide-for-edk2-contributors-and-maintainers&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951677089&sdata=Wo2qJFt7cHi5zZS96kCml7MZI%2B32v%2FiRqPdICvpTw5c%3D&reserved=0>
> * https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FCommit-Message-Format&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=uq8G6nGyLpa7m%2F0fD2pwrcM9uixbKs6SLTge8e77M%2FY%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FCommit-Message-Format&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951677089&sdata=RjfxykjBSMEU%2BqsYkAmDPl%2FIgvBTPx%2BCvSIOPexpcc8%3D&reserved=0>
> * https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FCommit-Signature-Format&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Mz8dUn2L8dFwJdlo4LbaIKt2JrWE%2Fn4tBtVWenK%2F8Ck%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FCommit-Signature-Format&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951687191&sdata=jPadwqi8wSOKaVOKU3o2JOegzvTqdz8o7bSLyT%2B0El8%3D&reserved=0>
>
> The goal is to post changes by opening a GitHub pull request and perform all
> code review activity using the GitHub web interface.  This proposal does not
> change any licenses or commit message requirements.  It does require all
> developers, maintainers, and reviewers to have GitHub accounts.
>
> One requirement that was collected from previous discussions on this topic is
> the need for an email archive of all patches and code review activities.  The
> existing GitHub features to produce an email archive were deemed insufficient.
> A proof of concept of a GitHub webhook has been implemented to provide the email
> archive service.  This email archive is read-only.  You will not be able to send
> emails to this archive or reply to emails in the archive.
>
> The sections below provide more details on the proposed GitHub pull request
> based code review process, details on the email archive service, and a set of
> remaining tasks make the email archive service production quality.  It does not
> make sense to support both the existing email-based code review and the GitHub
> pull request-based code review at the same time.  Instead, this proposal is to
> switch to the GitHub pull request-based code review and retire the email based
> code review process on the same date.
>
> The edk2 repository is using GitHub pull requests today to run automated
> CI checks on the code changes and allows a maintainer to set the `push` label to
> request the changes to be merged if all CI checks pass.  With this proposal,
> once the code review is complete and the commit messages have been updated, the
> same pull request can be used to perform a final set of CI checks and merge the
> changes into the master branch.
>
> I would like to collect feedback on this proposal and the email archive service
> over the next two weeks with close of comments on Friday May 22, 2020.  If all
> issues and concerns can be addressed, then I would like to see the community
> agree to make this change as soon as all remaining tasks are completed.
>
> # TianoCore Repositories to enable
>
> * [edk2](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951697080&sdata=YhFQ9Fxt8Y0kFdOTgY2v9vML9uCOpS9j5cKJj%2FPCToc%3D&reserved=0>)
> * [edk2-platforms](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-platforms&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=g8mgGL6B%2FRsvm3935OpZMctOTKUoeHGi8jPuCVKQjbI%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-platforms&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951697080&sdata=dG8NsNTDjSX05wQWXargPnJydEGQyvZCia%2BSOIUyG6o%3D&reserved=0>)
> * [edk2-non-osi](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-non-osi&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=9lrEsZWOpc3wqylKs7yF%2FzxYwZsUUamP3oUrWDWcHCc%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-non-osi&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951707067&sdata=Mkz1CUW26UJ2X6wIEgO0UGz0ZJqWhkXMFwc5v5420tU%3D&reserved=0>)
> * [edk2-test](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-test&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=8v205MD3HTYg3yLmGJS3SIDA5um9sVJfOa5CXViZjyU%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-test&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951707067&sdata=fzDqTRet6MgfwmVIq8mb5%2BgB3rAiVteCQklombfBMaU%3D&reserved=0>)
> * [edk2-libc](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-libc&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Tzt293HJzFnGSkh1mUBew8dAsaZ4axWq2ml8UhQ%2FSTI%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-libc&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951717077&sdata=wpaAwogDjzDwRzpoB5vInxeqtMkNoawyPFg1j3m9omw%3D&reserved=0>)
> * [edk2-staging](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-staging&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=bcNbt7Y7KoBrcnW4fAc4jbGgJL%2B4lYUkVLhYNo37OiM%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-staging&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951717077&sdata=5ehzG40XuHbF56QwzFHyY8krO1NwGDWnFVMZdZ9OUvM%3D&reserved=0>)
>
> # GitHub Pull Request Code Review Process
>
> **NOTE**: All steps below use [edk2](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951727054&sdata=IrlV0ptekGi4Sz%2FxBYhIa5LiRuI3jlKE9JzgwbKGP6o%3D&reserved=0>) as an
> example.  Several repositories are supported.
>
> ## Author/Developer Steps
>   * Create a personal fork of [edk2](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951727054&sdata=IrlV0ptekGi4Sz%2FxBYhIa5LiRuI3jlKE9JzgwbKGP6o%3D&reserved=0>)
>
>     https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fgetting-started-with-github%2Ffork-a-repo&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=umI3eqOh03qmt9YlPo33ujypu90YwImAvuxh5SlrM%2Bw%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fgetting-started-with-github%2Ffork-a-repo&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951737053&sdata=pC%2F9W90ZNlmGQFNy97ee1xQMJ93XC%2Bdo5lmIdZhnRk0%3D&reserved=0>
>
>   * Create a new branch from edk2/master in personal fork of edk2 repository.
>
>   * Add set of commits for new feature or bug fix to new branch.  Make sure to
>     follow the commit message format requirements.  The only change with this
>     RFC is that the Cc: lines to maintainers/reviewers should **not** be added.
>     The Cc: lines are still supported, but they should only be used to add
>     reviewers that do not have GitHub IDs or are not members of TianoCore.
>
>   * Push branch with new commits to personal fork
>   * Create a pull request against TianoCore edk2/master
>
>     https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fcollaborating-with-issues-and-pull-requests%2Fcreating-a-pull-request&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=2GVrQy0FGwd4CCeGveh99HL3zS1ekRfAAaKhhRiOMpU%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fcollaborating-with-issues-and-pull-requests%2Fcreating-a-pull-request&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951737053&sdata=sZjbyeRye35une4C2y5RQ83ah%2B01o4XhFNXsvIXT1l0%3D&reserved=0>
>
>   * If pull request has more than 1 commit, then fill in the pull request title
>     and decryption information for Patch #0.  Do not leave defaults.

s/decryption/description/

(Because I'm assuming this will turn into a wiki article at some point.)

>
>   * Do not assign reviewers.  The webhook assigns maintainers and reviewers to
>     the pull request and each commit in the pull request.
>
>   * If maintainers/reviewers provide feedback that requires changes, then make
>     add commits to the current branch with the requested changes.  Once all

s/make add/add/

>     changes are accepted on the current branch, reformulate the patch series and
>     commit comments as needed for perform a forced push to the branch in the
>     personal fork of the edk2 repository.  This step may be repeated if multiple
>     versions of the patch series are required to address all code review
>     feedback.

Do I understand correctly that this recommends the contributor first
push incremental patches on top of the series, then do a rebase
(squashing updates as necessary) and finally do a force-push, for the
next round of review?

To me as a reviewer, that's extra work. I'm used to locally comparing
the v(n) patch set to v(n+1) with git-range-diff, and/or with some
personal scripts. I wouldn't encourage incremental patches appended --
even temporarily -- to the branch, because (a) it's extra review work
(it requires me to review something that has zero chance to get into the
git history as-is), and (b) it superficially resembles the
github.com-specific bad practice called "squash on merge", and (c) it
runs the risk that the maintainer responsible for ultimately merging the
series ends up actually merging the incremental (= "fixup") patches in
isolation (without squashing them).

>
>   **OPEN**: How should minimum review period be set?  Labels?

Not sure about the best tooling. My recommendation would be to require
reviewers to start providing their feedback within one week.

One thing that I find important is that a maintainer can signal "I got
your work in my queue, but I may need more time". And a special case of
that are automated out-of-office responses. I think they are very
helpful (when a contributor feels they are bottlenecked on review), but
I'm not sure how one can configure that via github. I certainly would
not share my out-of-office times with github. (I set the start/end dates
in my email infrastructure, at the moment, but the out-of-office
messages it sends do not contain the dates either, on purpose.)

>
> ## TianoCore GitHub Email Archive Webhook Service Steps
>   * Receive an event that a new pull request was opened
>   * Evaluate the files modified by the entire pull request and each commit in
>     the pull request and cross references against `Maintainters.txt` in the root

s/cross references/cross reference them/ ?

>     of the repository to assign maintainers/reviewers to the pull request and
>     each commit in the pull request. Individual commit assignments are performed
>     by adding a commit comment of the following form:
>
>     [CodeReview] Review-request @mdkinney
>
>   * Generate and sends git patch review emails to the email archive.  Emails

s/sends/send/

>     are also sent to any Cc: tags in the commit messages.
>
>   * If the author/developer performs a forced push to the branch in their
>     personal fork of the edk2 repository, then a new set of patch review emails
>     with patch series Vx is sent to the email archive and any Cc: tags in commit
>     messages.
>
>   * Receive events associated with all code review activities and generate
>     and send emails to the email archive that shows all review comments and
>     all responses closely matching the email contents seen in the current email
>     based code review process.
>
>   * Generate and send email when pull request is merged or closed.
>
> ## Maintainer/Reviewer Steps
>
>   * Make sure GitHub configuration is setup to 'Watch' the repositories that
>     you have maintainer ship or review responsibilities and that email

s/maintainer ship/maintainership/

>     notifications from GitHub are enabled.  This enables email notifications
>     when a maintainer/reviewer is assigned to a pull request and individual
>     commits.
>
>     https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fmanaging-subscriptions-and-notifications-on-github%2Fconfiguring-notifications&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=OlkiyymcQi39P8%2FOJZG4yjh4h%2FHerkHBe5bCSQQFLOU%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fmanaging-subscriptions-and-notifications-on-github%2Fconfiguring-notifications&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951747044&sdata=FJwHjpFL7UFtZ9N12pSvQSXci3IqCJJVJXI9dDzlHHU%3D&reserved=0>
>
>   * Subscribe to the email archive associated with the TianoCore GitHub Email
>     Archive Webhook Service.
>
>     https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Ftianocore-code-review-poc&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=q0IuvS318pEkJU2td9xX87oIm0LbSlEvOvhpyOOFrE8%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Ftianocore-code-review-poc&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951747044&sdata=zbDuJd9lW3KoiuzGo6%2BU14Mnude7lZnbCeXQHJnR%2BCg%3D&reserved=0>

Important: as the name says ("-poc"), this is a Proof of Concept list,
for now. Once we're ready to switch over, I'll file an internal ticket
at RH to either rename the list, or (which is probably better) to create
a new list (no "-poc" suffix).

The second option seems more useful because then the webhook development
/ bugfixing (if any) could perhaps occur in parallel to the normal edk2
workflow.

>
>   * Review pull requests and commits assigned by the TianoCore GitHub Email
>     Archive Webhook Service and use the GitHub web UI to provide all review
>     feedback.
>
>     https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fcollaborating-with-issues-and-pull-requests%2Freviewing-changes-in-pull-requests&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=51Ljm3wUbBTWT8hcaBD1ZQznSROvAQqnoTzQmD6K%2FLY%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fcollaborating-with-issues-and-pull-requests%2Freviewing-changes-in-pull-requests&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951757040&sdata=Iav2d2fP2vw9mS9uA76tN3uQcIbl52D6BT963cD7ZC4%3D&reserved=0>
>
>   * Wait for Author/Developer to respond to all feedback and add commits with
>     code changes as needed to resolve all feedback.  This step may be repeated
>     if the developer/author need to produce multiple versions of the patch
>     series to address all feedback.

(same question about the incremental fixup patches as above)

>
>   * Once all feedback is addressed, add Reviewed-by, Acked-by, and Tested-by
>     responses on individual commits.  Or add Series-reviewed-by, Series-acked-by,
>     or Series-Tested-by responses to the entire pull request.
>
>   * Wait for Developer/Author to add tags to commit messages in the pull request.
>
>   * Perform final review of patches and commit message tags.  If there are not
>     issues, set the `push` label to run final set of CI checks and auto merge
>     the pull request into master.
>
> # Maintainers.txt Format Changes
>
> Add GitHub IDs of all maintainers and reviewers at the end of M: and R: lines
> in [].  For example:
>
>     M: Michael D Kinney <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>> [mdkinney]
>
> # TianoCore GitHub Email Archive Webhook Service
>
> Assign reviewers to commits in a GitHub pull request based on assignments
> documented in Maintainers.txt and generates an email archive of all pull request
> and code review activities.

s/generates/generate/

(or s/Assign/Assigns/)

>
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2-email-archive-webhook&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=7CJNJMEXrxoynjavmEwjzUyRbfNUIZ3FEG4kDRXvhI4%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2-email-archive-webhook&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951757040&sdata=FihVwcwfqewohsmqUECWG36TR1iCQVhVHq02iMZkEL8%3D&reserved=0>
>
> # Email Archive Subscription Service
>
> The emails are being delivered to the following RedHat email subscription
> service.  Please subscribe to receive the emails and to be able to view the
> email archives.
>
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Ftianocore-code-review-poc&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=q0IuvS318pEkJU2td9xX87oIm0LbSlEvOvhpyOOFrE8%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Ftianocore-code-review-poc&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951767033&sdata=482jeizloSfEobAk0F8%2BvP9FZptdF6oC90A%2BYSQZcds%3D&reserved=0>
>
> The email archives are at this link:
>
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2Findex.html&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=nedUfkmMmI5T6GtAxQCW4q6xt38%2FezeDYmfq6cpRD0M%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2Findex.html&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951767033&sdata=78HRihY2z%2Fll0qoC05RMh8I%2Bwri9rwIwblxNBYFPecw%3D&reserved=0>
>
> The following sections show some example pull requests and code reviews to
> help review the generated emails, their contents, and threading.
>
> ## Email Achieve Thread View
>
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fthread.html%2300289&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=GtrEudehfXiSU6ZwH2zKO35CPPPVk0ctZIzhkpI6DkE%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fthread.html%2300289&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951777024&sdata=%2BU8aFUi4kPBeJdlyW%2FERNtp5Fum5gZrWkgCTI8pwz14%3D&reserved=0>
>
> ## Example patch series with 1 patch
>
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fthread.html%2300340&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=ZGpI8%2BzIA9OMFm3pSCc2DQ4F5ZxtDSAXtjdFjD%2BY3NA%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fthread.html%2300340&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951777024&sdata=gXn%2B9z8MIdnJyUkk2xR1mK%2FL710jUDEeNhK74EUkb9A%3D&reserved=0>
>
> ## Example patch series with < 10 patches
>
> * https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00289.html&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=JyaUyvYfZD7b%2F2wN%2BpS%2B68b%2BwyKoZ3Rba4ol%2FyahQVU%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00289.html&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951787022&sdata=yvE8%2F%2FKOJUj%2BD4ZmtT3fZPAb2KBsGcAqxt%2FKwRKa6hA%3D&reserved=0>
> * https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00030.html&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=bQHIJIQq4Pri8iK3vPxMDMWz%2BKtXcyuPdhr8y7gFpXA%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00030.html&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951787022&sdata=zz5BgISidxyQVI%2BQihn%2Fq%2Fv9%2B5%2Fn2vpaeyvFbIiAigs%3D&reserved=0>
> * https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00018.html&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=uMIRGOq%2BVCOSwDzXkG4yueYS4ZJ7BWfsp3Z4%2B9lh6hE%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00018.html&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951797016&sdata=yN0WkrPdk1EEvyPwlqCG%2B6HTOpa3iT85w2QhXQJ2xYE%3D&reserved=0>
> * https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00008.html&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=3CBkdqDxRt4IxtECpWQdKJL%2Bf4HFqqHCXo4loxNTzAE%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00008.html&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951797016&sdata=qF2vw8XdQfY0ECclzhbpd60vIgTj2Bzl5RjC%2F4EdbA8%3D&reserved=0>
>
> ## Example patch series with > 80 patches
>
> * https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00198.html&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=fDfQnifOMzyzLMdP4xH8koKCiSj7ZiuYyrrSZXTf3d4%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00198.html&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951807013&sdata=uXAPx9iTDbQbiv40j3EiqIP51nlFLQ43ghvAsjt4dhs%3D&reserved=0>
> * https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00116.html&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=lcxA3tTna%2BdmTpcNMmPlS%2B47llMAcIEjhCEqxV7TDOc%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00116.html&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951807013&sdata=4ANIbH2YDPjN7H4gHGtbgNkAbM9EeVFOf%2FoWqTiXtiA%3D&reserved=0>
> * https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00035.html&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=CgvZ8e%2B7L4nacvRE35KqEyC%2F1CjDYP6wI10qn%2BoX39Y%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00035.html&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951817004&sdata=1jX%2FuPSMlRvzN9eZcv5U7k4JmwfJJwaVhDJ%2BrA6mPYc%3D&reserved=0>
>
> # Tasks to Complete
>
> * Create edk2-codereview repository for evaluation of new code review process.
> * Add GitHub IDs to Maintainers.txt in edk2-codereview repository
> * Update BaseTools/Scripts/GetMaintainer.py to be compatible with GitHub IDs at
>   the end of M: and R: statements
> * Update webhook to use Rabbit MQ to manage requests and emails
> * Determine if webhook requests must be serialized?  Current POC is serialized.
> * Make sure webhook has error handling for all unexpected events/states.
> * Add logging of all events and emails to webhook

The logging sounds very useful, thank you.

Whenever a log message relates to an email, please consider logging the
message-id of that email, if possible.

> * Add admin interface to webhook
> * Deploy webhook on a production server with 24/7 support
>
> # Ideas for Future Enhancements
>
> * Run PatchCheck.py before assigning maintainers/reviewers.
> * Add a simple check that fails if a single patch spans more than one package.

Hmmm, good idea in general, but there have been valid exceptions to this
rule.

> * Monitor comments for Reviewed-by, Acked-by, Tested-by, Series-Reviewed-by,
>   Series-Acked-by, Series-Tested-by made by assigned maintainers/reviewers.
>   Once all commits have required tags, auto update commit messages in the
>   branch and wait for maintainer to set the `Push` label to run CI and auto
>   merge if all CI checks pass.

Thank you for writing this up (and for implementing the webhook)!
Laszlo






[-- Attachment #2: Type: text/html, Size: 44390 bytes --]

Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

[Rebecca Cran]

On 5/14/20 3:26 PM, Bret Barkelew via groups.io wrote:

> I feel like this process is a good compromise. It’s not perfect (frankly, I’m a fan of enforced squash merges, which can maintain bisectability if managed well), but it allows for rapid iteration, ease of contribution, and approaches the workflow that many who have never used email to maintain a project would be familiar with.
>
> It’s code management for the Instagram generation, and I for one welcome our new insect overlords.

Or at least, that's what Microsoft is betting on! :D

Personally, I remain unconvinced about the usability of Github Pull 
Requests for a project the size of EDK2, but I hope to be proven wrong.


-- 
Rebecca Cran

Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

[Michael D Kinney]

Bret,

If the original submission is a single patch, and the code review
generates changes that are added as additional patches for review,
but the intent in the end is still a single patch, then squashing
them all at the end is correct.

Using the GitHub feature to squash them is a challenge because of 
the EDK II commit message requirements.  In order to make sure the
final commit message for the one commit is correct, the developer
should squash and edit the commit message.

There may be opportunities to automate these operations, but the
developer still needs to do the final review and have the ability
to do further edits of the commit message.

Mike

> -----Original Message-----
> From: rfc@edk2.groups.io <rfc@edk2.groups.io> On Behalf
> Of Bret Barkelew via groups.io
> Sent: Thursday, May 14, 2020 2:27 PM
> To: Kinney, Michael D <michael.d.kinney@intel.com>;
> devel@edk2.groups.io; lersek@redhat.com;
> rfc@edk2.groups.io
> Subject: Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc]
> GitHub Pull Request based Code Review Process
> 
> I feel like this process is a good compromise. It’s not
> perfect (frankly, I’m a fan of enforced squash merges,
> which can maintain bisectability if managed well), but
> it allows for rapid iteration, ease of contribution,
> and approaches the workflow that many who have never
> used email to maintain a project would be familiar
> with.
> 
> It’s code management for the Instagram generation, and
> I for one welcome our new insect overlords.
> 
> - Bret
> 
> From: Kinney, Michael
> D<mailto:michael.d.kinney@intel.com>
> Sent: Monday, May 11, 2020 1:43 PM
> To: Bret Barkelew<mailto:Bret.Barkelew@microsoft.com>;
> devel@edk2.groups.io<mailto:devel@edk2.groups.io>;
> lersek@redhat.com<mailto:lersek@redhat.com>;
> rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>; Kinney,
> Michael D<mailto:michael.d.kinney@intel.com>
> Subject: RE: [EXTERNAL] Re: [edk2-devel] [edk2-rfc]
> GitHub Pull Request based Code Review Process
> 
> Hi Bret,
> 
> This is a good point.
> 
> What I am proposing is the first version of the patch
> series submitted as a pull request.  Let the community
> do a complete review of the content.  The submitter can
> add patches to the end of the pull request addressing
> feedback and can even add patches that make changes to
> previous patches until all feedback/conversations are
> resolved.  This keeps the conversations complete and
> the conversations will also be archived to the email
> archive.
> 
> At this point, the developer can reformulate the patch
> series and do forced push of V2.  Reviewers can review
> the cleaned up patch series and repeat the process if
> there is more feedback, or move to final approval.
> 
> By doing all the work on a single pull request, we
> minimize the total number of pull requests in the repo.
> 
> An alternative approach would be to open a new pull
> request for each new version of the series.  This would
> preserve the GitHub conversations for each version of
> the pull request.  All the earlier ones would be
> closed/abandoned, and only the final one would be
> closed/merged.
> 
> Best regards,
> 
> Mike
> 
> From: Bret Barkelew <Bret.Barkelew@microsoft.com>
> Sent: Monday, May 11, 2020 1:10 PM
> To: devel@edk2.groups.io; lersek@redhat.com; Kinney,
> Michael D <michael.d.kinney@intel.com>;
> rfc@edk2.groups.io
> Subject: RE: [EXTERNAL] Re: [edk2-devel] [edk2-rfc]
> GitHub Pull Request based Code Review Process
> 
> As a counterpoint: if we force a new branch or force
> push on every tweak, we lose the “thread” of discussion
> on what caused the change, what changed as a result,
> and the easy hook for the original change requester to
> reply directly to the change as is.
> 
> - Bret
> 
> From: Laszlo Ersek via
> groups.io<mailto:lersek=redhat.com@groups.io>
> Sent: Monday, May 11, 2020 12:39 PM
> To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>;
> Kinney, Michael D<mailto:michael.d.kinney@intel.com>;
> rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>
> Subject: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub
> Pull Request based Code Review Process
> 
> On 05/09/20 04:59, Michael D Kinney wrote:
> > Hello,
> >
> > This is a proposal to change from the current email-
> based code review process to
> > a GitHub pull request-based code review process for
> all repositories maintained
> > in TianoCore.  The current email-based code review
> process and commit message
> > requirements are documented in Readme.md or
> Readme.rst at the root of
> > repositories along with a few Wiki pages:
> >
> > *
> https://nam06.safelinks.protection.outlook.com/?url=htt
> ps%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fblob%2Fmaste
> r%2FReadMe.rst&amp;data=02%7C01%7Cbret.barkelew%40micro
> soft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf8
> 6f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&am
> p;sdata=lVjWRLsBC3xJpyRFeDrGjFhMOzAgi2V3vsAPxj7lIDw%3D&
> amp;reserved=0<https://nam06.safelinks.protection.outlo
> ok.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2
> %2Fblob%2Fmaster%2FReadMe.rst&data=02%7C01%7CBret.Barke
> lew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%
> 7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482659
> 51667090&sdata=S%2Fu9iHwOFHKtYL7jeqIVGZLwDRbG%2F8%2BUm6
> qQxtLpwH0%3D&reserved=0>
> > *
> https://nam06.safelinks.protection.outlook.com/?url=htt
> ps%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io
> %2Fwiki%2FEDK-II-Development-
> Process&amp;data=02%7C01%7Cbret.barkelew%40microsoft.co
> m%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af
> 91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata
> =sgAhQxCpyjmzC%2FW%2BFiLLwaF2M8wscBz3k93ne25qUXs%3D&amp
> ;reserved=0<https://nam06.safelinks.protection.outlook.
> com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianoco
> re.github.io%2Fwiki%2FEDK-II-Development-
> Process&data=02%7C01%7CBret.Barkelew%40microsoft.com%7C
> b61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab
> 2d7cd011db47%7C1%7C0%7C637248265951667090&sdata=l4uZzb0
> JliBkFXCQ7YfNBXs3Aoky0RYQn5gVT34AlH4%3D&reserved=0>
> > *
> https://nam06.safelinks.protection.outlook.com/?url=htt
> ps%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io
> %2Fwiki%2FLaszlo%27s-unkempt-git-guide-for-edk2-
> contributors-and-
> maintainers&amp;data=02%7C01%7Cbret.barkelew%40microsof
> t.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f1
> 41af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;s
> data=eHP9fcPMw6yjqTU%2B%2BUZ3FZkq8jZeM1LTU6dGTzmFp4Q%3D
> &amp;reserved=0<https://nam06.safelinks.protection.outl
> ook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftia
> nocore.github.io%2Fwiki%2FLaszlo%27s-unkempt-git-guide-
> for-edk2-contributors-and-
> maintainers&data=02%7C01%7CBret.Barkelew%40microsoft.co
> m%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af
> 91ab2d7cd011db47%7C1%7C0%7C637248265951677089&sdata=Wo2
> qJFt7cHi5zZS96kCml7MZI%2B32v%2FiRqPdICvpTw5c%3D&reserve
> d=0>
> > *
> https://nam06.safelinks.protection.outlook.com/?url=htt
> ps%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io
> %2Fwiki%2FCommit-Message-
> Format&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com
> %7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af9
> 1ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=
> uq8G6nGyLpa7m%2F0fD2pwrcM9uixbKs6SLTge8e77M%2FY%3D&amp;
> reserved=0<https://nam06.safelinks.protection.outlook.c
> om/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocor
> e.github.io%2Fwiki%2FCommit-Message-
> Format&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb
> 61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2
> d7cd011db47%7C1%7C0%7C637248265951677089&sdata=RjfxykjB
> SMEU%2BqsYkAmDPl%2FIgvBTPx%2BCvSIOPexpcc8%3D&reserved=0
> >
> > *
> https://nam06.safelinks.protection.outlook.com/?url=htt
> ps%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io
> %2Fwiki%2FCommit-Signature-
> Format&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com
> %7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af9
> 1ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=
> Mz8dUn2L8dFwJdlo4LbaIKt2JrWE%2Fn4tBtVWenK%2F8Ck%3D&amp;
> reserved=0<https://nam06.safelinks.protection.outlook.c
> om/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocor
> e.github.io%2Fwiki%2FCommit-Signature-
> Format&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb
> 61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2
> d7cd011db47%7C1%7C0%7C637248265951687191&sdata=jPadwqi8
> wSOKaVOKU3o2JOegzvTqdz8o7bSLyT%2B0El8%3D&reserved=0>
> >
> > The goal is to post changes by opening a GitHub pull
> request and perform all
> > code review activity using the GitHub web interface.
> This proposal does not
> > change any licenses or commit message requirements.
> It does require all
> > developers, maintainers, and reviewers to have GitHub
> accounts.
> >
> > One requirement that was collected from previous
> discussions on this topic is
> > the need for an email archive of all patches and code
> review activities.  The
> > existing GitHub features to produce an email archive
> were deemed insufficient.
> > A proof of concept of a GitHub webhook has been
> implemented to provide the email
> > archive service.  This email archive is read-only.
> You will not be able to send
> > emails to this archive or reply to emails in the
> archive.
> >
> > The sections below provide more details on the
> proposed GitHub pull request
> > based code review process, details on the email
> archive service, and a set of
> > remaining tasks make the email archive service
> production quality.  It does not
> > make sense to support both the existing email-based
> code review and the GitHub
> > pull request-based code review at the same time.
> Instead, this proposal is to
> > switch to the GitHub pull request-based code review
> and retire the email based
> > code review process on the same date.
> >
> > The edk2 repository is using GitHub pull requests
> today to run automated
> > CI checks on the code changes and allows a maintainer
> to set the `push` label to
> > request the changes to be merged if all CI checks
> pass.  With this proposal,
> > once the code review is complete and the commit
> messages have been updated, the
> > same pull request can be used to perform a final set
> of CI checks and merge the
> > changes into the master branch.
> >
> > I would like to collect feedback on this proposal and
> the email archive service
> > over the next two weeks with close of comments on
> Friday May 22, 2020.  If all
> > issues and concerns can be addressed, then I would
> like to see the community
> > agree to make this change as soon as all remaining
> tasks are completed.
> >
> > # TianoCore Repositories to enable
> >
> > *
> [edk2](https://nam06.safelinks.protection.outlook.com/?
> url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&amp;dat
> a=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b
> 544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db4
> 7%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5
> Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0<https:/
> /nam06.safelinks.protection.outlook.com/?url=https%3A%2
> F%2Fgithub.com%2Ftianocore%2Fedk2&data=02%7C01%7CBret.B
> arkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebe
> b5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248
> 265951697080&sdata=YhFQ9Fxt8Y0kFdOTgY2v9vML9uCOpS9j5cKJ
> j%2FPCToc%3D&reserved=0>)
> > * [edk2-
> platforms](https://nam06.safelinks.protection.outlook.c
> om/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-
> platforms&amp;data=02%7C01%7Cbret.barkelew%40microsoft.
> com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141
> af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sda
> ta=g8mgGL6B%2FRsvm3935OpZMctOTKUoeHGi8jPuCVKQjbI%3D&amp
> ;reserved=0<https://nam06.safelinks.protection.outlook.
> com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-
> platforms&data=02%7C01%7CBret.Barkelew%40microsoft.com%
> 7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91
> ab2d7cd011db47%7C1%7C0%7C637248265951697080&sdata=dG8Ns
> NTDjSX05wQWXargPnJydEGQyvZCia%2BSOIUyG6o%3D&reserved=0>
> )
> > * [edk2-non-
> osi](https://nam06.safelinks.protection.outlook.com/?ur
> l=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-non-
> osi&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C
> 1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab
> 2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=9lr
> EsZWOpc3wqylKs7yF%2FzxYwZsUUamP3oUrWDWcHCc%3D&amp;reser
> ved=0<https://nam06.safelinks.protection.outlook.com/?u
> rl=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-non-
> osi&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61c
> e42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7c
> d011db47%7C1%7C0%7C637248265951707067&sdata=Mkz1CUW26UJ
> 2X6wIEgO0UGz0ZJqWhkXMFwc5v5420tU%3D&reserved=0>)
> > * [edk2-
> test](https://nam06.safelinks.protection.outlook.com/?u
> rl=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-
> test&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7
> C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91a
> b2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=8v
> 205MD3HTYg3yLmGJS3SIDA5um9sVJfOa5CXViZjyU%3D&amp;reserv
> ed=0<https://nam06.safelinks.protection.outlook.com/?ur
> l=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-
> test&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61
> ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7
> cd011db47%7C1%7C0%7C637248265951707067&sdata=fzDqTRet6M
> gfwmVIq8mb5%2BgB3rAiVteCQklombfBMaU%3D&reserved=0>)
> > * [edk2-
> libc](https://nam06.safelinks.protection.outlook.com/?u
> rl=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-
> libc&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7
> C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91a
> b2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Tz
> t293HJzFnGSkh1mUBew8dAsaZ4axWq2ml8UhQ%2FSTI%3D&amp;rese
> rved=0<https://nam06.safelinks.protection.outlook.com/?
> url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-
> libc&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61
> ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7
> cd011db47%7C1%7C0%7C637248265951717077&sdata=wpaAwogDjz
> DwRzpoB5vInxeqtMkNoawyPFg1j3m9omw%3D&reserved=0>)
> > * [edk2-
> staging](https://nam06.safelinks.protection.outlook.com
> /?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-
> staging&amp;data=02%7C01%7Cbret.barkelew%40microsoft.co
> m%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af
> 91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata
> =bcNbt7Y7KoBrcnW4fAc4jbGgJL%2B4lYUkVLhYNo37OiM%3D&amp;r
> eserved=0<https://nam06.safelinks.protection.outlook.co
> m/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-
> staging&data=02%7C01%7CBret.Barkelew%40microsoft.com%7C
> b61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab
> 2d7cd011db47%7C1%7C0%7C637248265951717077&sdata=5ehzG40
> XuHbF56QwzFHyY8krO1NwGDWnFVMZdZ9OUvM%3D&reserved=0>)
> >
> > # GitHub Pull Request Code Review Process
> >
> > **NOTE**: All steps below use
> [edk2](https://nam06.safelinks.protection.outlook.com/?
> url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&amp;dat
> a=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b
> 544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db4
> 7%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5
> Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0<https:/
> /nam06.safelinks.protection.outlook.com/?url=https%3A%2
> F%2Fgithub.com%2Ftianocore%2Fedk2&data=02%7C01%7CBret.B
> arkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebe
> b5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248
> 265951727054&sdata=IrlV0ptekGi4Sz%2FxBYhIa5LiRuI3jlKE9J
> zgwbKGP6o%3D&reserved=0>) as an
> > example.  Several repositories are supported.
> >
> > ## Author/Developer Steps
> >   * Create a personal fork of
> [edk2](https://nam06.safelinks.protection.outlook.com/?
> url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&amp;dat
> a=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b
> 544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db4
> 7%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5
> Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0<https:/
> /nam06.safelinks.protection.outlook.com/?url=https%3A%2
> F%2Fgithub.com%2Ftianocore%2Fedk2&data=02%7C01%7CBret.B
> arkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebe
> b5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248
> 265951727054&sdata=IrlV0ptekGi4Sz%2FxBYhIa5LiRuI3jlKE9J
> zgwbKGP6o%3D&reserved=0>)
> >
> >
> https://nam06.safelinks.protection.outlook.com/?url=htt
> ps%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fgetting-
> started-with-github%2Ffork-a-
> repo&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7
> C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91a
> b2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=um
> I3eqOh03qmt9YlPo33ujypu90YwImAvuxh5SlrM%2Bw%3D&amp;rese
> rved=0<https://nam06.safelinks.protection.outlook.com/?
> url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fgetti
> ng-started-with-github%2Ffork-a-
> repo&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61
> ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7
> cd011db47%7C1%7C0%7C637248265951737053&sdata=pC%2F9W90Z
> NlmGQFNy97ee1xQMJ93XC%2Bdo5lmIdZhnRk0%3D&reserved=0>
> >
> >   * Create a new branch from edk2/master in personal
> fork of edk2 repository.
> >
> >   * Add set of commits for new feature or bug fix to
> new branch.  Make sure to
> >     follow the commit message format requirements.
> The only change with this
> >     RFC is that the Cc: lines to
> maintainers/reviewers should **not** be added.
> >     The Cc: lines are still supported, but they
> should only be used to add
> >     reviewers that do not have GitHub IDs or are not
> members of TianoCore.
> >
> >   * Push branch with new commits to personal fork
> >   * Create a pull request against TianoCore
> edk2/master
> >
> >
> https://nam06.safelinks.protection.outlook.com/?url=htt
> ps%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fcollaboratin
> g-with-issues-and-pull-requests%2Fcreating-a-pull-
> request&amp;data=02%7C01%7Cbret.barkelew%40microsoft.co
> m%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af
> 91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata
> =2GVrQy0FGwd4CCeGveh99HL3zS1ekRfAAaKhhRiOMpU%3D&amp;res
> erved=0<https://nam06.safelinks.protection.outlook.com/
> ?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fcoll
> aborating-with-issues-and-pull-requests%2Fcreating-a-
> pull-
> request&data=02%7C01%7CBret.Barkelew%40microsoft.com%7C
> b61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab
> 2d7cd011db47%7C1%7C0%7C637248265951737053&sdata=sZjbyeR
> ye35une4C2y5RQ83ah%2B01o4XhFNXsvIXT1l0%3D&reserved=0>
> >
> >   * If pull request has more than 1 commit, then fill
> in the pull request title
> >     and decryption information for Patch #0.  Do not
> leave defaults.
> 
> s/decryption/description/
> 
> (Because I'm assuming this will turn into a wiki
> article at some point.)
> 
> >
> >   * Do not assign reviewers.  The webhook assigns
> maintainers and reviewers to
> >     the pull request and each commit in the pull
> request.
> >
> >   * If maintainers/reviewers provide feedback that
> requires changes, then make
> >     add commits to the current branch with the
> requested changes.  Once all
> 
> s/make add/add/
> 
> >     changes are accepted on the current branch,
> reformulate the patch series and
> >     commit comments as needed for perform a forced
> push to the branch in the
> >     personal fork of the edk2 repository.  This step
> may be repeated if multiple
> >     versions of the patch series are required to
> address all code review
> >     feedback.
> 
> Do I understand correctly that this recommends the
> contributor first
> push incremental patches on top of the series, then do
> a rebase
> (squashing updates as necessary) and finally do a
> force-push, for the
> next round of review?
> 
> To me as a reviewer, that's extra work. I'm used to
> locally comparing
> the v(n) patch set to v(n+1) with git-range-diff,
> and/or with some
> personal scripts. I wouldn't encourage incremental
> patches appended --
> even temporarily -- to the branch, because (a) it's
> extra review work
> (it requires me to review something that has zero
> chance to get into the
> git history as-is), and (b) it superficially resembles
> the
> github.com-specific bad practice called "squash on
> merge", and (c) it
> runs the risk that the maintainer responsible for
> ultimately merging the
> series ends up actually merging the incremental (=
> "fixup") patches in
> isolation (without squashing them).
> 
> >
> >   **OPEN**: How should minimum review period be set?
> Labels?
> 
> Not sure about the best tooling. My recommendation
> would be to require
> reviewers to start providing their feedback within one
> week.
> 
> One thing that I find important is that a maintainer
> can signal "I got
> your work in my queue, but I may need more time". And a
> special case of
> that are automated out-of-office responses. I think
> they are very
> helpful (when a contributor feels they are bottlenecked
> on review), but
> I'm not sure how one can configure that via github. I
> certainly would
> not share my out-of-office times with github. (I set
> the start/end dates
> in my email infrastructure, at the moment, but the out-
> of-office
> messages it sends do not contain the dates either, on
> purpose.)
> 
> >
> > ## TianoCore GitHub Email Archive Webhook Service
> Steps
> >   * Receive an event that a new pull request was
> opened
> >   * Evaluate the files modified by the entire pull
> request and each commit in
> >     the pull request and cross references against
> `Maintainters.txt` in the root
> 
> s/cross references/cross reference them/ ?
> 
> >     of the repository to assign maintainers/reviewers
> to the pull request and
> >     each commit in the pull request. Individual
> commit assignments are performed
> >     by adding a commit comment of the following form:
> >
> >     [CodeReview] Review-request @mdkinney
> >
> >   * Generate and sends git patch review emails to the
> email archive.  Emails
> 
> s/sends/send/
> 
> >     are also sent to any Cc: tags in the commit
> messages.
> >
> >   * If the author/developer performs a forced push to
> the branch in their
> >     personal fork of the edk2 repository, then a new
> set of patch review emails
> >     with patch series Vx is sent to the email archive
> and any Cc: tags in commit
> >     messages.
> >
> >   * Receive events associated with all code review
> activities and generate
> >     and send emails to the email archive that shows
> all review comments and
> >     all responses closely matching the email contents
> seen in the current email
> >     based code review process.
> >
> >   * Generate and send email when pull request is
> merged or closed.
> >
> > ## Maintainer/Reviewer Steps
> >
> >   * Make sure GitHub configuration is setup to
> 'Watch' the repositories that
> >     you have maintainer ship or review
> responsibilities and that email
> 
> s/maintainer ship/maintainership/
> 
> >     notifications from GitHub are enabled.  This
> enables email notifications
> >     when a maintainer/reviewer is assigned to a pull
> request and individual
> >     commits.
> >
> >
> https://nam06.safelinks.protection.outlook.com/?url=htt
> ps%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fmanaging-
> subscriptions-and-notifications-on-
> github%2Fconfiguring-
> notifications&amp;data=02%7C01%7Cbret.barkelew%40micros
> oft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86
> f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp
> ;sdata=OlkiyymcQi39P8%2FOJZG4yjh4h%2FHerkHBe5bCSQQFLOU%
> 3D&amp;reserved=0<https://nam06.safelinks.protection.ou
> tlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgit
> hub%2Fmanaging-subscriptions-and-notifications-on-
> github%2Fconfiguring-
> notifications&data=02%7C01%7CBret.Barkelew%40microsoft.
> com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141
> af91ab2d7cd011db47%7C1%7C0%7C637248265951747044&sdata=F
> JwHjpFL7UFtZ9N12pSvQSXci3IqCJJVJXI9dDzlHHU%3D&reserved=
> 0>
> >
> >   * Subscribe to the email archive associated with
> the TianoCore GitHub Email
> >     Archive Webhook Service.
> >
> >
> https://nam06.safelinks.protection.outlook.com/?url=htt
> ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Ftianoc
> ore-code-review-
> poc&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C
> 1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab
> 2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=q0I
> uvS318pEkJU2td9xX87oIm0LbSlEvOvhpyOOFrE8%3D&amp;reserve
> d=0<https://nam06.safelinks.protection.outlook.com/?url
> =https%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Fti
> anocore-code-review-
> poc&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61c
> e42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7c
> d011db47%7C1%7C0%7C637248265951747044&sdata=zbDuJd9lW3K
> oiuzGo6%2BU14Mnude7lZnbCeXQHJnR%2BCg%3D&reserved=0>
> 
> Important: as the name says ("-poc"), this is a Proof
> of Concept list,
> for now. Once we're ready to switch over, I'll file an
> internal ticket
> at RH to either rename the list, or (which is probably
> better) to create
> a new list (no "-poc" suffix).
> 
> The second option seems more useful because then the
> webhook development
> / bugfixing (if any) could perhaps occur in parallel to
> the normal edk2
> workflow.
> 
> >
> >   * Review pull requests and commits assigned by the
> TianoCore GitHub Email
> >     Archive Webhook Service and use the GitHub web UI
> to provide all review
> >     feedback.
> >
> >
> https://nam06.safelinks.protection.outlook.com/?url=htt
> ps%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fcollaboratin
> g-with-issues-and-pull-requests%2Freviewing-changes-in-
> pull-
> requests&amp;data=02%7C01%7Cbret.barkelew%40microsoft.c
> om%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141a
> f91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdat
> a=51Ljm3wUbBTWT8hcaBD1ZQznSROvAQqnoTzQmD6K%2FLY%3D&amp;
> reserved=0<https://nam06.safelinks.protection.outlook.c
> om/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fc
> ollaborating-with-issues-and-pull-requests%2Freviewing-
> changes-in-pull-
> requests&data=02%7C01%7CBret.Barkelew%40microsoft.com%7
> Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91a
> b2d7cd011db47%7C1%7C0%7C637248265951757040&sdata=Iav2d2
> fP2vw9mS9uA76tN3uQcIbl52D6BT963cD7ZC4%3D&reserved=0>
> >
> >   * Wait for Author/Developer to respond to all
> feedback and add commits with
> >     code changes as needed to resolve all feedback.
> This step may be repeated
> >     if the developer/author need to produce multiple
> versions of the patch
> >     series to address all feedback.
> 
> (same question about the incremental fixup patches as
> above)
> 
> >
> >   * Once all feedback is addressed, add Reviewed-by,
> Acked-by, and Tested-by
> >     responses on individual commits.  Or add Series-
> reviewed-by, Series-acked-by,
> >     or Series-Tested-by responses to the entire pull
> request.
> >
> >   * Wait for Developer/Author to add tags to commit
> messages in the pull request.
> >
> >   * Perform final review of patches and commit
> message tags.  If there are not
> >     issues, set the `push` label to run final set of
> CI checks and auto merge
> >     the pull request into master.
> >
> > # Maintainers.txt Format Changes
> >
> > Add GitHub IDs of all maintainers and reviewers at
> the end of M: and R: lines
> > in [].  For example:
> >
> >     M: Michael D Kinney
> <michael.d.kinney@intel.com<mailto:michael.d.kinney@int
> el.com>> [mdkinney]
> >
> > # TianoCore GitHub Email Archive Webhook Service
> >
> > Assign reviewers to commits in a GitHub pull request
> based on assignments
> > documented in Maintainers.txt and generates an email
> archive of all pull request
> > and code review activities.
> 
> s/generates/generate/
> 
> (or s/Assign/Assigns/)
> 
> >
> >
> https://nam06.safelinks.protection.outlook.com/?url=htt
> ps%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2-email-archive-
> webhook&amp;data=02%7C01%7Cbret.barkelew%40microsoft.co
> m%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af
> 91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata
> =7CJNJMEXrxoynjavmEwjzUyRbfNUIZ3FEG4kDRXvhI4%3D&amp;res
> erved=0<https://nam06.safelinks.protection.outlook.com/
> ?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2-email-
> archive-
> webhook&data=02%7C01%7CBret.Barkelew%40microsoft.com%7C
> b61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab
> 2d7cd011db47%7C1%7C0%7C637248265951757040&sdata=FihVwcw
> fqewohsmqUECWG36TR1iCQVhVHq02iMZkEL8%3D&reserved=0>
> >
> > # Email Archive Subscription Service
> >
> > The emails are being delivered to the following
> RedHat email subscription
> > service.  Please subscribe to receive the emails and
> to be able to view the
> > email archives.
> >
> >
> https://nam06.safelinks.protection.outlook.com/?url=htt
> ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Ftianoc
> ore-code-review-
> poc&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C
> 1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab
> 2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=q0I
> uvS318pEkJU2td9xX87oIm0LbSlEvOvhpyOOFrE8%3D&amp;reserve
> d=0<https://nam06.safelinks.protection.outlook.com/?url
> =https%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Fti
> anocore-code-review-
> poc&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61c
> e42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7c
> d011db47%7C1%7C0%7C637248265951767033&sdata=482jeizloSf
> EobAk0F8%2BvP9FZptdF6oC90A%2BYSQZcds%3D&reserved=0>
> >
> > The email archives are at this link:
> >
> >
> https://nam06.safelinks.protection.outlook.com/?url=htt
> ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
> re-code-review-
> poc%2Findex.html&amp;data=02%7C01%7Cbret.barkelew%40mic
> rosoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988b
> f86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&
> amp;sdata=nedUfkmMmI5T6GtAxQCW4q6xt38%2FezeDYmfq6cpRD0M
> %3D&amp;reserved=0<https://nam06.safelinks.protection.o
> utlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%
> 2Fprivate%2Ftianocore-code-review-
> poc%2Findex.html&data=02%7C01%7CBret.Barkelew%40microso
> ft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f
> 141af91ab2d7cd011db47%7C1%7C0%7C637248265951767033&sdat
> a=78HRihY2z%2Fll0qoC05RMh8I%2Bwri9rwIwblxNBYFPecw%3D&re
> served=0>
> >
> > The following sections show some example pull
> requests and code reviews to
> > help review the generated emails, their contents, and
> threading.
> >
> > ## Email Achieve Thread View
> >
> >
> https://nam06.safelinks.protection.outlook.com/?url=htt
> ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
> re-code-review-poc%2F2020-
> May%2Fthread.html%2300289&amp;data=02%7C01%7Cbret.barke
> lew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%
> 7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275
> 86678640&amp;sdata=GtrEudehfXiSU6ZwH2zKO35CPPPVk0ctZIzh
> kpI6DkE%3D&amp;reserved=0<https://nam06.safelinks.prote
> ction.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fm
> ailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-
> May%2Fthread.html%2300289&data=02%7C01%7CBret.Barkelew%
> 40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72
> f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C63724826595177
> 7024&sdata=%2BU8aFUi4kPBeJdlyW%2FERNtp5Fum5gZrWkgCTI8pw
> z14%3D&reserved=0>
> >
> > ## Example patch series with 1 patch
> >
> >
> https://nam06.safelinks.protection.outlook.com/?url=htt
> ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
> re-code-review-poc%2F2020-
> May%2Fthread.html%2300340&amp;data=02%7C01%7Cbret.barke
> lew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%
> 7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275
> 86678640&amp;sdata=ZGpI8%2BzIA9OMFm3pSCc2DQ4F5ZxtDSAXtj
> dFjD%2BY3NA%3D&amp;reserved=0<https://nam06.safelinks.p
> rotection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com
> %2Fmailman%2Fprivate%2Ftianocore-code-review-
> poc%2F2020-
> May%2Fthread.html%2300340&data=02%7C01%7CBret.Barkelew%
> 40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72
> f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C63724826595177
> 7024&sdata=gXn%2B9z8MIdnJyUkk2xR1mK%2FL710jUDEeNhK74EUk
> b9A%3D&reserved=0>
> >
> > ## Example patch series with < 10 patches
> >
> > *
> https://nam06.safelinks.protection.outlook.com/?url=htt
> ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
> re-code-review-poc%2F2020-
> May%2Fmsg00289.html&amp;data=02%7C01%7Cbret.barkelew%40
> microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f9
> 88bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275866786
> 40&amp;sdata=JyaUyvYfZD7b%2F2wN%2BpS%2B68b%2BwyKoZ3Rba4
> ol%2FyahQVU%3D&amp;reserved=0<https://nam06.safelinks.p
> rotection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com
> %2Fmailman%2Fprivate%2Ftianocore-code-review-
> poc%2F2020-
> May%2Fmsg00289.html&data=02%7C01%7CBret.Barkelew%40micr
> osoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf
> 86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951787022&s
> data=yvE8%2F%2FKOJUj%2BD4ZmtT3fZPAb2KBsGcAqxt%2FKwRKa6h
> A%3D&reserved=0>
> > *
> https://nam06.safelinks.protection.outlook.com/?url=htt
> ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
> re-code-review-poc%2F2020-
> May%2Fmsg00030.html&amp;data=02%7C01%7Cbret.barkelew%40
> microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f9
> 88bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275866786
> 40&amp;sdata=bQHIJIQq4Pri8iK3vPxMDMWz%2BKtXcyuPdhr8y7gF
> pXA%3D&amp;reserved=0<https://nam06.safelinks.protectio
> n.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailm
> an%2Fprivate%2Ftianocore-code-review-poc%2F2020-
> May%2Fmsg00030.html&data=02%7C01%7CBret.Barkelew%40micr
> osoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf
> 86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951787022&s
> data=zz5BgISidxyQVI%2BQihn%2Fq%2Fv9%2B5%2Fn2vpaeyvFbIiA
> igs%3D&reserved=0>
> > *
> https://nam06.safelinks.protection.outlook.com/?url=htt
> ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
> re-code-review-poc%2F2020-
> May%2Fmsg00018.html&amp;data=02%7C01%7Cbret.barkelew%40
> microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f9
> 88bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275866786
> 40&amp;sdata=uMIRGOq%2BVCOSwDzXkG4yueYS4ZJ7BWfsp3Z4%2B9
> lh6hE%3D&amp;reserved=0<https://nam06.safelinks.protect
> ion.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmai
> lman%2Fprivate%2Ftianocore-code-review-poc%2F2020-
> May%2Fmsg00018.html&data=02%7C01%7CBret.Barkelew%40micr
> osoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf
> 86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951797016&s
> data=yN0WkrPdk1EEvyPwlqCG%2B6HTOpa3iT85w2QhXQJ2xYE%3D&r
> eserved=0>
> > *
> https://nam06.safelinks.protection.outlook.com/?url=htt
> ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
> re-code-review-poc%2F2020-
> May%2Fmsg00008.html&amp;data=02%7C01%7Cbret.barkelew%40
> microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f9
> 88bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275866786
> 40&amp;sdata=3CBkdqDxRt4IxtECpWQdKJL%2Bf4HFqqHCXo4loxNT
> zAE%3D&amp;reserved=0<https://nam06.safelinks.protectio
> n.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailm
> an%2Fprivate%2Ftianocore-code-review-poc%2F2020-
> May%2Fmsg00008.html&data=02%7C01%7CBret.Barkelew%40micr
> osoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf
> 86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951797016&s
> data=qF2vw8XdQfY0ECclzhbpd60vIgTj2Bzl5RjC%2F4EdbA8%3D&r
> eserved=0>
> >
> > ## Example patch series with > 80 patches
> >
> > *
> https://nam06.safelinks.protection.outlook.com/?url=htt
> ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
> re-code-review-poc%2F2020-
> May%2Fmsg00198.html&amp;data=02%7C01%7Cbret.barkelew%40
> microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f9
> 88bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275866786
> 40&amp;sdata=fDfQnifOMzyzLMdP4xH8koKCiSj7ZiuYyrrSZXTf3d
> 4%3D&amp;reserved=0<https://nam06.safelinks.protection.
> outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman
> %2Fprivate%2Ftianocore-code-review-poc%2F2020-
> May%2Fmsg00198.html&data=02%7C01%7CBret.Barkelew%40micr
> osoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf
> 86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951807013&s
> data=uXAPx9iTDbQbiv40j3EiqIP51nlFLQ43ghvAsjt4dhs%3D&res
> erved=0>
> > *
> https://nam06.safelinks.protection.outlook.com/?url=htt
> ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
> re-code-review-poc%2F2020-
> May%2Fmsg00116.html&amp;data=02%7C01%7Cbret.barkelew%40
> microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f9
> 88bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275866786
> 40&amp;sdata=lcxA3tTna%2BdmTpcNMmPlS%2B47llMAcIEjhCEqxV
> 7TDOc%3D&amp;reserved=0<https://nam06.safelinks.protect
> ion.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmai
> lman%2Fprivate%2Ftianocore-code-review-poc%2F2020-
> May%2Fmsg00116.html&data=02%7C01%7CBret.Barkelew%40micr
> osoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf
> 86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951807013&s
> data=4ANIbH2YDPjN7H4gHGtbgNkAbM9EeVFOf%2FoWqTiXtiA%3D&r
> eserved=0>
> > *
> https://nam06.safelinks.protection.outlook.com/?url=htt
> ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
> re-code-review-poc%2F2020-
> May%2Fmsg00035.html&amp;data=02%7C01%7Cbret.barkelew%40
> microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f9
> 88bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275866786
> 40&amp;sdata=CgvZ8e%2B7L4nacvRE35KqEyC%2F1CjDYP6wI10qn%
> 2BoX39Y%3D&amp;reserved=0<https://nam06.safelinks.prote
> ction.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fm
> ailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-
> May%2Fmsg00035.html&data=02%7C01%7CBret.Barkelew%40micr
> osoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf
> 86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951817004&s
> data=1jX%2FuPSMlRvzN9eZcv5U7k4JmwfJJwaVhDJ%2BrA6mPYc%3D
> &reserved=0>
> >
> > # Tasks to Complete
> >
> > * Create edk2-codereview repository for evaluation of
> new code review process.
> > * Add GitHub IDs to Maintainers.txt in edk2-
> codereview repository
> > * Update BaseTools/Scripts/GetMaintainer.py to be
> compatible with GitHub IDs at
> >   the end of M: and R: statements
> > * Update webhook to use Rabbit MQ to manage requests
> and emails
> > * Determine if webhook requests must be serialized?
> Current POC is serialized.
> > * Make sure webhook has error handling for all
> unexpected events/states.
> > * Add logging of all events and emails to webhook
> 
> The logging sounds very useful, thank you.
> 
> Whenever a log message relates to an email, please
> consider logging the
> message-id of that email, if possible.
> 
> > * Add admin interface to webhook
> > * Deploy webhook on a production server with 24/7
> support
> >
> > # Ideas for Future Enhancements
> >
> > * Run PatchCheck.py before assigning
> maintainers/reviewers.
> > * Add a simple check that fails if a single patch
> spans more than one package.
> 
> Hmmm, good idea in general, but there have been valid
> exceptions to this
> rule.
> 
> > * Monitor comments for Reviewed-by, Acked-by, Tested-
> by, Series-Reviewed-by,
> >   Series-Acked-by, Series-Tested-by made by assigned
> maintainers/reviewers.
> >   Once all commits have required tags, auto update
> commit messages in the
> >   branch and wait for maintainer to set the `Push`
> label to run CI and auto
> >   merge if all CI checks pass.
> 
> Thank you for writing this up (and for implementing the
> webhook)!
> Laszlo
> 
> 
> 
> 
> 
> 
> 

Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

[Bret Barkelew]

[-- Attachment #1: Type: text/plain, Size: 47417 bytes --]

Agreed. Responsibility to verify the commit message when squashing is always something to be aware of.

With Github, the one who presses the “Close and Merge” (or whatever it’s called) button has the final say on the title and message. We can play with approaches to how the squash merge would work.

I would far prefer the approach of individual PRs for commits to allow for the squash flexibility (and is the strategy I think I would pursue with my PRs). For example, the VarPol PR would be broken up into 9 PRs for each final commit, and we can get them in one by one. Ideally, each one would be a small back and forth and then in. If it had been done that way to begin with, it would be over in a week and a half or so, rather than the multiple months that we’re now verging on.

- Bret

From: Michael D Kinney via groups.io<mailto:michael.d.kinney=intel.com@groups.io>
Sent: Thursday, May 14, 2020 6:19 PM
To: rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>; Bret Barkelew<mailto:Bret.Barkelew@microsoft.com>; devel@edk2.groups.io<mailto:devel@edk2.groups.io>; lersek@redhat.com<mailto:lersek@redhat.com>; Kinney, Michael D<mailto:michael.d.kinney@intel.com>
Subject: Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Bret,

If the original submission is a single patch, and the code review
generates changes that are added as additional patches for review,
but the intent in the end is still a single patch, then squashing
them all at the end is correct.

Using the GitHub feature to squash them is a challenge because of
the EDK II commit message requirements.  In order to make sure the
final commit message for the one commit is correct, the developer
should squash and edit the commit message.

There may be opportunities to automate these operations, but the
developer still needs to do the final review and have the ability
to do further edits of the commit message.

Mike

> -----Original Message-----
> From: rfc@edk2.groups.io <rfc@edk2.groups.io> On Behalf
> Of Bret Barkelew via groups.io
> Sent: Thursday, May 14, 2020 2:27 PM
> To: Kinney, Michael D <michael.d.kinney@intel.com>;
> devel@edk2.groups.io; lersek@redhat.com;
> rfc@edk2.groups.io
> Subject: Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc]
> GitHub Pull Request based Code Review Process
>
> I feel like this process is a good compromise. It’s not
> perfect (frankly, I’m a fan of enforced squash merges,
> which can maintain bisectability if managed well), but
> it allows for rapid iteration, ease of contribution,
> and approaches the workflow that many who have never
> used email to maintain a project would be familiar
> with.
>
> It’s code management for the Instagram generation, and
> I for one welcome our new insect overlords.
>
> - Bret
>
> From: Kinney, Michael
> D<mailto:michael.d.kinney@intel.com>
> Sent: Monday, May 11, 2020 1:43 PM
> To: Bret Barkelew<mailto:Bret.Barkelew@microsoft.com>;
> devel@edk2.groups.io<mailto:devel@edk2.groups.io>;
> lersek@redhat.com<mailto:lersek@redhat.com>;
> rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>; Kinney,
> Michael D<mailto:michael.d.kinney@intel.com>
> Subject: RE: [EXTERNAL] Re: [edk2-devel] [edk2-rfc]
> GitHub Pull Request based Code Review Process
>
> Hi Bret,
>
> This is a good point.
>
> What I am proposing is the first version of the patch
> series submitted as a pull request.  Let the community
> do a complete review of the content.  The submitter can
> add patches to the end of the pull request addressing
> feedback and can even add patches that make changes to
> previous patches until all feedback/conversations are
> resolved.  This keeps the conversations complete and
> the conversations will also be archived to the email
> archive.
>
> At this point, the developer can reformulate the patch
> series and do forced push of V2.  Reviewers can review
> the cleaned up patch series and repeat the process if
> there is more feedback, or move to final approval.
>
> By doing all the work on a single pull request, we
> minimize the total number of pull requests in the repo.
>
> An alternative approach would be to open a new pull
> request for each new version of the series.  This would
> preserve the GitHub conversations for each version of
> the pull request.  All the earlier ones would be
> closed/abandoned, and only the final one would be
> closed/merged.
>
> Best regards,
>
> Mike
>
> From: Bret Barkelew <Bret.Barkelew@microsoft.com>
> Sent: Monday, May 11, 2020 1:10 PM
> To: devel@edk2.groups.io; lersek@redhat.com; Kinney,
> Michael D <michael.d.kinney@intel.com>;
> rfc@edk2.groups.io
> Subject: RE: [EXTERNAL] Re: [edk2-devel] [edk2-rfc]
> GitHub Pull Request based Code Review Process
>
> As a counterpoint: if we force a new branch or force
> push on every tweak, we lose the “thread” of discussion
> on what caused the change, what changed as a result,
> and the easy hook for the original change requester to
> reply directly to the change as is.
>
> - Bret
>
> From: Laszlo Ersek via
> groups.io<mailto:lersek=redhat.com@groups.io>
> Sent: Monday, May 11, 2020 12:39 PM
> To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>;
> Kinney, Michael D<mailto:michael.d.kinney@intel.com>;
> rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>
> Subject: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub
> Pull Request based Code Review Process
>
> On 05/09/20 04:59, Michael D Kinney wrote:
> > Hello,
> >
> > This is a proposal to change from the current email-
> based code review process to
> > a GitHub pull request-based code review process for
> all repositories maintained
> > in TianoCore.  The current email-based code review
> process and commit message
> > requirements are documented in Readme.md or
> Readme.rst at the root of
> > repositories along with a few Wiki pages:
> >
> > *
> https://nam06.safelinks.protection.outlook.com/?url=htt
> ps%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fblob%2Fmaste
> r%2FReadMe.rst&amp;data=02%7C01%7Cbret.barkelew%40micro
> soft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf8
> 6f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&am
> p;sdata=lVjWRLsBC3xJpyRFeDrGjFhMOzAgi2V3vsAPxj7lIDw%3D&
> amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protection.outlo%2F&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611242055&amp;sdata=Gr4Mw1Yz36xH3riEM7yicL5DIKv1%2FVuXM%2FObxAIWbJo%3D&amp;reserved=0
> ok.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2
> %2Fblob%2Fmaster%2FReadMe.rst&data=02%7C01%7CBret.Barke
> lew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%
> 7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482659
> 51667090&sdata=S%2Fu9iHwOFHKtYL7jeqIVGZLwDRbG%2F8%2BUm6
> qQxtLpwH0%3D&reserved=0>
> > *
> https://nam06.safelinks.protection.outlook.com/?url=htt
> ps%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io
> %2Fwiki%2FEDK-II-Development-
> Process&amp;data=02%7C01%7Cbret.barkelew%40microsoft.co
> m%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af
> 91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata
> =sgAhQxCpyjmzC%2FW%2BFiLLwaF2M8wscBz3k93ne25qUXs%3D&amp
> ;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protection.outlook%2F&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611252057&amp;sdata=B0BrCucxkG8t6JzA0b113MLW5PocHmW54lhtOAQdR%2F4%3D&amp;reserved=0.
> com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianoco
> re.github.io%2Fwiki%2FEDK-II-Development-
> Process&data=02%7C01%7CBret.Barkelew%40microsoft.com%7C
> b61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab
> 2d7cd011db47%7C1%7C0%7C637248265951667090&sdata=l4uZzb0
> JliBkFXCQ7YfNBXs3Aoky0RYQn5gVT34AlH4%3D&reserved=0>
> > *
> https://nam06.safelinks.protection.outlook.com/?url=htt
> ps%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io
> %2Fwiki%2FLaszlo%27s-unkempt-git-guide-for-edk2-
> contributors-and-
> maintainers&amp;data=02%7C01%7Cbret.barkelew%40microsof
> t.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f1
> 41af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;s
> data=eHP9fcPMw6yjqTU%2B%2BUZ3FZkq8jZeM1LTU6dGTzmFp4Q%3D
> &amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protection.outl%2F&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611252057&amp;sdata=%2FfG2%2BrXhM02OXgg%2Fa1G8tBYx0mHIH8lTm%2FF7bKSh10M%3D&amp;reserved=0
> ook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftia
> nocore.github.io%2Fwiki%2FLaszlo%27s-unkempt-git-guide-
> for-edk2-contributors-and-
> maintainers&data=02%7C01%7CBret.Barkelew%40microsoft.co
> m%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af
> 91ab2d7cd011db47%7C1%7C0%7C637248265951677089&sdata=Wo2
> qJFt7cHi5zZS96kCml7MZI%2B32v%2FiRqPdICvpTw5c%3D&reserve
> d=0>
> > *
> https://nam06.safelinks.protection.outlook.com/?url=htt
> ps%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io
> %2Fwiki%2FCommit-Message-
> Format&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com
> %7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af9
> 1ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=
> uq8G6nGyLpa7m%2F0fD2pwrcM9uixbKs6SLTge8e77M%2FY%3D&amp;
> reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protection.outlook.c%2F&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611252057&amp;sdata=%2BfY0MvXLLyf%2Bt7w2VFkWgfnEGFjgqdJo7WsZmQN8yLE%3D&amp;reserved=0
> om/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocor
> e.github.io%2Fwiki%2FCommit-Message-
> Format&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb
> 61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2
> d7cd011db47%7C1%7C0%7C637248265951677089&sdata=RjfxykjB
> SMEU%2BqsYkAmDPl%2FIgvBTPx%2BCvSIOPexpcc8%3D&reserved=0
> >
> > *
> https://nam06.safelinks.protection.outlook.com/?url=htt
> ps%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io
> %2Fwiki%2FCommit-Signature-
> Format&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com
> %7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af9
> 1ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=
> Mz8dUn2L8dFwJdlo4LbaIKt2JrWE%2Fn4tBtVWenK%2F8Ck%3D&amp;
> reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protection.outlook.c%2F&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611252057&amp;sdata=%2BfY0MvXLLyf%2Bt7w2VFkWgfnEGFjgqdJo7WsZmQN8yLE%3D&amp;reserved=0
> om/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocor
> e.github.io%2Fwiki%2FCommit-Signature-
> Format&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb
> 61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2
> d7cd011db47%7C1%7C0%7C637248265951687191&sdata=jPadwqi8
> wSOKaVOKU3o2JOegzvTqdz8o7bSLyT%2B0El8%3D&reserved=0>
> >
> > The goal is to post changes by opening a GitHub pull
> request and perform all
> > code review activity using the GitHub web interface.
> This proposal does not
> > change any licenses or commit message requirements.
> It does require all
> > developers, maintainers, and reviewers to have GitHub
> accounts.
> >
> > One requirement that was collected from previous
> discussions on this topic is
> > the need for an email archive of all patches and code
> review activities.  The
> > existing GitHub features to produce an email archive
> were deemed insufficient.
> > A proof of concept of a GitHub webhook has been
> implemented to provide the email
> > archive service.  This email archive is read-only.
> You will not be able to send
> > emails to this archive or reply to emails in the
> archive.
> >
> > The sections below provide more details on the
> proposed GitHub pull request
> > based code review process, details on the email
> archive service, and a set of
> > remaining tasks make the email archive service
> production quality.  It does not
> > make sense to support both the existing email-based
> code review and the GitHub
> > pull request-based code review at the same time.
> Instead, this proposal is to
> > switch to the GitHub pull request-based code review
> and retire the email based
> > code review process on the same date.
> >
> > The edk2 repository is using GitHub pull requests
> today to run automated
> > CI checks on the code changes and allows a maintainer
> to set the `push` label to
> > request the changes to be merged if all CI checks
> pass.  With this proposal,
> > once the code review is complete and the commit
> messages have been updated, the
> > same pull request can be used to perform a final set
> of CI checks and merge the
> > changes into the master branch.
> >
> > I would like to collect feedback on this proposal and
> the email archive service
> > over the next two weeks with close of comments on
> Friday May 22, 2020.  If all
> > issues and concerns can be addressed, then I would
> like to see the community
> > agree to make this change as soon as all remaining
> tasks are completed.
> >
> > # TianoCore Repositories to enable
> >
> > *
> [edk2](https://nam06.safelinks.protection.outlook.com/?
> url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&amp;dat
> a=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b
> 544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db4
> 7%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5
> Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0<https:/
> /nam06.safelinks.protection.outlook.com/?url=https%3A%2
> F%2Fgithub.com%2Ftianocore%2Fedk2&data=02%7C01%7CBret.B
> arkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebe
> b5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248
> 265951697080&sdata=YhFQ9Fxt8Y0kFdOTgY2v9vML9uCOpS9j5cKJ
> j%2FPCToc%3D&reserved=0>)
> > * [edk2-
> platforms](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protection.outlook.c%2F&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611252057&amp;sdata=%2BfY0MvXLLyf%2Bt7w2VFkWgfnEGFjgqdJo7WsZmQN8yLE%3D&amp;reserved=0
> om/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-
> platforms&amp;data=02%7C01%7Cbret.barkelew%40microsoft.
> com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141
> af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sda
> ta=g8mgGL6B%2FRsvm3935OpZMctOTKUoeHGi8jPuCVKQjbI%3D&amp
> ;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protection.outlook%2F&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611252057&amp;sdata=B0BrCucxkG8t6JzA0b113MLW5PocHmW54lhtOAQdR%2F4%3D&amp;reserved=0.
> com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-
> platforms&data=02%7C01%7CBret.Barkelew%40microsoft.com%
> 7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91
> ab2d7cd011db47%7C1%7C0%7C637248265951697080&sdata=dG8Ns
> NTDjSX05wQWXargPnJydEGQyvZCia%2BSOIUyG6o%3D&reserved=0>
> )
> > * [edk2-non-
> osi](https://nam06.safelinks.protection.outlook.com/?ur
> l=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-non-
> osi&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C
> 1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab
> 2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=9lr
> EsZWOpc3wqylKs7yF%2FzxYwZsUUamP3oUrWDWcHCc%3D&amp;reser
> ved=0<https://nam06.safelinks.protection.outlook.com/?u
> rl=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-non-
> osi&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61c
> e42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7c
> d011db47%7C1%7C0%7C637248265951707067&sdata=Mkz1CUW26UJ
> 2X6wIEgO0UGz0ZJqWhkXMFwc5v5420tU%3D&reserved=0>)
> > * [edk2-
> test](https://nam06.safelinks.protection.outlook.com/?u
> rl=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-
> test&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7
> C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91a
> b2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=8v
> 205MD3HTYg3yLmGJS3SIDA5um9sVJfOa5CXViZjyU%3D&amp;reserv
> ed=0<https://nam06.safelinks.protection.outlook.com/?ur
> l=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-
> test&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61
> ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7
> cd011db47%7C1%7C0%7C637248265951707067&sdata=fzDqTRet6M
> gfwmVIq8mb5%2BgB3rAiVteCQklombfBMaU%3D&reserved=0>)
> > * [edk2-
> libc](https://nam06.safelinks.protection.outlook.com/?u
> rl=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-
> libc&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7
> C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91a
> b2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Tz
> t293HJzFnGSkh1mUBew8dAsaZ4axWq2ml8UhQ%2FSTI%3D&amp;rese
> rved=0<https://nam06.safelinks.protection.outlook.com/?
> url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-
> libc&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61
> ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7
> cd011db47%7C1%7C0%7C637248265951717077&sdata=wpaAwogDjz
> DwRzpoB5vInxeqtMkNoawyPFg1j3m9omw%3D&reserved=0>)
> > * [edk2-
> staging](https://nam06.safelinks.protection.outlook.com/
> /?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-
> staging&amp;data=02%7C01%7Cbret.barkelew%40microsoft.co
> m%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af
> 91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata
> =bcNbt7Y7KoBrcnW4fAc4jbGgJL%2B4lYUkVLhYNo37OiM%3D&amp;r
> eserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protection.outlook.co%2F&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611252057&amp;sdata=bR1nvNPVi%2BQ1NrpSJQlwdLTZB3SigLFQ3KV63kUTUzk%3D&amp;reserved=0
> m/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-
> staging&data=02%7C01%7CBret.Barkelew%40microsoft.com%7C
> b61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab
> 2d7cd011db47%7C1%7C0%7C637248265951717077&sdata=5ehzG40
> XuHbF56QwzFHyY8krO1NwGDWnFVMZdZ9OUvM%3D&reserved=0>)
> >
> > # GitHub Pull Request Code Review Process
> >
> > **NOTE**: All steps below use
> [edk2](https://nam06.safelinks.protection.outlook.com/?
> url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&amp;dat
> a=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b
> 544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db4
> 7%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5
> Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0<https:/
> /nam06.safelinks.protection.outlook.com/?url=https%3A%2
> F%2Fgithub.com%2Ftianocore%2Fedk2&data=02%7C01%7CBret.B
> arkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebe
> b5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248
> 265951727054&sdata=IrlV0ptekGi4Sz%2FxBYhIa5LiRuI3jlKE9J
> zgwbKGP6o%3D&reserved=0>) as an
> > example.  Several repositories are supported.
> >
> > ## Author/Developer Steps
> >   * Create a personal fork of
> [edk2](https://nam06.safelinks.protection.outlook.com/?
> url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&amp;dat
> a=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b
> 544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db4
> 7%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5
> Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0<https:/
> /nam06.safelinks.protection.outlook.com/?url=https%3A%2
> F%2Fgithub.com%2Ftianocore%2Fedk2&data=02%7C01%7CBret.B
> arkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebe
> b5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248
> 265951727054&sdata=IrlV0ptekGi4Sz%2FxBYhIa5LiRuI3jlKE9J
> zgwbKGP6o%3D&reserved=0>)
> >
> >
> https://nam06.safelinks.protection.outlook.com/?url=htt
> ps%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fgetting-
> started-with-github%2Ffork-a-
> repo&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7
> C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91a
> b2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=um
> I3eqOh03qmt9YlPo33ujypu90YwImAvuxh5SlrM%2Bw%3D&amp;rese
> rved=0<https://nam06.safelinks.protection.outlook.com/?
> url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fgetti
> ng-started-with-github%2Ffork-a-
> repo&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61
> ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7
> cd011db47%7C1%7C0%7C637248265951737053&sdata=pC%2F9W90Z
> NlmGQFNy97ee1xQMJ93XC%2Bdo5lmIdZhnRk0%3D&reserved=0>
> >
> >   * Create a new branch from edk2/master in personal
> fork of edk2 repository.
> >
> >   * Add set of commits for new feature or bug fix to
> new branch.  Make sure to
> >     follow the commit message format requirements.
> The only change with this
> >     RFC is that the Cc: lines to
> maintainers/reviewers should **not** be added.
> >     The Cc: lines are still supported, but they
> should only be used to add
> >     reviewers that do not have GitHub IDs or are not
> members of TianoCore.
> >
> >   * Push branch with new commits to personal fork
> >   * Create a pull request against TianoCore
> edk2/master
> >
> >
> https://nam06.safelinks.protection.outlook.com/?url=htt
> ps%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fcollaboratin
> g-with-issues-and-pull-requests%2Fcreating-a-pull-
> request&amp;data=02%7C01%7Cbret.barkelew%40microsoft.co
> m%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af
> 91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata
> =2GVrQy0FGwd4CCeGveh99HL3zS1ekRfAAaKhhRiOMpU%3D&amp;res
> erved=0<https://nam06.safelinks.protection.outlook.com/
> ?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fcoll
> aborating-with-issues-and-pull-requests%2Fcreating-a-
> pull-
> request&data=02%7C01%7CBret.Barkelew%40microsoft.com%7C
> b61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab
> 2d7cd011db47%7C1%7C0%7C637248265951737053&sdata=sZjbyeR
> ye35une4C2y5RQ83ah%2B01o4XhFNXsvIXT1l0%3D&reserved=0>
> >
> >   * If pull request has more than 1 commit, then fill
> in the pull request title
> >     and decryption information for Patch #0.  Do not
> leave defaults.
>
> s/decryption/description/
>
> (Because I'm assuming this will turn into a wiki
> article at some point.)
>
> >
> >   * Do not assign reviewers.  The webhook assigns
> maintainers and reviewers to
> >     the pull request and each commit in the pull
> request.
> >
> >   * If maintainers/reviewers provide feedback that
> requires changes, then make
> >     add commits to the current branch with the
> requested changes.  Once all
>
> s/make add/add/
>
> >     changes are accepted on the current branch,
> reformulate the patch series and
> >     commit comments as needed for perform a forced
> push to the branch in the
> >     personal fork of the edk2 repository.  This step
> may be repeated if multiple
> >     versions of the patch series are required to
> address all code review
> >     feedback.
>
> Do I understand correctly that this recommends the
> contributor first
> push incremental patches on top of the series, then do
> a rebase
> (squashing updates as necessary) and finally do a
> force-push, for the
> next round of review?
>
> To me as a reviewer, that's extra work. I'm used to
> locally comparing
> the v(n) patch set to v(n+1) with git-range-diff,
> and/or with some
> personal scripts. I wouldn't encourage incremental
> patches appended --
> even temporarily -- to the branch, because (a) it's
> extra review work
> (it requires me to review something that has zero
> chance to get into the
> git history as-is), and (b) it superficially resembles
> the
> github.com-specific bad practice called "squash on
> merge", and (c) it
> runs the risk that the maintainer responsible for
> ultimately merging the
> series ends up actually merging the incremental (=
> "fixup") patches in
> isolation (without squashing them).
>
> >
> >   **OPEN**: How should minimum review period be set?
> Labels?
>
> Not sure about the best tooling. My recommendation
> would be to require
> reviewers to start providing their feedback within one
> week.
>
> One thing that I find important is that a maintainer
> can signal "I got
> your work in my queue, but I may need more time". And a
> special case of
> that are automated out-of-office responses. I think
> they are very
> helpful (when a contributor feels they are bottlenecked
> on review), but
> I'm not sure how one can configure that via github. I
> certainly would
> not share my out-of-office times with github. (I set
> the start/end dates
> in my email infrastructure, at the moment, but the out-
> of-office
> messages it sends do not contain the dates either, on
> purpose.)
>
> >
> > ## TianoCore GitHub Email Archive Webhook Service
> Steps
> >   * Receive an event that a new pull request was
> opened
> >   * Evaluate the files modified by the entire pull
> request and each commit in
> >     the pull request and cross references against
> `Maintainters.txt` in the root
>
> s/cross references/cross reference them/ ?
>
> >     of the repository to assign maintainers/reviewers
> to the pull request and
> >     each commit in the pull request. Individual
> commit assignments are performed
> >     by adding a commit comment of the following form:
> >
> >     [CodeReview] Review-request @mdkinney
> >
> >   * Generate and sends git patch review emails to the
> email archive.  Emails
>
> s/sends/send/
>
> >     are also sent to any Cc: tags in the commit
> messages.
> >
> >   * If the author/developer performs a forced push to
> the branch in their
> >     personal fork of the edk2 repository, then a new
> set of patch review emails
> >     with patch series Vx is sent to the email archive
> and any Cc: tags in commit
> >     messages.
> >
> >   * Receive events associated with all code review
> activities and generate
> >     and send emails to the email archive that shows
> all review comments and
> >     all responses closely matching the email contents
> seen in the current email
> >     based code review process.
> >
> >   * Generate and send email when pull request is
> merged or closed.
> >
> > ## Maintainer/Reviewer Steps
> >
> >   * Make sure GitHub configuration is setup to
> 'Watch' the repositories that
> >     you have maintainer ship or review
> responsibilities and that email
>
> s/maintainer ship/maintainership/
>
> >     notifications from GitHub are enabled.  This
> enables email notifications
> >     when a maintainer/reviewer is assigned to a pull
> request and individual
> >     commits.
> >
> >
> https://nam06.safelinks.protection.outlook.com/?url=htt
> ps%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fmanaging-
> subscriptions-and-notifications-on-
> github%2Fconfiguring-
> notifications&amp;data=02%7C01%7Cbret.barkelew%40micros
> oft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86
> f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp
> ;sdata=OlkiyymcQi39P8%2FOJZG4yjh4h%2FHerkHBe5bCSQQFLOU%
> 3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protection.ou%2F&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611252057&amp;sdata=rMEZi8aziILY8pK5e%2BvnY3TgsvV4EqzQ2qXf8QvaqIc%3D&amp;reserved=0
> tlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgit
> hub%2Fmanaging-subscriptions-and-notifications-on-
> github%2Fconfiguring-
> notifications&data=02%7C01%7CBret.Barkelew%40microsoft.
> com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141
> af91ab2d7cd011db47%7C1%7C0%7C637248265951747044&sdata=F
> JwHjpFL7UFtZ9N12pSvQSXci3IqCJJVJXI9dDzlHHU%3D&reserved=
> 0>
> >
> >   * Subscribe to the email archive associated with
> the TianoCore GitHub Email
> >     Archive Webhook Service.
> >
> >
> https://nam06.safelinks.protection.outlook.com/?url=htt
> ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Ftianoc
> ore-code-review-
> poc&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C
> 1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab
> 2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=q0I
> uvS318pEkJU2td9xX87oIm0LbSlEvOvhpyOOFrE8%3D&amp;reserve
> d=0<https://nam06.safelinks.protection.outlook.com/?url
> =https%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Fti
> anocore-code-review-
> poc&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61c
> e42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7c
> d011db47%7C1%7C0%7C637248265951747044&sdata=zbDuJd9lW3K
> oiuzGo6%2BU14Mnude7lZnbCeXQHJnR%2BCg%3D&reserved=0>
>
> Important: as the name says ("-poc"), this is a Proof
> of Concept list,
> for now. Once we're ready to switch over, I'll file an
> internal ticket
> at RH to either rename the list, or (which is probably
> better) to create
> a new list (no "-poc" suffix).
>
> The second option seems more useful because then the
> webhook development
> / bugfixing (if any) could perhaps occur in parallel to
> the normal edk2
> workflow.
>
> >
> >   * Review pull requests and commits assigned by the
> TianoCore GitHub Email
> >     Archive Webhook Service and use the GitHub web UI
> to provide all review
> >     feedback.
> >
> >
> https://nam06.safelinks.protection.outlook.com/?url=htt
> ps%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fcollaboratin
> g-with-issues-and-pull-requests%2Freviewing-changes-in-
> pull-
> requests&amp;data=02%7C01%7Cbret.barkelew%40microsoft.c
> om%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141a
> f91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdat
> a=51Ljm3wUbBTWT8hcaBD1ZQznSROvAQqnoTzQmD6K%2FLY%3D&amp;
> reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protection.outlook.c%2F&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611252057&amp;sdata=%2BfY0MvXLLyf%2Bt7w2VFkWgfnEGFjgqdJo7WsZmQN8yLE%3D&amp;reserved=0
> om/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fc
> ollaborating-with-issues-and-pull-requests%2Freviewing-
> changes-in-pull-
> requests&data=02%7C01%7CBret.Barkelew%40microsoft.com%7
> Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91a
> b2d7cd011db47%7C1%7C0%7C637248265951757040&sdata=Iav2d2
> fP2vw9mS9uA76tN3uQcIbl52D6BT963cD7ZC4%3D&reserved=0>
> >
> >   * Wait for Author/Developer to respond to all
> feedback and add commits with
> >     code changes as needed to resolve all feedback.
> This step may be repeated
> >     if the developer/author need to produce multiple
> versions of the patch
> >     series to address all feedback.
>
> (same question about the incremental fixup patches as
> above)
>
> >
> >   * Once all feedback is addressed, add Reviewed-by,
> Acked-by, and Tested-by
> >     responses on individual commits.  Or add Series-
> reviewed-by, Series-acked-by,
> >     or Series-Tested-by responses to the entire pull
> request.
> >
> >   * Wait for Developer/Author to add tags to commit
> messages in the pull request.
> >
> >   * Perform final review of patches and commit
> message tags.  If there are not
> >     issues, set the `push` label to run final set of
> CI checks and auto merge
> >     the pull request into master.
> >
> > # Maintainers.txt Format Changes
> >
> > Add GitHub IDs of all maintainers and reviewers at
> the end of M: and R: lines
> > in [].  For example:
> >
> >     M: Michael D Kinney
> <michael.d.kinney@intel.com<mailto:michael.d.kinney@int
> el.com>> [mdkinney]
> >
> > # TianoCore GitHub Email Archive Webhook Service
> >
> > Assign reviewers to commits in a GitHub pull request
> based on assignments
> > documented in Maintainers.txt and generates an email
> archive of all pull request
> > and code review activities.
>
> s/generates/generate/
>
> (or s/Assign/Assigns/)
>
> >
> >
> https://nam06.safelinks.protection.outlook.com/?url=htt
> ps%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2-email-archive-
> webhook&amp;data=02%7C01%7Cbret.barkelew%40microsoft.co
> m%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af
> 91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata
> =7CJNJMEXrxoynjavmEwjzUyRbfNUIZ3FEG4kDRXvhI4%3D&amp;res
> erved=0<https://nam06.safelinks.protection.outlook.com/
> ?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2-email-
> archive-
> webhook&data=02%7C01%7CBret.Barkelew%40microsoft.com%7C
> b61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab
> 2d7cd011db47%7C1%7C0%7C637248265951757040&sdata=FihVwcw
> fqewohsmqUECWG36TR1iCQVhVHq02iMZkEL8%3D&reserved=0>
> >
> > # Email Archive Subscription Service
> >
> > The emails are being delivered to the following
> RedHat email subscription
> > service.  Please subscribe to receive the emails and
> to be able to view the
> > email archives.
> >
> >
> https://nam06.safelinks.protection.outlook.com/?url=htt
> ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Ftianoc
> ore-code-review-
> poc&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C
> 1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab
> 2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=q0I
> uvS318pEkJU2td9xX87oIm0LbSlEvOvhpyOOFrE8%3D&amp;reserve
> d=0<https://nam06.safelinks.protection.outlook.com/?url
> =https%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Fti
> anocore-code-review-
> poc&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61c
> e42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7c
> d011db47%7C1%7C0%7C637248265951767033&sdata=482jeizloSf
> EobAk0F8%2BvP9FZptdF6oC90A%2BYSQZcds%3D&reserved=0>
> >
> > The email archives are at this link:
> >
> >
> https://nam06.safelinks.protection.outlook.com/?url=htt
> ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
> re-code-review-
> poc%2Findex.html&amp;data=02%7C01%7Cbret.barkelew%40mic
> rosoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988b
> f86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&
> amp;sdata=nedUfkmMmI5T6GtAxQCW4q6xt38%2FezeDYmfq6cpRD0M
> %3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protection.o%2F&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611252057&amp;sdata=ybz14MP2cfsQUGM2JxpsZNq2q%2BZU0e835ZNpKpV9M9Y%3D&amp;reserved=0
> utlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%
> 2Fprivate%2Ftianocore-code-review-
> poc%2Findex.html&data=02%7C01%7CBret.Barkelew%40microso
> ft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f
> 141af91ab2d7cd011db47%7C1%7C0%7C637248265951767033&sdat
> a=78HRihY2z%2Fll0qoC05RMh8I%2Bwri9rwIwblxNBYFPecw%3D&re
> served=0>
> >
> > The following sections show some example pull
> requests and code reviews to
> > help review the generated emails, their contents, and
> threading.
> >
> > ## Email Achieve Thread View
> >
> >
> https://nam06.safelinks.protection.outlook.com/?url=htt
> ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
> re-code-review-poc%2F2020-
> May%2Fthread.html%2300289&amp;data=02%7C01%7Cbret.barke
> lew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%
> 7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275
> 86678640&amp;sdata=GtrEudehfXiSU6ZwH2zKO35CPPPVk0ctZIzh
> kpI6DkE%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.prote%2F&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611252057&amp;sdata=ug4axzMxZLKuGz8Eps6xepyEjv1er8atkiOZDpFca2c%3D&amp;reserved=0
> ction.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fm
> ailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-
> May%2Fthread.html%2300289&data=02%7C01%7CBret.Barkelew%
> 40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72
> f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C63724826595177
> 7024&sdata=%2BU8aFUi4kPBeJdlyW%2FERNtp5Fum5gZrWkgCTI8pw
> z14%3D&reserved=0>
> >
> > ## Example patch series with 1 patch
> >
> >
> https://nam06.safelinks.protection.outlook.com/?url=htt
> ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
> re-code-review-poc%2F2020-
> May%2Fthread.html%2300340&amp;data=02%7C01%7Cbret.barke
> lew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%
> 7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275
> 86678640&amp;sdata=ZGpI8%2BzIA9OMFm3pSCc2DQ4F5ZxtDSAXtj
> dFjD%2BY3NA%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.p%2F&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611262052&amp;sdata=7pNJG%2BcGgplnp0gOQmPx8ly3IgxLk7DBPW0liL%2BQ97c%3D&amp;reserved=0
> rotection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com
> %2Fmailman%2Fprivate%2Ftianocore-code-review-
> poc%2F2020-
> May%2Fthread.html%2300340&data=02%7C01%7CBret.Barkelew%
> 40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72
> f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C63724826595177
> 7024&sdata=gXn%2B9z8MIdnJyUkk2xR1mK%2FL710jUDEeNhK74EUk
> b9A%3D&reserved=0>
> >
> > ## Example patch series with < 10 patches
> >
> > *
> https://nam06.safelinks.protection.outlook.com/?url=htt
> ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
> re-code-review-poc%2F2020-
> May%2Fmsg00289.html&amp;data=02%7C01%7Cbret.barkelew%40
> microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f9
> 88bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275866786
> 40&amp;sdata=JyaUyvYfZD7b%2F2wN%2BpS%2B68b%2BwyKoZ3Rba4
> ol%2FyahQVU%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.p%2F&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611262052&amp;sdata=7pNJG%2BcGgplnp0gOQmPx8ly3IgxLk7DBPW0liL%2BQ97c%3D&amp;reserved=0
> rotection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com
> %2Fmailman%2Fprivate%2Ftianocore-code-review-
> poc%2F2020-
> May%2Fmsg00289.html&data=02%7C01%7CBret.Barkelew%40micr
> osoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf
> 86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951787022&s
> data=yvE8%2F%2FKOJUj%2BD4ZmtT3fZPAb2KBsGcAqxt%2FKwRKa6h
> A%3D&reserved=0>
> > *
> https://nam06.safelinks.protection.outlook.com/?url=htt
> ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
> re-code-review-poc%2F2020-
> May%2Fmsg00030.html&amp;data=02%7C01%7Cbret.barkelew%40
> microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f9
> 88bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275866786
> 40&amp;sdata=bQHIJIQq4Pri8iK3vPxMDMWz%2BKtXcyuPdhr8y7gF
> pXA%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protectio%2F&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611262052&amp;sdata=g37erq7Um78njCs8L%2BZ75MBqyGEoZrIEk1epGOJ5va0%3D&amp;reserved=0
> n.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailm
> an%2Fprivate%2Ftianocore-code-review-poc%2F2020-
> May%2Fmsg00030.html&data=02%7C01%7CBret.Barkelew%40micr
> osoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf
> 86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951787022&s
> data=zz5BgISidxyQVI%2BQihn%2Fq%2Fv9%2B5%2Fn2vpaeyvFbIiA
> igs%3D&reserved=0>
> > *
> https://nam06.safelinks.protection.outlook.com/?url=htt
> ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
> re-code-review-poc%2F2020-
> May%2Fmsg00018.html&amp;data=02%7C01%7Cbret.barkelew%40
> microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f9
> 88bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275866786
> 40&amp;sdata=uMIRGOq%2BVCOSwDzXkG4yueYS4ZJ7BWfsp3Z4%2B9
> lh6hE%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protect%2F&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611262052&amp;sdata=GcYd84S79b4L09WB1b40cfbx5y%2FREQnWZsljxNeAm2U%3D&amp;reserved=0
> ion.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmai
> lman%2Fprivate%2Ftianocore-code-review-poc%2F2020-
> May%2Fmsg00018.html&data=02%7C01%7CBret.Barkelew%40micr
> osoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf
> 86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951797016&s
> data=yN0WkrPdk1EEvyPwlqCG%2B6HTOpa3iT85w2QhXQJ2xYE%3D&r
> eserved=0>
> > *
> https://nam06.safelinks.protection.outlook.com/?url=htt
> ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
> re-code-review-poc%2F2020-
> May%2Fmsg00008.html&amp;data=02%7C01%7Cbret.barkelew%40
> microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f9
> 88bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275866786
> 40&amp;sdata=3CBkdqDxRt4IxtECpWQdKJL%2Bf4HFqqHCXo4loxNT
> zAE%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protectio%2F&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611262052&amp;sdata=g37erq7Um78njCs8L%2BZ75MBqyGEoZrIEk1epGOJ5va0%3D&amp;reserved=0
> n.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailm
> an%2Fprivate%2Ftianocore-code-review-poc%2F2020-
> May%2Fmsg00008.html&data=02%7C01%7CBret.Barkelew%40micr
> osoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf
> 86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951797016&s
> data=qF2vw8XdQfY0ECclzhbpd60vIgTj2Bzl5RjC%2F4EdbA8%3D&r
> eserved=0>
> >
> > ## Example patch series with > 80 patches
> >
> > *
> https://nam06.safelinks.protection.outlook.com/?url=htt
> ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
> re-code-review-poc%2F2020-
> May%2Fmsg00198.html&amp;data=02%7C01%7Cbret.barkelew%40
> microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f9
> 88bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275866786
> 40&amp;sdata=fDfQnifOMzyzLMdP4xH8koKCiSj7ZiuYyrrSZXTf3d
> 4%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protection%2F&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611262052&amp;sdata=oLtR%2Br3OOn3hqOQ6glrp56aXidbNCpd%2F8w06mY8j7dA%3D&amp;reserved=0.
> outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman
> %2Fprivate%2Ftianocore-code-review-poc%2F2020-
> May%2Fmsg00198.html&data=02%7C01%7CBret.Barkelew%40micr
> osoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf
> 86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951807013&s
> data=uXAPx9iTDbQbiv40j3EiqIP51nlFLQ43ghvAsjt4dhs%3D&res
> erved=0>
> > *
> https://nam06.safelinks.protection.outlook.com/?url=htt
> ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
> re-code-review-poc%2F2020-
> May%2Fmsg00116.html&amp;data=02%7C01%7Cbret.barkelew%40
> microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f9
> 88bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275866786
> 40&amp;sdata=lcxA3tTna%2BdmTpcNMmPlS%2B47llMAcIEjhCEqxV
> 7TDOc%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protect%2F&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611262052&amp;sdata=GcYd84S79b4L09WB1b40cfbx5y%2FREQnWZsljxNeAm2U%3D&amp;reserved=0
> ion.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmai
> lman%2Fprivate%2Ftianocore-code-review-poc%2F2020-
> May%2Fmsg00116.html&data=02%7C01%7CBret.Barkelew%40micr
> osoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf
> 86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951807013&s
> data=4ANIbH2YDPjN7H4gHGtbgNkAbM9EeVFOf%2FoWqTiXtiA%3D&r
> eserved=0>
> > *
> https://nam06.safelinks.protection.outlook.com/?url=htt
> ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
> re-code-review-poc%2F2020-
> May%2Fmsg00035.html&amp;data=02%7C01%7Cbret.barkelew%40
> microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f9
> 88bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275866786
> 40&amp;sdata=CgvZ8e%2B7L4nacvRE35KqEyC%2F1CjDYP6wI10qn%
> 2BoX39Y%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.prote%2F&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611262052&amp;sdata=m34ZxHM9zr3VFz25TIiYpmxnqR0hPT96UIJppWgU4os%3D&amp;reserved=0
> ction.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fm
> ailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-
> May%2Fmsg00035.html&data=02%7C01%7CBret.Barkelew%40micr
> osoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf
> 86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951817004&s
> data=1jX%2FuPSMlRvzN9eZcv5U7k4JmwfJJwaVhDJ%2BrA6mPYc%3D
> &reserved=0>
> >
> > # Tasks to Complete
> >
> > * Create edk2-codereview repository for evaluation of
> new code review process.
> > * Add GitHub IDs to Maintainers.txt in edk2-
> codereview repository
> > * Update BaseTools/Scripts/GetMaintainer.py to be
> compatible with GitHub IDs at
> >   the end of M: and R: statements
> > * Update webhook to use Rabbit MQ to manage requests
> and emails
> > * Determine if webhook requests must be serialized?
> Current POC is serialized.
> > * Make sure webhook has error handling for all
> unexpected events/states.
> > * Add logging of all events and emails to webhook
>
> The logging sounds very useful, thank you.
>
> Whenever a log message relates to an email, please
> consider logging the
> message-id of that email, if possible.
>
> > * Add admin interface to webhook
> > * Deploy webhook on a production server with 24/7
> support
> >
> > # Ideas for Future Enhancements
> >
> > * Run PatchCheck.py before assigning
> maintainers/reviewers.
> > * Add a simple check that fails if a single patch
> spans more than one package.
>
> Hmmm, good idea in general, but there have been valid
> exceptions to this
> rule.
>
> > * Monitor comments for Reviewed-by, Acked-by, Tested-
> by, Series-Reviewed-by,
> >   Series-Acked-by, Series-Tested-by made by assigned
> maintainers/reviewers.
> >   Once all commits have required tags, auto update
> commit messages in the
> >   branch and wait for maintainer to set the `Push`
> label to run CI and auto
> >   merge if all CI checks pass.
>
> Thank you for writing this up (and for implementing the
> webhook)!
> Laszlo
>
>
>
>
>
>
>





[-- Attachment #2: Type: text/html, Size: 61319 bytes --]

Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

[Laszlo Ersek]

On 05/14/20 23:26, Bret Barkelew via groups.io wrote:

> It’s code management for the Instagram generation

I find this an extremely good characterization!

And, I find the fact soul-destroying.

Laszlo

Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

[Laszlo Ersek]

On 05/15/20 06:49, Bret Barkelew via groups.io wrote:

> I would far prefer the approach of individual PRs for commits to
> allow for the squash flexibility (and is the strategy I think I would
> pursue with my PRs). For example, the VarPol PR would be broken up
> into 9 PRs for each final commit, and we can get them in one by one.
> Ideally, each one would be a small back and forth and then in. If it
> had been done that way to begin with, it would be over in a week and
> a half or so, rather than the multiple months that we’re now verging
> on.

This differs extremely from how we've been working on edk2-devel (or
from how any git-based project works that I've ever been involved with).
And I think the above workflow is out of scope, for migrating the edk2
process to github.

Again, the structuring of a patch series is a primary trait. Iterating
only on individual patches does not allow for the reordering /
restructuring of the patch series (dropping patches, reordering patches,
inserting patches, moving hunks between patches).

It's common that the necessity to revise an earlier patch emerges while
reworking a later patch. For instance, the git-rebase(1) manual
dedicates a separate section to "splitting commits".

In the initial evaluation of "web forges", Phabricator was one of the
"contestants". Phabricator didn't support the "patch series" concept at
all, it only supported review requests for individual patches, and it
supported setting up dependencies between them. So, for example, a
27-patch series would require 27 submissions and 26 dependencies.

Lacking support for the patch series concept was an immediate deal
breaker with Phabricator.

The longest patch series I've ever submitted to edk2-devel had 58
patches. It was SMM enablement for OVMF. It went from v1 to v5 (v5 was
merged), and the patch count varied significantly:

v1: 58 patches (25 Jul 2015)
v2: 41 patches ( 9 Oct 2015)
v3: 52 patches (15 Oct 2015)
v4: 41 patches ( 3 Nov 2015)
v5: 33 patches (27 Nov 2015)

(The significant drop in the patch count was due to Mike Kinney open
sourcing and upstreaming the *real* PiSmmCpuDxeSmm driver (which was
huge work in its own right), allowing me to drop the Quark-originated
32-bit-only PiSmmCpuDxeSmm variant, from my series.)

The contribution process should make difficult things possible, even if
that complicates simple things somewhat. A process that makes simple
things simple and difficult things impossible is useless. This is what
the Instagram generation seems to be missing.


I don't know why the VariablePolicy work took months. I can see the
following threads on the list:

* [edk2-devel] [PATCH v1 0/9] Add the VariablePolicy feature
  Fri, 10 Apr 2020 11:36:01 -0700

* [edk2-devel] [PATCH v2 00/12] Add the VariablePolicy feature
  Mon, 11 May 2020 23:46:23 -0700

I have two sets of comments:

(1) It's difficult to tell in retrospect (because the series seem to
have been posted with somewhat problematic threading), but the delay
apparently came from multiple sources.

(1a) Review was slow and spotty.

The v1 blurb received some comments in the first week after it was
posted. But the rest of the v1 series (the actual patches) received
feedback like this:

- v1 1/9: no feedback
- v1 2/9: 12 days after posting
- v1 3/9: 16 days after posting
- v1 4/9: no feedback
- v1 5/9: no feedback
- v1 6/9: no feedback
- v1 7/9: no feedback
- v1 8/9: no feedback
- v1 9/9: no feedback

(1b) There was also quite some time between the last response in the v1
thread (Apr 26th, as far as I can see), and the posting of the v2 series
(May 11th).

(1c) The v2 blurb got almost immediate, and numerous feedback (on the
day of posting, and the day after). Regarding the individual patches,
they didn't fare too well:

- v2 01/12: superficial comment on the day of posting from me (not a
            designated MdeModulePkg review), on the day of posting; no
            other feedback thus far
- v2 02/12: ditto
- v2 03/12: no feedback
- v2 04/12: superficial (coding style) comments on the day of posting
- v2 05/12: no feedback
- v2 06/12: no feedback
- v2 07/12: no feedback
- v2 08/12: no feedback
- v2 09/12: no feedback
- v2 10/12: no feedback
- v2 11/12: reasonably in-depth review from responsible co-maintainer
            (yours truly), on the day of posting
- v2 12/12: no feedback

In total, I don't think the current process takes the blame for the
delay. If reviewers don't care (or have no time) now, that problem will
not change with the transition to github.com.


(2) The VariablePolicy series is actually a good example that patch
series restructuring is important.

(2a) The patch count went from 9 (in v1) to 12 (in v2).

(2b) And under v2, Liming still pointed out: "To keep each commit build
pass, the patch set should first add new library instance, then add the
library instance into each platform DSC, last update Variable driver to
consume new library instance."

Furthermore, I requested enabling the feature in ArmVirtPkg too, and
maybe (based on owner feedback) UefiPayloadPkg.

Thus, the v2->v3 update will most likely bring about both patch order
changes, and an increased patch count.

Thanks
Laszlo

Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

[Bret Barkelew]

[-- Attachment #1: Type: text/plain, Size: 687 bytes --]

“… boundless and bare, The lone and level sands stretch far away.”

- Bret

From: Laszlo Ersek<mailto:lersek@redhat.com>
Sent: Friday, May 15, 2020 12:34 AM
To: rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>; Bret Barkelew<mailto:Bret.Barkelew@microsoft.com>; Kinney, Michael D<mailto:michael.d.kinney@intel.com>; devel@edk2.groups.io<mailto:devel@edk2.groups.io>
Subject: Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

On 05/14/20 23:26, Bret Barkelew via groups.io wrote:

> It’s code management for the Instagram generation

I find this an extremely good characterization!

And, I find the fact soul-destroying.

Laszlo


[-- Attachment #2: Type: text/html, Size: 2540 bytes --]

Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

[Bret Barkelew]

[-- Attachment #1: Type: text/plain, Size: 6496 bytes --]

I agree with some of your points, but I don’t believe that this calls for dependencies at all.
If a PR can pass CI with the changes, it’s functionally unordered.
And if  a PR can’t, it has to wait until the PRs that can are in.

This also allows the group to focus on getting one thing done at a time.

I use rebase all the time and agree that it’s very good at precise history management. If a given PR requires that level of control, those tools will always be there.

But just as you say that the simple should not preclude the difficult, the difficult 5% should not needlessly complicated the simple 95%.

For what it’s worth, this is all posturing on my part. I intend – and, indeed, am eager to – follow the process that we’ve been helping Mike to set up.

- Bret

From: Laszlo Ersek<mailto:lersek@redhat.com>
Sent: Friday, May 15, 2020 2:08 AM
To: rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>; Bret Barkelew<mailto:Bret.Barkelew@microsoft.com>; devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Kinney, Michael D<mailto:michael.d.kinney@intel.com>
Subject: Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

On 05/15/20 06:49, Bret Barkelew via groups.io wrote:

> I would far prefer the approach of individual PRs for commits to
> allow for the squash flexibility (and is the strategy I think I would
> pursue with my PRs). For example, the VarPol PR would be broken up
> into 9 PRs for each final commit, and we can get them in one by one.
> Ideally, each one would be a small back and forth and then in. If it
> had been done that way to begin with, it would be over in a week and
> a half or so, rather than the multiple months that we’re now verging
> on.

This differs extremely from how we've been working on edk2-devel (or
from how any git-based project works that I've ever been involved with).
And I think the above workflow is out of scope, for migrating the edk2
process to github.

Again, the structuring of a patch series is a primary trait. Iterating
only on individual patches does not allow for the reordering /
restructuring of the patch series (dropping patches, reordering patches,
inserting patches, moving hunks between patches).

It's common that the necessity to revise an earlier patch emerges while
reworking a later patch. For instance, the git-rebase(1) manual
dedicates a separate section to "splitting commits".

In the initial evaluation of "web forges", Phabricator was one of the
"contestants". Phabricator didn't support the "patch series" concept at
all, it only supported review requests for individual patches, and it
supported setting up dependencies between them. So, for example, a
27-patch series would require 27 submissions and 26 dependencies.

Lacking support for the patch series concept was an immediate deal
breaker with Phabricator.

The longest patch series I've ever submitted to edk2-devel had 58
patches. It was SMM enablement for OVMF. It went from v1 to v5 (v5 was
merged), and the patch count varied significantly:

v1: 58 patches (25 Jul 2015)
v2: 41 patches ( 9 Oct 2015)
v3: 52 patches (15 Oct 2015)
v4: 41 patches ( 3 Nov 2015)
v5: 33 patches (27 Nov 2015)

(The significant drop in the patch count was due to Mike Kinney open
sourcing and upstreaming the *real* PiSmmCpuDxeSmm driver (which was
huge work in its own right), allowing me to drop the Quark-originated
32-bit-only PiSmmCpuDxeSmm variant, from my series.)

The contribution process should make difficult things possible, even if
that complicates simple things somewhat. A process that makes simple
things simple and difficult things impossible is useless. This is what
the Instagram generation seems to be missing.


I don't know why the VariablePolicy work took months. I can see the
following threads on the list:

* [edk2-devel] [PATCH v1 0/9] Add the VariablePolicy feature
  Fri, 10 Apr 2020 11:36:01 -0700

* [edk2-devel] [PATCH v2 00/12] Add the VariablePolicy feature
  Mon, 11 May 2020 23:46:23 -0700

I have two sets of comments:

(1) It's difficult to tell in retrospect (because the series seem to
have been posted with somewhat problematic threading), but the delay
apparently came from multiple sources.

(1a) Review was slow and spotty.

The v1 blurb received some comments in the first week after it was
posted. But the rest of the v1 series (the actual patches) received
feedback like this:

- v1 1/9: no feedback
- v1 2/9: 12 days after posting
- v1 3/9: 16 days after posting
- v1 4/9: no feedback
- v1 5/9: no feedback
- v1 6/9: no feedback
- v1 7/9: no feedback
- v1 8/9: no feedback
- v1 9/9: no feedback

(1b) There was also quite some time between the last response in the v1
thread (Apr 26th, as far as I can see), and the posting of the v2 series
(May 11th).

(1c) The v2 blurb got almost immediate, and numerous feedback (on the
day of posting, and the day after). Regarding the individual patches,
they didn't fare too well:

- v2 01/12: superficial comment on the day of posting from me (not a
            designated MdeModulePkg review), on the day of posting; no
            other feedback thus far
- v2 02/12: ditto
- v2 03/12: no feedback
- v2 04/12: superficial (coding style) comments on the day of posting
- v2 05/12: no feedback
- v2 06/12: no feedback
- v2 07/12: no feedback
- v2 08/12: no feedback
- v2 09/12: no feedback
- v2 10/12: no feedback
- v2 11/12: reasonably in-depth review from responsible co-maintainer
            (yours truly), on the day of posting
- v2 12/12: no feedback

In total, I don't think the current process takes the blame for the
delay. If reviewers don't care (or have no time) now, that problem will
not change with the transition to github.com.


(2) The VariablePolicy series is actually a good example that patch
series restructuring is important.

(2a) The patch count went from 9 (in v1) to 12 (in v2).

(2b) And under v2, Liming still pointed out: "To keep each commit build
pass, the patch set should first add new library instance, then add the
library instance into each platform DSC, last update Variable driver to
consume new library instance."

Furthermore, I requested enabling the feature in ArmVirtPkg too, and
maybe (based on owner feedback) UefiPayloadPkg.

Thus, the v2->v3 update will most likely bring about both patch order
changes, and an increased patch count.

Thanks
Laszlo


[-- Attachment #2: Type: text/html, Size: 9419 bytes --]

Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

[Rebecca Cran]

On 5/15/20 1:34 AM, Laszlo Ersek wrote:

> On 05/14/20 23:26, Bret Barkelew via groups.io wrote:
>
>> It’s code management for the Instagram generation
> I find this an extremely good characterization!
>
> And, I find the fact soul-destroying.

I was working on a web project recently, and apparently people don't 
even check email any more! So someone had set up a Slack channel where 
Github pull requests were posted/linked, and we were supposed to react 
with thumbs-up, "OK" etc. emoji to indicate we'd seen/reviewed/accepted 
the request.


-- 

Rebecca Cran

Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

[Philippe Mathieu-Daudé]

Hi Bret,

On 5/15/20 5:43 PM, Bret Barkelew via groups.io wrote:
> I agree with some of your points, but I don’t believe that this calls for dependencies at all.

Which points are you disagreeing?

> If a PR can pass CI with the changes, it’s functionally unordered.
> And if  a PR can’t, it has to wait until the PRs that can are in.
> 
> This also allows the group to focus on getting one thing done at a time.
> 
> I use rebase all the time and agree that it’s very good at precise history management. If a given PR requires that level of control, those tools will always be there.
> 
> But just as you say that the simple should not preclude the difficult, the difficult 5% should not needlessly complicated the simple 95%.
> 
> For what it’s worth, this is all posturing on my part. I intend – and, indeed, am eager to – follow the process that we’ve been helping Mike to set up.
> 
> - Bret
> 
> From: Laszlo Ersek<mailto:lersek@redhat.com>
> Sent: Friday, May 15, 2020 2:08 AM
> To: rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>; Bret Barkelew<mailto:Bret.Barkelew@microsoft.com>; devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Kinney, Michael D<mailto:michael.d.kinney@intel.com>
> Subject: Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process
> 
> On 05/15/20 06:49, Bret Barkelew via groups.io wrote:
> 
>> I would far prefer the approach of individual PRs for commits to
>> allow for the squash flexibility (and is the strategy I think I would
>> pursue with my PRs). For example, the VarPol PR would be broken up
>> into 9 PRs for each final commit, and we can get them in one by one.
>> Ideally, each one would be a small back and forth and then in. If it
>> had been done that way to begin with, it would be over in a week and
>> a half or so, rather than the multiple months that we’re now verging
>> on.
> 
> This differs extremely from how we've been working on edk2-devel (or
> from how any git-based project works that I've ever been involved with).
> And I think the above workflow is out of scope, for migrating the edk2
> process to github.
> 
> Again, the structuring of a patch series is a primary trait. Iterating
> only on individual patches does not allow for the reordering /
> restructuring of the patch series (dropping patches, reordering patches,
> inserting patches, moving hunks between patches).
> 
> It's common that the necessity to revise an earlier patch emerges while
> reworking a later patch. For instance, the git-rebase(1) manual
> dedicates a separate section to "splitting commits".
> 
> In the initial evaluation of "web forges", Phabricator was one of the
> "contestants". Phabricator didn't support the "patch series" concept at
> all, it only supported review requests for individual patches, and it
> supported setting up dependencies between them. So, for example, a
> 27-patch series would require 27 submissions and 26 dependencies.
> 
> Lacking support for the patch series concept was an immediate deal
> breaker with Phabricator.
> 
> The longest patch series I've ever submitted to edk2-devel had 58
> patches. It was SMM enablement for OVMF. It went from v1 to v5 (v5 was
> merged), and the patch count varied significantly:
> 
> v1: 58 patches (25 Jul 2015)
> v2: 41 patches ( 9 Oct 2015)
> v3: 52 patches (15 Oct 2015)
> v4: 41 patches ( 3 Nov 2015)
> v5: 33 patches (27 Nov 2015)
> 
> (The significant drop in the patch count was due to Mike Kinney open
> sourcing and upstreaming the *real* PiSmmCpuDxeSmm driver (which was
> huge work in its own right), allowing me to drop the Quark-originated
> 32-bit-only PiSmmCpuDxeSmm variant, from my series.)
> 
> The contribution process should make difficult things possible, even if
> that complicates simple things somewhat. A process that makes simple
> things simple and difficult things impossible is useless. This is what
> the Instagram generation seems to be missing.
> 
> 
> I don't know why the VariablePolicy work took months. I can see the
> following threads on the list:
> 
> * [edk2-devel] [PATCH v1 0/9] Add the VariablePolicy feature
>    Fri, 10 Apr 2020 11:36:01 -0700
> 
> * [edk2-devel] [PATCH v2 00/12] Add the VariablePolicy feature
>    Mon, 11 May 2020 23:46:23 -0700
> 
> I have two sets of comments:
> 
> (1) It's difficult to tell in retrospect (because the series seem to
> have been posted with somewhat problematic threading), but the delay
> apparently came from multiple sources.
> 
> (1a) Review was slow and spotty.

IIUC it is easier for the "Instagram generation" to write a GitHub
plugin which ping an unmerged pullrequest for them, rather than tracking
their WiP and send a "ping" via an email client.

That reminds me of the Prophet tool:

Prophet: The first generate-and-validate tool that uses machine learning
techniques to learn useful knowledge from past human patches to
recognize correct patches. It is evaluated on the same benchmark as
GenProg and generate correct patches (i.e., equivalent to human patches)
for 18 out of 69 cases.

https://groups.csail.mit.edu/pac/patchgen/
https://en.wikipedia.org/wiki/Automatic_bug_fixing#C [8]

Use it as source, combined with a fuzzer that open GH pull-requests, and 
see if a patch get merged... /s

> 
> The v1 blurb received some comments in the first week after it was
> posted. But the rest of the v1 series (the actual patches) received
> feedback like this:
> 
> - v1 1/9: no feedback
> - v1 2/9: 12 days after posting
> - v1 3/9: 16 days after posting
> - v1 4/9: no feedback
> - v1 5/9: no feedback
> - v1 6/9: no feedback
> - v1 7/9: no feedback
> - v1 8/9: no feedback
> - v1 9/9: no feedback
> 
> (1b) There was also quite some time between the last response in the v1
> thread (Apr 26th, as far as I can see), and the posting of the v2 series
> (May 11th).
> 
> (1c) The v2 blurb got almost immediate, and numerous feedback (on the
> day of posting, and the day after). Regarding the individual patches,
> they didn't fare too well:
> 
> - v2 01/12: superficial comment on the day of posting from me (not a
>              designated MdeModulePkg review), on the day of posting; no
>              other feedback thus far
> - v2 02/12: ditto
> - v2 03/12: no feedback
> - v2 04/12: superficial (coding style) comments on the day of posting
> - v2 05/12: no feedback
> - v2 06/12: no feedback
> - v2 07/12: no feedback
> - v2 08/12: no feedback
> - v2 09/12: no feedback
> - v2 10/12: no feedback
> - v2 11/12: reasonably in-depth review from responsible co-maintainer
>              (yours truly), on the day of posting
> - v2 12/12: no feedback
> 
> In total, I don't think the current process takes the blame for the
> delay. If reviewers don't care (or have no time) now, that problem will
> not change with the transition to github.com.
> 
> 
> (2) The VariablePolicy series is actually a good example that patch
> series restructuring is important.
> 
> (2a) The patch count went from 9 (in v1) to 12 (in v2).
> 
> (2b) And under v2, Liming still pointed out: "To keep each commit build
> pass, the patch set should first add new library instance, then add the
> library instance into each platform DSC, last update Variable driver to
> consume new library instance."
> 
> Furthermore, I requested enabling the feature in ArmVirtPkg too, and
> maybe (based on owner feedback) UefiPayloadPkg.
> 
> Thus, the v2->v3 update will most likely bring about both patch order
> changes, and an increased patch count.
> 
> Thanks
> Laszlo
> 
> 
> 
> 

Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

[Nate DeSimone]

[-- Attachment #1.1: Type: text/plain, Size: 8034 bytes --]

Hi All,



I tend to agree with most of Laszlo's points. Specifically, that moving to pull requests will not fix the fact that maintainers are usually busy people and don't always give feedback in a punctual manner. Like Laszlo, I would also prefer that we do not squash patch series. My biggest reason for not squashing patch series is because when you put everything into a single commit, I have had to review commits with 500+ files changed. Opening git difftool on a commit like that is awful.



However, I would like to register my general endorsement for pull requests or some other web based system of code review… and I don’t have an Instagram account by the way :) Personally, I prefer Gerrit as I use it a lot with coreboot and other projects. But since we are using Github for hosting, pull requests are an easy switch and a logical choice. My main reason for being excited about pull requests mostly has to do with the amount of manual effort required to be a TianoCore maintainer right now. I have set up my email filter so that the mailing list is categorized like so:



[cid:image001.png@01D62D71.502B55E0]



Implementing the logic to parse the contents of emails to categorize them like this required me to define no less than 12 email filter rules in Microsoft Outlook, and I have to change my filtering logic every time I am added/removed from a Maintainers.txt file. I’m sure every other maintainer has spent a time separately implementing filtering logic like I have. This helps, but still for every thread, I have to go and check if one of the other maintainers has already reviewed/pushed that patch series yet, and if not review/push it. If I have ] feedback on a patch series, I have to categorize it as awaiting response from author and check up on it from time to time, sometimes I ping the author directly and remind them to send a new patch series. Implementing this state machine is a lot of manual work and it kind of feels like I’m a telephone operator in the 1950s. I greatly welcome automation here as I am sure it will increase the number of patch series I am able to review per hour.



Thanks,

Nate



-----Original Message-----
From: rfc@edk2.groups.io <rfc@edk2.groups.io> On Behalf Of Laszlo Ersek
Sent: Friday, May 15, 2020 2:08 AM
To: rfc@edk2.groups.io; bret.barkelew@microsoft.com; devel@edk2.groups.io; Kinney, Michael D <michael.d.kinney@intel.com>
Subject: Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process



On 05/15/20 06:49, Bret Barkelew via groups.io wrote:



> I would far prefer the approach of individual PRs for commits to allow

> for the squash flexibility (and is the strategy I think I would pursue

> with my PRs). For example, the VarPol PR would be broken up into 9 PRs

> for each final commit, and we can get them in one by one.

> Ideally, each one would be a small back and forth and then in. If it

> had been done that way to begin with, it would be over in a week and a

> half or so, rather than the multiple months that we’re now verging

> on.



This differs extremely from how we've been working on edk2-devel (or from how any git-based project works that I've ever been involved with).

And I think the above workflow is out of scope, for migrating the edk2 process to github.



Again, the structuring of a patch series is a primary trait. Iterating only on individual patches does not allow for the reordering / restructuring of the patch series (dropping patches, reordering patches, inserting patches, moving hunks between patches).



It's common that the necessity to revise an earlier patch emerges while reworking a later patch. For instance, the git-rebase(1) manual dedicates a separate section to "splitting commits".



In the initial evaluation of "web forges", Phabricator was one of the "contestants". Phabricator didn't support the "patch series" concept at all, it only supported review requests for individual patches, and it supported setting up dependencies between them. So, for example, a 27-patch series would require 27 submissions and 26 dependencies.



Lacking support for the patch series concept was an immediate deal breaker with Phabricator.



The longest patch series I've ever submitted to edk2-devel had 58 patches. It was SMM enablement for OVMF. It went from v1 to v5 (v5 was merged), and the patch count varied significantly:



v1: 58 patches (25 Jul 2015)

v2: 41 patches ( 9 Oct 2015)

v3: 52 patches (15 Oct 2015)

v4: 41 patches ( 3 Nov 2015)

v5: 33 patches (27 Nov 2015)



(The significant drop in the patch count was due to Mike Kinney open sourcing and upstreaming the *real* PiSmmCpuDxeSmm driver (which was huge work in its own right), allowing me to drop the Quark-originated 32-bit-only PiSmmCpuDxeSmm variant, from my series.)



The contribution process should make difficult things possible, even if that complicates simple things somewhat. A process that makes simple things simple and difficult things impossible is useless. This is what the Instagram generation seems to be missing.





I don't know why the VariablePolicy work took months. I can see the following threads on the list:



* [edk2-devel] [PATCH v1 0/9] Add the VariablePolicy feature

  Fri, 10 Apr 2020 11:36:01 -0700



* [edk2-devel] [PATCH v2 00/12] Add the VariablePolicy feature

  Mon, 11 May 2020 23:46:23 -0700



I have two sets of comments:



(1) It's difficult to tell in retrospect (because the series seem to have been posted with somewhat problematic threading), but the delay apparently came from multiple sources.



(1a) Review was slow and spotty.



The v1 blurb received some comments in the first week after it was posted. But the rest of the v1 series (the actual patches) received feedback like this:



- v1 1/9: no feedback

- v1 2/9: 12 days after posting

- v1 3/9: 16 days after posting

- v1 4/9: no feedback

- v1 5/9: no feedback

- v1 6/9: no feedback

- v1 7/9: no feedback

- v1 8/9: no feedback

- v1 9/9: no feedback



(1b) There was also quite some time between the last response in the v1 thread (Apr 26th, as far as I can see), and the posting of the v2 series (May 11th).



(1c) The v2 blurb got almost immediate, and numerous feedback (on the day of posting, and the day after). Regarding the individual patches, they didn't fare too well:



- v2 01/12: superficial comment on the day of posting from me (not a

            designated MdeModulePkg review), on the day of posting; no

            other feedback thus far

- v2 02/12: ditto

- v2 03/12: no feedback

- v2 04/12: superficial (coding style) comments on the day of posting

- v2 05/12: no feedback

- v2 06/12: no feedback

- v2 07/12: no feedback

- v2 08/12: no feedback

- v2 09/12: no feedback

- v2 10/12: no feedback

- v2 11/12: reasonably in-depth review from responsible co-maintainer

            (yours truly), on the day of posting

- v2 12/12: no feedback



In total, I don't think the current process takes the blame for the delay. If reviewers don't care (or have no time) now, that problem will not change with the transition to github.com.





(2) The VariablePolicy series is actually a good example that patch series restructuring is important.



(2a) The patch count went from 9 (in v1) to 12 (in v2).



(2b) And under v2, Liming still pointed out: "To keep each commit build pass, the patch set should first add new library instance, then add the library instance into each platform DSC, last update Variable driver to consume new library instance."



Furthermore, I requested enabling the feature in ArmVirtPkg too, and maybe (based on owner feedback) UefiPayloadPkg.



Thus, the v2->v3 update will most likely bring about both patch order changes, and an increased patch count.



Thanks

Laszlo









[-- Attachment #1.2: Type: text/html, Size: 15040 bytes --]

[-- Attachment #2: image001.png --]
[-- Type: image/png, Size: 2968 bytes --]

Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

[Laszlo Ersek]

On 05/19/20 09:21, Desimone, Nathaniel L wrote:

> However, I would like to register my general endorsement for pull
> requests or some other web based system of code review... and I don't
> have an Instagram account by the way :) Personally, I prefer Gerrit as
> I use it a lot with coreboot and other projects. But since we are
> using Github for hosting, pull requests are an easy switch and a
> logical choice. My main reason for being excited about pull requests
> mostly has to do with the amount of manual effort required to be a
> TianoCore maintainer right now.

My understanding is that, at this point, we're inevitably going to
migrate the contribution/review workflow to GitHub. I believe the switch
is going to happen once the email webhook has been deemed functional and
stable enough by the community.

Digression starts:

> Implementing the logic to parse the contents of emails to categorize
> them like this required me to define no less than 12 email filter
> rules in Microsoft Outlook, and I have to change my filtering logic
> every time I am added/removed from a Maintainers.txt file.

That seems strange. I have one rule per edk2-* list, for moving such
incoming email into the appropriate list folder. That's all.

While I read all the subject lines (skim all the threads) on edk2-devel,
generally, if you share reviewer or maintainer responsibilities for some
subsystem, then people posting patches for that subsystem are supposed
to CC you explicitly, in addition to messaging the list.

How you handle messages from then on may be a personal matter of course.
I simply tag ("star") such messages (patches / series pending my
review), and I revisit my "set of starred messages" every day (sometimes
multiple times per day).

> I'm sure every other maintainer has spent a time separately
> implementing filtering logic like I have. This helps, but still for
> every thread, I have to go and check if one of the other maintainers
> has already reviewed/pushed that patch series yet, and if not
> review/push it.

Checking whether others have commented is near trivial if your MUA
supports a threaded view.

Checking whether a co-maintainer of yours has pushed a given series is
also simple if they diligently report the fact of merging on the list
(in the subject patch threads).

> If I have feedback on a patch series, I have to categorize it as
> awaiting response from author and check up on it from time to time,
> sometimes I ping the author directly and remind them to send a new
> patch series.

I think this is not your job, as a reviewer/maintainer. Once your review
is complete, or blocked on a question you need an answer to, the ball is
back in the contributor's court. They can answer, or post the next
version, whenever they see fit. Until then, the most they can expect of
you is answering any further questions they might have for understanding
your previous feedback better. You need not push contributors to
complete their contributions.

> Implementing this state machine is a lot of manual work and it kind of
> feels like I'm a telephone operator in the 1950s. I greatly welcome
> automation here as I am sure it will increase the number of patch
> series I am able to review per hour.

"State machine" is a very good analogy! Personally, I don't find it
tiresome. Yes, it's important to recognize the events (= new emails)
that trigger transitions between states. (For example: when I complete a
review, when I get a new version of a series or a brand new series, when
I get asked a question.) Once I recognize those events correctly, I just
diligently massage said tags ("stars").

And I keep iterating over my set of "starred" messages; I do actual work
(e.g., reviews) in "bottom halves"; detached from new emails.

I don't find this a burden as I have to manage my "real life" with task
lists anyway. Without them, my real life would collapse in a week; so
it's nothing unusual for me. (And no, I don't allow shady cloud-based
automatisms to manage my life for me; I value my privacy way above my
comfort.)

Thanks!
Laszlo

Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

[Sean]

Nate/Laszlo,

Regarding a squash merge workflow.  I agree it can be abused and we all 
have seen terrible examples.  But a patch series that contains 500+ file 
changes isn't really much better.  Just because it is broken into 
multiple commits doesn't mean it is the right set of commits.

Anyway a squash merge workflow works amazingly well with and is 
optimized for a web based review and PR processes.  It allows a user to 
respond to changes, fix issues, learn thru the PR process, all while 
keeping complete track of the progression.  Then once all "status" 
checks and reviews are complete, it is squashed into a neat commit for 
mainline, containing only the relevant data in the message.

So, the ask is that we don't exclude squash merge workflows.  Those 
reviewing the PR can decide what is appropriate for the PR content 
submitted.  Just as you would request changes to the contents (or 
ordering) of a commit in a series, if the reviewers don't agree that the 
PR contents should be in a single commit then obviously it shouldn't be 
squashed to one.

Contributions like spelling fixes, typos, minor bug fixes, documentation 
additions/fixes, etc all are great examples of PRs that can easily 
leverage squash merges and this workflow significantly lowers the burden 
of the contribution and review process.  This workflow is also are much 
easier for casual or first time contributors.

I don't exactly know how we would enable this but I assume we could 
leverage tags or make it clear in the PR description.  First step is to 
get alignment that a squash merge workflow, while not appropriate for 
all contributions, is not something to be excluded.

Thanks
Sean



On 5/19/2020 12:21 AM, Nate DeSimone wrote:
> Hi All,
> 
> 
> 
> I tend to agree with most of Laszlo's points. Specifically, that moving to pull requests will not fix the fact that maintainers are usually busy people and don't always give feedback in a punctual manner. Like Laszlo, I would also prefer that we do not squash patch series. My biggest reason for not squashing patch series is because when you put everything into a single commit, I have had to review commits with 500+ files changed. Opening git difftool on a commit like that is awful.
> 
> 
> 
> However, I would like to register my general endorsement for pull requests or some other web based system of code review… and I don’t have an Instagram account by the way :) Personally, I prefer Gerrit as I use it a lot with coreboot and other projects. But since we are using Github for hosting, pull requests are an easy switch and a logical choice. My main reason for being excited about pull requests mostly has to do with the amount of manual effort required to be a TianoCore maintainer right now. I have set up my email filter so that the mailing list is categorized like so:
> 
> 
> 
> [cid:image001.png@01D62D71.502B55E0]
> 
> 
> 
> Implementing the logic to parse the contents of emails to categorize them like this required me to define no less than 12 email filter rules in Microsoft Outlook, and I have to change my filtering logic every time I am added/removed from a Maintainers.txt file. I’m sure every other maintainer has spent a time separately implementing filtering logic like I have. This helps, but still for every thread, I have to go and check if one of the other maintainers has already reviewed/pushed that patch series yet, and if not review/push it. If I have ] feedback on a patch series, I have to categorize it as awaiting response from author and check up on it from time to time, sometimes I ping the author directly and remind them to send a new patch series. Implementing this state machine is a lot of manual work and it kind of feels like I’m a telephone operator in the 1950s. I greatly welcome automation here as I am sure it will increase the number of patch series I am able to review per hour.
> 
> 
> 
> Thanks,
> 
> Nate
> 
> 
> 
> -----Original Message-----
> From: rfc@edk2.groups.io <rfc@edk2.groups.io> On Behalf Of Laszlo Ersek
> Sent: Friday, May 15, 2020 2:08 AM
> To: rfc@edk2.groups.io; bret.barkelew@microsoft.com; devel@edk2.groups.io; Kinney, Michael D <michael.d.kinney@intel.com>
> Subject: Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process
> 
> 
> 
> On 05/15/20 06:49, Bret Barkelew via groups.io wrote:
> 
> 
> 
>> I would far prefer the approach of individual PRs for commits to allow
> 
>> for the squash flexibility (and is the strategy I think I would pursue
> 
>> with my PRs). For example, the VarPol PR would be broken up into 9 PRs
> 
>> for each final commit, and we can get them in one by one.
> 
>> Ideally, each one would be a small back and forth and then in. If it
> 
>> had been done that way to begin with, it would be over in a week and a
> 
>> half or so, rather than the multiple months that we’re now verging
> 
>> on.
> 
> 
> 
> This differs extremely from how we've been working on edk2-devel (or from how any git-based project works that I've ever been involved with).
> 
> And I think the above workflow is out of scope, for migrating the edk2 process to github.
> 
> 
> 
> Again, the structuring of a patch series is a primary trait. Iterating only on individual patches does not allow for the reordering / restructuring of the patch series (dropping patches, reordering patches, inserting patches, moving hunks between patches).
> 
> 
> 
> It's common that the necessity to revise an earlier patch emerges while reworking a later patch. For instance, the git-rebase(1) manual dedicates a separate section to "splitting commits".
> 
> 
> 
> In the initial evaluation of "web forges", Phabricator was one of the "contestants". Phabricator didn't support the "patch series" concept at all, it only supported review requests for individual patches, and it supported setting up dependencies between them. So, for example, a 27-patch series would require 27 submissions and 26 dependencies.
> 
> 
> 
> Lacking support for the patch series concept was an immediate deal breaker with Phabricator.
> 
> 
> 
> The longest patch series I've ever submitted to edk2-devel had 58 patches. It was SMM enablement for OVMF. It went from v1 to v5 (v5 was merged), and the patch count varied significantly:
> 
> 
> 
> v1: 58 patches (25 Jul 2015)
> 
> v2: 41 patches ( 9 Oct 2015)
> 
> v3: 52 patches (15 Oct 2015)
> 
> v4: 41 patches ( 3 Nov 2015)
> 
> v5: 33 patches (27 Nov 2015)
> 
> 
> 
> (The significant drop in the patch count was due to Mike Kinney open sourcing and upstreaming the *real* PiSmmCpuDxeSmm driver (which was huge work in its own right), allowing me to drop the Quark-originated 32-bit-only PiSmmCpuDxeSmm variant, from my series.)
> 
> 
> 
> The contribution process should make difficult things possible, even if that complicates simple things somewhat. A process that makes simple things simple and difficult things impossible is useless. This is what the Instagram generation seems to be missing.
> 
> 
> 
> 
> 
> I don't know why the VariablePolicy work took months. I can see the following threads on the list:
> 
> 
> 
> * [edk2-devel] [PATCH v1 0/9] Add the VariablePolicy feature
> 
>    Fri, 10 Apr 2020 11:36:01 -0700
> 
> 
> 
> * [edk2-devel] [PATCH v2 00/12] Add the VariablePolicy feature
> 
>    Mon, 11 May 2020 23:46:23 -0700
> 
> 
> 
> I have two sets of comments:
> 
> 
> 
> (1) It's difficult to tell in retrospect (because the series seem to have been posted with somewhat problematic threading), but the delay apparently came from multiple sources.
> 
> 
> 
> (1a) Review was slow and spotty.
> 
> 
> 
> The v1 blurb received some comments in the first week after it was posted. But the rest of the v1 series (the actual patches) received feedback like this:
> 
> 
> 
> - v1 1/9: no feedback
> 
> - v1 2/9: 12 days after posting
> 
> - v1 3/9: 16 days after posting
> 
> - v1 4/9: no feedback
> 
> - v1 5/9: no feedback
> 
> - v1 6/9: no feedback
> 
> - v1 7/9: no feedback
> 
> - v1 8/9: no feedback
> 
> - v1 9/9: no feedback
> 
> 
> 
> (1b) There was also quite some time between the last response in the v1 thread (Apr 26th, as far as I can see), and the posting of the v2 series (May 11th).
> 
> 
> 
> (1c) The v2 blurb got almost immediate, and numerous feedback (on the day of posting, and the day after). Regarding the individual patches, they didn't fare too well:
> 
> 
> 
> - v2 01/12: superficial comment on the day of posting from me (not a
> 
>              designated MdeModulePkg review), on the day of posting; no
> 
>              other feedback thus far
> 
> - v2 02/12: ditto
> 
> - v2 03/12: no feedback
> 
> - v2 04/12: superficial (coding style) comments on the day of posting
> 
> - v2 05/12: no feedback
> 
> - v2 06/12: no feedback
> 
> - v2 07/12: no feedback
> 
> - v2 08/12: no feedback
> 
> - v2 09/12: no feedback
> 
> - v2 10/12: no feedback
> 
> - v2 11/12: reasonably in-depth review from responsible co-maintainer
> 
>              (yours truly), on the day of posting
> 
> - v2 12/12: no feedback
> 
> 
> 
> In total, I don't think the current process takes the blame for the delay. If reviewers don't care (or have no time) now, that problem will not change with the transition to github.com.
> 
> 
> 
> 
> 
> (2) The VariablePolicy series is actually a good example that patch series restructuring is important.
> 
> 
> 
> (2a) The patch count went from 9 (in v1) to 12 (in v2).
> 
> 
> 
> (2b) And under v2, Liming still pointed out: "To keep each commit build pass, the patch set should first add new library instance, then add the library instance into each platform DSC, last update Variable driver to consume new library instance."
> 
> 
> 
> Furthermore, I requested enabling the feature in ArmVirtPkg too, and maybe (based on owner feedback) UefiPayloadPkg.
> 
> 
> 
> Thus, the v2->v3 update will most likely bring about both patch order changes, and an increased patch count.
> 
> 
> 
> Thanks
> 
> Laszlo
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 

Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

[Nate DeSimone]

> On 5/19/20 01:40, Laszlo Ersek wrote:
> 
> That seems strange. I have one rule per edk2-* list, for moving such incoming
> email into the appropriate list folder. That's all.
> 
> While I read all the subject lines (skim all the threads) on edk2-devel,
> generally, if you share reviewer or maintainer responsibilities for some
> subsystem, then people posting patches for that subsystem are supposed to
> CC you explicitly, in addition to messaging the list.

I tend to make the assumption that people do not CC me on the patches that they are supposed to CC me on. So I set up my filtering rules to do a deep inspection of the message contents to see if it touches a package that I maintain.

> Checking whether others have commented is near trivial if your MUA
> supports a threaded view.
> 
> Checking whether a co-maintainer of yours has pushed a given series is also
> simple if they diligently report the fact of merging on the list (in the subject
> patch threads).

Yes, checking for comments is trivial. However, my fellow co-maintainers are not very diligent on sending push notifications. So when I see comments from one of my fellow co-maintainers I immediately ask myself the question: "Did they already push this, and does it make sense for me to spend time reviewing this patch series?" Answering that question involves a git pull and a review of history in gitk to see what has been done already.

> I think this is not your job, as a reviewer/maintainer. Once your review is
> complete, or blocked on a question you need an answer to, the ball is back in
> the contributor's court. They can answer, or post the next version, whenever
> they see fit. Until then, the most they can expect of you is answering any
> further questions they might have for understanding your previous feedback
> better. You need not push contributors to complete their contributions.

I think my experience is colored somewhat here. I'd say more than half the time, the contributor is another Intel employee. Often times, they are contributing code changes that I asked them to implement. :)

> "State machine" is a very good analogy! Personally, I don't find it tiresome.
> Yes, it's important to recognize the events (= new emails) that trigger
> transitions between states. (For example: when I complete a review, when I
> get a new version of a series or a brand new series, when I get asked a
> question.) Once I recognize those events correctly, I just diligently massage
> said tags ("stars").
> 
> And I keep iterating over my set of "starred" messages; I do actual work
> (e.g., reviews) in "bottom halves"; detached from new emails.
> 
> I don't find this a burden as I have to manage my "real life" with task lists
> anyway. Without them, my real life would collapse in a week; so it's nothing
> unusual for me. (And no, I don't allow shady cloud-based automatisms to
> manage my life for me; I value my privacy way above my
> comfort.)

Agreed that I also keep my personal task lists in a paper notebook and manage my real life list manually. However, my real life list is much smaller (since I have most of the context in my head already)... and its private. Everything I do on this mailing list is public anyway, so having some centralized service keep track of state transitions doesn't bother me. The "bottom half" of that state transition is going to generate a public email from my address, so it's not like the current state of the state machine that I'm running in my head is private.

Thanks,
Nate

> Thanks!
> Laszlo
> 
> 
> 

Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

[Nate DeSimone]

Hi Sean,

My recent spelling fix patch series is a good example of why this is a bad idea actually:

https://edk2.groups.io/g/devel/message/59779
https://edk2.groups.io/g/devel/message/59780
https://edk2.groups.io/g/devel/message/59781
https://edk2.groups.io/g/devel/message/59782
https://edk2.groups.io/g/devel/message/59783
https://edk2.groups.io/g/devel/message/59784
https://edk2.groups.io/g/devel/message/59785

Notice that I split along package boundaries, because the maintainers for each package is a different set of people. If my patch series was squashed at merge time... how do I know who reviewed what? If the commit set is not correct.. I tend to say so in my feedback :). The only sane way to squash this series would be to have a human re-write all the commit messages, which I am against.

Generally those that prefer an easily bisectable history have such preference mostly due to the usage of validators that immediately resort bisecting as a method to root cause an issue since they tend to not understand the code very well. Edk2 already has 12 years of non-bisectable history, so this method is going to be ineffective anyway.

With regard to sending squashed commits, I understand that those who are new may have difficulty sending a properly formatted patch series, but frankly attempting to shield them from having to learn I am strongly against. I suggest that Microsoft invest in its human capital similar to how Intel does. If you cannot figure out how to send a properly formatted patch series... then do your work on the internal codebase (or perhaps MU.) Within the Intel, having the skillset to contribute to TianoCore is considered a mark of prestige, and thus needs to be earned.

TLDR, I will reject squashed commits on any packages that I maintain.

Thanks,
Nate

> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Sean
> Sent: Tuesday, May 19, 2020 9:54 AM
> To: rfc@edk2.groups.io; Desimone, Nathaniel L
> <nathaniel.l.desimone@intel.com>; lersek@redhat.com;
> bret.barkelew@microsoft.com; devel@edk2.groups.io; Kinney, Michael D
> <michael.d.kinney@intel.com>
> Subject: Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review
> Process
> 
> Nate/Laszlo,
> 
> Regarding a squash merge workflow.  I agree it can be abused and we all
> have seen terrible examples.  But a patch series that contains 500+ file
> changes isn't really much better.  Just because it is broken into multiple
> commits doesn't mean it is the right set of commits.
> 
> Anyway a squash merge workflow works amazingly well with and is
> optimized for a web based review and PR processes.  It allows a user to
> respond to changes, fix issues, learn thru the PR process, all while keeping
> complete track of the progression.  Then once all "status"
> checks and reviews are complete, it is squashed into a neat commit for
> mainline, containing only the relevant data in the message.
> 
> So, the ask is that we don't exclude squash merge workflows.  Those
> reviewing the PR can decide what is appropriate for the PR content
> submitted.  Just as you would request changes to the contents (or
> ordering) of a commit in a series, if the reviewers don't agree that the PR
> contents should be in a single commit then obviously it shouldn't be
> squashed to one.
> 
> Contributions like spelling fixes, typos, minor bug fixes, documentation
> additions/fixes, etc all are great examples of PRs that can easily leverage
> squash merges and this workflow significantly lowers the burden of the
> contribution and review process.  This workflow is also are much easier for
> casual or first time contributors.
> 
> I don't exactly know how we would enable this but I assume we could
> leverage tags or make it clear in the PR description.  First step is to get
> alignment that a squash merge workflow, while not appropriate for all
> contributions, is not something to be excluded.
> 
> Thanks
> Sean
> 
> 
> 
> On 5/19/2020 12:21 AM, Nate DeSimone wrote:
> > Hi All,
> >
> >
> >
> > I tend to agree with most of Laszlo's points. Specifically, that moving to pull
> requests will not fix the fact that maintainers are usually busy people and
> don't always give feedback in a punctual manner. Like Laszlo, I would also
> prefer that we do not squash patch series. My biggest reason for not
> squashing patch series is because when you put everything into a single
> commit, I have had to review commits with 500+ files changed. Opening git
> difftool on a commit like that is awful.
> >
> >
> >
> > However, I would like to register my general endorsement for pull requests
> or some other web based system of code review… and I don’t have an
> Instagram account by the way :) Personally, I prefer Gerrit as I use it a lot with
> coreboot and other projects. But since we are using Github for hosting, pull
> requests are an easy switch and a logical choice. My main reason for being
> excited about pull requests mostly has to do with the amount of manual
> effort required to be a TianoCore maintainer right now. I have set up my
> email filter so that the mailing list is categorized like so:
> >
> >
> >
> > [cid:image001.png@01D62D71.502B55E0]
> >
> >
> >
> > Implementing the logic to parse the contents of emails to categorize them
> like this required me to define no less than 12 email filter rules in Microsoft
> Outlook, and I have to change my filtering logic every time I am
> added/removed from a Maintainers.txt file. I’m sure every other maintainer
> has spent a time separately implementing filtering logic like I have. This helps,
> but still for every thread, I have to go and check if one of the other
> maintainers has already reviewed/pushed that patch series yet, and if not
> review/push it. If I have ] feedback on a patch series, I have to categorize it
> as awaiting response from author and check up on it from time to time,
> sometimes I ping the author directly and remind them to send a new patch
> series. Implementing this state machine is a lot of manual work and it kind of
> feels like I’m a telephone operator in the 1950s. I greatly welcome
> automation here as I am sure it will increase the number of patch series I am
> able to review per hour.
> >
> >
> >
> > Thanks,
> >
> > Nate
> >
> >
> >
> > -----Original Message-----
> > From: rfc@edk2.groups.io <rfc@edk2.groups.io> On Behalf Of Laszlo
> > Ersek
> > Sent: Friday, May 15, 2020 2:08 AM
> > To: rfc@edk2.groups.io; bret.barkelew@microsoft.com;
> > devel@edk2.groups.io; Kinney, Michael D <michael.d.kinney@intel.com>
> > Subject: Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull
> > Request based Code Review Process
> >
> >
> >
> > On 05/15/20 06:49, Bret Barkelew via groups.io wrote:
> >
> >
> >
> >> I would far prefer the approach of individual PRs for commits to
> >> allow
> >
> >> for the squash flexibility (and is the strategy I think I would
> >> pursue
> >
> >> with my PRs). For example, the VarPol PR would be broken up into 9
> >> PRs
> >
> >> for each final commit, and we can get them in one by one.
> >
> >> Ideally, each one would be a small back and forth and then in. If it
> >
> >> had been done that way to begin with, it would be over in a week and
> >> a
> >
> >> half or so, rather than the multiple months that we’re now verging
> >
> >> on.
> >
> >
> >
> > This differs extremely from how we've been working on edk2-devel (or
> from how any git-based project works that I've ever been involved with).
> >
> > And I think the above workflow is out of scope, for migrating the edk2
> process to github.
> >
> >
> >
> > Again, the structuring of a patch series is a primary trait. Iterating only on
> individual patches does not allow for the reordering / restructuring of the
> patch series (dropping patches, reordering patches, inserting patches,
> moving hunks between patches).
> >
> >
> >
> > It's common that the necessity to revise an earlier patch emerges while
> reworking a later patch. For instance, the git-rebase(1) manual dedicates a
> separate section to "splitting commits".
> >
> >
> >
> > In the initial evaluation of "web forges", Phabricator was one of the
> "contestants". Phabricator didn't support the "patch series" concept at all, it
> only supported review requests for individual patches, and it supported
> setting up dependencies between them. So, for example, a 27-patch series
> would require 27 submissions and 26 dependencies.
> >
> >
> >
> > Lacking support for the patch series concept was an immediate deal
> breaker with Phabricator.
> >
> >
> >
> > The longest patch series I've ever submitted to edk2-devel had 58 patches.
> It was SMM enablement for OVMF. It went from v1 to v5 (v5 was merged),
> and the patch count varied significantly:
> >
> >
> >
> > v1: 58 patches (25 Jul 2015)
> >
> > v2: 41 patches ( 9 Oct 2015)
> >
> > v3: 52 patches (15 Oct 2015)
> >
> > v4: 41 patches ( 3 Nov 2015)
> >
> > v5: 33 patches (27 Nov 2015)
> >
> >
> >
> > (The significant drop in the patch count was due to Mike Kinney open
> > sourcing and upstreaming the *real* PiSmmCpuDxeSmm driver (which was
> > huge work in its own right), allowing me to drop the Quark-originated
> > 32-bit-only PiSmmCpuDxeSmm variant, from my series.)
> >
> >
> >
> > The contribution process should make difficult things possible, even if that
> complicates simple things somewhat. A process that makes simple things
> simple and difficult things impossible is useless. This is what the Instagram
> generation seems to be missing.
> >
> >
> >
> >
> >
> > I don't know why the VariablePolicy work took months. I can see the
> following threads on the list:
> >
> >
> >
> > * [edk2-devel] [PATCH v1 0/9] Add the VariablePolicy feature
> >
> >    Fri, 10 Apr 2020 11:36:01 -0700
> >
> >
> >
> > * [edk2-devel] [PATCH v2 00/12] Add the VariablePolicy feature
> >
> >    Mon, 11 May 2020 23:46:23 -0700
> >
> >
> >
> > I have two sets of comments:
> >
> >
> >
> > (1) It's difficult to tell in retrospect (because the series seem to have been
> posted with somewhat problematic threading), but the delay apparently
> came from multiple sources.
> >
> >
> >
> > (1a) Review was slow and spotty.
> >
> >
> >
> > The v1 blurb received some comments in the first week after it was posted.
> But the rest of the v1 series (the actual patches) received feedback like this:
> >
> >
> >
> > - v1 1/9: no feedback
> >
> > - v1 2/9: 12 days after posting
> >
> > - v1 3/9: 16 days after posting
> >
> > - v1 4/9: no feedback
> >
> > - v1 5/9: no feedback
> >
> > - v1 6/9: no feedback
> >
> > - v1 7/9: no feedback
> >
> > - v1 8/9: no feedback
> >
> > - v1 9/9: no feedback
> >
> >
> >
> > (1b) There was also quite some time between the last response in the v1
> thread (Apr 26th, as far as I can see), and the posting of the v2 series (May
> 11th).
> >
> >
> >
> > (1c) The v2 blurb got almost immediate, and numerous feedback (on the
> day of posting, and the day after). Regarding the individual patches, they
> didn't fare too well:
> >
> >
> >
> > - v2 01/12: superficial comment on the day of posting from me (not a
> >
> >              designated MdeModulePkg review), on the day of posting;
> > no
> >
> >              other feedback thus far
> >
> > - v2 02/12: ditto
> >
> > - v2 03/12: no feedback
> >
> > - v2 04/12: superficial (coding style) comments on the day of posting
> >
> > - v2 05/12: no feedback
> >
> > - v2 06/12: no feedback
> >
> > - v2 07/12: no feedback
> >
> > - v2 08/12: no feedback
> >
> > - v2 09/12: no feedback
> >
> > - v2 10/12: no feedback
> >
> > - v2 11/12: reasonably in-depth review from responsible co-maintainer
> >
> >              (yours truly), on the day of posting
> >
> > - v2 12/12: no feedback
> >
> >
> >
> > In total, I don't think the current process takes the blame for the delay. If
> reviewers don't care (or have no time) now, that problem will not change
> with the transition to github.com.
> >
> >
> >
> >
> >
> > (2) The VariablePolicy series is actually a good example that patch series
> restructuring is important.
> >
> >
> >
> > (2a) The patch count went from 9 (in v1) to 12 (in v2).
> >
> >
> >
> > (2b) And under v2, Liming still pointed out: "To keep each commit build
> pass, the patch set should first add new library instance, then add the library
> instance into each platform DSC, last update Variable driver to consume new
> library instance."
> >
> >
> >
> > Furthermore, I requested enabling the feature in ArmVirtPkg too, and
> maybe (based on owner feedback) UefiPayloadPkg.
> >
> >
> >
> > Thus, the v2->v3 update will most likely bring about both patch order
> changes, and an increased patch count.
> >
> >
> >
> > Thanks
> >
> > Laszlo
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> 
> 

Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

[Bret Barkelew]

[-- Attachment #1: Type: text/plain, Size: 16075 bytes --]

Nate, I believe you missed Sean’s point.

Each one of those packages should have been a separate PR.
Ergo, no information would have been lost in the squash.

Also, it’s not so much that we *can’t* learn. It’s that we choose not to. Around here, it’s a mark of prestige to not open doors with your face if it seems like there’s a better way. Makes it easier to focus on the work.

- Bret

From: Nate DeSimone via groups.io<mailto:nathaniel.l.desimone=intel.com@groups.io>
Sent: Tuesday, May 19, 2020 11:02 AM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; spbrogan@outlook.com<mailto:spbrogan@outlook.com>; rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>; lersek@redhat.com<mailto:lersek@redhat.com>; Bret Barkelew<mailto:Bret.Barkelew@microsoft.com>; Kinney, Michael D<mailto:michael.d.kinney@intel.com>
Subject: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Hi Sean,

My recent spelling fix patch series is a good example of why this is a bad idea actually:

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59779&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=fVz16E37%2BwW5pSgRxI45K7nWPDlIoS0HuI8UCGmEwjY%3D&amp;reserved=0
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59780&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=4q0lC1BSlSoQ3p0HGWwlph09HTjgJRo4nTO2Qx59%2Fjc%3D&amp;reserved=0
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59781&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=XQVSwPMXdpDJXj9nkuvq2fenwhNt6HGGZXsJwH5Bu8E%3D&amp;reserved=0
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59782&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=kCULGBc6%2Bifcn3cnPTV1odHI1ZUxuWQePN3POKKS3SM%3D&amp;reserved=0
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59783&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=SCOhUMdNXHIymGLaw9z3JTh%2Fe2BfaJaAyEC99EkG%2Fvg%3D&amp;reserved=0
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59784&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=epET6Wk30bIHQCvEDFLkeHEfmm9tzlxRrJ%2FQAuEfQFs%3D&amp;reserved=0
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59785&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=N8T7HjerJVvyGg94yMWjLm%2Fw7WDdXOdby1JpOYlPeVc%3D&amp;reserved=0

Notice that I split along package boundaries, because the maintainers for each package is a different set of people. If my patch series was squashed at merge time... how do I know who reviewed what? If the commit set is not correct.. I tend to say so in my feedback :). The only sane way to squash this series would be to have a human re-write all the commit messages, which I am against.

Generally those that prefer an easily bisectable history have such preference mostly due to the usage of validators that immediately resort bisecting as a method to root cause an issue since they tend to not understand the code very well. Edk2 already has 12 years of non-bisectable history, so this method is going to be ineffective anyway.

With regard to sending squashed commits, I understand that those who are new may have difficulty sending a properly formatted patch series, but frankly attempting to shield them from having to learn I am strongly against. I suggest that Microsoft invest in its human capital similar to how Intel does. If you cannot figure out how to send a properly formatted patch series... then do your work on the internal codebase (or perhaps MU.) Within the Intel, having the skillset to contribute to TianoCore is considered a mark of prestige, and thus needs to be earned.

TLDR, I will reject squashed commits on any packages that I maintain.

Thanks,
Nate

> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Sean
> Sent: Tuesday, May 19, 2020 9:54 AM
> To: rfc@edk2.groups.io; Desimone, Nathaniel L
> <nathaniel.l.desimone@intel.com>; lersek@redhat.com;
> bret.barkelew@microsoft.com; devel@edk2.groups.io; Kinney, Michael D
> <michael.d.kinney@intel.com>
> Subject: Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review
> Process
>
> Nate/Laszlo,
>
> Regarding a squash merge workflow.  I agree it can be abused and we all
> have seen terrible examples.  But a patch series that contains 500+ file
> changes isn't really much better.  Just because it is broken into multiple
> commits doesn't mean it is the right set of commits.
>
> Anyway a squash merge workflow works amazingly well with and is
> optimized for a web based review and PR processes.  It allows a user to
> respond to changes, fix issues, learn thru the PR process, all while keeping
> complete track of the progression.  Then once all "status"
> checks and reviews are complete, it is squashed into a neat commit for
> mainline, containing only the relevant data in the message.
>
> So, the ask is that we don't exclude squash merge workflows.  Those
> reviewing the PR can decide what is appropriate for the PR content
> submitted.  Just as you would request changes to the contents (or
> ordering) of a commit in a series, if the reviewers don't agree that the PR
> contents should be in a single commit then obviously it shouldn't be
> squashed to one.
>
> Contributions like spelling fixes, typos, minor bug fixes, documentation
> additions/fixes, etc all are great examples of PRs that can easily leverage
> squash merges and this workflow significantly lowers the burden of the
> contribution and review process.  This workflow is also are much easier for
> casual or first time contributors.
>
> I don't exactly know how we would enable this but I assume we could
> leverage tags or make it clear in the PR description.  First step is to get
> alignment that a squash merge workflow, while not appropriate for all
> contributions, is not something to be excluded.
>
> Thanks
> Sean
>
>
>
> On 5/19/2020 12:21 AM, Nate DeSimone wrote:
> > Hi All,
> >
> >
> >
> > I tend to agree with most of Laszlo's points. Specifically, that moving to pull
> requests will not fix the fact that maintainers are usually busy people and
> don't always give feedback in a punctual manner. Like Laszlo, I would also
> prefer that we do not squash patch series. My biggest reason for not
> squashing patch series is because when you put everything into a single
> commit, I have had to review commits with 500+ files changed. Opening git
> difftool on a commit like that is awful.
> >
> >
> >
> > However, I would like to register my general endorsement for pull requests
> or some other web based system of code review… and I don’t have an
> Instagram account by the way :) Personally, I prefer Gerrit as I use it a lot with
> coreboot and other projects. But since we are using Github for hosting, pull
> requests are an easy switch and a logical choice. My main reason for being
> excited about pull requests mostly has to do with the amount of manual
> effort required to be a TianoCore maintainer right now. I have set up my
> email filter so that the mailing list is categorized like so:
> >
> >
> >
> > [cid:image001.png@01D62D71.502B55E0]
> >
> >
> >
> > Implementing the logic to parse the contents of emails to categorize them
> like this required me to define no less than 12 email filter rules in Microsoft
> Outlook, and I have to change my filtering logic every time I am
> added/removed from a Maintainers.txt file. I’m sure every other maintainer
> has spent a time separately implementing filtering logic like I have. This helps,
> but still for every thread, I have to go and check if one of the other
> maintainers has already reviewed/pushed that patch series yet, and if not
> review/push it. If I have ] feedback on a patch series, I have to categorize it
> as awaiting response from author and check up on it from time to time,
> sometimes I ping the author directly and remind them to send a new patch
> series. Implementing this state machine is a lot of manual work and it kind of
> feels like I’m a telephone operator in the 1950s. I greatly welcome
> automation here as I am sure it will increase the number of patch series I am
> able to review per hour.
> >
> >
> >
> > Thanks,
> >
> > Nate
> >
> >
> >
> > -----Original Message-----
> > From: rfc@edk2.groups.io <rfc@edk2.groups.io> On Behalf Of Laszlo
> > Ersek
> > Sent: Friday, May 15, 2020 2:08 AM
> > To: rfc@edk2.groups.io; bret.barkelew@microsoft.com;
> > devel@edk2.groups.io; Kinney, Michael D <michael.d.kinney@intel.com>
> > Subject: Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull
> > Request based Code Review Process
> >
> >
> >
> > On 05/15/20 06:49, Bret Barkelew via groups.io wrote:
> >
> >
> >
> >> I would far prefer the approach of individual PRs for commits to
> >> allow
> >
> >> for the squash flexibility (and is the strategy I think I would
> >> pursue
> >
> >> with my PRs). For example, the VarPol PR would be broken up into 9
> >> PRs
> >
> >> for each final commit, and we can get them in one by one.
> >
> >> Ideally, each one would be a small back and forth and then in. If it
> >
> >> had been done that way to begin with, it would be over in a week and
> >> a
> >
> >> half or so, rather than the multiple months that we’re now verging
> >
> >> on.
> >
> >
> >
> > This differs extremely from how we've been working on edk2-devel (or
> from how any git-based project works that I've ever been involved with).
> >
> > And I think the above workflow is out of scope, for migrating the edk2
> process to github.
> >
> >
> >
> > Again, the structuring of a patch series is a primary trait. Iterating only on
> individual patches does not allow for the reordering / restructuring of the
> patch series (dropping patches, reordering patches, inserting patches,
> moving hunks between patches).
> >
> >
> >
> > It's common that the necessity to revise an earlier patch emerges while
> reworking a later patch. For instance, the git-rebase(1) manual dedicates a
> separate section to "splitting commits".
> >
> >
> >
> > In the initial evaluation of "web forges", Phabricator was one of the
> "contestants". Phabricator didn't support the "patch series" concept at all, it
> only supported review requests for individual patches, and it supported
> setting up dependencies between them. So, for example, a 27-patch series
> would require 27 submissions and 26 dependencies.
> >
> >
> >
> > Lacking support for the patch series concept was an immediate deal
> breaker with Phabricator.
> >
> >
> >
> > The longest patch series I've ever submitted to edk2-devel had 58 patches.
> It was SMM enablement for OVMF. It went from v1 to v5 (v5 was merged),
> and the patch count varied significantly:
> >
> >
> >
> > v1: 58 patches (25 Jul 2015)
> >
> > v2: 41 patches ( 9 Oct 2015)
> >
> > v3: 52 patches (15 Oct 2015)
> >
> > v4: 41 patches ( 3 Nov 2015)
> >
> > v5: 33 patches (27 Nov 2015)
> >
> >
> >
> > (The significant drop in the patch count was due to Mike Kinney open
> > sourcing and upstreaming the *real* PiSmmCpuDxeSmm driver (which was
> > huge work in its own right), allowing me to drop the Quark-originated
> > 32-bit-only PiSmmCpuDxeSmm variant, from my series.)
> >
> >
> >
> > The contribution process should make difficult things possible, even if that
> complicates simple things somewhat. A process that makes simple things
> simple and difficult things impossible is useless. This is what the Instagram
> generation seems to be missing.
> >
> >
> >
> >
> >
> > I don't know why the VariablePolicy work took months. I can see the
> following threads on the list:
> >
> >
> >
> > * [edk2-devel] [PATCH v1 0/9] Add the VariablePolicy feature
> >
> >    Fri, 10 Apr 2020 11:36:01 -0700
> >
> >
> >
> > * [edk2-devel] [PATCH v2 00/12] Add the VariablePolicy feature
> >
> >    Mon, 11 May 2020 23:46:23 -0700
> >
> >
> >
> > I have two sets of comments:
> >
> >
> >
> > (1) It's difficult to tell in retrospect (because the series seem to have been
> posted with somewhat problematic threading), but the delay apparently
> came from multiple sources.
> >
> >
> >
> > (1a) Review was slow and spotty.
> >
> >
> >
> > The v1 blurb received some comments in the first week after it was posted.
> But the rest of the v1 series (the actual patches) received feedback like this:
> >
> >
> >
> > - v1 1/9: no feedback
> >
> > - v1 2/9: 12 days after posting
> >
> > - v1 3/9: 16 days after posting
> >
> > - v1 4/9: no feedback
> >
> > - v1 5/9: no feedback
> >
> > - v1 6/9: no feedback
> >
> > - v1 7/9: no feedback
> >
> > - v1 8/9: no feedback
> >
> > - v1 9/9: no feedback
> >
> >
> >
> > (1b) There was also quite some time between the last response in the v1
> thread (Apr 26th, as far as I can see), and the posting of the v2 series (May
> 11th).
> >
> >
> >
> > (1c) The v2 blurb got almost immediate, and numerous feedback (on the
> day of posting, and the day after). Regarding the individual patches, they
> didn't fare too well:
> >
> >
> >
> > - v2 01/12: superficial comment on the day of posting from me (not a
> >
> >              designated MdeModulePkg review), on the day of posting;
> > no
> >
> >              other feedback thus far
> >
> > - v2 02/12: ditto
> >
> > - v2 03/12: no feedback
> >
> > - v2 04/12: superficial (coding style) comments on the day of posting
> >
> > - v2 05/12: no feedback
> >
> > - v2 06/12: no feedback
> >
> > - v2 07/12: no feedback
> >
> > - v2 08/12: no feedback
> >
> > - v2 09/12: no feedback
> >
> > - v2 10/12: no feedback
> >
> > - v2 11/12: reasonably in-depth review from responsible co-maintainer
> >
> >              (yours truly), on the day of posting
> >
> > - v2 12/12: no feedback
> >
> >
> >
> > In total, I don't think the current process takes the blame for the delay. If
> reviewers don't care (or have no time) now, that problem will not change
> with the transition to github.com.
> >
> >
> >
> >
> >
> > (2) The VariablePolicy series is actually a good example that patch series
> restructuring is important.
> >
> >
> >
> > (2a) The patch count went from 9 (in v1) to 12 (in v2).
> >
> >
> >
> > (2b) And under v2, Liming still pointed out: "To keep each commit build
> pass, the patch set should first add new library instance, then add the library
> instance into each platform DSC, last update Variable driver to consume new
> library instance."
> >
> >
> >
> > Furthermore, I requested enabling the feature in ArmVirtPkg too, and
> maybe (based on owner feedback) UefiPayloadPkg.
> >
> >
> >
> > Thus, the v2->v3 update will most likely bring about both patch order
> changes, and an increased patch count.
> >
> >
> >
> > Thanks
> >
> > Laszlo
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
>
>





[-- Attachment #2: Type: text/html, Size: 24433 bytes --]

Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

[Nate DeSimone]

[-- Attachment #1: Type: text/plain, Size: 17285 bytes --]

Hi Bret,

I believe you missed my point. I don’t want my patch series to be merged piece by piece; I want it merged all at once, in the order that I specified.

I tend to agree with Laszlo that you are choosing not to learn how to use Git properly. Commit early, commit often, perfect later, publish once is the Git best practice. You should not hide the sausage making, which is exactly what you are proposing. I find it unfortunate that you consider refusing to learn GIt best practices a mark of prestige.

Thanks,
Nate

From: <devel@edk2.groups.io> on behalf of "Bret Barkelew via groups.io" <bret.barkelew=microsoft.com@groups.io>
Reply-To: "devel@edk2.groups.io" <devel@edk2.groups.io>, "bret.barkelew@microsoft.com" <bret.barkelew@microsoft.com>
Date: Tuesday, May 19, 2020 at 12:35 PM
To: "devel@edk2.groups.io" <devel@edk2.groups.io>, "Desimone, Nathaniel L" <nathaniel.l.desimone@intel.com>, "spbrogan@outlook.com" <spbrogan@outlook.com>, "rfc@edk2.groups.io" <rfc@edk2.groups.io>, "lersek@redhat.com" <lersek@redhat.com>, "Kinney, Michael D" <michael.d.kinney@intel.com>
Subject: Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Nate, I believe you missed Sean’s point.

Each one of those packages should have been a separate PR.
Ergo, no information would have been lost in the squash.

Also, it’s not so much that we *can’t* learn. It’s that we choose not to. Around here, it’s a mark of prestige to not open doors with your face if it seems like there’s a better way. Makes it easier to focus on the work.

- Bret

From: Nate DeSimone via groups.io<mailto:nathaniel.l.desimone=intel.com@groups.io>
Sent: Tuesday, May 19, 2020 11:02 AM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; spbrogan@outlook.com<mailto:spbrogan@outlook.com>; rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>; lersek@redhat.com<mailto:lersek@redhat.com>; Bret Barkelew<mailto:Bret.Barkelew@microsoft.com>; Kinney, Michael D<mailto:michael.d.kinney@intel.com>
Subject: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Hi Sean,

My recent spelling fix patch series is a good example of why this is a bad idea actually:

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59779&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=fVz16E37%2BwW5pSgRxI45K7nWPDlIoS0HuI8UCGmEwjY%3D&amp;reserved=0
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59780&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=4q0lC1BSlSoQ3p0HGWwlph09HTjgJRo4nTO2Qx59%2Fjc%3D&amp;reserved=0
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59781&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=XQVSwPMXdpDJXj9nkuvq2fenwhNt6HGGZXsJwH5Bu8E%3D&amp;reserved=0
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59782&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=kCULGBc6%2Bifcn3cnPTV1odHI1ZUxuWQePN3POKKS3SM%3D&amp;reserved=0
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59783&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=SCOhUMdNXHIymGLaw9z3JTh%2Fe2BfaJaAyEC99EkG%2Fvg%3D&amp;reserved=0
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59784&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=epET6Wk30bIHQCvEDFLkeHEfmm9tzlxRrJ%2FQAuEfQFs%3D&amp;reserved=0
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59785&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=N8T7HjerJVvyGg94yMWjLm%2Fw7WDdXOdby1JpOYlPeVc%3D&amp;reserved=0

Notice that I split along package boundaries, because the maintainers for each package is a different set of people. If my patch series was squashed at merge time... how do I know who reviewed what? If the commit set is not correct.. I tend to say so in my feedback :). The only sane way to squash this series would be to have a human re-write all the commit messages, which I am against.

Generally those that prefer an easily bisectable history have such preference mostly due to the usage of validators that immediately resort bisecting as a method to root cause an issue since they tend to not understand the code very well. Edk2 already has 12 years of non-bisectable history, so this method is going to be ineffective anyway.

With regard to sending squashed commits, I understand that those who are new may have difficulty sending a properly formatted patch series, but frankly attempting to shield them from having to learn I am strongly against. I suggest that Microsoft invest in its human capital similar to how Intel does. If you cannot figure out how to send a properly formatted patch series... then do your work on the internal codebase (or perhaps MU.) Within the Intel, having the skillset to contribute to TianoCore is considered a mark of prestige, and thus needs to be earned.

TLDR, I will reject squashed commits on any packages that I maintain.

Thanks,
Nate

> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Sean
> Sent: Tuesday, May 19, 2020 9:54 AM
> To: rfc@edk2.groups.io; Desimone, Nathaniel L
> <nathaniel.l.desimone@intel.com>; lersek@redhat.com;
> bret.barkelew@microsoft.com; devel@edk2.groups.io; Kinney, Michael D
> <michael.d.kinney@intel.com>
> Subject: Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review
> Process
>
> Nate/Laszlo,
>
> Regarding a squash merge workflow.  I agree it can be abused and we all
> have seen terrible examples.  But a patch series that contains 500+ file
> changes isn't really much better.  Just because it is broken into multiple
> commits doesn't mean it is the right set of commits.
>
> Anyway a squash merge workflow works amazingly well with and is
> optimized for a web based review and PR processes.  It allows a user to
> respond to changes, fix issues, learn thru the PR process, all while keeping
> complete track of the progression.  Then once all "status"
> checks and reviews are complete, it is squashed into a neat commit for
> mainline, containing only the relevant data in the message.
>
> So, the ask is that we don't exclude squash merge workflows.  Those
> reviewing the PR can decide what is appropriate for the PR content
> submitted.  Just as you would request changes to the contents (or
> ordering) of a commit in a series, if the reviewers don't agree that the PR
> contents should be in a single commit then obviously it shouldn't be
> squashed to one.
>
> Contributions like spelling fixes, typos, minor bug fixes, documentation
> additions/fixes, etc all are great examples of PRs that can easily leverage
> squash merges and this workflow significantly lowers the burden of the
> contribution and review process.  This workflow is also are much easier for
> casual or first time contributors.
>
> I don't exactly know how we would enable this but I assume we could
> leverage tags or make it clear in the PR description.  First step is to get
> alignment that a squash merge workflow, while not appropriate for all
> contributions, is not something to be excluded.
>
> Thanks
> Sean
>
>
>
> On 5/19/2020 12:21 AM, Nate DeSimone wrote:
> > Hi All,
> >
> >
> >
> > I tend to agree with most of Laszlo's points. Specifically, that moving to pull
> requests will not fix the fact that maintainers are usually busy people and
> don't always give feedback in a punctual manner. Like Laszlo, I would also
> prefer that we do not squash patch series. My biggest reason for not
> squashing patch series is because when you put everything into a single
> commit, I have had to review commits with 500+ files changed. Opening git
> difftool on a commit like that is awful.
> >
> >
> >
> > However, I would like to register my general endorsement for pull requests
> or some other web based system of code review… and I don’t have an
> Instagram account by the way :) Personally, I prefer Gerrit as I use it a lot with
> coreboot and other projects. But since we are using Github for hosting, pull
> requests are an easy switch and a logical choice. My main reason for being
> excited about pull requests mostly has to do with the amount of manual
> effort required to be a TianoCore maintainer right now. I have set up my
> email filter so that the mailing list is categorized like so:
> >
> >
> >
> > [cid:image001.png@01D62D71.502B55E0]
> >
> >
> >
> > Implementing the logic to parse the contents of emails to categorize them
> like this required me to define no less than 12 email filter rules in Microsoft
> Outlook, and I have to change my filtering logic every time I am
> added/removed from a Maintainers.txt file. I’m sure every other maintainer
> has spent a time separately implementing filtering logic like I have. This helps,
> but still for every thread, I have to go and check if one of the other
> maintainers has already reviewed/pushed that patch series yet, and if not
> review/push it. If I have ] feedback on a patch series, I have to categorize it
> as awaiting response from author and check up on it from time to time,
> sometimes I ping the author directly and remind them to send a new patch
> series. Implementing this state machine is a lot of manual work and it kind of
> feels like I’m a telephone operator in the 1950s. I greatly welcome
> automation here as I am sure it will increase the number of patch series I am
> able to review per hour.
> >
> >
> >
> > Thanks,
> >
> > Nate
> >
> >
> >
> > -----Original Message-----
> > From: rfc@edk2.groups.io <rfc@edk2.groups.io> On Behalf Of Laszlo
> > Ersek
> > Sent: Friday, May 15, 2020 2:08 AM
> > To: rfc@edk2.groups.io; bret.barkelew@microsoft.com;
> > devel@edk2.groups.io; Kinney, Michael D <michael.d.kinney@intel.com>
> > Subject: Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull
> > Request based Code Review Process
> >
> >
> >
> > On 05/15/20 06:49, Bret Barkelew via groups.io wrote:
> >
> >
> >
> >> I would far prefer the approach of individual PRs for commits to
> >> allow
> >
> >> for the squash flexibility (and is the strategy I think I would
> >> pursue
> >
> >> with my PRs). For example, the VarPol PR would be broken up into 9
> >> PRs
> >
> >> for each final commit, and we can get them in one by one.
> >
> >> Ideally, each one would be a small back and forth and then in. If it
> >
> >> had been done that way to begin with, it would be over in a week and
> >> a
> >
> >> half or so, rather than the multiple months that we’re now verging
> >
> >> on.
> >
> >
> >
> > This differs extremely from how we've been working on edk2-devel (or
> from how any git-based project works that I've ever been involved with).
> >
> > And I think the above workflow is out of scope, for migrating the edk2
> process to github.
> >
> >
> >
> > Again, the structuring of a patch series is a primary trait. Iterating only on
> individual patches does not allow for the reordering / restructuring of the
> patch series (dropping patches, reordering patches, inserting patches,
> moving hunks between patches).
> >
> >
> >
> > It's common that the necessity to revise an earlier patch emerges while
> reworking a later patch. For instance, the git-rebase(1) manual dedicates a
> separate section to "splitting commits".
> >
> >
> >
> > In the initial evaluation of "web forges", Phabricator was one of the
> "contestants". Phabricator didn't support the "patch series" concept at all, it
> only supported review requests for individual patches, and it supported
> setting up dependencies between them. So, for example, a 27-patch series
> would require 27 submissions and 26 dependencies.
> >
> >
> >
> > Lacking support for the patch series concept was an immediate deal
> breaker with Phabricator.
> >
> >
> >
> > The longest patch series I've ever submitted to edk2-devel had 58 patches.
> It was SMM enablement for OVMF. It went from v1 to v5 (v5 was merged),
> and the patch count varied significantly:
> >
> >
> >
> > v1: 58 patches (25 Jul 2015)
> >
> > v2: 41 patches ( 9 Oct 2015)
> >
> > v3: 52 patches (15 Oct 2015)
> >
> > v4: 41 patches ( 3 Nov 2015)
> >
> > v5: 33 patches (27 Nov 2015)
> >
> >
> >
> > (The significant drop in the patch count was due to Mike Kinney open
> > sourcing and upstreaming the *real* PiSmmCpuDxeSmm driver (which was
> > huge work in its own right), allowing me to drop the Quark-originated
> > 32-bit-only PiSmmCpuDxeSmm variant, from my series.)
> >
> >
> >
> > The contribution process should make difficult things possible, even if that
> complicates simple things somewhat. A process that makes simple things
> simple and difficult things impossible is useless. This is what the Instagram
> generation seems to be missing.
> >
> >
> >
> >
> >
> > I don't know why the VariablePolicy work took months. I can see the
> following threads on the list:
> >
> >
> >
> > * [edk2-devel] [PATCH v1 0/9] Add the VariablePolicy feature
> >
> >    Fri, 10 Apr 2020 11:36:01 -0700
> >
> >
> >
> > * [edk2-devel] [PATCH v2 00/12] Add the VariablePolicy feature
> >
> >    Mon, 11 May 2020 23:46:23 -0700
> >
> >
> >
> > I have two sets of comments:
> >
> >
> >
> > (1) It's difficult to tell in retrospect (because the series seem to have been
> posted with somewhat problematic threading), but the delay apparently
> came from multiple sources.
> >
> >
> >
> > (1a) Review was slow and spotty.
> >
> >
> >
> > The v1 blurb received some comments in the first week after it was posted.
> But the rest of the v1 series (the actual patches) received feedback like this:
> >
> >
> >
> > - v1 1/9: no feedback
> >
> > - v1 2/9: 12 days after posting
> >
> > - v1 3/9: 16 days after posting
> >
> > - v1 4/9: no feedback
> >
> > - v1 5/9: no feedback
> >
> > - v1 6/9: no feedback
> >
> > - v1 7/9: no feedback
> >
> > - v1 8/9: no feedback
> >
> > - v1 9/9: no feedback
> >
> >
> >
> > (1b) There was also quite some time between the last response in the v1
> thread (Apr 26th, as far as I can see), and the posting of the v2 series (May
> 11th).
> >
> >
> >
> > (1c) The v2 blurb got almost immediate, and numerous feedback (on the
> day of posting, and the day after). Regarding the individual patches, they
> didn't fare too well:
> >
> >
> >
> > - v2 01/12: superficial comment on the day of posting from me (not a
> >
> >              designated MdeModulePkg review), on the day of posting;
> > no
> >
> >              other feedback thus far
> >
> > - v2 02/12: ditto
> >
> > - v2 03/12: no feedback
> >
> > - v2 04/12: superficial (coding style) comments on the day of posting
> >
> > - v2 05/12: no feedback
> >
> > - v2 06/12: no feedback
> >
> > - v2 07/12: no feedback
> >
> > - v2 08/12: no feedback
> >
> > - v2 09/12: no feedback
> >
> > - v2 10/12: no feedback
> >
> > - v2 11/12: reasonably in-depth review from responsible co-maintainer
> >
> >              (yours truly), on the day of posting
> >
> > - v2 12/12: no feedback
> >
> >
> >
> > In total, I don't think the current process takes the blame for the delay. If
> reviewers don't care (or have no time) now, that problem will not change
> with the transition to github.com.
> >
> >
> >
> >
> >
> > (2) The VariablePolicy series is actually a good example that patch series
> restructuring is important.
> >
> >
> >
> > (2a) The patch count went from 9 (in v1) to 12 (in v2).
> >
> >
> >
> > (2b) And under v2, Liming still pointed out: "To keep each commit build
> pass, the patch set should first add new library instance, then add the library
> instance into each platform DSC, last update Variable driver to consume new
> library instance."
> >
> >
> >
> > Furthermore, I requested enabling the feature in ArmVirtPkg too, and
> maybe (based on owner feedback) UefiPayloadPkg.
> >
> >
> >
> > Thus, the v2->v3 update will most likely bring about both patch order
> changes, and an increased patch count.
> >
> >
> >
> > Thanks
> >
> > Laszlo
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
>
>






[-- Attachment #2: Type: text/html, Size: 26969 bytes --]

Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

[Bret Barkelew]

[-- Attachment #1.1: Type: text/plain, Size: 20553 bytes --]

I will honor Mike Kinney’s efforts with my vote of confidence.
I think we’re headed in the right direction, even with some of the things that I disagree with.

In my history with TianoCore, I have learned to not be so quick to say “this is fucking stupid”. Every time I’ve done that, I’ve later discovered the reasons behind it, and even come to the conclusion that the designers were quite clever.

That said, I want to contribute. And I won’t with the current system. I hope to be able to with the future system.

- Bret

From: Desimone, Nathaniel L<mailto:nathaniel.l.desimone@intel.com>
Sent: Tuesday, May 19, 2020 12:59 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Bret Barkelew<mailto:Bret.Barkelew@microsoft.com>; spbrogan@outlook.com<mailto:spbrogan@outlook.com>; rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>; lersek@redhat.com<mailto:lersek@redhat.com>; Kinney, Michael D<mailto:michael.d.kinney@intel.com>
Subject: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Hi Bret,

I believe you missed my point. I don’t want my patch series to be merged piece by piece; I want it merged all at once, in the order that I specified.

I tend to agree with Laszlo that you are choosing not to learn how to use Git properly. Commit early, commit often, perfect later, publish once is the Git best practice. You should not hide the sausage making, which is exactly what you are proposing. I find it unfortunate that you consider refusing to learn GIt best practices a mark of prestige.

Thanks,
Nate

From: <devel@edk2.groups.io> on behalf of "Bret Barkelew via groups.io" <bret.barkelew=microsoft.com@groups.io>
Reply-To: "devel@edk2.groups.io" <devel@edk2.groups.io>, "bret.barkelew@microsoft.com" <bret.barkelew@microsoft.com>
Date: Tuesday, May 19, 2020 at 12:35 PM
To: "devel@edk2.groups.io" <devel@edk2.groups.io>, "Desimone, Nathaniel L" <nathaniel.l.desimone@intel.com>, "spbrogan@outlook.com" <spbrogan@outlook.com>, "rfc@edk2.groups.io" <rfc@edk2.groups.io>, "lersek@redhat.com" <lersek@redhat.com>, "Kinney, Michael D" <michael.d.kinney@intel.com>
Subject: Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Nate, I believe you missed Sean’s point.

Each one of those packages should have been a separate PR.
Ergo, no information would have been lost in the squash.

Also, it’s not so much that we *can’t* learn. It’s that we choose not to. Around here, it’s a mark of prestige to not open doors with your face if it seems like there’s a better way. Makes it easier to focus on the work.

- Bret

From: Nate DeSimone via groups.io<mailto:nathaniel.l.desimone=intel.com@groups.io>
Sent: Tuesday, May 19, 2020 11:02 AM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; spbrogan@outlook.com<mailto:spbrogan@outlook.com>; rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>; lersek@redhat.com<mailto:lersek@redhat.com>; Bret Barkelew<mailto:Bret.Barkelew@microsoft.com>; Kinney, Michael D<mailto:michael.d.kinney@intel.com>
Subject: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Hi Sean,

My recent spelling fix patch series is a good example of why this is a bad idea actually:

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59779&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=fVz16E37%2BwW5pSgRxI45K7nWPDlIoS0HuI8UCGmEwjY%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59779&data=02%7C01%7Cbret.barkelew%40microsoft.com%7Cbf6d5aed67374c95ca3508d7fc2f20ec%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255151662145479&sdata=m66IViN3G%2BbJpBwolRXf8d3BpWQeRXs495WYxnsD85M%3D&reserved=0>
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59780&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=4q0lC1BSlSoQ3p0HGWwlph09HTjgJRo4nTO2Qx59%2Fjc%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59780&data=02%7C01%7Cbret.barkelew%40microsoft.com%7Cbf6d5aed67374c95ca3508d7fc2f20ec%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255151662155477&sdata=VuYPqwcmYTrbiYdo%2B8K5irX8k6rgMgEoC2fY8eAocWA%3D&reserved=0>
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59781&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=XQVSwPMXdpDJXj9nkuvq2fenwhNt6HGGZXsJwH5Bu8E%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59781&data=02%7C01%7Cbret.barkelew%40microsoft.com%7Cbf6d5aed67374c95ca3508d7fc2f20ec%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255151662165472&sdata=lMrhfWKBWKGkjNnXJJy7%2BisrugTi0J%2FUkmtnj7Vxb7Q%3D&reserved=0>
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59782&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=kCULGBc6%2Bifcn3cnPTV1odHI1ZUxuWQePN3POKKS3SM%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59782&data=02%7C01%7Cbret.barkelew%40microsoft.com%7Cbf6d5aed67374c95ca3508d7fc2f20ec%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255151662165472&sdata=rltVYSQcSLG2sGP4d2awDIuWV11nYQcdxvqyPxDM4XE%3D&reserved=0>
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59783&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=SCOhUMdNXHIymGLaw9z3JTh%2Fe2BfaJaAyEC99EkG%2Fvg%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59783&data=02%7C01%7Cbret.barkelew%40microsoft.com%7Cbf6d5aed67374c95ca3508d7fc2f20ec%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255151662175469&sdata=b0J%2BL4nddt3g%2FmidZO61tlkgVqrRsufOUHGjRqh1CJM%3D&reserved=0>
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59784&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=epET6Wk30bIHQCvEDFLkeHEfmm9tzlxRrJ%2FQAuEfQFs%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59784&data=02%7C01%7Cbret.barkelew%40microsoft.com%7Cbf6d5aed67374c95ca3508d7fc2f20ec%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255151662185463&sdata=zhHUUmckRdY45oYIPodqP9r3Sh4Q4t%2FZYRLULOiAERs%3D&reserved=0>
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59785&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=N8T7HjerJVvyGg94yMWjLm%2Fw7WDdXOdby1JpOYlPeVc%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59785&data=02%7C01%7Cbret.barkelew%40microsoft.com%7Cbf6d5aed67374c95ca3508d7fc2f20ec%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255151662185463&sdata=MNUfWJhjb1UfPFciUJ2dcu9uWQwUNkS3PJO%2BeMkFVxA%3D&reserved=0>

Notice that I split along package boundaries, because the maintainers for each package is a different set of people. If my patch series was squashed at merge time... how do I know who reviewed what? If the commit set is not correct.. I tend to say so in my feedback :). The only sane way to squash this series would be to have a human re-write all the commit messages, which I am against.

Generally those that prefer an easily bisectable history have such preference mostly due to the usage of validators that immediately resort bisecting as a method to root cause an issue since they tend to not understand the code very well. Edk2 already has 12 years of non-bisectable history, so this method is going to be ineffective anyway.

With regard to sending squashed commits, I understand that those who are new may have difficulty sending a properly formatted patch series, but frankly attempting to shield them from having to learn I am strongly against. I suggest that Microsoft invest in its human capital similar to how Intel does. If you cannot figure out how to send a properly formatted patch series... then do your work on the internal codebase (or perhaps MU.) Within the Intel, having the skillset to contribute to TianoCore is considered a mark of prestige, and thus needs to be earned.

TLDR, I will reject squashed commits on any packages that I maintain.

Thanks,
Nate

> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Sean
> Sent: Tuesday, May 19, 2020 9:54 AM
> To: rfc@edk2.groups.io; Desimone, Nathaniel L
> <nathaniel.l.desimone@intel.com>; lersek@redhat.com;
> bret.barkelew@microsoft.com; devel@edk2.groups.io; Kinney, Michael D
> <michael.d.kinney@intel.com>
> Subject: Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review
> Process
>
> Nate/Laszlo,
>
> Regarding a squash merge workflow.  I agree it can be abused and we all
> have seen terrible examples.  But a patch series that contains 500+ file
> changes isn't really much better.  Just because it is broken into multiple
> commits doesn't mean it is the right set of commits.
>
> Anyway a squash merge workflow works amazingly well with and is
> optimized for a web based review and PR processes.  It allows a user to
> respond to changes, fix issues, learn thru the PR process, all while keeping
> complete track of the progression.  Then once all "status"
> checks and reviews are complete, it is squashed into a neat commit for
> mainline, containing only the relevant data in the message.
>
> So, the ask is that we don't exclude squash merge workflows.  Those
> reviewing the PR can decide what is appropriate for the PR content
> submitted.  Just as you would request changes to the contents (or
> ordering) of a commit in a series, if the reviewers don't agree that the PR
> contents should be in a single commit then obviously it shouldn't be
> squashed to one.
>
> Contributions like spelling fixes, typos, minor bug fixes, documentation
> additions/fixes, etc all are great examples of PRs that can easily leverage
> squash merges and this workflow significantly lowers the burden of the
> contribution and review process.  This workflow is also are much easier for
> casual or first time contributors.
>
> I don't exactly know how we would enable this but I assume we could
> leverage tags or make it clear in the PR description.  First step is to get
> alignment that a squash merge workflow, while not appropriate for all
> contributions, is not something to be excluded.
>
> Thanks
> Sean
>
>
>
> On 5/19/2020 12:21 AM, Nate DeSimone wrote:
> > Hi All,
> >
> >
> >
> > I tend to agree with most of Laszlo's points. Specifically, that moving to pull
> requests will not fix the fact that maintainers are usually busy people and
> don't always give feedback in a punctual manner. Like Laszlo, I would also
> prefer that we do not squash patch series. My biggest reason for not
> squashing patch series is because when you put everything into a single
> commit, I have had to review commits with 500+ files changed. Opening git
> difftool on a commit like that is awful.
> >
> >
> >
> > However, I would like to register my general endorsement for pull requests
> or some other web based system of code review… and I don’t have an
> Instagram account by the way :) Personally, I prefer Gerrit as I use it a lot with
> coreboot and other projects. But since we are using Github for hosting, pull
> requests are an easy switch and a logical choice. My main reason for being
> excited about pull requests mostly has to do with the amount of manual
> effort required to be a TianoCore maintainer right now. I have set up my
> email filter so that the mailing list is categorized like so:
> >
> >
> >
> > [cid:image001.png@01D62D71.502B55E0]
> >
> >
> >
> > Implementing the logic to parse the contents of emails to categorize them
> like this required me to define no less than 12 email filter rules in Microsoft
> Outlook, and I have to change my filtering logic every time I am
> added/removed from a Maintainers.txt file. I’m sure every other maintainer
> has spent a time separately implementing filtering logic like I have. This helps,
> but still for every thread, I have to go and check if one of the other
> maintainers has already reviewed/pushed that patch series yet, and if not
> review/push it. If I have ] feedback on a patch series, I have to categorize it
> as awaiting response from author and check up on it from time to time,
> sometimes I ping the author directly and remind them to send a new patch
> series. Implementing this state machine is a lot of manual work and it kind of
> feels like I’m a telephone operator in the 1950s. I greatly welcome
> automation here as I am sure it will increase the number of patch series I am
> able to review per hour.
> >
> >
> >
> > Thanks,
> >
> > Nate
> >
> >
> >
> > -----Original Message-----
> > From: rfc@edk2.groups.io <rfc@edk2.groups.io> On Behalf Of Laszlo
> > Ersek
> > Sent: Friday, May 15, 2020 2:08 AM
> > To: rfc@edk2.groups.io; bret.barkelew@microsoft.com;
> > devel@edk2.groups.io; Kinney, Michael D <michael.d.kinney@intel.com>
> > Subject: Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull
> > Request based Code Review Process
> >
> >
> >
> > On 05/15/20 06:49, Bret Barkelew via groups.io wrote:
> >
> >
> >
> >> I would far prefer the approach of individual PRs for commits to
> >> allow
> >
> >> for the squash flexibility (and is the strategy I think I would
> >> pursue
> >
> >> with my PRs). For example, the VarPol PR would be broken up into 9
> >> PRs
> >
> >> for each final commit, and we can get them in one by one.
> >
> >> Ideally, each one would be a small back and forth and then in. If it
> >
> >> had been done that way to begin with, it would be over in a week and
> >> a
> >
> >> half or so, rather than the multiple months that we’re now verging
> >
> >> on.
> >
> >
> >
> > This differs extremely from how we've been working on edk2-devel (or
> from how any git-based project works that I've ever been involved with).
> >
> > And I think the above workflow is out of scope, for migrating the edk2
> process to github.
> >
> >
> >
> > Again, the structuring of a patch series is a primary trait. Iterating only on
> individual patches does not allow for the reordering / restructuring of the
> patch series (dropping patches, reordering patches, inserting patches,
> moving hunks between patches).
> >
> >
> >
> > It's common that the necessity to revise an earlier patch emerges while
> reworking a later patch. For instance, the git-rebase(1) manual dedicates a
> separate section to "splitting commits".
> >
> >
> >
> > In the initial evaluation of "web forges", Phabricator was one of the
> "contestants". Phabricator didn't support the "patch series" concept at all, it
> only supported review requests for individual patches, and it supported
> setting up dependencies between them. So, for example, a 27-patch series
> would require 27 submissions and 26 dependencies.
> >
> >
> >
> > Lacking support for the patch series concept was an immediate deal
> breaker with Phabricator.
> >
> >
> >
> > The longest patch series I've ever submitted to edk2-devel had 58 patches.
> It was SMM enablement for OVMF. It went from v1 to v5 (v5 was merged),
> and the patch count varied significantly:
> >
> >
> >
> > v1: 58 patches (25 Jul 2015)
> >
> > v2: 41 patches ( 9 Oct 2015)
> >
> > v3: 52 patches (15 Oct 2015)
> >
> > v4: 41 patches ( 3 Nov 2015)
> >
> > v5: 33 patches (27 Nov 2015)
> >
> >
> >
> > (The significant drop in the patch count was due to Mike Kinney open
> > sourcing and upstreaming the *real* PiSmmCpuDxeSmm driver (which was
> > huge work in its own right), allowing me to drop the Quark-originated
> > 32-bit-only PiSmmCpuDxeSmm variant, from my series.)
> >
> >
> >
> > The contribution process should make difficult things possible, even if that
> complicates simple things somewhat. A process that makes simple things
> simple and difficult things impossible is useless. This is what the Instagram
> generation seems to be missing.
> >
> >
> >
> >
> >
> > I don't know why the VariablePolicy work took months. I can see the
> following threads on the list:
> >
> >
> >
> > * [edk2-devel] [PATCH v1 0/9] Add the VariablePolicy feature
> >
> >    Fri, 10 Apr 2020 11:36:01 -0700
> >
> >
> >
> > * [edk2-devel] [PATCH v2 00/12] Add the VariablePolicy feature
> >
> >    Mon, 11 May 2020 23:46:23 -0700
> >
> >
> >
> > I have two sets of comments:
> >
> >
> >
> > (1) It's difficult to tell in retrospect (because the series seem to have been
> posted with somewhat problematic threading), but the delay apparently
> came from multiple sources.
> >
> >
> >
> > (1a) Review was slow and spotty.
> >
> >
> >
> > The v1 blurb received some comments in the first week after it was posted.
> But the rest of the v1 series (the actual patches) received feedback like this:
> >
> >
> >
> > - v1 1/9: no feedback
> >
> > - v1 2/9: 12 days after posting
> >
> > - v1 3/9: 16 days after posting
> >
> > - v1 4/9: no feedback
> >
> > - v1 5/9: no feedback
> >
> > - v1 6/9: no feedback
> >
> > - v1 7/9: no feedback
> >
> > - v1 8/9: no feedback
> >
> > - v1 9/9: no feedback
> >
> >
> >
> > (1b) There was also quite some time between the last response in the v1
> thread (Apr 26th, as far as I can see), and the posting of the v2 series (May
> 11th).
> >
> >
> >
> > (1c) The v2 blurb got almost immediate, and numerous feedback (on the
> day of posting, and the day after). Regarding the individual patches, they
> didn't fare too well:
> >
> >
> >
> > - v2 01/12: superficial comment on the day of posting from me (not a
> >
> >              designated MdeModulePkg review), on the day of posting;
> > no
> >
> >              other feedback thus far
> >
> > - v2 02/12: ditto
> >
> > - v2 03/12: no feedback
> >
> > - v2 04/12: superficial (coding style) comments on the day of posting
> >
> > - v2 05/12: no feedback
> >
> > - v2 06/12: no feedback
> >
> > - v2 07/12: no feedback
> >
> > - v2 08/12: no feedback
> >
> > - v2 09/12: no feedback
> >
> > - v2 10/12: no feedback
> >
> > - v2 11/12: reasonably in-depth review from responsible co-maintainer
> >
> >              (yours truly), on the day of posting
> >
> > - v2 12/12: no feedback
> >
> >
> >
> > In total, I don't think the current process takes the blame for the delay. If
> reviewers don't care (or have no time) now, that problem will not change
> with the transition to github.com.
> >
> >
> >
> >
> >
> > (2) The VariablePolicy series is actually a good example that patch series
> restructuring is important.
> >
> >
> >
> > (2a) The patch count went from 9 (in v1) to 12 (in v2).
> >
> >
> >
> > (2b) And under v2, Liming still pointed out: "To keep each commit build
> pass, the patch set should first add new library instance, then add the library
> instance into each platform DSC, last update Variable driver to consume new
> library instance."
> >
> >
> >
> > Furthermore, I requested enabling the feature in ArmVirtPkg too, and
> maybe (based on owner feedback) UefiPayloadPkg.
> >
> >
> >
> > Thus, the v2->v3 update will most likely bring about both patch order
> changes, and an increased patch count.
> >
> >
> >
> > Thanks
> >
> > Laszlo
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
>
>






[-- Attachment #1.2: Type: text/html, Size: 28381 bytes --]

[-- Attachment #2: 801BEA439D70489191D6469EEA476862.png --]
[-- Type: image/png, Size: 140 bytes --]

Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

[Laszlo Ersek]

(+Leif, +Andrew)

Sean,

On 05/19/20 18:54, Sean Brogan wrote:
> Nate/Laszlo,
> 
> Regarding a squash merge workflow.  I agree it can be abused and we all
> have seen terrible examples.  But a patch series that contains 500+ file
> changes isn't really much better.  Just because it is broken into
> multiple commits doesn't mean it is the right set of commits.
> 
> Anyway a squash merge workflow works amazingly well with and is
> optimized for a web based review and PR processes.  It allows a user to
> respond to changes, fix issues, learn thru the PR process, all while
> keeping complete track of the progression.  Then once all "status"
> checks and reviews are complete, it is squashed into a neat commit for
> mainline, containing only the relevant data in the message.
> 
> So, the ask is that we don't exclude squash merge workflows.  Those
> reviewing the PR can decide what is appropriate for the PR content
> submitted.  Just as you would request changes to the contents (or
> ordering) of a commit in a series, if the reviewers don't agree that the
> PR contents should be in a single commit then obviously it shouldn't be
> squashed to one.
> 
> Contributions like spelling fixes, typos, minor bug fixes, documentation
> additions/fixes, etc all are great examples of PRs that can easily
> leverage squash merges and this workflow significantly lowers the burden
> of the contribution and review process.  This workflow is also are much
> easier for casual or first time contributors.
> 
> I don't exactly know how we would enable this but I assume we could
> leverage tags or make it clear in the PR description.  First step is to
> get alignment that a squash merge workflow, while not appropriate for
> all contributions, is not something to be excluded.

the scope for migrating the contribution & review workflows off the
mailing list and to github.com was set many months ago. That scope does
not include institutionalized changes to patch set structuring criteria.
The "git forge" evaluations that we had spent weeks/months on also
focused on how candidate systems would honor a patch series' structure;
i.e., how faithful the system would remain to the contributors' and
reviewers' shared intent, with a specific patch set.

Your proposal to "don't exclude squash merge workflows" is a trap. If we
tolerate that option -- which is obviously the sloppy, and hence more
convenient, option for some maintainers and some contributors, to the
detriment of the git history --, then almost every core maintainer will
use it as frequently as they can. In the long term, that will hurt many
consumers of the core code. It will limit the ability of people not
regularly dealing with a particular core module to file a fine-grained
bug report for that module, maybe even propose a fix. From the
regression analyst's side, if the bug report starts with "I have a
bisection log", that's already a good day. And your proposal would
destroy that option, because maintainers and people in general are
irrepairably lazy and undisciplined. We cannot post a community member
shoulder-by-shoulder with every core package reviewer/maintainer to
prevent the latter from approving a squash-on-merge, out of pure
laziness. I'm 100% sure the "option" to squash-on-merge would
*immediately* be abused for a lot more than just "typo fixes". There
isn't enough manpower to watch the watchers, so "no squash-on-merge"
needs to be a general rule.

I'm very sad that you're trying to wiggle such a crucial and intrusive
workflow change into the scope of this transition. Every time
squash-on-merge has come up over the years (regardless of this
transition), we've labeled it as one thing never to do, because it
destroys information (and/or even encourages not *creating* that
historical information in the first place, which is of course important
in reality).

Well, anyway, here's my feedback: if squash-on-merge is permitted in
edk2 or in basetools (or in any other external repository that's a hard
requirement for building edk2), that's a deal breaker for me, and I'll
hand in my resignation as a steward.

Maybe you'd consider that a win, I don't know -- but I couldn't remain a
steward with a straight face after failing to protect what I consider
one of the core values of open source / distributed development.

Thanks,
Laszlo

Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

[Nate DeSimone]

Hi Bret,

To be completely fair, I think we are splitting hairs on details here. I think both of us are in 90% agreement, and we are both passionate enough about our work to argue that last 10% to the grave.

I totally understand the desire for bisectability by the way. TianoCore is a huge codebase, the core modules have several extremely large functions, and very little in the way of explicit documentation. It has taken me years to learn how this beast works. I think it is possible to not squash every patch series and still maintain bisectability.

For example, your VariablePolicy patch series; we definitely want the patch that adds VariablePolicyLib to MdeModulePkg merged before the patch that adds it to OvmfPkg. But if the patch series is done carefully it can still be bisectable. In fact, bisectability will only be maintained iff we merge the entire series in the order that you/Michael sent it; if OvmfPkg gets merged first, then OvmfPkg will fail to build until the MdeModulePkg patch is merged. I don't think it would be the right thing to squash the OvmfPkg & MdeModulePkg patches together, as they really are distinct steps that you took on your journey towards making the VariablePolicy sausage.

Of course, there may be other patch series that would be logical to squash, especially if the author has not been careful to maintain bisectability. For example, I think of some patch series went a little overboard and could have been done in maybe 1-2 patches instead of 8-10. I would be happy to compromise with you and say that squashes can be done in circumstances where both the maintainer and the author agree to it.

Finally, I believe I can speak for everyone here that we all welcome your contributions. I think Mike and the rest of the community are trying to adjust the process to make contributing viable for a larger set of people. But at the same time, you must realize that TianoCore isn't just going to do everything exactly the same way that Microsoft does. You and Sean are expected to compromise with the rest of the community.

Thanks,
Nate

On 5/19/20, 1:11 PM, "rfc@edk2.groups.io on behalf of Bret Barkelew via groups.io" <rfc@edk2.groups.io on behalf of bret.barkelew=microsoft.com@groups.io> wrote:

    I will honor Mike Kinney’s efforts with my vote of confidence.
    I think we’re headed in the right direction, even with some of the things that I disagree with.

    In my history with TianoCore, I have learned to not be so quick to say “this is fucking stupid”. Every time I’ve done that, I’ve later discovered the reasons behind it, and even come to the conclusion that the designers were quite clever.

    That said, I want to contribute. And I won’t with the current system. I hope to be able to with the future system.

    - Bret

    From: Desimone, Nathaniel L<mailto:nathaniel.l.desimone@intel.com>
    Sent: Tuesday, May 19, 2020 12:59 PM
    To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Bret Barkelew<mailto:Bret.Barkelew@microsoft.com>; spbrogan@outlook.com<mailto:spbrogan@outlook.com>; rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>; lersek@redhat.com<mailto:lersek@redhat.com>; Kinney, Michael D<mailto:michael.d.kinney@intel.com>
    Subject: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

    Hi Bret,

    I believe you missed my point. I don’t want my patch series to be merged piece by piece; I want it merged all at once, in the order that I specified.

    I tend to agree with Laszlo that you are choosing not to learn how to use Git properly. Commit early, commit often, perfect later, publish once is the Git best practice. You should not hide the sausage making, which is exactly what you are proposing. I find it unfortunate that you consider refusing to learn GIt best practices a mark of prestige.

    Thanks,
    Nate

    From: <devel@edk2.groups.io> on behalf of "Bret Barkelew via groups.io" <bret.barkelew=microsoft.com@groups.io>
    Reply-To: "devel@edk2.groups.io" <devel@edk2.groups.io>, "bret.barkelew@microsoft.com" <bret.barkelew@microsoft.com>
    Date: Tuesday, May 19, 2020 at 12:35 PM
    To: "devel@edk2.groups.io" <devel@edk2.groups.io>, "Desimone, Nathaniel L" <nathaniel.l.desimone@intel.com>, "spbrogan@outlook.com" <spbrogan@outlook.com>, "rfc@edk2.groups.io" <rfc@edk2.groups.io>, "lersek@redhat.com" <lersek@redhat.com>, "Kinney, Michael D" <michael.d.kinney@intel.com>
    Subject: Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

    Nate, I believe you missed Sean’s point.

    Each one of those packages should have been a separate PR.
    Ergo, no information would have been lost in the squash.

    Also, it’s not so much that we *can’t* learn. It’s that we choose not to. Around here, it’s a mark of prestige to not open doors with your face if it seems like there’s a better way. Makes it easier to focus on the work.

    - Bret

    From: Nate DeSimone via groups.io<mailto:nathaniel.l.desimone=intel.com@groups.io>
    Sent: Tuesday, May 19, 2020 11:02 AM
    To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; spbrogan@outlook.com<mailto:spbrogan@outlook.com>; rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>; lersek@redhat.com<mailto:lersek@redhat.com>; Bret Barkelew<mailto:Bret.Barkelew@microsoft.com>; Kinney, Michael D<mailto:michael.d.kinney@intel.com>
    Subject: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

    Hi Sean,

    My recent spelling fix patch series is a good example of why this is a bad idea actually:

    https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59779&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=fVz16E37%2BwW5pSgRxI45K7nWPDlIoS0HuI8UCGmEwjY%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59779&data=02%7C01%7Cbret.barkelew%40microsoft.com%7Cbf6d5aed67374c95ca3508d7fc2f20ec%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255151662145479&sdata=m66IViN3G%2BbJpBwolRXf8d3BpWQeRXs495WYxnsD85M%3D&reserved=0>
    https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59780&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=4q0lC1BSlSoQ3p0HGWwlph09HTjgJRo4nTO2Qx59%2Fjc%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59780&data=02%7C01%7Cbret.barkelew%40microsoft.com%7Cbf6d5aed67374c95ca3508d7fc2f20ec%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255151662155477&sdata=VuYPqwcmYTrbiYdo%2B8K5irX8k6rgMgEoC2fY8eAocWA%3D&reserved=0>
    https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59781&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=XQVSwPMXdpDJXj9nkuvq2fenwhNt6HGGZXsJwH5Bu8E%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59781&data=02%7C01%7Cbret.barkelew%40microsoft.com%7Cbf6d5aed67374c95ca3508d7fc2f20ec%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255151662165472&sdata=lMrhfWKBWKGkjNnXJJy7%2BisrugTi0J%2FUkmtnj7Vxb7Q%3D&reserved=0>
    https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59782&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=kCULGBc6%2Bifcn3cnPTV1odHI1ZUxuWQePN3POKKS3SM%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59782&data=02%7C01%7Cbret.barkelew%40microsoft.com%7Cbf6d5aed67374c95ca3508d7fc2f20ec%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255151662165472&sdata=rltVYSQcSLG2sGP4d2awDIuWV11nYQcdxvqyPxDM4XE%3D&reserved=0>
    https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59783&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=SCOhUMdNXHIymGLaw9z3JTh%2Fe2BfaJaAyEC99EkG%2Fvg%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59783&data=02%7C01%7Cbret.barkelew%40microsoft.com%7Cbf6d5aed67374c95ca3508d7fc2f20ec%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255151662175469&sdata=b0J%2BL4nddt3g%2FmidZO61tlkgVqrRsufOUHGjRqh1CJM%3D&reserved=0>
    https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59784&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=epET6Wk30bIHQCvEDFLkeHEfmm9tzlxRrJ%2FQAuEfQFs%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59784&data=02%7C01%7Cbret.barkelew%40microsoft.com%7Cbf6d5aed67374c95ca3508d7fc2f20ec%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255151662185463&sdata=zhHUUmckRdY45oYIPodqP9r3Sh4Q4t%2FZYRLULOiAERs%3D&reserved=0>
    https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59785&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=N8T7HjerJVvyGg94yMWjLm%2Fw7WDdXOdby1JpOYlPeVc%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59785&data=02%7C01%7Cbret.barkelew%40microsoft.com%7Cbf6d5aed67374c95ca3508d7fc2f20ec%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255151662185463&sdata=MNUfWJhjb1UfPFciUJ2dcu9uWQwUNkS3PJO%2BeMkFVxA%3D&reserved=0>

    Notice that I split along package boundaries, because the maintainers for each package is a different set of people. If my patch series was squashed at merge time... how do I know who reviewed what? If the commit set is not correct.. I tend to say so in my feedback :). The only sane way to squash this series would be to have a human re-write all the commit messages, which I am against.

    Generally those that prefer an easily bisectable history have such preference mostly due to the usage of validators that immediately resort bisecting as a method to root cause an issue since they tend to not understand the code very well. Edk2 already has 12 years of non-bisectable history, so this method is going to be ineffective anyway.

    With regard to sending squashed commits, I understand that those who are new may have difficulty sending a properly formatted patch series, but frankly attempting to shield them from having to learn I am strongly against. I suggest that Microsoft invest in its human capital similar to how Intel does. If you cannot figure out how to send a properly formatted patch series... then do your work on the internal codebase (or perhaps MU.) Within the Intel, having the skillset to contribute to TianoCore is considered a mark of prestige, and thus needs to be earned.

    TLDR, I will reject squashed commits on any packages that I maintain.

    Thanks,
    Nate

    > -----Original Message-----
    > From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Sean
    > Sent: Tuesday, May 19, 2020 9:54 AM
    > To: rfc@edk2.groups.io; Desimone, Nathaniel L
    > <nathaniel.l.desimone@intel.com>; lersek@redhat.com;
    > bret.barkelew@microsoft.com; devel@edk2.groups.io; Kinney, Michael D
    > <michael.d.kinney@intel.com>
    > Subject: Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review
    > Process
    >
    > Nate/Laszlo,
    >
    > Regarding a squash merge workflow.  I agree it can be abused and we all
    > have seen terrible examples.  But a patch series that contains 500+ file
    > changes isn't really much better.  Just because it is broken into multiple
    > commits doesn't mean it is the right set of commits.
    >
    > Anyway a squash merge workflow works amazingly well with and is
    > optimized for a web based review and PR processes.  It allows a user to
    > respond to changes, fix issues, learn thru the PR process, all while keeping
    > complete track of the progression.  Then once all "status"
    > checks and reviews are complete, it is squashed into a neat commit for
    > mainline, containing only the relevant data in the message.
    >
    > So, the ask is that we don't exclude squash merge workflows.  Those
    > reviewing the PR can decide what is appropriate for the PR content
    > submitted.  Just as you would request changes to the contents (or
    > ordering) of a commit in a series, if the reviewers don't agree that the PR
    > contents should be in a single commit then obviously it shouldn't be
    > squashed to one.
    >
    > Contributions like spelling fixes, typos, minor bug fixes, documentation
    > additions/fixes, etc all are great examples of PRs that can easily leverage
    > squash merges and this workflow significantly lowers the burden of the
    > contribution and review process.  This workflow is also are much easier for
    > casual or first time contributors.
    >
    > I don't exactly know how we would enable this but I assume we could
    > leverage tags or make it clear in the PR description.  First step is to get
    > alignment that a squash merge workflow, while not appropriate for all
    > contributions, is not something to be excluded.
    >
    > Thanks
    > Sean
    >
    >
    >
    > On 5/19/2020 12:21 AM, Nate DeSimone wrote:
    > > Hi All,
    > >
    > >
    > >
    > > I tend to agree with most of Laszlo's points. Specifically, that moving to pull
    > requests will not fix the fact that maintainers are usually busy people and
    > don't always give feedback in a punctual manner. Like Laszlo, I would also
    > prefer that we do not squash patch series. My biggest reason for not
    > squashing patch series is because when you put everything into a single
    > commit, I have had to review commits with 500+ files changed. Opening git
    > difftool on a commit like that is awful.
    > >
    > >
    > >
    > > However, I would like to register my general endorsement for pull requests
    > or some other web based system of code review… and I don’t have an
    > Instagram account by the way :) Personally, I prefer Gerrit as I use it a lot with
    > coreboot and other projects. But since we are using Github for hosting, pull
    > requests are an easy switch and a logical choice. My main reason for being
    > excited about pull requests mostly has to do with the amount of manual
    > effort required to be a TianoCore maintainer right now. I have set up my
    > email filter so that the mailing list is categorized like so:
    > >
    > >
    > >
    > > [cid:image001.png@01D62D71.502B55E0]
    > >
    > >
    > >
    > > Implementing the logic to parse the contents of emails to categorize them
    > like this required me to define no less than 12 email filter rules in Microsoft
    > Outlook, and I have to change my filtering logic every time I am
    > added/removed from a Maintainers.txt file. I’m sure every other maintainer
    > has spent a time separately implementing filtering logic like I have. This helps,
    > but still for every thread, I have to go and check if one of the other
    > maintainers has already reviewed/pushed that patch series yet, and if not
    > review/push it. If I have ] feedback on a patch series, I have to categorize it
    > as awaiting response from author and check up on it from time to time,
    > sometimes I ping the author directly and remind them to send a new patch
    > series. Implementing this state machine is a lot of manual work and it kind of
    > feels like I’m a telephone operator in the 1950s. I greatly welcome
    > automation here as I am sure it will increase the number of patch series I am
    > able to review per hour.
    > >
    > >
    > >
    > > Thanks,
    > >
    > > Nate
    > >
    > >
    > >
    > > -----Original Message-----
    > > From: rfc@edk2.groups.io <rfc@edk2.groups.io> On Behalf Of Laszlo
    > > Ersek
    > > Sent: Friday, May 15, 2020 2:08 AM
    > > To: rfc@edk2.groups.io; bret.barkelew@microsoft.com;
    > > devel@edk2.groups.io; Kinney, Michael D <michael.d.kinney@intel.com>
    > > Subject: Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull
    > > Request based Code Review Process
    > >
    > >
    > >
    > > On 05/15/20 06:49, Bret Barkelew via groups.io wrote:
    > >
    > >
    > >
    > >> I would far prefer the approach of individual PRs for commits to
    > >> allow
    > >
    > >> for the squash flexibility (and is the strategy I think I would
    > >> pursue
    > >
    > >> with my PRs). For example, the VarPol PR would be broken up into 9
    > >> PRs
    > >
    > >> for each final commit, and we can get them in one by one.
    > >
    > >> Ideally, each one would be a small back and forth and then in. If it
    > >
    > >> had been done that way to begin with, it would be over in a week and
    > >> a
    > >
    > >> half or so, rather than the multiple months that we’re now verging
    > >
    > >> on.
    > >
    > >
    > >
    > > This differs extremely from how we've been working on edk2-devel (or
    > from how any git-based project works that I've ever been involved with).
    > >
    > > And I think the above workflow is out of scope, for migrating the edk2
    > process to github.
    > >
    > >
    > >
    > > Again, the structuring of a patch series is a primary trait. Iterating only on
    > individual patches does not allow for the reordering / restructuring of the
    > patch series (dropping patches, reordering patches, inserting patches,
    > moving hunks between patches).
    > >
    > >
    > >
    > > It's common that the necessity to revise an earlier patch emerges while
    > reworking a later patch. For instance, the git-rebase(1) manual dedicates a
    > separate section to "splitting commits".
    > >
    > >
    > >
    > > In the initial evaluation of "web forges", Phabricator was one of the
    > "contestants". Phabricator didn't support the "patch series" concept at all, it
    > only supported review requests for individual patches, and it supported
    > setting up dependencies between them. So, for example, a 27-patch series
    > would require 27 submissions and 26 dependencies.
    > >
    > >
    > >
    > > Lacking support for the patch series concept was an immediate deal
    > breaker with Phabricator.
    > >
    > >
    > >
    > > The longest patch series I've ever submitted to edk2-devel had 58 patches.
    > It was SMM enablement for OVMF. It went from v1 to v5 (v5 was merged),
    > and the patch count varied significantly:
    > >
    > >
    > >
    > > v1: 58 patches (25 Jul 2015)
    > >
    > > v2: 41 patches ( 9 Oct 2015)
    > >
    > > v3: 52 patches (15 Oct 2015)
    > >
    > > v4: 41 patches ( 3 Nov 2015)
    > >
    > > v5: 33 patches (27 Nov 2015)
    > >
    > >
    > >
    > > (The significant drop in the patch count was due to Mike Kinney open
    > > sourcing and upstreaming the *real* PiSmmCpuDxeSmm driver (which was
    > > huge work in its own right), allowing me to drop the Quark-originated
    > > 32-bit-only PiSmmCpuDxeSmm variant, from my series.)
    > >
    > >
    > >
    > > The contribution process should make difficult things possible, even if that
    > complicates simple things somewhat. A process that makes simple things
    > simple and difficult things impossible is useless. This is what the Instagram
    > generation seems to be missing.
    > >
    > >
    > >
    > >
    > >
    > > I don't know why the VariablePolicy work took months. I can see the
    > following threads on the list:
    > >
    > >
    > >
    > > * [edk2-devel] [PATCH v1 0/9] Add the VariablePolicy feature
    > >
    > >    Fri, 10 Apr 2020 11:36:01 -0700
    > >
    > >
    > >
    > > * [edk2-devel] [PATCH v2 00/12] Add the VariablePolicy feature
    > >
    > >    Mon, 11 May 2020 23:46:23 -0700
    > >
    > >
    > >
    > > I have two sets of comments:
    > >
    > >
    > >
    > > (1) It's difficult to tell in retrospect (because the series seem to have been
    > posted with somewhat problematic threading), but the delay apparently
    > came from multiple sources.
    > >
    > >
    > >
    > > (1a) Review was slow and spotty.
    > >
    > >
    > >
    > > The v1 blurb received some comments in the first week after it was posted.
    > But the rest of the v1 series (the actual patches) received feedback like this:
    > >
    > >
    > >
    > > - v1 1/9: no feedback
    > >
    > > - v1 2/9: 12 days after posting
    > >
    > > - v1 3/9: 16 days after posting
    > >
    > > - v1 4/9: no feedback
    > >
    > > - v1 5/9: no feedback
    > >
    > > - v1 6/9: no feedback
    > >
    > > - v1 7/9: no feedback
    > >
    > > - v1 8/9: no feedback
    > >
    > > - v1 9/9: no feedback
    > >
    > >
    > >
    > > (1b) There was also quite some time between the last response in the v1
    > thread (Apr 26th, as far as I can see), and the posting of the v2 series (May
    > 11th).
    > >
    > >
    > >
    > > (1c) The v2 blurb got almost immediate, and numerous feedback (on the
    > day of posting, and the day after). Regarding the individual patches, they
    > didn't fare too well:
    > >
    > >
    > >
    > > - v2 01/12: superficial comment on the day of posting from me (not a
    > >
    > >              designated MdeModulePkg review), on the day of posting;
    > > no
    > >
    > >              other feedback thus far
    > >
    > > - v2 02/12: ditto
    > >
    > > - v2 03/12: no feedback
    > >
    > > - v2 04/12: superficial (coding style) comments on the day of posting
    > >
    > > - v2 05/12: no feedback
    > >
    > > - v2 06/12: no feedback
    > >
    > > - v2 07/12: no feedback
    > >
    > > - v2 08/12: no feedback
    > >
    > > - v2 09/12: no feedback
    > >
    > > - v2 10/12: no feedback
    > >
    > > - v2 11/12: reasonably in-depth review from responsible co-maintainer
    > >
    > >              (yours truly), on the day of posting
    > >
    > > - v2 12/12: no feedback
    > >
    > >
    > >
    > > In total, I don't think the current process takes the blame for the delay. If
    > reviewers don't care (or have no time) now, that problem will not change
    > with the transition to github.com.
    > >
    > >
    > >
    > >
    > >
    > > (2) The VariablePolicy series is actually a good example that patch series
    > restructuring is important.
    > >
    > >
    > >
    > > (2a) The patch count went from 9 (in v1) to 12 (in v2).
    > >
    > >
    > >
    > > (2b) And under v2, Liming still pointed out: "To keep each commit build
    > pass, the patch set should first add new library instance, then add the library
    > instance into each platform DSC, last update Variable driver to consume new
    > library instance."
    > >
    > >
    > >
    > > Furthermore, I requested enabling the feature in ArmVirtPkg too, and
    > maybe (based on owner feedback) UefiPayloadPkg.
    > >
    > >
    > >
    > > Thus, the v2->v3 update will most likely bring about both patch order
    > changes, and an increased patch count.
    > >
    > >
    > >
    > > Thanks
    > >
    > > Laszlo
    > >
    > >
    > >
    > >
    > >
    > >
    > >
    > >
    > >
    > >
    > >
    >
    >






    

Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

[Bret Barkelew]

[-- Attachment #1: Type: text/plain, Size: 25097 bytes --]

I’ll pour another cup of tea to that.

- Bret

________________________________
From: Desimone, Nathaniel L <nathaniel.l.desimone@intel.com>
Sent: Tuesday, May 19, 2020 2:02:49 PM
To: rfc@edk2.groups.io <rfc@edk2.groups.io>; Bret Barkelew <Bret.Barkelew@microsoft.com>; devel@edk2.groups.io <devel@edk2.groups.io>; spbrogan@outlook.com <spbrogan@outlook.com>; lersek@redhat.com <lersek@redhat.com>; Kinney, Michael D <michael.d.kinney@intel.com>
Subject: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Hi Bret,

To be completely fair, I think we are splitting hairs on details here. I think both of us are in 90% agreement, and we are both passionate enough about our work to argue that last 10% to the grave.

I totally understand the desire for bisectability by the way. TianoCore is a huge codebase, the core modules have several extremely large functions, and very little in the way of explicit documentation. It has taken me years to learn how this beast works. I think it is possible to not squash every patch series and still maintain bisectability.

For example, your VariablePolicy patch series; we definitely want the patch that adds VariablePolicyLib to MdeModulePkg merged before the patch that adds it to OvmfPkg. But if the patch series is done carefully it can still be bisectable. In fact, bisectability will only be maintained iff we merge the entire series in the order that you/Michael sent it; if OvmfPkg gets merged first, then OvmfPkg will fail to build until the MdeModulePkg patch is merged. I don't think it would be the right thing to squash the OvmfPkg & MdeModulePkg patches together, as they really are distinct steps that you took on your journey towards making the VariablePolicy sausage.

Of course, there may be other patch series that would be logical to squash, especially if the author has not been careful to maintain bisectability. For example, I think of some patch series went a little overboard and could have been done in maybe 1-2 patches instead of 8-10. I would be happy to compromise with you and say that squashes can be done in circumstances where both the maintainer and the author agree to it.

Finally, I believe I can speak for everyone here that we all welcome your contributions. I think Mike and the rest of the community are trying to adjust the process to make contributing viable for a larger set of people. But at the same time, you must realize that TianoCore isn't just going to do everything exactly the same way that Microsoft does. You and Sean are expected to compromise with the rest of the community.

Thanks,
Nate

On 5/19/20, 1:11 PM, "rfc@edk2.groups.io on behalf of Bret Barkelew via groups.io" <rfc@edk2.groups.io on behalf of bret.barkelew=microsoft.com@groups.io> wrote:

    I will honor Mike Kinney’s efforts with my vote of confidence.
    I think we’re headed in the right direction, even with some of the things that I disagree with.

    In my history with TianoCore, I have learned to not be so quick to say “this is fucking stupid”. Every time I’ve done that, I’ve later discovered the reasons behind it, and even come to the conclusion that the designers were quite clever.

    That said, I want to contribute. And I won’t with the current system. I hope to be able to with the future system.

    - Bret

    From: Desimone, Nathaniel L<mailto:nathaniel.l.desimone@intel.com>
    Sent: Tuesday, May 19, 2020 12:59 PM
    To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Bret Barkelew<mailto:Bret.Barkelew@microsoft.com>; spbrogan@outlook.com<mailto:spbrogan@outlook.com>; rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>; lersek@redhat.com<mailto:lersek@redhat.com>; Kinney, Michael D<mailto:michael.d.kinney@intel.com>
    Subject: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

    Hi Bret,

    I believe you missed my point. I don’t want my patch series to be merged piece by piece; I want it merged all at once, in the order that I specified.

    I tend to agree with Laszlo that you are choosing not to learn how to use Git properly. Commit early, commit often, perfect later, publish once is the Git best practice. You should not hide the sausage making, which is exactly what you are proposing. I find it unfortunate that you consider refusing to learn GIt best practices a mark of prestige.

    Thanks,
    Nate

    From: <devel@edk2.groups.io> on behalf of "Bret Barkelew via groups.io" <bret.barkelew=microsoft.com@groups.io>
    Reply-To: "devel@edk2.groups.io" <devel@edk2.groups.io>, "bret.barkelew@microsoft.com" <bret.barkelew@microsoft.com>
    Date: Tuesday, May 19, 2020 at 12:35 PM
    To: "devel@edk2.groups.io" <devel@edk2.groups.io>, "Desimone, Nathaniel L" <nathaniel.l.desimone@intel.com>, "spbrogan@outlook.com" <spbrogan@outlook.com>, "rfc@edk2.groups.io" <rfc@edk2.groups.io>, "lersek@redhat.com" <lersek@redhat.com>, "Kinney, Michael D" <michael.d.kinney@intel.com>
    Subject: Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

    Nate, I believe you missed Sean’s point.

    Each one of those packages should have been a separate PR.
    Ergo, no information would have been lost in the squash.

    Also, it’s not so much that we *can’t* learn. It’s that we choose not to. Around here, it’s a mark of prestige to not open doors with your face if it seems like there’s a better way. Makes it easier to focus on the work.

    - Bret

    From: Nate DeSimone via groups.io<mailto:nathaniel.l.desimone=intel.com@groups.io>
    Sent: Tuesday, May 19, 2020 11:02 AM
    To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; spbrogan@outlook.com<mailto:spbrogan@outlook.com>; rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>; lersek@redhat.com<mailto:lersek@redhat.com>; Bret Barkelew<mailto:Bret.Barkelew@microsoft.com>; Kinney, Michael D<mailto:michael.d.kinney@intel.com>
    Subject: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

    Hi Sean,

    My recent spelling fix patch series is a good example of why this is a bad idea actually:

    https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59779&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C2edef9b2c759477da79708d7fc38039d%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255189829127641&amp;sdata=xYwOvgWRR2emIUFhy3CG%2Frxs774JyHIhlA0%2BrzV8dlU%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59779&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C2edef9b2c759477da79708d7fc38039d%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255189829127641&amp;sdata=xYwOvgWRR2emIUFhy3CG%2Frxs774JyHIhlA0%2BrzV8dlU%3D&amp;reserved=0>
    https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59780&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C2edef9b2c759477da79708d7fc38039d%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255189829127641&amp;sdata=Fvqm3EWSpquv3QSxvh8uhAK1tSxlz%2Fwd7EeeyBSMQis%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59780&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C2edef9b2c759477da79708d7fc38039d%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255189829137632&amp;sdata=vi503StyynvzgVF1JO6HeL0enBF0gpne%2FmZFa5nyx9Q%3D&amp;reserved=0>
    https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59781&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C2edef9b2c759477da79708d7fc38039d%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255189829137632&amp;sdata=YXgZgfwFGRiHzlN92j7jaf8hPaA58iD21A483yCesB8%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59781&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C2edef9b2c759477da79708d7fc38039d%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255189829137632&amp;sdata=YXgZgfwFGRiHzlN92j7jaf8hPaA58iD21A483yCesB8%3D&amp;reserved=0>
    https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59782&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C2edef9b2c759477da79708d7fc38039d%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255189829137632&amp;sdata=Cad7XxJOMK%2FEsczlQVu3DcITjVQbC1j797Q11DbAISU%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59782&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C2edef9b2c759477da79708d7fc38039d%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255189829137632&amp;sdata=Cad7XxJOMK%2FEsczlQVu3DcITjVQbC1j797Q11DbAISU%3D&amp;reserved=0>
    https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59783&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C2edef9b2c759477da79708d7fc38039d%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255189829137632&amp;sdata=3ToBlN2v4o6ip13o7isAMg29pMcmh9SBSvMIDojvl8o%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59783&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C2edef9b2c759477da79708d7fc38039d%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255189829137632&amp;sdata=3ToBlN2v4o6ip13o7isAMg29pMcmh9SBSvMIDojvl8o%3D&amp;reserved=0>
    https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59784&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C2edef9b2c759477da79708d7fc38039d%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255189829137632&amp;sdata=jic1pWXehzaiYdq3ihpR7uXZ9R0T0XdsUsc%2FHeAgpUo%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59784&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C2edef9b2c759477da79708d7fc38039d%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255189829137632&amp;sdata=jic1pWXehzaiYdq3ihpR7uXZ9R0T0XdsUsc%2FHeAgpUo%3D&amp;reserved=0>
    https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59785&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C2edef9b2c759477da79708d7fc38039d%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255189829147624&amp;sdata=LtOhDuYjnpe1OspUg33%2BEhSxnd0fG9COnCJjSrXJM9E%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59785&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C2edef9b2c759477da79708d7fc38039d%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255189829147624&amp;sdata=LtOhDuYjnpe1OspUg33%2BEhSxnd0fG9COnCJjSrXJM9E%3D&amp;reserved=0>

    Notice that I split along package boundaries, because the maintainers for each package is a different set of people. If my patch series was squashed at merge time... how do I know who reviewed what? If the commit set is not correct.. I tend to say so in my feedback :). The only sane way to squash this series would be to have a human re-write all the commit messages, which I am against.

    Generally those that prefer an easily bisectable history have such preference mostly due to the usage of validators that immediately resort bisecting as a method to root cause an issue since they tend to not understand the code very well. Edk2 already has 12 years of non-bisectable history, so this method is going to be ineffective anyway.

    With regard to sending squashed commits, I understand that those who are new may have difficulty sending a properly formatted patch series, but frankly attempting to shield them from having to learn I am strongly against. I suggest that Microsoft invest in its human capital similar to how Intel does. If you cannot figure out how to send a properly formatted patch series... then do your work on the internal codebase (or perhaps MU.) Within the Intel, having the skillset to contribute to TianoCore is considered a mark of prestige, and thus needs to be earned.

    TLDR, I will reject squashed commits on any packages that I maintain.

    Thanks,
    Nate

    > -----Original Message-----
    > From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Sean
    > Sent: Tuesday, May 19, 2020 9:54 AM
    > To: rfc@edk2.groups.io; Desimone, Nathaniel L
    > <nathaniel.l.desimone@intel.com>; lersek@redhat.com;
    > bret.barkelew@microsoft.com; devel@edk2.groups.io; Kinney, Michael D
    > <michael.d.kinney@intel.com>
    > Subject: Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review
    > Process
    >
    > Nate/Laszlo,
    >
    > Regarding a squash merge workflow.  I agree it can be abused and we all
    > have seen terrible examples.  But a patch series that contains 500+ file
    > changes isn't really much better.  Just because it is broken into multiple
    > commits doesn't mean it is the right set of commits.
    >
    > Anyway a squash merge workflow works amazingly well with and is
    > optimized for a web based review and PR processes.  It allows a user to
    > respond to changes, fix issues, learn thru the PR process, all while keeping
    > complete track of the progression.  Then once all "status"
    > checks and reviews are complete, it is squashed into a neat commit for
    > mainline, containing only the relevant data in the message.
    >
    > So, the ask is that we don't exclude squash merge workflows.  Those
    > reviewing the PR can decide what is appropriate for the PR content
    > submitted.  Just as you would request changes to the contents (or
    > ordering) of a commit in a series, if the reviewers don't agree that the PR
    > contents should be in a single commit then obviously it shouldn't be
    > squashed to one.
    >
    > Contributions like spelling fixes, typos, minor bug fixes, documentation
    > additions/fixes, etc all are great examples of PRs that can easily leverage
    > squash merges and this workflow significantly lowers the burden of the
    > contribution and review process.  This workflow is also are much easier for
    > casual or first time contributors.
    >
    > I don't exactly know how we would enable this but I assume we could
    > leverage tags or make it clear in the PR description.  First step is to get
    > alignment that a squash merge workflow, while not appropriate for all
    > contributions, is not something to be excluded.
    >
    > Thanks
    > Sean
    >
    >
    >
    > On 5/19/2020 12:21 AM, Nate DeSimone wrote:
    > > Hi All,
    > >
    > >
    > >
    > > I tend to agree with most of Laszlo's points. Specifically, that moving to pull
    > requests will not fix the fact that maintainers are usually busy people and
    > don't always give feedback in a punctual manner. Like Laszlo, I would also
    > prefer that we do not squash patch series. My biggest reason for not
    > squashing patch series is because when you put everything into a single
    > commit, I have had to review commits with 500+ files changed. Opening git
    > difftool on a commit like that is awful.
    > >
    > >
    > >
    > > However, I would like to register my general endorsement for pull requests
    > or some other web based system of code review… and I don’t have an
    > Instagram account by the way :) Personally, I prefer Gerrit as I use it a lot with
    > coreboot and other projects. But since we are using Github for hosting, pull
    > requests are an easy switch and a logical choice. My main reason for being
    > excited about pull requests mostly has to do with the amount of manual
    > effort required to be a TianoCore maintainer right now. I have set up my
    > email filter so that the mailing list is categorized like so:
    > >
    > >
    > >
    > > [cid:image001.png@01D62D71.502B55E0]
    > >
    > >
    > >
    > > Implementing the logic to parse the contents of emails to categorize them
    > like this required me to define no less than 12 email filter rules in Microsoft
    > Outlook, and I have to change my filtering logic every time I am
    > added/removed from a Maintainers.txt file. I’m sure every other maintainer
    > has spent a time separately implementing filtering logic like I have. This helps,
    > but still for every thread, I have to go and check if one of the other
    > maintainers has already reviewed/pushed that patch series yet, and if not
    > review/push it. If I have ] feedback on a patch series, I have to categorize it
    > as awaiting response from author and check up on it from time to time,
    > sometimes I ping the author directly and remind them to send a new patch
    > series. Implementing this state machine is a lot of manual work and it kind of
    > feels like I’m a telephone operator in the 1950s. I greatly welcome
    > automation here as I am sure it will increase the number of patch series I am
    > able to review per hour.
    > >
    > >
    > >
    > > Thanks,
    > >
    > > Nate
    > >
    > >
    > >
    > > -----Original Message-----
    > > From: rfc@edk2.groups.io <rfc@edk2.groups.io> On Behalf Of Laszlo
    > > Ersek
    > > Sent: Friday, May 15, 2020 2:08 AM
    > > To: rfc@edk2.groups.io; bret.barkelew@microsoft.com;
    > > devel@edk2.groups.io; Kinney, Michael D <michael.d.kinney@intel.com>
    > > Subject: Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull
    > > Request based Code Review Process
    > >
    > >
    > >
    > > On 05/15/20 06:49, Bret Barkelew via groups.io wrote:
    > >
    > >
    > >
    > >> I would far prefer the approach of individual PRs for commits to
    > >> allow
    > >
    > >> for the squash flexibility (and is the strategy I think I would
    > >> pursue
    > >
    > >> with my PRs). For example, the VarPol PR would be broken up into 9
    > >> PRs
    > >
    > >> for each final commit, and we can get them in one by one.
    > >
    > >> Ideally, each one would be a small back and forth and then in. If it
    > >
    > >> had been done that way to begin with, it would be over in a week and
    > >> a
    > >
    > >> half or so, rather than the multiple months that we’re now verging
    > >
    > >> on.
    > >
    > >
    > >
    > > This differs extremely from how we've been working on edk2-devel (or
    > from how any git-based project works that I've ever been involved with).
    > >
    > > And I think the above workflow is out of scope, for migrating the edk2
    > process to github.
    > >
    > >
    > >
    > > Again, the structuring of a patch series is a primary trait. Iterating only on
    > individual patches does not allow for the reordering / restructuring of the
    > patch series (dropping patches, reordering patches, inserting patches,
    > moving hunks between patches).
    > >
    > >
    > >
    > > It's common that the necessity to revise an earlier patch emerges while
    > reworking a later patch. For instance, the git-rebase(1) manual dedicates a
    > separate section to "splitting commits".
    > >
    > >
    > >
    > > In the initial evaluation of "web forges", Phabricator was one of the
    > "contestants". Phabricator didn't support the "patch series" concept at all, it
    > only supported review requests for individual patches, and it supported
    > setting up dependencies between them. So, for example, a 27-patch series
    > would require 27 submissions and 26 dependencies.
    > >
    > >
    > >
    > > Lacking support for the patch series concept was an immediate deal
    > breaker with Phabricator.
    > >
    > >
    > >
    > > The longest patch series I've ever submitted to edk2-devel had 58 patches.
    > It was SMM enablement for OVMF. It went from v1 to v5 (v5 was merged),
    > and the patch count varied significantly:
    > >
    > >
    > >
    > > v1: 58 patches (25 Jul 2015)
    > >
    > > v2: 41 patches ( 9 Oct 2015)
    > >
    > > v3: 52 patches (15 Oct 2015)
    > >
    > > v4: 41 patches ( 3 Nov 2015)
    > >
    > > v5: 33 patches (27 Nov 2015)
    > >
    > >
    > >
    > > (The significant drop in the patch count was due to Mike Kinney open
    > > sourcing and upstreaming the *real* PiSmmCpuDxeSmm driver (which was
    > > huge work in its own right), allowing me to drop the Quark-originated
    > > 32-bit-only PiSmmCpuDxeSmm variant, from my series.)
    > >
    > >
    > >
    > > The contribution process should make difficult things possible, even if that
    > complicates simple things somewhat. A process that makes simple things
    > simple and difficult things impossible is useless. This is what the Instagram
    > generation seems to be missing.
    > >
    > >
    > >
    > >
    > >
    > > I don't know why the VariablePolicy work took months. I can see the
    > following threads on the list:
    > >
    > >
    > >
    > > * [edk2-devel] [PATCH v1 0/9] Add the VariablePolicy feature
    > >
    > >    Fri, 10 Apr 2020 11:36:01 -0700
    > >
    > >
    > >
    > > * [edk2-devel] [PATCH v2 00/12] Add the VariablePolicy feature
    > >
    > >    Mon, 11 May 2020 23:46:23 -0700
    > >
    > >
    > >
    > > I have two sets of comments:
    > >
    > >
    > >
    > > (1) It's difficult to tell in retrospect (because the series seem to have been
    > posted with somewhat problematic threading), but the delay apparently
    > came from multiple sources.
    > >
    > >
    > >
    > > (1a) Review was slow and spotty.
    > >
    > >
    > >
    > > The v1 blurb received some comments in the first week after it was posted.
    > But the rest of the v1 series (the actual patches) received feedback like this:
    > >
    > >
    > >
    > > - v1 1/9: no feedback
    > >
    > > - v1 2/9: 12 days after posting
    > >
    > > - v1 3/9: 16 days after posting
    > >
    > > - v1 4/9: no feedback
    > >
    > > - v1 5/9: no feedback
    > >
    > > - v1 6/9: no feedback
    > >
    > > - v1 7/9: no feedback
    > >
    > > - v1 8/9: no feedback
    > >
    > > - v1 9/9: no feedback
    > >
    > >
    > >
    > > (1b) There was also quite some time between the last response in the v1
    > thread (Apr 26th, as far as I can see), and the posting of the v2 series (May
    > 11th).
    > >
    > >
    > >
    > > (1c) The v2 blurb got almost immediate, and numerous feedback (on the
    > day of posting, and the day after). Regarding the individual patches, they
    > didn't fare too well:
    > >
    > >
    > >
    > > - v2 01/12: superficial comment on the day of posting from me (not a
    > >
    > >              designated MdeModulePkg review), on the day of posting;
    > > no
    > >
    > >              other feedback thus far
    > >
    > > - v2 02/12: ditto
    > >
    > > - v2 03/12: no feedback
    > >
    > > - v2 04/12: superficial (coding style) comments on the day of posting
    > >
    > > - v2 05/12: no feedback
    > >
    > > - v2 06/12: no feedback
    > >
    > > - v2 07/12: no feedback
    > >
    > > - v2 08/12: no feedback
    > >
    > > - v2 09/12: no feedback
    > >
    > > - v2 10/12: no feedback
    > >
    > > - v2 11/12: reasonably in-depth review from responsible co-maintainer
    > >
    > >              (yours truly), on the day of posting
    > >
    > > - v2 12/12: no feedback
    > >
    > >
    > >
    > > In total, I don't think the current process takes the blame for the delay. If
    > reviewers don't care (or have no time) now, that problem will not change
    > with the transition to github.com.
    > >
    > >
    > >
    > >
    > >
    > > (2) The VariablePolicy series is actually a good example that patch series
    > restructuring is important.
    > >
    > >
    > >
    > > (2a) The patch count went from 9 (in v1) to 12 (in v2).
    > >
    > >
    > >
    > > (2b) And under v2, Liming still pointed out: "To keep each commit build
    > pass, the patch set should first add new library instance, then add the library
    > instance into each platform DSC, last update Variable driver to consume new
    > library instance."
    > >
    > >
    > >
    > > Furthermore, I requested enabling the feature in ArmVirtPkg too, and
    > maybe (based on owner feedback) UefiPayloadPkg.
    > >
    > >
    > >
    > > Thus, the v2->v3 update will most likely bring about both patch order
    > changes, and an increased patch count.
    > >
    > >
    > >
    > > Thanks
    > >
    > > Laszlo
    > >
    > >
    > >
    > >
    > >
    > >
    > >
    > >
    > >
    > >
    > >
    >
    >






    



[-- Attachment #2: Type: text/html, Size: 42660 bytes --]

Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

[Laszlo Ersek]

On 05/19/20 21:34, Bret Barkelew wrote:
> Nate, I believe you missed Sean’s point.
> 
> Each one of those packages should have been a separate PR.

And then we get to wrangle inter-PR dependencies.

Even if github.com supports that, it's a heavy-weight tool, and should
be used sparingly. Patches in a patch series are almost always
inter-dependent in some way, which indicates that many normal patch sets
would have to be split into multiple PRs.

> Ergo, no information would have been lost in the squash.
> 
> Also, it’s not so much that we *can’t* learn. It’s that we choose not to. Around here, it’s a mark of prestige to not open doors with your face if it seems like there’s a better way. Makes it easier to focus on the work.

Wrt. "open doors with your face", which I understand to be a retort to
Nate associating prestige with conforming to the current workflow:

I think the expression breaks the Code of Conduct:

  https://www.tianocore.org/coc.html

"Do not insult or put down other participants"

(... Before anyone suggests that I did the same when I called
maintainers & people en bloc "irrepairably lazy and undisciplined" in my
previous mail: that's a fact about humans.

People on average perform the minimum of work they can get away with,
for satisfying requirements and for reaching goals. It's natural. That's
why we have to set high standards. So that covers "lazy".

And "undisciplined" (= ignoring rules and good practices) is evidenced
frequently, with fixed BZs left open, posted patches not referenced in
the BZs they address, BZs ignored / left un-triaged for months and
years, pending patches ignored for weeks, reviewed patches left unmerged
for days or weeks, etc. I'm not throwing around accusations, just
showing that my statement was factual, hardly an insult. OTOH "open
doors with your face" is figurative speech, and I do consider it an insult.)

Thanks,
Laszlo

Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

[Nate DeSimone]

Hi Laszlo,

I think both myself and Bret may have gotten a little chippy. I think both of us are passionate about our work and that shows in the debate. I am happy to forgive Bret and hopefully he is with me as well.

Thanks,
Nate

On 5/19/20, 2:22 PM, "devel@edk2.groups.io on behalf of Laszlo Ersek" <devel@edk2.groups.io on behalf of lersek@redhat.com> wrote:

    On 05/19/20 21:34, Bret Barkelew wrote:
    > Nate, I believe you missed Sean’s point.
    > 
    > Each one of those packages should have been a separate PR.

    And then we get to wrangle inter-PR dependencies.

    Even if github.com supports that, it's a heavy-weight tool, and should
    be used sparingly. Patches in a patch series are almost always
    inter-dependent in some way, which indicates that many normal patch sets
    would have to be split into multiple PRs.

    > Ergo, no information would have been lost in the squash.
    > 
    > Also, it’s not so much that we *can’t* learn. It’s that we choose not to. Around here, it’s a mark of prestige to not open doors with your face if it seems like there’s a better way. Makes it easier to focus on the work.

    Wrt. "open doors with your face", which I understand to be a retort to
    Nate associating prestige with conforming to the current workflow:

    I think the expression breaks the Code of Conduct:

      https://www.tianocore.org/coc.html

    "Do not insult or put down other participants"

    (... Before anyone suggests that I did the same when I called
    maintainers & people en bloc "irrepairably lazy and undisciplined" in my
    previous mail: that's a fact about humans.

    People on average perform the minimum of work they can get away with,
    for satisfying requirements and for reaching goals. It's natural. That's
    why we have to set high standards. So that covers "lazy".

    And "undisciplined" (= ignoring rules and good practices) is evidenced
    frequently, with fixed BZs left open, posted patches not referenced in
    the BZs they address, BZs ignored / left un-triaged for months and
    years, pending patches ignored for weeks, reviewed patches left unmerged
    for days or weeks, etc. I'm not throwing around accusations, just
    showing that my statement was factual, hardly an insult. OTOH "open
    doors with your face" is figurative speech, and I do consider it an insult.)

    Thanks,
    Laszlo


    

Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

[Bret Barkelew]

[-- Attachment #1: Type: text/plain, Size: 3464 bytes --]

Agreed. :)

- Bret
________________________________
From: devel@edk2.groups.io <devel@edk2.groups.io> on behalf of Nate DeSimone via groups.io <nathaniel.l.desimone=intel.com@groups.io>
Sent: Tuesday, May 19, 2020 2:35:37 PM
To: devel@edk2.groups.io <devel@edk2.groups.io>; lersek@redhat.com <lersek@redhat.com>; Bret Barkelew <Bret.Barkelew@microsoft.com>; spbrogan@outlook.com <spbrogan@outlook.com>; rfc@edk2.groups.io <rfc@edk2.groups.io>; Kinney, Michael D <michael.d.kinney@intel.com>; Leif Lindholm (Nuvia address) <leif@nuviainc.com>; Andrew Fish <afish@apple.com>
Subject: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Hi Laszlo,

I think both myself and Bret may have gotten a little chippy. I think both of us are passionate about our work and that shows in the debate. I am happy to forgive Bret and hopefully he is with me as well.

Thanks,
Nate

On 5/19/20, 2:22 PM, "devel@edk2.groups.io on behalf of Laszlo Ersek" <devel@edk2.groups.io on behalf of lersek@redhat.com> wrote:

    On 05/19/20 21:34, Bret Barkelew wrote:
    > Nate, I believe you missed Sean’s point.
    >
    > Each one of those packages should have been a separate PR.

    And then we get to wrangle inter-PR dependencies.

    Even if github.com supports that, it's a heavy-weight tool, and should
    be used sparingly. Patches in a patch series are almost always
    inter-dependent in some way, which indicates that many normal patch sets
    would have to be split into multiple PRs.

    > Ergo, no information would have been lost in the squash.
    >
    > Also, it’s not so much that we *can’t* learn. It’s that we choose not to. Around here, it’s a mark of prestige to not open doors with your face if it seems like there’s a better way. Makes it easier to focus on the work.

    Wrt. "open doors with your face", which I understand to be a retort to
    Nate associating prestige with conforming to the current workflow:

    I think the expression breaks the Code of Conduct:

      https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.tianocore.org%2Fcoc.html&amp;data=02%7C01%7CBret.Barkelew%40microsoft.com%7C0b810c962b8045eb903108d7fc3c947f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255209426194060&amp;sdata=2nyvAPNoCddaBkvh9T4uZ5Tt%2Fpnjjwgw96YDoyiPLp8%3D&amp;reserved=0

    "Do not insult or put down other participants"

    (... Before anyone suggests that I did the same when I called
    maintainers & people en bloc "irrepairably lazy and undisciplined" in my
    previous mail: that's a fact about humans.

    People on average perform the minimum of work they can get away with,
    for satisfying requirements and for reaching goals. It's natural. That's
    why we have to set high standards. So that covers "lazy".

    And "undisciplined" (= ignoring rules and good practices) is evidenced
    frequently, with fixed BZs left open, posted patches not referenced in
    the BZs they address, BZs ignored / left un-triaged for months and
    years, pending patches ignored for weeks, reviewed patches left unmerged
    for days or weeks, etc. I'm not throwing around accusations, just
    showing that my statement was factual, hardly an insult. OTOH "open
    doors with your face" is figurative speech, and I do consider it an insult.)

    Thanks,
    Laszlo









[-- Attachment #2: Type: text/html, Size: 5322 bytes --]

Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

[Sean]

Laszlo,

First let me be clear, there is no desire or intent in any of these 
conversations/discussions for anyone to feel so distraught to give up 
this project, let alone someone so active and involved as yourself.

The basis for my perspective goes back to the conversations we had 
numerous years ago about being more inclusive and enabling more of the 
firmware development community to contribute and be involved in this 
project.  In my opinion this project needs help. It needs more 
maintainers, contributors, reviewers, testers, and active users. It 
needs people to write documentation, answer questions, triage bugs, and 
plan release cycles.  Without removing some of today's barriers, support 
will continue to decline and relevancy of this project will decline with it.

One of the first suggestions was to evaluate the contribution and review 
process, looking for places in the current process that are confusing, 
error prone, inefficient, or hard to drive consistently.  It was also 
important to evaluate trends in other successful open source projects. 
 From this the process moved towards evaluating GitHub based pull 
requests for the contribution and review process. That gets us to this 
discussion and in my opinion a slightly larger scope in that we are not 
trying to reproduce the current process using new tools but rather 
adjust the process to address the discussed issues leveraging these tools.

Another request from the community discussions years ago was to add 
testing capabilities to remove manual work and improve quality.  There 
has been a huge effort over the last year to enable a "core" CI system, 
practical/easy to use unit testing capabilities, and most recently 
"platform" CI. These features where developed and enabled to give 
contributors clear expectations for the quality needed for successful 
contribution.  In all of these efforts, my hope has been to enable more 
people to join this project.

Anyway, for what it is worth, I hope this long winded response provides 
some background into my perspective.  I'll respond to other comments 
below.




On 5/19/2020 1:41 PM, Laszlo Ersek wrote:
> (+Leif, +Andrew)
> 
> Sean,
> 
> On 05/19/20 18:54, Sean Brogan wrote:
>> Nate/Laszlo,
>>
>> Regarding a squash merge workflow.  I agree it can be abused and we all
>> have seen terrible examples.  But a patch series that contains 500+ file
>> changes isn't really much better.  Just because it is broken into
>> multiple commits doesn't mean it is the right set of commits.
>>
>> Anyway a squash merge workflow works amazingly well with and is
>> optimized for a web based review and PR processes.  It allows a user to
>> respond to changes, fix issues, learn thru the PR process, all while
>> keeping complete track of the progression.  Then once all "status"
>> checks and reviews are complete, it is squashed into a neat commit for
>> mainline, containing only the relevant data in the message.
>>
>> So, the ask is that we don't exclude squash merge workflows.  Those
>> reviewing the PR can decide what is appropriate for the PR content
>> submitted.  Just as you would request changes to the contents (or
>> ordering) of a commit in a series, if the reviewers don't agree that the
>> PR contents should be in a single commit then obviously it shouldn't be
>> squashed to one.
>>
>> Contributions like spelling fixes, typos, minor bug fixes, documentation
>> additions/fixes, etc all are great examples of PRs that can easily
>> leverage squash merges and this workflow significantly lowers the burden
>> of the contribution and review process.  This workflow is also are much
>> easier for casual or first time contributors.
>>
>> I don't exactly know how we would enable this but I assume we could
>> leverage tags or make it clear in the PR description.  First step is to
>> get alignment that a squash merge workflow, while not appropriate for
>> all contributions, is not something to be excluded.
> 
> the scope for migrating the contribution & review workflows off the
> mailing list and to github.com was set many months ago. That scope does
> not include institutionalized changes to patch set structuring criteria.
> The "git forge" evaluations that we had spent weeks/months on also
> focused on how candidate systems would honor a patch series' structure;
> i.e., how faithful the system would remain to the contributors' and
> reviewers' shared intent, with a specific patch set.

I hope the scope is to build an effective and efficient contribution 
process that helps current contributors deliver more while opening the 
door to the rest of the firmware community.  It should require less 
effort to contribute a change to edk2 than to maintain a downstream 
fork.  Today this is not true.

> 
> Your proposal to "don't exclude squash merge workflows" is a trap. If we
> tolerate that option -- which is obviously the sloppy, and hence more
> convenient, option for some maintainers and some contributors, to the
> detriment of the git history --, then almost every core maintainer will
> use it as frequently as they can. In the long term, that will hurt many
> consumers of the core code. It will limit the ability of people not
> regularly dealing with a particular core module to file a fine-grained
> bug report for that module, maybe even propose a fix. From the
> regression analyst's side, if the bug report starts with "I have a
> bisection log", that's already a good day. And your proposal would
> destroy that option, because maintainers and people in general are
> irrepairably lazy and undisciplined. We cannot post a community member
> shoulder-by-shoulder with every core package reviewer/maintainer to
> prevent the latter from approving a squash-on-merge, out of pure
> laziness. I'm 100% sure the "option" to squash-on-merge would
> *immediately* be abused for a lot more than just "typo fixes". There
> isn't enough manpower to watch the watchers, so "no squash-on-merge"
> needs to be a general rule.


I have trouble with this line of thinking. The maintainers are and 
should be considered the representatives of this code base.   They have 
a vested interest to enable this repository to work for them.  If they 
really are viewed as "sloppy" or "lazy" then we are destined to fail 
anyway.

Nothing in my statement of "don't exclude squash merge workflow" 
requested that we allow a PR to be squashed into a single commit that 
you believe should be a patch series. I do think those rules will need 
to be defined but that is needed today anyway.


> 
> I'm very sad that you're trying to wiggle such a crucial and intrusive
> workflow change into the scope of this transition. 

Not "trying to wiggle" anything, just focused on providing feedback and 
hoping to help develop an efficient and effective process using the 
tools available. See intro paragraph.

> Every time
> squash-on-merge has come up over the years (regardless of this
> transition), we've labeled it as one thing never to do, because it
> destroys information (and/or even encourages not *creating* that
> historical information in the first place, which is of course important
> in reality).
> 

You may have labelled it that way but given the wide spread use of this 
practice and my own great experiences enabling a broad team with mixed 
backgrounds using this practice, I personally haven't.  This community 
is a quiet one and I believe instead of speaking up, members just choose 
not to get involved.


> Well, anyway, here's my feedback: if squash-on-merge is permitted in
> edk2 or in basetools (or in any other external repository that's a hard
> requirement for building edk2), that's a deal breaker for me, and I'll
> hand in my resignation as a steward.
> 
> Maybe you'd consider that a win, I don't know -- but I couldn't remain a
> steward with a straight face after failing to protect what I consider
> one of the core values of open source / distributed development.
> 
> Thanks,
> Laszlo
> 

Thanks
Sean

Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

[Laszlo Ersek]

On 05/19/20 23:02, Desimone, Nathaniel L wrote:

> Of course, there may be other patch series that would be logical to
> squash, especially if the author has not been careful to maintain
> bisectability. For example, I think of some patch series went a
> little overboard and could have been done in maybe 1-2 patches
> instead of 8-10. I would be happy to compromise with you and say that
> squashes can be done in circumstances where both the maintainer and
> the author agree to it.

Important distinction:

(a) "squashing patches" is a 100% valid operation that some situations
fully justifiedly call for. Maintainers may ask for it, and contributors
may use it with or without being asked, if the situation calls for it.

(b) "squashing patches *on merge*" is intolerable.

The difference is whether there is a final human review for the
*post-squash* state before the merge occurs.

The valid case is when the contributor squashes some patches, resubmits
the review/pull request, the reviewer approves the *complete* work
(after performing another review, which may of course be incremental in
nature), and then the series is merged exactly as it was submitted.

The invalid case (squash on merge) is when the reviewer checks /
approves the series when it still contains incremental fixes as
broken-out patches, then squashes some patches (in the worst case: all
patches into one), and then merges the result. In this (invalid) case,
the complete work, in its final state (in the way it's going to land in
the git history) has not been reviewed by either submitter or reviewer,
incrementally or otherwise. This is why squash on merge is intolerable:
it places a sequence of commits into the git history that has never been
reviewed *verbatim* by either submitter or reviewer. It's a "blind
merge", to make up another term for illustration

Squashing is a 100% valid tool, I use it all the time. Squash-on-merge
is a catastrophic process failure.

Thanks
Laszlo

Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

[Sean]

When this is done in a PR with branch protections this works out 
differently and in my view your concerns are mitigated.

1. There isn't a partial squash operation.  All reviewers know that the 
final output of the PR is going to 1 commit.  Thus there is no confusion 
of what or how it is being committed to the target branch.

2. With GitHub branch protections requiring the PR only being merged if 
it is up-to-date with the target branch.  This means you have to push 
the button in github to merge in target and if any conflicts occur the 
PR is flagged and can't be completed without user involvement.  This 
would also give reviewers an opportunity to review the merge commit if 
necessary.

3. With GitHub status checks and branch policies correctly configured 
the builds are re-run every time the target branch changes. This means 
that if you have confidence in your PR gates catching most practical 
merge errors (at least the ones the submitter would catch) you have 
avoided this issue.  This is why the PR builds check every thing in the 
tree rather than just the incoming patch.

Again, this ask was not to create a lazy process or lower the quality of 
the code tree.  If there are legitimate gaps that a squash merge 
workflows creates, I am interested in finding solutions.  For example, 
the DCO requirement would need to be addressed.  But we can only start 
those conversations if we can get aligned on the idea.

Thanks
Sean




On 5/20/2020 10:05 AM, Laszlo Ersek wrote:
> On 05/19/20 23:02, Desimone, Nathaniel L wrote:
> 
>> Of course, there may be other patch series that would be logical to
>> squash, especially if the author has not been careful to maintain
>> bisectability. For example, I think of some patch series went a
>> little overboard and could have been done in maybe 1-2 patches
>> instead of 8-10. I would be happy to compromise with you and say that
>> squashes can be done in circumstances where both the maintainer and
>> the author agree to it.
> 
> Important distinction:
> 
> (a) "squashing patches" is a 100% valid operation that some situations
> fully justifiedly call for. Maintainers may ask for it, and contributors
> may use it with or without being asked, if the situation calls for it.
> 
> (b) "squashing patches *on merge*" is intolerable.
> 
> The difference is whether there is a final human review for the
> *post-squash* state before the merge occurs.
> 
> The valid case is when the contributor squashes some patches, resubmits
> the review/pull request, the reviewer approves the *complete* work
> (after performing another review, which may of course be incremental in
> nature), and then the series is merged exactly as it was submitted.
> 
> The invalid case (squash on merge) is when the reviewer checks /
> approves the series when it still contains incremental fixes as
> broken-out patches, then squashes some patches (in the worst case: all
> patches into one), and then merges the result. In this (invalid) case,
> the complete work, in its final state (in the way it's going to land in
> the git history) has not been reviewed by either submitter or reviewer,
> incrementally or otherwise. This is why squash on merge is intolerable:
> it places a sequence of commits into the git history that has never been
> reviewed *verbatim* by either submitter or reviewer. It's a "blind
> merge", to make up another term for illustration
> 
> Squashing is a 100% valid tool, I use it all the time. Squash-on-merge
> is a catastrophic process failure.
> 
> Thanks
> Laszlo
> 
> 
> 
> 

Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

[Laszlo Ersek]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=WINDOWS-1252, Size: 1021 bytes --]

off-topic, but for the record:

On 05/19/20 22:10, Bret Barkelew via groups.io wrote:

> In my history with TianoCore, I have learned to not be so quick to
> say “this is fucking stupid”. Every time I’ve done that, I’ve later
> discovered the reasons behind it, and even come to the conclusion
> that the designers were quite clever.

while I understand and appreciate the positive message here, that
particular present participle stands out to me like a sore thumb.

I couldn't resist, and I searched my edk2-devel archives for it (for the
four letter stem, that is), which go back to ~April 2012.

I'm reporting that in all these years, this has indeed been the first
use of the word.

(Not counting the base64 encodings of some binary files that were posted
to the list, in patches -- but those encodings hardly contain "words".)

Can we stay civil, please?

(And no, I'm not a prude; in fact I've shown such restraint in my own
word choices on this list that I can only congratulate myself.)

Thanks,
Laszlo

Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

[Laszlo Ersek]

On 05/20/20 00:25, Sean wrote:
> On 5/19/2020 1:41 PM, Laszlo Ersek wrote:

>> Your proposal to "don't exclude squash merge workflows" is a trap. If
>> we tolerate that option -- which is obviously the sloppy, and hence
>> more convenient, option for some maintainers and some contributors,
>> to the detriment of the git history --, then almost every core
>> maintainer will use it as frequently as they can. In the long term,
>> that will hurt many consumers of the core code. It will limit the
>> ability of people not regularly dealing with a particular core module
>> to file a fine-grained bug report for that module, maybe even propose
>> a fix. From the regression analyst's side, if the bug report starts
>> with "I have a bisection log", that's already a good day. And your
>> proposal would destroy that option, because maintainers and people in
>> general are irrepairably lazy and undisciplined. We cannot post a
>> community member shoulder-by-shoulder with every core package
>> reviewer/maintainer to prevent the latter from approving a
>> squash-on-merge, out of pure laziness. I'm 100% sure the "option" to
>> squash-on-merge would *immediately* be abused for a lot more than
>> just "typo fixes". There isn't enough manpower to watch the watchers,
>> so "no squash-on-merge" needs to be a general rule.
>
>
> I have trouble with this line of thinking. The maintainers are and
> should be considered the representatives of this code base.   They
> have a vested interest to enable this repository to work for them.  If
> they really are viewed as "sloppy" or "lazy" then we are destined to
> fail anyway.

You put it very well. "They have a vested interest to enable this
repository to work for them." Key part being "*for them*".

Core maintainers are responsible for making this repository work for a
lot larger camp than just themselves. Even if squash-on-merge satisfied
the requirements that core maintainers presented, squash-on-merge would
still hurt the larger community that depends on those packages.

The core-consumer community may not necessarily participate in the
day-to-day maintenance of the core packages, but they do report bugs and
even contributes bugfixes / occasional features, when their particular
use cases require those actions.

And squash-on-merge hurts those activities, down the road, because the
git history is instrumental to analyzing and learning the code base.

For example, the question "why do we call this function here?"
immediately leads to running "git blame" (possibly a series of git-blame
commands, to navigate past code movements and such). In the end
git-blame leads to a particular commit, and that commit is supposed to
answer the question. If the commit is huge (e.g. a squash of an entire
feature), then the question is not answered, and git-blame has been
rendered useless.


> Nothing in my statement of "don't exclude squash merge workflow"
> requested that we allow a PR to be squashed into a single commit that
> you believe should be a patch series.

If the button is there, maintainers will click it even in cases when
they shouldn't, and I won't be able to catch them. The result will not
necessarily hurt the maintainer (not at once, anyway), but it will harm
others that investigate the git history afterwards -- possibly years
later.

I can't watch all CoreFoobarPkg pull requests on github to prevent this.
On the other hand, I can, and do, monitor the edk2-devel list for
seriously mis-organized patch sets, especially for core packages where
I've formed an "I had better watch out for this core package"
impression.

I have made requests under core patch sets where I was mostly unfamiliar
with the technical subject *for the time being*, asking just for
improvements to the granularity of the series. Knowing the improved
granularity might very well help me *in the future*.


The mailing list equivalent of "squash-on-merge" would be the following:

- contributor posts v1 with patches 1/5 .. 5/5 (for example),

- reviewer requests updates A, B, and C,

- contributor posts (in response to the v1 blurb, i.e. 0/5) further
  patches 6/8, 7/8, 8/8

- reviewer checks the new patches and approves them, functionally,

- maintainer says "OK let me merge this",

- maintainer applies the patches (all 8 of them) from the list, on a
  local branch,

- maintainer runs a git rebase squashing the whole thing into a single
  patch,

- maintainer does *not* review the result,

- maintainer opens a PR with the resultant single patch,

- CI passes,

- the patch is merged.


With the list-based process, the disaster in the last step is mitigated
in at least three spots:

- All subscribers have a reasonably good chance to notice and intervene
  when the incremental fixups 6/8, 7/8, 8/8 are posted as followups to
  the v1 blurb, clearly with an intent to squash.

- Because the maintainer has to do *extra work* for the squashing, the
  natural laziness of the maintainer works *against* the disaster. Thus
  he or she will likely not perform the local rebase & squash. Instead
  they will ask the contributor to perform a *fine-grained* squash (i.e.
  squash each fixup into the one original patch where the fixup
  belongs), and to submit a v2 series.

- If someone interested in the git history catches (after the fact) that
  the maintainer merged a significantly different patch set from what
  had been posted and reviewed, they can raise a stern complaint on the
  list, and next time the maintainer will now better.

(This is not a theoretical option; I low-key follow both the list
traffic and the new commits in the git history (whenever I pull). In the
past I had reported several patch application violations (mismanaged
feedback tags, intrusive updates post-review, etc). Nowadays it's gotten
quite OK, thankfully, and I'm terrified of losing those improvements.)


If we just plaster a huge squash-on-merge button or checkbox over the
web UI, it *will* be abused (maintainer laziness will work *towards* the
disaster), with only a microscopic chance for me to prevent the abuse.

It's not that "I believe" that this or that *particular* series should
not be squashed. "Not squashing" is not the exception but the rule. The
*default* approach is that the submitter incorporates incremental fixes
into the series at the right stages, they maintain a proper series
structure over the iterations, and they propose revised versions of the
series in full. Squashing is the exception; for example one reason is,
"if you separate these changes from each other, then the tree will not
build in the middle; they belong together, please squash them, and
resubmit for review".


> I do think those rules will need to be defined but that is needed
> today anyway.

Rules are only as good as their enforcement is.

The question is not how nice it is to use squash-on-merge in the
minuscule set of situations when it might be justified; the question is
how difficult it would be to  prevent the inevitable abuses.

The list lets me advocate for proper git history hygiene reasonably
efficiently (although I still miss a bunch of warts due to lack of
capacity). With the squash-on-merge button or checkbox, the flood gates
would fly open. I won't stand for that (not as a steward anyway).

I think our world views differ fundamentally. I value the git history
*way* above my own comfort, and everyone else's (accounting for both
contributor and day-to-day maintainer roles). I guess you prefer the
reciprocal of that ratio.

Thanks,
Laszlo

Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

[Sean]

Laszlo,

I appreciate the back and forth.  I find email a challenge for this type 
of discussion because it leaves so much to individual interpretation and 
bias.  Anyway Thank you for having the discussion.  I hope others with 
opinions feel empowered to chime in and help this RFC go in the 
direction the community and stewards desire.

I am still in full support of the RFC and am ok to table my concerns 
that changing the tools without adapting the process will lead to a less 
than optimal workflow.  Anyway, I look forward to seeing if the "pull 
request based code review process" can help improve the communities 
collaboration and efficiency.

I have a few additional responses below that will clarify my thoughts 
but hopefully not invoke responses. :)

On 5/21/2020 6:30 AM, Laszlo Ersek wrote:
> On 05/20/20 00:25, Sean wrote:
>> On 5/19/2020 1:41 PM, Laszlo Ersek wrote:
> 
>>> Your proposal to "don't exclude squash merge workflows" is a trap. If
>>> we tolerate that option -- which is obviously the sloppy, and hence
>>> more convenient, option for some maintainers and some contributors,
>>> to the detriment of the git history --, then almost every core
>>> maintainer will use it as frequently as they can. In the long term,
>>> that will hurt many consumers of the core code. It will limit the
>>> ability of people not regularly dealing with a particular core module
>>> to file a fine-grained bug report for that module, maybe even propose
>>> a fix. From the regression analyst's side, if the bug report starts
>>> with "I have a bisection log", that's already a good day. And your
>>> proposal would destroy that option, because maintainers and people in
>>> general are irrepairably lazy and undisciplined. We cannot post a
>>> community member shoulder-by-shoulder with every core package
>>> reviewer/maintainer to prevent the latter from approving a
>>> squash-on-merge, out of pure laziness. I'm 100% sure the "option" to
>>> squash-on-merge would *immediately* be abused for a lot more than
>>> just "typo fixes". There isn't enough manpower to watch the watchers,
>>> so "no squash-on-merge" needs to be a general rule.
>>
>>
>> I have trouble with this line of thinking. The maintainers are and
>> should be considered the representatives of this code base.   They
>> have a vested interest to enable this repository to work for them.  If
>> they really are viewed as "sloppy" or "lazy" then we are destined to
>> fail anyway.
> 
> You put it very well. "They have a vested interest to enable this
> repository to work for them." Key part being "*for them*".
> 
> Core maintainers are responsible for making this repository work for a
> lot larger camp than just themselves. Even if squash-on-merge satisfied
> the requirements that core maintainers presented, squash-on-merge would
> still hurt the larger community that depends on those packages.
> 
> The core-consumer community may not necessarily participate in the
> day-to-day maintenance of the core packages, but they do report bugs and
> even contributes bugfixes / occasional features, when their particular
> use cases require those actions.
> 
> And squash-on-merge hurts those activities, down the road, because the
> git history is instrumental to analyzing and learning the code base.
> 
> For example, the question "why do we call this function here?"
> immediately leads to running "git blame" (possibly a series of git-blame
> commands, to navigate past code movements and such). In the end
> git-blame leads to a particular commit, and that commit is supposed to
> answer the question. If the commit is huge (e.g. a squash of an entire
> feature), then the question is not answered, and git-blame has been
> rendered useless.
> 
> 
>> Nothing in my statement of "don't exclude squash merge workflow"
>> requested that we allow a PR to be squashed into a single commit that
>> you believe should be a patch series.
> 
> If the button is there, maintainers will click it even in cases when
> they shouldn't, and I won't be able to catch them. The result will not
> necessarily hurt the maintainer (not at once, anyway), but it will harm
> others that investigate the git history afterwards -- possibly years
> later.
> 
> I can't watch all CoreFoobarPkg pull requests on github to prevent this.
> On the other hand, I can, and do, monitor the edk2-devel list for
> seriously mis-organized patch sets, especially for core packages where
> I've formed an "I had better watch out for this core package"
> impression.
> 
> I have made requests under core patch sets where I was mostly unfamiliar
> with the technical subject *for the time being*, asking just for
> improvements to the granularity of the series. Knowing the improved
> granularity might very well help me *in the future*.
> 
> 
> The mailing list equivalent of "squash-on-merge" would be the following:
> 
> - contributor posts v1 with patches 1/5 .. 5/5 (for example),
> 
> - reviewer requests updates A, B, and C,
> 
> - contributor posts (in response to the v1 blurb, i.e. 0/5) further
>    patches 6/8, 7/8, 8/8
> 
> - reviewer checks the new patches and approves them, functionally,
> 
> - maintainer says "OK let me merge this",
> 
> - maintainer applies the patches (all 8 of them) from the list, on a
>    local branch,
> 
> - maintainer runs a git rebase squashing the whole thing into a single
>    patch,
> 
> - maintainer does *not* review the result,
> 
> - maintainer opens a PR with the resultant single patch,
> 
> - CI passes,
> 
> - the patch is merged.
> 
> 

The above example should not be allowed in any process.
If a contribution was submitted as a patch series with 5 patches 
intentionally, then it would not be a candidate for a squash merge.  The 
squash merge workflow is only acceptable when it is agreed that the end 
result should be 1 patch.


> With the list-based process, the disaster in the last step is mitigated
> in at least three spots:
> 
> - All subscribers have a reasonably good chance to notice and intervene
>    when the incremental fixups 6/8, 7/8, 8/8 are posted as followups to
>    the v1 blurb, clearly with an intent to squash.
> 
> - Because the maintainer has to do *extra work* for the squashing, the
>    natural laziness of the maintainer works *against* the disaster. Thus
>    he or she will likely not perform the local rebase & squash. Instead
>    they will ask the contributor to perform a *fine-grained* squash (i.e.
>    squash each fixup into the one original patch where the fixup
>    belongs), and to submit a v2 series.
> 
> - If someone interested in the git history catches (after the fact) that
>    the maintainer merged a significantly different patch set from what
>    had been posted and reviewed, they can raise a stern complaint on the
>    list, and next time the maintainer will now better.
> 
> (This is not a theoretical option; I low-key follow both the list
> traffic and the new commits in the git history (whenever I pull). In the
> past I had reported several patch application violations (mismanaged
> feedback tags, intrusive updates post-review, etc). Nowadays it's gotten
> quite OK, thankfully, and I'm terrified of losing those improvements.)
> 
> 
> If we just plaster a huge squash-on-merge button or checkbox over the
> web UI, it *will* be abused (maintainer laziness will work *towards* the
> disaster), with only a microscopic chance for me to prevent the abuse.
> 
> It's not that "I believe" that this or that *particular* series should
> not be squashed. "Not squashing" is not the exception but the rule. The
> *default* approach is that the submitter incorporates incremental fixes
> into the series at the right stages, they maintain a proper series
> structure over the iterations, and they propose revised versions of the
> series in full. Squashing is the exception; for example one reason is,
> "if you separate these changes from each other, then the tree will not
> build in the middle; they belong together, please squash them, and
> resubmit for review".
> 
> 
>> I do think those rules will need to be defined but that is needed
>> today anyway.
> 
> Rules are only as good as their enforcement is.
> 
> The question is not how nice it is to use squash-on-merge in the
> minuscule set of situations when it might be justified; the question is
> how difficult it would be to  prevent the inevitable abuses.
> 

At time of writing i don't know any way to enforce this if the 
maintainers can not be relied upon.  Given my strong agreement with 
"Rules are only as good as their enforcement is." I don't see a 
practical way to resolve this and you seem content with the current 
solution.  Thanks for your diligence here.

> The list lets me advocate for proper git history hygiene reasonably
> efficiently (although I still miss a bunch of warts due to lack of
> capacity). With the squash-on-merge button or checkbox, the flood gates
> would fly open. I won't stand for that (not as a steward anyway).
> 
> I think our world views differ fundamentally. I value the git history
> *way* above my own comfort, and everyone else's (accounting for both
> contributor and day-to-day maintainer roles). I guess you prefer the
> reciprocal of that ratio.
> 
> Thanks,
> Laszlo
> 


Thanks
Sean

Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

[Andrew Fish]

> On May 20, 2020, at 10:21 AM, Sean <spbrogan@outlook.com> wrote:
> 
> When this is done in a PR with branch protections this works out differently and in my view your concerns are mitigated.
> 
> 1. There isn't a partial squash operation.  All reviewers know that the final output of the PR is going to 1 commit.  Thus there is no confusion of what or how it is being committed to the target branch.
> 

I use Stash/Bitbucket but even the UI is biased towards this. There is an Overview, Diff, and Commits tab. The default diff is the entire PR, you can go to commits tab and see a list of the commits/patch set and see only the diffs for those. I'm not sure how github does it. 

In our world we don't require the squash. We also have a set of command line tools that help automate common operations. 

Thanks,

Andrew Fish

> 2. With GitHub branch protections requiring the PR only being merged if it is up-to-date with the target branch.  This means you have to push the button in github to merge in target and if any conflicts occur the PR is flagged and can't be completed without user involvement.  This would also give reviewers an opportunity to review the merge commit if necessary.
> 
> 3. With GitHub status checks and branch policies correctly configured the builds are re-run every time the target branch changes. This means that if you have confidence in your PR gates catching most practical merge errors (at least the ones the submitter would catch) you have avoided this issue.  This is why the PR builds check every thing in the tree rather than just the incoming patch.
> 
> Again, this ask was not to create a lazy process or lower the quality of the code tree.  If there are legitimate gaps that a squash merge workflows creates, I am interested in finding solutions.  For example, the DCO requirement would need to be addressed.  But we can only start those conversations if we can get aligned on the idea.
> 
> Thanks
> Sean
> 
> 
> 
> 
> On 5/20/2020 10:05 AM, Laszlo Ersek wrote:
>> On 05/19/20 23:02, Desimone, Nathaniel L wrote:
>>> Of course, there may be other patch series that would be logical to
>>> squash, especially if the author has not been careful to maintain
>>> bisectability. For example, I think of some patch series went a
>>> little overboard and could have been done in maybe 1-2 patches
>>> instead of 8-10. I would be happy to compromise with you and say that
>>> squashes can be done in circumstances where both the maintainer and
>>> the author agree to it.
>> Important distinction:
>> (a) "squashing patches" is a 100% valid operation that some situations
>> fully justifiedly call for. Maintainers may ask for it, and contributors
>> may use it with or without being asked, if the situation calls for it.
>> (b) "squashing patches *on merge*" is intolerable.
>> The difference is whether there is a final human review for the
>> *post-squash* state before the merge occurs.
>> The valid case is when the contributor squashes some patches, resubmits
>> the review/pull request, the reviewer approves the *complete* work
>> (after performing another review, which may of course be incremental in
>> nature), and then the series is merged exactly as it was submitted.
>> The invalid case (squash on merge) is when the reviewer checks /
>> approves the series when it still contains incremental fixes as
>> broken-out patches, then squashes some patches (in the worst case: all
>> patches into one), and then merges the result. In this (invalid) case,
>> the complete work, in its final state (in the way it's going to land in
>> the git history) has not been reviewed by either submitter or reviewer,
>> incrementally or otherwise. This is why squash on merge is intolerable:
>> it places a sequence of commits into the git history that has never been
>> reviewed *verbatim* by either submitter or reviewer. It's a "blind
>> merge", to make up another term for illustration
>> Squashing is a 100% valid tool, I use it all the time. Squash-on-merge
>> is a catastrophic process failure.
>> Thanks
>> Laszlo
> 
> 
> 

Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

[Andrew Fish]

> On May 21, 2020, at 6:30 AM, Laszlo Ersek <lersek@redhat.com> wrote:
> 
> On 05/20/20 00:25, Sean wrote:
>> On 5/19/2020 1:41 PM, Laszlo Ersek wrote:
> 
>>> Your proposal to "don't exclude squash merge workflows" is a trap. If
>>> we tolerate that option -- which is obviously the sloppy, and hence
>>> more convenient, option for some maintainers and some contributors,
>>> to the detriment of the git history --, then almost every core
>>> maintainer will use it as frequently as they can. In the long term,
>>> that will hurt many consumers of the core code. It will limit the
>>> ability of people not regularly dealing with a particular core module
>>> to file a fine-grained bug report for that module, maybe even propose
>>> a fix. From the regression analyst's side, if the bug report starts
>>> with "I have a bisection log", that's already a good day. And your
>>> proposal would destroy that option, because maintainers and people in
>>> general are irrepairably lazy and undisciplined. We cannot post a
>>> community member shoulder-by-shoulder with every core package
>>> reviewer/maintainer to prevent the latter from approving a
>>> squash-on-merge, out of pure laziness. I'm 100% sure the "option" to
>>> squash-on-merge would *immediately* be abused for a lot more than
>>> just "typo fixes". There isn't enough manpower to watch the watchers,
>>> so "no squash-on-merge" needs to be a general rule.
>> 
>> 
>> I have trouble with this line of thinking. The maintainers are and
>> should be considered the representatives of this code base.   They
>> have a vested interest to enable this repository to work for them.  If
>> they really are viewed as "sloppy" or "lazy" then we are destined to
>> fail anyway.
> 
> You put it very well. "They have a vested interest to enable this
> repository to work for them." Key part being "*for them*".
> 
> Core maintainers are responsible for making this repository work for a
> lot larger camp than just themselves. Even if squash-on-merge satisfied
> the requirements that core maintainers presented, squash-on-merge would
> still hurt the larger community that depends on those packages.
> 
> The core-consumer community may not necessarily participate in the
> day-to-day maintenance of the core packages, but they do report bugs and
> even contributes bugfixes / occasional features, when their particular
> use cases require those actions.
> 
> And squash-on-merge hurts those activities, down the road, because the
> git history is instrumental to analyzing and learning the code base.
> 
> For example, the question "why do we call this function here?"
> immediately leads to running "git blame" (possibly a series of git-blame
> commands, to navigate past code movements and such). In the end
> git-blame leads to a particular commit, and that commit is supposed to
> answer the question. If the commit is huge (e.g. a squash of an entire
> feature), then the question is not answered, and git-blame has been
> rendered useless.
> 
> 
>> Nothing in my statement of "don't exclude squash merge workflow"
>> requested that we allow a PR to be squashed into a single commit that
>> you believe should be a patch series.
> 
> If the button is there, maintainers will click it even in cases when
> they shouldn't, and I won't be able to catch them. The result will not
> necessarily hurt the maintainer (not at once, anyway), but it will harm
> others that investigate the git history afterwards -- possibly years
> later.
> 
> I can't watch all CoreFoobarPkg pull requests on github to prevent this.
> On the other hand, I can, and do, monitor the edk2-devel list for
> seriously mis-organized patch sets, especially for core packages where
> I've formed an "I had better watch out for this core package"
> impression.
> 
> I have made requests under core patch sets where I was mostly unfamiliar
> with the technical subject *for the time being*, asking just for
> improvements to the granularity of the series. Knowing the improved
> granularity might very well help me *in the future*.
> 
> 
> The mailing list equivalent of "squash-on-merge" would be the following:
> 
> - contributor posts v1 with patches 1/5 .. 5/5 (for example),
> 
> - reviewer requests updates A, B, and C,
> 
> - contributor posts (in response to the v1 blurb, i.e. 0/5) further
>  patches 6/8, 7/8, 8/8
> 
> - reviewer checks the new patches and approves them, functionally,
> 
> - maintainer says "OK let me merge this",
> 
> - maintainer applies the patches (all 8 of them) from the list, on a
>  local branch,
> 
> - maintainer runs a git rebase squashing the whole thing into a single
>  patch,
> 
> - maintainer does *not* review the result,
> 
> - maintainer opens a PR with the resultant single patch,
> 
> - CI passes,
> 
> - the patch is merged.
> 
> 
> With the list-based process, the disaster in the last step is mitigated
> in at least three spots:
> 
> - All subscribers have a reasonably good chance to notice and intervene
>  when the incremental fixups 6/8, 7/8, 8/8 are posted as followups to
>  the v1 blurb, clearly with an intent to squash.
> 
> - Because the maintainer has to do *extra work* for the squashing, the
>  natural laziness of the maintainer works *against* the disaster. Thus
>  he or she will likely not perform the local rebase & squash. Instead
>  they will ask the contributor to perform a *fine-grained* squash (i.e.
>  squash each fixup into the one original patch where the fixup
>  belongs), and to submit a v2 series.
> 
> - If someone interested in the git history catches (after the fact) that
>  the maintainer merged a significantly different patch set from what
>  had been posted and reviewed, they can raise a stern complaint on the
>  list, and next time the maintainer will now better.
> 
> (This is not a theoretical option; I low-key follow both the list
> traffic and the new commits in the git history (whenever I pull). In the
> past I had reported several patch application violations (mismanaged
> feedback tags, intrusive updates post-review, etc). Nowadays it's gotten
> quite OK, thankfully, and I'm terrified of losing those improvements.)
> 
> 
> If we just plaster a huge squash-on-merge button or checkbox over the
> web UI, it *will* be abused (maintainer laziness will work *towards* the
> disaster), with only a microscopic chance for me to prevent the abuse.
> 
> It's not that "I believe" that this or that *particular* series should
> not be squashed. "Not squashing" is not the exception but the rule. The
> *default* approach is that the submitter incorporates incremental fixes
> into the series at the right stages, they maintain a proper series
> structure over the iterations, and they propose revised versions of the
> series in full. Squashing is the exception; for example one reason is,
> "if you separate these changes from each other, then the tree will not
> build in the middle; they belong together, please squash them, and
> resubmit for review".
> 
> 
>> I do think those rules will need to be defined but that is needed
>> today anyway.
> 
> Rules are only as good as their enforcement is.
> 

In my work world we require code review by a manager and that is the de facto enforcement mechanism. Basically there is always an owner to make sure the process was followed :)

Also in our world the squash is a developer choice. But we do have tools that insert the Bugzilla number in all the commits of the series, assist with the squash, etc. 

> The question is not how nice it is to use squash-on-merge in the
> minuscule set of situations when it might be justified; the question is
> how difficult it would be to  prevent the inevitable abuses.
> 
> The list lets me advocate for proper git history hygiene reasonably
> efficiently (although I still miss a bunch of warts due to lack of
> capacity). With the squash-on-merge button or checkbox, the flood gates
> would fly open. I won't stand for that (not as a steward anyway).
> 
> I think our world views differ fundamentally. I value the git history
> *way* above my own comfort, and everyone else's (accounting for both
> contributor and day-to-day maintainer roles). I guess you prefer the
> reciprocal of that ratio.
> 

I'd also point out that the processes you chose kind of defines your quanta of work. It is likely you would be willing to tackle a really big change as a large patch set, that you would likely break up into multiple PRs in a squash on commit world. In a squash on commit world you also might break a Bugzilla (BZ) up into dependent BZs, a tree of BZs. That might sound crazy, but when you work on a bigger project and there are BZs for EFI, T2, macOS, the Installer, and the RecoveryOS for a customer visible feature this tree of BZ might be familiar and make sense  to you. 

But I think the real argument for consistency is we have a rich git history that has value. We have made resource tradeoffs to have that rich git history so to me it makes the most sense, for these project, to try to preserve our past investment in git history. 

Thanks,

Andrew Fish

> Thanks,
> Laszlo
> 

Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

[Tomas Pilar (tpilar)]

I actually agree with you, when we migrated from reviewboard to github pull requests, I was sorely disappointed with the PR functionality and ergonomics.

Tomas Pilar

-----Original Message-----
From: rfc@edk2.groups.io <rfc@edk2.groups.io> On Behalf Of Rebecca Cran via groups.io
Sent: 14 May 2020 22:47
To: rfc@edk2.groups.io; bret.barkelew@microsoft.com; Kinney, Michael D <michael.d.kinney@intel.com>; devel@edk2.groups.io; lersek@redhat.com
Subject: Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

On 5/14/20 3:26 PM, Bret Barkelew via groups.io wrote:

> I feel like this process is a good compromise. It�s not perfect (frankly, I�m a fan of enforced squash merges, which can maintain bisectability if managed well), but it allows for rapid iteration, ease of contribution, and approaches the workflow that many who have never used email to maintain a project would be familiar with.
>
> It�s code management for the Instagram generation, and I for one welcome our new insect overlords.

Or at least, that's what Microsoft is betting on! :D

Personally, I remain unconvinced about the usability of Github Pull Requests for a project the size of EDK2, but I hope to be proven wrong.


--
Rebecca Cran





IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.