Re: [edk2-devel] [Patch 1/1] BaseTools/Scripts/PatchCheck: Error if commit modifies multiple packages

["Michael D Kinney" <michael.d.kinney@intel.com>]

Hi Ard,

Please review the analysis and proposal in the BZ and provide 
alternate proposals for the rules.

Thanks,

Mike

> -----Original Message-----
> From: Ard Biesheuvel <ardb@kernel.org>
> Sent: Tuesday, February 13, 2024 11:12 PM
> To: devel@edk2.groups.io; Kinney, Michael D
> <michael.d.kinney@intel.com>
> Cc: Rebecca Cran <rebecca@bsdio.com>; Liming Gao
> <gaoliming@byosoft.com.cn>; Feng, Bob C <bob.c.feng@intel.com>; Chen,
> Christine <yuwei.chen@intel.com>; Michael Kubacki
> <mikuback@linux.microsoft.com>
> Subject: Re: [edk2-devel] [Patch 1/1] BaseTools/Scripts/PatchCheck:
> Error if commit modifies multiple packages
> 
> On Wed, 14 Feb 2024 at 02:18, Michael D Kinney
> <michael.d.kinney@intel.com> wrote:
> >
> > REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4679
> >
> > Update PatchCheck.py to evaluate all the files modified in
> > each commit and generate an error if:
> > * A single commit modifies files in multiple packages
> > * A single commit modifies files in multiple non-package dirs
> > * A single commit modifies files in both a package and a
> >   non-package dir.
> >
> > Modifications to files in the root of the repository are not
> > evaluated.
> >
> > The set of packages are found by search for DEC files in the
> > repository. The list of DEC files in the repository is collected
> > with the following git command:
> >
> >   git ls-files *.dec
> >
> > The set of files added/modified by each commit is found using
> > the following git command:
> >
> >   git diff-tree --no-commit-id --name-only --diff-filter=AM -r
> <commit>
> >
> > The set of files deleted by each commit is found using the
> > following git command:
> >
> >   git diff-tree --no-commit-id --name-only --diff-filter=D -r
> <commit>
> >
> > Cc: Rebecca Cran <rebecca@bsdio.com>
> > Cc: Liming Gao <gaoliming@byosoft.com.cn>
> > Cc: Bob Feng <bob.c.feng@intel.com>
> > Cc: Yuwei Chen <yuwei.chen@intel.com>
> > Cc: Michael Kubacki <mikuback@linux.microsoft.com>
> > Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
> 
> Will this trigger pre-merge CI failures if the patch touches more than
> one package?
> 
> If so, I will need an opt-out/override for this.
> 
> 
> > ---
> >  BaseTools/Scripts/PatchCheck.py | 49
> +++++++++++++++++++++++++++++++++
> >  1 file changed, 49 insertions(+)
> >
> > diff --git a/BaseTools/Scripts/PatchCheck.py
> b/BaseTools/Scripts/PatchCheck.py
> > index 1675dcbd7321..988f152e38d7 100755
> > --- a/BaseTools/Scripts/PatchCheck.py
> > +++ b/BaseTools/Scripts/PatchCheck.py
> > @@ -665,6 +665,7 @@ class CheckGitCommits:
> >      """
> >
> >      def __init__(self, rev_spec, max_count):
> > +        dec_files = self.read_dec_files_from_git()
> >          commits = self.read_commit_list_from_git(rev_spec,
> max_count)
> >          if len(commits) == 1 and Verbose.level > Verbose.ONELINE:
> >              commits = [ rev_spec ]
> > @@ -681,9 +682,57 @@ class CheckGitCommits:
> >              self.ok &= EmailAddressCheck(email, 'Committer').ok
> >              patch = self.read_patch_from_git(commit)
> >              self.ok &= CheckOnePatch(commit, patch).ok
> > +            self.ok &= self.check_parent_packages (dec_files,
> commit)
> > +
> >          if not commits:
> >              print("Couldn't find commit matching:
> '{}'".format(rev_spec))
> >
> > +    def get_parent_packages(self, dec_files, commit, filter):
> > +        filelist = self.read_files_modified_from_git (commit,
> filter)
> > +        parents = set()
> > +        for file in filelist:
> > +            dec_found = False
> > +            for dec_file in dec_files:
> > +                if os.path.commonpath([dec_file, file]):
> > +                    dec_found = True
> > +                    parents.add(dec_file)
> > +            if not dec_found and os.path.dirname (file):
> > +                # No DEC file found and file is in a subdir
> > +                # Covers BaseTools, .github, .azurepipelines,
> .pytool
> > +                parents.add(file.split('/')[0])
> > +        return list(parents)
> > +
> > +    def check_parent_packages(self, dec_files, commit):
> > +        modified = self.get_parent_packages (dec_files, commit,
> 'AM')
> > +        if len (modified) > 1:
> > +            print("The commit adds/modifies files in multiple
> packages:\n *",
> > +                  '\n * '.join(modified))
> > +            self.ok = False
> > +        deleted = self.get_parent_packages (dec_files, commit, 'D')
> > +        if len (deleted) > 1:
> > +            print("The commit deletes files from multiple
> packages:\n *",
> > +                  '\n * '.join(deleted))
> > +            self.ok = False
> > +        return self.ok
> > +
> > +    def read_dec_files_from_git(self):
> > +        # run git ls-files *.dec
> > +        out = self.run_git('ls-files', '*.dec')
> > +        # return list of .dec files
> > +        try:
> > +            return out.split()
> > +        except:
> > +            return []
> > +
> > +    def read_files_modified_from_git(self, commit, filter):
> > +        # run git diff-tree --no-commit-id --name-only -r <commit>
> > +        out = self.run_git('diff-tree', '--no-commit-id', '--name-
> only',
> > +                           '--diff-filter=' + filter, '-r', commit)
> > +        try:
> > +            return out.split()
> > +        except:
> > +            return []
> > +
> >      def read_commit_list_from_git(self, rev_spec, max_count):
> >          # Run git to get the commit patch
> >          cmd = [ 'rev-list', '--abbrev-commit', '--no-walk' ]
> > --
> > 2.40.1.windows.1
> >
> >
> >
> > 
> >
> >


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#115462): https://edk2.groups.io/g/devel/message/115462
Mute This Topic: https://groups.io/mt/104345509/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-