[PATCH 0/2] Some refines to BaseTools/PatchCheck.py

[PATCH 0/2] Some refines to BaseTools/PatchCheck.py

[Hao Wu]

The following two refinements are made to PatchCheck.py:
a. Resolves misreport of 'Patch format error: diff found after end of
   patch', when a patch file contains a binary file change and any other
   changes after the binary change.

b. Adds detection of adding new binary files in a patch file or in a
   commit.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>

Hao Wu (2):
  BaseTools/PatchCheck.py: Fix misreport for binary changes in patch
  BaseTools/PatchCheck.py: Add warning info for new binary files

 BaseTools/Scripts/PatchCheck.py | 40 ++++++++++++++++++++++++----------------
 1 file changed, 24 insertions(+), 16 deletions(-)

-- 
2.12.0.windows.1

[PATCH 1/2] BaseTools/PatchCheck.py: Fix misreport for binary changes in patch

[Hao Wu]

For a patch file that:
1. Contains a binary change
2. Contains any other changes after the binary change

PatchCheck.py will complains with the following error:
* Patch format error: diff found after end of patch
   Line: literal XXXX

This commit resolves this misreport.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Hao Wu <hao.a.wu@intel.com>
---
 BaseTools/Scripts/PatchCheck.py | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/BaseTools/Scripts/PatchCheck.py b/BaseTools/Scripts/PatchCheck.py
index 7c3008233c..4bc697b1af 100755
--- a/BaseTools/Scripts/PatchCheck.py
+++ b/BaseTools/Scripts/PatchCheck.py
@@ -1,7 +1,7 @@
 ## @file
 #  Check a patch for various format issues
 #
-#  Copyright (c) 2015 - 2016, Intel Corporation. All rights reserved.<BR>
+#  Copyright (c) 2015 - 2017, Intel Corporation. All rights reserved.<BR>
 #
 #  This program and the accompanying materials are licensed and made
 #  available under the terms and conditions of the BSD License which
@@ -265,7 +265,7 @@ class GitDiffCheck:
             if line.startswith('@@ '):
                 self.state = PRE_PATCH
             elif len(line) >= 1 and line[0] not in ' -+' and \
-                 not line.startswith(r'\ No newline '):
+                 not line.startswith(r'\ No newline ') and not self.binary:
                 for line in self.lines[self.line_num + 1:]:
                     if line.startswith('diff --git'):
                         self.format_error('diff found after end of patch')
@@ -300,7 +300,7 @@ class GitDiffCheck:
         elif self.state == PATCH:
             if self.binary:
                 pass
-            if line.startswith('-'):
+            elif line.startswith('-'):
                 pass
             elif line.startswith('+'):
                 self.check_added_line(line[1:])
-- 
2.12.0.windows.1

[PATCH 2/2] BaseTools/PatchCheck.py: Add warning info for new binary files

[Hao Wu]

The commit adds the detection of adding new binary files in a patch file
or in a commit.

The following warning messages will be appended at the end of the script
output:

WARNING - The following binary files will be added into the repository:
  <BinaryFile1>
  <BinaryFile2>
  ...

Cc: Liming Gao <liming.gao@intel.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Hao Wu <hao.a.wu@intel.com>
---
 BaseTools/Scripts/PatchCheck.py | 34 +++++++++++++++++++++-------------
 1 file changed, 21 insertions(+), 13 deletions(-)

diff --git a/BaseTools/Scripts/PatchCheck.py b/BaseTools/Scripts/PatchCheck.py
index 4bc697b1af..7bc5736dbf 100755
--- a/BaseTools/Scripts/PatchCheck.py
+++ b/BaseTools/Scripts/PatchCheck.py
@@ -243,6 +243,7 @@ class GitDiffCheck:
         self.count = len(self.lines)
         self.line_num = 0
         self.state = START
+        self.new_bin = []
         while self.line_num < self.count and self.format_ok:
             line_num = self.line_num
             self.run()
@@ -254,6 +255,11 @@ class GitDiffCheck:
             return
         if self.ok:
             print('The code passed all checks.')
+        if self.new_bin:
+            print('\nWARNING - The following binary files will be added ' +
+                  'into the repository:')
+            for binary in self.new_bin:
+                print('  ' + binary)
 
     def run(self):
         line = self.lines[self.line_num]
@@ -276,21 +282,25 @@ class GitDiffCheck:
         if self.state == START:
             if line.startswith('diff --git'):
                 self.state = PRE_PATCH
-                self.set_filename(None)
+                self.filename = line[13:].split(' ',1)[0]
+                self.is_newfile = False
+                self.force_crlf = not self.filename.endswith('.sh')
             elif len(line.rstrip()) != 0:
                 self.format_error("didn't find diff command")
             self.line_num += 1
         elif self.state == PRE_PATCH:
-            if line.startswith('+++ b/'):
-                self.set_filename(line[6:].rstrip())
             if line.startswith('@@ '):
                 self.state = PATCH
                 self.binary = False
-            elif line.startswith('GIT binary patch'):
+            elif line.startswith('GIT binary patch') or \
+                 line.startswith('Binary files'):
                 self.state = PATCH
                 self.binary = True
+                if self.is_newfile:
+                    self.new_bin.append(self.filename)
             else:
                 ok = False
+                self.is_newfile = self.newfile_prefix_re.match(line)
                 for pfx in self.pre_patch_prefixes:
                     if line.startswith(pfx):
                         ok = True
@@ -320,22 +330,20 @@ class GitDiffCheck:
         'new mode ',
         'similarity index ',
         'rename ',
-        'Binary files ',
         )
 
     line_endings = ('\r\n', '\n\r', '\n', '\r')
 
-    def set_filename(self, filename):
-        self.hunk_filename = filename
-        if filename:
-            self.force_crlf = not filename.endswith('.sh')
-        else:
-            self.force_crlf = True
+    newfile_prefix_re = \
+        re.compile(r'''^
+                       index\ 0+\.\.
+                   ''',
+                   re.VERBOSE)
 
     def added_line_error(self, msg, line):
         lines = [ msg ]
-        if self.hunk_filename is not None:
-            lines.append('File: ' + self.hunk_filename)
+        if self.filename is not None:
+            lines.append('File: ' + self.filename)
         lines.append('Line: ' + line)
 
         self.error(*lines)
-- 
2.12.0.windows.1

Re: [PATCH 0/2] Some refines to BaseTools/PatchCheck.py

[Gao, Liming]

Reviewed-by: Liming Gao <liming.gao@intel.com>

>-----Original Message-----
>From: Wu, Hao A
>Sent: Thursday, June 08, 2017 3:49 PM
>To: edk2-devel@lists.01.org
>Cc: Wu, Hao A <hao.a.wu@intel.com>; Gao, Liming <liming.gao@intel.com>;
>Justen, Jordan L <jordan.l.justen@intel.com>
>Subject: [PATCH 0/2] Some refines to BaseTools/PatchCheck.py
>
>The following two refinements are made to PatchCheck.py:
>a. Resolves misreport of 'Patch format error: diff found after end of
>   patch', when a patch file contains a binary file change and any other
>   changes after the binary change.
>
>b. Adds detection of adding new binary files in a patch file or in a
>   commit.
>
>Cc: Liming Gao <liming.gao@intel.com>
>Cc: Jordan Justen <jordan.l.justen@intel.com>
>
>Hao Wu (2):
>  BaseTools/PatchCheck.py: Fix misreport for binary changes in patch
>  BaseTools/PatchCheck.py: Add warning info for new binary files
>
> BaseTools/Scripts/PatchCheck.py | 40 ++++++++++++++++++++++++---------
>-------
> 1 file changed, 24 insertions(+), 16 deletions(-)
>
>--
>2.12.0.windows.1