[Patch 0/2] BaseTools/GenFds: Make FDF parser mode flexible

[Patch 0/2] BaseTools/GenFds: Make FDF parser mode flexible

[Michael Kinney]

This patch series addresses the following issues:

https://bugzilla.tianocore.org/show_bug.cgi?id=132
https://bugzilla.tianocore.org/show_bug.cgi?id=141

This patch series replaces the following patch email:

https://lists.01.org/pipermail/edk2-devel/2016-October/002433.html

The logic for the OUTPUT_DIRECTORY was simplified and supports
files in more sections than just [FmpPayload] sections.

Cc: Kelly Steele <kelly.steele@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Michael Kinney <michael.d.kinney@intel.com>


Michael Kinney (2):
  BaseTools/GenFds: Support FDF sections in any order
  BaseTools/GenFds: Skip parse time OUTPUT_DIRECTORY file verify

 BaseTools/Source/Python/GenFds/FdfParser.py | 35 +++++++++--------------------
 1 file changed, 10 insertions(+), 25 deletions(-)

-- 
2.6.3.windows.1

[Patch 1/2] BaseTools/GenFds: Support FDF sections in any order

[Michael Kinney]

https://bugzilla.tianocore.org/show_bug.cgi?id=141

This patch updates EDK II FDF parser in GenFds to allow sections
to be placed in any order in the FDF file.

Cc: Kelly Steele <kelly.steele@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Michael Kinney <michael.d.kinney@intel.com>
---
 BaseTools/Source/Python/GenFds/FdfParser.py | 23 ++++-------------------
 1 file changed, 4 insertions(+), 19 deletions(-)

diff --git a/BaseTools/Source/Python/GenFds/FdfParser.py b/BaseTools/Source/Python/GenFds/FdfParser.py
index 02ae7c9..693128c 100644
--- a/BaseTools/Source/Python/GenFds/FdfParser.py
+++ b/BaseTools/Source/Python/GenFds/FdfParser.py
@@ -1385,25 +1385,10 @@ class FdfParser:
 
         try:
             self.Preprocess()
-            while self.__GetFd():
-                pass
-
-            while self.__GetFv():
-                pass
-
-            while self.__GetFmp():
-                pass
-
-            while self.__GetCapsule():
-                pass
-
-            while self.__GetVtf():
-                pass
-
-            while self.__GetRule():
-                pass
-            
-            while self.__GetOptionRom():
+            #
+            # Keep processing sections of the FDF until no new sections or a syntax error is found
+            #
+            while self.__GetFd() or self.__GetFv() or self.__GetFmp() or self.__GetCapsule() or self.__GetVtf() or self.__GetRule() or self.__GetOptionRom():
                 pass
 
         except Warning, X:
-- 
2.6.3.windows.1

[Patch 2/2] BaseTools/GenFds: Skip parse time OUTPUT_DIRECTORY file verify

[Michael Kinney]

https://bugzilla.tianocore.org/show_bug.cgi?id=132

This patch relaxes the file verification for all FILE statements
to not verify the file exists during initial FDF file parsing
if the file specified is in $(OUTPUT_DIRECTORY).  If a file is
specified does not exist when a section is processed, then a
build break will occur at that time instead.

Cc: Kelly Steele <kelly.steele@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Michael Kinney <michael.d.kinney@intel.com>
---
 BaseTools/Source/Python/GenFds/FdfParser.py | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/BaseTools/Source/Python/GenFds/FdfParser.py b/BaseTools/Source/Python/GenFds/FdfParser.py
index 693128c..b7edda9 100644
--- a/BaseTools/Source/Python/GenFds/FdfParser.py
+++ b/BaseTools/Source/Python/GenFds/FdfParser.py
@@ -3022,17 +3022,17 @@ class FdfParser:
     ## __VerifyFile
     #
     #    Check if file exists or not:
-    #      If current phase if GenFds, the file must exist;
-    #      If current phase is AutoGen and the file is not in $(OUTPUT_DIRECTORY), the file must exist
+    #      If the file is not in $(OUTPUT_DIRECTORY), then the file must exist
     #    @param FileName: File path to be verified.
     #
     def __VerifyFile(self, FileName):
         if FileName.replace('$(WORKSPACE)', '').find('$') != -1:
             return
-        if not GlobalData.gAutoGenPhase or not self.__GetMacroValue("OUTPUT_DIRECTORY") in FileName:
-            ErrorCode, ErrorInfo = PathClass(NormPath(FileName), GenFdsGlobalVariable.WorkSpaceDir).Validate()
-            if ErrorCode != 0:
-                EdkLogger.error("GenFds", ErrorCode, ExtraData=ErrorInfo)
+        if self.__GetMacroValue("OUTPUT_DIRECTORY") in FileName:
+            return
+        ErrorCode, ErrorInfo = PathClass(NormPath(FileName), GenFdsGlobalVariable.WorkSpaceDir).Validate()
+        if ErrorCode != 0:
+            EdkLogger.error("GenFds", ErrorCode, ExtraData=ErrorInfo)
 
     ## __GetCglSection() method
     #
-- 
2.6.3.windows.1

Re: [Patch 1/2] BaseTools/GenFds: Support FDF sections in any order

[Zhu, Yonghong]

This patch is fine to me.

Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com> 

Best Regards,
Zhu Yonghong

-----Original Message-----
From: Kinney, Michael D 
Sent: Saturday, October 08, 2016 5:33 AM
To: edk2-devel@lists.01.org
Cc: Steele, Kelly <kelly.steele@intel.com>; Zhu, Yonghong <yonghong.zhu@intel.com>; Gao, Liming <liming.gao@intel.com>
Subject: [Patch 1/2] BaseTools/GenFds: Support FDF sections in any order

https://bugzilla.tianocore.org/show_bug.cgi?id=141

This patch updates EDK II FDF parser in GenFds to allow sections to be placed in any order in the FDF file.

Cc: Kelly Steele <kelly.steele@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Michael Kinney <michael.d.kinney@intel.com>
---
 BaseTools/Source/Python/GenFds/FdfParser.py | 23 ++++-------------------
 1 file changed, 4 insertions(+), 19 deletions(-)

diff --git a/BaseTools/Source/Python/GenFds/FdfParser.py b/BaseTools/Source/Python/GenFds/FdfParser.py
index 02ae7c9..693128c 100644
--- a/BaseTools/Source/Python/GenFds/FdfParser.py
+++ b/BaseTools/Source/Python/GenFds/FdfParser.py
@@ -1385,25 +1385,10 @@ class FdfParser:
 
         try:
             self.Preprocess()
-            while self.__GetFd():
-                pass
-
-            while self.__GetFv():
-                pass
-
-            while self.__GetFmp():
-                pass
-
-            while self.__GetCapsule():
-                pass
-
-            while self.__GetVtf():
-                pass
-
-            while self.__GetRule():
-                pass
-            
-            while self.__GetOptionRom():
+            #
+            # Keep processing sections of the FDF until no new sections or a syntax error is found
+            #
+            while self.__GetFd() or self.__GetFv() or self.__GetFmp() or self.__GetCapsule() or self.__GetVtf() or self.__GetRule() or self.__GetOptionRom():
                 pass
 
         except Warning, X:
--
2.6.3.windows.1

Re: [Patch 1/2] BaseTools/GenFds: Support FDF sections in any order

[Kinney, Michael D]

Zhu Yonghong,

Thanks for the RB on the first patch in that series.

Your patch: 

  https://lists.01.org/pipermail/edk2-devel/2016-October/002507.html
  [Patch] BaseTools: Extend FMP to support FV statement and FD statement

is a better solution to the OUTPUT_DIRECTORY issue, so I am dropping 

  https://lists.01.org/pipermail/edk2-devel/2016-October/002481.html
  [Patch 2/2] BaseTools/GenFds: Skip parse time OUTPUT_DIRECTORY file verify

Thanks,

Mike

> -----Original Message-----
> From: Zhu, Yonghong
> Sent: Tuesday, October 11, 2016 9:13 PM
> To: Kinney, Michael D <michael.d.kinney@intel.com>; edk2-devel@lists.01.org
> Cc: Steele, Kelly <kelly.steele@intel.com>; Gao, Liming <liming.gao@intel.com>; Zhu,
> Yonghong <yonghong.zhu@intel.com>
> Subject: RE: [Patch 1/2] BaseTools/GenFds: Support FDF sections in any order
> 
> This patch is fine to me.
> 
> Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
> 
> Best Regards,
> Zhu Yonghong
> 
> -----Original Message-----
> From: Kinney, Michael D
> Sent: Saturday, October 08, 2016 5:33 AM
> To: edk2-devel@lists.01.org
> Cc: Steele, Kelly <kelly.steele@intel.com>; Zhu, Yonghong <yonghong.zhu@intel.com>;
> Gao, Liming <liming.gao@intel.com>
> Subject: [Patch 1/2] BaseTools/GenFds: Support FDF sections in any order
> 
> https://bugzilla.tianocore.org/show_bug.cgi?id=141
> 
> This patch updates EDK II FDF parser in GenFds to allow sections to be placed in any
> order in the FDF file.
> 
> Cc: Kelly Steele <kelly.steele@intel.com>
> Cc: Yonghong Zhu <yonghong.zhu@intel.com>
> Cc: Liming Gao <liming.gao@intel.com>
> Contributed-under: TianoCore Contribution Agreement 1.0
> Signed-off-by: Michael Kinney <michael.d.kinney@intel.com>
> ---
>  BaseTools/Source/Python/GenFds/FdfParser.py | 23 ++++-------------------
>  1 file changed, 4 insertions(+), 19 deletions(-)
> 
> diff --git a/BaseTools/Source/Python/GenFds/FdfParser.py
> b/BaseTools/Source/Python/GenFds/FdfParser.py
> index 02ae7c9..693128c 100644
> --- a/BaseTools/Source/Python/GenFds/FdfParser.py
> +++ b/BaseTools/Source/Python/GenFds/FdfParser.py
> @@ -1385,25 +1385,10 @@ class FdfParser:
> 
>          try:
>              self.Preprocess()
> -            while self.__GetFd():
> -                pass
> -
> -            while self.__GetFv():
> -                pass
> -
> -            while self.__GetFmp():
> -                pass
> -
> -            while self.__GetCapsule():
> -                pass
> -
> -            while self.__GetVtf():
> -                pass
> -
> -            while self.__GetRule():
> -                pass
> -
> -            while self.__GetOptionRom():
> +            #
> +            # Keep processing sections of the FDF until no new sections or a syntax
> error is found
> +            #
> +            while self.__GetFd() or self.__GetFv() or self.__GetFmp() or
> self.__GetCapsule() or self.__GetVtf() or self.__GetRule() or self.__GetOptionRom():
>                  pass
> 
>          except Warning, X:
> --
> 2.6.3.windows.1