public inbox for devel@edk2.groups.io
 help / color / mirror / Atom feed
* [Patch V2] BaseTools: Fix parsing multiple nest !include issue
@ 2018-06-28  6:27 Yonghong Zhu
  2018-06-28 20:48 ` Carsey, Jaben
  0 siblings, 1 reply; 3+ messages in thread
From: Yonghong Zhu @ 2018-06-28  6:27 UTC (permalink / raw)
  To: edk2-devel; +Cc: Yunhua Feng, Liming Gao

From: Yunhua Feng <yunhuax.feng@intel.com>

Fix the bug !include file in Components subsection meet syntax error.

Case example:
DSC components:
!include Test1.txt

Test1.txt:
 TestPkg/TestDriver.inf {
   <PcdsFixedAtBuild>
    PcdToken.PcdTest1 | "A"
    !include Test2.txt
  }

Test2.txt:
!include Test3.txt

Test3.txt:
PcdToken.PcdTest2 | "B"

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yunhua Feng <yunhuax.feng@intel.com>
---
 .../Source/Python/Workspace/MetaFileParser.py      | 28 ++++++++++------------
 1 file changed, 13 insertions(+), 15 deletions(-)

diff --git a/BaseTools/Source/Python/Workspace/MetaFileParser.py b/BaseTools/Source/Python/Workspace/MetaFileParser.py
index 8c860d594b..75ca630806 100644
--- a/BaseTools/Source/Python/Workspace/MetaFileParser.py
+++ b/BaseTools/Source/Python/Workspace/MetaFileParser.py
@@ -942,11 +942,18 @@ class DscParser(MetaFileParser):
             # subsection header
             elif Line[0] == TAB_OPTION_START and Line[-1] == TAB_OPTION_END:
                 self._SubsectionType = MODEL_META_DATA_SUBSECTION_HEADER
             # directive line
             elif Line[0] == '!':
-                self._DirectiveParser()
+                TokenList = GetSplitValueList(Line, ' ', 1)
+                if TokenList[0].upper() == "!INCLUDE":
+                    for Arch, ModuleType, DefaultStore in self._Scope:
+                        if self._SubsectionType != MODEL_UNKNOWN and Arch in OwnerId:
+                            self._Owner[-1] = OwnerId[Arch]
+                        self._DirectiveParser()
+                else:
+                    self._DirectiveParser()
                 continue
             if Line[0] == TAB_OPTION_START and not self._InSubsection:
                 EdkLogger.error("Parser", FILE_READ_FAILURE, "Missing the '{' before %s in Line %s" % (Line, Index+1), ExtraData=self.MetaFile)
 
             if self._InSubsection:
@@ -963,11 +970,11 @@ class DscParser(MetaFileParser):
             # Model, Value1, Value2, Value3, Arch, ModuleType, BelongsToItem=-1, BelongsToFile=-1,
             # LineBegin=-1, ColumnBegin=-1, LineEnd=-1, ColumnEnd=-1, Enabled=-1
             #
             for Arch, ModuleType, DefaultStore in self._Scope:
                 Owner = self._Owner[-1]
-                if self._SubsectionType != MODEL_UNKNOWN:
+                if self._SubsectionType != MODEL_UNKNOWN and Arch in OwnerId:
                     Owner = OwnerId[Arch]
                 self._LastItem = self._Store(
                                         self._ItemType,
                                         self._ValueList[0],
                                         self._ValueList[1],
@@ -1188,10 +1195,11 @@ class DscParser(MetaFileParser):
     @ParseMacro
     def _ComponentParser(self):
         if self._CurrentLine[-1] == '{':
             self._ValueList[0] = self._CurrentLine[0:-1].strip()
             self._InSubsection = True
+            self._SubsectionType = MODEL_UNKNOWN
         else:
             self._ValueList[0] = self._CurrentLine
 
     ## [LibraryClasses] section
     @ParseMacro
@@ -1560,28 +1568,18 @@ class DscParser(MetaFileParser):
 
             self.IncludedFiles.add (IncludedFile1)
 
             # set the parser status with current status
             Parser._SectionName = self._SectionName
-            if self._InSubsection:
-                Parser._SectionType = self._SubsectionType
-            else:
-                Parser._SectionType = self._SectionType
+            Parser._SubsectionType = self._SubsectionType
+            Parser._InSubsection = self._InSubsection
+            Parser._SectionType = self._SectionType
             Parser._Scope = self._Scope
             Parser._Enabled = self._Enabled
             # Parse the included file
             Parser.Start()
 
-            # update current status with sub-parser's status
-            self._SectionName = Parser._SectionName
-            if not self._InSubsection:
-                self._SectionType = Parser._SectionType
-            self._SubsectionType = Parser._SubsectionType
-            self._InSubsection = Parser._InSubsection
-
-            self._Scope = Parser._Scope
-            self._Enabled = Parser._Enabled
 
             # Insert all records in the table for the included file into dsc file table
             Records = IncludedFileTable.GetAll()
             if Records:
                 self._Content[self._ContentIndex:self._ContentIndex] = Records
-- 
2.12.2.windows.2



^ permalink raw reply related	[flat|nested] 3+ messages in thread

* Re: [Patch V2] BaseTools: Fix parsing multiple nest !include issue
  2018-06-28  6:27 [Patch V2] BaseTools: Fix parsing multiple nest !include issue Yonghong Zhu
@ 2018-06-28 20:48 ` Carsey, Jaben
  2018-06-29  5:41   ` Zhu, Yonghong
  0 siblings, 1 reply; 3+ messages in thread
From: Carsey, Jaben @ 2018-06-28 20:48 UTC (permalink / raw)
  To: Zhu, Yonghong, edk2-devel@lists.01.org; +Cc: Feng, YunhuaX, Gao, Liming

If you change the raw string ("!INCLUDE) to use the one defined in DataType.py (TAB_INCLUDE), then I like it.  Note that you will need to lower() instead of upper() to support this change.

Reviewed-by: Jaben Carsey <jaben.carsey@intel.com>

> -----Original Message-----
> From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of
> Yonghong Zhu
> Sent: Wednesday, June 27, 2018 11:27 PM
> To: edk2-devel@lists.01.org
> Cc: Feng, YunhuaX <yunhuax.feng@intel.com>; Gao, Liming
> <liming.gao@intel.com>
> Subject: [edk2] [Patch V2] BaseTools: Fix parsing multiple nest !include issue
> 
> From: Yunhua Feng <yunhuax.feng@intel.com>
> 
> Fix the bug !include file in Components subsection meet syntax error.
> 
> Case example:
> DSC components:
> !include Test1.txt
> 
> Test1.txt:
>  TestPkg/TestDriver.inf {
>    <PcdsFixedAtBuild>
>     PcdToken.PcdTest1 | "A"
>     !include Test2.txt
>   }
> 
> Test2.txt:
> !include Test3.txt
> 
> Test3.txt:
> PcdToken.PcdTest2 | "B"
> 
> Cc: Liming Gao <liming.gao@intel.com>
> Cc: Yonghong Zhu <yonghong.zhu@intel.com>
> Contributed-under: TianoCore Contribution Agreement 1.1
> Signed-off-by: Yunhua Feng <yunhuax.feng@intel.com>
> ---
>  .../Source/Python/Workspace/MetaFileParser.py      | 28 ++++++++++--------
> ----
>  1 file changed, 13 insertions(+), 15 deletions(-)
> 
> diff --git a/BaseTools/Source/Python/Workspace/MetaFileParser.py
> b/BaseTools/Source/Python/Workspace/MetaFileParser.py
> index 8c860d594b..75ca630806 100644
> --- a/BaseTools/Source/Python/Workspace/MetaFileParser.py
> +++ b/BaseTools/Source/Python/Workspace/MetaFileParser.py
> @@ -942,11 +942,18 @@ class DscParser(MetaFileParser):
>              # subsection header
>              elif Line[0] == TAB_OPTION_START and Line[-1] == TAB_OPTION_END:
>                  self._SubsectionType = MODEL_META_DATA_SUBSECTION_HEADER
>              # directive line
>              elif Line[0] == '!':
> -                self._DirectiveParser()
> +                TokenList = GetSplitValueList(Line, ' ', 1)
> +                if TokenList[0].upper() == "!INCLUDE":
> +                    for Arch, ModuleType, DefaultStore in self._Scope:
> +                        if self._SubsectionType != MODEL_UNKNOWN and Arch in
> OwnerId:
> +                            self._Owner[-1] = OwnerId[Arch]
> +                        self._DirectiveParser()
> +                else:
> +                    self._DirectiveParser()
>                  continue
>              if Line[0] == TAB_OPTION_START and not self._InSubsection:
>                  EdkLogger.error("Parser", FILE_READ_FAILURE, "Missing the '{'
> before %s in Line %s" % (Line, Index+1), ExtraData=self.MetaFile)
> 
>              if self._InSubsection:
> @@ -963,11 +970,11 @@ class DscParser(MetaFileParser):
>              # Model, Value1, Value2, Value3, Arch, ModuleType, BelongsToItem=-
> 1, BelongsToFile=-1,
>              # LineBegin=-1, ColumnBegin=-1, LineEnd=-1, ColumnEnd=-1,
> Enabled=-1
>              #
>              for Arch, ModuleType, DefaultStore in self._Scope:
>                  Owner = self._Owner[-1]
> -                if self._SubsectionType != MODEL_UNKNOWN:
> +                if self._SubsectionType != MODEL_UNKNOWN and Arch in
> OwnerId:
>                      Owner = OwnerId[Arch]
>                  self._LastItem = self._Store(
>                                          self._ItemType,
>                                          self._ValueList[0],
>                                          self._ValueList[1],
> @@ -1188,10 +1195,11 @@ class DscParser(MetaFileParser):
>      @ParseMacro
>      def _ComponentParser(self):
>          if self._CurrentLine[-1] == '{':
>              self._ValueList[0] = self._CurrentLine[0:-1].strip()
>              self._InSubsection = True
> +            self._SubsectionType = MODEL_UNKNOWN
>          else:
>              self._ValueList[0] = self._CurrentLine
> 
>      ## [LibraryClasses] section
>      @ParseMacro
> @@ -1560,28 +1568,18 @@ class DscParser(MetaFileParser):
> 
>              self.IncludedFiles.add (IncludedFile1)
> 
>              # set the parser status with current status
>              Parser._SectionName = self._SectionName
> -            if self._InSubsection:
> -                Parser._SectionType = self._SubsectionType
> -            else:
> -                Parser._SectionType = self._SectionType
> +            Parser._SubsectionType = self._SubsectionType
> +            Parser._InSubsection = self._InSubsection
> +            Parser._SectionType = self._SectionType
>              Parser._Scope = self._Scope
>              Parser._Enabled = self._Enabled
>              # Parse the included file
>              Parser.Start()
> 
> -            # update current status with sub-parser's status
> -            self._SectionName = Parser._SectionName
> -            if not self._InSubsection:
> -                self._SectionType = Parser._SectionType
> -            self._SubsectionType = Parser._SubsectionType
> -            self._InSubsection = Parser._InSubsection
> -
> -            self._Scope = Parser._Scope
> -            self._Enabled = Parser._Enabled
> 
>              # Insert all records in the table for the included file into dsc file table
>              Records = IncludedFileTable.GetAll()
>              if Records:
>                  self._Content[self._ContentIndex:self._ContentIndex] = Records
> --
> 2.12.2.windows.2
> 
> _______________________________________________
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel


^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: [Patch V2] BaseTools: Fix parsing multiple nest !include issue
  2018-06-28 20:48 ` Carsey, Jaben
@ 2018-06-29  5:41   ` Zhu, Yonghong
  0 siblings, 0 replies; 3+ messages in thread
From: Zhu, Yonghong @ 2018-06-29  5:41 UTC (permalink / raw)
  To: Carsey, Jaben, edk2-devel@lists.01.org
  Cc: Feng, YunhuaX, Gao, Liming, Zhu, Yonghong

When I commit this patch, I will change it to use  TAB_INCLUDE. Thanks.

Best Regards,
Zhu Yonghong


-----Original Message-----
From: Carsey, Jaben 
Sent: Friday, June 29, 2018 4:48 AM
To: Zhu, Yonghong <yonghong.zhu@intel.com>; edk2-devel@lists.01.org
Cc: Feng, YunhuaX <yunhuax.feng@intel.com>; Gao, Liming <liming.gao@intel.com>
Subject: RE: [edk2] [Patch V2] BaseTools: Fix parsing multiple nest !include issue

If you change the raw string ("!INCLUDE) to use the one defined in DataType.py (TAB_INCLUDE), then I like it.  Note that you will need to lower() instead of upper() to support this change.

Reviewed-by: Jaben Carsey <jaben.carsey@intel.com>

> -----Original Message-----
> From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of 
> Yonghong Zhu
> Sent: Wednesday, June 27, 2018 11:27 PM
> To: edk2-devel@lists.01.org
> Cc: Feng, YunhuaX <yunhuax.feng@intel.com>; Gao, Liming 
> <liming.gao@intel.com>
> Subject: [edk2] [Patch V2] BaseTools: Fix parsing multiple nest 
> !include issue
> 
> From: Yunhua Feng <yunhuax.feng@intel.com>
> 
> Fix the bug !include file in Components subsection meet syntax error.
> 
> Case example:
> DSC components:
> !include Test1.txt
> 
> Test1.txt:
>  TestPkg/TestDriver.inf {
>    <PcdsFixedAtBuild>
>     PcdToken.PcdTest1 | "A"
>     !include Test2.txt
>   }
> 
> Test2.txt:
> !include Test3.txt
> 
> Test3.txt:
> PcdToken.PcdTest2 | "B"
> 
> Cc: Liming Gao <liming.gao@intel.com>
> Cc: Yonghong Zhu <yonghong.zhu@intel.com>
> Contributed-under: TianoCore Contribution Agreement 1.1
> Signed-off-by: Yunhua Feng <yunhuax.feng@intel.com>
> ---
>  .../Source/Python/Workspace/MetaFileParser.py      | 28 ++++++++++--------
> ----
>  1 file changed, 13 insertions(+), 15 deletions(-)
> 
> diff --git a/BaseTools/Source/Python/Workspace/MetaFileParser.py
> b/BaseTools/Source/Python/Workspace/MetaFileParser.py
> index 8c860d594b..75ca630806 100644
> --- a/BaseTools/Source/Python/Workspace/MetaFileParser.py
> +++ b/BaseTools/Source/Python/Workspace/MetaFileParser.py
> @@ -942,11 +942,18 @@ class DscParser(MetaFileParser):
>              # subsection header
>              elif Line[0] == TAB_OPTION_START and Line[-1] == TAB_OPTION_END:
>                  self._SubsectionType = MODEL_META_DATA_SUBSECTION_HEADER
>              # directive line
>              elif Line[0] == '!':
> -                self._DirectiveParser()
> +                TokenList = GetSplitValueList(Line, ' ', 1)
> +                if TokenList[0].upper() == "!INCLUDE":
> +                    for Arch, ModuleType, DefaultStore in self._Scope:
> +                        if self._SubsectionType != MODEL_UNKNOWN and 
> + Arch in
> OwnerId:
> +                            self._Owner[-1] = OwnerId[Arch]
> +                        self._DirectiveParser()
> +                else:
> +                    self._DirectiveParser()
>                  continue
>              if Line[0] == TAB_OPTION_START and not self._InSubsection:
>                  EdkLogger.error("Parser", FILE_READ_FAILURE, "Missing the '{'
> before %s in Line %s" % (Line, Index+1), ExtraData=self.MetaFile)
> 
>              if self._InSubsection:
> @@ -963,11 +970,11 @@ class DscParser(MetaFileParser):
>              # Model, Value1, Value2, Value3, Arch, ModuleType, 
> BelongsToItem=- 1, BelongsToFile=-1,
>              # LineBegin=-1, ColumnBegin=-1, LineEnd=-1, ColumnEnd=-1,
> Enabled=-1
>              #
>              for Arch, ModuleType, DefaultStore in self._Scope:
>                  Owner = self._Owner[-1]
> -                if self._SubsectionType != MODEL_UNKNOWN:
> +                if self._SubsectionType != MODEL_UNKNOWN and Arch in
> OwnerId:
>                      Owner = OwnerId[Arch]
>                  self._LastItem = self._Store(
>                                          self._ItemType,
>                                          self._ValueList[0],
>                                          self._ValueList[1], @@ 
> -1188,10 +1195,11 @@ class DscParser(MetaFileParser):
>      @ParseMacro
>      def _ComponentParser(self):
>          if self._CurrentLine[-1] == '{':
>              self._ValueList[0] = self._CurrentLine[0:-1].strip()
>              self._InSubsection = True
> +            self._SubsectionType = MODEL_UNKNOWN
>          else:
>              self._ValueList[0] = self._CurrentLine
> 
>      ## [LibraryClasses] section
>      @ParseMacro
> @@ -1560,28 +1568,18 @@ class DscParser(MetaFileParser):
> 
>              self.IncludedFiles.add (IncludedFile1)
> 
>              # set the parser status with current status
>              Parser._SectionName = self._SectionName
> -            if self._InSubsection:
> -                Parser._SectionType = self._SubsectionType
> -            else:
> -                Parser._SectionType = self._SectionType
> +            Parser._SubsectionType = self._SubsectionType
> +            Parser._InSubsection = self._InSubsection
> +            Parser._SectionType = self._SectionType
>              Parser._Scope = self._Scope
>              Parser._Enabled = self._Enabled
>              # Parse the included file
>              Parser.Start()
> 
> -            # update current status with sub-parser's status
> -            self._SectionName = Parser._SectionName
> -            if not self._InSubsection:
> -                self._SectionType = Parser._SectionType
> -            self._SubsectionType = Parser._SubsectionType
> -            self._InSubsection = Parser._InSubsection
> -
> -            self._Scope = Parser._Scope
> -            self._Enabled = Parser._Enabled
> 
>              # Insert all records in the table for the included file into dsc file table
>              Records = IncludedFileTable.GetAll()
>              if Records:
>                  self._Content[self._ContentIndex:self._ContentIndex] 
> = Records
> --
> 2.12.2.windows.2
> 
> _______________________________________________
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel


^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, other threads:[~2018-06-29  5:41 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2018-06-28  6:27 [Patch V2] BaseTools: Fix parsing multiple nest !include issue Yonghong Zhu
2018-06-28 20:48 ` Carsey, Jaben
2018-06-29  5:41   ` Zhu, Yonghong

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox