[PATCH 1/1] BaseTools: Remove non-ascii character of StructurePcd comment

[PATCH 1/1] BaseTools: Remove non-ascii character of StructurePcd comment

[Yuwei Chen]

Currently, the ConvertFceToStructurePcd.py tool generate
StructurePcd dsc file with comments from UNI file including
non-ascii character. Following DSC spec, there should not have
non-ascii character in DSC file. This patch removes the non-ascii
character when adding the comment and changes the circle R to (R).

Signed-off-by: Yuwei Chen <yuwei.chen@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
---
 BaseTools/Scripts/ConvertFceToStructurePcd.py | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/BaseTools/Scripts/ConvertFceToStructurePcd.py b/BaseTools/Scripts/ConvertFceToStructurePcd.py
index 2052db8c4b69..2baabf2dd521 100644
--- a/BaseTools/Scripts/ConvertFceToStructurePcd.py
+++ b/BaseTools/Scripts/ConvertFceToStructurePcd.py
@@ -284,7 +284,15 @@ class Config(object):
         line=x.split('\n')[0]
         comment_list = value_re.findall(line) # the string \\... in "Q...." line
         comment_list[0] = comment_list[0].replace('//', '')
-        comment = comment_list[0].strip()
+        comment_ori = comment_list[0].strip()
+        comment = ""
+        for each in comment_ori:
+            if each != " " and "\x21" > each or each > "\x7E":
+                if bytes(each, 'utf-16') == b'\xff\xfe\xae\x00':
+                    each = '(R)'
+                else:
+                    each = ""
+            comment += each
         line=value_re.sub('',line) #delete \\... in "Q...." line
         list1=line.split(' ')
         value=self.value_parser(list1)
-- 
2.27.0.windows.1

Re: [PATCH 1/1] BaseTools: Remove non-ascii character of StructurePcd comment

[Bob Feng]

Reviewed-by: Bob Feng <bob.c.feng@intel.com>

-----Original Message-----
From: Chen, Christine <yuwei.chen@intel.com> 
Sent: Monday, June 28, 2021 4:47 PM
To: devel@edk2.groups.io
Cc: Feng, Bob C <bob.c.feng@intel.com>; Liming Gao <gaoliming@byosoft.com.cn>
Subject: [PATCH 1/1] BaseTools: Remove non-ascii character of StructurePcd comment

Currently, the ConvertFceToStructurePcd.py tool generate StructurePcd dsc file with comments from UNI file including non-ascii character. Following DSC spec, there should not have non-ascii character in DSC file. This patch removes the non-ascii character when adding the comment and changes the circle R to (R).

Signed-off-by: Yuwei Chen <yuwei.chen@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
---
 BaseTools/Scripts/ConvertFceToStructurePcd.py | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/BaseTools/Scripts/ConvertFceToStructurePcd.py b/BaseTools/Scripts/ConvertFceToStructurePcd.py
index 2052db8c4b69..2baabf2dd521 100644
--- a/BaseTools/Scripts/ConvertFceToStructurePcd.py
+++ b/BaseTools/Scripts/ConvertFceToStructurePcd.py
@@ -284,7 +284,15 @@ class Config(object):
         line=x.split('\n')[0]
         comment_list = value_re.findall(line) # the string \\... in "Q...." line
         comment_list[0] = comment_list[0].replace('//', '')
-        comment = comment_list[0].strip()
+        comment_ori = comment_list[0].strip()
+        comment = ""
+        for each in comment_ori:
+            if each != " " and "\x21" > each or each > "\x7E":
+                if bytes(each, 'utf-16') == b'\xff\xfe\xae\x00':
+                    each = '(R)'
+                else:
+                    each = ""
+            comment += each
         line=value_re.sub('',line) #delete \\... in "Q...." line
         list1=line.split(' ')
         value=self.value_parser(list1)
--
2.27.0.windows.1