From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from nwk-aaemail-lapp03.apple.com (nwk-aaemail-lapp03.apple.com [17.151.62.68]) by mx.groups.io with SMTP id smtpd.web10.83.1612568160324046313 for ; Fri, 05 Feb 2021 15:36:00 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@apple.com header.s=20180706 header.b=pPdNqUNl; spf=pass (domain: apple.com, ip: 17.151.62.68, mailfrom: afish@apple.com) Received: from pps.filterd (nwk-aaemail-lapp03.apple.com [127.0.0.1]) by nwk-aaemail-lapp03.apple.com (8.16.0.42/8.16.0.42) with SMTP id 115NURjl022151; Fri, 5 Feb 2021 15:35:59 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=apple.com; h=from : message-id : content-type : mime-version : subject : date : in-reply-to : cc : to : references; s=20180706; bh=YHRLsVEKWUWfCa1a1RDKiD4cUCNd1Ip+mr4ji3Hx5Q4=; b=pPdNqUNlVocKr7s3yj6MBM5IQ7mxq/AxDDkWeJLs417T6nE5PTlXAEbFDQcLskTgxS0E 3EmZFEsYlaRvg0f98m7AJTcd0a2ac6okpfkQy/3iCW9jegGGRCfhZ1uBEcBlRQmtUFlO d6gkxYq5aCl3icnQcqs0uxzO5dhuWTHGLENt3nz5dKV/dboDcNzYe8SOUjy4zv8IpuPb 0L/ERkRJ1GwS7PKOt5QjILIfK12T1lX3TLMn8d35WhsgvBUqwVJdtTc351TCbBubcxWJ KJtuL98/KCcE/SS5jChfSC3xNzoQQS3w86onVpGQHk/C5OpkBu4SUTfJfxdAjC0kaICU pA== Received: from rn-mailsvcp-mta-lapp02.rno.apple.com (rn-mailsvcp-mta-lapp02.rno.apple.com [10.225.203.150]) by nwk-aaemail-lapp03.apple.com with ESMTP id 36dr1v5juv-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Fri, 05 Feb 2021 15:35:59 -0800 Received: from rn-mailsvcp-mmp-lapp03.rno.apple.com (rn-mailsvcp-mmp-lapp03.rno.apple.com [17.179.253.16]) by rn-mailsvcp-mta-lapp02.rno.apple.com (Oracle Communications Messaging Server 8.1.0.7.20201203 64bit (built Dec 3 2020)) with ESMTPS id <0QO200WYOXJXCBB0@rn-mailsvcp-mta-lapp02.rno.apple.com>; Fri, 05 Feb 2021 15:35:57 -0800 (PST) Received: from process_milters-daemon.rn-mailsvcp-mmp-lapp03.rno.apple.com by rn-mailsvcp-mmp-lapp03.rno.apple.com (Oracle Communications Messaging Server 8.1.0.7.20201203 64bit (built Dec 3 2020)) id <0QO200G00XEIUM00@rn-mailsvcp-mmp-lapp03.rno.apple.com>; Fri, 05 Feb 2021 15:35:57 -0800 (PST) X-Va-A: X-Va-T-CD: da3a4df698400084da27c6ab403bcb35 X-Va-E-CD: 9d84617848399215bcacda3b28bf7f29 X-Va-R-CD: 8e8a47db792b9fc13122fef88649bc07 X-Va-CD: 0 X-Va-ID: 358a1793-8e8c-4dfe-b7a6-0854e67a6cda X-V-A: X-V-T-CD: da3a4df698400084da27c6ab403bcb35 X-V-E-CD: 9d84617848399215bcacda3b28bf7f29 X-V-R-CD: 8e8a47db792b9fc13122fef88649bc07 X-V-CD: 0 X-V-ID: 290ce731-e445-4640-a0d8-1ed13bbaea6c X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369,18.0.737 definitions=2021-02-05_15:2021-02-05,2021-02-05 signatures=0 Received: from [17.235.10.45] (unknown [17.235.10.45]) by rn-mailsvcp-mmp-lapp03.rno.apple.com (Oracle Communications Messaging Server 8.1.0.7.20201203 64bit (built Dec 3 2020)) with ESMTPSA id <0QO2004CKXJWJL00@rn-mailsvcp-mmp-lapp03.rno.apple.com>; Fri, 05 Feb 2021 15:35:57 -0800 (PST) From: "Andrew Fish" Message-id: <3B681CAB-2FD4-448D-86D2-59CD2576D698@apple.com> MIME-version: 1.0 (Mac OS X Mail 14.0 \(3654.20.0.2.1\)) Subject: Re: [edk2-devel] EDK2 Date: Fri, 05 Feb 2021 15:35:56 -0800 In-reply-to: Cc: edk2-devel-groups-io , "Pham, Tony Q" To: Mike Kinney References: X-Mailer: Apple Mail (2.3654.20.0.2.1) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369,18.0.737 definitions=2021-02-05_15:2021-02-05,2021-02-05 signatures=0 Content-type: multipart/alternative; boundary="Apple-Mail=_4B515475-8A47-472D-91FF-40044812201C" --Apple-Mail=_4B515475-8A47-472D-91FF-40044812201C Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii > On Feb 4, 2021, at 7:30 PM, Kinney, Michael D > wrote: >=20 > Hi Andrew, > > If the character is part of the code (not a comment), the ignoring the c= odec error could silently produce the incorrect FW behavior. > > I prefer a failure with a correct identification of the file/line # so t= he file can be fixed. > > The EDK II CI checks will not allow files in with these types of issues. > Mike, I agree the UNI file should be strict. I was also thinking about comments,= build config files etc?=20 Is there a definition of what is legal in a comment? The code that had the= crazy characters was not from TianoCore.=20 Thanks, Andrew Fish > Mike > > > > From: Andrew Fish >=20 > Sent: Thursday, February 4, 2021 4:11 PM > To: edk2-devel-groups-io >; Kinney, Michael D > > Cc: Pham, Tony Q > > Subject: Re: [edk2-devel] EDK2 > > >=20 >=20 > On Feb 4, 2021, at 3:58 PM, Michael D Kinney > wrote: > > Hi Tony, > > I suspect that one of your UNI files being processed by StrGather has an= invalid Unicode character in it. Can you review the UNI files in the modu= le that is being build when this error is generated? > > I would be better if this code identified the file/line number with the = issue, so that would be a good Bugzilla to enter. > > > Mike, > > I hit something like this too writing some Python. I think I ended up te= lling the codec to ignore errors, so that might be another option? This I t= hink the issue I saw was in C code.=20 > > Thanks, > > Andrew Fish >=20 >=20 > Mike > > From: devel@edk2.groups.io > On Behalf Of Pham, Tony Q > Sent: Thursday, February 4, 2021 1:39 PM > To: devel@edk2.groups.io > Subject: [edk2-devel] EDK2 > > Hi, > > I have a problem with build.py > > (Python 3.9.1 on win32) Traceback (most recent call last): > File "C:\edk2\BaseTools\Source\Python\build\build.py", line 2635, in M= ain > MyBuild.Launch() > File "C:\edk2\BaseTools\Source\Python\build\build.py", line 2433, in L= aunch > self._BuildModule() > File "C:\edk2\BaseTools\Source\Python\build\build.py", line 1895, in _= BuildModule > Ma.CreateCodeFile(True) > File "C:\edk2\BaseTools\Source\Python\AutoGen\ModuleAutoGen.py", line = 1832, in CreateCodeFile > for File in self.AutoGenFileList: > File "C:\edk2\BaseTools\Source\Python\Common\caching.py", line 28, in = __get__ > Value =3D obj.__dict__[self._function.__name__] =3D self._function(o= bj) > File "C:\edk2\BaseTools\Source\Python\AutoGen\ModuleAutoGen.py", line = 983, in AutoGenFileList > GenC.CreateCode(self, AutoGenC, AutoGenH, StringH, AutoGenUniIdf, Un= iStringBinBuffer, StringIdf, AutoGenUniIdf, IdfGenBinBuffer) > File "C:\edk2\BaseTools\Source\Python\AutoGen\GenC.py", line 2044, in = CreateCode > CreateUnicodeStringCode(Info, AutoGenC, StringH, UniGenCFlag, UniGen= BinBuffer) > File "C:\edk2\BaseTools\Source\Python\AutoGen\GenC.py", line 1706, in = CreateUnicodeStringCode > Header, Code =3D GetStringFiles(Info.UnicodeFileList, SrcList, IncLi= st, Info.IncludePathList, ['.uni', '.inf'], Info.Name, CompatibleMode, Shel= lMode, UniGenCFlag, UniGenBinBuffer, FilterInfo) > File "C:\edk2\BaseTools\Source\Python\AutoGen\StrGather.py", line 563,= in GetStringFiles > Uni =3D SearchString(Uni, sorted (FileList), IsCompatibleMode) > File "C:\edk2\BaseTools\Source\Python\AutoGen\StrGather.py", line 532,= in SearchString > for Line in Lines: > File "C:\Users\tqpham\AppData\Local\Programs\Python\Python39\lib\encod= ings\cp1252.py", line 23, in decode > return codecs.charmap_decode(input,self.errors,decoding_table)[0] > UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 5= 457: character maps to > > > - Failed - > Build end time: 13:36:22, Feb.04 2021 > Build total time: 00:00:02 >=20 --Apple-Mail=_4B515475-8A47-472D-91FF-40044812201C Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii


On Feb 4, 2021, at 7:30 PM, K= inney, Michael D <michael.d.kinney@intel.com> wrote:

Hi Andrew,
 
If the character is part of the code (not a comment= ), the ignoring the codec error could silently produce the incorrect FW beh= avior.
 
I pr= efer a failure with a correct identification of the file/line # so the file= can be fixed.
 
The EDK II CI checks will not allow files in with these types of iss= ues.
 

Mike,

I agree the UNI file should be strict. I was a= lso thinking about comments, build config files etc? 

Is there a definition of what i= s legal in a comment? The code that had the crazy characters was not from T= ianoCore. 

T= hanks,

Andrew Fis= h

=
Mike
 
 
 
From: Andrew Fish <afish@apple.com> <= /span>
Sent: Thursday, February 4, 2021 4:11 PM
To: edk2-deve= l-groups-io <devel@edk2.groups.io>; = Kinney, Michael D <michael.d.kinne= y@intel.com>
Cc: Pham, Tony Q <tony.q.pham@intel.com>
Subject: Re: [edk2-devel] EDK2
&n= bsp;
 


On Feb 4, 2021, at 3:58 PM, Michael D Kinney <michael.d.kinney@intel.com> wrote:<= o:p class=3D"">
 
Hi Tony,
 
I susp= ect that one of your UNI files being processed by StrGather has an invalid Unicode character i= n it.  Can you revie= w the UNI files in the module that is being build when this error is genera= ted?
 
I would be better if this code ident= ified the file/line number with the issue, so that would be a good Bugzilla= to enter.
 
 
Mike,
 
I hit something like this= too writing some Python. I think I ended up telling the codec to ignore er= rors, so that might be another option? This I think the issue I saw was in = C code. 
 
<= div class=3D"">
Thanks,
 
Andrew Fish


Mi= ke
<= span class=3D""> 
From:=  = devel@edk2.groups.io <devel@edk2.groups.io&= gt; On Beha= lf Of Pham, Tony QSent:&n= bsp;Thursday, February 4, 2021 1:39 PM
T= o: devel@edk2.= groups.io
Subject: [edk2-devel] EDK2
 
Hi,
 
I have a pr= oblem with build.py
 
<= /div>
(Python 3.9.1 on win32= ) Traceback (most recent call last):
  File "C:\edk2\Base= Tools\Source\Python\build\build.py", line 2635, in Main
 =    MyBuild.Launch()
  File "C:\edk2\BaseTools\S= ource\Python\build\build.py", line 2433, in Launch
  = ;  self._BuildModule()
  File "C:\edk2\BaseTools\Sou= rce\Python\build\build.py", line 1895, in _BuildModule
 &= nbsp;  Ma.CreateCodeFile(True)
  File "C:\edk2\BaseT= ools\Source\Python\AutoGen\ModuleAutoGen.py", line 1832, in CreateCodeFile<= o:p class=3D"">
    for File in self.AutoGenFileList:
  File "C:\edk2\BaseTools\Source\Python\Common\caching.py", lin= e 28, in __get__
    Value =3D obj.__dict__[sel= f._function.__name__] =3D self._function(obj)<= /div>
  File "C:\= edk2\BaseTools\Source\Python\AutoGen\ModuleAutoGen.py", line 983, in AutoGe= nFileList
    GenC.CreateCode(self, AutoGenC, = AutoGenH, StringH, AutoGenUniIdf, UniStringBinBuffer, StringIdf, AutoGenUni= Idf, IdfGenBinBuffer)
  File "C:\edk2\BaseTools\Source\Py= thon\AutoGen\GenC.py", line 2044, in CreateCode
  &n= bsp; CreateUnicodeStringCode(Info, AutoGenC, StringH, UniGenCFlag, UniGenBi= nBuffer)
  File "C:\edk2\BaseTools\Source\Python\AutoGe= n\GenC.py", line 1706, in CreateUnicodeStringCode
  =   Header, Code =3D GetStringFiles(Info.UnicodeFileList, SrcList, IncLi= st, Info.IncludePathList, ['.uni', '.inf'], Info.Name, CompatibleMode, Shel= lMode, UniGenCFlag, UniGenBinBuffer, FilterInfo)
  File "= C:\edk2\BaseTools\Source\Python\AutoGen\StrGather.py", line 563, in GetStri= ngFiles
    Uni =3D SearchString(Uni, sorted= (FileList), IsCompatibleMode)
  File "C:\edk2\BaseTools\= Source\Python\AutoGen\StrGather.py", line 532, in SearchString
    for Line in Lines:
  File "C:\Users\t= qpham\AppData\Local\Programs\Python\Python39\lib\encodings\cp1252.py", line= 23, in decode
    return codecs.charmap_decode= (input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'cha= rmap' codec can't decode byte 0x9d in position 5457: character maps to <= undefined>
 
<= div class=3D"">
 =
- Failed= -
<= span class=3D"">Build end time: 13:36:22, Feb.04 2021=
Build to= tal time: 00:00:02
=

--Apple-Mail=_4B515475-8A47-472D-91FF-40044812201C--