I found a couple of extensions for Visual Studio Code that will highlight bad characters. "Highlight Dodgy Characters" seems to do the trick. Sorry about that AMD Official Use Only thing. The wonders of modern email apps! Garrett Kirkendall SMTS Firmware Engineer 7171 Southwest Parkway, Austin, TX 78735 USA AMD facebook | amd.com From: Kinney, Michael D Sent: Friday, February 5, 2021 11:03 AM To: Kirkendall, Garrett ; devel@edk2.groups.io; Feng, Bob C ; Andrew Fish ; Kinney, Michael D Cc: Pham, Tony Q Subject: RE: [edk2-devel] EDK2 [AMD Official Use Only - Internal Distribution Only] [CAUTION: External Email] One I know the file path, I usually use a Notepad++ feature: Search-> Find Chars In Range-> Non ASCII Mike From: Kirkendall, Garrett > Sent: Friday, February 5, 2021 7:11 AM To: devel@edk2.groups.io; Feng, Bob C >; Kinney, Michael D >; Andrew Fish > Cc: Pham, Tony Q > Subject: RE: [edk2-devel] EDK2 [AMD Official Use Only - Internal Distribution Only] 0x9d is one of those pesky "smart quotes" many applications love to use. These are the double quote or single quote characters that slant left and right instead of the straight up and down like the ASCII versions. They can be very hard to track down in a source file because a lot of editors have very subtle slants to the smart quotes. This usually happens when you copy from one app like MS Word and paste into your source file. Garrett Kirkendall SMTS Firmware Engineer 7171 Southwest Parkway, Austin, TX 78735 USA AMD facebook | amd.com From: devel@edk2.groups.io > On Behalf Of Bob Feng via groups.io Sent: Thursday, February 4, 2021 11:40 PM To: devel@edk2.groups.io; Kinney, Michael D >; Andrew Fish > Cc: Pham, Tony Q > Subject: Re: [edk2-devel] EDK2 [CAUTION: External Email] Tony, This build failure should be caused by one of your c or header files have non-ascii characters. You may need to change the basetools' code to see which file has non-ascii characters Change C:\edk2\BaseTools\Source\Python\AutoGen\StrGather.py, Line 536 except: EdkLogger.error("UnicodeStringGather", AUTOGEN_ERROR, "SearchString: Error while processing file", File=File, RaiseError=False) raise to except: EdkLogger.error("UnicodeStringGather", AUTOGEN_ERROR, "SearchString: Error while processing file", File=File, RaiseError=True) Thanks, Bob From: devel@edk2.groups.io > On Behalf Of Michael D Kinney Sent: Friday, February 5, 2021 11:30 AM To: Andrew Fish >; edk2-devel-groups-io >; Kinney, Michael D > Cc: Pham, Tony Q > Subject: Re: [edk2-devel] EDK2 Hi Andrew, If the character is part of the code (not a comment), the ignoring the codec error could silently produce the incorrect FW behavior. I prefer a failure with a correct identification of the file/line # so the file can be fixed. The EDK II CI checks will not allow files in with these types of issues. Mike From: Andrew Fish > Sent: Thursday, February 4, 2021 4:11 PM To: edk2-devel-groups-io >; Kinney, Michael D > Cc: Pham, Tony Q > Subject: Re: [edk2-devel] EDK2 On Feb 4, 2021, at 3:58 PM, Michael D Kinney > wrote: Hi Tony, I suspect that one of your UNI files being processed by StrGather has an invalid Unicode character in it. Can you review the UNI files in the module that is being build when this error is generated? I would be better if this code identified the file/line number with the issue, so that would be a good Bugzilla to enter. Mike, I hit something like this too writing some Python. I think I ended up telling the codec to ignore errors, so that might be another option? This I think the issue I saw was in C code. Thanks, Andrew Fish Mike From: devel@edk2.groups.io > On Behalf Of Pham, Tony Q Sent: Thursday, February 4, 2021 1:39 PM To: devel@edk2.groups.io Subject: [edk2-devel] EDK2 Hi, I have a problem with build.py (Python 3.9.1 on win32) Traceback (most recent call last): File "C:\edk2\BaseTools\Source\Python\build\build.py", line 2635, in Main MyBuild.Launch() File "C:\edk2\BaseTools\Source\Python\build\build.py", line 2433, in Launch self._BuildModule() File "C:\edk2\BaseTools\Source\Python\build\build.py", line 1895, in _BuildModule Ma.CreateCodeFile(True) File "C:\edk2\BaseTools\Source\Python\AutoGen\ModuleAutoGen.py", line 1832, in CreateCodeFile for File in self.AutoGenFileList: File "C:\edk2\BaseTools\Source\Python\Common\caching.py", line 28, in __get__ Value = obj.__dict__[self._function.__name__] = self._function(obj) File "C:\edk2\BaseTools\Source\Python\AutoGen\ModuleAutoGen.py", line 983, in AutoGenFileList GenC.CreateCode(self, AutoGenC, AutoGenH, StringH, AutoGenUniIdf, UniStringBinBuffer, StringIdf, AutoGenUniIdf, IdfGenBinBuffer) File "C:\edk2\BaseTools\Source\Python\AutoGen\GenC.py", line 2044, in CreateCode CreateUnicodeStringCode(Info, AutoGenC, StringH, UniGenCFlag, UniGenBinBuffer) File "C:\edk2\BaseTools\Source\Python\AutoGen\GenC.py", line 1706, in CreateUnicodeStringCode Header, Code = GetStringFiles(Info.UnicodeFileList, SrcList, IncList, Info.IncludePathList, ['.uni', '.inf'], Info.Name, CompatibleMode, ShellMode, UniGenCFlag, UniGenBinBuffer, FilterInfo) File "C:\edk2\BaseTools\Source\Python\AutoGen\StrGather.py", line 563, in GetStringFiles Uni = SearchString(Uni, sorted (FileList), IsCompatibleMode) File "C:\edk2\BaseTools\Source\Python\AutoGen\StrGather.py", line 532, in SearchString for Line in Lines: File "C:\Users\tqpham\AppData\Local\Programs\Python\Python39\lib\encodings\cp1252.py", line 23, in decode return codecs.charmap_decode(input,self.errors,decoding_table)[0] UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 5457: character maps to - Failed - Build end time: 13:36:22, Feb.04 2021 Build total time: 00:00:02