Did you recently upgrade to Python 3.9. Are you using the latest version of BaseTools?

I see an issue that was fixed in BaseTools in August 2020 that may be related.

https://edk2.groups.io/g/devel/message/64002

https://github.com/tianocore/edk2/commit/5d8648345cd9ad729837118c731063c59edea192

Mike

From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Ajinkya Bhavalkar
Sent: Tuesday, September 27, 2022 5:51 AM
To: devel@edk2.groups.io
Cc: Siddalingesh Rajashekhar <srajashe@qti.qualcomm.com>; Shiv Kumar <shikum@qti.qualcomm.com>
Subject: [edk2-devel] MK BOOT.MXF.1.1.c1-00035-MAKENA-1 compilation is failing

Can someone please help with this.

This issue is observed on all our machines.

Command to compile:
python -u boot_images/boot_tools/buildex.py -t Makena -v WP -r DEBUG

Logs:

testSurrogatePairUnicodeCharInUtf8FileWithBom (CheckUnicodeSourceFiles.Tests) ... ok

testUtf16InUniFile (CheckUnicodeSourceFiles.Tests) ... FAIL

testValidUtf8File (CheckUnicodeSourceFiles.Tests) ... FAIL

testValidUtf8FileWithBom (CheckUnicodeSourceFiles.Tests) ... FAIL

======================================================================

FAIL: testUtf16InUniFile (CheckUnicodeSourceFiles.Tests)

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Source/Python/AutoGen/UniClassObject.py", line 375, in PreProcess

    FileIn = UniFileClassObject.OpenUniFile(LongFilePath(File.Path))

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Source/Python/AutoGen/UniClassObject.py", line 303, in OpenUniFile

    UniFileClassObject.VerifyUcs2Data(FileIn, FileName, Encoding)

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Source/Python/AutoGen/UniClassObject.py", line 312, in VerifyUcs2Data

    Ucs2Info = codecs.lookup('ucs-2')

LookupError: unknown encoding: ucs-2

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Tests/CheckUnicodeSourceFiles.py", line 60, in CheckFile

    BtUni.UniFileClassObject([path])

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Source/Python/AutoGen/UniClassObject.py", line 230, in __init__

    self.LoadUniFiles(FileList)

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Source/Python/AutoGen/UniClassObject.py", line 542, in LoadUniFiles

    self.LoadUniFile(File)

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Source/Python/AutoGen/UniClassObject.py", line 456, in LoadUniFile

    Lines = self.PreProcess(File)

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Source/Python/AutoGen/UniClassObject.py", line 381, in PreProcess

    EdkLogger.error("build", FILE_OPEN_FAILURE, ExtraData=File.Path);

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Source/Python/Common/EdkLogger.py", line 284, in error

    raise FatalError(ErrorCode)

Common.BuildToolError.FatalError: 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Tests/CheckUnicodeSourceFiles.py", line 79, in testUtf16InUniFile

    self.CheckFile('utf_16', shouldPass=True)

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Tests/CheckUnicodeSourceFiles.py", line 72, in CheckFile

    self.EdkErrorFailure(encoding, shouldPass)

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Tests/CheckUnicodeSourceFiles.py", line 55, in EdkErrorFailure

    self.ErrorFailure('EdkLogger.FatalError', encoding, shouldPass)

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Tests/CheckUnicodeSourceFiles.py", line 49, in ErrorFailure

    self.fail(msg)

AssertionError: EdkLogger.FatalError should not be generated for utf_16 data in a .uni file

======================================================================

FAIL: testValidUtf8File (CheckUnicodeSourceFiles.Tests)

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Source/Python/AutoGen/UniClassObject.py", line 375, in PreProcess

    FileIn = UniFileClassObject.OpenUniFile(LongFilePath(File.Path))

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Source/Python/AutoGen/UniClassObject.py", line 303, in OpenUniFile

    UniFileClassObject.VerifyUcs2Data(FileIn, FileName, Encoding)

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Source/Python/AutoGen/UniClassObject.py", line 312, in VerifyUcs2Data

    Ucs2Info = codecs.lookup('ucs-2')

LookupError: unknown encoding: ucs-2

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Tests/CheckUnicodeSourceFiles.py", line 60, in CheckFile

    BtUni.UniFileClassObject([path])

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Source/Python/AutoGen/UniClassObject.py", line 230, in __init__

    self.LoadUniFiles(FileList)

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Source/Python/AutoGen/UniClassObject.py", line 542, in LoadUniFiles

    self.LoadUniFile(File)

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Source/Python/AutoGen/UniClassObject.py", line 456, in LoadUniFile

    Lines = self.PreProcess(File)

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Source/Python/AutoGen/UniClassObject.py", line 381, in PreProcess

    EdkLogger.error("build", FILE_OPEN_FAILURE, ExtraData=File.Path);

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Source/Python/Common/EdkLogger.py", line 284, in error

    raise FatalError(ErrorCode)

Common.BuildToolError.FatalError: 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Tests/CheckUnicodeSourceFiles.py", line 112, in testValidUtf8File

    self.CheckFile(encoding='utf_8', shouldPass=True)

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Tests/CheckUnicodeSourceFiles.py", line 72, in CheckFile

    self.EdkErrorFailure(encoding, shouldPass)

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Tests/CheckUnicodeSourceFiles.py", line 55, in EdkErrorFailure

    self.ErrorFailure('EdkLogger.FatalError', encoding, shouldPass)

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Tests/CheckUnicodeSourceFiles.py", line 49, in ErrorFailure

    self.fail(msg)

AssertionError: EdkLogger.FatalError should not be generated for utf_8 data in a .uni file

======================================================================

FAIL: testValidUtf8FileWithBom (CheckUnicodeSourceFiles.Tests)

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Source/Python/AutoGen/UniClassObject.py", line 375, in PreProcess

    FileIn = UniFileClassObject.OpenUniFile(LongFilePath(File.Path))

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Source/Python/AutoGen/UniClassObject.py", line 303, in OpenUniFile

    UniFileClassObject.VerifyUcs2Data(FileIn, FileName, Encoding)

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Source/Python/AutoGen/UniClassObject.py", line 312, in VerifyUcs2Data

    Ucs2Info = codecs.lookup('ucs-2')

LookupError: unknown encoding: ucs-2

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Tests/CheckUnicodeSourceFiles.py", line 60, in CheckFile

    BtUni.UniFileClassObject([path])

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Source/Python/AutoGen/UniClassObject.py", line 230, in __init__

    self.LoadUniFiles(FileList)

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Source/Python/AutoGen/UniClassObject.py", line 542, in LoadUniFiles

    self.LoadUniFile(File)

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Source/Python/AutoGen/UniClassObject.py", line 456, in LoadUniFile

    Lines = self.PreProcess(File)

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Source/Python/AutoGen/UniClassObject.py", line 381, in PreProcess

    EdkLogger.error("build", FILE_OPEN_FAILURE, ExtraData=File.Path);

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Source/Python/Common/EdkLogger.py", line 284, in error

    raise FatalError(ErrorCode)

Common.BuildToolError.FatalError: 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Tests/CheckUnicodeSourceFiles.py", line 120, in testValidUtf8FileWithBom

    self.CheckFile(encoding=None, shouldPass=True, string=data)

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Tests/CheckUnicodeSourceFiles.py", line 72, in CheckFile

    self.EdkErrorFailure(encoding, shouldPass)

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Tests/CheckUnicodeSourceFiles.py", line 55, in EdkErrorFailure

    self.ErrorFailure('EdkLogger.FatalError', encoding, shouldPass)

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Tests/CheckUnicodeSourceFiles.py", line 49, in ErrorFailure

    self.fail(msg)

AssertionError: EdkLogger.FatalError should not be generated for None data in a .uni file

----------------------------------------------------------------------

Ran 282 tests in 1.492s

FAILED (failures=3)

make[1]: Leaving directory '/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Tests'

make: Leaving directory '/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools'

Done

[buildex.py] Generating image version file...

Generating /local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/boot/QcomPkg/Library/ImageVersionLib/qc_version.c

Done

[buildex.py] Building with arguments...

['/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Source/Python/build/build.py', '-p', 'QcomPkg/SocPkg/Makena/WP/Core.dsc', '-j', 'QcomPkg/SocPkg/Makena/WP/build_Core.log', '-w', '-a', 'AARCH64', '-b', 'DEBUG', '-t', 'CLANG100LINUX', '-D', 'BUILDROOT=/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images', '-D', 'BOOTROOT=/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/boot', '-D', 'REL=DEBUG', '-D', 'PRODMODE=DEBUGMODE', '-D', 'VAR=WP', '-D', 'COMPLIER=', '-D', 'FIRSTDLL=', '-D', 'SECONDDLL=', '-D', 'XBLSEC=', '-D', 'TARGETROOT=QcomPkg/SocPkg/Makena', '-D', 'TARGETID=SocMakena', '-D', 'TARGETPKG=SocPkg', '-D', 'QCOMDIR=', '-D', 'BOOTTOOLS=/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/boot_tools', '-D', 'EDK2ROOT=/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2', '-D', 'SECTOOLS_DIR=/pkg/sectools/glue/latest', '-D', 'SECTOOLSROOT=/pkg/sectools/glue/latest', '-D', 'COMPILER=CLANG100LINUX', '-D', 'AARCH=AARCH64']

Build environment: Linux-5.4.0-120-generic-x86_64-with-glibc2.27

Build start time: 18:17:25, Sep.27 2022

WORKSPACE        = /local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/boot

PACKAGES_PATH    = /local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/boot:/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/boot_tools:/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2:/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/sdk:/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/ssg

EDK_TOOLS_PATH   = /local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools

CONF_PATH        = /local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/Conf

Processing meta-data .

Architecture(s)  = AARCH64

Build target     = DEBUG

Toolchain        = CLANG100LINUX

Active Platform          = /local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/boot/QcomPkg/SocPkg/Makena/WP/Core.dsc

............Traceback (most recent call last):

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Source/Python/AutoGen/UniClassObject.py", line 375, in PreProcess

    FileIn = UniFileClassObject.OpenUniFile(LongFilePath(File.Path))

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Source/Python/AutoGen/UniClassObject.py", line 303, in OpenUniFile

    UniFileClassObject.VerifyUcs2Data(FileIn, FileName, Encoding)

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Source/Python/AutoGen/UniClassObject.py", line 312, in VerifyUcs2Data

    Ucs2Info = codecs.lookup('ucs-2')

LookupError: unknown encoding: ucs-2

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Source/Python/AutoGen/AutoGenWorker.py", line 268, in run

    Ma.CreateCodeFile(False)

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py", line 1846, in CreateCodeFile

    for File in self.AutoGenFileList:

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Source/Python/Common/caching.py", line 28, in __get__

    Value = obj.__dict__[self._function.__name__] = self._function(obj)

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py", line 983, in AutoGenFileList

    GenC.CreateCode(self, AutoGenC, AutoGenH, StringH, AutoGenUniIdf, UniStringBinBuffer, StringIdf, AutoGenUniIdf, IdfGenBinBuffer)

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Source/Python/AutoGen/GenC.py", line 2048, in CreateCode

    CreateUnicodeStringCode(Info, AutoGenC, StringH, UniGenCFlag, UniGenBinBuffer)

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Source/Python/AutoGen/GenC.py", line 1710, in CreateUnicodeStringCode

    Header, Code = GetStringFiles(Info.UnicodeFileList, SrcList, IncList, Info.IncludePathList, ['.uni', '.inf'], Info.Name, CompatibleMode, ShellMode, UniGenCFlag, UniGenBinBuffer, FilterInfo)

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Source/Python/AutoGen/StrGather.py", line 557, in GetStringFiles

    Uni = UniFileClassObject(sorted(UniFilList, key=lambda x: x.File), IsCompatibleMode, IncludePathList)

 File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Source/Python/AutoGen/UniClassObject.py", line 230, in __init__

    self.LoadUniFiles(FileList)

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Source/Python/AutoGen/UniClassObject.py", line 542, in LoadUniFiles

    self.LoadUniFile(File)

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Source/Python/AutoGen/UniClassObject.py", line 456, in LoadUniFile

    Lines = self.PreProcess(File)

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Source/Python/AutoGen/UniClassObject.py", line 381, in PreProcess

    EdkLogger.error("build", FILE_OPEN_FAILURE, ExtraData=File.Path);

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/BaseTools/Source/Python/Common/EdkLogger.py", line 284, in error

    raise FatalError(ErrorCode)

Common.BuildToolError.FatalError: 1

build.py...

: error 0001: File open failure

        /local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/edk2/NetworkPkg/Ip4Dxe/Ip4DxeStrings.uni

- Done -

Build end time: 18:17:38, Sep.27 2022

Build total time: 00:00:13

['python', '/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/boot_tools/image_header.py'] ['/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/Build/MakenaWP/Core/DEBUG_CLANG100LINUX/FV/FVMAIN_COMPACT.Fv', '/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/Build/MakenaWP/Core/DEBUG_CLANG100LINUX/FV/uefi.elf', '0x9F000000', 'elf', '64']

Traceback (most recent call last):

  File "/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/boot_tools/image_header.py", line 75, in <module>

    image_size = os.stat(source_full).st_size

FileNotFoundError: [Errno 2] No such file or directory: '/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/Build/MakenaWP/Core/DEBUG_CLANG100LINUX/FV/FVMAIN_COMPACT.Fv'

Traceback (most recent call last):

  File "boot_images/boot_tools/buildex.py", line 757, in process_component

    subprocess.check_output(tool_cmd + build_cmd)

  File "/usr/local/lib/python3.8/subprocess.py", line 411, in check_output

    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,

  File "/usr/local/lib/python3.8/subprocess.py", line 512, in run

    raise CalledProcessError(retcode, process.args,

subprocess.CalledProcessError: Command '['python', '/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/boot_tools/image_header.py', '/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/Build/MakenaWP/Core/DEBUG_CLANG100LINUX/FV/FVMAIN_COMPACT.Fv', '/local/mnt/workspace/Ajinkya/LPI_MK_XBL/boot_images/Build/MakenaWP/Core/DEBUG_CLANG100LINUX/FV/uefi.elf', '0x9F000000', 'elf', '64']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "boot_images/boot_tools/buildex.py", line 1741, in <module>

    main()

  File "boot_images/boot_tools/buildex.py", line 500, in main

    create_image(options,variant_flavor,build_config,tools_config)

  File "boot_images/boot_tools/buildex.py", line 574, in create_image

    dll_paths=process_components(options,variant_flavor,process_action,build_config,tools_config,merge_dict)

  File "boot_images/boot_tools/buildex.py", line 662, in process_components

    bin_path=process_component(options,variant_flavor,component,process_config,build_config,tools_config,merge_dict)

  File "boot_images/boot_tools/buildex.py", line 759, in process_component

    print("ERROR: buildex: execute: " ,tool_cmd , build_cmd + \

TypeError: can only concatenate list (not "str") to list