Hi Timothy,
This is an interesting bug report because it looks like something that should be caught by
EDK II CI
MdeModulePkg.dsc
does list the Brotli lib in a [Components] section, so it will always be built.
[Components.IA32, Components.X64,
Components.ARM, Components.AARCH64]
MdeModulePkg/Library/BrotliCustomDecompressLib/BrotliCustomDecompressLib.inf
MdeModulePkg/Library/LzmaCustomDecompressLib/LzmaCustomDecompressLib.inf
MdeModulePkg/Library/VarCheckUefiLib/VarCheckUefiLib.inf
MdeModulePkg/Core/Dxe/DxeMain.inf {
<LibraryClasses>
NULL|MdeModulePkg/Library/DxeCrc32GuidedSectionExtractLib/DxeCrc32GuidedSectionExtractLib.inf
}
However, a lib is allowed to have unresolved externals such as calling other libraries.
So unless the library is linked to a module, we do
not know if all the externals are resolved.
I am wondering if we need to expand package DSC builds to make sure all libraries that are built are also used in
at least one module with that package build.
You were able to produce this failure by attempting to use
Brotli with the DxeIpl module, so the unresolved
externals that are resolved by the missing source files were not noticed until this used case.
For example MdeModulePkg.dsc
could be extended to
DxeIpl with different NULL library instances for the different decompressors.
Mike
From: Lin, Kuei-Hung (Timothy) <Kuei-Hung.Lin@amd.com>
Sent: Monday, January 30, 2023 4:50 PM
To: Gao, Liming <gaoliming@byosoft.com.cn>; devel@edk2.groups.io; He, Jiangang <Jiangang.He@amd.com>; Chang, Abner <Abner.Chang@amd.com>; Kinney, Michael D <michael.d.kinney@intel.com>
Cc: Bi, Dandan <dandan.bi@intel.com>; Kirkendall, Garrett <Garrett.Kirkendall@amd.com>
Subject: Re: [edk2-devel] [PATCH V2] MdeModulePkg/Decompress: Add missing source file to Brotli library
[AMD Official Use Only - General]
The failure can be replicated with VS2019 and GCC5.
/usr/bin/ld: /tmp/cc7sinRE.ltrans0.ltrans.o: in function `DecodeLiteralBlockSwitch':
/home/XXX/edk2-stable202211/MdeModulePkg/Library/BrotliCustomDecompressLib/brotli/c/dec/decode.c:879: undefined reference to `_kBrotliPrefixCodeRanges'
/usr/bin/ld: /home/XXX/edk2-stable202211/MdeModulePkg/Library/BrotliCustomDecompressLib/brotli/c/dec/decode.c:1193: undefined reference to `_kBrotliContextLookupTable'
/usr/bin/ld: /tmp/cc7sinRE.ltrans0.ltrans.o: in function `DecodeCommandBlockSwitch':
/home/XXX/edk2-stable202211/MdeModulePkg/Library/BrotliCustomDecompressLib/brotli/c/dec/decode.c:879: undefined reference to `_kBrotliPrefixCodeRanges'
/usr/bin/ld: /tmp/cc7sinRE.ltrans0.ltrans.o: in function `DecodeDistanceBlockSwitch':
/home/XXX/edk2-stable202211/MdeModulePkg/Library/BrotliCustomDecompressLib/brotli/c/dec/decode.c:879: undefined reference to `_kBrotliPrefixCodeRanges'
/usr/bin/ld: /tmp/cc7sinRE.ltrans0.ltrans.o: in function `SafeDecodeLiteralBlockSwitch':
/home/XXX/edk2-stable202211/MdeModulePkg/Library/BrotliCustomDecompressLib/brotli/c/dec/decode.c:898: undefined reference to `_kBrotliPrefixCodeRanges'
/usr/bin/ld: /home/XXX/edk2-stable202211/MdeModulePkg/Library/BrotliCustomDecompressLib/brotli/c/dec/decode.c:1193: undefined reference to `_kBrotliContextLookupTable'
/usr/bin/ld: /tmp/cc7sinRE.ltrans0.ltrans.o: in function `SafeDecodeCommandBlockSwitch':
/home/XXX/edk2-stable202211/MdeModulePkg/Library/BrotliCustomDecompressLib/brotli/c/dec/decode.c:898: undefined reference to `_kBrotliPrefixCodeRanges'
/usr/bin/ld: /tmp/cc7sinRE.ltrans0.ltrans.o: in function `SafeDecodeDistanceBlockSwitch':
/home/XXX/edk2-stable202211/MdeModulePkg/Library/BrotliCustomDecompressLib/brotli/c/dec/decode.c:898: undefined reference to `_kBrotliPrefixCodeRanges'
/usr/bin/ld: /tmp/cc7sinRE.ltrans0.ltrans.o: in function `BrotliDecoderDestroyInstance':
/home/XXX/edk2-stable202211/MdeModulePkg/Library/BrotliCustomDecompressLib/brotli/c/dec/state.c:136: undefined reference to `BrotliSharedDictionaryDestroyInstance'
/usr/bin/ld: /tmp/cc7sinRE.ltrans0.ltrans.o: in function `BrotliUefiDecompress':
/home/XXX/edk2-stable202211/MdeModulePkg/Library/BrotliCustomDecompressLib/brotli/c/dec/state.c:87: undefined reference to `BrotliSharedDictionaryCreateInstance'
/usr/bin/ld: /tmp/cc7sinRE.ltrans0.ltrans.o: in function `BrotliUefiDecompress':
/home/XXX/edk2-stable202211/MdeModulePkg/Library/BrotliCustomDecompressLib/brotli/c/dec/decode.c:898: undefined reference to `_kBrotliPrefixCodeRanges'
/usr/bin/ld: /home/XXX/edk2-stable202211/MdeModulePkg/Library/BrotliCustomDecompressLib/brotli/c/dec/decode.c:1193: undefined reference to `_kBrotliContextLookupTable'
/usr/bin/ld: warning: creating DT_TEXTREL in a PIE
collect2: error: ld returned 1 exit status
...
build.py...
: error F002: Failed to build module
/home/XXX/edk2-stable202211/MdeModulePkg/Core/DxeIplPeim/DxeIpl.inf [X64, GCC5, RELEASE]
BrotliDecompressLib.lib(state.obj) : error LNK2001: unresolved external symbol BrotliDefaultAllocFunc
BrotliDecompressLib.lib(state.obj) : error LNK2001: unresolved external symbol BrotliSharedDictionaryCreateInstance
BrotliDecompressLib.lib(state.obj) : error LNK2001: unresolved external symbol BrotliSharedDictionaryDestroyInstance
BrotliDecompressLib.lib(decode.obj) : error LNK2001: unresolved external symbol _kBrotliPrefixCodeRanges
BrotliDecompressLib.lib(decode.obj) : error LNK2001: unresolved external symbol BrotliSharedDictionaryAttach
BrotliDecompressLib.lib(decode.obj) : error LNK2001: unresolved external symbol _kBrotliContextLookupTable
d:\XXX\Build\OvmfX64\RELEASE_VS2019\X64\MdeModulePkg\Core\DxeIplPeim\DxeIpl\DEBUG\DxeIpl.dll : fatal error
LNK1120: 7 unresolved externals
...
build.py...
: error 7000: Failed to execute command
C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\MSVC\14.29.30133\bin\Hostx86\x86\nmake.exe
/nologo tbuild [d:\XXX\Build\OvmfX64\RELEASE_VS2019\X64\MdeModulePkg\Core\DxeIplPeim\DxeIpl]
--
Timothy
From: gaoliming <gaoliming@byosoft.com.cn>
Sent: Saturday, January 28, 2023 11:35 PM
To: devel@edk2.groups.io <devel@edk2.groups.io>; Lin, Kuei-Hung (Timothy) <Kuei-Hung.Lin@amd.com>; He, Jiangang <Jiangang.He@amd.com>;
Chang, Abner <Abner.Chang@amd.com>; 'Kinney, Michael D' <michael.d.kinney@intel.com>
Cc: 'Bi, Dandan' <dandan.bi@intel.com>; Kirkendall, Garrett <Garrett.Kirkendall@amd.com>
Subject: 回复: [edk2-devel] [PATCH V2] MdeModulePkg/Decompress: Add missing source file
to Brotli library
|
Caution: This message originated from an External Source. Use proper caution when opening attachments,
clicking links, or responding. |
Timothy:
Thanks for your report. This is a real issue in Brotli decompress library. Which tool chain has been verified? GCC5 or VS2019 or Both?
Thanks
Liming
发件人:
devel@edk2.groups.io <devel@edk2.groups.io>
代表 Lin, Kuei-Hung (Timothy) via groups.io
发送时间: 2023年1月13日 6:18
收件人: He, Jiangang <Jiangang.He@amd.com>; Chang, Abner <Abner.Chang@amd.com>; Kinney, Michael D <michael.d.kinney@intel.com>;
devel@edk2.groups.io
抄送: Bi, Dandan <dandan.bi@intel.com>; Gao, Liming <gaoliming@byosoft.com.cn>; Kirkendall, Garrett <Garrett.Kirkendall@amd.com>
主题: Re: [edk2-devel] [PATCH V2] MdeModulePkg/Decompress: Add missing source file to Brotli library
[AMD Official Use Only - General]
This patch tried to fix following build failure when linking the Brotli decompress lib.
Steps to replicate the issue:
1 - Attach a NULL lib to DxeIpl.inf in any project's .DSC, say OvmfPkgX64.dsc
MdeModulePkg/Core/DxeIplPeim/DxeIpl.inf {
<LibraryClasses>
NULL|MdeModulePkg/Library/BrotliCustomDecompressLib/BrotliCustomDecompressLib.inf
}
2 - Follow the EDKII'2 package build steps to build that project. Build.py would compliains with following errors.
...
/home/edk2/edk2-stable202211/MdeModulePkg/Library/BrotliCustomDecompressLib/brotli/c/dec/decode.c:879:
undefined reference to `_kBrotliPrefixCodeRanges'
/usr/bin/ld: /home/edk2/edk2-stable202211/MdeModulePkg/Library/BrotliCustomDecompressLib/brotli/c/dec/decode.c:1193:
undefined reference to `_kBrotliContextLookupTable'
/usr/bin/ld: /tmp/ccKzzHN0.ltrans0.ltrans.o: in function `DecodeCommandBlockSwitch':
...
/usr/bin/ld: /tmp/ccKzzHN0.ltrans0.ltrans.o: in function `BrotliUefiDecompress':
/home/edk2/edk2-stable202211/MdeModulePkg/Library/BrotliCustomDecompressLib/brotli/c/dec/decode.c:898:
undefined reference to `_kBrotliPrefixCodeRanges'
/usr/bin/ld: /home/edk2/edk2-stable202211/MdeModulePkg/Library/BrotliCustomDecompressLib/brotli/c/dec/decode.c:1193:
undefined reference to `_kBrotliContextLookupTable'
--
Timothy
From: He, Jiangang <Jiangang.He@amd.com>
Sent: Thursday, January 12, 2023 8:08 AM
To: Chang, Abner <Abner.Chang@amd.com>; Kinney, Michael D <michael.d.kinney@intel.com>;
devel@edk2.groups.io <devel@edk2.groups.io>; Lin, Kuei-Hung (Timothy) <Kuei-Hung.Lin@amd.com>
Cc: Bi, Dandan <dandan.bi@intel.com>; Gao, Liming <gaoliming@byosoft.com.cn>; Kirkendall, Garrett <Garrett.Kirkendall@amd.com>;
Kinney, Michael D <michael.d.kinney@intel.com>
Subject: RE: [edk2-devel] [PATCH V2] MdeModulePkg/Decompress: Add missing source file to Brotli library
[AMD Official Use Only - General]
Timothy will answer the question.
From: Chang, Abner <Abner.Chang@amd.com>
Sent: Thursday, January 12, 2023 10:05 AM
To: Kinney, Michael D <michael.d.kinney@intel.com>;
devel@edk2.groups.io
Cc: Bi, Dandan <dandan.bi@intel.com>; Gao, Liming <gaoliming@byosoft.com.cn>; Kirkendall, Garrett <Garrett.Kirkendall@amd.com>;
Kinney, Michael D <michael.d.kinney@intel.com>; He, Jiangang <Jiangang.He@amd.com>
Subject: Re: [edk2-devel] [PATCH V2] MdeModulePkg/Decompress: Add missing source file to Brotli library
[AMD Official Use Only - General]
Hi Mike,
CC Jiangang, he can give the clear answer.
Thanks
Abner
From: Kinney, Michael D <michael.d.kinney@intel.com>
Sent: Thursday, January 12, 2023 11:23:48 PM
To: devel@edk2.groups.io <devel@edk2.groups.io>; Chang, Abner <Abner.Chang@amd.com>
Cc: Bi, Dandan <dandan.bi@intel.com>; Gao, Liming <gaoliming@byosoft.com.cn>; Kirkendall, Garrett <Garrett.Kirkendall@amd.com>;
Kinney, Michael D <michael.d.kinney@intel.com>
Subject: RE: [edk2-devel] [PATCH V2] MdeModulePkg/Decompress: Add missing source file to Brotli library
Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.
Hi Abner,
What was the failure before this change?
This looks like something that would always break the build of source files were missing.
Mike
> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Chang, Abner via groups.io
> Sent: Wednesday, January 11, 2023 9:08 PM
> To: devel@edk2.groups.io
> Cc: Bi, Dandan <dandan.bi@intel.com>; Gao, Liming <gaoliming@byosoft.com.cn>; Garrett Kirkendall <garrett.kirkendall@amd.com>;
> Abner Chang <abner.chang@amd.com>
> Subject: [edk2-devel] [PATCH V2] MdeModulePkg/Decompress: Add missing source file to Brotli library
>
> From: Abner Chang <abner.chang@amd.com>
>
> Add missing source files to Brotli library INF.
>
> Signed-off-by: Jiangang He <jiangang.he@amd.com>
> Cc: Dandan Bi <dandan.bi@intel.com>
> Cc: Liming Gao <gaoliming@byosoft.com.cn>
> Cc: Garrett Kirkendall <garrett.kirkendall@amd.com>
> Cc: Abner Chang <abner.chang@amd.com>
> ---
> .../BrotliCustomDecompressLib/BrotliCustomDecompressLib.inf | 5 +++++
> 1 file changed, 5 insertions(+)
>
> diff --git a/MdeModulePkg/Library/BrotliCustomDecompressLib/BrotliCustomDecompressLib.inf
> b/MdeModulePkg/Library/BrotliCustomDecompressLib/BrotliCustomDecompressLib.inf
> index 525e92408d6..244c2e35023 100644
> --- a/MdeModulePkg/Library/BrotliCustomDecompressLib/BrotliCustomDecompressLib.inf
> +++ b/MdeModulePkg/Library/BrotliCustomDecompressLib/BrotliCustomDecompressLib.inf
> @@ -5,6 +5,7 @@
> # Brotli was released on the website
https://github.com/google/brotli.
> #
> # Copyright (c) 2017 - 2020, Intel Corporation. All rights reserved.<BR>
> +# Copyright (C) 2023 Advanced Micro Devices, Inc. All rights reserved.<BR>
> #
> # SPDX-License-Identifier: BSD-2-Clause-Patent
> #
> @@ -41,6 +42,10 @@
> # Wrapper header files end #
> brotli/c/common/dictionary.c
> brotli/c/common/transform.c
> + brotli/c/common/context.c
> + brotli/c/common/platform.c
> + brotli/c/common/constants.c
> + brotli/c/common/shared_dictionary.c
> brotli/c/dec/bit_reader.c
> brotli/c/dec/decode.c
> brotli/c/dec/huffman.c
> --
> 2.37.1.windows.1
>
>
>
>
>