Role of DLINK2_FLAGS and link-time option consistency

Role of DLINK2_FLAGS and link-time option consistency

[Marvin Häuser]

Good day everyone,

I've been asked to check the following patch: https://edk2.groups.io/g/devel/message/102168

One thing I noticed was that -no-pie is added to DLINK2_FLAGS, not DLINK_FLAGS. For MSVC toolchains, DLINK2_FLAGS is used for a separate linker step [1] introduced by [2], which makes sense to me. For GCC* and CLANG* toolchains, however, I haven't really been able to understand its purpose. Generally, DLINK_FLAGS and DLINK2_FLAGS are used as part of the same command in the same fashion, e.g., [3]. The main difference I've been able to spot is that the linker step for 16-bit ASM files omits DLINK2_FLAGS, but passes DLINK_FLAGS [4]. As MSVC uses a completely different linker flags macro for these files and XCODE* toolchains use the static linker, it appears to be the semantics of DLINK2_FLAGS are different between toolchains. That's my confusion for the consumer side, I don't understand the exact purpose, or why 16-bit ASM in particular is special here. DLINK2_FLAGS unfortunately seems to date back to the "Sync BaseTools" SVN commits, so I didn't find much with blame.

For the producer side, I noticed --pie is passed for DLINK_FLAGS [5]. On the other hand, the IA32 GCC definitions override this with -no-pie, but in DLINK2_FLAGS (which the patch in question replicates). It's my understand that this means the linker step for 16-bit ASM files will be the only linker step in a IA32 build process to use --pie, while all other steps will have the -no-pie override. For the current codebase, there probably isn't much of a difference. However, for a well-defined build behaviour, don't we need to pass mostly the same linker flags (at the very least PIE-related flags) to all steps in the process? Is there a particular reason for how things work the way they do right now?

I've also tried to look at which kinds of flags are generally passed as DLINK_FLAGS vs DLINK2_FLAGS. For some reason, the linker script and related symbol definitions generally seem to be passed via DLINK2_FLAGS [8] as per [9]. I could imagine a possible rationale could be selective replacement, where a platform can discard-and-reassign DLINK2 to pass its own linker script? I didn't find any evidence for this though. With Pedro, I discussed various other possibilities for what could be the rationale for DLINK2_FLAGS, but pretty much failed. One was, that one of them is meant to be passed to the linker directly, while the other was supposed to be passed to the compiler (CC may be DLINK, or they may be different). However, there is a very inconsistent usage of "-Wl" between the two and LD does not support -Wl.

Another (rather unrelated) oddity we noticed is that CLANGDWARF passes different optimization levels to CC and DLINK2 [8], which I'm neither sure whether it is safe to do so, nor whether CC's -Oz really does much now, considering LTO will be on.

Any help with understanding the current design is appreciated, thanks!

Best regards,
Marvin


[1]
https://github.com/tianocore/edk2/blob/cdf6ff1719a9453351baec4bd32fcfc30e9ceeac/BaseTools/Conf/build_rule.template#L289-L291

[2]
https://github.com/tianocore/edk2/commit/578211b882fa99b1f7c82c3e5fd7eeee1510772c

[3]
https://github.com/tianocore/edk2/blob/cdf6ff1719a9453351baec4bd32fcfc30e9ceeac/BaseTools/Conf/build_rule.template#L297

[4]
https://github.com/tianocore/edk2/blob/cdf6ff1719a9453351baec4bd32fcfc30e9ceeac/BaseTools/Conf/build_rule.template#L503

[5]
https://github.com/tianocore/edk2/blob/cdf6ff1719a9453351baec4bd32fcfc30e9ceeac/BaseTools/Conf/tools_def.template#L1858

[6]
https://github.com/tianocore/edk2/blob/cdf6ff1719a9453351baec4bd32fcfc30e9ceeac/BaseTools/Conf/tools_def.template#L2303

[7]
https://github.com/tianocore/edk2/blob/cdf6ff1719a9453351baec4bd32fcfc30e9ceeac/BaseTools/Conf/tools_def.template#L1859

[8]
https://github.com/tianocore/edk2/blob/cdf6ff1719a9453351baec4bd32fcfc30e9ceeac/BaseTools/Conf/tools_def.template#L2937-L2939

[9]
https://github.com/tianocore/edk2/commit/6d732bbbc2b0463f367ceca381cb1861d52cf735

Re: [edk2-devel] Role of DLINK2_FLAGS and link-time option consistency

[Ard Biesheuvel]

On Fri, 7 Apr 2023 at 12:14, Marvin Häuser <mhaeuser@posteo.de> wrote:
>
> Good day everyone,
>
> I've been asked to check the following patch: https://edk2.groups.io/g/devel/message/102168
>
> One thing I noticed was that -no-pie is added to DLINK2_FLAGS, not DLINK_FLAGS. For MSVC toolchains, DLINK2_FLAGS is used for a separate linker step [1] introduced by [2], which makes sense to me. For GCC* and CLANG* toolchains, however, I haven't really been able to understand its purpose. Generally, DLINK_FLAGS and DLINK2_FLAGS are used as part of the same command in the same fashion, e.g., [3]. The main difference I've been able to spot is that the linker step for 16-bit ASM files omits DLINK2_FLAGS, but passes DLINK_FLAGS [4]. As MSVC uses a completely different linker flags macro for these files and XCODE* toolchains use the static linker, it appears to be the semantics of DLINK2_FLAGS are different between toolchains. That's my confusion for the consumer side, I don't understand the exact purpose, or why 16-bit ASM in particular is special here. DLINK2_FLAGS unfortunately seems to date back to the "Sync BaseTools" SVN commits, so I didn't find much with blame.
>
> For the producer side, I noticed --pie is passed for DLINK_FLAGS [5]. On the other hand, the IA32 GCC definitions override this with -no-pie, but in DLINK2_FLAGS (which the patch in question replicates). It's my understand that this means the linker step for 16-bit ASM files will be the only linker step in a IA32 build process to use --pie, while all other steps will have the -no-pie override. For the current codebase, there probably isn't much of a difference. However, for a well-defined build behaviour, don't we need to pass mostly the same linker flags (at the very least PIE-related flags) to all steps in the process? Is there a particular reason for how things work the way they do right now?
>
> I've also tried to look at which kinds of flags are generally passed as DLINK_FLAGS vs DLINK2_FLAGS. For some reason, the linker script and related symbol definitions generally seem to be passed via DLINK2_FLAGS [8] as per [9]. I could imagine a possible rationale could be selective replacement, where a platform can discard-and-reassign DLINK2 to pass its own linker script? I didn't find any evidence for this though. With Pedro, I discussed various other possibilities for what could be the rationale for DLINK2_FLAGS, but pretty much failed. One was, that one of them is meant to be passed to the linker directly, while the other was supposed to be passed to the compiler (CC may be DLINK, or they may be different). However, there is a very inconsistent usage of "-Wl" between the two and LD does not support -Wl.
>
> Another (rather unrelated) oddity we noticed is that CLANGDWARF passes different optimization levels to CC and DLINK2 [8], which I'm neither sure whether it is safe to do so, nor whether CC's -Oz really does much now, considering LTO will be on.
>
> Any help with understanding the current design is appreciated, thanks!
>

IIRC DLINK2_FLAGS is mainly being used (in the GCC case) for passing
options that need to come after previous occurrences of the same
option with a different value, in order for the DLINK2 one to take
precedence.

Re: [edk2-devel] Role of DLINK2_FLAGS and link-time option consistency

[Marvin Häuser]

[-- Attachment #1: Type: text/plain, Size: 503 bytes --]


> On 7. Apr 2023, at 13:02, Ard Biesheuvel <ardb@kernel.org> wrote:
> 
> IIRC DLINK2_FLAGS is mainly being used (in the GCC case) for passing
> options that need to come after previous occurrences of the same
> option with a different value, in order for the DLINK2 one to take
> precedence.

Thanks for the quick response! By that rationale, it should probably apply to the 16-bit ASM linker step as well and the rest can remain as-is (modulo that weird CLANGDWARF optimization situation)?


[-- Attachment #2: Type: text/html, Size: 3270 bytes --]

Re: [edk2-devel] Role of DLINK2_FLAGS and link-time option consistency

[Ard Biesheuvel]

(cc Ray)

On Fri, 7 Apr 2023 at 13:05, Marvin Häuser <mhaeuser@posteo.de> wrote:
>
>
> On 7. Apr 2023, at 13:02, Ard Biesheuvel <ardb@kernel.org> wrote:
>
> IIRC DLINK2_FLAGS is mainly being used (in the GCC case) for passing
> options that need to come after previous occurrences of the same
> option with a different value, in order for the DLINK2 one to take
> precedence.
>
>
> Thanks for the quick response! By that rationale, it should probably apply to the 16-bit ASM linker step as well and the rest can remain as-is (modulo that weird CLANGDWARF optimization situation)?
>

That build rule seems like dead code to me: we don't have any such
files in the repo, and no references to the .com output files that it
produces, afaict

(The last uses were removed in 1a48fda5315433661c2f3039a30aea5916c22267)

Maybe Ray can comment?

Re: [edk2-devel] Role of DLINK2_FLAGS and link-time option consistency

[Marvin Häuser]

[-- Attachment #1: Type: text/plain, Size: 1534 bytes --]


> On 7. Apr 2023, at 14:00, Ard Biesheuvel <ardb@kernel.org> wrote:
> 
> (cc Ray)
> 
> On Fri, 7 Apr 2023 at 13:05, Marvin Häuser <mhaeuser@posteo.de> wrote:
>> 
>> 
>> On 7. Apr 2023, at 13:02, Ard Biesheuvel <ardb@kernel.org> wrote:
>> 
>> IIRC DLINK2_FLAGS is mainly being used (in the GCC case) for passing
>> options that need to come after previous occurrences of the same
>> option with a different value, in order for the DLINK2 one to take
>> precedence.
>> 
>> 
>> Thanks for the quick response! By that rationale, it should probably apply to the 16-bit ASM linker step as well and the rest can remain as-is (modulo that weird CLANGDWARF optimization situation)?
>> 
> 
> That build rule seems like dead code to me: we don't have any such
> files in the repo

It appears there still is one file remaining:
https://github.com/tianocore/edk2/blob/797f526ae2a83811b0ccbde0138c65a9f137eba5/IntelFsp2Pkg/FspSecCore/Vtf0/Ia16/ResetVec.asm16

But it is not directly referenced, just included from a file with a different extension:
https://github.com/tianocore/edk2/blob/797f526ae2a83811b0ccbde0138c65a9f137eba5/IntelFsp2Pkg/FspSecCore/Vtf0/ResetVectorCode.asm#L11

Being FSP, the most important consumers will probably be downstream, so I will not propose to remove it.


> , and no references to the .com output files that it
> produces, afaict
> 
> (The last uses were removed in 1a48fda5315433661c2f3039a30aea5916c22267)
> 
> Maybe Ray can comment?

Ping? :)

Best regards,
Marvin


[-- Attachment #2: Type: text/html, Size: 2376 bytes --]