Role of DLINK2_FLAGS and link-time option consistency

["Marvin Häuser" <mhaeuser@posteo.de>]

Good day everyone,

I've been asked to check the following patch: https://edk2.groups.io/g/devel/message/102168

One thing I noticed was that -no-pie is added to DLINK2_FLAGS, not DLINK_FLAGS. For MSVC toolchains, DLINK2_FLAGS is used for a separate linker step [1] introduced by [2], which makes sense to me. For GCC* and CLANG* toolchains, however, I haven't really been able to understand its purpose. Generally, DLINK_FLAGS and DLINK2_FLAGS are used as part of the same command in the same fashion, e.g., [3]. The main difference I've been able to spot is that the linker step for 16-bit ASM files omits DLINK2_FLAGS, but passes DLINK_FLAGS [4]. As MSVC uses a completely different linker flags macro for these files and XCODE* toolchains use the static linker, it appears to be the semantics of DLINK2_FLAGS are different between toolchains. That's my confusion for the consumer side, I don't understand the exact purpose, or why 16-bit ASM in particular is special here. DLINK2_FLAGS unfortunately seems to date back to the "Sync BaseTools" SVN commits, so I didn't find much with blame.

For the producer side, I noticed --pie is passed for DLINK_FLAGS [5]. On the other hand, the IA32 GCC definitions override this with -no-pie, but in DLINK2_FLAGS (which the patch in question replicates). It's my understand that this means the linker step for 16-bit ASM files will be the only linker step in a IA32 build process to use --pie, while all other steps will have the -no-pie override. For the current codebase, there probably isn't much of a difference. However, for a well-defined build behaviour, don't we need to pass mostly the same linker flags (at the very least PIE-related flags) to all steps in the process? Is there a particular reason for how things work the way they do right now?

I've also tried to look at which kinds of flags are generally passed as DLINK_FLAGS vs DLINK2_FLAGS. For some reason, the linker script and related symbol definitions generally seem to be passed via DLINK2_FLAGS [8] as per [9]. I could imagine a possible rationale could be selective replacement, where a platform can discard-and-reassign DLINK2 to pass its own linker script? I didn't find any evidence for this though. With Pedro, I discussed various other possibilities for what could be the rationale for DLINK2_FLAGS, but pretty much failed. One was, that one of them is meant to be passed to the linker directly, while the other was supposed to be passed to the compiler (CC may be DLINK, or they may be different). However, there is a very inconsistent usage of "-Wl" between the two and LD does not support -Wl.

Another (rather unrelated) oddity we noticed is that CLANGDWARF passes different optimization levels to CC and DLINK2 [8], which I'm neither sure whether it is safe to do so, nor whether CC's -Oz really does much now, considering LTO will be on.

Any help with understanding the current design is appreciated, thanks!

Best regards,
Marvin


[1]
https://github.com/tianocore/edk2/blob/cdf6ff1719a9453351baec4bd32fcfc30e9ceeac/BaseTools/Conf/build_rule.template#L289-L291

[2]
https://github.com/tianocore/edk2/commit/578211b882fa99b1f7c82c3e5fd7eeee1510772c

[3]
https://github.com/tianocore/edk2/blob/cdf6ff1719a9453351baec4bd32fcfc30e9ceeac/BaseTools/Conf/build_rule.template#L297

[4]
https://github.com/tianocore/edk2/blob/cdf6ff1719a9453351baec4bd32fcfc30e9ceeac/BaseTools/Conf/build_rule.template#L503

[5]
https://github.com/tianocore/edk2/blob/cdf6ff1719a9453351baec4bd32fcfc30e9ceeac/BaseTools/Conf/tools_def.template#L1858

[6]
https://github.com/tianocore/edk2/blob/cdf6ff1719a9453351baec4bd32fcfc30e9ceeac/BaseTools/Conf/tools_def.template#L2303

[7]
https://github.com/tianocore/edk2/blob/cdf6ff1719a9453351baec4bd32fcfc30e9ceeac/BaseTools/Conf/tools_def.template#L1859

[8]
https://github.com/tianocore/edk2/blob/cdf6ff1719a9453351baec4bd32fcfc30e9ceeac/BaseTools/Conf/tools_def.template#L2937-L2939

[9]
https://github.com/tianocore/edk2/commit/6d732bbbc2b0463f367ceca381cb1861d52cf735