[edk2-devel] Expansion of ARCH in .dsc files

[edk2-devel] Expansion of ARCH in .dsc files

[Leif Lindholm via groups.io]

So... I originally tripped over my misunderstanding of <subject> when writing
https://github.com/tianocore/edk2/pull/6262 to get around something I
wanted to test in UefiPayloadPkg.

Now Ajan has run into at least an adjacent problem when trying to rework
https://github.com/tianocore/edk2/pull/6545 based on feedback I among
others provided.

Now, the thing I had misunderstood before is that when build is called
with multiple -a options,
the ARCH variable will expand to a space-separated list of *all* the
specified architectures.
A lot of Arm/AArch64 modules assume this variable only ever contains a
single architecture.

The thing is, I don't think this is clearly documented in the .dsc
specification at all. And it is
frequently used as a shorthand in there in a form that would only work
if build was called with
a single -a.
Now, this also means that there are nearly no correct users of ARCH in
edk2 or edk2-platforms.
Because "the list of all architectures being built simultaneously"
simply isn't very useful information
build-time. There are two correct uses in edk2, in EmulatorPkg.dsc and
in UefiPayloadPkg.dsc,
using '!if "IA32" in "$ARCH"', which as far as I can tell is the only
possible way to use the variable
correctly.
Slightly amusingly, EmulatorPkg.dsc also contains an incorrect use.

I guess the behaviour is a direct result of how the build command is
architected.
I guess the one question I have is - is this the intended behaviour?

Ard: I think we need to update some of our .dsc files...

Ajan: one somewhat hacky workaround I can think of to separate the
output images is to hardcode
the output image name in arch-specific Defines sections, like so:

diff --git a/ArmVirtPkg/ArmVirtQemu.dsc b/ArmVirtPkg/ArmVirtQemu.dsc
index 0bd8d85cab03..f07f2b91764e 100644
--- a/ArmVirtPkg/ArmVirtQemu.dsc
+++ b/ArmVirtPkg/ArmVirtQemu.dsc
@@ -12,12 +12,17 @@
 # Defines Section - statements that will be processed to create a Makefile.
 #
 ################################################################################
+[Defines.ARM]
+  OUTPUT_DIRECTORY               = Build/ArmVirtQemu-ARM
+
+[Defines.AARCH64]
+  OUTPUT_DIRECTORY               = Build/ArmVirtQemu-AARCH64
+
 [Defines]
   PLATFORM_NAME                  = ArmVirtQemu
   PLATFORM_GUID                  = 37d7e986-f7e9-45c2-8067-e371421a626c
   PLATFORM_VERSION               = 0.1
   DSC_SPECIFICATION              = 0x00010005
-  OUTPUT_DIRECTORY               = Build/ArmVirtQemu-$(ARCH)
   SUPPORTED_ARCHITECTURES        = AARCH64|ARM
   BUILD_TARGETS                  = DEBUG|RELEASE|NOOPT
   SKUID_IDENTIFIER               = DEFAULT


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#120967): https://edk2.groups.io/g/devel/message/120967
Mute This Topic: https://groups.io/mt/110483263/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] Expansion of ARCH in .dsc files

[Joey Vagedes via groups.io]

Leif,

For your awareness, Section 3.4 of the INF specification states that architectural modifiers for the [Defines] section tag are not permitted.

So that you can Ctrl+F and to find the exact language, here is a quote:

"architectural modifiers for the [Defines] section tag are not permitted"

https://tianocore-docs.github.io/edk2-InfSpecification/release-1.27/3_edk_ii_inf_file_format/34_[defines]_section.html#34-defines-section

Thanks,
Joey

-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Leif Lindholm via groups.io
Sent: Tuesday, January 7, 2025 11:05 AM
To: devel@edk2.groups.io
Cc: Ajan Zhong <ajan.zhong@newfw.com>; Ard Biesheuvel <ardb+tianocore@kernel.org>; Kinney, Michael D <michael.d.kinney@intel.com>
Subject: [EXTERNAL] [edk2-devel] Expansion of ARCH in .dsc files

So... I originally tripped over my misunderstanding of <subject> when writing
https://github.com/tianocore/edk2/pull/6262 to get around something I wanted to test in UefiPayloadPkg.

Now Ajan has run into at least an adjacent problem when trying to rework
https://github.com/tianocore/edk2/pull/6545 based on feedback I among others provided.

Now, the thing I had misunderstood before is that when build is called with multiple -a options, the ARCH variable will expand to a space-separated list of *all* the specified architectures.
A lot of Arm/AArch64 modules assume this variable only ever contains a single architecture.

The thing is, I don't think this is clearly documented in the .dsc specification at all. And it is frequently used as a shorthand in there in a form that would only work if build was called with a single -a.
Now, this also means that there are nearly no correct users of ARCH in
edk2 or edk2-platforms.
Because "the list of all architectures being built simultaneously"
simply isn't very useful information
build-time. There are two correct uses in edk2, in EmulatorPkg.dsc and in UefiPayloadPkg.dsc, using '!if "IA32" in "$ARCH"', which as far as I can tell is the only possible way to use the variable correctly.
Slightly amusingly, EmulatorPkg.dsc also contains an incorrect use.

I guess the behaviour is a direct result of how the build command is architected.
I guess the one question I have is - is this the intended behaviour?

Ard: I think we need to update some of our .dsc files...

Ajan: one somewhat hacky workaround I can think of to separate the output images is to hardcode the output image name in arch-specific Defines sections, like so:

diff --git a/ArmVirtPkg/ArmVirtQemu.dsc b/ArmVirtPkg/ArmVirtQemu.dsc index 0bd8d85cab03..f07f2b91764e 100644
--- a/ArmVirtPkg/ArmVirtQemu.dsc
+++ b/ArmVirtPkg/ArmVirtQemu.dsc
@@ -12,12 +12,17 @@
 # Defines Section - statements that will be processed to create a Makefile.
 #
 ################################################################################
+[Defines.ARM]
+  OUTPUT_DIRECTORY               = Build/ArmVirtQemu-ARM
+
+[Defines.AARCH64]
+  OUTPUT_DIRECTORY               = Build/ArmVirtQemu-AARCH64
+
 [Defines]
   PLATFORM_NAME                  = ArmVirtQemu
   PLATFORM_GUID                  = 37d7e986-f7e9-45c2-8067-e371421a626c
   PLATFORM_VERSION               = 0.1
   DSC_SPECIFICATION              = 0x00010005
-  OUTPUT_DIRECTORY               = Build/ArmVirtQemu-$(ARCH)
   SUPPORTED_ARCHITECTURES        = AARCH64|ARM
   BUILD_TARGETS                  = DEBUG|RELEASE|NOOPT
   SKUID_IDENTIFIER               = DEFAULT







-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#120969): https://edk2.groups.io/g/devel/message/120969
Mute This Topic: https://groups.io/mt/110483263/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] Expansion of ARCH in .dsc files

[Leif Lindholm via groups.io]

Hi Joey,

I appreciate the pointer.
However, that's the .inf spec.

The same language is not in the corresponding section of
https://tianocore-docs.github.io/edk2-DscSpecification/release-1.28/3_edk_ii_dsc_file_format/35_[defines]_section.html#35-defines-section

Regards,

Leif

On Tue, 7 Jan 2025 at 19:43, Joey Vagedes <joeyvagedes@microsoft.com> wrote:
>
> Leif,
>
> For your awareness, Section 3.4 of the INF specification states that architectural modifiers for the [Defines] section tag are not permitted.
>
> So that you can Ctrl+F and to find the exact language, here is a quote:
>
> "architectural modifiers for the [Defines] section tag are not permitted"
>
> https://tianocore-docs.github.io/edk2-InfSpecification/release-1.27/3_edk_ii_inf_file_format/34_[defines]_section.html#34-defines-section
>
> Thanks,
> Joey
>
> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Leif Lindholm via groups.io
> Sent: Tuesday, January 7, 2025 11:05 AM
> To: devel@edk2.groups.io
> Cc: Ajan Zhong <ajan.zhong@newfw.com>; Ard Biesheuvel <ardb+tianocore@kernel.org>; Kinney, Michael D <michael.d.kinney@intel.com>
> Subject: [EXTERNAL] [edk2-devel] Expansion of ARCH in .dsc files
>
> So... I originally tripped over my misunderstanding of <subject> when writing
> https://github.com/tianocore/edk2/pull/6262 to get around something I wanted to test in UefiPayloadPkg.
>
> Now Ajan has run into at least an adjacent problem when trying to rework
> https://github.com/tianocore/edk2/pull/6545 based on feedback I among others provided.
>
> Now, the thing I had misunderstood before is that when build is called with multiple -a options, the ARCH variable will expand to a space-separated list of *all* the specified architectures.
> A lot of Arm/AArch64 modules assume this variable only ever contains a single architecture.
>
> The thing is, I don't think this is clearly documented in the .dsc specification at all. And it is frequently used as a shorthand in there in a form that would only work if build was called with a single -a.
> Now, this also means that there are nearly no correct users of ARCH in
> edk2 or edk2-platforms.
> Because "the list of all architectures being built simultaneously"
> simply isn't very useful information
> build-time. There are two correct uses in edk2, in EmulatorPkg.dsc and in UefiPayloadPkg.dsc, using '!if "IA32" in "$ARCH"', which as far as I can tell is the only possible way to use the variable correctly.
> Slightly amusingly, EmulatorPkg.dsc also contains an incorrect use.
>
> I guess the behaviour is a direct result of how the build command is architected.
> I guess the one question I have is - is this the intended behaviour?
>
> Ard: I think we need to update some of our .dsc files...
>
> Ajan: one somewhat hacky workaround I can think of to separate the output images is to hardcode the output image name in arch-specific Defines sections, like so:
>
> diff --git a/ArmVirtPkg/ArmVirtQemu.dsc b/ArmVirtPkg/ArmVirtQemu.dsc index 0bd8d85cab03..f07f2b91764e 100644
> --- a/ArmVirtPkg/ArmVirtQemu.dsc
> +++ b/ArmVirtPkg/ArmVirtQemu.dsc
> @@ -12,12 +12,17 @@
>  # Defines Section - statements that will be processed to create a Makefile.
>  #
>  ################################################################################
> +[Defines.ARM]
> +  OUTPUT_DIRECTORY               = Build/ArmVirtQemu-ARM
> +
> +[Defines.AARCH64]
> +  OUTPUT_DIRECTORY               = Build/ArmVirtQemu-AARCH64
> +
>  [Defines]
>    PLATFORM_NAME                  = ArmVirtQemu
>    PLATFORM_GUID                  = 37d7e986-f7e9-45c2-8067-e371421a626c
>    PLATFORM_VERSION               = 0.1
>    DSC_SPECIFICATION              = 0x00010005
> -  OUTPUT_DIRECTORY               = Build/ArmVirtQemu-$(ARCH)
>    SUPPORTED_ARCHITECTURES        = AARCH64|ARM
>    BUILD_TARGETS                  = DEBUG|RELEASE|NOOPT
>    SKUID_IDENTIFIER               = DEFAULT
>
>
> 
>
>


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#120970): https://edk2.groups.io/g/devel/message/120970
Mute This Topic: https://groups.io/mt/110483263/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] Expansion of ARCH in .dsc files

[Ajan Zhong]

[-- Attachment #1: Type: text/plain, Size: 6341 bytes --]

Thanks Leif and Joey. :)

As suggested, I used the following content in the DSC file to pass all the builds:

[Defines.IA32]
OUTPUT_DIRECTORY = Build/UefiPayloadPkgIA32
FLASH_DEFINITION = UefiPayloadPkg/UefiPayloadPkg.fdf
[Defines.X64]
OUTPUT_DIRECTORY = Build/UefiPayloadPkgX64
FLASH_DEFINITION = UefiPayloadPkg/UefiPayloadPkg.fdf
[Defines.AARCH64]
OUTPUT_DIRECTORY = Build/UefiPayloadPkgAARCH64
FLASH_DEFINITION = UefiPayloadPkg/UefiPayloadPkg.fdf

I have tried following build commands, they all succeed to build corresponding binaries:
1. build -p UefiPayloadPkg/UefiPayloadPkg.dsc -b DEBUG -a IA32 -a X64 -a AARCH64 -t GCC5
2. python UefiPayloadPkg/UniversalPayloadBuild.py<http://UniversalPayloadBuild.py> -t GCC5 -b DEBUG --FIT
3. python UefiPayloadPkg/UniversalPayloadBuild.py<http://UniversalPayloadBuild.py> -t GCC5 -a IA32 -b DEBUG --FIT
4. python UefiPayloadPkg/UniversalPayloadBuild.py<http://UniversalPayloadBuild.py> -t GCC5 -a AARCH -b DEBUG --FIT

Now I am creating a new draft PR to fix all github CI failures which are not generated in my local.

Thanks again for your help.
> From: "Leif Lindholm"<leif.lindholm@oss.qualcomm.com>
> Date:  Wed, Jan 8, 2025, 4:14 AM
> Subject:  Re: [EXTERNAL] [edk2-devel] Expansion of ARCH in .dsc files
> To: "Joey Vagedes"<joeyvagedes@microsoft.com>
> Cc: "devel@edk2.groups.io"<devel@edk2.groups.io>, "Ajan Zhong"<ajan.zhong@newfw.com>, "Ard Biesheuvel"<ardb+tianocore@kernel.org>, "Kinney, Michael D"<michael.d.kinney@intel.com>
> Hi Joey,

> 
> I appreciate the pointer.

> However, that's the .inf spec.

> 
> The same language is not in the corresponding section of

> https://tianocore-docs.github.io/edk2-DscSpecification/release-1.28/3_edk_ii_dsc_file_format/35_[defines]_section.html#35-defines-section

> 
> Regards,

> 
> Leif

> 
> On Tue, 7 Jan 2025 at 19:43, Joey Vagedes <joeyvagedes@microsoft.com> wrote:

> >

> > Leif,

> >

> > For your awareness, Section 3.4 of the INF specification states that architectural modifiers for the [Defines] section tag are not permitted.

> >

> > So that you can Ctrl+F and to find the exact language, here is a quote:

> >

> > "architectural modifiers for the [Defines] section tag are not permitted"

> >

> > https://tianocore-docs.github.io/edk2-InfSpecification/release-1.27/3_edk_ii_inf_file_format/34_[defines]_section.html#34-defines-section

> >

> > Thanks,

> > Joey

> >

> > -----Original Message-----

> > From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Leif Lindholm via groups.io

> > Sent: Tuesday, January 7, 2025 11:05 AM

> > To: devel@edk2.groups.io

> > Cc: Ajan Zhong <ajan.zhong@newfw.com>; Ard Biesheuvel <ardb+tianocore@kernel.org>; Kinney, Michael D <michael.d.kinney@intel.com>

> > Subject: [EXTERNAL] [edk2-devel] Expansion of ARCH in .dsc files

> >

> > So... I originally tripped over my misunderstanding of <subject> when writing

> > https://github.com/tianocore/edk2/pull/6262 to get around something I wanted to test in UefiPayloadPkg.

> >

> > Now Ajan has run into at least an adjacent problem when trying to rework

> > https://github.com/tianocore/edk2/pull/6545 based on feedback I among others provided.

> >

> > Now, the thing I had misunderstood before is that when build is called with multiple -a options, the ARCH variable will expand to a space-separated list of *all* the specified architectures.

> > A lot of Arm/AArch64 modules assume this variable only ever contains a single architecture.

> >

> > The thing is, I don't think this is clearly documented in the .dsc specification at all. And it is frequently used as a shorthand in there in a form that would only work if build was called with a single -a.

> > Now, this also means that there are nearly no correct users of ARCH in

> > edk2 or edk2-platforms.

> > Because "the list of all architectures being built simultaneously"

> > simply isn't very useful information

> > build-time. There are two correct uses in edk2, in EmulatorPkg.dsc and in UefiPayloadPkg.dsc, using '!if "IA32" in "$ARCH"', which as far as I can tell is the only possible way to use the variable correctly.

> > Slightly amusingly, EmulatorPkg.dsc also contains an incorrect use.

> >

> > I guess the behaviour is a direct result of how the build command is architected.

> > I guess the one question I have is - is this the intended behaviour?

> >

> > Ard: I think we need to update some of our .dsc files...

> >

> > Ajan: one somewhat hacky workaround I can think of to separate the output images is to hardcode the output image name in arch-specific Defines sections, like so:

> >

> > diff --git a/ArmVirtPkg/ArmVirtQemu.dsc b/ArmVirtPkg/ArmVirtQemu.dsc index 0bd8d85cab03..f07f2b91764e 100644

> > --- a/ArmVirtPkg/ArmVirtQemu.dsc

> > +++ b/ArmVirtPkg/ArmVirtQemu.dsc

> > @@ -12,12 +12,17 @@

> >  # Defines Section - statements that will be processed to create a Makefile.

> >  #

> >  ################################################################################

> > +[Defines.ARM]

> > +  OUTPUT_DIRECTORY               = Build/ArmVirtQemu-ARM

> > +

> > +[Defines.AARCH64]

> > +  OUTPUT_DIRECTORY               = Build/ArmVirtQemu-AARCH64

> > +

> >  [Defines]

> >    PLATFORM_NAME                  = ArmVirtQemu

> >    PLATFORM_GUID                  = 37d7e986-f7e9-45c2-8067-e371421a626c

> >    PLATFORM_VERSION               = 0.1

> >    DSC_SPECIFICATION              = 0x00010005

> > -  OUTPUT_DIRECTORY               = Build/ArmVirtQemu-$(ARCH)

> >    SUPPORTED_ARCHITECTURES        = AARCH64|ARM

> >    BUILD_TARGETS                  = DEBUG|RELEASE|NOOPT

> >    SKUID_IDENTIFIER               = DEFAULT

> >

> >

> > 

> >

> >
> 


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#120997): https://edk2.groups.io/g/devel/message/120997
Mute This Topic: https://groups.io/mt/110483263/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-



[-- Attachment #2: Type: text/html, Size: 23713 bytes --]

Re: [edk2-devel] Expansion of ARCH in .dsc files

[Ajan Zhong]

[-- Attachment #1: Type: text/plain, Size: 7470 bytes --]

Hi all,
Just check outputs for each architecture, the FV cannot be generated if we use this method:
[Defines.X64]
OUTPUT_DIRECTORY = Build/UefiPayloadPkgX64
FLASH_DEFINITION = UefiPayloadPkg/UefiPayloadPkg.fdf

If we place FLASH_DEFINITION under the Defines section:
[Defines]
FLASH_DEFINITION = UefiPayloadPkg/UefiPayloadPkg.fdf
We will meet "error F003: Module built under multiple ARCHs ['AARCH64', 'X64']. Not able to determine which output to put into flash for Module MdeModulePkg/Universal/DevicePathDxe/DevicePathDxe.inf !" issue again.

Still working in progress to figure out other solutions.

> From: "Ajan Zhong"<ajan.zhong@newfw.com>
> Date:  Wed, Jan 8, 2025, 3:29 PM
> Subject:  Re: [EXTERNAL] [edk2-devel] Expansion of ARCH in .dsc files
> To: "Leif Lindholm"<leif.lindholm@oss.qualcomm.com>
> Cc: "Joey Vagedes"<joeyvagedes@microsoft.com>, "devel@edk2.groups.io"<devel@edk2.groups.io>, "Ard Biesheuvel"<ardb+tianocore@kernel.org>, "Kinney, Michael D"<michael.d.kinney@intel.com>
> Thanks Leif and Joey. :)
> 
> 
> As suggested, I used the following content in the DSC file to pass all the builds:
> 
> 
> [Defines.IA32]
> OUTPUT_DIRECTORY = Build/UefiPayloadPkgIA32
> FLASH_DEFINITION = UefiPayloadPkg/UefiPayloadPkg.fdf
> [Defines.X64]
> OUTPUT_DIRECTORY = Build/UefiPayloadPkgX64
> FLASH_DEFINITION = UefiPayloadPkg/UefiPayloadPkg.fdf
> [Defines.AARCH64]
> OUTPUT_DIRECTORY = Build/UefiPayloadPkgAARCH64
> FLASH_DEFINITION = UefiPayloadPkg/UefiPayloadPkg.fdf
> 
> 
> I have tried following build commands, they all succeed to build corresponding binaries:
> 1. build -p UefiPayloadPkg/UefiPayloadPkg.dsc -b DEBUG -a IA32 -a X64 -a AARCH64 -t GCC5
> 2. python UefiPayloadPkg/UniversalPayloadBuild.py<http://UniversalPayloadBuild.py> -t GCC5 -b DEBUG --FIT
> 3. python UefiPayloadPkg/UniversalPayloadBuild.py<http://UniversalPayloadBuild.py> -t GCC5 -a IA32 -b DEBUG --FIT
> 4. python UefiPayloadPkg/UniversalPayloadBuild.py<http://UniversalPayloadBuild.py> -t GCC5 -a AARCH -b DEBUG --FIT
> 
> 
> Now I am creating a new draft PR to fix all github CI failures which are not generated in my local.
> 
> 
> Thanks again for your help.
> From: "Leif Lindholm"<leif.lindholm@oss.qualcomm.com>
> Date:  Wed, Jan 8, 2025, 4:14 AM
> Subject:  Re: [EXTERNAL] [edk2-devel] Expansion of ARCH in .dsc files
> To: "Joey Vagedes"<joeyvagedes@microsoft.com>
> Cc: "devel@edk2.groups.io"<devel@edk2.groups.io>, "Ajan Zhong"<ajan.zhong@newfw.com>, "Ard Biesheuvel"<ardb+tianocore@kernel.org>, "Kinney, Michael D"<michael.d.kinney@intel.com>
> Hi Joey, 
> 
> 
> I appreciate the pointer. 
> However, that's the .inf spec. 
> 
> 
> The same language is not in the corresponding section of 
> https://tianocore-docs.github.io/edk2-DscSpecification/release-1.28/3_edk_ii_dsc_file_format/35_[defines]_section.html#35-defines-section 
> 
> 
> Regards, 
> 
> 
> Leif 
> 
> 
> On Tue, 7 Jan 2025 at 19:43, Joey Vagedes <joeyvagedes@microsoft.com> wrote: 
> > 
> > Leif, 
> > 
> > For your awareness, Section 3.4 of the INF specification states that architectural modifiers for the [Defines] section tag are not permitted. 
> > 
> > So that you can Ctrl+F and to find the exact language, here is a quote: 
> > 
> > "architectural modifiers for the [Defines] section tag are not permitted" 
> > 
> > https://tianocore-docs.github.io/edk2-InfSpecification/release-1.27/3_edk_ii_inf_file_format/34_[defines]_section.html#34-defines-section 
> > 
> > Thanks, 
> > Joey 
> > 
> > -----Original Message----- 
> > From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Leif Lindholm via groups.io<http://groups.io> 
> Sent: Tuesday, January 7, 2025 11:05 AM 
> > To: devel@edk2.groups.io 
> Cc: Ajan Zhong <ajan.zhong@newfw.com>; Ard Biesheuvel <ardb+tianocore@kernel.org>; Kinney, Michael D <michael.d.kinney@intel.com> 
> > Subject: [EXTERNAL] [edk2-devel] Expansion of ARCH in .dsc files 
> > 
> > So... I originally tripped over my misunderstanding of <subject> when writing 
> > https://github.com/tianocore/edk2/pull/6262 to get around something I wanted to test in UefiPayloadPkg. 
> > 
> > Now Ajan has run into at least an adjacent problem when trying to rework 
> > https://github.com/tianocore/edk2/pull/6545 based on feedback I among others provided. 
> > 
> > Now, the thing I had misunderstood before is that when build is called with multiple -a options, the ARCH variable will expand to a space-separated list of *all* the specified architectures. 
> > A lot of Arm/AArch64 modules assume this variable only ever contains a single architecture. 
> > 
> > The thing is, I don't think this is clearly documented in the .dsc specification at all. And it is frequently used as a shorthand in there in a form that would only work if build was called with a single -a. 
> > Now, this also means that there are nearly no correct users of ARCH in 
> > edk2 or edk2-platforms. 
> > Because "the list of all architectures being built simultaneously" 
> > simply isn't very useful information 
> > build-time. There are two correct uses in edk2, in EmulatorPkg.dsc and in UefiPayloadPkg.dsc, using '!if "IA32" in "$ARCH"', which as far as I can tell is the only possible way to use the variable correctly. 
> > Slightly amusingly, EmulatorPkg.dsc also contains an incorrect use. 
> > 
> > I guess the behaviour is a direct result of how the build command is architected. 
> > I guess the one question I have is - is this the intended behaviour? 
> > 
> > Ard: I think we need to update some of our .dsc files... 
> > 
> > Ajan: one somewhat hacky workaround I can think of to separate the output images is to hardcode the output image name in arch-specific Defines sections, like so: 
> > 
> > diff --git a/ArmVirtPkg/ArmVirtQemu.dsc b/ArmVirtPkg/ArmVirtQemu.dsc index 0bd8d85cab03..f07f2b91764e 100644 
> > --- a/ArmVirtPkg/ArmVirtQemu.dsc 
> > +++ b/ArmVirtPkg/ArmVirtQemu.dsc 
> > @@ -12,12 +12,17 @@ 
> >  # Defines Section - statements that will be processed to create a Makefile. 
> >  # 
> >  ################################################################################ 
> > +[Defines.ARM] 
> > +  OUTPUT_DIRECTORY               = Build/ArmVirtQemu-ARM 
> > + 
> > +[Defines.AARCH64] 
> > +  OUTPUT_DIRECTORY               = Build/ArmVirtQemu-AARCH64 
> > + 
> >  [Defines] 
> >    PLATFORM_NAME                  = ArmVirtQemu 
> >    PLATFORM_GUID                  = 37d7e986-f7e9-45c2-8067-e371421a626c 
> >    PLATFORM_VERSION               = 0.1 
> >    DSC_SPECIFICATION              = 0x00010005 
> > -  OUTPUT_DIRECTORY               = Build/ArmVirtQemu-$(ARCH) 
> >    SUPPORTED_ARCHITECTURES        = AARCH64|ARM 
> >    BUILD_TARGETS                  = DEBUG|RELEASE|NOOPT 
> >    SKUID_IDENTIFIER               = DEFAULT 
> > 
> > 
> >  
> > 
> >
> 
> 


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#120998): https://edk2.groups.io/g/devel/message/120998
Mute This Topic: https://groups.io/mt/110483263/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-



[-- Attachment #2: Type: text/html, Size: 52712 bytes --]

Re: [edk2-devel] Expansion of ARCH in .dsc files

[Gerd Hoffmann via groups.io]

  Hi,

> Because "the list of all architectures being built simultaneously"
> simply isn't very useful information build-time.

I think this is not intended for building all archs in simultaneously
(even though that often works too), but for firmware images using code
from multiple architectures.  That is (or used to be?) common in the x86
world, where firmware can run the PEI phase in 32-bit mode (aka IA32)
and the DXE phase in 64-bit mode (aka X64).

See OvmfPkg/OvmfPkgIa32X64.dsc

take care,
  Gerd



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#120972): https://edk2.groups.io/g/devel/message/120972
Mute This Topic: https://groups.io/mt/110483263/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] Expansion of ARCH in .dsc files

[Leif Lindholm via groups.io]

On Wed, 8 Jan 2025 at 15:01, Gerd Hoffmann <kraxel@redhat.com> wrote:
> > Because "the list of all architectures being built simultaneously"
> > simply isn't very useful information build-time.
>
> I think this is not intended for building all archs in simultaneously
> (even though that often works too), but for firmware images using code
> from multiple architectures.  That is (or used to be?) common in the x86
> world, where firmware can run the PEI phase in 32-bit mode (aka IA32)
> and the DXE phase in 64-bit mode (aka X64).
>
> See OvmfPkg/OvmfPkgIa32X64.dsc

Yes, indeed.
It feels like a bit of a hack introduced specifically for that
specific case (but not documented as such).

So, Ajan - I guess that means arguably the CI job should be reworked
to call the different arch builds independently. Then the problem
would go away even without the Defines.$ARCH change.

/
    Leif


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#120973): https://edk2.groups.io/g/devel/message/120973
Mute This Topic: https://groups.io/mt/110483263/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] Expansion of ARCH in .dsc files

[Ajan Zhong]

[-- Attachment #1: Type: text/plain, Size: 2165 bytes --]

Thanks all.

I checked packages which support multiple architectures (IA32 and X64), all these modules generate only one flash image for one architecture. And checked build process python scripts, we have no other way to build IA32/X64/AARCH64 at same time.

As suggested, I plan to update azure Ubuntu GCC5 pipeline to adapt this scenario. And will provide feedback after validation.

> From: "Leif Lindholm"<leif.lindholm@oss.qualcomm.com>
> Date:  Thu, Jan 9, 2025, 12:02 AM
> Subject:  Re: [edk2-devel] Expansion of ARCH in .dsc files
> To: "Gerd Hoffmann"<kraxel@redhat.com>
> Cc: <devel@edk2.groups.io>, "Ajan Zhong"<ajan.zhong@newfw.com>, "Ard Biesheuvel"<ardb+tianocore@kernel.org>, "Kinney, Michael D"<michael.d.kinney@intel.com>, "Michael Kubacki"<mikuback@linux.microsoft.com>
> On Wed, 8 Jan 2025 at 15:01, Gerd Hoffmann <kraxel@redhat.com> wrote:

> > > Because "the list of all architectures being built simultaneously"

> > > simply isn't very useful information build-time.

> >

> > I think this is not intended for building all archs in simultaneously

> > (even though that often works too), but for firmware images using code

> > from multiple architectures.  That is (or used to be?) common in the x86

> > world, where firmware can run the PEI phase in 32-bit mode (aka IA32)

> > and the DXE phase in 64-bit mode (aka X64).

> >

> > See OvmfPkg/OvmfPkgIa32X64.dsc

> 
> Yes, indeed.

> It feels like a bit of a hack introduced specifically for that

> specific case (but not documented as such).

> 
> So, Ajan - I guess that means arguably the CI job should be reworked

> to call the different arch builds independently. Then the problem

> would go away even without the Defines.$ARCH change.

> 
> /

>     Leif
> 


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#121002): https://edk2.groups.io/g/devel/message/121002
Mute This Topic: https://groups.io/mt/110483263/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-



[-- Attachment #2: Type: text/html, Size: 7326 bytes --]

Re: [edk2-devel] Expansion of ARCH in .dsc files

[Ajan]

[-- Attachment #1: Type: text/plain, Size: 1092 bytes --]

Hi all,
UefiPayloadPkg succeed to pass Azure Pipeline now, TARGET_UEFIPAYLOAD has been separated into two targets:
TARGET_UEFIPAYLOAD_IA32_X64 for GCC5 and VS20222, TARGET_UEFIPAYLOAD_AARCH64_GCC_ONLY for GCC5 only.

TARGET_UEFIPAYLOAD_IA32_X64:

Build.Pkgs: 'UefiPayloadPkg'

Build.Targets: 'DEBUG,RELEASE,NO-TARGET,NOOPT'

Build.Archlist: 'IA32,X64'

${{ if eq(parameters.tool_chain_tag, 'GCC5') }}:

TARGET_UEFIPAYLOAD_AARCH64_GCC_ONLY:

Build.Pkgs: 'UefiPayloadPkg'

Build.Targets: 'DEBUG,RELEASE,NO-TARGET,NOOPT'

Build.Archlist: 'AARCH64'

The changes have been submitted to EDK2 repo for review, here is the link if you have interest: https://github.com/tianocore/edk2/pull/6545

Thanks again.


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#121003): https://edk2.groups.io/g/devel/message/121003
Mute This Topic: https://groups.io/mt/110483263/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-



[-- Attachment #2: Type: text/html, Size: 3156 bytes --]