ARM PrePI Memory setup recommendations

ARM PrePI Memory setup recommendations

[Michael Zimmermann]

Hi,

I have some problems with my current setup and would like to ask you for
the best way to configure the following setup:
- I'm using PrePi, I don't need relocations because the loading address is
fixed(0xfe000000)
- The SystemMemory address range is unknown at compile time
- the system-memory consists of multiple ranges with holes so providing
just a base and a size is not enough
- There's no NOR, edk2 gets loaded into dram

my current setup:
PcdSystemMemoryBase = loading addr(0xfe000000)
PcdSystemMemorySize = 32MB
FD_BASE = PcdSystemMemoryBase
FD_SIZE = 4MB
in ArmPlatformGetVirtualMemoryMap I call BuildResourceDescriptorHob for all
DRAM memory ranges.
In PrePiMain after MemoryPeim I call BuildMemoryAllocationHob for some
reserved memory ranges

After running into out-of-memory errors during PrePi when using bigger FV's
I revisited everything and came across a PCD I somehow never looked at when
setting up my platform a year ago: PcdSystemMemoryUefiRegionSize.
this was hardcoded to 0x00e00000 which obviously caused allocation errors
while extracting big FV's even though PcdSystemMemorySize was big enough.

Since there already was this nice comment 'PcdSystemMemorySize - PcdFdSize'
I did exactly that: 32MB-4MB = 28MB.
Unfortunately after doing that MdeModulePkg/Core/Dxe/Gcd/Gcd.c starts
bugging that he can't find a resource hob that contains the PHIT Hob.

So now that my setup and problems are clear, the questions:
- Is it ok to define ResourceHob's that cover the PcdSystemMemory range?
- what can I do to fix the PHIT Hob problem? should PcdFdSize +
PcdSystemMemoryUefiRegionSize be smaller than PcdSystemMemorySize ?
- I took a quick look at ArmVirt where they change the PcdSystemMemory
range at runtime, but they use lots of custom code like their own
MemoryPeiLib, and I think everything but Xen uses PEIM instead of PrePi.
Also as I said I have more than one range so this would work anyway.

Thanks
Michael

Re: ARM PrePI Memory setup recommendations

[Ard Biesheuvel]

On 6 August 2016 at 13:05, Michael Zimmermann <sigmaepsilon92@gmail.com> wrote:
> Hi,
>
> I have some problems with my current setup and would like to ask you for the
> best way to configure the following setup:
> - I'm using PrePi, I don't need relocations because the loading address is
> fixed(0xfe000000)
> - The SystemMemory address range is unknown at compile time
> - the system-memory consists of multiple ranges with holes so providing just
> a base and a size is not enough
> - There's no NOR, edk2 gets loaded into dram
>
> my current setup:
> PcdSystemMemoryBase = loading addr(0xfe000000)
> PcdSystemMemorySize = 32MB
> FD_BASE = PcdSystemMemoryBase
> FD_SIZE = 4MB
> in ArmPlatformGetVirtualMemoryMap I call BuildResourceDescriptorHob for all
> DRAM memory ranges.
> In PrePiMain after MemoryPeim I call BuildMemoryAllocationHob for some
> reserved memory ranges
>
> After running into out-of-memory errors during PrePi when using bigger FV's
> I revisited everything and came across a PCD I somehow never looked at when
> setting up my platform a year ago: PcdSystemMemoryUefiRegionSize.
> this was hardcoded to 0x00e00000 which obviously caused allocation errors
> while extracting big FV's even though PcdSystemMemorySize was big enough.
>
> Since there already was this nice comment 'PcdSystemMemorySize - PcdFdSize'
> I did exactly that: 32MB-4MB = 28MB.
> Unfortunately after doing that MdeModulePkg/Core/Dxe/Gcd/Gcd.c starts
> bugging that he can't find a resource hob that contains the PHIT Hob.
>

Could you check the

gEmbeddedTokenSpaceGuid.PcdMemoryTypeEfiXXX|nnnn

definitions in your .DSC? Fairly recently, the core code was modified
to use the sum of these definitions as an estimate of the minimally
required allocation size, and it broke some other platforms in a
similar way.

> So now that my setup and problems are clear, the questions:
> - Is it ok to define ResourceHob's that cover the PcdSystemMemory range?
> - what can I do to fix the PHIT Hob problem? should PcdFdSize +
> PcdSystemMemoryUefiRegionSize be smaller than PcdSystemMemorySize ?
> - I took a quick look at ArmVirt where they change the PcdSystemMemory range
> at runtime, but they use lots of custom code like their own MemoryPeiLib,
> and I think everything but Xen uses PEIM instead of PrePi. Also as I said I
> have more than one range so this would work anyway.
>

ArmVirtQemu uses PEIM, since it runs from [emulated] NOR flash.
ArmVirtQemuKernel and ArmVirtXen use PrePi (the former is simply
ArmVirtQemu with the FD executing from DRAM, using the QEMU '-kernel'
option)

Re: ARM PrePI Memory setup recommendations

[Michael Zimmermann]

I'm using the defaults from the (now deleted) 2ndstage sample dsc:
  gEmbeddedTokenSpaceGuid.PcdMemoryTypeEfiACPIReclaimMemory|0
  gEmbeddedTokenSpaceGuid.PcdMemoryTypeEfiACPIMemoryNVS|0
  gEmbeddedTokenSpaceGuid.PcdMemoryTypeEfiReservedMemoryType|0
  gEmbeddedTokenSpaceGuid.PcdMemoryTypeEfiRuntimeServicesData|50
  gEmbeddedTokenSpaceGuid.PcdMemoryTypeEfiRuntimeServicesCode|20
  gEmbeddedTokenSpaceGuid.PcdMemoryTypeEfiBootServicesCode|400
  gEmbeddedTokenSpaceGuid.PcdMemoryTypeEfiBootServicesData|20000
  gEmbeddedTokenSpaceGuid.PcdMemoryTypeEfiLoaderCode|20
  gEmbeddedTokenSpaceGuid.PcdMemoryTypeEfiLoaderData|0

I figured that the PrePi's stack setup may cause this, especially since I'm
using all the memory up to 0xFFFFFFFF+1 which causes the logic to ignore
the last page(I think).
I'm now using PcdSystemMemorySize - PcdFdSize - EFI_PAGE_SIZE which seems
to work pretty good even though the "missing" memory was just like 900bytes
and not 4096.

Thanks
Michael

On Mon, Aug 8, 2016 at 10:50 AM, Ard Biesheuvel <ard.biesheuvel@linaro.org>
wrote:

> On 6 August 2016 at 13:05, Michael Zimmermann <sigmaepsilon92@gmail.com>
> wrote:
> > Hi,
> >
> > I have some problems with my current setup and would like to ask you for
> the
> > best way to configure the following setup:
> > - I'm using PrePi, I don't need relocations because the loading address
> is
> > fixed(0xfe000000)
> > - The SystemMemory address range is unknown at compile time
> > - the system-memory consists of multiple ranges with holes so providing
> just
> > a base and a size is not enough
> > - There's no NOR, edk2 gets loaded into dram
> >
> > my current setup:
> > PcdSystemMemoryBase = loading addr(0xfe000000)
> > PcdSystemMemorySize = 32MB
> > FD_BASE = PcdSystemMemoryBase
> > FD_SIZE = 4MB
> > in ArmPlatformGetVirtualMemoryMap I call BuildResourceDescriptorHob for
> all
> > DRAM memory ranges.
> > In PrePiMain after MemoryPeim I call BuildMemoryAllocationHob for some
> > reserved memory ranges
> >
> > After running into out-of-memory errors during PrePi when using bigger
> FV's
> > I revisited everything and came across a PCD I somehow never looked at
> when
> > setting up my platform a year ago: PcdSystemMemoryUefiRegionSize.
> > this was hardcoded to 0x00e00000 which obviously caused allocation errors
> > while extracting big FV's even though PcdSystemMemorySize was big enough.
> >
> > Since there already was this nice comment 'PcdSystemMemorySize -
> PcdFdSize'
> > I did exactly that: 32MB-4MB = 28MB.
> > Unfortunately after doing that MdeModulePkg/Core/Dxe/Gcd/Gcd.c starts
> > bugging that he can't find a resource hob that contains the PHIT Hob.
> >
>
> Could you check the
>
> gEmbeddedTokenSpaceGuid.PcdMemoryTypeEfiXXX|nnnn
>
> definitions in your .DSC? Fairly recently, the core code was modified
> to use the sum of these definitions as an estimate of the minimally
> required allocation size, and it broke some other platforms in a
> similar way.
>
> > So now that my setup and problems are clear, the questions:
> > - Is it ok to define ResourceHob's that cover the PcdSystemMemory range?
> > - what can I do to fix the PHIT Hob problem? should PcdFdSize +
> > PcdSystemMemoryUefiRegionSize be smaller than PcdSystemMemorySize ?
> > - I took a quick look at ArmVirt where they change the PcdSystemMemory
> range
> > at runtime, but they use lots of custom code like their own MemoryPeiLib,
> > and I think everything but Xen uses PEIM instead of PrePi. Also as I
> said I
> > have more than one range so this would work anyway.
> >
>
> ArmVirtQemu uses PEIM, since it runs from [emulated] NOR flash.
> ArmVirtQemuKernel and ArmVirtXen use PrePi (the former is simply
> ArmVirtQemu with the FD executing from DRAM, using the QEMU '-kernel'
> option)
>

Re: ARM PrePI Memory setup recommendations

[Ard Biesheuvel]

On 8 August 2016 at 11:18, Michael Zimmermann <sigmaepsilon92@gmail.com> wrote:
> I'm using the defaults from the (now deleted) 2ndstage sample dsc:
>   gEmbeddedTokenSpaceGuid.PcdMemoryTypeEfiACPIReclaimMemory|0
>   gEmbeddedTokenSpaceGuid.PcdMemoryTypeEfiACPIMemoryNVS|0
>   gEmbeddedTokenSpaceGuid.PcdMemoryTypeEfiReservedMemoryType|0
>   gEmbeddedTokenSpaceGuid.PcdMemoryTypeEfiRuntimeServicesData|50
>   gEmbeddedTokenSpaceGuid.PcdMemoryTypeEfiRuntimeServicesCode|20
>   gEmbeddedTokenSpaceGuid.PcdMemoryTypeEfiBootServicesCode|400
>   gEmbeddedTokenSpaceGuid.PcdMemoryTypeEfiBootServicesData|20000

20000 pages == 80 MB, so this value makes little sense if you have 32
MB of memory. Dropping this to a sane value fixed some problems we saw
with QEMU when running with the default 128 MB

>   gEmbeddedTokenSpaceGuid.PcdMemoryTypeEfiLoaderCode|20
>   gEmbeddedTokenSpaceGuid.PcdMemoryTypeEfiLoaderData|0
>
> I figured that the PrePi's stack setup may cause this, especially since I'm
> using all the memory up to 0xFFFFFFFF+1 which causes the logic to ignore the
> last page(I think).
> I'm now using PcdSystemMemorySize - PcdFdSize - EFI_PAGE_SIZE which seems to
> work pretty good even though the "missing" memory was just like 900bytes and
> not 4096.
>
> Thanks
> Michael
>
> On Mon, Aug 8, 2016 at 10:50 AM, Ard Biesheuvel <ard.biesheuvel@linaro.org>
> wrote:
>>
>> On 6 August 2016 at 13:05, Michael Zimmermann <sigmaepsilon92@gmail.com>
>> wrote:
>> > Hi,
>> >
>> > I have some problems with my current setup and would like to ask you for
>> > the
>> > best way to configure the following setup:
>> > - I'm using PrePi, I don't need relocations because the loading address
>> > is
>> > fixed(0xfe000000)
>> > - The SystemMemory address range is unknown at compile time
>> > - the system-memory consists of multiple ranges with holes so providing
>> > just
>> > a base and a size is not enough
>> > - There's no NOR, edk2 gets loaded into dram
>> >
>> > my current setup:
>> > PcdSystemMemoryBase = loading addr(0xfe000000)
>> > PcdSystemMemorySize = 32MB
>> > FD_BASE = PcdSystemMemoryBase
>> > FD_SIZE = 4MB
>> > in ArmPlatformGetVirtualMemoryMap I call BuildResourceDescriptorHob for
>> > all
>> > DRAM memory ranges.
>> > In PrePiMain after MemoryPeim I call BuildMemoryAllocationHob for some
>> > reserved memory ranges
>> >
>> > After running into out-of-memory errors during PrePi when using bigger
>> > FV's
>> > I revisited everything and came across a PCD I somehow never looked at
>> > when
>> > setting up my platform a year ago: PcdSystemMemoryUefiRegionSize.
>> > this was hardcoded to 0x00e00000 which obviously caused allocation
>> > errors
>> > while extracting big FV's even though PcdSystemMemorySize was big
>> > enough.
>> >
>> > Since there already was this nice comment 'PcdSystemMemorySize -
>> > PcdFdSize'
>> > I did exactly that: 32MB-4MB = 28MB.
>> > Unfortunately after doing that MdeModulePkg/Core/Dxe/Gcd/Gcd.c starts
>> > bugging that he can't find a resource hob that contains the PHIT Hob.
>> >
>>
>> Could you check the
>>
>> gEmbeddedTokenSpaceGuid.PcdMemoryTypeEfiXXX|nnnn
>>
>> definitions in your .DSC? Fairly recently, the core code was modified
>> to use the sum of these definitions as an estimate of the minimally
>> required allocation size, and it broke some other platforms in a
>> similar way.
>>
>> > So now that my setup and problems are clear, the questions:
>> > - Is it ok to define ResourceHob's that cover the PcdSystemMemory range?
>> > - what can I do to fix the PHIT Hob problem? should PcdFdSize +
>> > PcdSystemMemoryUefiRegionSize be smaller than PcdSystemMemorySize ?
>> > - I took a quick look at ArmVirt where they change the PcdSystemMemory
>> > range
>> > at runtime, but they use lots of custom code like their own
>> > MemoryPeiLib,
>> > and I think everything but Xen uses PEIM instead of PrePi. Also as I
>> > said I
>> > have more than one range so this would work anyway.
>> >
>>
>> ArmVirtQemu uses PEIM, since it runs from [emulated] NOR flash.
>> ArmVirtQemuKernel and ArmVirtXen use PrePi (the former is simply
>> ArmVirtQemu with the FD executing from DRAM, using the QEMU '-kernel'
>> option)
>
>

Re: ARM PrePI Memory setup recommendations

[Michael Zimmermann]

While that can be a problem in general it's definitely not a problem in my
case because the "MinimalMemorySizeNeeded" check is done after a
ResourceHob is found which contains the PHIT Hob.(this is the part which
broke).

And since at that point I've already registered resourcehob's for all
existing dram memory(usually between 512MB and 3GB) there's plenty of
memory for bootservices to use.

So I really think the only problem was that I had to take the stack size
into account when calculating PcdSystemMemorySize.

Thanks
Michael

On Mon, Aug 8, 2016 at 11:25 AM, Ard Biesheuvel <ard.biesheuvel@linaro.org>
wrote:

> On 8 August 2016 at 11:18, Michael Zimmermann <sigmaepsilon92@gmail.com>
> wrote:
> > I'm using the defaults from the (now deleted) 2ndstage sample dsc:
> >   gEmbeddedTokenSpaceGuid.PcdMemoryTypeEfiACPIReclaimMemory|0
> >   gEmbeddedTokenSpaceGuid.PcdMemoryTypeEfiACPIMemoryNVS|0
> >   gEmbeddedTokenSpaceGuid.PcdMemoryTypeEfiReservedMemoryType|0
> >   gEmbeddedTokenSpaceGuid.PcdMemoryTypeEfiRuntimeServicesData|50
> >   gEmbeddedTokenSpaceGuid.PcdMemoryTypeEfiRuntimeServicesCode|20
> >   gEmbeddedTokenSpaceGuid.PcdMemoryTypeEfiBootServicesCode|400
> >   gEmbeddedTokenSpaceGuid.PcdMemoryTypeEfiBootServicesData|20000
>
> 20000 pages == 80 MB, so this value makes little sense if you have 32
> MB of memory. Dropping this to a sane value fixed some problems we saw
> with QEMU when running with the default 128 MB
>
> >   gEmbeddedTokenSpaceGuid.PcdMemoryTypeEfiLoaderCode|20
> >   gEmbeddedTokenSpaceGuid.PcdMemoryTypeEfiLoaderData|0
> >
> > I figured that the PrePi's stack setup may cause this, especially since
> I'm
> > using all the memory up to 0xFFFFFFFF+1 which causes the logic to ignore
> the
> > last page(I think).
> > I'm now using PcdSystemMemorySize - PcdFdSize - EFI_PAGE_SIZE which
> seems to
> > work pretty good even though the "missing" memory was just like 900bytes
> and
> > not 4096.
> >
> > Thanks
> > Michael
> >
> > On Mon, Aug 8, 2016 at 10:50 AM, Ard Biesheuvel <
> ard.biesheuvel@linaro.org>
> > wrote:
> >>
> >> On 6 August 2016 at 13:05, Michael Zimmermann <sigmaepsilon92@gmail.com
> >
> >> wrote:
> >> > Hi,
> >> >
> >> > I have some problems with my current setup and would like to ask you
> for
> >> > the
> >> > best way to configure the following setup:
> >> > - I'm using PrePi, I don't need relocations because the loading
> address
> >> > is
> >> > fixed(0xfe000000)
> >> > - The SystemMemory address range is unknown at compile time
> >> > - the system-memory consists of multiple ranges with holes so
> providing
> >> > just
> >> > a base and a size is not enough
> >> > - There's no NOR, edk2 gets loaded into dram
> >> >
> >> > my current setup:
> >> > PcdSystemMemoryBase = loading addr(0xfe000000)
> >> > PcdSystemMemorySize = 32MB
> >> > FD_BASE = PcdSystemMemoryBase
> >> > FD_SIZE = 4MB
> >> > in ArmPlatformGetVirtualMemoryMap I call BuildResourceDescriptorHob
> for
> >> > all
> >> > DRAM memory ranges.
> >> > In PrePiMain after MemoryPeim I call BuildMemoryAllocationHob for some
> >> > reserved memory ranges
> >> >
> >> > After running into out-of-memory errors during PrePi when using bigger
> >> > FV's
> >> > I revisited everything and came across a PCD I somehow never looked at
> >> > when
> >> > setting up my platform a year ago: PcdSystemMemoryUefiRegionSize.
> >> > this was hardcoded to 0x00e00000 which obviously caused allocation
> >> > errors
> >> > while extracting big FV's even though PcdSystemMemorySize was big
> >> > enough.
> >> >
> >> > Since there already was this nice comment 'PcdSystemMemorySize -
> >> > PcdFdSize'
> >> > I did exactly that: 32MB-4MB = 28MB.
> >> > Unfortunately after doing that MdeModulePkg/Core/Dxe/Gcd/Gcd.c starts
> >> > bugging that he can't find a resource hob that contains the PHIT Hob.
> >> >
> >>
> >> Could you check the
> >>
> >> gEmbeddedTokenSpaceGuid.PcdMemoryTypeEfiXXX|nnnn
> >>
> >> definitions in your .DSC? Fairly recently, the core code was modified
> >> to use the sum of these definitions as an estimate of the minimally
> >> required allocation size, and it broke some other platforms in a
> >> similar way.
> >>
> >> > So now that my setup and problems are clear, the questions:
> >> > - Is it ok to define ResourceHob's that cover the PcdSystemMemory
> range?
> >> > - what can I do to fix the PHIT Hob problem? should PcdFdSize +
> >> > PcdSystemMemoryUefiRegionSize be smaller than PcdSystemMemorySize ?
> >> > - I took a quick look at ArmVirt where they change the PcdSystemMemory
> >> > range
> >> > at runtime, but they use lots of custom code like their own
> >> > MemoryPeiLib,
> >> > and I think everything but Xen uses PEIM instead of PrePi. Also as I
> >> > said I
> >> > have more than one range so this would work anyway.
> >> >
> >>
> >> ArmVirtQemu uses PEIM, since it runs from [emulated] NOR flash.
> >> ArmVirtQemuKernel and ArmVirtXen use PrePi (the former is simply
> >> ArmVirtQemu with the FD executing from DRAM, using the QEMU '-kernel'
> >> option)
> >
> >
>