UefiPayloadPkg: assert error in PciHostBridgeDxe

UefiPayloadPkg: assert error in PciHostBridgeDxe

[King Sumo]

[-- Attachment #1: Type: text/plain, Size: 1620 bytes --]

Hi,

When booting UefiPayloadPkg in my system (x86 Denverton SoC, coreboot) an assert error is generated in the PciHostBridgeDxe driver.
In the InitializePciHostBridge() function if all ASSERT's are ignored (by commenting out the code) the boot can move further on until it reaches the UEFI Shell.
Any clues?

Loading driver at 0x0007EE61000 EntryPoint=0x0007EE6CF52 PciHostBridgeDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 7EF1F698
ProtectUefiImageCommon - 0x7EF1F140
- 0x000000007EE61000 - 0x0000000000013000
SetUefiImageMemoryAttributes - 0x000000007EE61000 - 0x0000000000001000 (0x0000000000004008)
SetUefiImageMemoryAttributes - 0x000000007EE62000 - 0x0000000000010000 (0x0000000000020008)
SetUefiImageMemoryAttributes - 0x000000007EE72000 - 0x0000000000002000 (0x0000000000004008)
PROGRESS CODE: V03040002 I0
InitRootBridge: populated root bus 0, with room for 7 subordinate bus(es)
RootBridge: PciRoot(0x0)
Support/Attr: 10003 / 10003
DmaAbove4G: No
NoExtConfSpace: No
AllocAttr: 0 ()
Bus: 0 - 7 Translation=0
Io: 0 - 4FFF Translation=0
Mem: D4000000 - FE0FFFFF Translation=0
MemAbove4G: FFFFFFFFFFFFFFFF - 0 Translation=0
PMem: FFFFFFFFFFFFFFFF - 0 Translation=0
PMemAbove4G: FFFFFFFFFFFFFFFF - 0 Translation=0
PciHostBridgeDxe: IntersectMemoryDescriptor: desc [E0000000, F0000000) type 1 cap 870000000002600F conflicts with aperture [D4000000, FE100000) cap 1

ASSERT_EFI_ERROR (Status = Invalid Parameter)
ASSERT [PciHostBridgeDxe] /home/lxuser/occ/edk2/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c(488): !EFI_ERROR (Status)

Thanks,
Sumo

[-- Attachment #2: Type: text/html, Size: 2182 bytes --]

Re: [edk2-devel] UefiPayloadPkg: assert error in PciHostBridgeDxe

[Andrew Fish]

[-- Attachment #1: Type: text/plain, Size: 2308 bytes --]



> On Jul 2, 2020, at 3:54 PM, King Sumo <kingsumos@gmail.com> wrote:
> 
> Hi,
> 
> When booting UefiPayloadPkg in my system (x86 Denverton SoC, coreboot) an assert error is generated in the PciHostBridgeDxe driver.
> In the InitializePciHostBridge() function if all ASSERT's are ignored (by commenting out the code) the boot can move further on until it reaches the UEFI Shell.
> Any clues?
> 
> Loading driver at 0x0007EE61000 EntryPoint=0x0007EE6CF52 PciHostBridgeDxe.efi
> InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 7EF1F698
> ProtectUefiImageCommon - 0x7EF1F140
>   - 0x000000007EE61000 - 0x0000000000013000
> SetUefiImageMemoryAttributes - 0x000000007EE61000 - 0x0000000000001000 (0x0000000000004008)
> SetUefiImageMemoryAttributes - 0x000000007EE62000 - 0x0000000000010000 (0x0000000000020008)
> SetUefiImageMemoryAttributes - 0x000000007EE72000 - 0x0000000000002000 (0x0000000000004008)
> PROGRESS CODE: V03040002 I0
> InitRootBridge: populated root bus 0, with room for 7 subordinate bus(es)
> RootBridge: PciRoot(0x0)
>   Support/Attr: 10003 / 10003
>     DmaAbove4G: No
> NoExtConfSpace: No
>      AllocAttr: 0 ()
>            Bus: 0 - 7 Translation=0
>             Io: 0 - 4FFF Translation=0
>            Mem: D4000000 - FE0FFFFF Translation=0
>     MemAbove4G: FFFFFFFFFFFFFFFF - 0 Translation=0
>           PMem: FFFFFFFFFFFFFFFF - 0 Translation=0
>    PMemAbove4G: FFFFFFFFFFFFFFFF - 0 Translation=0
> PciHostBridgeDxe: IntersectMemoryDescriptor: desc [E0000000, F0000000) type 1 cap 870000000002600F conflicts with aperture [D4000000, FE100000) cap 1
>

It looks like the above request for "D4000000 - FE0FFFFF” overlaps with an existing mapping (E0000000, F0000000). The edk2 has something called GCD (Global Coherency Domain) that is kind of like malloc for MMIO space, and what part of the CPU address has DRAM. So it looks like 2 device think they own (E0000000, F0000000), and there can be only one. You may be getting lucky that your hardware works, but something seems miss configured. 

Thanks,

Andrew Fish

> ASSERT_EFI_ERROR (Status = Invalid Parameter)
> ASSERT [PciHostBridgeDxe] /home/lxuser/occ/edk2/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c(488): !EFI_ERROR (Status)
> 
> Thanks,
> Sumo
> 
> 


[-- Attachment #2: Type: text/html, Size: 3630 bytes --]

Re: [edk2-devel] UefiPayloadPkg: assert error in PciHostBridgeDxe

[Laszlo Ersek]

On 07/03/20 01:03, Andrew Fish via groups.io wrote:
> 
> 
>> On Jul 2, 2020, at 3:54 PM, King Sumo <kingsumos@gmail.com> wrote:
>>
>> Hi,
>>
>> When booting UefiPayloadPkg in my system (x86 Denverton SoC, coreboot) an assert error is generated in the PciHostBridgeDxe driver.
>> In the InitializePciHostBridge() function if all ASSERT's are ignored (by commenting out the code) the boot can move further on until it reaches the UEFI Shell.
>> Any clues?
>>
>> Loading driver at 0x0007EE61000 EntryPoint=0x0007EE6CF52 PciHostBridgeDxe.efi
>> InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 7EF1F698
>> ProtectUefiImageCommon - 0x7EF1F140
>>   - 0x000000007EE61000 - 0x0000000000013000
>> SetUefiImageMemoryAttributes - 0x000000007EE61000 - 0x0000000000001000 (0x0000000000004008)
>> SetUefiImageMemoryAttributes - 0x000000007EE62000 - 0x0000000000010000 (0x0000000000020008)
>> SetUefiImageMemoryAttributes - 0x000000007EE72000 - 0x0000000000002000 (0x0000000000004008)
>> PROGRESS CODE: V03040002 I0
>> InitRootBridge: populated root bus 0, with room for 7 subordinate bus(es)
>> RootBridge: PciRoot(0x0)
>>   Support/Attr: 10003 / 10003
>>     DmaAbove4G: No
>> NoExtConfSpace: No
>>      AllocAttr: 0 ()
>>            Bus: 0 - 7 Translation=0
>>             Io: 0 - 4FFF Translation=0
>>            Mem: D4000000 - FE0FFFFF Translation=0
>>     MemAbove4G: FFFFFFFFFFFFFFFF - 0 Translation=0
>>           PMem: FFFFFFFFFFFFFFFF - 0 Translation=0
>>    PMemAbove4G: FFFFFFFFFFFFFFFF - 0 Translation=0
>> PciHostBridgeDxe: IntersectMemoryDescriptor: desc [E0000000, F0000000) type 1 cap 870000000002600F conflicts with aperture [D4000000, FE100000) cap 1
>>
> 
> It looks like the above request for "D4000000 - FE0FFFFF” overlaps with an existing mapping (E0000000, F0000000). The edk2 has something called GCD (Global Coherency Domain) that is kind of like malloc for MMIO space, and what part of the CPU address has DRAM. So it looks like 2 device think they own (E0000000, F0000000), and there can be only one. You may be getting lucky that your hardware works, but something seems miss configured. 

PciHostBridgeDxe calls the PciHostBridgeGetRootBridges() function from
the platform's "PciHostBridgeLib" instance. The function outputs an
array of PCI_ROOT_BRIDGE structures that describe the PCI root bridges
on the system, including the various resource apertures for each bridge
(expressed as device address ranges).

Then PciHostBridgeDxe tries to allocate the described apertures from the
according resource type GCD spaces (after translating the address ranges
in question from device addresses to host addresses).

The attempt to allocate any one such aperture consists of three steps,
(1) make sure the GCD memory map covers the aperture with compatible
descriptor(s), (2) set the aperture range to uncacheable (for MMIO only;
not defined for IO), (3) actually allocate the aperture.

In step (1), there's a slight trick. We don't call gDS->AddIoSpace() /
gDS->AddMemorySpace() indiscriminately, because the platform may have
populated the GCD memory space / IO space maps before, such that there
could be a (partial or complete) overlap with the aperture being added.

Therefore the internal AddIoSpace() and AddMemoryMappedIoSpace()
functions are idempotent. They contrast every existent GCD space
descriptor with the aperture being added (including gaps, that is,
"nonexistent" type descriptors). For the case when both ranges are fully
distinct, nothing is done. If there is a partial or full overlap, and
the descriptor is "nonexistent", then the intersection is added with
gDS->AddIoSpace() / gDS->AddMemorySpace(). If there is a partial or full
overlap, and the descriptor type is *not* "nonexistent", then the
intersection's *compatibility* is checked, against the requested type
and capabilities of the aperture.

The above procedure ensures that, for any given resource aperture, the
GCD IO or memory space map covers the aperture, with nonzero (that is,
possibly more than one!) adjacent descriptor entries, such that each
descriptor intersecting with the aperture has compatible type and
capabilities with the aperture.

In turn, the error message seen above reports a platform misconfiguration.

It reports that the GCD memory space map already has an entry (a
descriptor) at [E0000000, F0000000), with type 1 (that is,
"EfiGcdMemoryTypeReserved" -- see "MdePkg/Include/Pi/PiDxeCis.h"). The
capabilities of this range are 0x8700_0000_0002_600F

At the same time, the platform's PciHostBridgeLib instance reported a
root bridge with an MMIO aperture at [D4000000, FE100000), with
capabilities 1.

This is a conflict. The capabilities don't even matter (we don't even
check whether the existent GCD descriptor's capabilities are a superset
of the aperture's), because the aperture requires GCD memory type
EfiGcdMemoryTypeMemoryMappedIo, but the GCD descriptor has type
EfiGcdMemoryTypeReserved.

In brief, the failure is due to the platform reporting a PCI root bridge
aperture such that it overlaps an area that is already listed as
"reserved" in the GCD memory space map. So this is a platform bug;
either in the "PciHostBridgeLib" instance, or in the module that
populates the GCD memory space map.

Thanks
Laszlo


> 
> Thanks,
> 
> Andrew Fish
> 
>> ASSERT_EFI_ERROR (Status = Invalid Parameter)
>> ASSERT [PciHostBridgeDxe] /home/lxuser/occ/edk2/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c(488): !EFI_ERROR (Status)
>>
>> Thanks,
>> Sumo
>>
>>
> 
> 
> 
> 
> 

Re: [edk2-devel] UefiPayloadPkg: assert error in PciHostBridgeDxe

[Andrew Fish]

[-- Attachment #1: Type: text/plain, Size: 6229 bytes --]



> On Jul 3, 2020, at 11:06 AM, Laszlo Ersek <lersek@redhat.com> wrote:
> 
> On 07/03/20 01:03, Andrew Fish via groups.io <http://groups.io/> wrote:
>> 
>> 
>>> On Jul 2, 2020, at 3:54 PM, King Sumo <kingsumos@gmail.com> wrote:
>>> 
>>> Hi,
>>> 
>>> When booting UefiPayloadPkg in my system (x86 Denverton SoC, coreboot) an assert error is generated in the PciHostBridgeDxe driver.
>>> In the InitializePciHostBridge() function if all ASSERT's are ignored (by commenting out the code) the boot can move further on until it reaches the UEFI Shell.
>>> Any clues?
>>> 
>>> Loading driver at 0x0007EE61000 EntryPoint=0x0007EE6CF52 PciHostBridgeDxe.efi
>>> InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 7EF1F698
>>> ProtectUefiImageCommon - 0x7EF1F140
>>>  - 0x000000007EE61000 - 0x0000000000013000
>>> SetUefiImageMemoryAttributes - 0x000000007EE61000 - 0x0000000000001000 (0x0000000000004008)
>>> SetUefiImageMemoryAttributes - 0x000000007EE62000 - 0x0000000000010000 (0x0000000000020008)
>>> SetUefiImageMemoryAttributes - 0x000000007EE72000 - 0x0000000000002000 (0x0000000000004008)
>>> PROGRESS CODE: V03040002 I0
>>> InitRootBridge: populated root bus 0, with room for 7 subordinate bus(es)
>>> RootBridge: PciRoot(0x0)
>>>  Support/Attr: 10003 / 10003
>>>    DmaAbove4G: No
>>> NoExtConfSpace: No
>>>     AllocAttr: 0 ()
>>>           Bus: 0 - 7 Translation=0
>>>            Io: 0 - 4FFF Translation=0
>>>           Mem: D4000000 - FE0FFFFF Translation=0
>>>    MemAbove4G: FFFFFFFFFFFFFFFF - 0 Translation=0
>>>          PMem: FFFFFFFFFFFFFFFF - 0 Translation=0
>>>   PMemAbove4G: FFFFFFFFFFFFFFFF - 0 Translation=0
>>> PciHostBridgeDxe: IntersectMemoryDescriptor: desc [E0000000, F0000000) type 1 cap 870000000002600F conflicts with aperture [D4000000, FE100000) cap 1
>>> 
>> 
>> It looks like the above request for "D4000000 - FE0FFFFF” overlaps with an existing mapping (E0000000, F0000000). The edk2 has something called GCD (Global Coherency Domain) that is kind of like malloc for MMIO space, and what part of the CPU address has DRAM. So it looks like 2 device think they own (E0000000, F0000000), and there can be only one. You may be getting lucky that your hardware works, but something seems miss configured. 
> 
> PciHostBridgeDxe calls the PciHostBridgeGetRootBridges() function from
> the platform's "PciHostBridgeLib" instance. The function outputs an
> array of PCI_ROOT_BRIDGE structures that describe the PCI root bridges
> on the system, including the various resource apertures for each bridge
> (expressed as device address ranges).
> 
> Then PciHostBridgeDxe tries to allocate the described apertures from the
> according resource type GCD spaces (after translating the address ranges
> in question from device addresses to host addresses).
> 
> The attempt to allocate any one such aperture consists of three steps,
> (1) make sure the GCD memory map covers the aperture with compatible
> descriptor(s), (2) set the aperture range to uncacheable (for MMIO only;
> not defined for IO), (3) actually allocate the aperture.
> 
> In step (1), there's a slight trick. We don't call gDS->AddIoSpace() /
> gDS->AddMemorySpace() indiscriminately, because the platform may have
> populated the GCD memory space / IO space maps before, such that there
> could be a (partial or complete) overlap with the aperture being added.
> 
> Therefore the internal AddIoSpace() and AddMemoryMappedIoSpace()
> functions are idempotent. They contrast every existent GCD space
> descriptor with the aperture being added (including gaps, that is,
> "nonexistent" type descriptors). For the case when both ranges are fully
> distinct, nothing is done. If there is a partial or full overlap, and
> the descriptor is "nonexistent", then the intersection is added with
> gDS->AddIoSpace() / gDS->AddMemorySpace(). If there is a partial or full
> overlap, and the descriptor type is *not* "nonexistent", then the
> intersection's *compatibility* is checked, against the requested type
> and capabilities of the aperture.
> 
> The above procedure ensures that, for any given resource aperture, the
> GCD IO or memory space map covers the aperture, with nonzero (that is,
> possibly more than one!) adjacent descriptor entries, such that each
> descriptor intersecting with the aperture has compatible type and
> capabilities with the aperture.
> 
> In turn, the error message seen above reports a platform misconfiguration.
> 
> It reports that the GCD memory space map already has an entry (a
> descriptor) at [E0000000, F0000000), with type 1 (that is,
> "EfiGcdMemoryTypeReserved" -- see "MdePkg/Include/Pi/PiDxeCis.h"). The
> capabilities of this range are 0x8700_0000_0002_600F
> 
> At the same time, the platform's PciHostBridgeLib instance reported a
> root bridge with an MMIO aperture at [D4000000, FE100000), with
> capabilities 1.
> 
> This is a conflict. The capabilities don't even matter (we don't even
> check whether the existent GCD descriptor's capabilities are a superset
> of the aperture's), because the aperture requires GCD memory type
> EfiGcdMemoryTypeMemoryMappedIo, but the GCD descriptor has type
> EfiGcdMemoryTypeReserved.
> 
> In brief, the failure is due to the platform reporting a PCI root bridge
> aperture such that it overlaps an area that is already listed as
> "reserved" in the GCD memory space map. So this is a platform bug;
> either in the "PciHostBridgeLib" instance, or in the module that
> populates the GCD memory space map.
> 

Laszlo,

Thanks for the detailed response. 

Sumo,

In your platforms DSC file you can or in 0x00100000 by hand into gEfiMdePkgTokenSpaceGuid.PcdDebugPrintErrorLevel to turn on DEBUG_GCD and get more DEBUG prints about GCD configuration. That may help you track down what is happening. 

Thanks,

Andrew Fish

> Thanks
> Laszlo
> 
> 
>> 
>> Thanks,
>> 
>> Andrew Fish
>> 
>>> ASSERT_EFI_ERROR (Status = Invalid Parameter)
>>> ASSERT [PciHostBridgeDxe] /home/lxuser/occ/edk2/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c(488): !EFI_ERROR (Status)
>>> 
>>> Thanks,
>>> Sumo
>>> 
>>> 
>> 
>> 
>> 


[-- Attachment #2: Type: text/html, Size: 51261 bytes --]

Re: [edk2-devel] UefiPayloadPkg: assert error in PciHostBridgeDxe

[King Sumo]

[-- Attachment #1: Type: text/plain, Size: 1320 bytes --]

On Fri, Jul 3, 2020 at 11:30 AM, Andrew Fish wrote:

> 
> At the same time, the platform's PciHostBridgeLib instance reported a
> root bridge with an MMIO aperture at [D4000000, FE100000), with
> capabilities 1.
> 
> This is a conflict. The capabilities don't even matter (we don't even
> check whether the existent GCD descriptor's capabilities are a superset
> of the aperture's), because the aperture requires GCD memory type
> EfiGcdMemoryTypeMemoryMappedIo, but the GCD descriptor has type
> EfiGcdMemoryTypeReserved.
> 
> In brief, the failure is due to the platform reporting a PCI root bridge
> aperture such that it overlaps an area that is already listed as
> "reserved" in the GCD memory space map. So this is a platform bug;
> either in the "PciHostBridgeLib" instance, or in the module that
> populates the GCD memory space map.

Thanks Andrew and Laszlo!

The map (E0000000, F0000000) is the PCIE Base Address, maybe BlSupportPeim of UefiPayloadPkg have added this in the GCD - I'm not sure if this is correct / normal. Looks like BlSupportPeim is adding to GCD several memory regions during the initialization.
Anyway, the PCIe root bridge MMIO aperture (D4000000, FE100000) looks pretty weird, I'll double check if coreboot is configuring the root port correctly.

Thanks,
Sumo

[-- Attachment #2: Type: text/html, Size: 3316 bytes --]

Re: [edk2-devel] UefiPayloadPkg: assert error in PciHostBridgeDxe

[King Sumo]

[-- Attachment #1: Type: text/plain, Size: 922 bytes --]

For anyone having this issue,the below patch detects the error in PciHostBridge aperture, from 9elements github repository: https://github.com/9elements/edk2-1/commit/6be829a631e3e0e7f6043c88cfef33f9a0588298

also available in system76 repo: https://github.com/system76/edk2.git

commit cf6caab535e84e5d3b3f1462d7ad57c7a914a5b8
Author: Patrick Rudolph <patrick.rudolph@9elements.com>
Date: Mon Oct 12 07:55:44 2020 -0600

UefiPayloadPkg/Library/PciHostBridgeLib: Add warning for coreboot/EDK2 incompability

EDK2 expects PCI bars to be continously be allocated without "gaps" of reserverd
memory in between.
coreboot places PCI bars anyware in the PCI MMIO space, interleaved with MMCONF
and reserved I/O MMIO space.

Warn about this behaviour and refuse to add the BAR to the PCI aperature as it would
cause the PciHostBridgeDxe fo fail.

Signed-off-by: Patrick Rudolph <patrick.rudolph@9elements.com>

[-- Attachment #2: Type: text/html, Size: 1168 bytes --]