Re: [edk2-devel] question about cxl device enumeration in pci bus driver

["Brian J. Johnson" <brian.johnson@hpe.com>]

[-- Attachment #1: Type: text/plain, Size: 5786 bytes --]

I see a long discussion on lore.kernel.org:

https://lore.kernel.org/linux-cxl/BYAPR12MB3336F5ADDEDBB6245BC2B48DBD489@BYAPR12MB3336.namprd12.prod.outlook.com/T/

This document also has some info on how CXL hotplug is handled:

https://cdrdv2-public.intel.com/643805/643805_CXL%20Memory%20Device%20SW%20Guide_Rev1p0.pdf

*Brian J. Johnson
*Enterprise X86 Lab

Hewlett Packard Enterprise

------------------------------------------------------------------------
*From:* Yoshinoya [mailto:yoshinoyatoko@163.com]
*Sent:* Monday, November 6, 2023 at 5:20 AM
*To:* devel@edk2.groups.io
*Cc:* jonathan.cameron@huawei.com, Laszlo Ersek <lersek@redhat.com>, 
kraxel@redhat.com, Ni, Ray <ray.ni@intel.com>, Sayanta Pattanayak 
<sayanta.pattanayak@arm.com>
*Subject:* [edk2-devel] question about cxl device enumeration in pci bus 
driver

> Hi,
> I have a question about cxl memory device hot-plug.
>
> For example:
> 1. When cold boot, the platform has only 512GB cxl type-3 memory
> 2. During OS runtime, user hot-plug another cxl type-3  memory device 
> expanding to 1TB.
>
> So, how did OS identify another 512GB space newly added without a reboot?
>
> Could anyone help to explain the procedure draftly?
>
> Thanks
>
>
>
>
> 在 2023-10-27 09:29:31，"Yoshinoya" <yoshinoyatoko@163.com> 写道：
>
>
>         Hi,
>         Thanks for reply!
>
>         I download code from this git
>         https://github.com/SayantaP-arm/edk2-platforms/
>
>         For this ARM edk2 sample package, it provided cxldxe driver
>         which being executed after UEFI DXE PciBus enumeration finishes.
>         This ppt (https://lpc.events/event/16/contributions/1254/
>         <https://lpc.events/event/16/contributions/1254/>)
>         describes good.
>
>         Intel also provied a CXL Type 3 memory device software guide.
>         This guide also describes system firmware boot sequence and
>         uefi boot sequence.
>         But i could not match these describes with standard UEFI BIOS
>         Boot flow, such as dxe phase's standard pci enumeration driver.
>         It sees needing add some cxl discovery code into dxe pci bus
>         driver.
>
>         Thanks
>
>
>
>
>
>         At 2023-10-26 21:35:38, "Jonathan Cameron via groups.io"<jonathan.cameron=huawei.com@groups.io>  wrote:
>         >On Thu, 26 Oct 2023 11:49:28 +0200
>         >"Laszlo Ersek" <lersek@redhat.com>  wrote:
>         >
>         >> On 10/26/23 10:33, Gerd Hoffmann wrote:
>         >> > On Thu, Oct 26, 2023 at 10:36:35AM +0800, Yoshinoya wrote:
>         >> >   
>         >> >> CXL Host Bridge / Root Port / Switch / Device enumeration / HDM Config, maybe could be integrated into pci drivers stack.  
>         >> > 
>         >> > Point being?  Can or should the firmware do anything useful with
>         >> > the CXL hardware?  If so, what exactly and why?
>         >> > 
>         >> > Current state of affairs is that the PCI stack does the usual PCI
>         >> > initialization (enumerate, assign resources to PCI bars) and leaves
>         >> > everything else to the OS.  
>         >> 
>         >> (I don't know what "HDM Config" stands for.)
>         >> 
>         >> The only utility for driving CXL devices from the firmware could be, AFAICT:
>         >> 
>         >> - booting off of such a device (or at least "supporting OS boot" in some
>         >> manner)
>         >> 
>         >> - using such a device for UEFI console purposes
>         >
>         >There are different models for how to use CXL devices and what's possible depends on the
>         >version of CXL.  CXL 1.1 wasn't great for standards defined discovery, so
>         >EDK2 platform logic basically has to do everything.
>         >
>         >The one mostly expected for early CXL servers, for backwards compatibility, makes
>         >setting up the CXL memory decoders (Host managed Device Memory - HDM) in all the
>         >components in the path to memory + locking them down an EDK2 problem.  They are then
>         >presented in the memory map and in SRAT, HMAT etc the same as normal DDR memory.
>         >Idea being that an old OS will be fine with that and doesn't have to be CXL aware
>         >at all.  Note this also involves walking the CDAT tables via DOE mailboxes in PCI
>         >config space to get the magic numbers needed to compute HMAT.
>         >
>         >The other model is to do very little in EDK2 and make entirely a problem for the OS.
>         >The logic is necessary anyway if you want to support hotplug etc, so use it for the
>         >cold plug paths 2.  That's all we've currently supported on QEMU.
>         >
>         >There was a presentation at Linux Plumbers last year on some out of tree support
>         >from ARM for doing the setup on a CXL 2.0 platform (I think) in EDK2 
>         >https://lpc.events/event/16/contributions/1254/  <https://lpc.events/event/16/contributions/1254/>
>         >But I guess it never went upstream.
>         >https://github.com/SayantaP-arm/edk2-platforms/tree/cxl-type-3
>         >
>         >+CC Sayanta
>         >
>         >Jonathan
>         >> 
>         >> Laszlo
>         >> 
>         >> 
>         >> 
>         >> 
>         >> 
>         >> 
>         >
>         >
>         >
>         >
>         >
>
> 


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#110877): https://edk2.groups.io/g/devel/message/110877
Mute This Topic: https://groups.io/mt/102173204/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-



[-- Attachment #2: Type: text/html, Size: 12817 bytes --]