Hi, Brian:

Thanks.

That is helpful.

It seems cxl device hot-plug feature depends on pcie bus native hot plug feature.

Not need bios asl code or smm handler code attends it.

OS kernel will cover all procedure.

At 2023-11-08 05:54:01, "Brian J. Johnson" <brian.johnson@hpe.com> wrote:

I see a long discussion on lore.kernel.org:

https://lore.kernel.org/linux-cxl/BYAPR12MB3336F5ADDEDBB6245BC2B48DBD489@BYAPR12MB3336.namprd12.prod.outlook.com/T/

This document also has some info on how CXL hotplug is handled:

https://cdrdv2-public.intel.com/643805/643805_CXL%20Memory%20Device%20SW%20Guide_Rev1p0.pdf

Brian J. Johnson
Enterprise X86 Lab

Hewlett Packard Enterprise

From: Yoshinoya [mailto:yoshinoyatoko@163.com]

Sent: Monday, November 6, 2023 at 5:20 AM

To: devel@edk2.groups.io

Cc: jonathan.cameron@huawei.com, Laszlo Ersek <lersek@redhat.com>, kraxel@redhat.com, Ni, Ray <ray.ni@intel.com>, Sayanta Pattanayak <sayanta.pattanayak@arm.com>

Subject: [edk2-devel] question about cxl device enumeration in pci bus driver
Hi,

I have a question about cxl memory device hot-plug.

For example:

1. When cold boot, the platform has only 512GB cxl type-3 memory

2. During OS runtime, user hot-plug another cxl type-3 memory device expanding to 1TB.

So, how did OS identify another 512GB space newly added without a reboot?

Could anyone help to explain the procedure draftly?

Thanks

在 2023-10-27 09:29:31，"Yoshinoya" <yoshinoyatoko@163.com> 写道：
Hi,

Thanks for reply!

I download code from this git https://github.com/SayantaP-arm/edk2-platforms/

For this ARM edk2 sample package, it provided cxldxe driver which being executed after UEFI DXE PciBus enumeration finishes.

This ppt (https://lpc.events/event/16/contributions/1254/) describes good.

Intel also provied a CXL Type 3 memory device software guide.

This guide also describes system firmware boot sequence and uefi boot sequence.

But i could not match these describes with standard UEFI BIOS Boot flow, such as dxe phase's standard pci enumeration driver.

It sees needing add some cxl discovery code into dxe pci bus driver.

Thanks
At 2023-10-26 21:35:38, "Jonathan Cameron via groups.io" <jonathan.cameron=huawei.com@groups.io> wrote:
>On Thu, 26 Oct 2023 11:49:28 +0200
>"Laszlo Ersek" <lersek@redhat.com> wrote:
>
>> On 10/26/23 10:33, Gerd Hoffmann wrote:
>> > On Thu, Oct 26, 2023 at 10:36:35AM +0800, Yoshinoya wrote:
>> >   
>> >> CXL Host Bridge / Root Port / Switch / Device enumeration / HDM Config, maybe could be integrated into pci drivers stack.  
>> > 
>> > Point being?  Can or should the firmware do anything useful with
>> > the CXL hardware?  If so, what exactly and why?
>> > 
>> > Current state of affairs is that the PCI stack does the usual PCI
>> > initialization (enumerate, assign resources to PCI bars) and leaves
>> > everything else to the OS.  
>> 
>> (I don't know what "HDM Config" stands for.)
>> 
>> The only utility for driving CXL devices from the firmware could be, AFAICT:
>> 
>> - booting off of such a device (or at least "supporting OS boot" in some
>> manner)
>> 
>> - using such a device for UEFI console purposes
>
>There are different models for how to use CXL devices and what's possible depends on the
>version of CXL.  CXL 1.1 wasn't great for standards defined discovery, so
>EDK2 platform logic basically has to do everything.
>
>The one mostly expected for early CXL servers, for backwards compatibility, makes
>setting up the CXL memory decoders (Host managed Device Memory - HDM) in all the
>components in the path to memory + locking them down an EDK2 problem.  They are then
>presented in the memory map and in SRAT, HMAT etc the same as normal DDR memory.
>Idea being that an old OS will be fine with that and doesn't have to be CXL aware
>at all.  Note this also involves walking the CDAT tables via DOE mailboxes in PCI
>config space to get the magic numbers needed to compute HMAT.
>
>The other model is to do very little in EDK2 and make entirely a problem for the OS.
>The logic is necessary anyway if you want to support hotplug etc, so use it for the
>cold plug paths 2.  That's all we've currently supported on QEMU.
>
>There was a presentation at Linux Plumbers last year on some out of tree support
>from ARM for doing the setup on a CXL 2.0 platform (I think) in EDK2 
>https://lpc.events/event/16/contributions/1254/
>But I guess it never went upstream.
>https://github.com/SayantaP-arm/edk2-platforms/tree/cxl-type-3
>
>+CC Sayanta
>
>Jonathan
>> 
>> Laszlo
>> 
>> 
>> 
>> 
>> 
>> 
>
>
>
>
>