From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from rn-mailsvcp-ppex-lapp35.apple.com (rn-mailsvcp-ppex-lapp35.apple.com [17.179.253.44]) by mx.groups.io with SMTP id smtpd.web11.474.1655931245700565460 for ; Wed, 22 Jun 2022 13:54:05 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@apple.com header.s=20180706 header.b=v9/VcHGi; spf=pass (domain: apple.com, ip: 17.179.253.44, mailfrom: afish@apple.com) Received: from pps.filterd (rn-mailsvcp-ppex-lapp35.rno.apple.com [127.0.0.1]) by rn-mailsvcp-ppex-lapp35.rno.apple.com (8.16.1.2/8.16.1.2) with SMTP id 25MKnw7H001989; Wed, 22 Jun 2022 13:54:05 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=apple.com; h=from : message-id : content-type : mime-version : subject : date : in-reply-to : cc : to : references; s=20180706; bh=hetoWY80aKAL1YN4xFVKhtOC01tA0Z3wW2B5u4YrLoo=; b=v9/VcHGiE9203sRI2Wvuum97J+P3s++X2SPbyu1UTjEwAtRkHuoUsDj6BeH6JePlmm8U BzT3OulKLKVAo43lZUHTObsuMc1p6pFtqBzTYtiA1+Efs6oQlvi50pzlVETbhoAVftC6 8trRbljCFKILyuj/UJfd3Z32BV7KsxaO8EVYD0hB75gAWbqSkqTHtZGd+pawFWjEIXPQ wiLPKg9gO0LEpqbRazJgoXOb5OpaFTDFfQLRW12r9n0/larNgOvBRXbxLMNgdGUxhj6t fXZVowPKKcG0W+t4Yet75X8Uzl0faIOamAHc1NO7Wan6+xjg25PeZrgGq9UqOmdBtF+L fQ== Received: from rn-mailsvcp-mta-lapp03.rno.apple.com (rn-mailsvcp-mta-lapp03.rno.apple.com [10.225.203.151]) by rn-mailsvcp-ppex-lapp35.rno.apple.com with ESMTP id 3gsa87fuhh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Wed, 22 Jun 2022 13:54:05 -0700 Received: from rn-mailsvcp-mmp-lapp02.rno.apple.com (rn-mailsvcp-mmp-lapp02.rno.apple.com [17.179.253.15]) by rn-mailsvcp-mta-lapp03.rno.apple.com (Oracle Communications Messaging Server 8.1.0.18.20220407 64bit (built Apr 7 2022)) with ESMTPS id <0RDW010PZCQ5WAK0@rn-mailsvcp-mta-lapp03.rno.apple.com>; Wed, 22 Jun 2022 13:54:05 -0700 (PDT) Received: from process_milters-daemon.rn-mailsvcp-mmp-lapp02.rno.apple.com by rn-mailsvcp-mmp-lapp02.rno.apple.com (Oracle Communications Messaging Server 8.1.0.18.20220407 64bit (built Apr 7 2022)) id <0RDW00Y00CIQJB00@rn-mailsvcp-mmp-lapp02.rno.apple.com>; Wed, 22 Jun 2022 13:54:05 -0700 (PDT) X-Va-A: X-Va-T-CD: b506847712214f169bff5b4513d3a88d X-Va-E-CD: a883af46aa02bf8091f2a6262c209ba2 X-Va-R-CD: 15da91710467abb124c65b2f9a04ce9d X-Va-CD: 0 X-Va-ID: 643ec944-b7bc-4db8-b322-61b53fb00e27 X-V-A: X-V-T-CD: b506847712214f169bff5b4513d3a88d X-V-E-CD: a883af46aa02bf8091f2a6262c209ba2 X-V-R-CD: 15da91710467abb124c65b2f9a04ce9d X-V-CD: 0 X-V-ID: a40bef54-8e10-4cf0-9188-24d3dd250099 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.517,18.0.883 definitions=2022-06-22_08:2022-06-22,2022-06-22 signatures=0 Received: from smtpclient.apple (unknown [17.235.53.23]) by rn-mailsvcp-mmp-lapp02.rno.apple.com (Oracle Communications Messaging Server 8.1.0.18.20220407 64bit (built Apr 7 2022)) with ESMTPSA id <0RDW00HFYCQ4HV00@rn-mailsvcp-mmp-lapp02.rno.apple.com>; Wed, 22 Jun 2022 13:54:05 -0700 (PDT) From: "Andrew Fish" Message-id: <64A70BD6-B1A0-4DDC-8B41-40222258F1EB@apple.com> MIME-version: 1.0 (Mac OS X Mail 15.0 \(3693.20.0.1.32\)) Subject: Re: [edk2-devel] Clarification of Memory management in PEI phase Date: Wed, 22 Jun 2022 13:54:03 -0700 In-reply-to: <76b8a649-c0e1-8ee8-4502-8d6010a76b14@hpe.com> Cc: devel@edk2.groups.io, ayushdevel1325@gmail.com To: "Brian J. Johnson" References: <500949D4-FFB9-4BC1-8655-430BC788765A@apple.com> <76b8a649-c0e1-8ee8-4502-8d6010a76b14@hpe.com> X-Mailer: Apple Mail (2.3693.20.0.1.32) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.517,18.0.883 definitions=2022-06-22_08:2022-06-22,2022-06-22 signatures=0 Content-type: multipart/alternative; boundary="Apple-Mail=_D6DDD31D-C0B2-48B8-9C61-8E8C392F3F0D" --Apple-Mail=_D6DDD31D-C0B2-48B8-9C61-8E8C392F3F0D Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Brian, I think all the PEI Allocate Pool does is make a HOB [1]. I don=E2=80=99t t= hink we can remove HOBs when memory is freed as that would change pointers= to pool. It may be possible to mark a region as free and allocate from tha= t list batch 1st, and just over allocate space if needed. It could be as si= mple as just adding a new HOB type to represent free pool.=20 [1] https://github.com/tianocore/edk2/blob/master/MdeModulePkg/Core/Pei/Mem= ory/MemoryServices.c#L878 Thanks, Andrew Fish > On Jun 22, 2022, at 12:39 PM, Brian J. Johnson wr= ote: >=20 > Sorry for the late response to this thread... >=20 > PEI has grown greatly in complexity, as Andrew pointed out. Do we (Tiano= Core community) think it's time to add a real pool manager to PEI? There's= getting to be quite a bit of post-DRAM-initialization PEI code on some pla= tforms. And really, having a limited amount of pre-DRAM memory available i= s an even better reason for having an effective pool allocator, so the memo= ry can be freed and reused. >=20 > FWIW we (HPE) needed to add a private pool allocator to manage memory for= a proprietary h/w initialization module which needs to run in post-DRAM PE= I, and depends on allocating and freeing memory in different phases of its = operation. >=20 > Brian J. Johnson >=20 > From: Ayush Singh [mailto:ayushdevel1325@gmail.com ] > Sent: Friday, June 10, 2022, 12:22 AM > To: Andrew Fish > Cc: devel@edk2.groups.io > Subject: [edk2-devel] Clarification of Memory management in PEI phase >=20 >> Thanks for the wonderful answer.=20 >>=20 >> Ayush Singh >>=20 >> On Thu, Jun 9 2022 at 01:26:58 PM -0700, Andrew Fish <= mailto:afish@apple.com> wrote: >>>=20 >>> On Jun 9, 2022, at 10:28 AM, Ayush Singh wrote: >>> =20 >>> Hello everyone, >>> =20 >>> Can anyone help me with understanding dynamic memory management in PEI >>> phase? In the UEFI Platform Integration Specification, version 1.7 >>> Errata A, Section 4.6, PEI Memory services are given which include: >>> =20 >>> 1. InstallPeiMemory() >>>=20 >>> This is basically: >>> (*PeiServices)->InstallPeiMemory (PeiServices, MemoryBegin, MemoryLengt= h); >>>=20 >>> This is how you tell the PEI Core the location of the memory that will = can be used in PEI.=20 >>>=20 >>> 2. AllocatePages() >>> 3. AllocatePool() >>> 4. CopyMem() >>> 5. SetMem() >>> 6. FreePages() >>> =20 >>> However, no `FreePool()` service seems to be present. So how is the >>> memory allocated using `AllocatePool()` freed? >>> =20 >>>=20 >>> It basically gets Freed when you transition to the DXE phase.=20 >>>=20 >>> To step back for a minute I think it is important to remember that the = main job of PEI is to initialize DRAM, and deal with S3 (resuming from susp= end to RAM). So as soon as you have DRAM you are kind done and ready for th= e DXE IPL so you can load the DXE Phase and start up EFI. Remember PEI is P= re EFI. The reality is programming DRAM is complex and lots of code got wri= tten, then lots more code got written and PEI has become large for some por= ts. That was never the intent. PEI is designed as a way to run C code when = you code is running from ROM and you don=C4=81=EF=BF=BD=EF=BF=BDt have any = DRAM. For x86 not having DRAM means you are using the cache as RAM. For som= e SoCs there is actually an SRAM you can use. Thus the PEI memory allocatio= n scheme is designed to deal with this very constrained environment.=20 >>>=20 >>> You start PEI with a heap and stack. You can also allocate HOBs (Hand O= ff Blocks). A pool allocation in PEI is just a HOB. See [1]. There is no wa= y to free a HOB. So the AllocatePool() kind of leaks into DXE too as an ent= ry in the HOB list. But when the OS called gBS->ExitBootServices() that fre= es all non runtime memory back to the OS.=20 >>>=20 >>> If you look a AllocatePages/FreePages you will see AllocatePages create= s a HOB that points to the memory region, and FreePages just marks that HOB= as not used. That code is also in this file [1].=20 >>>=20 >>> TL;DR there is no pool manager in PEI.=20 >>>=20 >>> [1] https://github.com/tianocore/edk2/blob/master/MdeModulePkg/Core/Pei= /Memory/MemoryServices.c#L878 >>>=20 >>>=20 >>> Thanks, >>>=20 >>> Andrew Fish >>>=20 >>> Ayush Singh >>> =20 >>> =20 >>> =20 >>> =20 >>> =20 >>>=20 >>=20 >>=20 > Brian >=20 > -------------------------------------------------------------------------= ----------------------------------------------------- > "It's OK to be stuck. 99% of the time in your own [research] work,=20 > you're stuck." > -- Mark Lawrence --Apple-Mail=_D6DDD31D-C0B2-48B8-9C61-8E8C392F3F0D Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 Brian,

I think all the PEI Allocate Pool does is = make a HOB [1]. I don=E2=80=99t think we can remove HOBs when memory is fre= ed as that would change  pointers to pool. It may be possible to mark = a region as free and allocate from that list batch 1st, and just over alloc= ate space if needed. It could be as simple as just adding a new HOB type to= represent free pool. 


[1] https://github.com/tianocore/edk2/blob/master/Md= eModulePkg/Core/Pei/Memory/MemoryServices.c#L878
<= br class=3D"">
Thanks,

Andrew Fish

<= blockquote type=3D"cite" class=3D"">
On Jun 22, 2022, at 12:= 39 PM, Brian J. Johnson <brian.johnson@hpe.com> wrote:

=20
Sorry for the late response to th= is thread...

PEI has grown greatly in complexity, as Andrew pointe= d out.  Do we (TianoCore community) think it's time to add a real pool manager to PEI?  There's getting to be quite a bit of post-DRAM-initialization PEI code on some platforms.  And really, having a limited amount of pre-DRAM memory available is an even better reason for having an effective pool allocator, so the memory can be freed and reused.

FWIW we (HPE) needed to add a private pool allocator = to manage memory for a proprietary h/w initialization module which needs to run in post-DRAM PEI, and depends on allocating and freeing memory in different phases of its operation.

Brian J. Johnson

From: Ayush Singh [mailto:ayushdevel1325@gmail.com]
Sent: Friday, June 10, 2022, 12:22 AM
To: Andrew Fish <afish@apple.com>
Cc: devel@edk2.groups.io
Subject: [edk2-devel] Clarification of Memory management in PEI phase
=20
Thanks for the wonderful answer. 

Ayush Singh

On Thu, Jun 9 2022 at 01:26:58 PM -0700, Andrew Fish = <afish@apple.com> wrote:
On Jun 9, 2022, at 10:28 = AM, Ayush Singh <ayushdevel1325@gmail.comw= rote: =20 Hello everyone, =20 Can anyone help me with understanding dynamic memory management in PEI phase? In the UEFI Platform Integration Specification, version 1.7 Errata A, Section 4.6, PEI Memory services are given which include: =20 1. InstallPeiMemory()
This is basically: (*PeiServices)->InstallPeiMemory (PeiServices, MemoryBegin, MemoryLength= ); This is how you tell the PEI Core the location of the memory that will can = be used in PEI.=20
2. AllocatePages() 3. AllocatePool() 4. CopyMem() 5. SetMem() 6. FreePages() =20 However, no `FreePool()` service seems to be present. So how is the memory allocated using `AllocatePool()` freed? =20
It basically gets Freed when you transition to the DXE phase.=20 To step back for a minute I think it is important to remember that the main= job of PEI is to initialize DRAM, and deal with S3 (resuming from suspend = to RAM). So as soon as you have DRAM you are kind done and ready for the DX= E IPL so you can load the DXE Phase and start up EFI. Remember PEI is Pre E= FI. The reality is programming DRAM is complex and lots of code got written= , then lots more code got written and PEI has become large for some ports. = That was never the intent. PEI is designed as a way to run C code when you = code is running from ROM and you don=C4=81=EF=BF=BD=EF=BF=BDt have any DRAM= . For x86 not having DRAM means you are using the cache as RAM. For some So= Cs there is actually an SRAM you can use. Thus the PEI memory allocation sc= heme is designed to deal with this very constrained environment.=20 You start PEI with a heap and stack. You can also allocate HOBs (Hand Off B= locks). A pool allocation in PEI is just a HOB. See [1]. There is no way to= free a HOB. So the AllocatePool() kind of leaks into DXE too as an entry i= n the HOB list. But when the OS called gBS->ExitBootServices() that free= s all non runtime memory back to the OS.=20 If you look a AllocatePages/FreePages you will see AllocatePages creates a = HOB that points to the memory region, and FreePages just marks that HOB as = not used. That code is also in this file [1].=20 TL;DR there is no pool manager in PEI.=20 [1] https://github.com/tianocore/edk2/blob/master/MdeModule= Pkg/Core/Pei/Memory/MemoryServices.c#L878 Thanks, Andrew Fish
Ayush Singh =20 =20 =20 =20 =20
=20
Brian

---------------------------------------------------------------------------= ---------------------------------------------------
   "It's OK to be stuck.  99% of the time in your own=
 [research] work,=20
    you're stuck."
                                           -- Mark Lawrence

--Apple-Mail=_D6DDD31D-C0B2-48B8-9C61-8E8C392F3F0D--