From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 8304F81F28 for ; Wed, 16 Nov 2016 17:18:42 -0800 (PST) Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga103.fm.intel.com with ESMTP; 16 Nov 2016 17:18:47 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.31,650,1473145200"; d="scan'208";a="1069558276" Received: from fmsmsx103.amr.corp.intel.com ([10.18.124.201]) by fmsmga001.fm.intel.com with ESMTP; 16 Nov 2016 17:18:47 -0800 Received: from fmsmsx115.amr.corp.intel.com (10.18.116.19) by FMSMSX103.amr.corp.intel.com (10.18.124.201) with Microsoft SMTP Server (TLS) id 14.3.248.2; Wed, 16 Nov 2016 17:18:47 -0800 Received: from shsmsx103.ccr.corp.intel.com (10.239.4.69) by fmsmsx115.amr.corp.intel.com (10.18.116.19) with Microsoft SMTP Server (TLS) id 14.3.248.2; Wed, 16 Nov 2016 17:18:46 -0800 Received: from shsmsx102.ccr.corp.intel.com ([169.254.2.239]) by SHSMSX103.ccr.corp.intel.com ([169.254.4.96]) with mapi id 14.03.0248.002; Thu, 17 Nov 2016 09:18:45 +0800 From: "Fan, Jeff" To: Laszlo Ersek , edk2-devel-01 Thread-Topic: [PATCH 4/4] UefiCpuPkg/MpInitLib: support 64-bit AP stack addresses Thread-Index: AQHSQGgUKTxtBGO7zk2iZmTrvJUT/KDcXX6g Date: Thu, 17 Nov 2016 01:18:44 +0000 Message-ID: <542CF652F8836A4AB8DBFAAD40ED192A4A2DFD98@shsmsx102.ccr.corp.intel.com> References: <20161117001754.4383-1-lersek@redhat.com> <20161117001754.4383-5-lersek@redhat.com> In-Reply-To: <20161117001754.4383-5-lersek@redhat.com> Accept-Language: zh-CN, en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiMDRkZTU3OGYtYzcwNy00MmIyLWE5N2ItZWY2N2U0ODk2YWUzIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX0lDIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE1LjkuNi42IiwiVHJ1c3RlZExhYmVsSGFzaCI6ImcxMkhJa1ZEeElkR005MEMyb3Z3U3ZMbHdIMHVoNkJwMVNZY3NXam1kUUE9In0= x-ctpclassification: CTP_IC x-originating-ip: [10.239.127.40] MIME-Version: 1.0 Subject: Re: [PATCH 4/4] UefiCpuPkg/MpInitLib: support 64-bit AP stack addresses X-BeenThere: edk2-devel@lists.01.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: EDK II Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Nov 2016 01:18:42 -0000 Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Laszlo, We have two solutions to fix stack > 4G issue. 1. Allocate AP stack buffer and all CPU MP data buffer under < 4G at the be= ginning. 2. Support AP stack buffer and all CPU MP data buffer > 4G as showed in you= r patch. For 1), it seems not necessary. For 2), besides your patch. We still need to update RelocateApLoop() in Dxe= MpLib.c to use one separate stack under 4G when paging disabled on long mod= e DXE. (Currently, we still use AP existing stack after paging disable= d) I prefer the 2), please go ahead to check-in this serial of patch. I will c= reate another patch to fix RelocateApLoop() stack issue. Reviewed-by: Jeff Fan Thanks! Jeff =20 -----Original Message----- From: Laszlo Ersek [mailto:lersek@redhat.com]=20 Sent: Thursday, November 17, 2016 8:18 AM To: edk2-devel-01 Cc: Fan, Jeff Subject: [PATCH 4/4] UefiCpuPkg/MpInitLib: support 64-bit AP stack addresse= s The cached "CPU_INFO_IN_HOB.ApTopOfStack" field currently has type UINT32. This is not ideal because the AP stacks are located within "CpuMpData->Buff= er", which is allocated with a plain AllocatePages() call in MpInitLibIniti= alize(): platform CpuMpPei included PEI RAM > 4GB result -------- ----------------- ------------- ------ Ia32 * n/a good Ia32X64 no n/a BAD Ia32X64 yes n/a good X64 no * BAD X64 yes no good X64 yes yes BAD - If we are on an Ia32X64 or X64 platform that does not include CpuMpPei, then CpuDxe cannot reuse the CPU_INFO_IN_HOB structures preallocated by CpuMpPei (through the CpuInitMpLib GUID HOB), and then AllocatePages() -- invoked first in 64-bit DXE -- could return an address outside of 32-bit address space. - If we are on an X64 platform where the permanent PEI RAM extends above the 32-bit address space, then the same issue can surface even if CpuMpPei is included: even the original allocation of the CPU_INFO_IN_HOB structures, by CpuMpPei, could be satisfied from above 4GB. The original "AP init" branch in "X64/MpFuncs.nasm" correctly considers a 6= 4-bit stack start: the "MP_CPU_EXCHANGE_INFO.StackStart" field has type UIN= TN, and the code uses QWORD addition and movement to set RSP from it. Adapt the "GetApicId" branch of "X64/MpFuncs.nasm": - change the type of "CPU_INFO_IN_HOB.ApTopOfStack" to UINT64, - remove the explicit truncation to UINT32 in InitializeApData(), - update the "GetNextProcNumber" iteration size to the new size of "CPU_INFO_IN_HOB", - set RSP with a QWORD movement from "CPU_INFO_IN_HOB.ApTopOfStack". Because the same CPU_INFO_IN_HOB structure is used by "Ia32/MpFuncs.nasm", = we have to update the "GetNextProcNumber" iteration size there as well. The ESP setting can be preserved as a DWORD movement from the original offs= et (decimal 12), since our integers are little endian. Cc: Jeff Fan Fixes: 845c5be1fd9bf7edfac4a103dfab70829686978f Contributed-under: TianoCore Contribution Agreement 1.0 Signed-off-by: Laszlo Ersek --- UefiCpuPkg/Library/MpInitLib/MpLib.h | 4 +++- UefiCpuPkg/Library/MpInitLib/MpLib.c | 8 ++++---- UefiCpuPkg/Library/MpInitLib/Ia32/MpFuncs.nasm | 2 +- UefiCpuPkg/Library/= MpInitLib/X64/MpFuncs.nasm | 5 ++--- 4 files changed, 10 insertions(+), 9 deletions(-) diff --git a/UefiCpuPkg/Library/MpInitLib/MpLib.h b/UefiCpuPkg/Library/MpIn= itLib/MpLib.h index 0ac777a099b1..f73a469ae84f 100644 --- a/UefiCpuPkg/Library/MpInitLib/MpLib.h +++ b/UefiCpuPkg/Library/MpInitLib/MpLib.h @@ -126,18 +126,20 @@ typedef struct { // // Basic CPU information saved in Guided HOB. // Because the contents will be shard between PEI and DXE, // we need to = make sure the each fields offset same in different // architecture. // +#pragma pack (1) typedef struct { UINT32 InitialApicId; UINT32 ApicId; UINT32 Health; - UINT32 ApTopOfStack; + UINT64 ApTopOfStack; } CPU_INFO_IN_HOB; +#pragma pack () =20 // // AP reset code information including code address and size, // this str= ucture will be shared be C code and assembly code. // It is natural aligned by design. // diff --git a/UefiCpuPkg/Library/MpInitLib/MpLib.c b/UefiCpuPkg/Library/MpIn= itLib/MpLib.c index 3c2e6d6b89d9..15dbfa1e7d6c 100644 --- a/UefiCpuPkg/Library/MpInitLib/MpLib.c +++ b/UefiCpuPkg/Library/MpInitLib/MpLib.c @@ -430,22 +430,22 @@ CollectProcessorCount ( **/ VOID InitializeApData = ( IN OUT CPU_MP_DATA *CpuMpData, IN UINTN ProcessorNumber, IN UINT32 BistData, - IN UINTN ApTopOfStack + IN UINT64 ApTopOfStack ) { CPU_INFO_IN_HOB *CpuInfoInHob; =20 CpuInfoInHob =3D (CPU_INFO_IN_HOB *) (UINTN) CpuMpData->CpuInfoInHob; CpuInfoInHob[ProcessorNumber].InitialApicId =3D GetInitialApicId (); CpuInfoInHob[ProcessorNumber].ApicId =3D GetApicId (); CpuInfoInHob[ProcessorNumber].Health =3D BistData; - CpuInfoInHob[ProcessorNumber].ApTopOfStack =3D (UINT32) ApTopOfStack; + CpuInfoInHob[ProcessorNumber].ApTopOfStack =3D ApTopOfStack; =20 CpuMpData->CpuData[ProcessorNumber].Waiting =3D FALSE; CpuMpData->CpuData[ProcessorNumber].CpuHealthy =3D (BistData =3D=3D 0) ?= TRUE : FALSE; if (CpuInfoInHob[ProcessorNumber].InitialApicId >=3D 0xFF) { // // Set x2APIC mode if there are any logical processor reporting @@ -47= 7,13 +477,13 @@ ApWakeupFunction ( UINTN ProcessorNumber; EFI_AP_PROCEDURE Procedure; VOID *Parameter; UINT32 BistData; volatile UINT32 *ApStartupSignalBuffer; CPU_INFO_IN_HOB *CpuInfoInHob; - UINTN ApTopOfStack; + UINT64 ApTopOfStack; =20 // // AP finished assembly code and begin to execute C code // CpuMpData =3D ExchangeInfo->CpuMpData; =20 @@ -497,13 +497,13 @@ ApWakeupFunction ( InterlockedIncrement ((UINT32 *) &CpuMpData->CpuCount); ProcessorNumber =3D NumApsExecuting; // // This is first time AP wakeup, get BIST information from AP stack // ApTopOfStack =3D CpuMpData->Buffer + (ProcessorNumber + 1) * CpuMpD= ata->CpuApStackSize; - BistData =3D *(UINT32 *) (ApTopOfStack - sizeof (UINTN)); + BistData =3D *(UINT32 *) ((UINTN) ApTopOfStack - sizeof (UINTN)); // // Do some AP initialize sync // ApInitializeSync (CpuMpData); // // Sync BSP's Control registers to APs diff --git a/UefiCpuPkg/Libra= ry/MpInitLib/Ia32/MpFuncs.nasm b/UefiCpuPkg/Library/MpInitLib/Ia32/MpFuncs.= nasm index 4bfa084c85a9..64e51d87ae24 100644 --- a/UefiCpuPkg/Library/MpInitLib/Ia32/MpFuncs.nasm +++ b/UefiCpuPkg/Library/MpInitLib/Ia32/MpFuncs.nasm @@ -178,13 +178,13 @@ GetProcessorNumber: lea eax, [esi + CpuInfoLocation] mov edi, [eax] =20 GetNextProcNumber: cmp [edi], edx ; APIC ID match? jz ProgramStack - add edi, 16 + add edi, 20 inc ebx jmp GetNextProcNumber =20 =20 ProgramStack: mov esp, [edi + 12] =20 diff --git a/UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm b/UefiCpuPkg/Lib= rary/MpInitLib/X64/MpFuncs.nasm index 138b97312b1d..aaabb50c5468 100644 --- a/UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm +++ b/UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm @@ -182,19 +182,18 @@ GetProcessorNumber: lea eax, [esi + CpuInfoLocation] mov edi, [eax] =20 GetNextProcNumber: cmp dword [edi], edx ; APIC ID match? jz ProgramStack - add edi, 16 + add edi, 20 inc ebx jmp GetNextProcNumber =20 =20 ProgramStack: - xor rsp, rsp - mov esp, dword [edi + 12] + mov rsp, qword [edi + 12] =20 CProcedureInvoke: push rbp ; Push BIST data at top of AP stack xor rbp, rbp ; Clear ebp for call stack trace push rbp mov rbp, rsp -- 2.9.2