From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) (using TLSv1 with cipher CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 5820681C8C for ; Thu, 3 Nov 2016 20:05:24 -0700 (PDT) Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga104.fm.intel.com with ESMTP; 03 Nov 2016 20:05:25 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.31,442,1473145200"; d="scan'208,217";a="27231829" Received: from orsmsx107.amr.corp.intel.com ([10.22.240.5]) by fmsmga006.fm.intel.com with ESMTP; 03 Nov 2016 20:05:25 -0700 Received: from orsmsx115.amr.corp.intel.com (10.22.240.11) by ORSMSX107.amr.corp.intel.com (10.22.240.5) with Microsoft SMTP Server (TLS) id 14.3.248.2; Thu, 3 Nov 2016 20:05:25 -0700 Received: from orsmsx113.amr.corp.intel.com ([169.254.9.250]) by ORSMSX115.amr.corp.intel.com ([10.22.240.11]) with mapi id 14.03.0248.002; Thu, 3 Nov 2016 20:05:25 -0700 From: "Kinney, Michael D" To: "Yao, Jiewen" , Laszlo Ersek , "Kinney, Michael D" CC: "Tian, Feng" , =?iso-8859-2?Q?Radim_Kr=3Fm=E1=3F?= , "edk2-devel@ml01.01.org" , Paolo Bonzini , "Fan, Jeff" , "Zeng, Star" Thread-Topic: [edk2] [PATCH 0/6] Enable SMM page level protection. Thread-Index: AQHSNZ8XvoslyVrIbk6dPNkCXTmtl6DIQKiAgAAj+QD//4+B8IAAh/AA//+jakCAAAFJoIAAdy0A//+M7dA= Date: Fri, 4 Nov 2016 03:05:23 +0000 Message-ID: References: <74D8A39837DF1E4DA445A8C0B3885C50386BE1E9@shsmsx102.ccr.corp.intel.com> In-Reply-To: <74D8A39837DF1E4DA445A8C0B3885C50386BE1E9@shsmsx102.ccr.corp.intel.com> Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ctpclassification: CTP_IC x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiMTZmZmZlNTYtOGRmOS00MzJhLThkNWEtN2QyZDkyOTgwNWJmIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX0lDIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE1LjkuNi42IiwiVHJ1c3RlZExhYmVsSGFzaCI6IlRtbXBrVHJJZ1ZQUktUZm5VdDQ0Z2JWTytTTFR4aXZ4ZHR4TGlwbUR4VkU9In0= x-originating-ip: [10.22.254.140] MIME-Version: 1.0 X-Content-Filtered-By: Mailman/MimeDel 2.1.21 Subject: Re: [PATCH 0/6] Enable SMM page level protection. X-BeenThere: edk2-devel@lists.01.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: EDK II Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Nov 2016 03:05:24 -0000 Content-Language: en-US Content-Type: text/plain; charset="iso-8859-2" Content-Transfer-Encoding: quoted-printable Jiewen, I remember now. Ovmf uses a different PCD setting for PcdCpuSmmSyncMode. !if $(SMM_REQUIRE) =3D=3D TRUE gUefiCpuPkgTokenSpaceGuid. PcdCpuSmmSyncMode |0x01 gUefiCpuPkgTokenSpaceGuid.PcdCpuSmmApSyncTimeout|100000 gUefiCpuPkgTokenSpaceGuid.PcdCpuSmmStackSize|0x4000 !endif ## Indicates the CPU synchronization method used when processing an SMI. # 0x00 - Traditional CPU synchronization method.
# 0x01 - Relaxed CPU synchronization method.
# @Prompt SMM CPU Synchronization Method. gUefiCpuPkgTokenSpaceGuid.PcdCpuSmmSyncMode|0x00|UINT8|0x60000014 Try setting is to default 0x0 value and I think all CPUs will come into SMM= . Mike From: Yao, Jiewen Sent: Thursday, November 3, 2016 7:56 PM To: Kinney, Michael D ; Laszlo Ersek Cc: Tian, Feng ; Radim Kr?m=E1? ; = edk2-devel@ml01.01.org; Paolo Bonzini ; Fan, Jeff ; Zeng, Star Subject: RE: [edk2] [PATCH 0/6] Enable SMM page level protection. Thanks Mike. I added "-cpu Nehalem -global ICH9-LPC.disable_s3=3D1", and get same resul= t. APs are not present. Thank you Yao Jiewen From: Kinney, Michael D Sent: Friday, November 4, 2016 10:50 AM To: Yao, Jiewen >; Laszlo= Ersek >; Kinney, Michael D > Cc: Tian, Feng >; Radim Kr?= m=E1? >; edk2-devel@ml01.01.o= rg; Paolo Bonzini >; Fan, Jeff >; Zeng, Star > Subject: RE: [edk2] [PATCH 0/6] Enable SMM page level protection. Jiewen, This is how I launch Qemu on Windows for OvmfPkgIa32X64.dsc. The flags are= not identical to yours. I have these additional flags: -cpu Nehalem -glob= al ICH9-LPC.disable_s3=3D1 set QEMU_PATH=3DC:\Program Files\Qemu set EDKII_BUILD_OUTPUT=3D%WORKSPACE%\Build\Ovmf3264\DEBUG_VS2015x86 echo %EDKII_BUILD_OUTPUT% start "Monitor" /B "c:\Program Files (x86)\teraterm\ttermpro.exe" localhost= :20717 /nossh start "Console" /B "c:\Program Files (x86)\teraterm\ttermpro.exe" localhost= :20716 /nosh start "QEMU" /B "%QEMU_PATH%\qemu-system-x86_64w.exe" -machine q35,smm=3Don= ,accel=3Dtcg -net none -cpu Nehalem -global ICH9-LPC.disable_s3=3D1 ^ -drive if=3Dpflash,format=3Draw,unit=3D0,file=3D%EDKII_BUILD_OUTPUT%\FV\OVM= F_CODE.fd,readonly=3Don ^ -drive if=3Dpflash,format=3Draw,unit=3D1,file=3D%EDKII_BUILD_OUTPUT%\FV\OVM= F_VARS.fd ^ -monitor tcp:localhost:20717,server ^ -serial tcp:localhost:20716,server ^ -smp 8 ^ Best regards, Mike From: Yao, Jiewen Sent: Thursday, November 3, 2016 6:16 PM To: Kinney, Michael D >; Laszlo Ersek > Cc: Tian, Feng >; Radim Kr?= m=E1? >; edk2-devel@ml01.01.o= rg; Paolo Bonzini >; Fan, Jeff >; Zeng, Star > Subject: RE: [edk2] [PATCH 0/6] Enable SMM page level protection. Thank you, Mike. Yes, I reproduced the issue and found out what is wrong. Here is detail: It seems OVMF never puts AP in SMM mode, even *before my patch series*. I rollback all my update and use e9d093. I add some debug message in PerformRemainingTasks(), I found below: *(mSmmMpSyncData->CpuData[0].Present) =3D 1 *(mSmmMpSyncData->CpuData[1].Present) =3D 0 *(mSmmMpSyncData->CpuData[2].Present) =3D 0 *(mSmmMpSyncData->CpuData[3].Present) =3D 0 *(mSmmMpSyncData->CpuData[4].Present) =3D 0 *(mSmmMpSyncData->CpuData[5].Present) =3D 0 *(mSmmMpSyncData->CpuData[6].Present) =3D 0 *(mSmmMpSyncData->CpuData[7].Present) =3D 0 It is never exposed before, because there is no code to call SmmStartupAp. I assume all APs should be present, so I added ASSERT. Based upon current OVMF situation, I will change ASSERT to a DEBUG_ERROR me= ssage. Last but not least important, it is unclear to me, it is a *bug* or a *feat= ure* on OVMF or QEMU. Anyone can confirm that? Thank you Yao Jiewen From: Kinney, Michael D Sent: Friday, November 4, 2016 8:10 AM To: Yao, Jiewen >; Laszlo= Ersek >; Kinney, Michael D > Cc: Tian, Feng >; Radim Kr?= m=E1? >; edk2-devel@ml01.01.o= rg; Paolo Bonzini >; Fan, Jeff >; Zeng, Star > Subject: RE: [edk2] [PATCH 0/6] Enable SMM page level protection. Jiewen, Try setting -smp 8 for 8 CPUs. Mike From: Yao, Jiewen Sent: Thursday, November 3, 2016 4:52 PM To: Laszlo Ersek > Cc: Tian, Feng >; Radim Kr?= m=E1? >; edk2-devel@ml01.01.o= rg; Kinney, Michael D >; Paolo Bonzini >; Fan, Jeff >; Zeng, Star > Subject: RE: [edk2] [PATCH 0/6] Enable SMM page level protection. Hi Laszlo I appreciate your help to validate the patch for me. Yes, I am using a windows QEMU binary. (qemu-system-x86_64.exe) A quick look at the code. 1) The ASSERT issue > ASSERT UefiCpuPkg/PiSmmCpuDxeSmm/SmmCpuMemoryManagement.c(468): !EFI_ERRO= R (Status) is caused by this line: Status =3D SmmBlockingStartupThisAp (FlushTlbOnCurrentProcessor, Inde= x, NULL); ASSERT_EFI_ERROR(Status); I am surprise to see it, because I think is an impossible case - InternalSm= mStartupThisAp(). if (CpuIndex >=3D gSmmCpuPrivate->SmmCoreEntryContext.NumberOfCpus || CpuIndex =3D=3D gSmmCpuPrivate->SmmCoreEntryContext.CurrentlyExecutin= gCpu || !(*(mSmmMpSyncData->CpuData[CpuIndex].Present)) || gSmmCpuPrivate->Operation[CpuIndex] =3D=3D SmmCpuRemove) { return EFI_INVALID_PARAMETER; } I am using below line to start QEMU: "qemu-system-x86_64.exe -machine q35,smm=3Don,accel=3Dtcg -smp 1 -drive if= =3Dpflash,format=3Draw,unit=3D0,file=3DOVMF_CODE.fd,readonly=3Don -drive if= =3Dpflash,format=3Draw,unit=3D1,file=3DOVMF_VARS.fd --serial COM8 -boot men= u=3Don,splash-time=3D7000 fat:rw:Test" Would you mind let me know what is difference between your environment and = mine? I will add more debug info to see what happened, which of above lines trigg= ers the error. 2) The unstable case is a headache. According to "RIP=3D000000000009f0fd", it seems outside of SMM right? What does this *KVM internal error. Suberror: 1* mean? Do you have any clue on what happened? Thank you Yao Jiewen From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of Lasz= lo Ersek Sent: Friday, November 4, 2016 5:43 AM To: Yao, Jiewen > Cc: Tian, Feng >; Radim Kr= =E8m=E1=F8 >; edk2-devel@ml01= .01.org; Kinney, Michael D >; Paolo Bonzini >; Fan, Jeff >; Zeng, Star > Subject: Re: [edk2] [PATCH 0/6] Enable SMM page level protection. On 11/03/16 07:53, Jiewen Yao wrote: > This series patch enables SMM page level protection. > Features are: > 1) PiSmmCore reports SMM PE image code/data information > in EdkiiPiSmmMemoryAttributeTable, if the SMM image is page aligned. > 2) PiSmmCpu consumes EdkiiPiSmmMemoryAttributeTable > and set XD for data page and RO for code page. > 3) PiSmmCpu enables Static Paging for X64 according to > PcdCpuSmmStaticPageTable. If it is true, 1G paging for above 4G > is used as long as it is supported. > 4) PiSmmCpu sets importance data structure to be read only, > such as Gdt, Idt, SmmEntrypoint, and PageTable itself. > > tested platform: > 1) Intel internal platform (X64). > 2) EDKII Quark IA32 > 3) EDKII Vlv2 X64 > 4) EDKII OVMF IA32 and IA32X64. Did you use a Windows QEMU binary for the OVMF tests? Please find my test results below, all done on KVM (3.10.0-514.el7.x86_64, if anyone is curious :)). > > Cc: Jeff Fan > > Cc: Feng Tian > > Cc: Star Zeng > > Cc: Michael D Kinney > > Cc: Laszlo Ersek > > Contributed-under: TianoCore Contribution Agreement 1.0 > Signed-off-by: Jiewen Yao > > > Jiewen Yao (6): > MdeModulePkg/Include: Add PiSmmMemoryAttributesTable.h > MdeModulePkg/dec: Add gEdkiiPiSmmMemoryAttributesTableGuid. > MdeModulePkg/PiSmmCore: Add MemoryAttributes support. > UefiCpuPkg/dec: Add PcdCpuSmmStaticPageTable. > UefiCpuPkg/PiSmmCpuDxeSmm: Add paging protection. > QuarkPlatformPkg/dsc: enable Smm paging protection. Legend: - "untested" means the test was not executed because the same test failed or proved unreliable in a less demanding configuration already, - "n/a" means a setting or test case was impossible, - "fail" and "unreliable" (lower case) are outside the scope of this series; they either capture the pre-series status, or are expected even with the series applied due to the pre-series status, - "FAIL" and "UNRELIABLE" mean regressions caused (or exposed) by the series. In all cases, 36 bits were used as address width in the CPU HOB (--> up to 64GB guest-phys address space). series OVMF = VCPU boot S3 resume # applied platform PcdCpuMaxLogicalProcessorNumber PcdCpuSmmStaticPageTabl= e topology result result -- ------- -------- ------------------------------- -----------------------= - -------- ------ --------- 1 no Ia32 64 n/a = 1x2x2 pass unreliable 2 no Ia32 255 n/a = 52x2x2 pass untested 3 no Ia32 255 n/a = 53x2x2 unreliable untested 4 no Ia32X64 64 n/a = 1x2x2 pass pass 5 no Ia32X64 255 n/a = 52x2x2 pass pass 6 no Ia32X64 255 n/a = 54x2x2 fail n/a 7 yes Ia32 64 FALSE = 1x2x2 FAIL n/a 8 yes Ia32 64 TRUE = 1x2x2 FAIL n/a 9 yes Ia32 255 FALSE = 52x2x2 untested untested 10 yes Ia32 255 FALSE = 53x2x2 untested untested 11 yes Ia32 255 TRUE = 52x2x2 untested untested 12 yes Ia32 255 TRUE = 53x2x2 untested untested 13 yes Ia32X64 64 FALSE = 1x2x2 pass UNRELIABLE 14 yes Ia32X64 64 TRUE = 1x2x2 FAIL n/a 15 yes Ia32X64 255 FALSE = 52x2x2 pass untested 16 yes Ia32X64 255 FALSE = 54x2x2 fail untested 17 yes Ia32X64 255 TRUE = 52x2x2 untested untested 18 yes Ia32X64 255 TRUE = 54x2x2 untested untested Notes for the baseline tests (1-6): * test 3: boot unreliable due to out-of-SMRAM (depends on what we do during boot) * test 5: S3 wakeup: 1 try, suspend takes very long, CPU intensive; wakeup is quick, works okay * test 6: boot fail: out of SMRAM (expected) Notes for the tests with the series applied: * test 7: normal boot is regressed even with PcdCpuSmmStaticPageTable=3DFALSE, relative to test 1: > SmmInstallProtocolInterface: [EfiSmmReadyToLockProtocol] 0 > Patch page table start ... > Patch page table done! > MemoryAttributesTable - NULL > SetPageTableAttributes > Start... > ConvertPageEntryAttribute 0x7FF9A067->0x7FF9A065 > > ASSERT_EFI_ERROR (Status =3D Invalid Parameter) > ASSERT UefiCpuPkg/PiSmmCpuDxeSmm/SmmCpuMemoryManagement.c(468): !EFI_ERRO= R (Status) Note that the processor model used in this test has no support for NX. * test 8: results identical to those of test 7 * test 13: S3 resume is unreliable (regression relative to test 4). End of OVMF log (Fedora guest): > S3_BOOT_SCRIPT_LIB_TERMINATE_OPCODE > S3BootScriptDone - Success > Call AsmDisablePaging64() to return to S3 Resume in PEI Phase > Install PPI: [PeiPostScriptTablePpi] > Install PPI: [EfiEndOfPeiSignalPpi] > Notify: PPI Guid: [EfiEndOfPeiSignalPpi], Peim notify entry point: 857895 > PeiMpInitLib: CpuMpEndOfPeiCallback () invoked > Transfer to 16bit OS waking vector - 9A1D0 QEMU log: > KVM internal error. Suberror: 1 > KVM internal error. Suberror: 1 > emulation failure > emulation failure > RAX=3D0000000000000001 RBX=3D0000000000000000 RCX=3D000000007ffdbf48 RDX= =3D000000007fede058 > RSI=3D0000000000000004 RDI=3D000000007fede040 RBP=3D0000000000000000 RSP= =3D000000007e19f000 > R8 =3D0000000000000000 R9 =3D0000000000000000 R10=3D0000000000000000 R11= =3D0000000000000000 > R12=3D0000000000000000 R13=3D0000000000000000 R14=3D0000000000000000 R15= =3D0000000000000000 > RIP=3D000000000009f0fd RFL=3D00010006 [-----P-] CPL=3D0 II=3D0 A20=3D1 SM= M=3D0 HLT=3D0 > ES =3D0030 0000000000000000 ffffffff 00c09300 DPL=3D0 DS [-WA] > CS =3D0038 0000000000000000 ffffffff 00a09b00 DPL=3D0 CS64 [-RA] > SS =3D0030 0000000000000000 ffffffff 00c09300 DPL=3D0 DS [-WA] > DS =3D0030 0000000000000000 ffffffff 00c09300 DPL=3D0 DS [-WA] > FS =3D0018 0000000000000000 ffffffff 00c09300 DPL=3D0 DS [-WA] > GS =3D0018 0000000000000000 ffffffff 00c09300 DPL=3D0 DS [-WA] > LDT=3D0000 0000000000000000 0000ffff 00008200 DPL=3D0 LDT > TR =3D0000 0000000000000000 0000ffff 00008b00 DPL=3D0 TSS64-busy > GDT=3D 000000007f294000 00000047 > IDT=3D 000000007f294048 00000fff > CR0=3De0000011 CR2=3D0000000000000000 CR3=3D000000007ff80000 CR4=3D000002= 20 > DR0=3D0000000000000000 DR1=3D0000000000000000 DR2=3D0000000000000000 DR3= =3D0000000000000000 > DR6=3D00000000ffff0ff0 DR7=3D0000000000000400 > EFER=3D0000000000000500 > Code=3D?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?= ? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?= ? ?? ?? ?? > RAX=3D0000000000000002 RBX=3D0000000000000000 RCX=3D000000007ffdbf48 RDX= =3D000000007fede070 > RSI=3D0000000000000004 RDI=3D000000007fede058 RBP=3D0000000000000000 RSP= =3D000000007e1a7000 > R8 =3D0000000000000000 R9 =3D0000000000000000 R10=3D0000000000000000 R11= =3D0000000000000000 > R12=3D0000000000000000 R13=3D0000000000000000 R14=3D0000000000000000 R15= =3D0000000000000000 > RIP=3D000000000009f0fd RFL=3D00010006 [-----P-] CPL=3D0 II=3D0 A20=3D1 SM= M=3D0 HLT=3D0 > ES =3D0030 0000000000000000 ffffffff 00c09300 DPL=3D0 DS [-WA] > CS =3D0038 0000000000000000 ffffffff 00a09b00 DPL=3D0 CS64 [-RA] > SS =3D0030 0000000000000000 ffffffff 00c09300 DPL=3D0 DS [-WA] > DS =3D0030 0000000000000000 ffffffff 00c09300 DPL=3D0 DS [-WA] > FS =3D0018 0000000000000000 ffffffff 00c09300 DPL=3D0 DS [-WA] > GS =3D0018 0000000000000000 ffffffff 00c09300 DPL=3D0 DS [-WA] > LDT=3D0000 0000000000000000 0000ffff 00008200 DPL=3D0 LDT > TR =3D0000 0000000000000000 0000ffff 00008b00 DPL=3D0 TSS64-busy > GDT=3D 000000007f294000 00000047 > IDT=3D 000000007f294048 00000fff > CR0=3De0000011 CR2=3D0000000000000000 CR3=3D000000007ff80000 CR4=3D000002= 20 > DR0=3D0000000000000000 DR1=3D0000000000000000 DR2=3D0000000000000000 DR3= =3D0000000000000000 > DR6=3D00000000ffff0ff0 DR7=3D0000000000000400 > EFER=3D0000000000000500 > Code=3D?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?= ? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?= ? ?? ?? ?? This happened 1 time out of the 2 times I ran the test. * test 14: boot regression relative to test 4. The symptoms are identical to those seen in test 7, that is: > SmmInstallProtocolInterface: [EfiSmmReadyToLockProtocol] 0 > Patch page table start ... > Patch page table done! > MemoryAttributesTable - NULL > SetPageTableAttributes > Start... > ConvertPageEntryAttribute 0x7FF98067->0x7FF98065 > > ASSERT_EFI_ERROR (Status =3D Invalid Parameter) > ASSERT UefiCpuPkg/PiSmmCpuDxeSmm/SmmCpuMemoryManagement.c(468): !EFI_ERRO= R (Status) Summary: - this series seems to break the boot with the Ia32 build of OVMF, regardless of PcdCpuSmmStaticPageTable (cases 7 and 8), - it seems to break the boot with the Ia32X64 build of OVMF, with PcdCpuSmmStaticPageTable=3DTRUE (case 14), - for the Ia32X64 build, with PcdCpuSmmStaticPageTable=3DFALSE, boot works, but S3 becomes unreliable (case 13), - because there was no successful boot with PcdCpuSmmStaticPageTable=3DTRUE, I couldn't check if and how the increased SMRAM demand would impact the maximum count of bootable VCPUs. (For this test, I should likely increase the guest-phys address width from 36 bits anyway.) Thanks Laszlo _______________________________________________ edk2-devel mailing list edk2-devel@lists.01.org https://lists.01.org/mailman/listinfo/edk2-devel