From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) (using TLSv1 with cipher CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id C95B081D41 for ; Thu, 3 Nov 2016 18:15:55 -0700 (PDT) Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga101.fm.intel.com with ESMTP; 03 Nov 2016 18:15:57 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.31,589,1473145200"; d="scan'208,217";a="27284043" Received: from fmsmsx104.amr.corp.intel.com ([10.18.124.202]) by fmsmga005.fm.intel.com with ESMTP; 03 Nov 2016 18:15:57 -0700 Received: from fmsmsx155.amr.corp.intel.com (10.18.116.71) by fmsmsx104.amr.corp.intel.com (10.18.124.202) with Microsoft SMTP Server (TLS) id 14.3.248.2; Thu, 3 Nov 2016 18:15:57 -0700 Received: from shsmsx103.ccr.corp.intel.com (10.239.4.69) by FMSMSX155.amr.corp.intel.com (10.18.116.71) with Microsoft SMTP Server (TLS) id 14.3.248.2; Thu, 3 Nov 2016 18:15:56 -0700 Received: from shsmsx102.ccr.corp.intel.com ([169.254.2.206]) by SHSMSX103.ccr.corp.intel.com ([169.254.4.139]) with mapi id 14.03.0248.002; Fri, 4 Nov 2016 09:15:54 +0800 From: "Yao, Jiewen" To: "Kinney, Michael D" , Laszlo Ersek CC: "Tian, Feng" , =?iso-8859-2?Q?Radim_Kr=3Fm=E1=3F?= , "edk2-devel@ml01.01.org" , Paolo Bonzini , "Fan, Jeff" , "Zeng, Star" Thread-Topic: [edk2] [PATCH 0/6] Enable SMM page level protection. Thread-Index: AQHSNZ8YswckCVz0V0usOM0dRj4P2qDHRTOAgACfFwD//4nfgIAAluYA Date: Fri, 4 Nov 2016 01:15:52 +0000 Message-ID: <74D8A39837DF1E4DA445A8C0B3885C50386BDFD3@shsmsx102.ccr.corp.intel.com> References: <1478156028-21572-1-git-send-email-jiewen.yao@intel.com> <74D8A39837DF1E4DA445A8C0B3885C50386BDEF5@shsmsx102.ccr.corp.intel.com> In-Reply-To: Accept-Language: zh-CN, en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.239.127.40] MIME-Version: 1.0 X-Content-Filtered-By: Mailman/MimeDel 2.1.21 Subject: Re: [PATCH 0/6] Enable SMM page level protection. X-BeenThere: edk2-devel@lists.01.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: EDK II Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Nov 2016 01:15:56 -0000 Content-Language: en-US Content-Type: text/plain; charset="iso-8859-2" Content-Transfer-Encoding: quoted-printable Thank you, Mike. Yes, I reproduced the issue and found out what is wrong. Here is detail: It seems OVMF never puts AP in SMM mode, even *before my patch series*. I rollback all my update and use e9d093. I add some debug message in PerformRemainingTasks(), I found below: *(mSmmMpSyncData->CpuData[0].Present) =3D 1 *(mSmmMpSyncData->CpuData[1].Present) =3D 0 *(mSmmMpSyncData->CpuData[2].Present) =3D 0 *(mSmmMpSyncData->CpuData[3].Present) =3D 0 *(mSmmMpSyncData->CpuData[4].Present) =3D 0 *(mSmmMpSyncData->CpuData[5].Present) =3D 0 *(mSmmMpSyncData->CpuData[6].Present) =3D 0 *(mSmmMpSyncData->CpuData[7].Present) =3D 0 It is never exposed before, because there is no code to call SmmStartupAp. I assume all APs should be present, so I added ASSERT. Based upon current OVMF situation, I will change ASSERT to a DEBUG_ERROR me= ssage. Last but not least important, it is unclear to me, it is a *bug* or a *feat= ure* on OVMF or QEMU. Anyone can confirm that? Thank you Yao Jiewen From: Kinney, Michael D Sent: Friday, November 4, 2016 8:10 AM To: Yao, Jiewen ; Laszlo Ersek ; K= inney, Michael D Cc: Tian, Feng ; Radim Kr?m=E1? ; = edk2-devel@ml01.01.org; Paolo Bonzini ; Fan, Jeff ; Zeng, Star Subject: RE: [edk2] [PATCH 0/6] Enable SMM page level protection. Jiewen, Try setting -smp 8 for 8 CPUs. Mike From: Yao, Jiewen Sent: Thursday, November 3, 2016 4:52 PM To: Laszlo Ersek > Cc: Tian, Feng >; Radim Kr?= m=E1? >; edk2-devel@ml01.01.o= rg; Kinney, Michael D >; Paolo Bonzini >; Fan, Jeff >; Zeng, Star > Subject: RE: [edk2] [PATCH 0/6] Enable SMM page level protection. Hi Laszlo I appreciate your help to validate the patch for me. Yes, I am using a windows QEMU binary. (qemu-system-x86_64.exe) A quick look at the code. 1) The ASSERT issue > ASSERT UefiCpuPkg/PiSmmCpuDxeSmm/SmmCpuMemoryManagement.c(468): !EFI_ERRO= R (Status) is caused by this line: Status =3D SmmBlockingStartupThisAp (FlushTlbOnCurrentProcessor, Inde= x, NULL); ASSERT_EFI_ERROR(Status); I am surprise to see it, because I think is an impossible case - InternalSm= mStartupThisAp(). if (CpuIndex >=3D gSmmCpuPrivate->SmmCoreEntryContext.NumberOfCpus || CpuIndex =3D=3D gSmmCpuPrivate->SmmCoreEntryContext.CurrentlyExecutin= gCpu || !(*(mSmmMpSyncData->CpuData[CpuIndex].Present)) || gSmmCpuPrivate->Operation[CpuIndex] =3D=3D SmmCpuRemove) { return EFI_INVALID_PARAMETER; } I am using below line to start QEMU: "qemu-system-x86_64.exe -machine q35,smm=3Don,accel=3Dtcg -smp 1 -drive if= =3Dpflash,format=3Draw,unit=3D0,file=3DOVMF_CODE.fd,readonly=3Don -drive if= =3Dpflash,format=3Draw,unit=3D1,file=3DOVMF_VARS.fd --serial COM8 -boot men= u=3Don,splash-time=3D7000 fat:rw:Test" Would you mind let me know what is difference between your environment and = mine? I will add more debug info to see what happened, which of above lines trigg= ers the error. 2) The unstable case is a headache. According to "RIP=3D000000000009f0fd", it seems outside of SMM right? What does this *KVM internal error. Suberror: 1* mean? Do you have any clue on what happened? Thank you Yao Jiewen From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of Lasz= lo Ersek Sent: Friday, November 4, 2016 5:43 AM To: Yao, Jiewen > Cc: Tian, Feng >; Radim Kr= =E8m=E1=F8 >; edk2-devel@ml01= .01.org; Kinney, Michael D >; Paolo Bonzini >; Fan, Jeff >; Zeng, Star > Subject: Re: [edk2] [PATCH 0/6] Enable SMM page level protection. On 11/03/16 07:53, Jiewen Yao wrote: > This series patch enables SMM page level protection. > Features are: > 1) PiSmmCore reports SMM PE image code/data information > in EdkiiPiSmmMemoryAttributeTable, if the SMM image is page aligned. > 2) PiSmmCpu consumes EdkiiPiSmmMemoryAttributeTable > and set XD for data page and RO for code page. > 3) PiSmmCpu enables Static Paging for X64 according to > PcdCpuSmmStaticPageTable. If it is true, 1G paging for above 4G > is used as long as it is supported. > 4) PiSmmCpu sets importance data structure to be read only, > such as Gdt, Idt, SmmEntrypoint, and PageTable itself. > > tested platform: > 1) Intel internal platform (X64). > 2) EDKII Quark IA32 > 3) EDKII Vlv2 X64 > 4) EDKII OVMF IA32 and IA32X64. Did you use a Windows QEMU binary for the OVMF tests? Please find my test results below, all done on KVM (3.10.0-514.el7.x86_64, if anyone is curious :)). > > Cc: Jeff Fan > > Cc: Feng Tian > > Cc: Star Zeng > > Cc: Michael D Kinney > > Cc: Laszlo Ersek > > Contributed-under: TianoCore Contribution Agreement 1.0 > Signed-off-by: Jiewen Yao > > > Jiewen Yao (6): > MdeModulePkg/Include: Add PiSmmMemoryAttributesTable.h > MdeModulePkg/dec: Add gEdkiiPiSmmMemoryAttributesTableGuid. > MdeModulePkg/PiSmmCore: Add MemoryAttributes support. > UefiCpuPkg/dec: Add PcdCpuSmmStaticPageTable. > UefiCpuPkg/PiSmmCpuDxeSmm: Add paging protection. > QuarkPlatformPkg/dsc: enable Smm paging protection. Legend: - "untested" means the test was not executed because the same test failed or proved unreliable in a less demanding configuration already, - "n/a" means a setting or test case was impossible, - "fail" and "unreliable" (lower case) are outside the scope of this series; they either capture the pre-series status, or are expected even with the series applied due to the pre-series status, - "FAIL" and "UNRELIABLE" mean regressions caused (or exposed) by the series. In all cases, 36 bits were used as address width in the CPU HOB (--> up to 64GB guest-phys address space). series OVMF = VCPU boot S3 resume # applied platform PcdCpuMaxLogicalProcessorNumber PcdCpuSmmStaticPageTabl= e topology result result -- ------- -------- ------------------------------- -----------------------= - -------- ------ --------- 1 no Ia32 64 n/a = 1x2x2 pass unreliable 2 no Ia32 255 n/a = 52x2x2 pass untested 3 no Ia32 255 n/a = 53x2x2 unreliable untested 4 no Ia32X64 64 n/a = 1x2x2 pass pass 5 no Ia32X64 255 n/a = 52x2x2 pass pass 6 no Ia32X64 255 n/a = 54x2x2 fail n/a 7 yes Ia32 64 FALSE = 1x2x2 FAIL n/a 8 yes Ia32 64 TRUE = 1x2x2 FAIL n/a 9 yes Ia32 255 FALSE = 52x2x2 untested untested 10 yes Ia32 255 FALSE = 53x2x2 untested untested 11 yes Ia32 255 TRUE = 52x2x2 untested untested 12 yes Ia32 255 TRUE = 53x2x2 untested untested 13 yes Ia32X64 64 FALSE = 1x2x2 pass UNRELIABLE 14 yes Ia32X64 64 TRUE = 1x2x2 FAIL n/a 15 yes Ia32X64 255 FALSE = 52x2x2 pass untested 16 yes Ia32X64 255 FALSE = 54x2x2 fail untested 17 yes Ia32X64 255 TRUE = 52x2x2 untested untested 18 yes Ia32X64 255 TRUE = 54x2x2 untested untested Notes for the baseline tests (1-6): * test 3: boot unreliable due to out-of-SMRAM (depends on what we do during boot) * test 5: S3 wakeup: 1 try, suspend takes very long, CPU intensive; wakeup is quick, works okay * test 6: boot fail: out of SMRAM (expected) Notes for the tests with the series applied: * test 7: normal boot is regressed even with PcdCpuSmmStaticPageTable=3DFALSE, relative to test 1: > SmmInstallProtocolInterface: [EfiSmmReadyToLockProtocol] 0 > Patch page table start ... > Patch page table done! > MemoryAttributesTable - NULL > SetPageTableAttributes > Start... > ConvertPageEntryAttribute 0x7FF9A067->0x7FF9A065 > > ASSERT_EFI_ERROR (Status =3D Invalid Parameter) > ASSERT UefiCpuPkg/PiSmmCpuDxeSmm/SmmCpuMemoryManagement.c(468): !EFI_ERRO= R (Status) Note that the processor model used in this test has no support for NX. * test 8: results identical to those of test 7 * test 13: S3 resume is unreliable (regression relative to test 4). End of OVMF log (Fedora guest): > S3_BOOT_SCRIPT_LIB_TERMINATE_OPCODE > S3BootScriptDone - Success > Call AsmDisablePaging64() to return to S3 Resume in PEI Phase > Install PPI: [PeiPostScriptTablePpi] > Install PPI: [EfiEndOfPeiSignalPpi] > Notify: PPI Guid: [EfiEndOfPeiSignalPpi], Peim notify entry point: 857895 > PeiMpInitLib: CpuMpEndOfPeiCallback () invoked > Transfer to 16bit OS waking vector - 9A1D0 QEMU log: > KVM internal error. Suberror: 1 > KVM internal error. Suberror: 1 > emulation failure > emulation failure > RAX=3D0000000000000001 RBX=3D0000000000000000 RCX=3D000000007ffdbf48 RDX= =3D000000007fede058 > RSI=3D0000000000000004 RDI=3D000000007fede040 RBP=3D0000000000000000 RSP= =3D000000007e19f000 > R8 =3D0000000000000000 R9 =3D0000000000000000 R10=3D0000000000000000 R11= =3D0000000000000000 > R12=3D0000000000000000 R13=3D0000000000000000 R14=3D0000000000000000 R15= =3D0000000000000000 > RIP=3D000000000009f0fd RFL=3D00010006 [-----P-] CPL=3D0 II=3D0 A20=3D1 SM= M=3D0 HLT=3D0 > ES =3D0030 0000000000000000 ffffffff 00c09300 DPL=3D0 DS [-WA] > CS =3D0038 0000000000000000 ffffffff 00a09b00 DPL=3D0 CS64 [-RA] > SS =3D0030 0000000000000000 ffffffff 00c09300 DPL=3D0 DS [-WA] > DS =3D0030 0000000000000000 ffffffff 00c09300 DPL=3D0 DS [-WA] > FS =3D0018 0000000000000000 ffffffff 00c09300 DPL=3D0 DS [-WA] > GS =3D0018 0000000000000000 ffffffff 00c09300 DPL=3D0 DS [-WA] > LDT=3D0000 0000000000000000 0000ffff 00008200 DPL=3D0 LDT > TR =3D0000 0000000000000000 0000ffff 00008b00 DPL=3D0 TSS64-busy > GDT=3D 000000007f294000 00000047 > IDT=3D 000000007f294048 00000fff > CR0=3De0000011 CR2=3D0000000000000000 CR3=3D000000007ff80000 CR4=3D000002= 20 > DR0=3D0000000000000000 DR1=3D0000000000000000 DR2=3D0000000000000000 DR3= =3D0000000000000000 > DR6=3D00000000ffff0ff0 DR7=3D0000000000000400 > EFER=3D0000000000000500 > Code=3D?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?= ? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?= ? ?? ?? ?? > RAX=3D0000000000000002 RBX=3D0000000000000000 RCX=3D000000007ffdbf48 RDX= =3D000000007fede070 > RSI=3D0000000000000004 RDI=3D000000007fede058 RBP=3D0000000000000000 RSP= =3D000000007e1a7000 > R8 =3D0000000000000000 R9 =3D0000000000000000 R10=3D0000000000000000 R11= =3D0000000000000000 > R12=3D0000000000000000 R13=3D0000000000000000 R14=3D0000000000000000 R15= =3D0000000000000000 > RIP=3D000000000009f0fd RFL=3D00010006 [-----P-] CPL=3D0 II=3D0 A20=3D1 SM= M=3D0 HLT=3D0 > ES =3D0030 0000000000000000 ffffffff 00c09300 DPL=3D0 DS [-WA] > CS =3D0038 0000000000000000 ffffffff 00a09b00 DPL=3D0 CS64 [-RA] > SS =3D0030 0000000000000000 ffffffff 00c09300 DPL=3D0 DS [-WA] > DS =3D0030 0000000000000000 ffffffff 00c09300 DPL=3D0 DS [-WA] > FS =3D0018 0000000000000000 ffffffff 00c09300 DPL=3D0 DS [-WA] > GS =3D0018 0000000000000000 ffffffff 00c09300 DPL=3D0 DS [-WA] > LDT=3D0000 0000000000000000 0000ffff 00008200 DPL=3D0 LDT > TR =3D0000 0000000000000000 0000ffff 00008b00 DPL=3D0 TSS64-busy > GDT=3D 000000007f294000 00000047 > IDT=3D 000000007f294048 00000fff > CR0=3De0000011 CR2=3D0000000000000000 CR3=3D000000007ff80000 CR4=3D000002= 20 > DR0=3D0000000000000000 DR1=3D0000000000000000 DR2=3D0000000000000000 DR3= =3D0000000000000000 > DR6=3D00000000ffff0ff0 DR7=3D0000000000000400 > EFER=3D0000000000000500 > Code=3D?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?= ? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?= ? ?? ?? ?? This happened 1 time out of the 2 times I ran the test. * test 14: boot regression relative to test 4. The symptoms are identical to those seen in test 7, that is: > SmmInstallProtocolInterface: [EfiSmmReadyToLockProtocol] 0 > Patch page table start ... > Patch page table done! > MemoryAttributesTable - NULL > SetPageTableAttributes > Start... > ConvertPageEntryAttribute 0x7FF98067->0x7FF98065 > > ASSERT_EFI_ERROR (Status =3D Invalid Parameter) > ASSERT UefiCpuPkg/PiSmmCpuDxeSmm/SmmCpuMemoryManagement.c(468): !EFI_ERRO= R (Status) Summary: - this series seems to break the boot with the Ia32 build of OVMF, regardless of PcdCpuSmmStaticPageTable (cases 7 and 8), - it seems to break the boot with the Ia32X64 build of OVMF, with PcdCpuSmmStaticPageTable=3DTRUE (case 14), - for the Ia32X64 build, with PcdCpuSmmStaticPageTable=3DFALSE, boot works, but S3 becomes unreliable (case 13), - because there was no successful boot with PcdCpuSmmStaticPageTable=3DTRUE, I couldn't check if and how the increased SMRAM demand would impact the maximum count of bootable VCPUs. (For this test, I should likely increase the guest-phys address width from 36 bits anyway.) Thanks Laszlo _______________________________________________ edk2-devel mailing list edk2-devel@lists.01.org https://lists.01.org/mailman/listinfo/edk2-devel