From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=209.132.183.28; helo=mx1.redhat.com; envelope-from=lersek@redhat.com; receiver=edk2-devel@lists.01.org Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 597AE2234435A for ; Sun, 28 Jan 2018 13:38:27 -0800 (PST) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 457C85F7B8; Sun, 28 Jan 2018 21:43:59 +0000 (UTC) Received: from lacos-laptop-7.usersys.redhat.com (ovpn-121-84.rdu2.redhat.com [10.10.121.84]) by smtp.corp.redhat.com (Postfix) with ESMTP id D6E5D5C260; Sun, 28 Jan 2018 21:43:57 +0000 (UTC) From: Laszlo Ersek To: Jian J Wang , edk2-devel@lists.01.org Cc: Ruiyu Ni , Jiewen Yao , Eric Dong References: <20180115085433.25008-1-jian.j.wang@intel.com> <20180115085433.25008-2-jian.j.wang@intel.com> Message-ID: <6264fc27-bf8d-5ee5-f631-31c28c32abeb@redhat.com> Date: Sun, 28 Jan 2018 22:43:56 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.5.2 MIME-Version: 1.0 In-Reply-To: X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Sun, 28 Jan 2018 21:43:59 +0000 (UTC) Subject: Re: [PATCH 1/6] UefiCpuPkg/MpInitLib: split wake up buffer into two parts X-BeenThere: edk2-devel@lists.01.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: EDK II Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Jan 2018 21:38:28 -0000 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit On 01/27/18 17:17, Laszlo Ersek wrote: > Hello Jian, > > On 01/15/18 09:54, Jian J Wang wrote: >> If PcdDxeNxMemoryProtectionPolicy is set to enable protection for memory >> of EfiBootServicesCode, EfiConventionalMemory, the BIOS will hang at a page >> fault exception during MP initialization. >> >> The root cause is that the AP wake up buffer, which is below 1MB and used >> to hold both AP init code and data, is type of EfiConventionalMemory (not >> really allocated because of potential conflict with legacy code), and is >> marked as non-executable. During the transition from real address mode >> to long mode, the AP init code has to enable paging which will then cause >> itself a page fault exception because it's just running in non-executable >> memory. >> >> The solution is splitting AP wake up buffer into two part: lower part is >> still below 1MB and shared with legacy system, higher part is really >> allocated memory of BootServicesCode type. The init code in the memory >> below 1MB will not enable paging but just switch to protected mode and >> jump to higher memory, in which the init code will enable paging and >> switch to long mode. >> >> Cc: Jiewen Yao >> Cc: Ruiyu Ni >> Cc: Eric Dong >> Cc: Laszlo Ersek >> Contributed-under: TianoCore Contribution Agreement 1.1 >> Signed-off-by: Jian J Wang >> --- >> UefiCpuPkg/Library/MpInitLib/DxeMpLib.c | 34 ++++++++++ >> UefiCpuPkg/Library/MpInitLib/Ia32/MpEqu.inc | 5 ++ >> UefiCpuPkg/Library/MpInitLib/Ia32/MpFuncs.nasm | 32 +++++----- >> UefiCpuPkg/Library/MpInitLib/MpLib.c | 45 +++++++++++++ >> UefiCpuPkg/Library/MpInitLib/MpLib.h | 22 +++++++ >> UefiCpuPkg/Library/MpInitLib/PeiMpLib.c | 23 +++++++ >> UefiCpuPkg/Library/MpInitLib/X64/MpEqu.inc | 5 +- >> UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm | 87 ++++++++++++++++---------- >> 8 files changed, 204 insertions(+), 49 deletions(-) > > This patch breaks OVMF on KVM. The symptom is that the guest crashes > and reboots as follows (infinite reboot loop): > >> Loading PEIM at 0x0007FEB0000 EntryPoint=0x0007FEB5C96 CpuMpPei.efi >> AP Loop Mode is 1 >> WakeupBufferStart = 9F000, WakeupBufferSize = 1000 >> -- crash & reboot here -- >> SecCoreStartupWithStack(0xFFFCC000, 0x820000) The following build options were used for this build: $ build -a X64 -p OvmfPkg/OvmfPkgX64.dsc -D SECURE_BOOT_ENABLE \ -t GCC48 -b NOOPT -D HTTP_BOOT_ENABLE The tree was built at 06c1f423e17f ("BeagleBoardPkg: reroute Firmware Vendor Pcd to MdeModulePkg", 2018-01-26). (This commit is listed at the top of the bisection log in my previous email.) Here's the KVM log up to the triple fault: > CPU-32283 [004] 13652.374591: kvm_entry: vcpu 2 > CPU-32283 [004] 13652.374594: kvm_exit: reason CR_ACCESS rip 0x3a info 0 0 > CPU-32283 [004] 13652.374595: kvm_cr: cr_write 0 = 0x60000013 > CPU-32283 [004] 13652.374596: kvm_entry: vcpu 2 > CPU-32283 [004] 13652.374597: kvm_exit: reason CR_ACCESS rip 0x9f06a info 4 0 > CPU-32283 [004] 13652.374598: kvm_cr: cr_write 4 = 0x20 > CPU-32283 [004] 13652.374603: kvm_entry: vcpu 2 > CPU-32283 [004] 13652.374604: kvm_exit: reason CR_ACCESS rip 0x9f075 info 103 0 > CPU-32283 [004] 13652.374605: kvm_cr: cr_write 3 = 0x800000 > CPU-32283 [004] 13652.374606: kvm_entry: vcpu 2 > CPU-32283 [004] 13652.374607: kvm_exit: reason MSR_READ rip 0x9f07d info 0 0 > CPU-32283 [004] 13652.374608: kvm_msr: msr_read c0000080 = 0x0 > CPU-32283 [004] 13652.374608: kvm_entry: vcpu 2 > CPU-32283 [004] 13652.374609: kvm_exit: reason MSR_WRITE rip 0x9f083 info 0 0 > CPU-32283 [004] 13652.374611: kvm_msr: msr_write c0000080 = 0x100 > CPU-32283 [004] 13652.374612: kvm_entry: vcpu 2 > CPU-32283 [004] 13652.374613: kvm_exit: reason CR_ACCESS rip 0x9f08c info 0 0 > CPU-32283 [004] 13652.374613: kvm_cr: cr_write 0 = 0xe0000013 > CPU-32283 [004] 13652.374620: kvm_entry: vcpu 2 > CPU-32283 [004] 13652.374622: kvm_exit: reason TRIPLE_FAULT rip 0x9f096 info 0 0 Offset 0x96 (relative to 0x9F000) is from "UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm": > 5c66d125eaae5 (Jeff Fan 2016-07-29 21:13:34 +0800 103) SkipEnableExecuteDisableBit: > f32bfe6d06142 (Jian J Wang 2017-12-29 09:12:54 +0800 104) ; > f32bfe6d06142 (Jian J Wang 2017-12-29 09:12:54 +0800 105) ; Enable PAE > f32bfe6d06142 (Jian J Wang 2017-12-29 09:12:54 +0800 106) ; > d94e5f672994f (Jeff Fan 2016-07-20 22:44:39 +0800 107) mov eax, cr4 > d94e5f672994f (Jeff Fan 2016-07-20 22:44:39 +0800 108) bts eax, 5 > d94e5f672994f (Jeff Fan 2016-07-20 22:44:39 +0800 109) mov cr4, eax > d94e5f672994f (Jeff Fan 2016-07-20 22:44:39 +0800 110) > f32bfe6d06142 (Jian J Wang 2017-12-29 09:12:54 +0800 111) ; > f32bfe6d06142 (Jian J Wang 2017-12-29 09:12:54 +0800 112) ; Load page table > f32bfe6d06142 (Jian J Wang 2017-12-29 09:12:54 +0800 113) ; > f32bfe6d06142 (Jian J Wang 2017-12-29 09:12:54 +0800 114) mov esi, Cr3Location ; Save CR3 in ecx > f32bfe6d06142 (Jian J Wang 2017-12-29 09:12:54 +0800 115) mov ecx, [ebx + esi] > d94e5f672994f (Jeff Fan 2016-07-20 22:44:39 +0800 116) mov cr3, ecx ; Load CR3 > d94e5f672994f (Jeff Fan 2016-07-20 22:44:39 +0800 117) > f32bfe6d06142 (Jian J Wang 2017-12-29 09:12:54 +0800 118) ; > f32bfe6d06142 (Jian J Wang 2017-12-29 09:12:54 +0800 119) ; Enable long mode > f32bfe6d06142 (Jian J Wang 2017-12-29 09:12:54 +0800 120) ; > d94e5f672994f (Jeff Fan 2016-07-20 22:44:39 +0800 121) mov ecx, 0c0000080h ; EFER MSR number > d94e5f672994f (Jeff Fan 2016-07-20 22:44:39 +0800 122) rdmsr ; Read EFER > d94e5f672994f (Jeff Fan 2016-07-20 22:44:39 +0800 123) bts eax, 8 ; Set LME=1 > d94e5f672994f (Jeff Fan 2016-07-20 22:44:39 +0800 124) wrmsr ; Write EFER > d94e5f672994f (Jeff Fan 2016-07-20 22:44:39 +0800 125) > f32bfe6d06142 (Jian J Wang 2017-12-29 09:12:54 +0800 126) ; > f32bfe6d06142 (Jian J Wang 2017-12-29 09:12:54 +0800 127) ; Enable paging > f32bfe6d06142 (Jian J Wang 2017-12-29 09:12:54 +0800 128) ; > d94e5f672994f (Jeff Fan 2016-07-20 22:44:39 +0800 129) mov eax, cr0 ; Read CR0 > d94e5f672994f (Jeff Fan 2016-07-20 22:44:39 +0800 130) bts eax, 31 ; Set PG=1 > d94e5f672994f (Jeff Fan 2016-07-20 22:44:39 +0800 131) mov cr0, eax ; Write CR0 > d94e5f672994f (Jeff Fan 2016-07-20 22:44:39 +0800 132) > f32bfe6d06142 (Jian J Wang 2017-12-29 09:12:54 +0800 133) ; > f32bfe6d06142 (Jian J Wang 2017-12-29 09:12:54 +0800 134) ; Far jump to 64-bit code > f32bfe6d06142 (Jian J Wang 2017-12-29 09:12:54 +0800 135) ; > f32bfe6d06142 (Jian J Wang 2017-12-29 09:12:54 +0800 136) mov edi, ModeHighMemoryLocation > f32bfe6d06142 (Jian J Wang 2017-12-29 09:12:54 +0800 137) add edi, ebx > f32bfe6d06142 (Jian J Wang 2017-12-29 09:12:54 +0800 138) jmp far [edi] <- here Thanks Laszlo