From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: redhat.com, ip: 209.85.221.68, mailfrom: pbonzini@redhat.com) Received: from mail-wr1-f68.google.com (mail-wr1-f68.google.com [209.85.221.68]) by groups.io with SMTP; Thu, 15 Aug 2019 09:21:32 -0700 Received: by mail-wr1-f68.google.com with SMTP id z11so2728843wrt.4 for ; Thu, 15 Aug 2019 09:21:31 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:openpgp:message-id :date:user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=0NnnyLAZl8sqrxHu4b/221KYMc1jXMKtn8iJbHOPjs4=; b=tW5hgcXZXv05BfvsjM0IWWrjn5s9IabF0x+4lhjBHneqIMmKPkDOQLgn0JLjaN5XWy FFCIAX2MqKc5KrZ0QOncU4tE/xjSfXahQ4vAVqwL0esQNGWNY5SljXzVmVXIt4t/PC7G 6TpL3VX1oOZdCueiQAtwig4lrCMKJk+riZLtd48TvSsWBDMvz3v4aBxF56BgGSqSAjMf nh5rh9bRVbls6iSH1L19OxxvElAvjuTnBuQWAzrbhYSv4j5zfhEmyaPApJGiRDgIjrVi DSqGb0p2/3u527nsmqyetGUIadpMsFZVzpNOupS5Dd4u16xfnvMlr5eafCe4YBWzY6jb DGlQ== X-Gm-Message-State: APjAAAXydmFhD/H5wMqvQyw1k7Si9ML8Jldfm/mT8K8OxFSRF1S7qv/h adiW0lXMl9YYc7c3GQ7N8QKaHg== X-Google-Smtp-Source: APXvYqxDjCalPqFnWeza9wR4BtPSi9Xqqfhjg4f/c/CIK+SoRJz8EiEUXuZLjNS4qYqugpf3lA+p+w== X-Received: by 2002:adf:c594:: with SMTP id m20mr6778454wrg.126.1565886090547; Thu, 15 Aug 2019 09:21:30 -0700 (PDT) Return-Path: Received: from ?IPv6:2001:b07:6468:f312:2cae:66cd:dd43:92d9? ([2001:b07:6468:f312:2cae:66cd:dd43:92d9]) by smtp.gmail.com with ESMTPSA id i66sm2018975wmg.2.2019.08.15.09.21.29 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 15 Aug 2019 09:21:30 -0700 (PDT) Subject: Re: [edk2-devel] CPU hotplug using SMM with QEMU+OVMF To: Laszlo Ersek , devel@edk2.groups.io, "Yao, Jiewen" Cc: edk2-rfc-groups-io , qemu devel list , Igor Mammedov , "Chen, Yingwen" , "Nakajima, Jun" , Boris Ostrovsky , Joao Marcal Lemos Martins , Phillip Goerl References: <8091f6e8-b1ec-f017-1430-00b0255729f4@redhat.com> <74D8A39837DF1E4DA445A8C0B3885C503F75B680@shsmsx102.ccr.corp.intel.com> <047801f8-624a-2300-3cf7-1daa1395ce59@redhat.com> <99219f81-33a3-f447-95f8-f10341d70084@redhat.com> From: Paolo Bonzini Openpgp: preference=signencrypt Message-ID: <6f8b9507-58d0-5fbd-b827-c7194b3b2948@redhat.com> Date: Thu, 15 Aug 2019 18:21:28 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 MIME-Version: 1.0 In-Reply-To: <99219f81-33a3-f447-95f8-f10341d70084@redhat.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit On 15/08/19 17:00, Laszlo Ersek wrote: > On 08/14/19 16:04, Paolo Bonzini wrote: >> On 14/08/19 15:20, Yao, Jiewen wrote: >>>> - Does this part require a new branch somewhere in the OVMF SEC code? >>>> How do we determine whether the CPU executing SEC is BSP or >>>> hot-plugged AP? >>> [Jiewen] I think this is blocked from hardware perspective, since the first instruction. >>> There are some hardware specific registers can be used to determine if the CPU is new added. >>> I don’t think this must be same as the real hardware. >>> You are free to invent some registers in device model to be used in OVMF hot plug driver. >> >> Yes, this would be a new operation mode for QEMU, that only applies to >> hot-plugged CPUs. In this mode the AP doesn't reply to INIT or SMI, in >> fact it doesn't reply to anything at all. >> >>>> - How do we tell the hot-plugged AP where to start execution? (I.e. that >>>> it should execute code at a particular pflash location.) >>> [Jiewen] Same real mode reset vector at FFFF:FFF0. >> >> You do not need a reset vector or INIT/SIPI/SIPI sequence at all in >> QEMU. The AP does not start execution at all when it is unplugged, so >> no cache-as-RAM etc. >> >> We only need to modify QEMU so that hot-plugged APIs do not reply to >> INIT/SIPI/SMI. >> >>> I don’t think there is problem for real hardware, who always has CAR. >>> Can QEMU provide some CPU specific space, such as MMIO region? >> >> Why is a CPU-specific region needed if every other processor is in SMM >> and thus trusted. > > I was going through the steps Jiewen and Yingwen recommended. > > In step (02), the new CPU is expected to set up RAM access. In step > (03), the new CPU, executing code from flash, is expected to "send board > message to tell host CPU (GPIO->SCI) -- I am waiting for hot-add > message." For that action, the new CPU may need a stack (minimally if we > want to use C function calls). > > Until step (03), there had been no word about any other (= pre-plugged) > CPUs (more precisely, Jiewen even confirmed "No impact to other > processors"), so I didn't assume that other CPUs had entered SMM. > > Paolo, I've attempted to read Jiewen's response, and yours, as carefully > as I can. I'm still very confused. If you have a better understanding, > could you please write up the 15-step process from the thread starter > again, with all QEMU customizations applied? Such as, unnecessary steps > removed, and platform specifics filled in. Sure. (01a) QEMU: create new CPU. The CPU already exists, but it does not start running code until unparked by the CPU hotplug controller. (01b) QEMU: trigger SCI (02-03) no equivalent (04) Host CPU: (OS) execute GPE handler from DSDT (05) Host CPU: (OS) Port 0xB2 write, all CPUs enter SMM (NOTE: New CPU will not enter CPU because SMI is disabled) (06) Host CPU: (SMM) Save 38000, Update 38000 -- fill simple SMM rebase code. (07a) Host CPU: (SMM) Write to CPU hotplug controller to enable new CPU (07b) Host CPU: (SMM) Send INIT/SIPI/SIPI to new CPU. (08a) New CPU: (Low RAM) Enter protected mode. (08b) New CPU: (Flash) Signals host CPU to proceed and enter cli;hlt loop. (09) Host CPU: (SMM) Send SMI to the new CPU only. (10) New CPU: (SMM) Run SMM code at 38000, and rebase SMBASE to TSEG. (11) Host CPU: (SMM) Restore 38000. (12) Host CPU: (SMM) Update located data structure to add the new CPU information. (This step will involve CPU_SERVICE protocol) (13) New CPU: (Flash) do whatever other initialization is needed (14) New CPU: (Flash) Deadloop, and wait for INIT-SIPI-SIPI. (15) Host CPU: (OS) Send INIT-SIPI-SIPI to pull new CPU in.. In other words, the cache-as-RAM phase of 02-03 is replaced by the INIT-SIPI-SIPI sequence of 07b-08a-08b. >> The QEMU DSDT could be modified (when secure boot is in effect) to OUT >> to 0xB2 when hotplug happens. It could write a well-known value to >> 0xB2, to be read by an SMI handler in edk2. > > I dislike involving QEMU's generated DSDT in anything SMM (even > injecting the SMI), because the AML interpreter runs in the OS. > > If a malicious OS kernel is a bit too enlightened about the DSDT, it > could willfully diverge from the process that we design. If QEMU > broadcast the SMI internally, the guest OS could not interfere with that. > > If the purpose of the SMI is specifically to force all CPUs into SMM > (and thereby force them into trusted state), then the OS would be > explicitly counter-interested in carrying out the AML operations from > QEMU's DSDT. But since the hotplug controller would only be accessible from SMM, there would be no other way to invoke it than to follow the DSDT's instruction and write to 0xB2. FWIW, real hardware also has plenty of 0xB2 writes in the DSDT or in APEI tables (e.g. for persistent store access). Paolo