From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by mx.groups.io with SMTP id smtpd.web11.9455.1614934780889056477 for ; Fri, 05 Mar 2021 00:59:41 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=FwKpVSnW; spf=pass (domain: redhat.com, ip: 216.205.24.124, mailfrom: pbonzini@redhat.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1614934778; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PzqhR/fgzj1HMUJeD9wbIdwjLC9R4My92hTHjNbFUBw=; b=FwKpVSnWUu9mwo6ptztU2alW44c8Oh/t8feu7V2RjdN/7YW36oPjs2SjniH2woFmruuExw ZIC6qkVBOY8/2diKXGeTuIcZNJzlW84cyWcGa9RQioyvNaQsSXr70/p/bx5sd/Pa/qqvm7 bbBz0F6n/irKoVm+OHYF1HcGSJUGqNE= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-231-Ds8x7TdNNXSu4W_4O_5fLg-1; Fri, 05 Mar 2021 03:59:36 -0500 X-MC-Unique: Ds8x7TdNNXSu4W_4O_5fLg-1 Received: by mail-wm1-f69.google.com with SMTP id a65so146660wmh.1 for ; Fri, 05 Mar 2021 00:59:36 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:to:references:from:subject:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=PzqhR/fgzj1HMUJeD9wbIdwjLC9R4My92hTHjNbFUBw=; b=f9K4rLrt2y4Fv3cHfr/+G2yTZTCml+dwuBru0psbQCkpk2LUmbSlfyaqTL2eBcjURs YbqpDTjYjBHDDOif1LOT5KfEtt7j3T2vDWBjivxdBnnVuoa+Zw9fW7nEpfepVTEdazKm +VasFfYAPuuqALUfjz2rFySIuAn+HpwPMTi8Okwqeh9GWmPlhmQ7EdHb0Isz20MnoWdV 9agZb4u4/Nyi8cGVs7zMtLTwzK6ER//0M5wvc7V9RQxf+vAnI7A/RKpuKxWAsOYLQQhd sPSnIYJzJeAu+x9xW0EqOu7ZcFFhuc7XXBM0MaUMF8XM5fz1oG2Ki5JHQCGzMmj2fmTL uHcg== X-Gm-Message-State: AOAM531g/sE9dvYW9O0uUOs1+PxQv1WaBFNOk+IgTFFtExbKZCvwvxsn 4YU6DtsCnS9M2CcxTGzoTkG9/YKiEJbVSkuxsi2l7TLeE1mT9pPDcmnDn+tBn9d2kRx2542EdE4 YqOC5bl94mDnvjA== X-Received: by 2002:adf:b60f:: with SMTP id f15mr8285682wre.83.1614934775350; Fri, 05 Mar 2021 00:59:35 -0800 (PST) X-Google-Smtp-Source: ABdhPJw3rYbHiAOaWiG4rW5ns+xuEFQkAwJ07sLe2lABwmsc7DfK0FbzfAW5hMWaEIgzmm8S+LaBdA== X-Received: by 2002:adf:b60f:: with SMTP id f15mr8285667wre.83.1614934775126; Fri, 05 Mar 2021 00:59:35 -0800 (PST) Return-Path: Received: from ?IPv6:2001:b07:6468:f312:c8dd:75d4:99ab:290a? ([2001:b07:6468:f312:c8dd:75d4:99ab:290a]) by smtp.gmail.com with ESMTPSA id x13sm3318814wrt.75.2021.03.05.00.59.34 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 05 Mar 2021 00:59:34 -0800 (PST) To: Laszlo Ersek , devel@edk2.groups.io, Tobin Feldman-Fitzthum References: <20210302204839.82042-1-tobin@linux.ibm.com> <9205.1614849670474335263@groups.io> <7775b8d9-ed8d-eb4f-0c1a-3552996cef90@redhat.com> From: "Paolo Bonzini" Subject: Re: [edk2-devel] [RFC PATCH 00/14] Firmware Support for Fast Live Migration for AMD SEV Message-ID: <7f9ab14b-40af-feff-3939-fc303daaa590@redhat.com> Date: Fri, 5 Mar 2021 09:59:33 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.7.0 MIME-Version: 1.0 In-Reply-To: <7775b8d9-ed8d-eb4f-0c1a-3552996cef90@redhat.com> Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=pbonzini@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit On 04/03/21 21:45, Laszlo Ersek wrote: > On 03/04/21 10:21, Paolo Bonzini wrote: >> Hi Tobin, >> >> as mentioned in the reply to the QEMU patches posted by Tobin, I >> think the firmware helper approach is very good, but there are some >> disadvantages in the idea of auxiliary vCPUs. These are especially >> true in the VMM, where it's much nicer to have a separate VM that >> goes through a specialized run loop; however, even in the firmware >> level there are some complications (as you pointed out) in letting >> MpService workers run after ExitBootServices. >> >> My idea would be that the firmware would start the VM as usual using >> the same launch data; then, the firmware would detect it was running >> as a migration helper VM during the SEC or PEI phases (for example >> via the GHCB or some other unencrypted communication area), and >> divert execution to the migration helper instead of proceeding to the >> next boot phase. This would be somewhat similar in spirit to how edk2 >> performs S3 resume, if my memory serves correctly. > > Very cool. You'd basically warm-reboot the virtual machine into a new > boot mode (cf. BOOT_WITH_FULL_CONFIGURATION vs. BOOT_ON_S3_RESUME in > OvmfPkg/PlatformPei). > > To me that's much more attractive than a "background job". > > The S3 parallel is great. What I'm missing is: > > - Is it possible to warm-reboot an SEV VM? (I vaguely recall that it's > not possible for SEV-ES at least.) Because, that's how we'd transfer > control to the early parts of the firmware again, IIUC your idea, while > preserving the memory contents. It's not exactly a warm reboot. It's two VMs booted at the same time, with exactly the same contents as far as encrypted RAM goes, but different unencrypted RAM. The difference makes one VM boot regularly and the other end up in the migration helper. The migration helper can be entirely contained in PEI, or it can even be its own OS, stored as a flat binary in the firmware. Whatever is easier. The divergence would happen much earlier than S3 though. It would have to happen before the APs are brought up, for example, and essentially before the first fw_cfg access if (as is likely) the migration helper VM does not have fw_cfg at all. That's why I brought up the possibility of diverging as soon as SEC. > - Who would initiate this process? S3 suspend is guest-initiated. (Not > that we couldn't use the guest agent, if needed.) > > (In case the idea is really about a separate VM, and not about rebooting > the already running VM, then I don't understand -- how would a separate > VM access the guest RAM that needs to be migrated?) Answering the other message: > (For some unsolicited personal information, now I feel less bad about > this idea never occurring to me -- I never knew about the KVM patch set > that would enable encryption context sharing. (TBH I thought that was > prevented, by design, in the SEV hardware...)) As far as the SEV hardware is concerned, a "VM" is defined by the ASID. The VM would be separate at the KVM level, but it would share the ASID (and thus the guest RAM) with the primary VM. So as far as the SEV hardware and the processor are concerned, the separate VM would be just one more VMCB that runs with that ASID. Only KVM knows that they are backed by different file descriptors etc. In fact, another advantage is that it would be much easier to scale the migration helper to multiple vCPUs. This is probably also a case for diverging much earlier than PEI, because a multi-processor migration helper running in PEI or DXE would require ACPI tables and a lot of infrastructure that is probably undesirable. Paolo