From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from us-smtp-delivery-74.mimecast.com (us-smtp-delivery-74.mimecast.com [216.205.24.74]) by mx.groups.io with SMTP id smtpd.web10.53712.1585583686462575050 for ; Mon, 30 Mar 2020 08:54:46 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=H4Shwis4; spf=pass (domain: redhat.com, ip: 216.205.24.74, mailfrom: lersek@redhat.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1585583684; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=clPe3SFEx7HEA/6SFjXSZyyYMwMwM/7D1Z5gyoaSy60=; b=H4Shwis4mko3S9gUf6cz6PC0dAdwOwfHfmm0JWeROJTlYMvKqCsr9XSGWNMzgwlKVWyxgE odEWQRlSe+QBzp9mL0CasB4U1/CWs0z6R1diI1N6SUoCuXCTBcnUBsWM2vnML27EpZZmQq kzzjirHCY/MkB2gLP4EetncCmpoIsak= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-297-t3oZ1usoOyKX3Z99JaL0Wg-1; Mon, 30 Mar 2020 11:54:36 -0400 X-MC-Unique: t3oZ1usoOyKX3Z99JaL0Wg-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 32335107ACC7; Mon, 30 Mar 2020 15:54:35 +0000 (UTC) Received: from lacos-laptop-7.usersys.redhat.com (ovpn-112-191.ams2.redhat.com [10.36.112.191]) by smtp.corp.redhat.com (Postfix) with ESMTP id 81F2DA0A91; Mon, 30 Mar 2020 15:54:33 +0000 (UTC) Subject: Re: [edk2-devel] [PATCH v3 13/17] OvmfPkg/PvScsiDxe: Setup requests and completions rings To: devel@edk2.groups.io, liran.alon@oracle.com Cc: nikita.leshchenko@oracle.com, aaron.young@oracle.com, jordan.l.justen@intel.com, ard.biesheuvel@linaro.org References: <20200328200100.60786-1-liran.alon@oracle.com> <20200328200100.60786-14-liran.alon@oracle.com> From: "Laszlo Ersek" Message-ID: Date: Mon, 30 Mar 2020 17:54:32 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: <20200328200100.60786-14-liran.alon@oracle.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit On 03/28/20 21:00, Liran Alon wrote: > These rings are shared memory buffers between host and device in which > a cyclic buffer is managed to send request descriptors from host to > device and receive completion descriptors from device to host. > > Note that because device may be constrained by IOMMU or guest may be run > under AMD SEV, we make sure to map these rings to device by using > PciIo->Map(). > > Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=2567 > Signed-off-by: Liran Alon > --- > OvmfPkg/PvScsiDxe/PvScsi.c | 219 ++++++++++++++++++++++++++++++++ > OvmfPkg/PvScsiDxe/PvScsi.h | 17 +++ > OvmfPkg/PvScsiDxe/PvScsiDxe.inf | 1 + > 3 files changed, 237 insertions(+) > > diff --git a/OvmfPkg/PvScsiDxe/PvScsi.c b/OvmfPkg/PvScsiDxe/PvScsi.c > index cf75884350ee..c7d367e83a2d 100644 > --- a/OvmfPkg/PvScsiDxe/PvScsi.c > +++ b/OvmfPkg/PvScsiDxe/PvScsi.c > @@ -11,11 +11,13 @@ > > #include > #include > +#include > #include > #include > #include > #include > #include > +#include > #include > > #include "PvScsi.h" > @@ -436,6 +438,207 @@ PvScsiRestorePciAttributes ( > ); > } > > +STATIC > +EFI_STATUS > +PvScsiAllocateSharedPages ( > + IN PVSCSI_DEV *Dev, > + IN UINTN Pages, > + OUT VOID **HostAddress, > + OUT PVSCSI_DMA_DESC *DmaDesc > + ) > +{ > + EFI_STATUS Status; > + UINTN NumberOfBytes; > + > + Status = Dev->PciIo->AllocateBuffer ( > + Dev->PciIo, > + AllocateAnyPages, > + EfiBootServicesData, > + Pages, > + HostAddress, > + EFI_PCI_ATTRIBUTE_MEMORY_CACHED > + ); > + if (EFI_ERROR (Status)) { > + return Status; > + } > + > + NumberOfBytes = EFI_PAGES_TO_SIZE (Pages); > + Status = Dev->PciIo->Map ( > + Dev->PciIo, > + EfiPciIoOperationBusMasterCommonBuffer, > + *HostAddress, > + &NumberOfBytes, > + &DmaDesc->DeviceAddress, > + &DmaDesc->Mapping > + ); > + if (EFI_ERROR (Status)) { > + goto FreeBuffer; > + } > + > + if (NumberOfBytes != EFI_PAGES_TO_SIZE (Pages)) { > + Status = EFI_OUT_OF_RESOURCES; > + goto Unmap; > + } > + > + return EFI_SUCCESS; > + > +Unmap: > + Dev->PciIo->Unmap (Dev->PciIo, DmaDesc->Mapping); > + > +FreeBuffer: > + Dev->PciIo->FreeBuffer (Dev->PciIo, Pages, *HostAddress); > + > + return Status; > +} > + > +STATIC > +VOID > +PvScsiFreeSharedPages ( > + IN PVSCSI_DEV *Dev, > + IN UINTN Pages, > + IN VOID *HostAddress, > + IN PVSCSI_DMA_DESC *DmaDesc > + ) > +{ > + Dev->PciIo->Unmap (Dev->PciIo, DmaDesc->Mapping); > + Dev->PciIo->FreeBuffer (Dev->PciIo, Pages, HostAddress); > +} > + > +STATIC > +EFI_STATUS > +PvScsiInitRings ( > + IN OUT PVSCSI_DEV *Dev > + ) > +{ > + EFI_STATUS Status; > + union { > + PVSCSI_CMD_DESC_SETUP_RINGS Cmd; > + UINT32 Uint32; > + } AlignedCmd; > + PVSCSI_CMD_DESC_SETUP_RINGS *Cmd; > + > + Cmd = &AlignedCmd.Cmd; > + > + Status = PvScsiAllocateSharedPages ( > + Dev, > + 1, > + (VOID **)&Dev->RingDesc.RingState, > + &Dev->RingDesc.RingStateDmaDesc > + ); > + if (EFI_ERROR (Status)) { > + return Status; > + } > + ZeroMem (Dev->RingDesc.RingState, EFI_PAGE_SIZE); > + > + Status = PvScsiAllocateSharedPages ( > + Dev, > + 1, > + (VOID **)&Dev->RingDesc.RingReqs, > + &Dev->RingDesc.RingReqsDmaDesc > + ); > + if (EFI_ERROR (Status)) { > + goto FreeRingState; > + } > + ZeroMem (Dev->RingDesc.RingReqs, EFI_PAGE_SIZE); > + > + Status = PvScsiAllocateSharedPages ( > + Dev, > + 1, > + (VOID **)&Dev->RingDesc.RingCmps, > + &Dev->RingDesc.RingCmpsDmaDesc > + ); > + if (EFI_ERROR (Status)) { > + goto FreeRingReqs; > + } > + ZeroMem (Dev->RingDesc.RingCmps, EFI_PAGE_SIZE); > + > + ZeroMem (Cmd, sizeof (*Cmd)); > + Cmd->ReqRingNumPages = 1; > + Cmd->CmpRingNumPages = 1; > + Cmd->RingsStatePPN = RShiftU64 ( > + Dev->RingDesc.RingStateDmaDesc.DeviceAddress, > + EFI_PAGE_SHIFT > + ); > + Cmd->ReqRingPPNs[0] = RShiftU64 ( > + Dev->RingDesc.RingReqsDmaDesc.DeviceAddress, > + EFI_PAGE_SHIFT > + ); > + Cmd->CmpRingPPNs[0] = RShiftU64 ( > + Dev->RingDesc.RingCmpsDmaDesc.DeviceAddress, > + EFI_PAGE_SHIFT > + ); > + > + STATIC_ASSERT ( > + sizeof (*Cmd) % sizeof (UINT32) == 0, > + "Cmd must be multiple of 32-bit words" > + ); > + Status = PvScsiWriteCmdDesc ( > + Dev, > + PvScsiCmdSetupRings, > + (UINT32 *)Cmd, > + sizeof (*Cmd) / sizeof (UINT32) > + ); > + if (EFI_ERROR (Status)) { > + goto FreeRingCmps; > + } > + > + return EFI_SUCCESS; > + > +FreeRingCmps: > + PvScsiFreeSharedPages ( > + Dev, > + 1, > + Dev->RingDesc.RingCmps, > + &Dev->RingDesc.RingCmpsDmaDesc > + ); > + > +FreeRingReqs: > + PvScsiFreeSharedPages ( > + Dev, > + 1, > + Dev->RingDesc.RingReqs, > + &Dev->RingDesc.RingReqsDmaDesc > + ); > + > +FreeRingState: > + PvScsiFreeSharedPages ( > + Dev, > + 1, > + Dev->RingDesc.RingState, > + &Dev->RingDesc.RingStateDmaDesc > + ); > + > + return Status; > +} > + > +STATIC > +VOID > +PvScsiFreeRings ( > + IN OUT PVSCSI_DEV *Dev > + ) > +{ > + PvScsiFreeSharedPages ( > + Dev, > + 1, > + Dev->RingDesc.RingCmps, > + &Dev->RingDesc.RingCmpsDmaDesc > + ); > + > + PvScsiFreeSharedPages ( > + Dev, > + 1, > + Dev->RingDesc.RingReqs, > + &Dev->RingDesc.RingReqsDmaDesc > + ); > + > + PvScsiFreeSharedPages ( > + Dev, > + 1, > + Dev->RingDesc.RingState, > + &Dev->RingDesc.RingStateDmaDesc > + ); > +} > + > STATIC > EFI_STATUS > PvScsiInit ( > @@ -466,6 +669,14 @@ PvScsiInit ( > goto RestorePciAttributes; > } > > + // > + // Init PVSCSI rings > + // > + Status = PvScsiInitRings (Dev); > + if (EFI_ERROR (Status)) { > + goto RestorePciAttributes; > + } > + > // > // Populate the exported interface's attributes > // > @@ -509,6 +720,14 @@ PvScsiUninit ( > IN OUT PVSCSI_DEV *Dev > ) > { > + // > + // Reset device to stop device usage of the rings. > + // This is required to safely free the rings. > + // > + PvScsiResetAdapter (Dev); If I understand correctly (at the end of the series): in order to save one of the (ultimately) two reset calls in PvScsiUninit(), namely - one for releasing the DMA buf - and the other (internal to PvScsiFreeRings()) for releasing the rings, you unnested the latter reset call from PvScsiFreeRings(), and added it manually to the error path of PvScsiInit(). To me, that's more brittle and more difficult to reason about than what I proposed, because now PvScsiFreeRings() does not completely undo PvScsiInitRings(). I specifically requested that we please not try to save on the two (seemingly) redundant reset calls in PvScsiUninit() -- see the paragraph containing "bad idea" here: http://mid.mail-archive.com/45e5950c-ec82-c24b-4c38-2912402747b2@redhat.com https://edk2.groups.io/g/devel/message/56425 (Note: we could be tempted to somehow "centralize" all of these Reset operations into a single spot. Bad idea. We are revoking the device's access rights to different resources, so the revocation operations will show up in different spots. It's a mere circumstance that the revocations all happen to be Reset operations.) I might be paranoid of course -- I just feel that maybe-superfluous reset operations on error paths are much better than silently corrupted guest memory and/or disk contents. You reacted to that message of mine, but not this paragraph specifically (it was snipped from your followup) -- so I thought you were OK with it. I'm a bit disappointed that you disagreed with my request *silently*. To summarize: technically, your solution is correct; from a code hygiene and resource ownership question, I'm convinced it is wrong. But, I'll live with it. Reviewed-by: Laszlo Ersek (Also, you didn't set up the "diff.orderFile" knob the way I requested in point (8): http://mid.mail-archive.com/0739202a-9b8a-759d-5809-2f2df69e9352@redhat.com https://edk2.groups.io/g/devel/message/56420 and so the header file changes are again at the end of the patch... Another thing I'll have to live with, I guess.) Laszlo > + > + PvScsiFreeRings (Dev); > + > PvScsiRestorePciAttributes (Dev); > } > > diff --git a/OvmfPkg/PvScsiDxe/PvScsi.h b/OvmfPkg/PvScsiDxe/PvScsi.h > index 5f611dbbc98c..6d23b6e1eccf 100644 > --- a/OvmfPkg/PvScsiDxe/PvScsi.h > +++ b/OvmfPkg/PvScsiDxe/PvScsi.h > @@ -15,12 +15,29 @@ > #include > #include > > +typedef struct { > + EFI_PHYSICAL_ADDRESS DeviceAddress; > + VOID *Mapping; > +} PVSCSI_DMA_DESC; > + > +typedef struct { > + PVSCSI_RINGS_STATE *RingState; > + PVSCSI_DMA_DESC RingStateDmaDesc; > + > + PVSCSI_RING_REQ_DESC *RingReqs; > + PVSCSI_DMA_DESC RingReqsDmaDesc; > + > + PVSCSI_RING_CMP_DESC *RingCmps; > + PVSCSI_DMA_DESC RingCmpsDmaDesc; > +} PVSCSI_RING_DESC; > + > #define PVSCSI_SIG SIGNATURE_32 ('P', 'S', 'C', 'S') > > typedef struct { > UINT32 Signature; > EFI_PCI_IO_PROTOCOL *PciIo; > UINT64 OriginalPciAttributes; > + PVSCSI_RING_DESC RingDesc; > UINT8 MaxTarget; > UINT8 MaxLun; > EFI_EXT_SCSI_PASS_THRU_PROTOCOL PassThru; > diff --git a/OvmfPkg/PvScsiDxe/PvScsiDxe.inf b/OvmfPkg/PvScsiDxe/PvScsiDxe.inf > index fcffc90d46c8..6200533698fc 100644 > --- a/OvmfPkg/PvScsiDxe/PvScsiDxe.inf > +++ b/OvmfPkg/PvScsiDxe/PvScsiDxe.inf > @@ -26,6 +26,7 @@ > OvmfPkg/OvmfPkg.dec > > [LibraryClasses] > + BaseLib > BaseMemoryLib > DebugLib > MemoryAllocationLib >