From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail02.groups.io (mail02.groups.io [66.175.222.108]) by spool.mail.gandi.net (Postfix) with ESMTPS id 7FF28AC0F4E for ; Fri, 3 Nov 2023 13:15:21 +0000 (UTC) DKIM-Signature: a=rsa-sha256; bh=1Aj2/tptU8Xpw+2Njl6lEfOM72JYN52tG1P8HgHPTeQ=; c=relaxed/simple; d=groups.io; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From:In-Reply-To:Precedence:List-Subscribe:List-Help:Sender:List-Id:Mailing-List:Delivered-To:Reply-To:List-Unsubscribe-Post:List-Unsubscribe:Content-Language:Content-Type:Content-Transfer-Encoding; s=20140610; t=1699017320; v=1; b=DPDv3Saj3pAjgu696htHtjjP/BwW07GcZa/i8PJu9l4HcmBKaxs3iFYRLrfbHodbzbkZsHcv 6PXQzd3hRbxH/z0Wdh/UhfS66kuN4DRg7W1MocfIjIB/eRsyqsb3l/76BTHK/8O9t1zEZFgY6Ab Acm7d6sQGmsBrvG6En59excc= X-Received: by 127.0.0.2 with SMTP id eXn8YY7687511xAauxsDJif4; Fri, 03 Nov 2023 06:15:20 -0700 X-Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.groups.io with SMTP id smtpd.web10.51555.1699017319432679494 for ; Fri, 03 Nov 2023 06:15:19 -0700 X-Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-140-4nicyIi2N1uZ7z9fJOhXKg-1; Fri, 03 Nov 2023 09:15:11 -0400 X-MC-Unique: 4nicyIi2N1uZ7z9fJOhXKg-1 X-Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 0C783811E7B; Fri, 3 Nov 2023 13:15:11 +0000 (UTC) X-Received: from [10.39.192.20] (unknown [10.39.192.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 01236492BE0; Fri, 3 Nov 2023 13:15:09 +0000 (UTC) Message-ID: <32d105da-7ccd-9acc-5cea-9a740bcc37f8@redhat.com> Date: Fri, 3 Nov 2023 14:15:08 +0100 MIME-Version: 1.0 Subject: Re: [edk2-devel] [PATCH v1] OvmfPkg/PlatformInitLib: Don't override user specified PciMmio64Size To: devel@edk2.groups.io, vivek.kasireddy@intel.com Cc: Gerd Hoffmann , Ard Biesheuvel , Dongwon Kim References: <20231103051519.277323-1-vivek.kasireddy@intel.com> From: "Laszlo Ersek" In-Reply-To: <20231103051519.277323-1-vivek.kasireddy@intel.com> X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.9 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Precedence: Bulk List-Subscribe: List-Help: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,lersek@redhat.com List-Unsubscribe-Post: List-Unsubscribe=One-Click List-Unsubscribe: X-Gm-Message-State: I0hvBXNVFRsQkLmHS3tCV1wUx7686176AA= Content-Language: en-US Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-GND-Status: LEGIT Authentication-Results: spool.mail.gandi.net; dkim=pass header.d=groups.io header.s=20140610 header.b=DPDv3Saj; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=redhat.com (policy=none); spf=pass (spool.mail.gandi.net: domain of bounce@groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce@groups.io On 11/3/23 06:15, Vivek Kasireddy wrote: > If the user specified a size for the PCI MMIO window via the option: > -fw_cfg name=3Dopt/ovmf/X-PciMmio64Mb,string=3D32768 > then this patch ensures that the mmio window is not resized again. >=20 > Essentially, this prevents the change introduced in the following > patch from taking effect: > commit ecb778d0ac62560aa172786ba19521f27bc3f650 > Author: Gerd Hoffmann > Date: Tue Oct 4 15:47:27 2022 +0200 >=20 > OvmfPkg/PlatformInitLib: dynamic mmio window size >=20 > In case we have a reliable PhysMemAddressWidth use that to dynamicall= y > size the 64bit address window. Allocate 1/8 of the physical address > space and place the window at the upper end of the address space. >=20 > The problem this patch is trying to solve is the VFIO mapping failures: > VFIO_MAP_DMA failed: Invalid argument > vfio_dma_map(0x557b2f2736d0, 0x380000000000, 0x1000000, 0x7f98ac400000) = =3D -22 (Invalid argument) > that occur when we try to passthrough the graphics device to the guest: > qemu-system-x86_64 -m 4096 -enable-kvm -cpu host -smp cores=3D4,threads= =3D2,sockets=3D1 > -device vfio-pci,host=3D0000:00:02.0 -bios OVMF.fd -nographic >=20 > The above failures seem to occur because of a mismatch between the > PhysMemAddressWidth and the Host IOMMU address width. More specifically, > if the PhysMemAddressWidth is bigger than the IOMMU address width, > VFIO fails to map the MMIO regions as the IOVAs would be larger > than the IOMMU aperture regions. When tested on modern Intel platforms > such as ADL, MTL, etc, OVMF determines PhysMemAddressWidth =3D 46 which > matches the Host address width but the IOMMU address width seems to > range anywhere from 38 to 48 depending on the IOMMU hardware > capabilities, version, etc. >=20 > One way to address this issue is if we ensure that PhysMemAddressWidth > matches IOMMU address width: > -cpu host,host-phys-bits=3Don,host-phys-bits-limit=3D > However, this requires the user to figure out the IOMMU address width; > which can be determined by looking at the 16-21 bits of the cap value: > cat /sys/devices/virtual/iommu/dmar0/intel-iommu/cap > or by reading the DMAR_CAP_REG register. But this does not seem like > a reasonable approach to solve this problem. Very nice problem description, already outlining the solution as well. >=20 > Therefore, this problem requires an OVMF specific solution to retain > the prior behavior. To this end, this patch reuses the X-PciMmio64Mb > option to opt-out of the behavior introduced in the above commit > instead of adding a new option or mechanism. No, the right solution is to enhance QEMU to query the host IOMMU address width. Then the following options arise: - either pass *both* the host CPU address width *and* the host IOMMU address width down to OVMF, and teach OVMF to pick the stricter limitation, for dynamically sizing the MMIO window - or let QEMU calulate the stricter width internally, and pass that (sole, scalar) piece of information down to OVMF. Teach OVMF to query this new piece of information, and size the MMIO window accordingly. Basically the QEMU command line-based workaround that you describe is what we need to automate (except we need a new information channel for it, because presenting the strict host IOMMU address width as the VCPU address width (via CPUID) to the guest is smelly). I agree that the proposed patch can function as a stop-gap, but the QEMU command line hack is already a stop-gap. And for the long term, this patch is not good enough. We should enhance the dynamic sizing, now that Gerd has put it into place. Thanks Laszlo >=20 > Cc: Gerd Hoffmann > Cc: Ard Biesheuvel > Cc: Laszlo Ersek > Cc: Dongwon Kim > Signed-off-by: Vivek Kasireddy > --- > OvmfPkg/Include/Library/PlatformInitLib.h | 1 + > OvmfPkg/Library/PlatformInitLib/MemDetect.c | 4 +++- > 2 files changed, 4 insertions(+), 1 deletion(-) >=20 > diff --git a/OvmfPkg/Include/Library/PlatformInitLib.h b/OvmfPkg/Include/= Library/PlatformInitLib.h > index 57b18b94d9..e8ea3defa2 100644 > --- a/OvmfPkg/Include/Library/PlatformInitLib.h > +++ b/OvmfPkg/Include/Library/PlatformInitLib.h > @@ -55,6 +55,7 @@ typedef struct { > BOOLEAN QemuFwCfgChecked; > BOOLEAN QemuFwCfgSupported; > BOOLEAN QemuFwCfgDmaSupported; > + BOOLEAN QemuFwCfgSizeSpecified; > } EFI_HOB_PLATFORM_INFO; > #pragma pack() > =20 > diff --git a/OvmfPkg/Library/PlatformInitLib/MemDetect.c b/OvmfPkg/Librar= y/PlatformInitLib/MemDetect.c > index 662e7e85bb..a53a1e24e4 100644 > --- a/OvmfPkg/Library/PlatformInitLib/MemDetect.c > +++ b/OvmfPkg/Library/PlatformInitLib/MemDetect.c > @@ -485,6 +485,7 @@ PlatformGetFirstNonAddress ( > case EFI_SUCCESS: > if (FwCfgPciMmio64Mb <=3D 0x1000000) { > PlatformInfoHob->PcdPciMmio64Size =3D LShiftU64 (FwCfgPciMmio64M= b, 20); > + PlatformInfoHob->QemuFwCfgSizeSpecified =3D TRUE; > break; > } > =20 > @@ -861,7 +862,8 @@ PlatformAddressWidthInitialization ( > } > =20 > PlatformAddressWidthFromCpuid (PlatformInfoHob, TRUE); > - if (PlatformInfoHob->PhysMemAddressWidth !=3D 0) { > + if (PlatformInfoHob->PhysMemAddressWidth !=3D 0 && > + !PlatformInfoHob->QemuFwCfgSizeSpecified) { > // physical address width is known > PlatformDynamicMmioWindow (PlatformInfoHob); > return; -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#110621): https://edk2.groups.io/g/devel/message/110621 Mute This Topic: https://groups.io/mt/102359124/7686176 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/leave/12367111/7686176/19134562= 12/xyzzy [rebecca@openfw.io] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-