From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail02.groups.io (mail02.groups.io [66.175.222.108]) by spool.mail.gandi.net (Postfix) with ESMTPS id 5A624941CC4 for ; Fri, 3 Nov 2023 14:07:27 +0000 (UTC) DKIM-Signature: a=rsa-sha256; bh=pZBm+vtbTcdxpQAne/l6E9otHpkPqJrJdjz9/j69D/Q=; c=relaxed/simple; d=groups.io; h=Message-ID:Date:MIME-Version:Subject:From:To:Cc:Reply-To:References:In-Reply-To:Precedence:List-Subscribe:List-Help:Sender:List-Id:Mailing-List:Delivered-To:List-Unsubscribe-Post:List-Unsubscribe:Content-Language:Content-Type:Content-Transfer-Encoding; s=20140610; t=1699020446; v=1; b=fZq3z5Tc8ozId6hTADxrmQxO9IVJN9FoNciNyJa1RZb89KNZnOKhGaIZSxiK2e+MZBwnosA3 MuVO+fu5PcUoEFL7R6BBzhWMtcZXpzAEWODuN8O+PKO37z8R9eAI1WCIW07NDl9msAJl2t8Fnls IkOuIeBN1s2oeauE48J7ttzY= X-Received: by 127.0.0.2 with SMTP id ROQIYY7687511xHtx58h8lb4; Fri, 03 Nov 2023 07:07:26 -0700 X-Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.groups.io with SMTP id smtpd.web11.52179.1699020445320935284 for ; Fri, 03 Nov 2023 07:07:25 -0700 X-Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-378--65JL-Y7OOWgvAGRQhKTCA-1; Fri, 03 Nov 2023 10:07:22 -0400 X-MC-Unique: -65JL-Y7OOWgvAGRQhKTCA-1 X-Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E97F9810FC4; Fri, 3 Nov 2023 14:07:14 +0000 (UTC) X-Received: from [10.39.192.20] (unknown [10.39.192.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id E31F32026D4C; Fri, 3 Nov 2023 14:07:13 +0000 (UTC) Message-ID: <360a4a15-ea24-b59f-d3a0-6ac3d5aef104@redhat.com> Date: Fri, 3 Nov 2023 15:07:12 +0100 MIME-Version: 1.0 Subject: Re: [edk2-devel] [PATCH v1] OvmfPkg/PlatformInitLib: Don't override user specified PciMmio64Size From: "Laszlo Ersek" To: devel@edk2.groups.io, vivek.kasireddy@intel.com Cc: Gerd Hoffmann , Ard Biesheuvel , Dongwon Kim Reply-To: devel@edk2.groups.io,lersek@redhat.com References: <20231103051519.277323-1-vivek.kasireddy@intel.com> <32d105da-7ccd-9acc-5cea-9a740bcc37f8@redhat.com> In-Reply-To: <32d105da-7ccd-9acc-5cea-9a740bcc37f8@redhat.com> X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Precedence: Bulk List-Subscribe: List-Help: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io List-Unsubscribe-Post: List-Unsubscribe=One-Click List-Unsubscribe: X-Gm-Message-State: AV9rqgRXC93oR8quOM18qBHWx7686176AA= Content-Language: en-US Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-GND-Status: LEGIT Authentication-Results: spool.mail.gandi.net; dkim=pass header.d=groups.io header.s=20140610 header.b=fZq3z5Tc; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=redhat.com (policy=none); spf=pass (spool.mail.gandi.net: domain of bounce@groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce@groups.io On 11/3/23 14:15, Laszlo Ersek wrote: > On 11/3/23 06:15, Vivek Kasireddy wrote: >> If the user specified a size for the PCI MMIO window via the option: >> -fw_cfg name=3Dopt/ovmf/X-PciMmio64Mb,string=3D32768 >> then this patch ensures that the mmio window is not resized again. >> >> Essentially, this prevents the change introduced in the following >> patch from taking effect: >> commit ecb778d0ac62560aa172786ba19521f27bc3f650 >> Author: Gerd Hoffmann >> Date: Tue Oct 4 15:47:27 2022 +0200 >> >> OvmfPkg/PlatformInitLib: dynamic mmio window size >> >> In case we have a reliable PhysMemAddressWidth use that to dynamical= ly >> size the 64bit address window. Allocate 1/8 of the physical address >> space and place the window at the upper end of the address space. >> >> The problem this patch is trying to solve is the VFIO mapping failures: >> VFIO_MAP_DMA failed: Invalid argument >> vfio_dma_map(0x557b2f2736d0, 0x380000000000, 0x1000000, 0x7f98ac400000) = =3D -22 (Invalid argument) >> that occur when we try to passthrough the graphics device to the guest: >> qemu-system-x86_64 -m 4096 -enable-kvm -cpu host -smp cores=3D4,threads= =3D2,sockets=3D1 >> -device vfio-pci,host=3D0000:00:02.0 -bios OVMF.fd -nographic >> >> The above failures seem to occur because of a mismatch between the >> PhysMemAddressWidth and the Host IOMMU address width. More specifically, >> if the PhysMemAddressWidth is bigger than the IOMMU address width, >> VFIO fails to map the MMIO regions as the IOVAs would be larger >> than the IOMMU aperture regions. When tested on modern Intel platforms >> such as ADL, MTL, etc, OVMF determines PhysMemAddressWidth =3D 46 which >> matches the Host address width but the IOMMU address width seems to >> range anywhere from 38 to 48 depending on the IOMMU hardware >> capabilities, version, etc. >> >> One way to address this issue is if we ensure that PhysMemAddressWidth >> matches IOMMU address width: >> -cpu host,host-phys-bits=3Don,host-phys-bits-limit=3D >> However, this requires the user to figure out the IOMMU address width; >> which can be determined by looking at the 16-21 bits of the cap value: >> cat /sys/devices/virtual/iommu/dmar0/intel-iommu/cap >> or by reading the DMAR_CAP_REG register. But this does not seem like >> a reasonable approach to solve this problem. >=20 > Very nice problem description, already outlining the solution as well. >=20 >> >> Therefore, this problem requires an OVMF specific solution to retain >> the prior behavior. To this end, this patch reuses the X-PciMmio64Mb >> option to opt-out of the behavior introduced in the above commit >> instead of adding a new option or mechanism. >=20 > No, the right solution is to enhance QEMU to query the host IOMMU > address width. Then the following options arise: >=20 > - either pass *both* the host CPU address width *and* the host IOMMU > address width down to OVMF, and teach OVMF to pick the stricter > limitation, for dynamically sizing the MMIO window >=20 > - or let QEMU calulate the stricter width internally, and pass that > (sole, scalar) piece of information down to OVMF. Teach OVMF to query > this new piece of information, and size the MMIO window accordingly. >=20 > Basically the QEMU command line-based workaround that you describe is > what we need to automate (except we need a new information channel for > it, because presenting the strict host IOMMU address width as the VCPU > address width (via CPUID) to the guest is smelly). >=20 > I agree that the proposed patch can function as a stop-gap, but the QEMU > command line hack is already a stop-gap. And for the long term, this > patch is not good enough. We should enhance the dynamic sizing, now that > Gerd has put it into place. ... I do agree however that the current behavior is strange -- the user specifies an explicit fw_cfg knob for OVMF, and OVMF ignores it (for whatever reason). I'd like to know what Gerd thinks of this. Personally I'd ACK the *code* in this patch, just to restore the correct priority between the dynamic sizing and the explicit fw_cfg knob, if: (a) the commit message referred to that exactly (i.e., the to the proper priority between these two configuration avenues), and (b) there were a promise to enhance QEMU and OVMF as I suggest above. I don't want the fw_cfg knob to stick around as a permanent excuse for not improving the dynamic sizing -- now that we *do have* dynamic sizing. BTW: I'd suggest renaming "QemuFwCfgSizeSpecified" to "PciMmio64MbOverride" or something like that. The important trait is that the value is an override (or direct setting) from the user; fw_cfg is incidental / irrelevant where the value is consumed. Thanks! Laszlo -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#110624): https://edk2.groups.io/g/devel/message/110624 Mute This Topic: https://groups.io/mt/102359124/7686176 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/leave/12367111/7686176/19134562= 12/xyzzy [rebecca@openfw.io] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-