From mboxrd@z Thu Jan  1 00:00:00 1970
Received: from mail-ej1-f45.google.com (mail-ej1-f45.google.com [209.85.218.45])
 by mx.groups.io with SMTP id smtpd.web09.5021.1616659362017822276
 for <devel@edk2.groups.io>;
 Thu, 25 Mar 2021 01:02:42 -0700
Authentication-Results: mx.groups.io;
 dkim=pass header.i=@solid-run-com.20150623.gappssmtp.com header.s=20150623 header.b=knD2c1es;
 spf=pass (domain: solid-run.com, ip: 209.85.218.45, mailfrom: jon@solid-run.com)
Received: by mail-ej1-f45.google.com with SMTP id w3so1411023ejc.4
        for <devel@edk2.groups.io>; Thu, 25 Mar 2021 01:02:41 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=solid-run-com.20150623.gappssmtp.com; s=20150623;
        h=mime-version:references:in-reply-to:from:date:message-id:subject:to
         :cc;
        bh=6a3wtVoHayl5teVRaiABcLCBbpI2qova5GsrGyOEfrA=;
        b=knD2c1eseRFy1Wo2Hb/SaMZuDF/k708TeKsKINk3lgUIy/YSQ46kQcCZAPoIb1isHv
         KaCq9ZLqoYo+iVbkdcxF1+vjS8otE7eRAM3v7OMHCp+xtwo3JonCUa6Wp11t7Lkk4+bP
         /IvtQp5snr+uDAdixlTzVOMC2gxqp19RmghjUjuzshZWHxJOND6ljqFN8W7xW3ZYN+9A
         /P6aHWSGPdIW9vquEO/UoK/Ep7hVdkzNcBf7SDGCtO/H7GzUrS3W5s7gcQYqo0AJV9QA
         RNlFIkfwUV9Rcec/ugQAdgDppqVkorFSLe0fWJvN357GPX/yhKpTJlrOdZnOPAl+Ekq7
         9ppA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:mime-version:references:in-reply-to:from:date
         :message-id:subject:to:cc;
        bh=6a3wtVoHayl5teVRaiABcLCBbpI2qova5GsrGyOEfrA=;
        b=he2Z5ZicL2Q7e822CXHPw4RXz71W/Z42Xlzlw7c2m0Sgtt0rz/R1i9z40tmNR3dnF9
         bHqbDU2dPPksePrlhUM7wGpyj+sK1vKQ6F0JD/QCzg5v/Sfw9eqYm41hwXwmRIuoTtBU
         qzquGAe299xr16ZR+DYJg3dXWprWqBDPGr+8oK9mlKndUYRmwEW8beW2yxXjNU3XTQEV
         sAgBprwzJrEFHkuleHfUXDNHtVV+0GC9UBwo2BmZPMhviQBIpW7Fg4V3jdcsFGE6QMvC
         uZESMBVK44Y4/hBk4KcedfMwe3DfF2qA8gH0Q0beH6y8ADt7eZ1DPNFVAb14gRIm2owp
         n+ug==
X-Gm-Message-State: AOAM532LqQ1qlXFdNPdHZhSTUHXhIcgm+8cbLtMmiLx2l7vikz9I+ROx
	aDB3RAtw0vDRT9B260QyhMBZ897CvO27ybf32Vswng==
X-Google-Smtp-Source: ABdhPJynSlI0OiScm+wRmq5s+XANkiNI64hX60WT5VJFfcTsE8cRcxxNafb+YvRrFXjNYEt6oOv/SYmbTvsFBJB+YtU=
X-Received: by 2002:a17:906:fc1c:: with SMTP id ov28mr7956076ejb.342.1616659360291;
 Thu, 25 Mar 2021 01:02:40 -0700 (PDT)
MIME-Version: 1.0
References: <CABdtJHsn8V6ZjZM0_tFkHNzg0gmg0sH81csDA6goHaO0DjM1Wg@mail.gmail.com>
 <986c079a-137f-c3e5-2e85-1aeccf148f8c@redhat.com>
In-Reply-To: <986c079a-137f-c3e5-2e85-1aeccf148f8c@redhat.com>
From: "Jon Nettleton" <jon@solid-run.com>
Date: Thu, 25 Mar 2021 09:02:03 +0100
Message-ID: <CABdtJHsWACL-db4t_i-taFTN05Ttup=NXvaNeX-1e-SRXrPY6Q@mail.gmail.com>
Subject: Re: [edk2-devel] SR-IOV setup in edk2
To: Laszlo Ersek <lersek@redhat.com>
Cc: devel@edk2.groups.io
Content-Type: text/plain; charset="UTF-8"

On Mon, Mar 22, 2021 at 8:04 PM Laszlo Ersek <lersek@redhat.com> wrote:
>
> On 03/21/21 13:46, Jon Nettleton wrote:
> > I am looking for some example code, or direction in how SR-IOV
> > functions are expected to be picked up by the platform pcie host
> > library so it can setup required bits like LUTs.  Currently the
> > HostLib is getting the gEfiPciIoProtocolGuid event and then setting
> > the controller up based on the BDF.  I see that PciScanBus is
> > detecting the VF's and calling PciAllocateBusNumber().
> >
> > PCI-IOV ScanBus - SubBusNumber - 0x2
> > PciBus: Discovered PPB @ [00|00|00]
> >
> > However my setup function is only ever triggered with the real device
> > BDF's...
>
> What do you mean by "setup function"?

Well currently we followed the methodology of most the other
PCI Host Bus drivers and as I said above get the Io event and then
finish the completion of the device setup.  This includes setting up
the Luts, adding the device to the SMMU tables, and a few other
operations.

The VFs for SR-IOV are different as they are treated as hot
pluggable devices in both edk2 and linux, however there is still
setup that needs to be done to the host controller to support the
VFs on the bus.  The PciScanBus is obviously partially wired to
do some enumeration for PCI-IOV but I am unclear where the
platform code should hook into this for the additional proving
and setup.

I have now implemented a driver connected to gEfiPciPlatformProtocolGuid
and I can get events passed in for the platform setup hooks, is this where
we should be doing the bus specific setup, rather than catching
gEfiPciIoProtocolGuid?

>
> If you have a UEFI driver that follows the UEFI driver model, i.e. it
> installs at least one instance of the Driver Binding protocol, then it's
> up to platform BDS to call ConnectController() on those devices that
> should be connected per platform policy.
>
> > Is it up to me to check for the VFs if SR-IOV is enabled and
> > set them up manually?  Is there another event I should be listening to
> > that will be triggered for VF's vs PF's?
>
> My guess is that platform BDS does not try to connect the PciIo
> instances that stand for VFs to any drivers.
>
> In the UEFI shell, try running
>
>   dh -d -v -p PciIo
>
> Pick a handle ID that appears to stand for a VF, then run

No handles are created for the VFs as they aren't setup yet.

Thanks,
Jon

>
>   connect <that handle>
>
> Thanks
> Laszlo
>
> >
> > Any pointers would be very helpful.
> >
> > -Jon
> >
> >
> > 
> >
> >
>