From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from nwk-aaemail-lapp03.apple.com (nwk-aaemail-lapp03.apple.com [17.151.62.68]) by mx.groups.io with SMTP id smtpd.web10.4505.1595102886977867051 for ; Sat, 18 Jul 2020 13:08:07 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@apple.com header.s=20180706 header.b=ESSOUxH7; spf=pass (domain: apple.com, ip: 17.151.62.68, mailfrom: afish@apple.com) Received: from pps.filterd (nwk-aaemail-lapp03.apple.com [127.0.0.1]) by nwk-aaemail-lapp03.apple.com (8.16.0.42/8.16.0.42) with SMTP id 06IK53CH031781; Sat, 18 Jul 2020 13:08:05 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=apple.com; h=from : content-type : mime-version : subject : date : references : to : in-reply-to : message-id; s=20180706; bh=6GkZ6UwvjbOqvonjyA+gVKY1nHu4G4FOX6IKtcN+4l4=; b=ESSOUxH7wrnCT2uPQCVZcDQUY4kLhVwDRFBVgLdBwuxihxlhD0lfC76rzrRmqjiL4/qi llLkujCXnwCntTIpS2TD9nbY7huBTjFCh3KJ38sxQXFp9tDlEj7UowemNj+cs7v+MOs6 MAEDHY34c+v8rkUEH53eSbdjrGCyErJeaOfdkdTv62gCxq9m7zZBojduT4+AvL6tAS39 RCexkVDkRWGBQrZdaIC09nO0BBkTnLUB0mgoqmhtr0UZieCc9G2ifDpeE8sDiaOtTan4 gkrzN2ugdSvgEABtjHouNYr4QU0KtJ8bnytDOeObNwf/9VlkKwwDPZcGVO6VDUJ4mtcM eg== Received: from rn-mailsvcp-mta-lapp04.rno.apple.com (rn-mailsvcp-mta-lapp04.rno.apple.com [10.225.203.152]) by nwk-aaemail-lapp03.apple.com with ESMTP id 32bwm27u67-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Sat, 18 Jul 2020 13:08:04 -0700 Received: from rn-mailsvcp-mmp-lapp03.rno.apple.com (rn-mailsvcp-mmp-lapp03.rno.apple.com [17.179.253.16]) by rn-mailsvcp-mta-lapp04.rno.apple.com (Oracle Communications Messaging Server 8.1.0.5.20200312 64bit (built Mar 12 2020)) with ESMTPS id <0QDO010ULL9GHC60@rn-mailsvcp-mta-lapp04.rno.apple.com>; Sat, 18 Jul 2020 13:08:04 -0700 (PDT) Received: from process_milters-daemon.rn-mailsvcp-mmp-lapp03.rno.apple.com by rn-mailsvcp-mmp-lapp03.rno.apple.com (Oracle Communications Messaging Server 8.1.0.5.20200312 64bit (built Mar 12 2020)) id <0QDO00D00L1NGF00@rn-mailsvcp-mmp-lapp03.rno.apple.com>; Sat, 18 Jul 2020 13:08:04 -0700 (PDT) X-Va-A: X-Va-T-CD: e0acb9dc03d22e4581b62f3d752335f3 X-Va-E-CD: 01093fc030ab03d36fd5e7ee208a4cd6 X-Va-R-CD: a3f879d2c1accc6e75c86140cc99a4ed X-Va-CD: 0 X-Va-ID: a8e605b2-a031-4bd8-b0de-b473ce95faf3 X-V-A: X-V-T-CD: e0acb9dc03d22e4581b62f3d752335f3 X-V-E-CD: 01093fc030ab03d36fd5e7ee208a4cd6 X-V-R-CD: a3f879d2c1accc6e75c86140cc99a4ed X-V-CD: 0 X-V-ID: 7e0be5fc-4fbb-4f1a-b275-e9ad6887f28c X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235,18.0.687 definitions=2020-07-18_11:2020-07-17,2020-07-18 signatures=0 Received: from [17.235.11.135] (unknown [17.235.11.135]) by rn-mailsvcp-mmp-lapp03.rno.apple.com (Oracle Communications Messaging Server 8.1.0.5.20200312 64bit (built Mar 12 2020)) with ESMTPSA id <0QDO004FJL9EFQ00@rn-mailsvcp-mmp-lapp03.rno.apple.com>; Sat, 18 Jul 2020 13:08:04 -0700 (PDT) From: "Andrew Fish" MIME-version: 1.0 (Mac OS X Mail 13.4 \(3608.80.23.2.2\)) Subject: Re: [edk2-devel] [edk2-discuss] Need memory barriers in IoLib for AARCH64 Date: Sat, 18 Jul 2020 13:08:02 -0700 References: <0d68b7da-01c9-f696-7eb4-ebbef8a5cf0f@redhat.com> To: edk2-devel-groups-io , wasim.khan@nxp.com In-reply-to: Message-id: <36CBD1C5-7DA7-4B73-83CD-75B91487C9BF@apple.com> X-Mailer: Apple Mail (2.3608.80.23.2.2) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235,18.0.687 definitions=2020-07-18_11:2020-07-17,2020-07-18 signatures=0 Content-type: multipart/alternative; boundary="Apple-Mail=_48EF5F26-0042-4378-8BAF-0649764B5B6C" --Apple-Mail=_48EF5F26-0042-4378-8BAF-0649764B5B6C Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Jul 16, 2020, at 11:43 PM, Wasim Khan wrote: >=20 >=20 >=20 >> -----Original Message----- >> From: Laszlo Ersek > >> Sent: Monday, July 13, 2020 9:02 PM >> To: devel@edk2.groups.io ; afish@apple.com= ; Wasim Khan >> > >> Cc: Mike Kinney >; liming.gao@intel.com ; Leif >> Lindholm (Nuvia address) > >> Subject: Re: [edk2-devel] [edk2-discuss] Need memory barriers in IoLib = for >> AARCH64 >>=20 >> On 07/12/20 18:54, Andrew Fish via groups.io wrote: >>>=20 >>>=20 >>>> On Jul 11, 2020, at 10:17 PM, Wasim Khan wrote: >>>>=20 >>>> Hello >>>>=20 >>>> Any comments ? >>>>=20 >>>=20 >>> I don=E2=80=99t see IoLibArm.c in master? I see IoLibNoIo.c. >>=20 >> That's due to the rename in commit 089e9c19a8c1 >> ("MdePkg/BaseIoLibIntrinsic: Rename IoLibArm.c=3D>IoLibNoIo.c", 2020-05= -07), >> which has been first included in edk2-stable202005. >>=20 >> I think Ard is away at the moment, so I'm adding Leif to the CC list. >>=20 >> Thanks >> Laszlo >>=20 >>> The MMIO function look like ARM assembler with the correct barrier >> instructions. The IO operations in this lib are the x86 in/out instruct= ions, so they >> just ASSERT on ARM. >>>=20 >>> On the X86 MemoryFence() is just a serializing intrinsic for the compi= ler to >> prevent optimizations from breaking the code, kind of like how you need= to >> make MMIO as volatile in C. >>>=20 >>> Thanks, >>>=20 >>> Andrew Fish >=20 > Thank you Andrew and Laszlo for you response.=20 > My problem was that because there is no MemoryFence() in MmioRead/Write = functions in IoLibNoIo.c, I was facing some serializing problem with AARCH6= 4. > I have two options: > Either put MemoryFence() in my code under edl2-platforms before and afte= r MmioRead/Write calls > OR > Add the MemoryFence() in MmioRead/Write itself for IoLibNoIo.c, like it = is done for IoLib.c . Because I don=E2=80=99t see any harm to make sure tha= t MmioRead/Write operation are serialized by using MemoryFence() in it.=20 >=20 > I prefer later option, if there is no specific reason for not adding Mem= oryFence() for IoLibNoIo.c. >=20 Look at BaseIoLibIntrinsicArmVirt.inf [1] I think it does what you want.= =20 I don=E2=80=99t know the history of the original ARM BaseIoLibIntrinsic po= rt, but it does look like there is an assumption that with volatile the com= piler would =E2=80=9Cdo the right thing=E2=80=9D. The ArmVirt flavor seems = to have the serializing instructions you are looking for?=20 [1] https://github.com/tianocore/edk2/blob/master/MdePkg/Library/BaseIoLib= Intrinsic/BaseIoLibIntrinsicArmVirt.inf [2] git log MdePkg/Library/BaseIoLibIntrinsic/BaseIoLibIntrinsicArmVirt.in= f=20 ... commit 4134f2bddcb68d2e20ed000cdf54abf3f1140904 Author: Ard Biesheuvel Date: Thu Jun 7 12:44:12 2018 +0200 MdePkg/BaseIoLibIntrinsic: make BaseIoLibIntrinsic safe for ArmVirt/KV= M KVM on ARM refuses to decode load/store instructions used to perform I/O to emulated devices, and instead relies on the exception syndrome information to describe the operand register, access size, etc. This is only possible for instructions that have a single input/output register (as opposed to ones that increment the offset register, or load/store pair instructions, etc). Otherwise, QEMU crashes with the following error error: kvm run failed Function not implemented R00=3D01010101 R01=3D00000008 R02=3D00000048 R03=3D08000820 R04=3D00000120 R05=3D7faaa0e0 R06=3D7faaa0dc R07=3D7faaa0e8 R08=3D7faaa0ec R09=3D7faaa088 R10=3D000000ff R11=3D00000080 R12=3Dff000000 R13=3D7fccfe08 R14=3D7faa835f R15=3D7faa887c PSR=3D800001f3 N--- T svc32 QEMU: Terminated and KVM produces a warning such as the following in the kernel log kvm [17646]: load/store instruction decoding not implemented The IoLib implementation provided by MdePkg/Library/BaseIoLibIntrinsic is based on C code, and when LTO is in effect, the MMIO accesses could be merged with, e.g., manipulations of the loop counter, producing opcodes that KVM does not support for emulated MMIO. So let's add a special ArmVirt flavor of this library that implements that actual load/store operations in assembler, ensuring that the instructions involved can be emulated by KVM. Contributed-under: TianoCore Contribution Agreement 1.1 Signed-off-by: Ard Biesheuvel Reviewed-by: Leif Lindholm Acked-by: Laszlo Ersek Reviewed-by: Liming Gao Thanks, Andrew Fish >>>=20 >>>>> -----Original Message----- >>>>> From: Wasim Khan >>>>> Sent: Friday, July 10, 2020 6:20 PM >>>>> To: michael.d.kinney@intel.com; liming.gao@intel.com; >>>>> devel@edk2.groups.io >>>>> Subject: [edk2-discuss] Need memory barriers in IoLib for AARCH64 >>>>>=20 >>>>> Hello, >>>>>=20 >>>>> MdePkg/Library/BaseIoLibIntrinsic/BaseIoLibIntrinsic.inf: >>>>> IoLib library uses IoLibArm.c for AARCH64/ARM architecture and >>>>> IoLib.c for other architectures. >>>>>=20 >>>>> While IoLib.c already has memory barriers in MmioWrite functions, >>>>> there barriers are missing in IoLibArm.c Is there any reason for >>>>> **not** adding these memory barriers in IoLibArm.c to guarantee that >>>>> all MMIO operations are serialized ? >>>>>=20 >>>>> I am facing some issues and I need to add memory barriers in >>>>> IoLibArm.c for >>>>> AARCH64 also . >>>>>=20 >>>>>=20 >>>>> Regards, >>>>> Wasim >>>>=20 >>>>=20 >>>>=20 >>>=20 >>>=20 >>>=20 >>>=20 >>>=20 >=20 >=20 >=20 --Apple-Mail=_48EF5F26-0042-4378-8BAF-0649764B5B6C Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8

On Jul 16, 2= 020, at 11:43 PM, Wasim Khan <wasim.khan@nxp.com> wrote:



-----Original Message---= --
From: Laszlo Ersek <lersek@redhat.com>
Sent: Monday, July 13,= 2020 9:02 PM
To: = devel@edk2.groups= .io; afish@apple.com; Wasim Khan
<wasim.khan@nxp.com>
Cc: Mike Kinney <michael.d.kinney@intel.com>; liming.gao@intel.com; Leif
Lindholm (Nuvia addre= ss) <leif@nuviainc.com>
Subject: Re: [edk2-devel] [edk2-discuss] Need memory b= arriers in IoLib for
AARCH64

On = 07/12/20 18:54, Andrew Fish via gro= ups.io wrote:


On Jul 11, 202= 0, at 10:17 PM, Wasim Khan <wasim.khan@nxp.com> wrote:

Hello=

Any comments ?

<= /blockquote>
I don=E2=80=99t see IoLibArm.c in master? I see = IoLibNoIo.c.

That's due to the re= name in commit 089e9c19a8c1
("MdePkg/BaseIoLibIntrinsic: Rena= me IoLibArm.c=3D>IoLibNoIo.c", 2020-05-07),
which has been= first included in edk2-stable202005.

I think = Ard is away at the moment, so I'm adding Leif to the CC list.

Thanks
Laszlo

The MMIO function look like ARM assemble= r with the correct barrier
instructions. The IO = operations in this lib are the x86 in/out instructions, so they
just ASSERT on ARM.
<= br class=3D"">On the X86 MemoryFence() is just a serializing intrinsic for = the compiler to
prevent optimizations from break= ing the code, kind of like how you need to
make MMIO as volat= ile in C.

Thanks,

Andrew Fish

Thank you Andr= ew and Laszlo for you response. =
My problem was = that because there is no MemoryFence() in MmioRead/Write functions in IoLib= NoIo.c, I was facing some serializing problem with AARCH64.
I have two options:
Either put MemoryFence() in my code u= nder edl2-platforms before and after MmioRead/Write calls
OR
Add the MemoryFence() in MmioRead/Write itself for IoLibN= oIo.c, like it is done for IoLib.c . Because I don=E2=80=99t see any harm t= o make sure that MmioRead/Write operation are serialized by using MemoryFen= ce() in it. 

I prefer later option, if there is no specific reason for not adding= MemoryFence() for IoLibNoIo.c.


Look at BaseIoLibIntrinsicArmVirt.inf [1] I think it does= what you want. 

I don=E2=80=99t k= now the history of the original ARM BaseIoLibIntrinsic port, but it does lo= ok like there is an assumption that with volatile the compiler would =E2=80= = =9Cdo the right thing=E2=80=9D. The ArmVirt flavor seems to have the seria= lizing instructions you are looking for? 


= [2] gi= t log MdePkg/Library/BaseIoLibIntrinsic/BaseIoLibIntrinsicArmVirt.inf 
...
commit 4134f2bddcb68d2e20ed000cdf54abf3f1140904
=
Author: Ard Biesheuvel <ard.biesheuvel@linaro.org= >
Date:   T= hu Jun 7 12:44:12 2018 +0200

    MdePkg/BaseIoLibIntrinsic: make BaseIoLibInt= rinsic safe for ArmVirt/KVM

    

    KVM on= ARM refuses to decode load/store instructions used to perform
=
    I/O to emulated devic= es, and instead relies on the exception syndrome
    information to describe the operand= register, access size, etc.
    This is only possible for instructions that have a sin= gle input/output
 =   register (as opposed to ones that increment the offset register, or=
    load/sto= re pair instructions, etc). Otherwise, QEMU crashes with the
    following error<= /div>

   &= nbsp;

      error: kvm run failed Function not= implemented
  &nb= sp;   R00=3D01010101 R01=3D00000008 R02=3D00000048 R03=3D08000820
      R04=3D= 00000120 R05=3D7faaa0e0 R06=3D7faaa0dc R07=3D7faaa0e8
      R08=3D7faaa0ec R09=3D7f= aaa088 R10=3D000000ff R11=3D00000080
      R12=3Dff000000 R13=3D7fccfe08 R14=3D7faa= 835f R15=3D7faa887c
&nb= sp;     PSR=3D800001f3 N--- T svc32
      QEMU: Terminated
    

    and KVM produces a warning such as the following= in the kernel log

    

      kvm [176= 46]: load/store instruction decoding not implemented

    =

    The IoLib implementation provided by MdePkg/Library/B= aseIoLibIntrinsic
<= span style=3D"font-variant-ligatures: no-common-ligatures" class=3D""> = ;   is based on C code, and when LTO is in effect, the MMIO accesses c= ould
    be m= erged with, e.g., manipulations of the loop counter, producing
=
    opcodes that KVM does= not support for emulated MMIO.

    

    = So let's add a special ArmVirt flavor of this library that implements
    that actual lo= ad/store operations in assembler, ensuring that the
    instructions involved can be e= mulated by KVM.

    
<= /p>

    Contributed-under:= TianoCore Contribution Agreement 1.1
    Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org&g= t;
    Review= ed-by: Leif Lindholm <leif.lindholm@linaro.org>
    Acked-by: Laszlo Ersek <lersek@redhat.com>
    Reviewed-by: Liming Gao <= ;liming.gao@intel.com>

Thanks,

Andrew Fish


-----Original Message-----
From: Wasim Khan
Sent: Friday, July 10, 2020 6:20 P= M
To: michael.d.kinney@intel.com; liming.gao@intel.com;
devel@edk2.groups.io
Subject: [= edk2-discuss] Need memory barriers in IoLib for AARCH64

Hello,

MdePkg/Library/BaseIoLibIntri= nsic/BaseIoLibIntrinsic.inf:
IoLib library uses IoLibArm.c fo= r AARCH64/ARM architecture and
IoLib.c for other architecture= s.

While IoLib.c already has memory barriers i= n MmioWrite functions,
there barriers are missing in IoLibArm= .c Is there any reason for
**not** adding these memory barrie= rs in IoLibArm.c to guarantee that
all MMIO operations are se= rialized ?

I am facing some issues and I need = to add memory barriers in
IoLibArm.c for
AARCH6= 4 also .


Regards,
Wasim











--Apple-Mail=_48EF5F26-0042-4378-8BAF-0649764B5B6C--