public inbox for devel@edk2.groups.io
 help / color / mirror / Atom feed
* [PATCH] ArmVirtPkg: implement KVM safe IoLib instance
@ 2018-06-05 11:05 Ard Biesheuvel
  2018-06-05 11:29 ` Leif Lindholm
  2018-06-05 13:04 ` Laszlo Ersek
  0 siblings, 2 replies; 4+ messages in thread
From: Ard Biesheuvel @ 2018-06-05 11:05 UTC (permalink / raw)
  To: edk2-devel; +Cc: lersek, leif.lindholm, Ard Biesheuvel

KVM on ARM refuses to decode load/store instructions used to perform
I/O to emulated devices, and instead relies on the exception syndrome
information to describe the operand register, access size, etc.
This is only possible for instructions that have a single input/output
register (as opposed to ones that increment the offset register, or
load/store pair instructions, etc). Otherwise, QEMU crashes with the
following error

  error: kvm run failed Function not implemented
  R00=01010101 R01=00000008 R02=00000048 R03=08000820
  R04=00000120 R05=7faaa0e0 R06=7faaa0dc R07=7faaa0e8
  R08=7faaa0ec R09=7faaa088 R10=000000ff R11=00000080
  R12=ff000000 R13=7fccfe08 R14=7faa835f R15=7faa887c
  PSR=800001f3 N--- T svc32
  QEMU: Terminated

and KVM produces a warning such as the following in the kernel log

  kvm [17646]: load/store instruction decoding not implemented

The IoLib implementation provided by MdePkg/Library/BaseIoLibIntrinsic
is based on C code, and when LTO is in effect, the MMIO accesses could
be merged with, e.g., manipulations of the loop counter, producing
opcodes that KVM does not support for emulated MMIO.

So instead, let's reimplement IoLib in a KVM safe manner.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
Yet another approach for the KVM MMIO emulation issue. Note that this one
(as well as the MdePkg) affect both AArch64 and ARM. This is deliberate,
given that there is no reason AArch64 should be immune to this: we simply
haven't triggered the issue yet.

 ArmVirtPkg/Library/ArmVirtIoLib/AArch64/ArmVirtMmio.S |  164 ++
 ArmVirtPkg/Library/ArmVirtIoLib/Arm/ArmVirtMmio.S     |  154 ++
 ArmVirtPkg/Library/ArmVirtIoLib/Arm/ArmVirtMmio.asm   |  165 ++
 ArmVirtPkg/Library/ArmVirtIoLib/ArmVirtIoLib.c        |  589 +++++
 ArmVirtPkg/Library/ArmVirtIoLib/ArmVirtIoLib.h        |  188 ++
 ArmVirtPkg/Library/ArmVirtIoLib/ArmVirtIoLib.inf      |   49 +
 ArmVirtPkg/Library/ArmVirtIoLib/IoHighLevel.c         | 2358 ++++++++++++++++++++
 ArmVirtPkg/Library/ArmVirtIoLib/IoLibMmioBuffer.c     |  413 ++++
 8 files changed, 4080 insertions(+)

diff --git a/ArmVirtPkg/Library/ArmVirtIoLib/AArch64/ArmVirtMmio.S b/ArmVirtPkg/Library/ArmVirtIoLib/AArch64/ArmVirtMmio.S
new file mode 100644
index 000000000000..47be68a3e783
--- /dev/null
+++ b/ArmVirtPkg/Library/ArmVirtIoLib/AArch64/ArmVirtMmio.S
@@ -0,0 +1,164 @@
+#
+#  Copyright (c) 2014-2018, Linaro Limited. All rights reserved.
+#
+#  This program and the accompanying materials are licensed and made available
+#  under the terms and conditions of the BSD License which accompanies this
+#  distribution.  The full text of the license may be found at
+#  http://opensource.org/licenses/bsd-license.php
+#
+#  THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED.
+#
+#
+
+.text
+.align 3
+
+GCC_ASM_EXPORT(MmioRead8Internal)
+GCC_ASM_EXPORT(MmioWrite8Internal)
+GCC_ASM_EXPORT(MmioRead16Internal)
+GCC_ASM_EXPORT(MmioWrite16Internal)
+GCC_ASM_EXPORT(MmioRead32Internal)
+GCC_ASM_EXPORT(MmioWrite32Internal)
+GCC_ASM_EXPORT(MmioRead64Internal)
+GCC_ASM_EXPORT(MmioWrite64Internal)
+
+//
+//  Reads an 8-bit MMIO register.
+//
+//  Reads the 8-bit MMIO register specified by Address. The 8-bit read value is
+//  returned. This function must guarantee that all MMIO read and write
+//  operations are serialized.
+//
+//  If 8-bit MMIO register operations are not supported, then ASSERT().
+//
+//  @param  Address The MMIO register to read.
+//
+//  @return The value read.
+//
+ASM_PFX(MmioRead8Internal):
+  ldrb    w0, [x0]
+  dmb     ld
+  ret
+
+//
+//  Writes an 8-bit MMIO register.
+//
+//  Writes the 8-bit MMIO register specified by Address with the value specified
+//  by Value and returns Value. This function must guarantee that all MMIO read
+//  and write operations are serialized.
+//
+//  If 8-bit MMIO register operations are not supported, then ASSERT().
+//
+//  @param  Address The MMIO register to write.
+//  @param  Value   The value to write to the MMIO register.
+//
+ASM_PFX(MmioWrite8Internal):
+  dmb     st
+  strb    w1, [x0]
+  ret
+
+//
+//  Reads a 16-bit MMIO register.
+//
+//  Reads the 16-bit MMIO register specified by Address. The 16-bit read value is
+//  returned. This function must guarantee that all MMIO read and write
+//  operations are serialized.
+//
+//  If 16-bit MMIO register operations are not supported, then ASSERT().
+//
+//  @param  Address The MMIO register to read.
+//
+//  @return The value read.
+//
+ASM_PFX(MmioRead16Internal):
+  ldrh    w0, [x0]
+  dmb     ld
+  ret
+
+//
+//  Writes a 16-bit MMIO register.
+//
+//  Writes the 16-bit MMIO register specified by Address with the value specified
+//  by Value and returns Value. This function must guarantee that all MMIO read
+//  and write operations are serialized.
+//
+//  If 16-bit MMIO register operations are not supported, then ASSERT().
+//
+//  @param  Address The MMIO register to write.
+//  @param  Value   The value to write to the MMIO register.
+//
+ASM_PFX(MmioWrite16Internal):
+  dmb     st
+  strh    w1, [x0]
+  ret
+
+//
+//  Reads a 32-bit MMIO register.
+//
+//  Reads the 32-bit MMIO register specified by Address. The 32-bit read value is
+//  returned. This function must guarantee that all MMIO read and write
+//  operations are serialized.
+//
+//  If 32-bit MMIO register operations are not supported, then ASSERT().
+//
+//  @param  Address The MMIO register to read.
+//
+//  @return The value read.
+//
+ASM_PFX(MmioRead32Internal):
+  ldr     w0, [x0]
+  dmb     ld
+  ret
+
+//
+//  Writes a 32-bit MMIO register.
+//
+//  Writes the 32-bit MMIO register specified by Address with the value specified
+//  by Value and returns Value. This function must guarantee that all MMIO read
+//  and write operations are serialized.
+//
+//  If 32-bit MMIO register operations are not supported, then ASSERT().
+//
+//  @param  Address The MMIO register to write.
+//  @param  Value   The value to write to the MMIO register.
+//
+ASM_PFX(MmioWrite32Internal):
+  dmb     st
+  str     w1, [x0]
+  ret
+
+//
+//  Reads a 64-bit MMIO register.
+//
+//  Reads the 64-bit MMIO register specified by Address. The 64-bit read value is
+//  returned. This function must guarantee that all MMIO read and write
+//  operations are serialized.
+//
+//  If 64-bit MMIO register operations are not supported, then ASSERT().
+//
+//  @param  Address The MMIO register to read.
+//
+//  @return The value read.
+//
+ASM_PFX(MmioRead64Internal):
+  ldr     x0, [x0]
+  dmb     ld
+  ret
+
+//
+//  Writes a 64-bit MMIO register.
+//
+//  Writes the 64-bit MMIO register specified by Address with the value specified
+//  by Value and returns Value. This function must guarantee that all MMIO read
+//  and write operations are serialized.
+//
+//  If 64-bit MMIO register operations are not supported, then ASSERT().
+//
+//  @param  Address The MMIO register to write.
+//  @param  Value   The value to write to the MMIO register.
+//
+ASM_PFX(MmioWrite64Internal):
+  dmb     st
+  str     x1, [x0]
+  ret
diff --git a/ArmVirtPkg/Library/ArmVirtIoLib/Arm/ArmVirtMmio.S b/ArmVirtPkg/Library/ArmVirtIoLib/Arm/ArmVirtMmio.S
new file mode 100644
index 000000000000..b5df7e9047b4
--- /dev/null
+++ b/ArmVirtPkg/Library/ArmVirtIoLib/Arm/ArmVirtMmio.S
@@ -0,0 +1,154 @@
+#
+#  Copyright (c) 2014-2018, Linaro Limited. All rights reserved.
+#
+#  This program and the accompanying materials are licensed and made available
+#  under the terms and conditions of the BSD License which accompanies this
+#  distribution.  The full text of the license may be found at
+#  http://opensource.org/licenses/bsd-license.php
+#
+#  THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED.
+#
+#
+
+#include <AsmMacroIoLib.h>
+
+//
+//  Reads an 8-bit MMIO register.
+//
+//  Reads the 8-bit MMIO register specified by Address. The 8-bit read value is
+//  returned. This function must guarantee that all MMIO read and write
+//  operations are serialized.
+//
+//  If 8-bit MMIO register operations are not supported, then ASSERT().
+//
+//  @param  Address The MMIO register to read.
+//
+//  @return The value read.
+//
+ASM_FUNC(MmioRead8Internal)
+  ldrb    r0, [r0]
+  dmb
+  bx      lr
+
+//
+//  Writes an 8-bit MMIO register.
+//
+//  Writes the 8-bit MMIO register specified by Address with the value specified
+//  by Value and returns Value. This function must guarantee that all MMIO read
+//  and write operations are serialized.
+//
+//  If 8-bit MMIO register operations are not supported, then ASSERT().
+//
+//  @param  Address The MMIO register to write.
+//  @param  Value   The value to write to the MMIO register.
+//
+ASM_FUNC(MmioWrite8Internal)
+  dmb     st
+  strb    r1, [r0]
+  bx      lr
+
+//
+//  Reads a 16-bit MMIO register.
+//
+//  Reads the 16-bit MMIO register specified by Address. The 16-bit read value is
+//  returned. This function must guarantee that all MMIO read and write
+//  operations are serialized.
+//
+//  If 16-bit MMIO register operations are not supported, then ASSERT().
+//
+//  @param  Address The MMIO register to read.
+//
+//  @return The value read.
+//
+ASM_FUNC(MmioRead16Internal)
+  ldrh    r0, [r0]
+  dmb
+  bx      lr
+
+//
+//  Writes a 16-bit MMIO register.
+//
+//  Writes the 16-bit MMIO register specified by Address with the value specified
+//  by Value and returns Value. This function must guarantee that all MMIO read
+//  and write operations are serialized.
+//
+//  If 16-bit MMIO register operations are not supported, then ASSERT().
+//
+//  @param  Address The MMIO register to write.
+//  @param  Value   The value to write to the MMIO register.
+//
+ASM_FUNC(MmioWrite16Internal)
+  dmb     st
+  strh    r1, [r0]
+  bx      lr
+
+//
+//  Reads a 32-bit MMIO register.
+//
+//  Reads the 32-bit MMIO register specified by Address. The 32-bit read value is
+//  returned. This function must guarantee that all MMIO read and write
+//  operations are serialized.
+//
+//  If 32-bit MMIO register operations are not supported, then ASSERT().
+//
+//  @param  Address The MMIO register to read.
+//
+//  @return The value read.
+//
+ASM_FUNC(MmioRead32Internal)
+  ldr     r0, [r0]
+  dmb
+  bx      lr
+
+//
+//  Writes a 32-bit MMIO register.
+//
+//  Writes the 32-bit MMIO register specified by Address with the value specified
+//  by Value and returns Value. This function must guarantee that all MMIO read
+//  and write operations are serialized.
+//
+//  If 32-bit MMIO register operations are not supported, then ASSERT().
+//
+//  @param  Address The MMIO register to write.
+//  @param  Value   The value to write to the MMIO register.
+//
+ASM_FUNC(MmioWrite32Internal)
+  dmb     st
+  str     r1, [r0]
+  bx      lr
+
+//
+//  Reads a 64-bit MMIO register.
+//
+//  Reads the 64-bit MMIO register specified by Address. The 64-bit read value is
+//  returned. This function must guarantee that all MMIO read and write
+//  operations are serialized.
+//
+//  If 64-bit MMIO register operations are not supported, then ASSERT().
+//
+//  @param  Address The MMIO register to read.
+//
+//  @return The value read.
+//
+ASM_FUNC(MmioRead64Internal)
+  ldrd    r0, r1, [r0]
+  dmb
+  bx      lr
+
+//
+//  Writes a 64-bit MMIO register.
+//
+//  Writes the 64-bit MMIO register specified by Address with the value specified
+//  by Value and returns Value. This function must guarantee that all MMIO read
+//  and write operations are serialized.
+//
+//  If 64-bit MMIO register operations are not supported, then ASSERT().
+//
+//  @param  Address The MMIO register to write.
+//  @param  Value   The value to write to the MMIO register.
+//
+ASM_FUNC(MmioWrite64Internal)
+  dmb     st
+  strd    r2, r3, [r0]
+  bx      lr
diff --git a/ArmVirtPkg/Library/ArmVirtIoLib/Arm/ArmVirtMmio.asm b/ArmVirtPkg/Library/ArmVirtIoLib/Arm/ArmVirtMmio.asm
new file mode 100644
index 000000000000..82add82f800a
--- /dev/null
+++ b/ArmVirtPkg/Library/ArmVirtIoLib/Arm/ArmVirtMmio.asm
@@ -0,0 +1,165 @@
+;
+;  Copyright (c) 2014-2018, Linaro Limited. All rights reserved.
+;
+;  This program and the accompanying materials are licensed and made available
+;  under the terms and conditions of the BSD License which accompanies this
+;  distribution.  The full text of the license may be found at
+;  http:;opensource.org/licenses/bsd-license.php
+;
+;  THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS,
+;  WITHOUT WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED.
+;
+
+
+AREA IoLibMmio, CODE, READONLY
+
+EXPORT MmioRead8Internal
+EXPORT MmioWrite8Internal
+EXPORT MmioRead16Internal
+EXPORT MmioWrite16Internal
+EXPORT MmioRead32Internal
+EXPORT MmioWrite32Internal
+EXPORT MmioRead64Internal
+EXPORT MmioWrite64Internal
+
+;
+;  Reads an 8-bit MMIO register.
+;
+;  Reads the 8-bit MMIO register specified by Address. The 8-bit read value is
+;  returned. This function must guarantee that all MMIO read and write
+;  operations are serialized.
+;
+;  If 8-bit MMIO register operations are not supported, then ASSERT().
+;
+;  @param  Address The MMIO register to read.
+;
+;  @return The value read.
+;
+MmioRead8Internal
+  ldrb    r0, [r0]
+  dmb
+  bx      lr
+
+;
+;  Writes an 8-bit MMIO register.
+;
+;  Writes the 8-bit MMIO register specified by Address with the value specified
+;  by Value and returns Value. This function must guarantee that all MMIO read
+;  and write operations are serialized.
+;
+;  If 8-bit MMIO register operations are not supported, then ASSERT().
+;
+;  @param  Address The MMIO register to write.
+;  @param  Value   The value to write to the MMIO register.
+;
+MmioWrite8Internal
+  dmb     st
+  strb    r1, [r0]
+  bx      lr
+
+;
+;  Reads a 16-bit MMIO register.
+;
+;  Reads the 16-bit MMIO register specified by Address. The 16-bit read value is
+;  returned. This function must guarantee that all MMIO read and write
+;  operations are serialized.
+;
+;  If 16-bit MMIO register operations are not supported, then ASSERT().
+;
+;  @param  Address The MMIO register to read.
+;
+;  @return The value read.
+;
+MmioRead16Internal
+  ldrh    r0, [r0]
+  dmb
+  bx      lr
+
+;
+;  Writes a 16-bit MMIO register.
+;
+;  Writes the 16-bit MMIO register specified by Address with the value specified
+;  by Value and returns Value. This function must guarantee that all MMIO read
+;  and write operations are serialized.
+;
+;  If 16-bit MMIO register operations are not supported, then ASSERT().
+;
+;  @param  Address The MMIO register to write.
+;  @param  Value   The value to write to the MMIO register.
+;
+MmioWrite16Internal
+  dmb     st
+  strh    r1, [r0]
+  bx      lr
+
+;
+;  Reads a 32-bit MMIO register.
+;
+;  Reads the 32-bit MMIO register specified by Address. The 32-bit read value is
+;  returned. This function must guarantee that all MMIO read and write
+;  operations are serialized.
+;
+;  If 32-bit MMIO register operations are not supported, then ASSERT().
+;
+;  @param  Address The MMIO register to read.
+;
+;  @return The value read.
+;
+MmioRead32Internal
+  ldr     r0, [r0]
+  dmb
+  bx      lr
+
+;
+;  Writes a 32-bit MMIO register.
+;
+;  Writes the 32-bit MMIO register specified by Address with the value specified
+;  by Value and returns Value. This function must guarantee that all MMIO read
+;  and write operations are serialized.
+;
+;  If 32-bit MMIO register operations are not supported, then ASSERT().
+;
+;  @param  Address The MMIO register to write.
+;  @param  Value   The value to write to the MMIO register.
+;
+MmioWrite32Internal
+  dmb     st
+  str     r1, [r0]
+  bx      lr
+
+;
+;  Reads a 64-bit MMIO register.
+;
+;  Reads the 64-bit MMIO register specified by Address. The 64-bit read value is
+;  returned. This function must guarantee that all MMIO read and write
+;  operations are serialized.
+;
+;  If 64-bit MMIO register operations are not supported, then ASSERT().
+;
+;  @param  Address The MMIO register to read.
+;
+;  @return The value read.
+;
+MmioRead64Internal
+  ldrd    r0, r1, [r0]
+  dmb
+  bx      lr
+
+;
+;  Writes a 64-bit MMIO register.
+;
+;  Writes the 64-bit MMIO register specified by Address with the value specified
+;  by Value and returns Value. This function must guarantee that all MMIO read
+;  and write operations are serialized.
+;
+;  If 64-bit MMIO register operations are not supported, then ASSERT().
+;
+;  @param  Address The MMIO register to write.
+;  @param  Value   The value to write to the MMIO register.
+;
+MmioWrite64Internal
+  dmb     st
+  strd    r2, r3, [r0]
+  bx      lr
+
+  END
diff --git a/ArmVirtPkg/Library/ArmVirtIoLib/ArmVirtIoLib.c b/ArmVirtPkg/Library/ArmVirtIoLib/ArmVirtIoLib.c
new file mode 100644
index 000000000000..ffaa87d4024c
--- /dev/null
+++ b/ArmVirtPkg/Library/ArmVirtIoLib/ArmVirtIoLib.c
@@ -0,0 +1,589 @@
+/** @file
+  I/O Library for ARM.
+
+  Copyright (c) 2006 - 2009, Intel Corporation. All rights reserved.<BR>
+  Portions copyright (c) 2008 - 2009, Apple Inc. All rights reserved.<BR>
+  Copyright (c) 2017, AMD Incorporated. All rights reserved.<BR>
+  Copyright (c) 2018, Linaro, Ltd. All rights reserved.<BR>
+
+  This program and the accompanying materials are licensed and made available
+  under the terms and conditions of the BSD License which accompanies this
+  distribution. The full text of the license may be found at
+  http://opensource.org/licenses/bsd-license.php.
+
+  THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS,
+  WITHOUT WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED.
+
+**/
+
+
+//
+// Include common header file for this module.
+//
+#include "ArmVirtIoLib.h"
+
+/**
+  Reads an 8-bit I/O port.
+
+  Reads the 8-bit I/O port specified by Port. The 8-bit read value is returned.
+  This function must guarantee that all I/O read and write operations are
+  serialized.
+
+  If 8-bit I/O port operations are not supported, then ASSERT().
+
+  @param  Port  The I/O port to read.
+
+  @return The value read.
+
+**/
+UINT8
+EFIAPI
+IoRead8 (
+  IN      UINTN                     Port
+  )
+{
+  ASSERT (FALSE);
+  return 0;
+}
+
+/**
+  Writes an 8-bit I/O port.
+
+  Writes the 8-bit I/O port specified by Port with the value specified by Value
+  and returns Value. This function must guarantee that all I/O read and write
+  operations are serialized.
+
+  If 8-bit I/O port operations are not supported, then ASSERT().
+
+  @param  Port  The I/O port to write.
+  @param  Value The value to write to the I/O port.
+
+  @return The value written the I/O port.
+
+**/
+UINT8
+EFIAPI
+IoWrite8 (
+  IN      UINTN                     Port,
+  IN      UINT8                     Value
+  )
+{
+  ASSERT (FALSE);
+  return Value;
+}
+
+/**
+  Reads a 16-bit I/O port.
+
+  Reads the 16-bit I/O port specified by Port. The 16-bit read value is returned.
+  This function must guarantee that all I/O read and write operations are
+  serialized.
+
+  If 16-bit I/O port operations are not supported, then ASSERT().
+
+  @param  Port  The I/O port to read.
+
+  @return The value read.
+
+**/
+UINT16
+EFIAPI
+IoRead16 (
+  IN      UINTN                     Port
+  )
+{
+  ASSERT (FALSE);
+  return 0;
+}
+
+/**
+  Writes a 16-bit I/O port.
+
+  Writes the 16-bit I/O port specified by Port with the value specified by Value
+  and returns Value. This function must guarantee that all I/O read and write
+  operations are serialized.
+
+  If 16-bit I/O port operations are not supported, then ASSERT().
+
+  @param  Port  The I/O port to write.
+  @param  Value The value to write to the I/O port.
+
+  @return The value written the I/O port.
+
+**/
+UINT16
+EFIAPI
+IoWrite16 (
+  IN      UINTN                     Port,
+  IN      UINT16                    Value
+  )
+{
+  ASSERT (FALSE);
+  return Value;
+}
+
+/**
+  Reads a 32-bit I/O port.
+
+  Reads the 32-bit I/O port specified by Port. The 32-bit read value is returned.
+  This function must guarantee that all I/O read and write operations are
+  serialized.
+
+  If 32-bit I/O port operations are not supported, then ASSERT().
+
+  @param  Port  The I/O port to read.
+
+  @return The value read.
+
+**/
+UINT32
+EFIAPI
+IoRead32 (
+  IN      UINTN                     Port
+  )
+{
+  ASSERT (FALSE);
+  return 0;
+}
+
+/**
+  Writes a 32-bit I/O port.
+
+  Writes the 32-bit I/O port specified by Port with the value specified by Value
+  and returns Value. This function must guarantee that all I/O read and write
+  operations are serialized.
+
+  If 32-bit I/O port operations are not supported, then ASSERT().
+
+  @param  Port  The I/O port to write.
+  @param  Value The value to write to the I/O port.
+
+  @return The value written the I/O port.
+
+**/
+UINT32
+EFIAPI
+IoWrite32 (
+  IN      UINTN                     Port,
+  IN      UINT32                    Value
+  )
+{
+  ASSERT (FALSE);
+  return Value;
+}
+
+/**
+  Reads a 64-bit I/O port.
+
+  Reads the 64-bit I/O port specified by Port. The 64-bit read value is returned.
+  This function must guarantee that all I/O read and write operations are
+  serialized.
+
+  If 64-bit I/O port operations are not supported, then ASSERT().
+  If Port is not aligned on a 64-bit boundary, then ASSERT().
+
+  @param  Port  The I/O port to read.
+
+  @return The value read.
+
+**/
+UINT64
+EFIAPI
+IoRead64 (
+  IN      UINTN                     Port
+  )
+{
+  ASSERT (FALSE);
+  return 0;
+}
+
+/**
+  Writes a 64-bit I/O port.
+
+  Writes the 64-bit I/O port specified by Port with the value specified by Value
+  and returns Value. This function must guarantee that all I/O read and write
+  operations are serialized.
+
+  If 64-bit I/O port operations are not supported, then ASSERT().
+  If Port is not aligned on a 64-bit boundary, then ASSERT().
+
+  @param  Port  The I/O port to write.
+  @param  Value The value to write to the I/O port.
+
+  @return The value written to the I/O port.
+
+**/
+UINT64
+EFIAPI
+IoWrite64 (
+  IN      UINTN                     Port,
+  IN      UINT64                    Value
+  )
+{
+  ASSERT (FALSE);
+  return 0;
+}
+
+/**
+  Reads an 8-bit I/O port fifo into a block of memory.
+
+  Reads the 8-bit I/O fifo port specified by Port.
+  The port is read Count times, and the read data is
+  stored in the provided Buffer.
+
+  This function must guarantee that all I/O read and write operations are
+  serialized.
+
+  If 8-bit I/O port operations are not supported, then ASSERT().
+
+  @param  Port    The I/O port to read.
+  @param  Count   The number of times to read I/O port.
+  @param  Buffer  The buffer to store the read data into.
+
+**/
+VOID
+EFIAPI
+IoReadFifo8 (
+  IN      UINTN                     Port,
+  IN      UINTN                     Count,
+  OUT     VOID                      *Buffer
+  )
+{
+  ASSERT (FALSE);
+}
+
+/**
+  Writes a block of memory into an 8-bit I/O port fifo.
+
+  Writes the 8-bit I/O fifo port specified by Port.
+  The port is written Count times, and the write data is
+  retrieved from the provided Buffer.
+
+  This function must guarantee that all I/O write and write operations are
+  serialized.
+
+  If 8-bit I/O port operations are not supported, then ASSERT().
+
+  @param  Port    The I/O port to write.
+  @param  Count   The number of times to write I/O port.
+  @param  Buffer  The buffer to retrieve the write data from.
+
+**/
+VOID
+EFIAPI
+IoWriteFifo8 (
+  IN      UINTN                     Port,
+  IN      UINTN                     Count,
+  IN      VOID                      *Buffer
+  )
+{
+  ASSERT (FALSE);
+}
+
+/**
+  Reads a 16-bit I/O port fifo into a block of memory.
+
+  Reads the 16-bit I/O fifo port specified by Port.
+  The port is read Count times, and the read data is
+  stored in the provided Buffer.
+
+  This function must guarantee that all I/O read and write operations are
+  serialized.
+
+  If 16-bit I/O port operations are not supported, then ASSERT().
+
+  @param  Port    The I/O port to read.
+  @param  Count   The number of times to read I/O port.
+  @param  Buffer  The buffer to store the read data into.
+
+**/
+VOID
+EFIAPI
+IoReadFifo16 (
+  IN      UINTN                     Port,
+  IN      UINTN                     Count,
+  OUT     VOID                      *Buffer
+  )
+{
+  ASSERT (FALSE);
+}
+
+/**
+  Writes a block of memory into a 16-bit I/O port fifo.
+
+  Writes the 16-bit I/O fifo port specified by Port.
+  The port is written Count times, and the write data is
+  retrieved from the provided Buffer.
+
+  This function must guarantee that all I/O write and write operations are
+  serialized.
+
+  If 16-bit I/O port operations are not supported, then ASSERT().
+
+  @param  Port    The I/O port to write.
+  @param  Count   The number of times to write I/O port.
+  @param  Buffer  The buffer to retrieve the write data from.
+
+**/
+VOID
+EFIAPI
+IoWriteFifo16 (
+  IN      UINTN                     Port,
+  IN      UINTN                     Count,
+  IN      VOID                      *Buffer
+  )
+{
+  ASSERT (FALSE);
+}
+
+/**
+  Reads a 32-bit I/O port fifo into a block of memory.
+
+  Reads the 32-bit I/O fifo port specified by Port.
+  The port is read Count times, and the read data is
+  stored in the provided Buffer.
+
+  This function must guarantee that all I/O read and write operations are
+  serialized.
+
+  If 32-bit I/O port operations are not supported, then ASSERT().
+
+  @param  Port    The I/O port to read.
+  @param  Count   The number of times to read I/O port.
+  @param  Buffer  The buffer to store the read data into.
+
+**/
+VOID
+EFIAPI
+IoReadFifo32 (
+  IN      UINTN                     Port,
+  IN      UINTN                     Count,
+  OUT     VOID                      *Buffer
+  )
+{
+  ASSERT (FALSE);
+}
+
+/**
+  Writes a block of memory into a 32-bit I/O port fifo.
+
+  Writes the 32-bit I/O fifo port specified by Port.
+  The port is written Count times, and the write data is
+  retrieved from the provided Buffer.
+
+  This function must guarantee that all I/O write and write operations are
+  serialized.
+
+  If 32-bit I/O port operations are not supported, then ASSERT().
+
+  @param  Port    The I/O port to write.
+  @param  Count   The number of times to write I/O port.
+  @param  Buffer  The buffer to retrieve the write data from.
+
+**/
+VOID
+EFIAPI
+IoWriteFifo32 (
+  IN      UINTN                     Port,
+  IN      UINTN                     Count,
+  IN      VOID                      *Buffer
+  )
+{
+  ASSERT (FALSE);
+}
+
+/**
+  Reads an 8-bit MMIO register.
+
+  Reads the 8-bit MMIO register specified by Address. The 8-bit read value is
+  returned. This function must guarantee that all MMIO read and write
+  operations are serialized.
+
+  If 8-bit MMIO register operations are not supported, then ASSERT().
+
+  @param  Address The MMIO register to read.
+
+  @return The value read.
+
+**/
+UINT8
+EFIAPI
+MmioRead8 (
+  IN      UINTN                     Address
+  )
+{
+  return MmioRead8Internal (Address);
+}
+
+/**
+  Writes an 8-bit MMIO register.
+
+  Writes the 8-bit MMIO register specified by Address with the value specified
+  by Value and returns Value. This function must guarantee that all MMIO read
+  and write operations are serialized.
+
+  If 8-bit MMIO register operations are not supported, then ASSERT().
+
+  @param  Address The MMIO register to write.
+  @param  Value   The value to write to the MMIO register.
+
+**/
+UINT8
+EFIAPI
+MmioWrite8 (
+  IN      UINTN                     Address,
+  IN      UINT8                     Value
+  )
+{
+  MmioWrite8Internal (Address, Value);
+  return Value;
+}
+
+/**
+  Reads a 16-bit MMIO register.
+
+  Reads the 16-bit MMIO register specified by Address. The 16-bit read value is
+  returned. This function must guarantee that all MMIO read and write
+  operations are serialized.
+
+  If 16-bit MMIO register operations are not supported, then ASSERT().
+
+  @param  Address The MMIO register to read.
+
+  @return The value read.
+
+**/
+UINT16
+EFIAPI
+MmioRead16 (
+  IN      UINTN                     Address
+  )
+{
+  return MmioRead16Internal (Address);
+}
+
+/**
+  Writes a 16-bit MMIO register.
+
+  Writes the 16-bit MMIO register specified by Address with the value specified
+  by Value and returns Value. This function must guarantee that all MMIO read
+  and write operations are serialized.
+
+  If 16-bit MMIO register operations are not supported, then ASSERT().
+
+  @param  Address The MMIO register to write.
+  @param  Value   The value to write to the MMIO register.
+
+**/
+UINT16
+EFIAPI
+MmioWrite16 (
+  IN      UINTN                     Address,
+  IN      UINT16                    Value
+  )
+{
+  ASSERT ((Address & 1) == 0);
+
+  MmioWrite16Internal (Address, Value);
+  return Value;
+}
+
+/**
+  Reads a 32-bit MMIO register.
+
+  Reads the 32-bit MMIO register specified by Address. The 32-bit read value is
+  returned. This function must guarantee that all MMIO read and write
+  operations are serialized.
+
+  If 32-bit MMIO register operations are not supported, then ASSERT().
+
+  @param  Address The MMIO register to read.
+
+  @return The value read.
+
+**/
+UINT32
+EFIAPI
+MmioRead32 (
+  IN      UINTN                     Address
+  )
+{
+  return MmioRead32Internal (Address);
+}
+
+/**
+  Writes a 32-bit MMIO register.
+
+  Writes the 32-bit MMIO register specified by Address with the value specified
+  by Value and returns Value. This function must guarantee that all MMIO read
+  and write operations are serialized.
+
+  If 32-bit MMIO register operations are not supported, then ASSERT().
+
+  @param  Address The MMIO register to write.
+  @param  Value   The value to write to the MMIO register.
+
+**/
+UINT32
+EFIAPI
+MmioWrite32 (
+  IN      UINTN                     Address,
+  IN      UINT32                    Value
+  )
+{
+  ASSERT ((Address & 3) == 0);
+
+  MmioWrite32Internal (Address, Value);
+  return Value;
+}
+
+/**
+  Reads a 64-bit MMIO register.
+
+  Reads the 64-bit MMIO register specified by Address. The 64-bit read value is
+  returned. This function must guarantee that all MMIO read and write
+  operations are serialized.
+
+  If 64-bit MMIO register operations are not supported, then ASSERT().
+
+  @param  Address The MMIO register to read.
+
+  @return The value read.
+
+**/
+UINT64
+EFIAPI
+MmioRead64 (
+  IN      UINTN                     Address
+  )
+{
+  ASSERT ((Address & 7) == 0);
+
+  return MmioRead64Internal (Address);
+}
+
+/**
+  Writes a 64-bit MMIO register.
+
+  Writes the 64-bit MMIO register specified by Address with the value specified
+  by Value and returns Value. This function must guarantee that all MMIO read
+  and write operations are serialized.
+
+  If 64-bit MMIO register operations are not supported, then ASSERT().
+
+  @param  Address The MMIO register to write.
+  @param  Value   The value to write to the MMIO register.
+
+**/
+UINT64
+EFIAPI
+MmioWrite64 (
+  IN      UINTN                     Address,
+  IN      UINT64                    Value
+  )
+{
+  ASSERT ((Address & 7) == 0);
+
+  MmioWrite64Internal (Address, Value);
+  return Value;
+}
diff --git a/ArmVirtPkg/Library/ArmVirtIoLib/ArmVirtIoLib.h b/ArmVirtPkg/Library/ArmVirtIoLib/ArmVirtIoLib.h
new file mode 100644
index 000000000000..98f4a7b76925
--- /dev/null
+++ b/ArmVirtPkg/Library/ArmVirtIoLib/ArmVirtIoLib.h
@@ -0,0 +1,188 @@
+/** @file
+  Common header file shared by all source files.
+
+  This file includes package header files, dependent library classes.
+
+  Copyright (c) 2007 - 2008, Intel Corporation. All rights reserved.<BR>
+  Copyright (c) 2018, Linaro, Ltd. All rights reserved.<BR>
+
+  This program and the accompanying materials are licensed and made available
+  under the terms and conditions of the BSD License which accompanies this
+  distribution. The full text of the license may be found at
+  http://opensource.org/licenses/bsd-license.php.
+
+  THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS,
+  WITHOUT WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED.
+
+**/
+
+#ifndef __ARMVIRTIOLIB_INTERNAL_H_
+#define __ARMVIRTIOLIB_INTERNAL_H_
+
+#include <Base.h>
+
+#include <Library/IoLib.h>
+#include <Library/BaseLib.h>
+#include <Library/DebugLib.h>
+
+/**
+  Reads an 8-bit MMIO register.
+
+  Reads the 8-bit MMIO register specified by Address. The 8-bit read value is
+  returned. This function must guarantee that all MMIO read and write
+  operations are serialized.
+
+  If 8-bit MMIO register operations are not supported, then ASSERT().
+
+  @param  Address The MMIO register to read.
+
+  @return The value read.
+
+**/
+UINT8
+EFIAPI
+MmioRead8Internal (
+  IN      UINTN                     Address
+  );
+
+/**
+  Writes an 8-bit MMIO register.
+
+  Writes the 8-bit MMIO register specified by Address with the value specified
+  by Value and returns Value. This function must guarantee that all MMIO read
+  and write operations are serialized.
+
+  If 8-bit MMIO register operations are not supported, then ASSERT().
+
+  @param  Address The MMIO register to write.
+  @param  Value   The value to write to the MMIO register.
+
+**/
+VOID
+EFIAPI
+MmioWrite8Internal (
+  IN      UINTN                     Address,
+  IN      UINT8                     Value
+  );
+
+/**
+  Reads a 16-bit MMIO register.
+
+  Reads the 16-bit MMIO register specified by Address. The 16-bit read value is
+  returned. This function must guarantee that all MMIO read and write
+  operations are serialized.
+
+  If 16-bit MMIO register operations are not supported, then ASSERT().
+
+  @param  Address The MMIO register to read.
+
+  @return The value read.
+
+**/
+UINT16
+EFIAPI
+MmioRead16Internal (
+  IN      UINTN                     Address
+  );
+
+/**
+  Writes a 16-bit MMIO register.
+
+  Writes the 16-bit MMIO register specified by Address with the value specified
+  by Value and returns Value. This function must guarantee that all MMIO read
+  and write operations are serialized.
+
+  If 16-bit MMIO register operations are not supported, then ASSERT().
+
+  @param  Address The MMIO register to write.
+  @param  Value   The value to write to the MMIO register.
+
+**/
+VOID
+EFIAPI
+MmioWrite16Internal (
+  IN      UINTN                     Address,
+  IN      UINT16                    Value
+  );
+
+/**
+  Reads a 32-bit MMIO register.
+
+  Reads the 32-bit MMIO register specified by Address. The 32-bit read value is
+  returned. This function must guarantee that all MMIO read and write
+  operations are serialized.
+
+  If 32-bit MMIO register operations are not supported, then ASSERT().
+
+  @param  Address The MMIO register to read.
+
+  @return The value read.
+
+**/
+UINT32
+EFIAPI
+MmioRead32Internal (
+  IN      UINTN                     Address
+  );
+
+/**
+  Writes a 32-bit MMIO register.
+
+  Writes the 32-bit MMIO register specified by Address with the value specified
+  by Value and returns Value. This function must guarantee that all MMIO read
+  and write operations are serialized.
+
+  If 32-bit MMIO register operations are not supported, then ASSERT().
+
+  @param  Address The MMIO register to write.
+  @param  Value   The value to write to the MMIO register.
+
+**/
+VOID
+EFIAPI
+MmioWrite32Internal (
+  IN      UINTN                     Address,
+  IN      UINT32                    Value
+  );
+
+/**
+  Reads a 64-bit MMIO register.
+
+  Reads the 64-bit MMIO register specified by Address. The 64-bit read value is
+  returned. This function must guarantee that all MMIO read and write
+  operations are serialized.
+
+  If 64-bit MMIO register operations are not supported, then ASSERT().
+
+  @param  Address The MMIO register to read.
+
+  @return The value read.
+
+**/
+UINT64
+EFIAPI
+MmioRead64Internal (
+  IN      UINTN                     Address
+  );
+
+/**
+  Writes a 64-bit MMIO register.
+
+  Writes the 64-bit MMIO register specified by Address with the value specified
+  by Value and returns Value. This function must guarantee that all MMIO read
+  and write operations are serialized.
+
+  If 64-bit MMIO register operations are not supported, then ASSERT().
+
+  @param  Address The MMIO register to write.
+  @param  Value   The value to write to the MMIO register.
+
+**/
+VOID
+EFIAPI
+MmioWrite64Internal (
+  IN      UINTN                     Address,
+  IN      UINT64                    Value
+  );
+
+#endif
diff --git a/ArmVirtPkg/Library/ArmVirtIoLib/ArmVirtIoLib.inf b/ArmVirtPkg/Library/ArmVirtIoLib/ArmVirtIoLib.inf
new file mode 100644
index 000000000000..b2eab7f8cbbf
--- /dev/null
+++ b/ArmVirtPkg/Library/ArmVirtIoLib/ArmVirtIoLib.inf
@@ -0,0 +1,49 @@
+## @file
+#  Instance of I/O Library using assembler, avoiding link time optimizations
+#  of these routines resulting in instructions that are unsupported by KVM
+#  when used to perform MMIO on regions that are backed by emulation.
+#
+#  Copyright (c) 2018, Linaro, Ltd. All rights reserved.<BR>
+#
+#  This program and the accompanying materials are licensed and made available
+#  under the terms and conditions of the BSD License which accompanies this
+#  distribution. The full text of the license may be found at
+#  http://opensource.org/licenses/bsd-license.php.
+#
+#  THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED.
+#
+##
+
+[Defines]
+  INF_VERSION                    = 0x0001001A
+  BASE_NAME                      = ArmVirtIoLib
+  FILE_GUID                      = e9723e7d-5df1-4165-9518-597faba42692
+  MODULE_TYPE                    = BASE
+  VERSION_STRING                 = 1.0
+  LIBRARY_CLASS                  = IoLib 
+
+#
+#  VALID_ARCHITECTURES           = ARM AARCH64
+#
+
+[Sources]
+  ArmVirtIoLib.c
+  ArmVirtIoLib.h
+  IoLibMmioBuffer.c
+  IoHighLevel.c
+
+[Sources.ARM]
+  Arm/ArmVirtMmio.S   | GCC
+  Arm/ArmVirtMmio.asm | RVCT
+
+[Sources.AARCH64]
+  AArch64/ArmVirtMmio.S
+
+[Packages]
+  ArmPkg/ArmPkg.dec
+  MdePkg/MdePkg.dec
+
+[LibraryClasses]
+  DebugLib
+  BaseLib
diff --git a/ArmVirtPkg/Library/ArmVirtIoLib/IoHighLevel.c b/ArmVirtPkg/Library/ArmVirtIoLib/IoHighLevel.c
new file mode 100644
index 000000000000..48c17fdae10f
--- /dev/null
+++ b/ArmVirtPkg/Library/ArmVirtIoLib/IoHighLevel.c
@@ -0,0 +1,2358 @@
+/** @file
+  High-level Io/Mmio functions.
+
+  All assertions for bit field operations are handled bit field functions in the
+  Base Library.
+
+  Copyright (c) 2006 - 2012, Intel Corporation. All rights reserved.<BR>
+  Copyright (c) 2018, Linaro, Ltd. All rights reserved.<BR>
+
+  This program and the accompanying materials are licensed and made available
+  under the terms and conditions of the BSD License which accompanies this
+  distribution. The full text of the license may be found at
+  http://opensource.org/licenses/bsd-license.php.
+
+  THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS,
+  WITHOUT WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED.
+
+  The following MdePkg IoLib instances contain the same copy of this file:
+
+    BaseIoLibIntrinsic
+    DxeIoLibCpuIo
+    PeiIoLibCpuIo
+
+**/
+
+#include "ArmVirtIoLib.h"
+
+/**
+  Reads an 8-bit I/O port, performs a bitwise OR, and writes the
+  result back to the 8-bit I/O port.
+
+  Reads the 8-bit I/O port specified by Port, performs a bitwise OR
+  between the read result and the value specified by OrData, and writes the
+  result to the 8-bit I/O port specified by Port. The value written to the I/O
+  port is returned. This function must guarantee that all I/O read and write
+  operations are serialized.
+
+  If 8-bit I/O port operations are not supported, then ASSERT().
+
+  @param  Port    The I/O port to write.
+  @param  OrData  The value to OR with the read value from the I/O port.
+
+  @return The value written back to the I/O port.
+
+**/
+UINT8
+EFIAPI
+IoOr8 (
+  IN      UINTN                     Port,
+  IN      UINT8                     OrData
+  )
+{
+  return IoWrite8 (Port, (UINT8) (IoRead8 (Port) | OrData));
+}
+
+/**
+  Reads an 8-bit I/O port, performs a bitwise AND, and writes the result back
+  to the 8-bit I/O port.
+
+  Reads the 8-bit I/O port specified by Port, performs a bitwise AND between
+  the read result and the value specified by AndData, and writes the result to
+  the 8-bit I/O port specified by Port. The value written to the I/O port is
+  returned. This function must guarantee that all I/O read and write operations
+  are serialized.
+
+  If 8-bit I/O port operations are not supported, then ASSERT().
+
+  @param  Port    The I/O port to write.
+  @param  AndData The value to AND with the read value from the I/O port.
+
+  @return The value written back to the I/O port.
+
+**/
+UINT8
+EFIAPI
+IoAnd8 (
+  IN      UINTN                     Port,
+  IN      UINT8                     AndData
+  )
+{
+  return IoWrite8 (Port, (UINT8) (IoRead8 (Port) & AndData));
+}
+
+/**
+  Reads an 8-bit I/O port, performs a bitwise AND followed by a bitwise
+  OR, and writes the result back to the 8-bit I/O port.
+
+  Reads the 8-bit I/O port specified by Port, performs a bitwise AND between
+  the read result and the value specified by AndData, performs a bitwise OR
+  between the result of the AND operation and the value specified by OrData,
+  and writes the result to the 8-bit I/O port specified by Port. The value
+  written to the I/O port is returned. This function must guarantee that all
+  I/O read and write operations are serialized.
+
+  If 8-bit I/O port operations are not supported, then ASSERT().
+
+  @param  Port    The I/O port to write.
+  @param  AndData The value to AND with the read value from the I/O port.
+  @param  OrData  The value to OR with the result of the AND operation.
+
+  @return The value written back to the I/O port.
+
+**/
+UINT8
+EFIAPI
+IoAndThenOr8 (
+  IN      UINTN                     Port,
+  IN      UINT8                     AndData,
+  IN      UINT8                     OrData
+  )
+{
+  return IoWrite8 (Port, (UINT8) ((IoRead8 (Port) & AndData) | OrData));
+}
+
+/**
+  Reads a bit field of an I/O register.
+
+  Reads the bit field in an 8-bit I/O register. The bit field is specified by
+  the StartBit and the EndBit. The value of the bit field is returned.
+
+  If 8-bit I/O port operations are not supported, then ASSERT().
+  If StartBit is greater than 7, then ASSERT().
+  If EndBit is greater than 7, then ASSERT().
+  If EndBit is less than StartBit, then ASSERT().
+
+  @param  Port      The I/O port to read.
+  @param  StartBit  The ordinal of the least significant bit in the bit field.
+                    Range 0..7.
+  @param  EndBit    The ordinal of the most significant bit in the bit field.
+                    Range 0..7.
+
+  @return The value read.
+
+**/
+UINT8
+EFIAPI
+IoBitFieldRead8 (
+  IN      UINTN                     Port,
+  IN      UINTN                     StartBit,
+  IN      UINTN                     EndBit
+  )
+{
+  return BitFieldRead8 (IoRead8 (Port), StartBit, EndBit);
+}
+
+/**
+  Writes a bit field to an I/O register.
+
+  Writes Value to the bit field of the I/O register. The bit field is specified
+  by the StartBit and the EndBit. All other bits in the destination I/O
+  register are preserved. The value written to the I/O port is returned.
+
+  If 8-bit I/O port operations are not supported, then ASSERT().
+  If StartBit is greater than 7, then ASSERT().
+  If EndBit is greater than 7, then ASSERT().
+  If EndBit is less than StartBit, then ASSERT().
+  If Value is larger than the bitmask value range specified by StartBit and EndBit, then ASSERT().
+
+  @param  Port      The I/O port to write.
+  @param  StartBit  The ordinal of the least significant bit in the bit field.
+                    Range 0..7.
+  @param  EndBit    The ordinal of the most significant bit in the bit field.
+                    Range 0..7.
+  @param  Value     The new value of the bit field.
+
+  @return The value written back to the I/O port.
+
+**/
+UINT8
+EFIAPI
+IoBitFieldWrite8 (
+  IN      UINTN                     Port,
+  IN      UINTN                     StartBit,
+  IN      UINTN                     EndBit,
+  IN      UINT8                     Value
+  )
+{
+  return IoWrite8 (
+           Port,
+           BitFieldWrite8 (IoRead8 (Port), StartBit, EndBit, Value)
+           );
+}
+
+/**
+  Reads a bit field in an 8-bit port, performs a bitwise OR, and writes the
+  result back to the bit field in the 8-bit port.
+
+  Reads the 8-bit I/O port specified by Port, performs a bitwise OR
+  between the read result and the value specified by OrData, and writes the
+  result to the 8-bit I/O port specified by Port. The value written to the I/O
+  port is returned. This function must guarantee that all I/O read and write
+  operations are serialized. Extra bits left in OrData are stripped.
+
+  If 8-bit I/O port operations are not supported, then ASSERT().
+  If StartBit is greater than 7, then ASSERT().
+  If EndBit is greater than 7, then ASSERT().
+  If EndBit is less than StartBit, then ASSERT().
+  If OrData is larger than the bitmask value range specified by StartBit and EndBit, then ASSERT().
+
+  @param  Port      The I/O port to write.
+  @param  StartBit  The ordinal of the least significant bit in the bit field.
+                    Range 0..7.
+  @param  EndBit    The ordinal of the most significant bit in the bit field.
+                    Range 0..7.
+  @param  OrData    The value to OR with the read value from the I/O port.
+
+  @return The value written back to the I/O port.
+
+**/
+UINT8
+EFIAPI
+IoBitFieldOr8 (
+  IN      UINTN                     Port,
+  IN      UINTN                     StartBit,
+  IN      UINTN                     EndBit,
+  IN      UINT8                     OrData
+  )
+{
+  return IoWrite8 (
+           Port,
+           BitFieldOr8 (IoRead8 (Port), StartBit, EndBit, OrData)
+           );
+}
+
+/**
+  Reads a bit field in an 8-bit port, performs a bitwise AND, and writes the
+  result back to the bit field in the 8-bit port.
+
+  Reads the 8-bit I/O port specified by Port, performs a bitwise AND between
+  the read result and the value specified by AndData, and writes the result to
+  the 8-bit I/O port specified by Port. The value written to the I/O port is
+  returned. This function must guarantee that all I/O read and write operations
+  are serialized. Extra bits left in AndData are stripped.
+
+  If 8-bit I/O port operations are not supported, then ASSERT().
+  If StartBit is greater than 7, then ASSERT().
+  If EndBit is greater than 7, then ASSERT().
+  If EndBit is less than StartBit, then ASSERT().
+  If AndData is larger than the bitmask value range specified by StartBit and EndBit, then ASSERT().
+
+  @param  Port      The I/O port to write.
+  @param  StartBit  The ordinal of the least significant bit in the bit field.
+                    Range 0..7.
+  @param  EndBit    The ordinal of the most significant bit in the bit field.
+                    Range 0..7.
+  @param  AndData   The value to AND with the read value from the I/O port.
+
+  @return The value written back to the I/O port.
+
+**/
+UINT8
+EFIAPI
+IoBitFieldAnd8 (
+  IN      UINTN                     Port,
+  IN      UINTN                     StartBit,
+  IN      UINTN                     EndBit,
+  IN      UINT8                     AndData
+  )
+{
+  return IoWrite8 (
+           Port,
+           BitFieldAnd8 (IoRead8 (Port), StartBit, EndBit, AndData)
+           );
+}
+
+/**
+  Reads a bit field in an 8-bit port, performs a bitwise AND followed by a
+  bitwise OR, and writes the result back to the bit field in the
+  8-bit port.
+
+  Reads the 8-bit I/O port specified by Port, performs a bitwise AND followed
+  by a bitwise OR between the read result and the value specified by
+  AndData, and writes the result to the 8-bit I/O port specified by Port. The
+  value written to the I/O port is returned. This function must guarantee that
+  all I/O read and write operations are serialized. Extra bits left in both
+  AndData and OrData are stripped.
+
+  If 8-bit I/O port operations are not supported, then ASSERT().
+  If StartBit is greater than 7, then ASSERT().
+  If EndBit is greater than 7, then ASSERT().
+  If EndBit is less than StartBit, then ASSERT().
+  If AndData is larger than the bitmask value range specified by StartBit and EndBit, then ASSERT().
+  If OrData is larger than the bitmask value range specified by StartBit and EndBit, then ASSERT().
+
+  @param  Port      The I/O port to write.
+  @param  StartBit  The ordinal of the least significant bit in the bit field.
+                    Range 0..7.
+  @param  EndBit    The ordinal of the most significant bit in the bit field.
+                    Range 0..7.
+  @param  AndData   The value to AND with the read value from the I/O port.
+  @param  OrData    The value to OR with the result of the AND operation.
+
+  @return The value written back to the I/O port.
+
+**/
+UINT8
+EFIAPI
+IoBitFieldAndThenOr8 (
+  IN      UINTN                     Port,
+  IN      UINTN                     StartBit,
+  IN      UINTN                     EndBit,
+  IN      UINT8                     AndData,
+  IN      UINT8                     OrData
+  )
+{
+  return IoWrite8 (
+           Port,
+           BitFieldAndThenOr8 (IoRead8 (Port), StartBit, EndBit, AndData, OrData)
+           );
+}
+
+/**
+  Reads a 16-bit I/O port, performs a bitwise OR, and writes the
+  result back to the 16-bit I/O port.
+
+  Reads the 16-bit I/O port specified by Port, performs a bitwise OR
+  between the read result and the value specified by OrData, and writes the
+  result to the 16-bit I/O port specified by Port. The value written to the I/O
+  port is returned. This function must guarantee that all I/O read and write
+  operations are serialized.
+
+  If 16-bit I/O port operations are not supported, then ASSERT().
+  If Port is not aligned on a 16-bit boundary, then ASSERT().
+
+  @param  Port    The I/O port to write.
+  @param  OrData  The value to OR with the read value from the I/O port.
+
+  @return The value written back to the I/O port.
+
+**/
+UINT16
+EFIAPI
+IoOr16 (
+  IN      UINTN                     Port,
+  IN      UINT16                    OrData
+  )
+{
+  return IoWrite16 (Port, (UINT16) (IoRead16 (Port) | OrData));
+}
+
+/**
+  Reads a 16-bit I/O port, performs a bitwise AND, and writes the result back
+  to the 16-bit I/O port.
+
+  Reads the 16-bit I/O port specified by Port, performs a bitwise AND between
+  the read result and the value specified by AndData, and writes the result to
+  the 16-bit I/O port specified by Port. The value written to the I/O port is
+  returned. This function must guarantee that all I/O read and write operations
+  are serialized.
+
+  If 16-bit I/O port operations are not supported, then ASSERT().
+  If Port is not aligned on a 16-bit boundary, then ASSERT().
+
+  @param  Port    The I/O port to write.
+  @param  AndData The value to AND with the read value from the I/O port.
+
+  @return The value written back to the I/O port.
+
+**/
+UINT16
+EFIAPI
+IoAnd16 (
+  IN      UINTN                     Port,
+  IN      UINT16                    AndData
+  )
+{
+  return IoWrite16 (Port, (UINT16) (IoRead16 (Port) & AndData));
+}
+
+/**
+  Reads a 16-bit I/O port, performs a bitwise AND followed by a bitwise
+  OR, and writes the result back to the 16-bit I/O port.
+
+  Reads the 16-bit I/O port specified by Port, performs a bitwise AND between
+  the read result and the value specified by AndData, performs a bitwise OR
+  between the result of the AND operation and the value specified by OrData,
+  and writes the result to the 16-bit I/O port specified by Port. The value
+  written to the I/O port is returned. This function must guarantee that all
+  I/O read and write operations are serialized.
+
+  If 16-bit I/O port operations are not supported, then ASSERT().
+  If Port is not aligned on a 16-bit boundary, then ASSERT().
+
+  @param  Port    The I/O port to write.
+  @param  AndData The value to AND with the read value from the I/O port.
+  @param  OrData  The value to OR with the result of the AND operation.
+
+  @return The value written back to the I/O port.
+
+**/
+UINT16
+EFIAPI
+IoAndThenOr16 (
+  IN      UINTN                     Port,
+  IN      UINT16                    AndData,
+  IN      UINT16                    OrData
+  )
+{
+  return IoWrite16 (Port, (UINT16) ((IoRead16 (Port) & AndData) | OrData));
+}
+
+/**
+  Reads a bit field of an I/O register.
+
+  Reads the bit field in a 16-bit I/O register. The bit field is specified by
+  the StartBit and the EndBit. The value of the bit field is returned.
+
+  If 16-bit I/O port operations are not supported, then ASSERT().
+  If Port is not aligned on a 16-bit boundary, then ASSERT().
+  If StartBit is greater than 15, then ASSERT().
+  If EndBit is greater than 15, then ASSERT().
+  If EndBit is less than StartBit, then ASSERT().
+
+  @param  Port      The I/O port to read.
+  @param  StartBit  The ordinal of the least significant bit in the bit field.
+                    Range 0..15.
+  @param  EndBit    The ordinal of the most significant bit in the bit field.
+                    Range 0..15.
+
+  @return The value read.
+
+**/
+UINT16
+EFIAPI
+IoBitFieldRead16 (
+  IN      UINTN                     Port,
+  IN      UINTN                     StartBit,
+  IN      UINTN                     EndBit
+  )
+{
+  return BitFieldRead16 (IoRead16 (Port), StartBit, EndBit);
+}
+
+/**
+  Writes a bit field to an I/O register.
+
+  Writes Value to the bit field of the I/O register. The bit field is specified
+  by the StartBit and the EndBit. All other bits in the destination I/O
+  register are preserved. The value written to the I/O port is returned. Extra
+  bits left in Value are stripped.
+
+  If 16-bit I/O port operations are not supported, then ASSERT().
+  If Port is not aligned on a 16-bit boundary, then ASSERT().
+  If StartBit is greater than 15, then ASSERT().
+  If EndBit is greater than 15, then ASSERT().
+  If EndBit is less than StartBit, then ASSERT().
+  If Value is larger than the bitmask value range specified by StartBit and EndBit, then ASSERT().
+
+  @param  Port      The I/O port to write.
+  @param  StartBit  The ordinal of the least significant bit in the bit field.
+                    Range 0..15.
+  @param  EndBit    The ordinal of the most significant bit in the bit field.
+                    Range 0..15.
+  @param  Value     The new value of the bit field.
+
+  @return The value written back to the I/O port.
+
+**/
+UINT16
+EFIAPI
+IoBitFieldWrite16 (
+  IN      UINTN                     Port,
+  IN      UINTN                     StartBit,
+  IN      UINTN                     EndBit,
+  IN      UINT16                    Value
+  )
+{
+  return IoWrite16 (
+           Port,
+           BitFieldWrite16 (IoRead16 (Port), StartBit, EndBit, Value)
+           );
+}
+
+/**
+  Reads a bit field in a 16-bit port, performs a bitwise OR, and writes the
+  result back to the bit field in the 16-bit port.
+
+  Reads the 16-bit I/O port specified by Port, performs a bitwise OR
+  between the read result and the value specified by OrData, and writes the
+  result to the 16-bit I/O port specified by Port. The value written to the I/O
+  port is returned. This function must guarantee that all I/O read and write
+  operations are serialized. Extra bits left in OrData are stripped.
+
+  If 16-bit I/O port operations are not supported, then ASSERT().
+  If Port is not aligned on a 16-bit boundary, then ASSERT().
+  If StartBit is greater than 15, then ASSERT().
+  If EndBit is greater than 15, then ASSERT().
+  If EndBit is less than StartBit, then ASSERT().
+  If OrData is larger than the bitmask value range specified by StartBit and EndBit, then ASSERT().
+
+  @param  Port      The I/O port to write.
+  @param  StartBit  The ordinal of the least significant bit in the bit field.
+                    Range 0..15.
+  @param  EndBit    The ordinal of the most significant bit in the bit field.
+                    Range 0..15.
+  @param  OrData    The value to OR with the read value from the I/O port.
+
+  @return The value written back to the I/O port.
+
+**/
+UINT16
+EFIAPI
+IoBitFieldOr16 (
+  IN      UINTN                     Port,
+  IN      UINTN                     StartBit,
+  IN      UINTN                     EndBit,
+  IN      UINT16                    OrData
+  )
+{
+  return IoWrite16 (
+           Port,
+           BitFieldOr16 (IoRead16 (Port), StartBit, EndBit, OrData)
+           );
+}
+
+/**
+  Reads a bit field in a 16-bit port, performs a bitwise AND, and writes the
+  result back to the bit field in the 16-bit port.
+
+  Reads the 16-bit I/O port specified by Port, performs a bitwise AND between
+  the read result and the value specified by AndData, and writes the result to
+  the 16-bit I/O port specified by Port. The value written to the I/O port is
+  returned. This function must guarantee that all I/O read and write operations
+  are serialized. Extra bits left in AndData are stripped.
+
+  If 16-bit I/O port operations are not supported, then ASSERT().
+  If Port is not aligned on a 16-bit boundary, then ASSERT().
+  If StartBit is greater than 15, then ASSERT().
+  If EndBit is greater than 15, then ASSERT().
+  If EndBit is less than StartBit, then ASSERT().
+  If AndData is larger than the bitmask value range specified by StartBit and EndBit, then ASSERT().
+
+  @param  Port      The I/O port to write.
+  @param  StartBit  The ordinal of the least significant bit in the bit field.
+                    Range 0..15.
+  @param  EndBit    The ordinal of the most significant bit in the bit field.
+                    Range 0..15.
+  @param  AndData   The value to AND with the read value from the I/O port.
+
+  @return The value written back to the I/O port.
+
+**/
+UINT16
+EFIAPI
+IoBitFieldAnd16 (
+  IN      UINTN                     Port,
+  IN      UINTN                     StartBit,
+  IN      UINTN                     EndBit,
+  IN      UINT16                    AndData
+  )
+{
+  return IoWrite16 (
+           Port,
+           BitFieldAnd16 (IoRead16 (Port), StartBit, EndBit, AndData)
+           );
+}
+
+/**
+  Reads a bit field in a 16-bit port, performs a bitwise AND followed by a
+  bitwise OR, and writes the result back to the bit field in the
+  16-bit port.
+
+  Reads the 16-bit I/O port specified by Port, performs a bitwise AND followed
+  by a bitwise OR between the read result and the value specified by
+  AndData, and writes the result to the 16-bit I/O port specified by Port. The
+  value written to the I/O port is returned. This function must guarantee that
+  all I/O read and write operations are serialized. Extra bits left in both
+  AndData and OrData are stripped.
+
+  If 16-bit I/O port operations are not supported, then ASSERT().
+  If Port is not aligned on a 16-bit boundary, then ASSERT().
+  If StartBit is greater than 15, then ASSERT().
+  If EndBit is greater than 15, then ASSERT().
+  If EndBit is less than StartBit, then ASSERT().
+  If AndData is larger than the bitmask value range specified by StartBit and EndBit, then ASSERT().
+  If OrData is larger than the bitmask value range specified by StartBit and EndBit, then ASSERT().
+
+  @param  Port      The I/O port to write.
+  @param  StartBit  The ordinal of the least significant bit in the bit field.
+                    Range 0..15.
+  @param  EndBit    The ordinal of the most significant bit in the bit field.
+                    Range 0..15.
+  @param  AndData   The value to AND with the read value from the I/O port.
+  @param  OrData    The value to OR with the result of the AND operation.
+
+  @return The value written back to the I/O port.
+
+**/
+UINT16
+EFIAPI
+IoBitFieldAndThenOr16 (
+  IN      UINTN                     Port,
+  IN      UINTN                     StartBit,
+  IN      UINTN                     EndBit,
+  IN      UINT16                    AndData,
+  IN      UINT16                    OrData
+  )
+{
+  return IoWrite16 (
+           Port,
+           BitFieldAndThenOr16 (IoRead16 (Port), StartBit, EndBit, AndData, OrData)
+           );
+}
+
+/**
+  Reads a 32-bit I/O port, performs a bitwise OR, and writes the
+  result back to the 32-bit I/O port.
+
+  Reads the 32-bit I/O port specified by Port, performs a bitwise OR
+  between the read result and the value specified by OrData, and writes the
+  result to the 32-bit I/O port specified by Port. The value written to the I/O
+  port is returned. This function must guarantee that all I/O read and write
+  operations are serialized.
+
+  If 32-bit I/O port operations are not supported, then ASSERT().
+  If Port is not aligned on a 32-bit boundary, then ASSERT().
+
+  @param  Port    The I/O port to write.
+  @param  OrData  The value to OR with the read value from the I/O port.
+
+  @return The value written back to the I/O port.
+
+**/
+UINT32
+EFIAPI
+IoOr32 (
+  IN      UINTN                     Port,
+  IN      UINT32                    OrData
+  )
+{
+  return IoWrite32 (Port, IoRead32 (Port) | OrData);
+}
+
+/**
+  Reads a 32-bit I/O port, performs a bitwise AND, and writes the result back
+  to the 32-bit I/O port.
+
+  Reads the 32-bit I/O port specified by Port, performs a bitwise AND between
+  the read result and the value specified by AndData, and writes the result to
+  the 32-bit I/O port specified by Port. The value written to the I/O port is
+  returned. This function must guarantee that all I/O read and write operations
+  are serialized.
+
+  If 32-bit I/O port operations are not supported, then ASSERT().
+  If Port is not aligned on a 32-bit boundary, then ASSERT().
+
+  @param  Port    The I/O port to write.
+  @param  AndData The value to AND with the read value from the I/O port.
+
+  @return The value written back to the I/O port.
+
+**/
+UINT32
+EFIAPI
+IoAnd32 (
+  IN      UINTN                     Port,
+  IN      UINT32                    AndData
+  )
+{
+  return IoWrite32 (Port, IoRead32 (Port) & AndData);
+}
+
+/**
+  Reads a 32-bit I/O port, performs a bitwise AND followed by a bitwise
+  OR, and writes the result back to the 32-bit I/O port.
+
+  Reads the 32-bit I/O port specified by Port, performs a bitwise AND between
+  the read result and the value specified by AndData, performs a bitwise OR
+  between the result of the AND operation and the value specified by OrData,
+  and writes the result to the 32-bit I/O port specified by Port. The value
+  written to the I/O port is returned. This function must guarantee that all
+  I/O read and write operations are serialized.
+
+  If 32-bit I/O port operations are not supported, then ASSERT().
+  If Port is not aligned on a 32-bit boundary, then ASSERT().
+
+  @param  Port    The I/O port to write.
+  @param  AndData The value to AND with the read value from the I/O port.
+  @param  OrData  The value to OR with the result of the AND operation.
+
+  @return The value written back to the I/O port.
+
+**/
+UINT32
+EFIAPI
+IoAndThenOr32 (
+  IN      UINTN                     Port,
+  IN      UINT32                    AndData,
+  IN      UINT32                    OrData
+  )
+{
+  return IoWrite32 (Port, (IoRead32 (Port) & AndData) | OrData);
+}
+
+/**
+  Reads a bit field of an I/O register.
+
+  Reads the bit field in a 32-bit I/O register. The bit field is specified by
+  the StartBit and the EndBit. The value of the bit field is returned.
+
+  If 32-bit I/O port operations are not supported, then ASSERT().
+  If Port is not aligned on a 32-bit boundary, then ASSERT().
+  If StartBit is greater than 31, then ASSERT().
+  If EndBit is greater than 31, then ASSERT().
+  If EndBit is less than StartBit, then ASSERT().
+
+  @param  Port      The I/O port to read.
+  @param  StartBit  The ordinal of the least significant bit in the bit field.
+                    Range 0..31.
+  @param  EndBit    The ordinal of the most significant bit in the bit field.
+                    Range 0..31.
+
+  @return The value read.
+
+**/
+UINT32
+EFIAPI
+IoBitFieldRead32 (
+  IN      UINTN                     Port,
+  IN      UINTN                     StartBit,
+  IN      UINTN                     EndBit
+  )
+{
+  return BitFieldRead32 (IoRead32 (Port), StartBit, EndBit);
+}
+
+/**
+  Writes a bit field to an I/O register.
+
+  Writes Value to the bit field of the I/O register. The bit field is specified
+  by the StartBit and the EndBit. All other bits in the destination I/O
+  register are preserved. The value written to the I/O port is returned. Extra
+  bits left in Value are stripped.
+
+  If 32-bit I/O port operations are not supported, then ASSERT().
+  If Port is not aligned on a 32-bit boundary, then ASSERT().
+  If StartBit is greater than 31, then ASSERT().
+  If EndBit is greater than 31, then ASSERT().
+  If EndBit is less than StartBit, then ASSERT().
+  If Value is larger than the bitmask value range specified by StartBit and EndBit, then ASSERT().
+
+  @param  Port      The I/O port to write.
+  @param  StartBit  The ordinal of the least significant bit in the bit field.
+                    Range 0..31.
+  @param  EndBit    The ordinal of the most significant bit in the bit field.
+                    Range 0..31.
+  @param  Value     The new value of the bit field.
+
+  @return The value written back to the I/O port.
+
+**/
+UINT32
+EFIAPI
+IoBitFieldWrite32 (
+  IN      UINTN                     Port,
+  IN      UINTN                     StartBit,
+  IN      UINTN                     EndBit,
+  IN      UINT32                    Value
+  )
+{
+  return IoWrite32 (
+           Port,
+           BitFieldWrite32 (IoRead32 (Port), StartBit, EndBit, Value)
+           );
+}
+
+/**
+  Reads a bit field in a 32-bit port, performs a bitwise OR, and writes the
+  result back to the bit field in the 32-bit port.
+
+  Reads the 32-bit I/O port specified by Port, performs a bitwise OR
+  between the read result and the value specified by OrData, and writes the
+  result to the 32-bit I/O port specified by Port. The value written to the I/O
+  port is returned. This function must guarantee that all I/O read and write
+  operations are serialized. Extra bits left in OrData are stripped.
+
+  If 32-bit I/O port operations are not supported, then ASSERT().
+  If Port is not aligned on a 32-bit boundary, then ASSERT().
+  If StartBit is greater than 31, then ASSERT().
+  If EndBit is greater than 31, then ASSERT().
+  If EndBit is less than StartBit, then ASSERT().
+  If OrData is larger than the bitmask value range specified by StartBit and EndBit, then ASSERT().
+
+  @param  Port      The I/O port to write.
+  @param  StartBit  The ordinal of the least significant bit in the bit field.
+                    Range 0..31.
+  @param  EndBit    The ordinal of the most significant bit in the bit field.
+                    Range 0..31.
+  @param  OrData    The value to OR with the read value from the I/O port.
+
+  @return The value written back to the I/O port.
+
+**/
+UINT32
+EFIAPI
+IoBitFieldOr32 (
+  IN      UINTN                     Port,
+  IN      UINTN                     StartBit,
+  IN      UINTN                     EndBit,
+  IN      UINT32                    OrData
+  )
+{
+  return IoWrite32 (
+           Port,
+           BitFieldOr32 (IoRead32 (Port), StartBit, EndBit, OrData)
+           );
+}
+
+/**
+  Reads a bit field in a 32-bit port, performs a bitwise AND, and writes the
+  result back to the bit field in the 32-bit port.
+
+  Reads the 32-bit I/O port specified by Port, performs a bitwise AND between
+  the read result and the value specified by AndData, and writes the result to
+  the 32-bit I/O port specified by Port. The value written to the I/O port is
+  returned. This function must guarantee that all I/O read and write operations
+  are serialized. Extra bits left in AndData are stripped.
+
+  If 32-bit I/O port operations are not supported, then ASSERT().
+  If Port is not aligned on a 32-bit boundary, then ASSERT().
+  If StartBit is greater than 31, then ASSERT().
+  If EndBit is greater than 31, then ASSERT().
+  If EndBit is less than StartBit, then ASSERT().
+  If AndData is larger than the bitmask value range specified by StartBit and EndBit, then ASSERT().
+
+  @param  Port      The I/O port to write.
+  @param  StartBit  The ordinal of the least significant bit in the bit field.
+                    Range 0..31.
+  @param  EndBit    The ordinal of the most significant bit in the bit field.
+                    Range 0..31.
+  @param  AndData   The value to AND with the read value from the I/O port.
+
+  @return The value written back to the I/O port.
+
+**/
+UINT32
+EFIAPI
+IoBitFieldAnd32 (
+  IN      UINTN                     Port,
+  IN      UINTN                     StartBit,
+  IN      UINTN                     EndBit,
+  IN      UINT32                    AndData
+  )
+{
+  return IoWrite32 (
+           Port,
+           BitFieldAnd32 (IoRead32 (Port), StartBit, EndBit, AndData)
+           );
+}
+
+/**
+  Reads a bit field in a 32-bit port, performs a bitwise AND followed by a
+  bitwise OR, and writes the result back to the bit field in the
+  32-bit port.
+
+  Reads the 32-bit I/O port specified by Port, performs a bitwise AND followed
+  by a bitwise OR between the read result and the value specified by
+  AndData, and writes the result to the 32-bit I/O port specified by Port. The
+  value written to the I/O port is returned. This function must guarantee that
+  all I/O read and write operations are serialized. Extra bits left in both
+  AndData and OrData are stripped.
+
+  If 32-bit I/O port operations are not supported, then ASSERT().
+  If Port is not aligned on a 32-bit boundary, then ASSERT().
+  If StartBit is greater than 31, then ASSERT().
+  If EndBit is greater than 31, then ASSERT().
+  If EndBit is less than StartBit, then ASSERT().
+  If AndData is larger than the bitmask value range specified by StartBit and EndBit, then ASSERT().
+  If OrData is larger than the bitmask value range specified by StartBit and EndBit, then ASSERT().
+
+  @param  Port      The I/O port to write.
+  @param  StartBit  The ordinal of the least significant bit in the bit field.
+                    Range 0..31.
+  @param  EndBit    The ordinal of the most significant bit in the bit field.
+                    Range 0..31.
+  @param  AndData   The value to AND with the read value from the I/O port.
+  @param  OrData    The value to OR with the result of the AND operation.
+
+  @return The value written back to the I/O port.
+
+**/
+UINT32
+EFIAPI
+IoBitFieldAndThenOr32 (
+  IN      UINTN                     Port,
+  IN      UINTN                     StartBit,
+  IN      UINTN                     EndBit,
+  IN      UINT32                    AndData,
+  IN      UINT32                    OrData
+  )
+{
+  return IoWrite32 (
+           Port,
+           BitFieldAndThenOr32 (IoRead32 (Port), StartBit, EndBit, AndData, OrData)
+           );
+}
+
+/**
+  Reads a 64-bit I/O port, performs a bitwise OR, and writes the
+  result back to the 64-bit I/O port.
+
+  Reads the 64-bit I/O port specified by Port, performs a bitwise OR
+  between the read result and the value specified by OrData, and writes the
+  result to the 64-bit I/O port specified by Port. The value written to the I/O
+  port is returned. This function must guarantee that all I/O read and write
+  operations are serialized.
+
+  If 64-bit I/O port operations are not supported, then ASSERT().
+  If Port is not aligned on a 64-bit boundary, then ASSERT().
+
+  @param  Port    The I/O port to write.
+  @param  OrData  The value to OR with the read value from the I/O port.
+
+  @return The value written back to the I/O port.
+
+**/
+UINT64
+EFIAPI
+IoOr64 (
+  IN      UINTN                     Port,
+  IN      UINT64                    OrData
+  )
+{
+  return IoWrite64 (Port, IoRead64 (Port) | OrData);
+}
+
+/**
+  Reads a 64-bit I/O port, performs a bitwise AND, and writes the result back
+  to the 64-bit I/O port.
+
+  Reads the 64-bit I/O port specified by Port, performs a bitwise AND between
+  the read result and the value specified by AndData, and writes the result to
+  the 64-bit I/O port specified by Port. The value written to the I/O port is
+  returned. This function must guarantee that all I/O read and write operations
+  are serialized.
+
+  If 64-bit I/O port operations are not supported, then ASSERT().
+  If Port is not aligned on a 64-bit boundary, then ASSERT().
+
+  @param  Port    The I/O port to write.
+  @param  AndData The value to AND with the read value from the I/O port.
+
+  @return The value written back to the I/O port.
+
+**/
+UINT64
+EFIAPI
+IoAnd64 (
+  IN      UINTN                     Port,
+  IN      UINT64                    AndData
+  )
+{
+  return IoWrite64 (Port, IoRead64 (Port) & AndData);
+}
+
+/**
+  Reads a 64-bit I/O port, performs a bitwise AND followed by a bitwise
+  OR, and writes the result back to the 64-bit I/O port.
+
+  Reads the 64-bit I/O port specified by Port, performs a bitwise AND between
+  the read result and the value specified by AndData, performs a bitwise OR
+  between the result of the AND operation and the value specified by OrData,
+  and writes the result to the 64-bit I/O port specified by Port. The value
+  written to the I/O port is returned. This function must guarantee that all
+  I/O read and write operations are serialized.
+
+  If 64-bit I/O port operations are not supported, then ASSERT().
+  If Port is not aligned on a 64-bit boundary, then ASSERT().
+
+  @param  Port    The I/O port to write.
+  @param  AndData The value to AND with the read value from the I/O port.
+  @param  OrData  The value to OR with the result of the AND operation.
+
+  @return The value written back to the I/O port.
+
+**/
+UINT64
+EFIAPI
+IoAndThenOr64 (
+  IN      UINTN                     Port,
+  IN      UINT64                    AndData,
+  IN      UINT64                    OrData
+  )
+{
+  return IoWrite64 (Port, (IoRead64 (Port) & AndData) | OrData);
+}
+
+/**
+  Reads a bit field of an I/O register.
+
+  Reads the bit field in a 64-bit I/O register. The bit field is specified by
+  the StartBit and the EndBit. The value of the bit field is returned.
+
+  If 64-bit I/O port operations are not supported, then ASSERT().
+  If Port is not aligned on a 64-bit boundary, then ASSERT().
+  If StartBit is greater than 63, then ASSERT().
+  If EndBit is greater than 63, then ASSERT().
+  If EndBit is less than StartBit, then ASSERT().
+
+  @param  Port      The I/O port to read.
+  @param  StartBit  The ordinal of the least significant bit in the bit field.
+                    Range 0..63.
+  @param  EndBit    The ordinal of the most significant bit in the bit field.
+                    Range 0..63.
+
+  @return The value read.
+
+**/
+UINT64
+EFIAPI
+IoBitFieldRead64 (
+  IN      UINTN                     Port,
+  IN      UINTN                     StartBit,
+  IN      UINTN                     EndBit
+  )
+{
+  return BitFieldRead64 (IoRead64 (Port), StartBit, EndBit);
+}
+
+/**
+  Writes a bit field to an I/O register.
+
+  Writes Value to the bit field of the I/O register. The bit field is specified
+  by the StartBit and the EndBit. All other bits in the destination I/O
+  register are preserved. The value written to the I/O port is returned. Extra
+  bits left in Value are stripped.
+
+  If 64-bit I/O port operations are not supported, then ASSERT().
+  If Port is not aligned on a 64-bit boundary, then ASSERT().
+  If StartBit is greater than 63, then ASSERT().
+  If EndBit is greater than 63, then ASSERT().
+  If EndBit is less than StartBit, then ASSERT().
+  If Value is larger than the bitmask value range specified by StartBit and EndBit, then ASSERT().
+
+  @param  Port      The I/O port to write.
+  @param  StartBit  The ordinal of the least significant bit in the bit field.
+                    Range 0..63.
+  @param  EndBit    The ordinal of the most significant bit in the bit field.
+                    Range 0..63.
+  @param  Value     The new value of the bit field.
+
+  @return The value written back to the I/O port.
+
+**/
+UINT64
+EFIAPI
+IoBitFieldWrite64 (
+  IN      UINTN                     Port,
+  IN      UINTN                     StartBit,
+  IN      UINTN                     EndBit,
+  IN      UINT64                    Value
+  )
+{
+  return IoWrite64 (
+           Port,
+           BitFieldWrite64 (IoRead64 (Port), StartBit, EndBit, Value)
+           );
+}
+
+/**
+  Reads a bit field in a 64-bit port, performs a bitwise OR, and writes the
+  result back to the bit field in the 64-bit port.
+
+  Reads the 64-bit I/O port specified by Port, performs a bitwise OR
+  between the read result and the value specified by OrData, and writes the
+  result to the 64-bit I/O port specified by Port. The value written to the I/O
+  port is returned. This function must guarantee that all I/O read and write
+  operations are serialized. Extra bits left in OrData are stripped.
+
+  If 64-bit I/O port operations are not supported, then ASSERT().
+  If Port is not aligned on a 64-bit boundary, then ASSERT().
+  If StartBit is greater than 63, then ASSERT().
+  If EndBit is greater than 63, then ASSERT().
+  If EndBit is less than StartBit, then ASSERT().
+  If OrData is larger than the bitmask value range specified by StartBit and EndBit, then ASSERT().
+
+  @param  Port      The I/O port to write.
+  @param  StartBit  The ordinal of the least significant bit in the bit field.
+                    Range 0..63.
+  @param  EndBit    The ordinal of the most significant bit in the bit field.
+                    Range 0..63.
+  @param  OrData    The value to OR with the read value from the I/O port.
+
+  @return The value written back to the I/O port.
+
+**/
+UINT64
+EFIAPI
+IoBitFieldOr64 (
+  IN      UINTN                     Port,
+  IN      UINTN                     StartBit,
+  IN      UINTN                     EndBit,
+  IN      UINT64                    OrData
+  )
+{
+  return IoWrite64 (
+           Port,
+           BitFieldOr64 (IoRead64 (Port), StartBit, EndBit, OrData)
+           );
+}
+
+/**
+  Reads a bit field in a 64-bit port, performs a bitwise AND, and writes the
+  result back to the bit field in the 64-bit port.
+
+  Reads the 64-bit I/O port specified by Port, performs a bitwise AND between
+  the read result and the value specified by AndData, and writes the result to
+  the 64-bit I/O port specified by Port. The value written to the I/O port is
+  returned. This function must guarantee that all I/O read and write operations
+  are serialized. Extra bits left in AndData are stripped.
+
+  If 64-bit I/O port operations are not supported, then ASSERT().
+  If Port is not aligned on a 64-bit boundary, then ASSERT().
+  If StartBit is greater than 63, then ASSERT().
+  If EndBit is greater than 63, then ASSERT().
+  If EndBit is less than StartBit, then ASSERT().
+  If AndData is larger than the bitmask value range specified by StartBit and EndBit, then ASSERT().
+
+  @param  Port      The I/O port to write.
+  @param  StartBit  The ordinal of the least significant bit in the bit field.
+                    Range 0..63.
+  @param  EndBit    The ordinal of the most significant bit in the bit field.
+                    Range 0..63.
+  @param  AndData   The value to AND with the read value from the I/O port.
+
+  @return The value written back to the I/O port.
+
+**/
+UINT64
+EFIAPI
+IoBitFieldAnd64 (
+  IN      UINTN                     Port,
+  IN      UINTN                     StartBit,
+  IN      UINTN                     EndBit,
+  IN      UINT64                    AndData
+  )
+{
+  return IoWrite64 (
+           Port,
+           BitFieldAnd64 (IoRead64 (Port), StartBit, EndBit, AndData)
+           );
+}
+
+/**
+  Reads a bit field in a 64-bit port, performs a bitwise AND followed by a
+  bitwise OR, and writes the result back to the bit field in the
+  64-bit port.
+
+  Reads the 64-bit I/O port specified by Port, performs a bitwise AND followed
+  by a bitwise OR between the read result and the value specified by
+  AndData, and writes the result to the 64-bit I/O port specified by Port. The
+  value written to the I/O port is returned. This function must guarantee that
+  all I/O read and write operations are serialized. Extra bits left in both
+  AndData and OrData are stripped.
+
+  If 64-bit I/O port operations are not supported, then ASSERT().
+  If Port is not aligned on a 64-bit boundary, then ASSERT().
+  If StartBit is greater than 63, then ASSERT().
+  If EndBit is greater than 63, then ASSERT().
+  If EndBit is less than StartBit, then ASSERT().
+  If AndData is larger than the bitmask value range specified by StartBit and EndBit, then ASSERT().
+  If OrData is larger than the bitmask value range specified by StartBit and EndBit, then ASSERT().
+
+  @param  Port      The I/O port to write.
+  @param  StartBit  The ordinal of the least significant bit in the bit field.
+                    Range 0..63.
+  @param  EndBit    The ordinal of the most significant bit in the bit field.
+                    Range 0..63.
+  @param  AndData   The value to AND with the read value from the I/O port.
+  @param  OrData    The value to OR with the result of the AND operation.
+
+  @return The value written back to the I/O port.
+
+**/
+UINT64
+EFIAPI
+IoBitFieldAndThenOr64 (
+  IN      UINTN                     Port,
+  IN      UINTN                     StartBit,
+  IN      UINTN                     EndBit,
+  IN      UINT64                    AndData,
+  IN      UINT64                    OrData
+  )
+{
+  return IoWrite64 (
+           Port,
+           BitFieldAndThenOr64 (IoRead64 (Port), StartBit, EndBit, AndData, OrData)
+           );
+}
+
+/**
+  Reads an 8-bit MMIO register, performs a bitwise OR, and writes the
+  result back to the 8-bit MMIO register.
+
+  Reads the 8-bit MMIO register specified by Address, performs a bitwise
+  OR between the read result and the value specified by OrData, and
+  writes the result to the 8-bit MMIO register specified by Address. The value
+  written to the MMIO register is returned. This function must guarantee that
+  all MMIO read and write operations are serialized.
+
+  If 8-bit MMIO register operations are not supported, then ASSERT().
+
+  @param  Address The MMIO register to write.
+  @param  OrData  The value to OR with the read value from the MMIO register.
+
+  @return The value written back to the MMIO register.
+
+**/
+UINT8
+EFIAPI
+MmioOr8 (
+  IN      UINTN                     Address,
+  IN      UINT8                     OrData
+  )
+{
+  return MmioWrite8 (Address, (UINT8) (MmioRead8 (Address) | OrData));
+}
+
+/**
+  Reads an 8-bit MMIO register, performs a bitwise AND, and writes the result
+  back to the 8-bit MMIO register.
+
+  Reads the 8-bit MMIO register specified by Address, performs a bitwise AND
+  between the read result and the value specified by AndData, and writes the
+  result to the 8-bit MMIO register specified by Address. The value written to
+  the MMIO register is returned. This function must guarantee that all MMIO
+  read and write operations are serialized.
+
+  If 8-bit MMIO register operations are not supported, then ASSERT().
+
+  @param  Address The MMIO register to write.
+  @param  AndData The value to AND with the read value from the MMIO register.
+
+  @return The value written back to the MMIO register.
+
+**/
+UINT8
+EFIAPI
+MmioAnd8 (
+  IN      UINTN                     Address,
+  IN      UINT8                     AndData
+  )
+{
+  return MmioWrite8 (Address, (UINT8) (MmioRead8 (Address) & AndData));
+}
+
+/**
+  Reads an 8-bit MMIO register, performs a bitwise AND followed by a bitwise
+  OR, and writes the result back to the 8-bit MMIO register.
+
+  Reads the 8-bit MMIO register specified by Address, performs a bitwise AND
+  between the read result and the value specified by AndData, performs a
+  bitwise OR between the result of the AND operation and the value specified by
+  OrData, and writes the result to the 8-bit MMIO register specified by
+  Address. The value written to the MMIO register is returned. This function
+  must guarantee that all MMIO read and write operations are serialized.
+
+  If 8-bit MMIO register operations are not supported, then ASSERT().
+
+
+  @param  Address The MMIO register to write.
+  @param  AndData The value to AND with the read value from the MMIO register.
+  @param  OrData  The value to OR with the result of the AND operation.
+
+  @return The value written back to the MMIO register.
+
+**/
+UINT8
+EFIAPI
+MmioAndThenOr8 (
+  IN      UINTN                     Address,
+  IN      UINT8                     AndData,
+  IN      UINT8                     OrData
+  )
+{
+  return MmioWrite8 (Address, (UINT8) ((MmioRead8 (Address) & AndData) | OrData));
+}
+
+/**
+  Reads a bit field of a MMIO register.
+
+  Reads the bit field in an 8-bit MMIO register. The bit field is specified by
+  the StartBit and the EndBit. The value of the bit field is returned.
+
+  If 8-bit MMIO register operations are not supported, then ASSERT().
+  If StartBit is greater than 7, then ASSERT().
+  If EndBit is greater than 7, then ASSERT().
+  If EndBit is less than StartBit, then ASSERT().
+
+  @param  Address   The MMIO register to read.
+  @param  StartBit  The ordinal of the least significant bit in the bit field.
+                    Range 0..7.
+  @param  EndBit    The ordinal of the most significant bit in the bit field.
+                    Range 0..7.
+
+  @return The value read.
+
+**/
+UINT8
+EFIAPI
+MmioBitFieldRead8 (
+  IN      UINTN                     Address,
+  IN      UINTN                     StartBit,
+  IN      UINTN                     EndBit
+  )
+{
+  return BitFieldRead8 (MmioRead8 (Address), StartBit, EndBit);
+}
+
+/**
+  Writes a bit field to a MMIO register.
+
+  Writes Value to the bit field of the MMIO register. The bit field is
+  specified by the StartBit and the EndBit. All other bits in the destination
+  MMIO register are preserved. The new value of the 8-bit register is returned.
+
+  If 8-bit MMIO register operations are not supported, then ASSERT().
+  If StartBit is greater than 7, then ASSERT().
+  If EndBit is greater than 7, then ASSERT().
+  If EndBit is less than StartBit, then ASSERT().
+  If Value is larger than the bitmask value range specified by StartBit and EndBit, then ASSERT().
+
+  @param  Address   The MMIO register to write.
+  @param  StartBit  The ordinal of the least significant bit in the bit field.
+                    Range 0..7.
+  @param  EndBit    The ordinal of the most significant bit in the bit field.
+                    Range 0..7.
+  @param  Value     The new value of the bit field.
+
+  @return The value written back to the MMIO register.
+
+**/
+UINT8
+EFIAPI
+MmioBitFieldWrite8 (
+  IN      UINTN                     Address,
+  IN      UINTN                     StartBit,
+  IN      UINTN                     EndBit,
+  IN      UINT8                     Value
+  )
+{
+  return MmioWrite8 (
+           Address,
+           BitFieldWrite8 (MmioRead8 (Address), StartBit, EndBit, Value)
+           );
+}
+
+/**
+  Reads a bit field in an 8-bit MMIO register, performs a bitwise OR, and
+  writes the result back to the bit field in the 8-bit MMIO register.
+
+  Reads the 8-bit MMIO register specified by Address, performs a bitwise
+  OR between the read result and the value specified by OrData, and
+  writes the result to the 8-bit MMIO register specified by Address. The value
+  written to the MMIO register is returned. This function must guarantee that
+  all MMIO read and write operations are serialized. Extra bits left in OrData
+  are stripped.
+
+  If 8-bit MMIO register operations are not supported, then ASSERT().
+  If StartBit is greater than 7, then ASSERT().
+  If EndBit is greater than 7, then ASSERT().
+  If EndBit is less than StartBit, then ASSERT().
+  If OrData is larger than the bitmask value range specified by StartBit and EndBit, then ASSERT().
+
+  @param  Address   The MMIO register to write.
+  @param  StartBit  The ordinal of the least significant bit in the bit field.
+                    Range 0..7.
+  @param  EndBit    The ordinal of the most significant bit in the bit field.
+                    Range 0..7.
+  @param  OrData    The value to OR with read value from the MMIO register.
+
+  @return The value written back to the MMIO register.
+
+**/
+UINT8
+EFIAPI
+MmioBitFieldOr8 (
+  IN      UINTN                     Address,
+  IN      UINTN                     StartBit,
+  IN      UINTN                     EndBit,
+  IN      UINT8                     OrData
+  )
+{
+  return MmioWrite8 (
+           Address,
+           BitFieldOr8 (MmioRead8 (Address), StartBit, EndBit, OrData)
+           );
+}
+
+/**
+  Reads a bit field in an 8-bit MMIO register, performs a bitwise AND, and
+  writes the result back to the bit field in the 8-bit MMIO register.
+
+  Reads the 8-bit MMIO register specified by Address, performs a bitwise AND
+  between the read result and the value specified by AndData, and writes the
+  result to the 8-bit MMIO register specified by Address. The value written to
+  the MMIO register is returned. This function must guarantee that all MMIO
+  read and write operations are serialized. Extra bits left in AndData are
+  stripped.
+
+  If 8-bit MMIO register operations are not supported, then ASSERT().
+  If StartBit is greater than 7, then ASSERT().
+  If EndBit is greater than 7, then ASSERT().
+  If EndBit is less than StartBit, then ASSERT().
+  If AndData is larger than the bitmask value range specified by StartBit and EndBit, then ASSERT().
+
+  @param  Address   The MMIO register to write.
+  @param  StartBit  The ordinal of the least significant bit in the bit field.
+                    Range 0..7.
+  @param  EndBit    The ordinal of the most significant bit in the bit field.
+                    Range 0..7.
+  @param  AndData   The value to AND with read value from the MMIO register.
+
+  @return The value written back to the MMIO register.
+
+**/
+UINT8
+EFIAPI
+MmioBitFieldAnd8 (
+  IN      UINTN                     Address,
+  IN      UINTN                     StartBit,
+  IN      UINTN                     EndBit,
+  IN      UINT8                     AndData
+  )
+{
+  return MmioWrite8 (
+           Address,
+           BitFieldAnd8 (MmioRead8 (Address), StartBit, EndBit, AndData)
+           );
+}
+
+/**
+  Reads a bit field in an 8-bit MMIO register, performs a bitwise AND followed
+  by a bitwise OR, and writes the result back to the bit field in the
+  8-bit MMIO register.
+
+  Reads the 8-bit MMIO register specified by Address, performs a bitwise AND
+  followed by a bitwise OR between the read result and the value
+  specified by AndData, and writes the result to the 8-bit MMIO register
+  specified by Address. The value written to the MMIO register is returned.
+  This function must guarantee that all MMIO read and write operations are
+  serialized. Extra bits left in both AndData and OrData are stripped.
+
+  If 8-bit MMIO register operations are not supported, then ASSERT().
+  If StartBit is greater than 7, then ASSERT().
+  If EndBit is greater than 7, then ASSERT().
+  If EndBit is less than StartBit, then ASSERT().
+  If AndData is larger than the bitmask value range specified by StartBit and EndBit, then ASSERT().
+  If OrData is larger than the bitmask value range specified by StartBit and EndBit, then ASSERT().
+
+  @param  Address   The MMIO register to write.
+  @param  StartBit  The ordinal of the least significant bit in the bit field.
+                    Range 0..7.
+  @param  EndBit    The ordinal of the most significant bit in the bit field.
+                    Range 0..7.
+  @param  AndData   The value to AND with read value from the MMIO register.
+  @param  OrData    The value to OR with the result of the AND operation.
+
+  @return The value written back to the MMIO register.
+
+**/
+UINT8
+EFIAPI
+MmioBitFieldAndThenOr8 (
+  IN      UINTN                     Address,
+  IN      UINTN                     StartBit,
+  IN      UINTN                     EndBit,
+  IN      UINT8                     AndData,
+  IN      UINT8                     OrData
+  )
+{
+  return MmioWrite8 (
+           Address,
+           BitFieldAndThenOr8 (MmioRead8 (Address), StartBit, EndBit, AndData, OrData)
+           );
+}
+
+/**
+  Reads a 16-bit MMIO register, performs a bitwise OR, and writes the
+  result back to the 16-bit MMIO register.
+
+  Reads the 16-bit MMIO register specified by Address, performs a bitwise
+  OR between the read result and the value specified by OrData, and
+  writes the result to the 16-bit MMIO register specified by Address. The value
+  written to the MMIO register is returned. This function must guarantee that
+  all MMIO read and write operations are serialized.
+
+  If 16-bit MMIO register operations are not supported, then ASSERT().
+  If Address is not aligned on a 16-bit boundary, then ASSERT().
+
+  @param  Address The MMIO register to write.
+  @param  OrData  The value to OR with the read value from the MMIO register.
+
+  @return The value written back to the MMIO register.
+
+**/
+UINT16
+EFIAPI
+MmioOr16 (
+  IN      UINTN                     Address,
+  IN      UINT16                    OrData
+  )
+{
+  return MmioWrite16 (Address, (UINT16) (MmioRead16 (Address) | OrData));
+}
+
+/**
+  Reads a 16-bit MMIO register, performs a bitwise AND, and writes the result
+  back to the 16-bit MMIO register.
+
+  Reads the 16-bit MMIO register specified by Address, performs a bitwise AND
+  between the read result and the value specified by AndData, and writes the
+  result to the 16-bit MMIO register specified by Address. The value written to
+  the MMIO register is returned. This function must guarantee that all MMIO
+  read and write operations are serialized.
+
+  If 16-bit MMIO register operations are not supported, then ASSERT().
+  If Address is not aligned on a 16-bit boundary, then ASSERT().
+
+  @param  Address The MMIO register to write.
+  @param  AndData The value to AND with the read value from the MMIO register.
+
+  @return The value written back to the MMIO register.
+
+**/
+UINT16
+EFIAPI
+MmioAnd16 (
+  IN      UINTN                     Address,
+  IN      UINT16                    AndData
+  )
+{
+  return MmioWrite16 (Address, (UINT16) (MmioRead16 (Address) & AndData));
+}
+
+/**
+  Reads a 16-bit MMIO register, performs a bitwise AND followed by a bitwise
+  OR, and writes the result back to the 16-bit MMIO register.
+
+  Reads the 16-bit MMIO register specified by Address, performs a bitwise AND
+  between the read result and the value specified by AndData, performs a
+  bitwise OR between the result of the AND operation and the value specified by
+  OrData, and writes the result to the 16-bit MMIO register specified by
+  Address. The value written to the MMIO register is returned. This function
+  must guarantee that all MMIO read and write operations are serialized.
+
+  If 16-bit MMIO register operations are not supported, then ASSERT().
+  If Address is not aligned on a 16-bit boundary, then ASSERT().
+
+  @param  Address The MMIO register to write.
+  @param  AndData The value to AND with the read value from the MMIO register.
+  @param  OrData  The value to OR with the result of the AND operation.
+
+  @return The value written back to the MMIO register.
+
+**/
+UINT16
+EFIAPI
+MmioAndThenOr16 (
+  IN      UINTN                     Address,
+  IN      UINT16                    AndData,
+  IN      UINT16                    OrData
+  )
+{
+  return MmioWrite16 (Address, (UINT16) ((MmioRead16 (Address) & AndData) | OrData));
+}
+
+/**
+  Reads a bit field of a MMIO register.
+
+  Reads the bit field in a 16-bit MMIO register. The bit field is specified by
+  the StartBit and the EndBit. The value of the bit field is returned.
+
+  If 16-bit MMIO register operations are not supported, then ASSERT().
+  If Address is not aligned on a 16-bit boundary, then ASSERT().
+  If StartBit is greater than 15, then ASSERT().
+  If EndBit is greater than 15, then ASSERT().
+  If EndBit is less than StartBit, then ASSERT().
+
+  @param  Address   The MMIO register to read.
+  @param  StartBit  The ordinal of the least significant bit in the bit field.
+                    Range 0..15.
+  @param  EndBit    The ordinal of the most significant bit in the bit field.
+                    Range 0..15.
+
+  @return The value read.
+
+**/
+UINT16
+EFIAPI
+MmioBitFieldRead16 (
+  IN      UINTN                     Address,
+  IN      UINTN                     StartBit,
+  IN      UINTN                     EndBit
+  )
+{
+  return BitFieldRead16 (MmioRead16 (Address), StartBit, EndBit);
+}
+
+/**
+  Writes a bit field to a MMIO register.
+
+  Writes Value to the bit field of the MMIO register. The bit field is
+  specified by the StartBit and the EndBit. All other bits in the destination
+  MMIO register are preserved. The new value of the 16-bit register is returned.
+
+  If 16-bit MMIO register operations are not supported, then ASSERT().
+  If Address is not aligned on a 16-bit boundary, then ASSERT().
+  If StartBit is greater than 15, then ASSERT().
+  If EndBit is greater than 15, then ASSERT().
+  If EndBit is less than StartBit, then ASSERT().
+  If Value is larger than the bitmask value range specified by StartBit and EndBit, then ASSERT().
+
+  @param  Address   The MMIO register to write.
+  @param  StartBit  The ordinal of the least significant bit in the bit field.
+                    Range 0..15.
+  @param  EndBit    The ordinal of the most significant bit in the bit field.
+                    Range 0..15.
+  @param  Value     The new value of the bit field.
+
+  @return The value written back to the MMIO register.
+
+**/
+UINT16
+EFIAPI
+MmioBitFieldWrite16 (
+  IN      UINTN                     Address,
+  IN      UINTN                     StartBit,
+  IN      UINTN                     EndBit,
+  IN      UINT16                    Value
+  )
+{
+  return MmioWrite16 (
+           Address,
+           BitFieldWrite16 (MmioRead16 (Address), StartBit, EndBit, Value)
+           );
+}
+
+/**
+  Reads a bit field in a 16-bit MMIO register, performs a bitwise OR, and
+  writes the result back to the bit field in the 16-bit MMIO register.
+
+  Reads the 16-bit MMIO register specified by Address, performs a bitwise
+  OR between the read result and the value specified by OrData, and
+  writes the result to the 16-bit MMIO register specified by Address. The value
+  written to the MMIO register is returned. This function must guarantee that
+  all MMIO read and write operations are serialized. Extra bits left in OrData
+  are stripped.
+
+  If 16-bit MMIO register operations are not supported, then ASSERT().
+  If Address is not aligned on a 16-bit boundary, then ASSERT().
+  If StartBit is greater than 15, then ASSERT().
+  If EndBit is greater than 15, then ASSERT().
+  If EndBit is less than StartBit, then ASSERT().
+  If OrData is larger than the bitmask value range specified by StartBit and EndBit, then ASSERT().
+
+  @param  Address   The MMIO register to write.
+  @param  StartBit  The ordinal of the least significant bit in the bit field.
+                    Range 0..15.
+  @param  EndBit    The ordinal of the most significant bit in the bit field.
+                    Range 0..15.
+  @param  OrData    The value to OR with read value from the MMIO register.
+
+  @return The value written back to the MMIO register.
+
+**/
+UINT16
+EFIAPI
+MmioBitFieldOr16 (
+  IN      UINTN                     Address,
+  IN      UINTN                     StartBit,
+  IN      UINTN                     EndBit,
+  IN      UINT16                    OrData
+  )
+{
+  return MmioWrite16 (
+           Address,
+           BitFieldOr16 (MmioRead16 (Address), StartBit, EndBit, OrData)
+           );
+}
+
+/**
+  Reads a bit field in a 16-bit MMIO register, performs a bitwise AND, and
+  writes the result back to the bit field in the 16-bit MMIO register.
+
+  Reads the 16-bit MMIO register specified by Address, performs a bitwise AND
+  between the read result and the value specified by AndData, and writes the
+  result to the 16-bit MMIO register specified by Address. The value written to
+  the MMIO register is returned. This function must guarantee that all MMIO
+  read and write operations are serialized. Extra bits left in AndData are
+  stripped.
+
+  If 16-bit MMIO register operations are not supported, then ASSERT().
+  If Address is not aligned on a 16-bit boundary, then ASSERT().
+  If StartBit is greater than 15, then ASSERT().
+  If EndBit is greater than 15, then ASSERT().
+  If EndBit is less than StartBit, then ASSERT().
+  If AndData is larger than the bitmask value range specified by StartBit and EndBit, then ASSERT().
+
+  @param  Address   The MMIO register to write.
+  @param  StartBit  The ordinal of the least significant bit in the bit field.
+                    Range 0..15.
+  @param  EndBit    The ordinal of the most significant bit in the bit field.
+                    Range 0..15.
+  @param  AndData   The value to AND with read value from the MMIO register.
+
+  @return The value written back to the MMIO register.
+
+**/
+UINT16
+EFIAPI
+MmioBitFieldAnd16 (
+  IN      UINTN                     Address,
+  IN      UINTN                     StartBit,
+  IN      UINTN                     EndBit,
+  IN      UINT16                    AndData
+  )
+{
+  return MmioWrite16 (
+           Address,
+           BitFieldAnd16 (MmioRead16 (Address), StartBit, EndBit, AndData)
+           );
+}
+
+/**
+  Reads a bit field in a 16-bit MMIO register, performs a bitwise AND followed
+  by a bitwise OR, and writes the result back to the bit field in the
+  16-bit MMIO register.
+
+  Reads the 16-bit MMIO register specified by Address, performs a bitwise AND
+  followed by a bitwise OR between the read result and the value
+  specified by AndData, and writes the result to the 16-bit MMIO register
+  specified by Address. The value written to the MMIO register is returned.
+  This function must guarantee that all MMIO read and write operations are
+  serialized. Extra bits left in both AndData and OrData are stripped.
+
+  If 16-bit MMIO register operations are not supported, then ASSERT().
+  If Address is not aligned on a 16-bit boundary, then ASSERT().
+  If StartBit is greater than 15, then ASSERT().
+  If EndBit is greater than 15, then ASSERT().
+  If EndBit is less than StartBit, then ASSERT().
+  If AndData is larger than the bitmask value range specified by StartBit and EndBit, then ASSERT().
+  If OrData is larger than the bitmask value range specified by StartBit and EndBit, then ASSERT().
+
+  @param  Address   The MMIO register to write.
+  @param  StartBit  The ordinal of the least significant bit in the bit field.
+                    Range 0..15.
+  @param  EndBit    The ordinal of the most significant bit in the bit field.
+                    Range 0..15.
+  @param  AndData   The value to AND with read value from the MMIO register.
+  @param  OrData    The value to OR with the result of the AND operation.
+
+  @return The value written back to the MMIO register.
+
+**/
+UINT16
+EFIAPI
+MmioBitFieldAndThenOr16 (
+  IN      UINTN                     Address,
+  IN      UINTN                     StartBit,
+  IN      UINTN                     EndBit,
+  IN      UINT16                    AndData,
+  IN      UINT16                    OrData
+  )
+{
+  return MmioWrite16 (
+           Address,
+           BitFieldAndThenOr16 (MmioRead16 (Address), StartBit, EndBit, AndData, OrData)
+           );
+}
+
+/**
+  Reads a 32-bit MMIO register, performs a bitwise OR, and writes the
+  result back to the 32-bit MMIO register.
+
+  Reads the 32-bit MMIO register specified by Address, performs a bitwise
+  OR between the read result and the value specified by OrData, and
+  writes the result to the 32-bit MMIO register specified by Address. The value
+  written to the MMIO register is returned. This function must guarantee that
+  all MMIO read and write operations are serialized.
+
+  If 32-bit MMIO register operations are not supported, then ASSERT().
+  If Address is not aligned on a 32-bit boundary, then ASSERT().
+
+  @param  Address The MMIO register to write.
+  @param  OrData  The value to OR with the read value from the MMIO register.
+
+  @return The value written back to the MMIO register.
+
+**/
+UINT32
+EFIAPI
+MmioOr32 (
+  IN      UINTN                     Address,
+  IN      UINT32                    OrData
+  )
+{
+  return MmioWrite32 (Address, MmioRead32 (Address) | OrData);
+}
+
+/**
+  Reads a 32-bit MMIO register, performs a bitwise AND, and writes the result
+  back to the 32-bit MMIO register.
+
+  Reads the 32-bit MMIO register specified by Address, performs a bitwise AND
+  between the read result and the value specified by AndData, and writes the
+  result to the 32-bit MMIO register specified by Address. The value written to
+  the MMIO register is returned. This function must guarantee that all MMIO
+  read and write operations are serialized.
+
+  If 32-bit MMIO register operations are not supported, then ASSERT().
+  If Address is not aligned on a 32-bit boundary, then ASSERT().
+
+  @param  Address The MMIO register to write.
+  @param  AndData The value to AND with the read value from the MMIO register.
+
+  @return The value written back to the MMIO register.
+
+**/
+UINT32
+EFIAPI
+MmioAnd32 (
+  IN      UINTN                     Address,
+  IN      UINT32                    AndData
+  )
+{
+  return MmioWrite32 (Address, MmioRead32 (Address) & AndData);
+}
+
+/**
+  Reads a 32-bit MMIO register, performs a bitwise AND followed by a bitwise
+  OR, and writes the result back to the 32-bit MMIO register.
+
+  Reads the 32-bit MMIO register specified by Address, performs a bitwise AND
+  between the read result and the value specified by AndData, performs a
+  bitwise OR between the result of the AND operation and the value specified by
+  OrData, and writes the result to the 32-bit MMIO register specified by
+  Address. The value written to the MMIO register is returned. This function
+  must guarantee that all MMIO read and write operations are serialized.
+
+  If 32-bit MMIO register operations are not supported, then ASSERT().
+  If Address is not aligned on a 32-bit boundary, then ASSERT().
+
+  @param  Address The MMIO register to write.
+  @param  AndData The value to AND with the read value from the MMIO register.
+  @param  OrData  The value to OR with the result of the AND operation.
+
+  @return The value written back to the MMIO register.
+
+**/
+UINT32
+EFIAPI
+MmioAndThenOr32 (
+  IN      UINTN                     Address,
+  IN      UINT32                    AndData,
+  IN      UINT32                    OrData
+  )
+{
+  return MmioWrite32 (Address, (MmioRead32 (Address) & AndData) | OrData);
+}
+
+/**
+  Reads a bit field of a MMIO register.
+
+  Reads the bit field in a 32-bit MMIO register. The bit field is specified by
+  the StartBit and the EndBit. The value of the bit field is returned.
+
+  If 32-bit MMIO register operations are not supported, then ASSERT().
+  If Address is not aligned on a 32-bit boundary, then ASSERT().
+  If StartBit is greater than 31, then ASSERT().
+  If EndBit is greater than 31, then ASSERT().
+  If EndBit is less than StartBit, then ASSERT().
+
+  @param  Address   The MMIO register to read.
+  @param  StartBit  The ordinal of the least significant bit in the bit field.
+                    Range 0..31.
+  @param  EndBit    The ordinal of the most significant bit in the bit field.
+                    Range 0..31.
+
+  @return The value read.
+
+**/
+UINT32
+EFIAPI
+MmioBitFieldRead32 (
+  IN      UINTN                     Address,
+  IN      UINTN                     StartBit,
+  IN      UINTN                     EndBit
+  )
+{
+  return BitFieldRead32 (MmioRead32 (Address), StartBit, EndBit);
+}
+
+/**
+  Writes a bit field to a MMIO register.
+
+  Writes Value to the bit field of the MMIO register. The bit field is
+  specified by the StartBit and the EndBit. All other bits in the destination
+  MMIO register are preserved. The new value of the 32-bit register is returned.
+
+  If 32-bit MMIO register operations are not supported, then ASSERT().
+  If Address is not aligned on a 32-bit boundary, then ASSERT().
+  If StartBit is greater than 31, then ASSERT().
+  If EndBit is greater than 31, then ASSERT().
+  If EndBit is less than StartBit, then ASSERT().
+  If Value is larger than the bitmask value range specified by StartBit and EndBit, then ASSERT().
+
+  @param  Address   The MMIO register to write.
+  @param  StartBit  The ordinal of the least significant bit in the bit field.
+                    Range 0..31.
+  @param  EndBit    The ordinal of the most significant bit in the bit field.
+                    Range 0..31.
+  @param  Value     The new value of the bit field.
+
+  @return The value written back to the MMIO register.
+
+**/
+UINT32
+EFIAPI
+MmioBitFieldWrite32 (
+  IN      UINTN                     Address,
+  IN      UINTN                     StartBit,
+  IN      UINTN                     EndBit,
+  IN      UINT32                    Value
+  )
+{
+  return MmioWrite32 (
+           Address,
+           BitFieldWrite32 (MmioRead32 (Address), StartBit, EndBit, Value)
+           );
+}
+
+/**
+  Reads a bit field in a 32-bit MMIO register, performs a bitwise OR, and
+  writes the result back to the bit field in the 32-bit MMIO register.
+
+  Reads the 32-bit MMIO register specified by Address, performs a bitwise
+  OR between the read result and the value specified by OrData, and
+  writes the result to the 32-bit MMIO register specified by Address. The value
+  written to the MMIO register is returned. This function must guarantee that
+  all MMIO read and write operations are serialized. Extra bits left in OrData
+  are stripped.
+
+  If 32-bit MMIO register operations are not supported, then ASSERT().
+  If Address is not aligned on a 32-bit boundary, then ASSERT().
+  If StartBit is greater than 31, then ASSERT().
+  If EndBit is greater than 31, then ASSERT().
+  If EndBit is less than StartBit, then ASSERT().
+  If OrData is larger than the bitmask value range specified by StartBit and EndBit, then ASSERT().
+
+  @param  Address   The MMIO register to write.
+  @param  StartBit  The ordinal of the least significant bit in the bit field.
+                    Range 0..31.
+  @param  EndBit    The ordinal of the most significant bit in the bit field.
+                    Range 0..31.
+  @param  OrData    The value to OR with read value from the MMIO register.
+
+  @return The value written back to the MMIO register.
+
+**/
+UINT32
+EFIAPI
+MmioBitFieldOr32 (
+  IN      UINTN                     Address,
+  IN      UINTN                     StartBit,
+  IN      UINTN                     EndBit,
+  IN      UINT32                    OrData
+  )
+{
+  return MmioWrite32 (
+           Address,
+           BitFieldOr32 (MmioRead32 (Address), StartBit, EndBit, OrData)
+           );
+}
+
+/**
+  Reads a bit field in a 32-bit MMIO register, performs a bitwise AND, and
+  writes the result back to the bit field in the 32-bit MMIO register.
+
+  Reads the 32-bit MMIO register specified by Address, performs a bitwise AND
+  between the read result and the value specified by AndData, and writes the
+  result to the 32-bit MMIO register specified by Address. The value written to
+  the MMIO register is returned. This function must guarantee that all MMIO
+  read and write operations are serialized. Extra bits left in AndData are
+  stripped.
+
+  If 32-bit MMIO register operations are not supported, then ASSERT().
+  If Address is not aligned on a 32-bit boundary, then ASSERT().
+  If StartBit is greater than 31, then ASSERT().
+  If EndBit is greater than 31, then ASSERT().
+  If EndBit is less than StartBit, then ASSERT().
+  If AndData is larger than the bitmask value range specified by StartBit and EndBit, then ASSERT().
+
+  @param  Address   The MMIO register to write.
+  @param  StartBit  The ordinal of the least significant bit in the bit field.
+                    Range 0..31.
+  @param  EndBit    The ordinal of the most significant bit in the bit field.
+                    Range 0..31.
+  @param  AndData   The value to AND with read value from the MMIO register.
+
+  @return The value written back to the MMIO register.
+
+**/
+UINT32
+EFIAPI
+MmioBitFieldAnd32 (
+  IN      UINTN                     Address,
+  IN      UINTN                     StartBit,
+  IN      UINTN                     EndBit,
+  IN      UINT32                    AndData
+  )
+{
+  return MmioWrite32 (
+           Address,
+           BitFieldAnd32 (MmioRead32 (Address), StartBit, EndBit, AndData)
+           );
+}
+
+/**
+  Reads a bit field in a 32-bit MMIO register, performs a bitwise AND followed
+  by a bitwise OR, and writes the result back to the bit field in the
+  32-bit MMIO register.
+
+  Reads the 32-bit MMIO register specified by Address, performs a bitwise AND
+  followed by a bitwise OR between the read result and the value
+  specified by AndData, and writes the result to the 32-bit MMIO register
+  specified by Address. The value written to the MMIO register is returned.
+  This function must guarantee that all MMIO read and write operations are
+  serialized. Extra bits left in both AndData and OrData are stripped.
+
+  If 32-bit MMIO register operations are not supported, then ASSERT().
+  If Address is not aligned on a 32-bit boundary, then ASSERT().
+  If StartBit is greater than 31, then ASSERT().
+  If EndBit is greater than 31, then ASSERT().
+  If EndBit is less than StartBit, then ASSERT().
+  If AndData is larger than the bitmask value range specified by StartBit and EndBit, then ASSERT().
+  If OrData is larger than the bitmask value range specified by StartBit and EndBit, then ASSERT().
+
+  @param  Address   The MMIO register to write.
+  @param  StartBit  The ordinal of the least significant bit in the bit field.
+                    Range 0..31.
+  @param  EndBit    The ordinal of the most significant bit in the bit field.
+                    Range 0..31.
+  @param  AndData   The value to AND with read value from the MMIO register.
+  @param  OrData    The value to OR with the result of the AND operation.
+
+  @return The value written back to the MMIO register.
+
+**/
+UINT32
+EFIAPI
+MmioBitFieldAndThenOr32 (
+  IN      UINTN                     Address,
+  IN      UINTN                     StartBit,
+  IN      UINTN                     EndBit,
+  IN      UINT32                    AndData,
+  IN      UINT32                    OrData
+  )
+{
+  return MmioWrite32 (
+           Address,
+           BitFieldAndThenOr32 (MmioRead32 (Address), StartBit, EndBit, AndData, OrData)
+           );
+}
+
+/**
+  Reads a 64-bit MMIO register, performs a bitwise OR, and writes the
+  result back to the 64-bit MMIO register.
+
+  Reads the 64-bit MMIO register specified by Address, performs a bitwise
+  OR between the read result and the value specified by OrData, and
+  writes the result to the 64-bit MMIO register specified by Address. The value
+  written to the MMIO register is returned. This function must guarantee that
+  all MMIO read and write operations are serialized.
+
+  If 64-bit MMIO register operations are not supported, then ASSERT().
+  If Address is not aligned on a 64-bit boundary, then ASSERT().
+
+  @param  Address The MMIO register to write.
+  @param  OrData  The value to OR with the read value from the MMIO register.
+
+  @return The value written back to the MMIO register.
+
+**/
+UINT64
+EFIAPI
+MmioOr64 (
+  IN      UINTN                     Address,
+  IN      UINT64                    OrData
+  )
+{
+  return MmioWrite64 (Address, MmioRead64 (Address) | OrData);
+}
+
+/**
+  Reads a 64-bit MMIO register, performs a bitwise AND, and writes the result
+  back to the 64-bit MMIO register.
+
+  Reads the 64-bit MMIO register specified by Address, performs a bitwise AND
+  between the read result and the value specified by AndData, and writes the
+  result to the 64-bit MMIO register specified by Address. The value written to
+  the MMIO register is returned. This function must guarantee that all MMIO
+  read and write operations are serialized.
+
+  If 64-bit MMIO register operations are not supported, then ASSERT().
+  If Address is not aligned on a 64-bit boundary, then ASSERT().
+
+  @param  Address The MMIO register to write.
+  @param  AndData The value to AND with the read value from the MMIO register.
+
+  @return The value written back to the MMIO register.
+
+**/
+UINT64
+EFIAPI
+MmioAnd64 (
+  IN      UINTN                     Address,
+  IN      UINT64                    AndData
+  )
+{
+  return MmioWrite64 (Address, MmioRead64 (Address) & AndData);
+}
+
+/**
+  Reads a 64-bit MMIO register, performs a bitwise AND followed by a bitwise
+  OR, and writes the result back to the 64-bit MMIO register.
+
+  Reads the 64-bit MMIO register specified by Address, performs a bitwise AND
+  between the read result and the value specified by AndData, performs a
+  bitwise OR between the result of the AND operation and the value specified by
+  OrData, and writes the result to the 64-bit MMIO register specified by
+  Address. The value written to the MMIO register is returned. This function
+  must guarantee that all MMIO read and write operations are serialized.
+
+  If 64-bit MMIO register operations are not supported, then ASSERT().
+  If Address is not aligned on a 64-bit boundary, then ASSERT().
+
+  @param  Address The MMIO register to write.
+  @param  AndData The value to AND with the read value from the MMIO register.
+  @param  OrData  The value to OR with the result of the AND operation.
+
+  @return The value written back to the MMIO register.
+
+**/
+UINT64
+EFIAPI
+MmioAndThenOr64 (
+  IN      UINTN                     Address,
+  IN      UINT64                    AndData,
+  IN      UINT64                    OrData
+  )
+{
+  return MmioWrite64 (Address, (MmioRead64 (Address) & AndData) | OrData);
+}
+
+/**
+  Reads a bit field of a MMIO register.
+
+  Reads the bit field in a 64-bit MMIO register. The bit field is specified by
+  the StartBit and the EndBit. The value of the bit field is returned.
+
+  If 64-bit MMIO register operations are not supported, then ASSERT().
+  If Address is not aligned on a 64-bit boundary, then ASSERT().
+  If StartBit is greater than 63, then ASSERT().
+  If EndBit is greater than 63, then ASSERT().
+  If EndBit is less than StartBit, then ASSERT().
+
+  @param  Address   The MMIO register to read.
+  @param  StartBit  The ordinal of the least significant bit in the bit field.
+                    Range 0..63.
+  @param  EndBit    The ordinal of the most significant bit in the bit field.
+                    Range 0..63.
+
+  @return The value read.
+
+**/
+UINT64
+EFIAPI
+MmioBitFieldRead64 (
+  IN      UINTN                     Address,
+  IN      UINTN                     StartBit,
+  IN      UINTN                     EndBit
+  )
+{
+  return BitFieldRead64 (MmioRead64 (Address), StartBit, EndBit);
+}
+
+/**
+  Writes a bit field to a MMIO register.
+
+  Writes Value to the bit field of the MMIO register. The bit field is
+  specified by the StartBit and the EndBit. All other bits in the destination
+  MMIO register are preserved. The new value of the 64-bit register is returned.
+
+  If 64-bit MMIO register operations are not supported, then ASSERT().
+  If Address is not aligned on a 64-bit boundary, then ASSERT().
+  If StartBit is greater than 63, then ASSERT().
+  If EndBit is greater than 63, then ASSERT().
+  If EndBit is less than StartBit, then ASSERT().
+  If Value is larger than the bitmask value range specified by StartBit and EndBit, then ASSERT().
+
+  @param  Address   The MMIO register to write.
+  @param  StartBit  The ordinal of the least significant bit in the bit field.
+                    Range 0..63.
+  @param  EndBit    The ordinal of the most significant bit in the bit field.
+                    Range 0..63.
+  @param  Value     The new value of the bit field.
+
+  @return The value written back to the MMIO register.
+
+**/
+UINT64
+EFIAPI
+MmioBitFieldWrite64 (
+  IN      UINTN                     Address,
+  IN      UINTN                     StartBit,
+  IN      UINTN                     EndBit,
+  IN      UINT64                    Value
+  )
+{
+  return MmioWrite64 (
+           Address,
+           BitFieldWrite64 (MmioRead64 (Address), StartBit, EndBit, Value)
+           );
+}
+
+/**
+  Reads a bit field in a 64-bit MMIO register, performs a bitwise OR, and
+  writes the result back to the bit field in the 64-bit MMIO register.
+
+  Reads the 64-bit MMIO register specified by Address, performs a bitwise
+  OR between the read result and the value specified by OrData, and
+  writes the result to the 64-bit MMIO register specified by Address. The value
+  written to the MMIO register is returned. This function must guarantee that
+  all MMIO read and write operations are serialized. Extra bits left in OrData
+  are stripped.
+
+  If 64-bit MMIO register operations are not supported, then ASSERT().
+  If Address is not aligned on a 64-bit boundary, then ASSERT().
+  If StartBit is greater than 63, then ASSERT().
+  If EndBit is greater than 63, then ASSERT().
+  If EndBit is less than StartBit, then ASSERT().
+  If OrData is larger than the bitmask value range specified by StartBit and EndBit, then ASSERT().
+
+  @param  Address   The MMIO register to write.
+  @param  StartBit  The ordinal of the least significant bit in the bit field.
+                    Range 0..63.
+  @param  EndBit    The ordinal of the most significant bit in the bit field.
+                    Range 0..63.
+  @param  OrData    The value to OR with read value from the MMIO register.
+
+  @return The value written back to the MMIO register.
+
+**/
+UINT64
+EFIAPI
+MmioBitFieldOr64 (
+  IN      UINTN                     Address,
+  IN      UINTN                     StartBit,
+  IN      UINTN                     EndBit,
+  IN      UINT64                    OrData
+  )
+{
+  return MmioWrite64 (
+           Address,
+           BitFieldOr64 (MmioRead64 (Address), StartBit, EndBit, OrData)
+           );
+}
+
+/**
+  Reads a bit field in a 64-bit MMIO register, performs a bitwise AND, and
+  writes the result back to the bit field in the 64-bit MMIO register.
+
+  Reads the 64-bit MMIO register specified by Address, performs a bitwise AND
+  between the read result and the value specified by AndData, and writes the
+  result to the 64-bit MMIO register specified by Address. The value written to
+  the MMIO register is returned. This function must guarantee that all MMIO
+  read and write operations are serialized. Extra bits left in AndData are
+  stripped.
+
+  If 64-bit MMIO register operations are not supported, then ASSERT().
+  If Address is not aligned on a 64-bit boundary, then ASSERT().
+  If StartBit is greater than 63, then ASSERT().
+  If EndBit is greater than 63, then ASSERT().
+  If EndBit is less than StartBit, then ASSERT().
+  If AndData is larger than the bitmask value range specified by StartBit and EndBit, then ASSERT().
+
+  @param  Address   The MMIO register to write.
+  @param  StartBit  The ordinal of the least significant bit in the bit field.
+                    Range 0..63.
+  @param  EndBit    The ordinal of the most significant bit in the bit field.
+                    Range 0..63.
+  @param  AndData   The value to AND with read value from the MMIO register.
+
+  @return The value written back to the MMIO register.
+
+**/
+UINT64
+EFIAPI
+MmioBitFieldAnd64 (
+  IN      UINTN                     Address,
+  IN      UINTN                     StartBit,
+  IN      UINTN                     EndBit,
+  IN      UINT64                    AndData
+  )
+{
+  return MmioWrite64 (
+           Address,
+           BitFieldAnd64 (MmioRead64 (Address), StartBit, EndBit, AndData)
+           );
+}
+
+/**
+  Reads a bit field in a 64-bit MMIO register, performs a bitwise AND followed
+  by a bitwise OR, and writes the result back to the bit field in the
+  64-bit MMIO register.
+
+  Reads the 64-bit MMIO register specified by Address, performs a bitwise AND
+  followed by a bitwise OR between the read result and the value
+  specified by AndData, and writes the result to the 64-bit MMIO register
+  specified by Address. The value written to the MMIO register is returned.
+  This function must guarantee that all MMIO read and write operations are
+  serialized. Extra bits left in both AndData and OrData are stripped.
+
+  If 64-bit MMIO register operations are not supported, then ASSERT().
+  If Address is not aligned on a 64-bit boundary, then ASSERT().
+  If StartBit is greater than 63, then ASSERT().
+  If EndBit is greater than 63, then ASSERT().
+  If EndBit is less than StartBit, then ASSERT().
+  If AndData is larger than the bitmask value range specified by StartBit and EndBit, then ASSERT().
+  If OrData is larger than the bitmask value range specified by StartBit and EndBit, then ASSERT().
+
+  @param  Address   The MMIO register to write.
+  @param  StartBit  The ordinal of the least significant bit in the bit field.
+                    Range 0..63.
+  @param  EndBit    The ordinal of the most significant bit in the bit field.
+                    Range 0..63.
+  @param  AndData   The value to AND with read value from the MMIO register.
+  @param  OrData    The value to OR with the result of the AND operation.
+
+  @return The value written back to the MMIO register.
+
+**/
+UINT64
+EFIAPI
+MmioBitFieldAndThenOr64 (
+  IN      UINTN                     Address,
+  IN      UINTN                     StartBit,
+  IN      UINTN                     EndBit,
+  IN      UINT64                    AndData,
+  IN      UINT64                    OrData
+  )
+{
+  return MmioWrite64 (
+           Address,
+           BitFieldAndThenOr64 (MmioRead64 (Address), StartBit, EndBit, AndData, OrData)
+           );
+}
diff --git a/ArmVirtPkg/Library/ArmVirtIoLib/IoLibMmioBuffer.c b/ArmVirtPkg/Library/ArmVirtIoLib/IoLibMmioBuffer.c
new file mode 100644
index 000000000000..0cceb575889c
--- /dev/null
+++ b/ArmVirtPkg/Library/ArmVirtIoLib/IoLibMmioBuffer.c
@@ -0,0 +1,413 @@
+/** @file
+  I/O Library MMIO Buffer Functions.
+
+  Copyright (c) 2007 - 2010, Intel Corporation. All rights reserved.<BR>
+  Copyright (c) 2018, Linaro, Ltd. All rights reserved.<BR>
+
+  This program and the accompanying materials are licensed and made available
+  under the terms and conditions of the BSD License which accompanies this
+  distribution. The full text of the license may be found at
+  http://opensource.org/licenses/bsd-license.php.
+
+  THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS,
+  WITHOUT WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED.
+
+**/
+
+#include "ArmVirtIoLib.h"
+
+/**
+  Copy data from the MMIO region to system memory by using 8-bit access.
+
+  Copy data from the MMIO region specified by starting address StartAddress
+  to system memory specified by Buffer by using 8-bit access. The total
+  number of byte to be copied is specified by Length. Buffer is returned.
+
+  If Length is greater than (MAX_ADDRESS - StartAddress + 1), then ASSERT().
+  If Length is greater than (MAX_ADDRESS - Buffer + 1), then ASSERT().
+
+
+  @param  StartAddress    The starting address for the MMIO region to be copied from.
+  @param  Length          The size, in bytes, of Buffer.
+  @param  Buffer          The pointer to a system memory buffer receiving the data read.
+
+  @return Buffer
+
+**/
+UINT8 *
+EFIAPI
+MmioReadBuffer8 (
+  IN  UINTN       StartAddress,
+  IN  UINTN       Length,
+  OUT UINT8       *Buffer
+  )
+{
+  UINT8   *ReturnBuffer;
+
+  ASSERT ((Length - 1) <=  (MAX_ADDRESS - StartAddress));
+  ASSERT ((Length - 1) <=  (MAX_ADDRESS - (UINTN) Buffer));
+
+  ReturnBuffer = Buffer;
+
+  while (Length-- != 0) {
+    *(Buffer++) = MmioRead8 (StartAddress++);
+  }
+
+  return ReturnBuffer;
+}
+
+/**
+  Copy data from the MMIO region to system memory by using 16-bit access.
+
+  Copy data from the MMIO region specified by starting address StartAddress
+  to system memory specified by Buffer by using 16-bit access. The total
+  number of byte to be copied is specified by Length. Buffer is returned.
+
+  If StartAddress is not aligned on a 16-bit boundary, then ASSERT().
+
+  If Length is greater than (MAX_ADDRESS - StartAddress + 1), then ASSERT().
+  If Length is greater than (MAX_ADDRESS - Buffer + 1), then ASSERT().
+
+  If Length is not aligned on a 16-bit boundary, then ASSERT().
+  If Buffer is not aligned on a 16-bit boundary, then ASSERT().
+
+  @param  StartAddress    The starting address for the MMIO region to be copied from.
+  @param  Length          The size, in bytes, of Buffer.
+  @param  Buffer          The pointer to a system memory buffer receiving the data read.
+
+  @return Buffer
+
+**/
+UINT16 *
+EFIAPI
+MmioReadBuffer16 (
+  IN  UINTN       StartAddress,
+  IN  UINTN       Length,
+  OUT UINT16      *Buffer
+  )
+{
+  UINT16    *ReturnBuffer;
+
+  ASSERT ((StartAddress & (sizeof (UINT16) - 1)) == 0);
+
+  ASSERT ((Length - 1) <=  (MAX_ADDRESS - StartAddress));
+  ASSERT ((Length - 1) <=  (MAX_ADDRESS - (UINTN) Buffer));
+
+  ASSERT ((Length & (sizeof (UINT16) - 1)) == 0);
+  ASSERT (((UINTN) Buffer & (sizeof (UINT16) - 1)) == 0);
+
+  ReturnBuffer = Buffer;
+
+  while (Length != 0) {
+    *(Buffer++) = MmioRead16 (StartAddress);
+    StartAddress += sizeof (UINT16);
+    Length -= sizeof (UINT16);
+  }
+
+  return ReturnBuffer;
+}
+
+/**
+  Copy data from the MMIO region to system memory by using 32-bit access.
+
+  Copy data from the MMIO region specified by starting address StartAddress
+  to system memory specified by Buffer by using 32-bit access. The total
+  number of byte to be copied is specified by Length. Buffer is returned.
+
+  If StartAddress is not aligned on a 32-bit boundary, then ASSERT().
+
+  If Length is greater than (MAX_ADDRESS - StartAddress + 1), then ASSERT().
+  If Length is greater than (MAX_ADDRESS - Buffer + 1), then ASSERT().
+
+  If Length is not aligned on a 32-bit boundary, then ASSERT().
+  If Buffer is not aligned on a 32-bit boundary, then ASSERT().
+
+  @param  StartAddress    The starting address for the MMIO region to be copied from.
+  @param  Length          The size, in bytes, of Buffer.
+  @param  Buffer          The pointer to a system memory buffer receiving the data read.
+
+  @return Buffer
+
+**/
+UINT32 *
+EFIAPI
+MmioReadBuffer32 (
+  IN  UINTN       StartAddress,
+  IN  UINTN       Length,
+  OUT UINT32      *Buffer
+  )
+{
+  UINT32    *ReturnBuffer;
+
+  ASSERT ((StartAddress & (sizeof (UINT32) - 1)) == 0);
+
+  ASSERT ((Length - 1) <=  (MAX_ADDRESS - StartAddress));
+  ASSERT ((Length - 1) <=  (MAX_ADDRESS - (UINTN) Buffer));
+
+  ASSERT ((Length & (sizeof (UINT32) - 1)) == 0);
+  ASSERT (((UINTN) Buffer & (sizeof (UINT32) - 1)) == 0);
+
+  ReturnBuffer = Buffer;
+
+  while (Length != 0) {
+    *(Buffer++) = MmioRead32 (StartAddress);
+    StartAddress += sizeof (UINT32);
+    Length -= sizeof (UINT32);
+  }
+
+  return ReturnBuffer;
+}
+
+/**
+  Copy data from the MMIO region to system memory by using 64-bit access.
+
+  Copy data from the MMIO region specified by starting address StartAddress
+  to system memory specified by Buffer by using 64-bit access. The total
+  number of byte to be copied is specified by Length. Buffer is returned.
+
+  If StartAddress is not aligned on a 64-bit boundary, then ASSERT().
+
+  If Length is greater than (MAX_ADDRESS - StartAddress + 1), then ASSERT().
+  If Length is greater than (MAX_ADDRESS - Buffer + 1), then ASSERT().
+
+  If Length is not aligned on a 64-bit boundary, then ASSERT().
+  If Buffer is not aligned on a 64-bit boundary, then ASSERT().
+
+  @param  StartAddress    The starting address for the MMIO region to be copied from.
+  @param  Length          The size, in bytes, of Buffer.
+  @param  Buffer          The pointer to a system memory buffer receiving the data read.
+
+  @return Buffer
+
+**/
+UINT64 *
+EFIAPI
+MmioReadBuffer64 (
+  IN  UINTN       StartAddress,
+  IN  UINTN       Length,
+  OUT UINT64      *Buffer
+  )
+{
+  UINT64    *ReturnBuffer;
+
+  ASSERT ((StartAddress & (sizeof (UINT64) - 1)) == 0);
+
+  ASSERT ((Length - 1) <=  (MAX_ADDRESS - StartAddress));
+  ASSERT ((Length - 1) <=  (MAX_ADDRESS - (UINTN) Buffer));
+
+  ASSERT ((Length & (sizeof (UINT64) - 1)) == 0);
+  ASSERT (((UINTN) Buffer & (sizeof (UINT64) - 1)) == 0);
+
+  ReturnBuffer = Buffer;
+
+  while (Length != 0) {
+    *(Buffer++) = MmioRead64 (StartAddress);
+    StartAddress += sizeof (UINT64);
+    Length -= sizeof (UINT64);
+  }
+
+  return ReturnBuffer;
+}
+
+
+/**
+  Copy data from system memory to the MMIO region by using 8-bit access.
+
+  Copy data from system memory specified by Buffer to the MMIO region specified
+  by starting address StartAddress by using 8-bit access. The total number
+  of byte to be copied is specified by Length. Buffer is returned.
+
+  If Length is greater than (MAX_ADDRESS - StartAddress + 1), then ASSERT().
+  If Length is greater than (MAX_ADDRESS -Buffer + 1), then ASSERT().
+
+
+  @param  StartAddress    The starting address for the MMIO region to be copied to.
+  @param  Length          The size, in bytes, of Buffer.
+  @param  Buffer          The pointer to a system memory buffer containing the data to write.
+
+  @return Buffer
+
+**/
+UINT8 *
+EFIAPI
+MmioWriteBuffer8 (
+  IN  UINTN         StartAddress,
+  IN  UINTN         Length,
+  IN  CONST UINT8   *Buffer
+  )
+{
+  VOID* ReturnBuffer;
+
+  ASSERT ((Length - 1) <=  (MAX_ADDRESS - StartAddress));
+  ASSERT ((Length - 1) <=  (MAX_ADDRESS - (UINTN) Buffer));
+
+  ReturnBuffer = (UINT8 *) Buffer;
+
+  while (Length-- != 0) {
+     MmioWrite8 (StartAddress++, *(Buffer++));
+  }
+
+  return ReturnBuffer;
+
+}
+
+/**
+  Copy data from system memory to the MMIO region by using 16-bit access.
+
+  Copy data from system memory specified by Buffer to the MMIO region specified
+  by starting address StartAddress by using 16-bit access. The total number
+  of byte to be copied is specified by Length. Buffer is returned.
+
+  If StartAddress is not aligned on a 16-bit boundary, then ASSERT().
+
+  If Length is greater than (MAX_ADDRESS - StartAddress + 1), then ASSERT().
+  If Length is greater than (MAX_ADDRESS -Buffer + 1), then ASSERT().
+
+  If Length is not aligned on a 16-bit boundary, then ASSERT().
+
+  If Buffer is not aligned on a 16-bit boundary, then ASSERT().
+
+  @param  StartAddress    The starting address for the MMIO region to be copied to.
+  @param  Length          The size, in bytes, of Buffer.
+  @param  Buffer          The pointer to a system memory buffer containing the data to write.
+
+  @return Buffer
+
+**/
+UINT16 *
+EFIAPI
+MmioWriteBuffer16 (
+  IN  UINTN        StartAddress,
+  IN  UINTN        Length,
+  IN  CONST UINT16 *Buffer
+  )
+{
+  UINT16    *ReturnBuffer;
+
+  ASSERT ((StartAddress & (sizeof (UINT16) - 1)) == 0);
+
+  ASSERT ((Length - 1) <=  (MAX_ADDRESS - StartAddress));
+  ASSERT ((Length - 1) <=  (MAX_ADDRESS - (UINTN) Buffer));
+
+  ASSERT ((Length & (sizeof (UINT16) - 1)) == 0);
+  ASSERT (((UINTN) Buffer & (sizeof (UINT16) - 1)) == 0);
+
+  ReturnBuffer = (UINT16 *) Buffer;
+
+  while (Length != 0) {
+    MmioWrite16 (StartAddress, *(Buffer++));
+
+    StartAddress += sizeof (UINT16);
+    Length -= sizeof (UINT16);
+  }
+
+  return ReturnBuffer;
+}
+
+
+/**
+  Copy data from system memory to the MMIO region by using 32-bit access.
+
+  Copy data from system memory specified by Buffer to the MMIO region specified
+  by starting address StartAddress by using 32-bit access. The total number
+  of byte to be copied is specified by Length. Buffer is returned.
+
+  If StartAddress is not aligned on a 32-bit boundary, then ASSERT().
+
+  If Length is greater than (MAX_ADDRESS - StartAddress + 1), then ASSERT().
+  If Length is greater than (MAX_ADDRESS -Buffer + 1), then ASSERT().
+
+  If Length is not aligned on a 32-bit boundary, then ASSERT().
+
+  If Buffer is not aligned on a 32-bit boundary, then ASSERT().
+
+  @param  StartAddress    The starting address for the MMIO region to be copied to.
+  @param  Length          The size, in bytes, of Buffer.
+  @param  Buffer          The pointer to a system memory buffer containing the data to write.
+
+  @return Buffer
+
+**/
+UINT32 *
+EFIAPI
+MmioWriteBuffer32 (
+  IN  UINTN        StartAddress,
+  IN  UINTN        Length,
+  IN  CONST UINT32 *Buffer
+  )
+{
+  UINT32    *ReturnBuffer;
+
+  ASSERT ((StartAddress & (sizeof (UINT32) - 1)) == 0);
+
+  ASSERT ((Length - 1) <=  (MAX_ADDRESS - StartAddress));
+  ASSERT ((Length - 1) <=  (MAX_ADDRESS - (UINTN) Buffer));
+
+  ASSERT ((Length & (sizeof (UINT32) - 1)) == 0);
+  ASSERT (((UINTN) Buffer & (sizeof (UINT32) - 1)) == 0);
+
+  ReturnBuffer = (UINT32 *) Buffer;
+
+  while (Length != 0) {
+    MmioWrite32 (StartAddress, *(Buffer++));
+
+    StartAddress += sizeof (UINT32);
+    Length -= sizeof (UINT32);
+  }
+
+  return ReturnBuffer;
+}
+
+/**
+  Copy data from system memory to the MMIO region by using 64-bit access.
+
+  Copy data from system memory specified by Buffer to the MMIO region specified
+  by starting address StartAddress by using 64-bit access. The total number
+  of byte to be copied is specified by Length. Buffer is returned.
+
+  If StartAddress is not aligned on a 64-bit boundary, then ASSERT().
+
+  If Length is greater than (MAX_ADDRESS - StartAddress + 1), then ASSERT().
+  If Length is greater than (MAX_ADDRESS -Buffer + 1), then ASSERT().
+
+  If Length is not aligned on a 64-bit boundary, then ASSERT().
+
+  If Buffer is not aligned on a 64-bit boundary, then ASSERT().
+
+  @param  StartAddress    The starting address for the MMIO region to be copied to.
+  @param  Length          The size, in bytes, of Buffer.
+  @param  Buffer          The pointer to a system memory buffer containing the data to write.
+
+  @return Buffer
+
+**/
+UINT64 *
+EFIAPI
+MmioWriteBuffer64 (
+  IN  UINTN        StartAddress,
+  IN  UINTN        Length,
+  IN  CONST UINT64 *Buffer
+  )
+{
+  UINT64    *ReturnBuffer;
+
+  ASSERT ((StartAddress & (sizeof (UINT64) - 1)) == 0);
+
+  ASSERT ((Length - 1) <=  (MAX_ADDRESS - StartAddress));
+  ASSERT ((Length - 1) <=  (MAX_ADDRESS - (UINTN) Buffer));
+
+  ASSERT ((Length & (sizeof (UINT64) - 1)) == 0);
+  ASSERT (((UINTN) Buffer & (sizeof (UINT64) - 1)) == 0);
+
+  ReturnBuffer = (UINT64 *) Buffer;
+
+  while (Length != 0) {
+    MmioWrite64 (StartAddress, *(Buffer++));
+
+    StartAddress += sizeof (UINT64);
+    Length -= sizeof (UINT64);
+  }
+
+  return ReturnBuffer;
+}
+
-- 
2.17.0



^ permalink raw reply related	[flat|nested] 4+ messages in thread

* Re: [PATCH] ArmVirtPkg: implement KVM safe IoLib instance
  2018-06-05 11:05 [PATCH] ArmVirtPkg: implement KVM safe IoLib instance Ard Biesheuvel
@ 2018-06-05 11:29 ` Leif Lindholm
  2018-06-05 13:04 ` Laszlo Ersek
  1 sibling, 0 replies; 4+ messages in thread
From: Leif Lindholm @ 2018-06-05 11:29 UTC (permalink / raw)
  To: Ard Biesheuvel; +Cc: edk2-devel, lersek

On Tue, Jun 05, 2018 at 01:05:43PM +0200, Ard Biesheuvel wrote:
> KVM on ARM refuses to decode load/store instructions used to perform
> I/O to emulated devices, and instead relies on the exception syndrome
> information to describe the operand register, access size, etc.
> This is only possible for instructions that have a single input/output
> register (as opposed to ones that increment the offset register, or
> load/store pair instructions, etc). Otherwise, QEMU crashes with the
> following error
> 
>   error: kvm run failed Function not implemented
>   R00=01010101 R01=00000008 R02=00000048 R03=08000820
>   R04=00000120 R05=7faaa0e0 R06=7faaa0dc R07=7faaa0e8
>   R08=7faaa0ec R09=7faaa088 R10=000000ff R11=00000080
>   R12=ff000000 R13=7fccfe08 R14=7faa835f R15=7faa887c
>   PSR=800001f3 N--- T svc32
>   QEMU: Terminated
> 
> and KVM produces a warning such as the following in the kernel log
> 
>   kvm [17646]: load/store instruction decoding not implemented
> 
> The IoLib implementation provided by MdePkg/Library/BaseIoLibIntrinsic
> is based on C code, and when LTO is in effect, the MMIO accesses could
> be merged with, e.g., manipulations of the loop counter, producing
> opcodes that KVM does not support for emulated MMIO.
> 
> So instead, let's reimplement IoLib in a KVM safe manner.
> 
> Contributed-under: TianoCore Contribution Agreement 1.1
> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> ---
> Yet another approach for the KVM MMIO emulation issue. Note that this one
> (as well as the MdePkg) affect both AArch64 and ARM. This is deliberate,
> given that there is no reason AArch64 should be immune to this: we simply
> haven't triggered the issue yet.

I'm happier with this version (at the same time as I'm sad we're
splitting this out).

Reviewed-by: Leif Lindholm <leif.lindholm@linaro.org>

>  ArmVirtPkg/Library/ArmVirtIoLib/AArch64/ArmVirtMmio.S |  164 ++
>  ArmVirtPkg/Library/ArmVirtIoLib/Arm/ArmVirtMmio.S     |  154 ++
>  ArmVirtPkg/Library/ArmVirtIoLib/Arm/ArmVirtMmio.asm   |  165 ++
>  ArmVirtPkg/Library/ArmVirtIoLib/ArmVirtIoLib.c        |  589 +++++
>  ArmVirtPkg/Library/ArmVirtIoLib/ArmVirtIoLib.h        |  188 ++
>  ArmVirtPkg/Library/ArmVirtIoLib/ArmVirtIoLib.inf      |   49 +
>  ArmVirtPkg/Library/ArmVirtIoLib/IoHighLevel.c         | 2358 ++++++++++++++++++++
>  ArmVirtPkg/Library/ArmVirtIoLib/IoLibMmioBuffer.c     |  413 ++++
>  8 files changed, 4080 insertions(+)


^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [PATCH] ArmVirtPkg: implement KVM safe IoLib instance
  2018-06-05 11:05 [PATCH] ArmVirtPkg: implement KVM safe IoLib instance Ard Biesheuvel
  2018-06-05 11:29 ` Leif Lindholm
@ 2018-06-05 13:04 ` Laszlo Ersek
  2018-06-07 10:44   ` Ard Biesheuvel
  1 sibling, 1 reply; 4+ messages in thread
From: Laszlo Ersek @ 2018-06-05 13:04 UTC (permalink / raw)
  To: Ard Biesheuvel, edk2-devel; +Cc: leif.lindholm

On 06/05/18 13:05, Ard Biesheuvel wrote:
> KVM on ARM refuses to decode load/store instructions used to perform
> I/O to emulated devices, and instead relies on the exception syndrome
> information to describe the operand register, access size, etc.
> This is only possible for instructions that have a single input/output
> register (as opposed to ones that increment the offset register, or
> load/store pair instructions, etc). Otherwise, QEMU crashes with the
> following error
> 
>   error: kvm run failed Function not implemented
>   R00=01010101 R01=00000008 R02=00000048 R03=08000820
>   R04=00000120 R05=7faaa0e0 R06=7faaa0dc R07=7faaa0e8
>   R08=7faaa0ec R09=7faaa088 R10=000000ff R11=00000080
>   R12=ff000000 R13=7fccfe08 R14=7faa835f R15=7faa887c
>   PSR=800001f3 N--- T svc32
>   QEMU: Terminated
> 
> and KVM produces a warning such as the following in the kernel log
> 
>   kvm [17646]: load/store instruction decoding not implemented
> 
> The IoLib implementation provided by MdePkg/Library/BaseIoLibIntrinsic
> is based on C code, and when LTO is in effect, the MMIO accesses could
> be merged with, e.g., manipulations of the loop counter, producing
> opcodes that KVM does not support for emulated MMIO.
> 
> So instead, let's reimplement IoLib in a KVM safe manner.
> 
> Contributed-under: TianoCore Contribution Agreement 1.1
> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> ---
> Yet another approach for the KVM MMIO emulation issue. Note that this one
> (as well as the MdePkg) affect both AArch64 and ARM. This is deliberate,
> given that there is no reason AArch64 should be immune to this: we simply
> haven't triggered the issue yet.
> 
>  ArmVirtPkg/Library/ArmVirtIoLib/AArch64/ArmVirtMmio.S |  164 ++
>  ArmVirtPkg/Library/ArmVirtIoLib/Arm/ArmVirtMmio.S     |  154 ++
>  ArmVirtPkg/Library/ArmVirtIoLib/Arm/ArmVirtMmio.asm   |  165 ++
>  ArmVirtPkg/Library/ArmVirtIoLib/ArmVirtIoLib.c        |  589 +++++
>  ArmVirtPkg/Library/ArmVirtIoLib/ArmVirtIoLib.h        |  188 ++
>  ArmVirtPkg/Library/ArmVirtIoLib/ArmVirtIoLib.inf      |   49 +
>  ArmVirtPkg/Library/ArmVirtIoLib/IoHighLevel.c         | 2358 ++++++++++++++++++++
>  ArmVirtPkg/Library/ArmVirtIoLib/IoLibMmioBuffer.c     |  413 ++++
>  8 files changed, 4080 insertions(+)

(1) I believe the edk2 nomenclature for library instances would suggest
we call this "BaseIoLibArmVirt" or "BaseIoLibKvm". However, I notice a
great many of our library instances under ArmVirtPkg/Library don't
follow that already, whereas we have several instances starting with
"ArmVirt*". So I value the consistency here; OK.


(2) I've compared some files:

(2a) BaseIoLibIntrinsic.inf <-> ArmVirtIoLib.inf:

- blurb updated,
- copyright notice updated,
- text rewrapped,
- INF_VERSION good,
- BASE_NAME good,
- FILE_GUID unique,
- VALID_ARCHITECTURES good,
- [Sources] etc good

OK.

(2b) IoLibMmioBuffer.c <-> IoLibMmioBuffer.c:

- copyright notice updated,
- text rewrapped,
- some trailing whitespace stripped,
- ArmVirtIoLib.h included rather than BaseIoLibIntrinsicInternal.h,

OK.

(2c) IoHighLevel.c <-> IoHighLevel.c:

- same as (2b),
- plus the reference to "IoLib instances" in the blurb has been updated
  to "MdePkg IoLib instances".

OK.

(2d) BaseIoLibIntrinsicInternal.h <-> ArmVirtIoLib.h:

- kept the common includes (see (2b) and (2c) #includes)
- added function declarations for the assembly functions

OK.

(2e) IoLibArm.c <-> ArmVirtIoLib.c:

This replaces the volatile pointer de-references with calls to the
assembly functions, which seems fine. However, some ASSERT()s about
alignment appear removed (in "read" functions only); is that
intentional? See MmioRead16(), MmioRead32().

Curiously, MmioRead64() preserves the ASSERT().

Otherwise, OK.


(3) New files (the assembly files): Question: why does ARM provide DMB
vs. DSB instructions? Answer: so that ARM experts can have a good
discussion about them every time they show up in new code. :P

(I mean, what chance do mere mortals have to get them right?...)


(4) When we added SEV customizations to IoLib (see
"BaseIoLibIntrinsicSev.inf" / commit b6d11d7c4678), we added those to
MdePkg. Can you please *briefly* investigate whether similar to commit
b6d11d7c4678 would be possible here?

I.e., introduce a new INF file called BaseIoLibIntrinsicArmVirt.inf, and
add "IoLibArmVirt.c" alongside the preexistent "IoLibArm.c". We already
have subdirs for Ia32 and X64 *.nasm, we could create Arm and AArch64
dirs as siblings. This way we could reuse "IoLibMmioBuffer.c" and
"IoHighLevel.c", perhaps?

Anyway I'm not trying to slow down this development by suggesting we add
it to MdePkg. So, if it ends up being too complex for MdePkg, I'm fine
with this patch, Package-wise.


In summary:
- Are the alignment ASSERT() removals in MmioRead16() and MmioRead32()
intentional?
- Can we make one *quick* attempt to put this into MdePkg, if you think
that's feasible?

Thanks!
Laszlo


^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [PATCH] ArmVirtPkg: implement KVM safe IoLib instance
  2018-06-05 13:04 ` Laszlo Ersek
@ 2018-06-07 10:44   ` Ard Biesheuvel
  0 siblings, 0 replies; 4+ messages in thread
From: Ard Biesheuvel @ 2018-06-07 10:44 UTC (permalink / raw)
  To: Laszlo Ersek; +Cc: edk2-devel@lists.01.org, Leif Lindholm

On 5 June 2018 at 15:04, Laszlo Ersek <lersek@redhat.com> wrote:
> On 06/05/18 13:05, Ard Biesheuvel wrote:
>> KVM on ARM refuses to decode load/store instructions used to perform
>> I/O to emulated devices, and instead relies on the exception syndrome
>> information to describe the operand register, access size, etc.
>> This is only possible for instructions that have a single input/output
>> register (as opposed to ones that increment the offset register, or
>> load/store pair instructions, etc). Otherwise, QEMU crashes with the
>> following error
>>
>>   error: kvm run failed Function not implemented
>>   R00=01010101 R01=00000008 R02=00000048 R03=08000820
>>   R04=00000120 R05=7faaa0e0 R06=7faaa0dc R07=7faaa0e8
>>   R08=7faaa0ec R09=7faaa088 R10=000000ff R11=00000080
>>   R12=ff000000 R13=7fccfe08 R14=7faa835f R15=7faa887c
>>   PSR=800001f3 N--- T svc32
>>   QEMU: Terminated
>>
>> and KVM produces a warning such as the following in the kernel log
>>
>>   kvm [17646]: load/store instruction decoding not implemented
>>
>> The IoLib implementation provided by MdePkg/Library/BaseIoLibIntrinsic
>> is based on C code, and when LTO is in effect, the MMIO accesses could
>> be merged with, e.g., manipulations of the loop counter, producing
>> opcodes that KVM does not support for emulated MMIO.
>>
>> So instead, let's reimplement IoLib in a KVM safe manner.
>>
>> Contributed-under: TianoCore Contribution Agreement 1.1
>> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
>> ---
>> Yet another approach for the KVM MMIO emulation issue. Note that this one
>> (as well as the MdePkg) affect both AArch64 and ARM. This is deliberate,
>> given that there is no reason AArch64 should be immune to this: we simply
>> haven't triggered the issue yet.
>>
>>  ArmVirtPkg/Library/ArmVirtIoLib/AArch64/ArmVirtMmio.S |  164 ++
>>  ArmVirtPkg/Library/ArmVirtIoLib/Arm/ArmVirtMmio.S     |  154 ++
>>  ArmVirtPkg/Library/ArmVirtIoLib/Arm/ArmVirtMmio.asm   |  165 ++
>>  ArmVirtPkg/Library/ArmVirtIoLib/ArmVirtIoLib.c        |  589 +++++
>>  ArmVirtPkg/Library/ArmVirtIoLib/ArmVirtIoLib.h        |  188 ++
>>  ArmVirtPkg/Library/ArmVirtIoLib/ArmVirtIoLib.inf      |   49 +
>>  ArmVirtPkg/Library/ArmVirtIoLib/IoHighLevel.c         | 2358 ++++++++++++++++++++
>>  ArmVirtPkg/Library/ArmVirtIoLib/IoLibMmioBuffer.c     |  413 ++++
>>  8 files changed, 4080 insertions(+)
>
> (1) I believe the edk2 nomenclature for library instances would suggest
> we call this "BaseIoLibArmVirt" or "BaseIoLibKvm". However, I notice a
> great many of our library instances under ArmVirtPkg/Library don't
> follow that already, whereas we have several instances starting with
> "ArmVirt*". So I value the consistency here; OK.
>
>
> (2) I've compared some files:
>
> (2a) BaseIoLibIntrinsic.inf <-> ArmVirtIoLib.inf:
>
> - blurb updated,
> - copyright notice updated,
> - text rewrapped,
> - INF_VERSION good,
> - BASE_NAME good,
> - FILE_GUID unique,
> - VALID_ARCHITECTURES good,
> - [Sources] etc good
>
> OK.
>
> (2b) IoLibMmioBuffer.c <-> IoLibMmioBuffer.c:
>
> - copyright notice updated,
> - text rewrapped,
> - some trailing whitespace stripped,
> - ArmVirtIoLib.h included rather than BaseIoLibIntrinsicInternal.h,
>
> OK.
>
> (2c) IoHighLevel.c <-> IoHighLevel.c:
>
> - same as (2b),
> - plus the reference to "IoLib instances" in the blurb has been updated
>   to "MdePkg IoLib instances".
>
> OK.
>
> (2d) BaseIoLibIntrinsicInternal.h <-> ArmVirtIoLib.h:
>
> - kept the common includes (see (2b) and (2c) #includes)
> - added function declarations for the assembly functions
>
> OK.
>
> (2e) IoLibArm.c <-> ArmVirtIoLib.c:
>
> This replaces the volatile pointer de-references with calls to the
> assembly functions, which seems fine. However, some ASSERT()s about
> alignment appear removed (in "read" functions only); is that
> intentional? See MmioRead16(), MmioRead32().
>
> Curiously, MmioRead64() preserves the ASSERT().
>
> Otherwise, OK.
>

No, that is an oversight. The whole pointed of wrapping the asm
functions in C functions was so that the ASSERT()s could be preserved.

>
> (3) New files (the assembly files): Question: why does ARM provide DMB
> vs. DSB instructions? Answer: so that ARM experts can have a good
> discussion about them every time they show up in new code. :P
>
> (I mean, what chance do mere mortals have to get them right?...)
>

This is about ordering vs. side effects. DMB manages the order in
which memory accesses are issued, DSB ensures that they have actually
completed.

>
> (4) When we added SEV customizations to IoLib (see
> "BaseIoLibIntrinsicSev.inf" / commit b6d11d7c4678), we added those to
> MdePkg. Can you please *briefly* investigate whether similar to commit
> b6d11d7c4678 would be possible here?
>
> I.e., introduce a new INF file called BaseIoLibIntrinsicArmVirt.inf, and
> add "IoLibArmVirt.c" alongside the preexistent "IoLibArm.c". We already
> have subdirs for Ia32 and X64 *.nasm, we could create Arm and AArch64
> dirs as siblings. This way we could reuse "IoLibMmioBuffer.c" and
> "IoHighLevel.c", perhaps?
>
> Anyway I'm not trying to slow down this development by suggesting we add
> it to MdePkg. So, if it ends up being too complex for MdePkg, I'm fine
> with this patch, Package-wise.
>
>
> In summary:
> - Are the alignment ASSERT() removals in MmioRead16() and MmioRead32()
> intentional?
> - Can we make one *quick* attempt to put this into MdePkg, if you think
> that's feasible?
>

I'll send out another patch.


^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2018-06-07 10:44 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2018-06-05 11:05 [PATCH] ArmVirtPkg: implement KVM safe IoLib instance Ard Biesheuvel
2018-06-05 11:29 ` Leif Lindholm
2018-06-05 13:04 ` Laszlo Ersek
2018-06-07 10:44   ` Ard Biesheuvel

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox