[PATCH v3 4/5] MdeModulePkg/UsbBotPei: Fix undefined behavior in signed left shift

[Hao Wu <hao.a.wu@intel.com>]

In function PeiUsbReadCapacity(), the following expression:
LastBlock = (Data.LastLba3 << 24) |
  (Data.LastLba2 << 16) |
  (Data.LastLba1 << 8) |
  Data.LastLba0;

(There is also a similar case in function PeiUsbReadFormattedCapacity().)

will involve undefined behavior in signed left shift operations.

Since Data.LastLbaX is of type UINT8, they will be promoted to int (32
bits, signed) first, and then perform the left shift operation.

According to the C11 spec, Section 6.5.7:
4 The result of E1 << E2 is E1 left-shifted E2 bit positions; vacated
  bits are filled with zeros. If E1 has an unsigned type, the value
  of the result is E1 * 2^E2 , reduced modulo one more than the
  maximum value representable in the result type. If E1 has a signed
  type and nonnegative value, and E1 * 2^E2 is representable in the
  result type, then that is the resulting value; otherwise, the
  behavior is undefined.

So if bit 7 of Data.LastLba3 is 1, (Data.LastLba3 << 24) will be out of
the range within int type. The undefined behavior of the signed left shift
might incur potential issues.

This commit will add an explicit UINT32 type cast for Data.LastLba3 to
refine the codes.

Cc: Feng Tian <feng.tian@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Hao Wu <hao.a.wu@intel.com>
---
 MdeModulePkg/Bus/Usb/UsbBotPei/PeiAtapi.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/MdeModulePkg/Bus/Usb/UsbBotPei/PeiAtapi.c b/MdeModulePkg/Bus/Usb/UsbBotPei/PeiAtapi.c
index eaad626d61..72bde756eb 100644
--- a/MdeModulePkg/Bus/Usb/UsbBotPei/PeiAtapi.c
+++ b/MdeModulePkg/Bus/Usb/UsbBotPei/PeiAtapi.c
@@ -1,7 +1,7 @@
 /** @file
 Pei USB ATATPI command implementations.
 
-Copyright (c) 1999 - 2015, Intel Corporation. All rights reserved.<BR>
+Copyright (c) 1999 - 2017, Intel Corporation. All rights reserved.<BR>
   
 This program and the accompanying materials
 are licensed and made available under the terms and conditions
@@ -274,7 +274,7 @@ PeiUsbReadCapacity (
   if (EFI_ERROR (Status)) {
     return EFI_DEVICE_ERROR;
   }
-  LastBlock = (Data.LastLba3 << 24) | (Data.LastLba2 << 16) | (Data.LastLba1 << 8) | Data.LastLba0;
+  LastBlock = ((UINT32) Data.LastLba3 << 24) | (Data.LastLba2 << 16) | (Data.LastLba1 << 8) | Data.LastLba0;
 
   if (LastBlock == 0xFFFFFFFF) {
     DEBUG ((EFI_D_INFO, "The usb device LBA count is larger than 0xFFFFFFFF!\n"));
@@ -346,7 +346,7 @@ PeiUsbReadFormattedCapacity (
     PeiBotDevice->Media2.LastBlock     = 0;
 
   } else {
-    LastBlock = (FormatData.LastLba3 << 24) | (FormatData.LastLba2 << 16) | (FormatData.LastLba1 << 8) | FormatData.LastLba0;
+    LastBlock = ((UINT32) FormatData.LastLba3 << 24) | (FormatData.LastLba2 << 16) | (FormatData.LastLba1 << 8) | FormatData.LastLba0;
     if (LastBlock == 0xFFFFFFFF) {
       DEBUG ((EFI_D_INFO, "The usb device LBA count is larger than 0xFFFFFFFF!\n"));
     }
-- 
2.12.0.windows.1