[PATCH v3 0/5] Resolve undefined behaviours in left shift OPs

[PATCH v3 0/5] Resolve undefined behaviours in left shift OPs

[Hao Wu]

Based on the feedbacks from Liming, the V3 series drops the patch for
MdeModulePkg/Crc32. Since the left shift operation "1 << Index" will be
removed by an under-reviewing patch series:

[Patch 0/2] Add CalculateCrc32() API in MdePkg BaseLib


V2 history:
According to the feebacks from Paolo, the following changes are made in
this version of patch series:
  a. Refine the code logic in Tpl.c to void left shifting the negative
     value. Also makes the code more readable;
  b. Remove the '~' operator before 'Time.Hour' in DxeNetLib.c, since it
     seems like an implementation choice for generating the seed;
  c. Use '1U' instead of '(UINT32)1' in Crc32.c.


V1 history:
The series resolves two kinds of undefined behaviours in left shift
operations:
  a. Left-shifting negative values;
  b. Left-shifting that incurs the result being out of range.

Cc: Steven Shi <steven.shi@intel.com>
Cc: Michael Kinney <michael.d.kinney@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Star Zeng <star.zeng@intel.com>
Cc: Eric Dong <eric.dong@intel.com>
Cc: Fu Siyuan <siyuan.fu@intel.com>
Cc: Ye Ting <ting.ye@intel.com>
Cc: Wu Jiaxin <jiaxin.wu@intel.com>
Cc: Qin Long <qin.long@intel.com>
Cc: Paolo Bonzini <pbonzini@redhat.com> 


Hao Wu (5):
  MdePkg/PrintLib: Fix possible negative value left shift
  MdeModulePkg/PrintLib: Fix possible negative value left shift
  MdeModulePkg/Tpl: Fix negative value left shift
  MdeModulePkg/DxeNetLib: Fix negative value left shift
  MdeModulePkg/AtaAtapiPassThru: Fix possible out of range left shift

 MdeModulePkg/Bus/Ata/AtaAtapiPassThru/AhciMode.c          |  4 ++--
 MdeModulePkg/Core/Dxe/Event/Tpl.c                         | 12 +++++++++---
 MdeModulePkg/Library/DxeNetLib/DxeNetLib.c                |  2 +-
 MdeModulePkg/Library/DxePrintLibPrint2Protocol/PrintLib.c |  2 +-
 MdePkg/Library/BasePrintLib/PrintLibInternal.c            |  2 +-
 5 files changed, 14 insertions(+), 8 deletions(-)

-- 
2.12.0.windows.1

[PATCH v3 1/5] MdePkg/PrintLib: Fix possible negative value left shift

[Hao Wu]

REF: https://bugzilla.tianocore.org/show_bug.cgi?id=702

Within function InternalPrintLibSPrintMarker(), possible left shift of a
negative value is found in:
"(*(ArgumentString + 1) << 8)"

which involves undefined behavior.

Since '*(ArgumentString + 1)' is of type CONST CHAR8 (signed), it will be
promoted to type int (signed) during the left shift operation. If
'*(ArgumentString + 1)' is a negative value, the behavior will be
undefined.

According to the C11 spec, Section 6.5.7:
> 4 The result of E1 << E2 is E1 left-shifted E2 bit positions; vacated
>   bits are filled with zeros. If E1 has an unsigned type, the value
>   of the result is E1 * 2^E2 , reduced modulo one more than the
>   maximum value representable in the result type. If E1 has a signed
>   type and nonnegative value, and E1 * 2^E2 is representable in the
>   result type, then that is the resulting value; otherwise, the
>   behavior is undefined.

This commit explicitly cast '*(ArgumentString + 1)' with UINT8 to resolve
this issue.

Cc: Steven Shi <steven.shi@intel.com>
Cc: Michael Kinney <michael.d.kinney@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Hao Wu <hao.a.wu@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
---
 MdePkg/Library/BasePrintLib/PrintLibInternal.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/MdePkg/Library/BasePrintLib/PrintLibInternal.c b/MdePkg/Library/BasePrintLib/PrintLibInternal.c
index cec5b3bc99..28d946472f 100644
--- a/MdePkg/Library/BasePrintLib/PrintLibInternal.c
+++ b/MdePkg/Library/BasePrintLib/PrintLibInternal.c
@@ -1165,7 +1165,7 @@ BasePrintLibSPrintMarker (
     // Copy the string into the output buffer performing the required type conversions
     //
     while (Index < Count) {
-      ArgumentCharacter = ((*ArgumentString & 0xff) | (*(ArgumentString + 1) << 8)) & ArgumentMask;
+      ArgumentCharacter = ((*ArgumentString & 0xff) | (((UINT8)*(ArgumentString + 1)) << 8)) & ArgumentMask;
 
       LengthToReturn += (1 * BytesPerOutputCharacter);
       if ((Flags & COUNT_ONLY_NO_PRINT) == 0 && Buffer != NULL) {
-- 
2.12.0.windows.1

[PATCH v3 2/5] MdeModulePkg/PrintLib: Fix possible negative value left shift

[Hao Wu]

REF: https://bugzilla.tianocore.org/show_bug.cgi?id=702

Within function InternalPrintLibSPrintMarker(), possible left shift of a
negative value is found in:
"(*(ArgumentString + 1) << 8)"

which involves undefined behavior.

Since '*(ArgumentString + 1)' is of type CONST CHAR8 (signed), it will be
promoted to type int (signed) during the left shift operation. If
'*(ArgumentString + 1)' is a negative value, the behavior will be
undefined.

According to the C11 spec, Section 6.5.7:
> 4 The result of E1 << E2 is E1 left-shifted E2 bit positions; vacated
>   bits are filled with zeros. If E1 has an unsigned type, the value
>   of the result is E1 * 2^E2 , reduced modulo one more than the
>   maximum value representable in the result type. If E1 has a signed
>   type and nonnegative value, and E1 * 2^E2 is representable in the
>   result type, then that is the resulting value; otherwise, the
>   behavior is undefined.

This commit explicitly cast '*(ArgumentString + 1)' with UINT8 to resolve
this issue.

Cc: Steven Shi <steven.shi@intel.com>
Cc: Michael Kinney <michael.d.kinney@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Hao Wu <hao.a.wu@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
---
 MdeModulePkg/Library/DxePrintLibPrint2Protocol/PrintLib.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/MdeModulePkg/Library/DxePrintLibPrint2Protocol/PrintLib.c b/MdeModulePkg/Library/DxePrintLibPrint2Protocol/PrintLib.c
index b58db8e011..56534e56c3 100644
--- a/MdeModulePkg/Library/DxePrintLibPrint2Protocol/PrintLib.c
+++ b/MdeModulePkg/Library/DxePrintLibPrint2Protocol/PrintLib.c
@@ -2108,7 +2108,7 @@ InternalPrintLibSPrintMarker (
     // Copy the string into the output buffer performing the required type conversions
     //
     while (Index < Count) {
-      ArgumentCharacter = ((*ArgumentString & 0xff) | (*(ArgumentString + 1) << 8)) & ArgumentMask;
+      ArgumentCharacter = ((*ArgumentString & 0xff) | (((UINT8)*(ArgumentString + 1)) << 8)) & ArgumentMask;
 
       LengthToReturn += (1 * BytesPerOutputCharacter);
       if ((Flags & COUNT_ONLY_NO_PRINT) == 0 && Buffer != NULL) {
-- 
2.12.0.windows.1

[PATCH v3 3/5] MdeModulePkg/Tpl: Fix negative value left shift

[Hao Wu]

REF: https://bugzilla.tianocore.org/show_bug.cgi?id=695

Within function CoreRestoreTpl(), left shift a negative value -2 is used
in:
"while (((-2 << NewTpl) & gEventPending) != 0) {"

which involves undefined behavior.

According to the C11 spec, Section 6.5.7:
> 4 The result of E1 << E2 is E1 left-shifted E2 bit positions; vacated
>   bits are filled with zeros. If E1 has an unsigned type, the value
>   of the result is E1 * 2^E2 , reduced modulo one more than the
>   maximum value representable in the result type. If E1 has a signed
>   type and nonnegative value, and E1 * 2^E2 is representable in the
>   result type, then that is the resulting value; otherwise, the
>   behavior is undefined.

This commit refines the code logic to avoid left shifting the negative
value.

Cc: Steven Shi <steven.shi@intel.com>
Cc: Eric Dong <eric.dong@intel.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Michael Kinney <michael.d.kinney@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Hao Wu <hao.a.wu@intel.com>
Reviewed-by: Star Zeng <star.zeng@intel.com>
---
 MdeModulePkg/Core/Dxe/Event/Tpl.c | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/MdeModulePkg/Core/Dxe/Event/Tpl.c b/MdeModulePkg/Core/Dxe/Event/Tpl.c
index 8ad0a33701..e3caf832b8 100644
--- a/MdeModulePkg/Core/Dxe/Event/Tpl.c
+++ b/MdeModulePkg/Core/Dxe/Event/Tpl.c
@@ -1,7 +1,7 @@
 /** @file
   Task priority (TPL) functions.
 
-Copyright (c) 2006 - 2015, Intel Corporation. All rights reserved.<BR>
+Copyright (c) 2006 - 2017, Intel Corporation. All rights reserved.<BR>
 This program and the accompanying materials
 are licensed and made available under the terms and conditions of the BSD License
 which accompanies this distribution.  The full text of the license may be found at
@@ -103,6 +103,7 @@ CoreRestoreTpl (
   )
 {
   EFI_TPL     OldTpl;
+  EFI_TPL     PendingTpl;
 
   OldTpl = gEfiCurrentTpl;
   if (NewTpl > OldTpl) {
@@ -123,8 +124,13 @@ CoreRestoreTpl (
   //
   // Dispatch any pending events
   //
-  while (((-2 << NewTpl) & gEventPending) != 0) {
-    gEfiCurrentTpl = (UINTN) HighBitSet64 (gEventPending);
+  while (gEventPending != 0) {
+    PendingTpl = (UINTN) HighBitSet64 (gEventPending);
+    if (PendingTpl <= NewTpl) {
+      break;
+    }
+
+    gEfiCurrentTpl = PendingTpl;
     if (gEfiCurrentTpl < TPL_HIGH_LEVEL) {
       CoreSetInterruptState (TRUE);
     }
-- 
2.12.0.windows.1

[PATCH v3 4/5] MdeModulePkg/DxeNetLib: Fix negative value left shift

[Hao Wu]

REF: https://bugzilla.tianocore.org/show_bug.cgi?id=698

Within function NetRandomInitSeed(), left shift a negative value is used
in:
"~Time.Hour << 24"

which involves undefined behavior.

Since Time.Hour is of type UINT8 (range from 0 to 23), hence ~Time.Hour
will be a negative value (of type int, signed).

According to the C11 spec, Section 6.5.7:
> 4 The result of E1 << E2 is E1 left-shifted E2 bit positions; vacated
>   bits are filled with zeros. If E1 has an unsigned type, the value
>   of the result is E1 * 2^E2 , reduced modulo one more than the
>   maximum value representable in the result type. If E1 has a signed
>   type and nonnegative value, and E1 * 2^E2 is representable in the
>   result type, then that is the resulting value; otherwise, the
>   behavior is undefined.

This commit will remove the '~' operator before 'Time.Hour', since it
seems like an implementation choice for generating the seed.

Cc: Steven Shi <steven.shi@intel.com>
Cc: Fu Siyuan <siyuan.fu@intel.com>
Cc: Ye Ting <ting.ye@intel.com>
Cc: Wu Jiaxin <jiaxin.wu@intel.com>
Cc: Qin Long <qin.long@intel.com>
Cc: Star Zeng <star.zeng@intel.com>
Cc: Eric Dong <eric.dong@intel.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Hao Wu <hao.a.wu@intel.com>
---
 MdeModulePkg/Library/DxeNetLib/DxeNetLib.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/MdeModulePkg/Library/DxeNetLib/DxeNetLib.c b/MdeModulePkg/Library/DxeNetLib/DxeNetLib.c
index 7cd7e3aca0..1ee77fe036 100644
--- a/MdeModulePkg/Library/DxeNetLib/DxeNetLib.c
+++ b/MdeModulePkg/Library/DxeNetLib/DxeNetLib.c
@@ -872,7 +872,7 @@ NetRandomInitSeed (
   UINT64                    MonotonicCount;
 
   gRT->GetTime (&Time, NULL);
-  Seed = (~Time.Hour << 24 | Time.Day << 16 | Time.Minute << 8 | Time.Second);
+  Seed = (Time.Hour << 24 | Time.Day << 16 | Time.Minute << 8 | Time.Second);
   Seed ^= Time.Nanosecond;
   Seed ^= Time.Year << 7;
 
-- 
2.12.0.windows.1

Re: [PATCH v3 4/5] MdeModulePkg/DxeNetLib: Fix negative value left shift

[Wu, Jiaxin]

Reviewed-by: Wu Jiaxin <jiaxin.wu@intel.com>



> -----Original Message-----
> From: Wu, Hao A
> Sent: Thursday, September 28, 2017 12:32 PM
> To: edk2-devel@lists.01.org
> Cc: Wu, Hao A <hao.a.wu@intel.com>; Shi, Steven <steven.shi@intel.com>;
> Fu, Siyuan <siyuan.fu@intel.com>; Ye, Ting <ting.ye@intel.com>; Wu, Jiaxin
> <jiaxin.wu@intel.com>; Long, Qin <qin.long@intel.com>; Zeng, Star
> <star.zeng@intel.com>; Dong, Eric <eric.dong@intel.com>; Paolo Bonzini
> <pbonzini@redhat.com>
> Subject: [PATCH v3 4/5] MdeModulePkg/DxeNetLib: Fix negative value left
> shift
> 
> REF: https://bugzilla.tianocore.org/show_bug.cgi?id=698
> 
> Within function NetRandomInitSeed(), left shift a negative value is used
> in:
> "~Time.Hour << 24"
> 
> which involves undefined behavior.
> 
> Since Time.Hour is of type UINT8 (range from 0 to 23), hence ~Time.Hour
> will be a negative value (of type int, signed).
> 
> According to the C11 spec, Section 6.5.7:
> > 4 The result of E1 << E2 is E1 left-shifted E2 bit positions; vacated
> >   bits are filled with zeros. If E1 has an unsigned type, the value
> >   of the result is E1 * 2^E2 , reduced modulo one more than the
> >   maximum value representable in the result type. If E1 has a signed
> >   type and nonnegative value, and E1 * 2^E2 is representable in the
> >   result type, then that is the resulting value; otherwise, the
> >   behavior is undefined.
> 
> This commit will remove the '~' operator before 'Time.Hour', since it
> seems like an implementation choice for generating the seed.
> 
> Cc: Steven Shi <steven.shi@intel.com>
> Cc: Fu Siyuan <siyuan.fu@intel.com>
> Cc: Ye Ting <ting.ye@intel.com>
> Cc: Wu Jiaxin <jiaxin.wu@intel.com>
> Cc: Qin Long <qin.long@intel.com>
> Cc: Star Zeng <star.zeng@intel.com>
> Cc: Eric Dong <eric.dong@intel.com>
> Cc: Paolo Bonzini <pbonzini@redhat.com>
> Contributed-under: TianoCore Contribution Agreement 1.1
> Signed-off-by: Hao Wu <hao.a.wu@intel.com>
> ---
>  MdeModulePkg/Library/DxeNetLib/DxeNetLib.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/MdeModulePkg/Library/DxeNetLib/DxeNetLib.c
> b/MdeModulePkg/Library/DxeNetLib/DxeNetLib.c
> index 7cd7e3aca0..1ee77fe036 100644
> --- a/MdeModulePkg/Library/DxeNetLib/DxeNetLib.c
> +++ b/MdeModulePkg/Library/DxeNetLib/DxeNetLib.c
> @@ -872,7 +872,7 @@ NetRandomInitSeed (
>    UINT64                    MonotonicCount;
> 
>    gRT->GetTime (&Time, NULL);
> -  Seed = (~Time.Hour << 24 | Time.Day << 16 | Time.Minute << 8 |
> Time.Second);
> +  Seed = (Time.Hour << 24 | Time.Day << 16 | Time.Minute << 8 |
> Time.Second);
>    Seed ^= Time.Nanosecond;
>    Seed ^= Time.Year << 7;
> 
> --
> 2.12.0.windows.1

Re: [PATCH v3 4/5] MdeModulePkg/DxeNetLib: Fix negative value left shift

[Fu, Siyuan]

Looks good to me.

Reviewed-by: Fu Siyuan <siyuan.fu@intel.com>

-----Original Message-----
From: Wu, Hao A 
Sent: Thursday, September 28, 2017 12:32 PM
To: edk2-devel@lists.01.org
Cc: Wu, Hao A <hao.a.wu@intel.com>; Shi, Steven <steven.shi@intel.com>; Fu, Siyuan <siyuan.fu@intel.com>; Ye, Ting <ting.ye@intel.com>; Wu, Jiaxin <jiaxin.wu@intel.com>; Long, Qin <qin.long@intel.com>; Zeng, Star <star.zeng@intel.com>; Dong, Eric <eric.dong@intel.com>; Paolo Bonzini <pbonzini@redhat.com>
Subject: [PATCH v3 4/5] MdeModulePkg/DxeNetLib: Fix negative value left shift

REF: https://bugzilla.tianocore.org/show_bug.cgi?id=698

Within function NetRandomInitSeed(), left shift a negative value is used
in:
"~Time.Hour << 24"

which involves undefined behavior.

Since Time.Hour is of type UINT8 (range from 0 to 23), hence ~Time.Hour
will be a negative value (of type int, signed).

According to the C11 spec, Section 6.5.7:
> 4 The result of E1 << E2 is E1 left-shifted E2 bit positions; vacated
>   bits are filled with zeros. If E1 has an unsigned type, the value
>   of the result is E1 * 2^E2 , reduced modulo one more than the
>   maximum value representable in the result type. If E1 has a signed
>   type and nonnegative value, and E1 * 2^E2 is representable in the
>   result type, then that is the resulting value; otherwise, the
>   behavior is undefined.

This commit will remove the '~' operator before 'Time.Hour', since it
seems like an implementation choice for generating the seed.

Cc: Steven Shi <steven.shi@intel.com>
Cc: Fu Siyuan <siyuan.fu@intel.com>
Cc: Ye Ting <ting.ye@intel.com>
Cc: Wu Jiaxin <jiaxin.wu@intel.com>
Cc: Qin Long <qin.long@intel.com>
Cc: Star Zeng <star.zeng@intel.com>
Cc: Eric Dong <eric.dong@intel.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Hao Wu <hao.a.wu@intel.com>
---
 MdeModulePkg/Library/DxeNetLib/DxeNetLib.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/MdeModulePkg/Library/DxeNetLib/DxeNetLib.c b/MdeModulePkg/Library/DxeNetLib/DxeNetLib.c
index 7cd7e3aca0..1ee77fe036 100644
--- a/MdeModulePkg/Library/DxeNetLib/DxeNetLib.c
+++ b/MdeModulePkg/Library/DxeNetLib/DxeNetLib.c
@@ -872,7 +872,7 @@ NetRandomInitSeed (
   UINT64                    MonotonicCount;
 
   gRT->GetTime (&Time, NULL);
-  Seed = (~Time.Hour << 24 | Time.Day << 16 | Time.Minute << 8 | Time.Second);
+  Seed = (Time.Hour << 24 | Time.Day << 16 | Time.Minute << 8 | Time.Second);
   Seed ^= Time.Nanosecond;
   Seed ^= Time.Year << 7;
 
-- 
2.12.0.windows.1

Re: [PATCH v3 4/5] MdeModulePkg/DxeNetLib: Fix negative value left shift

[Ye, Ting]

Reviewed-by: Ye Ting <ting.ye@intel.com> 

-----Original Message-----
From: Wu, Hao A 
Sent: Thursday, September 28, 2017 12:32 PM
To: edk2-devel@lists.01.org
Cc: Wu, Hao A <hao.a.wu@intel.com>; Shi, Steven <steven.shi@intel.com>; Fu, Siyuan <siyuan.fu@intel.com>; Ye, Ting <ting.ye@intel.com>; Wu, Jiaxin <jiaxin.wu@intel.com>; Long, Qin <qin.long@intel.com>; Zeng, Star <star.zeng@intel.com>; Dong, Eric <eric.dong@intel.com>; Paolo Bonzini <pbonzini@redhat.com>
Subject: [PATCH v3 4/5] MdeModulePkg/DxeNetLib: Fix negative value left shift

REF: https://bugzilla.tianocore.org/show_bug.cgi?id=698

Within function NetRandomInitSeed(), left shift a negative value is used
in:
"~Time.Hour << 24"

which involves undefined behavior.

Since Time.Hour is of type UINT8 (range from 0 to 23), hence ~Time.Hour will be a negative value (of type int, signed).

According to the C11 spec, Section 6.5.7:
> 4 The result of E1 << E2 is E1 left-shifted E2 bit positions; vacated
>   bits are filled with zeros. If E1 has an unsigned type, the value
>   of the result is E1 * 2^E2 , reduced modulo one more than the
>   maximum value representable in the result type. If E1 has a signed
>   type and nonnegative value, and E1 * 2^E2 is representable in the
>   result type, then that is the resulting value; otherwise, the
>   behavior is undefined.

This commit will remove the '~' operator before 'Time.Hour', since it seems like an implementation choice for generating the seed.

Cc: Steven Shi <steven.shi@intel.com>
Cc: Fu Siyuan <siyuan.fu@intel.com>
Cc: Ye Ting <ting.ye@intel.com>
Cc: Wu Jiaxin <jiaxin.wu@intel.com>
Cc: Qin Long <qin.long@intel.com>
Cc: Star Zeng <star.zeng@intel.com>
Cc: Eric Dong <eric.dong@intel.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Hao Wu <hao.a.wu@intel.com>
---
 MdeModulePkg/Library/DxeNetLib/DxeNetLib.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/MdeModulePkg/Library/DxeNetLib/DxeNetLib.c b/MdeModulePkg/Library/DxeNetLib/DxeNetLib.c
index 7cd7e3aca0..1ee77fe036 100644
--- a/MdeModulePkg/Library/DxeNetLib/DxeNetLib.c
+++ b/MdeModulePkg/Library/DxeNetLib/DxeNetLib.c
@@ -872,7 +872,7 @@ NetRandomInitSeed (
   UINT64                    MonotonicCount;
 
   gRT->GetTime (&Time, NULL);
-  Seed = (~Time.Hour << 24 | Time.Day << 16 | Time.Minute << 8 | Time.Second);
+  Seed = (Time.Hour << 24 | Time.Day << 16 | Time.Minute << 8 | 
+ Time.Second);
   Seed ^= Time.Nanosecond;
   Seed ^= Time.Year << 7;
 
--
2.12.0.windows.1

[PATCH v3 5/5] MdeModulePkg/AtaAtapiPassThru: Fix possible out of range left shift

[Hao Wu]

REF: https://bugzilla.tianocore.org/show_bug.cgi?id=699

Within function AhciModeInitialization(), left shift operations of 'BIT0'
in the following statements:
"if ((PortImplementBitMap & (BIT0 << Port)) != 0) {"

will incur possible out of range left shift when Port is 31, since
"1 << 31" is possible to exceed the range of type 'int' (signed).

According to the C11 spec, Section 6.5.7:
> 4 The result of E1 << E2 is E1 left-shifted E2 bit positions; vacated
>   bits are filled with zeros. If E1 has an unsigned type, the value
>   of the result is E1 * 2^E2 , reduced modulo one more than the
>   maximum value representable in the result type. If E1 has a signed
>   type and nonnegative value, and E1 * 2^E2 is representable in the
>   result type, then that is the resulting value; otherwise, the
>   behavior is undefined.

This commit explicitly cast 'BIT0' with UINT32 to resolve this issue.

Cc: Steven Shi <steven.shi@intel.com>
Cc: Star Zeng <star.zeng@intel.com>
Cc: Eric Dong <eric.dong@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Hao Wu <hao.a.wu@intel.com>
---
 MdeModulePkg/Bus/Ata/AtaAtapiPassThru/AhciMode.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/MdeModulePkg/Bus/Ata/AtaAtapiPassThru/AhciMode.c b/MdeModulePkg/Bus/Ata/AtaAtapiPassThru/AhciMode.c
index b954de8101..e6de5d65bc 100644
--- a/MdeModulePkg/Bus/Ata/AtaAtapiPassThru/AhciMode.c
+++ b/MdeModulePkg/Bus/Ata/AtaAtapiPassThru/AhciMode.c
@@ -1,7 +1,7 @@
 /** @file
   The file for AHCI mode of ATA host controller.
 
-  Copyright (c) 2010 - 2016, Intel Corporation. All rights reserved.<BR>
+  Copyright (c) 2010 - 2017, Intel Corporation. All rights reserved.<BR>
   (C) Copyright 2015 Hewlett Packard Enterprise Development LP<BR>
   This program and the accompanying materials
   are licensed and made available under the terms and conditions of the BSD License
@@ -2314,7 +2314,7 @@ AhciModeInitialization (
   }
 
   for (Port = 0; Port < EFI_AHCI_MAX_PORTS; Port ++) {
-    if ((PortImplementBitMap & (BIT0 << Port)) != 0) {
+    if ((PortImplementBitMap & (((UINT32)BIT0) << Port)) != 0) {
       //
       // According to AHCI spec, MaxPortNumber should be equal or greater than the number of implemented ports.
       //
-- 
2.12.0.windows.1

Re: [PATCH v3 5/5] MdeModulePkg/AtaAtapiPassThru: Fix possible out of range left shift

[Zeng, Star]

Reviewed-by: Star Zeng <star.zeng@intel.com>

-----Original Message-----
From: Wu, Hao A 
Sent: Thursday, September 28, 2017 12:32 PM
To: edk2-devel@lists.01.org
Cc: Wu, Hao A <hao.a.wu@intel.com>; Shi, Steven <steven.shi@intel.com>; Zeng, Star <star.zeng@intel.com>; Dong, Eric <eric.dong@intel.com>
Subject: [PATCH v3 5/5] MdeModulePkg/AtaAtapiPassThru: Fix possible out of range left shift

REF: https://bugzilla.tianocore.org/show_bug.cgi?id=699

Within function AhciModeInitialization(), left shift operations of 'BIT0'
in the following statements:
"if ((PortImplementBitMap & (BIT0 << Port)) != 0) {"

will incur possible out of range left shift when Port is 31, since
"1 << 31" is possible to exceed the range of type 'int' (signed).

According to the C11 spec, Section 6.5.7:
> 4 The result of E1 << E2 is E1 left-shifted E2 bit positions; vacated
>   bits are filled with zeros. If E1 has an unsigned type, the value
>   of the result is E1 * 2^E2 , reduced modulo one more than the
>   maximum value representable in the result type. If E1 has a signed
>   type and nonnegative value, and E1 * 2^E2 is representable in the
>   result type, then that is the resulting value; otherwise, the
>   behavior is undefined.

This commit explicitly cast 'BIT0' with UINT32 to resolve this issue.

Cc: Steven Shi <steven.shi@intel.com>
Cc: Star Zeng <star.zeng@intel.com>
Cc: Eric Dong <eric.dong@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Hao Wu <hao.a.wu@intel.com>
---
 MdeModulePkg/Bus/Ata/AtaAtapiPassThru/AhciMode.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/MdeModulePkg/Bus/Ata/AtaAtapiPassThru/AhciMode.c b/MdeModulePkg/Bus/Ata/AtaAtapiPassThru/AhciMode.c
index b954de8101..e6de5d65bc 100644
--- a/MdeModulePkg/Bus/Ata/AtaAtapiPassThru/AhciMode.c
+++ b/MdeModulePkg/Bus/Ata/AtaAtapiPassThru/AhciMode.c
@@ -1,7 +1,7 @@
 /** @file
   The file for AHCI mode of ATA host controller.
 
-  Copyright (c) 2010 - 2016, Intel Corporation. All rights reserved.<BR>
+  Copyright (c) 2010 - 2017, Intel Corporation. All rights 
+ reserved.<BR>
   (C) Copyright 2015 Hewlett Packard Enterprise Development LP<BR>
   This program and the accompanying materials
   are licensed and made available under the terms and conditions of the BSD License @@ -2314,7 +2314,7 @@ AhciModeInitialization (
   }
 
   for (Port = 0; Port < EFI_AHCI_MAX_PORTS; Port ++) {
-    if ((PortImplementBitMap & (BIT0 << Port)) != 0) {
+    if ((PortImplementBitMap & (((UINT32)BIT0) << Port)) != 0) {
       //
       // According to AHCI spec, MaxPortNumber should be equal or greater than the number of implemented ports.
       //
--
2.12.0.windows.1