From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=66.187.233.73; helo=mx1.redhat.com; envelope-from=lersek@redhat.com; receiver=edk2-devel@lists.01.org Received: from mx1.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 4F4F4220F33C2 for ; Thu, 15 Feb 2018 10:30:51 -0800 (PST) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B63444023141; Thu, 15 Feb 2018 18:36:43 +0000 (UTC) Received: from lacos-laptop-7.usersys.redhat.com (ovpn-120-84.rdu2.redhat.com [10.10.120.84]) by smtp.corp.redhat.com (Postfix) with ESMTP id A4DB02017DE3; Thu, 15 Feb 2018 18:36:42 +0000 (UTC) From: Laszlo Ersek To: edk2-devel-01 Cc: Bret Barkelew , Liming Gao , Michael D Kinney , Sean Brogan Date: Thu, 15 Feb 2018 19:36:35 +0100 Message-Id: <20180215183638.18578-2-lersek@redhat.com> In-Reply-To: <20180215183638.18578-1-lersek@redhat.com> References: <20180215183638.18578-1-lersek@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Thu, 15 Feb 2018 18:36:43 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Thu, 15 Feb 2018 18:36:43 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'lersek@redhat.com' RCPT:'' Subject: [PATCH 1/4] MdePkg/BaseSafeIntLib: fix undefined behavior in SafeInt64Sub() X-BeenThere: edk2-devel@lists.01.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: EDK II Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Feb 2018 18:30:51 -0000 The subtraction in the assignment SignedResult = Minuend - Subtrahend; is performed with unchecked INT64 operands. According to ISO C, if the mathematical result of signed integer subtraction cannot be represented in the result type, the behavior is undefined. (Refer to ISO C99 6.5p5. 6.2.5p9 only exempts unsigned integers, and 6.3.1.3p3 does not apply because it treats the conversion of integers that have been successfully evaluated first.) Replace the after-the-fact result checking with checks on the operands, and only perform the subtraction if it is safe. Cc: Bret Barkelew Cc: Liming Gao Cc: Michael D Kinney Cc: Sean Brogan Contributed-under: TianoCore Contribution Agreement 1.1 Signed-off-by: Laszlo Ersek --- MdePkg/Library/BaseSafeIntLib/SafeIntLib.c | 50 +++++++++++++++----- 1 file changed, 39 insertions(+), 11 deletions(-) diff --git a/MdePkg/Library/BaseSafeIntLib/SafeIntLib.c b/MdePkg/Library/BaseSafeIntLib/SafeIntLib.c index d846160ba0d1..8e857927b067 100644 --- a/MdePkg/Library/BaseSafeIntLib/SafeIntLib.c +++ b/MdePkg/Library/BaseSafeIntLib/SafeIntLib.c @@ -3837,27 +3837,55 @@ SafeInt64Sub ( ) { RETURN_STATUS Status; - INT64 SignedResult; if (Result == NULL) { return RETURN_INVALID_PARAMETER; } - SignedResult = Minuend - Subtrahend; - // - // Subtracting a positive number from a positive number never overflows. - // Subtracting a negative number from a negative number never overflows. - // If you subtract a negative number from a positive number, you expect a positive result. - // If you subtract a positive number from a negative number, you expect a negative result. - // Overflow if inputs vary in sign and the output does not have the same sign as the first input. + // * A Subtrahend of zero can never cause underflow or overflow. // - if (((Minuend < 0) != (Subtrahend < 0)) && - ((Minuend < 0) != (SignedResult < 0))) { + // * A positive Subtrahend can only cause underflow. The underflow condition + // is: + // + // (Minuend - Subtrahend) < MIN_INT64 + // + // Adding Subtrahend to both sides yields + // + // Minuend < (MIN_INT64 + Subtrahend) + // + // This condition can be coded directly in C because the RHS will neither + // underflow nor overflow. That is due to the starting condition: + // + // 0 < Subtrahend <= MAX_INT64 + // + // Adding MIN_INT64 to all three sides yields + // + // MIN_INT64 < (MIN_INT64 + Subtrahend) <= (MIN_INT64 + MAX_INT64) = -1 + // + // * A negative Subtrahend can only cause overflow. The overflow condition is + // + // (Minuend - Subtrahend) > MAX_INT64 + // + // Adding Subtrahend to both sides yields + // + // Minuend > (MAX_INT64 + Subtrahend) + // + // This condition can be coded directly in C because the RHS will neither + // underflow nor overflow. That is due to the starting condition: + // + // MIN_INT64 <= Subtrahend < 0 + // + // Adding MAX_INT64 to all three sides yields + // + // -1 = (MAX_INT64 + MIN_INT64) <= (MAX_INT64 + Subtrahend) < MAX_INT64 + // + if (((Subtrahend > 0) && (Minuend < (MIN_INT64 + Subtrahend))) || + ((Subtrahend < 0) && (Minuend > (MAX_INT64 + Subtrahend)))) { *Result = INT64_ERROR; Status = RETURN_BUFFER_TOO_SMALL; } else { - *Result = SignedResult; + *Result = Minuend - Subtrahend; Status = RETURN_SUCCESS; } -- 2.14.1.3.gb7cf6e02401b