From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=66.187.233.73; helo=mx1.redhat.com; envelope-from=lersek@redhat.com; receiver=edk2-devel@lists.01.org Received: from mx1.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 3373B22161150 for ; Thu, 15 Feb 2018 10:30:56 -0800 (PST) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 99E9D4072467; Thu, 15 Feb 2018 18:36:48 +0000 (UTC) Received: from lacos-laptop-7.usersys.redhat.com (ovpn-120-84.rdu2.redhat.com [10.10.120.84]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2B4AE200BC16; Thu, 15 Feb 2018 18:36:47 +0000 (UTC) From: Laszlo Ersek To: edk2-devel-01 Cc: Bret Barkelew , Liming Gao , Michael D Kinney , Sean Brogan Date: Thu, 15 Feb 2018 19:36:38 +0100 Message-Id: <20180215183638.18578-5-lersek@redhat.com> In-Reply-To: <20180215183638.18578-1-lersek@redhat.com> References: <20180215183638.18578-1-lersek@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Thu, 15 Feb 2018 18:36:48 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Thu, 15 Feb 2018 18:36:48 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'lersek@redhat.com' RCPT:'' Subject: [PATCH 4/4] MdePkg/BaseSafeIntLib: fix undefined behavior in SafeInt64Mult() X-BeenThere: edk2-devel@lists.01.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: EDK II Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Feb 2018 18:30:56 -0000 If we have to negate UnsignedResult (due to exactly one of Multiplicand and Multiplier being negative), and UnsignedResult is exactly MIN_INT64_MAGNITUDE (value 2^63), then the statement *Result = - ((INT64)UnsignedResult); invokes both implementation-defined behavior and undefined behavior. First, MIN_INT64_MAGNITUDE is not representable as INT64, therefore the result of the (inner) conversion (INT64)MIN_INT64_MAGNITUDE is implementation-defined, or an implementation-defined signal is raised, according to ISO C99 6.3.1.3p3. Second, if we assume that the C language implementation defines the conversion to INT64 simply as reinterpreting the bit pattern 0x8000_0000_0000_0000 as a signed integer in two's complement representation, then the conversion immediately produces the negative value MIN_INT64 (value -(2^63)). In turn, the (outer) negation -(MIN_INT64) invokes undefined behavior, because the mathematical result of the negation, namely 2^63, cannot be represented in an INT64 object. (Not even mentioning the fact that the mathematical result would be incorrect.) In practice, the undefined negation of MIN_INT64 happens to produce an unchanged, valid-looking result on x86, i.e. (-(MIN_INT64)) == MIN_INT64. We can summarize this as the undefined -- effectless -- negation canceling out the botched -- auto-negating -- implementation-defined conversion. Instead of relying on such behavior, dedicate a branch to this situation: assign MIN_INT64 directly. The branch can be triggered e.g. by multiplying (2^62) by (-2). Cc: Bret Barkelew Cc: Liming Gao Cc: Michael D Kinney Cc: Sean Brogan Contributed-under: TianoCore Contribution Agreement 1.1 Signed-off-by: Laszlo Ersek --- MdePkg/Library/BaseSafeIntLib/SafeIntLib.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/MdePkg/Library/BaseSafeIntLib/SafeIntLib.c b/MdePkg/Library/BaseSafeIntLib/SafeIntLib.c index de91ffeca2a5..c5f13d7e0828 100644 --- a/MdePkg/Library/BaseSafeIntLib/SafeIntLib.c +++ b/MdePkg/Library/BaseSafeIntLib/SafeIntLib.c @@ -4143,6 +4143,8 @@ SafeInt64Mult ( if (UnsignedResult > MIN_INT64_MAGNITUDE) { *Result = INT64_ERROR; Status = RETURN_BUFFER_TOO_SMALL; + } else if (UnsignedResult == MIN_INT64_MAGNITUDE) { + *Result = MIN_INT64; } else { *Result = - ((INT64)UnsignedResult); } -- 2.14.1.3.gb7cf6e02401b