From: Laszlo Ersek <lersek@redhat.com>
To: edk2-devel-01 <edk2-devel@lists.01.org>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>,
Liming Gao <liming.gao@intel.com>,
Michael D Kinney <michael.d.kinney@intel.com>,
Sean Brogan <sean.brogan@microsoft.com>
Subject: [PATCH 4/4] MdePkg/BaseSafeIntLib: fix undefined behavior in SafeInt64Mult()
Date: Thu, 15 Feb 2018 19:36:38 +0100 [thread overview]
Message-ID: <20180215183638.18578-5-lersek@redhat.com> (raw)
In-Reply-To: <20180215183638.18578-1-lersek@redhat.com>
If we have to negate UnsignedResult (due to exactly one of Multiplicand
and Multiplier being negative), and UnsignedResult is exactly
MIN_INT64_MAGNITUDE (value 2^63), then the statement
*Result = - ((INT64)UnsignedResult);
invokes both implementation-defined behavior and undefined behavior.
First, MIN_INT64_MAGNITUDE is not representable as INT64, therefore the
result of the (inner) conversion
(INT64)MIN_INT64_MAGNITUDE
is implementation-defined, or an implementation-defined signal is raised,
according to ISO C99 6.3.1.3p3.
Second, if we assume that the C language implementation defines the
conversion to INT64 simply as reinterpreting the bit pattern
0x8000_0000_0000_0000 as a signed integer in two's complement
representation, then the conversion immediately produces the negative
value MIN_INT64 (value -(2^63)). In turn, the (outer) negation
-(MIN_INT64)
invokes undefined behavior, because the mathematical result of the
negation, namely 2^63, cannot be represented in an INT64 object. (Not even
mentioning the fact that the mathematical result would be incorrect.) In
practice, the undefined negation of MIN_INT64 happens to produce an
unchanged, valid-looking result on x86, i.e. (-(MIN_INT64)) == MIN_INT64.
We can summarize this as the undefined -- effectless -- negation canceling
out the botched -- auto-negating -- implementation-defined conversion.
Instead of relying on such behavior, dedicate a branch to this situation:
assign MIN_INT64 directly. The branch can be triggered e.g. by multiplying
(2^62) by (-2).
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
---
MdePkg/Library/BaseSafeIntLib/SafeIntLib.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/MdePkg/Library/BaseSafeIntLib/SafeIntLib.c b/MdePkg/Library/BaseSafeIntLib/SafeIntLib.c
index de91ffeca2a5..c5f13d7e0828 100644
--- a/MdePkg/Library/BaseSafeIntLib/SafeIntLib.c
+++ b/MdePkg/Library/BaseSafeIntLib/SafeIntLib.c
@@ -4143,6 +4143,8 @@ SafeInt64Mult (
if (UnsignedResult > MIN_INT64_MAGNITUDE) {
*Result = INT64_ERROR;
Status = RETURN_BUFFER_TOO_SMALL;
+ } else if (UnsignedResult == MIN_INT64_MAGNITUDE) {
+ *Result = MIN_INT64;
} else {
*Result = - ((INT64)UnsignedResult);
}
--
2.14.1.3.gb7cf6e02401b
next prev parent reply other threads:[~2018-02-15 18:30 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-02-15 18:36 [PATCH 0/4] MdePkg/BaseSafeIntLib: fix undefined behavior in INT64 Sub/Add/Mult Laszlo Ersek
2018-02-15 18:36 ` [PATCH 1/4] MdePkg/BaseSafeIntLib: fix undefined behavior in SafeInt64Sub() Laszlo Ersek
2018-02-15 18:36 ` [PATCH 2/4] MdePkg/BaseSafeIntLib: fix undefined behavior in SafeInt64Add() Laszlo Ersek
2018-02-15 18:36 ` [PATCH 3/4] MdePkg/BaseSafeIntLib: clean up parentheses in MIN_INT64_MAGNITUDE Laszlo Ersek
2018-02-15 18:36 ` Laszlo Ersek [this message]
2018-02-16 11:28 ` [PATCH 0/4] MdePkg/BaseSafeIntLib: fix undefined behavior in INT64 Sub/Add/Mult Ard Biesheuvel
2018-02-16 20:44 ` Laszlo Ersek
2018-02-16 18:11 ` Kinney, Michael D
2018-02-16 20:49 ` Laszlo Ersek
2018-02-17 3:07 ` Kinney, Michael D
2018-02-21 11:00 ` Laszlo Ersek
2018-02-21 18:10 ` Ard Biesheuvel
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-list from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180215183638.18578-5-lersek@redhat.com \
--to=devel@edk2.groups.io \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox