From patchwork Fri Sep 19 01:40:03 2014
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
X-Patchwork-Submitter: Joseph Myers <joseph@codesourcery.com>
X-Patchwork-Id: 2914
Received: (qmail 20851 invoked by alias); 19 Sep 2014 01:40:10 -0000
Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm
Precedence: bulk
List-Id: <libc-alpha.sourceware.org>
List-Unsubscribe: <mailto:libc-alpha-unsubscribe-##L=##H@sourceware.org>
List-Subscribe: <mailto:libc-alpha-subscribe@sourceware.org>
List-Archive: <http://sourceware.org/ml/libc-alpha/>
List-Post: <mailto:libc-alpha@sourceware.org>
List-Help: <mailto:libc-alpha-help@sourceware.org>,
	<http://sourceware.org/ml/#faqs>
Sender: libc-alpha-owner@sourceware.org
Delivered-To: mailing list libc-alpha@sourceware.org
Received: (qmail 20840 invoked by uid 89); 19 Sep 2014 01:40:10 -0000
Authentication-Results: sourceware.org; auth=none
X-Virus-Found: No
X-Spam-SWARE-Status: No, score=-1.9 required=5.0 tests=AWL,
	BAYES_00 autolearn=ham version=3.3.2
X-HELO: relay1.mentorg.com
Date: Fri, 19 Sep 2014 01:40:03 +0000
From: "Joseph S. Myers" <joseph@codesourcery.com>
To: <libc-alpha@sourceware.org>
Subject: soft-fp: Fix _FP_TO_INT latent bug in overflow handling
Message-ID: <Pine.LNX.4.64.1409190139150.20342@digraph.polyomino.org.uk>
MIME-Version: 1.0

This patch, relative to a tree with
<https://sourceware.org/ml/libc-alpha/2014-09/msg00461.html> and
<https://sourceware.org/ml/libc-alpha/2014-09/msg00463.html> (pending
review) applied, fixes a latent bug in _FP_TO_INT regarding handling
of arguments with maximum exponent (infinities and NaNs).  If the
maximum exponent is below that calculated as an overflow threshold,
such values would incorrectly be treated as normal values for the
purposes of the conversion.  This could not occur for any of the
conversions actually occurring in glibc, libgcc or the Linux kernel
(the maximum exponent for float is, just, big enough to ensure
overflow for unsigned __int128), but would apply if soft-fp were used
for IEEE binary16.  Appropriate checks are inserted to ensure that the
maximum exponent is always treated as an overflowing exponent, and
never as a normal one.

Tested for powerpc-nofpu that the disassembly of installed shared
libraries is unchanged by this patch.

2014-09-19  Joseph Myers  <joseph@codesourcery.com>

	* soft-fp/op-common.h (_FP_TO_INT): Ensure maximum exponent is
	treated as invalid conversion, not as normal exponent.

diff --git a/soft-fp/op-common.h b/soft-fp/op-common.h
index 833658d..8c928af 100644
--- a/soft-fp/op-common.h
+++ b/soft-fp/op-common.h
@@ -1381,7 +1381,9 @@
 	  else								\
 	    FP_SET_EXCEPTION (FP_EX_INEXACT);				\
 	}								\
-      else if (X##_e >= _FP_EXPBIAS_##fs + rsize - (rsigned > 0 || X##_s) \
+      else if (X##_e >= (_FP_EXPMAX_##fs < _FP_EXPBIAS_##fs + rsize	\
+			 ? _FP_EXPMAX_##fs				\
+			 : _FP_EXPBIAS_##fs + rsize - (rsigned > 0 || X##_s)) \
 	       || (!rsigned && X##_s))					\
 	{								\
 	  /* Overflow or converting to the most negative integer.  */	\
@@ -1398,7 +1400,10 @@
 		r = ~r;							\
 	    }								\
 									\
-	  if (rsigned && X##_s && X##_e == _FP_EXPBIAS_##fs + rsize - 1) \
+	  if (_FP_EXPBIAS_##fs + rsize - 1 < _FP_EXPMAX_##fs		\
+	      && rsigned						\
+	      && X##_s							\
+	      && X##_e == _FP_EXPBIAS_##fs + rsize - 1)			\
 	    {								\
 	      /* Possibly converting to most negative integer; check the \
 		 mantissa.  */						\