Increase internal precision of ldbl-128ibm decimal printf

  When the signs differ, the precision of the conversion
could drop below 106 bits.  This strategy is
identical to the hexadecimal variant.

0x1.ffffffffffffffffffffffffff80p111L is an example of
a number which should, but does not round trip through
printf as a decimal number when printing with 32 digits
of precision.

Note, the printf test did not pick this up, as it
only tests up to 30 digits.  Long double can have
more at nominal precision.

2016-03-17  Paul E. Murphy  <murphyp@linux.vnet.ibm.com>

	* sysdeps/ieee754/ldbl-128ibm/ldbl2pm.c:
	(__mpn_extract_long_double): Carry 7 extra intermediate
	bits of precision to aide computing difference when
	signs differ.
---
 sysdeps/ieee754/ldbl-128ibm/ldbl2mpn.c | 21 +++++++++++++--------
 1 file changed, 13 insertions(+), 8 deletions(-)
  

"Paul E. Murphy" <murphyp@linux.vnet.ibm.com> writes:

> +  /* Hold 7 extra bits of precision in the mantissa.  This allows
> +     the normalizing shifts below to prevent losing precision when
> +     the signs differ and the exponents are sufficiently far apart.  */
> +  lo <<= 7;

That magical number should probably get a name.

>  	  /* we have a borrow from the hidden bit, so shift left 1.  */
> -	  hi = 0x0ffffffffffffeLL | (lo >> 51);
> -	  lo = 0x1fffffffffffffLL & (lo << 1);
> +	  hi = 0x000ffffffffffffeLL | (lo >> (52 + 7));
> +	  lo = 0x0fffffffffffffffLL & (lo << 1);

Not 51 + 7?

Andreas.
  

On Thu, 17 Mar 2016, Paul E. Murphy wrote:

> When the signs differ, the precision of the conversion
> could drop below 106 bits.  This strategy is
> identical to the hexadecimal variant.

I assume this was user-visible in a release, in which case it should have 
a bug filed in Bugzilla (if it doesn't already have one - I take it this 
is distinct from bug 5268?).

It should also have a testcase included in the patch (there are several 
tests in sysdeps/ieee754/ldbl-128ibm already for issues specific to this 
format - various of which use unions to test issues that can't be tested 
simply with 106-bit inputs GCC can represent, or where it's necessary to 
examine the two halves of a long double result separately).
  

On 03/17/2016 04:56 PM, Andreas Schwab wrote:
> "Paul E. Murphy" <murphyp@linux.vnet.ibm.com> writes:
> 
>> +  /* Hold 7 extra bits of precision in the mantissa.  This allows
>> +     the normalizing shifts below to prevent losing precision when
>> +     the signs differ and the exponents are sufficiently far apart.  */
>> +  lo <<= 7;
> 
> That magical number should probably get a name.

Suggestions? It was noted as an "IEEE 854 adjustment" constant in
printf_fphex.c, and otherwise integrated into the shifts later on.

INTERNAL_PRECISION_SHIFT? Setting it 1 should fix the I case noted
in the comments.

> 
>>  	  /* we have a borrow from the hidden bit, so shift left 1.  */
>> -	  hi = 0x0ffffffffffffeLL | (lo >> 51);
>> -	  lo = 0x1fffffffffffffLL & (lo << 1);
>> +	  hi = 0x000ffffffffffffeLL | (lo >> (52 + 7));
>> +	  lo = 0x0fffffffffffffffLL & (lo << 1);
> 
> Not 51 + 7?

That is intentional.  It should be moving the high bit of lo
lo into position 0.  This is identical to how the ldbl-128ibm
printf_fphex.c, which does round trip these numbers correctly.

I.e ((1<<53)-1)>>51 == 3.  The implicit bit is already factored
into lo, so the existing shift appears wrong here. 

> 
> Andreas.
>
  

On 03/17/2016 05:16 PM, Joseph Myers wrote:
> On Thu, 17 Mar 2016, Paul E. Murphy wrote:
> 
>> When the signs differ, the precision of the conversion
>> could drop below 106 bits.  This strategy is
>> identical to the hexadecimal variant.
> 
> I assume this was user-visible in a release, in which case it should have 
> a bug filed in Bugzilla (if it doesn't already have one - I take it this 
> is distinct from bug 5268?).
> 
> It should also have a testcase included in the patch (there are several 
> tests in sysdeps/ieee754/ldbl-128ibm already for issues specific to this 
> format - various of which use unions to test issues that can't be tested 
> simply with 106-bit inputs GCC can represent, or where it's necessary to 
> examine the two halves of a long double result separately).
> 

This came up internally. I think it is similar, the comments would indicate
it is a 1 bit internal precision loss which this patch does address.

I will figure out how to update the tests to catch this, and give the magic
shift amount a name.

Thanks,
Paul