Fix i386 memmove issue [BZ #22644]

  2018-03-14 15:59 GMT+01:00 H.J. Lu <hjl.tools@gmail.com>:
> On Wed, Mar 14, 2018 at 7:43 AM, Andrew Senkevich
> <andrew.n.senkevich@gmail.com> wrote:
>> 2018-02-19 11:13 GMT+01:00 Andreas Schwab <schwab@suse.de>:
>>> On Feb 19 2018, Andrew Senkevich <andrew.n.senkevich@gmail.com> wrote:
>>>
>>>> diff --git a/string/test-memmove.c b/string/test-memmove.c
>>>> index edc7a4c..8dc152b
>>>> --- a/string/test-memmove.c
>>>> +++ b/string/test-memmove.c
>>>> @@ -245,6 +245,49 @@ do_random_tests (void)
>>>>      }
>>>>  }
>>>>
>>>> +#if __SIZEOF_POINTER__ == 4
>>>> +static void
>>>> +do_test2 (void)
>>>> +{
>>>> +  uint32_t i;
>>>> +  uint32_t num = 0x20000000;
>>>> +  uint32_t * large_buf = mmap (0, sizeof(uint32_t) * num, PROT_READ |
>>>> PROT_WRITE,
>>>> +        MAP_PRIVATE | MAP_ANON, -1, 0);
>>>> +  if (large_buf == MAP_FAILED)
>>>> +    error (EXIT_FAILURE, errno, "large mmap failed");
>>>> +
>>>> +  if (!((uint32_t)(large_buf) < (0x80000000 - 128) && (0x80000000 +
>>>> 128) < (uint32_t)(&large_buf[num])))
>>>> +    {
>>>> +      error (0, 0,"allocated large memory doesn't cross 0x80000000 boundary");
>>>> +      ret = 1;
>>>> +      return;
>>>
>>> Please properly fold long lines, and remove the redundant parens.  Also,
>>> there is no guarantee that the address range is unallocated.
>>
>> Thanks, updated patch below.  Any comment or it is Ok for trunk?
>
> Please also test crossing 0x80000000 boundary on 64-bit systems.

Hi,

I extended test for 64-bit using MAP_FIXED which lets to hardcode
allocated address.  Manual says it is less portable, but without
MAP_FIXED I was unable to get proper address for 64bits.
I checked what test fails without fix of memcpy-sse2-unaligned
implementation.  Is it Ok for trunk?



--
WBR,
Andrew
  

On Mon, Mar 19, 2018 at 5:45 AM, Andrew Senkevich
<andrew.n.senkevich@gmail.com> wrote:
> 2018-03-14 15:59 GMT+01:00 H.J. Lu <hjl.tools@gmail.com>:
>> On Wed, Mar 14, 2018 at 7:43 AM, Andrew Senkevich
>> <andrew.n.senkevich@gmail.com> wrote:
>>> 2018-02-19 11:13 GMT+01:00 Andreas Schwab <schwab@suse.de>:
>>>> On Feb 19 2018, Andrew Senkevich <andrew.n.senkevich@gmail.com> wrote:
>>>>
>>>>> diff --git a/string/test-memmove.c b/string/test-memmove.c
>>>>> index edc7a4c..8dc152b
>>>>> --- a/string/test-memmove.c
>>>>> +++ b/string/test-memmove.c
>>>>> @@ -245,6 +245,49 @@ do_random_tests (void)
>>>>>      }
>>>>>  }
>>>>>
>>>>> +#if __SIZEOF_POINTER__ == 4
>>>>> +static void
>>>>> +do_test2 (void)
>>>>> +{
>>>>> +  uint32_t i;
>>>>> +  uint32_t num = 0x20000000;
>>>>> +  uint32_t * large_buf = mmap (0, sizeof(uint32_t) * num, PROT_READ |
>>>>> PROT_WRITE,
>>>>> +        MAP_PRIVATE | MAP_ANON, -1, 0);
>>>>> +  if (large_buf == MAP_FAILED)
>>>>> +    error (EXIT_FAILURE, errno, "large mmap failed");
>>>>> +
>>>>> +  if (!((uint32_t)(large_buf) < (0x80000000 - 128) && (0x80000000 +
>>>>> 128) < (uint32_t)(&large_buf[num])))
>>>>> +    {
>>>>> +      error (0, 0,"allocated large memory doesn't cross 0x80000000 boundary");
>>>>> +      ret = 1;
>>>>> +      return;
>>>>
>>>> Please properly fold long lines, and remove the redundant parens.  Also,
>>>> there is no guarantee that the address range is unallocated.
>>>
>>> Thanks, updated patch below.  Any comment or it is Ok for trunk?
>>
>> Please also test crossing 0x80000000 boundary on 64-bit systems.
>
> Hi,
>
> I extended test for 64-bit using MAP_FIXED which lets to hardcode
> allocated address.  Manual says it is less portable, but without
> MAP_FIXED I was unable to get proper address for 64bits.
> I checked what test fails without fix of memcpy-sse2-unaligned
> implementation.  Is it Ok for trunk?
>
> diff --git a/string/test-memmove.c b/string/test-memmove.c
> index edc7a4c..5920652 100644
> --- a/string/test-memmove.c
> +++ b/string/test-memmove.c
> @@ -24,6 +24,7 @@
>  # define TEST_NAME "memmove"
>  #endif
>  #include "test-string.h"
> +#include <support/test-driver.h>
>
>  char *simple_memmove (char *, const char *, size_t);
>
> @@ -245,6 +246,57 @@ do_random_tests (void)
>      }
>  }
>
> +#if __SIZEOF_POINTER__ == 4
> +# define ptr_type uint32_t
> +#else
> +# define ptr_type uint64_t
> +#endif

Please use uintptr_t instead.

> +static void
> +do_test2 (void)
> +{
> +  uint32_t num = 0x20000000;
> +  uint32_t * large_buf;
> +
> +  large_buf = mmap ((void*)0x70000000, num, PROT_READ | PROT_WRITE,
> +     MAP_PRIVATE | MAP_ANON | MAP_FIXED, -1, 0);
> +
> +  if (large_buf == MAP_FAILED)
> +    error (EXIT_FAILURE, errno, "Large mmap failed");

Please EXIT_UNSUPPORTED.

> +  uint32_t bytes_move = 0x80000000 - (ptr_type)large_buf;
> +  uint32_t arr_size = bytes_move / sizeof(uint32_t);
> +  uint32_t i;
> +
> +  FOR_EACH_IMPL (impl, 0)
> +    {
> +      for (i = 0; i < arr_size; i++)
> +        large_buf[i] = i;
> +
> +      uint32_t * dst = &large_buf[33];
> +
> +#ifdef TEST_BCOPY
> +      CALL (impl, (char *)large_buf, (char *)dst, bytes_move);
> +#else
> +      CALL (impl, (char *)dst, (char *)large_buf, bytes_move);
> +#endif
> +
> +      for (i = 0; i < arr_size; i++)
> +        {
> +          if (dst[i] != i)
> +     {
> +       error (0, 0,
> +      "Wrong result in function %s dst \"%p\" src \"%p\" offset \"%d\"",
> +      impl->name, dst, large_buf, i);
> +       ret = 1;
> +       break;
> +     }
> + }
> +    }
> +
> +  munmap((void *)large_buf, sizeof(uint32_t) * num);
> +}
> +
>  int
>  test_main (void)
>  {
> @@ -284,6 +336,9 @@ test_main (void)
>      }
>
>    do_random_tests ();
> +
> +  do_test2 ();
> +
>    return ret;
>  }
>
> diff --git a/sysdeps/i386/i686/multiarch/memcpy-sse2-unaligned.S
> b/sysdeps/i386/i686/multiarch/memcpy-sse2-unaligned.S
> index 9c3bbe7..9aa17de 100644
> --- a/sysdeps/i386/i686/multiarch/memcpy-sse2-unaligned.S
> +++ b/sysdeps/i386/i686/multiarch/memcpy-sse2-unaligned.S
> @@ -72,7 +72,7 @@ ENTRY (MEMCPY)
>   cmp %edx, %eax
>
>  # ifdef USE_AS_MEMMOVE
> - jg L(check_forward)
> + ja L(check_forward)
>
>  L(mm_len_0_or_more_backward):
>  /* Now do checks for lengths. We do [0..16], [16..32], [32..64], [64..128]
> @@ -81,7 +81,7 @@ L(mm_len_0_or_more_backward):
>   jbe L(mm_len_0_16_bytes_backward)
>
>   cmpl $32, %ecx
> - jg L(mm_len_32_or_more_backward)
> + ja L(mm_len_32_or_more_backward)
>
>  /* Copy [0..32] and return.  */
>   movdqu (%eax), %xmm0
> @@ -92,7 +92,7 @@ L(mm_len_0_or_more_backward):
>
>  L(mm_len_32_or_more_backward):
>   cmpl $64, %ecx
> - jg L(mm_len_64_or_more_backward)
> + ja L(mm_len_64_or_more_backward)
>
>  /* Copy [0..64] and return.  */
>   movdqu (%eax), %xmm0
> @@ -107,7 +107,7 @@ L(mm_len_32_or_more_backward):
>
>  L(mm_len_64_or_more_backward):
>   cmpl $128, %ecx
> - jg L(mm_len_128_or_more_backward)
> + ja L(mm_len_128_or_more_backward)
>
>  /* Copy [0..128] and return.  */
>   movdqu (%eax), %xmm0
> @@ -132,7 +132,7 @@ L(mm_len_128_or_more_backward):
>   add %ecx, %eax
>   cmp %edx, %eax
>   movl SRC(%esp), %eax
> - jle L(forward)
> + jbe L(forward)
>   PUSH (%esi)
>   PUSH (%edi)
>   PUSH (%ebx)
> @@ -269,7 +269,7 @@ L(check_forward):
>   add %edx, %ecx
>   cmp %eax, %ecx
>   movl LEN(%esp), %ecx
> - jle L(forward)
> + jbe L(forward)
>
>  /* Now do checks for lengths. We do [0..16], [0..32], [0..64], [0..128]
>   separately.  */
>
>
> --
> WBR,
> Andrew
  

On Mär 19 2018, Andrew Senkevich <andrew.n.senkevich@gmail.com> wrote:

> +static void
> +do_test2 (void)
> +{
> +  uint32_t num = 0x20000000;
> +  uint32_t * large_buf;
> +
> +  large_buf = mmap ((void*)0x70000000, num, PROT_READ | PROT_WRITE,
> +     MAP_PRIVATE | MAP_ANON | MAP_FIXED, -1, 0);

Since you are using MAP_FIXED this may overwrite an existing mapping.

Andreas.
  

On 03/19/2018 02:11 PM, Andreas Schwab wrote:
> On Mär 19 2018, Andrew Senkevich<andrew.n.senkevich@gmail.com>  wrote:
> 
>> +static void
>> +do_test2 (void)
>> +{
>> +  uint32_t num = 0x20000000;
>> +  uint32_t * large_buf;
>> +
>> +  large_buf = mmap ((void*)0x70000000, num, PROT_READ | PROT_WRITE,
>> +     MAP_PRIVATE | MAP_ANON | MAP_FIXED, -1, 0);
> Since you are using MAP_FIXED this may overwrite an existing mapping.

Leading to a hard-to-debug crash, maybe sporadically due to ASLR.  Yes, 
I have this concern as well.

There was a long, long Linux thread about a non-overriding MAP_FIXED 
variant, but as far as I can see, this has not been merged.  Maybe it 
would have helped here.

Is it very difficult to split out this test into a separate test file? 
Then link the whole thing statically, as non-PIE, and keep using 
MAP_FIXED.  This should make it quite likely that you don't override 
anything valuable.

Or you could parse /proc/self/maps to make sure that you don't override 
an existing mapping.  Yuck.

Thanks,
Florian
  

2018-03-19 14:17 GMT+01:00 Florian Weimer <fweimer@redhat.com>:
> On 03/19/2018 02:11 PM, Andreas Schwab wrote:
>>
>> On Mär 19 2018, Andrew Senkevich<andrew.n.senkevich@gmail.com>  wrote:
>>
>>> +static void
>>> +do_test2 (void)
>>> +{
>>> +  uint32_t num = 0x20000000;
>>> +  uint32_t * large_buf;
>>> +
>>> +  large_buf = mmap ((void*)0x70000000, num, PROT_READ | PROT_WRITE,
>>> +     MAP_PRIVATE | MAP_ANON | MAP_FIXED, -1, 0);
>>
>> Since you are using MAP_FIXED this may overwrite an existing mapping.
>
>
> Leading to a hard-to-debug crash, maybe sporadically due to ASLR.  Yes, I
> have this concern as well.
>
> There was a long, long Linux thread about a non-overriding MAP_FIXED
> variant, but as far as I can see, this has not been merged.  Maybe it would
> have helped here.
>
> Is it very difficult to split out this test into a separate test file? Then
> link the whole thing statically, as non-PIE, and keep using MAP_FIXED.  This
> should make it quite likely that you don't override anything valuable.

I think not very difficult, I will try this way.


--
WBR,
Andrew
  

On 19/03/18 13:17, Florian Weimer wrote:
> On 03/19/2018 02:11 PM, Andreas Schwab wrote:
>> On Mär 19 2018, Andrew Senkevich<andrew.n.senkevich@gmail.com>  wrote:
>>
>>> +static void
>>> +do_test2 (void)
>>> +{
>>> +  uint32_t num = 0x20000000;
>>> +  uint32_t * large_buf;
>>> +
>>> +  large_buf = mmap ((void*)0x70000000, num, PROT_READ | PROT_WRITE,
>>> +     MAP_PRIVATE | MAP_ANON | MAP_FIXED, -1, 0);
>> Since you are using MAP_FIXED this may overwrite an existing mapping.
> 
> Leading to a hard-to-debug crash, maybe sporadically due to ASLR.  Yes, I have this concern as well.
> 
> There was a long, long Linux thread about a non-overriding MAP_FIXED variant, but as far as I can see, this has not been merged.  Maybe it would 
> have helped here.
> 

i thought not using MAP_FIXED is the 'non-overriding MAP_FIXED variant'

if you use an address hint then the kernel will use that unless
it's not available and you can check the result.
  

On 03/19/2018 03:25 PM, Szabolcs Nagy wrote:
> i thought not using MAP_FIXED is the 'non-overriding MAP_FIXED variant'

In general, yes.  But I think there cases where the hint is ignored even 
if there isn't a pre-existing mapping.  Maybe mapping things at the 
middle of the (32-bit) address space is one such case?

Thanks,
Florian