[1/3] y2038: linux: Provide __futimes64 implementation

  This patch provides new __futimes64 explicit 64 bit function for setting file's
64 bit attributes for access and modification time (by specifying file
descriptor number).

Internally, the __utimensat64_helper function is used. This patch is necessary
for having architectures with __WORDSIZE == 32 Y2038 safe.

Moreover, a 32 bit version - __futimes has been refactored to internally use
__futimes64.

The __futimes is now supposed to be used on systems still supporting 32
bit time (__TIMESIZE != 64) - hence the necessary conversion of struct timeval
to 64 bit struct __timeval64.

The check if struct timevals' usec fields are in the range between 0 and 1000000
has been removed as Linux kernel performs it internally in the implementation
of utimensat (the conversion between struct __timeval64 and __timespec64 is not
relevant for this particular check).

Last but not least, checks for tvp{64} not being NULL have been preserved from
the original code as some legacy user space programs may rely on it.

Build tests:
./src/scripts/build-many-glibcs.py glibcs

Run-time tests:
- Run specific tests on ARM/x86 32bit systems (qemu):
  https://github.com/lmajewski/meta-y2038 and run tests:
  https://github.com/lmajewski/y2038-tests/commits/master

Above tests were performed with Y2038 redirection applied as well as without to
test the proper usage of both __futimes64 and __futimes.
---
 include/time.h                    |  3 +++
 sysdeps/unix/sysv/linux/futimes.c | 42 +++++++++++++++++--------------
 2 files changed, 26 insertions(+), 19 deletions(-)
  

Dear All,

> This patch provides new __futimes64 explicit 64 bit function for
> setting file's 64 bit attributes for access and modification time (by
> specifying file descriptor number).
> 
> Internally, the __utimensat64_helper function is used. This patch is
> necessary for having architectures with __WORDSIZE == 32 Y2038 safe.
> 
> Moreover, a 32 bit version - __futimes has been refactored to
> internally use __futimes64.
> 
> The __futimes is now supposed to be used on systems still supporting
> 32 bit time (__TIMESIZE != 64) - hence the necessary conversion of
> struct timeval to 64 bit struct __timeval64.
> 
> The check if struct timevals' usec fields are in the range between 0
> and 1000000 has been removed as Linux kernel performs it internally
> in the implementation of utimensat (the conversion between struct
> __timeval64 and __timespec64 is not relevant for this particular
> check).
> 
> Last but not least, checks for tvp{64} not being NULL have been
> preserved from the original code as some legacy user space programs
> may rely on it.
> 
> Build tests:
> ./src/scripts/build-many-glibcs.py glibcs
> 
> Run-time tests:
> - Run specific tests on ARM/x86 32bit systems (qemu):
>   https://github.com/lmajewski/meta-y2038 and run tests:
>   https://github.com/lmajewski/y2038-tests/commits/master
> 
> Above tests were performed with Y2038 redirection applied as well as
> without to test the proper usage of both __futimes64 and __futimes.
> ---
>  include/time.h                    |  3 +++
>  sysdeps/unix/sysv/linux/futimes.c | 42
> +++++++++++++++++-------------- 2 files changed, 26 insertions(+), 19
> deletions(-)
> 
> diff --git a/include/time.h b/include/time.h
> index b81ecd5e6e..c24066bf3a 100644
> --- a/include/time.h
> +++ b/include/time.h
> @@ -210,8 +210,11 @@ extern int __utimensat64_helper (int fd, const
> char *file, libc_hidden_proto (__utimensat64_helper);
>  
>  #if __TIMESIZE == 64
> +# define __futimes64 __futimes
>  # define __futimens64 __futimens
>  #else
> +extern int __futimes64 (int fd, const struct __timeval64 tvp64[2]);
> +libc_hidden_proto (__futimes64);
>  extern int __futimens64 (int fd, const struct __timespec64 tsp[2]);
>  libc_hidden_proto (__futimens64);
>  #endif
> diff --git a/sysdeps/unix/sysv/linux/futimes.c
> b/sysdeps/unix/sysv/linux/futimes.c index 4bea864470..21b41e4138
> 100644 --- a/sysdeps/unix/sysv/linux/futimes.c
> +++ b/sysdeps/unix/sysv/linux/futimes.c
> @@ -17,35 +17,39 @@
>     <https://www.gnu.org/licenses/>.  */
>  
>  #include <errno.h>
> -#include <sysdep.h>
> -#include <string.h>
>  #include <time.h>
> -#include <utime.h>
> -#include <sys/time.h>
> -#include <_itoa.h>
> -#include <fcntl.h>
> -
>  
>  /* Change the access time of the file associated with FD to TVP[0]
> and
> -   the modification time of FILE to TVP[1].
> +   the modification time of FILE to TVP[1].  */
> +int
> +__futimes64 (int fd, const struct __timeval64 tvp64[2])
> +{
> +  /* The utimensat system call expects timespec not timeval.  */
> +  struct __timespec64 ts64[2];
> +  if (tvp64 != NULL)
> +    {
> +      ts64[0] = timeval64_to_timespec64 (tvp64[0]);
> +      ts64[1] = timeval64_to_timespec64 (tvp64[1]);
> +    }
> +
> +  return __utimensat64_helper (fd, NULL, tvp64 ? &ts64[0] : NULL, 0);
> +}
> +
> +#if __TIMESIZE != 64
> +libc_hidden_def (__futimes64)
>  
> -   Starting with 2.6.22 the Linux kernel has the utimensat syscall
> which
> -   can be used to implement futimes.  */
>  int
>  __futimes (int fd, const struct timeval tvp[2])
>  {
> -  /* The utimensat system call expects timespec not timeval.  */
> -  struct timespec ts[2];
> +  struct __timeval64 tv64[2];
> +
>    if (tvp != NULL)
>      {
> -      if (tvp[0].tv_usec < 0 || tvp[0].tv_usec >= 1000000
> -          || tvp[1].tv_usec < 0 || tvp[1].tv_usec >= 1000000)
> -	return INLINE_SYSCALL_ERROR_RETURN_VALUE (EINVAL);
> -
> -      TIMEVAL_TO_TIMESPEC (&tvp[0], &ts[0]);
> -      TIMEVAL_TO_TIMESPEC (&tvp[1], &ts[1]);
> +      tv64[0] = valid_timeval_to_timeval64 (tvp[0]);
> +      tv64[1] = valid_timeval_to_timeval64 (tvp[1]);
>      }
>  
> -  return INLINE_SYSCALL (utimensat, 4, fd, NULL, tvp ? &ts : NULL,
> 0);
> +  return __futimes64 (fd, tvp ? &tv64[0] : NULL);
>  }
> +#endif
>  weak_alias (__futimes, futimes)

Gentle ping on this patch.


Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email: lukma@denx.de
  

Ok.

Andreas.
  

Hi Lukasz,

On 2/17/20 5:17 AM, Lukasz Majewski wrote:
> This patch provides new __futimes64 explicit 64 bit function for setting file's
> 64 bit attributes for access and modification time (by specifying file
> descriptor number).
> 
> Internally, the __utimensat64_helper function is used. This patch is necessary
> for having architectures with __WORDSIZE == 32 Y2038 safe.
> 
> Moreover, a 32 bit version - __futimes has been refactored to internally use
> __futimes64.
> 
> The __futimes is now supposed to be used on systems still supporting 32
> bit time (__TIMESIZE != 64) - hence the necessary conversion of struct timeval
> to 64 bit struct __timeval64.
> 
> The check if struct timevals' usec fields are in the range between 0 and 1000000
> has been removed as Linux kernel performs it internally in the implementation
> of utimensat (the conversion between struct __timeval64 and __timespec64 is not
> relevant for this particular check).
> 
> Last but not least, checks for tvp{64} not being NULL have been preserved from
> the original code as some legacy user space programs may rely on it.
> 
> Build tests:
> ./src/scripts/build-many-glibcs.py glibcs
> 
> Run-time tests:
> - Run specific tests on ARM/x86 32bit systems (qemu):
>   https://github.com/lmajewski/meta-y2038 and run tests:
>   https://github.com/lmajewski/y2038-tests/commits/master
> 
> Above tests were performed with Y2038 redirection applied as well as without to
> test the proper usage of both __futimes64 and __futimes.
> ---
>  include/time.h                    |  3 +++
>  sysdeps/unix/sysv/linux/futimes.c | 42 +++++++++++++++++--------------

ARC is 32-bit + TIMESIZE==64 but the new code is not getting built (instead
generic variant is). How do I override the generic fiel to be built instead ?

>  2 files changed, 26 insertions(+), 19 deletions(-)
> 
> diff --git a/include/time.h b/include/time.h
> index b81ecd5e6e..c24066bf3a 100644
> --- a/include/time.h
> +++ b/include/time.h
> @@ -210,8 +210,11 @@ extern int __utimensat64_helper (int fd, const char *file,
>  libc_hidden_proto (__utimensat64_helper);
>  
>  #if __TIMESIZE == 64
> +# define __futimes64 __futimes
>  # define __futimens64 __futimens
>  #else
> +extern int __futimes64 (int fd, const struct __timeval64 tvp64[2]);
> +libc_hidden_proto (__futimes64);
>  extern int __futimens64 (int fd, const struct __timespec64 tsp[2]);
>  libc_hidden_proto (__futimens64);
>  #endif
> diff --git a/sysdeps/unix/sysv/linux/futimes.c b/sysdeps/unix/sysv/linux/futimes.c
> index 4bea864470..21b41e4138 100644
> --- a/sysdeps/unix/sysv/linux/futimes.c
> +++ b/sysdeps/unix/sysv/linux/futimes.c
> @@ -17,35 +17,39 @@
>     <https://www.gnu.org/licenses/>.  */
>  
>  #include <errno.h>
> -#include <sysdep.h>
> -#include <string.h>
>  #include <time.h>
> -#include <utime.h>
> -#include <sys/time.h>
> -#include <_itoa.h>
> -#include <fcntl.h>
> -
>  
>  /* Change the access time of the file associated with FD to TVP[0] and
> -   the modification time of FILE to TVP[1].
> +   the modification time of FILE to TVP[1].  */
> +int
> +__futimes64 (int fd, const struct __timeval64 tvp64[2])
> +{
> +  /* The utimensat system call expects timespec not timeval.  */
> +  struct __timespec64 ts64[2];
> +  if (tvp64 != NULL)
> +    {
> +      ts64[0] = timeval64_to_timespec64 (tvp64[0]);
> +      ts64[1] = timeval64_to_timespec64 (tvp64[1]);
> +    }
> +
> +  return __utimensat64_helper (fd, NULL, tvp64 ? &ts64[0] : NULL, 0);
> +}
> +
> +#if __TIMESIZE != 64
> +libc_hidden_def (__futimes64)
>  
> -   Starting with 2.6.22 the Linux kernel has the utimensat syscall which
> -   can be used to implement futimes.  */
>  int
>  __futimes (int fd, const struct timeval tvp[2])
>  {
> -  /* The utimensat system call expects timespec not timeval.  */
> -  struct timespec ts[2];
> +  struct __timeval64 tv64[2];
> +
>    if (tvp != NULL)
>      {
> -      if (tvp[0].tv_usec < 0 || tvp[0].tv_usec >= 1000000
> -          || tvp[1].tv_usec < 0 || tvp[1].tv_usec >= 1000000)
> -	return INLINE_SYSCALL_ERROR_RETURN_VALUE (EINVAL);
> -
> -      TIMEVAL_TO_TIMESPEC (&tvp[0], &ts[0]);
> -      TIMEVAL_TO_TIMESPEC (&tvp[1], &ts[1]);
> +      tv64[0] = valid_timeval_to_timeval64 (tvp[0]);
> +      tv64[1] = valid_timeval_to_timeval64 (tvp[1]);
>      }
>  
> -  return INLINE_SYSCALL (utimensat, 4, fd, NULL, tvp ? &ts : NULL, 0);
> +  return __futimes64 (fd, tvp ? &tv64[0] : NULL);
>  }
> +#endif
>  weak_alias (__futimes, futimes)
>
  

On Mon, 6 Jul 2020 22:58:27 +0000
Vineet Gupta <Vineet.Gupta1@synopsys.com> wrote:

> Hi Lukasz,
> 
> On 2/17/20 5:17 AM, Lukasz Majewski wrote:
> > This patch provides new __futimes64 explicit 64 bit function for
> > setting file's 64 bit attributes for access and modification time
> > (by specifying file descriptor number).
> > 
> > Internally, the __utimensat64_helper function is used. This patch
> > is necessary for having architectures with __WORDSIZE == 32 Y2038
> > safe.
> > 
> > Moreover, a 32 bit version - __futimes has been refactored to
> > internally use __futimes64.
> > 
> > The __futimes is now supposed to be used on systems still
> > supporting 32 bit time (__TIMESIZE != 64) - hence the necessary
> > conversion of struct timeval to 64 bit struct __timeval64.
> > 
> > The check if struct timevals' usec fields are in the range between
> > 0 and 1000000 has been removed as Linux kernel performs it
> > internally in the implementation of utimensat (the conversion
> > between struct __timeval64 and __timespec64 is not relevant for
> > this particular check).
> > 
> > Last but not least, checks for tvp{64} not being NULL have been
> > preserved from the original code as some legacy user space programs
> > may rely on it.
> > 
> > Build tests:
> > ./src/scripts/build-many-glibcs.py glibcs
> > 
> > Run-time tests:
> > - Run specific tests on ARM/x86 32bit systems (qemu):
> >   https://github.com/lmajewski/meta-y2038 and run tests:
> >   https://github.com/lmajewski/y2038-tests/commits/master
> > 
> > Above tests were performed with Y2038 redirection applied as well
> > as without to test the proper usage of both __futimes64 and
> > __futimes. ---
> >  include/time.h                    |  3 +++
> >  sysdeps/unix/sysv/linux/futimes.c | 42
> > +++++++++++++++++--------------  
> 
> ARC is 32-bit + TIMESIZE==64 but the new code is not getting built
> (instead generic variant is). How do I override the generic fiel to
> be built instead ?

The futimes.c from sysdeps/unix/sysv/linux/ shall be built. 

The futimes.c is defined in following locations:
y2038-glibc/misc
y2038-glibc/sysdeps/unix/sysv/linux
y2038-glibc/sysdeps/mach/hurd

The first one is a stub. The last one is for hurd.
ARC shall use the middle one as well.

When I'm in doubt (or when for example code is auto generated to wrap a
syscall) I do use
https://github.com/lmajewski/meta-y2038/blob/master/README

with gdb to check which function is called. Maybe it would help you too?

> 
> >  2 files changed, 26 insertions(+), 19 deletions(-)
> > 
> > diff --git a/include/time.h b/include/time.h
> > index b81ecd5e6e..c24066bf3a 100644
> > --- a/include/time.h
> > +++ b/include/time.h
> > @@ -210,8 +210,11 @@ extern int __utimensat64_helper (int fd, const
> > char *file, libc_hidden_proto (__utimensat64_helper);
> >  
> >  #if __TIMESIZE == 64
> > +# define __futimes64 __futimes
> >  # define __futimens64 __futimens
> >  #else
> > +extern int __futimes64 (int fd, const struct __timeval64 tvp64[2]);
> > +libc_hidden_proto (__futimes64);
> >  extern int __futimens64 (int fd, const struct __timespec64 tsp[2]);
> >  libc_hidden_proto (__futimens64);
> >  #endif
> > diff --git a/sysdeps/unix/sysv/linux/futimes.c
> > b/sysdeps/unix/sysv/linux/futimes.c index 4bea864470..21b41e4138
> > 100644 --- a/sysdeps/unix/sysv/linux/futimes.c
> > +++ b/sysdeps/unix/sysv/linux/futimes.c
> > @@ -17,35 +17,39 @@
> >     <https://www.gnu.org/licenses/>.  */
> >  
> >  #include <errno.h>
> > -#include <sysdep.h>
> > -#include <string.h>
> >  #include <time.h>
> > -#include <utime.h>
> > -#include <sys/time.h>
> > -#include <_itoa.h>
> > -#include <fcntl.h>
> > -
> >  
> >  /* Change the access time of the file associated with FD to TVP[0]
> > and
> > -   the modification time of FILE to TVP[1].
> > +   the modification time of FILE to TVP[1].  */
> > +int
> > +__futimes64 (int fd, const struct __timeval64 tvp64[2])
> > +{
> > +  /* The utimensat system call expects timespec not timeval.  */
> > +  struct __timespec64 ts64[2];
> > +  if (tvp64 != NULL)
> > +    {
> > +      ts64[0] = timeval64_to_timespec64 (tvp64[0]);
> > +      ts64[1] = timeval64_to_timespec64 (tvp64[1]);
> > +    }
> > +
> > +  return __utimensat64_helper (fd, NULL, tvp64 ? &ts64[0] : NULL,
> > 0); +}
> > +
> > +#if __TIMESIZE != 64
> > +libc_hidden_def (__futimes64)
> >  
> > -   Starting with 2.6.22 the Linux kernel has the utimensat syscall
> > which
> > -   can be used to implement futimes.  */
> >  int
> >  __futimes (int fd, const struct timeval tvp[2])
> >  {
> > -  /* The utimensat system call expects timespec not timeval.  */
> > -  struct timespec ts[2];
> > +  struct __timeval64 tv64[2];
> > +
> >    if (tvp != NULL)
> >      {
> > -      if (tvp[0].tv_usec < 0 || tvp[0].tv_usec >= 1000000
> > -          || tvp[1].tv_usec < 0 || tvp[1].tv_usec >= 1000000)
> > -	return INLINE_SYSCALL_ERROR_RETURN_VALUE (EINVAL);
> > -
> > -      TIMEVAL_TO_TIMESPEC (&tvp[0], &ts[0]);
> > -      TIMEVAL_TO_TIMESPEC (&tvp[1], &ts[1]);
> > +      tv64[0] = valid_timeval_to_timeval64 (tvp[0]);
> > +      tv64[1] = valid_timeval_to_timeval64 (tvp[1]);
> >      }
> >  
> > -  return INLINE_SYSCALL (utimensat, 4, fd, NULL, tvp ? &ts : NULL,
> > 0);
> > +  return __futimes64 (fd, tvp ? &tv64[0] : NULL);
> >  }
> > +#endif
> >  weak_alias (__futimes, futimes)
> >   
> 




Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email: lukma@denx.de
  

On 7/7/20 12:21 AM, Lukasz Majewski wrote:
>> ARC is 32-bit + TIMESIZE==64 but the new code is not getting built
>> (instead generic variant is). How do I override the generic fiel to
>> be built instead ?
> The futimes.c from sysdeps/unix/sysv/linux/ shall be built. 
> 
> The futimes.c is defined in following locations:
> y2038-glibc/misc
> y2038-glibc/sysdeps/unix/sysv/linux
> y2038-glibc/sysdeps/mach/hurd
> 
> The first one is a stub. The last one is for hurd.
> ARC shall use the middle one as well.

It seems I mixed up the various *utime* files.
Following are currently building for ARC

sysdeps/unix/sysv/linux/generic/futimesat.c
sysdeps/unix/sysv/linux/generic/utimes.c    #2

sysdeps/unix/sysv/linux/futimes.c
sysdeps/unix/sysv/linux/futimens.c
sysdeps/unix/sysv/linux/utimensat.c

The issue is generic/utimes.c

Is that the right file to build. If yes then it needs fixing as it expects
__NR_utimensat (and thus require the aliasing hack in ARC sysdep.h)

#define __NR_utimensat		__NR_utimensat_time6

P.S. I know glibc wiki has a bunch of pages (from Arnd?) for the y2038 support.
Are there other (newer) docs which explain the whole song and dance or is it the
usual RTFC.

> When I'm in doubt (or when for example code is auto generated to wrap a
> syscall) I do use
> https://github.com/lmajewski/meta-y2038/blob/master/README
> 
> with gdb to check which function is called. Maybe it would help you too?

Yeah. I typically use build logs to see which exact file is built and then objdump
to corroborate.
  

On Tue, 7 Jul 2020, Vineet Gupta via Libc-alpha wrote:

> The issue is generic/utimes.c
> 
> Is that the right file to build. If yes then it needs fixing as it expects
> __NR_utimensat (and thus require the aliasing hack in ARC sysdep.h)

It was the right file to build, back when sysdeps/unix/sysv/linux/utimes.c 
required a utimes syscall to be present and architectures with the generic 
syscall interface didn't gave one.

Now, there is nothing in sysdeps/unix/sysv/linux/utimes.c that wouldn't 
work on asm-generic architectures.  So generic/utimes.c should be removed.  
Likewise, hppa/utimes.c should also be removed (now that 
sysdeps/unix/sysv/linux/utimes.c doesn't use the utimes syscalls, it 
doesn't matter than hppa got it late), and the __ASSUME_UTIMES definition 
in sysdeps/unix/sysv/linux/hppa/kernel-features.h should also be removed 
(because nothing will use it once the hppa utimes.c goes away).

Similarly, there is no longer a need for generic/futimesat.c so that can 
go away as well.
  

Hi Vineet,

> On 7/7/20 12:21 AM, Lukasz Majewski wrote:
> >> ARC is 32-bit + TIMESIZE==64 but the new code is not getting built
> >> (instead generic variant is). How do I override the generic fiel to
> >> be built instead ?  
> > The futimes.c from sysdeps/unix/sysv/linux/ shall be built. 
> > 
> > The futimes.c is defined in following locations:
> > y2038-glibc/misc
> > y2038-glibc/sysdeps/unix/sysv/linux
> > y2038-glibc/sysdeps/mach/hurd
> > 
> > The first one is a stub. The last one is for hurd.
> > ARC shall use the middle one as well.  
> 
> It seems I mixed up the various *utime* files.
> Following are currently building for ARC
> 
> sysdeps/unix/sysv/linux/generic/futimesat.c
> sysdeps/unix/sysv/linux/generic/utimes.c    #2
> 
> sysdeps/unix/sysv/linux/futimes.c
> sysdeps/unix/sysv/linux/futimens.c
> sysdeps/unix/sysv/linux/utimensat.c
> 
> The issue is generic/utimes.c
> 
> Is that the right file to build. If yes then it needs fixing as it
> expects __NR_utimensat (and thus require the aliasing hack in ARC
> sysdep.h)
> 
> #define __NR_utimensat		__NR_utimensat_time6
> 
> P.S. I know glibc wiki has a bunch of pages (from Arnd?) for the
> y2038 support. 

This wiki page was developed by Albert Aribaud and shall give you a
grasp about the work needed for Y2038 conversion.

However, it is now outdated as we already made some progress.

The most accurate list of supported/converted syscalls can be found
here:
https://github.com/lmajewski/y2038_glibc/commit/b94a7f1126dc5bd652ba3a856753e85e48a6e845

and the most up to date development branch for Y2038 conversion:
https://github.com/lmajewski/y2038_glibc/commits/y2038_edge

(I've sent yesterday a patch series to convert futex to futex_time64).

> Are there other (newer) docs which explain the whole
> song and dance or is it the usual RTFC.
> 
> > When I'm in doubt (or when for example code is auto generated to
> > wrap a syscall) I do use
> > https://github.com/lmajewski/meta-y2038/blob/master/README
> > 
> > with gdb to check which function is called. Maybe it would help you
> > too?  
> 
> Yeah. I typically use build logs to see which exact file is built and
> then objdump to corroborate.

Ok.

> 




Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email: lukma@denx.de