[5/6] Fix fflush handling for mmap files after ungetc (bug 32535)

Message ID 8ef14304-75b4-6938-1cd4-d44c3390f7e0@redhat.com (mailing list archive)
State New
Headers
Series Fix issues around flushing and file offsets for input files |

Checks

Context Check Description
redhat-pt-bot/TryBot-apply_patch success Patch applied to master at the time it was sent
linaro-tcwg-bot/tcwg_glibc_build--master-arm success Build passed
linaro-tcwg-bot/tcwg_glibc_build--master-aarch64 success Build passed
linaro-tcwg-bot/tcwg_glibc_check--master-arm success Test passed
linaro-tcwg-bot/tcwg_glibc_check--master-aarch64 success Test passed

Commit Message

Joseph Myers Jan. 14, 2025, 2:04 a.m. UTC
  As discussed in bug 32535, fflush fails on files opened for reading
using mmap after ungetc.  Fix the logic to handle this case and still
compute the file offset correctly.

Tested for x86_64.
---
 libio/fileops.c                | 12 ++++++---
 stdio-common/Makefile          |  1 +
 stdio-common/tst-fflush-mmap.c | 48 ++++++++++++++++++++++++++++++++++
 3 files changed, 57 insertions(+), 4 deletions(-)
 create mode 100644 stdio-common/tst-fflush-mmap.c
  

Comments

DJ Delorie Jan. 15, 2025, 7:17 a.m. UTC | #1
Some of the logic seems a little off to me, but it might be my brand new
understanding of how _IO_* works ;-)

Also, the test seems weak somehow.

Joseph Myers <josmyers@redhat.com> writes:
> diff --git a/libio/fileops.c b/libio/fileops.c
>  int
>  _IO_file_sync_mmap (FILE *fp)
>  {
> +  off64_t o = fp->_offset - (fp->_IO_read_end - fp->_IO_read_ptr);

To be consistent with logic elsewhere, this could have been:

> off64_t o = fp->_offset + (fp->_IO_read_ptr - fp->_IO_read_end);

But this logic doesn't reflect the true meaning of the underlying
pointers when _IO_in_backup() is true.  It ends up calculating the same
value, but it's applying fp->_offset to the wrong buffer in that case.
The effective offset of the backup buffer's end is the main buffer's
current read pointer.

This is how it's done elsewhere, though, so ok.

>    if (fp->_IO_read_ptr != fp->_IO_read_end)
>      {
> -      if (__lseek64 (fp->_fileno, fp->_IO_read_ptr - fp->_IO_buf_base,
> -		     SEEK_SET)
> -	  != fp->_IO_read_ptr - fp->_IO_buf_base)
> +      if (_IO_in_backup (fp))
> +	{
> +	  _IO_switch_to_main_get_area (fp);
> +	  o -= fp->_IO_read_end - fp->_IO_read_base;

Shouldn't one of these be _IO_read_ptr ?

> +	}
> +      if (__lseek64 (fp->_fileno, o, SEEK_SET) != o)

Ok.

>  	{
>  	  fp->_flags |= _IO_ERR_SEEN;
>  	  return EOF;
>  	}
>      }
> -  fp->_offset = fp->_IO_read_ptr - fp->_IO_buf_base;
> +  fp->_offset = o;
>    fp->_IO_read_end = fp->_IO_read_ptr = fp->_IO_read_base;
>    return 0;
>  }

Ok.

> diff --git a/stdio-common/Makefile b/stdio-common/Makefile
> index 06c9eaf426..703e8af7e8 100644
> --- a/stdio-common/Makefile
> +++ b/stdio-common/Makefile
> @@ -239,6 +239,7 @@ tests := \
>    tst-fdopen2 \
>    tst-ferror \
>    tst-fflush-all-input \
> +  tst-fflush-mmap \
>    tst-fgets \
>    tst-fgets2 \
>    tst-fileno \

Ok.

> diff --git a/stdio-common/tst-fflush-mmap.c b/stdio-common/tst-fflush-mmap.c
> +/* Test fflush after ungetc on files using mmap (bug 32535).
> +   Copyright (C) 2025 Free Software Foundation, Inc.
> +   This file is part of the GNU C Library.
> +
> +   The GNU C Library is free software; you can redistribute it and/or
> +   modify it under the terms of the GNU Lesser General Public
> +   License as published by the Free Software Foundation; either
> +   version 2.1 of the License, or (at your option) any later version.
> +
> +   The GNU C Library is distributed in the hope that it will be useful,
> +   but WITHOUT ANY WARRANTY; without even the implied warranty of
> +   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> +   Lesser General Public License for more details.
> +
> +   You should have received a copy of the GNU Lesser General Public
> +   License along with the GNU C Library; if not, see
> +   <https://www.gnu.org/licenses/>.  */
> +
> +#include <stdio.h>
> +
> +#include <support/check.h>
> +#include <support/temp_file.h>
> +#include <support/xstdio.h>
> +#include <support/xunistd.h>

Ok.

> +int
> +do_test (void)
> +{
> +  char *filename = NULL;
> +  int fd = create_temp_file ("tst-fflush-mmap", &filename);
> +  TEST_VERIFY_EXIT (fd != -1);
> +  xclose (fd);

Ok.

> +  /* Test fflush after ungetc (bug 32535).  */
> +  FILE *fp = xfopen (filename, "w");
> +  TEST_VERIFY (0 <= fputs ("test", fp));
> +  xfclose (fp);

Ok.

> +  fp = xfopen (filename, "rm");
> +  TEST_COMPARE (fgetc (fp), 't');
> +  TEST_COMPARE (ungetc ('u', fp), 'u');
> +  TEST_COMPARE (fflush (fp), 0);

Should we test that fflush() did the right thing, somehow?
Can we, for mmap'd files?

> +  xfclose (fp);
> +
> +  return 0;
> +}
> +
> +#include <support/test-driver.c>

Ok.
  
Joseph Myers Jan. 20, 2025, 10:56 p.m. UTC | #2
On Wed, 15 Jan 2025, DJ Delorie wrote:

> Some of the logic seems a little off to me, but it might be my brand new
> understanding of how _IO_* works ;-)
> 
> Also, the test seems weak somehow.
> 
> Joseph Myers <josmyers@redhat.com> writes:
> > diff --git a/libio/fileops.c b/libio/fileops.c
> >  int
> >  _IO_file_sync_mmap (FILE *fp)
> >  {
> > +  off64_t o = fp->_offset - (fp->_IO_read_end - fp->_IO_read_ptr);
> 
> To be consistent with logic elsewhere, this could have been:
> 
> > off64_t o = fp->_offset + (fp->_IO_read_ptr - fp->_IO_read_end);

This is following _IO_file_seekoff_mmap.

> >    if (fp->_IO_read_ptr != fp->_IO_read_end)
> >      {
> > -      if (__lseek64 (fp->_fileno, fp->_IO_read_ptr - fp->_IO_buf_base,
> > -		     SEEK_SET)
> > -	  != fp->_IO_read_ptr - fp->_IO_buf_base)
> > +      if (_IO_in_backup (fp))
> > +	{
> > +	  _IO_switch_to_main_get_area (fp);
> > +	  o -= fp->_IO_read_end - fp->_IO_read_base;
> 
> Shouldn't one of these be _IO_read_ptr ?

_IO_switch_to_main_get_area includes "fp->_IO_read_ptr = 
fp->_IO_read_base;".  (This logic is actually emulating the use of 
"fp->_IO_save_end - fp->_IO_save_base" in ftell and related functions.)

> > +  fp = xfopen (filename, "rm");
> > +  TEST_COMPARE (fgetc (fp), 't');
> > +  TEST_COMPARE (ungetc ('u', fp), 'u');
> > +  TEST_COMPARE (fflush (fp), 0);
> 
> Should we test that fflush() did the right thing, somehow?
> Can we, for mmap'd files?

I've added tests of the next characters read from the file, which is more 
or less what it means for fflush to do the right thing here (discard the 
character pushed with ungetc while preserving the offset).




Fix fflush handling for mmap files after ungetc (bug 32535)

As discussed in bug 32535, fflush fails on files opened for reading
using mmap after ungetc.  Fix the logic to handle this case and still
compute the file offset correctly.

Tested for x86_64.

---

Changed in v2: also verify characters read after fflush.
---
 libio/fileops.c                | 12 +++++---
 stdio-common/Makefile          |  1 +
 stdio-common/tst-fflush-mmap.c | 50 ++++++++++++++++++++++++++++++++++
 3 files changed, 59 insertions(+), 4 deletions(-)
 create mode 100644 stdio-common/tst-fflush-mmap.c

diff --git a/libio/fileops.c b/libio/fileops.c
index 97875d1eaf..7e370d12b2 100644
--- a/libio/fileops.c
+++ b/libio/fileops.c
@@ -858,17 +858,21 @@ libc_hidden_ver (_IO_new_file_sync, _IO_file_sync)
 int
 _IO_file_sync_mmap (FILE *fp)
 {
+  off64_t o = fp->_offset - (fp->_IO_read_end - fp->_IO_read_ptr);
   if (fp->_IO_read_ptr != fp->_IO_read_end)
     {
-      if (__lseek64 (fp->_fileno, fp->_IO_read_ptr - fp->_IO_buf_base,
-		     SEEK_SET)
-	  != fp->_IO_read_ptr - fp->_IO_buf_base)
+      if (_IO_in_backup (fp))
+	{
+	  _IO_switch_to_main_get_area (fp);
+	  o -= fp->_IO_read_end - fp->_IO_read_base;
+	}
+      if (__lseek64 (fp->_fileno, o, SEEK_SET) != o)
 	{
 	  fp->_flags |= _IO_ERR_SEEN;
 	  return EOF;
 	}
     }
-  fp->_offset = fp->_IO_read_ptr - fp->_IO_buf_base;
+  fp->_offset = o;
   fp->_IO_read_end = fp->_IO_read_ptr = fp->_IO_read_base;
   return 0;
 }
diff --git a/stdio-common/Makefile b/stdio-common/Makefile
index 06c9eaf426..703e8af7e8 100644
--- a/stdio-common/Makefile
+++ b/stdio-common/Makefile
@@ -239,6 +239,7 @@ tests := \
   tst-fdopen2 \
   tst-ferror \
   tst-fflush-all-input \
+  tst-fflush-mmap \
   tst-fgets \
   tst-fgets2 \
   tst-fileno \
diff --git a/stdio-common/tst-fflush-mmap.c b/stdio-common/tst-fflush-mmap.c
new file mode 100644
index 0000000000..3bddb909e0
--- /dev/null
+++ b/stdio-common/tst-fflush-mmap.c
@@ -0,0 +1,50 @@
+/* Test fflush after ungetc on files using mmap (bug 32535).
+   Copyright (C) 2025 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <https://www.gnu.org/licenses/>.  */
+
+#include <stdio.h>
+
+#include <support/check.h>
+#include <support/temp_file.h>
+#include <support/xstdio.h>
+#include <support/xunistd.h>
+
+int
+do_test (void)
+{
+  char *filename = NULL;
+  int fd = create_temp_file ("tst-fflush-mmap", &filename);
+  TEST_VERIFY_EXIT (fd != -1);
+  xclose (fd);
+
+  /* Test fflush after ungetc (bug 32535).  */
+  FILE *fp = xfopen (filename, "w");
+  TEST_VERIFY (0 <= fputs ("test", fp));
+  xfclose (fp);
+
+  fp = xfopen (filename, "rm");
+  TEST_COMPARE (fgetc (fp), 't');
+  TEST_COMPARE (ungetc ('u', fp), 'u');
+  TEST_COMPARE (fflush (fp), 0);
+  TEST_COMPARE (fgetc (fp), 't');
+  TEST_COMPARE (fgetc (fp), 'e');
+  xfclose (fp);
+
+  return 0;
+}
+
+#include <support/test-driver.c>
  
DJ Delorie Jan. 21, 2025, 2:28 a.m. UTC | #3
Joseph Myers <josmyers@redhat.com> writes:
>> > +	  _IO_switch_to_main_get_area (fp);
>> > +	  o -= fp->_IO_read_end - fp->_IO_read_base;
>> 
>> Shouldn't one of these be _IO_read_ptr ?
>
> _IO_switch_to_main_get_area includes "fp->_IO_read_ptr = 
> fp->_IO_read_base;".  (This logic is actually emulating the use of 
> "fp->_IO_save_end - fp->_IO_save_base" in ftell and related functions.)

Hmmm... that sounds like it's propogating a bad idea, in case
_IO_switch_to_main_get_area stops making that promise.  But that's a
problem for another day...



Reviewed-by: DJ Delorie <dj@redhat.com>
  

Patch

diff --git a/libio/fileops.c b/libio/fileops.c
index 97875d1eaf..7e370d12b2 100644
--- a/libio/fileops.c
+++ b/libio/fileops.c
@@ -858,17 +858,21 @@  libc_hidden_ver (_IO_new_file_sync, _IO_file_sync)
 int
 _IO_file_sync_mmap (FILE *fp)
 {
+  off64_t o = fp->_offset - (fp->_IO_read_end - fp->_IO_read_ptr);
   if (fp->_IO_read_ptr != fp->_IO_read_end)
     {
-      if (__lseek64 (fp->_fileno, fp->_IO_read_ptr - fp->_IO_buf_base,
-		     SEEK_SET)
-	  != fp->_IO_read_ptr - fp->_IO_buf_base)
+      if (_IO_in_backup (fp))
+	{
+	  _IO_switch_to_main_get_area (fp);
+	  o -= fp->_IO_read_end - fp->_IO_read_base;
+	}
+      if (__lseek64 (fp->_fileno, o, SEEK_SET) != o)
 	{
 	  fp->_flags |= _IO_ERR_SEEN;
 	  return EOF;
 	}
     }
-  fp->_offset = fp->_IO_read_ptr - fp->_IO_buf_base;
+  fp->_offset = o;
   fp->_IO_read_end = fp->_IO_read_ptr = fp->_IO_read_base;
   return 0;
 }
diff --git a/stdio-common/Makefile b/stdio-common/Makefile
index 06c9eaf426..703e8af7e8 100644
--- a/stdio-common/Makefile
+++ b/stdio-common/Makefile
@@ -239,6 +239,7 @@  tests := \
   tst-fdopen2 \
   tst-ferror \
   tst-fflush-all-input \
+  tst-fflush-mmap \
   tst-fgets \
   tst-fgets2 \
   tst-fileno \
diff --git a/stdio-common/tst-fflush-mmap.c b/stdio-common/tst-fflush-mmap.c
new file mode 100644
index 0000000000..fd3671f198
--- /dev/null
+++ b/stdio-common/tst-fflush-mmap.c
@@ -0,0 +1,48 @@ 
+/* Test fflush after ungetc on files using mmap (bug 32535).
+   Copyright (C) 2025 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <https://www.gnu.org/licenses/>.  */
+
+#include <stdio.h>
+
+#include <support/check.h>
+#include <support/temp_file.h>
+#include <support/xstdio.h>
+#include <support/xunistd.h>
+
+int
+do_test (void)
+{
+  char *filename = NULL;
+  int fd = create_temp_file ("tst-fflush-mmap", &filename);
+  TEST_VERIFY_EXIT (fd != -1);
+  xclose (fd);
+
+  /* Test fflush after ungetc (bug 32535).  */
+  FILE *fp = xfopen (filename, "w");
+  TEST_VERIFY (0 <= fputs ("test", fp));
+  xfclose (fp);
+
+  fp = xfopen (filename, "rm");
+  TEST_COMPARE (fgetc (fp), 't');
+  TEST_COMPARE (ungetc ('u', fp), 'u');
+  TEST_COMPARE (fflush (fp), 0);
+  xfclose (fp);
+
+  return 0;
+}
+
+#include <support/test-driver.c>