From patchwork Wed Jun 11 21:37:24 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adhemerval Zanella Netto X-Patchwork-Id: 1454 Received: (qmail 31660 invoked by alias); 11 Jun 2014 21:37:37 -0000 Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-alpha-owner@sourceware.org Delivered-To: mailing list libc-alpha@sourceware.org Received: (qmail 31593 invoked by uid 89); 11 Jun 2014 21:37:36 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-1.5 required=5.0 tests=AWL, BAYES_00, RP_MATCHES_RCVD, UNSUBSCRIBE_BODY autolearn=no version=3.3.2 X-HELO: e24smtp05.br.ibm.com Message-ID: <5398CC14.7050507@linux.vnet.ibm.com> Date: Wed, 11 Jun 2014 18:37:24 -0300 From: Adhemerval Zanella User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: "GNU C. Library" Subject: [RFC] [PATCH 2/2] libio: fmemopen rewrite to POSIX compliance References: <5398C951.3020403@linux.vnet.ibm.com> In-Reply-To: <5398C951.3020403@linux.vnet.ibm.com> X-TM-AS-MML: disable X-Content-Scanned: Fidelis XPS MAILER x-cbid: 14061121-1798-0000-0000-000002D955EF This patch added a new fmemopen version, for glibc 2.20, that aims to be POSIX complaint. It fixes some long-standing glibc fmemopen issues, such as: * it changes the way fseek with SEEK_END works on fmemopen to seek relative to buffer size instead of first '\0'. This is default mode and 'b' opening mode does not change internal behavior (bz#6544). * fix apending opening mode to use as start position either first null byte of len specified in function call (bz#13152 and #13151). * remove binary option 'b' and internal different handling (bz#12836) * fix seek/SEE_END with negative values (bz#14292). Old implementation is moved to libio/oldfmemopen.c and a compatibility symbol is provided to with old behavior for older symbols version (2.2). For this implementation I tried to both address and add testcases for BZ#14292, BZ#13152, BZ#13151, BZ#12836, and BZ#6544. The tst-fmemopen2 testcase is adjusted to reflect new implementation assumptions and a new test tst-fmemopen3 is added to check for some operations described in the bug for read and append mode. Tested on PPC64 and x86_64. PS: I haven't added a CL neither update the ABI files for all archs since it is RFC message. I intend to finish the patch will once we determine that this new implementation is the way we want for 2.20. diff --git a/include/stdio.h b/include/stdio.h index 9f2ea31..94ba5a5 100644 --- a/include/stdio.h +++ b/include/stdio.h @@ -152,7 +152,6 @@ libc_hidden_proto (fread_unlocked) libc_hidden_proto (fwrite_unlocked) libc_hidden_proto (fgets_unlocked) libc_hidden_proto (fputs_unlocked) -libc_hidden_proto (fmemopen) libc_hidden_proto (open_memstream) libc_hidden_proto (__libc_fatal) libc_hidden_proto (__vsprintf_chk) @@ -180,6 +179,9 @@ gets (char *__str) } # endif +extern FILE * __fmemopen (void *buf, size_t len, const char *mode); +libc_hidden_proto (__fmemopen) + __END_DECLS # endif diff --git a/libio/Makefile b/libio/Makefile index 56952ce..c76b37e 100644 --- a/libio/Makefile +++ b/libio/Makefile @@ -46,7 +46,7 @@ routines := \ __fbufsize __freading __fwriting __freadable __fwritable __flbf \ __fpurge __fpending __fsetlocking \ \ - libc_fatal fmemopen + libc_fatal fmemopen oldfmemopen tests = tst_swprintf tst_wprintf tst_swscanf tst_wscanf tst_getwc tst_putwc \ tst_wprintf2 tst-widetext test-fmemopen tst-ext tst-ext2 \ diff --git a/libio/Versions b/libio/Versions index 8df89d2..7ffa64d 100644 --- a/libio/Versions +++ b/libio/Versions @@ -148,6 +148,10 @@ libc { GLIBC_2.4 { open_wmemstream; } + GLIBC_2.20 { + # f* + fmemopen; + } GLIBC_PRIVATE { # Used by NPTL and librt __libc_fatal; diff --git a/libio/fmemopen.c b/libio/fmemopen.c index aee2696..64e2025 100644 --- a/libio/fmemopen.c +++ b/libio/fmemopen.c @@ -1,7 +1,6 @@ -/* Fmemopen implementation. - Copyright (C) 2000-2014 Free Software Foundation, Inc. +/* fmemopen implementation. + Copyright (C) 2014 Free Software Foundation, Inc. This file is part of the GNU C Library. - Contributed by Hanno Mueller, kontakt@hanno.de, 2000. The GNU C Library is free software; you can redistribute it and/or modify it under the terms of the GNU Lesser General Public @@ -17,54 +16,10 @@ License along with the GNU C Library; if not, see . */ -/* - * fmemopen() - "my" version of a string stream - * Hanno Mueller, kontakt@hanno.de - * - * - * I needed fmemopen() for an application that I currently work on, - * but couldn't find it in libio. The following snippet of code is an - * attempt to implement what glibc's documentation describes. - * - * - * - * I already see some potential problems: - * - * - I never used the "original" fmemopen(). I am sure that "my" - * fmemopen() behaves differently than the original version. - * - * - The documentation doesn't say wether a string stream allows - * seeks. I checked the old fmemopen implementation in glibc's stdio - * directory, wasn't quite able to see what is going on in that - * source, but as far as I understand there was no seek there. For - * my application, I needed fseek() and ftell(), so it's here. - * - * - "append" mode and fseek(p, SEEK_END) have two different ideas - * about the "end" of the stream. - * - * As described in the documentation, when opening the file in - * "append" mode, the position pointer will be set to the first null - * character of the string buffer (yet the buffer may already - * contain more data). For fseek(), the last byte of the buffer is - * used as the end of the stream. - * - * - It is unclear to me what the documentation tries to say when it - * explains what happens when you use fmemopen with a NULL - * buffer. - * - * Quote: "fmemopen [then] allocates an array SIZE bytes long. This - * is really only useful if you are going to write things to the - * buffer and then read them back in again." - * - * What does that mean if the original fmemopen() did not allow - * seeking? How do you read what you just wrote without seeking back - * to the beginning of the stream? - * - * - I think there should be a second version of fmemopen() that does - * not add null characters for each write. (At least in my - * application, I am not actually using strings but binary data and - * so I don't need the stream to add null characters on its own.) - */ +/* fmemopen() from 2.20 and forward works as defined by POSIX. It also + provides an older symbol, version 2.2.5, that behaves different regarding + SEEK_END (libio/oldfmemopen.c). */ + #include #include @@ -81,7 +36,8 @@ struct fmemopen_cookie_struct { char *buffer; int mybuffer; - int binmode; + int mode; + int append; size_t size; _IO_off64_t pos; size_t maxpos; @@ -91,13 +47,11 @@ struct fmemopen_cookie_struct static ssize_t fmemopen_read (void *cookie, char *b, size_t s) { - fmemopen_cookie_t *c; + fmemopen_cookie_t *c = (fmemopen_cookie_t *) cookie; - c = (fmemopen_cookie_t *) cookie; - - if (c->pos + s > c->size) + if (c->pos + s > c->maxpos) { - if ((size_t) c->pos == c->size) + if ((size_t) c->pos == c->maxpos) return 0; s = c->size - c->pos; } @@ -115,29 +69,28 @@ fmemopen_read (void *cookie, char *b, size_t s) static ssize_t fmemopen_write (void *cookie, const char *b, size_t s) { - fmemopen_cookie_t *c; + fmemopen_cookie_t *c = (fmemopen_cookie_t *) cookie;; + _IO_off64_t pos = c->append ? c->maxpos : c->pos; int addnullc; - c = (fmemopen_cookie_t *) cookie; + addnullc = (s == 0 || b[s - 1] != '\0'); - addnullc = c->binmode == 0 && (s == 0 || b[s - 1] != '\0'); - - if (c->pos + s + addnullc > c->size) + if (pos + s + addnullc > c->size) { - if ((size_t) (c->pos + addnullc) == c->size) + if ((size_t) (pos + addnullc) >= c->size) { __set_errno (ENOSPC); return 0; } - s = c->size - c->pos - addnullc; + s = c->size - pos - addnullc; } - memcpy (&(c->buffer[c->pos]), b, s); + memcpy (&(c->buffer[pos]), b, s); - c->pos += s; - if ((size_t) c->pos > c->maxpos) + pos += s; + if ((size_t) pos > c->maxpos) { - c->maxpos = c->pos; + c->maxpos = pos; if (addnullc) c->buffer[c->maxpos] = '\0'; } @@ -150,9 +103,7 @@ static int fmemopen_seek (void *cookie, _IO_off64_t *p, int w) { _IO_off64_t np; - fmemopen_cookie_t *c; - - c = (fmemopen_cookie_t *) cookie; + fmemopen_cookie_t *c = (fmemopen_cookie_t *) cookie; switch (w) { @@ -165,7 +116,7 @@ fmemopen_seek (void *cookie, _IO_off64_t *p, int w) break; case SEEK_END: - np = (c->binmode ? c->size : c->maxpos) - *p; + np = c->maxpos + *p; break; default: @@ -184,9 +135,7 @@ fmemopen_seek (void *cookie, _IO_off64_t *p, int w) static int fmemopen_close (void *cookie) { - fmemopen_cookie_t *c; - - c = (fmemopen_cookie_t *) cookie; + fmemopen_cookie_t *c = (fmemopen_cookie_t *) cookie; if (c->mybuffer) free (c->buffer); @@ -197,14 +146,13 @@ fmemopen_close (void *cookie) FILE * -fmemopen (void *buf, size_t len, const char *mode) +__fmemopen (void *buf, size_t len, const char *mode) { cookie_io_functions_t iof; fmemopen_cookie_t *c; if (__glibc_unlikely (len == 0)) { - einval: __set_errno (EINVAL); return NULL; } @@ -231,7 +179,8 @@ fmemopen (void *buf, size_t len, const char *mode) if (__glibc_unlikely ((uintptr_t) len > -(uintptr_t) buf)) { free (c); - goto einval; + __set_errno (EINVAL); + return NULL; } c->buffer = buf; @@ -244,13 +193,15 @@ fmemopen (void *buf, size_t len, const char *mode) c->size = len; - if (mode[0] == 'a') + if (mode[0] == 'r') + c->maxpos = len; + + c->append = mode[0] == 'a'; + if (c->append) c->pos = c->maxpos; else c->pos = 0; - c->binmode = mode[0] != '\0' && mode[1] == 'b'; - iof.read = fmemopen_read; iof.write = fmemopen_write; iof.seek = fmemopen_seek; @@ -258,4 +209,5 @@ fmemopen (void *buf, size_t len, const char *mode) return _IO_fopencookie (c, mode, iof); } -libc_hidden_def (fmemopen) +libc_hidden_def (__fmemopen) +versioned_symbol (libc, __fmemopen, fmemopen, GLIBC_2_20); diff --git a/libio/oldfmemopen.c b/libio/oldfmemopen.c new file mode 100644 index 0000000..4e85ebe --- /dev/null +++ b/libio/oldfmemopen.c @@ -0,0 +1,265 @@ +/* Fmemopen implementation. + Copyright (C) 2000-2014 Free Software Foundation, Inc. + This file is part of the GNU C Library. + Contributed by Hanno Mueller, kontakt@hanno.de, 2000. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +/* + * fmemopen() - "my" version of a string stream + * Hanno Mueller, kontakt@hanno.de + * + * + * I needed fmemopen() for an application that I currently work on, + * but couldn't find it in libio. The following snippet of code is an + * attempt to implement what glibc's documentation describes. + * + * + * + * I already see some potential problems: + * + * - I never used the "original" fmemopen(). I am sure that "my" + * fmemopen() behaves differently than the original version. + * + * - The documentation doesn't say wether a string stream allows + * seeks. I checked the old fmemopen implementation in glibc's stdio + * directory, wasn't quite able to see what is going on in that + * source, but as far as I understand there was no seek there. For + * my application, I needed fseek() and ftell(), so it's here. + * + * - "append" mode and fseek(p, SEEK_END) have two different ideas + * about the "end" of the stream. + * + * As described in the documentation, when opening the file in + * "append" mode, the position pointer will be set to the first null + * character of the string buffer (yet the buffer may already + * contain more data). For fseek(), the last byte of the buffer is + * used as the end of the stream. + * + * - It is unclear to me what the documentation tries to say when it + * explains what happens when you use fmemopen with a NULL + * buffer. + * + * Quote: "fmemopen [then] allocates an array SIZE bytes long. This + * is really only useful if you are going to write things to the + * buffer and then read them back in again." + * + * What does that mean if the original fmemopen() did not allow + * seeking? How do you read what you just wrote without seeking back + * to the beginning of the stream? + * + * - I think there should be a second version of fmemopen() that does + * not add null characters for each write. (At least in my + * application, I am not actually using strings but binary data and + * so I don't need the stream to add null characters on its own.) + */ + +#include "libioP.h" + +#if SHLIB_COMPAT (libc, GLIBC_2_2, GLIBC_2_20) + +#include +#include +#include +#include +#include +#include +#include + + +typedef struct fmemopen_cookie_struct fmemopen_cookie_t; +struct fmemopen_cookie_struct +{ + char *buffer; + int mybuffer; + int binmode; + size_t size; + _IO_off64_t pos; + size_t maxpos; +}; + + +static ssize_t +fmemopen_read (void *cookie, char *b, size_t s) +{ + fmemopen_cookie_t *c; + + c = (fmemopen_cookie_t *) cookie; + + if (c->pos + s > c->size) + { + if ((size_t) c->pos == c->size) + return 0; + s = c->size - c->pos; + } + + memcpy (b, &(c->buffer[c->pos]), s); + + c->pos += s; + if ((size_t) c->pos > c->maxpos) + c->maxpos = c->pos; + + return s; +} + + +static ssize_t +fmemopen_write (void *cookie, const char *b, size_t s) +{ + fmemopen_cookie_t *c; + int addnullc; + + c = (fmemopen_cookie_t *) cookie; + + addnullc = c->binmode == 0 && (s == 0 || b[s - 1] != '\0'); + + if (c->pos + s + addnullc > c->size) + { + if ((size_t) (c->pos + addnullc) == c->size) + { + __set_errno (ENOSPC); + return 0; + } + s = c->size - c->pos - addnullc; + } + + memcpy (&(c->buffer[c->pos]), b, s); + + c->pos += s; + if ((size_t) c->pos > c->maxpos) + { + c->maxpos = c->pos; + if (addnullc) + c->buffer[c->maxpos] = '\0'; + } + + return s; +} + + +static int +fmemopen_seek (void *cookie, _IO_off64_t *p, int w) +{ + _IO_off64_t np; + fmemopen_cookie_t *c; + + c = (fmemopen_cookie_t *) cookie; + + switch (w) + { + case SEEK_SET: + np = *p; + break; + + case SEEK_CUR: + np = c->pos + *p; + break; + + case SEEK_END: + np = (c->binmode ? c->size : c->maxpos) - *p; + break; + + default: + return -1; + } + + if (np < 0 || (size_t) np > c->size) + return -1; + + *p = c->pos = np; + + return 0; +} + + +static int +fmemopen_close (void *cookie) +{ + fmemopen_cookie_t *c; + + c = (fmemopen_cookie_t *) cookie; + + if (c->mybuffer) + free (c->buffer); + free (c); + + return 0; +} + + +FILE * +__old_fmemopen (void *buf, size_t len, const char *mode) +{ + cookie_io_functions_t iof; + fmemopen_cookie_t *c; + + if (__glibc_unlikely (len == 0)) + { + einval: + __set_errno (EINVAL); + return NULL; + } + + c = (fmemopen_cookie_t *) malloc (sizeof (fmemopen_cookie_t)); + if (c == NULL) + return NULL; + + c->mybuffer = (buf == NULL); + + if (c->mybuffer) + { + c->buffer = (char *) malloc (len); + if (c->buffer == NULL) + { + free (c); + return NULL; + } + c->buffer[0] = '\0'; + c->maxpos = 0; + } + else + { + if (__glibc_unlikely ((uintptr_t) len > -(uintptr_t) buf)) + { + free (c); + goto einval; + } + + c->buffer = buf; + + if (mode[0] == 'w') + c->buffer[0] = '\0'; + + c->maxpos = strnlen (c->buffer, len); + } + + c->size = len; + + if (mode[0] == 'a') + c->pos = c->maxpos; + else + c->pos = 0; + + c->binmode = mode[0] != '\0' && mode[1] == 'b'; + + iof.read = fmemopen_read; + iof.write = fmemopen_write; + iof.seek = fmemopen_seek; + iof.close = fmemopen_close; + + return _IO_fopencookie (c, mode, iof); +} +compat_symbol (libc, __old_fmemopen, fmemopen, GLIBC_2_2); +#endif diff --git a/stdio-common/Makefile b/stdio-common/Makefile index 5f8e534..5971e65 100644 --- a/stdio-common/Makefile +++ b/stdio-common/Makefile @@ -57,7 +57,7 @@ tests := tstscanf test_rdwr test-popen tstgetln test-fseek \ bug19 bug19a tst-popen2 scanf13 scanf14 scanf15 bug20 bug21 bug22 \ scanf16 scanf17 tst-setvbuf1 tst-grouping bug23 bug24 \ bug-vfprintf-nargs tst-long-dbl-fphex tst-fphex-wide tst-sprintf3 \ - bug25 tst-printf-round bug26 + bug25 tst-printf-round bug26 tst-fmemopen3 test-srcs = tst-unbputc tst-printf diff --git a/stdio-common/psiginfo.c b/stdio-common/psiginfo.c index 564d237..62a9671 100644 --- a/stdio-common/psiginfo.c +++ b/stdio-common/psiginfo.c @@ -60,7 +60,7 @@ void psiginfo (const siginfo_t *pinfo, const char *s) { char buf[512]; - FILE *fp = fmemopen (buf, sizeof (buf), "w"); + FILE *fp = __fmemopen (buf, sizeof (buf), "w"); if (fp == NULL) { const char *colon; diff --git a/stdio-common/tst-fmemopen.c b/stdio-common/tst-fmemopen.c index 756d6fe..2f2a598 100644 --- a/stdio-common/tst-fmemopen.c +++ b/stdio-common/tst-fmemopen.c @@ -39,7 +39,7 @@ do_test (void) if (fwrite (blah, 1, strlen (blah), fp) != strlen (blah)) { fclose (fp); - return 1; + return 2; } rewind (fp); diff --git a/stdio-common/tst-fmemopen2.c b/stdio-common/tst-fmemopen2.c index c2a4baa..65180f4 100644 --- a/stdio-common/tst-fmemopen2.c +++ b/stdio-common/tst-fmemopen2.c @@ -9,7 +9,8 @@ do_test (void) { int result = 0; char buf[100]; - FILE *fp = fmemopen (buf, sizeof (buf), "w"); +#define nbuf (sizeof (buf)) + FILE *fp = fmemopen (buf, nbuf, "w"); if (fp == NULL) { puts ("fmemopen failed"); @@ -39,7 +40,7 @@ do_test (void) o = ftello (fp); if (o != nstr) { - printf ("third ftello returned %ld, expected %zu\n", o, nstr); + printf ("third ftello returned %ld, expected %zu\n", o, nbuf); result = 1; } rewind (fp); diff --git a/stdio-common/tst-fmemopen3.c b/stdio-common/tst-fmemopen3.c new file mode 100644 index 0000000..1ff0920 --- /dev/null +++ b/stdio-common/tst-fmemopen3.c @@ -0,0 +1,206 @@ +/* fmemopen tests for append and read mode. + Copyright (C) 2014 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include +#include +#include +#include + +static void +print_buffer (const char *s, size_t n) +{ + size_t i; + for (i=0; i