From patchwork Wed Jul 12 14:52:29 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adhemerval Zanella Netto X-Patchwork-Id: 21560 Received: (qmail 57229 invoked by alias); 12 Jul 2017 14:52:38 -0000 Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-alpha-owner@sourceware.org Delivered-To: mailing list libc-alpha@sourceware.org Received: (qmail 57211 invoked by uid 89); 12 Jul 2017 14:52:37 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-26.4 required=5.0 tests=BAYES_00, GIT_PATCH_0, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, RCVD_IN_DNSWL_NONE, RCVD_IN_SORBS_SPAM, SPF_PASS autolearn=ham version=3.3.2 spammy= X-HELO: mail-qt0-f170.google.com X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=x35nYZYTLoJdJiGsM00p6D0cCyAjDqeA9UNxx2ulIF8=; b=HdqPY0F7IUr1LFCzy817yP1LeEnDjdyf9RgYjh1f/T+73JaY1YE0ypAXF1h7lZZvr+ U+iV/1IexDoduKCspaO5RaGE4FJ/ZzjYjesPZKpK4+Z9Udh8Iq4tS3aYF5S1bB++8TX9 a7wCfTyZQe12R2SaMwamNdxvKSO2MdqCYTQEHA0OJyYVbYzXixaQvoFXQQzy7kLjFxx9 6y1y4uwna/HNHkerjOG7FclNzKYhVUzchQNu/NJnU0jD35HaCbN22nSdLTUK6mk16mV7 ah6oYuimS5D54DB1k7KEU/Hv/y5NTGgimq1AcC5qVXoahlTyn0gRyhO23cZdtlKQAMrn nEbA== X-Gm-Message-State: AIVw111iSDtNVFUfucQSrnfTP6Toh6pC27Z1IKD0uxG2IKBAXQGMKxyJ 8l161cD+pYHchuQh9Xzvdw== X-Received: by 10.237.53.164 with SMTP id c33mr3399674qte.191.1499871152810; Wed, 12 Jul 2017 07:52:32 -0700 (PDT) Subject: Re: [PATCH] libio: Fix open_memstream flush (NULL) To: Florian Weimer Cc: libc-alpha@sourceware.org References: <1499703585-17933-1-git-send-email-adhemerval.zanella@linaro.org> <87a84ao83j.fsf@mid.deneb.enyo.de> From: Adhemerval Zanella Message-ID: <675937a5-84c5-130c-8834-eaeadfef0898@linaro.org> Date: Wed, 12 Jul 2017 11:52:29 -0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.2.1 MIME-Version: 1.0 In-Reply-To: <87a84ao83j.fsf@mid.deneb.enyo.de> On 11/07/2017 20:40, Florian Weimer wrote: > * Adhemerval Zanella: > >> POSIX specifies that the state and size after a fflush on a stream >> opened by open_memstream should be updated and current implementation >> does not act accordingly on a fflush (NULL) (meant to act on all opened >> streams). >> >> This patch fixes it for open_{w}memstream and also adjust the >> tst-memstream3 to use libsupport. > > Subject is wrong, it should be fflush. Ack, fixed locally. > > Please put the test case into a new file, and change it to call fflush > (NULL); it currently calls fflush (fp). As a result, it does not > actually test the change. Indeed, I used you suggestion to call _IO_mem_sync for EOF. For tests, used the previous tst-{w}memstream3.c one to avoid all the boilerplate required to add a new tests, but I think adding a new one should be better to reference the bug and backport. > > You also need to verify that this does not call malloc: > > #include > #include > #include > > int > main (void) > { > char * buffer = NULL; > size_t length = 0; > FILE *fp = open_memstream (&buffer, &length); > if (fp == NULL) > err (1, "open_memstream"); > abort (); > } > > I think we don't want the malloc call for a call to exit, either. > Similarly for __stack_chk_fail, but that's something we need to fix > there. I am not following, what exactly should not call malloc here? Since the open_memstream is a valid call, it will allocate both the initial internal FILE object (libio/memstream.c:77) and initial buffer (libio/memstream.c:84). Are you saying that we should start with an empty buffer instead? If it is the for I think we can do it, but it would require more intrusive changes I think. > >> +static int >> +_IO_mem_overflow (_IO_FILE *fp, int c) >> +{ >> + int ret = _IO_str_overflow (fp, c); >> + >> + struct _IO_FILE_memstream *mp = (struct _IO_FILE_memstream *) fp; >> + *mp->sizeloc = fp->_IO_write_ptr - fp->_IO_write_base; >> + >> + return ret; >> +} > > _IO_str_overflow does not correctly process an EOF argument, but > _IO_flush_all_lockp calls the overflow function with EOF. This is the > reason why the test with fflush (NULL) fails. > > The most direct fix appears to be to call _IO_mem_sync for c == EOF. > > Let me suggest again that libio changes are not appropriate during the > freeze. > So do you prefer to just add a workaround to disable the single-thread optimization for open_memstream and fix it on 2.27? IMHO this fix touches only the open_memstream itself, so it should not regress any other libio code so I would prefer than carry a workaround for 2.26. In any case, I updated the patch based on your review below. It passed both your threaded case for open_memstream that is failing on master the testcase for BZ#21735. --- [BZ #21735] * libio/memstream.c (_IO_mem_overflow): New function. (_IO_mem_jumps): User _IO_mem_overflow instead of _IO_str_overflow. (__open_memstream): Call _IO_link_in on the internal FILE object. * libio/wmemstream.c (_IO_wmem_overflow): New function. (_IO_wmem_jumps): User _IO_wmem_overflow instead of _IO_wstr_overflow. (__open_wmemstream): Call _IO_link_in on the internal FILE object. * libio/Makefile (test): Add tst-memstream4 and tst-wmemstream4. * libio/tst-memstream4.c: New file. * libio/tst-wmemstream4.c: Likewise. * libio/tst-memstream.h: Likewise. --- diff --git a/libio/Makefile b/libio/Makefile index a002a33..6ee809a 100644 --- a/libio/Makefile +++ b/libio/Makefile @@ -57,8 +57,8 @@ tests = tst_swprintf tst_wprintf tst_swscanf tst_wscanf tst_getwc tst_putwc \ tst-mmap-eofsync tst-mmap-fflushsync bug-mmap-fflush \ tst-mmap2-eofsync tst-mmap-offend bug-fopena+ bug-wfflush \ bug-ungetc2 bug-ftell bug-ungetc3 bug-ungetc4 tst-fopenloc2 \ - tst-memstream1 tst-memstream2 tst-memstream3 \ - tst-wmemstream1 tst-wmemstream2 tst-wmemstream3 \ + tst-memstream1 tst-memstream2 tst-memstream3 tst-memstream4 \ + tst-wmemstream1 tst-wmemstream2 tst-wmemstream3 tst-wmemstream4 \ bug-memstream1 bug-wmemstream1 \ tst-setvbuf1 tst-popen1 tst-fgetwc bug-wsetpos tst-fseek \ tst-fwrite-error tst-ftell-partial-wide tst-ftell-active-handler \ diff --git a/libio/memstream.c b/libio/memstream.c index f83d4a5..5fdd071 100644 --- a/libio/memstream.c +++ b/libio/memstream.c @@ -31,13 +31,14 @@ struct _IO_FILE_memstream static int _IO_mem_sync (_IO_FILE* fp) __THROW; static void _IO_mem_finish (_IO_FILE* fp, int) __THROW; +static int _IO_mem_overflow (_IO_FILE *fp, int c) __THROW; static const struct _IO_jump_t _IO_mem_jumps libio_vtable = { JUMP_INIT_DUMMY, JUMP_INIT (finish, _IO_mem_finish), - JUMP_INIT (overflow, _IO_str_overflow), + JUMP_INIT (overflow, _IO_mem_overflow), JUMP_INIT (underflow, _IO_str_underflow), JUMP_INIT (uflow, _IO_default_uflow), JUMP_INIT (pbackfail, _IO_str_pbackfail), @@ -87,6 +88,7 @@ __open_memstream (char **bufloc, _IO_size_t *sizeloc) return NULL; } _IO_init_internal (&new_f->fp._sf._sbf._f, 0); + _IO_link_in ((struct _IO_FILE_plus *) &new_f->fp._sf._sbf); _IO_JUMPS_FILE_plus (&new_f->fp._sf._sbf) = &_IO_mem_jumps; _IO_str_init_static_internal (&new_f->fp._sf, buf, _IO_BUFSIZ, buf); new_f->fp._sf._sbf._f._flags &= ~_IO_USER_BUF; @@ -137,3 +139,12 @@ _IO_mem_finish (_IO_FILE *fp, int dummy) _IO_str_finish (fp, 0); } + +static int +_IO_mem_overflow (_IO_FILE *fp, int c) +{ + /* Updates the returned size location on stream flush. */ + if (c == EOF) + return _IO_mem_sync (fp); + return _IO_str_overflow (fp, c); +} diff --git a/libio/tst-memstream.h b/libio/tst-memstream.h new file mode 100644 index 0000000..249a588 --- /dev/null +++ b/libio/tst-memstream.h @@ -0,0 +1,64 @@ +/* Common definitions for open_memstream tests. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include +#include +#include +#include +#include +#include + +#include + +#ifdef TEST_WCHAR +# include + +/* Straighforward implementation so tst-memstream3 could use check + fwrite on open_memstream. */ +static size_t __attribute__ ((used)) +fwwrite (const void *ptr, size_t size, size_t nmemb, FILE *arq) +{ + const wchar_t *wcs = (const wchar_t*) (ptr); + for (size_t s = 0; s < size; s++) + { + for (size_t n = 0; n < nmemb; n++) + if (fputwc (wcs[n], arq) == WEOF) + return n; + } + return size * nmemb; +} + +# define CHAR_T wchar_t +# define W(o) L##o +# define OPEN_MEMSTREAM open_wmemstream +# define PRINTF wprintf +# define FWRITE fwwrite +# define FPUTC fputwc +# define STRCMP wcscmp +#else +# define CHAR_T char +# define W(o) o +# define OPEN_MEMSTREAM open_memstream +# define PRINTF printf +# define FWRITE fwrite +# define FPUTC fputc +# define STRCMP strcmp +#endif + +#define S(s) S1 (s) +#define S1(s) #s diff --git a/libio/tst-memstream4.c b/libio/tst-memstream4.c new file mode 100644 index 0000000..d810063 --- /dev/null +++ b/libio/tst-memstream4.c @@ -0,0 +1,58 @@ +/* Test for open_memstream BZ #21735. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "tst-memstream.h" + +static void +mcheck_abort (enum mcheck_status ev) +{ + printf ("mecheck failed with status %d\n", (int) ev); + exit (1); +} + +static int +do_test (void) +{ + mcheck_pedantic (mcheck_abort); + + CHAR_T *buf; + size_t size; + + FILE *fp = OPEN_MEMSTREAM (&buf, &size); + if (fp == NULL) + FAIL_RET ("%s failed", S(OPEN_MEMSTREAM)); + + if (FPUTC (W('a'), fp) != W('a')) + FAIL_RET ("%s failed: %m", S(FPUTC)); + + if (fflush (NULL) != 0) + FAIL_RET ("fflush failed: %m"); + + if (size != 1) + FAIL_RET ("size != 1"); + + if (fileno (fp) != -1) + FAIL_RET ("fileno returned a valid integer descriptor"); + + if (fclose (fp) != 0) + FAIL_RET ("fclose: %m"); + + return 0; +} + +#include diff --git a/libio/tst-wmemstream4.c b/libio/tst-wmemstream4.c new file mode 100644 index 0000000..9dbf615 --- /dev/null +++ b/libio/tst-wmemstream4.c @@ -0,0 +1,20 @@ +/* Test for open_wmemstream BZ #21735. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#define TEST_WCHAR +#include diff --git a/libio/wmemstream.c b/libio/wmemstream.c index 5bc77f5..3aa5f8a 100644 --- a/libio/wmemstream.c +++ b/libio/wmemstream.c @@ -32,13 +32,14 @@ struct _IO_FILE_wmemstream static int _IO_wmem_sync (_IO_FILE* fp) __THROW; static void _IO_wmem_finish (_IO_FILE* fp, int) __THROW; +static int _IO_wmem_overflow (_IO_FILE *fp, int c) __THROW; static const struct _IO_jump_t _IO_wmem_jumps libio_vtable = { JUMP_INIT_DUMMY, JUMP_INIT (finish, _IO_wmem_finish), - JUMP_INIT (overflow, (_IO_overflow_t) _IO_wstr_overflow), + JUMP_INIT (overflow, (_IO_overflow_t) _IO_wmem_overflow), JUMP_INIT (underflow, (_IO_underflow_t) _IO_wstr_underflow), JUMP_INIT (uflow, (_IO_underflow_t) _IO_wdefault_uflow), JUMP_INIT (pbackfail, (_IO_pbackfail_t) _IO_wstr_pbackfail), @@ -89,6 +90,7 @@ open_wmemstream (wchar_t **bufloc, _IO_size_t *sizeloc) } _IO_no_init (&new_f->fp._sf._sbf._f, 0, 0, &new_f->wd, &_IO_wmem_jumps); _IO_fwide (&new_f->fp._sf._sbf._f, 1); + _IO_link_in ((struct _IO_FILE_plus *) &new_f->fp._sf._sbf); _IO_wstr_init_static (&new_f->fp._sf._sbf._f, buf, _IO_BUFSIZ / sizeof (wchar_t), buf); new_f->fp._sf._sbf._f._flags2 &= ~_IO_FLAGS2_USER_WBUF; @@ -142,3 +144,12 @@ _IO_wmem_finish (_IO_FILE *fp, int dummy) _IO_wstr_finish (fp, 0); } + +static int +_IO_wmem_overflow (_IO_FILE *fp, int c) +{ + /* Updates the returned size location on stream flush. */ + if (c == EOF) + return _IO_wmem_sync (fp); + return _IO_wstr_overflow (fp, c); +}