From patchwork Fri Apr 28 12:36:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ahelenia_Ziemia=C5=84ska?= X-Patchwork-Id: 68484 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 2B80C389942A for ; Fri, 28 Apr 2023 12:39:49 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 2B80C389942A DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1682685589; bh=tlNEtebp7KXbBMxWmreFZFf963d796h0ItWHP+dBXhY=; h=Date:To:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:From; b=LkISCF5p6bOh05X/a8J7J2TzLUsSvEle0cIwLdmGVjfWumbed5GTOJ3qU4IofTPjq Trkqf89VU9kkhhnooqTEGhEXfYOYF9bJgNur9NOi7M6CiwInsGGs+Yl3L1jMK2tNuN kEuFHnNtFMOhosKNAgUGbvWidqCwd137+lUEbwa8= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from tarta.nabijaczleweli.xyz (unknown [139.28.40.42]) by sourceware.org (Postfix) with ESMTP id 6BA9E385277B for ; Fri, 28 Apr 2023 12:36:33 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 6BA9E385277B Received: from tarta.nabijaczleweli.xyz (unknown [192.168.1.250]) by tarta.nabijaczleweli.xyz (Postfix) with ESMTPSA id 8DE357404 for ; Fri, 28 Apr 2023 14:36:29 +0200 (CEST) Date: Fri, 28 Apr 2023 14:36:28 +0200 To: libc-alpha@sourceware.org Subject: [PATCH v3 1/3] posix: add (failing) test for REG_STARTEND Message-ID: <9dee3d2ba84f09e883cf7a7dfcda486fda735382.1682685278.git.nabijaczleweli@nabijaczleweli.xyz> MIME-Version: 1.0 Content-Disposition: inline User-Agent: NeoMutt/20230407 X-Spam-Status: No, score=-9.7 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, KAM_INFOUSMEBIZ, RDNS_DYNAMIC, SPF_HELO_PASS, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: =?utf-8?b?0L3QsNCxIHZpYSBMaWJjLWFscGhh?= From: =?utf-8?q?Ahelenia_Ziemia=C5=84ska?= Reply-To: =?utf-8?b?0L3QsNCx?= Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Sender: "Libc-alpha" This test passes on NetBSD, the illumos gate, and musl with https://www.openwall.com/lists/musl/2023/04/20/2; it's nothing revolutionary and the behaviour it tests is largely guaranteed by the 4.4BSD-Lite manual; nevertheless, it currently fails with tst-reg-startend.c: ^a: a^@c: no match$ tst-reg-startend.c: ^a: a^@c: wanted {1, 2}, got {1, 4}$ tst-reg-startend.c: ^a: abc: no match$ tst-reg-startend.c: ^a: abc: wanted {1, 2}, got {1, 4}$ tst-reg-startend.c: ^a.c$: a^@c: no match$ tst-reg-startend.c: ^a.c$: abc: no match$ tst-reg-startend.c: ^a.*c$: a^@c: no match$ tst-reg-startend.c: ^a.*c$: abc: no match$ tst-reg-startend.c: ^a[^c]c$: a^@c: no match$ tst-reg-startend.c: ^a[^c]c$: abc: no match$ tst-reg-startend.c: ^a..: a^@c: no match$ tst-reg-startend.c: ^a..: abc: no match$ tst-reg-startend.c: ..c: a^@c: no match$ The test may also be compiled stand-alone (-DSTANDALONE) and on all platforms that have the interface (hence the macro to initialise regmatch_ts, which start with pointer fields on the illumos gate), for ease of testing and inclusion in other test suites. Signed-off-by: Ahelenia Ziemiańska --- Resending after a week, per guidelines. Clean rebase, includes the v2 of the test (idents renamed for compatibility), no changes to the non-test code. Keep me in CC, please. posix/Makefile | 1 + posix/tst-reg-startend.c | 124 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 125 insertions(+) create mode 100644 posix/tst-reg-startend.c diff --git a/posix/Makefile b/posix/Makefile index cc77e939ad..24aeb781ca 100644 --- a/posix/Makefile +++ b/posix/Makefile @@ -295,6 +295,7 @@ tests := \ tst-posix_spawn-setsid \ tst-preadwrite \ tst-preadwrite64 \ + tst-reg-startend \ tst-regcomp-truncated \ tst-regex \ tst-regex2 \ diff --git a/posix/tst-reg-startend.c b/posix/tst-reg-startend.c new file mode 100644 index 0000000000..c3bfac0359 --- /dev/null +++ b/posix/tst-reg-startend.c @@ -0,0 +1,124 @@ +/* Permission to use, copy, modify, and/or distribute this software for any + purpose with or without fee is hereby granted. + + THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. */ + +#include +#include +#include +#include +#include +#include + + +#define M(s, e) (regmatch_t) {.rm_so = s, .rm_eo = e} +#define MEQ(l, r) ((l).rm_so == (r).rm_so && (l).rm_eo == (r).rm_eo) + +static const regmatch_t bound = M(1, 4); + +static const char *const regex_ac[] = + {"^a", "c$", "^a.c$", "^a.*c$", "^a[^c]c$", "^a..", "..c", "[^z]c", NULL}; +static const char *const regex_aa[] = + {"^", "^a", "a$", "^\\(a\\).\\1$", "^a[^a]*", NULL}; +static const char *const data_ac[] = {"_a\0cdef", "_abcdef"}; +static const char *const data_aa[] = {"_a\0adef", "_abadef"}; +static const regmatch_t results_ac[] = + {M(1, 2), M(3, 4), M(1, 4), M(1, 4), M(1, 4), M(1, 4), M(1, 4), M(2, 4)}; +static const regmatch_t results_aa[] = + {M(1, 1), M(1, 2), M(3, 4), M(1, 4), M(1, 3)}; +static_assert(sizeof(regex_ac) / sizeof(*regex_ac) - 1 == + sizeof(results_ac) / sizeof(*results_ac), ""); +static_assert(sizeof(regex_aa) / sizeof(*regex_aa) - 1 == + sizeof(results_aa) / sizeof(*results_aa), ""); + + +static bool +testbunch (const char *const *regexes, const char *const data[static 2], + const regmatch_t *results) +{ +#define BASEERR(data) \ + err = true, \ + fprintf (stdout, __FILE__ ": %s: ", *regexes), \ + fwrite (data[i] + bound.rm_so, 1, bound.rm_eo - bound.rm_so, stdout) + + bool err = false; + for (; *regexes; ++regexes, ++results) + { + regex_t rgx; + assert (!regcomp (&rgx, *regexes, 0)); + + for (size_t i = 0; i < 2; ++i) + { + regmatch_t match = bound; + if (regexec (&rgx, data[i], 1, &match, REG_STARTEND)) + BASEERR(data), fputs (": no match\n", stdout); + + if (!MEQ(match, *results)) + BASEERR(data), fprintf (stdout, ": wanted {%d, %d}, got {%d, %d}\n", + (int)results->rm_so, (int)results->rm_eo, + (int)match.rm_so, (int)match.rm_eo); + } + + regfree(&rgx); + } + + return err; +} + + +static const char *const mb_data[2] = {"_aaćdef", "_aćdef"}; +static const bool mb_exp[] = {false, true}; + +static bool +testmb (void) +{ + bool err = false; + regex_t rgx; + const char *const regexes[] = {"ać"}; + assert (!regcomp (&rgx, *regexes, 0)); + + for (size_t i = 0; i < 2; ++i) + { + regmatch_t match = bound; + if (regexec (&rgx, mb_data[i], 1, &match, REG_STARTEND) == mb_exp[i]) + BASEERR(mb_data), fprintf (stdout, ": %s match\n", + mb_exp[i] ? "no" : "yes"); + + if (!MEQ(match, bound)) + BASEERR(mb_data), fprintf (stdout, ": wanted {%d, %d}, got {%d, %d}\n", + (int)bound.rm_so, (int)bound.rm_eo, + (int)match.rm_so, (int)match.rm_eo); + } + + regfree(&rgx); + return err; +} + + +static int +do_test (int argc, char **argv) +{ + (void) argc, (void) argv; + assert (setlocale (LC_ALL, "C.UTF-8")); + + return testbunch (regex_ac, data_ac, results_ac) || + testbunch (regex_aa, data_aa, results_aa) || + testmb (); +} + + +#ifndef STANDALONE +#include "../test-skeleton.c" +#else +int +main(int argc, char **argv) +{ + return do_test(argc, argv); +} +#endif From patchwork Fri Apr 28 12:36:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ahelenia_Ziemia=C5=84ska?= X-Patchwork-Id: 68481 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id CB24A3856DD9 for ; Fri, 28 Apr 2023 12:38:26 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org CB24A3856DD9 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1682685506; bh=r8aBqLjIvvNazEBFyRMa+xo9vEiqtY3WyfitmspF4bo=; h=Date:To:Subject:References:In-Reply-To:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=g3Z/DD+Tq47dstkCPzRnEDGY215BKQqVa62ZtF+VfAJtsTxAnschW4JmX0zR4NI8D SO/3mhtLkZCp+5cxfq43lf2Jhb8JjfnVXL752hUu93QK/7Oq8iVtNeXP+gJhzV0n8Y IehUaHEmilTQDWkc29WZVY2WgpUoSMoV9jZpVnsE= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from tarta.nabijaczleweli.xyz (unknown [139.28.40.42]) by sourceware.org (Postfix) with ESMTP id 0A4AA3854176 for ; Fri, 28 Apr 2023 12:36:44 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 0A4AA3854176 Received: from tarta.nabijaczleweli.xyz (unknown [192.168.1.250]) by tarta.nabijaczleweli.xyz (Postfix) with ESMTPSA id 5EB0A6CE6 for ; Fri, 28 Apr 2023 14:36:43 +0200 (CEST) Date: Fri, 28 Apr 2023 14:36:42 +0200 To: libc-alpha@sourceware.org Subject: [PATCH v3 2/3] posix: regcomp(): clear RE_DOT_NOT_NULL Message-ID: References: <9dee3d2ba84f09e883cf7a7dfcda486fda735382.1682685278.git.nabijaczleweli@nabijaczleweli.xyz> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <9dee3d2ba84f09e883cf7a7dfcda486fda735382.1682685278.git.nabijaczleweli@nabijaczleweli.xyz> User-Agent: NeoMutt/20230407 X-Spam-Status: No, score=-9.8 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, KAM_INFOUSMEBIZ, RDNS_DYNAMIC, SPF_HELO_PASS, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: =?utf-8?b?0L3QsNCxIHZpYSBMaWJjLWFscGhh?= From: =?utf-8?q?Ahelenia_Ziemia=C5=84ska?= Reply-To: =?utf-8?b?0L3QsNCx?= Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Sender: "Libc-alpha" The POSIX API always stops at first NUL so there's no change for that. The BSD REG_STARTEND API, with its explicit range, can include NULs within that range, and those NULs are matched with . and [^]. Heretofor, for a string of "a\0c", glibc would match "[^q]c", but not ".c". This is both inconsistent and nonconformant to BSD REG_STARTEND. With this patch, they're identical like you'd expect, and the tst-reg-startend.c: ..c: a^@c: no match$ failure is removed. Another approach would be to remove it from _RE_SYNTAX_POSIX_COMMON, but it's unclear to me what the custody chain is like for that and what other regex APIs glibc offers that could be affected by this. Signed-off-by: Ahelenia Ziemiańska --- Keep me in CC, please. posix/regcomp.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/posix/regcomp.c b/posix/regcomp.c index 647b18ba9e..cbd9bfc673 100644 --- a/posix/regcomp.c +++ b/posix/regcomp.c @@ -462,7 +462,7 @@ regcomp (regex_t *__restrict preg, const char *__restrict pattern, int cflags) { reg_errcode_t ret; reg_syntax_t syntax = ((cflags & REG_EXTENDED) ? RE_SYNTAX_POSIX_EXTENDED - : RE_SYNTAX_POSIX_BASIC); + : RE_SYNTAX_POSIX_BASIC) & ~RE_DOT_NOT_NULL; preg->buffer = NULL; preg->allocated = 0; From patchwork Fri Apr 28 12:36:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ahelenia_Ziemia=C5=84ska?= X-Patchwork-Id: 68487 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 82CA53887F4E for ; Fri, 28 Apr 2023 12:41:02 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 82CA53887F4E DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1682685662; bh=LZV95+QMHcEE1Pgsx1a8h4NxYyggPi27jZFW3nOH8p4=; h=Date:To:Subject:References:In-Reply-To:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=o6HVTkIiQnhT+rKiHkMZ3zFQ6U6KqVMI8zhq2gPfcEzPU1r+a60B+abd94Guh8/Zb USsQyokaQ0M+JLZVLD3FS54wSrEzW2yaJnCe1u+7qCEmr70FvN6pqoOzRUZXp01LEN f1qmFQNrehICDVstkAczpfewE8vGPip4RpyTwfY8= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from tarta.nabijaczleweli.xyz (unknown [139.28.40.42]) by sourceware.org (Postfix) with ESMTP id 461923857733 for ; Fri, 28 Apr 2023 12:36:56 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 461923857733 Received: from tarta.nabijaczleweli.xyz (unknown [192.168.1.250]) by tarta.nabijaczleweli.xyz (Postfix) with ESMTPSA id 9BB656CE8 for ; Fri, 28 Apr 2023 14:36:55 +0200 (CEST) Date: Fri, 28 Apr 2023 14:36:54 +0200 To: libc-alpha@sourceware.org Subject: [PATCH v3 3/3] posix: regexec(): fix REG_STARTEND, pmatch->rm_so != 0 w/^ anchor Message-ID: <7b9d17a0ffc408cfe5da6d4aac7fbb2f8a6997b3.1682685278.git.nabijaczleweli@nabijaczleweli.xyz> References: <9dee3d2ba84f09e883cf7a7dfcda486fda735382.1682685278.git.nabijaczleweli@nabijaczleweli.xyz> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <9dee3d2ba84f09e883cf7a7dfcda486fda735382.1682685278.git.nabijaczleweli@nabijaczleweli.xyz> User-Agent: NeoMutt/20230407 X-Spam-Status: No, score=-9.9 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, KAM_INFOUSMEBIZ, RDNS_DYNAMIC, SPF_HELO_PASS, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: =?utf-8?b?0L3QsNCxIHZpYSBMaWJjLWFscGhh?= From: =?utf-8?q?Ahelenia_Ziemia=C5=84ska?= Reply-To: =?utf-8?b?0L3QsNCx?= Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Sender: "Libc-alpha" re_search_internal () starts with /* If initial states with non-begbuf contexts have no elements, the regex must be anchored. If preg->newline_anchor is set, we'll never use init_state_nl, so do not check it. */ if (dfa->init_state->nodes.nelem == 0 && dfa->init_state_word->nodes.nelem == 0 && (dfa->init_state_nl->nodes.nelem == 0 || !preg->newline_anchor)) { if (start != 0 && last_start != 0) return REG_NOMATCH; start = last_start = 0; } and heretofor start and last_start (for example when "abc", {1, 2}, so matching just the "b") were != 0, and the return was taken for a "^b" regex, which is erroneous. Fix this by giving re_search_internal (string+rm_so, start=0), then fixing up the returned matches in an after-pass. This brings us to compatibility with the BSD spec and implementations. Signed-off-by: Ahelenia Ziemiańska --- Keep me in CC, please. posix/regexec.c | 41 ++++++++++++++++++++++++++++------------- 1 file changed, 28 insertions(+), 13 deletions(-) diff --git a/posix/regexec.c b/posix/regexec.c index bd0cd412d0..2ef868e1f6 100644 --- a/posix/regexec.c +++ b/posix/regexec.c @@ -187,38 +187,53 @@ static reg_errcode_t extend_buffers (re_match_context_t *mctx, int min_len); string; if REG_NOTEOL is set, then $ does not match at the end. Return 0 if a match is found, REG_NOMATCH if not, REG_BADPAT if - EFLAGS is invalid. */ + EFLAGS is invalid. + + If REG_STARTEND, the bounds are + [STRING + PMATCH->rm_so, STRING + PMATCH->rm_eo) + instead of the usual + [STRING, STRING + strlen(STRING)), + but returned matches are still referenced to STRING, + and matching is unaffected (i.e. "abc", {1, 2} matches regex "^b$"). + re_search_internal () has a built-in assumption of + (start != 0) <=> (^ doesn't match), so give it a truncated view + and fix up the matches afterward. */ int regexec (const regex_t *__restrict preg, const char *__restrict string, size_t nmatch, regmatch_t pmatch[_REGEX_NELTS (nmatch)], int eflags) { reg_errcode_t err; - Idx start, length; + Idx startoff = 0, length; re_dfa_t *dfa = preg->buffer; + size_t i = 0; if (eflags & ~(REG_NOTBOL | REG_NOTEOL | REG_STARTEND)) return REG_BADPAT; if (eflags & REG_STARTEND) { - start = pmatch[0].rm_so; - length = pmatch[0].rm_eo; + startoff = pmatch[0].rm_so; + string += startoff; + length = pmatch[0].rm_eo - startoff; } else - { - start = 0; - length = strlen (string); - } + length = strlen (string); lock_lock (dfa->lock); if (preg->no_sub) - err = re_search_internal (preg, string, length, start, length, - length, 0, NULL, eflags); - else - err = re_search_internal (preg, string, length, start, length, - length, nmatch, pmatch, eflags); + nmatch = 0; + err = re_search_internal (preg, string, length, 0, length, + length, nmatch, pmatch, eflags); lock_unlock (dfa->lock); + + if (err == REG_NOERROR && startoff) + for (i = 0; i < nmatch; ++i) + if (pmatch[i].rm_so != -1) + { + pmatch[i].rm_so += startoff; + pmatch[i].rm_eo += startoff; + } return err != REG_NOERROR; }