From patchwork Sun May 7 22:56:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ahelenia_Ziemia=C5=84ska?= X-Patchwork-Id: 68893 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 610BE3858C66 for ; Sun, 7 May 2023 22:56:32 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 610BE3858C66 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1683500192; bh=jmOnu3WIWc50SYlRsDZ7lvNk7kqFPFzfpV+INRSBJNM=; h=Date:Cc:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:From; b=nFy8jQjaTF0ZCr1MgUsloQkskhTAtpZ7VGynoaugUdukVs/Qzkm/LDhy9bMlwDWWb zCxge8jq8atezj4YR5GJJRAkXO49X9dAR1PCMZn9JWR+e4cuoR/LTDk/TJoa5BpoLG Dv2aIdcvUSl+LRobyqD38HTYN9O0xgNd1B6dP0PU= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from tarta.nabijaczleweli.xyz (unknown [139.28.40.42]) by sourceware.org (Postfix) with ESMTP id 5DF2D3858D28 for ; Sun, 7 May 2023 22:56:08 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 5DF2D3858D28 Received: from tarta.nabijaczleweli.xyz (unknown [192.168.1.250]) by tarta.nabijaczleweli.xyz (Postfix) with ESMTPSA id 5F6CC7A48 for ; Mon, 8 May 2023 00:56:07 +0200 (CEST) Date: Mon, 8 May 2023 00:56:06 +0200 Cc: libc-alpha@sourceware.org Subject: [PATCH v4 1/3] posix: add (failing) test for REG_STARTEND Message-ID: <1d5642ecb4bb477c9fd7e1ebaee868fe4ccbefc7.1683500149.git.nabijaczleweli@nabijaczleweli.xyz> MIME-Version: 1.0 Content-Disposition: inline User-Agent: NeoMutt/20230407 X-Spam-Status: No, score=-9.7 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, KAM_INFOUSMEBIZ, MISSING_HEADERS, RDNS_DYNAMIC, SPF_HELO_PASS, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: =?utf-8?b?0L3QsNCxIHZpYSBMaWJjLWFscGhh?= From: =?utf-8?q?Ahelenia_Ziemia=C5=84ska?= Reply-To: =?utf-8?b?0L3QsNCx?= Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Sender: "Libc-alpha" This test passes on NetBSD, the illumos gate, and musl with https://www.openwall.com/lists/musl/2023/04/20/2; it's nothing revolutionary and the behaviour it tests is largely guaranteed by the 4.4BSD-Lite manual; nevertheless, it currently fails with tst-reg-startend.c: ^a: a^@c: no match$ tst-reg-startend.c: ^a: a^@c: wanted {1, 2}, got {1, 4}$ tst-reg-startend.c: ^a: abc: no match$ tst-reg-startend.c: ^a: abc: wanted {1, 2}, got {1, 4}$ tst-reg-startend.c: ^a.c$: a^@c: no match$ tst-reg-startend.c: ^a.c$: abc: no match$ tst-reg-startend.c: ^a.*c$: a^@c: no match$ tst-reg-startend.c: ^a.*c$: abc: no match$ tst-reg-startend.c: ^a[^c]c$: a^@c: no match$ tst-reg-startend.c: ^a[^c]c$: abc: no match$ tst-reg-startend.c: ^a..: a^@c: no match$ tst-reg-startend.c: ^a..: abc: no match$ tst-reg-startend.c: ..c: a^@c: no match$ The test may also be compiled stand-alone (-DSTANDALONE) and on all platforms that have the interface (hence the macro to initialise regmatch_ts, which start with pointer fields on the illumos gate), for ease of testing and inclusion in other test suites. Signed-off-by: Ahelenia Ziemiańska Signed-off-by: Ahelenia Ziemiańska --- Resending after a week; clean rebase. posix/Makefile | 1 + posix/tst-reg-startend.c | 124 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 125 insertions(+) create mode 100644 posix/tst-reg-startend.c diff --git a/posix/Makefile b/posix/Makefile index cc77e939ad..24aeb781ca 100644 --- a/posix/Makefile +++ b/posix/Makefile @@ -295,6 +295,7 @@ tests := \ tst-posix_spawn-setsid \ tst-preadwrite \ tst-preadwrite64 \ + tst-reg-startend \ tst-regcomp-truncated \ tst-regex \ tst-regex2 \ diff --git a/posix/tst-reg-startend.c b/posix/tst-reg-startend.c new file mode 100644 index 0000000000..c3bfac0359 --- /dev/null +++ b/posix/tst-reg-startend.c @@ -0,0 +1,124 @@ +/* Permission to use, copy, modify, and/or distribute this software for any + purpose with or without fee is hereby granted. + + THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. */ + +#include +#include +#include +#include +#include +#include + + +#define M(s, e) (regmatch_t) {.rm_so = s, .rm_eo = e} +#define MEQ(l, r) ((l).rm_so == (r).rm_so && (l).rm_eo == (r).rm_eo) + +static const regmatch_t bound = M(1, 4); + +static const char *const regex_ac[] = + {"^a", "c$", "^a.c$", "^a.*c$", "^a[^c]c$", "^a..", "..c", "[^z]c", NULL}; +static const char *const regex_aa[] = + {"^", "^a", "a$", "^\\(a\\).\\1$", "^a[^a]*", NULL}; +static const char *const data_ac[] = {"_a\0cdef", "_abcdef"}; +static const char *const data_aa[] = {"_a\0adef", "_abadef"}; +static const regmatch_t results_ac[] = + {M(1, 2), M(3, 4), M(1, 4), M(1, 4), M(1, 4), M(1, 4), M(1, 4), M(2, 4)}; +static const regmatch_t results_aa[] = + {M(1, 1), M(1, 2), M(3, 4), M(1, 4), M(1, 3)}; +static_assert(sizeof(regex_ac) / sizeof(*regex_ac) - 1 == + sizeof(results_ac) / sizeof(*results_ac), ""); +static_assert(sizeof(regex_aa) / sizeof(*regex_aa) - 1 == + sizeof(results_aa) / sizeof(*results_aa), ""); + + +static bool +testbunch (const char *const *regexes, const char *const data[static 2], + const regmatch_t *results) +{ +#define BASEERR(data) \ + err = true, \ + fprintf (stdout, __FILE__ ": %s: ", *regexes), \ + fwrite (data[i] + bound.rm_so, 1, bound.rm_eo - bound.rm_so, stdout) + + bool err = false; + for (; *regexes; ++regexes, ++results) + { + regex_t rgx; + assert (!regcomp (&rgx, *regexes, 0)); + + for (size_t i = 0; i < 2; ++i) + { + regmatch_t match = bound; + if (regexec (&rgx, data[i], 1, &match, REG_STARTEND)) + BASEERR(data), fputs (": no match\n", stdout); + + if (!MEQ(match, *results)) + BASEERR(data), fprintf (stdout, ": wanted {%d, %d}, got {%d, %d}\n", + (int)results->rm_so, (int)results->rm_eo, + (int)match.rm_so, (int)match.rm_eo); + } + + regfree(&rgx); + } + + return err; +} + + +static const char *const mb_data[2] = {"_aaćdef", "_aćdef"}; +static const bool mb_exp[] = {false, true}; + +static bool +testmb (void) +{ + bool err = false; + regex_t rgx; + const char *const regexes[] = {"ać"}; + assert (!regcomp (&rgx, *regexes, 0)); + + for (size_t i = 0; i < 2; ++i) + { + regmatch_t match = bound; + if (regexec (&rgx, mb_data[i], 1, &match, REG_STARTEND) == mb_exp[i]) + BASEERR(mb_data), fprintf (stdout, ": %s match\n", + mb_exp[i] ? "no" : "yes"); + + if (!MEQ(match, bound)) + BASEERR(mb_data), fprintf (stdout, ": wanted {%d, %d}, got {%d, %d}\n", + (int)bound.rm_so, (int)bound.rm_eo, + (int)match.rm_so, (int)match.rm_eo); + } + + regfree(&rgx); + return err; +} + + +static int +do_test (int argc, char **argv) +{ + (void) argc, (void) argv; + assert (setlocale (LC_ALL, "C.UTF-8")); + + return testbunch (regex_ac, data_ac, results_ac) || + testbunch (regex_aa, data_aa, results_aa) || + testmb (); +} + + +#ifndef STANDALONE +#include "../test-skeleton.c" +#else +int +main(int argc, char **argv) +{ + return do_test(argc, argv); +} +#endif From patchwork Sun May 7 22:56:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ahelenia_Ziemia=C5=84ska?= X-Patchwork-Id: 68895 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 18764385828D for ; Sun, 7 May 2023 22:57:14 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 18764385828D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1683500234; bh=49vGja+lu7SJYFPm4KOpyoiuJcAufxCoDnj0M/7m7TU=; h=Date:Cc:Subject:References:In-Reply-To:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=FEJI2iFyGp+mUBTx7+kGvuvMfwvZO4Yf6J+X/RJI9hAh/haNartsJsxK3gcUBZ/Ku gBZ5JgVb9xhKlZ3F5JatUmniV8MkOJs5g7BvHGOTYS1RZnGNO9hpshClLpK8xrthK1 NYyYT7zGH93p9PIYWAHTF69/d0Dlimy3gWYcVNlc= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from tarta.nabijaczleweli.xyz (unknown [139.28.40.42]) by sourceware.org (Postfix) with ESMTP id 459E63858C74 for ; Sun, 7 May 2023 22:56:13 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 459E63858C74 Received: from tarta.nabijaczleweli.xyz (unknown [192.168.1.250]) by tarta.nabijaczleweli.xyz (Postfix) with ESMTPSA id B6E6178C4 for ; Mon, 8 May 2023 00:56:12 +0200 (CEST) Date: Mon, 8 May 2023 00:56:11 +0200 Cc: libc-alpha@sourceware.org Subject: [PATCH v4 2/3] posix: regcomp(): clear RE_DOT_NOT_NULL Message-ID: References: <1d5642ecb4bb477c9fd7e1ebaee868fe4ccbefc7.1683500149.git.nabijaczleweli@nabijaczleweli.xyz> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <1d5642ecb4bb477c9fd7e1ebaee868fe4ccbefc7.1683500149.git.nabijaczleweli@nabijaczleweli.xyz> User-Agent: NeoMutt/20230407 X-Spam-Status: No, score=-9.7 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, KAM_INFOUSMEBIZ, MISSING_HEADERS, RDNS_DYNAMIC, SPF_HELO_PASS, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: =?utf-8?b?0L3QsNCxIHZpYSBMaWJjLWFscGhh?= From: =?utf-8?q?Ahelenia_Ziemia=C5=84ska?= Reply-To: =?utf-8?b?0L3QsNCx?= Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Sender: "Libc-alpha" The POSIX API always stops at first NUL so there's no change for that. The BSD REG_STARTEND API, with its explicit range, can include NULs within that range, and those NULs are matched with . and [^]. Heretofor, for a string of "a\0c", glibc would match "[^q]c", but not ".c". This is both inconsistent and nonconformant to BSD REG_STARTEND. With this patch, they're identical like you'd expect, and the tst-reg-startend.c: ..c: a^@c: no match$ failure is removed. Another approach would be to remove it from _RE_SYNTAX_POSIX_COMMON, but it's unclear to me what the custody chain is like for that and what other regex APIs glibc offers that could be affected by this. Signed-off-by: Ahelenia Ziemiańska --- posix/regcomp.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/posix/regcomp.c b/posix/regcomp.c index 12650714c0..a928ef6c2d 100644 --- a/posix/regcomp.c +++ b/posix/regcomp.c @@ -462,7 +462,7 @@ regcomp (regex_t *__restrict preg, const char *__restrict pattern, int cflags) { reg_errcode_t ret; reg_syntax_t syntax = ((cflags & REG_EXTENDED) ? RE_SYNTAX_POSIX_EXTENDED - : RE_SYNTAX_POSIX_BASIC); + : RE_SYNTAX_POSIX_BASIC) & ~RE_DOT_NOT_NULL; preg->buffer = NULL; preg->allocated = 0; From patchwork Sun May 7 22:56:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ahelenia_Ziemia=C5=84ska?= X-Patchwork-Id: 68894 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id BCAA03858430 for ; Sun, 7 May 2023 22:56:42 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org BCAA03858430 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1683500202; bh=wBag7Xaap3GtycCMxUDXBkKWAZEXaX4Rx/Be9Cdu1Hk=; h=Date:Cc:Subject:References:In-Reply-To:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=pWYaVewTxxCAvSPcgSyPSiDjtABnz2dbdwL/fxH/iaMCbMMWxWdwSGbKwAuTFA5qo 8ygGrp2LDWfdI4OdbTDDiUQiEE7m94teu+COzeqdKkpcqzb9B8iZkv7xg6xOJCFYD+ Z03SjUiPSlTefet8GES4/WTuRuLPmKSMMJUfwxTA= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from tarta.nabijaczleweli.xyz (unknown [139.28.40.42]) by sourceware.org (Postfix) with ESMTP id D66933858414 for ; Sun, 7 May 2023 22:56:19 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org D66933858414 Received: from tarta.nabijaczleweli.xyz (unknown [192.168.1.250]) by tarta.nabijaczleweli.xyz (Postfix) with ESMTPSA id 31B5F75F4 for ; Mon, 8 May 2023 00:56:19 +0200 (CEST) Date: Mon, 8 May 2023 00:56:18 +0200 Cc: libc-alpha@sourceware.org Subject: [PATCH v4 3/3] posix: regexec(): fix REG_STARTEND, pmatch->rm_so != 0 w/^ anchor Message-ID: References: <1d5642ecb4bb477c9fd7e1ebaee868fe4ccbefc7.1683500149.git.nabijaczleweli@nabijaczleweli.xyz> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <1d5642ecb4bb477c9fd7e1ebaee868fe4ccbefc7.1683500149.git.nabijaczleweli@nabijaczleweli.xyz> User-Agent: NeoMutt/20230407 X-Spam-Status: No, score=-9.7 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, KAM_INFOUSMEBIZ, MISSING_HEADERS, RDNS_DYNAMIC, SPF_HELO_PASS, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: =?utf-8?b?0L3QsNCxIHZpYSBMaWJjLWFscGhh?= From: =?utf-8?q?Ahelenia_Ziemia=C5=84ska?= Reply-To: =?utf-8?b?0L3QsNCx?= Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Sender: "Libc-alpha" re_search_internal () starts with /* If initial states with non-begbuf contexts have no elements, the regex must be anchored. If preg->newline_anchor is set, we'll never use init_state_nl, so do not check it. */ if (dfa->init_state->nodes.nelem == 0 && dfa->init_state_word->nodes.nelem == 0 && (dfa->init_state_nl->nodes.nelem == 0 || !preg->newline_anchor)) { if (start != 0 && last_start != 0) return REG_NOMATCH; start = last_start = 0; } and heretofor start and last_start (for example when "abc", {1, 2}, so matching just the "b") were != 0, and the return was taken for a "^b" regex, which is erroneous. Fix this by giving re_search_internal (string+rm_so, start=0), then fixing up the returned matches in an after-pass. This brings us to compatibility with the BSD spec and implementations. Signed-off-by: Ahelenia Ziemiańska --- posix/regexec.c | 41 ++++++++++++++++++++++++++++------------- 1 file changed, 28 insertions(+), 13 deletions(-) diff --git a/posix/regexec.c b/posix/regexec.c index bd0cd412d0..2ef868e1f6 100644 --- a/posix/regexec.c +++ b/posix/regexec.c @@ -187,38 +187,53 @@ static reg_errcode_t extend_buffers (re_match_context_t *mctx, int min_len); string; if REG_NOTEOL is set, then $ does not match at the end. Return 0 if a match is found, REG_NOMATCH if not, REG_BADPAT if - EFLAGS is invalid. */ + EFLAGS is invalid. + + If REG_STARTEND, the bounds are + [STRING + PMATCH->rm_so, STRING + PMATCH->rm_eo) + instead of the usual + [STRING, STRING + strlen(STRING)), + but returned matches are still referenced to STRING, + and matching is unaffected (i.e. "abc", {1, 2} matches regex "^b$"). + re_search_internal () has a built-in assumption of + (start != 0) <=> (^ doesn't match), so give it a truncated view + and fix up the matches afterward. */ int regexec (const regex_t *__restrict preg, const char *__restrict string, size_t nmatch, regmatch_t pmatch[_REGEX_NELTS (nmatch)], int eflags) { reg_errcode_t err; - Idx start, length; + Idx startoff = 0, length; re_dfa_t *dfa = preg->buffer; + size_t i = 0; if (eflags & ~(REG_NOTBOL | REG_NOTEOL | REG_STARTEND)) return REG_BADPAT; if (eflags & REG_STARTEND) { - start = pmatch[0].rm_so; - length = pmatch[0].rm_eo; + startoff = pmatch[0].rm_so; + string += startoff; + length = pmatch[0].rm_eo - startoff; } else - { - start = 0; - length = strlen (string); - } + length = strlen (string); lock_lock (dfa->lock); if (preg->no_sub) - err = re_search_internal (preg, string, length, start, length, - length, 0, NULL, eflags); - else - err = re_search_internal (preg, string, length, start, length, - length, nmatch, pmatch, eflags); + nmatch = 0; + err = re_search_internal (preg, string, length, 0, length, + length, nmatch, pmatch, eflags); lock_unlock (dfa->lock); + + if (err == REG_NOERROR && startoff) + for (i = 0; i < nmatch; ++i) + if (pmatch[i].rm_so != -1) + { + pmatch[i].rm_so += startoff; + pmatch[i].rm_eo += startoff; + } return err != REG_NOERROR; }