From patchwork Fri Nov 26 09:33:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Jelinek X-Patchwork-Id: 48179 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id B2257385801A for ; Fri, 26 Nov 2021 09:33:36 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org B2257385801A DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1637919216; bh=ROcKh7ip5dZ3hIHfk7QlhI6e/99sqDqe+qYVHC0Nluk=; h=Date:To:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:Cc:From; b=UFcIBOyGnk3ZfRDaPQ1Pd8J/vSFLZU6qI3SrK7okHPr+PNa7GanICEEx8fj3CYQaU d0dwVqMNmk4xhhnUrPvaGswiQnSMrXW5HUmvQdHfmPcn4+9AUCsmo4fFmrbrzAAk/k IQj/DjYEbEU7xl57/kP7cOBtu4hFVhnCHyxWK468= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by sourceware.org (Postfix) with ESMTPS id C04EF3858D39 for ; Fri, 26 Nov 2021 09:33:07 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org C04EF3858D39 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-348-l9VrVETxNXOteySnnZ7U4A-1; Fri, 26 Nov 2021 04:33:06 -0500 X-MC-Unique: l9VrVETxNXOteySnnZ7U4A-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 4CF1581CCB9 for ; Fri, 26 Nov 2021 09:33:05 +0000 (UTC) Received: from tucnak.zalov.cz (unknown [10.39.192.23]) by smtp.corp.redhat.com (Postfix) with ESMTPS id C16AC7AD1E; Fri, 26 Nov 2021 09:33:04 +0000 (UTC) Received: from tucnak.zalov.cz (localhost [127.0.0.1]) by tucnak.zalov.cz (8.16.1/8.16.1) with ESMTPS id 1AQ9X1xw3955415 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT); Fri, 26 Nov 2021 10:33:01 +0100 Received: (from jakub@localhost) by tucnak.zalov.cz (8.16.1/8.16.1/Submit) id 1AQ9X1s83955414; Fri, 26 Nov 2021 10:33:01 +0100 Date: Fri, 26 Nov 2021 10:33:00 +0100 To: Jason Merrill Subject: [PATCH] libcpp: Fix up #__VA_OPT__ handling [PR103415] Message-ID: <20211126093300.GF2646553@tucnak> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Disposition: inline X-Spam-Status: No, score=-5.7 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H2, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jakub Jelinek via Gcc-patches From: Jakub Jelinek Reply-To: Jakub Jelinek Cc: gcc-patches@gcc.gnu.org Errors-To: gcc-patches-bounces+patchwork=sourceware.org@gcc.gnu.org Sender: "Gcc-patches" Hi! stringify_arg uses pfile->u_buff to create the string literal. Unfortunately, paste_tokens -> _cpp_lex_direct -> lex_number -> _cpp_unaligned_alloc can in some cases use pfile->u_buff too, which results in losing everything prepared for the string literal until the token pasting. The following patch fixes that by not calling paste_token during the construction of the string literal, but doing that before. All the tokens we are processing have been pushed into a token buffer using tokens_buff_add_token so it is fine if we paste some of them in that buffer (successful pasting creates a new token in that buffer), move following tokens if any to make it contiguous, pop (throw away) the extra tokens at the end and then do stringify_arg. Bootstrapped/regtested on x86_64-linux and i686-linux, ok for trunk? 2021-11-26 Jakub Jelinek PR preprocessor/103415 libcpp/ * macro.c (stringify_arg): Remove va_opt argument and va_opt handling. (replace_args): Adjust callers. For #__VA_OPT__, perform token pasting in a separate loop before stringify_arg call. gcc/testsuite/ * c-c++-common/cpp/va-opt-8.c: New test. Jakub --- libcpp/macro.c.jj 2021-11-23 10:26:30.595792246 +0100 +++ libcpp/macro.c 2021-11-25 17:55:39.149217299 +0100 @@ -295,7 +295,7 @@ static cpp_context *next_context (cpp_re static const cpp_token *padding_token (cpp_reader *, const cpp_token *); static const cpp_token *new_string_token (cpp_reader *, uchar *, unsigned int); static const cpp_token *stringify_arg (cpp_reader *, const cpp_token **, - unsigned int, bool); + unsigned int); static void paste_all_tokens (cpp_reader *, const cpp_token *); static bool paste_tokens (cpp_reader *, location_t, const cpp_token **, const cpp_token *); @@ -834,8 +834,7 @@ cpp_quote_string (uchar *dest, const uch /* Convert a token sequence FIRST to FIRST+COUNT-1 to a single string token according to the rules of the ISO C #-operator. */ static const cpp_token * -stringify_arg (cpp_reader *pfile, const cpp_token **first, unsigned int count, - bool va_opt) +stringify_arg (cpp_reader *pfile, const cpp_token **first, unsigned int count) { unsigned char *dest; unsigned int i, escape_it, backslash_count = 0; @@ -852,24 +851,6 @@ stringify_arg (cpp_reader *pfile, const { const cpp_token *token = first[i]; - if (va_opt && (token->flags & PASTE_LEFT)) - { - location_t virt_loc = pfile->invocation_location; - const cpp_token *rhs; - do - { - if (i == count) - abort (); - rhs = first[++i]; - if (!paste_tokens (pfile, virt_loc, &token, rhs)) - { - --i; - break; - } - } - while (rhs->flags & PASTE_LEFT); - } - if (token->type == CPP_PADDING) { if (source == NULL @@ -1945,8 +1926,7 @@ replace_args (cpp_reader *pfile, cpp_has if (src->flags & STRINGIFY_ARG) { if (!arg->stringified) - arg->stringified = stringify_arg (pfile, arg->first, arg->count, - false); + arg->stringified = stringify_arg (pfile, arg->first, arg->count); } else if ((src->flags & PASTE_LEFT) || (src != macro->exp.tokens && (src[-1].flags & PASTE_LEFT))) @@ -2066,11 +2046,49 @@ replace_args (cpp_reader *pfile, cpp_has { unsigned int count = start ? paste_flag - start : tokens_buff_count (buff); - const cpp_token *t - = stringify_arg (pfile, - start ? start + 1 - : (const cpp_token **) (buff->base), - count, true); + const cpp_token **first + = start ? start + 1 + : (const cpp_token **) (buff->base); + unsigned int i, j; + + /* Paste any tokens that need to be pasted. */ + for (i = 0, j = 0; i < count; i++, j++) + { + const cpp_token *token = first[i]; + + if (token->flags & PASTE_LEFT) + { + location_t virt_loc = pfile->invocation_location; + const cpp_token *rhs; + unsigned short flags = token->flags; + do + { + if (i == count) + abort (); + rhs = first[++i]; + if (!paste_tokens (pfile, virt_loc, &token, rhs)) + { + --i; + break; + } + } + while (rhs->flags & PASTE_LEFT); + if ((flags & PREV_WHITE) + && (token->flags & PREV_WHITE) == 0) + const_cast(token)->flags + |= PREV_WHITE; + } + + first[j] = token; + } + if (j != i) + { + while (i-- != j) + tokens_buff_remove_last_token (buff); + count = j; + } + + const cpp_token *t = stringify_arg (pfile, first, count); while (count--) tokens_buff_remove_last_token (buff); if (src->flags & PASTE_LEFT) --- gcc/testsuite/c-c++-common/cpp/va-opt-8.c.jj 2021-11-25 18:03:19.243595714 +0100 +++ gcc/testsuite/c-c++-common/cpp/va-opt-8.c 2021-11-25 18:02:12.712553219 +0100 @@ -0,0 +1,18 @@ +/* PR preprocessor/103415 */ +/* { dg-do run } */ +/* { dg-options "-std=gnu99" { target c } } */ +/* { dg-options "-std=c++20" { target c++ } } */ + +#define n(x, ...) = #__VA_OPT__(x##3) +#define o(x, ...) #__VA_OPT__(x##__VA_ARGS__##9) +const char *c n(1 2, 4); +const char *d = o(5 6, 7 8); + +int +main () +{ + if (__builtin_strcmp (c, "1 23") + || __builtin_strcmp (d, "5 67 89")) + __builtin_abort (); + return 0; +}