From patchwork Wed Aug 24 11:59:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: herron.philip@googlemail.com X-Patchwork-Id: 56984 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 2EF8B3955C84 for ; Wed, 24 Aug 2022 12:02:51 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by sourceware.org (Postfix) with ESMTPS id 2C189385022C; Wed, 24 Aug 2022 12:00:46 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 2C189385022C Authentication-Results: sourceware.org; dmarc=pass (p=quarantine dis=none) header.from=googlemail.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=googlemail.com Received: by mail-wr1-x431.google.com with SMTP id n7so20484431wrv.4; Wed, 24 Aug 2022 05:00:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20210112; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc; bh=hus3mtKbo35A1J/fT6U8f2ROeBZZopTU7h4IRTZ33nI=; b=R3S3gvL2VzIxmwn+w0uuwS/WVxEcMiuotpY1OfcC77xWIoixXvBjOfudNDnrd27Cna 8RNvvG+tPVK8XRllZFiLMDWGNiS/g4B2VFB8Nt4qNARge6j8mHWn0PNTC7lqxXk51N4U hGuDS0jNNVIm6XR81td3O/WBU/Vxp/rFZD8ArHKfAWP7PQuN5LO40EoQWAr0h5oerB0m fjWvz299MIyhIhzlWsahG+TCkTcA+ccrc6xdjq/C4hyqzyDvaVyQMF76ztHuY2LyAN1d LeKQ1/Yx7BbyMr0jjwxzmMXWpgFn2CdJP9oN6uLk2lWPAyFmKaqWxmwMgN/SW2oYQTaN UKvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc; bh=hus3mtKbo35A1J/fT6U8f2ROeBZZopTU7h4IRTZ33nI=; b=2P7Rz6ZdflZvZx0EZPzx3Zw30c2QiYH7n2lxXdIiR+YJwpBFvg91Kh5TdjI0RVo8qE 1r0SQichXEY+YlLfh7XbLRcUutcreHr0PCrcns8T7YhOM9Qy6Z3H8H2P/HVsSSe2q2/D qsBpWkmiG1i0wEQh4tp0hbR/jSduAFFN7f1rN1/1Z6qt2BmqustclEBg7o4U6EL9sZ5Z ac1b8+0AfyI8WJwz3AzUPsDnQdRjeJYO1HWR0VJxolBmCdtl6/NXdxiaBlD5a+/dOCYK /obEQ1wRwd+18J8TfyvEAsMkRE5GXw1HVxgk1NGaDwyMEbc7Cj/Pmf4iXYDe8DmMI6OS mC0A== X-Gm-Message-State: ACgBeo0kwSRwqEv5qqRFU/33DdBYeu0BEY4fcGf/r2uFG+ruIDosWR6x xOpGCd1OVNFNHa8DAzoi21JrlcNlLSs= X-Google-Smtp-Source: AA6agR5YPwm5v4Mx8rrdSOu3jjWmhAhBGUEq9+YIcR+LhES6/KAPifX3SX7BLFG1dT/tvBqoUyTIqw== X-Received: by 2002:a5d:61c7:0:b0:225:307b:b557 with SMTP id q7-20020a5d61c7000000b00225307bb557mr14965248wrv.402.1661342443070; Wed, 24 Aug 2022 05:00:43 -0700 (PDT) Received: from localhost.localdomain ([86.14.124.218]) by smtp.gmail.com with ESMTPSA id cc19-20020a5d5c13000000b0022571d43d32sm1697676wrb.21.2022.08.24.05.00.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Aug 2022 05:00:42 -0700 (PDT) From: herron.philip@googlemail.com X-Google-Original-From: philip.herron@embecosm.com To: gcc-patches@gcc.gnu.org Subject: [PATCH Rust front-end v2 11/37] gccrs: Add expansion pass for the Rust front-end Date: Wed, 24 Aug 2022 12:59:30 +0100 Message-Id: <20220824115956.737931-12-philip.herron@embecosm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220824115956.737931-1-philip.herron@embecosm.com> References: <20220824115956.737931-1-philip.herron@embecosm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-11.4 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, GIT_PATCH_0, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: philip.herron@embecosm.com Cc: Arthur Cohen , The Other , gcc-rust@gcc.gnu.org, Philip Herron Errors-To: gcc-patches-bounces+patchwork=sourceware.org@gcc.gnu.org Sender: "Gcc-patches" From: Arthur Cohen Arthur TODO Co-authored-by: Philip Herron Co-authored-by: The Other --- gcc/rust/expand/rust-attribute-visitor.cc | 3445 ++++++++++++++++++ gcc/rust/expand/rust-attribute-visitor.h | 316 ++ gcc/rust/expand/rust-macro-builtins.cc | 484 +++ gcc/rust/expand/rust-macro-builtins.h | 107 + gcc/rust/expand/rust-macro-expand.cc | 1012 +++++ gcc/rust/expand/rust-macro-expand.h | 366 ++ gcc/rust/expand/rust-macro-invoc-lexer.cc | 29 + gcc/rust/expand/rust-macro-invoc-lexer.h | 64 + gcc/rust/expand/rust-macro-substitute-ctx.cc | 312 ++ gcc/rust/expand/rust-macro-substitute-ctx.h | 93 + 10 files changed, 6228 insertions(+) create mode 100644 gcc/rust/expand/rust-attribute-visitor.cc create mode 100644 gcc/rust/expand/rust-attribute-visitor.h create mode 100644 gcc/rust/expand/rust-macro-builtins.cc create mode 100644 gcc/rust/expand/rust-macro-builtins.h create mode 100644 gcc/rust/expand/rust-macro-expand.cc create mode 100644 gcc/rust/expand/rust-macro-expand.h create mode 100644 gcc/rust/expand/rust-macro-invoc-lexer.cc create mode 100644 gcc/rust/expand/rust-macro-invoc-lexer.h create mode 100644 gcc/rust/expand/rust-macro-substitute-ctx.cc create mode 100644 gcc/rust/expand/rust-macro-substitute-ctx.h diff --git a/gcc/rust/expand/rust-attribute-visitor.cc b/gcc/rust/expand/rust-attribute-visitor.cc new file mode 100644 index 00000000000..8016f9430eb --- /dev/null +++ b/gcc/rust/expand/rust-attribute-visitor.cc @@ -0,0 +1,3445 @@ +// Copyright (C) 2020-2022 Free Software Foundation, Inc. + +// This file is part of GCC. + +// GCC is free software; you can redistribute it and/or modify it under +// the terms of the GNU General Public License as published by the Free +// Software Foundation; either version 3, or (at your option) any later +// version. + +// GCC is distributed in the hope that it will be useful, but WITHOUT ANY +// WARRANTY; without even the implied warranty of MERCHANTABILITY or +// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License +// for more details. + +// You should have received a copy of the GNU General Public License +// along with GCC; see the file COPYING3. If not see +// . + +#include "rust-attribute-visitor.h" +#include "rust-session-manager.h" + +namespace Rust { + +// Visitor used to expand attributes. +void +AttrVisitor::expand_struct_fields (std::vector &fields) +{ + for (auto it = fields.begin (); it != fields.end ();) + { + auto &field = *it; + + auto &field_attrs = field.get_outer_attrs (); + expander.expand_cfg_attrs (field_attrs); + if (expander.fails_cfg_with_expand (field_attrs)) + { + it = fields.erase (it); + continue; + } + + expander.push_context (MacroExpander::ContextType::TYPE); + + // expand sub-types of type, but can't strip type itself + auto &type = field.get_field_type (); + type->accept_vis (*this); + + maybe_expand_type (type); + + if (type->is_marked_for_strip ()) + rust_error_at (type->get_locus (), + "cannot strip type in this position"); + + expander.pop_context (); + + // if nothing else happens, increment + ++it; + } +} + +void +AttrVisitor::expand_tuple_fields (std::vector &fields) +{ + for (auto it = fields.begin (); it != fields.end ();) + { + auto &field = *it; + + auto &field_attrs = field.get_outer_attrs (); + expander.expand_cfg_attrs (field_attrs); + if (expander.fails_cfg_with_expand (field_attrs)) + { + it = fields.erase (it); + continue; + } + + // expand sub-types of type, but can't strip type itself + auto &type = field.get_field_type (); + type->accept_vis (*this); + if (type->is_marked_for_strip ()) + rust_error_at (type->get_locus (), + "cannot strip type in this position"); + + // if nothing else happens, increment + ++it; + } +} + +void +AttrVisitor::expand_function_params (std::vector ¶ms) +{ + expander.push_context (MacroExpander::ContextType::TYPE); + + for (auto it = params.begin (); it != params.end ();) + { + auto ¶m = *it; + + auto ¶m_attrs = param.get_outer_attrs (); + expander.expand_cfg_attrs (param_attrs); + if (expander.fails_cfg_with_expand (param_attrs)) + { + it = params.erase (it); + continue; + } + + // TODO: should an unwanted strip lead to break out of loop? + auto &pattern = param.get_pattern (); + pattern->accept_vis (*this); + if (pattern->is_marked_for_strip ()) + rust_error_at (pattern->get_locus (), + "cannot strip pattern in this position"); + + auto &type = param.get_type (); + type->accept_vis (*this); + + maybe_expand_type (type); + + if (type->is_marked_for_strip ()) + rust_error_at (type->get_locus (), + "cannot strip type in this position"); + + // increment + ++it; + } + + expander.pop_context (); +} + +void +AttrVisitor::expand_generic_args (AST::GenericArgs &args) +{ + // lifetime args can't be expanded + // FIXME: Can we have macro invocations for lifetimes? + + expander.push_context (MacroExpander::ContextType::TYPE); + + // expand type args - strip sub-types only + for (auto &arg : args.get_generic_args ()) + { + switch (arg.get_kind ()) + { + case AST::GenericArg::Kind::Type: { + auto &type = arg.get_type (); + type->accept_vis (*this); + maybe_expand_type (type); + + if (type->is_marked_for_strip ()) + rust_error_at (type->get_locus (), + "cannot strip type in this position"); + break; + } + case AST::GenericArg::Kind::Const: { + auto &expr = arg.get_expression (); + expr->accept_vis (*this); + maybe_expand_expr (expr); + + if (expr->is_marked_for_strip ()) + rust_error_at (expr->get_locus (), + "cannot strip expression in this position"); + break; + } + default: + break; + // FIXME: Figure out what to do here if there is ambiguity. Since the + // resolver comes after the expansion, we need to figure out a way to + // strip ambiguous values here + // TODO: Arthur: Probably add a `mark_as_strip` method to `GenericArg` + // or something. This would clean up this whole thing + } + } + + expander.pop_context (); + + // FIXME: Can we have macro invocations in generic type bindings? + // expand binding args - strip sub-types only + for (auto &binding : args.get_binding_args ()) + { + auto &type = binding.get_type (); + type->accept_vis (*this); + + if (type->is_marked_for_strip ()) + rust_error_at (type->get_locus (), + "cannot strip type in this position"); + } +} + +void +AttrVisitor::expand_qualified_path_type (AST::QualifiedPathType &path_type) +{ + expander.push_context (MacroExpander::ContextType::TYPE); + + auto &type = path_type.get_type (); + type->accept_vis (*this); + + maybe_expand_type (type); + + expander.pop_context (); + + if (type->is_marked_for_strip ()) + rust_error_at (type->get_locus (), "cannot strip type in this position"); + + if (path_type.has_as_clause ()) + { + auto &type_path = path_type.get_as_type_path (); + visit (type_path); + if (type_path.is_marked_for_strip ()) + rust_error_at (type_path.get_locus (), + "cannot strip type path in this position"); + } +} + +void +AttrVisitor::AttrVisitor::expand_closure_params ( + std::vector ¶ms) +{ + for (auto it = params.begin (); it != params.end ();) + { + auto ¶m = *it; + + auto ¶m_attrs = param.get_outer_attrs (); + expander.expand_cfg_attrs (param_attrs); + if (expander.fails_cfg_with_expand (param_attrs)) + { + it = params.erase (it); + continue; + } + + auto &pattern = param.get_pattern (); + pattern->accept_vis (*this); + if (pattern->is_marked_for_strip ()) + rust_error_at (pattern->get_locus (), + "cannot strip pattern in this position"); + + if (param.has_type_given ()) + { + expander.push_context (MacroExpander::ContextType::TYPE); + auto &type = param.get_type (); + type->accept_vis (*this); + + maybe_expand_type (type); + + if (type->is_marked_for_strip ()) + rust_error_at (type->get_locus (), + "cannot strip type in this position"); + + expander.pop_context (); + } + + // increment if found nothing else so far + ++it; + } +} + +void +AttrVisitor::expand_self_param (AST::SelfParam &self_param) +{ + if (self_param.has_type ()) + { + expander.push_context (MacroExpander::ContextType::TYPE); + auto &type = self_param.get_type (); + type->accept_vis (*this); + + maybe_expand_type (type); + + if (type->is_marked_for_strip ()) + rust_error_at (type->get_locus (), + "cannot strip type in this position"); + + expander.pop_context (); + } + /* TODO: maybe check for invariants being violated - e.g. both type and + * lifetime? */ +} + +void +AttrVisitor::expand_where_clause (AST::WhereClause &where_clause) +{ + // items cannot be stripped conceptually, so just accept visitor + for (auto &item : where_clause.get_items ()) + item->accept_vis (*this); +} + +void +AttrVisitor::expand_trait_function_decl (AST::TraitFunctionDecl &decl) +{ + // just expand sub-stuff - can't actually strip generic params themselves + for (auto ¶m : decl.get_generic_params ()) + param->accept_vis (*this); + + /* strip function parameters if required - this is specifically + * allowed by spec */ + expand_function_params (decl.get_function_params ()); + + if (decl.has_return_type ()) + { + expander.push_context (MacroExpander::ContextType::TYPE); + + auto &return_type = decl.get_return_type (); + return_type->accept_vis (*this); + + maybe_expand_type (return_type); + + if (return_type->is_marked_for_strip ()) + rust_error_at (return_type->get_locus (), + "cannot strip type in this position"); + + expander.pop_context (); + } + + if (decl.has_where_clause ()) + expand_where_clause (decl.get_where_clause ()); +} + +void +AttrVisitor::expand_trait_method_decl (AST::TraitMethodDecl &decl) +{ + // just expand sub-stuff - can't actually strip generic params themselves + for (auto ¶m : decl.get_generic_params ()) + param->accept_vis (*this); + + /* assuming you can't strip self param - wouldn't be a method + * anymore. spec allows outer attrs on self param, but doesn't + * specify whether cfg is used. */ + expand_self_param (decl.get_self_param ()); + + /* strip function parameters if required - this is specifically + * allowed by spec */ + expand_function_params (decl.get_function_params ()); + + if (decl.has_return_type ()) + { + expander.push_context (MacroExpander::ContextType::TYPE); + + auto &return_type = decl.get_return_type (); + return_type->accept_vis (*this); + + maybe_expand_type (return_type); + + if (return_type->is_marked_for_strip ()) + rust_error_at (return_type->get_locus (), + "cannot strip type in this position"); + + expander.pop_context (); + } + + if (decl.has_where_clause ()) + expand_where_clause (decl.get_where_clause ()); +} + +void +AttrVisitor::visit (AST::Token &) +{ + // shouldn't require? +} +void +AttrVisitor::visit (AST::DelimTokenTree &) +{ + // shouldn't require? +} +void +AttrVisitor::visit (AST::AttrInputMetaItemContainer &) +{ + // shouldn't require? +} +void +AttrVisitor::visit (AST::IdentifierExpr &ident_expr) +{ + // strip test based on outer attrs + expander.expand_cfg_attrs (ident_expr.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (ident_expr.get_outer_attrs ())) + { + ident_expr.mark_for_strip (); + return; + } +} +void +AttrVisitor::visit (AST::Lifetime &) +{ + // shouldn't require? +} +void +AttrVisitor::visit (AST::LifetimeParam &) +{ + // supposedly does not require - cfg does nothing +} +void +AttrVisitor::visit (AST::ConstGenericParam &) +{ + // likewise +} + +void +AttrVisitor::visit (AST::MacroInvocation ¯o_invoc) +{ + // initial strip test based on outer attrs + expander.expand_cfg_attrs (macro_invoc.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (macro_invoc.get_outer_attrs ())) + { + macro_invoc.mark_for_strip (); + return; + } + + // can't strip simple path + + // I don't think any macro token trees can be stripped in any way + + // TODO: maybe have cfg! macro stripping behaviour here? + expander.expand_invoc (macro_invoc, macro_invoc.has_semicolon ()); +} + +void +AttrVisitor::visit (AST::PathInExpression &path) +{ + // initial strip test based on outer attrs + expander.expand_cfg_attrs (path.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (path.get_outer_attrs ())) + { + path.mark_for_strip (); + return; + } + + for (auto &segment : path.get_segments ()) + { + if (segment.has_generic_args ()) + expand_generic_args (segment.get_generic_args ()); + } +} +void +AttrVisitor::visit (AST::TypePathSegment &) +{ + // shouldn't require +} +void +AttrVisitor::visit (AST::TypePathSegmentGeneric &segment) +{ + // TODO: strip inside generic args + + if (!segment.has_generic_args ()) + return; + + expand_generic_args (segment.get_generic_args ()); +} +void +AttrVisitor::visit (AST::TypePathSegmentFunction &segment) +{ + auto &type_path_function = segment.get_type_path_function (); + + for (auto &type : type_path_function.get_params ()) + { + type->accept_vis (*this); + if (type->is_marked_for_strip ()) + rust_error_at (type->get_locus (), + "cannot strip type in this position"); + } + + if (type_path_function.has_return_type ()) + { + expander.push_context (MacroExpander::ContextType::TYPE); + + auto &return_type = type_path_function.get_return_type (); + return_type->accept_vis (*this); + + maybe_expand_type (return_type); + + if (return_type->is_marked_for_strip ()) + rust_error_at (return_type->get_locus (), + "cannot strip type in this position"); + + expander.pop_context (); + } +} +void +AttrVisitor::visit (AST::TypePath &path) +{ + // this shouldn't strip any segments, but can strip inside them + for (auto &segment : path.get_segments ()) + segment->accept_vis (*this); +} +void +AttrVisitor::visit (AST::QualifiedPathInExpression &path) +{ + // initial strip test based on outer attrs + expander.expand_cfg_attrs (path.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (path.get_outer_attrs ())) + { + path.mark_for_strip (); + return; + } + + expand_qualified_path_type (path.get_qualified_path_type ()); + + for (auto &segment : path.get_segments ()) + { + if (segment.has_generic_args ()) + expand_generic_args (segment.get_generic_args ()); + } +} +void +AttrVisitor::visit (AST::QualifiedPathInType &path) +{ + expand_qualified_path_type (path.get_qualified_path_type ()); + + // this shouldn't strip any segments, but can strip inside them + for (auto &segment : path.get_segments ()) + segment->accept_vis (*this); +} + +void +AttrVisitor::visit (AST::LiteralExpr &expr) +{ + // initial strip test based on outer attrs + expander.expand_cfg_attrs (expr.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ())) + { + expr.mark_for_strip (); + return; + } +} +void +AttrVisitor::visit (AST::AttrInputLiteral &) +{ + // shouldn't require? +} +void +AttrVisitor::visit (AST::MetaItemLitExpr &) +{ + // shouldn't require? +} +void +AttrVisitor::visit (AST::MetaItemPathLit &) +{ + // shouldn't require? +} +void +AttrVisitor::visit (AST::BorrowExpr &expr) +{ + // initial strip test based on outer attrs + expander.expand_cfg_attrs (expr.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ())) + { + expr.mark_for_strip (); + return; + } + + /* strip any internal sub-expressions - expression itself isn't + * allowed to have external attributes in this position so can't be + * stripped. */ + auto &borrowed_expr = expr.get_borrowed_expr (); + borrowed_expr->accept_vis (*this); + if (borrowed_expr->is_marked_for_strip ()) + rust_error_at (borrowed_expr->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); +} +void +AttrVisitor::visit (AST::DereferenceExpr &expr) +{ + // initial strip test based on outer attrs + expander.expand_cfg_attrs (expr.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ())) + { + expr.mark_for_strip (); + return; + } + + /* strip any internal sub-expressions - expression itself isn't + * allowed to have external attributes in this position so can't be + * stripped. */ + auto &dereferenced_expr = expr.get_dereferenced_expr (); + dereferenced_expr->accept_vis (*this); + if (dereferenced_expr->is_marked_for_strip ()) + rust_error_at (dereferenced_expr->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); +} +void +AttrVisitor::visit (AST::ErrorPropagationExpr &expr) +{ + // initial strip test based on outer attrs + expander.expand_cfg_attrs (expr.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ())) + { + expr.mark_for_strip (); + return; + } + + /* strip any internal sub-expressions - expression itself isn't + * allowed to have external attributes in this position so can't be + * stripped. */ + auto &propagating_expr = expr.get_propagating_expr (); + propagating_expr->accept_vis (*this); + if (propagating_expr->is_marked_for_strip ()) + rust_error_at (propagating_expr->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); +} +void +AttrVisitor::visit (AST::NegationExpr &expr) +{ + // initial strip test based on outer attrs + expander.expand_cfg_attrs (expr.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ())) + { + expr.mark_for_strip (); + return; + } + + /* strip any internal sub-expressions - expression itself isn't + * allowed to have external attributes in this position so can't be + * stripped. */ + auto &negated_expr = expr.get_negated_expr (); + negated_expr->accept_vis (*this); + if (negated_expr->is_marked_for_strip ()) + rust_error_at (negated_expr->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); +} +void +AttrVisitor::visit (AST::ArithmeticOrLogicalExpr &expr) +{ + /* outer attributes never allowed before these. while cannot strip + * two direct descendant expressions, can strip ones below that */ + + /* should have no possibility for outer attrs as would be parsed + * with outer expr */ + auto &l_expr = expr.get_left_expr (); + l_expr->accept_vis (*this); + maybe_expand_expr (l_expr); + + /* should syntactically not have outer attributes, though this may + * not have worked in practice */ + auto &r_expr = expr.get_right_expr (); + r_expr->accept_vis (*this); + maybe_expand_expr (r_expr); + + // ensure that they are not marked for strip + if (expr.get_left_expr ()->is_marked_for_strip ()) + rust_error_at (expr.get_left_expr ()->get_locus (), + "cannot strip expression in this position - outer " + "attributes are never allowed " + "before binary op exprs"); + if (expr.get_right_expr ()->is_marked_for_strip ()) + rust_error_at (expr.get_right_expr ()->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); +} +void +AttrVisitor::visit (AST::ComparisonExpr &expr) +{ + /* outer attributes never allowed before these. while cannot strip + * two direct descendant expressions, can strip ones below that */ + + /* should have no possibility for outer attrs as would be parsed + * with outer expr */ + auto &l_expr = expr.get_left_expr (); + l_expr->accept_vis (*this); + maybe_expand_expr (l_expr); + + /* should syntactically not have outer attributes, though this may + * not have worked in practice */ + auto &r_expr = expr.get_right_expr (); + r_expr->accept_vis (*this); + maybe_expand_expr (r_expr); + + // ensure that they are not marked for strip + if (expr.get_left_expr ()->is_marked_for_strip ()) + rust_error_at (expr.get_left_expr ()->get_locus (), + "cannot strip expression in this position - outer " + "attributes are never allowed " + "before binary op exprs"); + if (expr.get_right_expr ()->is_marked_for_strip ()) + rust_error_at (expr.get_right_expr ()->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); +} +void +AttrVisitor::visit (AST::LazyBooleanExpr &expr) +{ + /* outer attributes never allowed before these. while cannot strip + * two direct descendant expressions, can strip ones below that */ + + /* should have no possibility for outer attrs as would be parsed + * with outer expr */ + auto &l_expr = expr.get_left_expr (); + l_expr->accept_vis (*this); + maybe_expand_expr (l_expr); + + /* should syntactically not have outer attributes, though this may + * not have worked in practice */ + auto &r_expr = expr.get_right_expr (); + r_expr->accept_vis (*this); + maybe_expand_expr (r_expr); + + // ensure that they are not marked for strip + if (expr.get_left_expr ()->is_marked_for_strip ()) + rust_error_at (expr.get_left_expr ()->get_locus (), + "cannot strip expression in this position - outer " + "attributes are never allowed " + "before binary op exprs"); + if (expr.get_right_expr ()->is_marked_for_strip ()) + rust_error_at (expr.get_right_expr ()->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); +} +void +AttrVisitor::visit (AST::TypeCastExpr &expr) +{ + /* outer attributes never allowed before these. while cannot strip + * direct descendant expression, can strip ones below that */ + + auto &casted_expr = expr.get_casted_expr (); + /* should have no possibility for outer attrs as would be parsed + * with outer expr */ + casted_expr->accept_vis (*this); + + // ensure that they are not marked for strip + if (casted_expr->is_marked_for_strip ()) + rust_error_at (casted_expr->get_locus (), + "cannot strip expression in this position - outer " + "attributes are never allowed before cast exprs"); + + // TODO: strip sub-types of type + auto &type = expr.get_type_to_cast_to (); + type->accept_vis (*this); + if (type->is_marked_for_strip ()) + rust_error_at (type->get_locus (), "cannot strip type in this position"); +} +void +AttrVisitor::visit (AST::AssignmentExpr &expr) +{ + expander.expand_cfg_attrs (expr.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ())) + { + expr.mark_for_strip (); + return; + } + + /* should have no possibility for outer attrs as would be parsed + * with outer expr */ + auto &l_expr = expr.get_left_expr (); + l_expr->accept_vis (*this); + maybe_expand_expr (l_expr); + + /* should syntactically not have outer attributes, though this may + * not have worked in practice */ + auto &r_expr = expr.get_right_expr (); + r_expr->accept_vis (*this); + maybe_expand_expr (r_expr); + + // ensure that they are not marked for strip + if (expr.get_left_expr ()->is_marked_for_strip ()) + rust_error_at (expr.get_left_expr ()->get_locus (), + "cannot strip expression in this position - outer " + "attributes are never allowed " + "before binary op exprs"); + if (expr.get_right_expr ()->is_marked_for_strip ()) + rust_error_at (expr.get_right_expr ()->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); +} +void +AttrVisitor::visit (AST::CompoundAssignmentExpr &expr) +{ + /* outer attributes never allowed before these. while cannot strip + * two direct descendant expressions, can strip ones below that */ + + /* should have no possibility for outer attrs as would be parsed + * with outer expr */ + auto &l_expr = expr.get_left_expr (); + l_expr->accept_vis (*this); + maybe_expand_expr (l_expr); + + /* should syntactically not have outer attributes, though this may + * not have worked in practice */ + auto &r_expr = expr.get_right_expr (); + r_expr->accept_vis (*this); + maybe_expand_expr (r_expr); + + // ensure that they are not marked for strip + if (expr.get_left_expr ()->is_marked_for_strip ()) + rust_error_at (expr.get_left_expr ()->get_locus (), + "cannot strip expression in this position - outer " + "attributes are never allowed " + "before binary op exprs"); + if (expr.get_right_expr ()->is_marked_for_strip ()) + rust_error_at (expr.get_right_expr ()->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); +} +void +AttrVisitor::visit (AST::GroupedExpr &expr) +{ + // initial strip test based on outer attrs + expander.expand_cfg_attrs (expr.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ())) + { + expr.mark_for_strip (); + return; + } + + /* strip test based on inner attrs - spec says these are inner + * attributes, not outer attributes of inner expr */ + expander.expand_cfg_attrs (expr.get_inner_attrs ()); + if (expander.fails_cfg_with_expand (expr.get_inner_attrs ())) + { + expr.mark_for_strip (); + return; + } + + /* strip any internal sub-expressions - expression itself isn't + * allowed to have external attributes in this position so can't be + * stripped. */ + auto &inner_expr = expr.get_expr_in_parens (); + inner_expr->accept_vis (*this); + if (inner_expr->is_marked_for_strip ()) + rust_error_at (inner_expr->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); +} +void +AttrVisitor::visit (AST::ArrayElemsValues &elems) +{ + /* apparently outer attributes are allowed in "elements of array + * expressions" according to spec */ + expand_pointer_allow_strip (elems.get_values ()); +} +void +AttrVisitor::visit (AST::ArrayElemsCopied &elems) +{ + /* apparently outer attributes are allowed in "elements of array + * expressions" according to spec. on the other hand, it would not + * make conceptual sense to be able to remove either expression. As + * such, not implementing. TODO clear up the ambiguity here */ + + // only intend stripping for internal sub-expressions + auto &copied_expr = elems.get_elem_to_copy (); + copied_expr->accept_vis (*this); + if (copied_expr->is_marked_for_strip ()) + rust_error_at (copied_expr->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); + + auto ©_count = elems.get_num_copies (); + copy_count->accept_vis (*this); + if (copy_count->is_marked_for_strip ()) + rust_error_at (copy_count->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); +} +void +AttrVisitor::visit (AST::ArrayExpr &expr) +{ + // initial strip test based on outer attrs + expander.expand_cfg_attrs (expr.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ())) + { + expr.mark_for_strip (); + return; + } + + /* strip test based on inner attrs - spec says there are separate + * inner attributes, not just outer attributes of inner exprs */ + expander.expand_cfg_attrs (expr.get_inner_attrs ()); + if (expander.fails_cfg_with_expand (expr.get_inner_attrs ())) + { + expr.mark_for_strip (); + return; + } + + /* assuming you can't strip away the ArrayElems type, but can strip + * internal expressions and whatever */ + expr.get_array_elems ()->accept_vis (*this); +} +void +AttrVisitor::visit (AST::ArrayIndexExpr &expr) +{ + /* it is unclear whether outer attributes are supposed to be + * allowed, but conceptually it wouldn't make much sense, but + * having expansion code anyway. TODO */ + // initial strip test based on outer attrs + expander.expand_cfg_attrs (expr.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ())) + { + expr.mark_for_strip (); + return; + } + + /* strip any internal sub-expressions - expression itself isn't + * allowed to have external attributes in this position so can't be + * stripped. */ + auto &array_expr = expr.get_array_expr (); + array_expr->accept_vis (*this); + if (array_expr->is_marked_for_strip ()) + rust_error_at (array_expr->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); + + auto &index_expr = expr.get_index_expr (); + index_expr->accept_vis (*this); + if (index_expr->is_marked_for_strip ()) + rust_error_at (index_expr->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); +} +void +AttrVisitor::visit (AST::TupleExpr &expr) +{ + /* according to spec, outer attributes are allowed on "elements of + * tuple expressions" */ + + // initial strip test based on outer attrs + expander.expand_cfg_attrs (expr.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ())) + { + expr.mark_for_strip (); + return; + } + + /* strip test based on inner attrs - spec says these are inner + * attributes, not outer attributes of inner expr */ + expander.expand_cfg_attrs (expr.get_inner_attrs ()); + if (expander.fails_cfg_with_expand (expr.get_inner_attrs ())) + { + expr.mark_for_strip (); + return; + } + + /* apparently outer attributes are allowed in "elements of tuple + * expressions" according to spec */ + expand_pointer_allow_strip (expr.get_tuple_elems ()); +} +void +AttrVisitor::visit (AST::TupleIndexExpr &expr) +{ + // initial strip test based on outer attrs + expander.expand_cfg_attrs (expr.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ())) + { + expr.mark_for_strip (); + return; + } + + /* wouldn't strip this directly (as outer attrs should be + * associated with this level), but any sub-expressions would be + * stripped. Thus, no need to erase when strip check called. */ + auto &tuple_expr = expr.get_tuple_expr (); + tuple_expr->accept_vis (*this); + if (tuple_expr->is_marked_for_strip ()) + rust_error_at (tuple_expr->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); +} +void +AttrVisitor::visit (AST::StructExprStruct &expr) +{ + // initial strip test based on outer attrs + expander.expand_cfg_attrs (expr.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ())) + { + expr.mark_for_strip (); + return; + } + + /* strip test based on inner attrs - spec says these are inner + * attributes, not outer attributes of inner expr */ + expander.expand_cfg_attrs (expr.get_inner_attrs ()); + if (expander.fails_cfg_with_expand (expr.get_inner_attrs ())) + { + expr.mark_for_strip (); + return; + } + + // strip sub-exprs of path + auto &struct_name = expr.get_struct_name (); + visit (struct_name); + if (struct_name.is_marked_for_strip ()) + rust_error_at (struct_name.get_locus (), + "cannot strip path in this position"); +} +void +AttrVisitor::visit (AST::StructExprFieldIdentifier &) +{ + // as no attrs (at moment, at least), no stripping possible +} +void +AttrVisitor::visit (AST::StructExprFieldIdentifierValue &field) +{ + /* as no attrs possible (at moment, at least), only sub-expression + * stripping is possible */ + auto &value = field.get_value (); + value->accept_vis (*this); + if (value->is_marked_for_strip ()) + rust_error_at (value->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); +} +void +AttrVisitor::visit (AST::StructExprFieldIndexValue &field) +{ + /* as no attrs possible (at moment, at least), only sub-expression + * stripping is possible */ + auto &value = field.get_value (); + value->accept_vis (*this); + if (value->is_marked_for_strip ()) + rust_error_at (value->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); +} +void +AttrVisitor::visit (AST::StructExprStructFields &expr) +{ + // initial strip test based on outer attrs + expander.expand_cfg_attrs (expr.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ())) + { + expr.mark_for_strip (); + return; + } + + /* strip test based on inner attrs - spec says these are inner + * attributes, not outer attributes of inner expr */ + expander.expand_cfg_attrs (expr.get_inner_attrs ()); + if (expander.fails_cfg_with_expand (expr.get_inner_attrs ())) + { + expr.mark_for_strip (); + return; + } + + // strip sub-exprs of path + auto &struct_name = expr.get_struct_name (); + visit (struct_name); + if (struct_name.is_marked_for_strip ()) + rust_error_at (struct_name.get_locus (), + "cannot strip path in this position"); + + /* spec does not specify whether expressions are allowed to be + * stripped at top level of struct fields, but I wouldn't think + * that they would be, so operating under the assumption that only + * sub-expressions can be stripped. */ + for (auto &field : expr.get_fields ()) + { + field->accept_vis (*this); + // shouldn't strip in this + } + + /* struct base presumably can't be stripped, as the '..' is before + * the expression. as such, can only strip sub-expressions. */ + if (expr.has_struct_base ()) + { + auto &base_struct_expr = expr.get_struct_base ().get_base_struct (); + base_struct_expr->accept_vis (*this); + if (base_struct_expr->is_marked_for_strip ()) + rust_error_at (base_struct_expr->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); + } +} +void +AttrVisitor::visit (AST::StructExprStructBase &expr) +{ + // initial strip test based on outer attrs + expander.expand_cfg_attrs (expr.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ())) + { + expr.mark_for_strip (); + return; + } + + /* strip test based on inner attrs - spec says these are inner + * attributes, not outer attributes of inner expr */ + expander.expand_cfg_attrs (expr.get_inner_attrs ()); + if (expander.fails_cfg_with_expand (expr.get_inner_attrs ())) + { + expr.mark_for_strip (); + return; + } + + // strip sub-exprs of path + auto &struct_name = expr.get_struct_name (); + visit (struct_name); + if (struct_name.is_marked_for_strip ()) + rust_error_at (struct_name.get_locus (), + "cannot strip path in this position"); + + /* struct base presumably can't be stripped, as the '..' is before + * the expression. as such, can only strip sub-expressions. */ + rust_assert (!expr.get_struct_base ().is_invalid ()); + auto &base_struct_expr = expr.get_struct_base ().get_base_struct (); + base_struct_expr->accept_vis (*this); + if (base_struct_expr->is_marked_for_strip ()) + rust_error_at (base_struct_expr->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); +} +void +AttrVisitor::visit (AST::CallExpr &expr) +{ + // initial strip test based on outer attrs + expander.expand_cfg_attrs (expr.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ())) + { + expr.mark_for_strip (); + return; + } + + /* should not be outer attrs on "function" expression - outer attrs + * should be associated with call expr as a whole. only sub-expr + * expansion is possible. */ + auto &function = expr.get_function_expr (); + function->accept_vis (*this); + if (function->is_marked_for_strip ()) + rust_error_at (function->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); + + /* spec says outer attributes are specifically allowed for elements + * of call expressions, so full stripping possible */ + // FIXME: Arthur: Figure out how to refactor this - This is similar to + // expanding items in the crate or stmts in blocks + expand_pointer_allow_strip (expr.get_params ()); + auto ¶ms = expr.get_params (); + for (auto it = params.begin (); it != params.end ();) + { + auto &stmt = *it; + + stmt->accept_vis (*this); + + auto final_fragment = expand_macro_fragment_recursive (); + if (final_fragment.should_expand ()) + { + // Remove the current expanded invocation + it = params.erase (it); + for (auto &node : final_fragment.get_nodes ()) + { + it = params.insert (it, node.take_expr ()); + it++; + } + } + else if (stmt->is_marked_for_strip ()) + it = params.erase (it); + else + it++; + } +} +void +AttrVisitor::visit (AST::MethodCallExpr &expr) +{ + // initial strip test based on outer attrs + expander.expand_cfg_attrs (expr.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ())) + { + expr.mark_for_strip (); + return; + } + + /* should not be outer attrs on "receiver" expression - outer attrs + * should be associated with call expr as a whole. only sub-expr + * expansion is possible. */ + auto &receiver = expr.get_receiver_expr (); + receiver->accept_vis (*this); + if (receiver->is_marked_for_strip ()) + rust_error_at (receiver->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); + + auto &method_name = expr.get_method_name (); + if (method_name.has_generic_args ()) + expand_generic_args (method_name.get_generic_args ()); + + /* spec says outer attributes are specifically allowed for elements + * of method call expressions, so full stripping possible */ + expand_pointer_allow_strip (expr.get_params ()); +} +void +AttrVisitor::visit (AST::FieldAccessExpr &expr) +{ + // initial strip test based on outer attrs + expander.expand_cfg_attrs (expr.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ())) + { + expr.mark_for_strip (); + return; + } + + /* should not be outer attrs on "receiver" expression - outer attrs + * should be associated with field expr as a whole. only sub-expr + * expansion is possible. */ + auto &receiver = expr.get_receiver_expr (); + receiver->accept_vis (*this); + if (receiver->is_marked_for_strip ()) + rust_error_at (receiver->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); +} +void +AttrVisitor::visit (AST::ClosureExprInner &expr) +{ + // initial strip test based on outer attrs + expander.expand_cfg_attrs (expr.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ())) + { + expr.mark_for_strip (); + return; + } + + /* strip closure parameters if required - this is specifically + * allowed by spec */ + expand_closure_params (expr.get_params ()); + + // can't strip expression itself, but can strip sub-expressions + auto &definition_expr = expr.get_definition_expr (); + definition_expr->accept_vis (*this); + if (definition_expr->is_marked_for_strip ()) + rust_error_at (definition_expr->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); +} + +void +AttrVisitor::visit (AST::BlockExpr &expr) +{ + // initial strip test based on outer attrs + expander.expand_cfg_attrs (expr.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ())) + { + expr.mark_for_strip (); + return; + } + + /* strip test based on inner attrs - spec says there are inner + * attributes, not just outer attributes of inner stmts */ + expander.expand_cfg_attrs (expr.get_inner_attrs ()); + if (expander.fails_cfg_with_expand (expr.get_inner_attrs ())) + { + expr.mark_for_strip (); + return; + } + + std::function (AST::SingleASTNode)> extractor + = [] (AST::SingleASTNode node) { return node.take_stmt (); }; + + expand_macro_children (MacroExpander::BLOCK, expr.get_statements (), + extractor); + + expander.push_context (MacroExpander::BLOCK); + + // strip tail expression if exists - can actually fully remove it + if (expr.has_tail_expr ()) + { + auto &tail_expr = expr.get_tail_expr (); + + tail_expr->accept_vis (*this); + maybe_expand_expr (tail_expr); + + if (tail_expr->is_marked_for_strip ()) + expr.strip_tail_expr (); + } + expander.pop_context (); +} + +void +AttrVisitor::visit (AST::ClosureExprInnerTyped &expr) +{ + // initial strip test based on outer attrs + expander.expand_cfg_attrs (expr.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ())) + { + expr.mark_for_strip (); + return; + } + + /* strip closure parameters if required - this is specifically + * allowed by spec */ + expand_closure_params (expr.get_params ()); + + expander.push_context (MacroExpander::ContextType::TYPE); + + // can't strip return type, but can strip sub-types + auto &type = expr.get_return_type (); + type->accept_vis (*this); + + maybe_expand_type (type); + + if (type->is_marked_for_strip ()) + rust_error_at (type->get_locus (), "cannot strip type in this position"); + + expander.pop_context (); + + // can't strip expression itself, but can strip sub-expressions + auto &definition_block = expr.get_definition_block (); + definition_block->accept_vis (*this); + if (definition_block->is_marked_for_strip ()) + rust_error_at (definition_block->get_locus (), + "cannot strip block expression in this position - outer " + "attributes not allowed"); +} +void +AttrVisitor::visit (AST::ContinueExpr &expr) +{ + // initial strip test based on outer attrs + expander.expand_cfg_attrs (expr.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ())) + { + expr.mark_for_strip (); + return; + } +} +void +AttrVisitor::visit (AST::BreakExpr &expr) +{ + // initial strip test based on outer attrs + expander.expand_cfg_attrs (expr.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ())) + { + expr.mark_for_strip (); + return; + } + + /* spec does not say that you can have outer attributes on + * expression, so assuming you can't. stripping for sub-expressions + * is the only thing that can be done */ + if (expr.has_break_expr ()) + { + auto &break_expr = expr.get_break_expr (); + + break_expr->accept_vis (*this); + + if (break_expr->is_marked_for_strip ()) + rust_error_at (break_expr->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); + } +} +void +AttrVisitor::visit (AST::RangeFromToExpr &expr) +{ + /* outer attributes never allowed before these. while cannot strip + * two direct descendant expressions, can strip ones below that */ + + /* should have no possibility for outer attrs as would be parsed + * with outer expr */ + expr.get_from_expr ()->accept_vis (*this); + /* should syntactically not have outer attributes, though this may + * not have worked in practice */ + expr.get_to_expr ()->accept_vis (*this); + + // ensure that they are not marked for strip + if (expr.get_from_expr ()->is_marked_for_strip ()) + rust_error_at (expr.get_from_expr ()->get_locus (), + "cannot strip expression in this position - outer " + "attributes are never allowed " + "before range exprs"); + if (expr.get_to_expr ()->is_marked_for_strip ()) + rust_error_at (expr.get_to_expr ()->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); +} +void +AttrVisitor::visit (AST::RangeFromExpr &expr) +{ + /* outer attributes never allowed before these. while cannot strip + * direct descendant expression, can strip ones below that */ + + /* should have no possibility for outer attrs as would be parsed + * with outer expr */ + auto &from_expr = expr.get_from_expr (); + + from_expr->accept_vis (*this); + + if (from_expr->is_marked_for_strip ()) + rust_error_at (from_expr->get_locus (), + "cannot strip expression in this position - outer " + "attributes are never allowed before range exprs"); +} +void +AttrVisitor::visit (AST::RangeToExpr &expr) +{ + /* outer attributes never allowed before these. while cannot strip + * direct descendant expression, can strip ones below that */ + + /* should syntactically not have outer attributes, though this may + * not have worked in practice */ + auto &to_expr = expr.get_to_expr (); + + to_expr->accept_vis (*this); + + if (to_expr->is_marked_for_strip ()) + rust_error_at (to_expr->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); +} +void +AttrVisitor::visit (AST::RangeFullExpr &) +{ + // outer attributes never allowed before these, so no stripping +} +void +AttrVisitor::visit (AST::RangeFromToInclExpr &expr) +{ + /* outer attributes never allowed before these. while cannot strip + * two direct descendant expressions, can strip ones below that */ + + /* should have no possibility for outer attrs as would be parsed + * with outer expr */ + expr.get_from_expr ()->accept_vis (*this); + /* should syntactically not have outer attributes, though this may + * not have worked in practice */ + expr.get_to_expr ()->accept_vis (*this); + + // ensure that they are not marked for strip + if (expr.get_from_expr ()->is_marked_for_strip ()) + rust_error_at (expr.get_from_expr ()->get_locus (), + "cannot strip expression in this position - outer " + "attributes are never allowed " + "before range exprs"); + if (expr.get_to_expr ()->is_marked_for_strip ()) + rust_error_at (expr.get_to_expr ()->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); +} +void +AttrVisitor::visit (AST::RangeToInclExpr &expr) +{ + /* outer attributes never allowed before these. while cannot strip + * direct descendant expression, can strip ones below that */ + + /* should syntactically not have outer attributes, though this may + * not have worked in practice */ + auto &to_expr = expr.get_to_expr (); + + to_expr->accept_vis (*this); + + if (to_expr->is_marked_for_strip ()) + rust_error_at (to_expr->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); +} +void +AttrVisitor::visit (AST::ReturnExpr &expr) +{ + // initial strip test based on outer attrs + expander.expand_cfg_attrs (expr.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ())) + { + expr.mark_for_strip (); + return; + } + + /* spec does not say that you can have outer attributes on + * expression, so assuming you can't. stripping for sub-expressions + * is the only thing that can be done */ + if (expr.has_returned_expr ()) + { + auto &returned_expr = expr.get_returned_expr (); + + returned_expr->accept_vis (*this); + + if (returned_expr->is_marked_for_strip ()) + rust_error_at (returned_expr->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); + } + /* TODO: conceptually, you would maybe be able to remove a returned + * expr - e.g. if you had conditional compilation returning void or + * returning a type. On the other hand, I think that function + * return type cannot be conditionally compiled, so I assumed you + * can't do this either. */ +} +void +AttrVisitor::visit (AST::UnsafeBlockExpr &expr) +{ + // initial strip test based on outer attrs + expander.expand_cfg_attrs (expr.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ())) + { + expr.mark_for_strip (); + return; + } + + // can't strip block itself, but can strip sub-expressions + auto &block_expr = expr.get_block_expr (); + block_expr->accept_vis (*this); + if (block_expr->is_marked_for_strip ()) + rust_error_at (block_expr->get_locus (), + "cannot strip block expression in this position - outer " + "attributes not allowed"); +} +void +AttrVisitor::visit (AST::LoopExpr &expr) +{ + // initial strip test based on outer attrs + expander.expand_cfg_attrs (expr.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ())) + { + expr.mark_for_strip (); + return; + } + + // can't strip block itself, but can strip sub-expressions + auto &loop_block = expr.get_loop_block (); + loop_block->accept_vis (*this); + if (loop_block->is_marked_for_strip ()) + rust_error_at (loop_block->get_locus (), + "cannot strip block expression in this position - outer " + "attributes not allowed"); +} +void +AttrVisitor::visit (AST::WhileLoopExpr &expr) +{ + // initial strip test based on outer attrs + expander.expand_cfg_attrs (expr.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ())) + { + expr.mark_for_strip (); + return; + } + + // can't strip predicate expr itself, but can strip sub-expressions + auto &predicate_expr = expr.get_predicate_expr (); + predicate_expr->accept_vis (*this); + if (predicate_expr->is_marked_for_strip ()) + rust_error_at (predicate_expr->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); + + // can't strip block itself, but can strip sub-expressions + auto &loop_block = expr.get_loop_block (); + loop_block->accept_vis (*this); + if (loop_block->is_marked_for_strip ()) + rust_error_at (loop_block->get_locus (), + "cannot strip block expression in this position - outer " + "attributes not allowed"); +} +void +AttrVisitor::visit (AST::WhileLetLoopExpr &expr) +{ + // initial strip test based on outer attrs + expander.expand_cfg_attrs (expr.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ())) + { + expr.mark_for_strip (); + return; + } + + for (auto &pattern : expr.get_patterns ()) + { + pattern->accept_vis (*this); + if (pattern->is_marked_for_strip ()) + rust_error_at (pattern->get_locus (), + "cannot strip pattern in this position"); + } + + // can't strip scrutinee expr itself, but can strip sub-expressions + auto &scrutinee_expr = expr.get_scrutinee_expr (); + scrutinee_expr->accept_vis (*this); + if (scrutinee_expr->is_marked_for_strip ()) + rust_error_at (scrutinee_expr->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); + + // can't strip block itself, but can strip sub-expressions + auto &loop_block = expr.get_loop_block (); + loop_block->accept_vis (*this); + if (loop_block->is_marked_for_strip ()) + rust_error_at (loop_block->get_locus (), + "cannot strip block expression in this position - outer " + "attributes not allowed"); +} +void +AttrVisitor::visit (AST::ForLoopExpr &expr) +{ + // initial strip test based on outer attrs + expander.expand_cfg_attrs (expr.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ())) + { + expr.mark_for_strip (); + return; + } + + // strip sub-patterns of pattern + auto &pattern = expr.get_pattern (); + pattern->accept_vis (*this); + if (pattern->is_marked_for_strip ()) + rust_error_at (pattern->get_locus (), + "cannot strip pattern in this position"); + + // can't strip scrutinee expr itself, but can strip sub-expressions + auto &iterator_expr = expr.get_iterator_expr (); + iterator_expr->accept_vis (*this); + if (iterator_expr->is_marked_for_strip ()) + rust_error_at (iterator_expr->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); + + // can't strip block itself, but can strip sub-expressions + auto &loop_block = expr.get_loop_block (); + loop_block->accept_vis (*this); + if (loop_block->is_marked_for_strip ()) + rust_error_at (loop_block->get_locus (), + "cannot strip block expression in this position - outer " + "attributes not allowed"); +} +void +AttrVisitor::visit (AST::IfExpr &expr) +{ + // rust playground test shows that IfExpr does support outer attrs, at least + // when used as statement + + // initial strip test based on outer attrs + expander.expand_cfg_attrs (expr.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ())) + { + expr.mark_for_strip (); + return; + } + + // can't strip condition expr itself, but can strip sub-expressions + auto &condition_expr = expr.get_condition_expr (); + condition_expr->accept_vis (*this); + maybe_expand_expr (condition_expr); + if (condition_expr->is_marked_for_strip ()) + rust_error_at (condition_expr->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); + + // can't strip if block itself, but can strip sub-expressions + auto &if_block = expr.get_if_block (); + if_block->accept_vis (*this); + if (if_block->is_marked_for_strip ()) + rust_error_at (if_block->get_locus (), + "cannot strip block expression in this position - outer " + "attributes not allowed"); +} +void +AttrVisitor::visit (AST::IfExprConseqElse &expr) +{ + // initial strip test based on outer attrs + expander.expand_cfg_attrs (expr.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ())) + { + expr.mark_for_strip (); + return; + } + + // can't strip condition expr itself, but can strip sub-expressions + auto &condition_expr = expr.get_condition_expr (); + condition_expr->accept_vis (*this); + maybe_expand_expr (condition_expr); + if (condition_expr->is_marked_for_strip ()) + rust_error_at (condition_expr->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); + + // can't strip if block itself, but can strip sub-expressions + auto &if_block = expr.get_if_block (); + if_block->accept_vis (*this); + if (if_block->is_marked_for_strip ()) + rust_error_at (if_block->get_locus (), + "cannot strip block expression in this position - outer " + "attributes not allowed"); + + // can't strip else block itself, but can strip sub-expressions + auto &else_block = expr.get_else_block (); + else_block->accept_vis (*this); + if (else_block->is_marked_for_strip ()) + rust_error_at (else_block->get_locus (), + "cannot strip block expression in this position - outer " + "attributes not allowed"); +} +void +AttrVisitor::visit (AST::IfExprConseqIf &expr) +{ + // initial strip test based on outer attrs + expander.expand_cfg_attrs (expr.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ())) + { + expr.mark_for_strip (); + return; + } + + // can't strip condition expr itself, but can strip sub-expressions + auto &condition_expr = expr.get_condition_expr (); + condition_expr->accept_vis (*this); + maybe_expand_expr (condition_expr); + if (condition_expr->is_marked_for_strip ()) + rust_error_at (condition_expr->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); + + // can't strip if block itself, but can strip sub-expressions + auto &if_block = expr.get_if_block (); + if_block->accept_vis (*this); + if (if_block->is_marked_for_strip ()) + rust_error_at (if_block->get_locus (), + "cannot strip block expression in this position - outer " + "attributes not allowed"); + + // can't strip if expr itself, but can strip sub-expressions + auto &conseq_if_expr = expr.get_conseq_if_expr (); + conseq_if_expr->accept_vis (*this); + if (conseq_if_expr->is_marked_for_strip ()) + rust_error_at (conseq_if_expr->get_locus (), + "cannot strip consequent if expression in this " + "position - outer attributes not allowed"); +} +void +AttrVisitor::visit (AST::IfExprConseqIfLet &expr) +{ + // initial strip test based on outer attrs + expander.expand_cfg_attrs (expr.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ())) + { + expr.mark_for_strip (); + return; + } + + // can't strip condition expr itself, but can strip sub-expressions + auto &condition_expr = expr.get_condition_expr (); + condition_expr->accept_vis (*this); + maybe_expand_expr (condition_expr); + if (condition_expr->is_marked_for_strip ()) + rust_error_at (condition_expr->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); + + // can't strip if block itself, but can strip sub-expressions + auto &if_block = expr.get_if_block (); + if_block->accept_vis (*this); + if (if_block->is_marked_for_strip ()) + rust_error_at (if_block->get_locus (), + "cannot strip block expression in this position - outer " + "attributes not allowed"); + + // can't strip if let expr itself, but can strip sub-expressions + auto &conseq_if_let_expr = expr.get_conseq_if_let_expr (); + conseq_if_let_expr->accept_vis (*this); + if (conseq_if_let_expr->is_marked_for_strip ()) + rust_error_at (conseq_if_let_expr->get_locus (), + "cannot strip consequent if let expression in this " + "position - outer attributes not " + "allowed"); +} +void +AttrVisitor::visit (AST::IfLetExpr &expr) +{ + // initial strip test based on outer attrs + expander.expand_cfg_attrs (expr.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ())) + { + expr.mark_for_strip (); + return; + } + + for (auto &pattern : expr.get_patterns ()) + { + pattern->accept_vis (*this); + if (pattern->is_marked_for_strip ()) + rust_error_at (pattern->get_locus (), + "cannot strip pattern in this position"); + } + + // can't strip value expr itself, but can strip sub-expressions + auto &value_expr = expr.get_value_expr (); + value_expr->accept_vis (*this); + if (value_expr->is_marked_for_strip ()) + rust_error_at (value_expr->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); + + // can't strip if block itself, but can strip sub-expressions + auto &if_block = expr.get_if_block (); + if_block->accept_vis (*this); + if (if_block->is_marked_for_strip ()) + rust_error_at (if_block->get_locus (), + "cannot strip block expression in this position - outer " + "attributes not allowed"); +} +void +AttrVisitor::visit (AST::IfLetExprConseqElse &expr) +{ + // initial strip test based on outer attrs + expander.expand_cfg_attrs (expr.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ())) + { + expr.mark_for_strip (); + return; + } + + for (auto &pattern : expr.get_patterns ()) + { + pattern->accept_vis (*this); + if (pattern->is_marked_for_strip ()) + rust_error_at (pattern->get_locus (), + "cannot strip pattern in this position"); + } + + // can't strip value expr itself, but can strip sub-expressions + auto &value_expr = expr.get_value_expr (); + value_expr->accept_vis (*this); + if (value_expr->is_marked_for_strip ()) + rust_error_at (value_expr->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); + + // can't strip if block itself, but can strip sub-expressions + auto &if_block = expr.get_if_block (); + if_block->accept_vis (*this); + if (if_block->is_marked_for_strip ()) + rust_error_at (if_block->get_locus (), + "cannot strip block expression in this position - outer " + "attributes not allowed"); + + // can't strip else block itself, but can strip sub-expressions + auto &else_block = expr.get_else_block (); + else_block->accept_vis (*this); + if (else_block->is_marked_for_strip ()) + rust_error_at (else_block->get_locus (), + "cannot strip block expression in this position - outer " + "attributes not allowed"); +} +void +AttrVisitor::visit (AST::IfLetExprConseqIf &expr) +{ + // initial strip test based on outer attrs + expander.expand_cfg_attrs (expr.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ())) + { + expr.mark_for_strip (); + return; + } + + for (auto &pattern : expr.get_patterns ()) + { + pattern->accept_vis (*this); + if (pattern->is_marked_for_strip ()) + rust_error_at (pattern->get_locus (), + "cannot strip pattern in this position"); + } + + // can't strip value expr itself, but can strip sub-expressions + auto &value_expr = expr.get_value_expr (); + value_expr->accept_vis (*this); + if (value_expr->is_marked_for_strip ()) + rust_error_at (value_expr->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); + + // can't strip if block itself, but can strip sub-expressions + auto &if_block = expr.get_if_block (); + if_block->accept_vis (*this); + if (if_block->is_marked_for_strip ()) + rust_error_at (if_block->get_locus (), + "cannot strip block expression in this position - outer " + "attributes not allowed"); + + // can't strip if expr itself, but can strip sub-expressions + auto &conseq_if_expr = expr.get_conseq_if_expr (); + conseq_if_expr->accept_vis (*this); + if (conseq_if_expr->is_marked_for_strip ()) + rust_error_at (conseq_if_expr->get_locus (), + "cannot strip consequent if expression in this " + "position - outer attributes not allowed"); +} +void +AttrVisitor::visit (AST::IfLetExprConseqIfLet &expr) +{ + // initial strip test based on outer attrs + expander.expand_cfg_attrs (expr.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ())) + { + expr.mark_for_strip (); + return; + } + + for (auto &pattern : expr.get_patterns ()) + { + pattern->accept_vis (*this); + if (pattern->is_marked_for_strip ()) + rust_error_at (pattern->get_locus (), + "cannot strip pattern in this position"); + } + + // can't strip value expr itself, but can strip sub-expressions + auto &value_expr = expr.get_value_expr (); + value_expr->accept_vis (*this); + if (value_expr->is_marked_for_strip ()) + rust_error_at (value_expr->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); + + // can't strip if block itself, but can strip sub-expressions + auto &if_block = expr.get_if_block (); + if_block->accept_vis (*this); + if (if_block->is_marked_for_strip ()) + rust_error_at (if_block->get_locus (), + "cannot strip block expression in this position - outer " + "attributes not allowed"); + + // can't strip if let expr itself, but can strip sub-expressions + auto &conseq_if_let_expr = expr.get_conseq_if_let_expr (); + conseq_if_let_expr->accept_vis (*this); + if (conseq_if_let_expr->is_marked_for_strip ()) + rust_error_at (conseq_if_let_expr->get_locus (), + "cannot strip consequent if let expression in this " + "position - outer attributes not " + "allowed"); +} +void +AttrVisitor::visit (AST::MatchExpr &expr) +{ + // initial strip test based on outer attrs + expander.expand_cfg_attrs (expr.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ())) + { + expr.mark_for_strip (); + return; + } + + // inner attr strip test + expander.expand_cfg_attrs (expr.get_inner_attrs ()); + if (expander.fails_cfg_with_expand (expr.get_inner_attrs ())) + { + expr.mark_for_strip (); + return; + } + + // can't strip scrutinee expr itself, but can strip sub-expressions + auto &scrutinee_expr = expr.get_scrutinee_expr (); + scrutinee_expr->accept_vis (*this); + if (scrutinee_expr->is_marked_for_strip ()) + rust_error_at (scrutinee_expr->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); + + // strip match cases + auto &match_cases = expr.get_match_cases (); + for (auto it = match_cases.begin (); it != match_cases.end ();) + { + auto &match_case = *it; + + // strip match case based on outer attributes in match arm + auto &match_arm = match_case.get_arm (); + expander.expand_cfg_attrs (match_arm.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (match_arm.get_outer_attrs ())) + { + // strip match case + it = match_cases.erase (it); + continue; + } + + for (auto &pattern : match_arm.get_patterns ()) + { + pattern->accept_vis (*this); + if (pattern->is_marked_for_strip ()) + rust_error_at (pattern->get_locus (), + "cannot strip pattern in this position"); + } + + /* assuming that guard expression cannot be stripped as + * strictly speaking you would have to strip the whole guard to + * make syntactical sense, which you can't do. as such, only + * strip sub-expressions */ + if (match_arm.has_match_arm_guard ()) + { + auto &guard_expr = match_arm.get_guard_expr (); + guard_expr->accept_vis (*this); + if (guard_expr->is_marked_for_strip ()) + rust_error_at (guard_expr->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); + } + + // strip sub-expressions from match cases + auto &case_expr = match_case.get_expr (); + case_expr->accept_vis (*this); + if (case_expr->is_marked_for_strip ()) + rust_error_at (case_expr->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); + + // increment to next case if haven't continued + ++it; + } +} +void +AttrVisitor::visit (AST::AwaitExpr &expr) +{ + // initial strip test based on outer attrs + expander.expand_cfg_attrs (expr.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ())) + { + expr.mark_for_strip (); + return; + } + + /* can't strip awaited expr itself, but can strip sub-expressions + * - this is because you can't have no expr to await */ + auto &awaited_expr = expr.get_awaited_expr (); + awaited_expr->accept_vis (*this); + if (awaited_expr->is_marked_for_strip ()) + rust_error_at (awaited_expr->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); +} +void +AttrVisitor::visit (AST::AsyncBlockExpr &expr) +{ + // initial strip test based on outer attrs + expander.expand_cfg_attrs (expr.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ())) + { + expr.mark_for_strip (); + return; + } + + // can't strip block itself, but can strip sub-expressions + auto &block_expr = expr.get_block_expr (); + block_expr->accept_vis (*this); + if (block_expr->is_marked_for_strip ()) + rust_error_at (block_expr->get_locus (), + "cannot strip block expression in this position - outer " + "attributes not allowed"); +} + +void +AttrVisitor::visit (AST::TypeParam ¶m) +{ + // outer attributes don't actually do anything, so ignore them + + if (param.has_type_param_bounds ()) + { + // don't strip directly, only components of bounds + for (auto &bound : param.get_type_param_bounds ()) + bound->accept_vis (*this); + } + + if (param.has_type ()) + { + expander.push_context (MacroExpander::ContextType::TYPE); + auto &type = param.get_type (); + type->accept_vis (*this); + + maybe_expand_type (type); + + if (type->is_marked_for_strip ()) + rust_error_at (type->get_locus (), + "cannot strip type in this position"); + + expander.pop_context (); + } +} +void +AttrVisitor::visit (AST::LifetimeWhereClauseItem &) +{ + // shouldn't require +} +void +AttrVisitor::visit (AST::TypeBoundWhereClauseItem &item) +{ + // for lifetimes shouldn't require + + expander.push_context (MacroExpander::ContextType::TYPE); + + auto &type = item.get_type (); + type->accept_vis (*this); + + maybe_expand_type (type); + + if (type->is_marked_for_strip ()) + rust_error_at (type->get_locus (), "cannot strip type in this position"); + + expander.pop_context (); + + // don't strip directly, only components of bounds + for (auto &bound : item.get_type_param_bounds ()) + bound->accept_vis (*this); +} +void +AttrVisitor::visit (AST::Method &method) +{ + // initial test based on outer attrs + expander.expand_cfg_attrs (method.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (method.get_outer_attrs ())) + { + method.mark_for_strip (); + return; + } + + // just expand sub-stuff - can't actually strip generic params themselves + for (auto ¶m : method.get_generic_params ()) + param->accept_vis (*this); + + /* assuming you can't strip self param - wouldn't be a method + * anymore. spec allows outer attrs on self param, but doesn't + * specify whether cfg is used. */ + expand_self_param (method.get_self_param ()); + + /* strip method parameters if required - this is specifically + * allowed by spec */ + expand_function_params (method.get_function_params ()); + + if (method.has_return_type ()) + { + expander.push_context (MacroExpander::ContextType::TYPE); + + auto &return_type = method.get_return_type (); + return_type->accept_vis (*this); + + maybe_expand_type (return_type); + + if (return_type->is_marked_for_strip ()) + rust_error_at (return_type->get_locus (), + "cannot strip type in this position"); + + expander.pop_context (); + } + + if (method.has_where_clause ()) + expand_where_clause (method.get_where_clause ()); + + /* body should always exist - if error state, should have returned + * before now */ + // can't strip block itself, but can strip sub-expressions + auto &block_expr = method.get_definition (); + block_expr->accept_vis (*this); + if (block_expr->is_marked_for_strip ()) + rust_error_at (block_expr->get_locus (), + "cannot strip block expression in this position - outer " + "attributes not allowed"); +} +void +AttrVisitor::visit (AST::Module &module) +{ + // strip test based on outer attrs + expander.expand_cfg_attrs (module.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (module.get_outer_attrs ())) + { + module.mark_for_strip (); + return; + } + + // A loaded module might have inner attributes + if (module.get_kind () == AST::Module::ModuleKind::LOADED) + { + // strip test based on inner attrs + expander.expand_cfg_attrs (module.get_inner_attrs ()); + if (expander.fails_cfg_with_expand (module.get_inner_attrs ())) + { + module.mark_for_strip (); + return; + } + } + + // Parse the module's items if they haven't been expanded and the file + // should be parsed (i.e isn't hidden behind an untrue or impossible cfg + // directive) + if (!module.is_marked_for_strip () + && module.get_kind () == AST::Module::ModuleKind::UNLOADED) + { + module.load_items (); + } + + // strip items if required + expand_pointer_allow_strip (module.get_items ()); +} +void +AttrVisitor::visit (AST::ExternCrate &extern_crate) +{ + // strip test based on outer attrs + expander.expand_cfg_attrs (extern_crate.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (extern_crate.get_outer_attrs ())) + { + extern_crate.mark_for_strip (); + return; + } + + if (!extern_crate.references_self ()) + { + Session &session = Session::get_instance (); + session.load_extern_crate (extern_crate.get_referenced_crate (), + extern_crate.get_locus ()); + } +} +void +AttrVisitor::visit (AST::UseTreeGlob &) +{ + // shouldn't require? +} +void +AttrVisitor::visit (AST::UseTreeList &) +{ + // shouldn't require? +} +void +AttrVisitor::visit (AST::UseTreeRebind &) +{ + // shouldn't require? +} +void +AttrVisitor::visit (AST::UseDeclaration &use_decl) +{ + // strip test based on outer attrs + expander.expand_cfg_attrs (use_decl.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (use_decl.get_outer_attrs ())) + { + use_decl.mark_for_strip (); + return; + } +} +void +AttrVisitor::visit (AST::Function &function) +{ + // initial test based on outer attrs + expander.expand_cfg_attrs (function.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (function.get_outer_attrs ())) + { + function.mark_for_strip (); + return; + } + + // just expand sub-stuff - can't actually strip generic params themselves + for (auto ¶m : function.get_generic_params ()) + param->accept_vis (*this); + + /* strip function parameters if required - this is specifically + * allowed by spec */ + expand_function_params (function.get_function_params ()); + + if (function.has_return_type ()) + { + expander.push_context (MacroExpander::ContextType::TYPE); + + auto &return_type = function.get_return_type (); + return_type->accept_vis (*this); + + maybe_expand_type (return_type); + + if (return_type->is_marked_for_strip ()) + rust_error_at (return_type->get_locus (), + "cannot strip type in this position"); + + expander.pop_context (); + } + + if (function.has_where_clause ()) + expand_where_clause (function.get_where_clause ()); + + /* body should always exist - if error state, should have returned + * before now */ + // can't strip block itself, but can strip sub-expressions + auto &block_expr = function.get_definition (); + block_expr->accept_vis (*this); + if (block_expr->is_marked_for_strip ()) + rust_error_at (block_expr->get_locus (), + "cannot strip block expression in this position - outer " + "attributes not allowed"); +} +void +AttrVisitor::visit (AST::TypeAlias &type_alias) +{ + // initial test based on outer attrs + expander.expand_cfg_attrs (type_alias.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (type_alias.get_outer_attrs ())) + { + type_alias.mark_for_strip (); + return; + } + + // just expand sub-stuff - can't actually strip generic params themselves + for (auto ¶m : type_alias.get_generic_params ()) + param->accept_vis (*this); + + if (type_alias.has_where_clause ()) + expand_where_clause (type_alias.get_where_clause ()); + + auto &type = type_alias.get_type_aliased (); + type->accept_vis (*this); + if (type->is_marked_for_strip ()) + rust_error_at (type->get_locus (), "cannot strip type in this position"); +} +void +AttrVisitor::visit (AST::StructStruct &struct_item) +{ + // initial test based on outer attrs + expander.expand_cfg_attrs (struct_item.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (struct_item.get_outer_attrs ())) + { + struct_item.mark_for_strip (); + return; + } + + // just expand sub-stuff - can't actually strip generic params themselves + for (auto ¶m : struct_item.get_generic_params ()) + param->accept_vis (*this); + + if (struct_item.has_where_clause ()) + expand_where_clause (struct_item.get_where_clause ()); + + /* strip struct fields if required - this is presumably + * allowed by spec */ + expand_struct_fields (struct_item.get_fields ()); +} +void +AttrVisitor::visit (AST::TupleStruct &tuple_struct) +{ + // initial test based on outer attrs + expander.expand_cfg_attrs (tuple_struct.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (tuple_struct.get_outer_attrs ())) + { + tuple_struct.mark_for_strip (); + return; + } + + // just expand sub-stuff - can't actually strip generic params themselves + for (auto ¶m : tuple_struct.get_generic_params ()) + param->accept_vis (*this); + + /* strip struct fields if required - this is presumably + * allowed by spec */ + expand_tuple_fields (tuple_struct.get_fields ()); + + if (tuple_struct.has_where_clause ()) + expand_where_clause (tuple_struct.get_where_clause ()); +} +void +AttrVisitor::visit (AST::EnumItem &item) +{ + // initial test based on outer attrs + expander.expand_cfg_attrs (item.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (item.get_outer_attrs ())) + { + item.mark_for_strip (); + return; + } +} +void +AttrVisitor::visit (AST::EnumItemTuple &item) +{ + // initial test based on outer attrs + expander.expand_cfg_attrs (item.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (item.get_outer_attrs ())) + { + item.mark_for_strip (); + return; + } + + /* strip item fields if required - this is presumably + * allowed by spec */ + expand_tuple_fields (item.get_tuple_fields ()); +} +void +AttrVisitor::visit (AST::EnumItemStruct &item) +{ + // initial test based on outer attrs + expander.expand_cfg_attrs (item.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (item.get_outer_attrs ())) + { + item.mark_for_strip (); + return; + } + + /* strip item fields if required - this is presumably + * allowed by spec */ + expand_struct_fields (item.get_struct_fields ()); +} +void +AttrVisitor::visit (AST::EnumItemDiscriminant &item) +{ + // initial test based on outer attrs + expander.expand_cfg_attrs (item.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (item.get_outer_attrs ())) + { + item.mark_for_strip (); + return; + } + + /* strip any internal sub-expressions - expression itself isn't + * allowed to have external attributes in this position so can't be + * stripped. */ + auto &expr = item.get_expr (); + expr->accept_vis (*this); + if (expr->is_marked_for_strip ()) + rust_error_at (expr->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); +} +void +AttrVisitor::visit (AST::Enum &enum_item) +{ + // initial test based on outer attrs + expander.expand_cfg_attrs (enum_item.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (enum_item.get_outer_attrs ())) + { + enum_item.mark_for_strip (); + return; + } + + // just expand sub-stuff - can't actually strip generic params themselves + for (auto ¶m : enum_item.get_generic_params ()) + param->accept_vis (*this); + + if (enum_item.has_where_clause ()) + expand_where_clause (enum_item.get_where_clause ()); + + /* strip enum fields if required - this is presumably + * allowed by spec */ + expand_pointer_allow_strip (enum_item.get_variants ()); +} +void +AttrVisitor::visit (AST::Union &union_item) +{ + // initial test based on outer attrs + expander.expand_cfg_attrs (union_item.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (union_item.get_outer_attrs ())) + { + union_item.mark_for_strip (); + return; + } + + // just expand sub-stuff - can't actually strip generic params themselves + for (auto ¶m : union_item.get_generic_params ()) + param->accept_vis (*this); + + if (union_item.has_where_clause ()) + expand_where_clause (union_item.get_where_clause ()); + + /* strip union fields if required - this is presumably + * allowed by spec */ + expand_struct_fields (union_item.get_variants ()); +} +void +AttrVisitor::visit (AST::ConstantItem &const_item) +{ + // initial test based on outer attrs + expander.expand_cfg_attrs (const_item.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (const_item.get_outer_attrs ())) + { + const_item.mark_for_strip (); + return; + } + + expander.push_context (MacroExpander::ContextType::TYPE); + + // strip any sub-types + auto &type = const_item.get_type (); + type->accept_vis (*this); + + maybe_expand_type (type); + + if (type->is_marked_for_strip ()) + rust_error_at (type->get_locus (), "cannot strip type in this position"); + + expander.pop_context (); + + /* strip any internal sub-expressions - expression itself isn't + * allowed to have external attributes in this position so can't be + * stripped. */ + auto &expr = const_item.get_expr (); + expr->accept_vis (*this); + if (expr->is_marked_for_strip ()) + rust_error_at (expr->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); +} +void +AttrVisitor::visit (AST::StaticItem &static_item) +{ + // initial test based on outer attrs + expander.expand_cfg_attrs (static_item.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (static_item.get_outer_attrs ())) + { + static_item.mark_for_strip (); + return; + } + + expander.push_context (MacroExpander::ContextType::TYPE); + + // strip any sub-types + auto &type = static_item.get_type (); + type->accept_vis (*this); + + maybe_expand_type (type); + + if (type->is_marked_for_strip ()) + rust_error_at (type->get_locus (), "cannot strip type in this position"); + + expander.pop_context (); + + /* strip any internal sub-expressions - expression itself isn't + * allowed to have external attributes in this position so can't be + * stripped. */ + auto &expr = static_item.get_expr (); + expr->accept_vis (*this); + if (expr->is_marked_for_strip ()) + rust_error_at (expr->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); +} +void +AttrVisitor::visit (AST::TraitItemFunc &item) +{ + // initial test based on outer attrs + expander.expand_cfg_attrs (item.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (item.get_outer_attrs ())) + { + item.mark_for_strip (); + return; + } + + expand_trait_function_decl (item.get_trait_function_decl ()); + + if (item.has_definition ()) + { + /* strip any internal sub-expressions - expression itself isn't + * allowed to have external attributes in this position so can't be + * stripped. */ + auto &block = item.get_definition (); + block->accept_vis (*this); + if (block->is_marked_for_strip ()) + rust_error_at (block->get_locus (), + "cannot strip block expression in this " + "position - outer attributes not allowed"); + } +} +void +AttrVisitor::visit (AST::TraitItemMethod &item) +{ + // initial test based on outer attrs + expander.expand_cfg_attrs (item.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (item.get_outer_attrs ())) + { + item.mark_for_strip (); + return; + } + + expand_trait_method_decl (item.get_trait_method_decl ()); + + if (item.has_definition ()) + { + /* strip any internal sub-expressions - expression itself isn't + * allowed to have external attributes in this position so can't be + * stripped. */ + auto &block = item.get_definition (); + block->accept_vis (*this); + if (block->is_marked_for_strip ()) + rust_error_at (block->get_locus (), + "cannot strip block expression in this " + "position - outer attributes not allowed"); + } +} +void +AttrVisitor::visit (AST::TraitItemConst &item) +{ + // initial test based on outer attrs + expander.expand_cfg_attrs (item.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (item.get_outer_attrs ())) + { + item.mark_for_strip (); + return; + } + + expander.push_context (MacroExpander::ContextType::TYPE); + + // strip any sub-types + auto &type = item.get_type (); + type->accept_vis (*this); + + maybe_expand_type (type); + + if (type->is_marked_for_strip ()) + rust_error_at (type->get_locus (), "cannot strip type in this position"); + + expander.pop_context (); + + /* strip any internal sub-expressions - expression itself isn't + * allowed to have external attributes in this position so can't be + * stripped */ + if (item.has_expression ()) + { + auto &expr = item.get_expr (); + expr->accept_vis (*this); + if (expr->is_marked_for_strip ()) + rust_error_at (expr->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); + } +} +void +AttrVisitor::visit (AST::TraitItemType &item) +{ + // initial test based on outer attrs + expander.expand_cfg_attrs (item.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (item.get_outer_attrs ())) + { + item.mark_for_strip (); + return; + } + + if (item.has_type_param_bounds ()) + { + // don't strip directly, only components of bounds + for (auto &bound : item.get_type_param_bounds ()) + bound->accept_vis (*this); + } +} +void +AttrVisitor::visit (AST::Trait &trait) +{ + // initial strip test based on outer attrs + expander.expand_cfg_attrs (trait.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (trait.get_outer_attrs ())) + { + trait.mark_for_strip (); + return; + } + + // strip test based on inner attrs + expander.expand_cfg_attrs (trait.get_inner_attrs ()); + if (expander.fails_cfg_with_expand (trait.get_inner_attrs ())) + { + trait.mark_for_strip (); + return; + } + + // just expand sub-stuff - can't actually strip generic params themselves + for (auto ¶m : trait.get_generic_params ()) + param->accept_vis (*this); + + if (trait.has_type_param_bounds ()) + { + // don't strip directly, only components of bounds + for (auto &bound : trait.get_type_param_bounds ()) + bound->accept_vis (*this); + } + + if (trait.has_where_clause ()) + expand_where_clause (trait.get_where_clause ()); + + std::function (AST::SingleASTNode)> extractor + = [] (AST::SingleASTNode node) { return node.take_trait_item (); }; + + expand_macro_children (MacroExpander::TRAIT, trait.get_trait_items (), + extractor); +} +void +AttrVisitor::visit (AST::InherentImpl &impl) +{ + // initial strip test based on outer attrs + expander.expand_cfg_attrs (impl.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (impl.get_outer_attrs ())) + { + impl.mark_for_strip (); + return; + } + + // strip test based on inner attrs + expander.expand_cfg_attrs (impl.get_inner_attrs ()); + if (expander.fails_cfg_with_expand (impl.get_inner_attrs ())) + { + impl.mark_for_strip (); + return; + } + + // just expand sub-stuff - can't actually strip generic params themselves + for (auto ¶m : impl.get_generic_params ()) + param->accept_vis (*this); + + expander.push_context (MacroExpander::ContextType::TYPE); + + auto &type = impl.get_type (); + type->accept_vis (*this); + + maybe_expand_type (type); + + if (type->is_marked_for_strip ()) + rust_error_at (type->get_locus (), "cannot strip type in this position"); + + expander.pop_context (); + + if (impl.has_where_clause ()) + expand_where_clause (impl.get_where_clause ()); + + std::function (AST::SingleASTNode)> + extractor = [] (AST::SingleASTNode node) { return node.take_impl_item (); }; + + expand_macro_children (MacroExpander::IMPL, impl.get_impl_items (), + extractor); +} +void +AttrVisitor::visit (AST::TraitImpl &impl) +{ + // initial strip test based on outer attrs + expander.expand_cfg_attrs (impl.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (impl.get_outer_attrs ())) + { + impl.mark_for_strip (); + return; + } + + // strip test based on inner attrs + expander.expand_cfg_attrs (impl.get_inner_attrs ()); + if (expander.fails_cfg_with_expand (impl.get_inner_attrs ())) + { + impl.mark_for_strip (); + return; + } + + // just expand sub-stuff - can't actually strip generic params themselves + for (auto ¶m : impl.get_generic_params ()) + param->accept_vis (*this); + + expander.push_context (MacroExpander::ContextType::TYPE); + + auto &type = impl.get_type (); + type->accept_vis (*this); + + maybe_expand_type (type); + + if (type->is_marked_for_strip ()) + rust_error_at (type->get_locus (), "cannot strip type in this position"); + + expander.pop_context (); + + auto &trait_path = impl.get_trait_path (); + visit (trait_path); + if (trait_path.is_marked_for_strip ()) + rust_error_at (trait_path.get_locus (), + "cannot strip typepath in this position"); + + if (impl.has_where_clause ()) + expand_where_clause (impl.get_where_clause ()); + + std::function (AST::SingleASTNode)> + extractor + = [] (AST::SingleASTNode node) { return node.take_trait_impl_item (); }; + + expand_macro_children (MacroExpander::TRAIT_IMPL, impl.get_impl_items (), + extractor); +} +void +AttrVisitor::visit (AST::ExternalStaticItem &item) +{ + // strip test based on outer attrs + expander.expand_cfg_attrs (item.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (item.get_outer_attrs ())) + { + item.mark_for_strip (); + return; + } + + expander.push_context (MacroExpander::ContextType::TYPE); + + auto &type = item.get_type (); + type->accept_vis (*this); + + maybe_expand_type (type); + + if (type->is_marked_for_strip ()) + rust_error_at (type->get_locus (), "cannot strip type in this position"); + + expander.pop_context (); +} +void +AttrVisitor::visit (AST::ExternalFunctionItem &item) +{ + // strip test based on outer attrs + expander.expand_cfg_attrs (item.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (item.get_outer_attrs ())) + { + item.mark_for_strip (); + return; + } + + // just expand sub-stuff - can't actually strip generic params themselves + for (auto ¶m : item.get_generic_params ()) + param->accept_vis (*this); + + /* strip function parameters if required - this is specifically + * allowed by spec */ + auto ¶ms = item.get_function_params (); + for (auto it = params.begin (); it != params.end ();) + { + auto ¶m = *it; + + auto ¶m_attrs = param.get_outer_attrs (); + expander.expand_cfg_attrs (param_attrs); + if (expander.fails_cfg_with_expand (param_attrs)) + { + it = params.erase (it); + continue; + } + + expander.push_context (MacroExpander::ContextType::TYPE); + + auto &type = param.get_type (); + type->accept_vis (*this); + + maybe_expand_type (type); + + if (type->is_marked_for_strip ()) + rust_error_at (type->get_locus (), + "cannot strip type in this position"); + + expander.pop_context (); + + // increment if nothing else happens + ++it; + } + /* NOTE: these are extern function params, which may have different + * rules and restrictions to "normal" function params. So expansion + * handled separately. */ + + /* TODO: assuming that variadic nature cannot be stripped. If this + * is not true, then have code here to do so. */ + + if (item.has_return_type ()) + { + expander.push_context (MacroExpander::ContextType::TYPE); + + auto &return_type = item.get_return_type (); + return_type->accept_vis (*this); + + maybe_expand_type (return_type); + + if (return_type->is_marked_for_strip ()) + rust_error_at (return_type->get_locus (), + "cannot strip type in this position"); + + expander.pop_context (); + } + + if (item.has_where_clause ()) + expand_where_clause (item.get_where_clause ()); +} + +void +AttrVisitor::visit (AST::ExternBlock &block) +{ + // initial strip test based on outer attrs + expander.expand_cfg_attrs (block.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (block.get_outer_attrs ())) + { + block.mark_for_strip (); + return; + } + + // strip test based on inner attrs + expander.expand_cfg_attrs (block.get_inner_attrs ()); + if (expander.fails_cfg_with_expand (block.get_inner_attrs ())) + { + block.mark_for_strip (); + return; + } + + std::function (AST::SingleASTNode)> + extractor + = [] (AST::SingleASTNode node) { return node.take_external_item (); }; + + expand_macro_children (MacroExpander::EXTERN, block.get_extern_items (), + extractor); +} + +// I don't think it would be possible to strip macros without expansion +void +AttrVisitor::visit (AST::MacroMatchFragment &) +{} +void +AttrVisitor::visit (AST::MacroMatchRepetition &) +{} +void +AttrVisitor::visit (AST::MacroMatcher &) +{} +void +AttrVisitor::visit (AST::MacroRulesDefinition &rules_def) +{ + // initial strip test based on outer attrs + expander.expand_cfg_attrs (rules_def.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (rules_def.get_outer_attrs ())) + { + rules_def.mark_for_strip (); + return; + } + + // I don't think any macro rules can be stripped in any way + + auto path = Resolver::CanonicalPath::new_seg (rules_def.get_node_id (), + rules_def.get_rule_name ()); + expander.resolver->get_macro_scope ().insert (path, rules_def.get_node_id (), + rules_def.get_locus ()); + expander.mappings->insert_macro_def (&rules_def); + rust_debug_loc (rules_def.get_locus (), "inserting macro def: [%s]", + path.get ().c_str ()); +} + +void +AttrVisitor::visit (AST::MetaItemPath &) +{} +void +AttrVisitor::visit (AST::MetaItemSeq &) +{} +void +AttrVisitor::visit (AST::MetaWord &) +{} +void +AttrVisitor::visit (AST::MetaNameValueStr &) +{} +void +AttrVisitor::visit (AST::MetaListPaths &) +{} +void +AttrVisitor::visit (AST::MetaListNameValueStr &) +{} + +void +AttrVisitor::visit (AST::LiteralPattern &) +{ + // not possible +} +void +AttrVisitor::visit (AST::IdentifierPattern &pattern) +{ + // can only strip sub-patterns of the inner pattern to bind + if (!pattern.has_pattern_to_bind ()) + return; + + auto &sub_pattern = pattern.get_pattern_to_bind (); + sub_pattern->accept_vis (*this); + if (sub_pattern->is_marked_for_strip ()) + rust_error_at (sub_pattern->get_locus (), + "cannot strip pattern in this position"); +} +void +AttrVisitor::visit (AST::WildcardPattern &) +{ + // not possible +} +void +AttrVisitor::visit (AST::RangePatternBoundLiteral &) +{ + // not possible +} +void +AttrVisitor::visit (AST::RangePatternBoundPath &bound) +{ + // can expand path, but not strip it directly + auto &path = bound.get_path (); + visit (path); + if (path.is_marked_for_strip ()) + rust_error_at (path.get_locus (), "cannot strip path in this position"); +} +void +AttrVisitor::visit (AST::RangePatternBoundQualPath &bound) +{ + // can expand path, but not strip it directly + auto &path = bound.get_qualified_path (); + visit (path); + if (path.is_marked_for_strip ()) + rust_error_at (path.get_locus (), "cannot strip path in this position"); +} +void +AttrVisitor::visit (AST::RangePattern &pattern) +{ + // should have no capability to strip lower or upper bounds, only expand + pattern.get_lower_bound ()->accept_vis (*this); + pattern.get_upper_bound ()->accept_vis (*this); +} +void +AttrVisitor::visit (AST::ReferencePattern &pattern) +{ + auto &sub_pattern = pattern.get_referenced_pattern (); + sub_pattern->accept_vis (*this); + if (sub_pattern->is_marked_for_strip ()) + rust_error_at (sub_pattern->get_locus (), + "cannot strip pattern in this position"); +} +void +AttrVisitor::visit (AST::StructPatternFieldTuplePat &field) +{ + // initial strip test based on outer attrs + expander.expand_cfg_attrs (field.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (field.get_outer_attrs ())) + { + field.mark_for_strip (); + return; + } + + // strip sub-patterns (can't strip top-level pattern) + auto &sub_pattern = field.get_index_pattern (); + sub_pattern->accept_vis (*this); + if (sub_pattern->is_marked_for_strip ()) + rust_error_at (sub_pattern->get_locus (), + "cannot strip pattern in this position"); +} +void +AttrVisitor::visit (AST::StructPatternFieldIdentPat &field) +{ + // initial strip test based on outer attrs + expander.expand_cfg_attrs (field.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (field.get_outer_attrs ())) + { + field.mark_for_strip (); + return; + } + + // strip sub-patterns (can't strip top-level pattern) + auto &sub_pattern = field.get_ident_pattern (); + sub_pattern->accept_vis (*this); + if (sub_pattern->is_marked_for_strip ()) + rust_error_at (sub_pattern->get_locus (), + "cannot strip pattern in this position"); +} +void +AttrVisitor::visit (AST::StructPatternFieldIdent &field) +{ + // initial strip test based on outer attrs + expander.expand_cfg_attrs (field.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (field.get_outer_attrs ())) + { + field.mark_for_strip (); + return; + } +} +void +AttrVisitor::visit (AST::StructPattern &pattern) +{ + // expand (but don't strip) path + auto &path = pattern.get_path (); + visit (path); + if (path.is_marked_for_strip ()) + rust_error_at (path.get_locus (), "cannot strip path in this position"); + + /* TODO: apparently struct pattern fields can have outer attrs. so can they + * be stripped? */ + if (!pattern.has_struct_pattern_elems ()) + return; + + auto &elems = pattern.get_struct_pattern_elems (); + + // assuming you can strip struct pattern fields + expand_pointer_allow_strip (elems.get_struct_pattern_fields ()); + + // assuming you can strip the ".." part + if (elems.has_etc ()) + { + expander.expand_cfg_attrs (elems.get_etc_outer_attrs ()); + if (expander.fails_cfg_with_expand (elems.get_etc_outer_attrs ())) + elems.strip_etc (); + } +} +void +AttrVisitor::visit (AST::TupleStructItemsNoRange &tuple_items) +{ + // can't strip individual patterns, only sub-patterns + for (auto &pattern : tuple_items.get_patterns ()) + { + pattern->accept_vis (*this); + + if (pattern->is_marked_for_strip ()) + rust_error_at (pattern->get_locus (), + "cannot strip pattern in this position"); + // TODO: quit stripping now? or keep going? + } +} +void +AttrVisitor::visit (AST::TupleStructItemsRange &tuple_items) +{ + // can't strip individual patterns, only sub-patterns + for (auto &lower_pattern : tuple_items.get_lower_patterns ()) + { + lower_pattern->accept_vis (*this); + + if (lower_pattern->is_marked_for_strip ()) + rust_error_at (lower_pattern->get_locus (), + "cannot strip pattern in this position"); + // TODO: quit stripping now? or keep going? + } + for (auto &upper_pattern : tuple_items.get_upper_patterns ()) + { + upper_pattern->accept_vis (*this); + + if (upper_pattern->is_marked_for_strip ()) + rust_error_at (upper_pattern->get_locus (), + "cannot strip pattern in this position"); + // TODO: quit stripping now? or keep going? + } +} +void +AttrVisitor::visit (AST::TupleStructPattern &pattern) +{ + // expand (but don't strip) path + auto &path = pattern.get_path (); + visit (path); + if (path.is_marked_for_strip ()) + rust_error_at (path.get_locus (), "cannot strip path in this position"); + + if (pattern.has_items ()) + pattern.get_items ()->accept_vis (*this); +} +void +AttrVisitor::visit (AST::TuplePatternItemsMultiple &tuple_items) +{ + // can't strip individual patterns, only sub-patterns + for (auto &pattern : tuple_items.get_patterns ()) + { + pattern->accept_vis (*this); + + if (pattern->is_marked_for_strip ()) + rust_error_at (pattern->get_locus (), + "cannot strip pattern in this position"); + // TODO: quit stripping now? or keep going? + } +} +void +AttrVisitor::visit (AST::TuplePatternItemsRanged &tuple_items) +{ + // can't strip individual patterns, only sub-patterns + for (auto &lower_pattern : tuple_items.get_lower_patterns ()) + { + lower_pattern->accept_vis (*this); + + if (lower_pattern->is_marked_for_strip ()) + rust_error_at (lower_pattern->get_locus (), + "cannot strip pattern in this position"); + // TODO: quit stripping now? or keep going? + } + for (auto &upper_pattern : tuple_items.get_upper_patterns ()) + { + upper_pattern->accept_vis (*this); + + if (upper_pattern->is_marked_for_strip ()) + rust_error_at (upper_pattern->get_locus (), + "cannot strip pattern in this position"); + // TODO: quit stripping now? or keep going? + } +} +void +AttrVisitor::visit (AST::TuplePattern &pattern) +{ + if (pattern.has_tuple_pattern_items ()) + pattern.get_items ()->accept_vis (*this); +} +void +AttrVisitor::visit (AST::GroupedPattern &pattern) +{ + // can't strip inner pattern, only sub-patterns + auto &pattern_in_parens = pattern.get_pattern_in_parens (); + + pattern_in_parens->accept_vis (*this); + + if (pattern_in_parens->is_marked_for_strip ()) + rust_error_at (pattern_in_parens->get_locus (), + "cannot strip pattern in this position"); +} +void +AttrVisitor::visit (AST::SlicePattern &pattern) +{ + // can't strip individual patterns, only sub-patterns + for (auto &item : pattern.get_items ()) + { + item->accept_vis (*this); + + if (item->is_marked_for_strip ()) + rust_error_at (item->get_locus (), + "cannot strip pattern in this position"); + // TODO: quit stripping now? or keep going? + } +} + +void +AttrVisitor::visit (AST::EmptyStmt &) +{ + // assuming no outer attributes, so nothing can happen +} +void +AttrVisitor::visit (AST::LetStmt &stmt) +{ + // initial strip test based on outer attrs + expander.expand_cfg_attrs (stmt.get_outer_attrs ()); + if (expander.fails_cfg_with_expand (stmt.get_outer_attrs ())) + { + stmt.mark_for_strip (); + return; + } + + // can't strip pattern, but call for sub-patterns + auto &pattern = stmt.get_pattern (); + pattern->accept_vis (*this); + if (pattern->is_marked_for_strip ()) + rust_error_at (pattern->get_locus (), + "cannot strip pattern in this position"); + + // similar for type + if (stmt.has_type ()) + { + expander.push_context (MacroExpander::ContextType::TYPE); + + auto &type = stmt.get_type (); + type->accept_vis (*this); + + maybe_expand_type (type); + + if (type->is_marked_for_strip ()) + rust_error_at (type->get_locus (), + "cannot strip type in this position"); + + expander.pop_context (); + } + + /* strip any internal sub-expressions - expression itself isn't + * allowed to have external attributes in this position so can't be + * stripped */ + if (stmt.has_init_expr ()) + { + auto &init_expr = stmt.get_init_expr (); + init_expr->accept_vis (*this); + + if (init_expr->is_marked_for_strip ()) + rust_error_at (init_expr->get_locus (), + "cannot strip expression in this position - outer " + "attributes not allowed"); + + maybe_expand_expr (init_expr); + } +} +void +AttrVisitor::visit (AST::ExprStmtWithoutBlock &stmt) +{ + // outer attributes associated with expr, so rely on expr + + // guard - should prevent null pointer expr + if (stmt.is_marked_for_strip ()) + return; + + // strip if expr is to be stripped + auto &expr = stmt.get_expr (); + expr->accept_vis (*this); + if (expr->is_marked_for_strip ()) + { + stmt.mark_for_strip (); + return; + } +} +void +AttrVisitor::visit (AST::ExprStmtWithBlock &stmt) +{ + // outer attributes associated with expr, so rely on expr + + // guard - should prevent null pointer expr + if (stmt.is_marked_for_strip ()) + return; + + // strip if expr is to be stripped + auto &expr = stmt.get_expr (); + expr->accept_vis (*this); + if (expr->is_marked_for_strip ()) + { + stmt.mark_for_strip (); + return; + } +} + +void +AttrVisitor::visit (AST::TraitBound &bound) +{ + // nothing in for lifetimes to strip + + // expand but don't strip type path + auto &path = bound.get_type_path (); + visit (path); + if (path.is_marked_for_strip ()) + rust_error_at (path.get_locus (), + "cannot strip type path in this position"); +} +void +AttrVisitor::visit (AST::ImplTraitType &type) +{ + // don't strip directly, only components of bounds + for (auto &bound : type.get_type_param_bounds ()) + bound->accept_vis (*this); +} +void +AttrVisitor::visit (AST::TraitObjectType &type) +{ + // don't strip directly, only components of bounds + for (auto &bound : type.get_type_param_bounds ()) + bound->accept_vis (*this); +} +void +AttrVisitor::visit (AST::ParenthesisedType &type) +{ + // expand but don't strip inner type + auto &inner_type = type.get_type_in_parens (); + inner_type->accept_vis (*this); + if (inner_type->is_marked_for_strip ()) + rust_error_at (inner_type->get_locus (), + "cannot strip type in this position"); +} +void +AttrVisitor::visit (AST::ImplTraitTypeOneBound &type) +{ + // no stripping possible + visit (type.get_trait_bound ()); +} +void +AttrVisitor::visit (AST::TraitObjectTypeOneBound &type) +{ + // no stripping possible + visit (type.get_trait_bound ()); +} +void +AttrVisitor::visit (AST::TupleType &type) +{ + // TODO: assuming that types can't be stripped as types don't have outer + // attributes + for (auto &elem_type : type.get_elems ()) + { + elem_type->accept_vis (*this); + if (elem_type->is_marked_for_strip ()) + rust_error_at (elem_type->get_locus (), + "cannot strip type in this position"); + } +} +void +AttrVisitor::visit (AST::NeverType &) +{ + // no stripping possible +} +void +AttrVisitor::visit (AST::RawPointerType &type) +{ + // expand but don't strip type pointed to + auto &pointed_type = type.get_type_pointed_to (); + pointed_type->accept_vis (*this); + if (pointed_type->is_marked_for_strip ()) + rust_error_at (pointed_type->get_locus (), + "cannot strip type in this position"); +} +void +AttrVisitor::visit (AST::ReferenceType &type) +{ + // expand but don't strip type referenced + auto &referenced_type = type.get_type_referenced (); + referenced_type->accept_vis (*this); + if (referenced_type->is_marked_for_strip ()) + rust_error_at (referenced_type->get_locus (), + "cannot strip type in this position"); +} +void +AttrVisitor::visit (AST::ArrayType &type) +{ + // expand but don't strip type referenced + auto &base_type = type.get_elem_type (); + base_type->accept_vis (*this); + if (base_type->is_marked_for_strip ()) + rust_error_at (base_type->get_locus (), + "cannot strip type in this position"); + + // same for expression + auto &size_expr = type.get_size_expr (); + size_expr->accept_vis (*this); + if (size_expr->is_marked_for_strip ()) + rust_error_at (size_expr->get_locus (), + "cannot strip expression in this position"); +} +void +AttrVisitor::visit (AST::SliceType &type) +{ + // expand but don't strip elem type + auto &elem_type = type.get_elem_type (); + elem_type->accept_vis (*this); + if (elem_type->is_marked_for_strip ()) + rust_error_at (elem_type->get_locus (), + "cannot strip type in this position"); +} +void +AttrVisitor::visit (AST::InferredType &) +{ + // none possible +} +void +AttrVisitor::visit (AST::BareFunctionType &type) +{ + // seem to be no generics + + // presumably function params can be stripped + auto ¶ms = type.get_function_params (); + for (auto it = params.begin (); it != params.end ();) + { + auto ¶m = *it; + + auto ¶m_attrs = param.get_outer_attrs (); + expander.expand_cfg_attrs (param_attrs); + if (expander.fails_cfg_with_expand (param_attrs)) + { + it = params.erase (it); + continue; + } + + expander.push_context (MacroExpander::ContextType::TYPE); + + auto &type = param.get_type (); + type->accept_vis (*this); + + maybe_expand_type (type); + + if (type->is_marked_for_strip ()) + rust_error_at (type->get_locus (), + "cannot strip type in this position"); + + expander.pop_context (); + + // increment if nothing else happens + ++it; + } + + /* TODO: assuming that variadic nature cannot be stripped. If this + * is not true, then have code here to do so. */ + + if (type.has_return_type ()) + { + // FIXME: Can we have type expansion in this position? + // In that case, we need to handle AST::TypeNoBounds on top of just + // AST::Types + auto &return_type = type.get_return_type (); + return_type->accept_vis (*this); + if (return_type->is_marked_for_strip ()) + rust_error_at (return_type->get_locus (), + "cannot strip type in this position"); + } + + // no where clause, apparently +} +void +AttrVisitor::maybe_expand_expr (std::unique_ptr &expr) +{ + auto final_fragment = expand_macro_fragment_recursive (); + if (final_fragment.should_expand ()) + expr = final_fragment.take_expression_fragment (); +} + +void +AttrVisitor::maybe_expand_type (std::unique_ptr &type) +{ + auto final_fragment = expand_macro_fragment_recursive (); + if (final_fragment.should_expand ()) + type = final_fragment.take_type_fragment (); +} +} // namespace Rust diff --git a/gcc/rust/expand/rust-attribute-visitor.h b/gcc/rust/expand/rust-attribute-visitor.h new file mode 100644 index 00000000000..0f9d1065334 --- /dev/null +++ b/gcc/rust/expand/rust-attribute-visitor.h @@ -0,0 +1,316 @@ +// Copyright (C) 2020-2022 Free Software Foundation, Inc. + +// This file is part of GCC. + +// GCC is free software; you can redistribute it and/or modify it under +// the terms of the GNU General Public License as published by the Free +// Software Foundation; either version 3, or (at your option) any later +// version. + +// GCC is distributed in the hope that it will be useful, but WITHOUT ANY +// WARRANTY; without even the implied warranty of MERCHANTABILITY or +// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License +// for more details. + +// You should have received a copy of the GNU General Public License +// along with GCC; see the file COPYING3. If not see +// . + +#include "rust-ast-visitor.h" +#include "rust-ast.h" +#include "rust-macro-expand.h" + +namespace Rust { +// Visitor used to expand attributes. +class AttrVisitor : public AST::ASTVisitor +{ +private: + MacroExpander &expander; + void maybe_expand_expr (std::unique_ptr &expr); + void maybe_expand_type (std::unique_ptr &expr); + +public: + AttrVisitor (MacroExpander &expander) : expander (expander) {} + + void expand_struct_fields (std::vector &fields); + void expand_tuple_fields (std::vector &fields); + void expand_function_params (std::vector ¶ms); + void expand_generic_args (AST::GenericArgs &args); + void expand_qualified_path_type (AST::QualifiedPathType &path_type); + void expand_closure_params (std::vector ¶ms); + void expand_self_param (AST::SelfParam &self_param); + void expand_where_clause (AST::WhereClause &where_clause); + void expand_trait_function_decl (AST::TraitFunctionDecl &decl); + void expand_trait_method_decl (AST::TraitMethodDecl &decl); + + /** + * Expand The current macro fragment recursively until it could not be + * expanded further. + * + * The return value checking works because correctly + * expanded fragment can never be an error (if the fragment can not be + * expanded, a stand-in error fragment will be returned; for fragments that + * could not be further expanded: the fragment prior to the expansion failure + * will be returned). + * + * @return Either the expanded fragment or an empty errored-out fragment + * indicating an expansion failure. + */ + AST::ASTFragment expand_macro_fragment_recursive () + { + auto fragment = expander.take_expanded_fragment (*this); + unsigned int original_depth = expander.expansion_depth; + auto final_fragment = AST::ASTFragment ({}, true); + + while (fragment.should_expand ()) + { + final_fragment = std::move (fragment); + expander.expansion_depth++; + // further expand the previously expanded macro fragment + auto new_fragment = expander.take_expanded_fragment (*this); + if (new_fragment.is_error ()) + break; + fragment = std::move (new_fragment); + } + expander.expansion_depth = original_depth; + return final_fragment; + } + + /** + * Expand a set of values, erasing them if they are marked for strip, and + * replacing them with expanded macro nodes if necessary. + * This function is slightly different from `expand_pointer_allow_strip` as + * it can only be called in certain expansion contexts - where macro + * invocations are allowed. + * + * @param ctx Context to use for macro expansion + * @param values Iterable reference over values to replace or erase + * @param extractor Function to call when replacing values with the content + * of an expanded AST node + */ + template + void expand_macro_children (MacroExpander::ContextType ctx, T &values, + std::function extractor) + { + expander.push_context (ctx); + + for (auto it = values.begin (); it != values.end ();) + { + auto &value = *it; + + // mark for stripping if required + value->accept_vis (*this); + + // recursively expand the children + auto final_fragment = expand_macro_fragment_recursive (); + + if (final_fragment.should_expand ()) + { + it = values.erase (it); + for (auto &node : final_fragment.get_nodes ()) + { + auto new_node = extractor (node); + if (new_node != nullptr && !new_node->is_marked_for_strip ()) + { + it = values.insert (it, std::move (new_node)); + it++; + } + } + } + else if (value->is_marked_for_strip ()) + { + it = values.erase (it); + } + else + { + ++it; + } + } + + expander.pop_context (); + } + + template void expand_pointer_allow_strip (T &values) + { + for (auto it = values.begin (); it != values.end ();) + { + auto &value = *it; + + // mark for stripping if required + value->accept_vis (*this); + if (value->is_marked_for_strip ()) + { + it = values.erase (it); + } + else + { + ++it; + } + } + } + + void visit (AST::Token &) override; + void visit (AST::DelimTokenTree &) override; + void visit (AST::AttrInputMetaItemContainer &) override; + void visit (AST::IdentifierExpr &ident_expr) override; + void visit (AST::Lifetime &) override; + void visit (AST::LifetimeParam &) override; + void visit (AST::ConstGenericParam &) override; + + void visit (AST::MacroInvocation ¯o_invoc) override; + + void visit (AST::PathInExpression &path) override; + void visit (AST::TypePathSegment &) override; + void visit (AST::TypePathSegmentGeneric &segment) override; + void visit (AST::TypePathSegmentFunction &segment) override; + void visit (AST::TypePath &path) override; + void visit (AST::QualifiedPathInExpression &path) override; + void visit (AST::QualifiedPathInType &path) override; + + void visit (AST::LiteralExpr &expr) override; + void visit (AST::AttrInputLiteral &) override; + void visit (AST::MetaItemLitExpr &) override; + void visit (AST::MetaItemPathLit &) override; + void visit (AST::BorrowExpr &expr) override; + void visit (AST::DereferenceExpr &expr) override; + void visit (AST::ErrorPropagationExpr &expr) override; + void visit (AST::NegationExpr &expr) override; + void visit (AST::ArithmeticOrLogicalExpr &expr) override; + void visit (AST::ComparisonExpr &expr) override; + void visit (AST::LazyBooleanExpr &expr) override; + void visit (AST::TypeCastExpr &expr) override; + void visit (AST::AssignmentExpr &expr) override; + void visit (AST::CompoundAssignmentExpr &expr) override; + void visit (AST::GroupedExpr &expr) override; + void visit (AST::ArrayElemsValues &elems) override; + void visit (AST::ArrayElemsCopied &elems) override; + void visit (AST::ArrayExpr &expr) override; + void visit (AST::ArrayIndexExpr &expr) override; + void visit (AST::TupleExpr &expr) override; + void visit (AST::TupleIndexExpr &expr) override; + void visit (AST::StructExprStruct &expr) override; + void visit (AST::StructExprFieldIdentifier &) override; + void visit (AST::StructExprFieldIdentifierValue &field) override; + + void visit (AST::StructExprFieldIndexValue &field) override; + void visit (AST::StructExprStructFields &expr) override; + void visit (AST::StructExprStructBase &expr) override; + void visit (AST::CallExpr &expr) override; + void visit (AST::MethodCallExpr &expr) override; + void visit (AST::FieldAccessExpr &expr) override; + void visit (AST::ClosureExprInner &expr) override; + + void visit (AST::BlockExpr &expr) override; + + void visit (AST::ClosureExprInnerTyped &expr) override; + void visit (AST::ContinueExpr &expr) override; + void visit (AST::BreakExpr &expr) override; + void visit (AST::RangeFromToExpr &expr) override; + void visit (AST::RangeFromExpr &expr) override; + void visit (AST::RangeToExpr &expr) override; + void visit (AST::RangeFullExpr &) override; + void visit (AST::RangeFromToInclExpr &expr) override; + void visit (AST::RangeToInclExpr &expr) override; + void visit (AST::ReturnExpr &expr) override; + void visit (AST::UnsafeBlockExpr &expr) override; + void visit (AST::LoopExpr &expr) override; + void visit (AST::WhileLoopExpr &expr) override; + void visit (AST::WhileLetLoopExpr &expr) override; + void visit (AST::ForLoopExpr &expr) override; + void visit (AST::IfExpr &expr) override; + void visit (AST::IfExprConseqElse &expr) override; + void visit (AST::IfExprConseqIf &expr) override; + void visit (AST::IfExprConseqIfLet &expr) override; + void visit (AST::IfLetExpr &expr) override; + void visit (AST::IfLetExprConseqElse &expr) override; + void visit (AST::IfLetExprConseqIf &expr) override; + void visit (AST::IfLetExprConseqIfLet &expr) override; + void visit (AST::MatchExpr &expr) override; + void visit (AST::AwaitExpr &expr) override; + void visit (AST::AsyncBlockExpr &expr) override; + void visit (AST::TypeParam ¶m) override; + void visit (AST::LifetimeWhereClauseItem &) override; + void visit (AST::TypeBoundWhereClauseItem &item) override; + void visit (AST::Method &method) override; + void visit (AST::Module &module) override; + void visit (AST::ExternCrate &crate) override; + void visit (AST::UseTreeGlob &) override; + void visit (AST::UseTreeList &) override; + void visit (AST::UseTreeRebind &) override; + void visit (AST::UseDeclaration &use_decl) override; + void visit (AST::Function &function) override; + void visit (AST::TypeAlias &type_alias) override; + void visit (AST::StructStruct &struct_item) override; + void visit (AST::TupleStruct &tuple_struct) override; + void visit (AST::EnumItem &item) override; + void visit (AST::EnumItemTuple &item) override; + void visit (AST::EnumItemStruct &item) override; + void visit (AST::EnumItemDiscriminant &item) override; + void visit (AST::Enum &enum_item) override; + void visit (AST::Union &union_item) override; + void visit (AST::ConstantItem &const_item) override; + void visit (AST::StaticItem &static_item) override; + void visit (AST::TraitItemFunc &item) override; + void visit (AST::TraitItemMethod &item) override; + void visit (AST::TraitItemConst &item) override; + void visit (AST::TraitItemType &item) override; + void visit (AST::Trait &trait) override; + void visit (AST::InherentImpl &impl) override; + void visit (AST::TraitImpl &impl) override; + void visit (AST::ExternalStaticItem &item) override; + void visit (AST::ExternalFunctionItem &item) override; + void visit (AST::ExternBlock &block) override; + + // I don't think it would be possible to strip macros without expansion + void visit (AST::MacroMatchFragment &) override; + void visit (AST::MacroMatchRepetition &) override; + void visit (AST::MacroMatcher &) override; + void visit (AST::MacroRulesDefinition &rules_def) override; + void visit (AST::MetaItemPath &) override; + void visit (AST::MetaItemSeq &) override; + void visit (AST::MetaWord &) override; + void visit (AST::MetaNameValueStr &) override; + void visit (AST::MetaListPaths &) override; + void visit (AST::MetaListNameValueStr &) override; + void visit (AST::LiteralPattern &) override; + void visit (AST::IdentifierPattern &pattern) override; + void visit (AST::WildcardPattern &) override; + void visit (AST::RangePatternBoundLiteral &) override; + void visit (AST::RangePatternBoundPath &bound) override; + void visit (AST::RangePatternBoundQualPath &bound) override; + void visit (AST::RangePattern &pattern) override; + void visit (AST::ReferencePattern &pattern) override; + void visit (AST::StructPatternFieldTuplePat &field) override; + void visit (AST::StructPatternFieldIdentPat &field) override; + void visit (AST::StructPatternFieldIdent &field) override; + void visit (AST::StructPattern &pattern) override; + void visit (AST::TupleStructItemsNoRange &tuple_items) override; + void visit (AST::TupleStructItemsRange &tuple_items) override; + void visit (AST::TupleStructPattern &pattern) override; + void visit (AST::TuplePatternItemsMultiple &tuple_items) override; + void visit (AST::TuplePatternItemsRanged &tuple_items) override; + void visit (AST::TuplePattern &pattern) override; + void visit (AST::GroupedPattern &pattern) override; + void visit (AST::SlicePattern &pattern) override; + + void visit (AST::EmptyStmt &) override; + void visit (AST::LetStmt &stmt) override; + void visit (AST::ExprStmtWithoutBlock &stmt) override; + void visit (AST::ExprStmtWithBlock &stmt) override; + + void visit (AST::TraitBound &bound) override; + void visit (AST::ImplTraitType &type) override; + void visit (AST::TraitObjectType &type) override; + void visit (AST::ParenthesisedType &type) override; + void visit (AST::ImplTraitTypeOneBound &type) override; + void visit (AST::TraitObjectTypeOneBound &type) override; + void visit (AST::TupleType &type) override; + void visit (AST::NeverType &) override; + void visit (AST::RawPointerType &type) override; + void visit (AST::ReferenceType &type) override; + void visit (AST::ArrayType &type) override; + void visit (AST::SliceType &type) override; + void visit (AST::InferredType &) override; + void visit (AST::BareFunctionType &type) override; +}; +} // namespace Rust diff --git a/gcc/rust/expand/rust-macro-builtins.cc b/gcc/rust/expand/rust-macro-builtins.cc new file mode 100644 index 00000000000..5eace13d197 --- /dev/null +++ b/gcc/rust/expand/rust-macro-builtins.cc @@ -0,0 +1,484 @@ +// Copyright (C) 2020-2022 Free Software Foundation, Inc. + +// This file is part of GCC. + +// GCC is free software; you can redistribute it and/or modify it under +// the terms of the GNU General Public License as published by the Free +// Software Foundation; either version 3, or (at your option) any later +// version. + +// GCC is distributed in the hope that it will be useful, but WITHOUT ANY +// WARRANTY; without even the implied warranty of MERCHANTABILITY or +// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License +// for more details. + +// You should have received a copy of the GNU General Public License +// along with GCC; see the file COPYING3. If not see +// . + +#include "rust-macro-builtins.h" +#include "rust-diagnostics.h" +#include "rust-expr.h" +#include "rust-session-manager.h" +#include "rust-macro-invoc-lexer.h" +#include "rust-lex.h" +#include "rust-parse.h" + +namespace Rust { +namespace { +std::unique_ptr +make_string (Location locus, std::string value) +{ + return std::unique_ptr ( + new AST::LiteralExpr (value, AST::Literal::STRING, + PrimitiveCoreType::CORETYPE_STR, {}, locus)); +} + +/* Match the end token of a macro given the start delimiter of the macro */ + +static inline TokenId +macro_end_token (AST::DelimTokenTree &invoc_token_tree, + Parser &parser) +{ + auto last_token_id = TokenId::RIGHT_CURLY; + switch (invoc_token_tree.get_delim_type ()) + { + case AST::DelimType::PARENS: + last_token_id = TokenId::RIGHT_PAREN; + rust_assert (parser.skip_token (LEFT_PAREN)); + break; + + case AST::DelimType::CURLY: + rust_assert (parser.skip_token (LEFT_CURLY)); + break; + + case AST::DelimType::SQUARE: + last_token_id = TokenId::RIGHT_SQUARE; + rust_assert (parser.skip_token (LEFT_SQUARE)); + break; + } + + return last_token_id; +} + +/* Parse a single string literal from the given delimited token tree, + and return the LiteralExpr for it. Allow for an optional trailing comma, + but otherwise enforce that these are the only tokens. */ + +std::unique_ptr +parse_single_string_literal (AST::DelimTokenTree &invoc_token_tree, + Location invoc_locus) +{ + MacroInvocLexer lex (invoc_token_tree.to_token_stream ()); + Parser parser (lex); + + auto last_token_id = macro_end_token (invoc_token_tree, parser); + + std::unique_ptr lit_expr = nullptr; + + if (parser.peek_current_token ()->get_id () == STRING_LITERAL) + { + lit_expr = parser.parse_literal_expr (); + parser.maybe_skip_token (COMMA); + if (parser.peek_current_token ()->get_id () != last_token_id) + { + lit_expr = nullptr; + rust_error_at (invoc_locus, "macro takes 1 argument"); + } + } + else if (parser.peek_current_token ()->get_id () == last_token_id) + rust_error_at (invoc_locus, "macro takes 1 argument"); + else + rust_error_at (invoc_locus, "argument must be a string literal"); + + parser.skip_token (last_token_id); + + return lit_expr; +} + +/* Treat PATH as a path relative to the source file currently being + compiled, and return the absolute path for it. */ + +std::string +source_relative_path (std::string path, Location locus) +{ + std::string compile_fname + = Session::get_instance ().linemap->location_file (locus); + + auto dir_separator_pos = compile_fname.rfind (file_separator); + + /* If there is no file_separator in the path, use current dir ('.'). */ + std::string dirname; + if (dir_separator_pos == std::string::npos) + dirname = std::string (".") + file_separator; + else + dirname = compile_fname.substr (0, dir_separator_pos) + file_separator; + + return dirname + path; +} + +/* Read the full contents of the file FILENAME and return them in a vector. + FIXME: platform specific. */ + +std::vector +load_file_bytes (const char *filename) +{ + RAIIFile file_wrap (filename); + if (file_wrap.get_raw () == nullptr) + { + rust_error_at (Location (), "cannot open filename %s: %m", filename); + return std::vector (); + } + + FILE *f = file_wrap.get_raw (); + fseek (f, 0L, SEEK_END); + long fsize = ftell (f); + fseek (f, 0L, SEEK_SET); + + std::vector buf (fsize); + + if (fread (&buf[0], fsize, 1, f) != 1) + { + rust_error_at (Location (), "error reading file %s: %m", filename); + return std::vector (); + } + + return buf; +} +} // namespace + +AST::ASTFragment +MacroBuiltin::assert (Location invoc_locus, AST::MacroInvocData &invoc) +{ + rust_debug ("assert!() called"); + + return AST::ASTFragment::create_error (); +} + +AST::ASTFragment +MacroBuiltin::file (Location invoc_locus, AST::MacroInvocData &invoc) +{ + auto current_file + = Session::get_instance ().linemap->location_file (invoc_locus); + auto file_str = AST::SingleASTNode (make_string (invoc_locus, current_file)); + + return AST::ASTFragment ({file_str}); +} + +AST::ASTFragment +MacroBuiltin::column (Location invoc_locus, AST::MacroInvocData &invoc) +{ + auto current_column + = Session::get_instance ().linemap->location_to_column (invoc_locus); + + auto column_no = AST::SingleASTNode (std::unique_ptr ( + new AST::LiteralExpr (std::to_string (current_column), AST::Literal::INT, + PrimitiveCoreType::CORETYPE_U32, {}, invoc_locus))); + + return AST::ASTFragment ({column_no}); +} + +/* Expand builtin macro include_bytes!("filename"), which includes the contents + of the given file as reference to a byte array. Yields an expression of type + &'static [u8; N]. */ + +AST::ASTFragment +MacroBuiltin::include_bytes (Location invoc_locus, AST::MacroInvocData &invoc) +{ + /* Get target filename from the macro invocation, which is treated as a path + relative to the include!-ing file (currently being compiled). */ + auto lit_expr + = parse_single_string_literal (invoc.get_delim_tok_tree (), invoc_locus); + if (lit_expr == nullptr) + return AST::ASTFragment::create_error (); + + std::string target_filename + = source_relative_path (lit_expr->as_string (), invoc_locus); + + std::vector bytes = load_file_bytes (target_filename.c_str ()); + + /* Is there a more efficient way to do this? */ + std::vector> elts; + for (uint8_t b : bytes) + { + elts.emplace_back ( + new AST::LiteralExpr (std::string (1, (char) b), AST::Literal::BYTE, + PrimitiveCoreType::CORETYPE_U8, + {} /* outer_attrs */, invoc_locus)); + } + + auto elems = std::unique_ptr ( + new AST::ArrayElemsValues (std::move (elts), invoc_locus)); + + auto array = std::unique_ptr ( + new AST::ArrayExpr (std::move (elems), {}, {}, invoc_locus)); + + auto borrow = std::unique_ptr ( + new AST::BorrowExpr (std::move (array), false, false, {}, invoc_locus)); + + auto node = AST::SingleASTNode (std::move (borrow)); + return AST::ASTFragment ({node}); +} + +/* Expand builtin macro include_str!("filename"), which includes the contents + of the given file as a string. The file must be UTF-8 encoded. Yields an + expression of type &'static str. */ + +AST::ASTFragment +MacroBuiltin::include_str (Location invoc_locus, AST::MacroInvocData &invoc) +{ + /* Get target filename from the macro invocation, which is treated as a path + relative to the include!-ing file (currently being compiled). */ + auto lit_expr + = parse_single_string_literal (invoc.get_delim_tok_tree (), invoc_locus); + if (lit_expr == nullptr) + return AST::ASTFragment::create_error (); + + std::string target_filename + = source_relative_path (lit_expr->as_string (), invoc_locus); + + std::vector bytes = load_file_bytes (target_filename.c_str ()); + + /* FIXME: Enforce that the file contents are valid UTF-8. */ + std::string str ((const char *) &bytes[0], bytes.size ()); + + auto node = AST::SingleASTNode (make_string (invoc_locus, str)); + return AST::ASTFragment ({node}); +} + +/* Expand builtin macro compile_error!("error"), which forces a compile error + during the compile time. */ +AST::ASTFragment +MacroBuiltin::compile_error (Location invoc_locus, AST::MacroInvocData &invoc) +{ + auto lit_expr + = parse_single_string_literal (invoc.get_delim_tok_tree (), invoc_locus); + if (lit_expr == nullptr) + return AST::ASTFragment::create_error (); + + std::string error_string = lit_expr->as_string (); + rust_error_at (invoc_locus, "%s", error_string.c_str ()); + + return AST::ASTFragment::create_error (); +} + +/* Expand builtin macro concat!(), which joins all the literal parameters + into a string with no delimiter. */ + +AST::ASTFragment +MacroBuiltin::concat (Location invoc_locus, AST::MacroInvocData &invoc) +{ + auto invoc_token_tree = invoc.get_delim_tok_tree (); + MacroInvocLexer lex (invoc_token_tree.to_token_stream ()); + Parser parser (lex); + + auto str = std::string (); + bool has_error = false; + + auto last_token_id = macro_end_token (invoc_token_tree, parser); + + /* NOTE: concat! could accept no argument, so we don't have any checks here */ + while (parser.peek_current_token ()->get_id () != last_token_id) + { + auto lit_expr = parser.parse_literal_expr (); + if (lit_expr) + { + str += lit_expr->as_string (); + } + else + { + auto current_token = parser.peek_current_token (); + rust_error_at (current_token->get_locus (), + "argument must be a constant literal"); + has_error = true; + // Just crash if the current token can't be skipped + rust_assert (parser.skip_token (current_token->get_id ())); + } + parser.maybe_skip_token (COMMA); + } + + parser.skip_token (last_token_id); + + if (has_error) + return AST::ASTFragment::create_error (); + + auto node = AST::SingleASTNode (make_string (invoc_locus, str)); + return AST::ASTFragment ({node}); +} + +/* Expand builtin macro env!(), which inspects an environment variable at + compile time. */ + +AST::ASTFragment +MacroBuiltin::env (Location invoc_locus, AST::MacroInvocData &invoc) +{ + auto invoc_token_tree = invoc.get_delim_tok_tree (); + MacroInvocLexer lex (invoc_token_tree.to_token_stream ()); + Parser parser (lex); + + auto last_token_id = macro_end_token (invoc_token_tree, parser); + + if (parser.peek_current_token ()->get_id () != STRING_LITERAL) + { + if (parser.peek_current_token ()->get_id () == last_token_id) + rust_error_at (invoc_locus, "env! takes 1 or 2 arguments"); + else + rust_error_at (parser.peek_current_token ()->get_locus (), + "argument must be a string literal"); + return AST::ASTFragment::create_error (); + } + + auto lit_expr = parser.parse_literal_expr (); + auto comma_skipped = parser.maybe_skip_token (COMMA); + + std::unique_ptr error_expr = nullptr; + + if (parser.peek_current_token ()->get_id () != last_token_id) + { + if (!comma_skipped) + { + rust_error_at (parser.peek_current_token ()->get_locus (), + "expected token: %<,%>"); + return AST::ASTFragment::create_error (); + } + if (parser.peek_current_token ()->get_id () != STRING_LITERAL) + { + rust_error_at (parser.peek_current_token ()->get_locus (), + "argument must be a string literal"); + return AST::ASTFragment::create_error (); + } + + error_expr = parser.parse_literal_expr (); + parser.maybe_skip_token (COMMA); + } + + if (parser.peek_current_token ()->get_id () != last_token_id) + { + rust_error_at (invoc_locus, "env! takes 1 or 2 arguments"); + return AST::ASTFragment::create_error (); + } + + parser.skip_token (last_token_id); + + auto env_value = getenv (lit_expr->as_string ().c_str ()); + + if (env_value == nullptr) + { + if (error_expr == nullptr) + rust_error_at (invoc_locus, "environment variable %qs not defined", + lit_expr->as_string ().c_str ()); + else + rust_error_at (invoc_locus, "%s", error_expr->as_string ().c_str ()); + return AST::ASTFragment::create_error (); + } + + auto node = AST::SingleASTNode (make_string (invoc_locus, env_value)); + return AST::ASTFragment ({node}); +} + +AST::ASTFragment +MacroBuiltin::cfg (Location invoc_locus, AST::MacroInvocData &invoc) +{ + // only parse if not already parsed + if (!invoc.is_parsed ()) + { + std::unique_ptr converted_input ( + invoc.get_delim_tok_tree ().parse_to_meta_item ()); + + if (converted_input == nullptr) + { + rust_debug ("DEBUG: failed to parse macro to meta item"); + // TODO: do something now? is this an actual error? + } + else + { + std::vector> meta_items ( + std::move (converted_input->get_items ())); + invoc.set_meta_item_output (std::move (meta_items)); + } + } + + /* TODO: assuming that cfg! macros can only have one meta item inner, like cfg + * attributes */ + if (invoc.get_meta_items ().size () != 1) + return AST::ASTFragment::create_error (); + + bool result = invoc.get_meta_items ()[0]->check_cfg_predicate ( + Session::get_instance ()); + auto literal_exp = AST::SingleASTNode (std::unique_ptr ( + new AST::LiteralExpr (result ? "true" : "false", AST::Literal::BOOL, + PrimitiveCoreType::CORETYPE_BOOL, {}, invoc_locus))); + + return AST::ASTFragment ({literal_exp}); +} + +/* Expand builtin macro include!(), which includes a source file at the current + scope compile time. */ + +AST::ASTFragment +MacroBuiltin::include (Location invoc_locus, AST::MacroInvocData &invoc) +{ + /* Get target filename from the macro invocation, which is treated as a path + relative to the include!-ing file (currently being compiled). */ + auto lit_expr + = parse_single_string_literal (invoc.get_delim_tok_tree (), invoc_locus); + if (lit_expr == nullptr) + return AST::ASTFragment::create_error (); + + std::string filename + = source_relative_path (lit_expr->as_string (), invoc_locus); + auto target_filename + = Rust::Session::get_instance ().include_extra_file (std::move (filename)); + + RAIIFile target_file (target_filename); + Linemap *linemap = Session::get_instance ().linemap; + + if (!target_file.ok ()) + { + rust_error_at (lit_expr->get_locus (), + "cannot open included file %qs: %m", target_filename); + return AST::ASTFragment::create_error (); + } + + rust_debug ("Attempting to parse included file %s", target_filename); + + Lexer lex (target_filename, std::move (target_file), linemap); + Parser parser (lex); + + auto parsed_items = parser.parse_items (); + bool has_error = !parser.get_errors ().empty (); + + for (const auto &error : parser.get_errors ()) + error.emit_error (); + + if (has_error) + { + // inform the user that the errors above are from a included file + rust_inform (invoc_locus, "included from here"); + return AST::ASTFragment::create_error (); + } + + std::vector nodes{}; + for (auto &item : parsed_items) + { + AST::SingleASTNode node (std::move (item)); + nodes.push_back (node); + } + + return AST::ASTFragment (nodes); +} + +AST::ASTFragment +MacroBuiltin::line (Location invoc_locus, AST::MacroInvocData &invoc) +{ + auto current_line + = Session::get_instance ().linemap->location_to_line (invoc_locus); + + auto line_no = AST::SingleASTNode (std::unique_ptr ( + new AST::LiteralExpr (std::to_string (current_line), AST::Literal::INT, + PrimitiveCoreType::CORETYPE_U32, {}, invoc_locus))); + + return AST::ASTFragment ({line_no}); +} + +} // namespace Rust diff --git a/gcc/rust/expand/rust-macro-builtins.h b/gcc/rust/expand/rust-macro-builtins.h new file mode 100644 index 00000000000..91f3727d450 --- /dev/null +++ b/gcc/rust/expand/rust-macro-builtins.h @@ -0,0 +1,107 @@ +// Copyright (C) 2020-2022 Free Software Foundation, Inc. + +// This file is part of GCC. + +// GCC is free software; you can redistribute it and/or modify it under +// the terms of the GNU General Public License as published by the Free +// Software Foundation; either version 3, or (at your option) any later +// version. + +// GCC is distributed in the hope that it will be useful, but WITHOUT ANY +// WARRANTY; without even the implied warranty of MERCHANTABILITY or +// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License +// for more details. + +// You should have received a copy of the GNU General Public License +// along with GCC; see the file COPYING3. If not see +// . + +#ifndef RUST_MACRO_BUILTINS_H +#define RUST_MACRO_BUILTINS_H + +#include "rust-ast.h" +#include "rust-location.h" + +/** + * This class provides a list of builtin macros implemented by the compiler. + * The functions defined are called "builtin transcribers" in that they replace + * the transcribing part of a macro definition. + * + * Like regular macro transcribers, they are responsible for building and + * returning an AST fragment: basically a vector of AST nodes put together. + * + * Unlike regular declarative macros where each match arm has its own associated + * transcriber, builtin transcribers are responsible for handling all match arms + * of the macro. This means that you should take extra care when implementing a + * builtin containing multiple match arms: You will probably need to do some + * lookahead in order to determine which match arm the user intended to use. + * + * An example of this is the `assert!()` macro: + * + * ``` + * macro_rules! assert { + * ($cond:expr $(,)?) => {{ ... }}; + * ($cond : expr, $ ($arg : tt) +) = > {{ ... }}; + * } + * ``` + * + * If more tokens exist beyond the optional comma, they need to be handled as + * a token-tree for a custom panic message. + * + * These builtin macros with empty transcribers are defined in the standard + * library. They are marked with a special attribute, `#[rustc_builtin_macro]`. + * When this attribute is present on a macro definition, the compiler should + * look for an associated transcriber in the mappings. Meaning that you must + * remember to insert your transcriber in the `builtin_macros` map of the + *`Mappings`. + * + * This map is built as a static variable in the `insert_macro_def()` method + * of the `Mappings` class. + */ + +/* If assert is defined as a macro this file will not parse, so undefine this + before continuing. */ +#ifdef assert +#undef assert +#endif + +namespace Rust { +class MacroBuiltin +{ +public: + static AST::ASTFragment assert (Location invoc_locus, + AST::MacroInvocData &invoc); + + static AST::ASTFragment file (Location invoc_locus, + AST::MacroInvocData &invoc); + + static AST::ASTFragment column (Location invoc_locus, + AST::MacroInvocData &invoc); + + static AST::ASTFragment include_bytes (Location invoc_locus, + AST::MacroInvocData &invoc); + + static AST::ASTFragment include_str (Location invoc_locus, + AST::MacroInvocData &invoc); + + static AST::ASTFragment compile_error (Location invoc_locus, + AST::MacroInvocData &invoc); + + static AST::ASTFragment concat (Location invoc_locus, + AST::MacroInvocData &invoc); + + static AST::ASTFragment env (Location invoc_locus, + AST::MacroInvocData &invoc); + + static AST::ASTFragment cfg (Location invoc_locus, + AST::MacroInvocData &invoc); + + static AST::ASTFragment include (Location invoc_locus, + AST::MacroInvocData &invoc); + + static AST::ASTFragment line (Location invoc_locus, + AST::MacroInvocData &invoc); +}; +} // namespace Rust + +#endif // RUST_MACRO_BUILTINS_H diff --git a/gcc/rust/expand/rust-macro-expand.cc b/gcc/rust/expand/rust-macro-expand.cc new file mode 100644 index 00000000000..1d57e394220 --- /dev/null +++ b/gcc/rust/expand/rust-macro-expand.cc @@ -0,0 +1,1012 @@ +// Copyright (C) 2020-2022 Free Software Foundation, Inc. + +// This file is part of GCC. + +// GCC is free software; you can redistribute it and/or modify it under +// the terms of the GNU General Public License as published by the Free +// Software Foundation; either version 3, or (at your option) any later +// version. + +// GCC is distributed in the hope that it will be useful, but WITHOUT ANY +// WARRANTY; without even the implied warranty of MERCHANTABILITY or +// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License +// for more details. + +// You should have received a copy of the GNU General Public License +// along with GCC; see the file COPYING3. If not see +// . + +#include "rust-macro-expand.h" +#include "rust-macro-substitute-ctx.h" +#include "rust-ast-full.h" +#include "rust-ast-visitor.h" +#include "rust-diagnostics.h" +#include "rust-parse.h" +#include "rust-attribute-visitor.h" + +namespace Rust { +AST::ASTFragment +MacroExpander::expand_decl_macro (Location invoc_locus, + AST::MacroInvocData &invoc, + AST::MacroRulesDefinition &rules_def, + bool semicolon) +{ + // ensure that both invocation and rules are in a valid state + rust_assert (!invoc.is_marked_for_strip ()); + rust_assert (!rules_def.is_marked_for_strip ()); + rust_assert (rules_def.get_macro_rules ().size () > 0); + + /* probably something here about parsing invoc and rules def token trees to + * token stream. if not, how would parser handle the captures of exprs and + * stuff? on the other hand, token trees may be kind of useful in rules def as + * creating a point where recursion can occur (like having + * "compare_macro_match" and then it calling itself when it finds delimiters) + */ + + /* find matching rule to invoc token tree, based on macro rule's matcher. if + * none exist, error. + * - specifically, check each matcher in order. if one fails to match, move + * onto next. */ + /* TODO: does doing this require parsing expressions and whatever in the + * invoc? if so, might as well save the results if referenced using $ or + * whatever. If not, do another pass saving them. Except this is probably + * useless as different rules could have different starting points for exprs + * or whatever. Decision trees could avoid this, but they have their own + * issues. */ + /* TODO: will need to modify the parser so that it can essentially "catch" + * errors - maybe "try_parse_expr" or whatever methods. */ + // this technically creates a back-tracking parser - this will be the + // implementation style + + /* then, after results are saved, generate the macro output from the + * transcriber token tree. if i understand this correctly, the macro + * invocation gets replaced by the transcriber tokens, except with + * substitutions made (e.g. for $i variables) */ + + /* TODO: it is probably better to modify AST::Token to store a pointer to a + * Lexer::Token (rather than being converted) - i.e. not so much have + * AST::Token as a Token but rather a TokenContainer (as it is another type of + * TokenTree). This will prevent re-conversion of Tokens between each type + * all the time, while still allowing the heterogenous storage of token trees. + */ + + AST::DelimTokenTree &invoc_token_tree = invoc.get_delim_tok_tree (); + + // find matching arm + AST::MacroRule *matched_rule = nullptr; + std::map matched_fragments; + for (auto &rule : rules_def.get_rules ()) + { + sub_stack.push (); + bool did_match_rule = try_match_rule (rule, invoc_token_tree); + matched_fragments = sub_stack.pop (); + + if (did_match_rule) + { + // // Debugging + // for (auto &kv : matched_fragments) + // rust_debug ("[fragment]: %s (%ld - %s)", kv.first.c_str (), + // kv.second.get_fragments ().size (), + // kv.second.get_kind () + // == MatchedFragmentContainer::Kind::Repetition + // ? "repetition" + // : "metavar"); + + matched_rule = &rule; + break; + } + } + + if (matched_rule == nullptr) + { + RichLocation r (invoc_locus); + r.add_range (rules_def.get_locus ()); + rust_error_at (r, "Failed to match any rule within macro"); + return AST::ASTFragment::create_error (); + } + + return transcribe_rule (*matched_rule, invoc_token_tree, matched_fragments, + semicolon, peek_context ()); +} + +void +MacroExpander::expand_invoc (AST::MacroInvocation &invoc, bool has_semicolon) +{ + if (depth_exceeds_recursion_limit ()) + { + rust_error_at (invoc.get_locus (), "reached recursion limit"); + return; + } + + AST::MacroInvocData &invoc_data = invoc.get_invoc_data (); + + // ?? + // switch on type of macro: + // - '!' syntax macro (inner switch) + // - procedural macro - "A token-based function-like macro" + // - 'macro_rules' (by example/pattern-match) macro? or not? "an + // AST-based function-like macro" + // - else is unreachable + // - attribute syntax macro (inner switch) + // - procedural macro attribute syntax - "A token-based attribute + // macro" + // - legacy macro attribute syntax? - "an AST-based attribute macro" + // - non-macro attribute: mark known + // - else is unreachable + // - derive macro (inner switch) + // - derive or legacy derive - "token-based" vs "AST-based" + // - else is unreachable + // - derive container macro - unreachable + + // lookup the rules for this macro + NodeId resolved_node = UNKNOWN_NODEID; + NodeId source_node = UNKNOWN_NODEID; + if (has_semicolon) + source_node = invoc.get_macro_node_id (); + else + source_node = invoc.get_pattern_node_id (); + auto seg + = Resolver::CanonicalPath::new_seg (source_node, + invoc_data.get_path ().as_string ()); + + bool found = resolver->get_macro_scope ().lookup (seg, &resolved_node); + if (!found) + { + rust_error_at (invoc.get_locus (), "unknown macro: [%s]", + seg.get ().c_str ()); + return; + } + + // lookup the rules + AST::MacroRulesDefinition *rules_def = nullptr; + bool ok = mappings->lookup_macro_def (resolved_node, &rules_def); + rust_assert (ok); + + auto fragment = AST::ASTFragment::create_error (); + + if (rules_def->is_builtin ()) + fragment + = rules_def->get_builtin_transcriber () (invoc.get_locus (), invoc_data); + else + fragment = expand_decl_macro (invoc.get_locus (), invoc_data, *rules_def, + has_semicolon); + + set_expanded_fragment (std::move (fragment)); +} + +/* Determines whether any cfg predicate is false and hence item with attributes + * should be stripped. Note that attributes must be expanded before calling. */ +bool +MacroExpander::fails_cfg (const AST::AttrVec &attrs) const +{ + for (const auto &attr : attrs) + { + if (attr.get_path () == "cfg" && !attr.check_cfg_predicate (session)) + return true; + } + return false; +} + +/* Determines whether any cfg predicate is false and hence item with attributes + * should be stripped. Will expand attributes as well. */ +bool +MacroExpander::fails_cfg_with_expand (AST::AttrVec &attrs) const +{ + // TODO: maybe have something that strips cfg attributes that evaluate true? + for (auto &attr : attrs) + { + if (attr.get_path () == "cfg") + { + if (!attr.is_parsed_to_meta_item ()) + attr.parse_attr_to_meta_item (); + + // DEBUG + if (!attr.is_parsed_to_meta_item ()) + rust_debug ("failed to parse attr to meta item, right before " + "cfg predicate check"); + else + rust_debug ("attr has been successfully parsed to meta item, " + "right before cfg predicate check"); + + if (!attr.check_cfg_predicate (session)) + { + // DEBUG + rust_debug ( + "cfg predicate failed for attribute: \033[0;31m'%s'\033[0m", + attr.as_string ().c_str ()); + + return true; + } + else + { + // DEBUG + rust_debug ("cfg predicate succeeded for attribute: " + "\033[0;31m'%s'\033[0m", + attr.as_string ().c_str ()); + } + } + } + return false; +} + +// Expands cfg_attr attributes. +void +MacroExpander::expand_cfg_attrs (AST::AttrVec &attrs) +{ + for (std::size_t i = 0; i < attrs.size (); i++) + { + auto &attr = attrs[i]; + if (attr.get_path () == "cfg_attr") + { + if (!attr.is_parsed_to_meta_item ()) + attr.parse_attr_to_meta_item (); + + if (attr.check_cfg_predicate (session)) + { + // split off cfg_attr + AST::AttrVec new_attrs = attr.separate_cfg_attrs (); + + // remove attr from vector + attrs.erase (attrs.begin () + i); + + // add new attrs to vector + attrs.insert (attrs.begin () + i, + std::make_move_iterator (new_attrs.begin ()), + std::make_move_iterator (new_attrs.end ())); + } + + /* do something - if feature (first token in tree) is in fact enabled, + * make tokens listed afterwards into attributes. i.e.: for + * [cfg_attr(feature = "wow", wow1, wow2)], if "wow" is true, then add + * attributes [wow1] and [wow2] to attribute list. This can also be + * recursive, so check for expanded attributes being recursive and + * possibly recursively call the expand_attrs? */ + } + else + { + i++; + } + } + attrs.shrink_to_fit (); +} + +void +MacroExpander::expand_crate () +{ + NodeId scope_node_id = crate.get_node_id (); + resolver->get_macro_scope ().push (scope_node_id); + + /* fill macro/decorator map from init list? not sure where init list comes + * from? */ + + // TODO: does cfg apply for inner attributes? research. + // the apparent answer (from playground test) is yes + + // expand crate cfg_attr attributes + expand_cfg_attrs (crate.inner_attrs); + + if (fails_cfg_with_expand (crate.inner_attrs)) + { + // basically, delete whole crate + crate.strip_crate (); + // TODO: maybe create warning here? probably not desired behaviour + } + // expand module attributes? + + push_context (ITEM); + + // expand attributes recursively and strip items if required + AttrVisitor attr_visitor (*this); + auto &items = crate.items; + for (auto it = items.begin (); it != items.end ();) + { + auto &item = *it; + + // mark for stripping if required + item->accept_vis (attr_visitor); + + auto fragment = take_expanded_fragment (attr_visitor); + if (fragment.should_expand ()) + { + // Remove the current expanded invocation + it = items.erase (it); + for (auto &node : fragment.get_nodes ()) + { + it = items.insert (it, node.take_item ()); + it++; + } + } + else if (item->is_marked_for_strip ()) + it = items.erase (it); + else + it++; + } + + pop_context (); + + // TODO: should recursive attribute and macro expansion be done in the same + // transversal? Or in separate ones like currently? + + // expand module tree recursively + + // post-process + + // extract exported macros? +} + +bool +MacroExpander::depth_exceeds_recursion_limit () const +{ + return expansion_depth >= cfg.recursion_limit; +} + +bool +MacroExpander::try_match_rule (AST::MacroRule &match_rule, + AST::DelimTokenTree &invoc_token_tree) +{ + MacroInvocLexer lex (invoc_token_tree.to_token_stream ()); + Parser parser (lex); + + AST::MacroMatcher &matcher = match_rule.get_matcher (); + + expansion_depth++; + if (!match_matcher (parser, matcher)) + { + expansion_depth--; + return false; + } + expansion_depth--; + + bool used_all_input_tokens = parser.skip_token (END_OF_FILE); + return used_all_input_tokens; +} + +bool +MacroExpander::match_fragment (Parser &parser, + AST::MacroMatchFragment &fragment) +{ + switch (fragment.get_frag_spec ().get_kind ()) + { + case AST::MacroFragSpec::EXPR: + parser.parse_expr (); + break; + + case AST::MacroFragSpec::BLOCK: + parser.parse_block_expr (); + break; + + case AST::MacroFragSpec::IDENT: + parser.parse_identifier_pattern (); + break; + + case AST::MacroFragSpec::LITERAL: + parser.parse_literal_expr (); + break; + + case AST::MacroFragSpec::ITEM: + parser.parse_item (false); + break; + + case AST::MacroFragSpec::TY: + parser.parse_type (); + break; + + case AST::MacroFragSpec::PAT: + parser.parse_pattern (); + break; + + case AST::MacroFragSpec::PATH: + parser.parse_path_in_expression (); + break; + + case AST::MacroFragSpec::VIS: + parser.parse_visibility (); + break; + + case AST::MacroFragSpec::STMT: { + auto restrictions = ParseRestrictions (); + restrictions.consume_semi = false; + parser.parse_stmt (restrictions); + break; + } + + case AST::MacroFragSpec::LIFETIME: + parser.parse_lifetime_params (); + break; + + // is meta attributes? + case AST::MacroFragSpec::META: + parser.parse_attribute_body (); + break; + + case AST::MacroFragSpec::TT: + parser.parse_token_tree (); + break; + + // i guess we just ignore invalid and just error out + case AST::MacroFragSpec::INVALID: + return false; + } + + // it matches if the parser did not produce errors trying to parse that type + // of item + return !parser.has_errors (); +} + +bool +MacroExpander::match_matcher (Parser &parser, + AST::MacroMatcher &matcher) +{ + if (depth_exceeds_recursion_limit ()) + { + rust_error_at (matcher.get_match_locus (), "reached recursion limit"); + return false; + } + + auto delimiter = parser.peek_current_token (); + + // this is used so we can check that we delimit the stream correctly. + switch (delimiter->get_id ()) + { + case LEFT_PAREN: { + if (!parser.skip_token (LEFT_PAREN)) + return false; + } + break; + + case LEFT_SQUARE: { + if (!parser.skip_token (LEFT_SQUARE)) + return false; + } + break; + + case LEFT_CURLY: { + if (!parser.skip_token (LEFT_CURLY)) + return false; + } + break; + default: + gcc_unreachable (); + } + + const MacroInvocLexer &source = parser.get_token_source (); + + for (auto &match : matcher.get_matches ()) + { + size_t offs_begin = source.get_offs (); + + switch (match->get_macro_match_type ()) + { + case AST::MacroMatch::MacroMatchType::Fragment: { + AST::MacroMatchFragment *fragment + = static_cast (match.get ()); + if (!match_fragment (parser, *fragment)) + return false; + + // matched fragment get the offset in the token stream + size_t offs_end = source.get_offs (); + sub_stack.insert_metavar ( + MatchedFragment (fragment->get_ident (), offs_begin, offs_end)); + } + break; + + case AST::MacroMatch::MacroMatchType::Tok: { + AST::Token *tok = static_cast (match.get ()); + if (!match_token (parser, *tok)) + return false; + } + break; + + case AST::MacroMatch::MacroMatchType::Repetition: { + AST::MacroMatchRepetition *rep + = static_cast (match.get ()); + if (!match_repetition (parser, *rep)) + return false; + } + break; + + case AST::MacroMatch::MacroMatchType::Matcher: { + AST::MacroMatcher *m + = static_cast (match.get ()); + expansion_depth++; + if (!match_matcher (parser, *m)) + { + expansion_depth--; + return false; + } + expansion_depth--; + } + break; + } + } + + switch (delimiter->get_id ()) + { + case LEFT_PAREN: { + if (!parser.skip_token (RIGHT_PAREN)) + return false; + } + break; + + case LEFT_SQUARE: { + if (!parser.skip_token (RIGHT_SQUARE)) + return false; + } + break; + + case LEFT_CURLY: { + if (!parser.skip_token (RIGHT_CURLY)) + return false; + } + break; + default: + gcc_unreachable (); + } + + return true; +} + +bool +MacroExpander::match_token (Parser &parser, AST::Token &token) +{ + // FIXME this needs to actually match the content and the type + return parser.skip_token (token.get_id ()); +} + +bool +MacroExpander::match_n_matches (Parser &parser, + AST::MacroMatchRepetition &rep, + size_t &match_amount, size_t lo_bound, + size_t hi_bound) +{ + match_amount = 0; + auto &matches = rep.get_matches (); + + const MacroInvocLexer &source = parser.get_token_source (); + while (true) + { + // If the current token is a closing macro delimiter, break away. + // TODO: Is this correct? + auto t_id = parser.peek_current_token ()->get_id (); + if (t_id == RIGHT_PAREN || t_id == RIGHT_SQUARE || t_id == RIGHT_CURLY) + break; + + // Skip parsing a separator on the first match, otherwise consume it. + // If it isn't present, this is an error + if (rep.has_sep () && match_amount > 0) + if (!match_token (parser, *rep.get_sep ())) + break; + + bool valid_current_match = false; + for (auto &match : matches) + { + size_t offs_begin = source.get_offs (); + switch (match->get_macro_match_type ()) + { + case AST::MacroMatch::MacroMatchType::Fragment: { + AST::MacroMatchFragment *fragment + = static_cast (match.get ()); + valid_current_match = match_fragment (parser, *fragment); + + // matched fragment get the offset in the token stream + size_t offs_end = source.get_offs (); + + // The main difference with match_matcher happens here: Instead + // of inserting a new fragment, we append to one. If that + // fragment does not exist, then the operation is similar to + // `insert_fragment` with the difference that we are not + // creating a metavariable, but a repetition of one, which is + // really different. + sub_stack.append_fragment ( + MatchedFragment (fragment->get_ident (), offs_begin, + offs_end)); + } + break; + + case AST::MacroMatch::MacroMatchType::Tok: { + AST::Token *tok = static_cast (match.get ()); + valid_current_match = match_token (parser, *tok); + } + break; + + case AST::MacroMatch::MacroMatchType::Repetition: { + AST::MacroMatchRepetition *rep + = static_cast (match.get ()); + valid_current_match = match_repetition (parser, *rep); + } + break; + + case AST::MacroMatch::MacroMatchType::Matcher: { + AST::MacroMatcher *m + = static_cast (match.get ()); + valid_current_match = match_matcher (parser, *m); + } + break; + } + } + // If we've encountered an error once, stop trying to match more + // repetitions + if (!valid_current_match) + break; + + match_amount++; + + // Break early if we notice there's too many expressions already + if (hi_bound && match_amount > hi_bound) + break; + } + + // Check if the amount of matches we got is valid: Is it more than the lower + // bound and less than the higher bound? + bool did_meet_lo_bound = match_amount >= lo_bound; + bool did_meet_hi_bound = hi_bound ? match_amount <= hi_bound : true; + + // If the end-result is valid, then we can clear the parse errors: Since + // repetitions are parsed eagerly, it is okay to fail in some cases + auto res = did_meet_lo_bound && did_meet_hi_bound; + if (res) + parser.clear_errors (); + + return res; +} + +bool +MacroExpander::match_repetition (Parser &parser, + AST::MacroMatchRepetition &rep) +{ + size_t match_amount = 0; + bool res = false; + + std::string lo_str; + std::string hi_str; + switch (rep.get_op ()) + { + case AST::MacroMatchRepetition::MacroRepOp::ANY: + lo_str = "0"; + hi_str = "+inf"; + res = match_n_matches (parser, rep, match_amount); + break; + case AST::MacroMatchRepetition::MacroRepOp::ONE_OR_MORE: + lo_str = "1"; + hi_str = "+inf"; + res = match_n_matches (parser, rep, match_amount, 1); + break; + case AST::MacroMatchRepetition::MacroRepOp::ZERO_OR_ONE: + lo_str = "0"; + hi_str = "1"; + res = match_n_matches (parser, rep, match_amount, 0, 1); + break; + default: + gcc_unreachable (); + } + + if (!res) + rust_error_at (rep.get_match_locus (), + "invalid amount of matches for macro invocation. Expected " + "between %s and %s, got %lu", + lo_str.c_str (), hi_str.c_str (), + (unsigned long) match_amount); + + rust_debug_loc (rep.get_match_locus (), "%s matched %lu times", + res ? "successfully" : "unsuccessfully", + (unsigned long) match_amount); + + // We have to handle zero fragments differently: They will not have been + // "matched" but they are still valid and should be inserted as a special + // case. So we go through the stack map, and for every fragment which doesn't + // exist, insert a zero-matched fragment. + auto &stack_map = sub_stack.peek (); + for (auto &match : rep.get_matches ()) + { + if (match->get_macro_match_type () + == AST::MacroMatch::MacroMatchType::Fragment) + { + auto fragment = static_cast (match.get ()); + auto it = stack_map.find (fragment->get_ident ()); + + if (it == stack_map.end ()) + sub_stack.insert_matches (fragment->get_ident (), + MatchedFragmentContainer::zero ()); + } + } + + return res; +} + +/** + * Helper function to refactor calling a parsing function 0 or more times + */ +static AST::ASTFragment +parse_many (Parser &parser, TokenId &delimiter, + std::function parse_fn) +{ + std::vector nodes; + while (true) + { + if (parser.peek_current_token ()->get_id () == delimiter) + break; + + auto node = parse_fn (); + nodes.emplace_back (std::move (node)); + } + + return AST::ASTFragment (std::move (nodes)); +} + +/** + * Transcribe 0 or more items from a macro invocation + * + * @param parser Parser to extract items from + * @param delimiter Id of the token on which parsing should stop + */ +static AST::ASTFragment +transcribe_many_items (Parser &parser, TokenId &delimiter) +{ + return parse_many (parser, delimiter, [&parser] () { + auto item = parser.parse_item (true); + return AST::SingleASTNode (std::move (item)); + }); +} + +/** + * Transcribe 0 or more external items from a macro invocation + * + * @param parser Parser to extract items from + * @param delimiter Id of the token on which parsing should stop + */ +static AST::ASTFragment +transcribe_many_ext (Parser &parser, TokenId &delimiter) +{ + return parse_many (parser, delimiter, [&parser] () { + auto item = parser.parse_external_item (); + return AST::SingleASTNode (std::move (item)); + }); +} + +/** + * Transcribe 0 or more trait items from a macro invocation + * + * @param parser Parser to extract items from + * @param delimiter Id of the token on which parsing should stop + */ +static AST::ASTFragment +transcribe_many_trait_items (Parser &parser, + TokenId &delimiter) +{ + return parse_many (parser, delimiter, [&parser] () { + auto item = parser.parse_trait_item (); + return AST::SingleASTNode (std::move (item)); + }); +} + +/** + * Transcribe 0 or more impl items from a macro invocation + * + * @param parser Parser to extract items from + * @param delimiter Id of the token on which parsing should stop + */ +static AST::ASTFragment +transcribe_many_impl_items (Parser &parser, TokenId &delimiter) +{ + return parse_many (parser, delimiter, [&parser] () { + auto item = parser.parse_inherent_impl_item (); + return AST::SingleASTNode (std::move (item)); + }); +} + +/** + * Transcribe 0 or more trait impl items from a macro invocation + * + * @param parser Parser to extract items from + * @param delimiter Id of the token on which parsing should stop + */ +static AST::ASTFragment +transcribe_many_trait_impl_items (Parser &parser, + TokenId &delimiter) +{ + return parse_many (parser, delimiter, [&parser] () { + auto item = parser.parse_trait_impl_item (); + return AST::SingleASTNode (std::move (item)); + }); +} + +/** + * Transcribe 0 or more statements from a macro invocation + * + * @param parser Parser to extract statements from + * @param delimiter Id of the token on which parsing should stop + */ +static AST::ASTFragment +transcribe_many_stmts (Parser &parser, TokenId &delimiter) +{ + auto restrictions = ParseRestrictions (); + restrictions.consume_semi = false; + + // FIXME: This is invalid! It needs to also handle cases where the macro + // transcriber is an expression, but since the macro call is followed by + // a semicolon, it's a valid ExprStmt + return parse_many (parser, delimiter, [&parser, restrictions] () { + auto stmt = parser.parse_stmt (restrictions); + return AST::SingleASTNode (std::move (stmt)); + }); +} + +/** + * Transcribe one expression from a macro invocation + * + * @param parser Parser to extract statements from + */ +static AST::ASTFragment +transcribe_expression (Parser &parser) +{ + auto expr = parser.parse_expr (); + + return AST::ASTFragment ({std::move (expr)}); +} + +/** + * Transcribe one type from a macro invocation + * + * @param parser Parser to extract statements from + */ +static AST::ASTFragment +transcribe_type (Parser &parser) +{ + auto type = parser.parse_type (); + + return AST::ASTFragment ({std::move (type)}); +} + +static AST::ASTFragment +transcribe_on_delimiter (Parser &parser, bool semicolon, + AST::DelimType delimiter, TokenId last_token_id) +{ + if (semicolon || delimiter == AST::DelimType::CURLY) + return transcribe_many_stmts (parser, last_token_id); + else + return transcribe_expression (parser); +} // namespace Rust + +static AST::ASTFragment +transcribe_context (MacroExpander::ContextType ctx, + Parser &parser, bool semicolon, + AST::DelimType delimiter, TokenId last_token_id) +{ + // The flow-chart in order to choose a parsing function is as follows: + // + // [switch special context] + // -- Item --> parser.parse_item(); + // -- Trait --> parser.parse_trait_item(); + // -- Impl --> parser.parse_impl_item(); + // -- Extern --> parser.parse_extern_item(); + // -- None --> [has semicolon?] + // -- Yes --> parser.parse_stmt(); + // -- No --> [switch invocation.delimiter()] + // -- { } --> parser.parse_stmt(); + // -- _ --> parser.parse_expr(); // once! + + // If there is a semicolon OR we are expanding a MacroInvocationSemi, then + // we can parse multiple items. Otherwise, parse *one* expression + + switch (ctx) + { + case MacroExpander::ContextType::ITEM: + return transcribe_many_items (parser, last_token_id); + break; + case MacroExpander::ContextType::TRAIT: + return transcribe_many_trait_items (parser, last_token_id); + break; + case MacroExpander::ContextType::IMPL: + return transcribe_many_impl_items (parser, last_token_id); + break; + case MacroExpander::ContextType::TRAIT_IMPL: + return transcribe_many_trait_impl_items (parser, last_token_id); + break; + case MacroExpander::ContextType::EXTERN: + return transcribe_many_ext (parser, last_token_id); + break; + case MacroExpander::ContextType::TYPE: + return transcribe_type (parser); + break; + default: + return transcribe_on_delimiter (parser, semicolon, delimiter, + last_token_id); + } +} + +static std::string +tokens_to_str (std::vector> &tokens) +{ + std::string str; + if (!tokens.empty ()) + { + str += tokens[0]->as_string (); + for (size_t i = 1; i < tokens.size (); i++) + str += " " + tokens[i]->as_string (); + } + + return str; +} + +AST::ASTFragment +MacroExpander::transcribe_rule ( + AST::MacroRule &match_rule, AST::DelimTokenTree &invoc_token_tree, + std::map &matched_fragments, + bool semicolon, ContextType ctx) +{ + // we can manipulate the token tree to substitute the dollar identifiers so + // that when we call parse its already substituted for us + AST::MacroTranscriber &transcriber = match_rule.get_transcriber (); + AST::DelimTokenTree &transcribe_tree = transcriber.get_token_tree (); + + auto invoc_stream = invoc_token_tree.to_token_stream (); + auto macro_rule_tokens = transcribe_tree.to_token_stream (); + + auto substitute_context + = SubstituteCtx (invoc_stream, macro_rule_tokens, matched_fragments); + std::vector> substituted_tokens + = substitute_context.substitute_tokens (); + + rust_debug ("substituted tokens: %s", + tokens_to_str (substituted_tokens).c_str ()); + + // parse it to an ASTFragment + MacroInvocLexer lex (std::move (substituted_tokens)); + Parser parser (lex); + + auto last_token_id = TokenId::RIGHT_CURLY; + + // this is used so we can check that we delimit the stream correctly. + switch (transcribe_tree.get_delim_type ()) + { + case AST::DelimType::PARENS: + last_token_id = TokenId::RIGHT_PAREN; + rust_assert (parser.skip_token (LEFT_PAREN)); + break; + + case AST::DelimType::CURLY: + rust_assert (parser.skip_token (LEFT_CURLY)); + break; + + case AST::DelimType::SQUARE: + last_token_id = TokenId::RIGHT_SQUARE; + rust_assert (parser.skip_token (LEFT_SQUARE)); + break; + } + + // see https://github.com/Rust-GCC/gccrs/issues/22 + // TL;DR: + // - Treat all macro invocations with parentheses, (), or square brackets, + // [], as expressions. + // - If the macro invocation has curly brackets, {}, it may be parsed as a + // statement depending on the context. + // - If the macro invocation has a semicolon at the end, it must be parsed + // as a statement (either via ExpressionStatement or + // MacroInvocationWithSemi) + + auto fragment + = transcribe_context (ctx, parser, semicolon, + invoc_token_tree.get_delim_type (), last_token_id); + + // emit any errors + if (parser.has_errors ()) + { + for (auto &err : parser.get_errors ()) + rust_error_at (err.locus, "%s", err.message.c_str ()); + return AST::ASTFragment::create_error (); + } + + // are all the tokens used? + bool did_delimit = parser.skip_token (last_token_id); + + bool reached_end_of_stream = did_delimit && parser.skip_token (END_OF_FILE); + if (!reached_end_of_stream) + { + const_TokenPtr current_token = parser.peek_current_token (); + rust_error_at (current_token->get_locus (), + "tokens here and after are unparsed"); + } + + return fragment; +} +} // namespace Rust diff --git a/gcc/rust/expand/rust-macro-expand.h b/gcc/rust/expand/rust-macro-expand.h new file mode 100644 index 00000000000..94d6702ecb8 --- /dev/null +++ b/gcc/rust/expand/rust-macro-expand.h @@ -0,0 +1,366 @@ +// Copyright (C) 2020-2022 Free Software Foundation, Inc. + +// This file is part of GCC. + +// GCC is free software; you can redistribute it and/or modify it under +// the terms of the GNU General Public License as published by the Free +// Software Foundation; either version 3, or (at your option) any later +// version. + +// GCC is distributed in the hope that it will be useful, but WITHOUT ANY +// WARRANTY; without even the implied warranty of MERCHANTABILITY or +// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License +// for more details. + +// You should have received a copy of the GNU General Public License +// along with GCC; see the file COPYING3. If not see +// . + +#ifndef RUST_MACRO_EXPAND_H +#define RUST_MACRO_EXPAND_H + +#include "rust-buffered-queue.h" +#include "rust-parse.h" +#include "rust-token.h" +#include "rust-ast.h" +#include "rust-macro.h" +#include "rust-hir-map.h" +#include "rust-name-resolver.h" +#include "rust-macro-invoc-lexer.h" + +// Provides objects and method prototypes for macro expansion + +namespace Rust { +// forward decls for AST +namespace AST { +class MacroInvocation; +} + +// Object used to store configuration data for macro expansion. +// NOTE: Keep all these items complying with the latest rustc. +struct ExpansionCfg +{ + // features? + // TODO: Add `features' when we have it. + unsigned int recursion_limit = 1024; + bool trace_mac = false; // trace macro + bool should_test = false; // strip #[test] nodes if false + bool keep_macs = false; // keep macro definitions + std::string crate_name = ""; +}; + +struct MatchedFragment +{ + std::string fragment_ident; + size_t token_offset_begin; + size_t token_offset_end; + + MatchedFragment (std::string identifier, size_t token_offset_begin, + size_t token_offset_end) + : fragment_ident (identifier), token_offset_begin (token_offset_begin), + token_offset_end (token_offset_end) + {} + + /** + * Empty constructor for uninitialized fragments + */ + MatchedFragment () : MatchedFragment ("", 0, 0) {} + + std::string as_string () const + { + return fragment_ident + "=" + std::to_string (token_offset_begin) + ":" + + std::to_string (token_offset_end); + } +}; + +class MatchedFragmentContainer +{ +public: + // Does the container refer to a simple metavariable, different from a + // repetition repeated once + enum class Kind + { + MetaVar, + Repetition, + }; + + MatchedFragmentContainer (std::vector fragments, + Kind kind = Kind::Repetition) + : fragments (fragments), kind (kind) + {} + + /** + * Create a valid fragment matched zero times. This is useful for repetitions + * which allow the absence of a fragment, such as * and ? + */ + static MatchedFragmentContainer zero () + { + return MatchedFragmentContainer ({}); + } + + /** + * Create a valid fragment matched one time + */ + static MatchedFragmentContainer metavar (MatchedFragment fragment) + { + return MatchedFragmentContainer ({fragment}, Kind::MetaVar); + } + + /** + * Add a matched fragment to the container + */ + void add_fragment (MatchedFragment fragment) + { + rust_assert (!is_single_fragment ()); + + fragments.emplace_back (fragment); + } + + size_t get_match_amount () const { return fragments.size (); } + const std::vector &get_fragments () const + { + return fragments; + } + // const std::string &get_fragment_name () const { return fragment_name; } + + bool is_single_fragment () const + { + return get_match_amount () == 1 && kind == Kind::MetaVar; + } + + const MatchedFragment get_single_fragment () const + { + rust_assert (is_single_fragment ()); + + return fragments[0]; + } + + const Kind &get_kind () const { return kind; } + +private: + /** + * Fragments matched `match_amount` times. This can be an empty vector + * in case having zero matches is allowed (i.e ? or * operators) + */ + std::vector fragments; + Kind kind; +}; + +class SubstitutionScope +{ +public: + SubstitutionScope () : stack () {} + + void push () { stack.push_back ({}); } + + std::map pop () + { + auto top = stack.back (); + stack.pop_back (); + return top; + } + + std::map &peek () + { + return stack.back (); + } + + /** + * Insert a new matched metavar into the current substitution map + */ + void insert_metavar (MatchedFragment fragment) + { + auto ¤t_map = stack.back (); + auto it = current_map.find (fragment.fragment_ident); + + if (it == current_map.end ()) + current_map.insert ({fragment.fragment_ident, + MatchedFragmentContainer::metavar (fragment)}); + else + gcc_unreachable (); + } + + /** + * Append a new matched fragment to a repetition into the current substitution + * map + */ + void append_fragment (MatchedFragment fragment) + { + auto ¤t_map = stack.back (); + auto it = current_map.find (fragment.fragment_ident); + + if (it == current_map.end ()) + current_map.insert ( + {fragment.fragment_ident, MatchedFragmentContainer ({fragment})}); + else + it->second.add_fragment (fragment); + } + + void insert_matches (std::string key, MatchedFragmentContainer matches) + { + auto ¤t_map = stack.back (); + auto it = current_map.find (key); + rust_assert (it == current_map.end ()); + + current_map.insert ({key, matches}); + } + +private: + std::vector> stack; +}; + +// Object used to store shared data (between functions) for macro expansion. +struct MacroExpander +{ + enum ContextType + { + ITEM, + BLOCK, + EXTERN, + TYPE, + TRAIT, + IMPL, + TRAIT_IMPL, + }; + + ExpansionCfg cfg; + unsigned int expansion_depth = 0; + + MacroExpander (AST::Crate &crate, ExpansionCfg cfg, Session &session) + : cfg (cfg), crate (crate), session (session), + sub_stack (SubstitutionScope ()), + expanded_fragment (AST::ASTFragment::create_error ()), + resolver (Resolver::Resolver::get ()), + mappings (Analysis::Mappings::get ()) + {} + + ~MacroExpander () = default; + + // Expands all macros in the crate passed in. + void expand_crate (); + + /* Expands a macro invocation - possibly make both + * have similar duck-typed interface and use templates?*/ + // should this be public or private? + void expand_invoc (AST::MacroInvocation &invoc, bool has_semicolon); + + // Expands a single declarative macro. + AST::ASTFragment expand_decl_macro (Location locus, + AST::MacroInvocData &invoc, + AST::MacroRulesDefinition &rules_def, + bool semicolon); + + void expand_cfg_attrs (AST::AttrVec &attrs); + bool fails_cfg (const AST::AttrVec &attr) const; + bool fails_cfg_with_expand (AST::AttrVec &attrs) const; + + bool depth_exceeds_recursion_limit () const; + + bool try_match_rule (AST::MacroRule &match_rule, + AST::DelimTokenTree &invoc_token_tree); + + AST::ASTFragment transcribe_rule ( + AST::MacroRule &match_rule, AST::DelimTokenTree &invoc_token_tree, + std::map &matched_fragments, + bool semicolon, ContextType ctx); + + bool match_fragment (Parser &parser, + AST::MacroMatchFragment &fragment); + + bool match_token (Parser &parser, AST::Token &token); + + bool match_repetition (Parser &parser, + AST::MacroMatchRepetition &rep); + + bool match_matcher (Parser &parser, + AST::MacroMatcher &matcher); + + /** + * Match any amount of matches + * + * @param parser Parser to use for matching + * @param rep Repetition to try and match + * @param match_amount Reference in which to store the ammount of succesful + * and valid matches + * + * @param lo_bound Lower bound of the matcher. When specified, the matcher + * will only succeed if it parses at *least* `lo_bound` fragments. If + * unspecified, the matcher could succeed when parsing 0 fragments. + * + * @param hi_bound Higher bound of the matcher. When specified, the matcher + * will only succeed if it parses *less than* `hi_bound` fragments. If + * unspecified, the matcher could succeed when parsing an infinity of + * fragments. + * + * @return true if matching was successful and within the given limits, false + * otherwise + */ + bool match_n_matches (Parser &parser, + AST::MacroMatchRepetition &rep, size_t &match_amount, + size_t lo_bound = 0, size_t hi_bound = 0); + + void push_context (ContextType t) { context.push_back (t); } + + ContextType pop_context () + { + rust_assert (!context.empty ()); + + ContextType t = context.back (); + context.pop_back (); + + return t; + } + + ContextType peek_context () { return context.back (); } + + void set_expanded_fragment (AST::ASTFragment &&fragment) + { + expanded_fragment = std::move (fragment); + } + + AST::ASTFragment take_expanded_fragment (AST::ASTVisitor &vis) + { + AST::ASTFragment old_fragment = std::move (expanded_fragment); + auto accumulator = std::vector (); + expanded_fragment = AST::ASTFragment::create_error (); + + for (auto &node : old_fragment.get_nodes ()) + { + expansion_depth++; + node.accept_vis (vis); + // we'll decide the next move according to the outcome of the macro + // expansion + if (expanded_fragment.is_error ()) + accumulator.push_back (node); // if expansion fails, there might be a + // non-macro expression we need to keep + else + { + // if expansion succeeded, then we need to merge the fragment with + // the contents in the accumulator, so that our final expansion + // result will contain non-macro nodes as it should + auto new_nodes = expanded_fragment.get_nodes (); + std::move (new_nodes.begin (), new_nodes.end (), + std::back_inserter (accumulator)); + expanded_fragment = AST::ASTFragment (accumulator); + } + expansion_depth--; + } + + return old_fragment; + } + +private: + AST::Crate &crate; + Session &session; + SubstitutionScope sub_stack; + std::vector context; + AST::ASTFragment expanded_fragment; + +public: + Resolver::Resolver *resolver; + Analysis::Mappings *mappings; +}; + +} // namespace Rust + +#endif diff --git a/gcc/rust/expand/rust-macro-invoc-lexer.cc b/gcc/rust/expand/rust-macro-invoc-lexer.cc new file mode 100644 index 00000000000..8a43d29e0d1 --- /dev/null +++ b/gcc/rust/expand/rust-macro-invoc-lexer.cc @@ -0,0 +1,29 @@ +#include "rust-macro-invoc-lexer.h" + +namespace Rust { + +const_TokenPtr +MacroInvocLexer::peek_token (int n) +{ + if ((offs + n) >= token_stream.size ()) + return Token::make (END_OF_FILE, Location ()); + + return token_stream.at (offs + n)->get_tok_ptr (); +} + +// Advances current token to n + 1 tokens ahead of current position. +void +MacroInvocLexer::skip_token (int n) +{ + offs += (n + 1); +} + +void +MacroInvocLexer::split_current_token (TokenId new_left __attribute__ ((unused)), + TokenId new_right + __attribute__ ((unused))) +{ + // FIXME + gcc_unreachable (); +} +} // namespace Rust diff --git a/gcc/rust/expand/rust-macro-invoc-lexer.h b/gcc/rust/expand/rust-macro-invoc-lexer.h new file mode 100644 index 00000000000..0fd4554d02f --- /dev/null +++ b/gcc/rust/expand/rust-macro-invoc-lexer.h @@ -0,0 +1,64 @@ +// Copyright (C) 2020-2022 Free Software Foundation, Inc. + +// This file is part of GCC. + +// GCC is free software; you can redistribute it and/or modify it under +// the terms of the GNU General Public License as published by the Free +// Software Foundation; either version 3, or (at your option) any later +// version. + +// GCC is distributed in the hope that it will be useful, but WITHOUT ANY +// WARRANTY; without even the implied warranty of MERCHANTABILITY or +// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License +// for more details. + +// You should have received a copy of the GNU General Public License +// along with GCC; see the file COPYING3. If not see +// . + +#ifndef RUST_MACRO_INVOC_LEXER_H +#define RUST_MACRO_INVOC_LEXER_H + +#include "rust-ast.h" + +namespace Rust { +class MacroInvocLexer +{ +public: + MacroInvocLexer (std::vector> stream) + : offs (0), token_stream (std::move (stream)) + {} + + // Returns token n tokens ahead of current position. + const_TokenPtr peek_token (int n); + + // Peeks the current token. + const_TokenPtr peek_token () { return peek_token (0); } + + // Advances current token to n + 1 tokens ahead of current position. + void skip_token (int n); + + // Skips the current token. + void skip_token () { skip_token (0); } + + // Splits the current token into two. Intended for use with nested generics + // closes (i.e. T> where >> is wrongly lexed as one token). Note that + // this will only work with "simple" tokens like punctuation. + void split_current_token (TokenId new_left, TokenId new_right); + + std::string get_filename () const + { + // FIXME + gcc_unreachable (); + return "FIXME"; + } + + size_t get_offs () const { return offs; } + +private: + size_t offs; + std::vector> token_stream; +}; +} // namespace Rust + +#endif // RUST_MACRO_INVOC_LEXER_H diff --git a/gcc/rust/expand/rust-macro-substitute-ctx.cc b/gcc/rust/expand/rust-macro-substitute-ctx.cc new file mode 100644 index 00000000000..9592d2d2a9e --- /dev/null +++ b/gcc/rust/expand/rust-macro-substitute-ctx.cc @@ -0,0 +1,312 @@ +#include "rust-macro-substitute-ctx.h" + +namespace Rust { + +std::vector> +SubstituteCtx::substitute_metavar (std::unique_ptr &metavar) +{ + auto metavar_name = metavar->get_str (); + + std::vector> expanded; + auto it = fragments.find (metavar_name); + if (it == fragments.end ()) + { + // Return a copy of the original token + expanded.push_back (metavar->clone_token ()); + } + else + { + // If we are expanding a metavar which has a lof of matches, we are + // currently expanding a repetition metavar - not a simple metavar. We + // need to error out and inform the user. + // Associated test case for an example: compile/macro-issue1224.rs + if (it->second.get_match_amount () != 1) + { + rust_error_at (metavar->get_locus (), + "metavariable is still repeating at this depth"); + rust_inform ( + metavar->get_locus (), + "you probably forgot the repetition operator: %<%s%s%s%>", "$(", + metavar->as_string ().c_str (), ")*"); + return expanded; + } + + // We only care about the vector when expanding repetitions. + // Just access the first element of the vector. + auto &frag = it->second.get_single_fragment (); + for (size_t offs = frag.token_offset_begin; offs < frag.token_offset_end; + offs++) + { + auto &tok = input.at (offs); + expanded.push_back (tok->clone_token ()); + } + } + + return expanded; +} + +bool +SubstituteCtx::check_repetition_amount (size_t pattern_start, + size_t pattern_end, + size_t &expected_repetition_amount) +{ + bool first_fragment_found = false; + bool is_valid = true; + + for (size_t i = pattern_start; i < pattern_end; i++) + { + if (macro.at (i)->get_id () == DOLLAR_SIGN) + { + auto &frag_token = macro.at (i + 1); + if (frag_token->get_id () == IDENTIFIER) + { + auto it = fragments.find (frag_token->get_str ()); + if (it == fragments.end ()) + { + // If the repetition is not anything we know (ie no declared + // metavars, or metavars which aren't present in the + // fragment), we can just error out. No need to paste the + // tokens as if nothing had happened. + rust_error_at (frag_token->get_locus (), + "metavar %s used in repetition does not exist", + frag_token->get_str ().c_str ()); + + is_valid = false; + } + + auto &fragment = it->second; + + size_t repeat_amount = fragment.get_match_amount (); + if (!first_fragment_found) + { + first_fragment_found = true; + expected_repetition_amount = repeat_amount; + } + else + { + if (repeat_amount != expected_repetition_amount + && !fragment.is_single_fragment ()) + { + rust_error_at ( + frag_token->get_locus (), + "different amount of matches used in merged " + "repetitions: expected %lu, got %lu", + (unsigned long) expected_repetition_amount, + (unsigned long) repeat_amount); + is_valid = false; + } + } + } + } + } + + return is_valid; +} + +std::vector> +SubstituteCtx::substitute_repetition ( + size_t pattern_start, size_t pattern_end, + std::unique_ptr separator_token) +{ + rust_assert (pattern_end < macro.size ()); + + size_t repeat_amount = 0; + if (!check_repetition_amount (pattern_start, pattern_end, repeat_amount)) + return {}; + + rust_debug ("repetition amount to use: %lu", (unsigned long) repeat_amount); + std::vector> expanded; + std::vector> new_macro; + + // We want to generate a "new macro" to substitute with. This new macro + // should contain only the tokens inside the pattern + for (size_t tok_idx = pattern_start; tok_idx < pattern_end; tok_idx++) + new_macro.emplace_back (macro.at (tok_idx)->clone_token ()); + + // Then, we want to create a subset of the matches so that + // `substitute_tokens()` can only see one fragment per metavar. Let's say we + // have the following user input: (1 145 'h') + // on the following match arm: ($($lit:literal)*) + // which causes the following matches: { "lit": [1, 145, 'h'] } + // + // The pattern (new_macro) is `$lit:literal` + // The first time we expand it, we want $lit to have the following token: 1 + // The second time, 145 + // The third and final time, 'h' + // + // In order to do so we must create "sub maps", which only contain parts of + // the original matches + // sub-maps: [ { "lit": 1 }, { "lit": 145 }, { "lit": 'h' } ] + // + // and give them to `substitute_tokens` one by one. + + for (size_t i = 0; i < repeat_amount; i++) + { + std::map sub_map; + for (auto &kv_match : fragments) + { + MatchedFragment sub_fragment; + + // FIXME: Hack: If a fragment is not repeated, how does it fit in the + // submap? Do we really want to expand it? Is this normal behavior? + if (kv_match.second.is_single_fragment ()) + sub_fragment = kv_match.second.get_single_fragment (); + else + sub_fragment = kv_match.second.get_fragments ()[i]; + + sub_map.insert ( + {kv_match.first, MatchedFragmentContainer::metavar (sub_fragment)}); + } + + auto substitute_context = SubstituteCtx (input, new_macro, sub_map); + auto new_tokens = substitute_context.substitute_tokens (); + + // Skip the first repetition, but add the separator to the expanded + // tokens if it is present + if (i != 0 && separator_token) + expanded.emplace_back (separator_token->clone_token ()); + + for (auto &new_token : new_tokens) + expanded.emplace_back (new_token->clone_token ()); + } + + // FIXME: We also need to make sure that all subsequent fragments + // contain the same amount of repetitions as the first one + + return expanded; +} + +static bool +is_rep_op (std::unique_ptr &tok) +{ + auto id = tok->get_id (); + return id == QUESTION_MARK || id == ASTERISK || id == PLUS; +} + +std::pair>, size_t> +SubstituteCtx::substitute_token (size_t token_idx) +{ + auto &token = macro.at (token_idx); + switch (token->get_id ()) + { + case IDENTIFIER: + rust_debug ("expanding metavar: %s", token->get_str ().c_str ()); + return {substitute_metavar (token), 1}; + case LEFT_PAREN: { + // We need to parse up until the closing delimiter and expand this + // fragment->n times. + rust_debug ("expanding repetition"); + + // We're in a context where macro repetitions have already been + // parsed and validated: This means that + // 1/ There will be no delimiters as that is an error + // 2/ There are no fragment specifiers anymore, which prevents us + // from reusing parser functions. + // + // Repetition patterns are also special in that they cannot contain + // "rogue" delimiters: For example, this is invalid, as they are + // parsed as MacroMatches and must contain a correct amount of + // delimiters. + // `$($e:expr ) )` + // ^ rogue closing parenthesis + // + // With all of that in mind, we can simply skip ahead from one + // parenthesis to the other to find the pattern to expand. Of course, + // pairs of delimiters, including parentheses, are allowed. + // `$($e:expr ( ) )` + // Parentheses are the sole delimiter for which we need a special + // behavior since they delimit the repetition pattern + + size_t pattern_start = token_idx + 1; + size_t pattern_end = pattern_start; + auto parentheses_stack = 0; + for (size_t idx = pattern_start; idx < macro.size (); idx++) + { + if (macro.at (idx)->get_id () == LEFT_PAREN) + { + parentheses_stack++; + } + else if (macro.at (idx)->get_id () == RIGHT_PAREN) + { + if (parentheses_stack == 0) + { + pattern_end = idx; + break; + } + parentheses_stack--; + } + } + + // Unreachable case, but let's make sure we don't ever run into it + rust_assert (pattern_end != pattern_start); + + std::unique_ptr separator_token = nullptr; + if (pattern_end + 1 <= macro.size ()) + { + auto &post_pattern_token = macro.at (pattern_end + 1); + if (!is_rep_op (post_pattern_token)) + separator_token = post_pattern_token->clone_token (); + } + + // Amount of tokens to skip + auto to_skip = 0; + // Parentheses + to_skip += 2; + // Repetition operator + to_skip += 1; + // Separator + if (separator_token) + to_skip += 1; + + return {substitute_repetition (pattern_start, pattern_end, + std::move (separator_token)), + pattern_end - pattern_start + to_skip}; + } + // TODO: We need to check if the $ was alone. In that case, do + // not error out: Simply act as if there was an empty identifier + // with no associated fragment and paste the dollar sign in the + // transcription. Unsure how to do that since we always have at + // least the closing curly brace after an empty $... + default: + rust_error_at (token->get_locus (), + "unexpected token in macro transcribe: expected " + "%<(%> or identifier after %<$%>, got %<%s%>", + get_token_description (token->get_id ())); + } + + // FIXME: gcc_unreachable() error case? + return {std::vector> (), 0}; +} + +std::vector> +SubstituteCtx::substitute_tokens () +{ + std::vector> replaced_tokens; + rust_debug ("expanding tokens"); + + for (size_t i = 0; i < macro.size (); i++) + { + auto &tok = macro.at (i); + if (tok->get_id () == DOLLAR_SIGN) + { + // Aaaaah, if only we had C++17 :) + // auto [expanded, tok_to_skip] = ... + auto p = substitute_token (i + 1); + auto expanded = std::move (p.first); + auto tok_to_skip = p.second; + + i += tok_to_skip; + + for (auto &token : expanded) + replaced_tokens.emplace_back (token->clone_token ()); + } + else + { + replaced_tokens.emplace_back (tok->clone_token ()); + } + } + + return replaced_tokens; +} + +} // namespace Rust diff --git a/gcc/rust/expand/rust-macro-substitute-ctx.h b/gcc/rust/expand/rust-macro-substitute-ctx.h new file mode 100644 index 00000000000..81dcab7643b --- /dev/null +++ b/gcc/rust/expand/rust-macro-substitute-ctx.h @@ -0,0 +1,93 @@ +// Copyright (C) 2020-2022 Free Software Foundation, Inc. + +// This file is part of GCC. + +// GCC is free software; you can redistribute it and/or modify it under +// the terms of the GNU General Public License as published by the Free +// Software Foundation; either version 3, or (at your option) any later +// version. + +// GCC is distributed in the hope that it will be useful, but WITHOUT ANY +// WARRANTY; without even the implied warranty of MERCHANTABILITY or +// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License +// for more details. + +// You should have received a copy of the GNU General Public License +// along with GCC; see the file COPYING3. If not see +// . + +#include "rust-ast.h" +#include "rust-macro-expand.h" + +namespace Rust { +class SubstituteCtx +{ + std::vector> &input; + std::vector> ¯o; + std::map &fragments; + + /** + * Find the repetition amount to use when expanding a repetition, and + * check that all fragments used respect that repetition amount + * + * @param pattern_start Start of the repetition pattern + * @param pattern_end End of the repetition pattern + * @param repeat_amount Reference to fill with the matched repetition amount + */ + bool check_repetition_amount (size_t pattern_start, size_t pattern_end, + size_t &repeat_amount); + +public: + SubstituteCtx (std::vector> &input, + std::vector> ¯o, + std::map &fragments) + : input (input), macro (macro), fragments (fragments) + {} + + /** + * Substitute a metavariable by its given fragment in a transcribing context, + * i.e. replacing $var with the associated fragment. + * + * @param metavar Metavariable to try and replace + * + * @return A token containing the associated fragment expanded into tokens if + * any, or the cloned token if no fragment was associated + */ + std::vector> + substitute_metavar (std::unique_ptr &metavar); + + /** + * Substitute a macro repetition by its given fragments + * + * @param pattern_start Start index of the pattern tokens + * @param pattern_end End index of the patterns tokens + * @param separator Optional separator to include when expanding tokens + * + * @return A vector containing the repeated pattern + */ + std::vector> + substitute_repetition (size_t pattern_start, size_t pattern_end, + std::unique_ptr separator); + + /** + * Substitute a given token by its appropriate representation + * + * @param token_idx Current token to try and substitute + * + * @return A token containing the associated fragment expanded into tokens if + * any, or the cloned token if no fragment was associated, as well as the + * amount of tokens that should be skipped before the next invocation. Since + * this function may consume more than just one token, it is important to skip + * ahead of the input to avoid mis-substitutions + */ + std::pair>, size_t> + substitute_token (size_t token_idx); + + /** + * Substitute all tokens by their appropriate representation + * + * @return A vector containing the substituted tokens + */ + std::vector> substitute_tokens (); +}; +} // namespace Rust