From patchwork Fri Nov 4 13:44:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lewis Hyatt X-Patchwork-Id: 59902 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 9A8723858426 for ; Fri, 4 Nov 2022 13:47:11 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 9A8723858426 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1667569631; bh=PC812SiOVBGUVJoaVuiv6GJNfMrdmsrF89V/ITCEmfk=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=upSv/xCtfqluTmJXKOGVVt2xss62gN+eh7irirx9SLdIIhf5zz3n9H5NwGNMyn5AY fPbN46OAwiLWZHcVygTEbZwuvnYfbB8m2yg6gW2IBli3GISCRf3syJ7CmGgry1S1fD wLwlIBSjHdkS7zrLWPNQMgsjU8IuR5HKUKsWN32Q= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mail-qt1-x82c.google.com (mail-qt1-x82c.google.com [IPv6:2607:f8b0:4864:20::82c]) by sourceware.org (Postfix) with ESMTPS id 212B3385841E for ; Fri, 4 Nov 2022 13:44:46 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 212B3385841E Received: by mail-qt1-x82c.google.com with SMTP id fz10so3035528qtb.3 for ; Fri, 04 Nov 2022 06:44:46 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PC812SiOVBGUVJoaVuiv6GJNfMrdmsrF89V/ITCEmfk=; b=WlOdbMsxkcIRlr3HcWyWnLojT+wJFf3GYymwqSN80bOP9DMuc19RDxE+4HRVJ0GalD C7tnkbZ+hEH8/GtDCq8KFzmTwnRRSsUOZsDuZECnSbH3LF6Ds5ioeG9B+xps4izhCJLX PqbSNvX0KEx2snUAnR/dVj7wZ1GXl6oKkG9fE40jMwWE9XLPhT53GkSySvnkJlAFq/3i Y/LomE/QFx6G3i18ell9Y/7W5fHCYQIdVCkHByjIGwPCIBJFXsjnFX+PRXif84Jex3ZL FoFY/d/RWm8jNPHMFq8mvBHcxOqXy8mPcpK0YixptErCVtZvBBgawYwQ00s3NEKyc3dl R89g== X-Gm-Message-State: ACrzQf0tVlPT1g8WXQ3LAHUrgczjDnG2+l81w+9aGqEHbCQdWaLH18zx KO6hvs6btnE/jOAAMcv+Wg4H1gDRTqA= X-Google-Smtp-Source: AMsMyM5m5DHl1t8kT/pNfnrPrzKusSj3tyo3F/8uUeJgoXuTBU36cz7o4nEv9PV2qcieiKooWbH2SA== X-Received: by 2002:ac8:5852:0:b0:3a5:7472:67c8 with SMTP id h18-20020ac85852000000b003a5747267c8mr97297qth.575.1667569485377; Fri, 04 Nov 2022 06:44:45 -0700 (PDT) Received: from localhost.localdomain (96-67-140-173-static.hfc.comcastbusiness.net. [96.67.140.173]) by smtp.gmail.com with ESMTPSA id ey21-20020a05622a4c1500b003988b3d5280sm2470577qtb.70.2022.11.04.06.44.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Nov 2022 06:44:44 -0700 (PDT) To: gcc-patches@gcc.gnu.org Subject: [PATCH 5/6] diagnostics: Support generated data in additional contexts Date: Fri, 4 Nov 2022 09:44:13 -0400 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-3038.9 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Lewis Hyatt via Gcc-patches From: Lewis Hyatt Reply-To: Lewis Hyatt Errors-To: gcc-patches-bounces+patchwork=sourceware.org@gcc.gnu.org Sender: "Gcc-patches" Add awareness that diagnostic locations may be in generated buffers rather than an actual file to other places in the diagnostics code that may care, most notably SARIF output (which needs to obtain its own snapshots of the code involved). For edit context output, which outputs fixit hints as diffs, for now just make sure we ignore generated data buffers. At the moment, there is no ability for a fixit hint to be generated in such a buffer. Because SARIF uses JSON as well, also add the ability to the json::string class to handle a buffer with nulls in the middle (since we place no restriction on LC_GEN content) by providing the option to specify the data length. gcc/ChangeLog: * diagnostic-format-sarif.cc (sarif_builder::xloc_to_fb): New function. (sarif_builder::maybe_make_physical_location_object): Support generated data locations. (sarif_builder::make_artifact_location_object): Likewise. (sarif_builder::maybe_make_region_object_for_context): Likewise. (sarif_builder::make_artifact_object): Likewise. (sarif_builder::maybe_make_artifact_content_object): Likewise. (get_source_lines): Likewise. * edit-context.cc (edit_context::apply_fixit): Ignore generated locations if one should make its way this far. * json.cc (string::string): Support non-null-terminated string. (string::print): Likewise. * json.h (class string): Likewise. --- gcc/diagnostic-format-sarif.cc | 86 +++++++++++++++++++++------------- gcc/edit-context.cc | 4 ++ gcc/json.cc | 17 +++++-- gcc/json.h | 5 +- 4 files changed, 75 insertions(+), 37 deletions(-) diff --git a/gcc/diagnostic-format-sarif.cc b/gcc/diagnostic-format-sarif.cc index 7110db4edd6..c2d18a1a16e 100644 --- a/gcc/diagnostic-format-sarif.cc +++ b/gcc/diagnostic-format-sarif.cc @@ -125,7 +125,10 @@ private: json::array *maybe_make_kinds_array (diagnostic_event::meaning m) const; json::object *maybe_make_physical_location_object (location_t loc); json::object *make_artifact_location_object (location_t loc); - json::object *make_artifact_location_object (const char *filename); + + typedef std::pair filename_or_buffer; + json::object *make_artifact_location_object (filename_or_buffer fb); + json::object *make_artifact_location_object_for_pwd () const; json::object *maybe_make_region_object (location_t loc) const; json::object *maybe_make_region_object_for_context (location_t loc) const; @@ -146,16 +149,17 @@ private: json::object *make_reporting_descriptor_object_for_cwe_id (int cwe_id) const; json::object * make_reporting_descriptor_reference_object_for_cwe_id (int cwe_id); - json::object *make_artifact_object (const char *filename); - json::object *maybe_make_artifact_content_object (const char *filename) const; - json::object *maybe_make_artifact_content_object (const char *filename, - int start_line, + json::object *make_artifact_object (filename_or_buffer fb); + json::object * + maybe_make_artifact_content_object (filename_or_buffer fb) const; + json::object *maybe_make_artifact_content_object (expanded_location xloc, int end_line) const; json::object *make_fix_object (const rich_location &rich_loc); json::object *make_artifact_change_object (const rich_location &richloc); json::object *make_replacement_object (const fixit_hint &hint) const; json::object *make_artifact_content_object (const char *text) const; int get_sarif_column (expanded_location exploc) const; + static filename_or_buffer xloc_to_fb (expanded_location xloc); diagnostic_context *m_context; @@ -166,7 +170,11 @@ private: diagnostic group. */ sarif_result *m_cur_group_result; - hash_set m_filenames; + /* If the second member is >0, then this is a buffer of generated content, + with that length, not a filename. */ + hash_set , + int_hash > + > m_filenames; bool m_seen_any_relative_paths; hash_set m_rule_id_set; json::array *m_rules_arr; @@ -588,6 +596,15 @@ sarif_builder::make_location_object (const diagnostic_event &event) return location_obj; } +/* Populate a filename_or_buffer pair from an expanded location. */ +sarif_builder::filename_or_buffer +sarif_builder::xloc_to_fb (expanded_location xloc) +{ + if (xloc.generated_data_len) + return filename_or_buffer (xloc.generated_data, xloc.generated_data_len); + return filename_or_buffer (xloc.file, 0); +} + /* Make a physicalLocation object (SARIF v2.1.0 section 3.29) for LOC, or return NULL; Add any filename to the m_artifacts. */ @@ -603,7 +620,7 @@ sarif_builder::maybe_make_physical_location_object (location_t loc) /* "artifactLocation" property (SARIF v2.1.0 section 3.29.3). */ json::object *artifact_loc_obj = make_artifact_location_object (loc); phys_loc_obj->set ("artifactLocation", artifact_loc_obj); - m_filenames.add (LOCATION_FILE (loc)); + m_filenames.add (xloc_to_fb (expand_location (loc))); /* "region" property (SARIF v2.1.0 section 3.29.4). */ if (json::object *region_obj = maybe_make_region_object (loc)) @@ -627,7 +644,7 @@ sarif_builder::maybe_make_physical_location_object (location_t loc) json::object * sarif_builder::make_artifact_location_object (location_t loc) { - return make_artifact_location_object (LOCATION_FILE (loc)); + return make_artifact_location_object (xloc_to_fb (expand_location (loc))); } /* The ID value for use in "uriBaseId" properties (SARIF v2.1.0 section 3.4.4) @@ -639,10 +656,12 @@ sarif_builder::make_artifact_location_object (location_t loc) or return NULL. */ json::object * -sarif_builder::make_artifact_location_object (const char *filename) +sarif_builder::make_artifact_location_object (filename_or_buffer fb) { json::object *artifact_loc_obj = new json::object (); + const auto filename = (fb.second ? special_fname_generated () : fb.first); + /* "uri" property (SARIF v2.1.0 section 3.4.3). */ artifact_loc_obj->set ("uri", new json::string (filename)); @@ -795,9 +814,7 @@ sarif_builder::maybe_make_region_object_for_context (location_t loc) const /* "snippet" property (SARIF v2.1.0 section 3.30.13). */ if (json::object *artifact_content_obj - = maybe_make_artifact_content_object (exploc_start.file, - exploc_start.line, - exploc_finish.line)) + = maybe_make_artifact_content_object (exploc_start, exploc_finish.line)) region_obj->set ("snippet", artifact_content_obj); return region_obj; @@ -1248,24 +1265,24 @@ sarif_builder::maybe_make_cwe_taxonomy_object () const /* Make an artifact object (SARIF v2.1.0 section 3.24). */ json::object * -sarif_builder::make_artifact_object (const char *filename) +sarif_builder::make_artifact_object (filename_or_buffer fb) { json::object *artifact_obj = new json::object (); /* "location" property (SARIF v2.1.0 section 3.24.2). */ - json::object *artifact_loc_obj = make_artifact_location_object (filename); + json::object *artifact_loc_obj = make_artifact_location_object (fb); artifact_obj->set ("location", artifact_loc_obj); /* "contents" property (SARIF v2.1.0 section 3.24.8). */ if (json::object *artifact_content_obj - = maybe_make_artifact_content_object (filename)) + = maybe_make_artifact_content_object (fb)) artifact_obj->set ("contents", artifact_content_obj); /* "sourceLanguage" property (SARIF v2.1.0 section 3.24.10). */ if (m_context->m_client_data_hooks) if (const char *source_lang = m_context->m_client_data_hooks->maybe_get_sarif_source_language - (filename)) + (fb.first)) artifact_obj->set ("sourceLanguage", new json::string (source_lang)); return artifact_obj; @@ -1331,16 +1348,21 @@ maybe_read_file (const char *filename) full contents of FILENAME. */ json::object * -sarif_builder::maybe_make_artifact_content_object (const char *filename) const +sarif_builder::maybe_make_artifact_content_object (filename_or_buffer fb) const { - char *text_utf8 = maybe_read_file (filename); - if (!text_utf8) - return NULL; - - json::object *artifact_content_obj = new json::object (); - artifact_content_obj->set ("text", new json::string (text_utf8)); - free (text_utf8); - + json::object *artifact_content_obj = nullptr; + if (fb.second) + { + artifact_content_obj = new json::object (); + artifact_content_obj->set ("text", new json::string (fb.first, + fb.second)); + } + else if (char *text_utf8 = maybe_read_file (fb.first)) + { + artifact_content_obj = new json::object (); + artifact_content_obj->set ("text", new json::string (text_utf8)); + free (text_utf8); + } return artifact_content_obj; } @@ -1348,15 +1370,14 @@ sarif_builder::maybe_make_artifact_content_object (const char *filename) const a freshly-allocated 0-terminated buffer containing them, or NULL. */ static char * -get_source_lines (const char *filename, - int start_line, +get_source_lines (expanded_location xloc, int end_line) { auto_vec result; - for (int line = start_line; line <= end_line; line++) + for (int line = xloc.line; line <= end_line; line++) { - char_span line_content = location_get_source_line (filename, line); + char_span line_content = location_get_source_line (xloc, line); if (!line_content.get_buffer ()) return NULL; result.reserve (line_content.length () + 1); @@ -1370,14 +1391,13 @@ get_source_lines (const char *filename, } /* Make an artifactContent object (SARIF v2.1.0 section 3.3) for the given - run of lines within FILENAME (including the endpoints). */ + run of lines starting at XLOC (including the endpoints). */ json::object * -sarif_builder::maybe_make_artifact_content_object (const char *filename, - int start_line, +sarif_builder::maybe_make_artifact_content_object (expanded_location xloc, int end_line) const { - char *text_utf8 = get_source_lines (filename, start_line, end_line); + char *text_utf8 = get_source_lines (xloc, end_line); if (!text_utf8) return NULL; diff --git a/gcc/edit-context.cc b/gcc/edit-context.cc index 6879ddd41b4..aa95bc0834f 100644 --- a/gcc/edit-context.cc +++ b/gcc/edit-context.cc @@ -301,8 +301,12 @@ edit_context::apply_fixit (const fixit_hint *hint) return false; if (start.column == 0) return false; + if (start.generated_data) + return false; if (next_loc.column == 0) return false; + if (next_loc.generated_data) + return false; edited_file &file = get_or_insert_file (start.file); if (!m_valid) diff --git a/gcc/json.cc b/gcc/json.cc index 974f8c36825..3ebe8495e96 100644 --- a/gcc/json.cc +++ b/gcc/json.cc @@ -190,6 +190,15 @@ string::string (const char *utf8) { gcc_assert (utf8); m_utf8 = xstrdup (utf8); + m_len = strlen (utf8); +} + +string::string (const char *utf8, size_t len) +{ + gcc_assert (utf8); + m_utf8 = XNEWVEC (char, len); + m_len = len; + memcpy (m_utf8, utf8, len); } /* Implementation of json::value::print for json::string. */ @@ -198,9 +207,9 @@ void string::print (pretty_printer *pp) const { pp_character (pp, '"'); - for (const char *ptr = m_utf8; *ptr; ptr++) + for (size_t i = 0; i != m_len; ++i) { - char ch = *ptr; + char ch = m_utf8[i]; switch (ch) { case '"': @@ -224,7 +233,9 @@ string::print (pretty_printer *pp) const case '\t': pp_string (pp, "\\t"); break; - + case '\0': + pp_string (pp, "\\0"); + break; default: pp_character (pp, ch); } diff --git a/gcc/json.h b/gcc/json.h index f272981259b..f7afd843dc5 100644 --- a/gcc/json.h +++ b/gcc/json.h @@ -156,16 +156,19 @@ class integer_number : public value class string : public value { public: - string (const char *utf8); + explicit string (const char *utf8); + string (const char *utf8, size_t len); ~string () { free (m_utf8); } enum kind get_kind () const final override { return JSON_STRING; } void print (pretty_printer *pp) const final override; const char *get_string () const { return m_utf8; } + size_t get_length () const { return m_len; } private: char *m_utf8; + size_t m_len; }; /* Subclass of value for the three JSON literals "true", "false",