diff mbox series

[COMMITTED] robots.txt: Disallow a few more bugzilla queries

Message ID 20231222205230.182419-1-mark@klomp.org
State New
Headers
Series [COMMITTED] robots.txt: Disallow a few more bugzilla queries |

Checks

Context Check Description
linaro-tcwg-bot/tcwg_gcc_build--master-arm fail Patch failed to apply
linaro-tcwg-bot/tcwg_gcc_build--master-aarch64 fail Patch failed to apply

Commit Message

Mark Wielaard Dec. 22, 2023, 8:52 p.m. UTC 
  Some spiders are hitting bugzilla hard generating dependency trees
or graphs, downloading large attachements or requesting all bugs
in xml format. Disallow all that.
---
 htdocs/robots.txt | 4 ++++
 1 file changed, 4 insertions(+)
diff mbox series

Patch

diff --git a/htdocs/robots.txt b/htdocs/robots.txt
index b9fc830d..057c5899 100644
--- a/htdocs/robots.txt
+++ b/htdocs/robots.txt
@@ -10,4 +10,8 @@  Disallow: /cgit/
 Disallow: /svn
 Disallow: /cgi-bin/
 Disallow: /bugzilla/buglist.cgi
+Disallow: /bugzilla/show_bug.cgi*ctype=xml*
+Disallow: /bugzilla/attachment.cgi
+Disallow: /bugzilla/showdependencygraph.cgi
+Disallow: /bugzilla/showdependencytree.cgi
 Crawl-Delay: 60