diff options
| author | Bruno Cornec <bruno@victoria.frmug.org> | 2026-03-09 12:11:37 +0100 |
|---|---|---|
| committer | Bruno Cornec <bruno@victoria.frmug.org> | 2026-03-09 12:11:37 +0100 |
| commit | b47efd9b4df85e56933c78357fc2ce347a7f49d0 (patch) | |
| tree | 56dd46f46558e51ca9c58ba7b60a8148741b1c5c /modules | |
| parent | a084b85270826e004e293ce3a2a9f6da2f97a0dd (diff) | |
| download | puppet-b47efd9b4df85e56933c78357fc2ce347a7f49d0.tar puppet-b47efd9b4df85e56933c78357fc2ce347a7f49d0.tar.gz puppet-b47efd9b4df85e56933c78357fc2ce347a7f49d0.tar.bz2 puppet-b47efd9b4df85e56933c78357fc2ce347a7f49d0.tar.xz puppet-b47efd9b4df85e56933c78357fc2ce347a7f49d0.zip | |
More Bots exluded in robots.txt - DataForSeoBot SemrushBot Applebot GPTBot SeznamBot)
Diffstat (limited to 'modules')
| -rw-r--r-- | modules/viewvc/files/robots.txt | 5 |
1 files changed, 5 insertions, 0 deletions
diff --git a/modules/viewvc/files/robots.txt b/modules/viewvc/files/robots.txt index dbb13834..e4534898 100644 --- a/modules/viewvc/files/robots.txt +++ b/modules/viewvc/files/robots.txt @@ -9,6 +9,11 @@ User-agent: Amazonbot User-agent: PetalBot User-agent: Bytespider User-agent: facebookexternalhit +User-agent: DataForSeoBot +User-agent: SemrushBot +User-agent: Applebot +User-agent: GPTBot +User-agent: SeznamBot Disallow: /*/tags/ Disallow: *?view=annotate* Disallow: *?annotate=* |