diff options
| author | Romain d'Alverny <rda@mageia.org> | 2011-12-21 18:46:04 +0000 |
|---|---|---|
| committer | Romain d'Alverny <rda@mageia.org> | 2011-12-21 18:46:04 +0000 |
| commit | 737afe6e46fba4d2f9ab3cf4bcb309cef17532d0 (patch) | |
| tree | 1da1e8ffc0780c2258cf2e5e64352cfa22cfeced /robots.txt | |
| parent | daea2d1ee6c370a5c88f24dc2070f79f3f24d40f (diff) | |
| download | www-737afe6e46fba4d2f9ab3cf4bcb309cef17532d0.tar www-737afe6e46fba4d2f9ab3cf4bcb309cef17532d0.tar.gz www-737afe6e46fba4d2f9ab3cf4bcb309cef17532d0.tar.bz2 www-737afe6e46fba4d2f9ab3cf4bcb309cef17532d0.tar.xz www-737afe6e46fba4d2f9ab3cf4bcb309cef17532d0.zip | |
robots.txt
Diffstat (limited to 'robots.txt')
| -rw-r--r-- | robots.txt | 29 |
1 files changed, 29 insertions, 0 deletions
diff --git a/robots.txt b/robots.txt new file mode 100644 index 000000000..dbc4707f9 --- /dev/null +++ b/robots.txt @@ -0,0 +1,29 @@ +# +# +# + +#Sitemap: /sitemap.xml + +User-Agent: * +Disallow: + + +User-Agent: bingbot +Disallow: /*/downloads/dl.php* +Disallow: /*/downloads/get/* + +User-Agent: Googlebot +Disallow: /*/downloads/dl.php* +Disallow: /*/downloads/get/* + +User-Agent: msnbot +Disallow: /*/downloads/dl.php* +Disallow: /*/downloads/get/* + +User-Agent: Ezooms +Disallow: /*/downloads/dl.php* +Disallow: /*/downloads/get/* + +User-Agent: MJ12bot +Disallow: /*/downloads/dl.php* +Disallow: /*/downloads/get/* |