Commit message (Collapse) | Author | Age | Files | Lines | |
---|---|---|---|---|---|
* | svnweb: add facebookexternalhit to robots.txt | Jani Välimaa | 2024-07-04 | 1 | -0/+1 |
| | |||||
* | svnweb: add more bots to robots.txt | Jani Välimaa | 2024-06-17 | 1 | -0/+4 |
| | |||||
* | Add two more robots to the svnweb robots.txt file | Dan Fandrich | 2024-04-25 | 1 | -1/+3 |
| | | | | | These two are hammering the site right now, causing severe slowdowns. Restrict the paths they are allowed to access. | ||||
* | viewvc/robots.txt: ban AhrefsBot | Nicolas Vigier | 2013-08-14 | 1 | -1/+2 |
| | | | | | It doesn't respect Crawl-delay, or disallowed URLs. So try to disallow everything for them. | ||||
* | viewvc/robots.txt: add more URLs | Nicolas Vigier | 2013-01-26 | 1 | -2/+2 |
| | |||||
* | viewvc/robots.txt: add Crowl-delay | Nicolas Vigier | 2013-01-26 | 1 | -0/+2 |
| | |||||
* | viewvc/robots.txt: add more URLs | Nicolas Vigier | 2013-01-26 | 1 | -2/+4 |
| | |||||
* | viewvc/robots.txt: add more URLs | Nicolas Vigier | 2013-01-26 | 1 | -1/+3 |
| | |||||
* | viewvc/robots.txt: add more URLs | Nicolas Vigier | 2013-01-26 | 1 | -0/+2 |
| | |||||
* | viewvc/robots.txt: filter ?sortby= urls | Nicolas Vigier | 2013-01-03 | 1 | -0/+1 |
| | |||||
* | viewvc/robots.txt: fix tags path | Nicolas Vigier | 2013-01-03 | 1 | -1/+1 |
| | |||||
* | viewvc: add robots.txt | Nicolas Vigier | 2013-01-03 | 1 | -0/+12 |