Directory: Computers - Internet - Search Engines and Directories - Search Engines - Search Engine Robots

Favicon
Robotstxt.org
Web robots FAQ, how to exclude robots, a list of robots, and articles.
http://www.robotstxt.org/
Favicon
List of Robot Agent Strings
A list of search robots and agent strings that show in logs after a visit. Includes links to information about each robot.
http://www.pgts.com.au/pgtsj/pgtsj0208d.html
Suggest Your Website || About Us

Copyright © 2001-2024 JoeAnt.com. All rights reserved.
Use of this site is subject to our Terms of Service.