where can I find a full Search Robot list
Printed From: Web Wiz Forums
Category: Web Wiz Web App Support Forums
Forum Name: Web Wiz Forums
Forum Description: Support forum for Web Wiz Forums application.
URL: https://forums.webwiz.net/forum_posts.asp?TID=15305
Printed Date: 13 April 2026 at 5:24am
Software Version: Web Wiz Forums 12.08 - https://www.webwizforums.com
Topic: where can I find a full Search Robot list
Posted By: Eran
Subject: where can I find a full Search Robot list
Date Posted: 31 May 2005 at 11:38am
| I saw that the forum have a list using the Request.ServerVariables("HTTP_USER_AGENT"). From where can I find a full list of the Search Robots that I could identify them? |
Replies:
Posted By: WebWiz-Bruce
Date Posted: 31 May 2005 at 4:18pm
|
I've not found anywhere yet that lists them all, it's a case of lots of
research on google to track down various spiders and what their user
agent ID is.
------------- https://www.webwiz.net/web-wiz-forums/forum-hosting.htm" rel="nofollow - Web Wiz Forums Hosting https://www.webwiz.net/web-hosting/windows-web-hosting.htm" rel="nofollow - ASP.NET Web Hosting |
Posted By: Eran
Date Posted: 01 June 2005 at 11:10am
|
bot the wwforum do have a small list of the main spiders.. I'm using that list but I know that it much bigger than that...
|
Posted By: sfd19
Date Posted: 01 June 2005 at 12:51pm
http://www.google.com/search?q=spiders+user+agents&meta= - http://www.google.com/search?q=spiders+user+agents&meta= http://www.psychedelix.com/agents1.html ------------- Politics, economy & social issues: http://www.studentsfordemocracy.net - StudentsforDemocracy.net |
Eran wrote: