Web Wiz Forums - Frequently Visiting Spider

Print Page | Close Window

Frequently Visiting Spider – Vegabondo

Printed From: Web Wiz Forums
Category: Web Wiz Web App Support Forums
Forum Name: Web Wiz Forums
Forum Description: Support forum for Web Wiz Forums application.
URL: https://forums.webwiz.net/forum_posts.asp?TID=30455
Printed Date: 31 March 2026 at 3:01am
Software Version: Web Wiz Forums 12.08 - https://www.webwizforums.com

Topic: Frequently Visiting Spider – Vegabondo

Posted By: IanSmithISA
Subject: Frequently Visiting Spider – Vegabondo
Date Posted: 22 August 2012 at 6:13am

Good morning,

I am just generally curious and was wondering about which and how often search engines where indexing my forum, so I added the following to the end of default.asp.

As I have virtually no visits this code is not putting much of a load onto the server, about 15 rows per day, if your forum has a lot of visitors you may not want to do this.

If Session("BrowserTypeLogged") = "" Then

openDatabase(strCon)

strSQL = "INSERT INTO tblActualAgents (Agent_char,OS_Char)" & _

" VALUES ('" & left(Request.ServerVariables("HTTP_USER_AGENT"),127) & "','" & left(Request.ServerVariables("HTTP_USER_AGENT"),127) & "')"

'Set error trapping

On Error Resume Next

adoCon.Execute(strSQL)

'If an error has occurred write an error to the page

If Err.Number <> 0 Then Call errorMsg("An error has occurred while executing SQL query on database.", "Insert Browser Type", "default.asp")

'Disable error trapping

On Error goto 0

CloseDatabase()

Session("BrowserTypeLogged") = "Done"

end if

And via SQL Server Enterprise Manager

CREATE TABLE [IanSmithcse_forum].[tblActualAgents](

Ref_id int IDENTITY(1,1) NOT NULL,

DateCreated_date datetime NOT NULL DEFAULT GetDate(),

Agent_char nchar(128) NULL,

OS_char nchar(128) NULL

) ON [PRIMARY]

Note that this table is not even indexed.

Querying this table I found this

2012-08-19 09:52:09.080 Mozilla/4.0 (compatible; Vagabondo/4.0; webcrawler at wise-guys dot nl; http://webagent.wise-guys.nl/; http://www.wise-guys.nl

2012-08-19 09:53:25.210 Mozilla/4.0 (compatible; Vagabondo/4.0; webcrawler at wise-guys dot nl; http://webagent.wise-guys.nl/; http://www.wise-guys.nl

2012-08-19 09:54:41.237 Mozilla/4.0 (compatible; Vagabondo/4.0; webcrawler at wise-guys dot nl; http://webagent.wise-

I left it for a while and as far as I can see Vagabondo just keeps on querying every 75 seconds. It is not me, all the other search engines seem much more sensible.

I have added Vagabondo to my robots.txt and this is respected. Anybody know anything about this spider, I have looked at http://www.wise-guys.nl and there is no obvious reason why I should care about them?

Bye

Ian