Web Wiz Forums - Robots.txt

Print Page | Close Window

Robots.txt

Printed From: Web Wiz Forums
Category: Web Wiz Web App Support Forums
Forum Name: Web Wiz Forums
Forum Description: Support forum for Web Wiz Forums application.
URL: https://forums.webwiz.net/forum_posts.asp?TID=19847
Printed Date: 11 April 2026 at 10:35pm
Software Version: Web Wiz Forums 12.08 - https://www.webwizforums.com

Topic: Robots.txt

Posted By: johnk
Subject: Robots.txt
Date Posted: 13 May 2006 at 1:52am

WWF v8

Got everything up and running last night. This morning I discovered that my site had been crawled and virtually every file associated with the forum had been indexed.

Any guidelines re excluding files/directories so that forum posts will be indexed but the "non content" files will be excluded?

Thanks

johnk

Replies:

Posted By: WebWiz-Bruce
Date Posted: 13 May 2006 at 10:40am

In the file includes/setup_options.asp set the following variable to True:-

Const blnNoFollowTagInLinks = False

-------------
https://www.webwiz.net/web-wiz-forums/forum-hosting.htm" rel="nofollow - Web Wiz Forums Hosting
https://www.webwiz.net/web-hosting/windows-web-hosting.htm" rel="nofollow - ASP.NET Web Hosting

Posted By: johnk
Date Posted: 15 May 2006 at 9:49am

Will this still allow posts to the forum to be crawled?

Thanks

johnk

Posted By: WebWiz-Bruce
Date Posted: 15 May 2006 at 10:53am

Yes, but any new posts that include links will not be followed by a rebot, therefore they shouldn't be indexed.

-------------
https://www.webwiz.net/web-wiz-forums/forum-hosting.htm" rel="nofollow - Web Wiz Forums Hosting
https://www.webwiz.net/web-hosting/windows-web-hosting.htm" rel="nofollow - ASP.NET Web Hosting