Web Wiz Support and Community Forums : Robots.txt https://forums.webwiz.net/ Copyright (c) 2006-2013 Web Wiz Forums - All Rights Reserved. Sat, 11 Apr 2026 19:30:30 +0000 Mon, 15 May 2006 10:53:41 +0000 http://blogs.law.harvard.edu/tech/rss Web Wiz Forums 12.08 360 https://forums.webwiz.net/RSS_post_feed.asp?TID=19847 <![CDATA[Web Wiz Support and Community Forums]]> https://forums.webwiz.net/forum_images/web_wiz_forums.png https://forums.webwiz.net/ <![CDATA[Robots.txt : Yes, but any new posts that include...]]> https://forums.webwiz.net/robots-txt_topic19847_post106975.html#106975 Author: WebWiz-Bruce
Subject: 19847
Posted: 15 May 2006 at 10:53am

Yes, but any new posts that include links will not be followed by a rebot, therefore they shouldn't be indexed.]]> Mon, 15 May 2006 10:53:41 +0000 https://forums.webwiz.net/robots-txt_topic19847_post106975.html#106975 <![CDATA[Robots.txt : Will this still allow posts to...]]> https://forums.webwiz.net/robots-txt_topic19847_post106959.html#106959 Author: johnk
Subject: 19847
Posted: 15 May 2006 at 9:49am

Will this still allow posts to the forum to be crawled?
 
Thanks
johnk
]]> Mon, 15 May 2006 09:49:43 +0000 https://forums.webwiz.net/robots-txt_topic19847_post106959.html#106959 <![CDATA[Robots.txt : In the file includes/setup_options.asp...]]> https://forums.webwiz.net/robots-txt_topic19847_post106878.html#106878 Author: WebWiz-Bruce
Subject: 19847
Posted: 13 May 2006 at 10:40am

In the file includes/setup_options.asp set the following variable to True:-

Const blnNoFollowTagInLinks = False
]]> Sat, 13 May 2006 10:40:24 +0000 https://forums.webwiz.net/robots-txt_topic19847_post106878.html#106878 <![CDATA[Robots.txt : WWF v8 Got everything up...]]> https://forums.webwiz.net/robots-txt_topic19847_post106861.html#106861 Author: johnk
Subject: 19847
Posted: 13 May 2006 at 1:52am

WWF v8
 
 
Got everything up and running last night. This morning I discovered that my site had been crawled and virtually every file associated with the forum had been indexed.
Any guidelines re excluding files/directories so that forum posts will be indexed but the "non content" files will be excluded?
 
Thanks
johnk
]]> Sat, 13 May 2006 01:52:28 +0000 https://forums.webwiz.net/robots-txt_topic19847_post106861.html#106861