Web Wiz Forums - Spiders

Print Page | Close Window

Spiders

Printed From: Web Wiz Forums
Category: Web Wiz Web App Support Forums
Forum Name: Web Wiz Forums
Forum Description: Support forum for Web Wiz Forums application.
URL: https://forums.webwiz.net/forum_posts.asp?TID=28138
Printed Date: 02 April 2026 at 4:42am
Software Version: Web Wiz Forums 12.08 - https://www.webwizforums.com

Topic: Spiders

Posted By: snadowitz
Subject: Spiders
Date Posted: 14 December 2009 at 3:44am

Hi - is there any way to:-

Make sure that web spiders/crawlers like googlebot can index the whole forum - including restricted access sections requiring user logon
Have some indication of wheter or not the spider/crawler has visited