Web Wiz Support and Community Forums : Google not spidering https://forums.webwiz.net/ Copyright (c) 2006-2013 Web Wiz Forums - All Rights Reserved. Tue, 07 Apr 2026 23:40:52 +0000 Tue, 10 Feb 2004 19:39:32 +0000 http://blogs.law.harvard.edu/tech/rss Web Wiz Forums 12.08 360 https://forums.webwiz.net/RSS_post_feed.asp?TID=9711 <![CDATA[Web Wiz Support and Community Forums]]> https://forums.webwiz.net/forum_images/web_wiz_forums.png https://forums.webwiz.net/ <![CDATA[Google not spidering : Ahh - that must be it. Thanks...]]> https://forums.webwiz.net/google-not-spidering_topic9711_post52177.html#52177 Author: peter
Subject: 9711
Posted: 10 February 2004 at 7:39pm

Ahh - that must be it.  Thanks a lot for that dpyers.]]> Tue, 10 Feb 2004 19:39:32 +0000 https://forums.webwiz.net/google-not-spidering_topic9711_post52177.html#52177 <![CDATA[Google not spidering : Google does an initial scan of...]]> https://forums.webwiz.net/google-not-spidering_topic9711_post52176.html#52176 Author: dpyers
Subject: 9711
Posted: 10 February 2004 at 7:36pm

Google does an initial scan of a site - just the home page and a couple of links off of it. Then returns 2-6 weeks later and performs a deep scan.]]> Tue, 10 Feb 2004 19:36:18 +0000 https://forums.webwiz.net/google-not-spidering_topic9711_post52176.html#52176 <![CDATA[Google not spidering : There is no reason in the code...]]> https://forums.webwiz.net/google-not-spidering_topic9711_post52149.html#52149 Author: WebWiz-Bruce
Subject: 9711
Posted: 10 February 2004 at 5:57pm

There is no reason in the code for google not to spider pages inside your forum.

Google spiders all the pages in the forum on this site and many other sites running web wiz forums.
]]> Tue, 10 Feb 2004 17:57:10 +0000 https://forums.webwiz.net/google-not-spidering_topic9711_post52149.html#52149 <![CDATA[Google not spidering : Google spiders the first page...]]> https://forums.webwiz.net/google-not-spidering_topic9711_post52127.html#52127 Author: peter
Subject: 9711
Posted: 10 February 2004 at 4:11pm

Google spiders the first page of my forum (and lists it) but isn't spidering the rest of it.  I know this for a fact.

Why is this?  I can't find one, but perhaps webwiz has some kind of spider=no tag on all the other pages??

It does spider the rest of the site (the forum is inside a /forum folder) but not the forum, other than  the first page.  You do NOT need to be logged in to view posts either.

If someone could help me with this I'd really appreciate it.



Edited by peter]]> Tue, 10 Feb 2004 16:11:27 +0000 https://forums.webwiz.net/google-not-spidering_topic9711_post52127.html#52127