Web Wiz Support and Community Forums : gogole robot https://forums.webwiz.net/ Copyright (c) 2006-2013 Web Wiz Forums - All Rights Reserved. Thu, 09 Apr 2026 04:21:38 +0000 Wed, 30 Aug 2006 23:36:37 +0000 http://blogs.law.harvard.edu/tech/rss Web Wiz Forums 12.08 360 https://forums.webwiz.net/RSS_post_feed.asp?TID=21149 <![CDATA[Web Wiz Support and Community Forums]]> https://forums.webwiz.net/forum_images/web_wiz_forums.png https://forums.webwiz.net/ <![CDATA[gogole robot : I have just put a robots.txt file...]]> https://forums.webwiz.net/gogole-robot_topic21149_post113366.html#113366 Author: jckruger
Subject: 21149
Posted: 30 August 2006 at 11:36pm

I have just put a robots.txt file in the root of my server and haven't seen any googlebots since.

Here is a reference you can use

From http://www.robotstxt.org/wc/faq.html "How do I stop robots scanning my site"

How do I prevent robots scanning my site?

The quick way to prevent robots visiting your site is put these two lines into the /robots.txt file on your server: 

User-agent: *Disallow: /

but its easy to be more selective than that.

]]> Wed, 30 Aug 2006 23:36:37 +0000 https://forums.webwiz.net/gogole-robot_topic21149_post113366.html#113366 <![CDATA[gogole robot : thank you so much... am a little...]]> https://forums.webwiz.net/gogole-robot_topic21149_post113218.html#113218 Author: kevsc
Subject: 21149
Posted: 27 August 2006 at 1:07pm

thank you so much... am a little less worried now, love the program just trying to figure everything out, is there a way of including location under their avatar?]]> Sun, 27 Aug 2006 13:07:58 +0000 https://forums.webwiz.net/gogole-robot_topic21149_post113218.html#113218 <![CDATA[gogole robot : -boRg- wrote:You need to be...]]> https://forums.webwiz.net/gogole-robot_topic21149_post113215.html#113215 Author: dfrancis
Subject: 21149
Posted: 27 August 2006 at 12:41pm

Originally posted by -boRg- -boRg- wrote:

You need to be a registered user in order to read a users profile.

If the Google Robot follows a link to a users profile it just gets a page saying that it needs to be registered in order to read that users profiles, so never actually reads the users actual profile.

If you are reading in the active users page that Google Robot is viewing 'username profile' then this is a bit in accurate as they are actually viewing a page saying it needs to be registered to view this users profile.

I shall look into changing this for the next version so that the active users page shows a more accurate message for this page.
 
 
You rock boRgman! (That's a good thing my kids tell me.)

I was thinking the same thing. I was thinking about offering a meta noindex follow and then a site map of public pages could help with SEO as well as customer service to the unregistered user.

]]> Sun, 27 Aug 2006 12:41:52 +0000 https://forums.webwiz.net/gogole-robot_topic21149_post113215.html#113215 <![CDATA[gogole robot : You need to be a registered user...]]> https://forums.webwiz.net/gogole-robot_topic21149_post113211.html#113211 Author: WebWiz-Bruce
Subject: 21149
Posted: 27 August 2006 at 11:43am

You need to be a registered user in order to read a users profile.

If the Google Robot follows a link to a users profile it just gets a page saying that it needs to be registered in order to read that users profiles, so never actually reads the users actual profile.

If you are reading in the active users page that Google Robot is viewing 'username profile' then this is a bit in accurate as they are actually viewing a page saying it needs to be registered to view this users profile.

I shall look into changing this for the next version so that the active users page shows a more accurate message for this page.
]]> Sun, 27 Aug 2006 11:43:46 +0000 https://forums.webwiz.net/gogole-robot_topic21149_post113211.html#113211 <![CDATA[gogole robot : I am a little confused, I have...]]> https://forums.webwiz.net/gogole-robot_topic21149_post113205.html#113205 Author: kevsc
Subject: 21149
Posted: 27 August 2006 at 9:45am

I am a little confused, I have closed all access for the guest account but the robot seems to get through still, any way of blocking this as I dont want user profiles being on public display.]]> Sun, 27 Aug 2006 09:45:05 +0000 https://forums.webwiz.net/gogole-robot_topic21149_post113205.html#113205