Web Wiz - Green Windows Web Hosting
Web Wiz Homepage Search Web Wiz

  New Posts New Posts RSS Feed - gogole robot
  FAQ FAQ  Forum Search   Events   Register Register  Login Login

gogole robot
 Post Reply Post Reply
Author
  Topic Search Topic Search  Topic Options Topic Options
kevsc View Drop Down
Newbie
Newbie
Avatar

Joined: 17 August 2006
Location: Australia
Status: Offline
Points: 21 		Post Options Post Options   Thanks (0) Thanks(0)   Quote kevsc Quote  Post ReplyReply Direct Link To This Post Topic: gogole robot
    Posted: 27 August 2006 at 9:45am
I am a little confused, I have closed all access for the guest account but the robot seems to get through still, any way of blocking this as I dont want user profiles being on public display.
Back to Top
WebWiz-Bruce View Drop Down
Admin Group
Admin Group
Avatar
Web Wiz Developer

Joined: 03 September 2001
Location: Bournemouth
Status: Offline
Points: 9844 		Post Options Post Options   Thanks (0) Thanks(0)   Quote WebWiz-Bruce Quote  Post ReplyReply Direct Link To This Post Posted: 27 August 2006 at 11:43am
You need to be a registered user in order to read a users profile.

If the Google Robot follows a link to a users profile it just gets a page saying that it needs to be registered in order to read that users profiles, so never actually reads the users actual profile.

If you are reading in the active users page that Google Robot is viewing 'username profile' then this is a bit in accurate as they are actually viewing a page saying it needs to be registered to view this users profile.

I shall look into changing this for the next version so that the active users page shows a more accurate message for this page.
Web Wiz Forums Hosting
ASP.NET Web Hosting
Back to Top
dfrancis View Drop Down
Senior Member
Senior Member


Joined: 16 March 2005
Location: United States
Status: Offline
Points: 442 		Post Options Post Options   Thanks (0) Thanks(0)   Quote dfrancis Quote  Post ReplyReply Direct Link To This Post Posted: 27 August 2006 at 12:41pm
Originally posted by -boRg- -boRg- wrote:

You need to be a registered user in order to read a users profile.

If the Google Robot follows a link to a users profile it just gets a page saying that it needs to be registered in order to read that users profiles, so never actually reads the users actual profile.

If you are reading in the active users page that Google Robot is viewing 'username profile' then this is a bit in accurate as they are actually viewing a page saying it needs to be registered to view this users profile.

I shall look into changing this for the next version so that the active users page shows a more accurate message for this page.
 
 
You rock boRgman! (That's a good thing my kids tell me.)

I was thinking the same thing. I was thinking about offering a meta noindex follow and then a site map of public pages could help with SEO as well as customer service to the unregistered user.
Back to Top
kevsc View Drop Down
Newbie
Newbie
Avatar

Joined: 17 August 2006
Location: Australia
Status: Offline
Points: 21 		Post Options Post Options   Thanks (0) Thanks(0)   Quote kevsc Quote  Post ReplyReply Direct Link To This Post Posted: 27 August 2006 at 1:07pm
thank you so much... am a little less worried now, love the program just trying to figure everything out, is there a way of including location under their avatar?
Back to Top
jckruger View Drop Down
Groupie
Groupie
Avatar

Joined: 18 April 2006
Location: Australia
Status: Offline
Points: 135 		Post Options Post Options   Thanks (0) Thanks(0)   Quote jckruger Quote  Post ReplyReply Direct Link To This Post Posted: 30 August 2006 at 11:36pm

I have just put a robots.txt file in the root of my server and haven't seen any googlebots since.

Here is a reference you can use

From http://www.robotstxt.org/wc/faq.html "How do I stop robots scanning my site"

How do I prevent robots scanning my site?

The quick way to prevent robots visiting your site is put these two lines into the /robots.txt file on your server: 

User-agent: *
Disallow: /

but its easy to be more selective than that.
Back to Top
 Post Reply Post Reply

Forum Jump Forum Permissions View Drop Down

Forum Software by Web Wiz Forums® version 12.08
Copyright ©2001-2026 Web Wiz Ltd.

Become a Fan on Facebook Follow us on X Connect with us on LinkedIn Web Wiz Blogs
About Web Wiz | Contact Web Wiz | Terms & Conditions | Cookies | Privacy Notice

Web Wiz is the trading name of Web Wiz Ltd. Company registration No. 05977755. Registered in England and Wales.
Registered office: Web Wiz Ltd, Unit 18, The Glenmore Centre, Fancy Road, Poole, Dorset, BH12 4FB, UK.

Prices exclude VAT at 20% unless otherwise stated. VAT No. GB988999105 - $, € prices shown as a guideline only.

Copyright ©2001-2026 Web Wiz Ltd. All rights reserved.