Web Wiz - Green Windows Web Hosting
Web Wiz Homepage Search Web Wiz

  New Posts New Posts RSS Feed - Search Engine Access
  FAQ FAQ  Forum Search   Events   Register Register  Login Login

Search Engine Access
 Post Reply Post Reply
Author
  Topic Search Topic Search  Topic Options Topic Options
snadowitz View Drop Down
Groupie
Groupie


Joined: 06 November 2009
Status: Offline
Points: 44 		Post Options Post Options   Thanks (0) Thanks(0)   Quote snadowitz Quote  Post ReplyReply Direct Link To This Post Topic: Search Engine Access
    Posted: 08 January 2010 at 10:12am
Hi- I noticed whilst looking at my statistics page that the yahoo and google search bots have been on the site for a long time

Guest 1 66.249.71.185 Today at 04:32 455 minutes Search Robot Google Writing Reply [Access Denied]
Still scabby!
Guest 2 67.195.112.158 Today at 00:33 694 minutes Search Robot Yahoo Viewing Topic
where to start?


The google bot seems stuck on this page for ages even though it cannot access it yet the yahoo bot page changes everytime I reload. How can I stop the google bot (in fact any bot) from trying to index pages it should not be indexing?
Back to Top
WebWiz-Bruce View Drop Down
Admin Group
Admin Group
Avatar
Web Wiz Developer

Joined: 03 September 2001
Location: Bournemouth
Status: Offline
Points: 9844 		Post Options Post Options   Thanks (0) Thanks(0)   Quote WebWiz-Bruce Quote  Post ReplyReply Direct Link To This Post Posted: 08 January 2010 at 11:41am
There is not much that can be done about this as Web Wiz Forums already has features in place to tell search engines not to follow links to these pages and not index them. Unfortunately search engine bots sometime ignore these requests and still try to access the page and are meet by an access denied page with the page set to not be indexed.
Web Wiz Forums Hosting
ASP.NET Web Hosting
Back to Top
 Post Reply Post Reply

Forum Jump Forum Permissions View Drop Down

Forum Software by Web Wiz Forums® version 12.08
Copyright ©2001-2026 Web Wiz Ltd.

Become a Fan on Facebook Follow us on X Connect with us on LinkedIn Web Wiz Blogs
About Web Wiz | Contact Web Wiz | Terms & Conditions | Cookies | Privacy Notice

Web Wiz is the trading name of Web Wiz Ltd. Company registration No. 05977755. Registered in England and Wales.
Registered office: Web Wiz Ltd, Unit 18, The Glenmore Centre, Fancy Road, Poole, Dorset, BH12 4FB, UK.

Prices exclude VAT at 20% unless otherwise stated. VAT No. GB988999105 - $, € prices shown as a guideline only.

Copyright ©2001-2026 Web Wiz Ltd. All rights reserved.