Web Wiz - Green Windows Web Hosting
Web Wiz Homepage Search Web Wiz

  New Posts New Posts RSS Feed - Prevent googlebot from opening reply forms etc
  FAQ FAQ  Forum Search   Events   Register Register  Login Login

Prevent googlebot from opening reply forms etc
 Post Reply Post Reply Page  <123
Author
  Topic Search Topic Search  Topic Options Topic Options
WebWiz-Bruce View Drop Down
Admin Group
Admin Group
Avatar
Web Wiz Developer

Joined: 03 September 2001
Location: Bournemouth
Status: Offline
Points: 9844 		Post Options Post Options   Thanks (0) Thanks(0)   Quote WebWiz-Bruce Quote  Post ReplyReply Direct Link To This Post Posted: 11 July 2007 at 2:38pm
Mattis have now implemented the nofollow on this forum to see how it goes for pages that only members can access like writing replies, and looking at the active users page it's no-longer full of listing of search engines trying index pages that they have no access to, so should save on some unwanted bandwidth use.
Web Wiz Forums Hosting
ASP.NET Web Hosting
Back to Top
WebWiz-Bruce View Drop Down
Admin Group
Admin Group
Avatar
Web Wiz Developer

Joined: 03 September 2001
Location: Bournemouth
Status: Offline
Points: 9844 		Post Options Post Options   Thanks (0) Thanks(0)   Quote WebWiz-Bruce Quote  Post ReplyReply Direct Link To This Post Posted: 11 July 2007 at 9:23pm
OK spoke to soon likes like all the major search engines, Yahoo, Google, etc. just ignore the nofollow directive and still visit the page anyway.
Web Wiz Forums Hosting
ASP.NET Web Hosting
Back to Top
Mattias Nordin View Drop Down
Groupie
Groupie


Joined: 30 March 2005
Status: Offline
Points: 60 		Post Options Post Options   Thanks (0) Thanks(0)   Quote Mattias Nordin Quote  Post ReplyReply Direct Link To This Post Posted: 12 July 2007 at 8:11am

Borg, i think that your change will work pretty well. The problem is probably that pages that allready exist in the organic google list will be vistited very often just to make sure the page still exists. Your change would hopfully prevent that the page will be addedd to the organic list in the first place. I don't think that the nofollow attribure will give the same signal as response 404.

Off topic. Nice layout on the forum. Something feels new ;)
Back to Top
ruycnd View Drop Down
Newbie
Newbie


Joined: 28 March 2007
Status: Offline
Points: 14 		Post Options Post Options   Thanks (0) Thanks(0)   Quote ruycnd Quote  Post ReplyReply Direct Link To This Post Posted: 16 July 2007 at 9:15am
Wink


Edited by ruycnd - 07 July 2011 at 6:40am
Back to Top
Mattias Nordin View Drop Down
Groupie
Groupie


Joined: 30 March 2005
Status: Offline
Points: 60 		Post Options Post Options   Thanks (0) Thanks(0)   Quote Mattias Nordin Quote  Post ReplyReply Direct Link To This Post Posted: 16 July 2007 at 9:22am

If i remeber correctly the "disallow" items in a robots.txt file needs to have a full path. I dont think that wildcards are supported. This would mean that it's not possible for borg to create a robots.txt page that would work for everyone becuase it's up to the user to install the script at any given location.

I think that the action borg allready have taken are good enough :)
Back to Top
 Post Reply Post Reply Page  <123

Forum Jump Forum Permissions View Drop Down

Forum Software by Web Wiz Forums® version 12.08
Copyright ©2001-2026 Web Wiz Ltd.

Become a Fan on Facebook Follow us on X Connect with us on LinkedIn Web Wiz Blogs
About Web Wiz | Contact Web Wiz | Terms & Conditions | Cookies | Privacy Notice

Web Wiz is the trading name of Web Wiz Ltd. Company registration No. 05977755. Registered in England and Wales.
Registered office: Web Wiz Ltd, Unit 18, The Glenmore Centre, Fancy Road, Poole, Dorset, BH12 4FB, UK.

Prices exclude VAT at 20% unless otherwise stated. VAT No. GB988999105 - $, € prices shown as a guideline only.

Copyright ©2001-2026 Web Wiz Ltd. All rights reserved.