Web Wiz - Green Windows Web Hosting

  New Posts New Posts RSS Feed - Prevent googlebot from opening reply forms etc
  FAQ FAQ  Forum Search   Events   Register Register  Login Login

Prevent googlebot from opening reply forms etc

 Post Reply Post Reply Page  123>
Author
Mattias Nordin View Drop Down
Groupie
Groupie


Joined: 30 March 2005
Status: Offline
Points: 60
Post Options Post Options   Thanks (0) Thanks(0)   Quote Mattias Nordin Quote  Post ReplyReply Direct Link To This Post Topic: Prevent googlebot from opening reply forms etc
    Posted: 26 June 2007 at 9:29pm
A suggestion, add "no follow" to all edit and reply buttons to prevent google from indexing pages like /new_reply_form.asp?M=Q&PID=136&PN=1&TR=1
 
I can see that google spent all day indexing pages that the googlebot should not have access to. If you have a lot of threads then google will have hundreds of unneccesary links to follow
 
You may solve this by adding the "nofollow" attribute to the links on all quote, reply, login and add new buttons. 
 
Examlpe, this is how it looks today:

...<a href="new_reply_form.asp?TID=360&PN=1&TR=1"><img src="forum_images/post_reply.gif"  border="0" align="absmiddle" alt="Post Reply"/></a>
<a href="new_topic_form.asp?FID=2"><img src="forum_images/new_post.gif"  border="0" align="absmiddle" alt="Post New Topic"/></a>...
To add a "nofollow" attribute on all links on a specific page use the meta tag:
<META name="ROBOTS" content="NOINDEX, NOFOLLOW">
The meta tag above works but the drawback is that all links on that page is affected and if the forum is included on someones webpage this option may not be to popular. Another solution that might work (depending on the spider) is to add rel="nofollow" and class="robots-nocontent" to the links.
 
Example:
 

<a href="new_topic_form.asp?FID=2" REL="nofollow" class="robots-nocontent"><img src="forum_images/new_post.gif"  border="0" align="absmiddle" alt="Post New Topic"/></a>...
I have to mention that i use one of the older versions of the 
forum, perhaps stuff has changed. Keep up the good work.
PS here you will find a list of all the lates google 
activity on my website. Note that googlebot has spent
time indexing many of the reply.asp pages. Googlebot list here DS
Best regards
Mattias 


Edited by Mattias Nordin - 26 June 2007 at 9:40pm
Back to Top
WebWiz-Bruce View Drop Down
Admin Group
Admin Group
Avatar
Web Wiz Developer

Joined: 03 September 2001
Location: Bournemouth
Status: Offline
Points: 9844
Post Options Post Options   Thanks (0) Thanks(0)   Quote WebWiz-Bruce Quote  Post ReplyReply Direct Link To This Post Posted: 27 June 2007 at 11:44am
The nofollow tag is already used in Web Wiz Forums but it is certainly worth adding in the places you mention
Back to Top
Mattias Nordin View Drop Down
Groupie
Groupie


Joined: 30 March 2005
Status: Offline
Points: 60
Post Options Post Options   Thanks (0) Thanks(0)   Quote Mattias Nordin Quote  Post ReplyReply Direct Link To This Post Posted: 27 June 2007 at 12:03pm
Thank you borg :)
Back to Top
Mattias Nordin View Drop Down
Groupie
Groupie


Joined: 30 March 2005
Status: Offline
Points: 60
Post Options Post Options   Thanks (0) Thanks(0)   Quote Mattias Nordin Quote  Post ReplyReply Direct Link To This Post Posted: 28 June 2007 at 4:02pm
While you're at it you might want to prevent the spiders from following all links in the calendar. Googlebot seems to be stupid enough to follow all kind of links to blank pages in the calendar...
 
If you're interested you may view the activity on this page. Newest at the bottom.
 
 
Back to Top
WebWiz-Bruce View Drop Down
Admin Group
Admin Group
Avatar
Web Wiz Developer

Joined: 03 September 2001
Location: Bournemouth
Status: Offline
Points: 9844
Post Options Post Options   Thanks (0) Thanks(0)   Quote WebWiz-Bruce Quote  Post ReplyReply Direct Link To This Post Posted: 28 June 2007 at 5:14pm
The problem with that is that people will want their calender events indexed by Google.

Have been looking into the issue of search engines following links to pages it can't index like the profile, replay, etc. pages and although Google and MSN often obey directives like rel="nofollow" allot of search engines like Yahoo completely ignore this but does obay class="robots-nocontent"

Even with a directive not to follow a link to a page often Google and MSN will still visit the page anyway although they would not index it.

The rel="nofollow" has been added to the next version to page links to pages that search engines are not permitted like reply, new post, etc. which should mean less visits to those pages from search engines but won't stop it completely, and should also help reduce bandwidth as have a number of large forums with lots of links to then where search engines are sucking up allot of bandwidth indexing pages it's not permitted on.
Back to Top
WebWiz-Bruce View Drop Down
Admin Group
Admin Group
Avatar
Web Wiz Developer

Joined: 03 September 2001
Location: Bournemouth
Status: Offline
Points: 9844
Post Options Post Options   Thanks (0) Thanks(0)   Quote WebWiz-Bruce Quote  Post ReplyReply Direct Link To This Post Posted: 28 June 2007 at 5:20pm
Sorry my mistake the class="robots-nocontent" will not work on links.

It is only used by Yahoo at the moment to omit sections of an XHTML page from being indexed, you place it on a div or span section, on a link it has no effect as the link will be followed anyway.

So it would seem that although rel="nofollow" will have some effect on Google and MSN there isn't anything that will prevent Yahoo from following a link.
Back to Top
Mattias Nordin View Drop Down
Groupie
Groupie


Joined: 30 March 2005
Status: Offline
Points: 60
Post Options Post Options   Thanks (0) Thanks(0)   Quote Mattias Nordin Quote  Post ReplyReply Direct Link To This Post Posted: 09 July 2007 at 12:52pm
Related link:
 
Google, Yahoo, MSN Unite On Support For Nofollow Attribute For Links
 
:)
Back to Top
WebWiz-Bruce View Drop Down
Admin Group
Admin Group
Avatar
Web Wiz Developer

Joined: 03 September 2001
Location: Bournemouth
Status: Offline
Points: 9844
Post Options Post Options   Thanks (0) Thanks(0)   Quote WebWiz-Bruce Quote  Post ReplyReply Direct Link To This Post Posted: 09 July 2007 at 1:23pm
Just a shame Yahoo didn't stick to this as this article is from 2005 and Yahoo presently seem to ignore rel="nofollow", but most of the other search engines seem to follow it.
Back to Top
 Post Reply Post Reply Page  123>

Forum Jump Forum Permissions View Drop Down

Forum Software by Web Wiz Forums® version 12.08
Copyright ©2001-2026 Web Wiz Ltd.


Become a Fan on Facebook Follow us on X Connect with us on LinkedIn Web Wiz Blogs
About Web Wiz | Contact Web Wiz | Terms & Conditions | Cookies | Privacy Notice

Web Wiz is the trading name of Web Wiz Ltd. Company registration No. 05977755. Registered in England and Wales.
Registered office: Web Wiz Ltd, Unit 18, The Glenmore Centre, Fancy Road, Poole, Dorset, BH12 4FB, UK.

Prices exclude VAT at 20% unless otherwise stated. VAT No. GB988999105 - $, € prices shown as a guideline only.

Copyright ©2001-2026 Web Wiz Ltd. All rights reserved.