Web Wiz - Green Windows Web Hosting

  New Posts New Posts RSS Feed - getting bots to crawl the whole site
  FAQ FAQ  Forum Search   Events   Register Register  Login Login

Topic Closedgetting bots to crawl the whole site

 Post Reply Post Reply
Author
Steve B View Drop Down
Newbie
Newbie


Joined: 28 March 2005
Location: United Kingdom
Status: Offline
Points: 2
Direct Link To This Post Topic: getting bots to crawl the whole site
    Posted: 28 March 2005 at 3:42pm
I have a website which has the front end built around MAMBO and a seperate forums using phpBB.  I have submitted both the url of the main site and a seperate url of the forums to the search engines but I have noticed that the bots are only crawling the forum index and not the forum or indiviual posts.  I have seen many topics/posts listed in search engines that have been indexed out of other forums but can't seem to get them to crawl the whole of mine.

Has anyone any ideas as to how I can sort out this problem please?


Back to Top
WebWiz-Bruce View Drop Down
Admin Group
Admin Group
Avatar
Web Wiz Developer

Joined: 03 September 2001
Location: Bournemouth
Status: Offline
Points: 9844
Direct Link To This Post Posted: 29 March 2005 at 4:09am
Maybe this is a problem you need to post on MAMBO or phpBB sites as they products competing against the software from this site!!

Now, if you had used the forum from written by and produced by this site, Web Wiz Forums, then it would have all pages indexed by search engines.
Back to Top
huwnet View Drop Down
Senior Member
Senior Member


Joined: 30 May 2003
Location: England
Status: Offline
Points: 1375
Direct Link To This Post Posted: 31 March 2005 at 5:24pm
You need to see the info on the phpbb faq about rewriting session ids for bots.

Or you could just wait for 3.x which will have it built in
Back to Top
Steve B View Drop Down
Newbie
Newbie


Joined: 28 March 2005
Location: United Kingdom
Status: Offline
Points: 2
Direct Link To This Post Posted: 02 April 2005 at 7:20pm

Thanks huwnet for the helpful reply.  After some more research I have managed to find a mod/hack that was written to address this issue.  This was implemented earlier in the week and seems to be working now.

boRg- I am sorry that my site is based on a competitor's software but would like to say that the whole of www.thephotographyforums.com has been based on software that was provided by the hosting company that we use, it wasn't chosen for any other reason.  I would not begin to know the benefits or disadvantages of either system, neither do I want to start sl*ging off either.  My site has had to much work done to it at this point for me to consider changing.

I found this site whilst looking for web promotion advice and find the free tools here very useful, from there I stumbled into your forums and posted for advice as I liked the general tone of the place, I didn't mean to cause any trouble.

Thanks again.


Back to Top
 Post Reply Post Reply

Forum Jump Forum Permissions View Drop Down

Forum Software by Web Wiz Forums® version 12.08
Copyright ©2001-2026 Web Wiz Ltd.


Become a Fan on Facebook Follow us on X Connect with us on LinkedIn Web Wiz Blogs
About Web Wiz | Contact Web Wiz | Terms & Conditions | Cookies | Privacy Notice

Web Wiz is the trading name of Web Wiz Ltd. Company registration No. 05977755. Registered in England and Wales.
Registered office: Web Wiz Ltd, Unit 18, The Glenmore Centre, Fancy Road, Poole, Dorset, BH12 4FB, UK.

Prices exclude VAT at 20% unless otherwise stated. VAT No. GB988999105 - $, € prices shown as a guideline only.

Copyright ©2001-2026 Web Wiz Ltd. All rights reserved.