Hello All,
I am having problem with google they send me this message about googlebot sucking my bandwith.
The end result was their Googlebot doesn’t handle sessions well:
"From the log snippet you provided, we can see that your site is using session IDs. As you’ve observed, session IDs can cause problems for our robots. Please disable session IDs for Googlebot so that our robots may crawl your site more efficiently."
can you provide me how forum doesn’t create a session for the Googlebot.?
I have found this script to block googlebot with session ID, and one of website saying this;
Im thinking a solution could be the following:
In the Global.asp file, create an IF, THEN, ELSE statement that says something about If googlebot. THEN don’t index sessions ELSE index the rest of the pages…
Can anyone make this MOD for us, it will really appriciated...
[QUOTE=PHP Code]
$isspider = is_spider();
if ($isspider != 'init') {
//turn off session id for spiders
ini_set(’session.use_trans_sid’, false);
}
function is_spider()
{
// don't know if this is a spider or not yet
$spiders = array(
"almaden.ibm.com" ,
"gigabot" ,
"appie 1.1" ,
"architext" ,
"ask jeeves",
"asterias2.0",
"augurfind" ,
"baiduspider" ,
"bannana_bot" ,
"bdcindexer" ,
"crawler" ,
"crawler@fast" ,
"docomo" ,
"fast-webcrawler",
"fluffy the spider",
"frooglebot",
"geobot",
"googlebot",
"mediapartners-google",
"gulliver",
"henrythemiragorobot",
"ia_archiver",
"infoseek",
"kit_fireball",
"lachesis",
"lycos_spider",
"mantraagent",
"mercator",
"msnbot",
"moget" ,
"muscatferret" ,
"nationaldirectory-webspider" ,
"naverrobot" ,
"ncsa beta" ,
"netresearchserver" ,
"ng/1.0" ,
"osis-project" ,
"polybot" ,
"pompos" ,
"scooter" ,
"seventwentyfour" ,
"sidewinder" ,
"sleek spider" ,
"slurp" ,
"steeler" ,
"szukacz" ,
"t-h-u-n-d-e-r-s-t-o-n-e" ,
"teoma" ,
"turnitinbot" ,