|
Title: Googlebot loves us Post by: Pitchpatch on March 19, 2011, 04:02 AM ... and is HAMMERING the site. I don't want to block googlebot entirely from indexing the site, but it's artificially inflating the hit stats to a ridiculous level. No obvious was to tame it without banning it via robots.txt or IP range.
I suppose I could regularly switch robots.txt on and off for short periods throughout the month... Title: Re: Googlebot loves us Post by: Pitchpatch on March 19, 2011, 04:30 AM Apparently SMF2 has better native handling of search bots. When SMF2 goes gold I'll look at upgrading.
Title: Re: Googlebot loves us Post by: Pitchpatch on April 01, 2011, 12:40 PM The downside of Google hammering us is the inflated pageview stats.
The upside is, Google's indexing the crap out of this place. http://www.google.com/search?hl=&q=site%3A10ptt.com (http://www.google.com/search?hl=&q=site%3A10ptt.com) Title: Bots no longer artificially bump stats Post by: Pitchpatch on May 10, 2011, 02:14 PM A change I made to the way hits get counted means search bots will no longer make an impression on stats. They'll still access and index the site as normal, but now we'll see a truer representation of http://visits per day (http://10ptt.com/smf/index.php?action=stats).
Title: Re: Googlebot loves us Post by: Tom & Jerry on May 10, 2011, 08:17 PM Cool story, Hansel.
Title: Re: Bots no longer artificially bump stats Post by: Pitchpatch on June 28, 2011, 01:50 PM A change I made to the way hits get counted means search bots will no longer make an impression on stats. They'll still access and index the site as normal, but now we'll see a truer representation of visits per day. Turns out that code change mentioned above only stopped bots increasing individual topic view counts within the forums. It didn't affect hit counts recorded in the overall stats. Made a change today that I hope will stop most bots from increasing the hit count. We'll see if the daily hit total goes down from here on. |