The 10-Page Torture Test
June 12, 2025, 01:46 PM *
Welcome, Guest. Please login or register.

Login with username, password and session length
News: Read: Screenwriting News from around the web (live)
 
   Home   Help Search Chat Login Register  
Pages: [1]   To Page Bottom
  Print  
Author Topic: Googlebot loves us  (Read 2457 times)
0 Members and 1 Guest are viewing this topic.
Pitchpatch
Rollercoaster on fire
Administrator
Mugwump
*****
Offline Offline

Posts: 757



« on: March 19, 2011, 04:02 AM »

... and is HAMMERING the site.  I don't want to block googlebot entirely from indexing the site, but it's artificially inflating the hit stats to a ridiculous level.  No obvious was to tame it without banning it via robots.txt or IP range.

I suppose I could regularly switch robots.txt on and off for short periods throughout the month...
Logged

NTSF:SD:SUV::
Pitchpatch
Rollercoaster on fire
Administrator
Mugwump
*****
Offline Offline

Posts: 757



« Reply #1 on: March 19, 2011, 04:30 AM »

Apparently SMF2 has better native handling of search bots.  When SMF2 goes gold I'll look at upgrading.
Logged

NTSF:SD:SUV::
Pitchpatch
Rollercoaster on fire
Administrator
Mugwump
*****
Offline Offline

Posts: 757



« Reply #2 on: April 01, 2011, 12:40 PM »

The downside of Google hammering us is the inflated pageview stats.

The upside is, Google's indexing the crap out of this place.  http://www.google.com/search?hl=&q=site%3A10ptt.com
« Last Edit: April 01, 2011, 12:42 PM by Pitchpatch » Logged

NTSF:SD:SUV::
Pitchpatch
Rollercoaster on fire
Administrator
Mugwump
*****
Offline Offline

Posts: 757



« Reply #3 on: May 10, 2011, 02:14 PM »

A change I made to the way hits get counted means search bots will no longer make an impression on stats.  They'll still access and index the site as normal, but now we'll see a truer representation of http://visits per day.


* Clipboard-1.jpg (32.29 KB, 700x224 - viewed 479 times.)
« Last Edit: May 10, 2011, 02:17 PM by Pitchpatch » Logged

NTSF:SD:SUV::
Tom & Jerry
Cat & Mouse
The True Black Meat
**
Offline Offline

Posts: 227



WWW
« Reply #4 on: May 10, 2011, 08:17 PM »

Cool story, Hansel.
« Last Edit: May 10, 2011, 08:22 PM by Tom & Jerry » Logged

Am I making sense?
Pitchpatch
Rollercoaster on fire
Administrator
Mugwump
*****
Offline Offline

Posts: 757



« Reply #5 on: June 28, 2011, 01:50 PM »

A change I made to the way hits get counted means search bots will no longer make an impression on stats.  They'll still access and index the site as normal, but now we'll see a truer representation of visits per day.

Turns out that code change mentioned above only stopped bots increasing individual topic view counts within the forums.  It didn't affect hit counts recorded in the overall stats.

Made a change today that I hope will stop most bots from increasing the hit count.  We'll see if the daily hit total goes down from here on.
Logged

NTSF:SD:SUV::
Pages: [1]   Back To Top
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF | SMF © Simple Machines Valid XHTML 1.0! Valid CSS!
Page created in 0.024 secs [18]