News:

Bored?  Looking to kill some time?  Want to chat with other SMF users?  Join us in IRC chat or Discord

Main Menu

How to reduce the number of google spiders...?

Started by CyprusGrump, January 25, 2013, 06:23:05 AM

Previous topic - Next topic

CyprusGrump

Hi,

I'm in the process of trying to reduce the number of hits my site receives as the traffic has been very high recently.

I've got bad behavior running and that is working well...

But at the moment I am overwhelmed with 'visitors' to the site - when I check the IP address it resolves to google.

Keen as I am to have Google crawling the site, is there a simple way to restrict the numbers...?

Thanks in advance for any advice...
I suppose I should put something in my signature shouldn't I?

Ricky.

You can use a good  robot.txt to disallow unwanted area on your site, there was a thread for that , try searching for it. It will eventually decrease amount of data google need to index. Secondly, read following:
http://support.google.com/webmasters/bin/answer.py?hl=en&answer=48620

CyprusGrump

Quote from: Ricky. on January 25, 2013, 06:29:22 AM
You can use a good  robot.txt to disallow unwanted area on your site, there was a thread for that , try searching for it. It will eventually decrease amount of data google need to index. Secondly, read following:
http://support.google.com/webmasters/bin/answer.py?hl=en&answer=48620

Thanks!
I suppose I should put something in my signature shouldn't I?

Advertisement: