News:

Join the Facebook Fan Page.

Main Menu

Google spider question

Started by delta5, September 06, 2017, 04:23:02 PM

Previous topic - Next topic

delta5

I just recently activated the calendar feature on 2.0.14 and now I am getting 4 or 5 Google spiders hitting my site quite often and they are always looking at the calendar. Is this normal?

shawnb61

Short answer: yes.   A new feature or bunch of posts may trigger Google pouring over EVERY nook & cranny for a while.   If your calendar goes out 100 years, it is quite possibly starting off looking at 365*100 entries, and then attempting every conceivable link off of each...   

It is fractal & recursive in nature.  It will die down and return to normal in a few days or maybe a couple weeks on the outside. 

If this is a big problem (cpu usage?) you can govern Google's activity either through use of a robots.txt file:
    https://time.ly/document/user-guide/troubleshooting/limiting-excessive-google-crawls/

Or Google's Webmaster Tools:
    https://www.google.com/webmasters/tools/home?hl=en
Address the process rather than the outcome.  Then, the outcome becomes more likely.   - Fripp

delta5

I don't really care that Google is hitting my site, I need all the exposure that I can get. It just seemed odd that it took such an interest in the calendar feature. At least we know the forum is getting indexed properly.

Advertisement: