SMF Support > SMF 2.1.x Support

Calendar

(1/1)

Bigguy:
Is there anyway to get spiders to stop indexing the calendar. I was searching on Google the other day for 2.1 sites and noticed there are a lot of calendars from 2.1 showing up. I would like mine not to show up in google. Any thoughts.

EDIT: It's not just google either. I went to try to make a sitemap the other day and the generator I used went to the calendar and took so long I stopped making it. It was just not worth it if it was going to take that long to generate a site map.

ziycon:
You can use robots.txt with something like below.


--- Code: ---Disallow: /*?action=calendar
--- End code ---

You'll have to wait until the bot in question downloads your updated robots.txt file as some bots don't read them every time they crawl a site.

Bigguy:
I will put that in my robot.txt file for sure. Is that the only part of the link I need to use. ?? Is there not a place in Google webmaster tools to add a link you do not want crawled as well. ??

ziycon:

--- Quote from: Bigguy on March 07, 2018, 06:57:20 PM ---I will put that in my robot.txt file for sure. Is that the only part of the link I need to use. ?? Is there not a place in Google webmaster tools to add a link you do not want crawled as well. ??

--- End quote ---
That should be all you need, see how it goes. The URL wont be indexed but it will still be followed.

You can use the 'Remove URLs Tool' but it only lasts for 90 days.

Bigguy:
Thank you sir. I will try this later tonight. I will not take up anymore of your time. We can mark this solved for now. :)

Navigation

[0] Message Index

Go to full version