News:

Wondering if this will always be free?  See why free is better.

Main Menu

Calendar

Started by Bigguy, March 07, 2018, 06:49:24 PM

Previous topic - Next topic

Bigguy

Is there anyway to get spiders to stop indexing the calendar. I was searching on Google the other day for 2.1 sites and noticed there are a lot of calendars from 2.1 showing up. I would like mine not to show up in google. Any thoughts.

EDIT: It's not just google either. I went to try to make a sitemap the other day and the generator I used went to the calendar and took so long I stopped making it. It was just not worth it if it was going to take that long to generate a site map.

ziycon

You can use robots.txt with something like below.

Disallow: /*?action=calendar

You'll have to wait until the bot in question downloads your updated robots.txt file as some bots don't read them every time they crawl a site.

Bigguy

I will put that in my robot.txt file for sure. Is that the only part of the link I need to use. ?? Is there not a place in Google webmaster tools to add a link you do not want crawled as well. ??

ziycon

Quote from: Bigguy on March 07, 2018, 06:57:20 PM
I will put that in my robot.txt file for sure. Is that the only part of the link I need to use. ?? Is there not a place in Google webmaster tools to add a link you do not want crawled as well. ??
That should be all you need, see how it goes. The URL wont be indexed but it will still be followed.

You can use the 'Remove URLs Tool' but it only lasts for 90 days.

Bigguy

Thank you sir. I will try this later tonight. I will not take up anymore of your time. We can mark this solved for now. :)

Advertisement: