New function in robots.txt

Started by Kindred, May 01, 2008, 10:04:20 AM

Previous topic - Next topic

Kindred

Apparently Yahoo, Google and MSN now respect a new line in your robots.txt. This is especially important, because apparently Google has been having issues with some people's submissions to their webmaster tools.

This seems like a GREAT idea, since it takes some of the manual management out of the sitemap....


QuoteSince working with Google and Microsoft to support a single format for submission with sitemaps, we have continued to discuss further enhancements to make it easy for webmasters to get their content to all search engines quickly.
All search crawlers recognize robots.txt, so it seemed like a good idea to use that mechanism to allow webmasters to share their sitemaps. You agreed and encouraged us to allow robots.txt discovery of sitemaps on our suggestion board. We took the idea to Google and Microsoft and are happy to announce today that you can now find your sitemaps in a uniform way across all participating engines. To do this, simply add the following line to your robots.txt file:

sitemap: http://www.example.tld/sitemap.xml



More http://www.ysearchblog.com/archives/000437.html
Слaва
Украинi

Please do not PM, IM or Email me with support questions.  You will get better and faster responses in the support boards.  Thank you.

"Loki is not evil, although he is certainly not a force for good. Loki is... complicated."

karlbenson

I've had it in mine for months
http://www.youposted.com/robots.txt

But to be honest, I've not seen any noticeable difference from Spiders visiting it.

Mine was already pinged directly to Google/Yahoo/MSN and submitted via Google Webmaster & MSN Webmaster Live

HoTmetal

Quote from: karlbenson on May 01, 2008, 10:10:22 AM
I've had it in mine for months
http://www.youposted.com/robots.txt

But to be honest, I've not seen any noticeable difference from Spiders visiting it.

Mine was already pinged directly to Google/Yahoo/MSN and submitted via Google Webmaster & MSN Webmaster Live
Hmm intresting post Kindred. I like to keep up on SEO & other things of this nature.

@ KB, yeah... though to be fair, I don't see search engines making dramatic spikes in pagerank within established sites. It seems once they get a baseline for how many users/hits/content(added & kept)/links/etc...
they don't move you up or down too much unless the new spike leads to a new trend. Still though, if anyone else can post their results, I'm more than curious

Advertisement: