Customizing SMF > Building Your Community and other Forum Advice

After "friendly URLs" should I exclude the old urls in robots.txt?

(1/1)

DaKrampus:
Since more than 6 months I have activated Search engine friendly URLs
But the old urls (i.e. index.php?topic=1100.0) are still in the google index. (nearly all of them, allthough the real urls are index.php/topic,1100.0.html)
to avoid duplicate content penalisation, should i add this line to my robots.txt?

--- Code: ---Disallow: /index.php?board=*
Disallow: /index.php?topic=*
--- End code ---
or will this remove the pages from the index without reindexing the new ones?
Help will be very appreciated
Da

Arantor:
You actually shouldn't have to - all SMF pages include a canonical URL declaration, telling search engines to use the other format.

Mind you, the 'search engine friendly' URL style is from years and years ago when they couldn't properly handle dynamic content, and is not necessary at all now. You could even turn it off and go back to how Google has it and not see any problems.

DaKrampus:
Thanx for the reply.
Actually google analytics tells me we have about 1200 links back from other sites, most of them being the new urls. so i wont switch it off..

I noticed we are sliding down in google. We used to be second or third on main keyword after wikipedia, we went down on place 8 or 9 since a month.. so I started wondering a bit.
i started adding a [page 1] [page 2] etc.. to all topic titles and descriptions having more than 1 page... but i wasnt about about the url.

I hope its not the reason.

Da

Navigation

[0] Message Index