Dynamic URLs vs. static URLs - Google Webmaster Central Blog

[青山 素子]

Hopefully this will help with those people interested in URL re-writing to "friendly" style URLs. The article below is from the Official Google Webmaster Central Blog.

If you are already using "friendly" URLs, you'll probably want to keep them or risk a great deal of loss in link equity, but for those considering a switch, this article sends a clear message:

Google recommends NOT rewriting dynamic URLs to static ones - leave them dynamic



Dynamic URLs vs. static URLs
Juliane Stiller and Kaspar Szymanski, Google Search Quality Team
Monday, September 22, 2008

Chatting with webmasters often reveals widespread beliefs that might have been accurate in the past, but are not necessarily up-to-date any more. This was the case when we recently talked to a couple of friends about the structure of a URL. One friend was concerned about using dynamic URLs, since (as she told us) "search engines can't cope with these." Another friend thought that dynamic URLs weren't a problem at all for search engines and that these issues were a thing of the past. One even admitted that he never understood the fuss about dynamic URLs in comparison to static URLs. For us, that was the moment we decided to read up on the topic of dynamic and static URLs. First, let's clarify what we're talking about:

What is a static URL?
A static URL is one that does not change, so it typically does not contain any url parameters. It can look like this: http://www.example.com/archive/january.htm. You can search for static URLs on Google by typing filetype:htm in the search field. Updating these kinds of pages can be time consuming, especially if the amount of information grows quickly, since every single page has to be hard-coded. This is why webmasters who deal with large, frequently updated sites like online shops, forum communities, blogs or content management systems may use dynamic URLs.

What is a dynamic URL?
If the content of a site is stored in a database and pulled for display on pages on demand, dynamic URLs maybe used. In that case the site serves basically as a template for the content. Usually, a dynamic URL would look something like this: http://code.google.com/p/google-checkout-php-sample-code/issues/detail?id=31. You can spot dynamic URLs by looking for characters like: ? = &. Dynamic URLs have the disadvantage that different URLs can have the same content. So different users might link to URLs with different parameters which have the same content. That's one reason why webmasters sometimes want to rewrite their URLs to static ones.

Should I try to make my dynamic URLs look static?
Following are some key points you should keep in mind while dealing with dynamic URLs:

• It's quite hard to correctly create and maintain rewrites that change dynamic URLs to static-looking URLs.
• It's much safer to serve us the original dynamic URL and let us handle the problem of detecting and avoiding problematic parameters.
• If you want to rewrite your URL, please remove unnecessary parameters while maintaining a dynamic-looking URL.
• If you want to serve a static URL instead of a dynamic URL you should create a static equivalent of your content.

Which can Googlebot read better, static or dynamic URLs?
We've come across many webmasters who, like our friend, believed that static or static-looking URLs were an advantage for indexing and ranking their sites. This is based on the presumption that search engines have issues with crawling and analyzing URLs that include session IDs or source trackers. However, as a matter of fact, we at Google have made some progress in both areas. While static URLs might have a slight advantage in terms of clickthrough rates because users can easily read the urls, the decision to use database-driven websites does not imply a significant disadvantage in terms of indexing and ranking. Providing search engines with dynamic URLs should be favored over hiding parameters to make them look static.

Let's now look at some of the widespread beliefs concerning dynamic URLs and correct some of the assumptions which spook webmasters. :)

Myth: "Dynamic URLs cannot be crawled."
Fact: We can crawl dynamic URLs and interpret the different parameters. We might have problems crawling and ranking your dynamic URLs if you try to make your urls look static and in the process hide parameters which offer the Googlebot valuable information. One recommendation is to avoid reformatting a dynamic URL to make it look static. It's always advisable to use static content with static URLs as much as possible, but in cases where you decide to use dynamic content, you should give us the possibility to analyze your URL structure and not remove information by hiding parameters and making them look static.

Myth: "Dynamic URLs are okay if you use fewer than three parameters."
Fact: There is no limit on the number of parameters, but a good rule of thumb would be to keep your URLs short (this applies to all URLs, whether static or dynamic). You may be able to remove some parameters which aren't essential for Googlebot and offer your users a nice looking dynamic URL. If you are not able to figure out which parameters to remove, we'd advise you to serve us all the parameters in your dynamic URL and our system will figure out which ones do not matter. Hiding your parameters keeps us from analyzing your URLs properly and we won't be able to recognize the parameters as such, which could cause a loss of valuable information.

Following are some questions we thought you might have at this point.

Does that mean I should avoid rewriting dynamic URLs at all?
That's our recommendation, unless your rewrites are limited to removing unnecessary parameters, or you are very diligent in removing all parameters that could cause problems. If you transform your dynamic URL to make it look static you should be aware that we might not be able to interpret the information correctly in all cases. If you want to serve a static equivalent of your site, you might want to consider transforming the underlying content by serving a replacement which is truly static. One example would be to generate files for all the paths and make them accessible somewhere on your site. However, if you're using URL rewriting (rather than making a copy of the content) to produce static-looking URLs from a dynamic site, you could be doing harm rather than good. Feel free to serve us your standard dynamic URL and we will automatically find the parameters which are unnecessary.

Can you give me an example?
If you have a dynamic URL which is in the standard format like foo?key1=value&key2=value2 we recommend that you leave the url unchanged, and Google will determine which parameters can be removed; or you could remove uncessary parameters for your users. Be careful that you only remove parameters which do not matter. Here's an example of a URL with a couple of parameters:

www.example.com/article/bin/answer.foo?language=en&answer=3&sid=98971298178906&query=URL

• language=en - indicates the language of the article
• answer=3 - the article has the number 3
• sid=8971298178906 - the session ID number is 8971298178906
• query=URL - the query with which the article was found is [url]

Not all of these parameters offer additional information. So rewriting the URL to www.example.com/article/bin/answer.foo?language=en&answer=3 probably would not cause any problems as all irrelevant parameters are removed.

The following are some examples of static-looking URLs which may cause more crawling problems than serving the dynamic URL without rewriting:

• www.example.com/article/bin/answer.foo/en/3/98971298178906/URL
• www.example.com/article/bin/answer.foo/language=en/answer=3/sid=98971298178906/query=URL
• www.example.com/article/bin/answer.foo/language/en/answer/3/sid/98971298178906/query/URL
• www.example.com/article/bin/answer.foo/en,3,98971298178906,URL

Rewriting your dynamic URL to one of these examples could cause us to crawl the same piece of content needlessly via many different URLs with varying values for session IDs (sid) and query. These forms make it difficult for us to understand that URL and 98971298178906 have nothing to do with the actual content which is returned via this URL. However, here's an example of a rewrite where all irrelevant parameters have been removed:

• www.example.com/article/bin/answer.foo/en/3

Although we are able to process this URL correctly, we would still discourage you from using this rewrite as it is hard to maintain and needs to be updated as soon as a new parameter is added to the original dynamic URL. Failure to do this would again result in a static looking URL which is hiding parameters. So the best solution is often to keep your dynamic URLs as they are. Or, if you remove irrelevant parameters, bear in mind to leave the URL dynamic as the above example of a rewritten URL shows:

• www.example.com/article/bin/answer.foo?language=en&answer=3

We hope this article is helpful to you and our friends to shed some light on the various assumptions around dynamic URLs. Please feel free to join our discussion group if you have any further questions.



Thanks to minstrel for the find. Red highlights are his.

Edit: Cleaned up the formatting to match the original post.

[karlbenson]

thanks for posting.

*bookmarked for linking people to at a later date

[Bancherd]

Thanks 

[AlphaHot1]

The fact that pretty urls are USER friendly besides being search engines friendly still remains.
And in the last example:

* www.example.com/article/bin/answer.foo/en/3

Although we are able to process this URL correctly, we would still discourage you from using this rewrite as it is hard to maintain and needs to be updated as soon as a new parameter is added to the original dynamic URL.

he simply states that the url si good but hard to maintain. And this is not always tha case obviously.

[青山 素子]

As I've said before, the people convinced this works wouldn't believe even those running the engines telling them it doesn't.

Of course, I rather trust that those who work daily on the search engines know what they are talking about. If they say that rewriting isn't helpful and can be bad, I'll trust they are speaking with knowledge that allows them to say that. It also helps when several people have done proper studies in the past showing the exact same thing.

As for the user-friendly part, I'm rather unconvinced. Take this topic for example. Would it be easier to remember:

hxxp://www.simplemachines.org/community/index.php?topic=263753

or

hxxp://www.simplemachines.org/community/SMF_Development/SMF_Feedback_and_Discussion/Dynamic_URLs_vs._static-URLs_-_Google_Webmaster_Central_Blog.html

?

I posit that remembering a six-digit number would be much easier than remembering that obscenely long text "friendly" URL.

Also, what if the topic moves? That would be worthless. The option would be to include some kind of marker to let the rewrites continue to function. Perhaps "Dynamic_URLs_vs._static-URLs_-_Google_Webmaster_Central_Blog-t-263753.html", but that defeats the purpose of making it look nicer.

[Charles Hill]

I would have to agree with rewriting URLs being incredibly difficult to maintain and fraught with problems.  As for what Motoko-chan said about trusting that "those who work daily on the search engines know what they are talking about".  I completely agree.

[karlbenson]

Its definately personal preference.

I think full seo urls are ugly.
hxxp://www.simplemachines.org/community/SMF_Development/SMF_Feedback_and_Discussion/Dynamic_URLs_vs._static-URLs_-_Google_Webmaster_Central_Blog.html

However if you use the method correctly when designing a script, it can make sense for ebay for example to do
ebay.com/cars
(as this is memorable and user friendly)
but not
ebay.com/cars-and-other-automobile-parts-and-another-url-keyword-stuffing-techniques.

For this reason I would really love to see Search Engines penalise using too many keywords or variables in the urls. (whether dynamic or static)

[matasanos]

good info

thanks!

[青山 素子]

For this reason I would really love to see Search Engines penalise using too many keywords or variables in the urls. (whether dynamic or static)

At least one study (done by WebCEO a year or two ago) determined that using highly-competitive keywords in the URL actually incurred a small penalty from Google at least.

[brianjw]

I have a question. Does this include the default feature in SMF "Search Engine Friendly URLs"?

I just uninstalled Pretty URLs thanks for this topic. However, will URLs like http://www.gamerzgarage.com/index.php/topic,276.html receive problems from search engines or is this one that google doesn't recommend?

[karlbenson]

SEF urls (default in smf). Shouldn't be much different than normal smf urls.

The only thing I would say though, once you decide on what type of urls to use, you should stick with them unless your prepared for negative seo in the short term (3-6 months) while search engines figure out your new url structure and replace old pages with new pages. (although the amount of negative seo, can be minimized).

[青山 素子]

I have a question. Does this include the default feature in SMF "Search Engine Friendly URLs"?

Yes, they count as well because they fake "static" URLs.

Agreed with Karl. If you are already using rewriting, you are stuck with it unless you want to go through re-indexing and not being ranked properly for several months. Also, all external links and bookmarks to topics will break.

[hartiberlin]

.

As for the user-friendly part, I'm rather unconvinced. Take this topic for example. Would it be easier to remember:

hxxp://www.simplemachines.org/community/index.php?topic=263753

or

hxxp://www.simplemachines.org/community/SMF_Development/SMF_Feedback_and_Discussion/Dynamic_URLs_vs._static-URLs_-_Google_Webmaster_Central_Blog.html

?

I posit that remembering a six-digit number would be much easier than remembering that obscenely long text "friendly" URL.

Hmm,
but what about if I give into the search field in Google:
static-URLs

Will then not the page be easier found when this page is indexed as
hxxp://www.simplemachines.org/community/SMF_Development/SMF_Feedback_and_Discussion/Dynamic_URLs_vs._static-URLs_-_Google_Webmaster_Central_Blog.html

as instead of:

hxxp://www.simplemachines.org/community/index.php?topic=263753


??


As there is the word:
static-URLs
inside the link, it will be listed much higher , maybe on the first page
on Google as Google favours domain names and URL names much
higher...
I have tested this myself already and with my domain
overunity.com
I am on location1 when you just type in
overunity
in the
Google search form...

So URL names really matter. Especially if you have keywords in them
which not many other sites have then you are ranked pretty high on
the search results...

But for instance, I wanted to be on page 1 with the
keyword
free energy

but only sites with the word
energy
or
freeenergy
inside the URL are listed on page one.
So you see, I can do what I want,
if I don´t have the word energy in my
URL name, I will not be listed on page 1 on Google,
although my content on my overunity.com
has very many topics, where the word free energy
will be displayed...

So Google always prefers URLs where the keywords are in there..
If you don´t have the keyword in your URL you have no chance to
get to page 1 on the search results...

Regards, Stefan.

[hartiberlin]

Agreed with Karl. If you are already using rewriting, you are stuck with it unless you want to go through re-indexing and not being ranked properly for several months. Also, all external links and bookmarks to topics will break.

Are you sure from your own experience ?

I had blocked the Google bot on my overunity.com
domain for about 3 weeks, cause it made too much traffic.

Then my site overunity.com
was removed from the Google index.
No more there.

After letting The GoogleBot  in again, I was again on location 1 on the
keyword:
overunity
after 3 or 4 days.

Also after changing a few links on my homepage I was relocated from page 4 to page 2
after just 3 or 4 days...
So it seems it is really fast crawling and rearranging the index nowadays.

Also the Googlebot always crawls my forum all day long,
also if I put the Revisit Meta Tag to 21 days.

Maybe it is because I have Google Adsense ads on my pages... I don´t know,
if this plays a major role, but it always crawls my site..


So I am still considdering to use SEO4SMF or Pretty URLs to get more keyword
related pages into the Google search results.

Regards, Stefan.

[brianjw]

So, then I will just leave my SMF URLs at the SMF default.

[Blind Bandit]

Ok so I have a question.

Would this mean that using SMF SEO mods (there are several) are a bad idea?

It seems like pretty URLs is out...

[青山 素子]

Hmm,
but what about if I give into the search field in Google:
static-URLs

Will then not the page be easier found when this page is indexed as
hxxp://www.simplemachines.org/community/SMF_Development/SMF_Feedback_and_Discussion/Dynamic_URLs_vs._static-URLs_-_Google_Webmaster_Central_Blog.html

as instead of:

hxxp://www.simplemachines.org/community/index.php?topic=263753

Nope. Google doesn't count URL text in the indexing. If that phrase appears in the content of the page, or in links to that page, it will likely show up in the results.

As there is the word:
static-URLs
inside the link, it will be listed much higher , maybe on the first page
on Google as Google favours domain names and URL names much
higher...
I have tested this myself already and with my domain
overunity.com
I am on location1 when you just type in
overunity
in the
Google search form...

So URL names really matter. Especially if you have keywords in them
which not many other sites have then you are ranked pretty high on
the search results...

Considering your domain is the keyword and it seems to be unique, that would be the reason for the ranking. Keywords in the domain name help, but not in the actual path.

But for instance, I wanted to be on page 1 with the
keyword
free energy

but only sites with the word
energy
or
freeenergy
inside the URL are listed on page one.
So you see, I can do what I want,
if I don´t have the word energy in my
URL name, I will not be listed on page 1 on Google,
although my content on my overunity.com
has very many topics, where the word free energy
will be displayed...

Doing a quick look, Wikipedia is #1. Typical since that domain is heavily linked, and has that phrase on the page eight times. The next two results have those words in their domain. The second result also has fifteen instances of the phrase on the page. The third has it five times. After that, it seems to be only for content or backlinks (#4 has the phrase in the title tags and also on the page content).

If you actually look, keyword-rich URLs have no effect on the rankings.

So Google always prefers URLs where the keywords are in there..
If you don´t have the keyword in your URL you have no chance to
get to page 1 on the search results...

Please provide me a proper study showing this. I find it very difficult to believe, especially when Google themselves say rewriting URLs is potentially harmful and when several studies have repeatedly said that keyword-rich URLs have almost no to negative effect.

Agreed with Karl. If you are already using rewriting, you are stuck with it unless you want to go through re-indexing and not being ranked properly for several months. Also, all external links and bookmarks to topics will break.

Are you sure from your own experience ?

Yes, I have had to do site restructuring and the rankings have issues for some time after until the indexes get refreshed. It's much easier with poorly-indexed sites than well-established ones.

After letting The GoogleBot  in again, I was again on location 1 on the
keyword:
overunity
after 3 or 4 days.

Yeah, it's in your domain name. I'm not surprised.

Also after changing a few links on my homepage I was relocated from page 4 to page 2
after just 3 or 4 days...
So it seems it is really fast crawling and rearranging the index nowadays.

What pages were relocated? The ones you changed? Did you see if the older page URLs were still getting hits?

Maybe it is because I have Google Adsense ads on my pages... I don´t know,
if this plays a major role, but it always crawls my site..

This seems to have a small effect on speeding crawlings. It can also account for why you see hits from Google all-day long. It's for serving ads with good context.

So I am still considdering to use SEO4SMF or Pretty URLs to get more keyword
related pages into the Google search results.

Do what you will, but don't be misinformed.

Also, please don't double-post. Edit your original, that's what the function is there for.

So, then I will just leave my SMF URLs at the SMF default.

That's what I'd do. There are much more effective ways to get better indexing than messing with your URLs.

Would this mean that using SMF SEO mods (there are several) are a bad idea?

If they rewrite the URLs, they are pointless and add extra overhead for no SEO benefit. There are plenty of other ways to help with indexing.

[hartiberlin]

@Motoko-chan
so do you think nowadays it is better not to install Pretty URLs or SEO4SMF
and just leave the default SMF URL settings and
just only use the Sitemap mod to help to index the whole forum
for inclusion in
Google Webmastertools  ?

[青山 素子]

For the past several years it has been best to leave dynamic URLs alone and concentrate on the important things of quality content and backlinks, along with some good HTML design.

You also don't need a sitemap, and it won't help your ranking if you do use it. All a sitemap does is helps to ensure the URLs you wish to be indexed are known. It's handy in many situations, but also not essential. It's an indexing helper, not a ranking helper.

[metallica48423]

i'll take this opportunity, just as fair warning, to note that there is a security bug in SEO4SMF which has not been patched by its developer.

Sorry for taking it off topic

[AlphaHot1]

As I've said before, the people convinced this works wouldn't believe even those running the engines telling them it doesn't. Of course, I rather trust that those who work daily on the search engines know what they are talking about. If they say that rewriting isn't helpful and can be bad, I'll trust they are speaking with knowledge that allows them to say that.

The only reason for search engines owners to tell you to do something is to ensure that people will write websites with legit contents and not try to trick the ranking algorithm. This doesn't mean necessarily that what they say will benefit your ranking.
Google says to not cloak sites, but we all know that you can gain rank with it.

So, I see the point and I agree to some extent, but I wouldn't take everything google blog says for granted.

After all they only care about one single thing: the relevancy of their search results.

It also helps when several people have done proper studies in the past showing the exact same thing.

Oh I don't think you can show data for that studies, there are too many variables in place, and google algorithms changed quite a lot in the past. The only one who can make a proper study is google or someone paid by google to make that study. But if it's so we are in case number one: they will show you what's they want to show you.
We just can guess or have empiric data.
Even google says to not invest money in SEO.

As for the user-friendly part, I'm rather unconvinced. Take this topic for example. Would it be easier to remember:

hxxp://www.simplemachines.org/community/index.php?topic=263753

or

hxxp://www.simplemachines.org/community/SMF_Development/SMF_Feedback_and_Discussion/Dynamic_URLs_vs._static-URLs_-_Google_Webmaster_Central_Blog.html

I posit that remembering a six-digit number would be much easier than remembering that obscenely long text "friendly" URL.

It should be rewritten to something like that:
http://www.simplemachines.org/Dynamic-URLs-vs.-static-URLs-Google-Webmaster-Central-Blog

and the point is not to remember the url, something that nobody does anyway, but to show a descriptive link.

Also, what if the topic moves? That would be worthless.

See this document: http://www.w3.org/Provider/Style/URI  "cool URIs don't change"

I'm not even saying that friendly URLs will make a site to rank higher but as the article says:

"So different users might link to URLs with different parameters which have the same content. That's one reason why webmasters sometimes want to rewrite their URLs to static ones.
That's one reason.

"static URLs might have a slight advantage in terms of clickthrough rates because users can easily read the urls"
That's another reason.

Then he continues:
"the decision to use database-driven websites does not imply a significant disadvantage in terms of indexing and ranking."
Ok.

But:
"Providing search engines with dynamic URLs should be favored over hiding parameters to make them look static."
this is in no way a logical conclusion of what has been said before.

The only thing I got from this article is that google has a new algorithm to index dinamyc links, and suddenly wants everyone to ditch static ones so they can test their new algoithm

[青山 素子]

It also helps when several people have done proper studies in the past showing the exact same thing.

Oh I don't think you can show data for that studies, there are too many variables in place, and google algorithms changed quite a lot in the past. The only one who can make a proper study is google or someone paid by google to make that study. But if it's so we are in case number one: they will show you what's they want to show you.
We just can guess or have empiric data.
Even google says to not invest money in SEO.

I know of a few studies that used hundreds of data points to determine the behavior of things. Even if it is the case that things have changed and there are constant changes, I can't point to any kind of change that loosens restrictions. At worst, the effect of a "hot" technique will be tuned towards neutral.

It should be rewritten to something like that:
http://www.simplemachines.org/Dynamic-URLs-vs.-static-URLs-Google-Webmaster-Central-Blog

and the point is not to remember the url, something that nobody does anyway, but to show a descriptive link.

And how does that help SEO in any way if the URL is ignored?

Also, what if the topic moves? That would be worthless.

See this document: http://www.w3.org/Provider/Style/URI  "cool URIs don't change"

Yep. This why even dynamic URLs like index.php?topic=263753 are good. If you spell out the board name (as SEO4SMF does at least), what happens when the topic moves to a new board? Do we remove the old link? Do we keep it working but 301 it? Do we just show the same content (which causes big search engine issues and possible duplicate content issues)?

The only thing I got from this article is that google has a new algorithm to index dinamyc links, and suddenly wants everyone to ditch static ones so they can test their new algoithm

Last I knew, Google has been reliably indexing dynamic URLs for years. Since the whole "rewriting" thing is a hot topic and a big problem if done badly, I see the article as a timely notice that the SEO "experts" are wrong in preaching conversion to static-looking URLs to "help indexing".

[Dannii]

I just uninstalled Pretty URLs thanks for this topic.

I don't recommend that at all. From the points mentioned in this topic I think the only one that is possibly relevant is that my mod can produce rather long URLs... though if that's a problem it can be changed.

If you uninstall it make sure you install the reverter package!

Please do explain specific problems with my mod if you know any...

Now Motoko-chan, please stop saying they're pointless. You know they're not.

Here are some reasons why I think textual URLs are useful:

• If you just add a link to a forum by posting a plain URL the URL's text will be used as it's link text, and so keywords could help
  
• Many people don't like clicking random links sent to them via IRC or IM, so have a textual URL will give them an idea of what the page is, and they might be more likely to open it
  
• Textual URLs are definitely easier to remember with FF3's awesomebar - you may not remember the whole URL, but you don't need to. It would work just as well without because the page title is the same... but if a topic title is changed it wouldn't anymore.
  
• Depending on what you're doing with your site, there's the possibility of changing software without changing URLs. I plan to move my blog from SMF to DokuWiki, and because I use pretty URLs I can recreate the same structure in DokuWiki and all my links will remain intact.

[AlphaHot1]

I know of a few studies that used hundreds of data points to determine the behavior of things. Even if it is the case that things have changed and there are constant changes, I can't point to any kind of change that loosens restrictions. At worst, the effect of a "hot" technique will be tuned towards neutral.

Ok, can I see those studies? And I didn't even said that friendly URLs will help SEO... I can't really see what's your point.

And how does that help SEO in any way if the URL is ignored?

There URL is ignored? What do you mean.

Yep. This why even dynamic URLs like index.php?topic=263753 are good. If you spell out the board name (as SEO4SMF does at least), what happens when the topic moves to a new board? Do we remove the old link? Do we keep it working but 301 it? Do we just show the same content (which causes big search engine issues and possible duplicate content issues)?

You made me dig this issue and I found something really interesting.

Dannii really made a wonderful mod (pretty urls: http://custom.simplemachines.org/mods/index.php?mod=636).

I explain: I just moved a topic in my dev forum.

case 1 without friendly URLs:
If you move a topic, a new topic with a new URL is created, and the old one still remains but the content is changed ("topic moved" etc. etc.)

case 2 with friendly URLs:
If you move a topic, a new one is created, with a different URL (just the category part changes). The old topic url (where the content is just "topic moved" etc. etc.) changes too: it's the same as the old one but with "moved-" added.
And here is the great thing: if you use the original URL you will be rewrited to the new one!

It's bettere explained with an example:
original post url: *ttp://italianlair.com/richieste-consigli/dubbi-dell'universo-rafc/
original category: "richieste e consigli"

moved post url: *ttp://italianlair.com/generale/dubbi-dell'universo-rafc/
moved category: "generale"

redirection post: *ttp://italianlair.com/richieste-consigli/spostato-dubbi-dell'universo-rafc/

Now, if you use the original post url you'll be rewrited to the new one. So those who knew the old link still will reach the same content.
The redirection post instead changes url, something that doesn't happen if you use dynamic. This is the right thing to do beacause the content is effectively changed.

This is clean, consistent and user-friendly and totally automatic: no maintenance from users or admin.

SO...

Last I knew, Google has been reliably indexing dynamic URLs for years. Since the whole "rewriting" thing is a hot topic and a big problem if done badly, I see the article as a timely notice that the SEO "experts" are wrong in preaching conversion to static-looking URLs to "help indexing".

Well, this contradicts what's is posted in that very same article: they state that google HAD problems indexing dynamic URLs.
I agree that there was a lot of fuss on this URL-rewritng thing and probabily it's not as hot as someone might believe, but I still believe that they are helpful for SEO.

In the end what the article says is that  that rewrinting URLs is bad if is done bad. I'm thankful to google for this unbelievable eye-opener...

[brianjw]

Dannii, I will reinstall the mod. But I want to be sure that it won't cause problems with Google... This is a big debate and debates never end with facts, they end with opinions...

[karlbenson]

Indeed Brian.

Search Engine Optimization has almost always been about 'beliefs, guestimates, estimates, guess work, trial and error".
It is very rare that search Engines come out specifically to be catagorical on something.

This Google article does dispel alot of the negative myths about dynamic urls.  But does suggest there can be a very slight advantage on a couple of occasions (as pointed out by Dannii).  Although it would be debatable and personal preference whether its worth it for your site.

[Charles Hill]

You are wise, karl.  My personal preference is that it just isn't worth the trouble.... Time spent on getting rewriting to work effectively and consistently would be better spent developing other things.

[brianjw]

You are wise, karl.  My personal preference is that it just isn't worth the trouble.... Time spent on getting rewriting to work effectively and consistently would be better spent developing other things.

But if it is already developed for you as an SMF Mod. Such as pretty urls.

[Charles Hill]

It does not convert all pages of my site.  The forum on my site is not the only thing that makes dynamic URLs so  I would have to modify that mod to work with the other dynamic parts of my site... and the mod conflicts with all other things that I try to do with .htaccess.

[AlphaHot1]

I agree with karlbenson.

Charles, so we aren't going to see your blog mod with static urls anytime soon? I hoped...

[Charles Hill]

I'm afraid not.  I have much more useful features to develop first.

[brianjw]

Charles Hill, it does interfere with the .htaccess you currently have set. But all you have to do is go into Subs-PrettyUrls.php in ./Sources/ and find

Code Select Expand

Build the new .htaccess file and add the .htaccess contents in that area. It works fine on my site.

[Dannii]

He might have individual rules that actually are incompatible with the mod's rules too...

[wildcat180]

Let me preface this by saying I'm a beginner when it comes to web stuff.  I'm just someone trying to build my own website and have it do something more than cost me monthly in hosting.  SEO is such a hot topic right now, my head is spinning from all I've read lately.

That said, I use Google's webmaster tools.  I have been having an issue with the Googlebot not doing anything more than indexing the index page of my forum.  In trying to read their help files to find out why, I came across this from Google's own site regarding their bot:  (http://www.google.com/support/webmasters/bin/answer.py?answer=70897&hl=en#1)

"Indexing     

Googlebot processes each of the pages it crawls in order to compile a massive index of all the words it sees and their location on each page. In addition, we process information included in key content tags and attributes, such as Title tags and ALT attributes. Googlebot can process many, but not all, content types. For example, we cannot process the content of some rich media files or dynamic pages. "

I may have misunderstood that, but taking that to mean "dynamic URLs", I did a search to see if SMF uses dynamic URLs (hey, I said I was new at this! ) and I came across this topic here.  After reading this and feeling like this topic totally went against what I *thought* Google was saying, I did some more digging on Google's site and came across this in their "creating a Google friendly URL structure":  (http://www.google.com/support/webmasters/bin/answer.py?answer=76329&query=dynamic+URLs&topic=&type=)

" A site's URL structure should be as simple as possible. Consider organizing your content so that URLs are constructed logically and in a manner that is most intelligible to humans (when possible, readable words rather than long ID numbers). For example, if you're searching for information about aviation, a URL like http://en.wikipedia.org/wiki/Aviation will help you decide whether to click that link. A URL like http://www.example.com/index.php?id_sezione=360&sid=3a5ebc944f41daa6f849f730f1, is much less appealing to users.

Consider using punctuation in your URLs. The URL http://www.example.com/green-dress.html is much more useful to us than http://www.example.com/greendress.html. We recommend that you use hyphens (-) instead of underscores (_) in your URLs.

Overly complex URLs, especially those containing multiple parameters, can cause a problems for crawlers by creating unnecessarily high numbers of URLs that point to identical or similar content on your site. As a result, Googlebot may consume much more bandwidth than necessary, or may be unable to completely index all the content on your site."

Then this towards the end of that same article:

"To avoid potential problems with URL structure, we recommend the following:

    * Consider using a robots.txt file to block Googlebot's access to problematic URLs. Typically, you should consider blocking dynamic URLs, such as URLs that generate search results, or URLs that can create infinite spaces, such as calendars. Using regular expressions in your robots.txt file can allow you to easily block large numbers of URLs. "


To me, and granted I'm very much the newbie, it sounds like Google is advocating the use of text based URLs, not going against it and favoring dynamic URLs like this blog suggests.  Can someone clarify for me, because at this point, I'm SUPER confused and that's not going to help my site any...

[青山 素子]

In the first article, Google is just stating that they can't parse all content. Since dynamic URLs sometimes generate some special content, they can't guarantee that will be indexed. SMF uses plain HTML, so it is fine.

In the second article, they are talking about bad dynamic URLs that can lead to infinite loops, broken pages, etc. SMF avoids adding superfluous parameters, and actively lets Google know which pages should be indexed (via noindex tags on certain URLs) to avoid this problem.

[wildcat180]

So, dynamic URLs that aren't messed with are usually OK?

[青山 素子]

The dynamic URLs SMF uses should be perfectly fine. Some systems go rather crazy with parameters, and that is where the problems can occur.

[Nao 尚]

Thanks to Motoko-chan for pointing me to this topic. I don't have much to add on the subject, but you're starting to know me -- I will still post a 52-page book about my thoughts. Feel free to skip it, eheh

First of all, my position about Pretty URLs. I'm pretty fond, if I may say, of the Pretty URLs mod myself. I wanted that kind of mod on my board, I found Pretty URLs, Dannii was kind enough to share his knowledge of regular expressions with me (which really made me become fond of them as well -- and without that, I wouldn't be maintaining Aeva today), and even though I was totally ignorant of SVN and mod making at the time (which made me unable to work correctly with Dannii, which I deeply regret), I often wonder whether or not I should go back and start helping again.

Now, regarding the pointlessness of URL prettification.

I quickly read through the topic, and it occurs to me that, maybe I was too quick and I missed it, but there was no mention of one of Pretty URLs' peculiarities. (Although Motoko-chan's latest posts are going in the same direction as I will be going.)

Basically, what the mod does, is turn some specific parameters into static URLs. action=xxx will generate a specific URL for these. board=xxx will do the same. action=profile;u=xxx will do the same. Apart from these, the rest of the URL is untransformed.

Which means that this kind of URL:

ttp://noisen.com/index.php?topic=4997.105;action=markasread;sa=topic;t=228046;sesc=xxx

Will become:

ttp://nao.noisen.com/4997/chapeau-bas-docteur-who/105/?action=markasread;sa=topic;t=228046;sesc=xxx

(Mind you, this is a custom implementation of the mod.)

Basically, I think Google would be bothered if the URL looked like this instead:

ttp://nao.noisen.com/4997-chapeau-bas-docteur-who-105-action-markasread-sa-topic-t-228046-sesc-xxx.html

Which could then confuse Google, because it couldn't determine what is the topic's title, and what is the specific action to do. In PrettyURLs' case, the URL only turns the most important settings (topic ID + topic title + page number) into static content. The "small details" that change the way the contents are shown, remain in the query string. Additionally, measures can be taken (actually, I did) to ask Google not to index such pages that are variations on the same URL and return the same contents, such as 4997.msg228039 -> that way, even less pages will be recorded, and Google will have a better chance to figure out the site's structure.

All I mean is -- Dannii thought of it all, I think. His system is solid, and it's not in my nature to praise someone's work when I don't truly believe it's commendable. Actually, I think the Google blog post made me even more confident that Pretty URLs is an excellent mod for users who would like to help Google find their way inside their website.

If you made it through here, congrats!

[青山 素子]

I still hold by my stance that this proves that SMF's default URLs are quite fine as they are.

[Dannii]

Of course they're quite fine. And mods like these won't make a direct impact on SEO.

But I think my mod (and others like it) will help your forum indirectly. Do not discount how often people post straight URLs into forums. And additionally, they're much more people-friendly. If someone sends me random numbered URLs (especially on something like IRC) I'm unlikely to open it. If they send me a textual URL maybe I will.

[ccraciun]

And finally, for newbie users like me, two questions:

- i guess it's better to start with SMF's standard URLs and later change, if needed, to more complex ones (like those from Danii's mod);
my case: right after launching my forum, i enabled SMF's option SEF URLs (Admin | Features & Options | Basic); after a few days, i noticed in the analytics logs that my site was found by searching after a pretty popular expression (useful freeware programs, spelled in romanian) and i had the first two links in SERP!! Unfortunately, both linking to the same subject, description etc. After a few more days, only the dynamic link remained and the other vanished; now i disabled SEF URLs option, in order to prevent duplicate content, but both of them still work: programe freeware utile, and the same result also, with "index.php/topic,7.0.html". Is this normal? I should add a disallow line in robots.txt, to prevent the indexing of SEF URLs or Dynamic URLs? (if needed, an example would be great).
Now, the statement related to Pretty URLs mod: i would use this mod after getting familiar with the proper configuration of robots.txt, for instance. I noticed that most important for google is the content that give you a good position in SERP and not how the link is spelled; maybe linking, pretty URLs and other SEO workarounds are good in achieving high PR and/or good impression to visitors.

- if i choose to use Pretty URLs mod, the other two types of linking (dynamic and SEF) will still work? Would that be ok, or i should disallow them in robots.txt?

I asked these two questions here, because for experimented users like you (Motoko-chan, Danii, Nao etc) maybe taking a decision would be simple, but for those like me  who are reading this topic, the answers may banish some of the fog. 

Thanks

[青山 素子]

- i guess it's better to start with SMF's standard URLs and later change, if needed, to more complex ones (like those from Danii's mod);

Generally a bad idea. Pick one thing and work with it. Otherwise, you have to manage redirects so the old URLs will redirect to the correct ones and you'll still get proper handling for search engines and stuff. If you do change, be prepared for it to be a bit of a pain,

After a few more days, only the dynamic link remained and the other vanished; now i disabled SEF URLs option, in order to prevent duplicate content, but both of them still work: programe freeware utile, and the same result also, with "index.php/topic,7.0.html". Is this normal? I should add a disallow line in robots.txt, to prevent the indexing of SEF URLs or Dynamic URLs? (if needed, an example would be great).

Remove the rewrite rules in the htaccess that the option inserted. Be prepared to lose credit from those links then.

- if i choose to use Pretty URLs mod, the other two types of linking (dynamic and SEF) will still work? Would that be ok, or i should disallow them in robots.txt?

I believe Pretty URLs redirects the dynamic defaults to the new style, so you won't need to worry. Of course, best to confirm with the author in the support topic for that mod.

[ccraciun]

Thanks Motoko-chan!

Three more questions though:
- i looked for .htaccess file on my site and it's empty (has 0 bytes); mine is located in /public_html, the root of SMF installation; i should look somewhere else?
- serching for duplicate content problem i guess i found the issue that mess things up; i used "index,follow", as i saw in some popular sites (not SMF...). I will modify this to default "noindex", should i modify something else?
- if i choose dynamic URLs and "noindex" in index.template.php, the SEF URLs will get the "noindex" tag automatically or i should use some robots.txt "disallow" line for those "index.php/topic,x.0.html" links?