News:

Want to get involved in developing SMF, then why not lend a hand on our github!

Main Menu

Dynamic URLs vs. static URLs - Google Webmaster Central Blog

Started by 青山 素子, September 23, 2008, 05:43:36 PM

Previous topic - Next topic

青山 素子

Hopefully this will help with those people interested in URL re-writing to "friendly" style URLs. The article below is from the Official Google Webmaster Central Blog.

If you are already using "friendly" URLs, you'll probably want to keep them or risk a great deal of loss in link equity, but for those considering a switch, this article sends a clear message:

Google recommends NOT rewriting dynamic URLs to static ones - leave them dynamic




Dynamic URLs vs. static URLs
Juliane Stiller and Kaspar Szymanski, Google Search Quality Team
Monday, September 22, 2008

Chatting with webmasters often reveals widespread beliefs that might have been accurate in the past, but are not necessarily up-to-date any more. This was the case when we recently talked to a couple of friends about the structure of a URL. One friend was concerned about using dynamic URLs, since (as she told us) "search engines can't cope with these." Another friend thought that dynamic URLs weren't a problem at all for search engines and that these issues were a thing of the past. One even admitted that he never understood the fuss about dynamic URLs in comparison to static URLs. For us, that was the moment we decided to read up on the topic of dynamic and static URLs. First, let's clarify what we're talking about:

What is a static URL?
A static URL is one that does not change, so it typically does not contain any url parameters. It can look like this: http://www.example.com/archive/january.htm. You can search for static URLs on Google by typing filetype:htm in the search field. Updating these kinds of pages can be time consuming, especially if the amount of information grows quickly, since every single page has to be hard-coded. This is why webmasters who deal with large, frequently updated sites like online shops, forum communities, blogs or content management systems may use dynamic URLs.

What is a dynamic URL?
If the content of a site is stored in a database and pulled for display on pages on demand, dynamic URLs maybe used. In that case the site serves basically as a template for the content. Usually, a dynamic URL would look something like this: http://code.google.com/p/google-checkout-php-sample-code/issues/detail?id=31. You can spot dynamic URLs by looking for characters like: ? = &. Dynamic URLs have the disadvantage that different URLs can have the same content. So different users might link to URLs with different parameters which have the same content. That's one reason why webmasters sometimes want to rewrite their URLs to static ones.

Should I try to make my dynamic URLs look static?
Following are some key points you should keep in mind while dealing with dynamic URLs:

  • It's quite hard to correctly create and maintain rewrites that change dynamic URLs to static-looking URLs.
  • It's much safer to serve us the original dynamic URL and let us handle the problem of detecting and avoiding problematic parameters.
  • If you want to rewrite your URL, please remove unnecessary parameters while maintaining a dynamic-looking URL.
  • If you want to serve a static URL instead of a dynamic URL you should create a static equivalent of your content.

Which can Googlebot read better, static or dynamic URLs?
We've come across many webmasters who, like our friend, believed that static or static-looking URLs were an advantage for indexing and ranking their sites. This is based on the presumption that search engines have issues with crawling and analyzing URLs that include session IDs or source trackers. However, as a matter of fact, we at Google have made some progress in both areas. While static URLs might have a slight advantage in terms of clickthrough rates because users can easily read the urls, the decision to use database-driven websites does not imply a significant disadvantage in terms of indexing and ranking. Providing search engines with dynamic URLs should be favored over hiding parameters to make them look static.

Let's now look at some of the widespread beliefs concerning dynamic URLs and correct some of the assumptions which spook webmasters. :)

Myth: "Dynamic URLs cannot be crawled."
Fact: We can crawl dynamic URLs and interpret the different parameters. We might have problems crawling and ranking your dynamic URLs if you try to make your urls look static and in the process hide parameters which offer the Googlebot valuable information. One recommendation is to avoid reformatting a dynamic URL to make it look static. It's always advisable to use static content with static URLs as much as possible, but in cases where you decide to use dynamic content, you should give us the possibility to analyze your URL structure and not remove information by hiding parameters and making them look static.

Myth: "Dynamic URLs are okay if you use fewer than three parameters."
Fact: There is no limit on the number of parameters, but a good rule of thumb would be to keep your URLs short (this applies to all URLs, whether static or dynamic). You may be able to remove some parameters which aren't essential for Googlebot and offer your users a nice looking dynamic URL. If you are not able to figure out which parameters to remove, we'd advise you to serve us all the parameters in your dynamic URL and our system will figure out which ones do not matter. Hiding your parameters keeps us from analyzing your URLs properly and we won't be able to recognize the parameters as such, which could cause a loss of valuable information.

Following are some questions we thought you might have at this point.

Does that mean I should avoid rewriting dynamic URLs at all?
That's our recommendation,
unless your rewrites are limited to removing unnecessary parameters, or you are very diligent in removing all parameters that could cause problems. If you transform your dynamic URL to make it look static you should be aware that we might not be able to interpret the information correctly in all cases. If you want to serve a static equivalent of your site, you might want to consider transforming the underlying content by serving a replacement which is truly static. One example would be to generate files for all the paths and make them accessible somewhere on your site. However, if you're using URL rewriting (rather than making a copy of the content) to produce static-looking URLs from a dynamic site, you could be doing harm rather than good. Feel free to serve us your standard dynamic URL and we will automatically find the parameters which are unnecessary.

Can you give me an example?
If you have a dynamic URL which is in the standard format like foo?key1=value&key2=value2 we recommend that you leave the url unchanged, and Google will determine which parameters can be removed; or you could remove uncessary parameters for your users. Be careful that you only remove parameters which do not matter. Here's an example of a URL with a couple of parameters:

www.example.com/article/bin/answer.foo?language=en&answer=3&sid=98971298178906&query=URL


  • language=en - indicates the language of the article
  • answer=3 - the article has the number 3
  • sid=8971298178906 - the session ID number is 8971298178906
  • query=URL - the query with which the article was found is [url]

Not all of these parameters offer additional information. So rewriting the URL to www.example.com/article/bin/answer.foo?language=en&answer=3 probably would not cause any problems as all irrelevant parameters are removed.

The following are some examples of static-looking URLs which may cause more crawling problems than serving the dynamic URL without rewriting:


Rewriting your dynamic URL to one of these examples could cause us to crawl the same piece of content needlessly via many different URLs with varying values for session IDs (sid) and query. These forms make it difficult for us to understand that URL and 98971298178906 have nothing to do with the actual content which is returned via this URL. However, here's an example of a rewrite where all irrelevant parameters have been removed:


Although we are able to process this URL correctly, we would still discourage you from using this rewrite as it is hard to maintain and needs to be updated as soon as a new parameter is added to the original dynamic URL. Failure to do this would again result in a static looking URL which is hiding parameters. So the best solution is often to keep your dynamic URLs as they are. Or, if you remove irrelevant parameters, bear in mind to leave the URL dynamic as the above example of a rewritten URL shows:


We hope this article is helpful to you and our friends to shed some light on the various assumptions around dynamic URLs. Please feel free to join our discussion group if you have any further questions.




Thanks to minstrel for the find. Red highlights are his.

Edit: Cleaned up the formatting to match the original post.
Motoko-chan
Director, Simple Machines

Note: Unless otherwise stated, my posts are not representative of any official position or opinion of Simple Machines.


karlbenson

thanks for posting.

*bookmarked for linking people to at a later date


AlphaHot1

The fact that pretty urls are USER friendly besides being search engines friendly still remains.
And in the last example:

* www.example.com/article/bin/answer.foo/en/3

Although we are able to process this URL correctly, we would still discourage you from using this rewrite as it is hard to maintain and needs to be updated as soon as a new parameter is added to the original dynamic URL.


he simply states that the url si good but hard to maintain. And this is not always tha case obviously.

青山 素子

#4
As I've said before, the people convinced this works wouldn't believe even those running the engines telling them it doesn't.

Of course, I rather trust that those who work daily on the search engines know what they are talking about. If they say that rewriting isn't helpful and can be bad, I'll trust they are speaking with knowledge that allows them to say that. It also helps when several people have done proper studies in the past showing the exact same thing.

As for the user-friendly part, I'm rather unconvinced. Take this topic for example. Would it be easier to remember:

hxxp://www.simplemachines.org/community/index.php?topic=263753

or

hxxp://www.simplemachines.org/community/SMF_Development/SMF_Feedback_and_Discussion/Dynamic_URLs_vs._static-URLs_-_Google_Webmaster_Central_Blog.html

?

I posit that remembering a six-digit number would be much easier than remembering that obscenely long text "friendly" URL.

Also, what if the topic moves? That would be worthless. The option would be to include some kind of marker to let the rewrites continue to function. Perhaps "Dynamic_URLs_vs._static-URLs_-_Google_Webmaster_Central_Blog-t-263753.html", but that defeats the purpose of making it look nicer.
Motoko-chan
Director, Simple Machines

Note: Unless otherwise stated, my posts are not representative of any official position or opinion of Simple Machines.


Charles Hill

I would have to agree with rewriting URLs being incredibly difficult to maintain and fraught with problems.  As for what Motoko-chan said about trusting that "those who work daily on the search engines know what they are talking about".  I completely agree.

karlbenson

Its definately personal preference.

I think full seo urls are ugly.
hxxp://www.simplemachines.org/community/SMF_Development/SMF_Feedback_and_Discussion/Dynamic_URLs_vs._static-URLs_-_Google_Webmaster_Central_Blog.html

However if you use the method correctly when designing a script, it can make sense for ebay for example to do
ebay.com/cars
(as this is memorable and user friendly)
but not
ebay.com/cars-and-other-automobile-parts-and-another-url-keyword-stuffing-techniques.

For this reason I would really love to see Search Engines penalise using too many keywords or variables in the urls. (whether dynamic or static)


青山 素子

Quote from: karlbenson on September 26, 2008, 02:35:08 PM
For this reason I would really love to see Search Engines penalise using too many keywords or variables in the urls. (whether dynamic or static)

At least one study (done by WebCEO a year or two ago) determined that using highly-competitive keywords in the URL actually incurred a small penalty from Google at least.
Motoko-chan
Director, Simple Machines

Note: Unless otherwise stated, my posts are not representative of any official position or opinion of Simple Machines.


brianjw

I have a question. Does this include the default feature in SMF "Search Engine Friendly URLs"?

I just uninstalled Pretty URLs thanks for this topic. ;) However, will URLs like http://www.gamerzgarage.com/index.php/topic,276.html receive problems from search engines or is this one that google doesn't recommend?

karlbenson

SEF urls (default in smf). Shouldn't be much different than normal smf urls.

The only thing I would say though, once you decide on what type of urls to use, you should stick with them unless your prepared for negative seo in the short term (3-6 months) while search engines figure out your new url structure and replace old pages with new pages. (although the amount of negative seo, can be minimized).

青山 素子

Quote from: brianjw on September 28, 2008, 10:19:20 AM
I have a question. Does this include the default feature in SMF "Search Engine Friendly URLs"?

Yes, they count as well because they fake "static" URLs.

Agreed with Karl. If you are already using rewriting, you are stuck with it unless you want to go through re-indexing and not being ranked properly for several months. Also, all external links and bookmarks to topics will break.
Motoko-chan
Director, Simple Machines

Note: Unless otherwise stated, my posts are not representative of any official position or opinion of Simple Machines.


hartiberlin

#12
Quote from: Motoko-chan on September 26, 2008, 12:50:19 PM
.

As for the user-friendly part, I'm rather unconvinced. Take this topic for example. Would it be easier to remember:

hxxp://www.simplemachines.org/community/index.php?topic=263753

or

hxxp://www.simplemachines.org/community/SMF_Development/SMF_Feedback_and_Discussion/Dynamic_URLs_vs._static-URLs_-_Google_Webmaster_Central_Blog.html

?

I posit that remembering a six-digit number would be much easier than remembering that obscenely long text "friendly" URL.


Hmm,
but what about if I give into the search field in Google:
static-URLs

Will then not the page be easier found when this page is indexed as
hxxp://www.simplemachines.org/community/SMF_Development/SMF_Feedback_and_Discussion/Dynamic_URLs_vs._static-URLs_-_Google_Webmaster_Central_Blog.html

as instead of:

hxxp://www.simplemachines.org/community/index.php?topic=263753


??


As there is the word:
static-URLs
inside the link, it will be listed much higher , maybe on the first page
on Google as Google favours domain names and URL names much
higher...
I have tested this myself already and with my domain
overunity.com
I am on location1 when you just type in
overunity
in the
Google search form...

So URL names really matter. Especially if you have keywords in them
which not many other sites have then you are ranked pretty high on
the search results...

But for instance, I wanted to be on page 1 with the
keyword
free energy

but only sites with the word
energy
or
freeenergy
inside the URL are listed on page one.
So you see, I can do what I want,
if I don´t have the word energy in my
URL name, I will not be listed on page 1 on Google,
although my content on my overunity.com
has very many topics, where the word free energy
will be displayed...

So Google always prefers URLs where the keywords are in there..
If you don´t have the keyword in your URL you have no chance to
get to page 1 on the search results...

Regards, Stefan.

hartiberlin

#13
Quote from: Motoko-chan on September 28, 2008, 12:16:22 PM


Agreed with Karl. If you are already using rewriting, you are stuck with it unless you want to go through re-indexing and not being ranked properly for several months. Also, all external links and bookmarks to topics will break.

Are you sure from your own experience ?

I had blocked the Google bot on my overunity.com
domain for about 3 weeks, cause it made too much traffic.

Then my site overunity.com
was removed from the Google index.
No more there.

After letting The GoogleBot  in again, I was again on location 1 on the
keyword:
overunity
after 3 or 4 days.

Also after changing a few links on my homepage I was relocated from page 4 to page 2
after just 3 or 4 days...
So it seems it is really fast crawling and rearranging the index nowadays.

Also the Googlebot always crawls my forum all day long,
also if I put the Revisit Meta Tag to 21 days.

Maybe it is because I have Google Adsense ads on my pages... I don´t know,
if this plays a major role, but it always crawls my site..


So I am still considdering to use SEO4SMF or Pretty URLs to get more keyword
related pages into the Google search results.

Regards, Stefan.

brianjw

So, then I will just leave my SMF URLs at the SMF default. :)

Blind Bandit

Ok so I have a question.

Would this mean that using SMF SEO mods (there are several) are a bad idea?

It seems like pretty URLs is out...

青山 素子

Quote from: hartiberlin on September 28, 2008, 03:16:35 PM
Hmm,
but what about if I give into the search field in Google:
static-URLs

Will then not the page be easier found when this page is indexed as
hxxp://www.simplemachines.org/community/SMF_Development/SMF_Feedback_and_Discussion/Dynamic_URLs_vs._static-URLs_-_Google_Webmaster_Central_Blog.html

as instead of:

hxxp://www.simplemachines.org/community/index.php?topic=263753

Nope. Google doesn't count URL text in the indexing. If that phrase appears in the content of the page, or in links to that page, it will likely show up in the results.


Quote from: hartiberlin on September 28, 2008, 03:16:35 PM
As there is the word:
static-URLs
inside the link, it will be listed much higher , maybe on the first page
on Google as Google favours domain names and URL names much
higher...
I have tested this myself already and with my domain
overunity.com
I am on location1 when you just type in
overunity
in the
Google search form...

So URL names really matter. Especially if you have keywords in them
which not many other sites have then you are ranked pretty high on
the search results...

Considering your domain is the keyword and it seems to be unique, that would be the reason for the ranking. Keywords in the domain name help, but not in the actual path.


Quote from: hartiberlin on September 28, 2008, 03:16:35 PM
But for instance, I wanted to be on page 1 with the
keyword
free energy

but only sites with the word
energy
or
freeenergy
inside the URL are listed on page one.
So you see, I can do what I want,
if I don´t have the word energy in my
URL name, I will not be listed on page 1 on Google,
although my content on my overunity.com
has very many topics, where the word free energy
will be displayed...

Doing a quick look, Wikipedia is #1. Typical since that domain is heavily linked, and has that phrase on the page eight times. The next two results have those words in their domain. The second result also has fifteen instances of the phrase on the page. The third has it five times. After that, it seems to be only for content or backlinks (#4 has the phrase in the title tags and also on the page content).

If you actually look, keyword-rich URLs have no effect on the rankings.


Quote from: hartiberlin on September 28, 2008, 03:16:35 PM
So Google always prefers URLs where the keywords are in there..
If you don´t have the keyword in your URL you have no chance to
get to page 1 on the search results...

Please provide me a proper study showing this. I find it very difficult to believe, especially when Google themselves say rewriting URLs is potentially harmful and when several studies have repeatedly said that keyword-rich URLs have almost no to negative effect.


Quote from: hartiberlin on September 28, 2008, 03:27:23 PM
Quote from: Motoko-chan on September 28, 2008, 12:16:22 PM
Agreed with Karl. If you are already using rewriting, you are stuck with it unless you want to go through re-indexing and not being ranked properly for several months. Also, all external links and bookmarks to topics will break.

Are you sure from your own experience ?

Yes, I have had to do site restructuring and the rankings have issues for some time after until the indexes get refreshed. It's much easier with poorly-indexed sites than well-established ones.


Quote from: hartiberlin on September 28, 2008, 03:16:35 PM
After letting The GoogleBot  in again, I was again on location 1 on the
keyword:
overunity
after 3 or 4 days.

Yeah, it's in your domain name. I'm not surprised.


Quote from: hartiberlin on September 28, 2008, 03:16:35 PM
Also after changing a few links on my homepage I was relocated from page 4 to page 2
after just 3 or 4 days...
So it seems it is really fast crawling and rearranging the index nowadays.

What pages were relocated? The ones you changed? Did you see if the older page URLs were still getting hits?


Quote from: hartiberlin on September 28, 2008, 03:16:35 PM
Maybe it is because I have Google Adsense ads on my pages... I don´t know,
if this plays a major role, but it always crawls my site..

This seems to have a small effect on speeding crawlings. It can also account for why you see hits from Google all-day long. It's for serving ads with good context.


Quote from: hartiberlin on September 28, 2008, 03:16:35 PM
So I am still considdering to use SEO4SMF or Pretty URLs to get more keyword
related pages into the Google search results.

Do what you will, but don't be misinformed.

Also, please don't double-post. Edit your original, that's what the function is there for.


Quote from: brianjw on September 28, 2008, 04:03:57 PM
So, then I will just leave my SMF URLs at the SMF default. :)

That's what I'd do. There are much more effective ways to get better indexing than messing with your URLs.


Quote from: Blind Bandit on September 28, 2008, 04:06:00 PM
Would this mean that using SMF SEO mods (there are several) are a bad idea?

If they rewrite the URLs, they are pointless and add extra overhead for no SEO benefit. There are plenty of other ways to help with indexing.
Motoko-chan
Director, Simple Machines

Note: Unless otherwise stated, my posts are not representative of any official position or opinion of Simple Machines.


hartiberlin

@Motoko-chan
so do you think nowadays it is better not to install Pretty URLs or SEO4SMF
and just leave the default SMF URL settings and
just only use the Sitemap mod to help to index the whole forum
for inclusion in
Google Webmastertools  ?

青山 素子

For the past several years it has been best to leave dynamic URLs alone and concentrate on the important things of quality content and backlinks, along with some good HTML design.

You also don't need a sitemap, and it won't help your ranking if you do use it. All a sitemap does is helps to ensure the URLs you wish to be indexed are known. It's handy in many situations, but also not essential. It's an indexing helper, not a ranking helper.
Motoko-chan
Director, Simple Machines

Note: Unless otherwise stated, my posts are not representative of any official position or opinion of Simple Machines.


metallica48423

i'll take this opportunity, just as fair warning, to note that there is a security bug in SEO4SMF which has not been patched by its developer.

Sorry for taking it off topic :)
Justin O'Leary
Ex-Project Manager
Ex-Lead Support Specialist

QuoteMicrosoft wants us to "Imagine life without walls"...
I say, "If there are no walls, who needs Windows?"


Useful Links:
Online Manual!
How to Help us Help you
Search
Settings Repair Tool

Advertisement: