Topic: Forum SEO is a myth

[Arantor]

Last year I posted an article on a site about the mythology of trying to search engine optimise forums. I'm reposting because I think it's an interesting point for discussion.

I have said publicly that trying to 'optimise' a forum is futile. Here's why I think that's the case.


1. You're Trying To Game The System

Let me get this straight. You're trying to attract the attention of, amongst other things, one of the most powerful companies in the world, which is staffed with Smart People, probably smarter than both you and I combined in many cases. You're trying to attract their attention, and do so by gaming a system built by Smart People.

It's like going to the doctors, and suggesting a diagnosis before you get there. And at the clinic, the doctor is likely going to ignore your diagnosis (especially if they have any sense), and evaluate you themselves, before making a recommendation.

You know what? That's what Google does, mostly. Sure, you can help them a little, but by and large it isn't your site's SEO they're evaluating, and it definitely isn't the prime factor they rank you on. It's the site as a whole. Length of time it's been there, responsiveness of pages, and its content.


2. SEO works on carefully crafted content.

Now, unless you're weird, your average public forum is filled with juicy content that's contributed by the members, not the staff. And unless you're going to sit and mercilessly edit content, or beat your community members with a big stick, odds are quite strongly that it won't be the keyword-laden, search engine "optimised" masterpiece that the SEO experts recommend you should have.

I wonder why that is. Is it because you have no control over it?

On a blog, where you're the sole author, you can control the content, the presentation - every aspect of the page from the opening DOCTYPE to the closing </html>. But on a forum, odds are you just don't have that flexibility. Sure, you can modify the layout, the code, whatever, but the most important thing for search engines is the one thing you have little real control over: content. And content is utterly king for search engines.

You can adjust your page to have header tags (H1 through H6), you can adjust the link follow elements, you can do all the so-called tricks, but since you're only affecting a tiny percentage of your content, it actually makes very little difference.


3. Sitemaps are next to useless on a forum.

A sitemap, for those who aren't really aware of it, is a list of URLs a crawler should visit. Not a "must index these pages" list. Nor even a "only these pages kthx" list. But simply a "here's what we think you'd be interested in" list.

Now, on a smaller website, or website where the content is more finely controlled, you probably would want a sitemap. You get to set relative priorities of pages, plus a 'how frequently updated this page is' factor. This does actually help a search engine, for the kinds of sites that that applies to.

Note I said "for the kinds of sites that that applies to". A forum is not one of those kinds of site. A forum, by nature, has a totally dynamic, primarily user driven update schedule. It's not like a blog where the author writes as and when he feels like it, or even on a schedule.

You will also likely know that a crawler is quite capable of traversing links by itself; it doesn't need to be told the list of pages it should index, and where it's dynamic and likely to change from one day to the next, adding a sitemap just means you're building an ever changing list of links, which an engine is smart enough to find for itself without your help.

There is only, and I do mean only, one reason you might consider a sitemap, and that is if you've managed to hide a board from general view but still want it indexed without it being linked any other way. If you create a sub-board (or child board) of a parent, such that the parent is not available to the public but the child board is, the child board can be browsed and indexed, but won't show on the board index.

This, however, isn't standard practice, meaning that it's the sort of thing you don't actually need to do on the vast bulk of forums out there.


4. Pretty URLs doesn't help.

At this point I know that I'm going to be near lynched by claims of B-B-B-BUT THE SEO GUIDE SAID... It's crap. It's absolute, utter crap.

Well, what are pretty URLs? The idea is that the URL contains key words relating to the page at hand, so search engines rank it higher. It's also easier to remember if you're going to type it in.

Search engines generally don't use the URL itself in any kind of ranking, or if they do (Google, at least, is known not to use the full URL at this time in ranking though it can consider the domain on its own, but all that may change) it won't be significant at all.

Now before we get into the 'but every little helps' argument, let me put this forward first. Since Google has stated page load speed is actually a factor in the ranking process (albeit not a huge one, but likely more important than the URL), and that pretty URLs on a forum is not a small calculation (though, SlammedDime's SimpleSEF mod does it surprisingly fast), you'd probably be better off dropping the pretty URLs for the performance boost.

That said, there is one possible argument for pretty URLs on a forum, but it's pretty weaksauce: click through. When you're on the Google-or-Bing-or-whatnot page of results, you'll see the URL of the page, and if it's short, and it looks nice, there is a tiny fraction of a percentage of an outside chance that you'd click on it where you wouldn't otherwise. Most seasoned net users, however, don't care.

Now, other sites DO use pretty URLs. And it works for them. Why? Why not forums?

Let's have a quick look - off the top of my head, Facebook and Wikipedia both do. But, curiously, it's not for search engine ranking reasons (and anyone that tells you it is, is BS'ing you) In both cases, it's because articles are likely to be actually typed in by users directly.

For example, http://www.facebook.com/arantor is a prime example. It's a pretty URL.

Or http://en.wikipedia.org/wiki/Meat_Loaf (which is reachable through http://en.wikipedia.org/wiki/Meat Loaf in most browsers) Again, it's memorable.

So why wouldn't this approach work on a forum? After all, what's stopping this page from being index.php/Chit-Chat/Forum-SEO-is-a-myth ? Well, partly [here] don't have any pretty URLs mods installed, but if it did, I know full well it wouldn't generate that as a URL.

As I've said earlier, a forum is typically, mostly, community generated, which often means little regards to the normal SEO stomping grounds of things like titles. I won't dwell on it but on the forum community here, titles such as "Help!!!!!!!!!!!!!!!!!!!!!!" are not uncommon. As you can imagine, that's not very SEO friendly.

That's the first thing: the page's title will be directly tied to its URL, and if you have a meaningful title, you have a meaningful URL. The majority of forums do not produce meaningful thread titles.

The second thing is what happens if you get a duplicate? Well, on a conventional blog or other site, duplicates of names are much less common, but on any decent sized forum it starts to become very common - and you would get into the realms of having index.php/board/topic, index.php/board/topic2, index.php/board/topic3, which isn't very meaningful either.

So already you've drifted off track in terms of having a meaningful URL. What both the pretty URL mods do for SMF (this software) is embed the thread's id number into the URL. This has three effects.

Firstly, it makes the thread URL unique, which is great.

Secondly, it means an expensive trip to the database is avoided because you have the id number handy and don't have to look it up like you do elsewhere, which means site performance isn't degraded as much.

Lastly, but by far most importantly, it makes the URL unmemorable. Which sort of makes it pointless because the whole idea is to have MEMORABLE URLs.

And seriously, how often do people type URLs in? Well, on social services, and linking to their friends, not that uncommonly perhaps. But general forum links? Very, very often you won't type the whole URL in, you'll either copy/paste it or you'll just use a URL shortening service - either way, the pretty URL phenomena just doesn't work on forums.


Closing thoughts

I hear a lot of people saying how effective these tips are, and I'd love to actually collect stats on whether it does actually have an effect or not, because I believe that forum growth is IN SPITE of the above, not because of it.

Why? Well, if you're putting that time and effort into the site to check SEO, to check all these things work, you're making more content and content is really the key. If you're actively pushing the site, you'll grow, not because you played some voodoo magic to game a system built by people smarter than you.

[DoctorMalboro]

Honestly, I don't care about SEO or anything like that on my blog. It just a waste of time, but a good way to get easy money by people who "believes" in SEO magicians...

[青山 素子]

Your ideas are intriguing to me and I wish to subscribe to your newsletter.

Seriously though, I agree with your points. It's near impossible to massage user-contributed content in places like a forum unless one is willing to heavily re-write ever post a user makes. Should one actually try doing that, I think they will find out quickly that people don't appreciate such things.

I especially agree with the notes on "friendly urls". Using something such as that is expensive (in computation time), so it should be avoided. As for the readability and "friendlyness" of them, let's make a few real-world examples.

First, the standard SMF URL:

hxxp://www.example.com/index.php?topic=394582

Here, you only really have to remember the topic ID number. It's not as easy to memorize as some things, but it's not super-humanly difficult.

Now, a sample "friendly" URL:

hxxp://www.example.com/topic/should-britian-eat-so-much-fish-394582.html

I think that perhaps this might be easier to memorize than some of the more common topic titles, but you suffer a bit here from the length and time. Four hours later, if you try to pull the page up on a different computer you not only have to still know the topic ID, but now you have to try and remember the words too! Was it Britian or England? Eat or Consume? Of course, if the rewrite is working correctly (pulling only the ID at the end), the following should take you to the same topic:

hxxp://www.example.com/topic/nudity-buffer-394582.html

This means, of course, that the software will need to find some way of signaling to a search engine that such a URL as the above is not the correct URL either through the fairly-new canonical link tag in the page header, or via redirect to the "correct" URL. Of course, this means more computation time is spent making sure that the URLs are preserved instead of, you know, serving the page.

As for blogs, the canonical URL usually has a date in front of it to prevent URL collisions, which make "friendly" URLs on blog posts an oxymoron as you then have to also remember the year and month of the post and possibly the day as well.



On the topic of using heading tags the right way (in other words, not sprinkinlg h1-h6 in random areas you think are important but as actual outline elements), I support taking the time to do so if it is possible. Anything that can help the search engine distinguish between content and wrapper should be encouraged. This is especially important if you have the advantage of getting quality content-filled posts and topics.

[ARG01]

I have seen incidents where start up companies would pay well over $1,000 for SEO services. I can understand this to a limit for product sales or distribution and, if the site content was well written. But for a basic forum site? No way.

 

[Arantor]

That one came from arantor.org last year but most of the thoughts I have about the 'way of things' tend to appear on Innovate, not Imitate, and interestingly enough I recently posted again on the same matter, though much less forum-specifically. (Here's the second article, off InI, more about SEO being uninnovative in general)


Thing is, though, the same is true of blogs... the majority of them that have nice long 'friendly' URLs are invariably far longer than I could ever be bothered to type, I'd just copy/paste it again. The one difference with blogs is that you can pass in just the year/month and often get a list of topics from that period which could be useful for narrowing it down - assuming you found the right month/year to start with.

I have seen incidents where start up companies would pay well over $1,000 for SEO services. I can understand this to a limit for product sales or distribution and, if the site content was well written. But for a basic forum site? No way.

Yup... how do you optimise something you have no real control over? Even a huge forum with masses of content can do little more than nibbling at the edges.

[SlammedDime]

As the author of Sitemap and SimpleSEF... I almost wholeheartedly agree... however with some points about your sitemap observations and Motoko's on 'pretty urls'...

From Google's own webmaster tools...

Sitemaps are particularly helpful if:

Your site has dynamic content.
Your site has pages that aren't easily discovered by Googlebot during the crawl process—for example, pages featuring rich AJAX or images.
Your site is new and has few links to it. (Googlebot crawls the web by following links from one page to another, so if your site isn't well linked, it may be hard for us to discover it.)
Your site has a large archive of content pages that are not well linked to each other, or are not linked at all.

I'd say the very first one applies well to forums, and the third may apply to many forums as well that aren't well established yet.

Furthermore...

The optional priority tag describes the relative priority of all pages on your site. This priority can range from 1.0 (extremely important) to 0.1 (not important at all).
...
Setting the priority of pages will never affect your site's ranking in Google search results. Search engines use this information when selecting between URLs on the same site, so you can use this tag to increase the likelihood that your more important pages are present in a search index.

Sitemap uses dynamic priority based on how old a topic is and how many posts it contains, so more recent topics will have a higher priority and older topics have a lower.

And also...

The optional changefreq indicates how frequently the page is likely to change.
...
The value of this tag is considered a hint and not a command. Even though search engine crawlers consider this information when making decisions, they may crawl pages marked "hourly" less frequently than that, and they may crawl pages marked "yearly" more frequently than that. It is also likely that crawlers will periodically crawl pages marked "never" so that they can handle unexpected changes to those pages.

So while this is only a hint, it is somewhat valid, and again Sitemap calculates this dynamically based on the number of posts and age of the topic/board.  If a topic is active, it is changed more frequently thus should have a higher frequency for the search engines.

First, the standard SMF URL:

hxxp://www.example.com/index.php?topic=394582

Here, you only really have to remember the topic ID number. It's not as easy to memorize as some things, but it's not super-humanly difficult.

Now, a sample "friendly" URL:

hxxp://www.example.com/topic/should-britian-eat-so-much-fish-394582.html

I think that perhaps this might be easier to memorize than some of the more common topic titles, but you suffer a bit here from the length and time. Four hours later, if you try to pull the page up on a different computer you not only have to still know the topic ID, but now you have to try and remember the words too! Was it Britian or England? Eat or Consume? Of course, if the rewrite is working correctly (pulling only the ID at the end), the following should take you to the same topic:

hxxp://www.example.com/topic/nudity-buffer-394582.html

This means, of course, that the software will need to find some way of signaling to a search engine that such a URL as the above is not the correct URL either through the fairly-new canonical link tag in the page header, or via redirect to the "correct" URL. Of course, this means more computation time is spent making sure that the URLs are preserved instead of, you know, serving the page.

This is exactly what SimpleSEF does... only uses the topic ID part, so directs to the right topic regardless of what friendly words you put in front of it.  I don't really see this as an issue for search engines unless people go posting links with invalid words but a correct topic (but in the case of SMF 2, at least there is the canonical tag, albeit, however lightly used by search engines.

[Arantor]

I'd say the very first one applies well to forums, and the third may apply to many forums as well that aren't well established yet.

'Dynamic' content can refer to an awful lot of things. It's more for sites that have regular content updates rather than totally dynamic-driven sites like forums.

Sitemap uses dynamic priority based on how old a topic is and how many posts it contains, so more recent topics will have a higher priority and older topics have a lower.

Yes, I was aware of that - but at best it's a vague indicator, in ranges rather than the more limited, more targeted variant that Google et al are expecting.

So while this is only a hint, it is somewhat valid, and again Sitemap calculates this dynamically based on the number of posts and age of the topic/board.  If a topic is active, it is changed more frequently thus should have a higher frequency for the search engines.

Except most people seem to think that's it's gospel and a command for 'only index this' rather than anything else.

This is exactly what SimpleSEF does... only uses the topic ID part, so directs to the right topic regardless of what friendly words you put in front of it.

Good to know, but while it achieves that, it still breaks the 'memorable URL' mentality attached....

[青山 素子]

This is exactly what SimpleSEF does... only uses the topic ID part, so directs to the right topic regardless of what friendly words you put in front of it.  I don't really see this as an issue for search engines unless people go posting links with invalid words but a correct topic (but in the case of SMF 2, at least there is the canonical tag, albeit, however lightly used by search engines.

Heh, posting links with funny URLs might be interesting.

I never said it was a problem with search engines (they'll ignore the URL mostly, anyway), just that you're no better off for the much-argued point of it being easier to type directly or remember, and in fact might be worse off if you must use the correct phrase. The argument that the URL will help drive users to click on the link ignores that the page title is the biggest text on the search engine result page and is also a bit better for SEO purposes anyway.

[SleePy]

I should point out that while this topic mentions SEO, the contents posted in it has seem to of focused on SEF urls.  These are two things really, even though SEF falls under SEO.  Search Engine Friendly URLs is being discussed and has the myth that some bots couldn't handle urls with query strings in them (any bot nowdays that can't won't have very many users using their search). Then there is SEO, which is optimizing your actual page to give the search engine better and more direct content relevance to that page and thus to search results.

I don't believe in SEF, although have made a few mods and even use SimpleSEF on one of my sites.  The reason for using it was consistency with the reset of the site that didn't contain any query strings and was meant to keep it clean and organized.  I could of left it out, but it made the pages very easy to remember and doesn't break out of the structure I have for the site.

SEO on the other hand is important.  While many will argue against it, I see doing proper (x)HTML and other front end programing as key to ensuring that content is well indexed.  Not having a bunch of poor coding, random images, and random characters meant as a alt character for when a image can't be displayed, will help keep screen readers and search engines from misinterpreting their usage.

[Arantor]

The reason I focused on SEF URLs is because it's the number one 'SEO technique' promoted out there, and for all the wrong reasons. Using it for nice looking URLs because that's clean and tidy is a separate thing altogether - not for SEO benefits.

SEO is important in and of itself - it just doesn't apply to forums by definition. The examples you mention such as valid X/HTML and proper alt tags for images, they're not really SEO things at all, they're actually accessibility - it just happens that people who put the time in getting accessibility and other things right improve the site as a whole, not because of SEO.

SEO, ultimately, is about targetting the content of your site at search engines, not actual users. Except that a forum who targets content at search engines isn't likely to be particularly popular.

[bloc]

Can only agreed to whats mentioned about SEF urls. SEO, true, its more than just SEF, IMHO the biggest advantage is already done in the theme: using headers tags properly and not abusing semantic tags for layout(like lists just to place a content side by side), that is what the markup should do and will then be enough for the crawlers. I have used lots of divs in my themes, up to a point of real "divitis" lol, but it doesn't seem to affect sites using them, at least not for the SEO...they are just ignored, as they should be, a div is just a container tag. Using a actual header tag will be much more important - in which default theme and my themes does of course.The only problem is that we under-use the semantic tags..for lack of knowledge or even wondering what the tag should apply to in a forum? lol With CSS3 coming full on thats even more important I guess.

There's no denial that Google finds the smallest site anyway, as long as its on the net lol.Will it rank it high? Now, thats another area altogether - and Arantor pointed out the important parts-  so trying to "cheat" is almost hilarious considering how long Google have been in this game.

I think they are onto us(them)..

[WhiteEagle]

I only use sitemaps for my sites powered by SMF (and other dynamic-systems like link directories, Content Management Systems, blogs, etc.) so I can submit a sitemap to Google Webmasters. Not that it's done me any benefit, but I just like doing it. I rely more on word of mouth and social websites like Facebook and Twitter to get my sites out there.

[Dannii]

I see the benefit of pretty URLs not in being somehow easier to remember but that when someone posts one in IRC or whatever you'll have an idea of what it's about before you open the page. But now that people are using URL shorteners it might not be as useful...

Arantor, do you have information showing that search engines no longer care about URLs which are the content of an <a>? If so that would mean pretty URLs no longer help in the common situation when someone just posts a link and doesn't use the [url] syntax.

[Arantor]

Well, search engines have been known to take the content of the link text itself as for ranking, but only as a minor factor, from what I understood they'd long since preferred to have a shorter phrase that's still meaningful (and, as at least one SEO 'expert' has recommended in the past, the link title should match the destination page's title or have a meaningful relationship to it)

I guess there could be a small amount of ranking based on having a bare pretty URL with keywords in it, but again that brings us back to user-driven content having meaningful titles and thus meaningful URLs, which for primarily user driven content (which was the scope of the article, i.e. for *forums*) doesn't tend to happen.

[mashby]

The more time you spend on optimizing <meta keywords> or worrying about how your URL is constructed and looks to the human eye is time you really should be spending on content people would care to read. That's one of the biggest bang for you buck you'll ever imagine. Content is king. Keywords and URLs are peons. Think about it this way. Let's say I have a page that had the absolute best keywords and the best URL you'll ever see (put on your imagination caps please). But the page content was this:

How is that effective or useful? Focus your content on being useful, informative, linkable. That's one of the best directions SEO has taken. Content is king. Pictures of bunnies with pancakes on their head isn't quite so much.

[DoctorMalboro]

I wish I could translate this and put it on my blog...

[Arantor]

The article or the picture of a bunny with a pancake on its head?

[DoctorMalboro]

both

[Arantor]

Feel free to repost the article translated, provided that you make it clear it's not the original article (but a translation) and that a link back is given here to its source. (Consider the article as being Creative Commons Attribution-ShareAlike)

[DoctorMalboro]

You made me sound like a leecher, of course i'll do that. Just in another time, now i'm short of it...