Do SMF posts appear in search engines? (Something to be concerned about)

[kizer]

This is what I did by searching around on the forum. Two days later I was spidered and Im spidered nearly every day. Sometimes multiple times a day with google.

http://www.simplemachines.org/community/index.php?topic=214583.msg1384123#msg1384123

Other search engines seem to do their thing on their own.

[Ricky.]

I like SMF in every way, but I'm sure that it has a problem with search engines, and when I use the search function within my own site some things don't show up the first time, sometimes I have to search it 3 times for it to show up. I'm not sure if thats related to the search engines tho. Just wanted to bring it up to see if they are related or not.

You are talking about SEARCH FEATURE in SMF and thats different thing.
Here we are actually discussing how SMF is treated by Search engines like Yahoo and google etc.

[eline]

Just to add here..
I switched from phpBB to SMF becz of SEO.. no to mention, now my SMF forum tops loads of keywords in google and yahoo.

Could You explain a little more ?

[Ricky.]

smf have few things like "noindex" directive for duplicate pages and good mods available.. u can work upon it and make it good seo..
Though now phpbb3 also features similar stuff also..

[CraigPal]

Other than that it may be worth trying to work out how the search engines get hold of the dynamic urls, when SE friendly urls are enabled.

I've wondered the same. Could the search engines be getting a hold of these dynamic URLs by users of gmail, yahoo, or hotmail, who receive them in their email notifications? That's the only way I think they could be getting them.

__________________
Kubuntu Forum

[Low]

This thread has just confirmed what I have been thinking for so long. Even SPAMMy rehashed content show above my forum posts  for the same title/keyword search.

I have every possible seo mod available keyword-URL's. H1 tags, brilliant meta data (unique/rellevant), backlinks, unique content etc etc.. whilst a VB or PHBB forum has nothing but is listed first page on Google for the title/keywords.

Such a shame.

[青山 素子]

This thread has just confirmed what I have been thinking for so long. Even SPAMMy rehashed content show above my forum posts  for the same title/keyword search.

Well, your topics do have the appearance of being similar to other spam content, so you'll need to fight harder to rank for your area simply because of the huge glut of stuff out there that has been around for a much longer time and are all interlinked.

I have every possible seo mod available keyword-URL's.

This has been shown to have virtual no impact on ranking. In fact, Google recommends not rewriting URLs. Also, if you have made that move recently, you will be penalized along with all the old index and credibility being gone. When you switch URL schemes, it is like starting over completely.

H1 tags

If you are tossing them in willy-nilly, you could actually confuse the search engine spiders to the structure of your content. This would hurt indexing of your content.

brilliant meta data (unique/rellevant)

That would have been very helpful ten years ago. Google has never trusted meta tags and all the other major search engines stopped using them in ranking calculations over 6 years ago. As a bonus, your competitors can easily look at your site and determine the keywords most important to you - giving them a competitive advantage.

backlinks

That can be helpful if you get linked with good quality link text and if the sites themselves have content in the same area as you. Doing a quick look, Google only sees two links to your site and Yahoo! sees 185.

whilst a VB or PHBB forum has nothing but is listed first page on Google for the title/keywords.

How long have those forums been around? Age is rewarded here as established sites are given greater credibility by default. A one to two year lead is a small advantage. How old are their domain names? Once again, older is better. Also, domains that expire more than a year away seem to have a small advantage. Do those domain names have words similar to search terms in them? Having a domain named similar to search terms does give you a boost. How many backlinks do you see for those sites? A larger number of backlinks and better quality ones also provide an advantage.

I think some of your problems are related to using tactics that no longer work and are potentially damaging (random header tags around content that isn't a header, for example). Another part of your problem is residing in an area that is heavy with spam - you could be hitting a filter and not knowing it simply because of your content area.

Also, you really need to update to the latest SMF. The version you are running has some security flaws that have been patched.

[Low]

This thread has just confirmed what I have been thinking for so long. Even SPAMMy rehashed content show above my forum posts  for the same title/keyword search.

Well, your topics do have the appearance of being similar to other spam content, so you'll need to fight harder to rank for your area simply because of the huge glut of stuff out there that has been around for a much longer time and are all interlinked.

I have every possible seo mod available keyword-URL's.

This has been shown to have virtual no impact on ranking. In fact, Google recommends not rewriting URLs. Also, if you have made that move recently, you will be penalized along with all the old index and credibility being gone. When you switch URL schemes, it is like starting over completely.

H1 tags

If you are tossing them in willy-nilly, you could actually confuse the search engine spiders to the structure of your content. This would hurt indexing of your content.

brilliant meta data (unique/rellevant)

That would have been very helpful ten years ago. Google has never trusted meta tags and all the other major search engines stopped using them in ranking calculations over 6 years ago. As a bonus, your competitors can easily look at your site and determine the keywords most important to you - giving them a competitive advantage.

backlinks

That can be helpful if you get linked with good quality link text and if the sites themselves have content in the same area as you. Doing a quick look, Google only sees two links to your site and Yahoo! sees 185.

whilst a VB or PHBB forum has nothing but is listed first page on Google for the title/keywords.

How long have those forums been around? Age is rewarded here as established sites are given greater credibility by default. A one to two year lead is a small advantage. How old are their domain names? Once again, older is better. Also, domains that expire more than a year away seem to have a small advantage. Do those domain names have words similar to search terms in them? Having a domain named similar to search terms does give you a boost. How many backlinks do you see for those sites? A larger number of backlinks and better quality ones also provide an advantage.

I think some of your problems are related to using tactics that no longer work and are potentially damaging (random header tags around content that isn't a header, for example). Another part of your problem is residing in an area that is heavy with spam - you could be hitting a filter and not knowing it simply because of your content area.

Also, you really need to update to the latest SMF. The version you are running has some security flaws that have been patched.

I understand you defending SMF so gracefully but alas you are way off imho.

For one you have NO idea as to what domain I'm referring to. Second.. are you seriously saying meta data has no effect on ranking or was you joking? lol.

As for backlinks, I have over 800 for the site YOU are referring to according to Google WMT

Also, who says I use random h1/header tags when all headers are for the topic content.. relevant!

I tried to update but like usual it says forbidden.

Again.. Motoko-chan, I understand you defending SMF against this problem and replying with what I consider ok reasons, but 15 pages of proof must say something right?

My reply was not a rant, more like reverse shycology in that I can't wait for the day SMF becomes perfect in every way.

Anyhoo, I love SMF and will stick with it forever!

[青山 素子]

Edit:

I realize my post below might be a bit harsh, but I've just grown a bit tired of seeing so many people encourage bad optimization practices that either do nothing but waste time and resources, or are potentially harmful.

That does not mean that I feel SMF is perfect. There are likely lots of things that could be done better. However, no amount of "you need to do better" will help. What is needed are actual citations of what can be done and why it is helpful. That kind of thing is what put in the noindex tag for duplicate content so late in the 1.1 cycle. For the 2.0 series, one of our beta testers and later team members did an in-depth study on what could be done to make SMF better to index. They provided statistics, empirical evidence, and showed that those things were actually beneficial.

If you feel SMF is having indexing troubles, please investigate why and tell us. If you know a technique we could be using, tell us how we can implement it and where it is shown to be good by outside independent studies.



And now on to the original content, so you can understand the comments in my previous post.

For one you have NO idea as to what domain I'm referring to.

Since you did not provide one, I went with the site in your profile (yoebo). If that was not the site, I apologize.

Second.. are you seriously saying meta data has no effect on ranking or was you joking? lol.

I'm serious. Monika Henziger, Research Director at Google stated as much: "Currently we don't trust metadata because we are afraid of being manipulated." That was published in the Journal of Internet Cataloging in Volume 5(1) in 2002. SearchEngineWatch tags the last major search engine supporting meta keywords for ranking as AltaVista dropping that in July 2002. (Death of a Meta Tag).

Most people who follow SEO practices closely can tell you this, just like I did. Even SEOmoz on their Search Engine Ranking Factors list places the importance at 1.2 out of 5, with the comments from experts indicating that it's barely worth your time.

As for backlinks, I have over 800 for the site YOU are referring to according to Google WMT

That's good. Google's public facilities are a little constrained for various reasons. However, the huge ratio difference (2:800+) in private to public count indicates to me that very few of those links carry any weight. In other words, they are doing very very little to help your ranking.

Also, who says I use random h1/header tags when all headers are for the topic content.. relevant!

But do they make a proper outline? Remember that header tags (h1, h2, h3, ...) are for headers. Remember that search engine indexing doesn't quite have the spacial reasoning of a human. Tags help provide hints on how the content is structured. If you give those hints, make sure they make sense.

As a practice, open the source from a random page and go through your header tags from top down. Create an outline based on these tags. Does that outline make sense? If it does not, then you are probably confusing the search engines. In my opinion, it is better to offer no outline guidance than to provide confusing guidance.

I tried to update but like usual it says forbidden.

That sounds like a hosting issue, possibly an issue with mod_security. You really should contact your hosting provider.

Again.. Motoko-chan, I understand you defending SMF against this problem and replying with what I consider excuses, but 15 pages of proof must say something right?

Point to the proof in these pages. Most of it is arguing over keywords in URLs, meta tags, and other factors that even the true experts in the field say is of low use. Show me one good post that has proper citations and shows something SMF should be doing.

Anyhoo, I love SMF and will stick with it forever!

Glad to hear it. You might also be glad to hear that our upcoming 2.0 release has had many changes to help improve indexing of topics and posts.

Also, if you want a quick guide of good optimizations/changes to make to help get indexed better, that SEOmoz article is quite good (even if it was back in 2007).  SEOmoz: Google Search Engine Ranking Factors