Do SMF posts appear in search engines? (Something to be concerned about)

[Kǝmac]

Is it possible to delete the comma in the search engine friendly URLs?

It won't change anything because search engines don't get SEF URLs (I've reported that to devs long time ago). Turn off cookies in a web browser and check it. Problems with SEO are probably caused not by a form of URLs, but by a enormous amount of diffrent links produced by SMF.

[KGIII]

I am boarderline convinced that Google WANTS you to rank poorly - that way you want to buy ads.

That is a popular conspiracy theory.  Look at it this way:  it used to be easy to get indexed in Google and returned in the search results.  Now it is very difficult.  What changed?  They went public.  They now have to generate huge profits to justify their astronomical stock price. 
[/quote]

I am not personally big on theories without evidence. One thing I have noticed lately is that I keep seeing ads for Google's products listed in their AdSense ads which, as near as I can tell, is a giant conflict of interest if not entirely immoral as well. I was a fan of Google until they started doing everything that was against what they were when they started. Now I mostly get junk when I search there.

[confusion]

Google is, for better or for worse, THE search engine.  They dominate the mindshare of web surfers.  They got that way because they were highly innovative in the results they provided.  Because of that, everyone and their dog is trying to get their web sites to the top of google's SERP.  Google has had to respond by being very selective about what they index, what they don't, what yields placement improvement and what consitutes someone trying to game them.

Be very clear - Google is one of the largest advertising firms.  They also have a search engine.  And an email service.  And a suite of on-line office tools.  And collaboration, development, imaging, payment, financial, online advertising and community tools.  They fully intend to become the Internet.  That's not a conspiracy theory - they want to become to the internet what Microsoft is to the desktop.  They are full of incredibly smart people, and I believe they are quickly obtaining their goal.

But, it's important to understand that the search engine is just a component of google, and one that doesn't make a lot of money at that, directly at least.   

[nitins60]

Whatever it maybe, my site is submitted to all search engines throug a custom made script, it submitted to all SE. Even Google indexed my site! Problem s, it's not indexing ONLY forum topics/related things. All other pages which are not related to FORUM are listed

[Toadmund]

Care to share your custom made script? PM me?

Anyway, how the hell can one's site ever get popular if google doesn't list it adequately enough?
Word of mouth, other forums i guess?

It's like the proverbial saying: "How can I get the job experience that they want, if I can't get a job doing that job?"

[Dannii]

Word of mouth is always the best way to get popular.

[KGIII]

Word of mouth is always the best way to get popular.

Amen. I guess they call it organic now. Go figure?

[Dannii]

And btw, incoming links is sort of like the internets' version of word of mouth. Spam is the internets' version of viral marketing. BOOO@Spam!

[KGIII]

And btw, incoming links is sort of like the internets' version of word of mouth. Spam is the internets' version of viral marketing. BOOO@Spam!

Bah... It is the chit chat section.

Boo is installing a billion and ten lines of code manually for a killer mod. Thanks again and worth all the frustration and even hand-typing some I missed only a comma.

I am getting better I tell ya!

[motumbo]

And btw, incoming links is sort of like the internets' version of word of mouth. Spam is the internets' version of viral marketing. BOOO@Spam!

No, incoming links are not like word of mouth.  Incoming links, these days, is typically the result of one webmaster linking to another webmaster, often reciprocally, in hopes of making Google think they are more popular than they are.

True word of mouth is one web surfer telling another web surfer about a cool website, not one web publisher asking another web publisher for links.  Most web surfers are not publishers.  Most people looking for information want the information they are looking for and are not the least bit interested if the website has two thousand incoming links or zero.

If you go to the library to do research for a college paper, do you want books and resources that give you the information you need to do your task or do you want the library to only stock what is popular with other library patrons?

Google heavily weighs incoming links.  Google tells webmasters to get more incoming links to get into its search engine.  Google is working against itself by advising webmasters to engage in behavior designed to do nothing but trick Google.     

[confusion]

Google heavily weighs incoming links.  Google tells webmasters to get more incoming links to get into its search engine.  Google is working against itself by advising webmasters to engage in behavior designed to do nothing but trick Google.    

Google uses incoming links to establish authority on a subject.  In the early days, you could get away with using link farms to get your name on the map, but that's just not so any longer.  The pagerank of the site linking weighs very heavily.  If you have 2 or 3 PR >5 sites linking to you, that's going to do more for your SERP than hundreds of PR < 2 sites.

People are constantly finding ways to "game the system" with google.  And yes, there are quick ways to get to the top 3, but you won't stay there.  SEO forums are FILLED with comments like: "I did XXX and went from page 10000 for my keyword to #3 in 2 weeks, but now my site has disappeared!  please help!".  It's like the story of the bull and the turkey.  BS can get you to the top, but it won't keep you there.

The current popular scams now are:

multi-tiered link farms, where I link to you, you link to someone else, they link back to me, all as part of an arrangement.  It's hard to correlate that kind of activity, but I believe google has recently started to catch on.

Writing articles.  This started out (IMO) as a fairly honest thing years ago where people could get their (and their site's) name out by writing authoritative articles.  Nowadays, you get people writing about how to shave their dogs, how to get your oil changed at jiffy lube, and crazy stuff like that.  There's still some good stuff there, but google will ultimately be forced to penalized the whole process, IMO.

Domain farms.  Domains are cheap.  I've done this.  I own over 50 domains personally, though not out of the intention to game google, but in the misguided belief that I will do something with them all.  But, I CAN get a new site indexed and part of the SERP in under a day, which is impressive, IMO.  I'm not that sophisticated, so I can imagine that someone who is determined and makes this their livelyhood could likely leverage this technique well.

Save any fundamental problems with the software that is causing google to unfairly penalize the site (seems like it MAY be the case with SMF), doing the things that make your site a REALLY useful resource for your keyword targets is the only road to long term success.  Good content, active site, no spam, etc, and a bit of marketing will spur other quality sites to link to you.   

[motumbo]

Google heavily weighs incoming links.  Google tells webmasters to get more incoming links to get into its search engine.  Google is working against itself by advising webmasters to engage in behavior designed to do nothing but trick Google.     

Google uses incoming links to establish authority on a subject.  In the early days, you could get away with using link farms to get your name on the map, but that's just not so any longer. 

Nothing could be further from the truth.  I know of a few websites that have set up tons of websites all linking to the main site. 

I found an interesting Google group called "Google Groups: AJAX World" while searching Yahoo.  So, I searched Yahoo and Google using the name of the Google group and this is what I found:

Yahoo has the Google Group in position 1.

Google:  Not in top 100.

Google does have a French Ajax World Google Group around position 15.  But it isn't the group we are looking for.

Yahoo does a better job finding Google Groups than Google does!!!

[confusion]

Nothing could be further from the truth.  I know of a few websites that have set up tons of websites all linking to the main site. 

Absolutely, it's still done.  It doesn't have the effect it used to on SERP/PR status, though.  If you are not careful, it can get you sandboxed.

I found an interesting Google group called "Google Groups: AJAX World" while searching Yahoo.  So, I searched Yahoo and Google using the name of the Google group and this is what I found:

Yahoo has the Google Group in position 1.

Google:  Not in top 100.

Google does have a French Ajax World Google Group around position 15.  But it isn't the group we are looking for.

Yahoo does a better job finding Google Groups than Google does!!!

My comments were not a plug for the accuracy of google as a search engine.  I believe google has had to implement so many crazy algorithms to avoid spammers that you are far better off searching yahoo or msn if you really need to find something.  The fact remains, though, that despite it's troubles, it is THE search engine people use, and as such, it's what site owners care about.

Take for instance my syslog site.  On google, I am #3, on MSN I am #4 and on yahoo I am #2.  An analysis of the web server logs over the past 4 years should >92% of search engine referrals come from google and it's pretty steady month-to-month still.

So, yes, it is easier to get your site on "my keyword" in the top 5 on msn or yahoo, and it is also true that people looking for information on "my keyword" would be better off searching in msn or yahoo, but few people actually do.   

[Ensiferous]

Absolutely, it's still done.  It doesn't have the effect it used to on SERP/PR status, though.  If you are not careful, it can get you sandboxed.

You don't get sandboxed, you start out sandboxed, once you're out you're out for good (unless domain changes owner etc) you can however get removed/penalized heavily by overdoing stuff.

One thing I noticed though is that all pages have the same keywords, thus increasing the changes of pages being in supplemental hell.

[motumbo]

One thing I noticed though is that all pages have the same keywords, thus increasing the changes of pages being in supplemental hell.

My understanding from reading Matt Cutts blog is that one of the primary determinants of whether or not your pages end up in supplemental is the number of incoming links to the page (another determinant is the number of parameters in your query strings). 

Most of my pages are in Supplemental.  Matt Cutts claims that being in Supplemental isn't bad.  But my experience has been that I can't find my Supplemental pages in the search results.  If my page is indexed and in Supplemental, I can't find it even if I search for a phrase in quotes on the page. 

Google's Supplemental Index is bad news.   

[motumbo]

My comments were not a plug for the accuracy of google as a search engine.  I believe google has had to implement so many crazy algorithms to avoid spammers that you are far better off searching yahoo or msn if you really need to find something.  The fact remains, though, that despite it's troubles, it is THE search engine people use, and as such, it's what site owners care about.

I'm glad to hear that you are a sensible person.  You are correct that Google is THE search engine.  It is used by close to 50% of web searchers.

However, does Google need to be THE search engine?  Would it be possible for enough webmasters to get together and encourage our users to ditch Google in favor of Yahoo or Live because Google has so many problems, puts pages in Supplemental Hell, doesn't put pages in SERPS for lack of incoming links no matter how relevant the page is, etc?

The nice thing about the internet is that the little guy can have a big voice.

Take for instance my syslog site.  On google, I am #3, on MSN I am #4 and on yahoo I am #2.  An analysis of the web server logs over the past 4 years should >92% of search engine referrals come from google and it's pretty steady month-to-month still.

Google's marketshare isn't that high!  Does everybody find you using the same keywords?  Perhaps certain demographics slant toward Google.

So, yes, it is easier to get your site on "my keyword" in the top 5 on msn or yahoo, and it is also true that people looking for information on "my keyword" would be better off searching in msn or yahoo, but few people actually do.   

For now...

There often exists a commonality in what is good for two different parties.  It is best for search users and webmasters alike that searches be done not with Google.  At the very least, I'd hope that a bunch of webmasters publically stating their case about Google would ecourage them to get their act together and go back to what they used to be.

[moraidh]

I selected no indexation by google when I installed my forum.

Reading here, I think this may have been a good idea.

My site's main articles are indexed by Google, so it's not a problem that the forum isn't if spambots rely on Google indexation for their databases.

I haven't had a single spam registration since changing from phpBB2 to SMF. 

[majo]

Have you already tried that in your site, Ben_S?

Went to your Liverpool FC forum, congrats, that's a really huge forum. But still are you not concerned  that out of 2.3 million posts, only 1,000+ entries appear in Google search results, and most are not even threads or posts?

http://www.google.com/search?hl=en&q=site%3Aredandwhitekop.com%2Fforum&btnG=Google+Search

Google.com - site:redandwhitekop.com/forum

Results 1 - 10 of about 1,640 from redandwhitekop.com/forum for . (0.04 seconds)

maybe you should try next time as :

http://www.google.com/search?q=site:redandwhitekop.com&hl=en&start=70&sa=N

not /forum  

[Slamlander]

All I can say is that if you're not concerned with your site not ranking high in the search engines, it means you're merely a newbie webmaster or your site caters to only a few people. Ranking in search engines is a must if you want more traffic and more earnings.

Only if your earnings are ad-based. I won't/don't run ads.

[geezmo]

@Slamlander, again it's all about organic search. If searchers can't find your site via a search engine, there's a big problem with your site. Of course, you can say you can use Adwords, but for those who can't afford to advertise right now, organic referral is one, if not the only, way visitors can discover the site.

@majo, the /forum suffix was used in that search because we were only testing how many pages in the forum (thus /forum) appear as SERP. Using "http://www.google.com/search?q=site:redandwhitekop.com&hl=en&start=70&sa=N" will give you results such as articles, blog pages, etc. which are not the subject of this forum test. Again using the /forum site search, only 6,700 results appear in Google considering the fact that more than 80,000 topics and 2,000,000 posts are in the forum.