Do SMF posts appear in search engines? (Something to be concerned about)

[nitins60]

I found one site www.scriptsocket.com! There is a free sitemap generator! It will be useful for who want to submit sitemap to google

[青山 素子]

I personally use GSiteCrawler for making my sitemaps. It has some nice reporting options that are useful in many occasions.

[Ben_S]

Have you already tried that in your site, Ben_S?

Went to your Liverpool FC forum, congrats, that's a really huge forum. But still are you not concerned  that out of 2.3 million posts, only 1,000+ entries appear in Google search results, and most are not even threads or posts?

http://www.google.com/search?hl=en&q=site%3Aredandwhitekop.com%2Fforum&btnG=Google+Search

Google.com - site:redandwhitekop.com/forum

Results 1 - 10 of about 1,640 from redandwhitekop.com/forum for . (0.04 seconds)

Back on the climb since adding profiles to robots.txt

http://www.google.com/search?num=100&hl=en&safe=off&c2coff=1&client=firefox-a&rls=org.mozilla%3Aen-US%3Aofficial&hs=0Xm&q=site%3Aredandwhitekop.com%2Fforum&btnG=Search
Results 1 - 100 of about 30,900 from redandwhitekop.com/forum. (0.07 seconds)

Certainly scope for improvement but so far it's going up every day.

Ranking well for terms such as torrent help (result 3) despite not being a site remotely about torrents, which is nice.

[destalk]

Indeed. I found that excluding profiles was pretty crucial. Much more important than excluding .msg urls IMHO. In fact I still don't exclude .msg urls. It speeds up indexing and Google seems happy and capable of resolving this to the 'root' url most of the time. I tried leaving the noindex code from SMF 1.1 for a few weeks and got a sharp reduction in indexed files and visitor traffic.

Could be a coincidence of course. You do one thing, Google does this, you do another thing... drives you barmy in the end. Best to concentrate on content..

[khoking]

My site has more than 1200+ index at google.com now! Last week or two weeks ago was only 300+...I am very happy and hopefully google will index my site more and more soon!

site:pentaxworld.com

My site is only 2 months old, with Total Posts: 4918. Of course google is not indexing all posts/threads but at least I can see some threads being indexed now.

I dont use SEO, just the default SMF URL type. If they can index SMF site, I believe they should be able to index mine as well

[Farmacija]

I found one site www.scriptsocket.com! There is a free sitemap generator! It will be useful for who want to submit sitemap to google

no, this generator doesn't index all pages of forum

[destalk]

Many people have commented about issues with the SMF forum software and Google. In the past there do seem to have some indexing issues. Things I have noticed people express concerns about include;

- threads being indexed with duplicate content due to the .msg url system.
- profiles being given undue priority in indexing whilst threads remaining unindexed.
- threads not being indexed at a level below the main boards.
- threads being put into Google's supplemental index.

Well I checked the latest SMF forum listings in Google today and they are impressive, regardless as to how you look at it;

site:www.simplemachines.org/community

http://www.google.com/search?hl=en&q=site%3Awww.simplemachines.org%2Fcommunity&btnG=Search

and

site:www.simplemachines.org

http://www.google.com/search?hl=en&q=site%3Awww.simplemachines.org&btnG=Search

Some points in particular that I noticed have changed in these results;

1. The majority of the first few search results are ordinary threads, not member profiles, which always used to be a problem in Google. This is especially interesting as the simplemachines.org web site does not exclude member profiles via robots.txt. Be interesting to know if these changes are due to any kind page structure change of the SMF development edition, or just Google getting better at indexing the site?

2. Many of those threads are indexed at the 'root' of the url, rather than the .msg version. Which is a testament to the implementation of of the noindex feature of the new version of SMF. Still some way to go, but that's just a matter of waiting for Google to get around to update its index.

3. There are very few pages in Google's supplemental index - the bulk are in the main Google index. A sign that Google sees each topic as having 'unique' content. I.E. No duplicate content issues.

4. There are no new PHPSESSID indexed URLs. Which shows that other changes to the forum software a while ago are working well.

It seems clear that even the default SMF installation is being spidered quite well by Google.

[Dannii]

I have to agree with that

[Joshua Dickerson]

I actually get completely different results. 7 on /community and 14 on the site root. 110,000 displayed in the supplemental results. Might just be my Google server.

[Daniel Hofverberg]

I get the same as groundup.

[destalk]

110,000 displayed in the supplemental results

How strange. That's what I got last month.

List of Google data centres if anyone is curious.

http://www.vaughns-1-pagers.com/internet/google-data-centers.htm

And it is true, on some data centres I also get only 7 results for site:simplemachines.org/community

The rest are not supplemental though. You just need to click on more results. Which does kind of indicate a 'similar content' issue.

But perhaps we still need to watch this space for a while. Let's hope it's going in the right direction.

[khoking]

I believe with more and more people using SMF, google will change their index structure on SMF forum and make it better index.

Search engines need to be smart in indexing (showing up good search results), so that people will like or love it more. If not, then there might be another Oogle coming out soon...

[GPGrieco]

I am not sure about vbullitin, but smf comes with no form of SEO on the sites. If someone wants to be ranked high then they would need to have some SEO done to their site. If any one needs it, I can do SEO. If you would like more info please e-mail me at GPGrieco(at)msn(dot)com

[metallica48423]

I believe with more and more people using SMF, google will change their index structure on SMF forum and make it better index.

I highly doubt that they will change their indexes for the benefit of one group of people or one software.

I don't see that happening

[Dannii]

I am not sure about vbullitin, but smf comes with no form of SEO on the sites. If someone wants to be ranked high then they would need to have some SEO done to their site. If any one needs it, I can do SEO. If you would like more info please e-mail me at GPGrieco(at)msn(dot)com

That's not true. SMF now uses a meta noindex rule very effectively, which is probably one of the most important SEO practices.

[Isaac]

I am not sure about vbullitin, but smf comes with no form of SEO on the sites. If someone wants to be ranked high then they would need to have some SEO done to their site. If any one needs it, I can do SEO. If you would like more info please e-mail me at GPGrieco(at)msn(dot)com

Why don't you just share your knowledge with all of us?

[klra]

Ok, I'm going to resurect this thread as I can't find anything more relevent.

Currently running SMF 1.1.4
Posts- 48070
Threads - 4819
Forum age - 1 year

Most of the forum is private (to protect the tech content, long story).

My main issue is that the forums aren't showing up on Google search when the 2 most important keywords are used for a search - KLR650 & forums.

Yahoo doesn't see it either, though MSN sees it as the 4th on the results.

I really don't care if Google indexes the threads of the forums, what I really want is the main forum index to show up when a basic search is done.

It's the 2nd largest KLR650 forum-

http://klrworld.com/forums/

Any help/recommendations are appreciated !

[klra]

I'll also add that I have seen up to 130 spiders crawling at once, and usually at least 5 are there at all times.

[falguni1]

hmmmmmmmmmm important

[metallica48423]

Do you have adequate backlinks (other sites linking to you)?  If not, you will not be indexed very highly in google's eyes.

By your restrictions, you are also limiting what can be indexed of your page, this can and probably will affect your indexing and position as well.  I would reccommend NOT using the disallow guests option.  Instead disallow them entry into the boards you don't want them to see by removing the board access from the guests group.

Using an SEO mod would have almost no effect for you since you are effectively blocking the content.  Without unique content you will not be indexed very highly.  Having a page doesn't guarantee good indexing .