Do SMF posts appear in search engines? (Something to be concerned about)

Started by geezmo, September 06, 2006, 09:16:08 PM

Previous topic - Next topic

Harzem

Google is funny. There are thousands of SMF forums using "Dilber MC" theme, and all have a small copyright text at the bottom right. When I searrch google for "Dilber MC", I don't get them. When searching via yahoo, I can see almost all of them. Google started to be too picky.

geezmo

R-S_Doug, thanks buddy for the extra publicity by using me as example ;) Yup, I've mentioned that same problem in the first page of this thread. Very, very, very few posts and threads appear as a serp in Google, what appears normally are the profile and user statistics info in SMF.

HarzeM, I've posted the same observation in this thread. Some SMF posts appear in Yahoo but there's usually none or very few in Google. There's something in the Google search ranking engine that hates SMF posts. Unfortunately, more than 80% of internet users use Google to search so it's not that much value to appear in Yahoo or MSN or Altavista. I only need my SMF posts to rank high, or simply just appear as a serp, in Google, that's it.

Joshua Dickerson

Geezmo: I agree that SEO is a major webmaster responsibility. I think it ranks up somewhere under good content (#1), usability (#2), then acccesibility (#3), so maybe #4 ;)

If you have any suggestions on how to make SMF more SEO'd, we are all ears.
Come work with me at Promenade Group



Need help? See the wiki. Want to help SMF? See the wiki!

Did you know you can help develop SMF? See us on Github.

How have you bettered the world today?

geezmo

Hi groundup, I think right now we can only guess. Nobody knows how Google's search engine works exactly so we won't know why it dislikes SMF posts.

The other guys have made a few guesess, such as the duplicate content issue, and I just tried using the codes Ben_S gave for the robots.txt file. It will take months before we see any results.

I think one good example is webdigity's site. Posts in his SMF forum are appearing in Google because of his Archive mod and other modifications. Not sure though how long it took the forum for those to appear as SERPs. I've been using his archive mod for more than a month now. No results in Google yet, but I'm crossing my fingers.

Toadmund

Welp, I'm giving this a shot:
http://mods.simplemachines.org/index.php?mod=339
I hope it searches for the meat under the plastic wrap (topic catagories), at least when I 'view source', the meta tag content has the topic subject title (name of the meat).

ie Title: Deliver the Liver (which is in the meta tag)
Subject: Liver is awesome with onions on buttered bread! Liver has that velvity powdery yummyness whose flavour bursts out with onions, bread and butter, of course, some salt, pepper, garlic and soya sauce couldn't hurt!

Which I think is good (that it's in the meta tag, liver too).

Now, I just have to wait a few weeks to see what happens, knock on wood, or 'pound on keyboard' ;)

Here is what my archive looks like:
http://www.reversedisorder.com/forum/archive.php

R-S_Doug

Toadmund,
Please note that the archive mod at no 339 in the SMF mods list will not work with SEO friendly URLs, as it uses query strings. If you email Nikolas at webdigity, or register on his forum and pm him, he will send you the code for his archive mod, which will.
Doug.

Toadmund

I am doing er, have done just that R-S_Doug.
I hope my email made it, I had issues with thunderbird, think it's fixed now though.

Garry1953

I'm also new to SMF, and have been trying to find out why the forum wont show in Google search results. Its interesting that both Yahoo, and MSN seem to find it OK.

P.S. Iv'e also sent a request for the mod to Nikolas.

Ben_S

Quote from: geezmo on October 26, 2006, 10:18:01 AM
Google.com - site:redandwhitekop.com/forum

Results 1 - 10 of about 1,640 from redandwhitekop.com/forum for . (0.04 seconds)

Update, results initially dropped down to about 700 or so when the .msg pages etc were dropped but is now rising by about 300 results a day, currently on 3,580 so hopefully within a few months there will be a decent amount of topics listed.

SMF 1.1 Final will have a meta robots noindex tag on the .msg, print pages etc which should have the same effect as the robots.txt.
Liverpool FC Forum with 14 million+ posts.

Farmacija

www.farmaceuti.com
www.farmaceuti.com/tekstovi

alchemy

action=printpage

still indexing in SMF 1.1 Final

Guess we still need to add some of Ben_S suggestions to robots.txt

Farmacija

and i think that some wap version of smf are better index by google then version for internet browsers
www.farmaceuti.com
www.farmaceuti.com/tekstovi

Sverre

Upgraded our live forum to 1.1 earlier today, and it seems Yahoo likes the change A LOT. If the insane amount of crawling results in better indexing remains to be seen though...

Farmacija

Yes like yahoo bot go crazy . in my forum there is 10 times more yahoo bots then before upgrading on 1.1.
www.farmaceuti.com
www.farmaceuti.com/tekstovi

青山 素子

Yahoo's bot just likes to spider sites. I've had it almost take down some sites before (spidering about 20 sites running heavy db-driven stuff on the same server = insane loads).
Motoko-chan
Director, Simple Machines

Note: Unless otherwise stated, my posts are not representative of any official position or opinion of Simple Machines.


Farmacija

yes but so much yahoo bots at one time... it doesnt happen before, with 1.09 version...
www.farmaceuti.com
www.farmaceuti.com/tekstovi

Lilac

Elliquiy Adult Role playing is a mature roleplaying community with some 500,000 posts in over 100 boards.

I don't know everything.  Please ask questions in the support forums instead of PMing me!

forumposters

I get visitors from Yahoo, MSN, and Google.  However, most of my visitors come from Google for the query "forum posters".

KGIII

I'm not sure how well this will format:


Google    4346      49.4 %    4352      48.2 %
MSN       1847      21 %      1888       20.9 %
Yahoo     1618     18.3 %    1642       18.1 %


The forum's been up and running (after a rather long period of downtime) for just about 2 months. The above is pulled from the AWStats script for the month of November - the only full month we've had with the new forum in use.

I took a trip to Google and specified the domain and then +reply to ensure it would only show me if it listed the subjects themselves:

http://www.google.com/search?hl=en&lr=&q=site%3Aforum.kgiii.info+%2Breply

(I'm pretty sure this isn't going to be nearly as helpful as people were hoping.)

Anyhow...

That listed 130 results which properly indicates the 130 topics (minus a few in a private section and a couple of new ones that haven't been indexed yet) that are available to it. Err... It has done this since I first installed SMF? The only changes that I've ever really made were to actually swap out the meta tags in there so that they represented the content better and, honestly, Google doesn't give any great weight to meta tags anyhow seeing as the entire online world decided to mis-use them.

I only have a PR of 4 the last time I looked, up from a 0 in the start of November.

I guess what I'm saying is that maybe you're trying too hard or doing something that you shouldn't be doing? I haven't done anything, there aren't any tips, tweaks, or major changes I've made. My results aren't spectacular but the forum was just re-created in October after having been down for quite a while so who knows what it will turn into?

Most of the traffic comes looking for stuff about IE7, Vista, or DVD/CD problems. Google traditionally ranks anything I ever touch really low while MSN/Live Search seems to love anything I do. I really don't have any great pearls of wisdom here - just that, well, I didn't make any great effort to get indexed or the likes.

Edited to fix up the code section for readability.

My PC Support Forum
Please ask in-thread before PMing
                   SMF Help
                   Visit My Blog

How can we improve the support process?:
http://www.simplemachines.org/community/index.php?topic=163533.0

SMF vs. Godzilla? Who do you think will win?

Witte

Hmm, just checked my stats on google webmaster, I now have around 700 pages listed, and am getting hits from many different google sites...Belgium, Canada, france, Germany, Hungary, India, Holland, Sweden UK, and the US. I made a sitmap using a tool that removes the sessionID crap from the url, I think that helped a lot. I also submitted the sitemap to Yahoo, and within a couple of days I have up to 67 spiders on my site. Googlebot visits more than once a day.

I am on 1.1, and decided to remove the SEO mod...because of the robots-noindex that was added in final. That had me worried at first, it looked like it would just tell all the robots not to index the page...I hope I am right. I also have the archive mod from Nikolas, with the link to it as the first link on the page...

Just checked yahoo, and they have indexed 291 pages (From the archive and the normal site), but I only submitted to them last week...

Advertisement: