Does GOOGLE spider SMF without mod?

[zom]

I was with phpbb and change to SMF. I'm with SMF for 15 days now and google stil does not index posts. (Under phpbb all posts where indexed without problem every day).

The forum has now about 20 000 messages and this indexing is a problem. Is there a mod a anything to help google spidering ?

thank you

[Cypher7]

mine is not running any mods and it is indexing fine... search for tkd fighters needed in google, which is one of my topic subjects and you'll see it's first in the list... ok granted it still shows my old Yabb.cgi link (which you can still click on and will bring you to SMF, but at least the second link is index.php !!

[zom]

for me I don't understand why messge are the profile ones, with message by author...
But normal pages are not indexed... 
what explaination for that?

my SMF:
http://www.celebrites-selection.com/phorum/

[Burpee]

Hmm.. I don't know... the tdk fighters link doesn't make google go to the thread directly either...

Personally, I set the option to display links like this: index.php/topic,344.new

The only publically available posts on my forum are being found by google ... so you might want to try enabling that option in your settings, if you really need to have it fixed quickly... it's apache servers only though...

Also, just tested this board on google...

[Ben_S]

http://www.google.com/search?q=allinurl:++site:simplemachines.org&hl=en&lr=&ie=UTF-8&safe=off&as_qdr=all&start=0&sa=N&filter=0

[Cypher7]

Hmm.. I don't know... the tdk fighters link doesn't make google go to the thread directly either...

I think the reason it still shows the Yabb one is cos that post was started when I had my board as Yabb gold. Now my yabb.cgi file (which is still uploaded) is a redirect to /forum... which is why the link goes to the board index.

this google search shows a few links to my main site, and also a link to a topic in the forum...

[Skoen]

Every three days I see about 15 Google robots crawling my forum. And I'm not running any mods. It's the meta tags that also helps alot. If you have the same meta tags in your forum as you have on your main website, plus a few links directly to the forum from the main site and otherwise, Google will crawl your site every once in a while to keep itself updated with your hits. Plus, you are very likely, if you have your meta tags set properly, to become one of the first ten hits on almost every search word that contains items you know is on your forum.

[Ben_S]

Unless things have changed, Google places very little value on meta tags.

[Cypher7]

True Ben... I get most of my hits from strategically used keywords on my main page...

[Thunderace]

I was with phpbb and change to SMF. I'm with SMF for 15 days now and google stil does not index posts. (Under phpbb all posts where indexed without problem every day).

The forum has now about 20 000 messages and this indexing is a problem. Is there a mod a anything to help google spidering ?

thank you

Unless you are a high PR site google wont dig very deep.

Make the forums 2 levels down max, my advice would be to build a site map with a "one link" access to each of your forums, google will start indexing from there I recon.

[kegobeer]

I've found the best way to get spidered is exposure, exposure, exposure.  I belong to a bunch of community forums, post quite a bit (always put my site's url in my sig/profile), and always use the same username.  The more popular the sites you post on, the more traffic you'll generate.

Seems like Google has indexed my site pretty well with no mods.  For example, if you type DezinaControl in Google, both results link to posts on my site.

[Thunderace]

I've found the best way to get spidered is exposure, exposure, exposure.  I belong to a bunch of community forums, post quite a bit (always put my site's url in my sig/profile), and always use the same username.  The more popular the sites you post on, the more traffic you'll generate.

Seems like Google has indexed my site pretty well with no mods.  For example, if you type DezinaControl in Google, both results link to posts on my site.

That's called link popularity  always get as many links incomming as possible, as long as they're high PR and relevant, and make sure your links have tags that match your keywords. (link reputation)

[zom]

Hmm.. I don't know... the tdk fighters link doesn't make google go to the thread directly either...

Personally, I set the option to display links like this: index.php/topic,344.new

how do you do that?

[Thunderace]

SMF works fine like all php scripts a site map speeds things up.

If you want more use mod_rewrite rule (find on the web)

[zom]

could it be possible to have a sit_map page to help indexing like with phpbb. This php page contains all the leads to all the posts directely. If you know  scripting this could help: here is the code for phpbb:


<?php
/***************************************************************************
* Save this file as:   site_map.php (or anything you like)
* Version:      Friday, Oct 4, 2002
* Email:      [email protected]
* Purpose of hack:   Basically generates a list of topics and
*      displays them with link to the topic. Goal
*      is to provide search engines like Google
*      with a static page of links to dynamic pages
*      You should link to this page from your sites
*      home page somewhere.
* Demo:      http://www.aussiecelebs.com/forums/site_map.php
* Tested on:   phpBB 2.01, 2.02
*
***************************************************************************/

/***************************************************************************
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
***************************************************************************/

echo "

<!-- Start header here. Make sure you dont use any \" s -->

<html>
<head>

</head>
<h1>TITLE</h1>
<!-- End Header -->

";

define('IN_PHPBB', true);
$phpbb_root_path = './';
include($phpbb_root_path . 'extension.inc');
include($phpbb_root_path . 'common.'.$phpEx);

echo "<h2>Catégories</h2>";

$result = mysql_query("SELECT cat_title, cat_id FROM phpbb_categories ORDER BY cat_title");
while($row=  mysql_fetch_assoc($result))
{
      echo "<a href='index.php?c=".$row["cat_id"]."'>".$row["cat_title"]."</a><br>";

}

echo "<h2>All forums</h2>";

$result = mysql_query("SELECT forum_id, forum_name, forum_desc FROM phpbb_forums ORDER BY forum_name");
while($row=  mysql_fetch_assoc($result))
{
      echo "<a href='viewforum.php?f=".$row["forum_id"]."'>".$row["forum_name"]."</a>"." - ".$row["forum_desc"]."<br>";

}

echo "<h2>All posts</h2>";

$result = mysql_query("SELECT topic_title, topic_id FROM phpbb_topics ORDER BY topic_title");
while($row=  mysql_fetch_assoc($result))
{
      echo "<a href='viewtopic.php?t=".$row["topic_id"]."'>".$row["topic_title"]."</a><br>";

}

echo "

<!-- Start footer here. -->

</body>
</html>

<!-- End footer -->

";

?>

[Shadow's Pawn]

I have to highly recommend the mod_rewrite function.  You'll have to rewrite your urls in the source files, but I did this on one of my sites which has almost 60,000 posts and every single post is on google.  So are all my user's profiles.  This gets my site very high rankings for a wide variety of search terms.

[zom]

does anyone have an .htaccess for those mod rewrite ?

[flyingmembers]

errmm, how do you do this mod rewrite function?

thanks

I have to highly recommend the mod_rewrite function.  You'll have to rewrite your urls in the source files, but I did this on one of my sites which has almost 60,000 posts and every single post is on google.  So are all my user's profiles.  This gets my site very high rankings for a wide variety of search terms.

[SeaOfSin]

I would like to know about mod rewrite as well!  I have quite a few topics show on google, but no posts showing.

Sea Of Sin

[[Unknown]]

Search for mod_rewrite.

-[Unknown]