Search Engine Friendly URLs

[CA3LE]

First of all SMF is the shiz!  Converting from phpBB has shown me all the stuff I was missing... I will never look back!

NOW, on to the issue

Am I missing something here?  I have read many posts about this but am still scratching my head.

All I want is for the URL's to look like

THIS
http://testmy.net/forum/index.php/topic,1628.html
(better yet http://testmy.net/topic,1628.html)

NOT THIS
http://testmy.net/forum/index.php?topic=1628.0


Linux server with Apache 1.3.33
PHP 4.3.10
Show URLs without ?'s is checked

I have even at one point commented out and added to /$SMFROOT$/Sources/QueryString.php

//   elseif (!empty($modSettings['queryless_urls']) && !$context['server']['is_cgi'] && $context['server']['is_apache'])
      elseif (!empty($modSettings['queryless_urls']))

(sadly did nothing for me)


phpinfo()  if you need it  -------  just ask what portion you want

Can you see why this is not working with my server, does it need to be in SAPI mode... from what I have read it does, can someone help me to understand how I can get PHP in sapi mode.   Or whatever I need to do.  I run a dedicated server so it's not a matter of "calling my host" ... anything that needs to be done I can take care of, I just need a nudge in the right direction.

(oh yea.. real quick, whoever helps me... spell check does not work either... maybe related? -- lemme know what I need installed for this, I read a while back but have forgotten.)

Can someone please help 

- Thanks in advance!

[Oldiesmann]

I don't know enough about PHP to tell you how to get it in SAPI mode, but [Unknown] can probably tell you that. As far as spell check - you need to have ASpell installed and then re-compile PHP with "-- with-pspell" in the configure command... ASpell can be downloaded at http://aspell.sf.net

[Alexandre P.]

For the Search Engine Friendly URLs, in you Admin Center > Edit features and options > Basic Forum Features > Search engine friendly URLs, there should be a button.  A popup will open and tells you if this function will work on your forum.

[CA3LE]

clicked it .. and it says...

This changes the format of URLs a little so search engines will like them better. They will look like index.php/action_profile/u_1.

This feature will work on your server.

Close window

[CA3LE]

By the way... as for the spell checking I did compile and install aspell... but I forgot to re-build apache... DUR, now it works great.

[[Unknown]]

Could you give the actual link to your forum?

-[Unknown]

[CA3LE]

Could you give the actual link to your forum?

-[Unknown]

http://www.testmy.net/forum/


thanks [Unknown]

[[Unknown]]

Well, it does, at least, look like they should work on your server.

http://testmy.net/forum/index.php/board,5.0

Try this for the last function in QueryString.php:

Code Select Expand

// Rewrite URLs to include the session ID.
function ob_sessrewrite($buffer)
{
global $scripturl, $modSettings, $user_info, $context;

// If $scripturl is set to nothing, or the SID is not defined (SSI?) just quit.
if ($scripturl == '' || !defined('SID'))
return $buffer;

// Do nothing if the session is cookied, or they are a crawler - guests are caught by redirectexit().  This doesn't work below PHP 4.3.0, because it makes the output buffer bigger.
if (empty($_COOKIE) && SID != '' && (!$user_info['is_guest'] || (strpos($_SERVER['HTTP_USER_AGENT'], 'Mozilla') !== false || strpos($_SERVER['HTTP_USER_AGENT'], 'Opera') !== false)))
$buffer = preg_replace('/"' . preg_quote($scripturl, '/') . '(?!\?' . preg_quote(SID, '/') . ')(\?)?/', '"' . $scripturl . '?' . SID . '&', $buffer);
// You can't do both, because session_start() won't catch the session if you do.  But this should work even in 4.2.x, just not CGI.
else
$buffer = preg_replace('/"' . preg_quote($scripturl, '/') . '\?((?:board|topic)=[^#"]+)(#[^"]*)?"/e', "'\"' . \$scripturl . '/' . strtr('\$1', '&;=', '//,') . '.html\$2\"'", $buffer);

// Return the changed buffer.
return $buffer;
}

-[Unknown]

[CA3LE]

Well, it does, at least, look like they should work on your server.

http://testmy.net/forum/index.php/board,5.0

Try this for the last function in QueryString.php:

Code Select Expand

// Rewrite URLs to include the session ID.
function ob_sessrewrite($buffer)
{
global $scripturl, $modSettings, $user_info, $context;

// If $scripturl is set to nothing, or the SID is not defined (SSI?) just quit.
if ($scripturl == '' || !defined('SID'))
return $buffer;

// Do nothing if the session is cookied, or they are a crawler - guests are caught by redirectexit().  This doesn't work below PHP 4.3.0, because it makes the output buffer bigger.
if (empty($_COOKIE) && SID != '' && (!$user_info['is_guest'] || (strpos($_SERVER['HTTP_USER_AGENT'], 'Mozilla') !== false || strpos($_SERVER['HTTP_USER_AGENT'], 'Opera') !== false)))
$buffer = preg_replace('/"' . preg_quote($scripturl, '/') . '(?!\?' . preg_quote(SID, '/') . ')(\?)?/', '"' . $scripturl . '?' . SID . '&', $buffer);
// You can't do both, because session_start() won't catch the session if you do.  But this should work even in 4.2.x, just not CGI.
else
$buffer = preg_replace('/"' . preg_quote($scripturl, '/') . '\?((?:board|topic)=[^#"]+)(#[^"]*)?"/e', "'\"' . \$scripturl . '/' . strtr('\$1', '&;=', '//,') . '.html\$2\"'", $buffer);

// Return the changed buffer.
return $buffer;
}

-[Unknown]

Changed it.. still no luck .... ?

[[Unknown]]

Very strange.  It's working for a few of the URLs but not all of them.  I'm not sure why that would happen, as they look the same to me.

-[Unknown]

[CA3LE]

Just a follow up, I got this fixed.

For anyone who needs to know in the future... here was the fix for my specific application

I edited

Sources/BoardIndex.php
Sources/Display.php
Sources/MessageIndex.php

to reflect the new URLs for instance

Code Select Expand


'<a href="' . $scripturl . '?topic=' .


becomes

Code Select Expand


'<a href="' . $scripturl2 . '/topic-' .


and do the same format for topics if you'd like... don't bother changing throughout the entire forum, that isn't the point to this, message indexes and board indexes are all you need to change, what I have edited is actually more than I needed to do I think... but hey, it works great so I'm not going to get too picky.

and I of course defined

Code Select Expand


$scripturl2 = '/';


I don't remember where I added that... but it should be able to be added to index.php if your lazy.  The only reason I defined that instead of just leaving it $scripturl is because I wanted it escaped from the forum directory.


and added the following in .htaccess

RewriteEngine On
RewriteRule ^topic-([^/]+)[/]?$ /forum/index.php?topic=$1
RewriteRule ^board-([^/]+)[/]?$ /forum/index.php?board=$1

End result was the board URLs look like  >> http://testmy.net/board-5 and topics for instance look like http://testmy.net/topic-2160 -

By the way, I have noticed that this will still work with SMF even if you don't check "Show URLs without ?'s" in the options -- but don't fool yourself into thinking all is good... session errors occur if this isn't checked.

a few days ago Inktomi Slurp had over 300 bots in my forum spidering the crap out of it... then the other day I saw MSNBot, EchO!,  Walhello appie, WISENutbot, LinkWalker and Netcraft going deeper than I have ever logged.  Googlebot should be in there in a few days.. that one will be interesting.  Google has a cache of about 7200 pages of my site and I have about 21,000 posts total in the forum... I should be able to tell success when Google gets their hands on my site again and I re-query.  I will try and remember to post here with my findings ;-)

Thank you for your help... I guess I was just misunderstanding the use of that option.  I thought it would change all the links...

and [unknown] - from what I have read in this forum... you are a stand up kinda guy, congratulations.. and VERY nice scripting indeed.

[CA3LE]

in 8 days google has added over 2500 more of my pages .. I will keep updating... this is only the start because google and other engines have been in my forums very heavy the past week.

[Daevien]

Don't forget to add $scripturl2 to the global vars section as well at the top of the files

[CA3LE]

Don't forget to add $scripturl2 to the global vars section as well at the top of the files

Yea I did explain that wrong a bit... you actually can do it about 3 different ways, one including removing $scripturl2 completely... but you know what I meant  ;-)

Anyway, I checked Google again and a query for site:testmy.net is now showing 18,000+ entries ~ much better than the previous 7200 (or whatever it was) ~ this number is still on the upswing and Googles bots (along with all the other guys) are in my forum all the time now keeping stuff up-to-date ~~~ this change really boosted my visibility

[flyingmembers]

I see Ca3le came up with one solution, but having followed his posts I came up with a few problems...can anyone help me and go through this post again...I'm sure It'll benefit others.

Also you might want to check out:
http://www.submitexpress.com/analyzer/

Then enter your forum url and check how many pages are spidered, then start the changes below and then try it again...

If anyone can help me on this, that'll be great!

Just a follow up, I got this fixed.

For anyone who needs to know in the future... here was the fix for my specific application

I edited

Sources/BoardIndex.php
Sources/Display.php
Sources/MessageIndex.php

to reflect the new URLs for instance

Code Select Expand


'<a href="' . $scripturl . '?topic=' .


becomes

Code Select Expand


'<a href="' . $scripturl2 . '/topic-' .


and do the same format for topics if you'd like... don't bother changing throughout the entire forum, that isn't the point to this, message indexes and board indexes are all you need to change, what I have edited is actually more than I needed to do I think... but hey, it works great so I'm not going to get too picky.

and I of course defined

Code Select Expand


$scripturl2 = '/';


I don't remember where I added that... but it should be able to be added to index.php if your lazy.  The only reason I defined that instead of just leaving it $scripturl is because I wanted it escaped from the forum directory.


and added the following in .htaccess

RewriteEngine On
RewriteRule ^topic-([^/]+)[/]?$ /forum/index.php?topic=$1
RewriteRule ^board-([^/]+)[/]?$ /forum/index.php?board=$1

End result was the board URLs look like  >> http://testmy.net/board-5 and topics for instance look like http://testmy.net/topic-2160 -

By the way, I have noticed that this will still work with SMF even if you don't check "Show URLs without ?'s" in the options -- but don't fool yourself into thinking all is good... session errors occur if this isn't checked.

a few days ago Inktomi Slurp had over 300 bots in my forum spidering the crap out of it... then the other day I saw MSNBot, EchO!,  Walhello appie, WISENutbot, LinkWalker and Netcraft going deeper than I have ever logged.  Googlebot should be in there in a few days.. that one will be interesting.  Google has a cache of about 7200 pages of my site and I have about 21,000 posts total in the forum... I should be able to tell success when Google gets their hands on my site again and I re-query.  I will try and remember to post here with my findings ;-)

Thank you for your help... I guess I was just misunderstanding the use of that option.  I thought it would change all the links...

and [unknown] - from what I have read in this forum... you are a stand up kinda guy, congratulations.. and VERY nice scripting indeed.

[[Unknown]]

Search engines barely look at meta tags these days...

-[Unknown]

[flyingmembers]

But if you run most forums through:
http://www.submitexpress.com/analyzer/

You'll see the actual topic posts aren't spidered at all, even if Search Friendly Mod switched on in Apache.

Hope either yourself or Ca3le could help me with the text above as I entered it into my forum and still couldn't get it working...I think I need an "Idiot's Guide" in order to use the above code...

[Daevien]

Basically, you are opening the Sources/BoardIndex.php, Sources/Display.php & Sources/MessageIndex.php files and changing  the lines that contain stuff like

Code Select Expand


'<a href="' . $scripturl . '?topic=' .


into stuff like

Code Select Expand


'<a href="' . $scripturl2 . '/topic-' .


You can also change the ones that are for the board listings as well, just search for ?board= and change them the same way as you changes the ones for the topics.

Also you need to go look at the lines that declare the global variables such as

Code Select Expand


global $scripturl, $txt, $db_prefix, $modSettings, $context, $settings, $options, $sourcedir;


and add $scripturl2 as in like

Code Select Expand


global $scripturl, $scripturl2, $txt, $db_prefix, $modSettings, $context, $settings, $options, $sourcedir;


$scripturl2 is set somewhere (i think i used the base index.php file when i did it to one of my sites) to be the base url of your site (http://yoursite.com [nofollow] or http://yoursite.com/forums [nofollow] for example, also just / should work, i think i defined my full url in any case) and then it add the /topic- to the url giving you something like http://yoursite.com/topic-1234 [nofollow] for topic 1234

If you used $scripturl which is always defined by smf for it's own purposes you could end with some funky urls, hence the extra variable

The modifications set what SMF returns for linking to urls in the board index, etc instead of the standard way.

But... That's only half of it. You still need to have your web server to redirect the /topic- urls to the proper locations

For apache servers, you can do it with .htaccess files as long as they are enabled (usually are for commercial hosting companies) and do what was suggest above:

Code Select Expand


RewriteEngine On
RewriteRule ^topic-([^/]+)[/]?$ /forum/index.php?topic=$1
RewriteRule ^board-([^/]+)[/]?$ /forum/index.php?board=$1


For other servers, I can't tell you right off what you would have to do for the equiv settings, maybe if you aren't running apache, you can tell me what software you do use and i'll see what I can find.

(Note, I hacked up CA3LE's initial posts into a slightly different layout with some different explanations. Hopefully It made ands ense and was all correct but as I just got home from a long shift and it's late at night, I give you no guarantee )

I may look into doing up a quick mod for this, I'll see what time I have over the next few days if it would be something peopel would use.

Code Select Expand

[[Unknown]]

But if you run most forums through:
http://www.submitexpress.com/analyzer/

You'll see the actual topic posts aren't spidered at all, even if Search Friendly Mod switched on in Apache.

So, obviously, that tool knows absolutely nothing.

-[Unknown]

[flyingmembers]

Thanks Jaeren, but I personally still encountered the same problems.  If you check out my site:
http://www.flyingmembers.com/forums/

All the pages have HTML which is great, however individual posts aren't showing the /forums/ url bit, can I break up your post....

"Basically, you are opening the Sources/BoardIndex.php..."

If you search for '<a href="' . $scripturl . '?topic=' . on Sources/BoardIndex.php you won't find any matches.  I found x2 matches on Sources/Display.php and x4 on Sources/MessageIndex.php.

Am I supposed to replace all instances of .?topic= to /topic- ?

"Also you need to go look at the lines that declare the global variables..."

Could you confirm that "scripturl2" needs to be placed on all .template.php files such as:
global $context, $settings, $options, $scripturl, $scripturl2, $txt, $modSettings;

"$scripturl2 is set somewhere..."

The definition of scripturl2 needs to go in forum/index.php near the top at?:
// Get everything started up...
define('SMF', 1);
@set_magic_quotes_runtime(0);
error_reporting(E_ALL);
$scripturl2 = 'http://www.mywebsite/forum/';

"For apache servers, you can do it with .htaccess files..."

Finally, does the .htaccess file go into the Root directory or the actual forum/ directory?

RewriteEngine On
RewriteRule ^topic-([^/]+)[/]?$ /forum/index.php?topic=$1
RewriteRule ^board-([^/]+)[/]?$ /forum/index.php?board=$1

[flyingmembers]

I don't suppose anyone could help me with this...I don't mind to pay if the above could be clarified.

Thanks

[Daevien]

You need to change any part that looks like (or very close to)

<a href="' . $scripturl . '?topic=

to

<a href="' . $scripturl2 . '/topic=

You are changing the ?topic stuff to /topic- which is correct but you also need to change the base url returned or else you'll get stuff like domain/forums/index.php?/topic-1 which will return errors.

With the scripturl2 in the global vars part, I set it in the three files you editted, anything else shouldn't refer to those locations so wouldn't need to set that.

The .htaccess could be done within either directory I believe, I've only done it with the base site address though which is what the original person did as well. From what you seem to be doing, you want it to include the /forums/ bit when you give a url, ie: domain.com/forums/topic-1 [nofollow]? Or do you want it to be set to domain/topic-1? If it's right off the domain, you would want the .htaccess file in the root directory for your domain. If you move your .htaccess from the current location (which i think is under the forums directory) to the root, you should be in business for the topics being http://www.flyingmembers.com/topic-1.0 [nofollow]

Currently if you go to http://www.flyingmembers.com/forums/topic-1.0 [nofollow] it does in fact work but that's not really much shorter of a url.

I think this post made sense, just got off a crazy work schedule and then slept for 18 hours straight, I may not be making sense yet (if i ever do

Jaeren

[flyingmembers]

Thanks, so what I did was pretty much right (I added the .htaccess file in the root before), however when you mention:

"but you also need to change the base url returned or else you'll get
stuff like domain/forums/index.php?/topic-1 which will return errors."

I think this is the problem...what does this actually mean?

Thanks again...

[Daevien]

Case 1
$scripturl2 should be set to http://www.flyingmembers.com/forums/ [nofollow] if you want it to show as http://www.flyingmembers.com/forums/topic-1 [nofollow]
Your htaccess should be in the forums directory then I believe

Case 2

$scripturil set to http://www.flyingmembers.com/ [nofollow] if you have your htaccess in the root and want it to show as http://www.flyingmembers.com/topic-1 [nofollow]

[Daevien]

I see you got it going, grats

[CA3LE]

But if you run most forums through:
http://www.submitexpress.com/analyzer/

You'll see the actual topic posts aren't spidered at all, even if Search Friendly Mod switched on in Apache.

So, obviously, that tool knows absolutely nothing.

-[Unknown]

hahahaha, yep

[shads]

I got error when I follow this step.. I succesfully converted all the dynamic urls i want to change into static but i have error in my logs.


8: Undefined index: topic_first_message
File: /home/x/public_html/forums/Themes/default/Display.template.php (eval?)
Line: 669


Anyone who can help me fix this? thanks 

[talkaboutpop.com]

I am sorry but when I go to Admin Center/ Edit Features and Options there is NO Search Engine Friendly URL's

Am I in need of an upgrade because I don't think there is much point me trying to do much else when I cannot even get this sorted...........

For the Search Engine Friendly URLs, in you Admin Center > Edit features and options > Basic Forum Features > Search engine friendly URLs, there should be a button.  A popup will open and tells you if this function will work on your forum.

[[Unknown]]

8: Undefined index: topic_first_message
File: /home/x/public_html/forums/Themes/default/Display.template.php (eval?)
Line: 669

Are you using an older template or non standard theme?

I am sorry but when I go to Admin Center/ Edit Features and Options there is NO Search Engine Friendly URL's

Am I in need of an upgrade because I don't think there is much point me trying to do much else when I cannot even get this sorted...........

Have you made sure you expanded all the different sections?

-[Unknown]

[flyingmembers]

Did you make sure the search friendly URL's button is ticked?  Also have you setup the .htaccess file.

I had problems, but read the instructions about x10 times and it does really work.

I got error when I follow this step.. I succesfully converted all the dynamic urls i want to change into static but i have error in my logs.


8: Undefined index: topic_first_message
File: /home/x/public_html/forums/Themes/default/Display.template.php (eval?)
Line: 669


Anyone who can help me fix this? thanks 

[talkaboutpop.com]

As I said before I can find the option "search engine friendly URLS" in this section

Is that what it is actually called? This is very frustrating when I cannot even figure this out.

I am not blind but this option does not appear to be there!

[[Unknown]]

I believe it's called "show URLs without ?s" or similar.

-[Unknown]

[talkaboutpop.com]

Thank you, That was not that obvious you know

I finally got the URLs sorted at my website www.talkaboutpop.com [nofollow]

Right now Google has only 4 pages in its sites for my site so I expect this to improve rapidly soon... or should I?

Is there a code I can add to .httaccess to encourage google to do it's business...

Go wild... wild i tell you WILD!!!!!!!!!

THANKS EVERYONE FOR YOUR POSTS xxx

[shads]

8: Undefined index: topic_first_message
File: /home/x/public_html/forums/Themes/default/Display.template.php (eval?)
Line: 669

Are you using an older template or non standard theme?

I am sorry but when I go to Admin Center/ Edit Features and Options there is NO Search Engine Friendly URL's

Am I in need of an upgrade because I don't think there is much point me trying to do much else when I cannot even get this sorted...........

Have you made sure you expanded all the different sections?

-[Unknown]

I just want to update my post here.. Its fixed now .. thanks 

[talkaboutpop.com]

BAD NEWS....

Google crawled my site the 30th APRIL but only crawled the front page and did not delve any further...

Any wanna check out www.talkaboutpop.com [nofollow] and see if I'm doing anything wrong?

Tommy

[talkaboutpop.com]

Help anyone?

[Ben_S]

Be patent, it takes a while from a googlebot crawl to any updates in googles listing.

[[Unknown]]

http://www.google.com/search?q=site:talkaboutpop.com
http://www.google.com/search?q=link:28o_std9UOoJ:www.talkaboutpop.com/
http://www.google.com/search?q=%22talkaboutpop.com%22

Looks like it's just not getting around enough yet... it's not like you just get indexed completely because you want to.

-[Unknown]

[shads]

$context['page_index'] = constructPageIndex($scripturl . '?topic=' . $topic, $_REQUEST['start'], $topicinfo['numReplies'] + 1, $modSettings['defaultMaxMessages'], true);


in this part of the display.php .. is it possible to change the dot? e.g
topic=128.15  into topic=128/15

I really dont know how to edit that dot thing. anyone help pls 

[[Unknown]]

Yes, but you'd also have to make it parse it correctly.

Why don't you like the dot?  Google doesn't seem to mind the dot.

-[Unknown]

[shads]

How do i do that? 

im having problem  converting them with html extension..

so i rather want  them to appear like directories. 

Can you help me please  how to parse that?

Ive check and tried to edit querystring.php but no success for me. 

[JustJa]

I am not very clever in coding but I try to follow the instructions anyway.
I have done changes in my

Boardindex
display
Messageindex


Now I come to the forum and see the boards but when I click on a topic it comes : http://topic=572.0
and the horrible message that the page cannot be found!
Any Good soul out there to help?
Thanks a lot!

[JustJa]

Hi everybody. If you do not have Apache like me, don´t do this.
It will be just the matter of changing all those codes on the script and than you have to undo everything again!
I wanted so much to have my pages also showing on google 500 times, but no..
At least my forum is working normaly again 

thanks Ben!

[NOLA]

Hey guys,
I wanted to get this topic going again because I feel it one of the most important ones I have read for some time on SMF...... and I have only just found it after repeated searches for "Google spidering etc."

By making SMF get indexed/crawled more often, everybody wins (SMF,website owners). It should be stickyed to the freakin home page!!!

www.testmy.net has more links than COCA FREAKIN COLA. Are you kidding me!!!

I think that says it all...... Not sure about the goose egg w/ msn but at 50k who cares.......

www.Coke.com             35,096    376    376    19,944    14,400
www.testmy.net/forum   50,108   304   304   0   49,500

Results from: http://tools.marketleap.com/


+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Back on topic.....

Hi everybody. If you do not have Apache like me, don´t do this.

Hey Grodden,
What do we poor souls do about this then?
The whole reason for adding a forum to my site was to increase my search engine numbers.

Don't take this the wrong way guys, but I have noticed w/ some of the themes for smf that it seems to be designed around NOT wanting the spiders to crawl them???

ex, No index, no follow meta tags included in the aqua theme....

Why is that? More importantly, what do us non apache users do about it??

My forum is full of keyword rich content that is doing the main site little or no good and it could be........

Please make a module/script/package or anything but please lets not let this topic die......

THX
NOLA