Pretty URLs

[Mick.]

viulian, interesting, is that httpd used widely? Maybe we could make it generate the rules for it automatically.

DarrenDriven and BlueDevil: try the latest snapshot on the downloads page

Hey this update "prettyurls-base_20071116.tar.gz" seems to work just fine.  Thanx.

[viulian]

viulian, interesting, is that httpd used widely? Maybe we could make it generate the rules for it automatically.

Don't know exactly how spread it is.. This is a copy/paste from the author's website:

nginx has been running for more than three years on many heavily loaded Russian sites including Rambler (RamblerMedia.com). In March 2007 about 20% of all Russian virtual hosts were served or proxied by nginx. According to Google Online Security Blog nginx serves or proxies about 4% of all Internet virtual hosts, although Netcraft shows much less percent.

I've mainly chosen it as I had to keep the memory consumption low, and the few people I saw blogging about nginx prised it's stability and no memory leaks. I intend to blog about it myself after a month or so to see how things go.

Here's another quote from this blog:

I have been using Lighttpd for almost a year and Nginx for a month on my servers. I know that they were created to be massively scalable, solving the C10k problem.

I think the technically aware people will start using lighttpd and nginx more and more, it's just that not being like Apache, its hard to convert. For lighttpd, for example, you need to rewrite Apache's mod-rewrite rules using LUA language (which myself I don't know) plus, as far as I read, they are server dependent, not directory dependent (as you would put in a .htaccess file inside in a directory).

But anyway, I am happy to use SMF + PrettyURL on a small OpenVZ VPS, and the results are impressive.
SMF 2.0 (I read a while ago) will also support PostgreSQL, I guess that would mean independence from MySQL too. As your module gives a lot of value to SMF, making it less Apache dependent will give a plus to SMF as a whole and give it a lot of flexibility.

Thank you for the clever way of only using a couple of rules  It only took me 4-5 hours to convert (understanding how things work with nginx included) instead of weeks!

[Dannii]

I know that for Lighttpd you have to put all the rules in a central config file. I was thinking (if people asked) that the mod could generate the rules and just put them in a text file for the user to copy over manually. Perhaps it would be possible to get SMF to modify that file directly. Might require nasty permissions though...

[Riotblade]

I just installed it and enabled filters.  I'm getting a blank index, anyone know why?

Okay I installed the latest snapshot and it works.  However, some topics aren't accessible and leads back to the topic index.  This topic in question contains a single quote(').  Is this the problem?

THanks

[Dannii]

Did you have an older version installed before the snapshot? You have to clear the database entries for it to work.

[Riotblade]

Oh, how would I clear the database entries for it?

[viulian]

ܝܠܕܟ I think a wiki page should be made ( on http://code.google.com/p/prettyurls/w/list ) which describes these informations - too many people are asking how to do X or Y, which has already been answered a lot of times on the [now 49 pages length] topic.

Also, when I try to copy paste your name here - inserting it some text, I get the whole text put in reverse order!

Riotblade You should empty the tables: smf_pretty_topic_urls and smf_pretty_urls_cache [in case smf_ is your database prefix. If not, replace with your prefix.]

Then access the forum again. It will take a while till it answers (as it recomputes the content from those tables) but it should come back properly.

PS: make sure you empty them and not DROP them [in phpmyadmin those buttons are one next to each other and I dropped it one time; had to recreate the table structure from backup]

[viulian]

A separate issue: on my forum:

Database size: 94Mb
smf_pretty_urls_cache: 43Mb
smf_messages table: 35Mb
.....

I noticed that a lot of entries in the table also include the Forum URL..

For example, I have ~ 358,000 rows in that table. My Forum URL has a length of 33 chars. That means that removing the unnecessarily URL (which is the same, minus a couple of outside links) on all rows, I would save 11.81 Mb!

Memory is also an issue, as DumpDatabase.php limits the total amount of memory to be used during backup procedure to 128Mb.
For unaware people (just like me till last hour or so  ) this will make the web backup procedure in SMF fail with an error such as:

Allowed memory size of 134217728 bytes exhausted (tried to allocate 50914081 bytes) in /usr/local/nginx/html/community/Sources/DumpDatabase.php on line 253

Issue is that you don't see this error, because if you let the default 'compress' on, the PHP script will die internally, and all you end up with is an 0 lenght .sql.gz file.

I just increased the memory to 256Mb - seems to be working so far

[Dannii]

Riotblade, use this file: http://prettyurls.googlecode.com/svn/trunk/uninstall.php

Viulian, just empty the table before backing it up That's what I always do.
I am working on making the cache table more efficient though.

[Nao 尚]

For example, I have ~ 358,000 rows in that table. My Forum URL has a length of 33 chars. That means that removing the unnecessarily URL (which is the same, minus a couple of outside links) on all rows, I would save 11.81 Mb!

My custom version of PrettyURLs removes $scripturl from the URL so it's much shorter in effect.
Also, in the future it will be possible (scheduled task) to clear the URL cache when a cache entry is too old. (Say, over one month.) I think Eldacar is planning to take it to the next level.
The ideal would be to delete an entry when it hasn't been used for a couple of days, but I don't know if it's possible to UPDATE a field when accessing a field in read mode... I don't think it is. (Or, it would take too much time to do. Performance is critical here.)

[tkam]

I get a "hacking attempt..." error message when I tried to install this mod.  I'm using the latest development snapshot version.  My forum is running smf 1.1.4 and the mods i have installed are:

1.     Auto Embed Video Clips      1.4   
2.    Gallery 2 Integration    1.0beta3.1    
3.    Fix Message Empty Error For Embed Video Mods    1.0    
4.    TinyPortal    0.983
5.    Remove Post Modified Time    1.1    

[viulian]

My custom version of PrettyURLs removes $scripturl from the URL so it's much shorter in effect.
Also, in the future it will be possible (scheduled task) to clear the URL cache when a cache entry is too old. (Say, over one month.) I think Eldacar is planning to take it to the next level.
The ideal would be to delete an entry when it hasn't been used for a couple of days, but I don't know if it's possible to UPDATE a field when accessing a field in read mode... I don't think it is. (Or, it would take too much time to do. Performance is critical here.)

These are good ideas, however, from my 'bird eye' view over the problem, they appear as 'hacks' which are trying to bring memory consumption down while trying to keep performance reasonable.
But there are still issues: if the scheduled task empties the links which are over 1 month old, all my links will probably get dropped (as I've just set up the table) and recreated after 1 month.
Then for the next 30 days nothing happens, then again, all is dropped and gets recreated.
About the other idea: what is faster, a query from an item is a big table, or computing it every time.. again, it depends on the hardware, available memory, load at that moment, etc.. Again, not a safe solution from my point of view.

I would just accept the cache size and that's it. It's scalable.
Small forum, small cache, no memory requirements, hosting price low.
Huge forum, huge memory requirements, lots of CPU power already there, it doesn't matter if cache was also high - you pay up a lot already - 512Mb more or less doesn't count.
Maybe, for huge forums, an option should be added to use a separate database just for cache, running on a dedicated computer connected with the given forum via a gigabit lan.

But I won't recommend on tweaking a clean code to the max just to squeeze some memory/performance.

When I posted, I was frustrated because I figured out why backups did not work [I wrote that Java SMF Backup using SMF' built in backup system]; having half of database for pretty URL seemed unnecessary. After a while, I realized that those 11Mb saved using $scriptUrl aren't in fact a big deal, database would have been 83Mb instead of 94Mb.

As long as this module is not part of the core for everything to happen instantly, without the need to intercept each URL and check it against a cache [I assume this is the way Pretty URL works, haven't checked], I accept it it's OK the way it is

[Dannii]

tkam, anything in the error log?

viulian, the mod is designed to be optimised for runtime, not for backing it up. I think one huge table is the most efficient way to do this, as the index should (hopefully) mean that the entries can be found in the same speed regardless of the table length.

[tkam]

The error log shows the following:

Hacking attempt...

REPLACE INTO smf_pretty_topic_urls
(ID_TOPIC, pretty_url)
VALUES (8, "welcome-everyone!"), (70, "regarding-discussion-in-the-for-sale-forums")

There's many more items listed after the "VALUES" so it looks like it's having problems at the stage where it's attempting to fill the pretty_url related tables in the DB.  Any ideas?

[Dannii]

Try adding this line before the query in Subs-PrettyUrls.php

Code Select Expand

$modSettings['disableQueryCheck'] = true;

[Greyhound]

Got a quick question....

I installed the pretty urls mod, great idea.  I like that it captures the forum name.  I changed one of my forum names but the url still shows the old forum name in the url link. 

Anyway to change that?  Thanks!

[Dannii]

Not easily, yet. If you uninstall it, delete the 'pretty_board_lookup' and 'pretty_board_urls' settings and then reinstall the mod, then you should get the new names.

[Nao 尚]

There's many more items listed after the "VALUES" so it looks like it's having problems at the stage where it's attempting to fill the pretty_url related tables in the DB.  Any ideas?

A possible reason: one of the topic titles got stripped to 70 characters, and it ended with a "-". And two of the topics had the same name (or at least the same first 70 characters), so Pretty URLs added a "-t" followed by the topic ID to differentiate them. Then we get two "--" in a row and your SQL server thinks it is a comment.

Obviously, PrettyURLs should deal with this and remove extra dashes in the final query...

$pretty_text = preg_replace('/-+/', '-', $pretty_text);

[Dannii]

Ahhh, that explains how those two dashes got there!

[L.G.S]

Kind of a big problem which seems to have happened recently.

I get this on a white error page when trying to save a change to profiles which have content in the signatures:

Service Temporarily Unavailable

I do not get this error with blank signatures though.

My members also get it.