Traffic capacity of a Joomla / SMF bridged site

[Dylan Malone]

Oristo - you know I love your bridge - been using various versions of it on my sites since the early days.  And Kindred, I don't know you, but your posts are fantastic too.

Here's the problem I'm bumping up against.  I run a political website, Joomla 1.0.5 and SMF 1.1 RC2 - bridged.  When our candidate has a busy news day, we'll have 400-500 users at any given time all day long (this is using SMF's default 30 minute activity measurement).

On those days the dedicated server runs at 40-70% CPU load, with a few spikes to 100% here and there that seem to last a minute or two.  Also there are times when the MySQL database seems to be overloaded, as Joomla pages will display a "This site is temporarily unavailable, please contact the administrator" and SMF pages (if accessed directly outside the bridge) will display the equivalent SMF message about the database being unavailable.  The database problem is transient, and seems to occur only during the very highest loads.

Does this sound like a normal amount of server load and performance?  The site is a little sluggish when it gets beyond a few hundred users - and borders on annoying when it reaches the 500 range.

Am I running the busiest known Joomla / SMF bridged site?  If there are others running busier sites, what are they doing to keep things running smoothly?

P.S. Here's a link to the site, sorry for the lackluster visuals... I'm working on a much improved template written from scratch using the advice in the templating thread here.

http://www.algore.org [nofollow]

Thanks!

[Aravot]

May I make a suggestion (not related to your problem) if you are going to keep the forum unwrapped put a link / tab somewhere in the forum to return to Joomla.

Edited:
Newsletter subscription seems to work even without an email address.

[Dylan Malone]

May I make a suggestion (not related to your problem) if you are going to keep the forum unwrapped put a link / tab somewhere in the forum to return to Joomla.

Thanks for the advice.  In fact I don't like the forum unwrapped, but had to make an emergency upgrade from Mambo 4.x to Joomla.  The all-too-well-known security flaws in Mambo, coupled with a high profile politician made us a huge target for hackers.  We must have tempted fate a bit too long, the site was nailed.

It wasn't trivial to move the site to Joomla 1.0.5 and SMF RC2 because of a handful of components and some other problems.  I essentially started over with fresh installs of Joomla and SMF-- the site is running pretty plain vanilla now as we restore all the interesting functionality that made it worthwhile in the first place.  We'll be wrapping the forum back in with the new template next week. 

[Orstio]

As for other large bridged sites:

http://www.40konline.com/mos/

http://www.goosemooose.com

I believe GooseMoose may have some tips on altering some database tables for very large sites to improve database performance.

Thanks for the advice.  In fact I don't like the forum unwrapped, but had to make an emergency upgrade from Mambo 4.x to Joomla.  The all-too-well-known security flaws in Mambo, coupled with a high profile politician made us a huge target for hackers.  We must have tempted fate a bit too long, the site was nailed.

You're aware that Mambo 4.5.3h fixes just as many security issues as Joomla, right?

[Dylan Malone]

I believe GooseMoose may have some tips on altering some database tables for very large sites to improve database performance.

Thanks, I'll contact the admins at both sites to see what their experiences have been.  However, though they have more registered users and a larger post count - that appears to the result of simply having their forums online longer (we reset all our data less than a year ago).  The "most online" count at both sites was lower than ours.

Does it sound like I'm at the top end of what's realistic for a single server Joomla / SMF site?]

Our "Most Online Ever:" counter is telling us 679.  And that was before we rebuilt the server for better performance... I imagine the number would have been higher but the site went offline that time.

Also, I expect the site to double in size within 6-12 months... possibly more (though it's pure conjecture at this point).  Are there any sites running Jommla, SMF, or especially both... that are REALY big?  Or is this just asking the impossible... and I'm going to have to start using load balancing, round robin DNS, etc. (a bunch of high end stuff I know nothing about so far)

You're aware that Mambo 4.5.3h fixes just as many security issues as Joomla, right?

I didn't know that actually... thanks - I'll investigate both forks from now on then.   

[Aravot]

I didn't know that actually... thanks - I'll investigate both forks from now on then.   

Go with Joomla, more active than mambo

[Goosemoose]

Hi Dylan,
There's A LOT you should be doing with a site that big. If you haven't done the tweaks you are going to be suffering greatly. First, please let us know what the specs are with your server. You hopefully have a dedicated server with at least 2 gigs of ram. Please find out the details. You also want to upload this file to your forum directory and give us a link to it. It will tell a lot about the server and how well it's running.

Download status.php here: http://www.simplemachines.org/community/index.php?action=dlattach;topic=49999.0;attach=11768
There are quite a few other important suggestions here: http://www.simplemachines.org/community/index.php?topic=49999.0 the MOST important one being the conversion of tables to InnoDB. This is very important with large sites.

Make sure you have eAccelerator or another php accelerator installed.
Make sure you turn on SMF's caching feature once you've done this.

Make sure you have Joomla caching on.

Let us know how it goes once you do some of these things

[chadness]

That's a huge site!  I can see why it would be busier than most.  I wouldn't be surprised if that's the record for a bridged site.

By the way, the right column doesn't resize to fit the calendar in Firefox 1.5.

[Dylan Malone]

You also want to upload this file to your forum directory and give us a link to it. It will tell a lot about the server and how well it's running.

Thank you, thank you.    I'll be implementing your suggestions one at a time over the coming week.  For what it's worth, I've uploaded status.php now for a baseline:

http://www.algore.org/forum/status.php [nofollow]

Weird how it doesn't seem able to get memory and mhz ratings on my server.

The machine is an Xserve G5, single processor, with 1.5 Gigabytes of RAM.  2 Gigahertz, but G5s are pretty good CPUs- that's more horsepower than it sounds like by mhz alone.  This was a $3000+ machine about 18 months ago.

I have no idea if the output of the status.php is okay, the site is pretty quiet this week, and the server's not working very hard.  It does say:

Opened vs. Open tables:
(table_cache)    1101.4688 (should be <= 80)

That seems bad, but I haven't done any searching to try to figure out what that means yet.  Thanks again.

[Dylan Malone]

edit: decided to reply to myself with an update on my progress...

I've created a my.cnf file to improve MySQL performance and the status.php report.

I hope I haven't gone too far... I essentially took the my-large.cnf file I found (one of the default configs that comes with the distribution) made a couple minor changes and saved it to etc/my.cnf

With 1.5 Gigs of RAM for the web / MySQL / mail server I'm not sure if this config is swinging the pendulum too far the other way though...  I also set max_connections = 300

Does anyone know a downside to tripling the default 100 connections?  I notice that this variable wasn't included in the default my-large.cnf config file.

I'm going to enable SMF and Joomla caching next and see if anything breaks.  Thanks for the pointers!

[bluevoodu]

You also want to upload this file to your forum directory and give us a link to it. It will tell a lot about the server and how well it's running.

I have no idea if the output of the status.php is okay, the site is pretty quiet this week, and the server's not working very hard.  It does say:

Opened vs. Open tables:
(table_cache)    1101.4688 (should be <= 80)

That seems bad, but I haven't done any searching to try to figure out what that means yet.  Thanks again.

did you find out what that is?  Mine is around 91... not much more than 80... but I am curious about that as well.

BV

[Dylan Malone]

did you find out what that is?  Mine is around 91... not much more than 80... but I am curious about that as well.

I don't understand it well enough to paraphrase, but I got the general idea after reading this (Google is your friend):

http://www.databasejournal.com/features/mysql/article.php/3367871 [nofollow]

I guess that MySQL doesn't have some magical way of self-configuring to match the RAM of the server it finds itself on... apparently it's our job as server admins to tell these applications how we want them to share the resources.  Considering my terrifying lack of insight into the inner workings of UNIX this is scary, but I think the changes I made today will help.

[Goosemoose]

Your stats look pretty good from what I can see, but I can't see your forum page load time. If you enable that, it will help. Record a few load times for your board index before enabling SMF caching, you'll be AMAZED at the difference. Joomla caching isn't the greatest, but it will still help Make sure you install a php accelerator.