Simple Machines Community Forum

SMF Support => Server Performance and Configuration => Topic started by: Goosemoose on September 12, 2004, 05:13:37 PM

Title: Forum using up all system resources?
Post by: Goosemoose on September 12, 2004, 05:13:37 PM
hi guys,
 my host just shut down my site because the forum started to hog all their system resources "This was shut down due to the fact that your site has a heavy enough mysql load to shut down the physical server."
  They made a backup of the database but won't turn it back on until I fix the problem. But since its not running I don't know how to go about this. If you have any ideas let me know.
Title: Re: Forum using up all system resources?
Post by: [Unknown] on September 12, 2004, 05:37:21 PM
Do you have the facility to check the database?

If you do, can you see if any of the topics have a "numViews" higher than logically possible?  Meaning, more than say 10000 but with few replies?

Was your host using MySQL 3 or MySQL 4?

-[Unknown]
Title: Re: Forum using up all system resources?
Post by: Goosemoose on September 12, 2004, 05:40:43 PM
I'm downloading the database right now. I'll look at it and let you know what I find.
Title: Re: Forum using up all system resources?
Post by: Goosemoose on September 12, 2004, 05:47:02 PM
The backup I downloaded that they created for me is full of MYD and MYI files of each table. When I try to view them in notepad I see only junk. What should I use to view this and what table should I be looking for. Usually I would just look around but since there are hundred of files now it's harder to find.
Title: Re: Forum using up all system resources?
Post by: [Unknown] on September 12, 2004, 05:50:52 PM
You might be able to view those if you install MySQL on your own server, create a database, and copy these files into mysql\data\databasename.  Those are the raw MySQL database files....

-[Unknown]
Title: Re: Forum using up all system resources?
Post by: Goosemoose on September 12, 2004, 06:10:16 PM
Okay, I am installing in on my machine right now. They were running mysql 3.0 as it is illegal for them to install 4.0 due to legal reasons.
Title: Re: Forum using up all system resources?
Post by: Ben_S on September 12, 2004, 06:39:15 PM
If they think it's illegal to install MySQL 4, then they are talking rubbish and I highly doubt that they have a clue what they are talking about when saying your forum was a resource hog.
Title: Re: Forum using up all system resources?
Post by: Goosemoose on September 12, 2004, 06:47:46 PM
Here is what they told me (webintellects.net):
"We are running MySQl 3. Version 4 is restricted by its license not to be commercially distributed so our software provider could not upgrade if they wanted to. "
Title: Re: Forum using up all system resources?
Post by: kegobeer on September 12, 2004, 06:52:52 PM
[offtopic]
Time for you to switch hosts, I think.  ;D

If they are using MySQL Pro then they probably can't just upgrade - but as a host I know I would stay on the edge to keep my customers happy.

There are plenty of hosts that stay at the forefront!
[/offtopic]
Title: Re: Forum using up all system resources?
Post by: [Unknown] on September 12, 2004, 07:00:52 PM
Sigh.  Stupid hosts.

MySQL 3 was licensed under the lesser GPL.  MySQL 4 is licensed under the GPL.  Yes, there's a HUGE difference there, but they've completely and utterly missed it.

You see, MySQL 4 just cannot be sold.  Period.  End of story.  However, open source would not exist if people could not profit or at least break even off of it.  Don't think we're communists - only people who don't understand do.

You can sell SERVICES for MySQL, such as hosting, without problem.  This is the INTENT.  You're SUPPOSED to sell services.  That's how it WORKS.

http://www.mysql.com/company/legal/licensing/opensource-license.html

Even if it weren't, it's $595.  Compare that with Microsoft SQL (http://www.microsoft.com/sql/howtobuy/default.asp).  So even then, how cheap is your host?

GPL here:
http://www.gnu.org/copyleft/gpl.html

LGPL here:
http://www.gnu.org/copyleft/lesser.html

-[Unknown]
Title: Re: Forum using up all system resources?
Post by: Ben_S on September 12, 2004, 07:03:35 PM
I imagine the problem is they use something crappy like Ensim which comes with MySQL 3x and they for whatever reason cant/wont upgrade to MySQL 4x themselves.
Title: Re: Forum using up all system resources?
Post by: Goosemoose on September 12, 2004, 07:32:24 PM
But everything has been running fine for months. I haven't changed a thing since I installed smf months ago. They are using control maestro,  WEBppliance Pro 3.5.21-10
 
 
Title: Re: Forum using up all system resources?
Post by: [Unknown] on September 12, 2004, 07:48:39 PM
SMF was designed with MySQL 4 in mind.  The problem is, MySQL 3 and 4 act very differently in some situations, and what is slow on one may be fast on the other.  In fact, there have been situations where we have had to pick between two methods - one fast on 4, another fast on 3.  Obviously, the descision could only logically be made to go with the current version (4) not the old and no longer available 3.

It still should not be slow, persay, but I can understand it performing worse.  Additionally, MySQL 4 is simply faster overall, and when you account for query caching it is faster indeed.  If your host is really concerned about their server's performance, they should be willing to upgrade MySQL - otherwise, they really are not worth your money.

MySQL 4 has been available for quite a while now.  Your host probably just over-reacted to a peak period.

-[Unknown]
Title: Re: Forum using up all system resources?
Post by: Goosemoose on September 12, 2004, 07:51:07 PM
They took the entire site down. Then after I got a hold of them they put it back up and supposedly the resources shot up immediately so they took it down again. Right now they are waiting on me to fix the problem which I can't really do.
Title: Re: Forum using up all system resources?
Post by: [Unknown] on September 12, 2004, 07:52:23 PM
Edit Settings.php and put it in maintenance mode.  See if it "shoots up again" then too.

Because that would obviously be a crock :P.

-[Unknown]
Title: Re: Forum using up all system resources?
Post by: Goosemoose on September 12, 2004, 08:04:44 PM
I have put it in maitenance mode and asked them to turn it back on. Hopefully they will do it soon.
Title: Re: Forum using up all system resources?
Post by: Amacythe on September 12, 2004, 08:38:20 PM
Why go through all this trouble... the easiest solution would be to find a host that doesn't have their collective heads up their... ahem... find a host that knows what they are doing.
Title: Re: Forum using up all system resources?
Post by: Goosemoose on September 12, 2004, 10:52:04 PM
Why go through all this trouble... the easiest solution would be to find a host that doesn't have their collective heads up their... ahem... find a host that knows what they are doing.

I've been using this host for years and haven't had a problem. I think the person who got stuck as tech support on a sunday just isn't very knowledgeable. I usually get much better answers and quick response. I'll find out when I call my usual contact tommorrow. I have hundreds of thousands of messages and over 4,000 users so it's not a small board. Hopefully I can get it resolved soon, otherwise I will definately think of changing.

Title: Re: Forum using up all system resources?
Post by: Goosemoose on September 13, 2004, 01:09:14 AM
Do you have the facility to check the database?

If you do, can you see if any of the topics have a "numViews" higher than logically possible?  Meaning, more than say 10000 but with few replies?

Was your host using MySQL 3 or MySQL 4?

-[Unknown]

Okay, I finally got a server running on my local machine and checked out the database. The highest sumViews was 5600 and there were 400 replies to it so I don't think there were any rogue threads. Any other ideas?
Title: Re: Forum using up all system resources?
Post by: [Unknown] on September 13, 2004, 01:45:25 AM
Hmm.. at the time it was dropped, how many people were listed in the smf_log_online table?

-[Unknown]
Title: Re: Forum using up all system resources?
Post by: Goosemoose on September 13, 2004, 01:56:09 AM
It looks like there were only 22 on at the time they took it down. The top for the day was 48. Nothing abnormal.
Title: Re: Forum using up all system resources?
Post by: [Unknown] on September 13, 2004, 02:00:40 AM
I don't suppose you could get them to look at the access log, load averages, or anything else from that time?  I can't imagine how else it could be suddenly using up enough resources to physically damage the server.  Unless the tech on duty was wasted or something.

-[Unknown]
Title: Re: Forum using up all system resources?
Post by: Goosemoose on September 13, 2004, 02:03:06 AM
I could sure ask, what exactly do you want me to ask them to look for?
Title: Re: Forum using up all system resources?
Post by: [Unknown] on September 13, 2004, 02:11:18 AM
I could sure ask, what exactly do you want me to ask them to look for?

Well, one possibility might be someone contacting the forum many times per second over a period of more than a few minutes.  I was consider that almost a DoS attack, and that's why I asked you to check numViews - this has been done before by people with malicious intents to other forums, and can cause high load.

I'm also wondering how bad the load averages really were.  On this server, they're currently.. "load average: 0.34, 0.71, 0.66" which, while not totally ideal of course (they used to be lower, strangely...) are perfectly fine.  Your first post, at least, made it sound like your account was disabled for fear of physical damage to the server.  For this, I might expect the load averages to be higher than 20 or so - although, I am not by any means a linux expert.  Being able to see the load average might give me an idea of how bad the issue is.

You might also want to ask if there is ANYTHING your host change on your server in the last week.  It could be that they even changed some minor setting in php.ini - obviously, I don't think such could have caused high-load problems, but it's possible and moreover, it's a start.

Right now, because you say your forum was performing okay in the past and you were not shut down by your host before now, I have nothing else to really go on.  It would be considerably different if you knew your host had changed something, or if you had had some problems before... but sudden problems are a lot harder to track down.

Of course, there is another question too (but I assume you've already asked yourself this..) have *you* changed anything in SMF's configuration or an .htaccess file in the past while yourself?  I doubt that's a problem either... but again, it may at least be something to go off of.

-[Unknown]
Title: Re: Forum using up all system resources?
Post by: A.M.A on September 13, 2004, 04:56:14 AM
would not 'persistent connection' when activated on a non DS causes a high load!
Title: Re: Forum using up all system resources?
Post by: Goosemoose on September 13, 2004, 01:46:40 PM
Okay the forum is in maintenance mode and they have turned mysql back on. According to the tech there have been no changes over the past weel that he is "aware of". It's at http://www.goosemoose.com/rfc if you want to take a look at it. I am still not taking it out of maintenance mode yet though.
Title: Re: Forum using up all system resources?
Post by: Goosemoose on September 13, 2004, 02:21:29 PM
Okay, I took it out of maintenance mode and all looks ok. Note that I still made no changes to the site or db.
Title: Re: Forum using up all system resources?
Post by: Goosemoose on September 13, 2004, 02:24:14 PM
Strike that, they took it down within 5 minutes of it being out of mainentance mode, it started doing the same thing.
Title: Re: Forum using up all system resources?
Post by: [Unknown] on September 13, 2004, 09:13:28 PM
Have you considered upgrading to RC1?

-[Unknown]
Title: Re: Forum using up all system resources?
Post by: Goosemoose on September 13, 2004, 09:44:55 PM
I'm in the process of doing it right now. I had uploaded my beta 4 files on accident so now I'm going back up then to RC1. Do you think that will fix the problem? Has anyone else had anything similar?
Title: Re: Forum using up all system resources?
Post by: [Unknown] on September 13, 2004, 11:26:07 PM
Well, the search functionality was a bit slower than desirable in Beta 5.  Still, if you had no problems until now it doesn't make any sense.

-[Unknown]
Title: Re: Forum using up all system resources?
Post by: Goosemoose on September 14, 2004, 04:10:15 PM
Well I upgraded to RC1 and turned off maintenance mode and the forum crashed the server as soon as it was taken out of maintenance mode. I saw it myself as I could load the first few pages then it started going slower and slower until smf told me the db connection was lost. The host claims nothing has changed recently and I know I haven't changed any files for months. Does anybody have any other ideas? They are telling me now that they probably won't turn it back on.
Title: Re: Forum using up all system resources?
Post by: Grudge on September 14, 2004, 04:16:01 PM
You 100% sure that persistant connections isn't somehow on?
Title: Re: Forum using up all system resources?
Post by: Goosemoose on September 14, 2004, 04:45:08 PM
I just checked and smf has persistent connection unchecked
Title: Re: Forum using up all system resources?
Post by: Tony Reid on September 14, 2004, 05:17:17 PM
It sounds like an issue with the server to me.

I'd move hosts.
Title: Re: Forum using up all system resources?
Post by: [Unknown] on September 14, 2004, 07:02:30 PM
If you want to give me FTP access I'll try to figure out what's eating load...?

-[Unknown]
Title: Re: Forum using up all system resources?
Post by: Goosemoose on September 15, 2004, 12:02:25 AM
I imagine the problem is they use something crappy like Ensim which comes with MySQL 3x and they for whatever reason cant/wont upgrade to MySQL 4x themselves.
Yup , they were using Ensim!
Title: Re: Forum using up all system resources?
Post by: Amacythe on September 15, 2004, 01:07:04 AM
You say they were using Ensim, does this mean they have upgraded or are going to?
Title: Re: Forum using up all system resources?
Post by: Goosemoose on September 16, 2004, 01:33:54 AM
well they still are using it but I moved mine to another server running plesk where i have my own virutual server, where everything seems to be running okay, though a bit slow.
Title: Re: Forum using up all system resources?
Post by: [Unknown] on September 16, 2004, 01:42:15 AM
well they still are using it but I moved mine to another server running plesk where i have my own virutual server, where everything seems to be running okay, though a bit slow.

Can I see a phpinfo?  MySQL version information (not the stuff on the phpinfo!)?

-[Unknown]
Title: Re: Forum using up all system resources?
Post by: Goosemoose on September 16, 2004, 02:03:44 AM
http://209.216.209.81/phpinfo.php
mysql is still version 3.
Title: Re: Forum using up all system resources?
Post by: [Unknown] on September 16, 2004, 03:13:10 AM
http://209.216.209.81/phpinfo.php
mysql is still version 3.

Egad!  They have so many things compiled in.. ugh.

What do you/are you going to use the server for?  Mainly SMF and static content?  If so, you might consider upgrading to PHP 5.  Even if not, you should upgrade to 4.3.8 (and soon a 4.3.9) which includes some security fixes ;).

The configure line also includes a lot of things it might not need.  I'd take out --enable-mbrege, --enable-dio (never heard of that?), --enable-trans-sid, and --with-ncurses at least.

It's also nice to note that MySQL 4 is significantly faster, and is the current stable release.  MySQL 3 is now an "archived" download.

http://downloads.mysql.com/archives.php?p=mysql-3.23

MySQL 4 supports the all-powerful-and-beloved feature we call "query caching", which is just generally awesome.

You seem to be on Apache 2, which is good.

Also, since you're on a dedicated server, you might try using a persistant connection.  This should be faster too ;).

-[Unknown]
Title: Re: Forum using up all system resources?
Post by: Goosemoose on September 17, 2004, 03:40:55 AM
Yeah i'm just using smf and static content. I'll look into the upgrades. Check out the forum and tell me what you think about the speed.