Simple Machines Community Forum

SMF Support => Server Performance and Configuration => Topic started by: gasman on November 25, 2005, 03:31:08 PM

Title: SMF overloading my server's CPU
Post by: gasman on November 25, 2005, 03:31:08 PM
I have received a warning from my hosting company (lunarpages) that a php script is overloading the server's CPU. I am on a shared hosting plan, and the php script is using huge amounts of the server's CPU at its peak.

QuoteYour account is utilizing excessive resources, causing a significant
degradation of services on the server. This is a shared environment and we
can not allow one user to utilize the majority of the resources on a server
as it affects all users adversely.

Load Averages: 6.71 1.25 0.8

Top Process %CPU 79.0 /usr/bin/php
Top Process %CPU 64.0 /usr/bin/php
Top Process %CPU 47.0 /usr/bin/php

This has been going on for well over a week. I have been warned that unless I modify or disable the script, my account will be suspended. They can't tell me exactly which script is causing the problem, and have suggested I disable scripts one by one and see if it makes a difference to CPU usage.

This seems to have happened after I upgraded to SMS1.1 RC1.

I have no other php scripts on my site, apart from the forum. I have a cgi script that I disabled but it made no differrence to my server's CPU usage.

I have a small-medium forum, 1000+ members, up to 60 or 70 online at peak times.

I have been moved to a lower-spec server until the situation is resolved. My server's specs: PHP version 4.4.1, Apache version 1.3.34 (Unix), MySQL version 4.0.25-standard

Status.php is here: http://www.good-music-guide.com/status.php (it changes rapidly, so try refreshing)


Googlebot has been active recently, indexing pages from the forum.

My questions are:
   

cheers
Rob Lang
www.good-music-guide.com/forum
Title: Re: SMF overloading my server's CPU
Post by: Compuart on November 28, 2005, 08:28:39 AM
Quote from: gasman on November 25, 2005, 03:31:08 PMdoes SMF use this much CPU power? (Surely not!)
No, it shouldn't and in general doesn't.
QuoteWhy only since the upgrade to 1.1
Not sure, but things should have to get faster, especially if you've enabled caching.
Quoteand what can I do to lower it?
Check README: Checklist for performance problems (http://www.simplemachines.org/community/index.php?topic=49999.0)

QuoteWould disabling the search function help? Or creating a fulltext index?
Most definitely! Searching without a fulltext index can be hurting your server.
Title: Re: SMF overloading my server's CPU
Post by: gasman on November 28, 2005, 06:44:15 PM
Thanks for your reply.

I am having continuing server CPU load problems.

My host asked me to disable the forum for a 24 hour period to see if it made a difference to CPU loads, and it seems to have made a big difference:

QuoteHello,
Here are a set of updated statistics for your site. It looks like CPU usage dropped dramatically during this period. Please let us know if you have any questions.

Nov 28th 2005
good-music-guide.com 8.40 1.69 0.8

Nov 27th 2005
good-music-guide.com 0.53 0.32 0.1

Nov 26th 2005
good-music-guide.com 3.05 0.84 0.2


What I have done so far:
1. disabled guest access
2. Robots.txt to disallow googlebot
3. Disabled search (and created fulltext file for when I re-enable)

My status.php (http://www.good-music-guide.com/status.php) still shows numbers that are way out of range. Unfortunately, I have no idea what they mean or how to optimise them.

How do I enable caching?
What (and how) can I do to optimise my database? (I have very limited MySQL abilities).
Why on earth is the forum using >8% average CPU load????

I will be suspended from my account if I can't get this resolved.

Any advise MUCH appreciated.

cheers
Rob

Title: Re: SMF overloading my server's CPU
Post by: Ben_S on November 28, 2005, 07:04:13 PM
Move host, your cu rrent host is rubbish would be my advice.

Nothing you have posted shows that it is your account that caused the high load.

Your forum is not causing any harm to their server, ask the to show you proper figures that show it is, I'll bet they can't, what you have posted thus far does not show that your forum is causing any harm.

Looks like an oversold host to me.... Which is a shame since they have a good reputation. Their MySQL is also very poorly configured.

Why not invite them here to respond themselves?
Title: Re: SMF overloading my server's CPU
Post by: gasman on November 29, 2005, 05:11:45 AM
Thanks for the advice, Ben.

I agree, that Lunarpages are not providing enough specific data to enable me to identify which particular php script within SMF is causing the problem. However, it does seem that the forum is the problem.  Our test of 24 hours downtime reduced CPU load to acceptable amounts, and now that the forum is up again, CPU load is sky high again.

My concern is this: either it is a weird setting in Lunarpages version of php or MySQL, in which case I can't do anything from my side, or it is a funny setting on the forum. I am hoping somebody can identify a simple setting which eases the load on the CPU.

If all else fails, I may have to change hosts. Big hassle.

Is anyone else having this problem? In particular, is anyone else hosted with Lunarpages on a shared plan, with a SMF forum on their site?

cheers
Rob Lang
Title: Re: SMF overloading my server's CPU
Post by: gasman on November 29, 2005, 05:19:18 AM
As an additional note, I received a email from Lunarpages saying that we should be careful using any formmail scripts, as they can be hijacked and used to send SPAM. I have been getting a huge amount of Non-Delivery Report emails lately from somebody promoting their Ultimate Online Pharmacy. I didn't think much of it - just thought it was email spoofing.

Is it possible someone could hack into the mail system of the forum and use it to send spam?
Title: Re: SMF overloading my server's CPU
Post by: Ben_S on November 29, 2005, 05:29:51 AM
Unlikly, there are no known security issues.

Do you have any mod installed?
Title: Re: SMF overloading my server's CPU
Post by: Angie on November 29, 2005, 05:43:46 AM
There is no known issue to use the smf´s mailfunction for DoS or DDoS attacks.
Btw: most DDos or DoS attacks make use of the insecure formular (GET instead POST and inadequate filtering classes) not of the mailfunction itself.

the problem "high load": ask your hosting company for detailed informations.

We have on our smallest server 3 big smf, 1 smf is highly modded (using ajax features).
And we use 2 big java chat servers (both with own httpd) on this server.
Additionaly we produce on this server "tickers".
Tickers are autogenerated pictures (approx. 70.000 a day) ...

Here our Server load:

Processor: AMD Sempron(tm) 2600+ (1833.218MHz)
Load averages: 0.08, 0.09, 0.08
Current processes: 124 (120 sleeping, 3 running, 1 zombie)
Processes by CPU: httpd2- (41) 2.4%, (other) (27) 0.1% 
Memory usage: 87.394% (436412k / 499360k)

as you can see, smf is not producing high server load.

lg angie
Title: Re: SMF overloading my server's CPU
Post by: gasman on November 29, 2005, 06:09:01 AM
Thanks Angie and Ben

The only package I have installed is the MySQL 5.0.12 Compatibility Update, ironic as I have now been shifted to a lower spec server with MySQL v.4.

Good to see what CPU loads other SMF packages produce.

I'm SURE that my little forum couldn't be causing the high CPU load alone!

I have examined one of the email headers from the Non-Delivery Report emails. I'm getting hundreds of these each day in my catch-all (default) account. I'm no expert in email headers, but the first "Received:" header (with the earliest timestamp) seems to originate from good-music-guide.com - i.e. my website. I believe this part of the header is the most difficult to forge. Does this mean the SPAM is originating from my mailserver? I.e. someone has hacked into my account to send SPAM? Or am I being naiive?

Here is one of the hundreds of emails I'm getting each day, with headers. Naturally, the Return-Path ([email protected]) is not a valid email address of mine (though good-music-guide.com is my domain), so I simply assumed I was a victim of email spoofing, and these undeliverable emails bounce back to my catch-all.

However, good-music-guide.com appears as the very first Received From header (highlighted):

QuoteReturn-Path: <[email protected]>
Received: (qmail 3350 invoked from network); 26 Nov 2005 05:27:15 -0000
Received: from sgrelayg1.core.theplanet.net (195.92.195.145)
  by thallium.smartgroups.com with SMTP; 26 Nov 2005 05:27:15 -0000
Received: from [211.162.131.165] (helo=143184544)
   by sgrelayg1.core.theplanet.net with smtp (Exim 3.22 #1)
   id 1Efrn4-0005cR-00
   for [email protected]; Sat, 26 Nov 2005 04:36:00 +0000
Received: from good-music-guide.com  (144286640 [143387216])
   by glomedical.com (Qmailv1) with ESMTP id 8A93C69BCB
   for <[email protected]>; Fri, 25 Nov 2005 23:35:17 -0500
Date: Fri, 25 Nov 2005 23:35:17 -0500
From: Doctor <[email protected]>
X-Mailer: The Bat! (v2.00.2) Personal
X-Priority: 3
Message-ID: <[email protected]>
To: Mc <[email protected]>
Subject: The Ultimate Online Pharmaceutical
MIME-Version: 1.0
Content-Type: multipart/alternative;
   boundary="----------EFA2D01C28DB155"
X-Virus-Scanned: by Ameriserv.net Anti-Virus E-Gateway

This is a multi-part message in MIME format.

------------EFA2D01C28DB155
Content-Type: text/plain
Content-Transfer-Encoding: 7bit

Vlabgra $3.3
Leviltra $3.3
Ciaalis $3.7
Imityrex $16.4
Flomyax $2.2
Ultrlam $0.78
Vionxx $4.75
Ambfien $2.2
Valtium $0.97
Xanahx $1.09
Sogma $3
Merimdia $2.2 


visit our website
http://nergron.com/?YLRYEXXFEdR1RfVkd0RlxTQUBSQ11GREYfUVxZ

___
Best regards,
Online Pharmaceuticals

qwefgfgjs XFEdR1RfVkd0RlxTQUBSQ11GREYfUVxZ

------------EFA2D01C28DB155
Content-Type: text/html
Content-Transfer-Encoding: 7bit

<html>
<body>
<b>Vlagrra - $3.3 <br>
Lebvitra - $3.3<br>
Cialiqs - $3.7<br>
Imlitrex - $16.4<br>
Flmomax - $2.2<br>
Ulxtram - $0.78<br>
Viwoxx - $4.75 <br>
Amcbien - $2.2<br>
Vaglium - $0.97 <br>
Xaenax - $1.09<br>
Suoma - $3 <br>
Mseridia - $2.2</b><br>
<br>
<br>
<a href="http://nergron.com/?YLRYEXXFEdR1RfVkd0RlxTQUBSQ11GREYfUVxZ"><strong>visit our website</strong></a><br>
Title: Re: SMF overloading my server's CPU
Post by: Angie on November 29, 2005, 06:43:55 AM
when you have doubts if you are a victim of DoS or not, ask your host to show you the /var/log/messages

lg angie

Title: Re: SMF overloading my server's CPU
Post by: MAZZA on November 29, 2005, 11:58:13 AM
I think you have a received from header there because it's a bouncemail from your own mailserver. Not the original mail.
Title: Re: SMF overloading my server's CPU
Post by: gasman on November 29, 2005, 03:06:11 PM
Thanks Angie and MAZZA

I have been assured by Lunarpages that the emails are not originating from my site, so its not a DoS attack.

They also agree that SMF usually does not use such high server resources. They have suggested that there may be some broken tables in the database.

Well, this is where I reach my limits of my expertise. I have used the forum's own "optimise tables" and the "find and repair errors" functions in the "forum maintenance" section of admin, but I suspect I need to investigate using phpMyAdmin. Could anyone lead me by the hand and show me how to check for broken tables using phpMyAdmin?

My status.php is still showing way high readings espescially in the  Load averages, and the opened vs open tables:
QuoteBasic Information
November 29, 2005, 12:08:45 PM
Operating System: CentOS release 3.6 (Final) 
Processor: Intel® Xeon(TM) CPU 2.80GHz (2791.087MHz)
Load averages: 5.20, 5.51, 5.66
Current processes: 204 (196 sleeping, 7 running, 1 zombie)
Processes by CPU: php (8) 26.0%, x (2) 22.8%, spamd (3) 2.3%, httpd (85) 1.2%, (other) (53) 0.4% 
Memory usage: 97.843% (3774072k / 3857288k)
Swap: 0.698% (58544k / 8385920k) 

MySQL Statistics
MySQL 4.0.x
Connections per second:  5.6368 
Kilobytes received per second:  4.8524 
Kilobytes sent per second:  7.7874 
Queries per second:  85.7308 
Percentage of slow queries:  0 
Opened vs. Open tables:
(table_cache)  1527.6719 (should be <= 80) 
Table cache usage:
(table_cache)  1 (should be >= 0.5 and <= 0.9) 

Any ideas?

cheers
Rob
Title: Re: SMF overloading my server's CPU
Post by: Trekkie101 on November 29, 2005, 03:13:39 PM
Is this shared account?


Just, "Total Posts: 176590" would explain the trouble if this is a shared account.
Title: Re: SMF overloading my server's CPU
Post by: Angie on November 29, 2005, 04:11:32 PM
Quote from: gasman on November 29, 2005, 03:06:11 PM
Thanks Angie and MAZZA

I have been assured by Lunarpages that the emails are not originating from my site, so its not a DoS attack.

They also agree that SMF usually does not use such high server resources. They have suggested that there may be some broken tables in the database.

Well, this is where I reach my limits of my expertise. I have used the forum's own "optimise tables" and the "find and repair errors" functions in the "forum maintenance" section of admin, but I suspect I need to investigate using phpMyAdmin. Could anyone lead me by the hand and show me how to check for broken tables using phpMyAdmin?

My status.php is still showing way high readings espescially in the  Load averages, and the opened vs open tables:
QuoteBasic Information
November 29, 2005, 12:08:45 PM
Operating System: CentOS release 3.6 (Final) 
Processor: Intel® Xeon(TM) CPU 2.80GHz (2791.087MHz)
Load averages: 5.20, 5.51, 5.66
Current processes: 204 (196 sleeping, 7 running, 1 zombie)
Processes by CPU: php (8) 26.0%, x (2) 22.8%, spamd (3) 2.3%, httpd (85) 1.2%, (other) (53) 0.4% 
Memory usage: 97.843% (3774072k / 3857288k)
Swap: 0.698% (58544k / 8385920k) 

MySQL Statistics
MySQL 4.0.x
Connections per second:  5.6368 
Kilobytes received per second:  4.8524 
Kilobytes sent per second:  7.7874 
Queries per second:  85.7308 
Percentage of slow queries:  0 
Opened vs. Open tables:
(table_cache)  1527.6719 (should be <= 80) 
Table cache usage:
(table_cache)  1 (should be >= 0.5 and <= 0.9) 

Any ideas?

cheers
Rob

even such many posts cant produce so high server loads!
check with phpMyAdmin or similiar tool your tables

Lg Angie
Title: Re: SMF overloading my server's CPU
Post by: Trekkie101 on November 29, 2005, 04:59:46 PM
Quoteeven such many posts cant produce so high server loads!
check with phpMyAdmin or similiar tool your tables

Lg Angie

Your missing the point, shared server, lots of people, crammed on one server, each making loads of resource drains and this BIG site suddenly drives it up the wall.

This very community is not even twice as big and already was too big for a lesser powered dedicated server. Its on its own server just now.
Title: Re: SMF overloading my server's CPU
Post by: Ben_S on November 29, 2005, 05:26:55 PM
How active is his forum though, number of posts aren't really that critical, it's how many people are active.

Also the celeron this site was on was certaily not a server ;D
Title: Re: SMF overloading my server's CPU
Post by: gasman on November 29, 2005, 06:09:42 PM
Yes, it is on a shared hosting plan - always has been, and never had this problem before.

It has a total of about 1300 members, but only about 50-60 online at once at peak times. Most of the members have not visited in weeks, or months. It has a total of 176000 posts. It gets about 500-700 new posts per day. There are no mods installed, ecept the MySQL v.5 compatibility mod. No funny stuff allowed - only txt and jpg attachments, no flash, simple gif/jpg avatars. No file sharing.

The vast bulk of the forum is simple php which outputs mainly text pages. Currently searching is disabled. It is on a Celeron sever now, only because they took it off the main servers pending review of CPU load problems.

The status.php (http://www.good-music-guide.com/status.php) output continues to show high CPU loads, even when there are only 5 or 6 members online. It also shows what I think are problems with the tables and table cache usage in the MySQL database, but I am not expert enough to interpret these.

cheers
Rob Lang
Title: Re: SMF overloading my server's CPU
Post by: gasman on November 29, 2005, 06:22:05 PM
Out of interest, compare my forum (www.good-music-guide.com/forum) with this forum (http://www.simplemachines.org/community/index.php)

This forum has 3 times as many active users at any one time, about the same number of posts, and similar numbers of new posts per day.

yet, when we compare the corresponding status.php outputs, there is a huge difference between both CPU loads, as well as the database parameters.

www.good-music-guide.com/status.php  vs  www.simplemachines.org/community/status.php

What I am really after is for somebody to help me interpret the status.php output, and let me know what settings or changes I can make to optimise it, especially with regards to the tables and table cache numbers, which seem to be way off.

Please don't give up on me!

cheers
Rob Lang
Title: Re: SMF overloading my server's CPU
Post by: Joshua Dickerson on November 29, 2005, 06:24:17 PM
Generally, MySQL doesn't make your CPU load that high. It usually eats memory faster than CPU. Some things that might eat your CPU are avatar resizing (quite possibly the culprit) and extreme output compression level.

What happens if you downgrade to 1.0? If you think it is MySQL, try creating a backup of the tables, dropping them, then recreating them. Maybe dropping the entire database and recreating it. I doubt it is MySQL because you said your host moved you to another server.

Might want to tell your host to use the thread cache. It is a pretty good feature. Why is InnoDB disabled? Ask your host to see the slow query log.
Title: Re: SMF overloading my server's CPU
Post by: gasman on November 29, 2005, 07:27:51 PM
Thanks groundup!

I'll disable user-selected avatars for a while and see if it helps.

Can we degrade to an older version of SMF? I imagine only by deleted current files and re-installing an older version. Would that be compatible with the existing database?

Yes, the website (?including database) was changed to a new server by the host, so I doubt that I can fix it by simply re-installing the database from a backup.

I have had output compression off the entire time.

cheers
Rob
Title: Re: SMF overloading my server's CPU
Post by: Ben_S on November 29, 2005, 07:46:45 PM
You cannot downgrade.
Title: Re: SMF overloading my server's CPU
Post by: Joshua Dickerson on November 30, 2005, 12:26:29 AM
Quote from: Ben_S on November 29, 2005, 07:46:45 PM
You cannot downgrade.
If you backed up you could ;)
Title: Re: SMF overloading my server's CPU
Post by: Angie on November 30, 2005, 03:13:11 AM
QuoteYour missing the point, shared server, lots of people, crammed on one server, each making loads of resource drains and this BIG site suddenly drives it up the wall.

This very community is not even twice as big and already was too big for a lesser powered dedicated server. Its on its own server just now.

Big site or not, posts will be loaded in pages, not all at once ...
a big site is producing high load, when many user are online at once.

this community has many user and guests online, often more than 300.

@gasman disable also "hostname" if not already disabled.

Lg Angie
Title: Re: SMF overloading my server's CPU
Post by: Ben_S on November 30, 2005, 05:21:30 AM
Quote from: Angie on November 30, 2005, 03:13:11 AM
Big site or not, posts will be loaded in pages, not all at once ...
a big site is producing high load, when many user are online at once.

Actually if it's searched a lot and it's not using InnoDB for it's log_search tables and it's not using a fulltext index, that could cause a problem.
Title: Re: SMF overloading my server's CPU
Post by: gasman on November 30, 2005, 05:31:09 AM
I have completely disabled search.

Anyways, I will try to do a database dump and then reload the database. If that fails, I will reload the forum software.

Fingers crossed.

Cheers, Rob
Title: Re: SMF overloading my server's CPU
Post by: ovoboy on December 06, 2005, 07:14:36 PM
Hi Gasman,


I have an account with lunarpages and they're complayning about the same problem aswell.
They moved my account to a smaller server while I'm trying to resolve the problem.
Did you get any luck with yours?

Thanks
Title: Re: SMF overloading my server's CPU
Post by: tX on December 06, 2005, 07:34:13 PM
If your whole site even is taking up that much spu power of the server the ohnly reason could be is they have a half fryed AMD duron 800 in there, thats underclocked to 400.
Title: Re: SMF overloading my server's CPU
Post by: SleePy on December 06, 2005, 07:35:08 PM
ahhh. i had the same problem but i got it fixed! running rc1 as well.
What i did is installed PHP 4.4.0 Compatibility Update. lunar has 4.4.1
i do not know why but after a few days this dropped by cpu usage way under the limit.
Also ssi request from anything outside the forum directory seems to take this up high as well. no clue as to why so i would do something about that if you got ssi requests from outside the forum directory.

if you want ill download the page on the support ticket i had with lunar till it got fixed.

-edit-
how we found out we had to install the update is we actually called the tech support and they told us in better detail what was causing it and how to fix it :)
Title: Re: SMF overloading my server's CPU
Post by: Joshua Dickerson on December 06, 2005, 08:08:11 PM
SleePy: you should tell them to upgrade ;)
Title: Re: SMF overloading my server's CPU
Post by: SleePy on December 06, 2005, 09:07:17 PM
lunar has many options on their shared servers. if you request to be moved to a server were you need something that your current server does not. if enough people request it they will add that to a couple of servers and move people around. If it is an overwhelming request they would apply it to almost all their shared servers.

i think its stupid though because i have to request to be moved to a server with pspell so i can use spell check? some things are stupid... really...

when lunar first question me with it i was like wtf? smf says right on their homepage
QuoteEven huge forums only need minimal resources
??? so why was i used almost 50% of the cpu resources?
Title: Re: SMF overloading my server's CPU
Post by: Joshua Dickerson on December 07, 2005, 12:47:04 AM
Ask them for PHP 5.1 citing tons of new functions and numerous performance enhancements, emphasize security enhancements (even though 4.4.1 should be as secure as 5)
Title: Re: SMF overloading my server's CPU
Post by: ovoboy on December 07, 2005, 07:18:05 AM
Quote from: SleePy on December 06, 2005, 07:35:08 PM
ahhh. i had the same problem but i got it fixed! running rc1 as well.
What i did is installed PHP 4.4.0 Compatibility Update. lunar has 4.4.1
i do not know why but after a few days this dropped by cpu usage way under the limit.
Also ssi request from anything outside the forum directory seems to take this up high as well. no clue as to why so i would do something about that if you got ssi requests from outside the forum directory.

if you want ill download the page on the support ticket i had with lunar till it got fixed.

-edit-
how we found out we had to install the update is we actually called the tech support and they told us in better detail what was causing it and how to fix it :)

Hi SleePy,

Can you send me the trouble ticket so that I can show the guys in lunarpages?
They are going to suspend my account tomorow

:D

Thanks
Title: Re: SMF overloading my server's CPU
Post by: Uzbekistan on December 07, 2005, 03:30:55 PM
Hi guys, I am having the same issue - my hoster has suspended my shared account because of "resources abuse". (ace-host)
Quote
username somedomain.com 4.02 17.55 16.7
Top Process %CPU 99.9 /usr/bin/php index.php
Top Process %CPU 23.0 [php ]
Top Process %CPU 18.0 [php ]

Please help me! Which hosters have no issues with smf?

PS. It is the second time it happened. I installed SMF in early summer. Since then I had a couple major problems with SMF, and once critical, when my account was suspended. Then i though of another script and removed it. But problems persisted... Now hosters are going to block my account forever! WTF!
Title: Re: SMF overloading my server's CPU
Post by: Ben_S on December 07, 2005, 04:32:57 PM
If you don't provide any info about your forum noone can help you.
Title: Re: SMF overloading my server's CPU
Post by: SleePy on December 07, 2005, 04:45:41 PM
Well since it gots some presonal information i just copied it and will remove that info and show you here.

Quote
Message by Thain on Fri, 18th Nov 2005 7:52 pm
Hello,

Your account is utilizing excessive resources, causing a significant degradation of services on the server. This is a shared environment and we can not allow one user to utilize the majority of the resources on a server as it affects all users adversely. Because of this, you have been temporarily moved to the Quantz server. A detail of the problem is shown below:

*** *****.com 1.71 1.20 0.5
Top Process %CPU 12.0 /usr/bin/php
Top Process %CPU 11.0 /usr/bin/php
Top Process %CPU 10.0 /usr/bin/php

11-17-05
*** *****.com 0.34 0.93 0.1
Top Process %CPU 15.0 /usr/bin/php
Top Process %CPU 14.0 /usr/bin/php
Top Process %CPU 13.0 /usr/bin/php

11-16-05
*** *****.com 0.50 0.94 0.1
Top Process %CPU 22.0 /usr/bin/php
Top Process %CPU 13.0 /usr/bin/php
Top Process %CPU 12.0 /usr/bin/php

We moved you to Quantz first to solve the issue of the degraded service on Perseus and second as a courtesy in order to prevent any downtime on your site to allow you time to rectify the situation. Currently, your site is not suitable for a shared hosting environment.

Quantz is not a production server and is not guaranteed to run in the same manner as your old server nor will it support all functions you may have. This is a transitional server only and is not intended to be a permanent placement. Please make a decision on how you would like to proceed within seven (7) days. After seven (7) days, your account will be suspended if no other arrangements are made.

You have the following options:

• You may upgrade your package to the Enterprise Plan. The Enterprise Plan is a larger package designed for current customers who have outgrown our published shared hosting packages. For more information on the features and pricing, please see http://helpdesk.lunarpages.com/faq.php?do=article&articleid=202
• You may upgrade your package to our Dedicated hosting plan. For more information on the features and pricing, please see http://helpdesk.lunarpages.com/faq.php?do=article&articleid=334
• [if applicable]You may take steps to correct the problem. This must be done before the seven (7) days are up. You will need to let the technician know the steps you took to correct the issue and get approval before being returned to a production server.
• You may look for a dedicated server or other hosting solution. Please remember that propagation takes up to seventy two (72) hours so if you decide on changing hosts you should allow enough time for propagation.

Thank you for your cooperation in this matter. We value you as a customer and want to work with you toward a solution that is mutually beneficial. Please let us know as quickly as possible how you would like to proceed.

Your message on Fri, 18th Nov 2005 9:19 pm
Hello,
I do not know what is taking up that much PHP resources.
Is there a way that i may view what files are being accessed the most and what files are using the most resources?
we have a forum on our site and im hoping it is not it as it is supposed to not use much resources to run.

Message by Marcus on Tue, 22nd Nov 2005 8:14 am
Hi,

We apologize for the delay in response. In reviewing your account, I have noticed that you are running a forum as well as a membership profile site. Here are your top 7 most visited URLs:

1 1638757 69.83% http://*****.com/forums/index.php
2 244824 10.43% http://www.*****.com/home.htm
3 163716 6.98% - (Direct Request)
4 156792 6.68% http://*****.com/forums/
5 30429 1.30% http://www.*****.com/forums/
6 28279 1.20% http://www.*****.com/forums/index.php
7 26674 1.14% http://*****.com/home.htm

As you can see number 1 and 6 are the most prevalent as they are PHP files and PHP is what is causing excessive CPU usage. You can see that your forums index page is the most visited page. This information can be viewed by looking at webalizer stats in Web/FTP stats in cpanel.

Your message on Wed, 23rd Nov 2005 2:29 am
Thanks for the response.
After some searching we found out the the forums we were running had an upgrade to upgrade the php it used to the version that was closer to what was in cpanel.
We Think this has fixed it but if you wouldn't mind watching it for a few days for us to tell us if it has fixed.

Message by Marcus on Wed, 23rd Nov 2005 7:39 am
Hi,

We will continue to monitor your account for usage as it shouldn't take much to get it below acceptible levels.

Message by Marcus on Wed, 23rd Nov 2005 7:41 am
Hi,

I have pulled your usage stats for tonight and they are as follows:

11/22/2005
*** *****.com 0.88 0.34 0.1
Top Process %CPU 31.0 /usr/bin/php
Top Process %CPU 27.0 /usr/bin/php
Top Process %CPU 22.0 /usr/bin/php

Your overall usage is ok, however, we still your php usage to drop below 20%

If you have any further questions in the meantime, please do not hesitate to contact us.

Your message on Wed, 23rd Nov 2005 10:12 pm
Is there a way that i can be told what is using all the php requests? i have no idea what php files are causing this much php usage. my only guess would be forums.
Could it be because allot of our pages now use php? like our forums, home.htm, 2 subdomains, and a few other files all use php. Could the cause of the over usage of php be that we are using to much php?

Message by erik.chen on Thu, 24th Nov 2005 2:28 am
Hello,

The top processes are generated by one of your scripts when it is activated. So the top processes are the highest CPU% that your script generates at one time. Your usage for today has definetly decreased though. We continue to monitor your usage since the decreased traffic may have to do with the holidays. If all is well by next Monday, we will move you back. Here are your stats :

*** *****.com 0.36 0.39 0.0
Top Process %CPU 8.4 gzip
Top Process %CPU 7.4 gzip
Top Process %CPU 7.3 /usr/bin/php

Thanks

Message by erik.chen on Sat, 26th Nov 2005 2:04 am
Hello,

Your usage has gone down enough and I have moved you back to perseus. I have resent your critical account info to ***********

Thanks

At the End here after he said it did not go down i went and removed something from our .htaccess i think was causing it. some reason every html page was being processed though the php engine cuss of this
AddHandler x-httpd-php .html
AddHandler x-httpd-php .htm


removing that and all my ssi from stuff outside the forums directory took it down. 

ps. the only reason the gzip was up above php is i just did a home directory backup and sql backup about 2 hours before he posted this. it was part of my weekly backup and a backup before the site was going to go down so it would be back if we lost our account.
Title: Re: SMF overloading my server's CPU
Post by: Ben_S on December 07, 2005, 04:47:53 PM
And how big is your forum, how many online and over what time period.

Appologies, too many people posting to the same threads, would be an idea if people started their own threads with the relevant info, keeping track of various posters in a single thread is difficult.
Title: Re: SMF overloading my server's CPU
Post by: SleePy on December 07, 2005, 04:52:14 PM
read my sig it tells :)

i do not need support. im helping these people because i went though the same thing but i am now ok.
Title: Re: SMF overloading my server's CPU
Post by: ovoboy on December 07, 2005, 05:36:00 PM
Hi,

My site was moved to quantz also.
One thing that I can't understand, what you upgraded PHP? the forum to 1.1?
Was it you that did the upgraded or it was lunarpages?
Title: Re: SMF overloading my server's CPU
Post by: Uzbekistan on December 07, 2005, 05:41:50 PM
Quote from: Ben_S on December 07, 2005, 04:32:57 PM
If you don't provide any info about your forum noone can help you.
Pardon, I was annoyed by ace-host. my fault...

I was running SMF 1.1 RC1 with Arcade MOD installed. I had about 1700 users and about 50-60 users online. It is not a big site (compared to this one).
Shared server had Apache 1.3 installed, PHP4 (don't remember exact versioning).
What surprises me is that it workes fine for sometime and then gives me errors every two months :(.
Title: Re: SMF overloading my server's CPU
Post by: SleePy on December 07, 2005, 05:43:04 PM
http://lordsofclantribe.com/forums/Packages/php-4.4.0-compat.tar.gz

thats the modification i applied. use package manger and upload it and run it.
Title: Re: SMF overloading my server's CPU
Post by: Ben_S on December 07, 2005, 05:47:47 PM
If http://www.ace-host.net/ is the host you mean, then frankly, looking at their prices I'm not surprised you are having preoblems.
Title: Re: SMF overloading my server's CPU
Post by: ovoboy on December 07, 2005, 05:55:13 PM
Many Thanks Sleepy,


I've just applied the upgrade, I've spoke to the people from lunarpages, i'm crossing my finguers for everything to work
Title: Re: SMF overloading my server's CPU
Post by: Uzbekistan on December 08, 2005, 04:50:40 AM
Quote from: SleePy on December 07, 2005, 05:43:04 PM
http://lordsofclantribe.com/forums/Packages/php-4.4.0-compat.tar.gz

thats the modification i applied. use package manger and upload it and run it.


I thought that smf 1.1 RC1 was already compatible with PHP4.4 :( And it worked fine for a few months now...

People, could you recommend me a good host? I am thinking of hostgator & lunarpages.
Regards.
Title: Re: SMF overloading my server's CPU
Post by: Oldiesmann on December 08, 2005, 09:52:40 AM
Hostgator is a good one. I wouldn't recommend Lunarpages though.

Others:
http://www.achohosting.com
http://www.killhost.com
Title: Re: SMF overloading my server's CPU
Post by: ovoboy on December 09, 2005, 06:07:23 PM
Well, nothing worked for me.

My CPU levels usage are still too high (that's what they say).
I'm thinking on uninstalling SMF and installing it again, do you think that whould do or I will end in the same place?
Anyone has other clues?

my phpinfo file is this http://www.geracaox.com/phpinfo.php (http://www.geracaox.com/phpinfo.php)
the forum has 5381 users 13776 messages and 60 concurrent users.

Thanks
Title: Re: SMF overloading my server's CPU
Post by: SleePy on December 09, 2005, 07:50:33 PM
try to remove any ssi.

also make sure that only .php files are being used by the php engine.
this can be checked in Apache handlers.
and user added one make sure its not trying to use the php engine especially if the file extension is used allot on your server.
Title: Re: SMF overloading my server's CPU
Post by: Uzbekistan on December 12, 2005, 11:42:48 AM
Quote from: ovoboy on December 09, 2005, 06:07:23 PM
Well, nothing worked for me.

My CPU levels usage are still too high (that's what they say).
I'm thinking on uninstalling SMF and installing it again, do you think that whould do or I will end in the same place?
Anyone has other clues?

my phpinfo file is this http://www.geracaox.com/phpinfo.php (http://www.geracaox.com/phpinfo.php)
the forum has 5381 users 13776 messages and 60 concurrent users.

Thanks

Go to cpanel -> MySql Databases and click "Repair" opposite your smf database. You'll have to wait for some time. It worked for me before. For some reason smf "breaks" tables in database.
Title: Re: SMF overloading my server's CPU
Post by: Ben_S on December 12, 2005, 12:14:30 PM
Quote from: Uzbek on December 12, 2005, 11:42:48 AM
Go to cpanel -> MySql Databases and click "Repair" opposite your smf database. You'll have to wait for some time. It worked for me before. For some reason smf "breaks" tables in database.

No it doesn't all SMF does is issues commands to MySQL. If a table crashes, then thats down to MySQL, usually because the hosts server is overloaded and MySQL exits abnormally.

Anyway, I don't see anything in the original posters message that says that he had issues with broken / crashed tables.
Title: Re: SMF overloading my server's CPU
Post by: gasman on December 14, 2005, 01:07:26 AM
Well, I finally got kicked off Lunarpages hosting because of CPU overloading. To their credit, they let me keep everything up until I could make backups and transfer to a new host.

I have gone with a virtual private server (with PowerVPS). Although much more expensive, I can transfer all (4) of my websites that were previously scattered across several shared hosting plans, and consolidate them on "my" server.

I can now monitor "my" CPU usage directly and install whatever server components I wish.  :D

However, I still see huge CPU usage from the forum. Up to 8% (on a quad CPU server) average per 24 hour period!

The single most resource intensive php script is the avatar loader (I forget the exact name of script - I'll edit post to add it later).

Reloading the forum software and database made no difference.
I have no SSI calls from the forum.
I did "Repair" MySQL tables.
I disabled search.
I banned guests (to get rid of search engine spiders from hogging the resources)

None of this made a difference.

I'll look into the php version and see if the php 4.4 compatibility mod makes a difference.

At least, since now it is "my" server, I don't get harrassed by my host about the CPU usage!

cheers
Rob

Title: Re: SMF overloading my server's CPU
Post by: redone on December 14, 2005, 01:14:03 AM
I don't know how big your forum is rob but there are some big forums running SMF these days. If you need further help getting the most out of your server I am sure you will find it here.

Thanks for the update.

:)
Title: Re: SMF overloading my server's CPU
Post by: Ben_S on December 14, 2005, 06:02:03 AM
WHM's monitoring things aren't exactly the most accurate things.
Title: Re: SMF overloading my server's CPU
Post by: Joshua Dickerson on December 14, 2005, 10:48:10 AM
gasman: it isn't really 'your' usage. You still share the server with someone else, it just appears that you don't.
Title: Re: SMF overloading my server's CPU
Post by: David Wong on February 26, 2006, 07:40:45 PM
Okay, I'm glad I found this thread because there appears to be something seriously wrong with Lunarpages' ability to host SMF.  I, too, got kicked off the shared account because of high usage (50 to 100 people on the forums at one time).  I moved to their entry-level dedicated machine, then had to move off of it because it, too, couldn't handle the load of the forums.

As I said on another thread, I'm on their 3.0 GHz dedicated machine with a RAM upgrade to 2 GB.  And my CPU is STILL running at 25 to 40%.  I asked Lunarpages tech support staff, they suggested I upgrade again.  Again, the same 50 to 100 people on at a time.

I assume there's something seriously wrong with their mysql configuration, but I have no idea what to do about it.  I'm not a mysql programmer, I'm just a lowly webmaster who knows FTP and HTML and how to find out where my audience is getting referred from. 

But seriously, what could LP be doing so wrong?  Their dedicated hosting is suspiciously cheap, but I've always been afraid to move for fear I'll wind up paying twice as much and having the same problems.  But they really are clueless when it comes to resource issues.

Title: Re: SMF overloading my server's CPU
Post by: sirwoogie on February 26, 2006, 09:57:31 PM
Boy, they must really have something wrong in their configurations. I've been following this thread and some others (http://www.simplemachines.org/community/index.php?topic=73460.0) that all have issues with LP. I've never hosted with them, but it sounds to me it would be wise to take your money elsewhere. I have several hosts with 1and1 on dedicated that do not even come close to your issue. Looking at their prices, you're best to find another dedicated machine for the price range you are in now with companies that would more than likely not have any issue with your workload.

My forum is well over 366 posts per day with upwards of 170 people online pushing almost 500k messages. It's a 3Ghz machine with a gig of RAM running Gentoo on 2.6 (http://forum.arcadecontrols.com/status.php). I rarely go above 10% utilization. We host well over 20 domains, email (spam/virus scan), mysql, etc. on the single host with a single 120 IDE drive; that 10% includes those components.

May I suggest you take a look at the Hosts and Hosting (http://www.simplemachines.org/community/index.php?board=4.0) forum for good hosts, or review WebHostingTalk (http://www.webhostingtalk.com) for some others.

Best of luck!