SMF overloading my server's CPU

Started by gasman, November 25, 2005, 03:31:08 PM

Previous topic - Next topic

Ben_S

Liverpool FC Forum with 14 million+ posts.

Joshua Dickerson

Come work with me at Promenade Group



Need help? See the wiki. Want to help SMF? See the wiki!

Did you know you can help develop SMF? See us on Github.

How have you bettered the world today?

Angie

QuoteYour missing the point, shared server, lots of people, crammed on one server, each making loads of resource drains and this BIG site suddenly drives it up the wall.

This very community is not even twice as big and already was too big for a lesser powered dedicated server. Its on its own server just now.

Big site or not, posts will be loaded in pages, not all at once ...
a big site is producing high load, when many user are online at once.

this community has many user and guests online, often more than 300.

@gasman disable also "hostname" if not already disabled.

Lg Angie
Greetings from Angie (fiancé of mediman ;-))


Ben_S

Quote from: Angie on November 30, 2005, 03:13:11 AM
Big site or not, posts will be loaded in pages, not all at once ...
a big site is producing high load, when many user are online at once.

Actually if it's searched a lot and it's not using InnoDB for it's log_search tables and it's not using a fulltext index, that could cause a problem.
Liverpool FC Forum with 14 million+ posts.

gasman

I have completely disabled search.

Anyways, I will try to do a database dump and then reload the database. If that fails, I will reload the forum software.

Fingers crossed.

Cheers, Rob

ovoboy

Hi Gasman,


I have an account with lunarpages and they're complayning about the same problem aswell.
They moved my account to a smaller server while I'm trying to resolve the problem.
Did you get any luck with yours?

Thanks

tX

If your whole site even is taking up that much spu power of the server the ohnly reason could be is they have a half fryed AMD duron 800 in there, thats underclocked to 400.

SleePy

#27
ahhh. i had the same problem but i got it fixed! running rc1 as well.
What i did is installed PHP 4.4.0 Compatibility Update. lunar has 4.4.1
i do not know why but after a few days this dropped by cpu usage way under the limit.
Also ssi request from anything outside the forum directory seems to take this up high as well. no clue as to why so i would do something about that if you got ssi requests from outside the forum directory.

if you want ill download the page on the support ticket i had with lunar till it got fixed.

-edit-
how we found out we had to install the update is we actually called the tech support and they told us in better detail what was causing it and how to fix it :)
Jeremy D ~ Site Team / SMF Developer ~ GitHub Profile ~ Join us on IRC @ Libera.chat/#smf ~ Support the SMF Support team!

Joshua Dickerson

SleePy: you should tell them to upgrade ;)
Come work with me at Promenade Group



Need help? See the wiki. Want to help SMF? See the wiki!

Did you know you can help develop SMF? See us on Github.

How have you bettered the world today?

SleePy

lunar has many options on their shared servers. if you request to be moved to a server were you need something that your current server does not. if enough people request it they will add that to a couple of servers and move people around. If it is an overwhelming request they would apply it to almost all their shared servers.

i think its stupid though because i have to request to be moved to a server with pspell so i can use spell check? some things are stupid... really...

when lunar first question me with it i was like wtf? smf says right on their homepage
QuoteEven huge forums only need minimal resources
??? so why was i used almost 50% of the cpu resources?
Jeremy D ~ Site Team / SMF Developer ~ GitHub Profile ~ Join us on IRC @ Libera.chat/#smf ~ Support the SMF Support team!

Joshua Dickerson

Ask them for PHP 5.1 citing tons of new functions and numerous performance enhancements, emphasize security enhancements (even though 4.4.1 should be as secure as 5)
Come work with me at Promenade Group



Need help? See the wiki. Want to help SMF? See the wiki!

Did you know you can help develop SMF? See us on Github.

How have you bettered the world today?

ovoboy

Quote from: SleePy on December 06, 2005, 07:35:08 PM
ahhh. i had the same problem but i got it fixed! running rc1 as well.
What i did is installed PHP 4.4.0 Compatibility Update. lunar has 4.4.1
i do not know why but after a few days this dropped by cpu usage way under the limit.
Also ssi request from anything outside the forum directory seems to take this up high as well. no clue as to why so i would do something about that if you got ssi requests from outside the forum directory.

if you want ill download the page on the support ticket i had with lunar till it got fixed.

-edit-
how we found out we had to install the update is we actually called the tech support and they told us in better detail what was causing it and how to fix it :)

Hi SleePy,

Can you send me the trouble ticket so that I can show the guys in lunarpages?
They are going to suspend my account tomorow

:D

Thanks

Uzbekistan

#32
Hi guys, I am having the same issue - my hoster has suspended my shared account because of "resources abuse". (ace-host)
Quote
username somedomain.com 4.02 17.55 16.7
Top Process %CPU 99.9 /usr/bin/php index.php
Top Process %CPU 23.0 [php ]
Top Process %CPU 18.0 [php ]

Please help me! Which hosters have no issues with smf?

PS. It is the second time it happened. I installed SMF in early summer. Since then I had a couple major problems with SMF, and once critical, when my account was suspended. Then i though of another script and removed it. But problems persisted... Now hosters are going to block my account forever! WTF!

Ben_S

If you don't provide any info about your forum noone can help you.
Liverpool FC Forum with 14 million+ posts.

SleePy

#34
Well since it gots some presonal information i just copied it and will remove that info and show you here.

Quote
Message by Thain on Fri, 18th Nov 2005 7:52 pm
Hello,

Your account is utilizing excessive resources, causing a significant degradation of services on the server. This is a shared environment and we can not allow one user to utilize the majority of the resources on a server as it affects all users adversely. Because of this, you have been temporarily moved to the Quantz server. A detail of the problem is shown below:

*** *****.com 1.71 1.20 0.5
Top Process %CPU 12.0 /usr/bin/php
Top Process %CPU 11.0 /usr/bin/php
Top Process %CPU 10.0 /usr/bin/php

11-17-05
*** *****.com 0.34 0.93 0.1
Top Process %CPU 15.0 /usr/bin/php
Top Process %CPU 14.0 /usr/bin/php
Top Process %CPU 13.0 /usr/bin/php

11-16-05
*** *****.com 0.50 0.94 0.1
Top Process %CPU 22.0 /usr/bin/php
Top Process %CPU 13.0 /usr/bin/php
Top Process %CPU 12.0 /usr/bin/php

We moved you to Quantz first to solve the issue of the degraded service on Perseus and second as a courtesy in order to prevent any downtime on your site to allow you time to rectify the situation. Currently, your site is not suitable for a shared hosting environment.

Quantz is not a production server and is not guaranteed to run in the same manner as your old server nor will it support all functions you may have. This is a transitional server only and is not intended to be a permanent placement. Please make a decision on how you would like to proceed within seven (7) days. After seven (7) days, your account will be suspended if no other arrangements are made.

You have the following options:

• You may upgrade your package to the Enterprise Plan. The Enterprise Plan is a larger package designed for current customers who have outgrown our published shared hosting packages. For more information on the features and pricing, please see http://helpdesk.lunarpages.com/faq.php?do=article&articleid=202
• You may upgrade your package to our Dedicated hosting plan. For more information on the features and pricing, please see http://helpdesk.lunarpages.com/faq.php?do=article&articleid=334
• [if applicable]You may take steps to correct the problem. This must be done before the seven (7) days are up. You will need to let the technician know the steps you took to correct the issue and get approval before being returned to a production server.
• You may look for a dedicated server or other hosting solution. Please remember that propagation takes up to seventy two (72) hours so if you decide on changing hosts you should allow enough time for propagation.

Thank you for your cooperation in this matter. We value you as a customer and want to work with you toward a solution that is mutually beneficial. Please let us know as quickly as possible how you would like to proceed.

Your message on Fri, 18th Nov 2005 9:19 pm
Hello,
I do not know what is taking up that much PHP resources.
Is there a way that i may view what files are being accessed the most and what files are using the most resources?
we have a forum on our site and im hoping it is not it as it is supposed to not use much resources to run.

Message by Marcus on Tue, 22nd Nov 2005 8:14 am
Hi,

We apologize for the delay in response. In reviewing your account, I have noticed that you are running a forum as well as a membership profile site. Here are your top 7 most visited URLs:

1 1638757 69.83% http://*****.com/forums/index.php
2 244824 10.43% http://www.*****.com/home.htm
3 163716 6.98% - (Direct Request)
4 156792 6.68% http://*****.com/forums/
5 30429 1.30% http://www.*****.com/forums/
6 28279 1.20% http://www.*****.com/forums/index.php
7 26674 1.14% http://*****.com/home.htm

As you can see number 1 and 6 are the most prevalent as they are PHP files and PHP is what is causing excessive CPU usage. You can see that your forums index page is the most visited page. This information can be viewed by looking at webalizer stats in Web/FTP stats in cpanel.

Your message on Wed, 23rd Nov 2005 2:29 am
Thanks for the response.
After some searching we found out the the forums we were running had an upgrade to upgrade the php it used to the version that was closer to what was in cpanel.
We Think this has fixed it but if you wouldn't mind watching it for a few days for us to tell us if it has fixed.

Message by Marcus on Wed, 23rd Nov 2005 7:39 am
Hi,

We will continue to monitor your account for usage as it shouldn't take much to get it below acceptible levels.

Message by Marcus on Wed, 23rd Nov 2005 7:41 am
Hi,

I have pulled your usage stats for tonight and they are as follows:

11/22/2005
*** *****.com 0.88 0.34 0.1
Top Process %CPU 31.0 /usr/bin/php
Top Process %CPU 27.0 /usr/bin/php
Top Process %CPU 22.0 /usr/bin/php

Your overall usage is ok, however, we still your php usage to drop below 20%

If you have any further questions in the meantime, please do not hesitate to contact us.

Your message on Wed, 23rd Nov 2005 10:12 pm
Is there a way that i can be told what is using all the php requests? i have no idea what php files are causing this much php usage. my only guess would be forums.
Could it be because allot of our pages now use php? like our forums, home.htm, 2 subdomains, and a few other files all use php. Could the cause of the over usage of php be that we are using to much php?

Message by erik.chen on Thu, 24th Nov 2005 2:28 am
Hello,

The top processes are generated by one of your scripts when it is activated. So the top processes are the highest CPU% that your script generates at one time. Your usage for today has definetly decreased though. We continue to monitor your usage since the decreased traffic may have to do with the holidays. If all is well by next Monday, we will move you back. Here are your stats :

*** *****.com 0.36 0.39 0.0
Top Process %CPU 8.4 gzip
Top Process %CPU 7.4 gzip
Top Process %CPU 7.3 /usr/bin/php

Thanks

Message by erik.chen on Sat, 26th Nov 2005 2:04 am
Hello,

Your usage has gone down enough and I have moved you back to perseus. I have resent your critical account info to ***********

Thanks

At the End here after he said it did not go down i went and removed something from our .htaccess i think was causing it. some reason every html page was being processed though the php engine cuss of this
AddHandler x-httpd-php .html
AddHandler x-httpd-php .htm


removing that and all my ssi from stuff outside the forums directory took it down. 

ps. the only reason the gzip was up above php is i just did a home directory backup and sql backup about 2 hours before he posted this. it was part of my weekly backup and a backup before the site was going to go down so it would be back if we lost our account.
Jeremy D ~ Site Team / SMF Developer ~ GitHub Profile ~ Join us on IRC @ Libera.chat/#smf ~ Support the SMF Support team!

Ben_S

#35
And how big is your forum, how many online and over what time period.

Appologies, too many people posting to the same threads, would be an idea if people started their own threads with the relevant info, keeping track of various posters in a single thread is difficult.
Liverpool FC Forum with 14 million+ posts.

SleePy

read my sig it tells :)

i do not need support. im helping these people because i went though the same thing but i am now ok.
Jeremy D ~ Site Team / SMF Developer ~ GitHub Profile ~ Join us on IRC @ Libera.chat/#smf ~ Support the SMF Support team!

ovoboy

Hi,

My site was moved to quantz also.
One thing that I can't understand, what you upgraded PHP? the forum to 1.1?
Was it you that did the upgraded or it was lunarpages?

Uzbekistan

Quote from: Ben_S on December 07, 2005, 04:32:57 PM
If you don't provide any info about your forum noone can help you.
Pardon, I was annoyed by ace-host. my fault...

I was running SMF 1.1 RC1 with Arcade MOD installed. I had about 1700 users and about 50-60 users online. It is not a big site (compared to this one).
Shared server had Apache 1.3 installed, PHP4 (don't remember exact versioning).
What surprises me is that it workes fine for sometime and then gives me errors every two months :(.

SleePy

Jeremy D ~ Site Team / SMF Developer ~ GitHub Profile ~ Join us on IRC @ Libera.chat/#smf ~ Support the SMF Support team!

Advertisement: