forum suspended by hostmonster due to excessive MYSQL queries

[caper12]

Hello. My forum was recently suspended without warning by Hostmonster (reason: site causing performance problems). I called them after I optimized the tables and repaired them. I also was running simple portal an disabled that. They reactivated the account and told me that if it happens again, it's "strike two"

So now I am paranoid and want to figure out what the problem was. The techs at Hostmonster told me I need to keep an eye on the slow queries logs. I don't know what to look for in the logs.

Here is a link to my status.php. Can someone please have a look at it and give me some input on the numbers?

hxxp:www.flamewarriors.net/status.php [nonactive]

[青山 素子]

Hmmm....

UNLIMITED Hosting Space
UNLIMITED Site Bandwidth
Host UNLIMITED Domains

For $5.95 a month? Something tells me that's the real problem.


What kind of traffic are you seeing daily?

[caper12]

Hmmm....

UNLIMITED Hosting Space
UNLIMITED Site Bandwidth
Host UNLIMITED Domains

For $5.95 a month? Something tells me that's the real problem.


What kind of traffic are you seeing daily?

Not a lot, I have only about 200 active members. Can anyone look at my stat.php and see if they can tell me anything? Hostmonster is a huge company and I have several other websites hosted with them and no problems.

Thanks for any help.

Here are the stats from November

Total Hits    1661704
Total Files    757210
Total Pages    473249
Total Visits    31299
Total KBytes    13498402
Total Unique Sites    3117
Total Unique URLs    637
Total Unique Referrers    245
Total Unique User Agents    941
.                   Avg             Max
Hits per Hour    2307    10807
Hits per Day    55390    128908
Files per Day    25240    54786
Pages per Day    15774    38667
Visits per Day    1043    2781
KBytes per Day    449947    1170003

[vbgamer45]

See if you have any caching options under Server Settings => Caching that will help with some queries.

[RobertMfromLI]

I do not know if this is your problem, but here's my suggestion (we ran into this a number of times when we were using them - it was a near regular occurrence):




You have two options that may help, IF the problems are being caused by the same miscreant who hit us:

   
• Add an entry (the FIRST entry) into robots.txt to institute a crawl delay. Make it 100 or more.
• BAN MSNBot. ALL of them.

The first suggestion worked well for us for a while. Then suddenly MSNBot ignores the directive (we've made NO changes to robots.txt... it simply starts ignoring it). Then it'll start honoring it. Then it ignores it for a few months.


We found (some time last year when it went on yet another rampage) that 300GB of bandwidth was being used by those bots. And only 250GB by the forum users and other bots. Even during the "early days" when GoogleBot was hitting the pages pretty heavy (a few thousand requests more a day than MSNBot), we didnt notice them, as they seem to spread out their spidering. MSNBots would come in swarms of 200-300 at a time. Often, 20-30 of them would be on the same page. And occassionally, a few of them would simply repeatedly grab the same file (in some cases, a few THOUSAND times in a day).


Of course, you need to evaluate your need (or lack thereof) for listings on Bing. Us, we don't care. Out of search engine referrals, MSNBot accounts for under 5% (sometimes as low as 3%) of referrals (and the number is dropping). YMMV.


This, btw, was the EXACT cause of our problem when we used Hostmonster.
We never ran into this problem again (while we were with them), once we updated robots.txt. Though since (just recently), MSNBot doesnt seem to be respecting the directive in robots.txt anymore.


Best,
Robert

[RobertMfromLI]

There ARE other possibilities, such as:

   
• Your forum is being attacked by spammers or other miscreants
• You have other software on the system that has been compromised (ZenCart?)

But... that (MSNBot) was what caused our issues. Check your logs. Don't go by the spider logs... look for 207.46.*.* and 65.52.*.*, 65.53.*.*, 65.54.*.*, 65.55.*.*


For whatever reason, we've had lots of MSNBots using those IPs that don't seem to be sending msnbot as their UA.


Best,
Robert

[cactusdan]

Can I ask what your code looked like to do this?

I'm no coding monkey

Many thanks!

-Dan

[Aleksi "Lex" Kilpinen]

Seriously, seeing any warning about database activity from the host, should make one think about changing to a better host. Oversellers like that one tend to not like forums and other dynamic sites, because they can actually cause some load to the servers - unlike small static pages, or images etc. Stored there.

[Ensiferous]

Seriously, seeing any warning about database activity from the host, should make one think about changing to a better host. Oversellers like that one tend to not like forums and other dynamic sites, because they can actually cause some load to the servers - unlike small static pages, or images etc. Stored there.

Err, or it could just be because uploaded avatars by default cause tons of queries, which is a legitimate reason for saying someone is causing too much load.

[Aleksi "Lex" Kilpinen]

Seriously, seeing any warning about database activity from the host, should make one think about changing to a better host. Oversellers like that one tend to not like forums and other dynamic sites, because they can actually cause some load to the servers - unlike small static pages, or images etc. Stored there.

Err, or it could just be because uploaded avatars by default cause tons of queries, which is a legitimate reason for saying someone is causing too much load.

Well - true, it can of course do that - and then it may be a legitimate cause for the host to act on it, but then the host should really give some hard data too, about what is wrong, and not just, "excessive MYSQL queries", or "site causing performance problems" - which are the "template" causes for most oversellers, that don't want to - or can not - offer any more specific reasons...

[Ensiferous]

I wasn't defending the host, just the fact they promise unlimited is already a warning indicator. What to take away from my previous post is that there might actually be a legitimate problem.

[RobertMfromLI]

Seriously, seeing any warning about database activity from the host, should make one think about changing to a better host. Oversellers like that one tend to not like forums and other dynamic sites, because they can actually cause some load to the servers - unlike small static pages, or images etc. Stored there.

Which is exactly what we did.