News:

SMF 2.1.4 has been released! Take it for a spin! Read more.

Main Menu

Due to High Stress on the server....

Started by pierse, August 23, 2005, 07:36:17 AM

Previous topic - Next topic

klumy


Joseph

Joseph Fung
Co-Founder & CEO, Kiite
Co-Founder TribeHR, Lewis Media, SMF

Bonk

Search function, search function, search function. It still needs serious work. I'm coming close to taking a look at it myself.

Ben_S

It may happen now and then, every time I have used it of late it's been available.

It's only sensible to disable features in the event of high load so as no avoid locking everything up.

Even with a decent server, with a large board there will always be times when load is high.
Liverpool FC Forum with 14 million+ posts.

Bonk

#44
The search function (at least on 1.0.5) causes horrid resource use, even if there is only two users on the board, if one is searching the other must wait for that search before he is served any pages... it is not load related, it is code related.

I'm pretty sure the same thing is happening here on 1.1 still. If you experience a long delay in a page load, go to the who's online list and you're sure to find someone viewing search results. All users should not have to wait for one user's search result... I think I'll look at the search code later today to see if I can figure out why this is the case.

Bonk

#45
Case in point:

Just moments ago:
QuotePage created in 8.942 seconds with 13 queries.
When loading:
http://www.simplemachines.org/community/index.php?board=2.0

A quick peek at:
http://www.simplemachines.org/community/index.php?action=who
shows:
Quote
Bonk     08:45:28 AM     Viewing Who's Online.
Spidla     08:45:00 AM     Searching the forum.
a user searching the forum ....   :o  ;)

The search function still needs work. I'm sure the server is fine.


Ben_S

#46
1.1's search is vastly improved, although I think compuart is testing a new search option here.

Try a search, everything I've tried searching for completes in under 2 seconds. Most likely it's something else like view unread posts, web stats generaton, backup or the dreaded show the posts of this user which is a nightmare.

Search in 1.1 is pretty good.
Liverpool FC Forum with 14 million+ posts.

Bonk

Searched for "search":
Quote
Page created in 3.065 seconds with 25 queries.

But the question is did was everyone else on the board waiting for that search too?

If so this is a way to perform a DoS attack on an SMF board, that would shut it down completely, requiring removal of the search function.

Bonk

I just tried another search for "starfleet command dynaverse":

I got:
Quote
Your query was not specific enough. Try using larger words, or less common phrases.

An inappropriate response.

Ben_S

There would have been some wait, but nowhere near the full 3 seconds. If someone did try to DoS the board with search the load would rise anyway and it would disable itself.

Also I'm fairly sure 1.1 has an option to stop search being flooded.

There are much more resource heavy things than search.

Quote from: Bonk on November 17, 2005, 09:04:36 AM
I just tried another search for "starfleet command dynaverse":

I got:
Quote
Your query was not specific enough. Try using larger words, or less common phrases.

An inappropriate response.

I'll give you that one ;D, but like I mentioned, search is still undergoing development.
Liverpool FC Forum with 14 million+ posts.

Bonk

Right, to prevent such a DoS search must be disabled...  :(

StarWars Fan

Quote from: Ben_S on November 17, 2005, 09:05:45 AM
If someone did try to DoS the board with search the load would rise anyway and it would disable itself.

I've disabled Search for Guests on my board just because of what I've seen here about that...

Bonk

Quote from: That's The Hacker to you on November 17, 2005, 09:09:14 AM
Quote from: Ben_S on November 17, 2005, 09:05:45 AM
If someone did try to DoS the board with search the load would rise anyway and it would disable itself.

I've disabled Search for Guests on my board just because of what I've seen here about that...

That is unfortunate for your guests.  :(

Anyhoo, its good to know that the search function is still a work in progress that is actively being worked on. Hopefully SMF will get it working optimally soon. Its a pretty important (if not the most important) function of forum software.

Keep at it!  :D

StarWars Fan

Quote from: Bonk on November 17, 2005, 09:12:49 AM
That is unfortunate for your guests.  :(

No biggie - all they have to do is register and then they can search...

Then you know who is Searching - not some Guest trying to overload your board...

And if the Search is so CPU intensive then you're being a better "neighbor" on a shared hosting plan...

And the most important Members on your board still have a Search function that isn't Fuseboxed shut...

Ben_S

In all honesty, unless you have say half a million posts it's not going to be that much of an issue, and if you have half a million posts, you would certainly be better of using 1.1 with it's fulltext search option.
Liverpool FC Forum with 14 million+ posts.

Bonk

#55
Quote from: Ben_S on November 17, 2005, 09:46:57 AM
In all honesty, unless you have say half a million posts it's not going to be that much of an issue, and if you have half a million posts, you would certainly be better of using 1.1 with it's fulltext search option.

We have 416712 posts currently.

I will use 1.1 when it is no longer a release candidate.

Also I fear that all the mods (not features) we have in use with 1.0.5, will not be compatible with 1.1 and make our users, moderators and admins very unhappy.

I do not like the concept of mods and templates at all because of this. I potentially have to rebuild the forums with each new SMF release.

Unfortunately I did not start the forums I am now managing, so I was stuck with mods that our users and moderators and admins have gotten used to and will not go without.

Note: our server is quad xeons with 1GB of ram and a huge raid array - running php as an apache module with eAccelerator and the search function still cripples it. (Windows server 2003, Apache 2.0.55, php 5.0.5, MySQL 5.0.15 and my own compile of eacclerator 0.9.3 for php 5.0.5 as that arnot fellow is way to slow with it...)

Note my posted fixes for SMF 1.0.5 on php 5.0.5 and MySQL 5.0.15


Sooo...... hurry up with 1.1.0 then, and make its search function non-crippling and incorporate all the features that have been passed off to modders when they should just be features of SMF.  >:( :P

Ben_S

1.1 CVS snapshots works fine here for me with 1.5 million posts on a p4 3Ghz, 1.5GB ram and a single SATA hard drive.

Would hate to think about using 1.0x's search feature with a load of posts. lD
Liverpool FC Forum with 14 million+ posts.

Bonk

#57
Quote from: Ben_S on November 17, 2005, 10:56:51 AM
1.1 CVS snapshots works fine here for me with 1.5 million posts on a p4 3Ghz, 1.5GB ram and a single SATA hard drive.

Would hate to think about using 1.0x's search feature with a load of posts. lD

Now you see my problem.

I do not have access to the SMF CVS repository.

Hopefully 1.1.0 will be released soon, hopefully my modded db will be compatible with it. I do not look forward to the work of the update, (with all the mods and templates) but look forward to the result. I may just end up not using any mods and using only the default template to avoid trouble and make future updates easier.

Ben_S

RC1 generally will be more stable than whats in the CVS anyway, I've always run the CVS snapshots to help with  beta testing, sometimes not with the best of results but hey, it helps SMF improve as a product with largish boards using it.
Liverpool FC Forum with 14 million+ posts.

JayBachatero

I'm running from the CVS too and the search has been improved.  The search is having problems at the moment like Ben_S said Compuart is working on it.  By RC2 the search should be more stable.
Follow me on Twitter

"HELP!!! I've fallen and I can't get up"
This moment has been brought to you by LifeAlert

Advertisement: