Simple Machines Community Forum

Simple Machines => News and Updates => Topic started by: dschwab9 on April 16, 2009, 03:15:08 PM

Title: New SQL Server
Post by: dschwab9 on April 16, 2009, 03:15:08 PM
Hey guys, just wanted to give everyone an update on the site status.

We have just completed migrating our SQL databases to a brand new box. The new machine is a 2.5GHz 8 core machine with 12GB of RAM and 6 146GB 10,000 RPM SAS drives in RAID 10. This should dramatically improve the speed of the forums.

One minor issue - during the migration, I lost connectivity to the old database server, requiring a hard reboot, which in turn corrupted a couple of tables. Due to this issue, there are roughly 4 hours worth of posts from this morning missing. If your post disappeared, I apologize.

If anyone finds any other issues, please report them in this topic and I will look into it.
Title: Re: New SQL Server
Post by: Fustrate on April 16, 2009, 03:19:22 PM
/me likes this news.
 
It's kind of scary loading a page this fast after the last month and a half... thanks for the awesome fright!
Title: Re: New SQL Server
Post by: grafitus on April 16, 2009, 03:20:00 PM
Vaov.Be Better..
Title: Re: New SQL Server
Post by: shadow82x on April 16, 2009, 03:22:33 PM
Very good news!

Quote
It's kind of scary loading a page this fast after the last month and a half... thanks for the awesome fright!
Agreed :P
Title: Re: New SQL Server
Post by: Deaks on April 16, 2009, 03:23:06 PM
darn you I lost about 10000 posts cause of you.


Nah I didnt I lost about 30 but ah well end of day its great to be back and have a faster server.

Great job and nice server specs  :)
Title: Re: New SQL Server
Post by: Aleksi "Lex" Kilpinen on April 16, 2009, 03:25:08 PM
Hmm... My last post is from Today at 10:50:05 - and now it's 22:23, so thats 12 hours minus maintanance time - missing, since I've been spamming the support boards since morning to the maintanence.... ::)

EDIT: But guess this is still good news - I just feel a bit sorry for all the lost support posts...
Title: Re: New SQL Server
Post by: SleePy on April 16, 2009, 03:25:20 PM
Thanks Derek :D

What I couldn't do with a server like that :P
Title: Re: New SQL Server
Post by: Fustrate on April 16, 2009, 03:26:48 PM
Thanks Derek :D

What I couldn't do with a server like that :P
Run Crysis.
Title: Re: New SQL Server
Post by: Tristan Perry on April 16, 2009, 03:42:08 PM
Wow, very amazing server :) Will definitely be able to handle much of what is thrown at it.
Title: Re: New SQL Server
Post by: Eliana Tamerin on April 16, 2009, 03:53:45 PM
Thanks Derek :D

What I couldn't do with a server like that :P
Run Crysis.

Bad graphics card, it'd look horrible. All black with little white text... *shudders*

Thanks Derek, for all your hard work on this.
Title: Re: New SQL Server
Post by: Spaceman-Spiff on April 16, 2009, 04:00:42 PM
That's ok, with an 8-core CPU, we can do software rendering of Crysis!

 ;D
Title: Re: New SQL Server
Post by: Adish - (F.L.A.M.E.R) on April 16, 2009, 04:02:10 PM
I think i lost about 70-100 posts. I resolved about 2 pages od Support Topics..!! Comon get them back.. I have to go through them again... :( marked many solved today...
Title: Re: New SQL Server
Post by: Aleksi "Lex" Kilpinen on April 16, 2009, 04:05:09 PM
I think i lost about 70-100 posts. I resolved about 2 pages od Support Topics..!! Comon get them back.. I have to go through them again... :( marked many solved today...
My initial thoughts too - but after a little thought, with the speed the forum runs now, I guess we'll soon catch up ;)
Title: Re: New SQL Server
Post by: Adish - (F.L.A.M.E.R) on April 16, 2009, 04:06:52 PM
Aah, i would love to post again, but seriously, i might have marked over 20 topics solved today and posted about 100.. i was on a post count of about 130 when it was in maintanance.. (not sure though)
Title: Re: New SQL Server
Post by: Fustrate on April 16, 2009, 04:07:58 PM
Then stop complaining and go solve them again ;)
Title: Re: New SQL Server
Post by: Aleksi "Lex" Kilpinen on April 16, 2009, 04:09:16 PM
Then stop complaining and go solve them again ;)
Repetition is the mother of all learning... ;)
Title: Re: New SQL Server
Post by: Adish - (F.L.A.M.E.R) on April 16, 2009, 04:09:59 PM
I am on it boss.. :P but after sometime.. If it could get back, i think it would be a great thing. (just to be sure if we arnt getting it back..)
Title: Re: New SQL Server
Post by: Adish - (F.L.A.M.E.R) on April 16, 2009, 04:13:40 PM
Just reporting in a problem.. (maybe!) I dont see any unread posts even though people are making posts in different sections. Unread replies works fine.
Title: Re: New SQL Server
Post by: Aleksi "Lex" Kilpinen on April 16, 2009, 04:18:28 PM
Search seems to not work, "Unable to access the search daemon"
Title: Re: New SQL Server
Post by: Dzonny on April 16, 2009, 04:36:47 PM
I'm glad that community is speeded up.. :)
/me is happy
Title: Re: New SQL Server
Post by: dschwab9 on April 16, 2009, 04:39:01 PM
Will look at search shortly. I think I need to reconfigure Sphynx to work with the new server.
Title: Re: New SQL Server
Post by: dschwab9 on April 16, 2009, 04:48:41 PM
Search should be working now.
Title: Re: New SQL Server
Post by: GravuTrad on April 16, 2009, 05:12:23 PM
The forum isn't putted in maintenance mode before this kind of operation?

for the search problem, it was like each change of server, sphinx problem like you said. ;)
http://www.simplemachines.org/community/index.php?topic=295395.msg1949737#msg1949737 (http://www.simplemachines.org/community/index.php?topic=295395.msg1949737#msg1949737)
http://www.simplemachines.org/community/index.php?topic=295395.msg1954602#msg1954602 (http://www.simplemachines.org/community/index.php?topic=295395.msg1954602#msg1954602)
http://www.simplemachines.org/community/index.php?topic=217492.0

Thanks derek

nb: the requests go really faster now, thanks.
Title: Re: New SQL Server
Post by: dschwab9 on April 16, 2009, 05:25:18 PM
It's all great news, except... If it was planned, couldn't the team have performed a backup before upgrading?
We did - that's why it was only 4 hours instead of days worth of stuff ::)

The forum isn't putted in maintenance mode before this kind of operation?
It was in maintenance mode....
Title: Re: New SQL Server
Post by: Eliana Tamerin on April 16, 2009, 05:42:12 PM
You didn't see the blank page with black text that said "We're upgrading the server." ?
Title: Re: New SQL Server
Post by: JBlaze on April 16, 2009, 05:46:49 PM
/me lost ~35 posts :(

All for the better though :)
Title: Re: New SQL Server
Post by: GravuTrad on April 16, 2009, 06:05:01 PM
Yes i had it, but if you are in maintenance mode, you do a backup of your bases and you reinject them in the new server, you don't lose any posts.

So the migration process is on the fly between the two servers? (like do plesk for example)
no backup before? to can redo it as much as you want?
Title: Re: New SQL Server
Post by: TheDisturbedOne on April 16, 2009, 06:25:09 PM
Hey guys, just wanted to give everyone an update on the site status.

We have just completed migrating our SQL databases to a brand new box. The new machine is a 2.5GHz 8 core machine with 12GB of RAM and 6 146GB 10,000 RPM SAS drives in RAID 10. This should dramatically improve the speed of the forums.

One minor issue - during the migration, I lost connectivity to the old database server, requiring a hard reboot, which in turn corrupted a couple of tables. Due to this issue, there are roughly 4 hours worth of posts from this morning missing. If your post disappeared, I apologize.

If anyone finds any other issues, please report them in this topic and I will look into it.
A quick questiion.  About how big is the SQL database for the community forums?
Title: Re: New SQL Server
Post by: bullbreedluverz on April 16, 2009, 06:48:22 PM
hey its great to have the server issues sorted

but just a thought

as is done on most sites if staff plan to put the site in maintenance mode it would be nice if an announcement was made to inform members that scheduled work will be taking place

if there was an announcement made then i didnt see it but i see nothing in news & updates and i recieved no email/newsletter
Title: Re: New SQL Server
Post by: Nolt on April 16, 2009, 06:49:36 PM
Great job... and thanks for this upgrade.
Title: Re: New SQL Server
Post by: JBlaze on April 16, 2009, 08:59:44 PM
Just a quick question...

Can the lost posts be recovered or not?
Title: Re: New SQL Server
Post by: Fustrate on April 16, 2009, 09:00:17 PM
As I understand it, no.
Title: Re: New SQL Server
Post by: SleePy on April 16, 2009, 09:07:53 PM
For only 4 hours of posts, I don't think it is worth it really.

Its only 4 hours, it isn't like its 4 months.
Title: Re: New SQL Server
Post by: GravuTrad on April 16, 2009, 09:24:52 PM
I believe the real question was, for 4 hours of post, why don't redo the migration...
This question was important to answer for derek and the team, the choose has been taken, it's now to late...

so...4 hours of posts is not a lot for each member...maybe it's more with the total of the members of this forum...but the choices have been made. It's over now. ;)

We have to go to something else now...life continues lol.... :D
Title: Re: New SQL Server
Post by: JBlaze on April 16, 2009, 09:26:46 PM
I was just asking :)

The nice conversation I was having in the Chit Chat board is now gone :(
Title: Re: New SQL Server
Post by: Adish - (F.L.A.M.E.R) on April 16, 2009, 09:30:36 PM
Please dont do further migration for now until you are sure with the totally. I finally finished the 2 pages of Support topics again! ...Phew!
Title: Re: New SQL Server
Post by: JBlaze on April 16, 2009, 09:32:00 PM
Also, on another note, I dont know if this was covered but i have lost 15+ PM's, some of which were important :(
Title: Re: New SQL Server
Post by: IchBin™ on April 16, 2009, 10:03:06 PM
I think you guys missed out on some key points. They put the forum in maintenance mode to do the DB migration. Trying to do a backup BEFORE they do the migration. Derek lost the connection during this, so it sounds like he had to roll back to a previous backup. There's no point in trying to recover if the site has been opened up again... Especially since it sounds like he doesn't have anything to restore to other than a previous backup before the disconnect.

Derek,

Thanks for your hard work. From what I hear your work schedule has been really hellish for you lately. The fact that you had the time, and were able to give it to SMF to fix our issues is very generous of you. Thank you for your hard work and dedication. You da man! This site is faster than it has been in long time for me. Two thumbs up for you.


Title: Re: New SQL Server
Post by: aldo on April 16, 2009, 11:43:16 PM
Sweet! Seems faster, but what do I know? ::)
Title: Re: New SQL Server
Post by: Aleksi "Lex" Kilpinen on April 17, 2009, 01:15:57 AM
I would like to say my thanks as well, my site hasn't really been  too fast lately (since we're really limited on cash and have to make compromises with limited resources :P ) and still SMF seemed even slower lately - but after this change - I want a new server too!  :P
Title: Re: New SQL Server
Post by: Yigal on April 17, 2009, 01:54:03 AM
Well...Everything comes with strings ^^.
Title: Re: New SQL Server
Post by: JBlaze on April 17, 2009, 02:21:19 AM
I would like to say my thanks as well, my site hasn't really been  too fast lately (since we're really limited on cash and have to make compromises with limited resources :P ) and still SMF seemed even slower lately - but after this change - I want a new server too!  :P

As do I. My server just... well... SUCKS! :)
Title: Re: New SQL Server
Post by: Yigal on April 17, 2009, 02:23:33 AM
What serve are you running on?

btw If I didn't catch it, what host is Simple Machines running under?
Title: Re: New SQL Server
Post by: JBlaze on April 17, 2009, 02:25:06 AM
Shared hosting by HostMonster.

Downtime has been @5-10% this month. It's getting ridiculous.
Title: Re: New SQL Server
Post by: Yigal on April 17, 2009, 02:27:18 AM
Lol, change it, to something like what this site has...XD

Wow, these guys can afford this software....just look at all the members in the charter area....about 500 ... now times that by 50.

$25,000 right there!  Plus all of the donations...insanity!  Imagine, if they started charging for each download!
Title: Re: New SQL Server
Post by: JBlaze on April 17, 2009, 02:28:27 AM
Lol, change it, to something like what this site has...XD

Wow, these guys can afford this software....just look at all the members in the charter area....about 500 ... now times that by 50.

$25,000 right there!  Plus all of the donations...insanity!  Imagine, if they started charging for each download!

Luckily, the only thing about SMF we will ever have to pay for is the CM :) And that's optional :P
Title: Re: New SQL Server
Post by: Yigal on April 17, 2009, 02:31:05 AM
Good! Imagine all these members having to pay each 50 bucks.

They'd make millions...but then again...who would pay 50 buck just to sign up ^^.
Title: Re: New SQL Server
Post by: Dragooon on April 17, 2009, 02:32:29 AM
Lol, change it, to something like what this site has...XD

Wow, these guys can afford this software....just look at all the members in the charter area....about 500 ... now times that by 50.

$25,000 right there!  Plus all of the donations...insanity!  Imagine, if they started charging for each download!
For a site like this, that money would literally flow down the drain.
Title: Re: New SQL Server
Post by: JBlaze on April 17, 2009, 02:33:02 AM
who would pay 50 buck just to sign up ^^.

I would.
Title: Re: New SQL Server
Post by: Yigal on April 17, 2009, 02:33:50 AM
Yea just to sign up?  Or get support and stuff?

OK....we are 2 very big spammers...we gotta stop this XD.
Title: Re: New SQL Server
Post by: metallica48423 on April 17, 2009, 02:37:20 AM
The community database here is nearing 3GB

We own all of our own hardware (aside from the mirror servers, which were donated) and colocate it in a datacenter in Atlanta, Georgia.

If i remember right, we currently have two webservers, one devel/email server, and two database servers (one master, one slave for replication/backup (this server was the one we used to replace the db server with the fried power supply)).  I believe the old db2 server will be made a webserver, but dschwab9 can probably clarify on that :P

Needless to say, dschwab9's been working hard on getting us set up for redundancy.  Particularly with how the loss of our db server occured, the fact that we had a spare available saved us hours (and possibly days) of downtime.  Put simply, we'd have been having a harder time than we were ;)

I think that answers all the appropriate questions :P

Lastly, we're not about to start charging for SMF or to register here... so none of that, now :p
Title: Re: New SQL Server
Post by: Dragooon on April 17, 2009, 02:38:28 AM
Great work Dschwab, it must be tiring to maintain a site this big. Keep up the great work.
Title: Re: New SQL Server
Post by: Eliana Tamerin on April 17, 2009, 02:41:43 AM
Lastly, we're not about to start charging for SMF or to register here... so none of that, now :p

WHAT?!

/me puts away her million dollar tickets for registering and grumbles about how Justin's always squashing her money-making schemes... :P
Title: Re: New SQL Server
Post by: Yigal on April 17, 2009, 02:45:10 AM
Imagine one person to maintain this whole site ... I'd go nuts if it were me!

Eliana! Hi! Remember me? You used to always help me out with SMF problem :P
Title: Re: New SQL Server
Post by: JBlaze on April 17, 2009, 02:46:11 AM
Only thing harder than supporting a site?

Supporting a pregnant girlfriend :P
Title: Re: New SQL Server
Post by: Yigal on April 17, 2009, 02:47:36 AM
Awwee :P
Title: Re: New SQL Server
Post by: JBlaze on April 17, 2009, 02:53:01 AM
Awwee :P

You wouldn't be saying that if you were in my shoes
Title: Re: New SQL Server
Post by: Fustrate on April 17, 2009, 03:00:27 AM
Awwee :P

You wouldn't be saying that if you were in my shoes
PM your shoes to me, please :) I'd like to put some words in your mouth!
Title: Re: New SQL Server
Post by: islam2hamy on April 17, 2009, 03:03:25 AM
Very good news!
Title: Re: New SQL Server
Post by: niko on April 17, 2009, 04:55:32 AM
Finally! Site seems to work well now hopefully it keeps that way :)
Title: Re: New SQL Server
Post by: Yigal on April 17, 2009, 04:39:54 PM
Just a few glitches sometimes, the site turns white for 2 seconds.
Title: Re: New SQL Server
Post by: CoreISP on April 18, 2009, 09:01:48 AM
One minor issue - during the migration, I lost connectivity to the old database server, requiring a hard reboot, which in turn corrupted a couple of tables. Due to this issue, there are roughly 4 hours worth of posts from this morning missing. If your post disappeared, I apologize.

If anyone finds any other issues, please report them in this topic and I will look into it.

Ahhh so that's why I was missing something! :P

Thanks! :)
Does this mean the hickups are gone? ;D Yes!! :)
Title: Re: New SQL Server
Post by: Adish - (F.L.A.M.E.R) on April 18, 2009, 09:27:42 AM
One minor issue - during the migration, I lost connectivity to the old database server, requiring a hard reboot, which in turn corrupted a couple of tables. Due to this issue, there are roughly 4 hours worth of posts from this morning missing. If your post disappeared, I apologize.

If anyone finds any other issues, please report them in this topic and I will look into it.

Ahhh so that's why I was missing something! :P

Thanks! :)
Does this mean the hickups are gone? ;D Yes!! :)

I havnt seen hiccups from the day of SQL change.. That was a relief!!
Title: Re: New SQL Server
Post by: Yigal on April 18, 2009, 12:35:11 PM
^^. Let's hope this mySQL server stays with us for a long time!
Title: Re: New SQL Server
Post by: GravuTrad on April 18, 2009, 08:38:20 PM
I have few slow when i click on the answer and edit button to posting...
Title: Re: New SQL Server
Post by: Yigal on April 18, 2009, 08:48:34 PM
It could possibly be your internet connection...for me it work under 0.5 seconds!
Title: Re: New SQL Server
Post by: GravuTrad on April 18, 2009, 08:53:55 PM
hmm nono, fast elsewhere (and with many connexions)...only slow for these two operations...
Title: Re: New SQL Server
Post by: Yigal on April 18, 2009, 08:54:39 PM
Oh, well it's weird ^^, works for  me really fast :)
Title: Re: New SQL Server
Post by: Adish - (F.L.A.M.E.R) on April 18, 2009, 09:05:09 PM
hmm nono, fast elsewhere (and with many connexions)...only slow for these two operations...

Might be the way how your browse handles Ajax..

Try google chrome, but turn off WYSIWYG editor when you use it... It turns total mess in there..
Title: Re: New SQL Server
Post by: GravuTrad on April 18, 2009, 09:08:40 PM
I use most of the time Firefox, and opera too. not the others...
Title: Re: New SQL Server
Post by: Adish - (F.L.A.M.E.R) on April 18, 2009, 09:10:39 PM
I use most of the time Firefox, and opera too. not the others...

Firefox should be fine. I use it aswell. I dont know about Opera. Many people here dont use it i suppose. Maybe its not quite compatible. Shall be resolved soon as you have reported and brought into notice. :)
Title: Re: New SQL Server
Post by: Nibogo on April 18, 2009, 10:30:38 PM
Really nice, great news SMF Team :)
Title: Re: New SQL Server
Post by: GravuTrad on April 19, 2009, 05:43:10 AM
Firefox is not fine for me with these two actions. i suppose too a little problem with ajax but i don't know which one...
Title: Re: New SQL Server
Post by: Aleksi "Lex" Kilpinen on April 19, 2009, 05:48:49 AM
Firefox is not fine for me with these two actions. i suppose too a little problem with ajax but i don't know which one...
Could be related: http://www.simplemachines.org/community/index.php?topic=305379.0
Title: Re: New SQL Server
Post by: GravuTrad on April 19, 2009, 05:59:25 AM
yes surely this:

http://www.simplemachines.org/community/index.php?topic=305379.msg2021551#msg2021551
Title: Re: New SQL Server
Post by: Kenny01 on April 19, 2009, 03:22:09 PM
12GB of RAM
12GB of RAM for how many traffic par day?
Title: Re: New SQL Server
Post by: Tristan Perry on April 19, 2009, 03:31:39 PM
12GB of RAM
12GB of RAM for how many traffic par day?
Lots :P
Title: Re: New SQL Server
Post by: projectgz on April 25, 2009, 06:07:38 AM
good stuff
Title: Re: New SQL Server
Post by: dr.wills on April 25, 2009, 03:43:00 PM
darn, it's fast. I took than fourth a second to make a post  :P
Title: Re: New SQL Server
Post by: Yigal on April 25, 2009, 04:04:31 PM
Is there anything bigger than this kind of software?
Title: Re: New SQL Server
Post by: dschwab9 on April 26, 2009, 01:01:36 AM
Is there anything bigger than this kind of software?

What kind of software are you referring to?
Title: Re: New SQL Server
Post by: Yigal on April 26, 2009, 01:03:44 AM
I mean is this the ultimate MySQL version...or is there something higher with like, 20 GB or RAM?
Title: Re: New SQL Server
Post by: Dragooon on April 26, 2009, 01:07:19 AM
I mean is this the ultimate MySQL version...or is there something higher with like, 20 GB or RAM?
There are servers used to provide VPS hosting with 32 GB of RAM. I'm sure there are more powerful MySQL servers somewhere, Google anyone?
Title: Re: New SQL Server
Post by: Fustrate on April 26, 2009, 01:26:25 AM
Google distributes their service among tons of shipping containers filled with little motherboards and stuff like that, IIRC. Read an article about it a little while ago.

per dragooon's "lol" in irc, here's what I read (and haven't reread for accuracy of my statement): Google uncloaks once-secret server (http://news.cnet.com/8301-1001_3-10209580-92.html)
Title: Re: New SQL Server
Post by: Kenny01 on April 26, 2009, 02:16:43 AM
I mean is this the ultimate MySQL version...or is there something higher with like, 20 GB or RAM?
There are servers used to provide VPS hosting with 32 GB of RAM. I'm sure there are more powerful MySQL servers somewhere, Google anyone?
32 GB of RAM!!!
My God, this must be a site with millions of hit par day.
Title: Re: New SQL Server
Post by: dschwab9 on April 26, 2009, 03:04:46 AM
The new database server here is a HP DL360 G5, which supports up to 32GB of RAM. So, we can actually add a considerable amount of RAM to the current box There are other models available that support 64, 128, and even 256GB.

The 64 bit Linux 2.6 Kernel has no practical limit to the amount of RAM it can support - you are limited by the number of addresses the 64 bit processor can handle, which equates to something really crazy, like 16 Exabytes.

Not sure if MySQL has a limit to how much RAM it can use - I assume it can use whatever the operating system has available.

My God, this must be a site with millions of hit par day.
It takes an amazing amount of server hardware to run a very high traffic site. Here's a quick list of what we have here at Simple Machines:

Primary SQL Server: Dual 2.5GHz Quad Core, 12GB RAM
Slave SQL Server, Search Server: 2.5GHz Quad Core, 8 GB RAM
Primary File Server, Memcache Server: 2.5GHz Quad Core, 8 GB RAM
Secondary File Server: 2.5GHz Quad Core, 4 GB RAM (Not yet in service)
Web Server 1: 2.8GHz Dual Core, 2 GB RAM
Web Server 2: 3.2GHz Dual Core, 2 GB RAM
Mail, Development, Monitoring Server (Running VMware ESX): 2.8GHz Dual Core, 2 GB RAM

The last 3 on the list will all be getting a bump to 4GB RAM next time I visit the datacenter
Title: Re: New SQL Server
Post by: Tristan Perry on April 26, 2009, 05:08:42 AM
Wow, some very powerful stuff dschwab9. Never realised it needed that much in terms of processing and memory power. Out of interest, are you running mostly Xeon processors?
Title: Re: New SQL Server
Post by: dschwab9 on April 26, 2009, 05:24:31 AM
The new SQL server has Xeons, the older SQL and file servers are Core 2 Quads (basically the same as the Xeon without multi-CPU support). The Web server are AMD Athlon 64 X2's.

As far as traffic goes, so far this month we have served 54.4 million page views and 92 million hits from our main datacenter. The main SQL server is averaging 230 queries per second.

The challenge is the growth has been pretty big - doubled in the last month or so. Very hard to keep up with feeding that with more hardware for an organization run by volunteers to produce a free product  :)
Title: Re: New SQL Server
Post by: Tristan Perry on April 26, 2009, 05:45:59 AM
That's crazy stuff :) Good job, for sure. I remember when SMF had just started - running everything on one server would have been fine :P

Loads seem fine at the moment; I hope (sure you will) that you'll be fine to handle the growth; doubling in the last month or so can't be easy!
Title: Re: New SQL Server
Post by: dschwab9 on April 26, 2009, 05:53:57 AM
Sorry, I meant last few months - since november or so. Doubling in one month would indeed be scary :)
Title: Re: New SQL Server
Post by: Kenny01 on April 26, 2009, 06:01:41 AM
Growth is a good sign of smf success to compare to other forums bla bla bla softwares.
Title: Re: New SQL Server
Post by: Tristan Perry on April 26, 2009, 07:16:43 AM
Sorry, I meant last few months - since november or so. Doubling in one month would indeed be scary :)
Ah yes, that would make more sense :) Still, doubling in the last few months is still pretty rapid growth.
Title: Re: New SQL Server
Post by: Kenny01 on April 26, 2009, 08:15:37 AM
As far as traffic goes, so far this month we have served 54.4 million page views and 92 million hits from our main datacenter. The main SQL server is averaging 230 queries per second.
I dream of this kind of traffic in my site.
Title: Re: New SQL Server
Post by: Joshua Dickerson on April 26, 2009, 07:12:26 PM
When you double the amount of memory and the additional file server goes on line, will they all be utilized by memcache?
Title: Re: New SQL Server
Post by: Kenny01 on April 26, 2009, 08:57:27 PM
When you double the amount of memory and the additional file server goes on line, will they all be utilized by memcache?
I think so, because memcache is install in a server, all php files in that server will use it.
Title: Re: New SQL Server
Post by: Joshua Dickerson on April 26, 2009, 09:40:41 PM
Kenny... umm... I am not exactly sure what you are saying but no, not all PHP files use memcache. memcache is not installed by default either.
Title: Re: New SQL Server
Post by: Kenny01 on April 27, 2009, 09:23:25 AM
I never mean it's install by defaut.
Title: Re: New SQL Server
Post by: dschwab9 on April 27, 2009, 11:33:08 AM
Memcache is a distributed cache. You can actually install it on any machine on the network and allocate how ever much RAM you want. For example, you can allocate 500 megs each on 3 different machines and point PHP at all 3 of them, and it will load balance.

We currently one one 3GB memcache instance here, which seems to work good.
Title: Re: New SQL Server
Post by: Kenny01 on April 27, 2009, 12:15:08 PM
You're too good man.
Title: Re: New SQL Server
Post by: Bono_ on April 28, 2009, 06:45:12 AM
The new SQL server has Xeons, the older SQL and file servers are Core 2 Quads (basically the same as the Xeon without multi-CPU support). The Web server are AMD Athlon 64 X2's.

As far as traffic goes, so far this month we have served 54.4 million page views and 92 million hits from our main datacenter. The main SQL server is averaging 230 queries per second.

The challenge is the growth has been pretty big - doubled in the last month or so. Very hard to keep up with feeding that with more hardware for an organization run by volunteers to produce a free product  :)

It seems weird that you needed new db server, you gave all info maybe you could share how big is your db? It seems like huge upgrade if by my calculation mysql would need only 3-4GB of RAM.
5x less page views and running everything on one server, so I'm just trying to compare.
QPS 165

Good job, forum loads really fast.
Title: Re: New SQL Server
Post by: Spiritwarrior on April 28, 2009, 09:47:36 PM
Great work, the forum is fast and haven't had any timeout errors since then.
Title: Re: New SQL Server
Post by: Joshua Dickerson on April 29, 2009, 05:12:37 AM
Bono_ what are your calculations?
Title: Re: New SQL Server
Post by: Bono_ on April 29, 2009, 09:33:13 AM
Bono_ what are your calculations?

For SMF usually size of db is 100k posts = 100MB.
Title: Re: New SQL Server
Post by: Tristan Perry on April 29, 2009, 09:40:02 AM
Bono_ what are your calculations?

For SMF usually size of db is 100k posts = 100MB.
I'd agree with that as a rough guide, give or take 25% from that. However remember that the docs site also (AFAIK) runs off the forum database too, meaning the database size is higher than 'normal'
Title: Re: New SQL Server
Post by: Dragooon on April 29, 2009, 10:20:43 AM
Go to the 3rd page, metallica said that the community DB is nearing 3 GB.
Title: Re: New SQL Server
Post by: Joshua Dickerson on April 29, 2009, 12:41:08 PM
Bono_: your calculations really don't have much to do with how much RAM is needed for the server.
Title: Re: New SQL Server
Post by: Bono_ on April 30, 2009, 05:50:48 AM
Bono_: your calculations really don't have much to do with how much RAM is needed for the server.
I just wondered how big is database, sure you will not buy server that will match your current needs. You will try to get one that can handle raise in traffic in next year or two.

It is nice that you shared specs, usually people hide them like it's top secret.
Title: Re: New SQL Server
Post by: reviewsquad on April 30, 2009, 10:32:11 AM
nice info  ;D
Title: Re: New SQL Server
Post by: Yigal on April 30, 2009, 09:18:08 PM
Question.  Is SMF the most popular forum software at current?
Title: Re: New SQL Server
Post by: Joshua Dickerson on April 30, 2009, 09:19:39 PM
Question.  Is SMF the most popular forum software at current?
No.
Title: Re: New SQL Server
Post by: Yigal on April 30, 2009, 09:20:00 PM
Oh, well then who is :P
Title: Re: New SQL Server
Post by: IchBin™ on May 01, 2009, 12:07:35 AM
These two forums have been around far longer than most. IMO, they are the most popular.
Free - phpBB
Paid - vBulletin
Title: Re: New SQL Server
Post by: Yigal on May 01, 2009, 12:12:19 AM
Out of all those, I think that Simple Machines LLC is much better than phpBB and vBulletin.  I've tried out many other forums, I personally like this one the best.

I hope other feel the same way.
Title: Re: New SQL Server
Post by: Kenny01 on May 01, 2009, 01:15:51 AM
Out of all those, I think that Simple Machines LLC is much better than phpBB and vBulletin.  I've tried out many other forums, I personally like this one the best.

I hope other feel the same way.
You're correct.
Title: Re: New SQL Server
Post by: Aleksi "Lex" Kilpinen on May 01, 2009, 03:48:35 AM
I've had quite a few "arguments" with a friend of mine that's a devoted phpbb fan. Usually things come down to preference in the end, different things done in different ways - and one likes the other better. :) Both are actually quite good forum softwares - but I couldn't think of swtiching over to phpbb myself ;)
Title: Re: New SQL Server
Post by: Kenny01 on May 01, 2009, 03:52:11 AM
Creating topics and posting is much easier with smf than vBulletin.
Title: Re: New SQL Server
Post by: Dragooon on May 01, 2009, 04:05:49 AM
Creating topics and posting is much easier with smf than vBulletin.
I believe its a matter of choice. Some might think the same in vB's favor. I personally prefer SMF.
Title: Re: New SQL Server
Post by: wjhdiz on May 01, 2009, 11:11:07 AM
Do you use MySQL DB or else?

Thanks.
Title: Re: New SQL Server
Post by: Tristan Perry on May 01, 2009, 11:19:45 AM
Do you use MySQL DB or else?

Thanks.
I'm 99.9% certain they use a MySQL server - remember there was a time when SMF only supported MySQL, and it seems unlikely that SMF would have migrated to a new DB system.
Title: Re: New SQL Server
Post by: Yigal on May 01, 2009, 01:33:55 PM
Creating topics and posting is much easier with smf than vBulletin.
I believe its a matter of choice. Some might think the same in vB's favor. I personally prefer SMF.
True.  Also, in phpBB, it's really hard to install modifications.  When here, it's just like that.
Title: Re: New SQL Server
Post by: metallica48423 on May 02, 2009, 02:14:26 PM
Yes, we use a *very* customized LAMP setup.
Title: Re: New SQL Server
Post by: neetusharma on May 04, 2009, 06:10:00 AM
hi gd news
Title: Re: New SQL Server
Post by: ოկtђ on May 30, 2009, 08:21:46 AM
Wow its great.
Title: Re: New SQL Server
Post by: /\takan on June 22, 2009, 04:42:31 AM
Thanks..
Title: Re: New SQL Server
Post by: Eliana Tamerin on June 22, 2009, 12:21:50 PM
Yes, we use a *very* customized LAMP setup.

So customized, that when one of Amy's dogs sing, the server does a fsck. :P
Title: Re: New SQL Server
Post by: Ben K on August 16, 2009, 04:31:11 PM
Can some one upload a picture of that "monstor" ? :D
Title: Re: New SQL Server
Post by: dschwab9 on August 20, 2009, 10:45:05 PM
Can some one upload a picture of that "monstor" ? :D

Not very clear. but here's a couple of pics.
Title: Re: New SQL Server
Post by: Yigal on August 20, 2009, 10:48:24 PM
Wow....do you have people sitting there or somethin' :P
Title: Re: New SQL Server
Post by: dschwab9 on August 20, 2009, 10:52:21 PM
Wow....do you have people sitting there or somethin' :P

No, but I keep updated pics of the rack layout in case I have to walk someone through handling an issue remotely  ::)
Title: Re: New SQL Server
Post by: Aleksi "Lex" Kilpinen on August 21, 2009, 12:58:32 AM
Wow....do you have people sitting there or somethin' :P

No, but I keep updated pics of the rack layout in case I have to walk someone through handling an issue remotely  ::)

That's really a good idea. :)
Title: Re: New SQL Server
Post by: 青山 素子 on August 21, 2009, 11:03:59 AM
No, but I keep updated pics of the rack layout in case I have to walk someone through handling an issue remotely  ::)

That's really a good idea. :)

Great idea indeed, but that wouldn't have helped at the previous data center my company was with.

They banned all cameras from the data center floor. Then again, they were the main peering point for the fiber loop in the region and had companies like Bandai (there was a full-size Power Ranger / Super Sentai doll in their cage), Washington Mutual, and Yahoo! on the floor so it may have been justified for privacy.
Title: Re: New SQL Server
Post by: Amacythe on August 22, 2009, 04:38:32 AM
Wow....do you have people sitting there or somethin' :P

Of course we do... dschwab9 is kept chained in the cage ;)
Title: Re: New SQL Server
Post by: ѕησω on August 22, 2009, 09:01:41 AM
Wow....do you have people sitting there or somethin' :P

Of course we do... dschwab9 is kept chained in the cage ;)

Who comes to feed him now the oldiess is in marketing
Title: Re: New SQL Server
Post by: Yigal on August 23, 2009, 10:09:51 PM
We have an automatic system that feeds it him.  xD
Title: Re: New SQL Server
Post by: Bas on September 01, 2009, 02:31:18 PM
Hey guys, just wanted to give everyone an update on the site status.

We have just completed migrating our SQL databases to a brand new box. The new machine is a 2.5GHz 8 core machine with 12GB of RAM and 6 146GB 10,000 RPM SAS drives in RAID 10. This should dramatically improve the speed of the forums.

The number of cores is unimportant for database servers, sorry but it doesn't do much.
And 10K SAS drives? Do they still make those?
I would use 15K SAS drives and put them in RAID5.
As for the CPU, bad choice, you better have gone 2~4 AMD DUAL/QUAD core in NUMA config, that will give you a memory bandwidth that's incredible.
Cores only help if you have enough memory-speed to support it, databases typical need a lot of memory speed, CPU speed is second to that.
12GB, sorry, useless, as your memory speed is too slow for 8 cores to keep busy at all, I bet most cores are idle.
Title: Re: New SQL Server
Post by: Arantor on September 01, 2009, 06:11:21 PM
Actually, the number of cores does make a difference; MySQL scales differently on different cores. Plus when you have sufficient RAM to cache things in memory (and/or have MEMORY based tables in MySQL), ability to process in parallel is actually kind of important too.

Remember, how this site is configured is not the same as other forums, and likely has a lot pushed into memory-based storage.
Title: Re: New SQL Server
Post by: rsw686 on September 01, 2009, 08:30:33 PM
The number of cores is unimportant for database servers, sorry but it doesn't do much.
And 10K SAS drives? Do they still make those?
I would use 15K SAS drives and put them in RAID5.
As for the CPU, bad choice, you better have gone 2~4 AMD DUAL/QUAD core in NUMA config, that will give you a memory bandwidth that's incredible.
Cores only help if you have enough memory-speed to support it, databases typical need a lot of memory speed, CPU speed is second to that.
12GB, sorry, useless, as your memory speed is too slow for 8 cores to keep busy at all, I bet most cores are idle.

I agree with the 15k drives. The more IOPs you can push the better. However whats the point of 15k drives if you want to run them in RAID 5. RAID 5 has a write overhead that RAID 10 doesn't have. The IOP load of a database server will benefit from RAID 10.

12GB of memory should be more than sufficient for this site. If each post averaged 2KB the database with 2 million posts would be around 4GB plus indexes, which doesn't even come close to 12GB. As long as it is running a 64bit OS mySQL should have plenty of memory to cache with.
Title: Re: New SQL Server
Post by: dschwab9 on September 03, 2009, 08:56:31 PM
Guys, I hate to be the bad guy, but that is incorrect. Processing cores and memory trump everything with the type of workload we have here. MySQL multithreads very will and the load stays distributed fairly evenly. We are currently running 16 MySQL threads and performance is great.

Disk speed - yeah, faster is always better, but the difference in performance between a 10K and a 15K drive is minimal and not worth the added cost in our situation. We have a relatively small dataset - around 10GB with lots of traffic - over 300 queries per second at times. Why would you want to have no memory and fast disks that you thrash constantly? Having lots of RAM and heavily using the query and table caches outperforms fast disks by a long shot. The bottleneck is the GigE connecting everything together, not disk, memory, or CPU speed. Just checked the server and we are currently using about 11GB of RAM.

Also, yes, RAID 5 isn't the best for a database server. However, we are not running RAID 5.  ;D
Title: Re: New SQL Server
Post by: Kill Em All on September 04, 2009, 12:47:49 AM
Genius! Very excellent post dschwab9! Well informative. :)
Title: Re: New SQL Server
Post by: Hell Raiser on September 04, 2009, 06:54:21 AM
11GB RAM

Short it down to 8GB and let me borrow other 3GB's of RAM for my PC :P
Title: Re: New SQL Server
Post by: Bas on September 04, 2009, 12:53:16 PM
I do agree RAID10 is faster if you don't have enough drives.

But I do not agree on the number of cores, it's already proven that many cores without the much needed memory band-weight is useless.
For everything the CPU does it's pulling data from the memory and pushing the result back.
Caching does a bit, but upto a point, it still needs to write it to the memory...and in the end to the harddisk.

I doubt you are even able to get all cores fully loaded at all with MySQL...sure a CPU bench running in cache could, but real life....no way.

This problem isn't new, it was already there at the 486DX100 ages, the CPU was too fast for the memory and hardly did any better then the 486DX66....and in those days the memory speed wasn't that far back on the CPU speed.
8 cores, with terribly limited band-weight? No way...sorry...waste of money on the CPU, 4 core would have worked just the same.
Title: Re: New SQL Server
Post by: Arantor on September 04, 2009, 03:54:22 PM
For many typical apps, I've no doubt you may be correct, Bas, but this thread has seen comment from the admin of this forum, and indeed I know the admin of one of the largest SMF forums worldwide has seen this thread too.

For the load that SMF generates, how it generates it and how it works in terms of MySQL it is simply more CPU intensive than other apps, when you have properly built indexes stored in memory, plus memory tables.


Out of interest, you mention your dedicated server: how big's your forum (we can't see it, the link appears to be broken)?
Title: Re: New SQL Server
Post by: Sabre™ on September 04, 2009, 06:35:47 PM
Out of interest, you mention your dedicated server: how big's your forum (we can't see it, the link appears to be broken)?

Typo in his link, [THIS] (http://forum-en.msi.com/) seems to be it.
Title: Re: New SQL Server
Post by: rsw686 on September 04, 2009, 06:42:27 PM
If http://forum-en.msi.com/ is running on the server in Bas's sig there is something not configured right as the page load time was 6.5 seconds.
Title: Re: New SQL Server
Post by: Arantor on September 04, 2009, 06:42:47 PM
Thanks Sabre. Interesting.

A forum of 890k posts is no lightweight, but the load is slightly different for 890k posts/83k members vs 2m posts / 200k members, especially considering the extra things built into this site that do hook into the forum code, such as the mod site, the support site and so on.

The load on this forum is not the same as it is on a straight forum as a result and more of the operations here are thus more CPU intensive (particularly around the mod site) than I/O intensive.
Title: Re: New SQL Server
Post by: dschwab9 on September 05, 2009, 12:11:48 PM
The number of cores is the primary reason we upgraded. Our old database server was a quad core and our slow downs were due to all the cores being maxed out at peak times. We didn't have any issues with disk IO. The new server with additional cores fixed the problems.
Title: Re: New SQL Server
Post by: Bas on September 06, 2009, 04:44:21 AM
If http://forum-en.msi.com/ is running on the server in Bas's sig there is something not configured right as the page load time was 6.5 seconds.

That's not a normal load time, but it can happen when many people are searching at the same time.
We need the max possible search index and that almost doubles the database size, and when many people use it, it can slow it down a bit sometimes.
If you have a forum without a lot of searching that is a lot easier to configure and way less hardware is needed.
Title: Re: New SQL Server
Post by: Yigal on September 08, 2009, 03:05:08 AM
11GB RAM

Short it down to 8GB and let me borrow other 3GB's of RAM for my PC :P

I have 4 :P

The server has 12GB not 11 :P
Title: Re: New SQL Server
Post by: dschwab9 on September 09, 2009, 05:29:01 AM
That's not a normal load time, but it can happen when many people are searching at the same time.
We need the max possible search index and that almost doubles the database size, and when many people use it, it can slow it down a bit sometimes.
If you have a forum without a lot of searching that is a lot easier to configure and way less hardware is needed.

You should seriously consider using sphinx if you're not already. We run a dedicated Sphinx server for this site and searches happen very fast with no measurable load on the web or database servers.
Title: Re: New SQL Server
Post by: Hell Raiser on September 09, 2009, 05:38:27 AM
11GB RAM

Short it down to 8GB and let me borrow other 3GB's of RAM for my PC :P

I have 4 :P

The server has 12GB not 11 :P

O...oO you have 4GB  :P

Throw some GB's on me  :P
Title: Re: New SQL Server
Post by: Something like that on September 13, 2009, 02:16:54 AM
The number of cores is the primary reason we upgraded. Our old database server was a quad core and our slow downs were due to all the cores being maxed out at peak times. We didn't have any issues with disk IO. The new server with additional cores fixed the problems.

If you're running more than 4 InnoDB threads, you should update to MySQL 5.4 as it fixes the scalability issues. Otherwise, if you are running more than 4, reducing to 4 InnoDB threads will actually improve performance. :)
Title: Re: New SQL Server
Post by: Something like that on September 13, 2009, 02:31:13 AM
As far as traffic goes, so far this month we have served 54.4 million page views and 92 million hits from our main datacenter. The main SQL server is averaging 230 queries per second.

The challenge is the growth has been pretty big - doubled in the last month or so. Very hard to keep up with feeding that with more hardware for an organization run by volunteers to produce a free product  :)

Damn. I do only 4.5 million page views per month, not 60 (though 30 million PHP requests a month). And I only average 90 queries per second. But I do it all in 1 GB of RAM ;)
Title: Re: New SQL Server
Post by: Bas on September 16, 2009, 05:46:52 AM
That's not a normal load time, but it can happen when many people are searching at the same time.
We need the max possible search index and that almost doubles the database size, and when many people use it, it can slow it down a bit sometimes.
If you have a forum without a lot of searching that is a lot easier to configure and way less hardware is needed.

You should seriously consider using sphinx if you're not already. We run a dedicated Sphinx server for this site and searches happen very fast with no measurable load on the web or database servers.

Derek, I have found the reason.
It was Microsofts poor MSNbot 2.0b, it misbehaves and ignores robots.txt so I completly banned that bot via .htcaccess on their IP's.
That bot is hitting our server extremely hard, probably other sites like this one too.
Simply entered this into it:

#Block Inktomi and MSNbot 2.0b
order allow,deny
deny from 74.6
deny from 72.30
deny from 65.55.207
deny from 65.55.106
deny from 65.55.107
allow from all

And hoppa, problem solved ;D
One does not want misbehaving bots...and Inktomi and MSNbot 2.0b are the worst on the planet.

Yesterday MSNbot 2.0b hit our website with about 50000 pageviews! That 1 every 2 seconds.
Even after I set crawldelay to 120 seconds, all others obey my setting, MSNbot 2.0b does not, and I suspect it to do even near 90k a day, as the peaks are very high.
Not even Google does anywhere close to that.
Title: Re: New SQL Server
Post by: Something like that on September 16, 2009, 08:53:37 AM
I'd love to have another 50,000 pageviews. I'd watch my CPU usage go up 3% :D
Title: Re: New SQL Server
Post by: Bas on September 16, 2009, 09:38:59 AM
The load wouldn't go up that much, but it puts useless stress on the database/harddisks as well as useless bandweight.
I rather have it feed pages quickly to users then feeding some troll MSNbot 2.0b. (=Bing that nobody cares about anyway :P)