News:

Want to get involved in developing SMF, then why not lend a hand on our github!

Main Menu

New SQL Server

Started by dschwab9, April 16, 2009, 03:15:08 PM

Previous topic - Next topic

dschwab9

Quote from: Yigal on April 25, 2009, 04:04:31 PM
Is there anything bigger than this kind of software?

What kind of software are you referring to?

Yigal

I mean is this the ultimate MySQL version...or is there something higher with like, 20 GB or RAM?
Yigal V.
Contact me by PM


do yourself a favour and not annoy support helpers by spamming or messaging for support
1.x Support | 2.x Support | 5Mods

Dragooon

Quote from: Yigal on April 26, 2009, 01:03:44 AM
I mean is this the ultimate MySQL version...or is there something higher with like, 20 GB or RAM?
There are servers used to provide VPS hosting with 32 GB of RAM. I'm sure there are more powerful MySQL servers somewhere, Google anyone?

Fustrate

#83
Google distributes their service among tons of shipping containers filled with little motherboards and stuff like that, IIRC. Read an article about it a little while ago.

per dragooon's "lol" in irc, here's what I read (and haven't reread for accuracy of my statement): Google uncloaks once-secret server
Steven Hoffman
Former Team Member, 2009-2012

Kenny01

Quote from: Dragooon on April 26, 2009, 01:07:19 AM
Quote from: Yigal on April 26, 2009, 01:03:44 AM
I mean is this the ultimate MySQL version...or is there something higher with like, 20 GB or RAM?
There are servers used to provide VPS hosting with 32 GB of RAM. I'm sure there are more powerful MySQL servers somewhere, Google anyone?
32 GB of RAM!!!
My God, this must be a site with millions of hit par day.

dschwab9

The new database server here is a HP DL360 G5, which supports up to 32GB of RAM. So, we can actually add a considerable amount of RAM to the current box There are other models available that support 64, 128, and even 256GB.

The 64 bit Linux 2.6 Kernel has no practical limit to the amount of RAM it can support - you are limited by the number of addresses the 64 bit processor can handle, which equates to something really crazy, like 16 Exabytes.

Not sure if MySQL has a limit to how much RAM it can use - I assume it can use whatever the operating system has available.

Quote from: Kenny01 on April 26, 2009, 02:16:43 AM
My God, this must be a site with millions of hit par day.
It takes an amazing amount of server hardware to run a very high traffic site. Here's a quick list of what we have here at Simple Machines:

Primary SQL Server: Dual 2.5GHz Quad Core, 12GB RAM
Slave SQL Server, Search Server: 2.5GHz Quad Core, 8 GB RAM
Primary File Server, Memcache Server: 2.5GHz Quad Core, 8 GB RAM
Secondary File Server: 2.5GHz Quad Core, 4 GB RAM (Not yet in service)
Web Server 1: 2.8GHz Dual Core, 2 GB RAM
Web Server 2: 3.2GHz Dual Core, 2 GB RAM
Mail, Development, Monitoring Server (Running VMware ESX): 2.8GHz Dual Core, 2 GB RAM

The last 3 on the list will all be getting a bump to 4GB RAM next time I visit the datacenter

Tristan Perry

Wow, some very powerful stuff dschwab9. Never realised it needed that much in terms of processing and memory power. Out of interest, are you running mostly Xeon processors?

dschwab9

#87
The new SQL server has Xeons, the older SQL and file servers are Core 2 Quads (basically the same as the Xeon without multi-CPU support). The Web server are AMD Athlon 64 X2's.

As far as traffic goes, so far this month we have served 54.4 million page views and 92 million hits from our main datacenter. The main SQL server is averaging 230 queries per second.

The challenge is the growth has been pretty big - doubled in the last month or so. Very hard to keep up with feeding that with more hardware for an organization run by volunteers to produce a free product  :)

Tristan Perry

That's crazy stuff :) Good job, for sure. I remember when SMF had just started - running everything on one server would have been fine :P

Loads seem fine at the moment; I hope (sure you will) that you'll be fine to handle the growth; doubling in the last month or so can't be easy!

dschwab9

Sorry, I meant last few months - since november or so. Doubling in one month would indeed be scary :)

Kenny01

Growth is a good sign of smf success to compare to other forums bla bla bla softwares.

Tristan Perry

Quote from: dschwab9 on April 26, 2009, 05:53:57 AM
Sorry, I meant last few months - since november or so. Doubling in one month would indeed be scary :)
Ah yes, that would make more sense :) Still, doubling in the last few months is still pretty rapid growth.

Kenny01

Quote from: dschwab9 on April 26, 2009, 05:24:31 AM
As far as traffic goes, so far this month we have served 54.4 million page views and 92 million hits from our main datacenter. The main SQL server is averaging 230 queries per second.
I dream of this kind of traffic in my site.

Joshua Dickerson

When you double the amount of memory and the additional file server goes on line, will they all be utilized by memcache?
Come work with me at Promenade Group



Need help? See the wiki. Want to help SMF? See the wiki!

Did you know you can help develop SMF? See us on Github.

How have you bettered the world today?

Kenny01

Quote from: groundup on April 26, 2009, 07:12:26 PM
When you double the amount of memory and the additional file server goes on line, will they all be utilized by memcache?
I think so, because memcache is install in a server, all php files in that server will use it.

Joshua Dickerson

Kenny... umm... I am not exactly sure what you are saying but no, not all PHP files use memcache. memcache is not installed by default either.
Come work with me at Promenade Group



Need help? See the wiki. Want to help SMF? See the wiki!

Did you know you can help develop SMF? See us on Github.

How have you bettered the world today?

Kenny01

I never mean it's install by defaut.

dschwab9

Memcache is a distributed cache. You can actually install it on any machine on the network and allocate how ever much RAM you want. For example, you can allocate 500 megs each on 3 different machines and point PHP at all 3 of them, and it will load balance.

We currently one one 3GB memcache instance here, which seems to work good.

Kenny01


Bono_

Quote from: dschwab9 on April 26, 2009, 05:24:31 AM
The new SQL server has Xeons, the older SQL and file servers are Core 2 Quads (basically the same as the Xeon without multi-CPU support). The Web server are AMD Athlon 64 X2's.

As far as traffic goes, so far this month we have served 54.4 million page views and 92 million hits from our main datacenter. The main SQL server is averaging 230 queries per second.

The challenge is the growth has been pretty big - doubled in the last month or so. Very hard to keep up with feeding that with more hardware for an organization run by volunteers to produce a free product  :)

It seems weird that you needed new db server, you gave all info maybe you could share how big is your db? It seems like huge upgrade if by my calculation mysql would need only 3-4GB of RAM.
5x less page views and running everything on one server, so I'm just trying to compare.
QPS 165

Good job, forum loads really fast.

Advertisement: