Server setup and new server!

LiroyvH · April 04, 2011, 06:10:25 PM

So, after a lot of hickups and errors and all that today, we are proud to announce our new server setup!

It is currently in beta, so dont be surprised if there can be a error from time to time. (Alhtough that g*dd**n high stress error should finally be gone now

) Tests are still being run and only a small part is completely moved to the new setup, this is for the sake of testing and stability.

The new setup uses nginx.

We have also added a new webserver to our pool, .136. The third webserver powering SimpleMachines

You will see it at the bottom of the page from time to time. Nginx has been configured to balance the load over all 3 webservers now.
So, a long and warm welcome to our third webserver!! May it process thy requests without failure and in a speedy manner.

The third webserver is powered by CentOS, running in it's own environment on one of our vMware systems.

Me and SleePy have been working on testing, installing, bug hunting, firewalls going berserk, vmware changes, setup changes, error recovery and OMG MAJOR PROBLEM!!1!! recovery for a few hours to ensure everything runs as we want it. (You may have noticed a couple of downtimes and errors during this timeframe

)
However: If you notice any problems or issues, please do let us know.

Most and *a lot* of kudo's to SleePy!!! He has built the full initial setup for nginx and put a lot of his time in to testing it in his own environment and testing configurations compatible with our server setup and worked very hard to get it up on our systems! So a very big thanks, congratulations and compliments to SleePy!
SleePy: Thank you!

Happy foruming guys! We hope you enjoy the new setup as much as our servers currently seem to love it

The load has reduced a lot.

In case you are wondering: No. The DDoS attacks have not stopped and there are still measures countering those attacks constantly as we speak. This new setup just helps it a lot and provides significant performance improvements.

Enjoy!!

~ CoreISP & SleePy

SleePy · April 04, 2011, 06:15:21 PM

* SleePy wants a vacation now...

Adish - (F.L.A.M.E.R) · April 04, 2011, 06:28:23 PM

Thank you SleePy and CoreISP. I hope that you have a peaceful sleep soon enough.

For everyone else,
The server is like a top secret thingy. If you come anywhere near 10 miles radius of this server, you may get shot in places you may not like!

redone · April 04, 2011, 06:55:28 PM

Nice work guys. Nothing ever goes to plan when your working on a project no matter how big or small.

~RedOne

Joseph · April 04, 2011, 06:59:36 PM

Well done, guys. Good to hear things are back up and running.

wiecher · April 04, 2011, 07:05:37 PM

Nice work

ziycon · April 04, 2011, 07:07:51 PM

Best of look with the new server, load balancing is the gift of gods!

busterone · April 04, 2011, 07:45:35 PM

Nice work to both of you.

bloc · April 07, 2011, 12:57:01 PM

Very nicely done, I think I notice a increase in response time now. And as a newfound user of Nginx myself, good call using it

Joker™ · April 07, 2011, 01:03:13 PM

Yup, site is running very smooth on my side from past couple of days. Great works guys

.

Robert. · April 08, 2011, 03:59:38 AM

Great work both

kat · April 08, 2011, 06:53:11 AM

I thought the place seemed a bit more zippy, today!

Nothing could possibly go wro

LiroyvH · April 08, 2011, 09:29:13 AM

Thank you all.

On a good note, it seems to be holding very well. The nginx setup plus load balancing plus security seems to be very effective. We havent really had major load issues. Just on one of the servers the load was slightly raised but due to some circumstances, this was to be expected and will not affect general performance.

Dzonny · April 10, 2011, 08:21:17 AM

Works fine for now, great work guys

Hj Ahmad Rasyid Hj Ismail · April 10, 2011, 09:04:53 AM

Feel good. Congratulation. Good work guys!

MotherBoard · April 11, 2011, 12:21:00 PM

Nice work, anything to continue our progression

NanoSector · April 14, 2011, 03:36:44 PM

Very nice job, both

Illori · April 16, 2011, 08:03:13 AM

I just got a high stress warning on 10.0.100.134 (10.0.100.135)

LiroyvH · April 16, 2011, 08:43:44 AM

That's possible due to .135 getting multiple extra requests at this time.
Will be very sporadic, but possible while this setup is still being tested.

Sleezy · April 16, 2011, 11:10:57 AM

Good job guys!

Arantor · April 25, 2011, 07:22:22 AM

I've had a high stress (white screen level) warning today and had several yesterday evening.

LiroyvH · April 25, 2011, 07:28:55 AM

Hi,

yes we are aware of that. The servers had highly raised ddos activity yesterday which caused many problems.
Things should have stabilised by now though

Arantor · April 25, 2011, 07:31:05 AM

Um, that high stress warning was minutes before I posted, which is why I posted...

TechTitan · April 25, 2011, 08:52:35 AM

Yeah, i get that too, plus the forum page is very slow to load..

LiroyvH · April 25, 2011, 11:54:08 AM

That's odd, but the high stress warnings are indeed possible at the moment as we are working with reduced server power at this moment.

Working on the systems as we speak.

redone · April 26, 2011, 05:08:28 PM

Anybody would think there is only one website in the world to attack or something! Good luck in your fight.

~RedOne

Nidjo123 · April 26, 2011, 07:45:42 PM

DDoS is piece of cake for SMF server team.

live627 · April 26, 2011, 09:13:32 PM

Quote from: Nidjo123 on April 26, 2011, 07:45:42 PM
DDoS is piece of cake for SMF server team.

Hardly. Managing a server is tough in and of itself. Many have tried and failed. Very few succeed.

Nidjo123 · April 27, 2011, 03:59:20 PM

But they're doing awesome job in that case.

Dzonny · June 13, 2011, 06:15:28 AM

Stress today after few months of great work

LiroyvH · June 13, 2011, 07:02:12 AM

I havent seen anything disturbing, but with the large traffic for 2.0, I can imagine it slips from time to time.

Dzonny · June 13, 2011, 07:06:43 AM

Yes i guess so, just wanted to point you out if that can help you.
Everything works fine now.

LiroyvH · June 13, 2011, 07:13:17 AM

Thanks

Unfortunately there is not too much to do about it now, it's only stressing on very few occasions which at the moment is quite logical and imho isnt worth the effort to isolate. Tweak more, affect performance more.

As long as it is running stable 99,8% of the time it's fine imo in the current traffic situation.

live627 · June 14, 2011, 05:03:57 AM

You talkin to a ghost? Or they removed their posts?

LiroyvH · June 14, 2011, 08:08:52 AM

Quote from: live627 on June 14, 2011, 05:03:57 AM
You talkin to a ghost? Or they removed their posts?

I'm talking to Dzonny, maybe you put him on ignore or something?

Or the other way around, of course... In which case you wouldnt read this post anyway :')

live627 · June 15, 2011, 12:31:32 AM

Nope. My ignore list has two items inn it, neither of whom are Dzonny.

LiroyvH · June 15, 2011, 06:33:58 AM

Odd, and you cant see his posts above mine? That's really weird :p

Aleksi "Lex" Kilpinen · June 15, 2011, 06:36:39 AM

You two just look like two peas in a pod - It's hard to tell you apart

Dzonny · June 15, 2011, 12:21:31 PM

Quote from: Aleksi "Lex" Kilpinen on June 15, 2011, 06:36:39 AM
You two just look like two peas in a pod - It's hard to tell you apart

haha, yeah, blame on the Reaper

青山素子 · June 15, 2011, 05:03:26 PM

Quote from: Dzonny on June 15, 2011, 12:21:31 PM
haha, yeah, blame on the Reaper

Maybe, but just remember: don't fear the reaper.

Farmacija · June 19, 2011, 11:44:38 AM

Again problems with server?!
I can access only to community, not for http://custom.simplemachines.org/ or http://wiki.simplemachines.org/smf/Main_Page since yesterday.
It is stressed again?

LiroyvH · June 19, 2011, 12:08:01 PM

That was a DNS error, it should be resolved now. Works for me.

We are however experiencing some technical difficulties with our setup at the moment. I am looking in to the unstable behaviour as we speak.

kat · June 19, 2011, 01:26:12 PM

Good on ya, Guvnor!

Farmacija · June 19, 2011, 01:31:12 PM

Its working for now.

Burridge · June 19, 2011, 01:37:24 PM

Ahh, wasn't just me who couldn't access the site today then O_o. Glad to know it's fixed/getting fixed.

LiroyvH · June 19, 2011, 01:42:12 PM

Quote from: Burridge on June 19, 2011, 01:37:24 PM
Ahh, wasn't just me who couldn't access the site today then O_o. Glad to know it's fixed/getting fixed.

That was unfortunately due to our file server going down. This has been fixed now

Burridge · June 19, 2011, 01:45:17 PM

Quote from: CoreISP on June 19, 2011, 01:42:12 PM
Quote from: Burridge on June 19, 2011, 01:37:24 PM
Ahh, wasn't just me who couldn't access the site today then O_o. Glad to know it's fixed/getting fixed.

That was unfortunately due to our file server going down. This has been fixed now

I need to look at my twitter more often tbh

Or get SMF tweets emailed to me x)

LiroyvH · June 19, 2011, 01:53:44 PM

Hehe, yeah, I did announce it on Twitter

Usually, the latest news if something is messed up serverside can be found there

Though I was glad it had been silent for a few months, heh.

rd · June 19, 2011, 03:42:58 PM

Thanks guys! I'm happy SMF is backup and nginx should provide a good and stable environment

Alex' Manson · June 20, 2011, 12:03:31 AM

Good Job,
And.. Welcome To The New Server! LoL...
And just want to comment on something, well.. actually it's a message to the one's DDOS'ing the servers, G.E.T A L.I.F.E A.N.D G.T.F.O

News:

Server setup and new server!

wiecher

bloc

kat

kat