Server maintenance

LiroyvH · October 24, 2011, 04:38:09 PM

Dear users,

We have planned required maintenance to our server infrastructure.
This maintenance causes us to have downtime several times, both short and a bit longer in time.
(Exact timeframe cannot be predicted)
We will of course do our utmost best to keep the interruptions as short as possible.

The maintenance is scheduled to start at:
Sunday, October 30 2011.
The first maintenance session will most likely be the one causing the longest interruption.
Other dates will be released when they are planned.

For more information, please see all details about this maintenance in the Team Blog article:
http://www.simplemachines.org/community/index.php?topic=457092

To be able to receive quick updates regarding the progress and status, please:
Follow us on Twitter
Add us on Facebook

If you require urgent support when our community is down, please visit our backup forum. Make sure to bookmark it!!
http://www.simplemachinesforum.org/
Please do not open support topics there unless it is really urgent and cant wait a few hours.

Thank you

Regards,
- Servers Team
Simple Machines

Looking · October 24, 2011, 04:47:17 PM

Out of curiosity what kind of hosting setup runs this community?

LiroyvH · October 24, 2011, 04:49:23 PM

I will not go in to full details, but we have several servers that do different jobs such as virtual machine hosts, webservers, SQL servers, file servers, mail servers, etc.
All of them combined is what powers Simple Machines.

We own all hardware.

Looking · October 24, 2011, 05:23:32 PM

Thanks for the reply, that must be quite a setup to maintain with updates and monitoring user activity here.

It also explains the below page stats staying on par:

QuotePage created in 0.077 seconds with 25 queries.
Page served by: 10.0.100.135 (10.0.100.135)

SleePy · October 24, 2011, 05:34:39 PM

It isn't always easy, but both of our primary server admins are very experience professionals in this

Robert. · October 25, 2011, 10:47:09 AM

Thanks for the news, glad to hear simplemachinesforum.org will still be available

Good luck folks

LiroyvH · October 31, 2011, 02:18:46 PM

Quick note:
Welcome back!

The upgrades didnt went as smoothly as they should have gone, as you may have noticed

We will post more details later.

As some of you may have noticed: the MOD site currently replies with "access denied".
We apologise for the inconvenience, but we have to make some changes before opening that up again. This should not take too long.

Hang in there!

Thank you for your patience!

Antes · October 31, 2011, 02:21:48 PM

Thanks for info.

I can access the mod site now.

LiroyvH · October 31, 2011, 02:22:45 PM

You mean customize?
You shouldnt be able to access the mods section

Illori · October 31, 2011, 02:23:38 PM

Quote from: Antes on October 31, 2011, 02:21:48 PM
I can access the mod site now.

http://custom.simplemachines.org/mods/

no you cant

QuoteOops, what happened?
Sorry, but you're not allowed to access the file you requested.

Robert. · October 31, 2011, 02:24:16 PM

Good news and thanks all for your hard work

Antes · October 31, 2011, 02:38:43 PM

edit : now mine gone too thank you Illori

SleePy · October 31, 2011, 02:45:55 PM

It should be fixed now. But it may go off-line intermediately.

Herman's Mixen · October 31, 2011, 02:57:43 PM

* The Burglar! gives CoreISP and Derek a nice break after doing this maint everything is smooth again

good work and keep it up

gisfreak · October 31, 2011, 10:17:57 PM

good work, and congrats

LiroyvH · October 31, 2011, 10:39:21 PM

Thank you

So this has been quite a nightmare for us system administrators.
We will post a more detailed report later, but first of all I would like to thank:

- dschwab9 (Derek) for spending a significant ammount of time both in and out of the datacenter, patching configurations, comming up with workarounds, upgrading hardware, etc.

- SleePy for spending a lot of time looking through configs and helping where he could

- WireSix Technicians, for bearing with us on letting them run to our file server multiple times

They have been a great help to allow us to continue doing remote jobs
- I guess myself (sounds so wrong

) for spending tons of time on transferring files, udating configs, figure out temporary workarounds for our hardware issues, making patches and modifications, installing software and mitigating DDoS attacks, all to get us in a running state again and working with Derek with the datacenter jobs that had to be done.
- Anyone else that may have indirectly helped us

And thank you all for your patience in this matter!!

Happy forum-ing!!

- Liroy

mashby · October 31, 2011, 10:44:05 PM

Thanks to all who got everything back up! Running nicely at the moment.

SleePy · October 31, 2011, 11:00:36 PM

Well maybe I am under estimating myself. But both Derek and Liroy deserve a good thanks for the lack of sleep from this all

. I just annoyed them and kept them thinking

dschwab9 · November 01, 2011, 02:48:29 AM

Just a heads up - the mail server and backend utility server that processes the mail queue are still down, so no emails are going out from the site. We're currently working on recovering those.

Due to a hardware malfunction with the RAID controller in our NFS server, it's taking forever to migrate everything.

Oldiesmann · November 01, 2011, 02:58:17 AM

Thanks for the update Derek. Hopefully you'll have things back up and running like normal again soon.

News:

Server maintenance