Questions about forum outage this morning

[tdennis]

I started getting database error emails around 2AM this morning. When I tried to access my forum around 7:15AM this is what I got :

    Notice: Undefined variable: boarddir in /home/americ82/public_html/forum/index.php on line 47

Notice: Undefined variable: sourcedir in /home/americ82/public_html/forum/index.php on line 52

Warning: require_once(/QueryString.php): failed to open stream: No such file or directory in /home/americ82/public_html/forum/index.php on line 52

Fatal error: require_once(): Failed opening required '/QueryString.php' (include_path='.:/usr/lib/php:/usr/local/lib/php') in /home/americ82/public_html/forum/index.php on line 52

So I reported it to my host since I had not so much as logged into Cpanel in at least 3 months, and had made no changes/updates to the database. My host said it was:

On checking the logs, we could see the following errors are getting logged:-

[22-Sep-2019 12:15:35 UTC] PHP Fatal error:  require_once(): Failed opening required '/QueryString.php' (include_path='.:/usr/lib/php:/usr/local/lib/php') in /home/americ82/public_html/forum/index.php on line 52
[22-Sep-2019 12:15:35 UTC] PHP Notice:  Undefined variable: boarddir in /home/americ82/public_html/forum/index.php on line 47
[22-Sep-2019 12:15:35 UTC] PHP Notice:  Undefined variable: sourcedir in /home/americ82/public_html/forum/index.php on line 52
[22-Sep-2019 12:15:35 UTC] PHP Notice:  Undefined variable: sourcedir in /home/americ82/public_html/forum/index.php on line 52
[22-Sep-2019 12:15:35 UTC] PHP Warning:  require_once(/QueryString.php): failed to open stream: No such file or directory in /home/americ82/public_html/forum/index.php on line 52
[22-Sep-2019 12:15:35 UTC] PHP Fatal error:  require_once(): Failed opening required '/QueryString.php' (include_path='.:/usr/lib/php:/usr/local/lib/php') in /home/americ82/public_html/forum/index.php on line 52

This seems like an issue with the code you are using on the website. Since our expertise in Web Development is limited, we recommend contacting your developer and ask them to have a look at this. If your Web Developer is not available you can seek the help of our in-house Web Developer by opening a support ticket to our Web Development department.

Please note that our Web Development Department is available from Monday to Friday, 10.00 am to 5.00 pm EST/EDT.

Which kind of ticked me off since nothing had been changed on our end, I suspected they did some maintenance during the night and caused the problem. I pushed back and told them I had similiar things like the happen before and they always found something and fixed the problem.

They finally fixed the problem and here is what they said caused the problem:

While checking we could find that the Settings.php file was empty. We have now restored this file from the latest available backup and the site is loading fine. Please refer the screenshot attached.

Please let us know if you have any further queries.

Thank you.

Now if this was something that I should have been been responsible for, should have taken care of I want to do it but right now I have no idea what caused the problem or how I should have been able to find the problem.

Would like to hear from others.
Dennis

[Arantor]

There is a known bug in SMF 2.0 that can cause Settings.php to be wiped out, breaking the entire site.

A possible fix was introduced in 2.1 for it but as a workaround, you can turn off the setting in Admin > Server Settings called 'Send emails on database connection error'.

The issue is that at some point SMF couldn't talk to the database (which can happen on a busy server, even if it's not just you on it), it failed, it tried to send an email, and also track the last time this happened so you don't get floods of emails for a brief problem. Only problem is, if it happens too close together you can end up trying to overwrite the Settings.php (with the time of last error) while another page is trying to do the same thing and end up wiping it out entirely.

I personally turn that setting off all the time.

[lurkalot]

I personally turn that setting off all the time.

I learnt something, wasn't aware of that bug Thanks..   Am I correct in assuming this is already turned off by default?  I ask because all mine appear to be off already.

[Arantor]

Nope, this is on by default, but I've been advocating for years for it to either be off by default or simply removed - in most cases whatever issue causes it to trigger is long since gone by the time you could otherwise get to it, and if it's not, you'll notice it anyway by simply going to your forum...

[Antechinus]

Just went looking for that setting, since this seems like a very handy tip. It's actually on the Admin > Server Settings > Database and Paths page.

Seventh setting from the top. I had to look twice because it doesn't have a Help icon next to it, which makes it a bit inconspicuous.

[tdennis]

There is a known bug in SMF 2.0 that can cause Settings.php to be wiped out, breaking the entire site.

A possible fix was introduced in 2.1 for it but as a workaround, you can turn off the setting in Admin > Server Settings called 'Send emails on database connection error'.

The issue is that at some point SMF couldn't talk to the database (which can happen on a busy server, even if it's not just you on it), it failed, it tried to send an email, and also track the last time this happened so you don't get floods of emails for a brief problem. Only problem is, if it happens too close together you can end up trying to overwrite the Settings.php (with the time of last error) while another page is trying to do the same thing and end up wiping it out entirely.

I personally turn that setting off all the time.

I will turn that setting off.
Thanks for the info.
Dennis

[tdennis]

Nope, this is on by default, but I've been advocating for years for it to either be off by default or simply removed - in most cases whatever issue causes it to trigger is long since gone by the time you could otherwise get to it, and if it's not, you'll notice it anyway by simply going to your forum...

Found it and it's now cut off.
Thanks to all.

Solved

[shadav]

well that's good to know..... goes and turns that off.....