Occasional "There has been a problem with the database!" Email notice

[djr33]

The email notification looks like this:

There has been a problem with the database!

This is a notice email to let you know that SMF could not connect to the database, contact your host if this continues.

The forum works fine! But once in a while I get that email.

I find the message annoying, and I don't know what to do to fix it-- I can't ever replicate the problem loading the forum myself.


I've had an SMF 1.0 forum running for a few years and I periodically get this warning about it-- I added some custom code there to print out the user agent string and I found that it was often bots (the Bing crawler sometimes). I assume that it is the server being overwhelmed by too many requests.

Today I received 6 emails about my newer SMF 2.0 forum with the same message.


I've looked through a few older threads here, but the answers weren't helpful-- they were just suggesting the database info was incorrect, and that's not true. My forum works very well. But sometimes I also get these emails.

Questions:

1. What should I do, if anything?

2. Should I just ignore these emails? Can I turn them off?

3. Is there some kind of flood control that would slow the bots down so that they don't use up all of the available database connections?

4. Could this affect (lower) my search engine ratings?

5. What's the point of this in the SMF code? Is there any debugging info/strategy available?

I don't really understand the message-- if the forum isn't working, then I don't need an email every time someone tries to load the page. That'll be obvious when I can't load the forum myself. But if the forum is working, then these messages aren't helpful. And if it's only not working sometimes, then clearly this isn't a database problem per se, but rather something about usage limits that isn't helped by the vague message. Contacting my host with "sometimes SMF claims there is a problem, but I can't replicate it" won't help either.


I'm using shared GoDaddy hosting, if that matters. I can provide a link to my forum(s), but it won't matter-- they work just fine. I've never been able to replicate a problem myself, although I imagine that if I were to try to load a page the moment that the bot is trying to load the page that I might also have this problem.




Edit:
FYI, the emails I received in the last 24 hours:
9:46pm 9:38am 9:50am 9:54am 10:38am 10:59am

For the other (1.0) forum, the emails don't follow a pattern, sometimes with months in between and sometimes anywhere from one to about a dozen messages in a day.

[margarett]

Your MySQL died momentarily. That's something you need to address with your host.

That's an automated email sent to notify you that your forum is down. You should be more concerned about your forum being down regularly, than about SMF telling you so.

[djr33]

Your MySQL died momentarily. That's something you need to address with your host.

Is that speculation?

Why does this only occur with bots? Isn't it more likely due to too many simultaneous requests than with normal users?

That's an automated email sent to notify you that your forum is down. You should be more concerned about your forum being down regularly, than about SMF telling you so.

I've never seen it down and no one has informed me that it's down. Plus, isn't very rare downtime expected?
To be clear, I've been lucky enough to see some of these emails immediately and I've checked the forum-- it's been up within about 60-120 seconds of the original error. Saying the forum is "down" is an exaggeration, given that I can't even observe it being down. It might indeed be down for 2 seconds at a time once in a while, but I don't know what to do about that.


If I do go to my host, what should I tell them? The email doesn't contain any useful information.

[margarett]

Your MySQL died momentarily. That's something you need to address with your host.

Is that speculation?

No, it isn't.

Why does this only occur with bots? Isn't it more likely due to too many simultaneous requests than with normal users?

It probably doesn't really just occur with bots, more or less randomly, a genuine user will be affected. The "too many" simultaneous is relative... Too many for some people is nothing for other people
How many simultaneous bots/users do you have?

Please note that those issues are not... new in GD...

[Arantor]

Well, every page requires a connection to the server. If the server is busy that page, the connection can be refused (leading to an error)

If the connection drops part way through the page (usually because it's that busy), again you'll get the error.

Sure you can turn the emails off, but all it will do is pretend nothing's wrong - it will still be generating fatal error messages to the user who receives it (including search engines) but it won't email you when it does so. You will think everything is fine but something is wrong under the hood.

It seems that, ultimately, overloaded hosts are more prone to this than not. I've not seen issues like this on my VPS for some time, and the last time it did, was immediately after I'd patched MySQL and it refused to restart properly...

[djr33]

No, it isn't.

How do you know that "MySQL died" rather than hitting a quota on simultaneous connections?

It probably doesn't really just occur with bots, more or less randomly, a genuine user will be affected.

I agree. But as I said, I modified the email code in my SMF1 forum and I haven't once received an email with a non-bot user agent string. This makes me think that it's possibly caused by spiders crawling the forum too rapidly.

The "too many" simultaneous is relative... Too many for some people is nothing for other people

I'm not describing it as a feeling I have-- I'm suggesting it might be hitting some limit set by the host, the number of simultaneous database connections permitted, either a hard limit or perhaps one that arises out of usage.

How many simultaneous bots/users do you have?

On the old forum, which was around for a while, there are currently 13 guests online. It's not an active forum (little active posting for the last year or so), but it's still regularly indexed by bots.
On the new forum, it's usually much lower than that, a handful of guests+users at any time. (That's growing, though. It was just launched this month.)
So it's nothing overwhelming and I'm sure some other forums have many many more users/guests than that.
This adds to the reasons why I suspect it's because of bots making too many requests-- one bot requesting several hundred or several dozen pages at one time trying to crawl the whole site. That's why I asked above if there's some way to slow them down.

Please note that those issues are not... new in GD...

Can you explain?
Is this just something that happens with GoDaddy? There are several reasons I might want/need a new host in the future. If this is just a GoDaddy bug, I suppose I can disable the email message for now and just deal with it. Realistically this is still probably 99% uptime. In terms of actually affecting users, I'd be more concerned about the rare instances where the forum loads slowly, also probably a normal part of using GoDaddy, but nothing to panic about.

Well, every page requires a connection to the server. If the server is busy that page, the connection can be refused (leading to an error)

If the connection drops part way through the page (usually because it's that busy), again you'll get the error.

Right. That's what I expect. I don't know what's causing it to be too busy though, or if there's anything I can do about it.

Sure you can turn the emails off, but all it will do is pretend nothing's wrong - it will still be generating fatal error messages to the user who receives it (including search engines) but it won't email you when it does so. You will think everything is fine but something is wrong under the hood.

I understand. But if there's nothing I can do to fix it, why worry about the warnings? If there is something I can do to fix it, let me know.

It seems that, ultimately, overloaded hosts are more prone to this than not. I've not seen issues like this on my VPS for some time, and the last time it did, was immediately after I'd patched MySQL and it refused to restart properly...

So are you suggesting that this kind of very rare MySQL error is unusual and should be fixable on a shared hosting account? And what do you mean about "overloaded host"? Maybe it has nothing to do with my account at all and some other domain on the same server (or MySQL server) is experiencing heavy traffic?




In the end, I feel like this is a "check engine light" problem-- the car's check engine light is on, no one can figure out exactly what the problem is, and more than that there's not much to do about it. But the light is annoying and keeps suggesting something very important is wrong-- the car is going to break! Many people just reset the light (or have the mechanic do that for them).

[Arantor]

Right. That's what I expect. I don't know what's causing it to be too busy though, or if there's anything I can do about it.

You're on a shared server. That means you have potentially *hundreds* of sites sharing one MySQL server. Each making connections. Each making queries. Each one potentially the source of the server being too busy.

I understand. But if there's nothing I can do to fix it, why worry about the warnings? If there is something I can do to fix it, let me know.

Therein lies the problem.

So are you suggesting that this kind of very rare MySQL error is unusual and should be fixable on a shared hosting account?

Nope, it's not actually that rare at all. We see users reporting it every few weeks. The correlation, though, is the more loaded a server is, the more likely it is to happen. It has never, to my knowledge, happened here for example (on this multi-million post forum), nor on the other forums that we look after that are also multi-million post forums. But they're all on dedicated systems.

Like I said, I've never seen it strike out of the blue on my VPS (and I've been running forums on that thing for nearly 8 years)

And what do you mean about "overloaded host"? Maybe it has nothing to do with my account at all and some other domain on the same server (or MySQL server) is experiencing heavy traffic?

Bingo. We've seen cases of GD servers hosting *thousands* of sites off a single server, though these days it tends to be in the *hundreds* not the thousands bracket.

the car's check engine light is on, no one can figure out exactly what the problem is, and more than that there's not much to do about it. But the light is annoying and keeps suggesting something very important is wrong-- the car is going to break! Many people just reset the light (or have the mechanic do that for them).

Sadly, yes.

[ApplianceJunk]

the car's check engine light is on, no one can figure out exactly what the problem is, and more than that there's not much to do about it. But the light is annoying and keeps suggesting something very important is wrong-- the car is going to break! Many people just reset the light (or have the mechanic do that for them).

I'm using shared GoDaddy hosting, if that matters.

I would be looking for a new mechanic/host.

[djr33]

So, in summary: this warning is actually telling me "Warning: you're using GoDaddy" and I should just ignore it?

I'm not overly concerned about only 99%+ uptime. I'd just like to know if there's something that I should actually do in response to the warnings.

[Illori]

a LOT of people run into issues using godaddy, so in otherwords you may be loosing members or potential members due to this. go find a non-oversold host and you will get better support and your forum will run better... and this error will go away as well.

[djr33]

Interestingly enough, I'm pretty sure this just happened on this website. So I suppose the admins just got an email complaining that the database couldn't be loaded.

From my perspective, it's not big deal. It said "sorry, that didn't work, try again". So I tried again, and it worked. Now I'm continuing to post happily.

I didn't think to take a screen capture at the time.

In the end, I feel like this is a normal part of running a forum, not something we need a panicked-sounding email about. Having a notice in the admin CP that the database didn't load 12 times in a month or something (that is, 12 page loads out of  many, many thousands!) is not too bad.

I'm not expecting SMF (or my server) to be perfect. But it does concern me when there's an error that I can't fix. Maybe this should be considered a performance issue, something to be monitored, but not an error?

[Arantor]

On this website? No. Unless you had a white screen saying almost nothing on it (which I doubt, from your description), that's not what happened.

Question of the day: if there is a database error, where does one save the log of database errors exactly?

[djr33]

Unless you had a white screen saying almost nothing on it (which I doubt, from your description), that's not what happened.

I now feel silly for not having taken a screen capture and not reading the text in detail, but it was definitely a "white screen" with relatively little text.
It said "server busy" with some other details.

It's possible I misread it and this was about the HTTP server rather than the database connection, but I think it said something about "SMF"... then again that might be the default error page because this website is called "SMF", rather than just being about the software running the website. (Have you seen Inception? SMF is a forum within a forum...)

Anyway, it was a white error page saying that the server was busy and to try again. I'd recognize it again but I don't remember the details and I'm not sure how I'd check.

Question of the day: if there is a database error, where does one save the log of database errors exactly?

Haha. That's a great point.
But given that this is obviously an exception to the concept of storing things in a database (for the reason you stated), why not use flat files? SMF obviously already uses flat files when it modifies the settings and updates various pages, so adding in an extra database-tracking.php file wouldn't be terrible. Not pretty, but the situation isn't either.

[Arantor]

Server busy is not the same as database error. The messages are quite distinct.

SMF, with the load balancing settings enabled, will disable various things when under serious load to the point where it will disable the entire forum with a truly minimal error at worst. That's what happened there.

But given that this is obviously an exception to the concept of storing things in a database (for the reason you stated), why not use flat files? SMF obviously already uses flat files when it modifies the settings and updates various pages

There are only a few places where you can actually edit flat files: Settings.php for some truly core settings (passwords, paths), the registration agreement and the language files can be changed.

Virtually every setting is in the database along with everything else.

I should note something for historical context: YaBB, SMF's ancestor, used flat files. There are some very serious performance and scalability issues with doing this, not to mention security concerns. YaBB SE, a splinter project, did some conversion to the database but not complete, SMF goes further - and frankly even more should really be done.

In most server configurations we actually recommend most files, especially Settings.php, be read only - and only made writable when necessary. This doesn't make it easy.

Adding in such a tracking file does not help anything, it's a potential security risk, it's a scalability issue and won't tell you anything particularly useful anyway.

[Kindred]

no. SMF only uses flat files for the settings - because you need the settings in order to connect to the database.

Attempting to store an error into a flat file like you suggest could quickly lead to a RACE condition.


And no...   the server busy message you got here happens when our servers get a little clogged.   Our server team is working on replacing the hardware and getting things better so that doesn't happen.   But it is not related to your database problem, at all.

as for the error email....   Chances are the same condition was present long before you started getting the emails. The newer versions just repaired the problem that stopped it from getting sent.

[djr33]

Server busy is not the same as database error. The messages are quite distinct.

it is not related to your database problem, at all.

Alright.
I've never actually send the database error page, so I wasn't sure what it looked like. (Personally for security reasons I'd be happy to call it just a simple server-is-busy error and tell them to reload. Users don't need more info than that. I assumed that was the case.)
Thanks for clearing that up anyway.

There are only a few places where you can actually edit flat files: Settings.php for some truly core settings (passwords, paths), the registration agreement and the language files can be changed.

Virtually every setting is in the database along with everything else.

You're forgetting about updates and mods.
All I'm saying is that it's already a function in SMF to modify files (so, for example, you can't run SMF without the right permissions on a server and the ability to write files), so I don't see a huge problem with that idea.

I should note something for historical context: YaBB, SMF's ancestor, used flat files. There are some very serious performance and scalability issues with doing this, not to mention security concerns. YaBB SE, a splinter project, did some conversion to the database but not complete, SMF goes further - and frankly even more should really be done.

Sure. I agree. I would never suggest using flat files for something like a whole forum. But just using one to store info about database errors would be fine.
It would be just like the agreement text. In fact, there's a much better reason to store this as a flat file (obvious) than that-- I've always wondered, but it works just fine.

as for the error email....   Chances are the same condition was present long before you started getting the emails. The newer versions just repaired the problem that stopped it from getting sent.

Amusing. The error had an error. And now there's no more error error, but I do get error emails about an error I can't fix. (Long day I guess.)

Attempting to store an error into a flat file like you suggest could quickly lead to a RACE condition.

Without a DB, it's hard to avoid that. While in theory that is a problem, is that a critical issue? If two people experience downtime within milliseconds of each other then you don't really need to know about both instances (although it would hide the spread of the problem and how many users actually saw it).
To turn this around, the email method is almost insane in that sense: if there are really users overlapping by milliseconds with this problem, then your inbox will be flooded with thousands of emails. I'd rather miss a few in the log than have multiple emails in a second.

Clearly that brings up an important point: the scale (frequency) of this problem is relevant. I just find it annoying. It's not to the point where my email inbox is unusable with the flood of new messages. If the DB server died for a day, that might occur. And that would be an awful experience for the admin, haha.

it's a potential security risk,

Why?
Settings.php include the database password. If that's secure and available, then surely relatively harmless information like when a database wasn't available could also be effectively secured-- store it as PHP file comments if you want. The whole point of using PHP is that it's secure. If you don't want to use a .txt file, that's fine, .php is the same thing but can be secure.
Plus, what is the actual harm, aside from potential embarrassment? Are you concerned about someone using it for a database-targetted DDOS attack or something? I suppose but you don't really need logs for that (even if they make the aim more transparent). They could still see the error pages.
All that would be included in the file would be timestamps, each one indicating a problem. There's no more information in the emails! (It wouldn't be a terrible idea either way to add things like the page, the IP, the user agent string, etc.)

it's a scalability issue

Nah. It's a bigger security issue if you get thousands of emails. That doesn't seem to happen (at least not for me-- I might go insane if so), and if it's a problem you can either clear the logs every month (or at X length) or split them into a new file at that point.
You could even send out an email summary from the logs every once in a while, before clearing them.

Adding in such a tracking file does not help anything, ... and won't tell you anything particularly useful anyway.

Ok, that is crucial! This is the core issue.
If a log "does not help anything" and "won't tell you anything particularly useful" then why are the emails any different? That's what I've been trying to say.
Either this is critical, useful, helpful information or it shouldn't be emailed or logged.
I think that a log of this would be relevant, and I'd certainly rather be able to check it on the forum than by email.


Now, sure, if the database server really dies it would be great to be informed by email. But it rarely (if ever) is actually dead when we look at it again, and if it were dead we'd be getting a constant stream of email messages.
If email notification is necessary, then the proper (but complicated) way to do this would be a cron job that looks at the log to see if there's a pattern of repeated downtime over the last five minutes. If so, send a (single) very panicked email. If not, it's just a little hiccup that doesn't need to bug my email inbox-- I can check on it in the admin CP.




Anyway, I can function like this. I'll just delete the emails as they come every few days. In a way it's almost reassuring: I never get hundreds at one time, so that means my database server is never down for more than a few milliseconds

[Arantor]

All I'm saying is that it's already a function in SMF to modify files (so, for example, you can't run SMF without the right permissions on a server and the ability to write files), so I don't see a huge problem with that idea.

No, I'm not. Best practice is to run with everything possible to be read only to BE read only and elevate permissions if needed. Virtually every "I've been hacked" story is the result of leaving an installation insecure by not following this basic practice.

To turn this around, the email method is almost insane in that sense: if there are really users overlapping by milliseconds with this problem, then your inbox will be flooded with thousands of emails. I'd rather miss a few in the log than have multiple emails in a second.

If the server is properly configured it won't be more than one an hour if my memory serves.

Settings.php include the database password. If that's secure and available, then surely relatively harmless information like when a database wasn't available could also be effectively secured-- store it as PHP file comments if you want. The whole point of using PHP is that it's secure. If you don't want to use a .txt file, that's fine, .php is the same thing but can be secure.

Secure to whom exactly? If you have a PHP file writable by PHP itself it has permissions that are too high. Now consider, a PHP file that can be written by PHP on a lot of configurations means it can be written by any PHP process, without proper server hardening. Like, say, someone else's account hacking into your files and then serving up malware to your users. This does happen, though thankfully it happens a lot less now than it used to because server configuration is getting smarter.

If a log "does not help anything" and "won't tell you anything particularly useful" then why are the emails any different? That's what I've been trying to say.

Because there are two sets of circumstances that can cause it and when I suggested the change 3 years ago, it was as a quick and dirty fix to a more pressing problem. Time was when only an actual database connection failure would send you the error. Until we encountered cascade failure: connection dropped rather than failed, which would cause an error that couldn't be logged, which would cause an error that couldn't be logged,etc. etc. In response I suggested the safest, quickest way out of that: the DB fatal error message. At the time I certainly didn't expect it to be included in the core code and a full scale fix to the problem is impractical to carry out in a 2.0.x scale release.

The probable reality is that you server isn't even down but merely that PHP was disconnected for some reason, usually overloading.

[djr33]

If the server is properly configured it won't be more than one an hour if my memory serves.

How?
I'm perplexed by that. The only way to only send one per hour is to know whether one was sent earlier. Knowing implies a log. So... isn't it solved in one way or another then?

Also, that's crucial information missing from the email message: it should read "this message implies that the database may have been down for up to an hour at this time".

However, that doesn't appear to be happening:

FYI, the emails I received in the last 24 hours:
9:46pm 9:38am 9:50am 9:54am 10:38am 10:59am

I don't know exactly what you mean by "properly configured", but I'm on a widely used standard shared hosting account, so that's a major problem if you expect everyone using SMF to have dedicated hosting. I'm not claiming a dedicated host (with expert configuration) is not desirable; I'm just saying that in reality that isn't what a lot of people have available: is SMF only recommended for use in a dedicated hosting environment?

Alternatively, GoDaddy might be doing something extremely weird with their servers. I genuinely don't know. But it's not something I can address, and I'd need to have detailed debugging info to try to approach this question with them. I'm willing to try that, but it's hard to get very far with tech support. I'm not a server expert, but I often find myself wanting to ask "can you please put someone on the phone who actually knows more than I do?" Then I'd need some evidence to back it up.

Secure to whom exactly? If you have a PHP file writable by PHP itself it has permissions that are too high. Now consider, a PHP file that can be written by PHP on a lot of configurations means it can be written by any PHP process, without proper server hardening. Like, say, someone else's account hacking into your files and then serving up malware to your users. This does happen, though thankfully it happens a lot less now than it used to because server configuration is getting smarter.

SMF changes Settings.php and any other files on update. SMF runs on PHP. I'm completely lost at your comment here.
I could imagine that the installer presents a security concern and that is why it is deleted and then the other files are given lower permissions. Fine.
But the automatic updating process, also used for mods, actually modifies the files as well. Clearly PHP is able to (even if it's indirectly) modify the PHP files on the server.

Are you suggesting that automatic installation of updates and mods is a bad idea?

Because there are two sets of circumstances that can cause it and when I suggested the change 3 years ago, it was as a quick and dirty fix to a more pressing problem. Time was when only an actual database connection failure would send you the error. Until we encountered cascade failure: connection dropped rather than failed, which would cause an error that couldn't be logged, which would cause an error that couldn't be logged,etc. etc. In response I suggested the safest, quickest way out of that: the DB fatal error message. At the time I certainly didn't expect it to be included in the core code and a full scale fix to the problem is impractical to carry out in a 2.0.x scale release.

I understand the practical considerations. This is complicated and would take a lot of time and alterations to fix. That's fine.
The problem is that at a minimum the current message, sent only to the forum administrators, is confusing and missing helpful information. For example, it could simply read "Your database didn't load. There are a number of possible reasons for this that cannot be diagnosed by SMF at this time. Please keep an eye on your forum's performance and contact your host if you frequently receive this message."

The probable reality is that you server isn't even down but merely that PHP was disconnected for some reason, usually overloading.

Do you mean PHP or MySQL?
Either way, I agree. If the server is still around the 99% uptime range, then I don't think it is a huge problem.
It is, obviously, a concern if it happens especially often.

[Arantor]

I'm perplexed by that. The only way to only send one per hour is to know whether one was sent earlier. Knowing implies a log. So... isn't it solved in one way or another then?

No, because the way it's done can cause a forum to become damaged in 2.0 (and fixing it in 2.0 is difficult because of the botched way it's always been done and I haven't figured out a good patch, in 2.0 it's stored directly in Settings.php which can be the subject of a race condition), and in 2.1 it's better but still has all of the same concerns as above, because in 2.1 it is stored as the last time in a flat PHP file.

I'm not claiming a dedicated host (with expert configuration) is not desirable; I'm just saying that in reality that isn't what a lot of people have available: is SMF only recommended for use in a dedicated hosting environment?

No, on a dedicated hosting environment this problem never happens in the first place. On a properly configured shared hosting environment it wouldn't be sufficiently overloaded that MySQL has to drop connections to manage the load.

GoDaddy and others are in a race to the bottom, with hundreds or even thousands of sites on a single server. Hosts, even shared hosts, that are not so overloaded do not exhibit the same symptoms. Funny that.

SMF changes Settings.php and any other files on update. SMF runs on PHP. I'm completely lost at your comment here.

SMF only changes Settings.php if it needs to (which in a general case should be almost never -- except in this logging situation). I run with it read only 99% of the time along with the rest of all of my installations and only have the things writable that need to be.

I could imagine that the installer presents a security concern and that is why it is deleted and then the other files are given lower permissions. Fine.
But the automatic updating process, also used for mods, actually modifies the files as well. Clearly PHP is able to (even if it's indirectly) modify the PHP files on the server.

Only if you let it. On my server I can't install packages out of the box, because I have to explicitly up the permissions before installing things, before putting the permissions back to normal afterwards. This is completely intentional but helps ensure the security of my setup.

Leaving it writable all the time IS a security risk, plain and simple.

Let me put it this way: I used to develop for a rival forum software that was built on top of SMF (legitimately). I completely removed the package manager and built one from scratch to avoid all the security issues that are presented by SMF's. It's not feasible for me to do this for SMF 2.1 but it is part of the plan for 3.0 to move back in that direction.

Are you suggesting that automatic installation of updates and mods is a bad idea?

Only if you don't properly secure your site. Like most users don't.

The problem is that at a minimum the current message, sent only to the forum administrators, is confusing and missing helpful information. For example, it could simply read "Your database didn't load. There are a number of possible reasons for this that cannot be diagnosed by SMF at this time. Please keep an eye on your forum's performance and contact your host if you frequently receive this message."

Except that means a substantial code change because there is only one code path. Both sets of circumstances go to one place - we'd have to create an entire new branch of code for that, to differentiate the two. This is not a simple job and is infeasible to include in 2.0, not great for 2.1. But as I said, it was written as a two line patch to prevent hundreds of thousands of errors happening in the space of a few seconds, to the point where entire servers were being taken down. I wish I'd never investigated now, all the hassle it's caused.

Do you mean PHP or MySQL?

A script written in the PHP language connects to the MySQL server. All other sites on the same server are going to be doing the same. Something causes that connection from SMF(PHP) to MySQL to fail, causing this error.
Either way, I agree. If the server is still around the 99% uptime range, then I don't think it is a huge problem.
It is, obviously, a concern if it happens especially often.[/quote]

Of course it is, but everyone just goes ERMAHGERD and turns it off because they just assume that SMF is being stupid when it's doing the safest thing it can actually do in the circumstances. I find it funny that the recommendation from some hosts was to actually reconnect on connection failure (which means *increasing* the connection contention rather than reducing it, thus making the problem worse)

[djr33]

Complicated. Thanks for taking the time to discuss it.

Why not change the email message? No technical changes whatsoever, just a change in the explanation (perhaps a link to [a cleaned up version of] this or another thread, even).

Simply explaining that shared hosting can cause this is helpful information. "Upgrade to a dedicated host if you want. If not you may see this sometimes."

No, because the way it's done can cause a forum to become damaged in 2.0 (and fixing it in 2.0 is difficult because of the botched way it's always been done and I haven't figured out a good patch, in 2.0 it's stored directly in Settings.php which can be the subject of a race condition), and in 2.1 it's better but still has all of the same concerns as above, because in 2.1 it is stored as the last time in a flat PHP file.

It's stored in Settings.php?
Then Settings.php is writable by PHP?

Maybe I'm mixing up what you think is optimal and what is actually in practice.

is SMF only recommended for use in a dedicated hosting environment?

No, ...

GoDaddy and others are in a race to the bottom, with hundreds or even thousands of sites on a single server. Hosts, even shared hosts, that are not so overloaded do not exhibit the same symptoms. Funny that.

I'm confused. So SMF won't run well on GoDaddy and other major shared hosts but SMF is still intended to be used with them?
I don't disagree with you at all and I will upgrade if it becomes a practical concern (less than, say 99% uptime, for a start). There are reason I'd want to do this anyway (projects under development) but I'm not at that point yet, so the price for the moment is workable. I've used GoDaddy for a while for relatively light weight websites and I'm happy with it most of the time. Bizarre bugs do some up though (I'm having some very odd DNS issues on another site for example, which is a completely unrelated topic).

SMF only changes Settings.php if it needs to (which in a general case should be almost never -- except in this logging situation). I run with it read only 99% of the time along with the rest of all of my installations and only have the things writable that need to be.

PHP changes Settings.php. It can do it. There may be an issue of a race condition, but I can't see a greater security problem given that it is possible to alter it with PHP.
Hackers don't take advantage of the frequency with which you do operations but of the fact that you can do operations. Settings.php is either secure or it is not.

Only if you let it. On my server I can't install packages out of the box, because I have to explicitly up the permissions before installing things, before putting the permissions back to normal afterwards. This is completely intentional but helps ensure the security of my setup.

Sure. That's safer.

Leaving it writable all the time IS a security risk, plain and simple.

Let me put it this way: I used to develop for a rival forum software that was built on top of SMF (legitimately). I completely removed the package manager and built one from scratch to avoid all the security issues that are presented by SMF's. It's not feasible for me to do this for SMF 2.1 but it is part of the plan for 3.0 to move back in that direction.

Ok, so actually then SMF has (potential) security issues, with the default settings. (Software isn't perfect.)
Therefore, under the default configuration, there would be no particular problem in itself of having a flat file log for this. Your objection is that it's one step forward and two steps back. Fair enough. (For SMF 2.0 or maybe 2.1 then, it might still be a good idea, not that it's clear what to do later.)

Only if you don't properly secure your site. Like most users don't.

So you would suggest securing your server, then temporarily unsecuring it, then running the PHP-based update, then resecuring it?
Yes, that would work. I didn't follow that before. You'd actually need to manually change server settings (outside of PHP/SMF) in order to make changes. That's fine and secure. But I don't know that it's the ideal for widespread software like this given that many users don't do that (and may not want to). It's a major tradeoff-- you certainly have a point.

Except that means a substantial code change because there is only one code path. Both sets of circumstances go to one place - we'd have to create an entire new branch of code for that, to differentiate the two. This is not a simple job and is infeasible to include in 2.0, not great for 2.1. But as I said, it was written as a two line patch to prevent hundreds of thousands of errors happening in the space of a few seconds, to the point where entire servers were being taken down. I wish I'd never investigated now, all the hassle it's caused.

Hm, I see.
Would it be a particularly bad idea to change "both" messages, then? "Either X or Y happened. SMF can't diagnose which at this time."

Of course it is, but everyone just goes ERMAHGERD and turns it off because they just assume that SMF is being stupid when it's doing the safest thing it can actually do in the circumstances.

Right. It's effectively useless, at least for this situation. It's a bad situation without a clear fix, I suppose, at least without major rewrites. The only thing I can see is altering the message (in all cases) to explain what the possibilities are. I'd be satisfied with that, at least as a patch.

I find it funny that the recommendation from some hosts was to actually reconnect on connection failure (which means *increasing* the connection contention rather than reducing it, thus making the problem worse)

Indeed.
Have you thought about a delay?
Assuming the server isn't overwhelmed all the time and just sometimes there are peaks in activity, then maybe a 5 second delay ("The server is busy. Retrying in 5 seconds...") would work.
Of course this is far from the top of the list of things to be concerned about because it's a simple as the user refreshing the page. Almost every user on the internet gets that: something's broken; refresh.