Simple Machines Community Forum

Archived Boards and Threads... => Archived Boards => SMF Feedback and Discussion => Topic started by: kat on April 23, 2012, 02:37:18 PM

Title: SMF admin panel backup is crap! Can we have a patch release to remove it please?
Post by: kat on April 23, 2012, 02:37:18 PM
Our poor users are thinking that they're safe, because, like the good admins that they are (Some, anyway), they've backed-up, not knowing that the SMF backup "feature" is a load of old pony.

This has caused serious problems, for some. Indeed, it's still doing so.

It's been reported and whinged about for bloody yonks, too.

I've mentioned it, in the wiki. But, of course, not everyone knows about that.

So, can we finally just dump it, please?

Patch both the v1.x and v2 branches?

Like... NOW? Or, even better, yesterday?

This has been a public service announcement from the Pissed-Off Party. (http://www.katzy.dsl.pipex.com/Smileys/c018.gif)(http://www.katzy.dsl.pipex.com/Smileys/c017.gif)

(http://www.katzy.dsl.pipex.com/Smileys/c016.gif)

Edited to remove Freudian slip of the keyboard. :P
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Robert. on April 23, 2012, 02:38:34 PM
I agree, it costed me my forum once because I had no idea that admin panel backups didn't work. So, I'd say yes :)
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: 青山 素子 on April 23, 2012, 02:41:40 PM
Taking backups is all well and good, but they should always be tested. It's the same thing with any kind of minimal-use item like carbon monoxide or smoke detectors - those should be checked once a year. At the least, important database backups should be checked on a similar schedule.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Sir Osis of Liver on April 23, 2012, 02:43:12 PM

   (http://www.thekrashsite.com/pics/banghead.gif)

Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: kat on April 23, 2012, 02:43:32 PM
Yeah, but this is abysmal stuff.

If the devs are too hard-pressed, or don't give a ****** (No dig intended), is this something that could be made into a mod?

Although, if it was done as a mod, not everyone would even know of it's existence, I guess.

Soooo....
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: a10 on April 23, 2012, 06:10:38 PM
Fix it or remove it! Any software can be delivered with some potentially serious\fatal flow, but it's a disgrace if it's not dealt with properly after it has been discovered.

Those who are often on the forum\wiki pick up any crucial info quite quickly and gets phpmyadmin up and running asap, but seemingly some come here for help after the fact, and discover the hidden truth.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Bigguy on April 23, 2012, 06:18:50 PM
I say fix it. It's a nice part of SMF when it works. If it can be fixed then I say leave it, if not chuck it out like your throwin out the old bath water. :)
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: TheListener on April 23, 2012, 06:20:32 PM
Quote from: Bigguy on April 23, 2012, 06:18:50 PM
chuck it out like your throwin out the old bath water. :)

I'm sure the neighbours would be thrilled.

;D
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: LiroyvH on April 23, 2012, 06:21:52 PM
Did the topic title really have to be in capitals? :P
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Adish - (F.L.A.M.E.R) on April 23, 2012, 07:03:45 PM
A fix may be plausible. However, it could be resource intensive and could take quite some time depending on the server resources. At this point, all I can say is that we can create a sticky and give them a warning that they check the backup before proceeding. Sometimes you do get a good backup, while sometimes you don't. It all depends on the server functionality.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Sir Osis of Liver on April 23, 2012, 09:53:20 PM
Quote from: (F.L.A.M.E.R) on April 23, 2012, 07:03:45 PM
Sometimes you do get a good backup, while sometimes you don't.

That's really the whole point - it's unreliable, and it's a critical function.  There's no error or any indication that the backup has failed.  An experienced user will notice that the backup file is way too small, but not everyone is an experienced user.  If it's a major problem to fix it, shouldn't be too difficult to issue a patch that removes it altogether.

Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: 青山 素子 on April 24, 2012, 01:37:30 AM
I do have to admit that having something that doesn't work reliably is probably more dangerous than not offering it at all.

Still, people should be testing their backups instead of just relying on hope.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: kat on April 24, 2012, 06:19:01 AM
Quote from: CoreISP on April 23, 2012, 06:21:52 PM
Did the topic title really have to be in capitals? :P

I thought so, for two reasons.

1) As has been pointed out, this is a pretty major, critical flaw, when you think about it.

2) It's been mentioned at least four zillion times and squat's been done about it.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: CircleDock on April 24, 2012, 06:34:10 AM
Quote from: K@ on April 24, 2012, 06:19:01 AM
Quote from: CoreISP on April 23, 2012, 06:21:52 PM
Did the topic title really have to be in capitals? :P

I thought so, for two reasons.

1) As has been pointed out, this is a pretty major, critical flaw, when you think about it.

2) It's been mentioned at least four zillion times and squat's been done about it.

Ummm... You're part of the Team, you discovered it, wrote about it and, I assume, reported it -- and it hasn't been fixed???!!

Oh bloody wonderful.

And just how are you supposed to test if a backup is working? If it's faulty, you'll probably lose (part of) your site in the process.

>:(
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: ziycon on April 24, 2012, 06:35:24 AM
I'm working on a very basic mod at the moment for backups, its nearly done.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: kat on April 24, 2012, 06:47:45 AM
Quote from: CircleDock on April 24, 2012, 06:34:10 AMUmmm... You're part of the Team, you discovered it, wrote about it and, I assume, reported it -- and it hasn't been fixed???!!

I suspect it's a case of "Perceived priorities", CD.

As I'd kinda unilaterally decided that it was time to shout about it, I did. ;)

I'm expecting a bit of flak over it, naturally. ;)
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: emanuele on April 24, 2012, 07:22:08 AM
Quote from: CircleDock on April 24, 2012, 06:34:10 AM
And just how are you supposed to test if a backup is working? If it's faulty, you'll probably lose (part of) your site in the process.
If you want to test a backup you do it off-line on your own computer or at least on another database, of course you *never* do it with your live site.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: kat on April 24, 2012, 07:32:44 AM
You can often tell just from the size of the file, I would assume.

I've never used the thing, myself, because, when I very first used SMF, I realised that there's nowhere in the Admin section to actually restore what you save, there.

So, I found the CPanel way of doing it, PDQ.

But, for some nooBs, it may not be that obvious.

I've always kinda assumed that we all want SMF to be as user-friendly as possible and this fails, in that, big-time.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Sir Osis of Liver on April 24, 2012, 03:07:37 PM
Quote from: 青山 素子 on April 24, 2012, 01:37:30 AM
Still, people should be testing their backups instead of just relying on hope.

Quote from: emanuele on April 24, 2012, 07:22:08 AM
If you want to test a backup you do it off-line on your own computer or at least on another database, of course you *never* do it with your live site.

Bear in mind that the same noobs who don't recognize the backup problem are also unlikely to know how to safely test a backup, and would certainly run the risk of trashing their forum if they tried it on a production board, as emanuele points out.

Placing the onus on the user does not excuse a serious deficiency in a critical function.

Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Adish - (F.L.A.M.E.R) on April 24, 2012, 03:17:26 PM
Quote from: Krash. on April 23, 2012, 09:53:20 PM
Quote from: (F.L.A.M.E.R) on April 23, 2012, 07:03:45 PM
Sometimes you do get a good backup, while sometimes you don't.

That's really the whole point - it's unreliable, and it's a critical function.  There's no error or any indication that the backup has failed.  An experienced user will notice that the backup file is way too small, but not everyone is an experienced user.  If it's a major problem to fix it, shouldn't be too difficult to issue a patch that removes it altogether.


The thing is, the backup method implemented is based on the server configurations. Now, we have to work through the very widely used server configuration to work on a fix for this or an alternate way which does not abuse the server resources would be a good idea. I am pretty sure this idea was brought forward previously from what I remember. Something was started back then but never got implemented either.

Many FREE hosts don't allow database backup's for free. In order to help them out, this backup method is sort of a work around. At least something is better than nothing. But I have to admit, there could be a better way out there.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: kat on April 24, 2012, 03:21:09 PM
Quote from: (F.L.A.M.E.R) on April 24, 2012, 03:17:26 PMAt least something is better than nothing.

Actually, I disagree, with that.

If it's unreliable, it's worse than nothing.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Adish - (F.L.A.M.E.R) on April 24, 2012, 03:24:23 PM
I meant it for the free hosting users. We could just have it as an addon would be the quickest fix.

For users with paid hosting and permissions to backup their database more efficiently, we should definitely look for an alternative.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: kat on April 24, 2012, 03:26:09 PM
Why bother, though?

CPanel has everything you need.

Why take up space with something that really isn't necessary?

Especially as you need CPanel to restore it, anyway.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Adish - (F.L.A.M.E.R) on April 24, 2012, 03:29:16 PM
Because FREE hosting services use something called vista panel or lxadmin or their custom control panels which don't have everything. This backup utility is a benefit for them.

Now, talking from the marketing point of view, I don't know how many forums run under free hosting with paid domains or free domains such as .tk or .co.cc or use subdomains. Roughly 5% around the world would be a guess. Now, do we want to take away that advantage from those 5% users using SMF?
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: kat on April 24, 2012, 03:38:28 PM
So, how would they restore these backups?

Seems they can't. So, again, why bother saving a backup that they can't restore?
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Sir Osis of Liver on April 24, 2012, 03:40:06 PM

If the backup function works properly in a specific hosting environment, is it possible for the software to detect the host platform and only show the backup option to users on a compatible host?

Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Adish - (F.L.A.M.E.R) on April 24, 2012, 03:45:13 PM
Quote from: K@ on April 24, 2012, 03:38:28 PM
So, how would they restore these backups?

Seems they can't. So, again, why bother saving a backup that they can't restore?
AFAIK, I think that backup is given in .tar.gz format? You extract it and an .sql file pops out. You go to phpmyadmin and restore it there. Sometimes, there is even .tar.gz support in phpmyadmin which lets you use that straight away. So, upload it using the import area and done.

Quote from: Krash. on April 24, 2012, 03:40:06 PM

If the backup function works properly in a specific hosting environment, is it possible for the software to detect the host platform and only show the backup option to users on a compatible host?


It is not just the hosting platform (I suppose you mean control panels and permissions set?) , but we could possibly make use of the phpinfo details and show up a warning if the variables set by the hosting company may not be suitable for a good backup copy to be generated. Again, some hosting companies disable phpinfo by default and users have to enable it using their .htaccess. So, that may be another issue to tackle.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Sir Osis of Liver on April 24, 2012, 03:56:44 PM
Quote from: (F.L.A.M.E.R) on April 24, 2012, 03:45:13 PM
You go to phpmyadmin and restore it there.

If user has to restore it with phpmyadmin, why not backup with phpmyadmin?  Why bother risking a bad backup with a balky function that, more often than not, doesn't work?

Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: kat on April 24, 2012, 03:57:30 PM
So, they need to learn phpmyadmin, anyway.

So, as I said, the backup's superfluous. As they need to learn about phpmyadmin, they might as well backup, there, too.

NINJAD! :)
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: IchBin™ on April 24, 2012, 04:10:44 PM
I say we ditch it like the plague. There are far better tools out there, and there's really no need for us to reinvent the wheel in trying to fix it. Backups should be done on the host level, not the forum level IMO. And yes, I've shouted about this before too.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Adish - (F.L.A.M.E.R) on April 24, 2012, 04:14:08 PM
See it from this point of view:
- A free hosting user with no access to phpmyadmin can not backup their database or they have to pay for it.
- They use the backup utility and backup their database.
- They then go to another free host that can restore their database and continue further with their forum OR a paid host with access to phpmyadmin and they restore the backup themselves.

This gives room for free users to at least use possible work around.

So, ditching it doesn't seem very viable unless it can be used as a modification to perform backups from the next major release.
Fixing it does seem viable, but it won't be fixed in the core software until the next major release and that is if the devs decide to do it.

Encouraging users to use the right tools if available would be the best way ahead right now.

PS: I am not against your ideas, I am just trying to look at both the sides and suggest the best way ahead.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Chalky on April 24, 2012, 04:24:01 PM
So how about simply adding a message to the Backup option saying "This tool is unreliable - please do not use if other options are available to you"?

Maybe I'm being simplistic but this would cover all options....
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: kat on April 24, 2012, 04:26:05 PM
From where I see it, that just makes SMF look silly(er) for putting something unreliable for something that is, after all, critical.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Chalky on April 24, 2012, 04:33:31 PM
That depends if you think something silly looks less silly just because you don't point out that it's silly ;)

I agree that it should be trashed altogether if it doesn't work properly but as F.L.A.M.E.R.  seemed so adamant that there is justification in keeping it, I was trying to find some middle ground (http://sabrinova.com/Smileys/fantasticsmileys/tiphat.gif)
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: kat on April 24, 2012, 04:35:36 PM
Reading through the topic, F.L.A.M.E.R. seems to be in a minority of one, thus far. ;)
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Adish - (F.L.A.M.E.R) on April 24, 2012, 04:36:27 PM
I am all for trashing it really. :) I am just thinking it from all users perspective. :)
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Kindred on April 24, 2012, 04:40:55 PM
actually, except on a site with several million posts, I have never had an issue with the internal backup function.

and Flamer raises a good point ---    it is a way for folks with free hosting to GET their data in order to transfer it to another host or install a test forum locally.
So, I am against ditching it completely.

It should either be corrected to work better or perhaps replaced with an external (third part, predesigned) function
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: kat on April 24, 2012, 04:41:35 PM
Seriously, I think dumping it, totally, would be of great benefit for our users.

We can point them to the wiki, to learn how to backup, and we won't get any more "****** my backup's useless!" posts into the bargain.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: IchBin™ on April 24, 2012, 04:45:35 PM
Most free hosts that I ever looked at removed the option to even use the database backup section, as well as the option to install mods and themes. So I'm not sure the free host option is really a valid one at this point.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Adish - (F.L.A.M.E.R) on April 24, 2012, 04:48:31 PM
Ichbin,
The free hosts you are talking about are the one's with SMF already pre-loaded. I am talking about the free hosting services which allow you to use your own paid domain or .tk or .co.cc or their sub domain. There are thousands of free hosts out there like those. They don't have any restrictions except putting an advert on the forum.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: IchBin™ on April 24, 2012, 06:17:54 PM
If they don't have restrictions, then they are allowed to upload a script like big dump? They're allowed to use better tools than what we can provide?
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Antechinus on April 24, 2012, 06:43:40 PM
I'm in two minds about this. Yes, our current backup script does have limitations. However, it used to do perfect backups until my site reached 20 meg zipped, at which point I changed to using a cron job for backups.

I did find the built-in script very valuable in the early stages of running the site as a complete n00b who knew absolutely nothing.  I realise it is not as reliable on all servers, and that getting perfect 20 meg zips from it is unusual.

The reasons why it did so well on our box must be partly due to server config, and I'm happy to provide details of that if they are useful. The other thing is that I ran up against the default limit for script execution time very early on, which was truncating the backups to bugger all. I noticed that straight away when I got a backup that was smaller than the previous one. Searching here quickly provided me with a way to up the limit to something more useful.

Really, anyone who is interested in backing up their site should be keeping an eye out for anomalies like that. It's just basic common sense IMO. If people are not prepared to keep an eye out for bloody obvious malfunctions, then there's probably not a lot we can do for them.

I'd be in favour of adding information about the scripts limitations asap, and then looking at whether it can be improved.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: butchs on April 24, 2012, 08:18:14 PM
I used to back up from both php and SMF.  Now I will just use php.    :-X
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: ziycon on April 25, 2012, 04:07:26 AM
Personally I think that there should be a way to take backups from within SM but for obvious reason you wont be able to restore a backup through the interface. It helps with people that aren't as technical and makes it easier for they to have piece of mind instead of having to rely on someone else or their host to make sure they have a recent backup in a worst case situation.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: emanuele on April 25, 2012, 12:23:54 PM
So, I just finished a dump (plain text for the moment) of a 2.8 GB database with the internal SMF backup utility (MySQL only at the moment), it's a bit slow (20 minutes...that is not so good for consistency, but is still probably better than nothing).

The question is: should I debug it, test it with PostgreSQL, with compressed output, try to find few other optimizations and commit it or commit the complete drop of the tool?
Let me know.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Sir Osis of Liver on April 25, 2012, 01:39:35 PM

It should work for all users, at all experience levels, across all server platforms.  If it can't be fixed to do that, it should be removed, because the result can be disastrous for the users most likely to rely on it, and least likely to understand what's happening.

Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Kindred on April 25, 2012, 02:38:58 PM
Well, I disagree. :P

I think that a little debugging and optimizations would go a long way rather than dropping the tool (which has never failed for me, mind you - I can't recreate the problem on 3 different servers/sites. LOL!)
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Adish - (F.L.A.M.E.R) on April 25, 2012, 02:40:35 PM
Oh! And it is also dependently on browser execution time too. Some browsers give up after a few minutes of wait time. So, that is another problem that needs tackling if we going to get all perfect :P
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Kindred on April 25, 2012, 02:41:41 PM
well, I don't know about that  --- since even the phpmyadmin backup fails for me on my 3.7 gb database over a browser.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: kat on April 25, 2012, 04:16:12 PM
So, maybe a note saying "Don't trust this method of backing-up your database, coz it's bollox. Use your CPanel, or equivalent, if you can." would be best, to keep F.L.A.M.E.R. and Kindred happy, then?
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: vbgamer45 on April 25, 2012, 04:19:04 PM
Quote from: Kindred on April 25, 2012, 02:38:58 PM
Well, I disagree. :P

I think that a little debugging and optimizations would go a long way rather than dropping the tool (which has never failed for me, mind you - I can't recreate the problem on 3 different servers/sites. LOL!)
Maybe have some kind of progress bar and do it in chunks instead of all at once. Like the upgrade tool. And just have the chunks written to disk.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: emanuele on April 25, 2012, 04:34:24 PM
Quote from: Krash. on April 25, 2012, 01:39:35 PM
It should work for all users, at all experience levels, across all server platforms.
Something so comprehensive is totally impossible. ;)

What I can say is: I wrote down this code, is not so different from the previous, it "simply" split the backup in chunks (well not exactly, not only, but that's not the point here).
The limits are more or less (I still have to optimize it a bit) the limits that phpMyAdmin has: almost none with plain text dumps, few with zipped dumps because of the way phpMyAdmin handles the backup I discovered they are almost useless if bigger then a certain size, pmyadmin concatenates gzencode/bzcompress outputs, but these cannot be concatenated, resulting in a file that has only the first echo readable, at least here on my box).

Additionally it takes time. That's not something related to server overheating or the fact you should stay in front of the computer waiting for the backup to finish, but is something related to data consistency. Think about it: if the backup takes 20 minutes it means that if something happens during these 20 minutes (a post, a new topic, a ban, whatever) this could be lost. Or, since the backup scans the entire database from the first table to the last, it could create inconsistent data: if new member registers while the backup is at {db_prefix}messages and this new user posts a message, in the backup the message could appear, but the corresponding user will not (because {db_prefix}members is before {db_prefix}messages).
So for "big databases" the maintenance mode would be mandatory in order to be sure the backup is "good enough".

There are few things to take in consideration.

Quote from: Kindred on April 25, 2012, 02:38:58 PM
(which has never failed for me, mind you - I can't recreate the problem on 3 different servers/sites. LOL!)
Just because you don't use it? :P
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Kindred on April 25, 2012, 04:47:16 PM
lol, silly...    I have used it, previously on my smaller forums.

When I need to do a quick backup without going through the trouble of logging into the site's control panel
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: kat on April 25, 2012, 05:03:40 PM
Quote from: vbgamer45 on April 25, 2012, 04:19:04 PM
Maybe have some kind of progress bar and do it in chunks instead of all at once. Like the upgrade tool. And just have the chunks written to disk.

The problem with doing it in chunks on the server, is that for the entire time those backups are being written, they have to be written somewhere that is server-writable (i.e. 777) and thus can be readily tampered with.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Adish - (F.L.A.M.E.R) on April 25, 2012, 05:12:22 PM
PhpMyAdmin does give out incomplete backup's too by the way no matter if it is plain text .sql or compressed in .zip .tar or .tar.gz. The settings are dependent on my.cnf config by the host. Many times, they have a mysql dump setting which stops the backup once a certain size is reached.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Antechinus on April 25, 2012, 07:34:02 PM
Quote from: Krash. on April 25, 2012, 01:39:35 PMIt should work for all users, at all experience levels, across all server platforms.  If it can't be fixed to do that, it should be removed, because the result can be disastrous for the users most likely to rely on it, and least likely to understand what's happening.

Analogy: if a certain medicine is not effective in all cases, it should be removed from distribution. :)
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Sir Osis of Liver on April 25, 2012, 08:28:43 PM

No, but if a medication is found to be harmful to a majority of users, it should, and sometimes is ($$$$ notwithstanding) taken off the market.  It's not a question of the backup feature being useful to a minority of users, and harmless to others.  It can, and does, trash forums.

Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: 青山 素子 on April 25, 2012, 08:41:59 PM
Quote from: Krash. on April 25, 2012, 08:28:43 PM
No, but if a medication is found to be harmful to a majority of users, it should, and sometimes is ($$$$ notwithstanding) taken off the market.  It's not a question of the backup feature being useful to a minority of users, and harmless to others.  It can, and does, trash forums.

If you wanted your analogy to be correct, the backup feature would have to break existing boards. I haven't seen any reports of such. Rather, the tool lacks efficacy in providing working backups under some conditions. That's more a matter of "this drug doesn't work in 20% of cases" than "this drug is causing strokes".
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Sir Osis of Liver on April 25, 2012, 08:46:33 PM

This thread was spun off (by K@) from yet another thread about a board that was trashed and had an unuseable backup from the admin feature.

Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Antechinus on April 25, 2012, 08:51:16 PM
Ok, I'm interested. Why was the backup unusable? Was it truncated, or was there some other problem?
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Bigguy on April 25, 2012, 09:24:53 PM
Is there no way to at least hook into cpanels backup feature and use it through SMF. ???
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: mashby on April 25, 2012, 09:28:35 PM
Interesting. I had posted some questions before I did an upgrade of a site from 1.1.x to 2.0.2 and I had written that I was going to backup the database using the SMF admin panel. I was advised to use cPanel instead and not to rely on the SMF backup. So, which is it?

Realistically speaking, using cPanel is the most effective way to do backups, right? What tool does the administrators of this site use to do a backup?
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: busterone on April 25, 2012, 09:58:58 PM
I use SSH myself, but I realize that a large portion of new users may not have command line access, nor know how to do it if they do have access. It is by far the best export and import method in my opinion though.
My experience with SMF's admin backup was shoddy from the start. I tried it for S@G on several hosts and had the same result each time. Anytime a DB had grown beyond around 15 mb native, the backup failed. 
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Bigguy on April 25, 2012, 10:02:05 PM
@ Mashby: As far as I know cpanel is the most reliable backup you can get.

I need to learn SSH for sure. That's one of the next things I will do when I get time.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: 青山 素子 on April 25, 2012, 10:23:34 PM
Quote from: mashby on April 25, 2012, 09:28:35 PM
I was advised to use cPanel instead and not to rely on the SMF backup. So, which is it?

Realistically speaking, using cPanel is the most effective way to do backups, right? What tool does the administrators of this site use to do a backup?

In order of preference:
Using mysqldump directly avoids all the potential problems from a web-based solution including timeouts and memory limits. It's also usually lighter on resource usage and faster. If that isn't available, using the hoster-provided tool is the next best as these are sometimes configured so as to avoid some of the limits imposed on customer sites. The last resort is using the SMF tool which works okay, but is bound by any restrictions the host imposes on customer sites.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Sir Osis of Liver on April 25, 2012, 10:25:58 PM
Quote from: Antechinus on April 25, 2012, 08:51:16 PM
Ok, I'm interested. Why was the backup unusable? Was it truncated, or was there some other problem?

Here is the original thread -

http://www.simplemachines.org/community/index.php?topic=474878.msg3320443#top

Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: mashby on April 25, 2012, 10:47:05 PM
So, to fix the SMF part, which as I understand it is third in order of effectiveness (thanks for the analysis Motoko!), the DEVs would have to make it at least as good as cPanel, or ideally, as good as mysqldump. I guess my question would be why waste the time to do that when there are better, existing tools? I get where Flamer is going with the concern regarding free hosts. In that case, though, don't you get what you paid for? If it's an easy fix, great, but this issue seems to have been lingering for quite some time which makes me believe it's not all that easy, or something along those lines. I'm still wondering what the administrators of this site use to do a backup?
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Sir Osis of Liver on April 25, 2012, 10:54:19 PM

Can the problem be addressed from the other end?  Is there any way the backup utility can be made smart enough to recognize server errors, or compare the backup with the db, and display an error if backup fails?

Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Yağız... on April 25, 2012, 11:38:25 PM
What about this?
http://www.simplemachines.org/community/index.php?topic=18350.0

I used it on 1.1 and still using on 2.0, works like a charm.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: ApplianceJunk on April 26, 2012, 12:30:54 AM
Quote from: CoreISP on April 23, 2012, 06:21:52 PM
Did the topic title really have to be in capitals? :P

Typing with paws... :P
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: 青山 素子 on April 26, 2012, 01:25:20 AM
Quote from: mashby on April 25, 2012, 10:47:05 PM
I guess my question would be why waste the time to do that when there are better, existing tools? I get where Flamer is going with the concern regarding free hosts. In that case, though, don't you get what you paid for? If it's an easy fix, great, but this issue seems to have been lingering for quite some time which makes me believe it's not all that easy, or something along those lines.

Perhaps move this functionality to a modification?


Quote from: mashby on April 25, 2012, 10:47:05 PM
I'm still wondering what the administrators of this site use to do a backup?

As this site is on its own servers, mysqldump, last I knew.


Quote from: Krash. on April 25, 2012, 10:54:19 PM
Can the problem be addressed from the other end?  Is there any way the backup utility can be made smart enough to recognize server errors, or compare the backup with the db, and display an error if backup fails?

Possibly, some of this could be done. Also, segmenting the backup could be helpful. Comparing wouldn't help much as it would basically require re-reading the file into some temporary table and then comparing, which would be a real pain. I don't think the effort needed would be worth the result.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Angelina Belle on April 26, 2012, 09:27:31 AM
I've had a problem with the internal backup, and I have a small site with not very many posts.

I agree that Half a backup is NOT BETTER than no backup at all.

[Unknown]'s old backup and restore scripts are reliable for me.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Adish - (F.L.A.M.E.R) on April 26, 2012, 10:58:50 AM
Last I knew, cPanel runs on perl and that is why the PHP limitations don't tend to cause any issues. SMF could possibly do that, but I don't know if perl is compatible with anything except CentOS and ubuntu. If it is, well and good. If it is not, we hit another road block. :P
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Angelina Belle on April 26, 2012, 11:03:06 AM
[unknown]'s backup and restore scripts are reliable for me, and they are written in php.  Are they reliable for larger forums?
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Adish - (F.L.A.M.E.R) on April 26, 2012, 11:08:10 AM
Not sure, I could try and test it out. I don't even know where that topic is. Would you please be kind enough to point me to that topic? :)
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Angelina Belle on April 26, 2012, 11:33:52 AM
It is really sad that the docs don't cover this: http://wiki.simplemachines.org/smf/Backup

I had to use google to search for "create_backup.php" and found
http://www.simplemachines.org/community/index.php?topic=18350.20 and
http://www.simplemachines.org/community/index.php?topic=184356.0


Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: MrPhil on April 26, 2012, 01:04:23 PM
QuotePeople can simply examine the backup to see if it's good...
Most people wouldn't know if a backup was actually any good or not, just by looking at it. Even doing a test restore isn't foolproof, if the backup quit part way through while leaving a valid .sql file. SMF's backups can have some very subtle problems, including

These are only the ones I know about. Some of them are fixed in my sig > Fixes. All of them can be fixed after the fact by editing the .sql file, except for loss of data due to timeouts, but it's delicate and time-consuming work (I know -- I've fixed several large backups for people).

QuoteSomething is usually better than nothing...
Absolutely not. A backup should be complete and reliable, or you shouldn't produce one. People are relying on this backup to protect them, and it looks very bad when it's no good.

QuoteNot all hosts offer database backup and restore functions...
That is a problem. Question: if there is no way to restore a database, what is a user on such as host supposed to do with their SMF backup? Perhaps the solution would be to offer as a separate download utility (i.e., not built into SMF, to discourage people from using it) a simple repaired backup and restore utility. It could set the forum into maintenance mode, and run in smallish chunks to keep from getting timeouts. What say?
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: 青山 素子 on April 26, 2012, 01:09:24 PM
Quote from: (F.L.A.M.E.R) on April 26, 2012, 10:58:50 AM
Last I knew, cPanel runs on perl and that is why the PHP limitations don't tend to cause any issues.

Plesk uses PHP and doesn't have issues either (at least with backups). The real problem is the limitations on client websites.


Quote from: (F.L.A.M.E.R) on April 26, 2012, 10:58:50 AM
SMF could possibly do that, but I don't know if perl is compatible with anything except CentOS and ubuntu. If it is, well and good. If it is not, we hit another road block. :P

Perl runs on nearly everything, including Windows, Solaris, AIX, HP-UX, and every flavor of Linux and BSD UNIX. You'll still run into the issue that not all hosts allow perl scripts, and you'll still be dealing with any timeouts the host has on client sites.

The best solution is to pull out the integrated backup tool, turn it into a supported modification, and improve that code as best as can be done.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Angelina Belle on April 26, 2012, 01:22:26 PM
Mr.Phil -- doesn't unknown's create_backup and restore_backup do that? Do the job in chunks, to avoid timeouts?
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: MrPhil on April 26, 2012, 03:49:22 PM
If it's been repaired (if needed) to avoid the problems I listed, it might just fit the ticket. Should it be included as a standalone utility in SMF (not directly called by SMF), or just offered for download? That is, do we want to encourage users to back up/restore using phpMyAdmin wherever possible, and only use this utility as a last resort? Should such a utility be optimized in any way for SMF (what ways)? Should it only back up/restore {smf_prefix} tables, in case the user has only one shared database, or can it be a general purpose utility?
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Angelina Belle on April 26, 2012, 04:21:46 PM
As it is, it is general purpose.  If you want to back up only one prefix, you specify the prefix.
You have to type in all the "settings.php stuff" yourself.

It would be great to have someone look through the code critically.  It is not complicated.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: emanuele on April 26, 2012, 04:37:16 PM
Quote from: MrPhil on April 26, 2012, 01:04:23 PM
  • The backup can end in the middle due to timeouts, and the poor user is none the wiser.
What I have right now is a code that is as good as phpMyAdmin is (as far as I can test on my own computer).
It still rely on apache_reset_timeout or set_time_limit, I'll check later other ways to do the backup.

Just added something like this to the backup page (wording needs a lot of love... :P):
http://img337.imageshack.us/img337/8521/screen61r.png

Quote from: MrPhil on April 26, 2012, 01:04:23 PM
  • SMF backs up in chunks of 250 records per INSERT. If there are 250*N records, you end up with a superfluous INSERT with no data.
That shouldn't happen with the code I wrote.

Quote from: MrPhil on April 26, 2012, 01:04:23 PM
  • Field names should be within ` because sometimes there are reserved keywords used as field names.
That is already like that in 2.0 as far as I can see:
$data = 'INSERT INTO `' . $tableName . '`' . $crlf . "\t" . '(`' . implode('`, `', $fields) . '`)' . $crlf . 'VALUES ';
$schema_create .= ' `' . $row['Field'] . '` ' . $row['Type'] . ($row['Null'] != 'YES' ? ' NOT NULL' : '');
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: IchBin™ on April 26, 2012, 04:51:16 PM
I think putting a message about our tool in the backup page not working or not being reliable looks pretty amateur'ish. I think I'd prefer we point to the scripts that Unknown wrote. And just update those tools if they need it.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Angelina Belle on April 26, 2012, 05:06:53 PM
Knowing about a bug and not admitting it looks ????.
I think it would make sense to own up to "a bug" in the backup page.

Disable it in 2.1, if it can't be replaced "in time".

Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Antechinus on April 26, 2012, 05:07:36 PM
Quote from: AngelinaBelle on April 26, 2012, 11:03:06 AM
[unknown]'s backup and restore scripts are reliable for me, and they are written in php.  Are they reliable for larger forums?
Corn job running straight from mysqldump. Sorted. Easy. Works. No extra shiz required. :)
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Antechinus on April 26, 2012, 05:08:55 PM
Quote from: MrPhil on April 26, 2012, 03:49:22 PM
If it's been repaired (if needed) to avoid the problems I listed, it might just fit the ticket. Should it be included as a standalone utility in SMF (not directly called by SMF), or just offered for download? That is, do we want to encourage users to back up/restore using phpMyAdmin wherever possible, and only use this utility as a last resort? Should such a utility be optimized in any way for SMF (what ways)? Should it only back up/restore {smf_prefix} tables, in case the user has only one shared database, or can it be a general purpose utility?
Why phpMyAdmin? Honestly, it's not the best way of tackling backups and restoring.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: IchBin™ on April 26, 2012, 05:39:40 PM
Quote from: AngelinaBelle on April 26, 2012, 05:06:53 PM
Knowing about a bug and not admitting it looks ????.
I think it would make sense to own up to "a bug" in the backup page.

Disable it in 2.1, if it can't be replaced "in time".

I'm not talking about that. I'm saying let's not show that we have half assed code in our product. Either completely remove the option and perhaps point to the right tool, or fix the one so we don't have to put such a message in our product.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: emanuele on April 26, 2012, 05:42:48 PM
Quote from: AngelinaBelle on April 26, 2012, 05:06:53 PM
Disable it in 2.1, if it can't be replaced "in time".
It can be replaced by something that works better, it cannot be replaced by something that works *in every possible condition*, but not because "we" are not good at writing such tool, but simply because it's impossible.

SMF is written in php and run in a very wide range of server configurations (from the cheap free overseller host to dedicated servers full of ram, space and almost without limits.
The point is: even on such super-duper server a backup made with php is *not* always reliable. It cannot be. But the same is true for *any* php script attempting to do a backup on large databases (phpMyAdmin included...and their authors should know something about MySQL).

I already wrote many of the reasons why such backups are not reliable, now it's up to you: do you want something or should we drop it completely?
Keep in mind that the *only* 100% reliable alternative is the command line, of course many people using SMF don't have access to such tool...

Quote from: IchBin™ on April 26, 2012, 05:39:40 PM
Quote from: AngelinaBelle on April 26, 2012, 05:06:53 PM
Knowing about a bug and not admitting it looks ????.
I think it would make sense to own up to "a bug" in the backup page.

Disable it in 2.1, if it can't be replaced "in time".

I'm not talking about that. I'm saying let's not show that we have half assed code in our product. Either completely remove the option and perhaps point to the right tool, or fix the one so we don't have to put such a message in our product.
On the code I'm using it's not a problem of fix anything.
As I said before I did a backup of a 2.8 GB database with that code (on my localhost), the point is that in a real-life scenario this backup is unreliable because it took me 20 minutes to do it. And the same would be true for phpMyAdmin.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: 青山 素子 on April 26, 2012, 06:24:55 PM
Quote from: IchBin™ on April 26, 2012, 05:39:40 PM
I'm saying let's not show that we have half assed code in our product.

Looking at the page in the new doc wiki, and ugh... It's totally non-professional in tone and way too conversational. Can I just rewrite it in whole or do I need to run a total rewrite through somewhere first?
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Angelina Belle on April 27, 2012, 08:16:51 AM
Regarding the reliability of MySql backup written in php:
Thanks for pointing that out, emanuele.  I was not aware.  I obviously have not tested out [unknown]'s script on a larg forum.
I've only seen that it works better than the inbuilt utility on my little forum running at 1&1, and I don't know why.

Would it be possible to include, with whatever backup thingy we eventually go with, some kind of guideline that the admin can read if they care to do so?
This backup is likely to work in the following cases, and unlikely to work in the following cases and here's how you can check if it worked properly.
Is there any way that the script itself can figure out if the backup failed?

Regarding the wiki:
It's a wiki.  Generally, on a wiki, we write first and ask questions later. Be bold.  But respectufl.
If two people care deeply about the wording of one document, they should discuss it in the documentation helpers board.
If there is a big problem, I will lock the document until "wiki war" is over.

We have two sets of standards for documents.  Those documenting the actual "pages" that admins see while using the software should be dry and non-personal.

How-tos, tutorials,  and FAQs can be written in a more "friendly" style.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: MrPhil on April 27, 2012, 09:36:41 AM
Quote from: Antechinus on April 26, 2012, 05:08:55 PM
Why phpMyAdmin? Honestly, it's not the best way of tackling backups and restoring.

If there is a better tool that is available on nearly every system, let's recommend it. My first priority would be to suggest tools that are likely to be on every system. If we package a better backup and restore utility with SMF, fine, let's tell people to use it (but it has to be bug free!). If it's a tool that many users are going to have to download and install (separately from SMF), I'm not so sure that's the right road to take.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: MrPhil on April 27, 2012, 09:46:41 AM
Quote from: emanuele on April 26, 2012, 05:42:48 PM
It can be replaced by something that works better, it cannot be replaced by something that works *in every possible condition*, but not because "we" are not good at writing such tool, but simply because it's impossible.

SMF is written in php and run in a very wide range of server configurations (from the cheap free overseller host to dedicated servers full of ram, space and almost without limits.
The point is: even on such super-duper server a backup made with php is *not* always reliable. It cannot be. But the same is true for *any* php script attempting to do a backup on large databases (phpMyAdmin included...and their authors should know something about MySQL).
Granted, it is impossible to prove that any given non-trivial program is 100% correct. However, the issue at hand is that SMF's current backup is known to be riddled with serious bugs that would have been revealed with more than cursory testing.

Quote
As I said before I did a backup of a 2.8 GB database with that code (on my localhost), the point is that in a real-life scenario this backup is unreliable because it took me 20 minutes to do it. And the same would be true for phpMyAdmin.
The only problem I know of with phpMyAdmin (and maybe straight MySQL command line itself) is that it's subject to process timeouts. That can be cured by breaking up the job into smaller pieces in some manner. Otherwise, it seems to be quite reliable.

A further note (in favor of an SMF-specific backup): any backup that takes more than a second or two to back up the entire database should automatically put the forum into maintenance mode so that the database is not changed during the backup. This prevents the backup from being inconsistent.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: MrPhil on April 27, 2012, 10:08:59 AM
Quote from: AngelinaBelle on April 26, 2012, 05:06:53 PM
Knowing about a bug and not admitting it looks ????.
I think it would make sense to own up to "a bug" in the backup page.

Disable it in 2.1, if it can't be replaced "in time".

Knowing about a serious bug and not saying anything about it (hiding it) verges on criminal behavior. Simple Machines could actually be found liable for damages if someone were to sue for loss to their business. Listing a relatively minor bug and its probable impact, with the promise that we're working on it and will have a fix soon, is permissible. Putting out software with known serious bugs, whether documented or not, is unacceptable. The software or function should be disabled, or release held up until it's fixed. This applies to all levels of SMF, not just 2.1. We've got to get out of the mindset that serious bugs in downlevel branches are OK, because we have a new leading edge version out. Unless we want to go out and cyber-attack SMF 1 installations to force them to upgrade to SMF 2, we have to accept that people will be running 1.1 and even 1.0 for a long time to come. I wouldn't be surprised if there were still YaBB (SE) installations running!
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Kindred on April 27, 2012, 11:04:06 AM
Actually, I disagree with the last contention, MrPhil.

Bugs in previous versions which are known and acknowledged (or more importantly, fixed) in new releases can require people to upgrade. Sorry... but if someone refuses to upgrade when there is a fixed version out there, then what happens is on their own head. If someone doesn't keep track of the things fixed in upgrades, then THEY are acting stupidly.

So, fixing it (my preference) to some reasonable standard or removing it (I hope not) in 2.1 is acceptable.
1.0 is at end of life... ignore it.
1.1 has been stated, numerous times, to be in security release mode only...  If people really want to make a stink about it, then kill the function in 1.1 with the next security release, but I see no point in wasting time on a version which is outdated and replaced.
(honestly, if it were up to me, I would encourage the immediate sunset of 1.0 and the planned sunset of 1.1 by next year (instead of hanging on to outdated versions like we seem to do) -- but that's neither here not there, for this issue.)

and as for your statement "would have been revealed with more than cursory testing", I disagree....  I don't think I have actually had it fail on me, across 8 sites on 5 different hosts, machines, etc. I'll admit, it does seem to be an issue with some folks. Not sure what the common factor is which causes it to fail for them, but I'd hardly say that it was so obviously broken that it should have been immediately obvious (considering the length of our beta process, really?)
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: MrPhil on April 27, 2012, 12:24:17 PM
Testing was clearly inadequate. For example, the code outputs INSERTs with 250 records at a time. Testing should have been done with 499, 500, and 501 records for some table, just to make sure the 250 splitting works OK. This would have revealed a problem at 500 (250*N) records with an INSERT with no data.

I'll agree that it's hard to design a test to pick up everything. Who would think to test a hex field with value 123e4567, which MySQL will barf on if it's not in quotes? (tries to handle it as a floating point number) Older versions did not put ` around field names, which might not be a problem with vanilla SMF, but does show a problem with mods which add fields with reserved keywords as names. Good coding practice would try to account for that, but even thorough testing on vanilla SMF might not have displayed the problem.

As for your contention that older versions need not be fixed, I must respectfully disagree. At the very least, the next/last 1.0 update should disable backup (if it's not to be fixed) and the next 1.1 security update should do the same. If people want to stay at older 1.x levels, that's their lookout, but I feel that SM has an obligation towards its community to make available patches to fix bugs which seriously compromise the system's functioning, even if, strictly speaking, they're not security related. Needless to say, 2.0 should get the full treatment, and not wait for some future 2.1.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Kindred on April 27, 2012, 01:08:18 PM
well, obviously, I disagree...   but I'll leave it to the active team folks to decide how they specifically want to handle it.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Sir Osis of Liver on April 27, 2012, 02:40:30 PM

If a feature has limited functionality, but that functionality is considered to be of sufficient value to put up with the cases where it doesn't work, I can understand keeping it, but only if the feature displays appropriate error messages when it fails.  The very bad thing about the admin backup is that it always appears successful, making users believe they have a good backup when they actually do not. 

Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Norv on April 27, 2012, 02:48:53 PM
Quote from: MrPhil on April 27, 2012, 12:24:17 PM
I feel that SM has an obligation towards its community to make available patches to fix bugs which seriously compromise the system's functioning, even if, strictly speaking, they're not security related. Needless to say, 2.0 should get the full treatment, and not wait for some future 2.1.

This is how it is. For as long as they're in active development, older versions may receive backported fixes for major/critical issues, that could affect critical functionality or compromise the system's functioning. We have done that in the past, for SMF 1.0.x even, and of course SMF 1.1.x, as well as 2.0.x. It does apply even to 1.0.x, on some issues. And of course, during ALL the time until 1.0.x reaches EOL. Whether an issue is critically needed or not, is the question, not that we do that. We might also not do it the same way, too.

It has been under question whether this is indeed the case of the backup functionality. It doesn't compromise one's system, though. There are cases of hitting resource limits and such, and there is no warning, error, message, something to tell the user that it may go wrong too. That's wrong, in the sense that when it happens, it gives a false sense of security.


Please do share your feedback on Ema's tool presented here:

Quote from: emanuele on April 26, 2012, 04:37:16 PM
Quote from: MrPhil on April 26, 2012, 01:04:23 PM
  • The backup can end in the middle due to timeouts, and the poor user is none the wiser.
What I have right now is a code that is as good as phpMyAdmin is (as far as I can test on my own computer).
It still rely on apache_reset_timeout or set_time_limit, I'll check later other ways to do the backup.

Just added something like this to the backup page (wording needs a lot of love... :P):
http://img337.imageshack.us/img337/8521/screen61r.png

Quote from: MrPhil on April 26, 2012, 01:04:23 PM
  • SMF backs up in chunks of 250 records per INSERT. If there are 250*N records, you end up with a superfluous INSERT with no data.
That shouldn't happen with the code I wrote.

Quote from: MrPhil on April 26, 2012, 01:04:23 PM
  • Field names should be within ` because sometimes there are reserved keywords used as field names.
That is already like that in 2.0 as far as I can see:
$data = 'INSERT INTO `' . $tableName . '`' . $crlf . "\t" . '(`' . implode('`, `', $fields) . '`)' . $crlf . 'VALUES ';
$schema_create .= ' `' . $row['Field'] . '` ' . $row['Type'] . ($row['Null'] != 'YES' ? ' NOT NULL' : '');


Thank you very much for looking into it, Ema. Perhaps having it available for testing would be even better, when possible, for people to take it for a spin.
As for wording, can you propose an alternative, better wording?
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Norv on April 27, 2012, 02:56:10 PM
Quote from: Krash. on April 25, 2012, 10:25:58 PM
Quote from: Antechinus on April 25, 2012, 08:51:16 PM
Ok, I'm interested. Why was the backup unusable? Was it truncated, or was there some other problem?

Here is the original thread -

http://www.simplemachines.org/community/index.php?topic=474878.msg3320443#top

Let me get this straight.
1/  the user asks how to restore their backup.
2/  we are, instead, telling them the backup is "corrupted". (?)
3/  this discussion is opened, to report that another backup was corrupted. (?)

I must be missing something here?
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Sir Osis of Liver on April 27, 2012, 03:07:49 PM

I think the point was to caution the OP that the backup may not be useable.  Not knowing the experience level of the user, importing a bad backup could further damage the forum db.  These support topics are problematic because it's often difficult to determine exactly what the user is dealing with, and the unreliability of admin backup must be considered.

Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: emanuele on April 27, 2012, 03:14:32 PM
Quote from: N. N. on April 27, 2012, 02:48:53 PM
Thank you very much for looking into it, Ema. Perhaps having it available for testing would be even better, when possible, for people to take it for a spin.
I will, I will, but remember my award, I would be sure I didn't put something strange in it. :P
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: kat on April 27, 2012, 03:33:06 PM
Boy, I sure started something, with this, didn't I? (http://www.katzy.dsl.pipex.com/Smileys/c018.gif)
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: MrPhil on April 27, 2012, 04:59:46 PM
@N.N., there are at least two failure categories for SMF backup (as currently implemented). One is catastrophic (partial backup due to some resource limitation, with no warning to the user). If the user is relying on that backup to restore their system, they're screwed. Might as well start over from scratch with their forum. The other is annoying and potentially much work to fix (250*N record problem, certain hex strings needing quotes, wrong default for an integer, reserved keywords as field names, and possibly others). These, at least, can be fixed by manually editing the .sql file, but it can be tricky work.

Are 100% of backups bad? No, but a distressingly high percentage are. Should the first thing a user asking how to restore their backup be told is that it's probably bad? Perhaps not, but they should be told there's a non-trivial chance it's no good.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: MrPhil on April 27, 2012, 05:02:28 PM
Quote from: K@ on April 27, 2012, 03:33:06 PM
Boy, I sure started something, with this, didn't I?

Bad kitty! Hold out your paw so I can slap it!

Hey, sheath those claws!!!
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Sir Osis of Liver on April 27, 2012, 09:45:14 PM
(http://www.thekrashsite.com/pics/ak47cat-ori.gif)

Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: kat on April 28, 2012, 10:07:07 AM
Quote from: MrPhil on April 27, 2012, 05:02:28 PMBad kitty! Hold out your paw so I can slap it!

Hey, sheath those claws!!!

(http://www.katzy.dsl.pipex.com/LOLCats/IMayHaveFaults.jpg)
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Angelina Belle on April 30, 2012, 01:03:51 PM
I can understand  that the incomplete backup problem was missed in beta testing.

It has been mentioned before, in support issues for some time.

It seems that we have had a failure to communicate -- somewhere between support team and dev team, the severity of these issues was lost, and the fix was never made.

I thought SMF 1.0.x  had already reached EOL? http://www.simplemachines.org/community/index.php?topic=472913.0
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Burridge on April 30, 2012, 04:34:41 PM
Quote from: AngelinaBelle on April 30, 2012, 01:03:51 PM
I thought SMF 1.0.x  had already reached EOL? http://www.simplemachines.org/community/index.php?topic=472913.0
According to that link, that's just the announcement - EOL doesn't happen until December 31st this year. If that is what you meant, then I misinterpreted your comment, and feel free to ignore mine...

EDIT: Lol, got a warning telling me the title of the topic was too long, haha.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: user1234 on April 30, 2012, 05:13:47 PM
Quote from: 医生唱片骑师 on April 23, 2012, 02:38:34 PM
I agree, it costed me my forum once because I had no idea that admin panel backups didn't work. So, I'd say yes :)

It cost me mine too, as Mr. Phil will attest that it put to my knees in tears (Hey Mr. Phil!) and as K@ will surely attest, as he was bearing with me throughout the adventure. I was a couple weeks trying to get it back on line, along with blowing through 4 different hosts, while being dogged by DDoS attacks at every turn. Funny thing was, after all that, all I really needed to do was to rename the database!

I back mine up every couple of days or sooner, and then I test it in another website that I have, that I uploaded SMF onto purely for test purposes (that site doesn't otherwise need a forum).
I "import" the db, and then after I get the OK, I physically look at the forum online to make sure the database looks just like it did on it's own site, then I "select all" tables and click "drop". I do it a couple times a month but I had a scare today, and got that sinking feeling again, that may encourage me to test it even weekly.

And as far as this thread subject goes, then I guess I just have to abandon that handy in-forum database backup. It worked on every backup this month except the one I tried today, for which I repeatedly got an instant "done" message but the backup is 0 kb.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Angelina Belle on April 30, 2012, 08:05:14 PM
 Oh well, when you are as old as I am, December 31st feels like tomorrow   :-[   
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: kat on May 01, 2012, 07:24:04 AM
Quote from: AngelinaBelle on April 30, 2012, 08:05:14 PM
when you are as old as I am

pffft!

Kids, eh? ;)
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: MrPhil on May 01, 2012, 10:15:30 AM
Quote from: K@ on May 01, 2012, 07:24:04 AM
pffft!

Kids, eh? ;)

Dad-gum kiddies! Let 'em start out the way I did, with FORTRAN on punch cards! That'll learn 'em! And another thing -- I had to write a whole database in binary, but we couldn't afford '0's in those days, and...
[nurse] Come on, Grandpa. It's time for your medications!
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: kat on May 01, 2012, 01:06:26 PM
Huh! FORTRAN hadn't been invented!

We used slates and chisels!
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Adish - (F.L.A.M.E.R) on May 01, 2012, 01:32:51 PM
How about we do this?
- Create an algorithm which could give the users a good idea of the backup size and is dispalyed on the forum backup page.
- Instead of letting users download the backup, place the backup in a default directory /backup or give them an option to rename it to something else through their maintenance area? (The /backup directory can not be viewable from the browser by default and a fix permission option will be a plus.)

How does this help?
- If the size is not near to what we suggest, then the backup is corrupted for sure and user can look into an alternative.
- Reduces chances of a bad copy being downloaded due to slow internet, browser crashing etc.. on the user side or module crash/restart from the host side.

What say?
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Sir Osis of Liver on May 01, 2012, 03:23:17 PM
Quote from: MrPhil on May 01, 2012, 10:15:30 AM
.... FORTRAN on punch cards!

Been there, done that.  (http://www.thekrashsite.com/pics/facepalm.gif)

Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Antechinus on May 01, 2012, 05:41:19 PM
Me too. Don't wanna do it again either.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: MrPhil on May 01, 2012, 05:46:35 PM
Quote from: (F.L.A.M.E.R) on May 01, 2012, 01:32:51 PM
What say?

Hmm. Will all of the databases tell us how large variable character fields are, so we can add them up? The big problem will be that unless we can come up with the exact expected size, users will deluge support with "My backup is 3 bytes smaller than what it should be! What went wrong?" A rough estimate is no good -- it must be exact.

What might be just as effective is to do a query on each table to count the number of records, and make sure after spitting out the backup that we output that many records. Naturally, the forum would be in maintenance mode throughout the backup. I think it would be sufficient to make sure we output all the records (i.e., I don't expect to see partial records). If it quits in the middle, couldn't we detect that, too? Maybe we should just look at the end of the .sql file to see if there is anything that looks like an error message (e.g., due to timeout) and report failure.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: emanuele on May 01, 2012, 05:51:08 PM
@FLAMER: what you are describing is more or less what you can find in the snap I posted few minutes ago: there are warnings if the backup is at risk, there are instructions (well not yet :P) on what's the best option and if the database is over a certain size the button is disabled if the forum is not in maintenance mode.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Adish - (F.L.A.M.E.R) on May 01, 2012, 10:46:16 PM
MrPhil,
A perfect size may just increase the server resource usage from how I see it. May be, under Forum Maintenance, we can give them a option of "Calculate database size now" and then they can correlate it with that, or a button on the backup page. Having it automatically calculated every time they backup the area might not be a good idea unless you think it wont be very resource intense.

Yep, looking at the last few lines will give us the answers straight away, but that may need the use of vi or nano command on the server side which would be something that is disabled as a user. Not sure, how this will work otherwise.

emanuele,
Putting the forum into maintenance mode seems to be a good idea when the forum is over a certain size. Annd... I am a little lost in this topic lol. Which post are you talking about? :P
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: kat on May 02, 2012, 05:43:49 AM
I'm no coder, as you all well-know. So, this suggestion could be total bollox and be laughable.

Thinking about archivers and the like, though, is it possible to do the equivalent of a Cyclic redundancy check? Maybe a checksum?
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: MrPhil on May 02, 2012, 10:22:00 AM
Quote from: (F.L.A.M.E.R) on May 01, 2012, 10:46:16 PM
Yep, looking at the last few lines will give us the answers straight away, but that may need the use of vi or nano command on the server side
So long as you haven't yet compressed/gzipped the .sql file, it can be opened, seek to the end, back up a few hundred bytes, and read in the tail end of it. If it doesn't contain any sort of bad-looking message, you're golden. Close the file and compress it if desired.

Quote from: K@ on May 02, 2012, 05:43:49 AM
Thinking about archivers and the like, though, is it possible to do the equivalent of a Cyclic redundancy check? Maybe a checksum?
Yeah, but what are you going to compare it against to see if it's a valid file? The backup will be different every time. There's no fixed CRC/MD5 value to check against.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: kat on May 02, 2012, 11:46:09 AM
Ah. Yeah, I had a feeling that that might be the sticking-point.

I wondered if there could be a check done, onsite, to say "This file should be xKB in size, when you get it, which, when the file gets download, would be matched against that.

Like, back in days of yore, the ol' "Verify" thing did, with tape saves, on the Spectrum, etc.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Chalky on May 02, 2012, 01:47:54 PM
[offtopic]

I had a Spectrum, 48K - I broke my heart when I spilt orange juice on it and it wouldn't work anymore.  No more "Horace Goes Skiing" for me....  :'(

[/offtopic]
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Sir Osis of Liver on May 02, 2012, 01:52:00 PM

Can the backup function automatically place the forum in and out of maintenace mode?

Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: MrPhil on May 02, 2012, 02:05:11 PM
Quote from: K@ on May 02, 2012, 11:46:09 AM
I wondered if there could be a check done, onsite, to say "This file should be xKB in size, when you get it, which, when the file gets download, would be matched against that.

Unfortunately, that doesn't tell you whether the backup file was complete in the first place, only that it was downloaded correctly. Even, then, going from Linux to Windows PC, the line-ends will change from \n to CRLF, inflating the size... you'd have to count the number of records and figure how much the size would change. Even then, I'm sure it would leave most forum owners thoroughly confused.

If the backup process is killed in the middle due to a timeout or other resource limitation, isn't there always a message at the end that we could look for?

Quote from: Krash. on May 02, 2012, 01:52:00 PM
Can the backup function automatically place the forum in and out of maintenace mode?

Of course it could (and should). The Settings.php file would have to be updated, so it would be a good reason to pull out the $maintenance variable into its own file (included into Settings.php), just like it would be a good idea to pull out the timestamp of the last database error into its own file (hint hint).
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: kat on May 02, 2012, 02:21:01 PM
Well, it was worth asking. ;)


Quote from: Sabrinova on May 02, 2012, 01:47:54 PM
[offtopic]

I had a Spectrum, 48K - I broke my heart when I spilt orange juice on it and it wouldn't work anymore.  No more "Horace Goes Skiing" for me....  :'(

[/offtopic]

I've still got mine, somewhere...
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: Angelina Belle on May 02, 2012, 03:13:45 PM
When the backup ends in the middle, it just ends.
I'll see if I can dig one up...
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: emanuele on May 02, 2012, 03:18:14 PM
Quote from: MrPhil on May 02, 2012, 02:05:11 PM
Quote from: Krash. on May 02, 2012, 01:52:00 PM
Can the backup function automatically place the forum in and out of maintenace mode?

Of course it could (and should). The Settings.php file would have to be updated, so it would be a good reason to pull out the $maintenance variable into its own file (included into Settings.php), just like it would be a good idea to pull out the timestamp of the last database error into its own file (hint hint).
It could but it shouldn't, without prominent notice at least.
And even so, you would be "in trouble" if the script dies due to a timeout or whatever error because it wouldn't be able to restore the forum to normal mode automatically (yes, that would be a notice that the backup is not good...not very user-friendly and with too many implications... and anyway I still prefer if the admin takes a moment to understand what he is doing and why he gets a warning).
As I wrote at the moment the devel version disables the button if the forum's size is big (well, of course this is an estimate) and maintenance is off, preventing (client side, but if you are crazy enough to force the download you know what you are risking) the download.
Title: Re: SMF ADMIN PANEL BACKUP IS CRAP! CAN WE HAVE A PATCH RELEASE TO REMOVE IT PLEASE?
Post by: MrPhil on May 02, 2012, 05:08:32 PM
Quote from: emanuele on May 02, 2012, 03:18:14 PM
Quote from: Krash. on May 02, 2012, 01:52:00 PM
Can the backup function automatically place the forum in and out of maintenace mode?

It could but it shouldn't, without prominent notice at least.
Of course the admin should be notified if the forum is being put into maintenance mode (which it should be for any size backup). The notice could include a quick reminder of what to do/check if for some reason it doesn't come back within a reasonable time.

Quote
And even so, you would be "in trouble" if the script dies due to a timeout or whatever error because it wouldn't be able to restore the forum to normal mode automatically
And that's why you remind the admin each time what's going on.

Quote
I still prefer if the admin takes a moment to understand what he is doing and why he gets a warning).
Incurable dreamer, eh? Most admins have no idea what's going on inside of SMF and cannot understand why this or that is happening. They just want and need a cookbook. Most admins are looking for an appliance, not a full time hobby.

Quote
As I wrote at the moment the devel version disables the button if the forum's size is big (well, of course this is an estimate) and maintenance is off, preventing (client side, but if you are crazy enough to force the download you know what you are risking) the download.
I feel that maintenance mode (1) should be turned on regardless of the estimated backup size, and that we need to do something to examine the log or backup to see if something bad happened. Do we have any experience with backup problems as to what the end of the file looks like? Does it have any consistent pattern to indicate success or failure?

Put a comment at the top of the .sql file saying that the SMF backup created it. Put a comment at the bottom of the .sql file saying "end of backup". If we see that, is there any reason we can't assume that everything in-between is OK and turn off maintenance mode?

P.S. The maintenance message in Settings.php should then be changed to something about "system maintenance is being performed..." to cover both backups and restores. Or, the text string could be dynamically updated (as a separate, included file) to indicate what's going on.