Anyone willing to test some things for me?

Started by [Unknown], October 17, 2004, 01:22:35 AM

Previous topic - Next topic

[Unknown]

I'm going to say right now, if you have a big (or moderate-size) YaBB forum please skip down below....

The first is pretty easy.  I'm attaching a database restoration tool.  It features the following:

  • Restoration from a file on the server (uploaded with FTP not HTTP.)
  • Timeout and overload prevention.
  • Doesn't require SMF to be installed.
  • Extracts zip and gzip archived dumps.

In other words, no more upload time outs... no more timeouts from too many queries... just plain high-grade database backup restoration ;).

Except one small thing... it needs testing.  I'm not going to release this, and later find out that in some cases the timeout prevention skips data.  Bad bad.  Not for a restoration utility... so what I'd like is someone with a decent size database (bigger = better) to give it a test, in a test database or even on a test server, to no harm or anything... and report results.

If you're game, it's attached.  Thanks a lot!

Okay, here's number two.  It's not as easy, but I'm hoping to get some "hits".  It's for the new YaBB converter I'm writing.

You see, I love YaBB.  I really do.  It's great.  The problem is, Perl is going out.. and for large forums... well, I also like SMF better (sorry anyone from the YaBB team.)  If you're here, especially if you're reading this, it's quite possible you agree - and want to convert over.

However, this is usually not easy, for larger forums.  Currently, you have to first go to YaBB SE, then SMF.  Even without that, the YaBB converter will quickly time out if you have even a decent-size forum.  I don't think a converter even exists for larger YaBB forums... it may have seemed impossible to escape.

This converter features timeout prevention.  Direct conversion to SMF.  All those very very nice things we work hard for in SMF and its tools/converters.

But, I need a forum with a bunch of posts to test it with.  Preferrably, more than 80,000.  That should be enough to give it a good amount of growing pains and testing.  And larger is probably even better.

So, if you have one I could test with... and a server with MySQL (or are willing to transfer the entire database of files to me) please send me a message, email me, or reply here.  I don't know how many want to convert... but this is a bit of a challenge, and that makes it enough ;).

Thanks!
-[Unknown]

Ben_S

#1
Will test it tonight, my database is currently about 340MB so that should be large enough.

Ended up popping out and it's nearly 1am so will try it tomorrow.

Ended up going to the pub, will be at the footy tomorrow too so earliest I will get to try it is wednesday and I'll probably be overworked, will see what I can do.
Liverpool FC Forum with 14 million+ posts.

[Unknown]

Heh... well, I decided, "hey, wait a minute... I've got one right here!" and downloaded the database dump for this database here ;).

It didn't work properly; really funny, I accidentally used ";" but didn't parse those, so it kept restarting from the beginning (I had tested it previously only with a shorter dump.)  I fixed that, however, and added a percentage (appoximatation) of completion.  It's looking better, imho ;).

The data restoration looked okay, but I'd still love it if you could test it.. and least to make sure it runs, but it'd be nice to make sure the data works properly too.  This database was only 30 mb (gzipped) anyway.

Attached is the updated version...

-[Unknown]

Ben_S

Liverpool FC Forum with 14 million+ posts.

Ben_S

Ok, seemed to start restoring, but have killed it as the load went upto 12 and my board stopped responding, guess I should test it on my local test box rather than my server.
Liverpool FC Forum with 14 million+ posts.

[Unknown]

Here's another, as promised, to create backups - saem timeout prevention, gzips optionally, took about 5 minutes to do this site without affecting load in the slightest.

Again, testing is a wonderful, wonderful thing ;).  Don't everyone volunteer at once, now :P.

-[Unknown]

Anguz

SMF databases only or will it work with a YaBB SE one as well?
Cristián Lávaque http://cristianlavaque.com

[Unknown]


Ben_S

I'll test the restore one again, but on a test box, only issue is, it only has 256 MB ram, Athlon 2600+ processor though (shame it's wasted and just sits there backing up my site nightly).

Anyway it may not perform too well with so little ram.
Liverpool FC Forum with 14 million+ posts.

Anguz

First with permissions set to 644 I got these errors:


Warning: fopen(/home/anguz/public_html/database_2004-10-25.sql.gz): failed to open stream: Permission denied in /home/anguz/public_html/create_backup.php on line 185

Warning: fwrite(): supplied argument is not a valid stream resource in /home/anguz/public_html/create_backup.php on line 265

etc...


Then changed the permissions to 777 and got these:


Warning: fopen(/home/anguz/public_html/database_2004-10-25.sql.gz): failed to open stream: Permission denied in /home/anguz/public_html/create_backup.php on line 172

Warning: fwrite(): supplied argument is not a valid stream resource in /home/anguz/public_html/create_backup.php on line 182

Warning: fwrite(): supplied argument is not a valid stream resource in /home/anguz/public_html/create_backup.php on line 210

Warning: fwrite(): supplied argument is not a valid stream resource in /home/anguz/public_html/create_backup.php on line 231

Warning: fwrite(): supplied argument is not a valid stream resource in /home/anguz/public_html/create_backup.php on line 265

Warning: fwrite(): supplied argument is not a valid stream resource in /home/anguz/public_html/create_backup.php on line 271

etc...


Did I do something wrong?
Cristián Lávaque http://cristianlavaque.com

[Unknown]

Quote from: Ben_S on October 25, 2004, 08:26:27 PM
Anyway it may not perform too well with so little ram.

Ideal testing conditions!

Anguz, what if you create a zero byte file with that name in that directory, and make it 777?

-[Unknown]

Anguz

Cristián Lávaque http://cristianlavaque.com

Anguz

It didn't work the first time, then I gave it 777 and it did. Had to turn off the compression because it changed the file extension (I created the 0b file with .sql).

The resulting file is almost 50 megs in size. Haven't checked it to see if it's complete though and can't right now, but will later.
Cristián Lávaque http://cristianlavaque.com

Miyuki-chan

#13
I tried it (create_backup.php) and it worked perfectly for me. ^_^ Will this script be integrated in a future version of smf?

Jazhawk

Quote from: Miyuki-chan on October 27, 2004, 09:33:25 PM
I tried it (create_backup.php) and it worked perfectly for me. ^_^ Will this script be integrated in a future version of smf?

I have the same question Unknown.

-Jazhawk

[Unknown]

I'm not sure.  Currently, it's a separate tool.

-[Unknown]

kisii

Unknwon , Are still looking for more servers you can test on?I have one that you work on...I also dio email you about it but I have not haerd a singel word from you.

[Unknown]

I am very very busy at the moment, sorry.

-[Unknown]

kisii


oscar2001

Tried this on a test forum just for the hell of it but it only restored 3 tables from a database dump (the dump was in sql.gz format) it said the restore was successfull. Am I missing something? 

Advertisement: