Urgent:Want to make conversion faster as well a problm in converting a big board

[Ricky.]

SMF Version: SMF 1.1.4
Hi,
I am trying to convert a large phpBB forum into smf board.
Problem I am facing of duplicate entry but I figured it that it happens if I click on continue button manually.

Actually, my internet connection line is poor and it drops occasionally and then conversion get stucks, so I had to click continue manually and then it says "duplicate entry " and then I have to start again.

So ..
1. is there any way that I can use my server's whole resource to make  conversion faster (I have dedicated box for the forum)
2. If possible, if my connection drops, can I continue from the place where I last left without getting duplicate key error ?

PLEASE HELP... its URGET
PS: Its been 14 hrs.. since I am trying it.. It ran successful for 8 hrs once but then my connection dropped and I had to start again.

I really need it to be fast, I am running it from many hours and only 30% done yet..

[JayBachatero]

How big is your board?  Also do you have access to the command line?

[Ricky.]

not really big but in phpBB it has 650000+ posts and yes I do have access to command line.

PS: its  almost 8 hr since the conversion is running and I guess its has not even crossed 300000 posts .. If there is way to make it fast.. I dont' mind as again.. only this board is running on that server,..

[JayBachatero]

Ok you can use this convert file to convert via command line.  You need to runt he following command.

/path/to/php -f /path/to/convert.php -- --path_to=/path/to/smf --path_from=/path/from/phpbb --convert_script=phpbb_to_smf.sql --db_pass=smf-database-pass --debug

Code Select Expand


Usage: /path/to/php -f convert.php -- [OPTION]...

    --path_to               Path to SMF.
    --path_from             Path to the software that you are converting from.
    --convert_script        The name of the script. (old_forum_to_smf.sql)
    --db_pass               SMF database password. "The MySQL password (for verification only.)"
    --debug                 Output debugging information.


[Ricky.]

Thankyou Jay, My conversion is still running and if it again fails then I will run command line version.
Just little doubt..

is it --convert_script = phbb2_2_smf.sql or you are referring to some other file ?

[JayBachatero]

Ooops.  Yes it's phpb_to_smf.sql.  We wouldn't want you going from SMF to phpBB would we?

[Ricky.]

Hi Jay..

Seems that CLI is not working :

I am getting following errors :

Code Select Expand

Notice: Undefined variable:  result in /path/to/my/smf/convert.php on line 2085

Notice: Undefined variable:  result in /path/to/my/smf/convert.php on line 2086

Warning: mysql_fetch_assoc(): supplied argument is not a valid MySQL result resource in /path/to/my/smf/convert.php on line 1200

Warning: mysql_free_result(): supplied argument is not a valid MySQL result resource in /path/to/my/smf/convert.php on line 1202

Notice: Undefined variable:  result in /path/to/my/smf/convert.php on line 2085

Notice: Undefined variable:  result in /path/to/my/smf/convert.php on line 2086

Warning: mysql_fetch_assoc(): supplied argument is not a valid MySQL result resource in /path/to/my/smf/convert.php on line 1208

Warning: mysql_free_result(): supplied argument is not a valid MySQL result resource in /path/to/my/smf/convert.php on line 1214

Notice: Undefined variable:  result in /path/to/my/smf/convert.php on line 2085

Notice: Undefined variable:  result in /path/to/my/smf/convert.php on line 2086

Warning: mysql_fetch_assoc(): supplied argument is not a valid MySQL result resource in /path/to/my/smf/convert.php on line 1225

Warning: mysql_free_result(): supplied argument is not a valid MySQL result resource in /path/to/my/smf/convert.php on line 1226

Notice: Undefined variable:  result in /path/to/my/smf/convert.php on line 2085

Notice: Undefined variable:  result in /path/to/my/smf/convert.php on line 2086

Warning: mysql_num_rows(): supplied argument is not a valid MySQL result resource in /path/to/my/smf/convert.php on line 1234

Warning: mysql_free_result(): supplied argument is not a valid MySQL result resource in /path/to/my/smf/convert.php on line 1236

Notice: Undefined variable:  result in /path/to/my/smf/convert.php on line 2085

Notice: Undefined variable:  result in /path/to/my/smf/convert.php on line 2086

Warning: mysql_fetch_assoc(): supplied argument is not a valid MySQL result resource in /path/to/my/smf/convert.php on line 1242

Warning: mysql_free_result(): supplied argument is not a valid MySQL result resource in /path/to/my/smf/convert.php on line 1243

Notice: Undefined variable:  result in /path/to/my/smf/convert.php on line 2085

Notice: Undefined variable:  result in /path/to/my/smf/convert.php on line 2086

Warning: mysql_fetch_assoc(): supplied argument is not a valid MySQL result resource in /path/to/my/smf/convert.php on line 1249

Warning: mysql_free_result(): supplied argument is not a valid MySQL result resource in /path/to/my/smf/convert.php on line 1250

Notice: Undefined variable:  result in /path/to/my/smf/convert.php on line 2085

Notice: Undefined variable:  result in /path/to/my/smf/convert.php on line 2086

Notice: Undefined variable:  result in /path/to/my/smf/convert.php on line 2085

Notice: Undefined variable:  result in /path/to/my/smf/convert.php on line 2086

Warning: mysql_fetch_assoc(): supplied argument is not a valid MySQL result resource in /path/to/my/smf/convert.php on line 1274

Warning: mysql_free_result(): supplied argument is not a valid MySQL result resource in /path/to/my/smf/convert.php on line 1276

Notice: Undefined variable:  result in /path/to/my/smf/convert.php on line 2085

Notice: Undefined variable:  result in /path/to/my/smf/convert.php on line 2086

Warning: mysql_fetch_assoc(): supplied argument is not a valid MySQL result resource in /path/to/my/smf/convert.php on line 1282

Warning: mysql_free_result(): supplied argument is not a valid MySQL result resource in /path/to/my/smf/convert.php on line 1294

Notice: Undefined variable:  result in /path/to/my/smf/convert.php on line 2085

Notice: Undefined variable:  result in /path/to/my/smf/convert.php on line 2086

Notice: Undefined variable:  result in /path/to/my/smf/convert.php on line 2085

Notice: Undefined variable:  result in /path/to/my/smf/convert.php on line 2086

Warning: mysql_fetch_assoc(): supplied argument is not a valid MySQL result resource in /path/to/my/smf/convert.php on line 1319

Warning: mysql_free_result(): supplied argument is not a valid MySQL result resource in /path/to/my/smf/convert.php on line 1321

Notice: Undefined variable:  result in /path/to/my/smf/convert.php on line 2085

Notice: Undefined variable:  result in /path/to/my/smf/convert.php on line 2086

Warning: mysql_num_rows(): supplied argument is not a valid MySQL result resource in /path/to/my/smf/convert.php on line 1367

Warning: mysql_fetch_assoc(): supplied argument is not a valid MySQL result resource in /path/to/my/smf/convert.php on line 1369

Warning: mysql_free_result(): supplied argument is not a valid MySQL result resource in /path/to/my/smf/convert.php on line 1381

Notice: Undefined variable:  result in /path/to/my/smf/convert.php on line 2085

Notice: Undefined variable:  result in /path/to/my/smf/convert.php on line 2086

Warning: mysql_fetch_assoc(): supplied argument is not a valid MySQL result resource in /path/to/my/smf/convert.php on line 1403

Warning: mysql_free_result(): supplied argument is not a valid MySQL result resource in /path/to/my/smf/convert.php on line 1408

Notice: Undefined variable:  result in /path/to/my/smf/convert.php on line 2085

Notice: Undefined variable:  result in /path/to/my/smf/convert.php on line 2086

Warning: mysql_fetch_assoc(): supplied argument is not a valid MySQL result resource in /path/to/my/smf/convert.php on line 1469

Warning: mysql_free_result(): supplied argument is not a valid MySQL result resource in /path/to/my/smf/convert.php on line 1471

Notice: Undefined variable:  result in /path/to/my/smf/convert.php on line 2085

Notice: Undefined variable:  result in /path/to/my/smf/convert.php on line 2086

Warning: mysql_fetch_assoc(): supplied argument is not a valid MySQL result resource in /path/to/my/smf/convert.php on line 1507

Warning: mysql_free_result(): supplied argument is not a valid MySQL result resource in /path/to/my/smf/convert.php on line 1526

Notice: Undefined variable:  result in /path/to/my/smf/convert.php on line 2085

Notice: Undefined variable:  result in /path/to/my/smf/convert.php on line 2086

Notice: Undefined variable:  result in /path/to/my/smf/convert.php on line 2085

Notice: Undefined variable:  result in /path/to/my/smf/convert.php on line 2086

Notice: Undefined variable:  result in /path/to/my/smf/convert.php on line 2085

Notice: Undefined variable:  result in /path/to/my/smf/convert.php on line 2086

Warning: mysql_fetch_row(): supplied argument is not a valid MySQL result resource in /path/to/my/smf/convert.php on line 1549

Warning: mysql_free_result(): supplied argument is not a valid MySQL result resource in /path/to/my/smf/convert.php on line 1550

Notice: Undefined variable:  result in /path/to/my/smf/convert.php on line 2085

Notice: Undefined variable:  result in /path/to/my/smf/convert.php on line 2086

Warning: mysql_fetch_assoc(): supplied argument is not a valid MySQL result resource in /path/to/my/smf/convert.php on line 2252

Notice: Undefined variable:  result in /path/to/my/smf/convert.php on line 2085

Notice: Undefined variable:  result in /path/to/my/smf/convert.php on line 2086

Notice: Undefined variable:  result in /path/to/my/smf/convert.php on line 2085

Notice: Undefined variable:  result in /path/to/my/smf/convert.php on line 2086

Warning: mysql_fetch_assoc(): supplied argument is not a valid MySQL result resource in /path/to/my/smf/convert.php on line 2252

Notice: Undefined variable:  result in /path/to/my/smf/convert.php on line 2085

Notice: Undefined variable:  result in /path/to/my/smf/convert.php on line 2086


Any idea how can I make web based convert.php to do more work at a time.?

Hey, I just noticed that your new convert.php has option to select which material you want to convert .. so can do conversion in parts ?
ie. so It won't touch other part.. ie. if I did posts at once then later can do private messages etc.. and it won't touch posts.. right ?

[Ricky.]

Just to add :
my phpBB is 2.0.17 ..but I don't think it is problm as I did a conversion on small board few months back and was smooth.
Still on this board whtver got converted was good.. only first later of signature are missing .. ( im using latest phpbb2_to_smf.sql as found in this phpBB2 thread)

Update:
Though ur new convert.php has option what to convert and what to not but it is converting everything even if you have not selected that area.

[JayBachatero]

The selective conversion is not working yet.  That converter is still in development.  About the errors.  I converter a few boards via CLI and it worked fine.  If you want you can IM me on AIM, Yahoo, MSN or Gtalk and see what's going on.

[Ricky.]

Is it possible that webased convert.php do conversion in one go.. all at once without thinking about server load ??

[JayBachatero]

No.  It it does then apache would time out and so on.  You have a pretty big forum so it will take a while.

[Ricky.]

Seems it is not going to be easy, initially conversion was pretty fast ie. around 10000 posts every 5 minute or so and now seems its converting only 250 posts a minute.

Earlier CPU usage was around 70% and now dropped to 10-20%, so what I see that somehow now its converting at slow rates. at this rate it will take around 33 more hours and practically this impossible for me to keep running my terminal uninterrupted to get process completed.

Any suggestion ?

[JayBachatero]

That's weird.  There must be something that is slowing it down.  How many posts in total have converted?

[Ricky.]

after straight 10hrs, only 259000 posts were converted.

Sadly, I got it reseted again .. has just started again and this time with measures so that it wont' get disturbed even if my connection drops

[Ricky.]

Well, first 100000 posts are done fast. I guess its now slowing down..
Any logic or reason that it might slow down after certain amount of posts getting processed ?

[SleePy]

Do you have a maximum queries per hour or other limits on your mysql user account?
I would see this being the only limitation here. Otherwise it could just be server usage. Do you know how much of the cpu is being used and for what?

[JayBachatero]

He has a dedicated box so I don't see this as an issue.  It might be something with the indexes for tha table.

[Ricky.]

Its going on and on ..

almost 12 hours completed since I had to start again and still posts are under conversion so may take many more hours as then PMS and other stuff would get convrted and I also heard that it will take quite some time when it will calculate forum stats.

[JayBachatero]

How many posts have converted?  Also what are the colation of the tables?  UTF-8 or something else.

[Ricky.]

They are UTF..
Anyways.. its done after 15 hours I guess..

But did not complete successfully but all posts are converted.. it exited on "Moderator" stuff. 

So smileys, board persmission, avatars etc. are left.
I think I can redo all remainining stuff myself.

I logged and recounted forum stats as they were also not updated.

So forum is done.. now I am doing customization. No relief yet..
A huge list of "THINGS to DO " is remainng before opening forum to member !