[SMF Converter] Yabb - 2.1

[JayBachatero]

Try the one in the first page.

[evgenydeep]

Hello Jay,
Currently, on my forum Yabb 2.1 I have 
1 216 568 messages,
28 659 subj,
12 226 users.
I'm trying to convert all staff to SMF 1.1.4 (fresh install of course).
After 2 times(it took a few hours to fulfill), I have the same result - blank board. All user data converted well(it seems well).

Now I install 2 additional SMF :-) and trying to convert all data.

During converting there are numerous errors, like

Notice: Undefined index: convert_script in /home/forumgr/public_html/smf01/convert.php on line 580

Warning: Invalid argument supplied for foreach() in /home/forumgr/public_html/smf01/convert.php on line 583

Notice: Undefined index: convert_script in /home/forumgr/public_html/smf01/convert.php on line 588
Successful.
Recalculating forum statistics...
Notice: Undefined variable: result in /home/forumgr/public_html/smf01/convert.php on line 1827

Notice: Undefined variable: result in /home/forumgr/public_html/smf01/convert.php on line 1828

Warning: mysql_fetch_assoc(): supplied argument is not a valid MySQL result resource in /home/forumgr/public_html/smf01/convert.php on line 935

Warning: mysql_free_result(): supplied argument is not a valid MySQL result resource in /home/forumgr/public_html/smf01/convert.php on line 945

Notice: Undefined variable: result in /home/forumgr/public_html/smf01/convert.php on line 1827

Notice: Undefined variable: result in /home/forumgr/public_html/smf01/convert.php on line 1828

Warning: mysql_fetch_assoc(): supplied argument is not a valid MySQL result resource in /home/forumgr/public_html/smf01/convert.php on line 953

Warning: mysql_free_result(): supplied argument is not a valid MySQL result resource in /home/forumgr/public_html/smf01/convert.php on line 963

Notice: Undefined variable: result in /home/forumgr/public_html/smf01/convert.php on line 1827

Notice: Undefined variable: result in /home/forumgr/public_html/smf01/convert.php on line 1828

Warning: mysql_fetch_assoc(): supplied argument is not a valid MySQL result resource in /home/forumgr/public_html/smf01/convert.php on line 976

Warning: mysql_free_result(): supplied argument is not a valid MySQL result resource in /home/forumgr/public_html/smf01/convert.php on line 989

Notice: Undefined variable: result in /home/forumgr/public_html/smf01/convert.php on line 1827

Notice: Undefined variable: result in /home/forumgr/public_html/smf01/convert.php on line 1828

Warning: mysql_fetch_assoc(): supplied argument is not a valid MySQL result resource in /home/forumgr/public_html/smf01/convert.php on line 1035

Warning: mysql_free_result(): supplied argument is not a valid MySQL result resource in /home/forumgr/public_html/smf01/convert.php on line 1037

Notice: Undefined variable: result in /home/forumgr/public_html/smf01/convert.php on line 1827

Notice: Undefined variable: result in /home/forumgr/public_html/smf01/convert.php on line 1828

Warning: mysql_fetch_assoc(): supplied argument is not a valid MySQL result resource in /home/forumgr/public_html/smf01/convert.php on line 1043

Warning: mysql_free_result(): supplied argument is not a valid MySQL result resource in /home/forumgr/public_html/smf01/convert.php on line 1049

Notice: Undefined variable: result in /home/forumgr/public_html/smf01/convert.php on line 1827

Notice: Undefined variable: result in /home/forumgr/public_html/smf01/convert.php on line 1828

Warning: mysql_fetch_assoc(): supplied argument is not a valid MySQL result resource in /home/forumgr/public_html/smf01/convert.php on line 1060

Warning: mysql_free_result(): supplied argument is not a valid MySQL result resource in /home/forumgr/public_html/smf01/convert.php on line 1061

Notice: Undefined variable: result in /home/forumgr/public_html/smf01/convert.php on line 1827

Notice: Undefined variable: result in /home/forumgr/public_html/smf01/convert.php on line 1828

Warning: mysql_num_rows(): supplied argument is not a valid MySQL result resource in /home/forumgr/public_html/smf01/convert.php on line 1069

Warning: mysql_free_result(): supplied argument is not a valid MySQL result resource in /home/forumgr/public_html/smf01/convert.php on line 1071

Notice: Undefined variable: result in /home/forumgr/public_html/smf01/convert.php on line 1827

Notice: Undefined variable: result in /home/forumgr/public_html/smf01/convert.php on line 1828

Warning: mysql_fetch_assoc(): supplied argument is not a valid MySQL result resource in /home/forumgr/public_html/smf01/convert.php on line 1077

Warning: mysql_free_result(): supplied argument is not a valid MySQL result resource in /home/forumgr/public_html/smf01/convert.php on line 1078

Notice: Undefined variable: result in /home/forumgr/public_html/smf01/convert.php on line 1827

Notice: Undefined variable: result in /home/forumgr/public_html/smf01/convert.php on line 1828

Warning: mysql_fetch_assoc(): supplied argument is not a valid MySQL result resource in /home/forumgr/public_html/smf01/convert.php on line 1084

Warning: mysql_free_result(): supplied argument is not a valid MySQL result resource in /home/forumgr/public_html/smf01/convert.php on line 1085

Notice: Undefined variable: result in /home/forumgr/public_html/smf01/convert.php on line 1827

Notice: Undefined variable: result in /home/forumgr/public_html/smf01/convert.php on line 1828

Notice: Undefined variable: result in /home/forumgr/public_html/smf01/convert.php on line 1827

Notice: Undefined variable: result in /home/forumgr/public_html/smf01/convert.php on line 1828

Warning: mysql_fetch_assoc(): supplied argument is not a valid MySQL result resource in /home/forumgr/public_html/smf01/convert.php on line 1109

Warning: mysql_free_result(): supplied argument is not a valid MySQL result resource in /home/forumgr/public_html/smf01/convert.php on line 1111

Notice: Undefined variable: result in /home/forumgr/public_html/smf01/convert.php on line 1827

Notice: Undefined variable: result in /home/forumgr/public_html/smf01/convert.php on line 1828

Warning: mysql_fetch_assoc(): supplied argument is not a valid MySQL result resource in /home/forumgr/public_html/smf01/convert.php on line 1117

Warning: mysql_free_result(): supplied argument is not a valid MySQL result resource in /home/forumgr/public_html/smf01/convert.php on line 1129

Notice: Undefined variable: result in /home/forumgr/public_html/smf01/convert.php on line 1827

Notice: Undefined variable: result in /home/forumgr/public_html/smf01/convert.php on line 1828

Notice: Undefined variable: result in /home/forumgr/public_html/smf01/convert.php on line 1827

Notice: Undefined variable: result in /home/forumgr/public_html/smf01/convert.php on line 1828

Warning: mysql_fetch_assoc(): supplied argument is not a valid MySQL result resource in /home/forumgr/public_html/smf01/convert.php on line 1154

Warning: mysql_free_result(): supplied argument is not a valid MySQL result resource in /home/forumgr/public_html/smf01/convert.php on line 1156

Notice: Undefined variable: result in /home/forumgr/public_html/smf01/convert.php on line 1827

Notice: Undefined variable: result in /home/forumgr/public_html/smf01/convert.php on line 1828

Warning: mysql_fetch_row(): supplied argument is not a valid MySQL result resource in /home/forumgr/public_html/smf01/convert.php on line 1192

Warning: mysql_free_result(): supplied argument is not a valid MySQL result resource in /home/forumgr/public_html/smf01/convert.php on line 1193

Notice: Undefined variable: result in /home/forumgr/public_html/smf01/convert.php on line 1827

Notice: Undefined variable: result in /home/forumgr/public_html/smf01/convert.php on line 1828

Warning: mysql_fetch_assoc(): supplied argument is not a valid MySQL result resource in /home/forumgr/public_html/smf01/convert.php on line 1231

Warning: mysql_free_result(): supplied argument is not a valid MySQL result resource in /home/forumgr/public_html/smf01/convert.php on line 1236

Notice: Undefined variable: result in /home/forumgr/public_html/smf01/convert.php on line 1827

Notice: Undefined variable: result in /home/forumgr/public_html/smf01/convert.php on line 1828

Warning: mysql_fetch_assoc(): supplied argument is not a valid MySQL result resource in /home/forumgr/public_html/smf01/convert.php on line 1297

Warning: mysql_free_result(): supplied argument is not a valid MySQL result resource in /home/forumgr/public_html/smf01/convert.php on line 1299

Notice: Undefined variable: result in /home/forumgr/public_html/smf01/convert.php on line 1827

Notice: Undefined variable: result in /home/forumgr/public_html/smf01/convert.php on line 1828

Warning: mysql_fetch_assoc(): supplied argument is not a valid MySQL result resource in /home/forumgr/public_html/smf01/convert.php on line 1335

Warning: mysql_free_result(): supplied argument is not a valid MySQL result resource in /home/forumgr/public_html/smf01/convert.php on line 1354

Notice: Undefined variable: result in /home/forumgr/public_html/smf01/convert.php on line 1827

Notice: Undefined variable: result in /home/forumgr/public_html/smf01/convert.php on line 1828

Notice: Undefined variable: result in /home/forumgr/public_html/smf01/convert.php on line 1827

Notice: Undefined variable: result in /home/forumgr/public_html/smf01/convert.php on line 1828

Notice: Undefined variable: result in /home/forumgr/public_html/smf01/convert.php on line 1827

Notice: Undefined variable: result in /home/forumgr/public_html/smf01/convert.php on line 1828

Warning: mysql_fetch_row(): supplied argument is not a valid MySQL result resource in /home/forumgr/public_html/smf01/convert.php on line 1377

Warning: mysql_free_result(): supplied argument is not a valid MySQL result resource in /home/forumgr/public_html/smf01/convert.php on line 1378

In today converting one another type of error appeared(it caused stop converting process):
INSERT INTO `forumgr_smf01db`.smf_pm_recipients
(ID_PM, ID_MEMBER, labels, is_read)
SELECT pm.ID_PM, mem.ID_MEMBER, -1 AS labels, 1 AS is_read
FROM (`forumgr_smf01db`.smf_personal_messages AS pm, `forumgr_smf01db`.smf_members AS mem)
WHERE mem.memberName = pm.temp_toName
AND pm.temp_toName != '';
Caused the error:
Duplicate entry '1-462' for key 1

Is your convertor is for 1.1.3 only?
May be I have to insatll not 1.1.4, but 1.1.3?
Can you give me any suggestion - how to finish process?

[evgenydeep]

PS:
I tried 6 times, unfortunately, results are vary but not too much...
1) script works very slow, it takes 3-4 hours and after that time script end it's work with numerous of mistakes. After that SMF board became empty, only Category and subforms is viewable. No messages, no topics...

Is it possible to divide convert script into several scripts?

[evgenydeep]

pPS: it's became strange more and more ...
I ran convert yesterday at evening/night, now is midday and convert is still
"Converting personal messages... Incomplete"
it works is about 12-14 hours.

I have MySQL 4.1.22 on FreeBSD 5.3 x86.

I found that converting process begin working slowly when converting each of part is close to the end. E.g.
Converting personal messages... Incomplete.
Script converted ~150 000 messages quickly(totally is about 152 000), and last 2 000 processing by 10-20 pieces per request...

[JayBachatero]

Umm how many personal messages do you have in total?  Also it might be better running the conversion locally since you seem to have a pretty big forum.  Would run faster if you run it locally.

[evgenydeep]

I don't know how many personal messages on my forum.
For now:
1 390 120 messages
      30 144 subjects
      12 290 users
I have tried locally, but locally I have Apache 2.2.8+PHP 5.2.5+MySQL 5.0.51a installed in Win32.
I tried on win32... but your script doesn't work after first step. Only
   Converting membergroups... Successful.
   Converting members...
and nothing.
Also, on win32 there is problem with names(uppercase and lowcase )...
So... I prefer converting is better on production server.
I tried to run Lynx locally... but not too much faster.

Is it possible to divide converting script somehow? Divide converting members, converting personal messages, converting boards?

How do you think - is it possible that problems idue to old version of MySQL 4.1.22?

[evgenydeep]

I have other small forum, it is the same Yabb2.1, there test conversion passed well.
But with this one... :-(

I really like SMF... and mods for it, but... I already 3-4 days can't convert my yabb2.1 to smf :-(
I'm already began looking to vBulletin :-(

I not lost my hope for good migration to SMF yet...

[evgenydeep]

I run locally converting and I got the same problem as on production server.
When count of personal messages, inserted in MySQL reached ~145 000, then converting process became very slow. Only 10-100 rows inserted per one transaction.

[evgenydeep]

I guessed the problem is with accessing to 55 000 files(~220 MB), which located on one directory.
Jay, is it possible to add to script possibility to load files to the memory by portions... e.g per 100MB, and then access to them directly from RAM? It will speed up process, i'm sure.

[JayBachatero]

Ummmm.  If you can give me the FTP info I can try running the conversion and see what exactly is going on.  There must be something going on with a wiery or something.  So if I can get access to phpMyAdmin and FTP I can debug this for you and convert the forum. .

Converting from flatfile will always be an issue since you have to look at EVERY single file that ends with .abc and the parse the data and so on.  Maybe making PHP use more memory when converting PM's.  But it shouldn't be so much of a problem if posts were converted successfully.  That means that it should handle PMs as well w/o too much trouble.

[evgenydeep]

Thanks for your proposal!
Today is too late for me, I'll wait results of local converting tomorrow then write PM to you.

[JayBachatero]

Alright.  I should have some time to do it tomorrow while at work.  So just shoot me a PM if anything.

[evgenydeep]

"News" :-)
I waited 2 days of converting...
In table personal_messages was 705 000 rows.
After ~140 000 rows convert sript began work very slow and insert ~100rows each 25 seconds
With help of grep I aprroximately calculate how much of personal messages should contian my forum ~  350 000.
So, more than 700 000 rows looks strange. I have stopped converting.

I'll prepare FTP and rest data for you... right now then will PM you.

[SleePy]

I actually think it may be a bug in the conversion process why it does that.

It shouldn't slow down. Though it is possible jay coded in the timeout script to detect when sections take longer to convert due to slower queries and changed the size of data it is converting at once.

You could change the default size it is trying to use when converting PMs at a time as well.

As said I think a local convert would be better in the long run. You wouldn't affect server resources and when you are done you can just import the sql file to your mysql database and then setup your SMF forum to point to that database.

[evgenydeep]

I prepeared all things, I wait for NS update...

May be following data will be useful for someone who want to use convertor, especially in case of converting big boards. I collected this during exploring this forum and looking my log-files:

1) MySQL should not be runned under Strict mode
Simple way to decide problem it is to run query in MySQL:
SET @@global.sql_mode=''

2) If you see error message saying "MySQL server has gone away" probably this is:
#08S01Got a packet bigger than 'max_allowed_packet' bytes
http://dev.mysql.com/doc/refman/5.0/en/packet-too-large.html
Simple way to decide problem it is to run query in MySQL:
SET GLOBAL max_allowed_packet=10000000;

3) if your convert just finished and no new step appear  this may happened  due to
PHP Fatal error:  Maximum execution time of 30 seconds exceeded

How to fix: Edit the maximum_execution_time in your php.ini
Default is 30 seconds, I increased it to 300 seconds and works still fine.
Also I add to Apache settings: php_value max_execution_time 300
And don't forget to restart apache and php.

[evgenydeep]

Jay, I sent all info in PM

[SleePy]

Thanks for the info Useful for others to know

[b4sakenxx]

Hi everyone, I'm new to SMF and have been trying to convert from YaBB2.1
The farthest I've gotten is to the poll votes conversion where it gave me the error below:

Code Select Expand


Converting poll votes... Unsuccessful!
This query:
ALTER TABLE `ibaraki_smf`.smf_poll_choices
ADD PRIMARY KEY (ID_POLL, ID_CHOICE);
Caused the error:

Duplicate entry '16777215-0' for key 1

I found the file on page 15 of this topic that was supposed to fix the poll vote problem, but I never got that far with that file.  It always stopped at topics i think.

I've gone back to the file attached to the first post.  Any help would be greatly appreciated, thank you.

[SleePy]

Please do not post your question in multiple places.
As well if you do a search you will see that similar errors have been posted before and have been answered.

First off make sure you got the latest file which is attached to the first post in this topic.

Make sure for one you are not clicking continue and trying to speed up the process of conversion. If javascript is enabled on the browser it will automatically count down and continue the conversion at a nice rate to not put a big load on the server.

It shouldn't still be happening as a key like tha shouldn't be added :|

[JayBachatero]

Try this file and see if that fixes the poll issues.  http://www.simplemachines.org/community/index.php?topic=122338.msg1220273#msg1220273