News:

Wondering if this will always be free?  See why free is better.

Main Menu

[WIP - Converter] IPB 3.2/3.3/3.4

Started by emanuele, May 17, 2012, 03:44:15 PM

Previous topic - Next topic

sorensonfan41

Good day!  I'm helping a friend convert his IPB 3.4.6 board to SMF 2.0.6.  I ran the converter and it seems like it was successful in converting the users, passwords, messages, posts, forums, etc.  All looks well....but with one issue....

On the IPB board, there's over 300,000 posts and over 23,000 topics.  When I finish the conversion process (which looks to be successful), I only find 18,246 posts and 1,259 topics were actually converted.  It adds up to almost a year's worth of posts and topics along with older topics that were "sticked" or "pinned".

I'd really like to be able to convert all 300,000+ posts and the 23,000+ topics successfully.  I think the script is somehow timing out and causing the script to advance to the next step....but it's not throwing any kind of error message (and the script runs really fast and completes really fast).

How would I be able to convert the rest of the posts?  There are two separate databases in use, one for IPB and one for SMF (but with same user with appropriate permissions on both databases).

Thank you for your hard work and thank you for any help you may be able to provide!

mentalist

QuoteIt adds up to almost a year's worth of posts and topics along with older topics that were "sticked" or "pinned".
Mhh, that sounds like you converted an older database instead of the live forum.. Are you sure you converted the proper database?

sorensonfan41

Quote from: TE on December 16, 2013, 10:47:04 PM
QuoteIt adds up to almost a year's worth of posts and topics along with older topics that were "sticked" or "pinned".
Mhh, that sounds like you converted an older database instead of the live forum.. Are you sure you converted the proper database?

It was the live IPB forum that was converted to SMF.  (and it's still live and going as I type this)

mentalist

that's odd...  the members have been fully converted?

chris319

It seems the threads "archived" by the IPB software is the problem. Unfortunately we have over 22,000 such "archived" threads.

Does anyone know enough about IPB archiving to know if the converter scripts can be made to unarchive these threads? It seems to take forever if you let the Invision software do it.

emanuele

Nope, never had a chance to see it, but... well, I suspect that if IPB gives you the tool to archive and "unarchive", it's better to let the creator of the script handle this part of the job for the best result. ;)


Take a peek at what I'm doing! ;D




Hai bisogno di supporto in Italiano?

Aiutateci ad aiutarvi: spiegate bene il vostro problema: no, "non funziona" non è una spiegazione!!
1) Cosa fai,
2) cosa ti aspetti,
3) cosa ottieni.

sorensonfan41

Ok, we've got everything converted over.  It was a success for the most part.  All the posts and topics are now there.  We had to manually clean up and reorganize a couple of things (no big deal).

Here's is one minor annoyance, though.....and it affects a great deal of posts.

Anywhere there's a quotation mark or an apostrophe, the converter added a \ before the respective symbol.  I looked through this thread and someone suggested a SQL command for such a problem on one of the previous pages.  I tried it in phpMyAdmin and it didn't do anything (no rows were modified....and it didn't mess anything up....but it also didn't fix anything).  Here's an example.....

QuoteI caught an interview the other night from CBS This Morning circa 1990-1991. Mark Goodson was talking about his company legacy, and how some games didn\'t seem to have the pizazz necessary to survive the 90s, but To Tell the Truth was going to try.



What game shows of the past have the hardest time finding relevancy today? The right star and venue could make anything a success, sure, but I don\'t imagine we\'ll see You Bet Your Life make a comeback anywhere near the form it was originally successful in.



As you start to go forward in the overall game show timeline, the question gets muddier. Is To Tell the Truth itself past the diminishing point of audience attention spans? For me personally, it\'s a show I run hot and cold on. Sometimes I\'m in the mood to play along, other times the questioning makes me reach for the remote.



We\'ve also talked about Password before. The last time it was on TV, it was jazzed up significantly from the earlier versions. But a lot of us respond to Jimmy Fallon\'s late night Password skits, and they do them with some regularity. Could a form of that work in the same vain as @midnight, perhaps?

Is there any easy way to fix that?  If there's nothing that can be done about it.....it's not like the forum is going to implode, quit working, or whatever.  If it can be fixed easily, though (without messing up the forum)....it definitely would be great.  Also, is there any way the extra space in between the paragraphs can be fixed?  If not, no big deal.

Thank you for your help so far and for all of your hard work in creating a usable conversion script for IPB to SMF!

emanuele

From phpMyAdmin:
UPDATE smf_messages SET body = REPLACE ( body, "\'","'" );


Take a peek at what I'm doing! ;D




Hai bisogno di supporto in Italiano?

Aiutateci ad aiutarvi: spiegate bene il vostro problema: no, "non funziona" non è una spiegazione!!
1) Cosa fai,
2) cosa ti aspetti,
3) cosa ottieni.

sorensonfan41

Quote from: emanuele on December 23, 2013, 09:12:21 AM
From phpMyAdmin:
UPDATE smf_messages SET body = REPLACE ( body, "\'","'" );

I tried that in phpMyAdmin and it says 0 rows were updated (and the posts are still showing up like that).

ToVarg

Hi! ipb v.3.3.3, smf 2.0.6 using сonverter shows the error "Unknown system variable 'SQL_MAX_JOIN_SIZE'". XAMPP on Mac OS X

emanuele

Quote from: sorensonfan41 on December 23, 2013, 04:47:37 PM
Quote from: emanuele on December 23, 2013, 09:12:21 AM
From phpMyAdmin:
UPDATE smf_messages SET body = REPLACE ( body, "\'","'" );

I tried that in phpMyAdmin and it says 0 rows were updated (and the posts are still showing up like that).
That's odd...
Pick a random message (for example the one you posted above), take note of the id (it's the number after msg in the link of the post, for example in http://www.simplemachines.org/community/index.php?topic=477019.msg3650014#msg3650014 the id is 3650014), go to phpmyadmin, find that message and post a screenshot of the result with (if possible) the entire message showing (or at least a piece including a \').


Take a peek at what I'm doing! ;D




Hai bisogno di supporto in Italiano?

Aiutateci ad aiutarvi: spiegate bene il vostro problema: no, "non funziona" non è una spiegazione!!
1) Cosa fai,
2) cosa ti aspetti,
3) cosa ottieni.

emanuele

Quote from: ToVarg on January 04, 2014, 04:57:29 AM
Hi! ipb v.3.3.3, smf 2.0.6 using сonverter shows the error "Unknown system variable 'SQL_MAX_JOIN_SIZE'". XAMPP on Mac OS X
No idea about the consequences, you can remove this block from convert.php:
// Attempt to allow big selects, only for mysql so far though.
if ($smcFunc['db_title'] == 'MySQL')
{
$results = $smcFunc['db_query']('', "SELECT @@SQL_BIG_SELECTS, @@SQL_MAX_JOIN_SIZE", array('security_override' => true));
list($big_selects, $sql_max_join) = $smcFunc['db_fetch_row']($results);

// Only waste a query if its worth it.
if (empty($big_selects) || ($big_selects != 1 && $big_selects != '1'))
$smcFunc['db_query']('', "SET @@SQL_BIG_SELECTS = 1", array('security_override' => true));

// Lets set MAX_JOIN_SIZE to something we should
if (empty($sql_max_join) || ($sql_max_join == '18446744073709551615' && $sql_max_join == '18446744073709551615'))
$smcFunc['db_query']('', "SET @@SQL_MAX_JOIN_SIZE = 18446744073709551615", array('security_override' => true));
}


Take a peek at what I'm doing! ;D




Hai bisogno di supporto in Italiano?

Aiutateci ad aiutarvi: spiegate bene il vostro problema: no, "non funziona" non è una spiegazione!!
1) Cosa fai,
2) cosa ti aspetti,
3) cosa ottieni.

sorensonfan41

#212
Quote from: emanuele on January 06, 2014, 03:10:31 PM
That's odd...
Pick a random message (for example the one you posted above), take note of the id (it's the number after msg in the link of the post, for example in http://www.simplemachines.org/community/index.php?topic=477019.msg3650014#msg3650014 the id is 3650014), go to phpmyadmin, find that message and post a screenshot of the result with (if possible) the entire message showing (or at least a piece including a \').

Here you go....(server names, complete database names, poster info/IP/email/etc. are blacked out)

emanuele



Take a peek at what I'm doing! ;D




Hai bisogno di supporto in Italiano?

Aiutateci ad aiutarvi: spiegate bene il vostro problema: no, "non funziona" non è una spiegazione!!
1) Cosa fai,
2) cosa ti aspetti,
3) cosa ottieni.

sorensonfan41

Quote from: emanuele on January 06, 2014, 05:18:49 PM
meh... that's SMF trying to be smart...
Use the query in this gist:
https://gist.github.com/emanuele45/8290814 [nofollow]

raw version:
https://gist.github.com/emanuele45/8290814/raw/4cb7896e91930d7dc26c16ee5700a7cc772a72fc/gistfile1.sql [nofollow]

I tried that a moment ago.  It affected 138,597 rows (and took 5.6446 seconds) -- that's between 1/3 and 1/2 of our messages table (307,788 as of the time of typing this).  I've been randomly going through the threads and it seems like it only affected the very old threads and didn't do anything for the newer threads.  I ran it again and it said 0 rows affected.

It also didn't fix the \s on quotation marks.

I wonder why it only affected the older posts and not the newer ones?

emanuele

Pick a "recent" message that has not been affected by the query and see what the db values look like.
With quotation marks you mean double quotes? "?
That requires a different query, but let's solve one at a time. ;)


Take a peek at what I'm doing! ;D




Hai bisogno di supporto in Italiano?

Aiutateci ad aiutarvi: spiegate bene il vostro problema: no, "non funziona" non è una spiegazione!!
1) Cosa fai,
2) cosa ti aspetti,
3) cosa ottieni.

sorensonfan41

Quote from: emanuele on January 07, 2014, 07:42:58 AM
Pick a "recent" message that has not been affected by the query and see what the db values look like.
With quotation marks you mean double quotes? "?
That requires a different query, but let's solve one at a time. ;)

Yes, double quotes as in "s.

Here's a pic of another message that wasn't affected after that SQL query that you gave me and I tried yesterday (after backing up the database, of course).  The message was from Nov. 2013.

brubsyoo

QuoteSorry, the database connection information used in the specified installation of SMF cannot access the installation of Invision Power Board 3.2. This may either mean that the installation doesn't exist, or that the Database account used does not have permissions to access it.

The error that was received from the Database was: SELECT command denied to user '[databaseprefix]_simplu'@'localhost' for table '[databaseprefix]members'

I got this, the converter doesn't run.
Also, i'm converting a 3.3.x board, is that the problem?

emanuele

Quote from: sorensonfan41 on January 07, 2014, 12:54:07 PM
Here's a pic of another message that wasn't affected after that SQL query that you gave me and I tried yesterday (after backing up the database, of course).  The message was from Nov. 2013.
mmm... okay, so it seems that some of the single quotes were converted from \' to \&# 039;
Try this query:
https://gist.github.com/emanuele45/8290814/raw/eaec53760d227964a9bdc1ef632814b66b06ddaf/gistfile1.sql


Take a peek at what I'm doing! ;D




Hai bisogno di supporto in Italiano?

Aiutateci ad aiutarvi: spiegate bene il vostro problema: no, "non funziona" non è una spiegazione!!
1) Cosa fai,
2) cosa ti aspetti,
3) cosa ottieni.

emanuele

Quote from: brubsyoo on January 07, 2014, 02:05:01 PM
QuoteSorry, the database connection information used in the specified installation of SMF cannot access the installation of Invision Power Board 3.2. This may either mean that the installation doesn't exist, or that the Database account used does not have permissions to access it.

The error that was received from the Database was: SELECT command denied to user '[databaseprefix]_simplu'@'localhost' for table '[databaseprefix]members'

I got this, the converter doesn't run.
Also, i'm converting a 3.3.x board, is that the problem?
As explained in the error there is a problem with the mysql users, the one you have used for SMF doesn't have access to the IPB installation, you should be able to give permissions (at least SELECT) from the control panel of your host.


Take a peek at what I'm doing! ;D




Hai bisogno di supporto in Italiano?

Aiutateci ad aiutarvi: spiegate bene il vostro problema: no, "non funziona" non è una spiegazione!!
1) Cosa fai,
2) cosa ti aspetti,
3) cosa ottieni.

Advertisement: