Blank page with IPB 2.0.x converter

[Jerigord]

I'm attempting to convert an IPB 2.0.x board to SMF and I'm running into a blank screen error.  I've tried three different versions of convert.php with the most recent being the one linked to in this post: http://www.simplemachines.org/community/index.php?topic=140741.0

I have also used the SQL file attached to the post at the top of this forum.  The first screen (where I define paths and enter the password) loads as expected.  After I submit that page, it runs for 30-45 seconds and then stops executing with a blank page.  I'm not getting any errors in the log and PHP errors are normally displayed on screen on my host (though I guess it could be a silent error).  Any thoughts?

[Jerigord]

Some additional information: It appears to be importing data since the SMF tables are populating with information from my IPB installation, but I'm still getting a blank screen.  I'm going one checked option at a time so I can monitor the process, but that's not going to be feasible when I get to the posts import.

[ThorstenE]

in most cases this is a timeout error from the webserver.. try this:

edit the invision2_to_smf.sql
find

Code Select Expand

---* {$to_prefix}messages
replace with

Code Select Expand

---* {$to_prefix}messages 200
the 200 means that 200 messages at the same time are converted (default is 500)
maybe this helps..

[Jerigord]

A good idea, except that the SQL I'm using already has the 200 in it.  :-(  It's also giving the blank screen even if I tell it to only do Members.

[ThorstenE]

do you have access to edit the php.ini from your webserver? if possible change the max_execution_timeout to a higher value...
another way would be to install a local webserver environment (XAMPP for example) and to convert your forums locally..

[Jerigord]

Unfortunately, I can't edit php.ini locally nor do I have SSH access to run it CLI, but installing the forums on my system temporarily and doing the conversion locally might be an option.  I'll give it a go.  Thanks for that idea.  While I'm attempting that, I'll still keep this thread around in case someone has any other ideas for what might be happening.

[Jerigord]

And it looks like it's not working 100% on my local install either.  I got WAMP running and reconfigured the two boards to run locally.  After that, I ran the converter as before.  It's not giving me a while screen now, but it is failing part way through the topics import with no message.  The screen refreshes with notes that members, categories, and boards were successful, but the topics line just reads "Converting topics..."  This is especially fun since I told it not to import any of those things.  It also doesn't appear to be automatically pausing every few hundred and prompting me to continue, but I may have imagined it would do that and it doesn't really work that way.

Can anyone confirm if this is a problem with the convert.php I've downloaded?  I've got the one from TE's signature.

[SleePy]

The topics conversion is refreshing? Are you sure it just hasn't finished converting those yet?

The selection of things to convert shouldn't be showing. I thought I fixed that long time ago to not show yet. That feature isn't ready/fully operational.

[Jerigord]

I'm definitely getting a list of checkboxes to select what to convert.  I downloaded the convert.php this morning, but I can grab another copy if I somehow got the wrong one.  After I put in the directories and password and submit the form, it processes for a very long time before refreshing the screen, at which point it shows the success messages along with the line about topics.  It then sits there and doesn't attempt to process anything nor does it attempt to reconnect.  It's effectively acting like a finished, rendered page...aside from the fact that the process didn't 100% finish.

I can't send a screenshot right now since I closed my window, but I can run it again when I go to dinner and get you the result afterward.  It takes 20-30 minutes to die and takes almost all of my system resources while it's running, but that's not surprising given the system I'm running it on.

[SleePy]

Ahh, Found the bug that made it display those. It was using or instead of and for a check.

How big is your IPB board?

[Jerigord]

The SQL dump is around 200 MB uncompressed.  It's probably a little bit less in DB, but that's a good rough idea.  It's not a small board, but I imagine there are bigger ones out there.  :-)

Is there a new version of convert.php I should try or should I just be patient?

[Jerigord]

Found the new convert.php in the thread.  I'll give it a go overnight and let you know how it works.

[SleePy]

Well if its 200 mb of data uncompressed I would say it most likely is a medium sized board (I was hoping for topic count, post count and member counts though).

It shouldn't take to long to convert that.
Check the mysql process tab to see if you notice any queries coming from convert.php during this process.
Also check the url, there is a substep which will increase each time it refreshes that lets it know where in the current step it is.

[Jerigord]

Approximately 100 members, 31,000 topics, 438,000 posts.  It's a private board we've been running for years and we don't generally prune old posts.

I ran into the same issue with the new convert.php.  It stopped at the topics step and the screen refreshed with "Converting topics..." and nothing afterward.  The URL has step=1 in it and no substeps.

[SleePy]

does it continue to refresh?

Check the mysql process tab to see if you notice any queries coming from convert.php during this process.

[Jerigord]

No, it doesn't continue to refresh.  Running a show full processlist is painful, but shows one dead query:

SELECT
        t.tid AS ID_TOPIC, t.pinned AS isSticky, t.forum [nofollow]_id AS ID_BOARD,
        t.starter_id AS ID_MEMBER_STARTED, t.last_poster_id AS ID_MEMBER_UPDATED,
        pl.pid [nofollow] AS ID_POLL, t.posts AS numReplies, t.views AS numViews,
        MIN(p.pid) AS ID_FIRST_MSG, MAX(p.pid) AS ID_LAST_MSG,
        t.state = 'closed' AS locked
FROM (`soultoa_ivbd1`._sttopics AS t, `soultoa_ivbd1`._stposts AS p)
        LEFT JOIN `soultoa_ivbd1`._stpolls AS pl ON (pl.tid = t.tid)
WHERE p.topic_id = t.tid
GROUP BY t.tid
HAVING ID_FIRST_MSG != 0
        AND ID_LAST_MSG != 0
LIMIT 2500, 500

That one hung out in there until I prematurely killed the conversion so I could actually post this message.  Do you need me to run an EXPLAIN on the query or check join counts or anything?

[SleePy]

Did mysql say if that query was copying to the temp table or not?

[Jerigord]

It didn't say anything about that as far as I could see.  It mentioned that the process was dead even though its execution time was continually increasing.

[SleePy]

Would be interesting to know.
That is a normal query it is running. That is in fact one of the queries that it would be running to try and get information about the topic so it can insert that into the smf_topics table.

Can you add EXPLAIN to the start of that as well then run it?
This should produce some detailed results of the query. If you could post them here I can see if they mean anything important.

It could also just be to many queries at once or to large of data that we are trying to grab from the topics table. Which we can lower the limit to a smaller number if you want to try that.

[Jerigord]

Sorry for the delay; I wasn't able to work on things this weekend.  The EXPLAIN doesn't show much, but maybe it will give you some insight.

+----+-------------+-------+------+---------------+----------+---------+---------------------+-------+---------------------------------+
| id | select_type | table | type | possible_keys | key      | key_len | ref                 | rows  | Extra                           |
+----+-------------+-------+------+---------------+----------+---------+---------------------+-------+---------------------------------+
|  1 | SIMPLE      | t     | ALL  | PRIMARY       | NULL     | NULL    | NULL                | 31032 | Using temporary; Using filesort |
|  1 | SIMPLE      | p     | ref  | topic_id      | topic_id | 4       | soultoa_ivbd1.t.tid |     6 | Using index                     |
|  1 | SIMPLE      | pl    | ALL  | NULL          | NULL     | NULL    | NULL                |   589 |                                 |
+----+-------------+-------+------+---------------+----------+---------+---------------------+-------+---------------------------------+

[SleePy]

I tells me we are using a file sort on the topics table.

The best thing I could suggest here is we use a smaller select to grab less data at once.

Find:

Code Select Expand

---* {$to_prefix}topics

Replace:

Code Select Expand

---* {$to_prefix}topics 200

This will make it select less at once and may help.

[Jerigord]

Still failing when I set it to 200.  :-(

[SleePy]

Does 100 or 50 give you the same results?

Do you have a really big post that either has a lot of data or a lot of posts?

[Jerigord]

50 gives the same result.  We might have a large post, but I'll have to do some searching to know for sure.

[SleePy]

If you look at the query that would be in the mysql process list, you will see a "LIMIT xxx, 50". What ever xxx is, is where it currently is at. That means the big topic is somewhere after that number. Those numbers since its the topic conversion would be the topic ids.

[Jerigord]

I just set the values down to 5 and ran the query manually in MySQL rather than through the browser.  It took 16 minutes to execute for just those five lines.  The counts were low as well, so it's not like we've got huge posts in the first five.  Any other ideas I can check?

[SleePy]

Well we could try to add some indexes topics table in ipb and see if that helps speed things up.

I don't have IPB so I am just going to say, try going to structure and adding a few indexes for some of the columns. How many columns you use in it is up to you.

[Jerigord]

I'll give it a go and let you know if anything improves.

[Oldiesmann]

Do you still need help with this Jerigord?