[SMF Converter] Invisionfree/ActiveBoards/iPBfree

[CF]

That means that the URL was entered incorrectly in hxxp:settings.py [nonactive]. The most likely cause is a missing slash at the end.

That, of course, fixed the problem and the cookie.txt file was generated.

I was then able to run the "members_topics_posts_smf.py" file, which finished much faster than I thought it would.  Just a few minutes in fact.  But it completed with verification of only slightly fewer posts than I manually counted (look at members table and add up total posts from each user for grand total).

Is it normal for the crawl to take such a short time?  (for slightly less than 20,000 posts and just fewer than 30 members)?

Next, I am trying a test run by importing the data into SMF 1.1.16 running locally on wamp  (I want to see and understand everything before I make the final decision to move to SMF, buy hosting, and go live).  It appears that I successfully imported the categories_fourms.sql and members.sql SQL batch files.  But when I try to import the topics_posts_smf.sql file, I get the following error:



Once again, any help would be appreciated.

Thanks

EDIT:

Additional Information:
-the "topics_posts_smf.sql" file is 6,925 kb and, when opened with universal viewer, seems to contain the posts contents.

[nneonneo]

Is it normal for the crawl to take such a short time?  (for slightly less than 20,000 posts and just fewer than 30 members)?

A board of that size can be crawled in less than 10 minutes if the server is cooperative. The crawler downloads each thread page and each member page, so the time taken is roughly proportional to the number of threads plus the number of users. If you have only a few threads with many pages each, the crawl can be very fast.

Next, I am trying a test run by importing the data into SMF 1.1.16 running locally on wamp  (I want to see and understand everything before I make the final decision to move to SMF, buy hosting, and go live).  It appears that I successfully imported the categories_fourms.sql and members.sql SQL batch files.  But when I try to import the topics_posts_smf.sql file, I get the following error:

[No data was received to import]

How big is the file? The default WAMP upload size is probably around 2MB, so if the topics_posts.sql file is bigger than that then the import will fail.

You have a few options:

1) Zip the file by itself (creating topics_posts.zip) and try uploading that. Zipping the file should shrink it by a substantial margin, up to 95% in many cases.
2) Edit the WAMP configuration to increase PHP's maximum file upload size (upload_max_filesize).
3) Use the provided split_sql.py utility (double-click to run) to split the SQL file into several smaller SQL files. Choose a filesize that is below the import limit; 800000 bytes (~780KB) usually works well.

I recommend trying the first option first, as it should work given your WAMP configuration.

[CF]

How big is the file? The default WAMP upload size is probably around 2MB, so if the topics_posts.sql file is bigger than that then the import will fail.

You have a few options:

1) Zip the file by itself (creating topics_posts.zip) and try uploading that. Zipping the file should shrink it by a substantial margin, up to 95% in many cases.
2) Edit the WAMP configuration to increase PHP's maximum file upload size (upload_max_filesize).
3) Use the provided split_sql.py utility (double-click to run) to split the SQL file into several smaller SQL files. Choose a filesize that is below the import limit; 800000 bytes (~780KB) usually works well.

I recommend trying the first option first, as it should work given your WAMP configuration.

Wow, right there on the image I uploaded, it says the maximum size and I never even noticed it.  Sorry, and thanks for being so patient. (Note: I edited my previous post with the size info while you were posting.)

I zipped the entire file and got it below the threshold.  I got a "Import has been successfully finished, 102 queries executed. (topics_posts_smf.sql.zip)" message, but the main forum is still showing zero threads and zero posts and the individual boards are empty when clicking from the main forum page.  However, threads and posts can be accessed by going through the " View the most recent posts on the forum" or the "More stats" link and clicking on the thread topics listed under "Top 10 Threads".


EDIT:  In the interim, I browsed through the threads I can access through the above-mentioned "back doors."  Wow.  Better transfer than I could have hoped for.  Formatting, photos, quotes . . . all seem good.  This is an awesome service you provide nneonneo. 

[nneonneo]

There are more instructions after the import; see Instructions.txt.

[CF]

There are more instructions after the import; see Instructions.txt.

Crap.  I thought I had read that somewhere.  I'm just full of fail today.

(But please see the edit to my above post.)


And that worked without a hitch.

[neojin3]

Dear nneonneo, my conversion from Zetaboards to SMF led to 3403 topics while my Zetaboards recount shows that I actually have 3542 topics. Any idea what would be the problem?
None of the sections are password protected.
There are certain sections that are accessible only to mods and 1 section that is only accessible to me.
There are no hidden sections but I'm not sure if limited access qualifies as "hidden".
Some topics are hidden and others are queued.

[nneonneo]

Can you find a specific topic in the conversion that went missing? It might be a miscount on ZetaBoards.

[neojin3]

I wouldn't even know where to start to look for that. Zetaboards has recounted twice, once automatically and once by my request, so I don't think it's a miscount. Could the crawler have missed topics by members who have been deleted? I have a lot of that.

Here's the conditions in case you missed some of my edited-in stuff:
None of the sections are password protected.
There are certain sections that are accessible only to mods and 1 section that is only accessible to me.
There are no hidden sections but I'm not sure if limited access qualifies as "hidden".
Some topics are hidden and others are queued.

[nneonneo]

Compare a few of the forums between Zetaboards and the new converted board. Set the number of topics per page to the same value so that you can tell whether the same topics show up in the same place.

EDIT: I've heard a fair bit about divergent counts with ZetaBoards, but so far, nobody has been able to find a specific topic which fails to be converted. If you can find such a topic, I can figure out how to fix the converter.

[neojin3]

I haven't gotten to installing the new board yet. I've encountered a problem installing.

An SMF-based free forum-hosting service is installing my data as one of their boards, but this is what the staff emailed me as the problem they have with my data: "This is not a full MySQL backup, please send me all the tables".

I gave them the following files:
1. topics_posts
2. members
3. categories_forums
What are they looking for?

[nneonneo]

They're looking for a full database backup.

Either
a) import into a local SMF installation (use XAMPP or something to install a local webserver), or
b) tell them to just import those into a blank board.

[neojin3]

Dear nneonneo, how long does this installation take usually, and how complicated is it? He hasn't gotten back to me in 2 days..

[neojin3]

The host is asking me what version of SMF the SQL files convert to..

[nneonneo]

1.1.x.

[Dream of Omnimaga]

It seems your script for Invisionfree might need to be updated for a few reasons:

1) All topics ends up being assigned to category #0.
2) The cookie script no longer works. It says it was unable to get the cookies. I thought this was because of the new Guest Welcome box feature IF added recently like the one on Zetaboards, but even after disabling it it still didn't work. As a result, before converting I had to manually create a blank cookie.txt file.
3) Would it be possible to allow parsing posts that has attachments in them? Your script choke on them and won't convert any topic containing attached images. It gives the following error:

Code Select Expand

ValueError: unconverted data remains:  <br />
<br />
<strong><span class='edit'>Attached File ( Number of downloads: 6 )</span></strong>
<br />
[url=http://z9.invisionfree.com/omnimaga/index.php?s=15c41ace77fe66124b05d32519c9af1b&amp;act=Attach&amp;type=post&amp;id=13584907' title='Download attachment]<img src='http://209.85.48.10/html/mime_types/quicktime.gif' border='0' alt='Attached File' />[/url]
&nbsp;[url=http://z9.invisionfree.com/omnimaga/index.php?s=15c41ace77fe66124b05d32519c9af1b&amp;act=Attach&amp;type=post&amp;id=13584907' title='Download attachment]GAME01.8xg [b](3.87 kb)[/b][/url]</div>

Invisionfree has no option to make attachments invisible and I don't want to delete those attachments completely already (for historical/nostalgia reasons, after the posts are converted, I want to keep the original board online, intact, as it has been since our move). I don't mind if it doesn't include the actual attachments, but just so it at least parse the post content above the attached files.

Note, I tried this on both SMF 1.1.5 (the required version) and 1.1.16 (the latest), after following all instructions to import topics and posts correctly, and it didn't work correctly. The Zetaboards conversion worked fine, though.

[nneonneo]

What version of the Invisionfree converter are you using? I just tried version 0.2.1 (the one posted on the first topic), and it correctly grabs the cookie, assigns all categories and converts posts containing both binary and image attachments.

Try using 'http://z<number>.invisionfree.com/yourforum/index.php?' as the URL. The s<number> servers sometimes behave strangely.

[Dream of Omnimaga]

I used the one attached in the 2nd post of the topic. No version number was specified in the readme. I effectively used z9 too.

Could it be because I use custom board wrappers and CSS on the two Invisionfree boards?

[nneonneo]

The version number is at the top of the file you run.

When you say "effectively", what do you mean? If you are trying to convert through a custom domain or some frame-wrapping domain, be aware that it may not work correctly or at all.

[Dream of Omnimaga]

I mean I had used z9.invisionfree.com, not s9.invisionfree.com. No custom domain name were used, either.

Could it be that I use Python 2.7 instead of 2.6? I couldn't find 2.6 anywhere...

[nneonneo]

2.7 works; it's what I'm using. Disable every customization and use the default theme, then try again.