Advertisement:

Author Topic: Converting from YaBB 2.2.3  (Read 34335 times)

Offline trepanne

  • Semi-Newbie
  • *
  • Posts: 26
Converting from YaBB 2.2.3
« on: October 04, 2008, 12:05:13 AM »
OK, following sleepy's suggestion in this post, I first upgraded my YaBB2 RC forum to the current version (YaBB 2.2.3).  The upgrade seems to have gone fairly successfully.

Then I tried to convert the YaBB 2.2.3 forum to SMF.  This is not working.  It's failing with this error message:
Code: [Select]
Successful.
 Converting membergroups... Unsuccessful!
 This query:
DELETE FROM `forum`.permissions
 WHERE id_group > 4;
 Caused the error:
 
Table 'forum.permissions' doesn't exist

I'm using the convert.php file from this sticky post, and i'm using the yab22_to_smf.php file from this sticky post.

Thanks for helping!

Offline AzJose

  • Semi-Newbie
  • *
  • Posts: 40
  • Gender: Male
Re: Converting from YaBB 2.2.3
« Reply #1 on: October 04, 2008, 03:22:34 AM »
I used this one tonight on yabb 2.2.2 and worked barring a couple of minor issues.

This Message

Hopes this helps and hope the SMF guys don't mind me trying to help another user  :)

Offline trepanne

  • Semi-Newbie
  • *
  • Posts: 26
Re: Converting from YaBB 2.2.3
« Reply #2 on: October 04, 2008, 08:25:15 AM »
Well thanks a lot AzJose; I certainly appreciate all the help I can get.

As it turns out, though, my problem was careless user error... I had blown away the DB & recreated it empty, so of course the code wasn't able to find the table it was looking for.

But I did use the file you pointed me to, and it successfully converted my forum.  Hooray!

However, the conversion process did not succeed in maintaining the attachments from the old site.  Anybody got a suggestion for how to fix that?

TIA

Offline trepanne

  • Semi-Newbie
  • *
  • Posts: 26
Re: Converting from YaBB 2.2.3
« Reply #3 on: October 04, 2008, 09:20:53 AM »
It looks like the conversion process didn't copy over the attachment files, so I did that manually.  Still not working though.

As well, for posts that had been edited... both the original text and the edited text are deleted in the converted version.

SMF looks pretty jazzy; I'd like to get it up & running.  However, preserving fidelity of the post text and the attachments is about the highest priority I've got.

Any help would be much appreciated.

Offline SleePy

  • Site Team Lead
  • SMF Master
  • *
  • Posts: 29,939
  • Gender: Male
  • Thats his happy face.
    • jdarwood007 on GitHub
    • @jdarwood on Twitter
    • SleePy Code - My personal site
Re: Converting from YaBB 2.2.3
« Reply #4 on: October 05, 2008, 01:18:18 PM »
Was there data in the attachments table?

Jeremy D — Site Team / SMF Developer
Support the SMF Support team!
Profiles:
GitHub

Offline trepanne

  • Semi-Newbie
  • *
  • Posts: 26
Re: Converting from YaBB 2.2.3
« Reply #5 on: October 06, 2008, 05:49:53 PM »
Quote
Was there data in the attachments table?
nope, a complete blank:
Code: [Select]
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 7306909
Server version: 5.0.33-log FreeBSD port: mysql-server-5.0.33

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> use forum
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed

mysql> select * from attachments;
Empty set (0.00 sec)


Offline SleePy

  • Site Team Lead
  • SMF Master
  • *
  • Posts: 29,939
  • Gender: Male
  • Thats his happy face.
    • jdarwood007 on GitHub
    • @jdarwood on Twitter
    • SleePy Code - My personal site
Re: Converting from YaBB 2.2.3
« Reply #6 on: October 07, 2008, 12:25:37 PM »
Looks like that is our issue. It didn't even convert any attachments.

In your yabb files. In either Paths.pl or Variables/Paths.pl

Is there a line in there that has a "uploaddir" variable?
Jeremy D — Site Team / SMF Developer
Support the SMF Support team!
Profiles:
GitHub

Offline trepanne

  • Semi-Newbie
  • *
  • Posts: 26
Re: Converting from YaBB 2.2.3
« Reply #7 on: October 07, 2008, 12:32:02 PM »
Quote
In your yabb files. In either Paths.pl or Variables/Paths.pl
Is there a line in there that has a "uploaddir" variable?

Yes, it's set to ~/web/public/forum/Attachments (but actually an absolute path that is correct, not a relative path).

Offline SleePy

  • Site Team Lead
  • SMF Master
  • *
  • Posts: 29,939
  • Gender: Male
  • Thats his happy face.
    • jdarwood007 on GitHub
    • @jdarwood on Twitter
    • SleePy Code - My personal site
Re: Converting from YaBB 2.2.3
« Reply #8 on: October 09, 2008, 11:37:10 AM »
I am guessing that SMF is using the same path for attachments?

That would be the only explanation here.
Jeremy D — Site Team / SMF Developer
Support the SMF Support team!
Profiles:
GitHub

Offline trepanne

  • Semi-Newbie
  • *
  • Posts: 26
Re: Converting from YaBB 2.2.3
« Reply #9 on: October 09, 2008, 12:33:52 PM »
Quote
I am guessing that SMF is using the same path for attachments?
How would I verify?

Pretend that you're talking to a very slow child, and you won't be too far wrong...

Offline SleePy

  • Site Team Lead
  • SMF Master
  • *
  • Posts: 29,939
  • Gender: Male
  • Thats his happy face.
    • jdarwood007 on GitHub
    • @jdarwood on Twitter
    • SleePy Code - My personal site
Re: Converting from YaBB 2.2.3
« Reply #10 on: October 09, 2008, 12:50:09 PM »
In the SMF database go to the smf_settings table.
Search for the variable "attachmentUploadDir".

Is that the same path as your yabb one?
Jeremy D — Site Team / SMF Developer
Support the SMF Support team!
Profiles:
GitHub

Offline trepanne

  • Semi-Newbie
  • *
  • Posts: 26
Re: Converting from YaBB 2.2.3
« Reply #11 on: October 09, 2008, 04:43:26 PM »
Code: [Select]
In the SMF database go to the smf_settings table.
Search for the variable "attachmentUploadDir".

Is that the same path as your yabb one?
No, it isn't. The path is set to ~/web/public/smf/attachments (i.e. the correct path).

Offline SleePy

  • Site Team Lead
  • SMF Master
  • *
  • Posts: 29,939
  • Gender: Male
  • Thats his happy face.
    • jdarwood007 on GitHub
    • @jdarwood on Twitter
    • SleePy Code - My personal site
Re: Converting from YaBB 2.2.3
« Reply #12 on: October 09, 2008, 07:59:03 PM »
I am at a lost here then. There should of been data converted then.

If you go to your SMF database.
There is a smf_convert table.

Search by type and see if there is any one that exists with "msg_attach".
I wonder if they didn't even get transferred over from the original conversion.

Edit, I think I possibly see the bug now. Give me a second to make the changes.

Edit2, Try the attached one.
« Last Edit: October 09, 2008, 08:02:08 PM by SleePy »
Jeremy D — Site Team / SMF Developer
Support the SMF Support team!
Profiles:
GitHub

Offline trepanne

  • Semi-Newbie
  • *
  • Posts: 26
Re: Converting from YaBB 2.2.3
« Reply #13 on: October 09, 2008, 08:39:53 PM »
OK, I think we're getting somewhere.

Now the conversion process copies over a bunch of files into smf/attachments/ as you'd expect.  However, each file copied over is only 7.0K in size, and the original filename has been rewritten as something like
"1_d41d8cd98f00b204e9800998ecf8427e" (i.e. an incrementing index, then an underscore, then a hex hash, with no file extension).

I tried using diff on some of these files, and the ones I checked all were identical.  Examining the contents with cat, I see the desired filenames (along with some other metadata from PDF attachments) listed as data inside the files.

Does that help?

Thanks for the support, Sleepy.  Am I the only one using this converter?


Offline SleePy

  • Site Team Lead
  • SMF Master
  • *
  • Posts: 29,939
  • Gender: Male
  • Thats his happy face.
    • jdarwood007 on GitHub
    • @jdarwood on Twitter
    • SleePy Code - My personal site
Re: Converting from YaBB 2.2.3
« Reply #14 on: October 09, 2008, 09:27:18 PM »
Well that is the encoded file name. SMF encodes the file name for protection and security. It shouldn't be putting any data into the files. It keeps that stored in the database.

I don't think you are the only one using this converter. Although the 2.2 converter for Yabb is fairly new so keep that in mind. I rewrote it totally and I did make a few mistakes it looks like with attachments. I didn't have any attachments or didn't see any attachments in the test board I had :P

Try this file if you don't mind. I hopefully found that bug that made them all the same file size. I wasn't getting the correct file name copied over so it was bugging out.
Jeremy D — Site Team / SMF Developer
Support the SMF Support team!
Profiles:
GitHub

Offline trepanne

  • Semi-Newbie
  • *
  • Posts: 26
Re: Converting from YaBB 2.2.3
« Reply #15 on: October 09, 2008, 10:35:29 PM »
All right sleepy, this is working, thanks.

Now I've got the actual files stored inside smf/attachments/, and browsing the forum, they are properly linked to their posts.  They can be downloaded & viewed correctly.  So far as I can tell, this functionality is now working.  Thanks!

There's another bug I can see, though.  In posts which have been modified after initial posting, the modifications aren't getting converted over to SMF; instead the posts are truncated where they were modified.  As well, it looks like posts which contain certain kinds of whitespace characters are getting truncated there too.

I've got posts where text has been cut&pasted from HTML documents; looks like SMF truncates right at the newline.

Any idea what might be causing this?  Thanks again.  I really do appreciate the help; what I've seen of SMF really makes me want to convert.

Offline SleePy

  • Site Team Lead
  • SMF Master
  • *
  • Posts: 29,939
  • Gender: Male
  • Thats his happy face.
    • jdarwood007 on GitHub
    • @jdarwood on Twitter
    • SleePy Code - My personal site
Re: Converting from YaBB 2.2.3
« Reply #16 on: October 10, 2008, 12:30:34 AM »
Well that is odd.

In that sample text you sent me. One of those posts was edited right? I believe it was the 2nd or 3rd reply that was edited?

That seems odd though :|
Jeremy D — Site Team / SMF Developer
Support the SMF Support team!
Profiles:
GitHub

Offline trepanne

  • Semi-Newbie
  • *
  • Posts: 26
Re: Converting from YaBB 2.2.3
« Reply #17 on: October 10, 2008, 08:20:54 AM »
Quote
In that sample text you sent me. One of those posts was edited right? I believe it was the 2nd or 3rd reply that was edited?

I'm afraid you must have me confused with somebody else, because I haven't sent you any sample text.  Would it be helpful if I did so?

In addition to this "edited post" bug, I was also trying to figure out why the conversion process is truncating posts that contain newline characters cut&pasted from rendered web pages.

I tried cutting & pasting the offending text into a new post on SMF, and that was no problem.  It was just the conversion process that choked on this.

Offline SleePy

  • Site Team Lead
  • SMF Master
  • *
  • Posts: 29,939
  • Gender: Male
  • Thats his happy face.
    • jdarwood007 on GitHub
    • @jdarwood on Twitter
    • SleePy Code - My personal site
Re: Converting from YaBB 2.2.3
« Reply #18 on: October 10, 2008, 03:31:48 PM »
Oh sorry that was. To many people :P

It would help to see a regular post then an edited post. I don't get what could of changed that caused this, but we could try and fix it.

As well the sample text for the site thing. The converter isn't touching any line endings, so I wonder if its just an issue how SMF is able to handle it since it didn't go through its cleanup functions that posts normally go through.
Jeremy D — Site Team / SMF Developer
Support the SMF Support team!
Profiles:
GitHub

Offline trepanne

  • Semi-Newbie
  • *
  • Posts: 26
Re: Converting from YaBB 2.2.3
« Reply #19 on: October 11, 2008, 12:11:41 AM »
Sleepy,

I sent you a PM with some sample text that's getting mangled in conversion.

Thanks for the help!

Offline trepanne

  • Semi-Newbie
  • *
  • Posts: 26
Re: Converting from YaBB 2.2.3
« Reply #20 on: October 11, 2008, 12:54:00 AM »
Okay, following up on this:

We're discussing two apparent bugs:
1) The YaBB converter seems to truncate edited posts, and
2) The YaBB converter seems to truncate posts with certain kinds of whitespace.

I am unable to replicate problem (1).  I suspect that it's actually not true; it's just that the post edits contained bad whitespace, so got truncated.

The actual bug is (2).  I opened up a Python interpreter, then cut&pasted a post from the old YaBB forum that was getting truncated as a Python string so that I could analyze each character of the whitespace.

It looks like the conversion process is choking on '\xc2\xa0' in UTF-8, which is what an HTML ' ' translates to.

So instead of setting up my new SMF forum with UTF-8 character set, as I'd been doing, I instead tried setting it up as ISO-8859-1.  This seems to fix all the problems (although I'll be looking around for any more bugs).

I think I can get by with ISO-8859-1, because all the forum content is in English... but just a heads-up, it seems that the YaBB converter isn't compatible with UTF-8, if that's an issue for anybody else.

Thanks again for your support, it's much appreciated.

Offline SleePy

  • Site Team Lead
  • SMF Master
  • *
  • Posts: 29,939
  • Gender: Male
  • Thats his happy face.
    • jdarwood007 on GitHub
    • @jdarwood on Twitter
    • SleePy Code - My personal site
Re: Converting from YaBB 2.2.3
« Reply #21 on: October 13, 2008, 02:55:28 PM »
Hmmm..

Doesn't make much sense why the converter isn't understanding the UTF-8 characters.
I guess we could try to replace those characters with a preg_replace if you think that is going to work for you.

If you are using non latin characters, then you are bound to have issues if you are not using UTF-8 for SMF.
Jeremy D — Site Team / SMF Developer
Support the SMF Support team!
Profiles:
GitHub

Offline trepanne

  • Semi-Newbie
  • *
  • Posts: 26
Re: Converting from YaBB 2.2.3
« Reply #22 on: October 14, 2008, 12:18:29 AM »
Sleepy, my problem is basically solved.  I really don't need nonroman characters, so latin charset is good enough for me.  I'm just providing some feedback on what I found to help you or anybody else who encounters a similar problem in the future.

Thanks very much, you really helped me get it working.  We're subjecting SMF to serious testing in the process of switching over.

Offline schlafe

  • Newbie
  • *
  • Posts: 1
Re: Converting from YaBB 2.2.3
« Reply #23 on: February 21, 2014, 01:10:23 PM »
I am hesitant to start a new thread as my issue relates directly to the aforementioned problems.  I have almost 100% successfully converted a YaBB 2.5 AE forum over to SMF via IamTheBoy's instructions located here:http://www.simplemachines.org/community/index.php?topic=457735

I utilized the yabb24_to_smf.php and converter.php located in mrtoner's reply #11 here:
http://www.simplemachines.org/community/index.php?topic=344465.msg2778613#msg2778613

Direct link to yabb24_to_smf.php and converter.php zip file: http://www.simplemachines.org/community/index.php?action=dlattach;topic=344465.0;attach=156540

I can confirm the issue of posts getting truncated was resolved by using the  ISO-8859-1 standard rather than UTF-8.

Attachments have transferred and are properly associated with every post that has not been edited.

The only remaining issue is that attachments associated with YaBB posts that have been edited hare no longer properly associated with the posts and do not appear in the thread.

Any advice with regard to isolating the cause and potentially resolving the issue would be greatly appreciated.

Thank you,

josh