YaBB2.1 -> SMF - Personal Messages Problem

[JMB]

After finally getting the forum converted (many thanks SleePy!), I've noticed that there are some issues with Personal Messages.

First, the date of all members PM's is December 31, 1969, 11:00:00 PM. Looking at the database, the msgtime field has 0 for all PM's for all members.

Second, only PM's in members inboxes where converted. All PM's in outboxes are missing. It also appears that PM's in members PM Storage are missing too.

Any ideas?

[ccbtimewiz]

That date is the default date when the date cannot be fetched.

I would say I could help you out, but at this point I'm afraid there is little you can do without having to do an immense amount of work.

[JMB]

Hi ccbtimewiz,

As I'm still in the testing phase, running everything locally, I don't mind doing the conversion from scratch again, if its something that can be fixed by editing the conversion scripts.

[JMB]

OK, I found an old version of yabb21_to_smf.php I'd used to convert a forum at the beginning of the year. The PM dates converted OK back then, so I compared the two files.

In cstep4, I changed:

Code Select Expand


'msgtime' => empty($userData[$i][2]) ? (int) $userData[$i][2] : '0',



to:

Code Select Expand


'msgtime' => $userData[$i][2],



which was in the old file.

I reran the converter and this time it populated the msgtime field properly.

So, that's one thing fixed. Still the Outbox and Storage PM's missing though.

[SleePy]

Thanks for the first one. the bug was I used "empty" instead of "!empty" in that code.

The converter forces the pms to not show in the outbox, it sets it as deleted by sender.

You can try this, I haven't tested:
Find:

Code Select Expand

                    'deleted_by_sender' => 1,

Change to:

Code Select Expand

                    'deleted_by_sender' => 0,

I am not sure of the repercussions of this though without testing it.

[JMB]

Hi SleePy.

Yep, !empty works. 

I tried changing deleted_by_sender to 0. It populates outboxes with sent messages that haven't been deleted by the receiver. That's a good start!  Is there any way to process the .outbox files and bring all the messages back into the outbox?

[SleePy]

All messages should be in the outbox now that the user has sent?
That change just tells SMF that the person who sent it didn't delete it, so now it shows in their outbox. So this means every message sent is now showing in their outbox.

[JMB]

All messages should be in the outbox now that the user has sent?
That change just tells SMF that the person who sent it didn't delete it, so now it shows in their outbox. So this means every message sent is now showing in their outbox.

No, because only messages in the .msg files where converted. So that's only messages that members had in their inbox. Those are now in the senders outbox, but some messages are missing from the outbox. The missing messages are ones that the receiver had deleted, so they never got converted as they weren't in the .msg files. Those missing messages are in the .outbox files that don't seem to get converted.

[SleePy]

Hmm, I don't know how easy it would be to fix that.
The message then no longer exists in the .msg file for the user it was sent to then. So the importing would need to check that out. Or it would need to add more messages into the pms table but force the messages to always either have been deleted by sender or receiver.

[JMB]

Yeah, something like that! Sounds a bit tricky, but I'm no programmer. 

Then there is the PM Storage box on YaBB. That's not getting converted either. I would think that would be more simple to do. Just import all the messages in the .imstore files and add them to the users inbox. Looks like the format of the .msg and .imstore files is the same.

[SleePy]

Are the messages in those files the ones that where missing from the outboxes of the others?
If that is true, then that is a little easier to fix.

[JMB]

I'm trying to get the logic of this clear in my head... 

YaBB has an Inbox, Outbox and Storage box for PM's. The Inbox messages are in .msg files, the Outbox messages in .outbox files and the Storage messages in .imstore files.

So far, all .msg files have been converted and messages put in members inboxes on SMF. Then we changed the deleted_by_sender to 0. This made sent messages appear in members outboxes, but only messages that the receiver hadn't deleted. OK, that's were we're at...

The .outbox files, contain all messages that the member has in their outbox, so there will be duplicates with the messages that have already been imported, the ones that the receiver hadn't deleted, the ones that are now in the outboxes. But all the rest of the messages that should be in the outboxes are in this file too.

The .imstore file is like an archive for messages and contains other messages not present in either .msg or .outbox files. As SMF doesn't appear to have an archive/storage bit for PM's, it makes sense that they get put into the users inbox.

I think I understand all that! Does it make sense to you?

[SleePy]

I think I may have a way to fix it..
Currently working on the yabb 2.2 converter. I will test it out with the new changes once I fix other issues and backport them.

[JMB]

Great news. Thanks SleePy. 

[JMB]

Hi SleePy,

Any updates or news on this issue?

[SleePy]

I was working on this. Finals is nearby though for the summer quarter so I haven't looked at it much.

You can try it if you want. I can't say if it works for sure as I haven't even tested it.
What I did to the conversion is have it check for either a .msg or .outbox file. If it found one then it worked on that file using a variable to decide if it was in the outbox or not to tell it what to do.

[JMB]

Great, thanks SleePy, I'll give it a try!

I've given myself a crash course in PHP and MySQL, so I'm getting the hang of all this. 

So, I may even be able to work a few things out myself! 

I'll give it a go and report back.

Thanks again!

[JMB]

OK, just tried that and it doesn't work. No PM's get converted at all. 

The change to this line seems to kill it:

Code Select Expand


if (strrchr($entry, '.') != '.msg' || strrchr($entry, '.') != '.outbox')


If I change it to:

Code Select Expand


if (strrchr($entry, '.') != '.outbox')


It converts all the posts in .outbox as expected. But having both expressions with the OR doesn't work at all...

[SleePy]

Oh, logic error. It should be and

Code Select Expand

            if (strrchr($entry, '.') != '.msg' && strrchr($entry, '.') != '.outbox')

Did it convert them correctly and put them in the outbox of that user btw?

[JMB]

Ah ha! Logic was never my strong-point! 

OK, that's better. The inbox looks fine.

The outbox has messages in it, but not the right ones. They appear to be copies of messages sent to that member that the sender has in their outbox, but with the the member listed as the sender! Does that make sense?

Example for a user called Admin.

In Admin's outbox are 16 messages, all with Admin's name next to them, as you would expect being in Admin's outbox. The messages are not ones sent by Admin, but by other members to Admin.

So, I'm assuming these are all the messages to Admin that are in other members outboxes.

[SleePy]

Ok try this.

Find:

Code Select Expand

                    'fromName' => substr(htmlspecialchars($userData[$i][0]), 0, 255),

Replace:

Code Select Expand

                    'fromName' => $is_sent ? htmlspecialchars(substr($entry, 0, -4)) : substr(htmlspecialchars($userData[$i][0]), 0, 255),

Find (Should only be a couple lines down from the first one):

Code Select Expand

                    'temp_toName' => htmlspecialchars(substr($entry, 0, -4)),

Replace:

Code Select Expand

                    'temp_toName' => $is_sent ? substr(htmlspecialchars($userData[$i][0]), 0, 255) : htmlspecialchars(substr($entry, 0, -4)),

That should fix it.

[JMB]

Tried that. No messages in the outbox at all... 

[SleePy]

Find:

Code Select Expand

                WHERE mem.memberName = pm.temp_toName
                    AND pm.temp_toName != ''
                    AND pm.deletedBySender != 0");

Replace:

Code Select Expand

                WHERE mem.memberName = pm.temp_toName
                    AND pm.temp_toName != ''");

[JMB]

OK, still no messages in Outbox and I've got some duplicates in Inbox.

For example a message from a member named Futile and next to it the same message from a member called Futile.ou, who is marked as a guest.

[JMB]

Any progress on this problem SleePy?

It's the final hurdle for me to be able to convert to SMF. 

[SleePy]

No I haven't had time to look into getting this done right.
I plan on it in the up coming days since finals are almost over.

[JMB]

No worries. Good luck with the finals! 

[JMB]

Any luck with figuring this one out SleePy? 

[SleePy]

I have had people say pms are converting correctly for the yabb 2.2 converter now after I did some work to it. I was able to get both the inbox and outbox to convert data.

I plan on posting that as a beta script by the end of the week most likely. Just need to hear back from 1 more person.
I could most likely back port some of the changes. Which would basically be just where the information was in the array. I should be able to pick that up off the yabb 2.1 converter.

[stealth8080]

so any news with outbox pm problem?

I just converted yabb 2.2.3 forums to smf 1.1.6 and users didn't have outbox messages.

how to fix it, thanks.

[SleePy]

Did you use the converter file attached to the Yabb 2.2 beta topic that is a sticky in this board?

[stealth8080]

Did you use the converter file attached to the Yabb 2.2 beta topic that is a sticky in this board?

yea, I used it a the messages weren't converted.

[SleePy]

Is there any data in the smf_personal_messages and smf_pm_recipients tables?

[JMB]

As you seem to have managed to get this working on the 2.2 converter, I made a copy of my board and upgraded to 2.2.3.

I ran the latest SMF converter and it stops right at the start with this error:

Converting membergroups... Unsuccessful!
This query:

REPLACE INTO `porcupin_ptfsmf`.smf_membergroups
(id_group, group_name, online_color, min_posts, stars)
VALUES (2, SUBSTRING('Global Moderator', 1, 80), SUBSTRING('blue', 1, 20), '-1', SUBSTRING('5#stargmod.gif', 1, 255)),
(1, SUBSTRING('PTF Administrator', 1, 80), SUBSTRING('red', 1, 20), '-1', SUBSTRING('5#staradmin.gif', 1, 255)),
(3, SUBSTRING('PTF Moderator', 1, 80), SUBSTRING('green', 1, 20), '-1', SUBSTRING('5#starmod.gif', 1, 255));

Caused the error:

Unknown column 'group_name' in 'field list'

Any ideas?

[SleePy]

Which file did you get?
Those columns it is trying to use, are SMF 2.0 columns.

[JMB]

Which file did you get?
Those columns it is trying to use, are SMF 2.0 columns.

I used the file for 2.2 in the sticky post.

[SleePy]

Oh sorry, I reattached the file. I accidentally attached the version for 2.0

[JMB]

Lol!

I'll try it again then!

[JMB]

I finally found some time to try this again. Converter ran really fast!! Converted the whole forum in under 8 minutes, compared to over 3 hours using the 2.1 converter! Well done on that! 

Most things worked fine, but still problems with PM's.

All PM's in the inbox have a date of either January 01, 1970, 12:00:00 AM or January 01, 1970, 12:00:02 AM.

All PM's in inbox have the IP address for the subject and the subject for the name of the sender.

Message text is OK.

The are no PM's in the outbox.

[SleePy]

Alright.

This one any better?
I was off on a couple indexes.

If after this we can't get the pms to convert the outboxes, I will use the dummy function I have there to do this I guess.

[JMB]

Cool that worked much better, PM's in the Inbox are fine. Outbox has some stuff in but it's not right.

Taking my Admin account as an example, all the PM's in the Outbox say they where sent from Admin, which is correct. However, the message itself seems to be the original message sent from another user that Admin was replying to and not Admin's reply.

The subject line doesn't have Re: at the start as a YaBB reply would have.

Only a few are there, probably the ones that the user who sent the message still has in their Outbox.

Also the Sent to: bit, omits the name, for example:

« Sent to: on: August 21, 2008, 08:11:29 AM »

[SleePy]

Hmm, Alright. I guess we do things the hard way...

I may need time to program this.