News:

Wondering if this will always be free?  See why free is better.

Main Menu

[PHP-Nuke] Conversion fixes and issues.

Started by Rumbaar, August 25, 2008, 08:47:14 AM

Previous topic - Next topic

Rumbaar

Well using the current smf_1-1-5_phpnuke_converter on a pre 7.0 version of php-nuke.

Test run, had to make two changes to the sql file for it not error out on mismatch integer issues.
From
v.vote_id AS ID_POLL

To
IF(v.vote_id <> 0, v.vote_id, 0) AS ID_POLL

From
p.post_edit_time AS modifiedTime

To
IF(p.post_edit_time <> 0, p.post_edit_time, 0) AS modifiedTime



But after all that I found the very first character of peoples signatures had been missed in the conversion process.  I can't see why in the sql file it would be doing this.
REPLACE(u.user_sig, '\n', '<br />') AS signatureAny assistance would be benefit when I go for a live conversion.

Or is it this piece of code?
$row['signature'] = substr($row['signature'], 1, 65534);Would changing the 1 to 0 be viable?  As I don't know why it's currently set to start 1 character in.
"An important reward for a job well done is a personal sense of worthwhile achievement."

[ Themes ]

ThorstenE

$row['signature'] = substr($row['signature'], 0, 65534);
should fix it..
I think the author has swapped mysql SUBSTR & php substr();  ;)

The mysql-version starts with '1' , the php-version with '1'

SleePy

Thanks Rumbaar.

I made the changes in my source files, I will commit them soon to svn..

Yes TE, I made a mistake with that :P php starts its index at 0, while mysql starts at 1.
Jeremy D ~ Site Team / SMF Developer ~ GitHub Profile ~ Join us on IRC @ Libera.chat/#smf ~ Support the SMF Support team!

Rumbaar

Sweet, I'll make the changes for the final test run and see how I go.   Also wasn't there an addition to the script to handle article/news posts for php-nuke?  I know I was suppose to look at that myself many months ago, but I'm lazy and you're such a skilled operator :)
"An important reward for a job well done is a personal sense of worthwhile achievement."

[ Themes ]

Rumbaar

Okay Sleepy looks like I found another glitch in the conversion of the person avatar path details in relation to server side avatars
$row['avatar'] = substr('gallery/' . $row['avatar'], 1, 255);

Once again it starts 1 character in and misses the first 'g' in gallery path

I assume this would fix it:
$row['avatar'] = substr('gallery/' . $row['avatar'], 0, 255);

I'm also hoping to be able to convert the nuke_stats_hour table into workable information to insert it into the smf_log_activity table.  I don't care about topics, posts,   registers,mostOn but would like to be able to insert hits and date.

Now it's stored:
date hits topics posts registers mostOn
2007-11-19  1

With hist across 24 hours (0-23)  How/what would be the best way to take all days hits and combine then.  Create the date entry based on that day and add the combined days hits.
"An important reward for a job well done is a personal sense of worthwhile achievement."

[ Themes ]

SleePy

How is it stored in php-nuke?

The best way I can think of would be to just do a $no_add = true; and manually prepare to insert the data yourself based on what you gain on data.
So you would only look at the day, month and year instead. If data already exists in the database for it, just update it and add to it, otherwise it would add a new fresh one.
Jeremy D ~ Site Team / SMF Developer ~ GitHub Profile ~ Join us on IRC @ Libera.chat/#smf ~ Support the SMF Support team!

Rumbaar

It's stored like I've defined in the last code box.  So you have a line for each hour of each day.  There is a daily stats table, but it doesn't seem to be populated with the daily hits.

There is currently nothing in the SMF side of the stats table.

But I might try an export and then subtotals summary in excel, then use that data as in UPDATE/INSERT.  I know how to manipulate data in that, than I do with PHP/mySQL.

Thx for your input.  All in all the conversion went well now, other than moving over previously deleted members.  But that it most likely due to the nature of the members management mod I had, that actually never totally removed them from the database.
"An important reward for a job well done is a personal sense of worthwhile achievement."

[ Themes ]

SleePy

That table information you showed me looks like the SMF log_activity table though?
Jeremy D ~ Site Team / SMF Developer ~ GitHub Profile ~ Join us on IRC @ Libera.chat/#smf ~ Support the SMF Support team!

Rumbaar

#8
So sorry Sleepy, yes that is the SMF table.  I'm sure I meant to post the php-nuke table originally.

Sample of a single days stats (my first day). nuke_stats_hour
year month date hour hits
2003  0
2003 6 5 1 0
2003 6 5 2 0
2003 6 5 3 0
2003 6 5 4 0
2003 6 5 5 0
2003 6 5 6 0
2003 6 5 7 0
2003 6 5 8 0
2003 6 5 9 0
2003 6 5 10 0
2003 6 5 11 0
2003 6 5 12 0
2003 6 5 13 0
2003 6 5 14 0
2003 6 5 15 0
2003 6 5 16 0
2003 6 5 17 0
2003 6 5 18 39
2003 6 5 19 36
2003 6 5 20 93
2003 6 5 21 46
2003 6 5 22 19
2003 6 5 23 52


What I was thinking of doing is a sub-total in excel on each change of day.  Then using those subtotals to populate the INSERT command to update the SMF's stats table.  Not sure how 'old' dates after new date will affect it though.

There is a day stats table (nuke_stats_date), but it's hits field isn't populated.  Which would have made it a lot easier
year month date hits
2003 1 1 0
2003 1 2 0
2003 1 3 0
"An important reward for a job well done is a personal sense of worthwhile achievement."

[ Themes ]

Rumbaar

#9
Okay looks like I've found another issue or thought.

The conversion and movement of avatars.  In php-nuke they are stored under a gallery folder and then sub-folders.  So the converter copies them and adds the 'gallery' folder again.  You get a sub-sub gallery folder.

So it becomes /gallery/gallery/folder/avatar.gif

As the SMF can't handle sub folders they are lost when a person tries to set the avatar again or even modify their account details.  I'll have to work out how to do this globally via a SQL command, but might want to look into this for the conversion process for others.

But I'm not sure how the galleries are handling in later or other php-nuke configurations.

I'm going to get around that issue by running this or a variant to remove the extra gallery sub-folders from the field
UPDATE `smf_members`
SET `avatar` = SUBSTRING(`avatar`,16,255)
WHERE `avatar` LIKE "gallery/gallery%"




Okay another issue, and this might also related to any phpBB conversion as well I haven't looked, but as SMF doesn't have ANNOUNCEMENT threads they seem to be ignored in the conversion process and lost.  I think at least turning them into stickies rather than just dumping them back to basic posts would be best.

So a change in the SQL code to also look for ANNOUNCEMENTS (type = 2)
---* {$to_prefix}topics
SELECT
t.topic_id AS ID_TOPIC, t.topic_type = 1 OR t.topic_type = 2 AS isSticky,




I've since updated my stats via export and tidy up via excel, then re-importation.  It's given me my desired daily hits stats.

I'm now looking at the conversion possibility in regards 'nuke news' entries.  Though it might still be beyond my php/SQL capabilities to make it automated.  The creation of topics via automation will be among my issues, as will combination and formatting of entries that might contain normally restricted code.


Finally during the copying of the avatar files/folders they were transferred with the final owner of 'nobody', which I'm not sure if this can be fixed.  As I had to get my host to CHOWN the files/folders to gain access to them again.
"An important reward for a job well done is a personal sense of worthwhile achievement."

[ Themes ]

SleePy

I think if we work with a GROUP BY and then COUNT, it may be possible to do this..
I am lazy here and don't want to manually create that table to test it though, Can I have a data dump :P
Working with PHP wouldn't be to hard though, its just a simple select to see if it exists under those where conditions, if it doesn't, insert it, otherwise update it.

For the avatar gallery. Does changing this help it?
elseif ($row['user_avatar_type'] == 3)
    $row['avatar'] = substr('gallery/' . $row['avatar'], 0, 255);


to:
elseif ($row['user_avatar_type'] == 3)
    $row['avatar'] = substr($row['avatar'], 0, 255);


Fixed the last one in phpnuke and phpbb2 and 3 converters (in SVN only though)

The copying of files can't be helped. That is because php is running as nobody ;)
Jeremy D ~ Site Team / SMF Developer ~ GitHub Profile ~ Join us on IRC @ Libera.chat/#smf ~ Support the SMF Support team!

Rumbaar

#11
I've since done all the updating, collating and inserting of the 'old' stats data now Sleepy.  But I can easily supply a data dump or sample data dump of the applicable 'old' nuke stats tables if you still want to look into this for future conversions?

That change probably would have helped, but I've moved on from the initial conversion.  I've moved the files server side and performed the SQL command to remove the extra gallery folder where applicable.

I doubt you're being lazy Sleepy, you can't be all things for all people.  So I appreciate the assistance.

Now for the 'nobody' issue can a note be added after the conversion process to inform people of the fact.  So they can be aware and address it or know why they can't delete files/folders that have been copied.  Some might not get the reason why they no longer have control of their files, and not know what to do to fix it.


I've also since extracted, converted and imported the nuke_stories (frontpage news) from my php-nuke db.  Now this was mixture of automated and manual action.  But with a combination of knowledge I'm sure it could be developed into a fully automated script run separately after conversion.

There are only 4 variables that need to be called upon.  Start ID_TOPIC, ID_MSG, ID_BOARD & ID_MEMBER.  Which could all be asked at run time, then populated automatically as it extracts the data from the stories table.  So the TOPIC and MSG would auto increment (unless there is alt method) from your first number (people will have input their last IDs).  The BOARD and MEMBER would be static and you can get the posterName/posterEmail/posterIP from the MEMBER ID.  CONCAT the two stories fields (HOMETEXT, BODYTEXT) and parse it for special characters (this is where I failed as well, for automation).

I'll have to learn more to be able to design this myself, I have the theory and idea in my head but lack of skill/knowledge to get it to code.


Okay found another one, which I found before but forgot about.  Each user_level in php-nuke as default as 1, and administrators are 2.  So the following code sets all base users to administrator group!
IF(u.user_level = 1, 1, mg.ID_GROUP) AS ID_GROUP,
Change too:
IF(u.user_level = 2, 1, 0) AS ID_GROUP,


As for the gallery issues there is another change that needs to be made, so it copies the phpBB avatars and sub-folders correctly.  As it has gallery as default and then other sub-folders with it.  When copying the whole directory, you don't need to add the gallery folder.
From
@mkdir($smf_avatar_directory . '/gallery', 0777);
copy_dir($phpbb_avatar_gallery_path, $smf_avatar_directory . '/gallery');


To
copy_dir($phpbb_avatar_gallery_path, $smf_avatar_directory);

On a second run and test, it seemed to copy the avatars correctly.  In base folder and all sub-folders on their own and accessible to SMF.
"An important reward for a job well done is a personal sense of worthwhile achievement."

[ Themes ]

Advertisement: