Like the title says. The converter is not converting over all Topics. The Stickied topics are not being converted and a number of Announcements also.
Furthermore, all of my users are Administrators and my Moderators and Admins are plain users. I can't get in to fix this since my account dosen't have the privelages.
Not to mention that the User registration dates are NOT being imported. Everyone is set at the year 1900.
Can someone help me with this?
for the first point:
try to search the topics who was sticky and announcement manually
the converter did not take the attributes, it convert all topics like normal ones; you should be able to found them, edit and provide them with sticky and or announc. attribute
I convert myself a phpBb forum about 100.000 posts and all goes well, users, private forums, permission, profiles, dates, etc.
try again, I tried 8 times until I succeed.
That's NOT the case. The missing topics are NOT there. Either they were converted wrong or SMF will not recognize them. Either way, they are not visible on the Front End.
I can now confirm that the convertor does NOT convert phpBB user Permissions over correctly. IT just copies them to the SMF members table.
However, I still don't have an explanation for the missing posts.
What happens if you click on "Find and Repair Errors" under "Forum Maintenance" in the admin center?
I don't know what to tell you about the dates.
OK, It claims that it has found threads & messages with no poster attatched to them. I told it to fix them and now the post count is close to what it was before.
The Dates thing was because of the extra tables manipulated by the PHP-Nuke team. I did a conversion to phpBB standalone and then to SMF.
I still havent figured out the Administrators problem. ALSO because phpBB allows unusual characters in the user name, its probabaly creating errors.
I don't have enough MYSQL experience to write my own querys. Can anyone help me out? Please. If I don't get these problems fixed, I'm gonna have no other choice but to stick with phpBB.
I can help you out.
As far as the usernames are concerned - we have a couple of options here. We can either replace the strange characters with something else, or we can strip them out - your choice...
Admins - to make a user an adminsitrator:
Click on their username (to go to their profile).
Click on "Account Related Settings".
Under "Primary Group", select "Administrator".
Click "Update Profile".
I know about those options BUT since I have about 500 members, that will take WEEKS to finish manually.
IS there some kind of SQL command that I can run to make this quick?
What strange characters are you worried about? Yes, it can be done quickly - but what exactly needs to be done quickly?
-[Unknown]
Well I read in another thread about usernames having spaces and other certain characters could cause SMF to act weirdly. But I didn't find a way to fix it. Since my SQL knowledge is limited, I wouldn't know how to fix it other than a PHP script that someone else would have written.
It should be fine. Are you having any problems there, or are you just worried?
Some of the 1.0.x converters didn't do statistics right at the end. Repairing and recounting will fix that. I hope to have this completely resolved for 1.1.
It's also possible some of the topics may have been damaged in your phpBB database.
-[Unknown]
Quote from: [Unknown] on April 19, 2005, 10:16:35 PM
It should be fine. Are you having any problems there, or are you just worried?
Some of the 1.0.x converters didn't do statistics right at the end. Repairing and recounting will fix that. I hope to have this completely resolved for 1.1.
It's also possible some of the topics may have been damaged in your phpBB database.
-[Unknown]
I was just concerned that the names thing might mess something up.
Also I have no Idea if something is messed up in phpBB since there is no real way to tell that I know of. phpBB doesen't have a repair script like the great SMF.
2 days ago tried migration with phpbb2_to_smf.php. so far so good, sticky in phpbb are sticky in smf, but announcement was not converted as sticky.
by the way, so far the best convertor from phpbb (ipb and vb have more glitches). 450k+ messages in less than 10 hours (dual xeon 733 + 1g ram + scsi raid5, debian testing, apache1.xx).
if there was a way to do it not from the web server - something shell-like or even a win32 app which would work with sql dumps... but 10-12 hours of downtime is far too much. or maybe "incremental" install?
incremental - split the script, first import the users, then the threads/postings. i can stop for a day or two the new user's registration. and if i can prepare everything and switch quickly, having only to "fill up" the last day or two messages, it will be nice.
wwell edi
PS unknown, 1.1 version of the script or the board? when (if soon we may wait a little for the migration)
PS/2 great board. let's kill that blackhole phpbb2, we do already have a new king.
As for the time needed to convert, your estimation is far to high. I did the complete switch last Saturday with a phpBB database of 80K+ postings.
1. Installed SMF on new server at new hosting provider (old forum still alive)
2. Closing of old forum
3. Dumped phpBB database from old server of old hosting provider to local (5 min)
4. Uploaded old_phpBB.sql to new server (40 min, due to slow upload connection)
5. Used bigdump.php to build phpBB database with old data in the same database as smf (10 min)
6. Used the converter online to convert data from phpBB to smf (roughly 5 min)
7. Opening of new forum (if you don't care for permissions, etc.)
So the minimum time the forum was out of order was 60 min for 80K postings. Actually I used more than 120 minutes because I had trouble with permissions and groups. But the minimum time counted for your database should be around 6-7 hours altogether. Faster if you have a faster upload connection than me (128KBit/s) and a faster server (plain 2.4 P4 with 80G IDE harddisk, 512M memory).
wobo
10x for the hardware hint, i got a p4/3.06/2g/(p) ata5 for the "server" - im using the dual xeon, which is p3/733 only for the conversion. maybe it will be wise if i do lock down the real host, and do the conversion right on site. no, it is not about the connection, lucikly, i do have a quite quick link to my hosting place - about 500-600 kbytes/s, so it is not a problem.
anyway, it seems that the cpu speed is most important in that case.
i will do some other tests. the difference between
Quote
6. Used the converter online to convert data from phpBB to smf (roughly 5 min)
and more than 5 hours, NOT COUNTING sql imports et cetera, "pure" converter time... is far too big.
how big was the dump?
mine is around
gzip -l bgmamma-mysql-bgmamma-dump-20050504-0240-EEST.gz
compressed uncompressed ratio uncompressed_name
135603130 758994365 82.1% bgmamma-mysql-bgmamma-dump-20050504-0240-EEST
this is with incomplete search index (the problems forcing me to migrate was primairly the almost-not-functioning search system).
if anyone else have timings about migration on a know hardware, please, share ;)
wwell edi
i can and will check the times again, tonite or at the weekend.
Quote from: ediww on May 04, 2005, 05:49:46 AM
how big was the dump?
mine is around uncompressed 135603130
this is with incomplete search index (the problems forcing me to migrate was primairly the almost-not-functioning search system).
Ah! I did not dump any search related tables! That reduced my 190MB dump to a mere 120MB! You don't need those tables anyway when moving the database. It is even recommended to clean them out every now and then.
wobo
wobo,
i've edited my post after posting - uncompressed size is over 700 megs, w/io the search index. the index is huge, that's right, but in this archive there is no index file.
wwell edi
I feel like such a newbie compared to you all. LOL
I tried to switch from phpBB to SMF and it wasn't working with the conversion tool. I had no access to my database(s), so I finally just gave up and switched hosts. Now - how do I take my phpbb database backup file (generated by phpBB) and put it into SMF?
Should I upload the data I downloaded from my previous installation of phpBB first, then install SMF and attempt the conversion again?
Did you do a complete dump of your phpBB database (with structure) ? If you are unsure, open the file of the dump and look, if it starts with "CREATE TABLE" (after some informational chitchat). If so, it is a complete dump.
All you have to do is upload the dump file to your new server (anywhere there) and create the phpBB tables IN THE SAME DATABASE as your smf tables. As most servers don't allow for doing this in one single command you may use some tool (like bigdump.php) for this.
After you have both, smf and phpBB in the same database, you can start the conversion tool. After that you can erase the phpBB tables.
Beware for one thing! If in your phpBB forum you have moved threads from one board to another and you left the hint in the old board about the movement, these placeholders will come up in SMF as empty threads with '-1' as number of replies. Search for this in the database and erase all lines with the '-1'. This kept me "searching for lost replies" for nearly 2 hours!
wobo
I did get it worked out. :) I switched to a new host and they ended up dumping my database (full backup, by the way, and it was 313MB!). I did get convert - and now everyone's happy! :D
today was the D-day. no turning back.
500k+ postings, 5k+ users. phpmyadmin says for both bgmamma and smf (they are still in one database)
64 table(s) Sum 11,142,906 -- 824.7 MB 5.3 MB
debian sarge (current testing), apache 1.3.33, local mysql server (4.0.24), php4.3.10, not counting any uploads et cetera
p4 3.06,2gb of ram, pata disks with softwre raid1, with 27MB/s (hdparms says).
done in around 2,5 hours (05:30 am to 08:00 am), "pure" converter time, tweaked the script a little to avoid the timed wait
setTimeout("doAutoSubmit();", 0);
on line 2928
since was the only user.
(it is not a good idea to do so, except if you have many posts and you are the only user, and you are in a hurry).
SO:
I.for sure there are missing topics. strange: i've got the same number of messages, but i really can not find the messages from a particular topic. i've still got both my boards running, if anyone of the devs is interested, can show where and what.
II.second problem, if in phpbb2 is a "topic" which is only a link to a moved topic (numviews=-1), it gets converted but points to nowhere (generates error).
III. the announcement topics. better, leave them just as sticky.
IV.there is a problem with the polls:
2: max(): Array must contain atleast one element in Poll.php line 362. going to check it right now.
i guess it is because nobody voted or something, but it is not for that topic.
that's all. going right now to see that poll s**t. so far so good.
V. duplicate username resulted from "illegal" char.
in phpbb i've had two different users with usernames "eli" and 'eli===". the second got converted as "Eli" also, resulting with two users with same username, and different ID_MEMBER
ediww,
Sorry that you've had some problems, I'll try to address some of them:
Firstly, could you go to the SMF admin centre and run "Find and Fix any Errors". This should sort out the moved topics, hopefully also the polls and maybe the other bits. The poll error is because their don't seem to be any options for that poll - so maybe the conversion hasn't fixed that?
As for usernames with duplicate names - we really should of protected against that! I'm afraid you will have to manually edit that using phpMyAdmin - although we could give you another converter to fix these things if you don't mind running it again?
I'm concerned about the "missing" messages - could you run the repair error thing and let us know how that got on.
Cheers,
Grudge
grudge,
so far, and it becomes fun to dig. i've got a problem with the polls, and i've manually fixed it - there was only one really important poll. there was entries for who voted, the poll existed, but the options (w/o &'" or any special chars) was missing.
i've put by hand them in the db and everything is ok (it was a poll about choosing a moderator so it was important to do quickly). so, they are suspicious to me, but i will check them better later. for now, maybe the poll import have problems, but in that moment i do not have a problem with poll import ;)
but with the topics...
i've got two, not, 3 types of problems.
1 is solved already- this was the "moved" pointers. just checked for topics with -1 replies and removed them.
2
next, i've pulled from the database the topic id's from the phpbb and smf (all who have moved field = 0). removed all the new topics (tbe board is live) and compared the id's.
then, pulled the message (post) id's from the database, and checked the numbers, topic per topic.
nd i've got about 300 topics, which have the messages imported but the record in the topics database is missing. checked with simple shell scripts, no error with the numbers. did not check the content, but several random picked are equal.
3 th i've got 2 topics which have messages in phpbb but the id's are not the same in the smf and phpbb2.
i will check with the moderators, it is possible that these topics was one, but split to two.
wwell edi
PS i will leave the repair options for now.. is there a doc explaining what is done with them in detail somewhere - or i must check the sources?
ediww,
Glad to see you are working through it :)
The most important problem seems to be these messages without topic entries. I assume you are saying that the messages table has entries with an ID_TOPIC value, but no entry in the topics table exists with this value - very odd.
I do suggest you look at the repair options - the source code is in RepairBoards.php. I would first say backing up my be advised before trying to do it. In the case of messages without topic entries, repair will actually CREATE those entries for you, and put them in a "salvage board". You can then check they are correct and move them into the relevant actual boards. Of course you could edit RepairBoards.php to make sure they just get put in the correct board, as ID_BOARD is now stored in the messages table.
On a side note, to help us fix these problems in the future, could you please post an example line from phpBB which *didn't* get converted over (Assuming your phpBB install is still in place). For example, if a message with a topic_id of 364 didn't have the topics entry created, do:
SELECT * FROM phpBB_topics WHERE topic_id = 364
And post the result here, so I can see if there was an obvious reason for the error.
For the other problems, personally I would have removed the "moved topics" pointers by deleting entries with a moved_topic_id value which wasn't equal to 0, as there is a *chance* that phpBB could put the wrong numReplies value in for a topic which wasn't moved.
The third problem does sound like someones split the topic - especialy if one of the numbers is very high compared to the other. The converter desn't do any clever stuff creating additional entries at conversion.
With the polls are you saying that any options with special characters in them were missing? You have any examples of what characters they were?
Cheers,
Matt
grudge,
i've fixed the missing topics "by hand" - using phpmyadmin, and some mysqldump|grep, i've managed to make the missing entries quite easily, fixing the posts/forums and polls attached. i really don't wanted to sort out 315 threads by hand - the effort would be at least the same. everything is just fine after that sewing finished;) i will double-check with the repair script.
If you need something, please, pm me and tell me exactly what (and where to send)- i do still have both smf and phpbb databases.
actually, i will perform an test import on other machine tomorrow if i do have some time, from the same phpbb2 dump. it will be interesting to see if the missing things will be the same.
but, let me know if this will be of any help (i.e. someone will fix something in the import scripts).
wwell edi
PS just to make it clear: the import _IS_ good, and in the matter of fact, it is much cleaner than the result from other boards import scripts.
After having moved my phpBB to SMF - although not quite as large a beast as edi's! - I had similar results:
1. Missing messages
Strolling through the new smf I realized there were topics with a reply number of '-1'. After checking both phpBB and smf database I realized that all those topics were topics whose messages were moved to another board. In phpBB you could leave a "shadow" topic which - on klick - redirects the user to the new location of the thread. These 'shadow' topics are the ones which turned up with "NoOfReplies=-1" in SMF.
2. Polls
All converted polls which were closed befor convertion contained no data, neither the questions nor the results. The only data displayed are the total of given votes. I have not found any cause or solution to this, only the common fact that it effects only closed polls. A bit annoying but not really a killer.
That was all there was. Everything else was converted. There were no problems with duplicate names because of special chars.
wobo
The moved topics (replies = -1) - how should they be converted? Should a message be made for them, or should they be ignored?
-[Unknown]
In the converter for 1.1, I made it so moved topic notices would just be ignored. The alternative would be to make a topic out of them with a "moved" notice, but then you'd have to hardcode the text for said topic, making it annoying for non-english forums.
in phpbb you can have some special chars in the usernames - like & ' et cetera. they do convert as html'zed entities. if it can not be allowed, let's at least have some info about these users, some list or, maybe, email message to the users affected? i've checked the login username before and after conversion (using the phpbb's and the new, smf tables) and did get the differences... but after user compliant.
"-1" topics are pointers. if you move, in phpbb, a topic, you have a choice to leave a "pointer" - which is in fact a link to the moved topic. not a bad feature, imho. it will not be bad to have similar thing in smf.
wwell edi
ok I did a creepy script that will replace phpBB shadows (pointers) with a move topic:
[convertShadows.php]
<?php
include_once 'Settings.php';
mysql_connect($db_server, $db_user, $db_passwd);
mysql_select_db($db_name);
$fetch = mysql_query("
SELECT ID_TOPIC, ID_BOARD, ID_FIRST_MSG
FROM {$db_prefix}topics
WHERE numReplies = -1 AND ID_FIRST_MSG = ID_LAST_MSG
")or die(mysql_error());
$i = 0;
$move = 'moved'; //Change it to your own language
while ($row = mysql_fetch_assoc($fetch)) {
$fetch_2 = mysql_query("
SELECT * FROM {$db_prefix}messages
WHERE ID_MSG=$row[ID_FIRST_MSG] LIMIT 1
")or die(mysql_error());
$msg = mysql_fetch_assoc($fetch_2);
$topicID_fetch = mysql_query("
SELECT ID_TOPIC FROM {$db_prefix}topics
WHERE ID_LAST_MSG='$msg[ID_MSG]'
AND numReplies != -1
LIMIT 1
")or die(mysql_error());
$topic_id = mysql_fetch_row($topicID_fetch);
$topic_id = $topic_id[0];
mysql_query("
INSERT INTO {$db_prefix}messages
VALUES(NULL, $row[ID_TOPIC], $row[ID_BOARD], $msg[posterTime], $msg[ID_MEMBER],
'$move: $msg[subject]', '$msg[posterName]', '$msg[posterEmail]', '$msg[posterIP]',
$msg[smileysEnabled], $msg[modifiedTime], '$msg[modifiedName]',
'[iurl]$boardurl/index.php?topic=$topic_id.0[/iurl]', 'moved')
")or die(mysql_error());
$id_fetch = mysql_query("
SELECT ID_MSG FROM {$db_prefix}messages
WHERE posterTime = $msg[posterTime] AND
body = '[iurl]$boardurl/index.php?topic=$topic_id.0[/iurl]'
AND icon = 'moved'
LIMIT 1
")or die(mysql_error());
$last_id = mysql_fetch_row($id_fetch);
mysql_query("
UPDATE {$db_prefix}topics
SET
ID_FIRST_MSG=$last_id[0],
ID_LAST_MSG=$last_id[0],
numReplies=0
WHERE ID_TOPIC=$row[ID_TOPIC]
")or die(mysql_error());
$i++;
}
echo "<b>$i</b> Shadow topics where converted succesfully!";
?>
you have to put this script in the root dir of SMF and run it ONCE!
just change the value of $move to your language..
I know it's very bad and I don't like it but it works :P
suggestions to improve it?
of course this is tested on 1.0! I don't know if it's compatible with 1.1 or not.
After edi's recommendation I've moved my forum site to smf. Then I've tried to post new topic. As a result i got topic with one old (not related message) and my new posting inside. I've deleted not related one and I've got strange result - topic with -1 replies and no messages. Then i manually deleted all topics with -1 replies (26 of 2750) and then all messages with no topic (54 of ~130000).
mysql> delete from smf_topics where numReplies=-1;
Query OK, 23 rows affected (0.00 sec)
mysql> delete from smf_messages where id_topic not in (select id_topic from smf_topics);
Query OK, 54 rows affected (0.90 sec)
I had no time problems - on overloaded hosting, athlon 1800+ it took about 15-20 minutes to convert the database (130487 msg).
My current problem is that all users are able to see all hidden forums. I am not exactly sure how permissions system in smf works, but i will try to understand and fix this too.