Simple Machines Community Forum

SMF Support => Converting to SMF => Topic started by: JayBachatero on January 08, 2007, 09:39:10 PM

Title: UTF-8 Conversion
Post by: JayBachatero on January 08, 2007, 09:39:10 PM
I'm looking into adding UTF-8 support to the converter.  Before I do that I want to try this first to see if it will work.  After installing SMF go to Admin > Forum Maintenance > Convert to UTF-8.  That will convert the tables and column to the UTF-8 collation.  After that try the converter.  Doing that should work for the most part.  If not I'll look at another alternative.
Title: Re: UTF-8 Conversion
Post by: JayBachatero on January 18, 2007, 12:55:33 AM
Ok this is a test converter to test UTF-8 stuff out.  Let me know how it goes.
Title: Re: UTF-8 Conversion
Post by: madru on January 18, 2007, 01:13:13 AM
Negative. Same thing and this time I got an error. I ticked the new option of converting to UTF-8 (I did a fast clean install again) and here is what I got:
Converting ranks... Successful.
Converting groups... Successful.
Converting members... Successful.
Converting additional member groups... Successful.
Converting categories... Successful.
Converting boards... Successful.
Converting topics... Successful.
Converting posts (this may take some time)... Successful.
Converting polls... Successful.
Converting poll options... Successful.
Converting poll votes... Successful.
Converting personal messages (step 1)... Successful.
Converting personal messages (step 2)... Successful.
Converting topic notifications... Successful.
Converting board access... Successful.
Converting group access... Successful.
Converting moderators... Successful.
Converting avatar gallery images...
Fatal error: Cannot redeclare copy_dir() (previously declared in /var/www/site/forum/convert.php:2108) in /var/www/site/forum/convert.php(682) : eval()'d code on line 19
Title: Re: UTF-8 Conversion
Post by: JayBachatero on January 18, 2007, 01:17:54 AM
Ok that error is due to a change that I made in convert.php.  I have to update the phpbb2_to_smf.sql to reflect the change.  I'll update it now and post in the topic.
Title: Re: UTF-8 Conversion
Post by: odbayarb2000 on February 17, 2007, 11:55:29 PM
I have already converted my forum and it turned into alien language lol.
Is there any way to change the character settings into UTF-8 after everything has done. I mean After converted.
Please help me.
Also I want to translate SMF into my own language. How should I start. Please show me the way.
Thanks for your support
Title: Re: UTF-8 Conversion
Post by: odbayarb2000 on February 19, 2007, 02:39:45 AM
Hey Guys,
I want to convert my phpbb 2.0.22 forum into SMF 1.1.2.
I have done everything as suggested as here.
The forum has converted successfully. But unfortunately,   I have character set error.
My entire forum is non-english. What should I do now. Please help me ASAP.  
Title: Re: UTF-8 Conversion
Post by: JayBachatero on February 26, 2007, 10:30:08 AM
What collation are the tables set to for SMF?
Title: Re: UTF-8 Conversion
Post by: whitewash on March 06, 2007, 02:32:06 AM
Where can I find the phpbb2_to_smf.sql for UTF8?
Title: Re: UTF-8 Conversion
Post by: JayBachatero on March 26, 2007, 03:17:36 PM
Ok give this one a try.  Install SMF with the correct charset that you want to use then convert.
Title: Re: UTF-8 Conversion
Post by: boki on March 27, 2007, 09:04:46 AM

I've tried and got this error message:

QuoteConverting...
Converting ranks... Successful.
Converting groups... Successful.
Converting members... Successful.
Converting additional member groups... Successful.
Converting categories... Successful.
Converting boards... Successful.
Converting topics... Successful.
Converting posts (this may take some time)... Successful.
Converting polls... Successful.
Converting poll options... Successful.
Converting poll votes... Successful.
Converting personal messages (step 1)... Successful.
Converting personal messages (step 2)... Successful.
Converting topic notifications... Successful.
Converting board access... Successful.
Converting group access... Successful.
Converting moderators... Successful.
Converting avatar gallery images...
Fatal error: Cannot redeclare copy_dir() (previously declared in /home/drugacen/public_html/smf/convert.php:2096) in /home/drugacen/public_html/smf/convert.php(671) : eval()'d code on line 1
Title: Re: UTF-8 Conversion
Post by: JayBachatero on March 28, 2007, 12:10:10 AM
What converter are you using?  I mean the .sql file.
Title: Re: UTF-8 Conversion
Post by: boki on March 28, 2007, 11:06:01 AM
I'm using the .sql file from "download" page, as you posted only php file on this forum...  
Title: Re: UTF-8 Conversion
Post by: JayBachatero on March 28, 2007, 01:31:59 PM
Where are you converting from?  What software?
Title: Re: UTF-8 Conversion
Post by: boki on March 28, 2007, 01:37:04 PM
I'm converting from phpBB 2.0.21
Title: Re: UTF-8 Conversion
Post by: JayBachatero on March 28, 2007, 01:48:10 PM
Try using the converter in this topic: phpBB 2.0.19+ (http://www.simplemachines.org/community/index.php?topic=142124) along with the convert.php from here.
Title: Re: UTF-8 Conversion
Post by: boki on March 28, 2007, 02:03:23 PM
Nothing... still the same problem
Title: Re: UTF-8 Conversion
Post by: JayBachatero on March 28, 2007, 03:12:35 PM
Meaning same UTF-8 problem or conversion problem?
Title: Re: UTF-8 Conversion
Post by: haco on July 30, 2007, 01:04:10 PM
thank you very much
Title: Re: UTF-8 Conversion
Post by: koyukaranlik on July 30, 2007, 03:33:14 PM
thanks
Title: Re: UTF-8 Conversion
Post by: hamade_ima on August 01, 2007, 06:06:27 AM
thanks
Title: Re: UTF-8 Conversion
Post by: CSpili on August 08, 2007, 10:09:27 AM
I'm having some problems converting from the greek language of MyBB 1.2 to SMF 1.1.3.  Still shows alien figures unfortunately :(
Title: Re: UTF-8 Conversion
Post by: JayBachatero on August 08, 2007, 10:23:56 AM
Did you setup smf to the correct chatset?  Also did you download the correct language pack.
Title: Re: UTF-8 Conversion
Post by: beate_r on August 18, 2007, 07:50:07 AM
What about running recode or iconv on a database dump and then importing the new database to SMF? Has anyone tried?
Title: Re: UTF-8 Conversion
Post by: Sarge on August 19, 2007, 07:52:57 AM
Be sure to export and import the database with the correct charsets:
http://textsnippets.com/posts/show/84

The sed syntax posted in the link above has never been correct for me in CentOS and Ubuntu (invalid arguments, I think), but the alternate syntax (posted in the last comment) always worked fine:
sed -r 's/latin1/utf8/g' dump.sql > dump_utf.sql

If you can't import because you don't have shell access, you can add
SET NAMES utf8
or
SET NAMES 'utf8'
at the start of the database dump file. If you do this, save the dump file in the correct charset! Alternatively, use/modify a restore script that can "SET NAMES utf8" when importing.
Title: Re: UTF-8 Conversion
Post by: kicho on August 26, 2007, 07:34:36 AM
Quote from: Sarge on August 19, 2007, 07:52:57 AM
Be sure to export and import the database with the correct charsets:
http://textsnippets.com/posts/show/84 (http://textsnippets.com/posts/show/84)

The sed syntax posted in the link above has never been correct for me in CentOS and Ubuntu (invalid arguments, I think), but the alternate syntax (posted in the last comment) always worked fine:
sed -r 's/latin1/utf8/g' dump.sql > dump_utf.sql

If you can't import because you don't have shell access, you can add
SET NAMES utf8
or
SET NAMES 'utf8'
at the start of the database dump file. If you do this, save the dump file in the correct charset! Alternatively, use/modify a restore script that can "SET NAMES utf8" when importing.


Can anyone do this for me? Pretty, pretty, pretty please. :(
Title: Re: UTF-8 Conversion
Post by: Sarge on August 26, 2007, 07:48:18 AM
kicho, I can try. Send me the details via PM. I prefer access to the original (unconverted) phpBB database, if possible.
Title: Re: UTF-8 Conversion
Post by: kicho on August 26, 2007, 07:58:39 AM
U have PM ;)
Title: Re: UTF-8 Conversion
Post by: CSpili on August 27, 2007, 12:52:37 PM
Jay, sorry for the delay, I was on vacation...  At the moment it's still showing alien.  DB config is UTF-8, mybb has latin1 as usual, and smf is installed with utf-8 support.

Well, at the moment, the problem still resides.  Apparently, the SQL DB is installed as utf-8, mybb has latin1 as usual and the board is in greek (iso 8859-7), however, the converter probably understands it as 8859-1 and there's no option to change it in -7 in the convert db @ the admin panel or when I'm converting the forum.

Please advise.
Thanks in advance,
Constantinos
Title: Re: UTF-8 Conversion
Post by: Sarge on August 27, 2007, 02:27:01 PM
CSpili, if you don't hear back from Jay sooner, I will try to help you as soon as I finish helping kicho (above) :)
Title: Re: UTF-8 Conversion
Post by: sektor on August 27, 2007, 06:20:14 PM
Hello,

Any idea why i get a "Hacking attempt..." message when i try to convert to UTF8?
Title: Re: UTF-8 Conversion
Post by: Sarge on August 28, 2007, 02:18:09 AM
Quote from: al0000 on August 27, 2007, 06:20:14 PM
Any idea why i get a "Hacking attempt..." message when i try to convert to UTF8?

Can you please provide more information? For example, the forum software you're converting from, where you downloaded convert.php and the .sql file, and how you tried the conversion.

A step-by-step example would help us in determining whether the issue can be replicated or is specific to your case.
Title: Re: UTF-8 Conversion
Post by: Sarge on August 30, 2007, 05:54:41 AM
CSpili, I managed to repair kicho's forum, so I can work on yours. Send me a PM. :)
Title: Re: UTF-8 Conversion
Post by: CSpili on September 01, 2007, 08:02:09 AM
you've got PM Sarge
Title: Re: UTF-8 Conversion
Post by: Sarge on September 02, 2007, 04:40:32 AM
Quote from: CSpili on September 01, 2007, 08:02:09 AM
you've got PM Sarge

OK, I got the info. Text and varchar columns in your MyBB database tables seem to be in latin1, whereas connections to MySQL are made in utf8. This causes Greek characters to get garbled during transfer.

First of all, put your MyBB forum in "Maintenance Mode" (I don't know what the relevant setting is in MyBB). Then get a full backup of MyBB database tables, or delete the SMF tables and backup the database. I have found backups generated by cPanel to be pretty reliable. Keep this backup somewhere safe -- don't use it for the procedure described below.

Create a database backup using a modified create_backup.php (attached) -- this modified tool creates a database backup using the latin1 charset, regardless of the charset that MySQL uses by default (utf8 in your case) to transfer data. You can specify the table prefix (probably mybb_) so that you get only MyBB tables.

Assuming you're running Windows on your home computer, download and install the Win32 version of GNU sed from here:
http://gnuwin32.sourceforge.net/downlinks/sed.php

Open a command prompt (Start > All Programs > Accessories > Command Prompt, in Windows XP), type the following command and hit Enter:

path-to-sed s/latin1/utf8/g path-to-db-backup.sql > path-to-db-backup-sed.sql

Replace path-to-sed, path-to-db-backup.sql and path-to-db-backup-sed.sql with correct values. For example, if your (uncompressed) database dump is saved as backup.sql in C:\ and sed.exe is located in C:\Program Files\GnuWin32\bin, the correct command would be:

"C:\Program Files\GnuWin32\bin\sed" s/latin1/utf8/g C:\backup.sql > C:\backup-sed.sql

Then import the created backup-sed.sql to phpMyAdmin, cPanel or whatever you use to import database backups. Hopefully, your MyBB installation should now be in UTF-8 and you should be able to convert to SMF without any problems.

There are some possible issues with codepage conversions, for which you can use a text editor that allows you to specify the codepage; UltraEdit-32  seems to support this.

If you run into any issues or are afraid of doing the above by yourself, you can send me cPanel login details via PM and I will carry out the conversion process for you.
Title: Re: UTF-8 Conversion
Post by: designer0307 on September 05, 2007, 08:01:34 AM
Hi ,
I have similar problem as you guys but still i can't manage to fix it.
The matter is simple.I have just converted phpbb to SMF and everything works fine except encoding.I don't have  "ąśźć" (polish signs) in the posts converted.I only see"? ? ?". Phpbb datebase is encoded "latin1_swedish_ci" in phpmy Admin same with SMF tables. Smf is fresh installed without"utf8" marked during installation. I tried to mark it and then convert but problem remains the same.
Simply,converter changes proper polish signs to question marks.How i can fix this?

If someone can help me i would appreciate it.
Title: Re: UTF-8 Conversion
Post by: Sarge on September 05, 2007, 02:14:37 PM
designer0307, first of all, welcome to SMF! :)

I suspect the accented characters problem has to do with database table charset and/or collation.

Post the configuration variables mentioned in this post:
http://www.simplemachines.org/community/index.php?topic=165442.msg1056581#msg1056581
Title: Re: UTF-8 Conversion
Post by: designer0307 on September 05, 2007, 03:33:33 PM
 That's what i thought too but it seems that it's not the case. Phpbb tables are encoded in latin1_swedish_ci so i changed smf tables to latin1 swedish ci and then converted but there were still question marks in the posts;/

I tried installing smf in utf8 but this doesn't help at all.When i did it,  phpbb tables were latin1_swedish_ci and smf tables(in the same datebase) were utf8_general ci.

Here are the variables u asked for.I hope this will help a  bit.


character set client     utf8
(Global value)    latin1
character set connection    utf8
(Global value)    latin1
character set database    latin1
character set results    utf8
default_charset(Global value)    latin1
character set server    latin1
character set system    utf8
character sets dir    /usr/share/mysql/charsets/
collation connection    utf8_unicode_ci
(Global value)    latin1_swedish_ci
collation database    latin1_swedish_ci
collation server    latin1_swedish_ci


default charset -no value
_SERVER HTTP_ACCEPT_CHARSET"]   -ISO-8859-2, utf-8;q=0.7,*;q=0.7
_ENV["HTTP_ACCEPT_CHARSET"]   -ISO-8859-2, utf-8;q=0.7,*;q=0.7


Ps.I installed polish language pack as well and i used default smf template.I tried bot packs:

* smf-1-1-2_polish_iso-8859-2.zip
* smf-1-1-2_polish_utf-8.zip
Title: Re: UTF-8 Conversion
Post by: Sarge on September 05, 2007, 04:15:31 PM
Link to your phpBB forum? Send it to me via PM if, for some reason, you don't want to post it publicly.

Are you using SMF now? When converting from phpBB, new posts, members etc. in your new SMF forum will be lost, although I can help with posts.
Title: Re: UTF-8 Conversion
Post by: designer0307 on September 05, 2007, 05:32:31 PM
Sure thing.I have modified phpbb colled "phpbb by przemo".It converts  with ease though. i am using phpbb2 converter for this purpose.The only problem is only encoding.
http://forum.forumwow.net

I want to convert it to SMF and add tiny portal to it so i am not currently using SMF.Only fresh clean installation.

This is what i managed to convert:
http://forumwow.net/smf

I don't care about anything byt posts.

Thx for helping me ;)
Title: Re: UTF-8 Conversion
Post by: Sarge on September 05, 2007, 06:07:16 PM
I see. The data gets saved as ISO-8859-2, which is also indicated by the encoding used in the phpBB forum pages (View -> Character Encoding in Firefox).

Do Polish characters show up correctly in new posts in SMF?

I will be try to help Jay add support for multiple character sets in the converters, but meanwhile I can solve the issue for you. PM me your access details (cPanel, FTP, SMF Admin accounts) and I will do the conversion for you.

Let me know if you want to keep the existing SMF installation (and new posts and members), or redo the conversion from phpBB and lose the new posts. Personally, I prefer the second method, because it's cleaner (read my previous post), but it's up to you.
Title: Re: UTF-8 Conversion
Post by: designer0307 on September 05, 2007, 06:51:08 PM
Private message sent.
Thx in advance

Ps.New posts dispaly correctly.
Title: Re: UTF-8 Conversion
Post by: Sarge on September 24, 2007, 05:45:21 AM
CSpili, have you managed to convert your Greek myBB forum to SMF? If not, I can send you an updated converter that may help with the character set issue.

kicho and designer0307, how are your converted forums running? :)
Title: Re: UTF-8 Conversion
Post by: Kiri` Von Antertynwae on October 22, 2007, 06:38:40 PM
Hello.

I have same problem as @designer0307 - wanna convert phpBB by using converter.php to smf 1.1.4 with polish sign posts but they still be ? ? ? but in database signs be normal...

Any idea to fix that problem?
Title: Re: UTF-8 Conversion
Post by: SleePy on October 22, 2007, 11:59:00 PM
Did you go to in SMF
Admin -> Forum Maintenance
And click to convert to UTF-8?
Title: Re: UTF-8 Conversion
Post by: Kiri` Von Antertynwae on October 23, 2007, 01:56:19 AM
Yea, but didn't work...   :-\
Title: Re: UTF-8 Conversion
Post by: Sarge on November 07, 2007, 02:36:35 AM
Quote from: Kiri` Von Antertynwae on October 23, 2007, 01:56:19 AM
Yea, but didn't work...   :-\

Sorry for the delayed response. Have you installed the Polish-utf8 language pack?
Title: Re: UTF-8 Conversion
Post by: target_locked on December 03, 2007, 03:57:58 AM
Hi All,
I can not backup and restore my DB with UTF-8 charset in PHPMyadmin, there is no option for UTF-8?
After backup and restore, the UTF-8 collation become "latin1_swedish_ci" and my Vietnamese language can not be displayed correctly. How to change it back to UTF-8?
Any one experienced this, pls advice.
Thanks.
Title: Re: UTF-8 Conversion
Post by: Sarge on December 03, 2007, 06:27:17 AM
Quote from: target_locked on December 03, 2007, 03:57:58 AM
Hi All,
I can not backup and restore my DB with UTF-8 charset in PHPMyadmin, there is no option for UTF-8?
After backup and restore, the UTF-8 collation become "latin1_swedish_ci" and my Vietnamese language can not be displayed correctly. How to change it back to UTF-8?
Any one experienced this, pls advice.
Thanks.

Go to SMF Admin > Forum Maintenance and run "Convert the database and data to UTF-8". If this option is not available and you have "Convert HTML-entities to UTF-8 characters" instead, but the collation of your DB tables and table fields is latin1_swedish_ci, you probably have to delete the global_character_set variable from the table smf_settings in phpMyAdmin; then you should see the option "Convert the database and data to UTF-8".

See this post for more info:
http://www.simplemachines.org/community/index.php?topic=166743.msg1151417#msg1151417

Hopefully it will work for you! :)
Title: Re: UTF-8 Conversion
Post by: target_locked on December 03, 2007, 11:47:38 PM
Quote from: Sarge on December 03, 2007, 06:27:17 AM
Go to SMF Admin > Forum Maintenance and run "Convert the database and data to UTF-8". If this option is not available and you have "Convert HTML-entities to UTF-8 characters" instead, but the collation of your DB tables and table fields is latin1_swedish_ci, you probably have to delete the global_character_set variable from the table smf_settings in phpMyAdmin; then you should see the option "Convert the database and data to UTF-8".

See this post for more info:
http://www.simplemachines.org/community/index.php?topic=166743.msg1151417#msg1151417

Hopefully it will work for you! :)

That doesn't work.
After run "Convert the database and data to UTF-8" and come back to forum, everything is same.

I just want to back up my current DB (which is working normally) 100% exactly the same, but after deleting the DB and restore/import it by PHPMyadmin, it become different DB.
Title: Re: UTF-8 Conversion
Post by: Sarge on December 06, 2007, 02:33:07 AM
Quote from: target_locked on December 03, 2007, 11:47:38 PM
That doesn't work.
After run "Convert the database and data to UTF-8" and come back to forum, everything is same.

I just want to back up my current DB (which is working normally) 100% exactly the same, but after deleting the DB and restore/import it by PHPMyadmin, it become different DB.

How are you backing up the database? Are you doing it from phpMyAdmin or SMF? What options do you select during export and import?
Title: Re: UTF-8 Conversion
Post by: Naglfar on December 17, 2007, 05:15:58 PM
Hello.

I have exacly the same problem as designer0307. Conversion from "phpbb by przemo" goes fine, except polish signs. Phpbb tables are in latin1_swedish_ci encoding, SMF tables too. After conversion polish signs are replaced with strange symbols. Changing encoding in web browser doesn't help.

Sarge, you have solved designer0307's problem. Can you write how you did that?

Thanks in advance.
Title: Re: UTF-8 Conversion
Post by: Sarge on December 18, 2007, 02:28:18 AM
Quote from: Naglfar on December 17, 2007, 05:15:58 PM
Sarge, you have solved designer0307's problem. Can you write how you did that?

What is selected as the (Character) Encoding (from the View menu) in your browser when you go to your phpBB forum?
Title: Re: UTF-8 Conversion
Post by: Naglfar on December 18, 2007, 07:46:18 AM
Quote from: Sarge on December 18, 2007, 02:28:18 AMWhat is selected as the (Character) Encoding (from the View menu) in your browser when you go to your phpBB forum?

"Phpbb by przemo" looks proper in ISO-8859-2 encoding.
Title: Re: UTF-8 Conversion
Post by: Sarge on December 18, 2007, 11:30:29 AM
Here's what I did for designer0307. I'm not sure if the guide below applies in your case, so let me know how it goes for you.

0) First, get a full backup of the database, including the structure. I suggest getting a backup from cPanel, if you have it, but backups from phpMyAdmin should be OK too. Be sure to test the downloaded backup, to make sure it's not corrupted -- this can happen, for example, when the download has been interrupted. This is very important!

1) Install SMF using the default options. Do not select UTF-8 in the install page. If you install SMF on a different database than the phpBB installation, make sure that the database user for SMF has access to the phpBB database.

2) Download the UTF-8 Converter (convert.php) attached to this post:
http://www.simplemachines.org/community/index.php?topic=140917.msg1023841#msg1023841

Open it in a text editor. Find:

mysql_query("SET NAMES $db_character_set");


Replace with:

mysql_query("SET NAMES latin2");


If it doesn't work, try latin1 instead of latin2.

Save it as cvt_latin2.php (or cvt_latin1.php) after making the change and upload it to the main directory of your SMF forum. Also upload the latest phpbb2_to_smf.sql, attached to this post:
http://www.simplemachines.org/community/index.php?topic=142124.msg905972#msg905972

3) Run the converter as usual, for example from this URL:
http://www.yoursite.com/smf/cvt_latin2.php
Delete cvt_latin2.php after the conversion.

Let me know if you get any errors during the conversion.

4) After the conversion has been finished successfully, go to Admin (Administracja in Polish) --> Forum Maintenance (Obsluga forum). Click on "Convert the database and data to UTF-8" (Przekonwertuj baze danych i dane na UTF-8). In the next page, select ISO-8859-2 (if it doesn't work, select ISO-8859-1) for "Data character set" (Standard kodowania danych) and click the Proceed (Dalej) button.

5) After the UTF-8 conversion has finished, upload the Polish UTF-8 language pack files.

6) Switch the language to Polish in your profile or in Admin > Server Settings (Ustawienia serwera) > "Default Forum Language" (Domyslny jezyk forum) to Polish.

Note 1: Since characters are already converted to UTF-8, I think SMF displays posted Polish characters correctly even when using the English language pack, maybe even with other language-utf8 packs.

Note 2: After conversion, character (page) encoding in your browser should be selected as Unicode (UTF-8). Central European (ISO-8859-2) doesn't work for UTF-8 characters, except for the English alphabet and some other common characters.

Let me know if you find any problems with the above, or if you need more information.
Title: Re: UTF-8 Conversion
Post by: Sarge on December 20, 2007, 10:17:57 PM
Naglfar, any news about the conversion?
Title: Re: UTF-8 Conversion
Post by: newk on March 19, 2008, 02:18:55 PM
Is choosing UTF-8 when importing database in PHPmyadmin another way to convert database to UTF-8?
Title: Re: UTF-8 Conversion
Post by: forepro on March 23, 2008, 10:16:07 AM
Hi

I followed Sarge's procedure (not exactly the same though) and I was able to convert FireBoard to SMF UTF-8 successfully!

http://www.simplemachines.org/community/index.php?topic=228775.msg1477176#msg1477176

cheers!
Title: Re: UTF-8 Conversion
Post by: SleePy on April 13, 2008, 12:07:19 AM
Quote from: newk on March 19, 2008, 02:18:55 PM
Is choosing UTF-8 when importing database in PHPmyadmin another way to convert database to UTF-8?

No,
When you convert to UTF-8 SMF does some additional stuff such as changing the charset it uses on output
Title: Re: UTF-8 Conversion
Post by: hoo on April 26, 2008, 11:58:15 PM
Sarge can you help me?  with this converting i tried manually edited base .sql and converted severeal times but no results ;/

This instruction:
QuoteHere's what I did for designer0307. I'm not sure if the guide below applies in your case, so let me know how it goes for you.

0) First, get a full backup of the database, including the structure. I suggest getting a backup from cPanel, if you have it, but backups from phpMyAdmin should be OK too. Be sure to test the downloaded backup, to make sure it's not corrupted -- this can happen, for example, when the download has been interrupted. This is very important!

1) Install SMF using the default options. Do not select UTF-8 in the install page. If you install SMF on a different database than the phpBB installation, make sure that the database user for SMF has access to the phpBB database.

2) Download the UTF-8 Converter (convert.php) attached to this post:
http://www.simplemachines.org/community/index.php?topic=140917.msg1023841#msg1023841

Open it in a text editor. Find:
Code: [Select]
mysql_query("SET NAMES $db_character_set");

Replace with:
Code: [Select]
mysql_query("SET NAMES latin2");

If it doesn't work, try latin1 instead of latin2.

Save it as cvt_latin2.php (or cvt_latin1.php) after making the change and upload it to the main directory of your SMF forum. Also upload the latest phpbb2_to_smf.sql, attached to this post:
http://www.simplemachines.org/community/index.php?topic=142124.msg905972#msg905972

3) Run the converter as usual, for example from this URL:
http://www.yoursite.com/smf/cvt_latin2.php
Delete cvt_latin2.php after the conversion.

Let me know if you get any errors during the conversion.

4) After the conversion has been finished successfully, go to Admin (Administracja in Polish) --> Forum Maintenance (Obsluga forum). Click on "Convert the database and data to UTF-8" (Przekonwertuj baze danych i dane na UTF-8). In the next page, select ISO-8859-2 (if it doesn't work, select ISO-8859-1) for "Data character set" (Standard kodowania danych) and click the Proceed (Dalej) button.

5) After the UTF-8 conversion has finished, upload the Polish UTF-8 language pack files.

6) Switch the language to Polish in your profile or in Admin > Server Settings (Ustawienia serwera) > "Default Forum Language" (Domyslny jezyk forum) to Polish.

Note 1: Since characters are already converted to UTF-8, I think SMF displays posted Polish characters correctly even when using the English language pack, maybe even with other language-utf8 packs.

Note 2: After conversion, character (page) encoding in your browser should be selected as Unicode (UTF-8). Central European (ISO-8859-2) doesn't work for UTF-8 characters, except for the English alphabet and some other common characters.

Let me know if you find any problems with the above, or if you need more information.

doesn't work for me ;/, please can you help me in that?
Title: Re: UTF-8 Conversion
Post by: Sarge on May 10, 2008, 03:21:27 AM
Sorry for the delay, hoo. What didn't work? Can you tell us what character sets and collations are used in the original database, and what did you do (step by step)?
Title: Re: UTF-8 Conversion
Post by: mymktp1 on May 13, 2008, 04:19:05 AM
Hi I currently am having the same problem as many other people have been having and was wondering if anyone could help me... I have a phpBB forum in THAI language  which I checked in mySql that the Collation is all in Latin-1 which should be in UTF-8 ... and I don't know and kind of hesitated to change this since I don't really know what I'm doing... I have successfully converted phpbb to smf...  but the characters don't come out right at all... and I'm thinking collation set in my phpBB is probably the problem... can anyone help me with this?
Title: Re: UTF-8 Conversion
Post by: SleePy on May 13, 2008, 11:54:29 AM
mymktp1,

http://www.simplemachines.org/community/index.php?topic=239123.msg1542073#new

I was helping you there. Please don't double post your question.
Title: Re: UTF-8 Conversion
Post by: Ni©holąś™ on June 02, 2008, 02:47:49 PM
Thanks :)
Title: Re: UTF-8 Conversion
Post by: mobilewo on July 10, 2008, 05:41:08 AM
Sorry, the database connection information used in the specified installation of SMF cannot access the installation of phpBB3. This may either mean that the installation doesn't exist, or that the MySQL account used does not have permissions to access it.

The error MySQL gave was: SELECT command denied to user 'mobilewo'@'WEB10' for table 'phpbb_users'
Title: Re: UTF-8 Conversion
Post by: ThorstenE on July 10, 2008, 05:48:26 AM
Quote from: mobilewo on July 10, 2008, 05:41:08 AM
The error MySQL gave was: SELECT command denied to user 'mobilewo'@'WEB10' for table 'phpbb_users'
Common conversion errors (http://www.simplemachines.org/community/index.php?topic=146192.0)
Title: Re: UTF-8 Conversion
Post by: fiolek on July 24, 2008, 06:18:08 PM
Hi,
I am new here and I am trying to convert my nuke forum to smf, but I still have ? instead of polish letters. I tried to change collation to latin1, latin2, utf8 I to convert nuke db to utf but i still have te same problem, can anyone help me with this?
Title: Re: UTF-8 Conversion
Post by: Sarge on July 25, 2008, 12:48:44 AM
Quote from: fiolek on July 24, 2008, 06:18:08 PM
Hi,
I am new here and I am trying to convert my nuke forum to smf, but I still have ? instead of polish letters. I tried to change collation to latin1, latin2, utf8 I to convert nuke db to utf but i still have te same problem, can anyone help me with this?

1) Can you tell us what collations are used in the original database (for both the forum tables and table columns), and what did you do (step by step)?

2) Follow the steps in reply #53 (http://www.simplemachines.org/community/index.php?topic=140917.msg1348511#msg1348511), but replace:

mysql_query("SET NAMES $db_character_set");

with:

mysql_query("SET NAMES latin2 COLLATE latin2_polish_ci");


If that doesn't work, try:

mysql_query("SET CHARACTER SET latin2 COLLATE latin2_polish_ci");

Title: Re: UTF-8 Conversion
Post by: fiolek on July 25, 2008, 05:06:49 AM
I've got:
character set client     utf8
(Wartość globalna)    latin1
character set connection    utf8
(Wartość globalna)    latin1
character set database    latin1
character set results    utf8
(Wartość globalna)    latin1
character set server    latin1
character set system    utf8
character sets dir    /usr/share/mysql/charsets/
collation connection    utf8_general_ci
(Wartość globalna)    latin1_swedish_ci
collation database    latin1_swedish_ci
collation server    latin1_swedish_ci

in db with nuke I have latin2 (i tried also utf8)

Step by step I tried eg instruction from this topic

Quotemysql_query("SET NAMES $db_character_set");
but in nuke converter I haven't it
Title: Re: UTF-8 Conversion
Post by: Sarge on July 25, 2008, 05:47:17 AM
Quote from: fiolek on July 25, 2008, 05:06:49 AM
Quotemysql_query("SET NAMES $db_character_set");
but in nuke converter I haven't it

Get the latest convert.php (http://www.simplemachines.org/community/index.php?action=dlattach;topic=140741.0;attach=61415) and find this:

mysql_query("SET NAMES $charset");


(Apparently, $db_character_set has been replaced with $charset in recent versions.)
Title: Re: UTF-8 Conversion
Post by: fiolek on July 25, 2008, 06:06:53 AM
But it still doesn't work. What collation i db should I use?
Title: Re: UTF-8 Conversion
Post by: Sarge on July 25, 2008, 06:26:22 AM
Quote from: fiolek on July 25, 2008, 05:06:49 AM
I've got:
in db with nuke I have latin2 (i tried also utf8)

What collation are the tables and the table columns?

What is the page encoding in your browser? (View > Character Encoding in Firefox, View > Encoding in IE)
Title: Re: UTF-8 Conversion
Post by: fiolek on July 25, 2008, 06:36:28 AM
Latin2 but I convert i tried also on utf8.
In my browser it is iso
Title: Re: UTF-8 Conversion
Post by: Sarge on July 25, 2008, 06:42:39 AM
Quote from: fiolek on July 25, 2008, 06:36:28 AM
Latin2 but I convert i tried also on utf8.
In my browser it is iso

What latin2 collation? latin2_general_ci, latin2_croatian_ci, or what?
What ISO encoding is autoselected in your browser? ISO-8859-1, -2, or what?
Title: Re: UTF-8 Conversion
Post by: fiolek on July 25, 2008, 06:56:18 AM
latin2_general_ci and iso 8859-1
Title: Re: UTF-8 Conversion
Post by: Sarge on July 25, 2008, 07:00:49 AM
If you change the page encoding to ISO-8859-1 or ISO-8859-2 for your SMF forum pages from the browser, do the Polish characters show up correctly?
Title: Re: UTF-8 Conversion
Post by: fiolek on July 25, 2008, 07:14:55 AM
No, i've got also ? instead of polish signs in my db
Title: Re: UTF-8 Conversion
Post by: Kvark on July 25, 2008, 08:46:16 AM
May I ask some help? My phpBB was gone (phpMy Admin don see it) but converter from phpBB to SMF read folder of phpBB instalations place and some how get data back, but its in utf maybe becouse i install SMF in 1251 charset, and i dont remember wich cahrset was in BB also is it posible some how insert post into SMF but dont loose 1251 charset? 
Title: Re: UTF-8 Conversion
Post by: Sarge on July 25, 2008, 09:51:08 AM
Quote from: fiolek on July 25, 2008, 07:14:55 AM
No, i've got also ? instead of polish signs in my db

Link to your forum? Feel free to send it via PM if you don't want to make it public.

If you want to give me access, I can try doing the conversion myself.
Title: Re: UTF-8 Conversion
Post by: Sarge on July 25, 2008, 10:36:10 AM
Quote from: Kvark on July 25, 2008, 08:46:16 AM
May I ask some help? My phpBB was gone (phpMy Admin don see it) but converter from phpBB to SMF read folder of phpBB instalations place and some how get data back, but its in utf maybe becouse i install SMF in 1251 charset, and i dont remember wich cahrset was in BB also is it posible some how insert post into SMF but dont loose 1251 charset?

If phpMyAdmin cannot see the DB but the converter can, something is wrong there. Contact your host about it.

UTF-8 supports cp1251 characters, you just have to convert them to UTF-8 format. But without knowing what exactly the original character set is, results can be unpredictable.
Title: Re: UTF-8 Conversion
Post by: nackgr on March 28, 2009, 06:25:06 AM
i have a vb 3.8.1  with latin1 swedish_ci collation  forum is at greek lang  to view it  i use iso 8859-7 and i want to convert it  to  smf 


i convert it  but  characters are  #%@#!2$ what should i do ?
i cant connect to admin panel  bcause users dont work ... it says wrong password
Title: Re: UTF-8 Conversion
Post by: ThorstenE on March 28, 2009, 09:13:36 AM
which character set is your SMF? you should install SMF with latin1 swedish_ci  (ISO 8859-1) and try again.
Title: Re: UTF-8 Conversion
Post by: nackgr on March 28, 2009, 04:04:28 PM
ok i convert it  i did what u said ... but now i cant log in  ...  says wrong password

if i  change  browser's code to iso 8859-7 i see everything fine  greek too .. but i cant   log on
Title: Re: UTF-8 Conversion
Post by: ThorstenE on March 28, 2009, 05:08:23 PM
mhh, maybe you should install SMF with iso 8859-7 encoding and then convert your vBulletin. Sorry, thought ISO-8859-1 had fixed it.
Title: Re: UTF-8 Conversion
Post by: nackgr on March 28, 2009, 06:55:42 PM
post etc.. are fine  with (ISO 8859-1)  db  i change  it to  8859-7 at the browser and everything is fine  only the users are dead anyway i will test what  did u  propose tommorow and i will post results
Title: Re: UTF-8 Conversion
Post by: nackgr on March 29, 2009, 04:16:17 AM
converter dont have  8859-7 convert option ... so still i cant log on like a user  (wrong password )
Title: Re: UTF-8 Conversion
Post by: Sarge on April 03, 2009, 04:56:41 AM
Quote from: nackgr on March 28, 2009, 06:55:42 PM
post etc.. are fine  with (ISO 8859-1)  db  i change  it to  8859-7 at the browser and everything is fine  only the users are dead anyway i will test what  did u  propose tommorow and i will post results

You should add the smf_1-1-8_greek (http://download.simplemachines.org/?languages;lang=greek) language pack and set it as the forum default. This pack uses Windows-1253 (http://en.wikipedia.org/wiki/Windows-1253) instead of ISO-8859-7 (http://en.wikipedia.org/wiki/ISO_8859-7), so Ά ("Greek capital letter alpha with tonos") might not show up correctly, but I can make an ISO-8859-7 pack for you if all other characters are displayed correctly.