Encoding problem ? Please help... :-)

[MacDo]

Hello,

I'm sorry posting here but I get no answer on the french child board. I really need to know if something is wrong and how to get it work... :-)

Well, here is the full story, but rather short.

I installed first phpbb2 (yes, was an error, I know :-)) and moved few days after on smf, updating database using the script given on the smf site. All was ok, fantastic : I'm now using smf 1.1RC2 and I enjoy it. (I'm using linux and utf8 on my personal computer)

Some days later, I wanted to integrate the news fader on my web site. I managed using the SSI functions.
BUT the accented characters (like é, ê, à, ...) appears correctly in the news fader in the SMF forum, but appears as � in my website. Huh.

I thought about an encoding problem in the DB. I saw in phpmyadmin that all the tables was encoded with latin1_swedish_ci. I did some

Koodi [Valitse] Laajenna

ALTER TABLE `smf_tables` DEFAULT CHARACTER SET UTF8 COLLATE utf8_general_ci;where smf_tables are the tables in my database (smf_attachments, smf_boards, smf_message, ...). Then, all the collations are utf8_general_ci. But the fields are encoded with latin1_swedish_ci... This seems not be a problem but I think that the accented problem I have with my integrated newsfader can be linked to the field encoding.

Am I wrong ?

What can I do to solve this problem ?

Thank you very much for your help !

[Elmacik]

Because the collations of columns (fields) are set individually.
For example if you look at the structure of messages table, you'll see that message body field is still in latin1_swedish_ci

[MacDo]

So How can I change the encoding of all the text fields of all the tables in my db (named 'forum') from latin1_swedish_ci to utf8_general_ci ?
I'm not familiar with phpmyadmin... 

Thank you once more for your help !

[Elmacik]

Just change them by editing the fields (click onto pencil icon next to the field)
So you can change collation there.

[MacDo]

Yes, ok... Thank you very much. I changed the collation for all latin1_swedish_ci fields. I set to utf8_general_ci but this does not solve the problem with the fader. The accented characters shows correctly in the SMF fader but not in the website.

Could you help me ? I really don't understand what could be the problem and how to solve it... 

[MacDo]

  Anyone could help me ?? Please...

[Compuart]

Make sure the website uses the same character set as the forum site. Check the source code of a forum page and look for <meta http-equiv="Content-Type" content="text/html; charset=XXXXX" />. Make sure such a line (with the same character set) is in your web site.

[MacDo]

All my webpages contains this metatag :

Koodi [Valitse] Laajenna

<meta http-equiv="Content-type" content="text/html; charset=UTF-8" />
I think it's ok, isn't it ?

[Compuart]

That depends on what your forum character set is. Unless you've changed it, the default forum character set would be ISO-8859-1. If you try that, it might show the characters correctly.

[MacDo]

Well ! You are right. But the problem is not over...

I edited the index.french.php file and replaced $txt['lang_character_set'] = 'ISO-8859-1'; by $txt['lang_character_set'] = 'UTF-8'; I had to change $txt['smf56'] and on other I forgot.
It solves the fader problem !! Thank you very very much !!!

But when I go to the forum, I see that all accented characters are shown as � so I have to correct all of them. Ok, there is not so much posts so I can do it.

Was it right to modify the charset where I did it ?

[Compuart]

It's a matter of choice. If you prefer UTF-8 as your character set, then you should indeed change everything into UTF-8, but that invalidates data that was originally stored as ISO-8859-1. As of 1.1 RC3 (which will soon be released) SMF allows conversion of forum data to UTF-8. Until then you'd have to do it manually.

[MacDo]

I think I will change everything into UTF-8 and make the RC3 update when available. I have around 15 posts so It will not be a huge work.

I cannot understand why I have a problem with 'à'... All other accented characters are OK but the 'à' seems to be coded by "Ã&nbsp;" in the DB and appears as �.

An idea, master Compuart ? :-)

[Compuart]

Yes, in the UTF-8 character set the à is broken down into two bytes 0xC3 and 0xA0. In 1.1 RC2 the 0xA0 byte is not always properly handled. Therefor 1.1 RC2 does not fully support UTF-8. 1.1 RC3 will fix this of course

[MacDo]

Ok, thank you very much for your help :-) I'll wait for RC3 :-)

By the way, you could maybe help me for the other question I have here ? :-)