RC3 -> RC4: Special Characters appear as �

[busy_one]

After Upgrading to RC4 all the special characters in my german-utf8 board appear as � - most posts are just empty boxes with occasional quote-boxes and list-elements.

New posts appear correctly, fixing the characters works also. But looking in the database shows that new/corrected messages are saved not UTF-8: ö should be displayed this way, but looks like this ö

The collation is utf8_general_ci.

Any advise how to fix this?

[Norv]

Can we have a link to your board please?
A test account (normal user account) would also be appreciated.

Does your forum's Settings.php file contain a line like:

Code Select Expand


$global_character_set = 'utf8';


[busy_one]

I added $global_character_set = 'utf8'; but that didn't change a thing.

Here you can reply as guest:
http://forum.freigeisterchen.de/index.php/board,2.0.html [nofollow]

First message is fixed, rest is broken.

This is open for testing:
http://forum.freigeisterchen.de/index.php/board,12.0.html [nofollow]

[Norv]

Can you please link to a post which does not show right? Sorry, but looking around, I only found posts with no international characters, or where the international characters do show correctly.
(like this for example: http://forum.freigeisterchen.de/index.php/topic,7.0.html)

[busy_one]

sry, you picked the other one I fixed (terms of usage)

Here is no content visible:
http://forum.freigeisterchen.de/index.php/topic,14.0.html [nofollow]

here's a screenshot of that message attached.

[LosElchos]

Hello

I had the same problem, but it's easy.
Our forum is in UTF-8 German language but the mysql DB connection from SMF 2.0 RC4 isn't UTF-8.

How too resolve this problem:
- Open your sources/Subs-Db-mysql.php file
- Goto line 360
- change the code

Code Select Expand

}
if (empty($db_unbuffered))

Code Select Expand

}
mysql_query("SET NAMES 'UTF8';",$connection);
if (empty($db_unbuffered))

Now it works fine

Sorry for my bad english.
@Developers please fix it !

Greets from germany
Elch

[busy_one]

It works again!

Never mind your english as long as your PHP is so awesome! 

I suspected something like that, but weren't able to verify, less fix it - many, many thanks LosElchos!

[Norv]

Thank you for sharing your solution!

Moving to bug reports.

[Norv]

Please accept my apologies for a huge typo: in Settings.php, UTF8 forums should normally have the line:

Code Select Expand


$db_character_set = 'utf8';


Was this in your Settings file before the upgrade? Specially, was it in the file after the upgrade? Is it in Settings_bak.php by any chance?

[LosElchos]

@Norv
Hi, I had check this, but there wasn't an entry in the old Settings_bak.php or in the new Settings.php.
After I added this line in the settings, and remove my code, the DB connection works fine.

But why it works before? In the 2.0 RC3?

I think the best way is that UTF8 should be the standard for SMF, all languages and secial characters are supported and there are no conflicts between iso and utf8 or an other collation.

Nice Weekend and thanks for your help

greets from germany
Elch

[Northtech]

...
I think the best way is that UTF8 should be the standard for SMF, all languages and secial characters are supported and there are no conflicts between iso and utf8 or an other collation.
...

+1000. all in utf-8... it would be just fine.

[Norv]

I can confirm this is a bug. It is a bug in the upgrader since 2.0 RC1.2 (!), though only some particular "arrangement" of lines in Settings.php appears to trigger it - thus it is rather rare. In any case, the upgrade process might remove that line itself - the bak file is overwritten as well. It's being fixed as we speak.

About making SMF UTF8, I believe we are taking that into account for future versions. There may be some trade-offs, but perhaps the overall advantages for the admin to handle and for SMF developers' code maintenance may be worth it.

[busy_one]

it's not fixed in RC5?

After installing RC5 it's the same  � all over again.

please post the bug-ID at least.

[Norv]

We have added a fix to the upgrade, and couldn't replicate the bug anymore in the tests done.

Can you please tell, did you add the line mentioned here
http://www.simplemachines.org/community/index.php?topic=407762.msg2841033#msg2841033
to your Settings.php file after the RC4 upgrade?

Does it exist now, after the upgrade to RC5?

[busy_one]

Thanks Norv - that did it!