Non-ASCII character make posts disappear?

[ArMaP]

In a forum I help administer we made a new installation and, with phpMyAdmin, I exported the old database and imported the data into a new one. At first it all looked OK, but I must have made a "zig" instead of a "zag" somewhere and some posts are now missing.

After a little search through the forum and the database I found out that what was causing the problem was the horizontal ellipsis character (...), removing it from the post (directly in the database, the only way I could find to access the posts' text) solved the problem. Replacing it with the corresponding HTML entity code also worked. I also found a post with apostrophes (probably copied from an email, according to the context of that post) that were also making the whole post appear blank.

Can anyone tell me what I did wrong and how to solve it?

Thanks in advance.

Edited to add that, although I could try running an update command on the database to replace the known culprits would work, it's better to know how to really do it, in case I have to do this process again.

I forgot to add that inserting those characters on a new post works as expected, although they appear as strange characters on the database.

[Kindred]

Change your forum to UTF8

See the FAQ/wiki on how to do that

[ArMaP]

I thought about that, but the old forum wasn't in UTF8, or at least it didn't look like it to me.

I'll try that.

PS: is that something we choose while installing? I don't remember. 

[Sir Osis of Liver]

Look in the database and check the table collation.  If the db is already utf8, I believe the conversion will fail.  If that happens, manually add this line to Settings.php -

Code Select Expand



     $db_character_set = 'utf8';



[ArMaP]

Thanks everybody for the answers, converting to UTF-8 worked.

[whoey]

FYI, this also worked for us, we had a severe crash that wiped the config file, and the repaired config file was missing the utf-8 line