News:

SMF 2.1.4 has been released! Take it for a spin! Read more.

Main Menu

Convert to UTF-8?

Started by Petter B, March 07, 2017, 03:14:34 AM

Previous topic - Next topic

Petter B

I run a swedish forum and have never had problems with our letters åäö.
Only in some parts of the Tapatalk-app they look weird. Now I installed SMFpacks Social Login and when someone register a new account via FB, åäö doesn't work.
I get Petter Bergström instead of Petter Bergström

Is there an easy way to fix this or do I need to convert the forum to UTF-8?
I guess it´s not now because the language file says: Swedish ISO-8859-1

There is a setting in the forum admin: Convert database och data till UTF-8.
Should I press the convert-button? (feels scary...)
and do I in that case need to do all this first:
QuoteHow to convert to UTF-8[edit]
This procedure will work well if you have used the recommended character sets in your language files in the past.
1. Start with a backup of your database. Character set conversions nearly always work correctly, but it is best to be prepared for the unexpected.
2. Check your default language file (Administration Center » Languages » Edit Languages ) and make a note of the character set used.
3. Go to Forum Maintenance > Convert the database and data to UTF-8 (this option will only be available if SMF detects a database version which supports UTF-8).
4. Select the character sets for your data (member posts) and database. By default, SMF will choose the character set of your default language file.
5. Press proceed, and your database will be converted. Depending on the size of your database, the conversion process might stop temporarily from time-to-time to avoid overloading the server. If this was successful, your forum should be converted to UTF-8.
6. For each of language pack currently in use, replace it with the UTF-8 version of that language pack.
7. Once all the UTF-8 language packs have been installed, convert the language settings of each user by running this query:
UPDATE smf_members
SET lngfile = CONCAT(lngfile, '-utf8')
WHERE lngfile != ''
8. In your Administration Center, change the default language, ensuring that you choose the UTF-8 version.
9. Check to see if all your data was properly converted.
10. If any of your posts contain HTML entities, you will want to convert these to UTF-8 as well by running "Convert HTML-entities to UTF-8 characters"

Illori

yes you need to convert to UTF-8.

Petter B

OK thanks.
So I just do the list above?
I have a very big forum, are there chances this will break it?

Illori

then do a backup of your database in case of any issue you can always revert back.

Petter B

Instead of downloading the BIG database, is it OK to just make copy of it in the phpMyAdmin?
(don´t remember the command for this...)


shawnb61

That looks like an excellent set of instructions... 

Something to look out for:
I recall some mods needed to be reinstalled, specifically the ones that made entries to the language files. 
Address the process rather than the outcome.  Then, the outcome becomes more likely.   - Fripp

shawnb61

Quote from: Petter B on March 07, 2017, 08:17:42 AM
Instead of downloading the BIG database, is it OK to just make copy of it in the phpMyAdmin?
(don´t remember the command for this...)

That - very helpful - feature is under 'Operations' in phpMyAdmin. 

I do that all the time on my test environment under localhost.   

I have never tried it on my ISP, the main reason is that in the past, any DBs not created under their database cPanel applet were not properly accessible.  (E.g., if created using a CREATE DATABASE in pmpMyAdmin it ended up in limbo...  They really wanted DBs created via cPanel...)   

So...   I would experiment with the phpMyAdmin COPY function to confirm it works as expected & the results are usable. 
Address the process rather than the outcome.  Then, the outcome becomes more likely.   - Fripp

Petter B

Tried this in my dev-server now but got:

QuoteDuplicate entry 'vår-4328' for key 'PRIMARY'
Fil: /home/d/dev/www/Sources/ManageMaintenance.php
Rad: 705

NOTE: It appears that your database may need to be upgraded. Your forum files belonging to the SMF version 2.0.13, while your database belong to version 2.0.2. It is very possible that this error can be resolved if you're running the latest version of Upgrade.php.

What/where is Upgrade.php and should I run it?

Illori

you dont need to run upgrade.php there are no database changes to apply. you do need to fix the duplicate entry though.

admin -> security and moderation enable include database query in the error log, run the convert again and check your error log for what query is failing then go from there to fix the issue.

Petter B


Petter B

#11
After a few tries I put forum in maintainance mode and got to run the conversion for a couple of minutes but then I got 500 (Internal Server Error)...  >:(

And now I can not log in as admin since my password contains the letter ä.
Glad it´s only my test forumet that I screwed up..  :P

I´m gonna forget this whole UTF8-thing, forum works fine without it. Except for social login via FB. Is there a workaround for that?

Kindred

well 2.1 is ONLY UTF-8, so you'll have to do it sometime...

as for facebook, you would have to ask the mod author
Слaва
Украинi

Please do not PM, IM or Email me with support questions.  You will get better and faster responses in the support boards.  Thank you.

"Loki is not evil, although he is certainly not a force for good. Loki is... complicated."

Petter B

Thanks, giving the test forum another try.
About this:
QuoteUPDATE smf_members
SET lngfile = CONCAT(lngfile, '-utf8')
WHERE lngfile != ''

-should I run it exactly like this or put the name of the language file in it somewhere?

Arantor

Exactly like that. It works for all languages.

Petter B

#15
Hmmm.
Seems like only 14 rows got affected, and no difference in forum, still strange characters...

"Affected rows: 15 (Query took 0.3936 sec)"

@rjen

Did you check this post:

http://www.simplemachines.org/community/index.php?topic=357401.msg2435528#msg2435528

QuoteChanging the users' language settings is not done in any file. You go into phpMyAdmin and in the SQL tab run the given query to change everyone's <language> to <language>-utf8. Of course, first you will check to see if any (or all) are already -utf8 (which means the help page needs updating!). After running the SQL to update the entries, list them all again to make sure all where changed and that you don't have any unexpected language files that you need to add (e.g., you had "albanian", but you forgot to load "albanian-utf8". Make sure that your default language is likewise updated. Make sure that Settings.php has the entry for "UTF8" now added to it, so that the pages will be displayed with UTF8 encoding rather than Latin-1.

Running SMF 2.1 with latest TinyPortal at www.fjr-club.nl

Petter B

Thanks! It seems like the field is empty for all except 15 members.
How can I change the command:
UPDATE smf_members
SET lngfile = CONCAT(lngfile, '-utf8')
WHERE lngfile != ''

So that everyone becomes Swedish UTF-8 ( sv_SE.utf8  ) ?

Arantor

You don't change every record, blank means "use the forum default". So if the forum default is that, they'll use that.

Petter B

Yes, I have set swedish UTF-8 as standard.
OK, what might be wrong then?
Still strange letters or words cut of after åäö...

Advertisement: