Advertisement:

Author Topic: Before I upgrade my forum to UTF-8 ..  (Read 655 times)

Offline MobileCS

  • Jr. Member
  • **
  • Posts: 238
Before I upgrade my forum to UTF-8 ..
« on: February 12, 2021, 11:05:52 PM »
I noticed that 48,000+ members on my forum do not have a language set. These are all the latest members. I saw in the instructions that I need to do a "SET lngfile = CONCAT(lngfile, '-utf8')".

My questions are :

Why is SMF not setting the default "English" to "lngfile" in smf_members in the database?

Do I need those rows to be populated if I have the forum set to "English (UTF-8)" and disable "Enable user-selectable language support"?


Also, I've noticed that half my mods do not have any english-utf8.php language files.

Do I need to create those files for them? or will they run fine without them after the conversion?

Offline shawnb61

  • Dev. Team Mentor
  • Developer
  • SMF Hero
  • *
  • Posts: 3,113
    • sbulen on GitHub
Re: Before I upgrade my forum to UTF-8 ..
« Reply #1 on: February 12, 2021, 11:36:05 PM »
Blank = use the forum default.  You're good there as long as you change the forum default.

For english, there no difference between the UTF8 & non-UTF8 versions.  If missing text for your mods, I am pretty sure you can just copy them. 
Address the process rather than the outcome.  Then, the outcome becomes more likely.   - Fripp

Offline Kindred

  • The Mean One
  • Support Specialist
  • SMF Legend
  • *
  • Posts: 60,210
  • Gender: Male
    • Kindred-999 on GitHub
Re: Before I upgrade my forum to UTF-8 ..
« Reply #2 on: February 13, 2021, 01:16:48 AM »
Not quite true... there is ONE line difference...ISO vs UTF8
Please do not PM, IM or Email me with support questions.  You will get better and faster responses in the support boards.  Thank you.

"Loki is not evil, although he is certainly not a force for good. Loki is... complicated."

Online @rjen

  • Sr. Member
  • ****
  • Posts: 708
  • Gender: Male
    • FJR-club Nederland
Re: Before I upgrade my forum to UTF-8 ..
« Reply #3 on: February 13, 2021, 02:53:28 AM »
Not quite true... there is ONE line difference...ISO vs UTF8

I am pretty sure that line actually does not DO anything...
Running SMF 2.0 with Tinyportal 2.0.1 at www.fjr-club.nl
Testing SMF 2.1 with Tinyportal 2.1.0 at test2.fjr-club.nl

Offline Kindred

  • The Mean One
  • Support Specialist
  • SMF Legend
  • *
  • Posts: 60,210
  • Gender: Male
    • Kindred-999 on GitHub
Re: Before I upgrade my forum to UTF-8 ..
« Reply #4 on: February 13, 2021, 09:15:53 AM »
It does. It's a setting and is required to actually DISPLAY the site in UTF8
Please do not PM, IM or Email me with support questions.  You will get better and faster responses in the support boards.  Thank you.

"Loki is not evil, although he is certainly not a force for good. Loki is... complicated."

Offline shawnb61

  • Dev. Team Mentor
  • Developer
  • SMF Hero
  • *
  • Posts: 3,113
    • sbulen on GitHub
Re: Before I upgrade my forum to UTF-8 ..
« Reply #5 on: February 13, 2021, 09:43:27 AM »
It does. It's a setting and is required to actually DISPLAY the site in UTF8

Actually that's what global_character_set is for.  But it's true that the language string is used when global_character_set isn't set.

It's also displayed on the language screen.  If nothing else,  it's confusing to leave saying ISO & should be changed.


Address the process rather than the outcome.  Then, the outcome becomes more likely.   - Fripp

Offline shawnb61

  • Dev. Team Mentor
  • Developer
  • SMF Hero
  • *
  • Posts: 3,113
    • sbulen on GitHub
Re: Before I upgrade my forum to UTF-8 ..
« Reply #6 on: February 13, 2021, 12:09:58 PM »
Note the original question & response was for language files for mods

Address the process rather than the outcome.  Then, the outcome becomes more likely.   - Fripp

Offline MobileCS

  • Jr. Member
  • **
  • Posts: 238
Re: Before I upgrade my forum to UTF-8 ..
« Reply #7 on: February 13, 2021, 12:26:38 PM »
Ok, just to clarify ...

If a mod is missing the english-utf8.php file, it will just use the normal english.php as a language file when UTF-8 is enabled?

Offline shawnb61

  • Dev. Team Mentor
  • Developer
  • SMF Hero
  • *
  • Posts: 3,113
    • sbulen on GitHub
Re: Before I upgrade my forum to UTF-8 ..
« Reply #8 on: February 13, 2021, 01:19:42 PM »
I would give that a shot.  If the labels are missing, I'd rename it to the utf8 version. 
Address the process rather than the outcome.  Then, the outcome becomes more likely.   - Fripp

Offline MobileCS

  • Jr. Member
  • **
  • Posts: 238
Re: Before I upgrade my forum to UTF-8 ..
« Reply #9 on: February 17, 2021, 11:41:56 PM »
Are there any issues with using "utf8mb4_general_ci" character set in SMF 2.0.18?

I noticed on a fresh install, SMF uses a legacy "utf8_general_ci". MySQL says not to use this and to use utf8mb4 instead.

Offline shawnb61

  • Dev. Team Mentor
  • Developer
  • SMF Hero
  • *
  • Posts: 3,113
    • sbulen on GitHub
Re: Before I upgrade my forum to UTF-8 ..
« Reply #10 on: February 17, 2021, 11:56:14 PM »
I suggest sticking with utf8.  The DB & code both need changes to properly support utf8mb4.
Address the process rather than the outcome.  Then, the outcome becomes more likely.   - Fripp

Offline MobileCS

  • Jr. Member
  • **
  • Posts: 238
Re: Before I upgrade my forum to UTF-8 ..
« Reply #11 on: February 19, 2021, 12:14:30 AM »
The utf-8 conversion went fine, but the htmlentities -> utf-8 had this error :

Unsupported operand types: int / string

Any ideas how to solve that? Do I need to run this conversion? What would happen if I skipped that step?

I've restored my forum backup so everything is fine right now and back to iso-8859-1.


Offline shawnb61

  • Dev. Team Mentor
  • Developer
  • SMF Hero
  • *
  • Posts: 3,113
    • sbulen on GitHub
Re: Before I upgrade my forum to UTF-8 ..
« Reply #12 on: February 19, 2021, 12:31:04 AM »
If you do not do the step, there is no visible difference, though within the DB, some characters may be represented by html entities instead of characters. 

I.e., under the covers, they look like:
😪😫🥱😴😌😛🤓🤓🤓🤓🤓
Instead of:
😪😫🥱😴😌😛🤓🤓🤓🤓🤓

I have never seen that error before.  There should be more info in the error log...  Do you have any other info?  E.g., line #?
Address the process rather than the outcome.  Then, the outcome becomes more likely.   - Fripp

Offline MobileCS

  • Jr. Member
  • **
  • Posts: 238
Re: Before I upgrade my forum to UTF-8 ..
« Reply #13 on: February 19, 2021, 03:39:58 AM »
No errors showed in the error log. I don't see anything in Apache logs either which is really weird.