Advertisement:
« previous next »
Pages: [1]   Go Down

Author Topic: Emojis are causing PMs to not send full content  (Read 2083 times)

Offline mark7144

  • Sophist Member
  • *****
  • Posts: 1,174
Emojis are causing PMs to not send full content
« on: May 28, 2018, 07:14:53 AM »
I've noticed that when you insert emojis (via phone keyboard) into a PM, everything after that point fails to reach the recipient. However, the full contents actually shows in the email notification they get.

Does anyone know how I can fix this?
Logged

Offline GigaWatt

  • The Smiley Guy
  • SMF Friend
  • SMF Hero
  • *
  • Posts: 2,187
  • Gender: Male
    • Macedonian electronics forum
Re: Emojis are causing PMs to not send full content
« Reply #1 on: May 28, 2018, 08:29:44 AM »
Probably a UTF-8 database problem. See if the content column in the private messages table is set as UTF-8.

I would advise you not to insert emojis in posts and PMs. I had a similar problem on my forum and I've asked the members (in a separate thread) not to insert emojis in posts, but to use the standard smileys available in the post editor.

Basically, emojis are some sort of unicode characters and I have no idea if they are UTF-8 standardized. They work on mobile devices, but they are not like normal smileys, like the ones on forums created by 2 or more ANSI characters. They are separate symbols.
Logged
"This is really a generic concept about human thinking - when faced with large tasks we're naturally inclined to try to break them down into a bunch of smaller tasks that together make up the whole."

"A 500 error loosely translates to the webserver saying, "WTF?"..."

Offline mark7144

  • Sophist Member
  • *****
  • Posts: 1,174
Re: Emojis are causing PMs to not send full content
« Reply #2 on: May 28, 2018, 08:44:20 AM »
Thanks for replying.

So currently under smf_personal_messages, the 'body' collation is set as 'utf8_general_ci'. So to fix this do I simply change that to 'utf8mb4_unicode_ci'? Is that relatively safe to do?
Logged

Offline Illori

  • Project Manager
  • SMF Legend
  • *
  • Posts: 52,251
Re: Emojis are causing PMs to not send full content
« Reply #3 on: May 28, 2018, 08:49:55 AM »
are you using SMF 1.1.*? if so i would suggest you upgrade to SMF 2.0. i believe SMF 2.0 has support for emojis at least to the point of not breaking the post like you are seeing.
Logged

Offline mark7144

  • Sophist Member
  • *****
  • Posts: 1,174
Re: Emojis are causing PMs to not send full content
« Reply #4 on: May 28, 2018, 08:52:18 AM »
I do want to upgrade but I'm holding back for now as it's such a daunting task. My website is heavily integrated using SSI code and my theme is basically custom - I made it mobile friendly myself which took a long time. Also, this particular forum of mine is past its peak now so I don't mind just trying to keep it chugging along. :)
Logged

Offline Illori

  • Project Manager
  • SMF Legend
  • *
  • Posts: 52,251
Re: Emojis are causing PMs to not send full content
« Reply #5 on: May 28, 2018, 08:56:13 AM »
do keep in mind that SMF 1.1.* is not getting security patches any longer.
Logged

Offline mark7144

  • Sophist Member
  • *****
  • Posts: 1,174
Re: Emojis are causing PMs to not send full content
« Reply #6 on: May 28, 2018, 02:45:19 PM »
Yeah, I admit that's a little unsettling.
Logged

Offline GigaWatt

  • The Smiley Guy
  • SMF Friend
  • SMF Hero
  • *
  • Posts: 2,187
  • Gender: Male
    • Macedonian electronics forum
Re: Emojis are causing PMs to not send full content
« Reply #7 on: May 28, 2018, 05:42:12 PM »
Quote from: mark7144 on May 28, 2018, 08:44:20 AM
So currently under smf_personal_messages, the 'body' collation is set as 'utf8_general_ci'. So to fix this do I simply change that to 'utf8mb4_unicode_ci'? Is that relatively safe to do?

I would advise you not to do that, you may break forum PMs (make characters question marks or unreadable characters). "utf8_general_ci" is the way to go. If you decide to do this, do a database backup first.

And I just noticed that this post is in the 1.1.x. As far as I know, SMF 1.1.x doesn't officially support UTF-8 out of the box (someone correct me if I'm wrong). There are "hacks" that make 1.1.x UTF-8 compatible, but I wouldn't go further than "utf-8_general_ci".

Please post results if you happen to try it and it works ;).
Logged
"This is really a generic concept about human thinking - when faced with large tasks we're naturally inclined to try to break them down into a bunch of smaller tasks that together make up the whole."

"A 500 error loosely translates to the webserver saying, "WTF?"..."

Offline mark7144

  • Sophist Member
  • *****
  • Posts: 1,174
Re: Emojis are causing PMs to not send full content
« Reply #8 on: May 28, 2018, 06:02:03 PM »
It was already set as utf8_general_ci originally. Did you mean to say that?

Anyway, I actually tried changing it to 'utf8mb4_unicode_ci' before I read your post. And while emojis only show as question marks, it did resolve the major flaw of missing private message content. So far not noticing any other issues thankfully.

Please let me know your thoughts? :)
Logged

Offline GigaWatt

  • The Smiley Guy
  • SMF Friend
  • SMF Hero
  • *
  • Posts: 2,187
  • Gender: Male
    • Macedonian electronics forum
Re: Emojis are causing PMs to not send full content
« Reply #9 on: May 28, 2018, 06:54:39 PM »
Quote from: mark7144 on May 28, 2018, 06:02:03 PM
It was already set as utf8_general_ci originally. Did you mean to say that?

No, I meant if you set the column from "utf-8_general_ci" to "utf8mb4_unicode_ci", post results ;).

As for the other part of your post... good to know ;). I wouldn't set this collation as this might cause problems in future versions of SMF and database conversions from one version to another, but it's your forum, your choice :).

If there was a way to block emojis from mobile devices, I would implement it on my forum, but as far as I know, there isn't one, so I just hope the members obey the "no emoji request" i posted in a thread in their PMs. As for the public part of the forum, I usually do manual edits, although I haven't done one in a while, not since I posted the request.

Just to be clear, there are no problems with emojis on SMF 2.0.x. On some browsers they load, on others, they don't (I think PPAPI browsers loaded them correctly, NPAPI didn't... well, not all of them, I usually use Pale Moon, a Firefox fork and it loaded the emojis correctly), but the topics and posts are accessible and readable. Though, I have to admit, I haven't looked in the error log :S.

So, the bottom line is, I asked the members not to use emojis as a precaution. I don't know what decisions will be made in future versions of server OSes, database managers, SMF, PHP, etc. Most likely, things will go in an upward direction (will support these symbols out of the box in future versions), but better safe than sorry ;). Besides, it's not such a big deal, they could always use the ANSI combinations to generate smileys. Sure, it's a bit more complicated, but hey... that's life :P.
Logged
"This is really a generic concept about human thinking - when faced with large tasks we're naturally inclined to try to break them down into a bunch of smaller tasks that together make up the whole."

"A 500 error loosely translates to the webserver saying, "WTF?"..."

Offline mark7144

  • Sophist Member
  • *****
  • Posts: 1,174
Re: Emojis are causing PMs to not send full content
« Reply #10 on: May 29, 2018, 04:04:41 AM »
Ok great. Thanks for your help here :)
Logged

Offline GigaWatt

  • The Smiley Guy
  • SMF Friend
  • SMF Hero
  • *
  • Posts: 2,187
  • Gender: Male
    • Macedonian electronics forum
Re: Emojis are causing PMs to not send full content
« Reply #11 on: May 29, 2018, 09:32:25 AM »
No problem :).

And don't forget to mark the topic as solved ;).
Logged
"This is really a generic concept about human thinking - when faced with large tasks we're naturally inclined to try to break them down into a bunch of smaller tasks that together make up the whole."

"A 500 error loosely translates to the webserver saying, "WTF?"..."
Pages: [1]   Go Up
« previous next »