News:

Want to get involved in developing SMF, then why not lend a hand on our github!

Main Menu

iPhone smileys (emojis) cuts off posts

Started by hebron, July 30, 2014, 05:41:38 AM

Previous topic - Next topic

hebron

Hi

I am having issues with iPhone smileys on my forum, whenever a user inserts a iPhone smiley the post cuts off... So the first iPhone smiley basically decides the length of the post. Users report having this issue both when using Tapatalk and when posting directly on the website.

I have googled, and searched this forum without much luck. Can anyone point me in the right direction? Debugging is made more difficult by the fact that I do not have a iPhone... But any pointers on what might cause posts to cut is greatly appreciated.

I am using SMF 2.0.8, with Norwegian UTF-8 language.

Thank you.

Arantor

Link to an example where this is happening?

hebron

#2
Quote from: ‽ on July 30, 2014, 05:43:27 AM
Link to an example where this is happening?

Thank you your quick response. Here are some examples:
<links removed>

There are no strange symbols in the posts, they just cut. I've checked the database, and it doesn't contain the rest of the text either.

Arantor

What's the collation of the body column in smf_messages?

hebron


Arantor

Interesting. I was going to suspect a mismatch between database encoding and everything else but you're UTF-8 through and through.

This may well prove to be a bug in SMF itself (since the posting routine predates the iPhone!)

hebron

That was my initial suspicion as well, but I didn't find any. If it is indeed a bug in the SMF system I find it strange that I can't seem to find anyone else with the same problem. I'm guessing a lot of people use their iPhone when posting.

I do know that whenever someone sends me iPhone smileys my BlackBerry doesn't understand any of them and just prints squares.

Any hints to what source file I can start digging in? Or where I go from here?

BTW: The forum is normally closed for guests, i opened it temporary. Can I close it again?

Arantor

Sure, you can definitely close it again.

The thing about such emoji is that they are special characters and something doesn't like them. I've not seen them used here, though.

What mods do you have installed?

hebron

My mods:
Tapatalk SMF 2.0 Plugin   4.0.1
SMF Gallery Lite   3.7
SimplePortal   2.3.5

Theme:
Blue Star Theme

In addition to that I have written a custom scheduled task and action feature.

Kindred

are you posting form your iphone to your site through tapatalk or directly?

(and, either way, the reason it might not have been noticed is that this is the first time I have heard of someone using the iphone emoji rather than the standard smilies)
Слaва
Украинi

Please do not PM, IM or Email me with support questions.  You will get better and faster responses in the support boards.  Thank you.

"Loki is not evil, although he is certainly not a force for good. Loki is... complicated."

hebron

Quote from: Kindred on July 30, 2014, 07:16:11 AM
are you posting form your iphone to your site through tapatalk or directly?

(and, either way, the reason it might not have been noticed is that this is the first time I have heard of someone using the iphone emoji rather than the standard smilies)
Hi, I have not posted myself. But I've had a couple of members that have experienced the error, and they have tested and confirmed that it indeed was the iPhone smileys that caused the post to cut. One member tried posting both through Tapatalk and directly, with the same result. The post cut off at the first iPhone smiley.

It should be very easy to replicate, but I don't have, or have access to an iPhone... I had one of the members send the iPhone emotions to me on e-mail, and they show up as black squares in my webmail, like this:

Hm, that was interesting! When I tried to copy and paste the black squares from my webmail I got the follow error message when previewing this message: The following error or errors occurred while posting this message: The message body was left empty. And all my text duplicated and screwed up the entire entry.

I am going to try and get the HTML code for the symbol and add it to the censored words. Let you know how that goes.

Kindred

well, that is the point...  I don't think that iPhone emojis have an html code...

wow...   actually, I see LOTS of articles on going the other way - converting html to emoji, but - with albeit a quick search, nothing about converting iphone emoji back to html.


edit - found one..
http://www.iamcal.com/emoji-in-web-apps/

and, from this article it looks like it is a definite bother and would require new code to handle...

blech.
I hate apple.
Слaва
Украинi

Please do not PM, IM or Email me with support questions.  You will get better and faster responses in the support boards.  Thank you.

"Loki is not evil, although he is certainly not a force for good. Loki is... complicated."

hebron

You are right, gawd damn Apple...

Thank you for the article, I think I'm going to try and test with implementing "Emoji for PHP" solution in my posting source. But it seems like more of a work around than a solution. Maybe a more permanent solution to this is something to consider for future SMF versions? Seems like the use of emojis isn't going to decline any time soon.

hebron

On second thought, I think I'm going to try and filter them out and remove them instead. Supporting them is not really important, but I can't have them cutting my members posts. That gets a bit frustrating.

I'm thinking a regex replace on the submitted post title and body. Any ideas and pointers would be greatly appreciated, and thank you for pointing me in the right direction :-)

hebron

Okey, it seems that I can use http://htmlpurifier.org/ to clear out the unwanted symbols. But I am having some problems locating where the text is actually saved, I tried applying it to the beginning of function Post2 in Sources/Post.php, but this only seems to apply to the preview...Any ideas?

Illori

touchpal keyboard on android has emoji's

i can enter them just fine, post the message and the message seems to be 100% empty as there was no other text in it.

phpmyadmin has nothing as the message body at all. [tested with current 2.1 which is utf-8 by default]

margarett

I'm posting from iPhone and this is an emoji
Se forem conduzir, não bebam. Se forem beber... CHAMEM-ME!!!! :D

QuoteOver 90% of all computer problems can be traced back to the interface between the keyboard and the chair

margarett

Well, it seems real :o
I posted a bigger message with more text and emojis...
Se forem conduzir, não bebam. Se forem beber... CHAMEM-ME!!!! :D

QuoteOver 90% of all computer problems can be traced back to the interface between the keyboard and the chair

Illori

Quote from: margarett on July 30, 2014, 01:50:11 PM
Well, it seems real :o
I posted a bigger message with more text and emojis...

yep tested again with android/touchpal and can duplicate the same with 2.1. i just think that utf-8 cant handle the characters so it removes them?

hebron

So I have added this to the beginning of the Post2 function in Source/Post.php:

require_once '/path/to/library/HTMLPurifier.auto.php';

$config = HTMLPurifier_Config::createDefault();
$purifier = new HTMLPurifier($config);
$_REQUEST['message'] = $purifier->purify($_REQUEST['message']);


Right after:

        // No need!
        $context['robot_no_index'] = true;


And I no longer get an error when previewing a post with emojis, they are just removed and all text is kept. But no change when I post, then all text after the first emojis is lost. How can I also apply this filter when posting? I've read trough the function but am no closer to understanding why...

Advertisement: