Advertisement:

Author Topic: Character corruption upon inline edit after 2.0.16 update  (Read 1218 times)

Offline spiros

  • Language Moderator
  • SMF Hero
  • *
  • Posts: 1,796
  • Gender: Male
  • A different point of view
    • spiros.doikas on Facebook
    • doikas on LinkedIn
    • @greektranslator on Twitter
    • Greek Translation
Character corruption upon inline edit after 2.0.16 update
« on: December 31, 2019, 06:13:55 AM »
When using inline edit on topic level, I get non-ASCII character conversion in subject and body text to Unicode Character Names. It happens with all tested themes. It also happens in a vanilla forum just after installing 2.0.16

https://www.translatum.gr/forum/index.php
« Last Edit: December 31, 2019, 06:56:34 AM by spiros »

Offline bulatus

  • Newbie
  • *
  • Posts: 4
Re: Character corruption upon inline edit after 2.0.16 update
« Reply #1 on: January 02, 2020, 02:58:22 AM »
Hello everyone!!!
I confirm that after updating to 2.0.17 there was a problem with encoding. utf8
How do I disable this feature?

Offline spiros

  • Language Moderator
  • SMF Hero
  • *
  • Posts: 1,796
  • Gender: Male
  • A different point of view
    • spiros.doikas on Facebook
    • doikas on LinkedIn
    • @greektranslator on Twitter
    • Greek Translation
Re: Character corruption upon inline edit after 2.0.16 update
« Reply #2 on: January 02, 2020, 05:26:37 AM »
I had to remove the inline edit button in the theme template.

Offline bulatus

  • Newbie
  • *
  • Posts: 4
Re: Character corruption upon inline edit after 2.0.16 update
« Reply #3 on: January 02, 2020, 05:32:11 AM »
Hi!!! How did you delete this button? In what file and what line???

Offline spiros

  • Language Moderator
  • SMF Hero
  • *
  • Posts: 1,796
  • Gender: Male
  • A different point of view
    • spiros.doikas on Facebook
    • doikas on LinkedIn
    • @greektranslator on Twitter
    • Greek Translation
Re: Character corruption upon inline edit after 2.0.16 update
« Reply #4 on: January 02, 2020, 05:36:58 AM »
Depends on your theme. For example in default theme, Display.template.php, remove this

Code: [Select]
// Can the user modify the contents of this post?  Show the modify inline image.
if ($message['can_modify'])
echo '
<img src="', $settings['images_url'], '/icons/modify_inline.gif" alt="', $txt['modify_msg'], '" title="', $txt['modify_msg'], '" class="modifybutton" id="modify_button_', $message['id'], '" style="cursor: ', ($context['browser']['is_ie5'] || $context['browser']['is_ie5.5'] ? 'hand' : 'pointer'), '; display: none;" onclick="oQuickModify.modifyMsg(\'', $message['id'], '\')" />';

Offline Erros

  • Newbie
  • *
  • Posts: 1
Re: Character corruption upon inline edit after 2.0.16 update
« Reply #5 on: January 02, 2020, 08:02:56 PM »
I had to remove the inline edit button in the theme template.
Just leave script.js from 2.0.15 version. For me after that inline edit works well again.

Online shawnb61

  • Developer
  • SMF Hero
  • *
  • Posts: 1,711
    • sbulen on GitHub
Re: Character corruption upon inline edit after 2.0.16 update
« Reply #6 on: January 02, 2020, 09:13:36 PM »
Just leave script.js from 2.0.15 version. For me after that inline edit works well again.

Pretty sure that will re-introduce two separate bugs - the multi-byte preview problem, and the Firefox code select bug. 
Address the process rather than the outcome.  Then, the outcome becomes more likely.   - Fripp

Offline Vanomas

  • Newbie
  • *
  • Posts: 2
Re: Character corruption upon inline edit after 2.0.16 update
« Reply #7 on: January 03, 2020, 10:16:27 AM »
Pretty sure that will re-introduce two separate bugs - the multi-byte preview problem, and the Firefox code select bug.
I can't confirm it has returned, but I think it's much lesser evil than destructive consequences of that unpleasant behavior for Cyrillic world.

Offline spiros

  • Language Moderator
  • SMF Hero
  • *
  • Posts: 1,796
  • Gender: Male
  • A different point of view
    • spiros.doikas on Facebook
    • doikas on LinkedIn
    • @greektranslator on Twitter
    • Greek Translation
Re: Character corruption upon inline edit after 2.0.16 update
« Reply #8 on: January 03, 2020, 11:20:33 AM »
Absolutely.
It is difficult for someone who runs Latin-character-based forums to appreciate the destructiveness of this bug  :)

Online shawnb61

  • Developer
  • SMF Hero
  • *
  • Posts: 1,711
    • sbulen on GitHub
Re: Character corruption upon inline edit after 2.0.16 update
« Reply #9 on: January 03, 2020, 12:37:33 PM »
Understood.  Just sharing the tradeoffs. 
Address the process rather than the outcome.  Then, the outcome becomes more likely.   - Fripp

Offline delysid

  • Semi-Newbie
  • *
  • Posts: 13
  • Gender: Male
  • https://wotcheats.ru
    • lsdmax on GitHub
    • https://wotcheats.ru
Re: Character corruption upon inline edit after 2.0.16 update
« Reply #10 on: January 04, 2020, 05:09:31 PM »
Just leave script.js from 2.0.15 version. For me after that inline edit works well again.

Pretty sure that will re-introduce two separate bugs - the multi-byte preview problem, and the Firefox code select bug.

What errors do you write about if users edit their posts and this will kill all the forums?
It is like a terrorist attack.

Offline spiros

  • Language Moderator
  • SMF Hero
  • *
  • Posts: 1,796
  • Gender: Male
  • A different point of view
    • spiros.doikas on Facebook
    • doikas on LinkedIn
    • @greektranslator on Twitter
    • Greek Translation
Re: Character corruption upon inline edit after 2.0.16 update
« Reply #11 on: January 05, 2020, 02:54:21 AM »
Indeed, pretty minor (and not affecting the text output) compared to this.

Offline delysid

  • Semi-Newbie
  • *
  • Posts: 13
  • Gender: Male
  • https://wotcheats.ru
    • lsdmax on GitHub
    • https://wotcheats.ru
Re: Character corruption upon inline edit after 2.0.16 update
« Reply #12 on: January 06, 2020, 09:27:33 AM »
In my forum https://wotcheats.ru [nofollow] (Russian lang) this FIX work very well.
Change script.js
Code: [Select]
// Convert a string to an 8 bit representation (like in PHP).
String.prototype.php_to8bit = function ()
{
if (smf_charset == 'UTF-8')
{
return this;
}
















else if (this.oCharsetConversion.from.length == 0)


TO

Code: [Select]
// Convert a string to an 8 bit representation (like in PHP).
String.prototype.php_to8bit = function ()
{
if (smf_charset == 'UTF-8')
{
var n, sReturn = '';

for (var i = 0, iTextLen = this.length; i < iTextLen; i++)
{
n = this.charCodeAt(i);
if (n < 128)
sReturn += String.fromCharCode(n)
else if (n < 2048)
sReturn += String.fromCharCode(192 | n >> 6) + String.fromCharCode(128 | n & 63);
else if (n < 65536)
sReturn += String.fromCharCode(224 | n >> 12) + String.fromCharCode(128 | n >> 6 & 63) + String.fromCharCode(128 | n & 63);
else
sReturn += String.fromCharCode(240 | n >> 18) + String.fromCharCode(128 | n >> 12 & 63) + String.fromCharCode(128 | n >> 6 & 63) + String.fromCharCode(128 | n & 63);
}

return sReturn;

}
else if (this.oCharsetConversion.from.length == 0)


Online shawnb61

  • Developer
  • SMF Hero
  • *
  • Posts: 1,711
    • sbulen on GitHub
Re: Character corruption upon inline edit after 2.0.16 update
« Reply #13 on: January 07, 2020, 01:03:52 AM »
This issue is tracked in the internal log as #123/PR#124 & will target 2.0.18.

Any changes you make must be reversed before you apply 2.0.18. 

The changes I would recommend are:

Find this:
Code: [Select]
if (smf_charset == 'UTF-8')
{
return this;
}

Replace with:
Code: [Select]
if (smf_charset == 'UTF-8')
{
var n, sReturn = '';

// Recode from UTF16 (native .js) to UTF8
for (var i = 0, iTextLen = this.length; i < iTextLen; i++)
{
// Below xFFFF, UTF16 simply = the code points
n = this.charCodeAt(i);
if (n < 128)
sReturn += String.fromCharCode(n);
else if (n < 2048)
sReturn += String.fromCharCode(192 | n >> 6) + String.fromCharCode(128 | n & 63);
// 0xD800 - 0xDBFF
else if (n >= 55296 && n <= 56319)
{
// In this range, this is the beginning of a surrogate pair, where 4-byte utf8 chars are
n = 65536 + ((n & 1023) << 10) + (this.charCodeAt(i + 1) & 1023);
sReturn += String.fromCharCode(240 | n >> 18) + String.fromCharCode(128 | n >> 12 & 63) + String.fromCharCode(128 | n >> 6 & 63) + String.fromCharCode(128 | n & 63);
// Skip next char, already used...
i++;
}
else
sReturn += String.fromCharCode(224 | n >> 12) + String.fromCharCode(128 | n >> 6 & 63) + String.fromCharCode(128 | n & 63);
}

return sReturn;
}

And also find this:
Code: [Select]
String.prototype.php_urlencode = function()
{
if (smf_charset == 'UTF-8')
return encodeURIComponent(this);

return escape(this).replace(/\+/g, '%2b').replace('*', '%2a').replace('/', '%2f').replace('@', '%40');
}

Replace with:
Code: [Select]
String.prototype.php_urlencode = function()
{
return escape(this).replace(/\+/g, '%2b').replace('*', '%2a').replace('/', '%2f').replace('@', '%40');
}

I believe this will address all of the issues.
Address the process rather than the outcome.  Then, the outcome becomes more likely.   - Fripp

Offline delysid

  • Semi-Newbie
  • *
  • Posts: 13
  • Gender: Male
  • https://wotcheats.ru
    • lsdmax on GitHub
    • https://wotcheats.ru
Re: Character corruption upon inline edit after 2.0.16 update
« Reply #14 on: January 07, 2020, 03:31:08 AM »
I believe this will address all of the issues.
Checked. All is well. Thanks

Offline Hatshepsut

  • Semi-Newbie
  • *
  • Posts: 69
  • Gender: Male
    • bgnacionalisti.org/ on Facebook
    • Bulgarian Nationalist Forum
Re: Character corruption upon inline edit after 2.0.16 update
« Reply #15 on: January 16, 2020, 02:13:02 AM »
This fix doesn't work for me :(
I am using Bulgarian language (Bulgarian UTF-8 language pack). I have removed 'Modify' button from post body and problem disappeared.

Forum details:

Forum version: SMF 2.0.17 (more detailed)
Current SMF version: SMF 2.0.17
GD version: bundled (2.1.0 compatible)
Database Server: MariaDB
MySQL version: 10.3.21-MariaDB-log
PHP: 5.4.45
Server version: Apache

Offline spiros

  • Language Moderator
  • SMF Hero
  • *
  • Posts: 1,796
  • Gender: Male
  • A different point of view
    • spiros.doikas on Facebook
    • doikas on LinkedIn
    • @greektranslator on Twitter
    • Greek Translation
Re: Character corruption upon inline edit after 2.0.16 update
« Reply #16 on: January 16, 2020, 03:12:43 AM »
I would recommend updating php.