E-mail notifications: Entities in Greek characters message body

[spiros]

Not sure why this happens. Any help appreciated.

[shawnb61]

This is my understanding...

You'll see in the very first line of that image that it's displaying the message in 7-bit.  SMF sends messages out in both 7-bit for folks with much older mail clients and an base64 encoded format for newer clients.

Your email client normally picks one.  (And normally does a better job displaying it than that...)

I think the 7-bit version is slowly becoming more problematic for newer mail transports and newer mail clients.  They really should do better than that, it's not that complicated... Yet here we are.  My host actually strips the 7-bit version and only sends the encoded version when my forum sends email.

In SMF3.0, we're probably removing 7-bit entirely.  More here:
https://github.com/SimpleMachines/SMF/issues/8288

So...  What to do...

My input:
 - If the awkward display is restricted to specific email clients, I'd discourage the use of those clients.
 - I'd try delivering mail with & without html; they're processed a little differently & that might help.

If neither of the above work, and this is a problem for many users, you could try removing the 7-bit processing from your sendmail code, in Sub-Post.php.

[shawnb61]

I heartily suggest an upgrade to 2.1.  There have been a few mail fixes in 2.1, and it will provide access to supported versions of php, mysql. 

[spiros]

I fully get it that 2.1 is a must. The point is my forum is so heavily customized (also customized Sphinx and search results) that it will be hell to redo all that, given that there is a lot of work going on SMF 3.0 too. So my idea is if possible go directly to 3.0.

Not sure what change should I make on Subs-Post.php (could you supply a Find/Replace instruction?), I tried this but did not improve things.

Code Select Expand

$message = str_replace(array("\r", "\n"), array('', $line_break), $message);
// Decode any HTML entities (so "Σ" becomes "Σ", etc.)
$message = html_entity_decode($message, ENT_QUOTES, 'UTF-8');

PS. Interestingly, the subject of the e-mail had no encoding issue with Greek.

[Kindred]

Waiting for smf 3.0 is a poor choice - there no telling his long it will actualy be

[shawnb61]

This won't be a little tweak.  This will be removing several lines of code...

I've never done this before, & I haven't looked at 2.0 code in 3+ years.  But I can give you some guidance as to how I would approach this if I had to get rid of the alternate encodings, based on 2.1 code.

First it helps to see what we're doing...

If you look at the payload part of a forum email in raw format (email clients should let you view a message in 'raw' or 'original' format), it will look something like this, with two copies of the message payload:
You cannot view this attachment.

We want to remove all but the last one only, which would be the one that matches your encoding:
You cannot view this attachment.

I believe - not coded or tested - this will require removing pretty much everything highlighted in red here:
You cannot view this attachment.


If someone has actually done this before, it would be good to hear from you!

[spiros]

Thank you. If I get this right

These 2 instances

Code Select Expand

$message .= $encoded_message . $line_break . '--' . $mime_boundary . '--';
Must become this (first instance)

Code Select Expand

$message .= $encoded_message . $line_break . '';
Or this (second instance):

Code Select Expand

$message .= $encoded_message . $line_break . ';
And the other change should be:

Code Select Expand

$message 'Content-Type: text/plain; charset=' . $charset . $line_break;
And here is the full thing:

   

Code Select Expand

// Using mime, as it allows to send a plain unencoded alternative.
$headers .= 'Mime-Version: 1.0' . $line_break;

// Sending HTML?  Let's plop in some basic stuff, then.
if ($send_html)
{

// This is the actual HTML message, prim and proper.  If we wanted images, they could be inlined here (with multipart/related, etc.)
list($charset, $html_message, $encoding) = mimespecialchars($orig_message, false, $hotmail_fix, $line_break);
$message 'Content-Type: text/html; charset=' . $charset . $line_break;
$message .= 'Content-Transfer-Encoding: ' . ($encoding == '' ? '7bit' : $encoding) . $line_break . $line_break;
$message .= $html_message . $line_break . '';
}
// Text is good too.
else
{

// Now add an encoded message using the forum's character set.
list ($charset, $encoded_message, $encoding) = mimespecialchars($orig_message, false, false, $line_break);
$message 'Content-Type: text/plain; charset=' . $charset . $line_break;
$message .= 'Content-Transfer-Encoding: ' . $encoding . $line_break . $line_break;
$message .= $encoded_message . $line_break . '--' . $mime_boundary . '--';
}

[shawnb61]

A couple of those you need to change the .= to an =

When you start building the string.  I was trying to circle the .

[spiros]

Thanks! Pardon my ignorance, still not clear about these lines:

This:

Code Select Expand

$message .= $html_message . $line_break . '';
...
$message .= $encoded_message . $line_break . '';
This?

Code Select Expand

$message .= $html_message . $line_break . ';
...
$message .= $encoded_message . $line_break . ';

Or simply this?

Code Select Expand

$message .= $html_message . $line_break;
...
$message .= $encoded_message . $line_break;

Here is the full thing again:

Code Select Expand

// Using mime, as it allows to send a plain unencoded alternative.
$headers .= 'Mime-Version: 1.0' . $line_break;

// Sending HTML?  Let's plop in some basic stuff, then.
if ($send_html)
{

// This is the actual HTML message, prim and proper.  If we wanted images, they could be inlined here (with multipart/related, etc.)
list($charset, $html_message, $encoding) = mimespecialchars($orig_message, false, $hotmail_fix, $line_break);
$message = 'Content-Type: text/html; charset=' . $charset . $line_break;
$message .= 'Content-Transfer-Encoding: ' . ($encoding == '' ? '7bit' : $encoding) . $line_break . $line_break;
$message .= $html_message . $line_break . '';
}
// Text is good too.
else
{

// Now add an encoded message using the forum's character set.
list ($charset, $encoded_message, $encoding) = mimespecialchars($orig_message, false, false, $line_break);
$message = 'Content-Type: text/plain; charset=' . $charset . $line_break;
$message .= 'Content-Transfer-Encoding: ' . $encoding . $line_break . $line_break;
$message .= $encoded_message . $line_break . '';
}

[shawnb61]

First or 3rd should work.

[shawnb61]

Any luck?

[spiros]

It cut off the body part, but then I reverted to using SMTP, so everything was ok with SMTP and port 587 -:)