Message has lines too long for transport

[shawnb61]

I spent some time trying to reproduce this, & couldn't.  My initial thoughts above were that some contraints were exceeded due to the insertion of "New Announcment" in the subject.  But I couldn't reproduce that in SMF.  The subject remained properly intact & was sent to my test gmail account properly.

I also made a 20000 character, single-line post in Cyrillic, then announced it...   That also went out fine & was received properly by gmail. 

But what stands out above is that there is a portion of the email that was treated as 7-bit and entity encoded, and not base64 encoded.  I suspect both Kindred & vbgamer45 were on the right track...  There is *something* in that text that triggered special processing. 

None of my attempts produced that behavior, it was all base64 encoded, and sent and received properly by gmail. 

I am not familiar enough with that code to understand what would trigger entity encoding vs base64.  I may look again over the weekend. 

Interesting problem.

[vbgamer45]

Might also check out exim mailserver settings. google and look at error messages

[live627]

But what stands out above is that there is a portion of the email that was treated as 7-bit and entity encoded, and not base64 encoded.  I suspect both Kindred & vbgamer45 were on the right track...  There is *something* in that text that triggered special processing. 

Those lines are too long and must be wrapped.

[shawnb61]

Xvost -

I am still trying to reproduce the issue - with no luck.  Two more questions:
1) Does it happen on other announcements/mailers, or only when you announce that particular topic?
2) Could you send me the text of the problem message?  From within Quick Edit mode, so I get all the exact BBC as posted.  Just send via PM so I can reproduce. 

Thanks.

BTW - Very nice site.  I have a number of friends that follow trains & their history.  In fact, I have a close friend who is traveling to see one of the "Big Boys" up close this weekend.
https://en.wikipedia.org/wiki/Union_Pacific_Big_Boy

One or two of the Big Boys are still in operation, attracting crowds when they visit small towns. 

[Diego Andrés]

Also just another piece of information from the discoveries made last year of users using the PSL MOD, SMF sendmail() function doesn't give a flying ****** about the things it receives, and expects the strings to be already trimmed.

So for example, in this scenario, AnnouncementSend() could be receiving a 5000 line subject, and sendmail() won't care because neither function is trimming it like Post()

Code Select Expand

        // Make sure the subject isn't too long - taking into account special characters.
        if ($smcFunc['strlen']($form_subject) > 100)
            $form_subject = $smcFunc['substr']($form_subject, 0, 100);

[Chen Zhen]

Do you use CPanel from your webhost platform?
It might be related to a bug in slightly older versions of CPanel where Exim was updated but CPanel's settings for it were not updated. If you change the settings manually it may resolve your issue.

ref.https://webdesires.co.uk/blog/cpanel-message-has-lines-too-long-for-transport/


Failing that I'd start with disabling any mods that mess with emails such as your Email Subject Length mod & use some trial & error to see if one of them is the culprit.

[shawnb61]

It is possible that this is a bug, but I cannot reproduce it.  My host is clearly filtering/altering the data, making it hard/impossible to reproduce.

SMF sendmail usually sends the mail in two different formats: 7-bit plain text (in theory for older clients) and utf8, which is base64 encoded & chunked up into 80 byte chunks.

My suspicion is that the 7-bit portion of the email sent is not being checked for a 1000 byte limit per line as it should.  It basically just sends the text.  I don't see any length check in the sendmail/mimespecialchars routines for 7-bit. 
https://stackoverflow.com/questions/25710599/content-transfer-encoding-7bit-or-8-bit

Since I cannot find this 1000-byte check occuring in the code, I logged an issue:
https://github.com/SimpleMachines/SMF/issues/8288

The reason I cannot confirm it is that it appears my host strips the 7-bit entirely...

When I send something that looks like this to my SMTP server:

Code Select Expand

X-Mailer: SMF
Mime-Version: 1.0
content-type: multipart/alternative; boundary="SMF-a937f56445fd3145753c93750a6c9850"
content-transfer-encoding: 7bit
Content-Length: 35094

Восени 2020-го року, я вирішив, що технічно формат проекту не
...
--SMF-a937f56445fd3145753c93750a6c9850
content-type: text/plain; charset=UTF-8
content-transfer-encoding: base64

TGluazogaHR0cHM6Ly93d3cuc2ltcGxlbWFjaGluZXMub3JnL2NvbW11bml0eS9pbmRleC5waHA/
bXNnPTQxNzY2ODkNCg0KQW5vdGhlciBsaW5rOmhlcmUuDQoNCmh0bWwgbGluazogbGluayB0ZXh0
DQoNCtCS0L7RgdC10L3QuCAyMDIwLdCz0L4g0YDQvtC60YMsINGPINCy0LjRgNGW0YjQuNCyLCDR

My mail client receives something that looks like this:

Code Select Expand

X-Mailer: SMF
Mime-Version: 1.0
content-type: multipart/alternative; boundary="SMF-a937f56445fd3145753c93750a6c9850"
content-transfer-encoding: 7bit

--SMF-a937f56445fd3145753c93750a6c9850
content-type: text/plain; charset=UTF-8
content-transfer-encoding: base64

TGluazogaHR0cHM6Ly93d3cuc2ltcGxlbWFjaGluZXMub3JnL2NvbW11bml0eS9pbmRleC5waHA/
bXNnPTQxNzY2ODkNCg0KQW5vdGhlciBsaW5rOmhlcmUuDQoNCmh0bWwgbGluazogbGluayB0ZXh0
DQoNCtCS0L7RgdC10L3QuCAyMDIwLdCz0L4g0YDQvtC60YMsINGPINCy0LjRgNGW0YjQuNCyLCDR


[shawnb61]

This problem is much worse for folks with languages with multi-byte charsets. 

E.g., this 198-character line in Ukranian:

Code Select Expand

Восени 2020-го року, я вирішив, що технічно формат проекту не відповідає вимогам часу. Тому рішуче почав перебудовувати сайт на новий движок - WordPress. Планувалось, та й так і сталося, що найбільш

Becomes this 1080 byte line in the 7bit chunk of an SMF email body, which is out of compliance with the RFC:

Code Select Expand

Восени 2020-го року, я вирішив, що технічно формат проекту не відповідає вимогам часу. Тому рішуче почав перебудовувати сайт на новий движок - WordPress. Планувалось, та й так і сталося, що найбільш

As a result, the 1000 byte limitation is much tougher on languages with multibyte characters. 

The fix is a more difficult question...  Introducing linebreaks into those long lines will chop up the source content in an unnatural way.  It'd be ugly. 

I wonder if we can just abandon the 7-bit encoding altogether, and just keep the base64 encoded utf8. 

[Kindred]

Yes, that's what I would suggest that we do

[Xvost]

The further, the more interesting. As the previous problem was solved with the help of a crutch, here is a new one. Now the same, but with password recovery emails.

Error message:

Code Select Expand

This message was created automatically by mail delivery software.

A message that you sent could not be delivered to one or more of its
recipients. This is a permanent error. The following address(es) failed:

  [email protected]
    message has lines too long for transport


Reporting-MTA: dns; ova.in.ua

Action: failed
Final-Recipient: rfc822;[email protected]
Status: 5.0.0


Return-path: <[email protected]>
Received: from ova.in.ua ([95.215.157.210]:49970)
by ova.in.ua with esmtpa (Exim 4.96)
(envelope-from <[email protected]>)
id 1sQjdu-002tL8-2u
for [email protected];
Mon, 08 Jul 2024 10:22:38 +0200
Subject: =?UTF-8?B?0J3QvtCy0LjQuSDQv9Cw0YDQvtC70Ywg0LTQu9GPINCf0L7RgdGCINCS0L7RgNGB0LrQu9Cw?=
To: <[email protected]>
From: "=?UTF-8?B?0J/QvtGB0YIg0JLQvtGA0YHQutC70LA=?=" <[email protected]>
Date: Mon, 08 Jul 2024 08:10:58 -0000
Message-ID: <[email protected]>
X-Mailer: SMF
Mime-Version: 1.0
content-type: multipart/alternative; boundary="SMF-c1cadc99ce0a620dabeee9e10332e0ba"
content-transfer-encoding: 7bit
X-Exim-DSN-Information: Due to administrative limits only headers are returned

The standard text of the password recovery letter in Ukrainian:

Code Select Expand

Вітаємо ...,
Цей лист було надіслано у зв'язку з тим, що до вашого облікового запису було застосовано функцію відновлення пароля. Щоб встановити новий пароль, перейдіть за наступним посиланням:
...

IP: ...
Ім'я користувача: ...

Regards,
...

[Xvost]

I did a little research, why in one case the letters with the Ukrainian text are sent correctly, in the other - they are not sent and we get a response from the mail server about an error.

It turned out that in all cases where the letter was not sent and caused an error message from the server, the text contained an apostrophe character (U+0027):

Code Select Expand

ʼ
If you correct the text of the worksheet template so as to avoid using the apostrophe, the sheets will be added correctly, without any errors.

Those same people are fussing about those who announced the announcement of a deal. The first words are written with an apostrophe:

Восени 2020-го року, я вирішив, що технічно формат проекту не відповідає вимогам часу. Тому рішуче почав перебудовувати сайт на новий движок - WordPress. Планувалось, та й так і сталося, що найбільш цікаві матеріали будуть перенесені у вигляді окремих статей. Але час та вимоги, що з'явилися, розставили всі крапки над «ї».

[shawnb61]

That's interesting.  Apostrophes are totally valid, & cause no issues that I can see in my tests. 

If issues return, I suggest editing the template to keep paragraphs short - fewer than 140 Cyrillic characters between line feeds.

Another possibility is to step you thru removing the 7bit encoding altogether.  It's a pretty simple edit, and would look better than a bunch of short paragraphs & sentences.  (But might be a problem if you have users with very old email clients...)

My host just strips them entirely if there is valid utf8/base64 content.  It might be worthwhile asking your host if that is a possibility also.

[Xvost]

@shawnb61 looks like you are right. It's not about the apostrophe here. Revised the entire Email message templates, editing everything so that each paragraph did not exceed 140 characters. Everything seems to work.