Amazon SES (email) Duplicate Mime-Version Error

[bigdy]

SMF 2.0.5

Trying to use Amazon SES for email and am running into some errors. The errors seem to be intermittent (sometimes the emails send, sometimes they don't) without any rhyme or reason for why they might come up.  Amazon SES is an SMTP service which uses TLS to send emails.  I have SMF configured to send emails via SMTP on:

SMTP Server: tls://email-smtp.us-east-1.amazonaws.com
SMTP Port: 465

Ran into problems sending Mail. Error: 554 Transaction failed: Duplicate header 'MIME-Version'.

Location:

http://www.mywebsite.com/index.php?scheduled=mailq;ts=1377035123 [nofollow]

The error was also present
Type of error: General

I've searched the forum and found one other, old, unsolved post about this but lacking much details. 

The only 2 places mime-version show up in the php files are:

Sources\Subs-Post.php

Code Select Expand

// Using mime, as it allows to send a plain unencoded alternative.
$headers .= 'Mime-Version: 1.0' . $line_break;
$headers .= 'Content-Type: multipart/alternative; boundary="' . $mime_boundary . '"' . $line_break;
$headers .= 'Content-Transfer-Encoding: 7bit' . $line_break;

Sources\ScheduledTasks.php

Code Select Expand


$result = smtp_mail(array($email['to']), $email['subject'], $email['body'], $email['send_html'] ? $email['headers'] : 'Mime-Version: 1.0' . "\r\n" . $email['headers']);

// Hopefully it sent?
if (!$result)
$failed_emails[] = array($e

Does anyone have any suggestions on what might be done to ensure email can be sent? 

Thanks!

[bigdy]

Not able to edit my own posts.  So as an update I did find something that works.  I've gone into mail_queue and edited the headers field.

This does NOT get sent:

Code Select Expand

From: "MY WEBSITE" <[email protected]>
Return-Path: [email protected]
Date: Tue, 20 Aug 2013 23:18:42 -0000
Message-ID: <[email protected]>
X-Mailer: SMF
[b]Mime-Version: 1.0[/b]
Content-Type: multipart/alternative; boundary="SMF-UNIQUE-CODE"
Content-Transfer-Encoding: 7bit

Removing "Mime-Version: 1.0" from each individual field in the mail queue DOES send the email.

In the email that was received to our test user (after Mime-Version was removed from the mail_queue) the headers look like this:

Code Select Expand

DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple;
s=UNIQUE-CODE; d=mywebsite.com; t=1377048300;
h=Subject:To:[b]Mime-Version:From[/b]:Date:Message-ID:Content-Type:Content-Transfer-Encoding;
bh=UNIQUE-CODE=;
b=UNIQUE-CODE
7UNIQUE-CODE
UNIQUE-CODE=
Subject: Topic reply: test
To: <[email protected]>
[b]Mime-Version: 1.0[/b]
From: "MY WEBSITE" <[email protected]>
Return-Path: [email protected]
Date: Wed, 21 Aug 2013 01:25:00 +0000
Message-ID: <[email protected]>
X-Mailer: SMF
Content-Type: multipart/alternative; boundary="SMF-UNIQUE-CODE"
Content-Transfer-Encoding: 7bit
X-SES-Outgoing: DATE / TIME


START OF THE MESSAGE


So for some reason, even though I removed the Mime-Version header from the databanse, there it is in the email.

Can anyone make sense of this?

[Arantor]

Does it actually reach the destination? Does it have unusual characters in the message (like accented letters)?

[bigdy]

Does it actually reach the destination? Does it have unusual characters in the message (like accented letters)?

Thanks for your reply.  It looks like removing Mime-Version: 1.0 DOES reach it's final destination.  In my test case it sends through Amazon and reaches a Google Apps hosted domain.

I went a bit further and changed Sources\Subs-Post.php:

Code Select Expand

$headers .= 'Mime-Version: 1.0' . $line_break;
TO

Code Select Expand

// $headers .= 'Mime-Version: 1.0' . $line_break;

Now all emails are being delivered.  I'm lost. lol.

[Arantor]

What about when there are unusual characters in the email body?

[bigdy]

What type of unusual character?  Is there an example I could try to see what happens?   I will try right now. 

Thank you so much, too.

[Arantor]

Like accented characters. Anything that isn't a regular ASCII character will do.

EDIT: You need to do it through the normal posting/announcement/whatever interface and not directly in the mail queue table.

[bigdy]

Thank you for the edit!  That's actually what I was going to try to do too. 

I tested it via making an announcement with ASCII characters.  The email DID go through but the characters (å ä ö) were not in the email. The other part of the message, all 5 words, were there.

I tested this via Newsletter Pro (SMFHack plugin) with the same characters and the email did go through, including the ASCII words in there.

I'm actually not able to even post "å ä ö" to our our forum.  The forum looks to be stripping them out. 

[Arantor]

Hmmm, interesting. That suggests something interesting is going on with the missing header. I need to set this up and start thoroughly investigating.

[bigdy]

Just to note that I was able to send email via SMTP using Google for Apps.  Right now, using the "fix" in post #2, emails / notifications are able to be sent via Amazon SES.   

This seems to be an issue with SMF and Amazon SES.  I was looking for an email sent via SMTP on Google but I can't locate one.  Sending via PHP mail worked just fine as well. 

The other post that referenced this issue: http://www.simplemachines.org/community/index.php?topic=479458.0

Screenshot showing we have too many installed plugins: http://i.imgur.com/Z5gOJZu.png [nofollow]

Thank you for your help here!  I know enough to piece stuff together and follow directions, but developing and troubleshooting ...... eeeeeek.

[Arantor]

Hmm... not a lot of people are using Amazon SES at present so I really do need to look into this a bit more thoroughly.

I can well imagine that duplicate headers could be produced actually, but the protocol doesn't enforce singles of all headers, so that's just SES being... well... SES.

What concerns me is that you're now not sending that header at all as far as I can tell and that may have a knock on effect elsewhere (as other mail relays may reject it and/or class it as spam)

[bigdy]

The interesting thing is that with the 'fix' from post #2 the header still reports.

Mime-Version: 1.

Would I be able to private message you a complete header example / checker URL? 

[Arantor]

I'm not sure what that would tell me at this stage. It's been a while since I looked at the email code, I really need to go back through it and the best way for me to do that is to do it myself with the tools I can throw at it.

[bigdy]

OK.  I've made sure to subscribe to this post so PLEASE post if I can be of ANY help. 

If it would help troubleshooting I could setup an Amazon SES email account for testing purposes.  Please let me know Thank you.

[bigdy]

There were a few emails that were getting caught in the send mail queue, even after applying "the fix" from post #2. I decided to go back and do a check to see what Google SMTP looked liked, here is what I found. Headers are from THE SAME MESSAGE. I caught the headers in mail_queue before it was sent.

This test was conducted with default header settings... I did not apply any fixes for this test.  Should have actually included what the headers look like with the fix applied. Edit coming.

Here is the header from the database mail_queue (before it was sent):

From: "MY WEBSITE" <[email protected]>
Return-Path: [email protected]
Date: Wed, 21 Aug 2013 19:25:28 -0000
Message-ID: <[email protected]>
X-Mailer: SMF
Mime-Version: 1.0
Content-Type: multipart/alternative; boundary="SMF-9ea4d13cfaeee7c8f4808a380ecd8a26"
Content-Transfer-Encoding: 7bit

Here is the header received on Gmail (sent via Google Apps, to Google Apps email):

Subject: Topic reply: Here's to a test
To: <[email protected]>
Mime-Version: 1.0
From: "MY WEBSITE" <[email protected]>
Return-Path: [email protected]
Date: Wed, 21 Aug 2013 19:25:28 -0000
Message-ID: <[email protected]>
X-Mailer: SMF
Mime-Version: 1.0
Content-Type: multipart/alternative; boundary="SMF-9ea4d13cfaeee7c8f4808a380ecd8a26"
Content-Transfer-Encoding: 7bit

So there are indeed two Mime-Version headers in the email...  now to do some trials.



The following is a test with the fix from post #2 (commented out Mime-Version from Sources\Subs-Post.php):


As seen in mail_queue:

From: "MY WEBSITE" <[email protected]>
Return-Path: [email protected]
Date: Wed, 21 Aug 2013 19:36:35 -0000
Message-ID: <[email protected]>
X-Mailer: SMF
Content-Type: multipart/alternative; boundary="SMF-7b11018a58bd83ccfbfef3f68668947c"
Content-Transfer-Encoding: 7bit

Here is the header received on Gmail (sent via Google Apps, to Google Apps email):

Subject: Topic reply: random stuff
To: <[email protected]>
Mime-Version: 1.0
From: "MY Website" <[email protected]>
Return-Path: [email protected]
Date: Wed, 21 Aug 2013 19:36:35 -0000
Message-ID: <[email protected]>
X-Mailer: SMF
Content-Type: multipart/alternative; boundary="SMF-7b11018a58bd83ccfbfef3f68668947c"
Content-Transfer-Encoding: 7bit

[shumilica]

Can I bump this topic?
I had 2 nights wasted on this, I activated the plugin 'email inactive users' and wanted to send the messages through amazon ses (aprox. 10.000), but they didn't seem to go from mail queue. If I wanted to ask for a password change the mail would go through, so the setup for amazon ses was fine, dkim signed and everything. But when I wanted to send those notificiation emails to inactive users it just wouldn't work. Also the birthday emails weren't sent.
And I didn't had anything in the error log.

Commenting out the line specified by bigdy resolved the issue. You should check this out more thoroughly because amazon ses is starting to get used more and more often.

[shumilica]

One interesting thing, commenting out that line will let the mails that are sent through the newsletter option integrated in SMF without the 'MIME-VERSION: 1' part. But notifications such as new topic reply, birthday emails and password resets are sent with MIME-VERSION.

So i uncommented that part and erased MIME-VERSION:1 from Sources\ScheduledTasks.php

Code Select Expand


$result = smtp_mail(array($email['to']), $email['subject'], $email['body'], $email['send_html'] ? $email['headers'] : 'Mime-Version: 1.0' . "\r\n" . $email['headers']);


with this

Code Select Expand


$result = smtp_mail(array($email['to']), $email['subject'], $email['body'], $email['send_html'] ? $email['headers'] : '' . "\r\n" . $email['headers']);


From my test now it sends always with one mime-version

[Molten Bear]

I too am seeing this error with Amazon SES.  Duplicate Mime

The edit above in the ScheduledTasks.php doesn't work correctly, the email contains header information.  Yes, the email is sent but it looks awful.

Removing the "Mime-1" code in ScheduledTasks.php results in sent emails but OTHER errors:
Ran into problems sending Mail. Error: 554 Transaction failed: Expected parameter value, got "null"

This refers to Scheduled task emails, like Birthday notice, etc.

So, whatever emails that run through BOTH ScheduledTasks.php AND Subs-post.php get MIME-1 added twice.

[Molten Bear]

To be clear:

1. With standard SMF 2.0.9 code and SES as SMTP mail server.  All emails are failing on send.  "Duplicate MIME" error messages.

2. With ['Mime-Version: 1.0' . "\r\n" .] removed from ScheduledTasks.php:
- the Mail queue can be resent and all emails get sent and delivered and look like normal email. The message headers contain Mime version info (presumably going through Subs-post.php)
- I can open a user profile and send email, that works too.
- Scheduled emails, birthday, expiration etc emails fail with "null" error message, missing MIME header info

[Oldiesmann]

Ok, so this seems to be an SMF issue to some extent. I'll look into it more when I get a chance and see if I can come up with a proper fix for you.