News:

Bored?  Looking to kill some time?  Want to chat with other SMF users?  Join us in IRC chat or Discord

Main Menu

Hotmail/Live/Outlook rejects all our mails (RFC 5322)

Started by huns, October 23, 2016, 03:07:45 AM

Previous topic - Next topic

huns

So a few days ago I started receiving emails from the new members of the community who wanted to register on our forums.
They told me the emails didn't arrive. I found that very weird, since we use SendGrid, so our mails get sent 100%, and they do get delivered too.

So, I looked at the logs, and this is what I saw:
REASON550 5.7.0 (COL004-MC1F11) Message could not be delivered. Please ensure the message is RFC 5322 compliant.
REASON550 5.7.0 (COL004-MC4F1) Message could not be delivered. Please ensure the message is RFC 5322 compliant.

And so on, and so on.
I checked Google, and it seems the email format is not OK. It worked okay so far, so I don't know, maybe MS changed something.
Here are the "solutions":
https://stackoverflow.com/questions/22433687/how-to-format-an-email-that-hotmail-outlook-is-happy-with
https://answers.microsoft.com/en-us/outlook_com/forum/oemail-ocompose/message-could-not-be-delivered-please-ensure-the/87a52762-7d08-467e-85a2-120721c2dd8e

Now I thought about writing a simple mod as I always do, but this kinda concerns every SMF user, so I thought it would be better to bring it to the public.
Anyone seen this? Can the SMF team write an official hotfix?

Woraphat

I'll be following this one.  We have been fighting some e-mails not being received by members for a long time.

huns

If you run your own email server just check the .log file and you can see the reply from Hotmail/Outlook. You could even just grep for the same RFC number.
On Sendgrid, it's under Activity.

Amazon SES would be also great to use with SMF, but it never worked.

Shambles

Could be a tightening up of the validation, like Google did earlier this year.

Try the small code change described below, just to see if makes any difference:

http://www.simplemachines.org/community/index.php?topic=544856.msg3870010#msg3870010

huns

Dear Shambles!

This is related to the SMTP, your code, that is.
The problem is in the sent email's structure.

The email gets sent 100%, and it's delivered.
But Hotmail rejects it, because the FROM or some similar part is not RFC compliant.

Basically the sent email's content would have to be edited.

Thanks for your quick reply by the way!

radu81

try to do a test on https://www.mail-tester.com/

I am not using sendrid, I use a similar service and recently I had problems with gmail accounts, one IP of the smtp provider was in a blacklist
sorry for my bad english

huns

It's a bad score, but it complains about the same thing basically.

> -2.599   FROM_NO_USER   From: has no local-part before @ sign

There is this too:
> Listed in SORBS-SPAM

I contacted support about this one.
But the email gets rejected due to the no user thing, it does not end up in spam, it's nowhere.

UPDATE: Support replied under 15 minutes, they moved us to another IP.
But then again, the IP was never the problem. Our mail did not end up as spam, it ended up being rejected.

radu81

With sendgrid you can get a 10 score and your e-mails will be delivered (if they remove the ip from blacklist or change the IP)

Try to post a link with mail-tester results. Do you have access to your DNS zone?
sorry for my bad english

aegersz

i had the same problem BUT without the RFC 5322 bit AND this may help you.

what really helped was to email Microsoft Support and tell them to address any delivery issue. I did and this was their reply (yours may be different):

We have completed reviewing the IP(s) you submitted. The following table contains the results of our investigation.

Conditionally mitigated
xxx.xxx.xxx.xxx [removed]
Our investigation has determined that the above IP(s) qualify for conditional mitigation. These IP(s) have been unblocked, but may be subject to low daily email limits until they have established a good reputation.

Please note that mitigating this issue does not guarantee that your email will be delivered to a user's inbox.

Ongoing complaints from users will result in removal of the mitigation.

Mitigation may take 24 - 48 hours to replicate completely throughout our system.

The configuration of my Linux VPS (SMF 2.0 with 160+ mods & some assorted manual tweaks) can be found here and notes on my mods can be found here (warning: those links will take you to a drug related forum). My (House) music DJ dedication page is here

huns

Look guys, I am very thankful for your responses, but I have to repeat.
The problem is NOT the spam list. It is NOT, NOT the spam list.

SMF already messes this up once. If you install a vanilla SMF and try to use specific SMTP servers, you will see that it cannot talk to TLS servers for example properly. Hacks here and there, it works. Yay.

Basically. You have the email.
It must have given parts.
If your email gets REJECTED, you are doing something terribly wrong.
It's not that your email is spam, looks bad, anything, it IS bad, wrong.

So anyhow...
I thought this topic will notify the team, they will check, maybe write a fix sometime.
But, hey, no probs, I will write it myself. I just thought it's better if everyone gets the fix and all.

(Or we just move to other engine, our forum has like 5-10 hotfixes in mods already that I wrote... lol.
Not sure if the 2.1 upgrade - if once it will release - would even install.)

To reply:
@aegersz: I know. We had IP problems with VPS and I signed us up to the Hotmail Junk Mail Reporting Program or what it was called.
It's easy, do-able (though you have to have an inbox for postmaster), that's it.
This is the easiest way of doing it.

@radu81: I have full control over the server and the stack, everything.
It's our own 1:1 rented machine. (Crazy, it costs the same as a VPS, except it's 10x more powerful. Gotta love capitalism.)
As I mentioned to Aegersz, before we used our own local stack, and I used JMRP and the other provider programs, that worked.
But this is different now. This is the email being malformed, not correct.
The PHP code that prepares the email structure has to be fixed.
Or the engine (SMF) has to be changed down (phpBB, vBulletin, ipb, fluxbb - now punbb).
Though most of our community would frown if I paid for an engine that is not open-source, so ipb/vb goes out of that list sadly.
Guess it will be phpBB.


Topic can be closed.

rosewillrnx

Any info from SMF would be great. I have members complaining about not getting password reminders. It is the same problem. Some get the mail others simply do not no matter where they check.

Of course other members just never come back. I imagine it is happening on other forums too and just not being noticed. I have a guy that watches the activity pretty regularly and sees people request the password and never log in. Sometimes they use the contact button we had to create for guests for this issue. Other times we lose a member.

This is a real issue that needs tracked down in the original source.

@huns. Did you write a solution? I would be interested in helping test it.


vbgamer45

http://stackoverflow.com/questions/22433687/how-to-format-an-email-that-hotmail-outlook-is-happy-with
It might be the replyto line
Reply-To: "Testing Site" <[email protected]>

In smf it says
$headers .= $from !== null ? 'Reply-To: <' . $from . '>' . $line_break : '';
Community Suite for SMF - Take your forum to the next level built for SMF, Gallery,Store,Classifieds,Downloads,more!

SMFHacks.com -  Paid Modifications for SMF

Mods:
EzPortal - Portal System for SMF
SMF Gallery Pro
SMF Store SMF Classifieds Ad Seller Pro

rosewillrnx

Thanks vbgamer45. I think your on the right track. I have been searching every where I can think of. Could you point me in the direction of the file to look at?




vbgamer45

Community Suite for SMF - Take your forum to the next level built for SMF, Gallery,Store,Classifieds,Downloads,more!

SMFHacks.com -  Paid Modifications for SMF

Mods:
EzPortal - Portal System for SMF
SMF Gallery Pro
SMF Store SMF Classifieds Ad Seller Pro

rosewillrnx

Hi Again, thanks for the direction.
I commented out both header tags and I am going to try this out:
$headers = From: "Chiang Rai Ties" <[email protected]> . $line_break;
of course I used my own address.
I tested it and got one message so I know it works at least as well as the original code. And I left the reply to out as it is the same address.

Ill post back with results.

rosewillrnx

So as far as I can tell it's still misfiring. We get some new member notifications and some go unreceived. One really strange thing is we have the forum set to notify two addresses. It does once in a while. Other times one might get one.... Or the other. It doesn't make any sense. Why wouldn't it be consistent? Always tossing out the same address? Wherever it is dying it is hurting our forum.

Are the admins here getting all of their notifications? Or are they having the same problem?

Members, in this forum. Are you getting a notifications for every topic you select?

Is it really just mail hosts throwing these message out randomly? Why doesn't it happen across the board. Everyone's messages from everywhere. I think if it was mailhosts it would be a common problem. People would complain ALL the time about not getting important messages.

GL700Wing

Quote from: huns on October 23, 2016, 03:07:45 AM
So a few days ago I started receiving emails from the new members of the community who wanted to register on our forums.
They told me the emails didn't arrive. I found that very weird, since we use SendGrid, so our mails get sent 100%, and they do get delivered too.
I first became aware of this problem on my SMF 2.0 forum late last year when I noticed an increasing number of new members who had registered using a Microsoft email address (eg, hotmail.com, live.com, outlook.com, msn.com or windowslive.com) but not activated their account.

I searched this forum for reports of others experiencing the same problems - and a possible solution - but couldn't find anything.  I didn't post about it here because I was extremely busy with other commitments and really wouldn't have had the time to properly respond to requests for additional information.

In the end, and because I was concerned about the number of new members who could not activate their accounts, I installed the Restrict Email Providers on Registration mod and configured it to prevent new members registering using Microsoft email addresses.  In addition, I also did some further customisation to prevent existing users from changing their email address to a Microsoft email address.

I implemented this change in January and since that time I've only had about one new member a month who has not activated their account (I manually send them an email when this happens).
Life doesn't have to be perfect to be wonderful ...

rosewillrnx

Hey folks, no change.

This is a lousy solution but its my only option at the moment.

http://chiangraities.com/index.php?action=reminder


This code is working surprisingly more frequently than with the variable values.
// Construct the mail headers...
/**$headers = 'From: "' . $from_name . '" <' . (empty($modSettings['mail_from']) ? $webmaster_email : $modSettings['mail_from']) . '>' . $line_break;
$headers .= $from !== null ? 'Reply-To: <' . $from . '>' . $line_break : '';*/
$headers = 'From: "Chiang Rai Ties Friendly Forum" <[email protected]>' . $line_break;
//$headers .= 'Return-Path: ' . (empty($modSettings['mail_from']) ? $webmaster_email : $modSettings['mail_from']) . $line_break;
$headers .= 'Return-Path: "Chiang Rai Ties From" <[email protected]>' . $line_break;
$headers .= 'Date: ' . gmdate('D, d M Y H:i:s') . ' -0000' . $line_break;


More mails come but the password reminder still fails frequently.

huns

Wow guys I am terribly sorry.
I found the fix.

The "from" was really, in fact, empty.
Now.... There are two possible causes:
1) The latest hotfix patch cleared that sql value.
2) Microsoft stepped up it's game, and it accepted mails until the last few days.

Either way, check your Settings, see if you have that filled.
(I think modifying the registration message also helps with delivers.)

Advertisement: