2.0.17 Email failure with Dreamhost and Couldn't get mail server response codes

[Dena]

This topic may belong in Bug Reports but as I am unsure, I am starting here.
I am new to PHP and Simple machines but I have had 4 years as a moderator and 40 years assembler experience. I have reached the limit of what I can figure out and without learning a lot more, I am at a dead end.

The host server is Dreamhost. I know your high opinion of it but the deed was done before I had a say in the matter. The person who did the install is out of touch so am somewhat limited on what I can report. The system logs for the mail service require a higher security level than I have.

Email isn't being sent and as far as I can tell, the configuration is what Dreamhost requires. The Simple Machines error log is returning "Couldn't get mail server response codes". The funny part is the email was being sent at one point and stopped working. I suspect it was about the time the secure connection certificate was installed but I can't swear to it as a good deal more was going on at the time. 

In searching this web site, I have found this topic (modification follows). The  suggested fix was to comment the second line and replace it with the third line.

Code: (Replace With) [Select]
      // EHLO could be understood to mean encrypted hello...
//      if (server_parse('EHLO ' . $modSettings['smtp_host'], $socket, null) == '250')
      if (server_parse('EHLO ' . 'localhost', $socket, null) == '250')

Ok, I can handle that so I pulled up the code in our copy of 2.0.17 and - - glup, maybe this isn't going to be quite so easy. It appears an attempt was made to resolve the problem but they took a different approach. Was the new approach more right or does it still have issues. I don't understand enough about this section of code to attempt to tamper with it so it's time to yell for help.

   // Start off by using the stored mail server.
   $helo = $modSettings['smtp_host'];

   // Try and determine this server's name.
   if (function_exists('gethostname') && gethostname() !== false)
      $helo = gethostname();
   elseif (function_exists('php_uname'))
      $helo = php_uname('n');
   elseif (!empty($_SERVER['SERVER_NAME']))
      $helo = $_SERVER['SERVER_NAME'];

   if ($modSettings['mail_type'] == 1 && $modSettings['smtp_username'] != '' && $modSettings['smtp_password'] != '')
   {
      // EHLO could be understood to mean encrypted hello...
      if (server_parse('EHLO ' . $helo, $socket, null) == '250')

In closing, please help if you can. The site is only about 5 weeks old and it appears we may soon have as many spammers as members. I suspect an opposing web site may have given the spammers a little help and without email verification, spammers are becoming a real issue.

[Shambles]

Code Select Expand

   // Start off by using the stored mail server.
   $helo = $modSettings['smtp_host'];

   // Try and determine this server's name.
   if (function_exists('gethostname') && gethostname() !== false)
      $helo = gethostname();
   elseif (function_exists('php_uname'))
      $helo = php_uname('n');
   elseif (!empty($_SERVER['SERVER_NAME']))
      $helo = $_SERVER['SERVER_NAME'];


Try forcing "localhost" into the equation just to see if it gets you over the line:

Code Select Expand

   // Start off by using the stored mail server.
   $helo = $modSettings['smtp_host'];

   // Try and determine this server's name.
   if (function_exists('gethostname') && gethostname() !== false)
      $helo = gethostname();
   elseif (function_exists('php_uname'))
      $helo = php_uname('n');
   elseif (!empty($_SERVER['SERVER_NAME']))
      $helo = $_SERVER['SERVER_NAME'];

   $helo = 'localhost';


[Torngate]

Chiming in here as someone who's also experiencing this issue - Didn't work for me.

Just defining it still returned a "Couldn't get mail server response codes", AWS hosts here.

(Just reporting my tests)

[Dena]

Ok, I know I have been gone for a long time. I was locked out of the account for three weeks because somebody else was assigned control of the account and there can be only one. We have a sharing arrangement so I am back in.

I attempted the fix and there was no joy in Mudville. Still behaving as before. I had a brain storm and came up with a way to solve this problem. I figured if I could determine the mail server, we might know what the return codes are. It turns out they are using PHP mail() [nofollow] and the document contains a link to the documentation for the package [nofollow]. The documentation looks a little strange to me but I have never worked with PHP and that may explain a few things. If nobody comes up with it first, my goal will be to learn enough PHP that I can understand it and see if I can work out a fix.

I just realized I don't have enough posts for propers links so I can verify what I posted is correct. Should you have difficult extracting the information form the link, let me know and I will see if I made a mistake.

[shawnb61]

Smtp?  Ssl/tls?

If so, try putting "ssl://" in front of the server name.

[Dena]

I started with smtp.dreamhost.com [nofollow] and changed it to ssl://smpt.dreamhost.com. The return code changed to

Ran into problems sending Mail. Error: 550 5.7.1 Sender domain not allowed. Please read: http://dhurl.org/20b D157.

Figuring I might not have understood the instructions, I changed it to ssl://dreamhost.com and for my efforts received

Could not connect to SMTP host: 110 : Connection timed out

Nice try but I don't think we are there yet.

[shawnb61]

Have you contacted dreamhost for the proper settings?

https://help.dreamhost.com/hc/en-us/articles/214918038-Email-client-configuration-overview

[Dena]

Have you contacted dreamhost for the proper settings?

That battle was the first one I fought and I had several exchanges. It was complicated by the fact we started with 2.0.15 and the PHP was up to date on the host system. Finding the error in the log was near impossible and that is why we upgraded to 2.0.17. I am reasonably sure these are the correct settings as I can see an item removed from the queue and then after a few seconds, replaced for a latter attempt. The package limits you to 100 emails an hour so the setting are set to one a minute.

Dreamhost doesn't seem to have the support team required for BBS support but instead seems to be more web site orientated with email using their canned mailing packages.They have had spammer issues in the past so it is possible to send email from their platform. The only question is what is Simple Machines not doing that they expect.

[shawnb61]

I looked at the dreamhost documentation, & it looks like for smtp you need to use:

• server:  ssl://smtp.dreamhost.com
• port:  try 587; if that doesn't work, try 465
• & username & password for the email account you are using.

"Couldn't get server response codes" means you're not properly connecting.  There is a problem with one of the settings above. 

That email account must be properly setup on the DH side, of course.  You should be able to test that it works using *any* email client, & send and receive email using it - completely independent of SMF.  If that email account doesn't work on its own, you won't get very far...    (My host has a concept of a main email account, tied to my cpanel ID, that is to be used under these circumstances.) 

If DH folks are confused, ask for the proper smtp settings for WP, wordpress. 

[Dena]

The first test

ssl://smtp.dreamhost.com   port  465

Ran into problems sending Mail. Error: 550 5.7.1 Sender domain not allowed. Please read: http://dhurl.org/20b D157

The second test, Really hit the bonanza on error messages.

ssl://smtp.dreamhost.com   port  587

Could not connect to SMTP host: 0 :

2: fsockopen(): unable to connect to ssl://smtp.dreamhost.com:587 (Unknown error)

2: fsockopen(): Failed to enable crypto

2: fsockopen(): SSL operation failed with code 1. OpenSSL Error messages:
error:1408F10B:SSL routines:ssl3_get_record:wrong version number

I ran the test just to make sure I didn't miss anything and because the last time I tested this stuff the error log was so flooded that I could't pick out the error messages. Now I can see them it verifies the past decision. Dream host is pretty stuck on port 465 for email as I tried out I think two older standards and they failed as well. As for the server name, they are pretty fussy about what is acceptable. From what I saw, not long ago they were a leader in spam production. To solve this problem, they made two minor little changes. They don't allow you to use a server other than theirs and any email must have a from domain associated with their site. This ensures all email that originates on their site is traceable back to their site and you can't get around this restriction by using another email server that is out of their control. This means our member will not be able to email another member with the site email function as the sender domain will not be acceptable to Dreamhost. They can PM and the PM email will get through.

Only good news is I set up a bunch of verification question, 15 so far and a spambot hasn't made an account in two days. I know that's not going to last but it gives me more time to work on other solutions.

[shawnb61]

So - does it work with 465 if you use a dreamhost.com email user id & password?

[Dena]

So - does it work with 465 if you use a dreamhost.com [nofollow] email user id & password?

It throws off fewer errors and the errors indicate we are closer to where we should be. The fact that we aren't getting a response code sort of indicates we are getting a connection to something and it's not unhappy with what we are connected to. The fact that we aren't getting a response code suggest the operation procedures to the point where an attempt was made but for some reason, we don't know what the results are.

To me, this suggest that that the email software doesn't behave the way other mail servers do. Looking at the code above, you will see that it is looking several places for a response code and it appears there is another that isn't tested for.

Years ago we went to higher level languages so software would be portable between systems. It appears that interface calls are becoming non standard so software is losing it's portability. Sounds like job security to  me.   

[shawnb61]

Unfortunately that response code message covers lots of bases.  Pretty sure you get that with a bad id/password.  Thats where my suspicions are.  SMF mail works just fine via smtp with the right 4 values entered.

Can you connect with that id/pw using gmail or some other client?   Take smf out of the equation & prove it works at all first.

[Dena]

The secret of making Simple Machines work with Dreamhost Email. Yes, I got it working.

I have Shambles modification but I suspect it may not be affecting things. The reason is the code is for secure connections and as I will explain latter, I am using an unsecured connection. Now the settings.

Maximum emails to send per minute - 1
On a shared server, Dreamhost limits you to 100 emails an hour (anti spam feature and resource issue). 2 a minute would be above the limit but if you knew your average would low, you could use a larger number. If you exceed 100 an hour very often, they will disable email. If you do it occasionally they will delay sends to keep you within the limit.

Maximum amount of emails to send per page load - 5
While you can only send one email a minute, Simple machines seems to have a problem keeping that schedule so the send process is slow. I suspect Simple machine grabs a page of 5 in this case, sends them all and waits 5 minutes. A larger number seems to keep a better schedule and you don't end up with one minute being two.

Mail Type - Smtp
That's what Dreamhost wants so that's what they get.

SMTP server - smtp.dreamhost.com [nofollow]
Again, Dreamhost requirement and I haven't found anything else that connects properly

SMTP port - 587
Dreamhost supports two ports, 587 and 465. Port 465 is the one you're supposed to use because it's secure but it appears to return a different return code and/or behaves differently in Simple Machines. Changing to 587 is an unsecured connection but for most of the email, a secure connection isn't really that important. This was the part that I was missing and it is what solved the problem. There still is an issue in the code with secure connections but I am not worrying about it for now.

The other got you with Dreamhost is the sender's domain must be Dreamhost or your domain hosted on Dreamhost. Failure to this will cause your email to be routed to a folder on the server and you will need to go in and clean it out occasionally. This means that sending another member an email using the Simple machine email isn't going to work. This was done as an anti spam feature so all email sent from the server is traceable back to the source.

While the problem is solved, it's possible that this should be moved to the bug report area. On the other hand, it is a resource for somebody dealing with Dreamhost. I leave it with the staff to decide where this thread should go.

[shawnb61]

Thanks for the info!  Very helpful.

Did you have to specify, ssl://, e.g., "ssl://smtp.dreamhost.com" or did it only work without that?

What would be the bug?  Everything above is a DH limitation - mails per minute, domain, port, etc.  SMF supports all those configurations.

[Dena]

If you use ssl://, it thinks your using another server and that is forbidden.  All you need is the smtp.dreamhost.com [nofollow] but if you have a higher level package with better email, you might be able to use smtp.yourdomain.xxx [nofollow] . We are using the $20 a month and that means shared everything including the IP address.

The primary issue is secure server doesn't work and you have to fall back to an unsecured email. This isn't a problem now but software upgrade or policy change could require you to use only secure email.

The other issues are minor and can be lived with. It appears that one email a minutes with a page size of one is much slower than indicated. Bumping the page size up as I did resolved the issue. It would be better if we could set emails sent per hour as have to configure a number 40 below what I am permitted by the hosting site. It's not really a problem at the moment because we are only generating about 100 emails a day. It may never be a problem because we could upgrade our hosting package and that would set a higher send limit.

[shawnb61]

Again, thank you, very helpful.  Glad you got it working.

465 is secure only, which SMF supports.  I don't know why DH wouldn't accept it.  My best guess is it's tied to the fact that they rejected ssl://. 

567 is (usually) SMARTTLS, which is "I'll accept a secure or insecure connection & attempt to upgrade it to secure".  So it could be either secure or insecure, if the secure connection was rejected by either side.  But DH owns both sides of that channel, since they are hosting SMF for you & also providing the email...  If 567 is not secure, that's on them.

SMF could be better (explicit SMARTTLS is supported in 2.1, not 2.0), but it really feels like the unusual limitations imposed by DH are a major part of the problem here.

[Dena]

Dreamhost isn't exactly one of the most popular hosting sites around here and there are probably a few reasons why. Unfortunately as well, when you have to control spam coming off your host, you end up with some restrictions that aren't very popular.

At least things are working for the most part and I think I may have a way around the domain email issue with forwarded email accounts. Everybody can have a forwarded account (I think), they can put it in their profile and the problem is worked around again. Dreamhost is happy and the members are happy. Now all I have to do is find out the maximum number of forwarded emails I can define.