Simple Machines Community Forum

SMF Development => Feature Requests => Topic started by: Night09 on August 30, 2014, 10:31:53 PM

Title: smf mail
Post by: Night09 on August 30, 2014, 10:31:53 PM
Is there going to be any upgrade of the mail system in 2.1?

Google now classes smf mail as an insecure app and Insists that less secure security is enabled In your account settings before mail will successfully send.

This may already have been addressed but I'm off  my phone atm so haven't looked into it too deep.
Title: Re: smf mail
Post by: Dragooon on August 31, 2014, 03:43:03 AM
Uh not sure how SMF can help with that. Setting up SMTP with DKIM and SPF might help with that.
Title: Re: smf mail
Post by: Night09 on August 31, 2014, 04:52:19 AM
Technically though its not those needing setting up as im mailing via google as a standard account so dont use the records to send mail.

Heres an edited version of the mail I recieved last night.

Quote

We recently blocked a sign-in attempt to your Google Account.

Sign in attempt details
Date & Time: Sunday, August 31, 2014 1:31:06 AM UTC
Location: England, UK

If this wasn't you
Please review your Account Activity page at https://security.google.com/settings/security/activity (https://security.google.com/settings/security/activity) to see if anything looks suspicious. Whoever tried to sign in to your account knows your password; we recommend that you change it right away.

If this was you
You can switch to an app made by Google such as Gmail to access your account (recommended) or change your settings at https://www.google.com/settings/security/lesssecureapps (https://www.google.com/settings/security/lesssecureapps) so that your account is no longer protected by modern security standards.

To learn more, see https://support.google.com/accounts/answer/6010255 (https://support.google.com/accounts/answer/6010255).

Sincerely,
The Google Accounts team


I tried what the mail says and it works now but would prefer to keep the security up.

QuoteUpgrade to a more secure app that uses the most up to date security measures. All Google products, like Gmail, use the latest security measures.

Googles saying SMF isnt using the latest security standards to log into the account so what is it missing?
Title: Re: smf mail
Post by: Night09 on September 02, 2014, 06:20:43 PM
Ive looked into this a bit more and Google have effectively upgraded the mail servers to encrypted HTTPS to try and stop likes of the NSA snooping into stuff sent back and too between servers.

This is technically now an SMF issue as other platforms will endeavour to upgrade if not already and also it renders the mail system half useless unless your willing to accept a lower security level on the Google side.

I dont know what would need to be done to get mail sending as encrypted HTTPS to meet the new standard or if its major work or not. I will be looking further into this though as its going to have to be addressed at some point. Who knows Google might one day simply say no more unencrypted mail allowed. In that case everyones high and dry without a fix.

If you have any ideas on this it might get the ball rolling but please no naysayers trying to convert opinion to 'Its not needed bla bla' Technically we dont need any tech to survive but since its here lets at least do our best to make it work right. ;)
Title: Re: smf mail
Post by: Kindred on September 02, 2014, 06:36:33 PM
and this is yet another demonstration of why we do not support third party connections as a standard part of SMF...

google, facebook, etc all change their APIs - and any update to out code requires a full release - which is not a minor task.
Title: Re: smf mail
Post by: live627 on September 02, 2014, 07:30:47 PM
Quote from: Kindred on September 02, 2014, 06:36:33 PM
and this is yet another demonstration of why we do not support third party connections as a standard part of SMF...

google, facebook, etc all change their APIs - and any update to out code requires a full release - which is not a minor task.
"don't use third-party systems because they change and we don't" - that's how I read it...
Title: Re: smf mail
Post by: Kindred on September 02, 2014, 11:51:13 PM
well, yes and no...

they change (anything) which requires us to make a major change - because releasing a new version is not a minor thing.

Honestly, I think all third party interactions should be separated as mods....  much easier to update for the people who care, without requiring a system update
Title: Re: smf mail
Post by: Hj Ahmad Rasyid Hj Ismail on September 03, 2014, 12:07:34 AM
Quote from: Night09 on August 31, 2014, 04:52:19 AM
Technically though its not those needing setting up as im mailing via google as a standard account so dont use the records to send mail.

Heres an edited version of the mail I recieved last night.

Quote

We recently blocked a sign-in attempt to your Google Account.

Sign in attempt details
Date & Time: Sunday, August 31, 2014 1:31:06 AM UTC
Location: England, UK

If this wasn't you
Please review your Account Activity page at https://security.google.com/settings/security/activity (https://security.google.com/settings/security/activity) to see if anything looks suspicious. Whoever tried to sign in to your account knows your password; we recommend that you change it right away.

If this was you
You can switch to an app made by Google such as Gmail to access your account (recommended) or change your settings at https://www.google.com/settings/security/lesssecureapps (https://www.google.com/settings/security/lesssecureapps) so that your account is no longer protected by modern security standards.

To learn more, see https://support.google.com/accounts/answer/6010255 (https://support.google.com/accounts/answer/6010255).

Sincerely,
The Google Accounts team


I tried what the mail says and it works now but would prefer to keep the security up.

QuoteUpgrade to a more secure app that uses the most up to date security measures. All Google products, like Gmail, use the latest security measures.

Googles saying SMF isnt using the latest security standards to log into the account so what is it missing?

If you are using gmail to send email from your SMF forum, do read this first: http://www.simplemachines.org/community/index.php?topic=504772.0
Title: Re: smf mail
Post by: JBlaze on September 03, 2014, 12:12:03 AM
Quote from: Kindred on September 02, 2014, 11:51:13 PM
well, yes and no...

they change (anything) which requires us to make a major change - because releasing a new version is not a minor thing.

Honestly, I think all third party interactions should be separated as mods....  much easier to update for the people who care, without requiring a system update

I agree, although the way I would do it is to make everything a module. You have your SMF core which contains all necessary functions and whatnot, then you have your modules such as the forum itself, user panel, messages, admin, etc. This way, instead of updating the entire system, you can update each module as needed.
Title: Re: smf mail
Post by: Kindred on September 03, 2014, 12:13:16 AM
that was one of the goals for the smCore that Norv and Fustrate were working on an abandoned.

We'll see what happens when we start working on SMF 3.0
Title: Re: smf mail
Post by: Dragooon on September 03, 2014, 07:13:24 AM
Well supporting SMTP over SSL should be our problem, its not a third party thing.
Title: Re: smf mail
Post by: Night09 on September 03, 2014, 01:51:17 PM
QuoteIf you are using gmail to send email from your SMF forum, do read this first: http://www.simplemachines.org/community/index.php?topic=504772.0 (http://www.simplemachines.org/community/index.php?topic=504772.0)

Thats an option I looked at Aphrasis but it means one person has to recieve the code and not sure how it would work with scheduled queue's where hundreds a day may be sent. You may be limited as Google would be assuming your sending a single mail at a time.

To use HTTPS just to narrow it all down a bit would it be the changing of a single template to accept the new encryption or would it need to really go deeper. Im assuming Apps and other Sites that meet the criteria have had to either be recoded to work or natively worked with the build process.

The main issue broken down is this: You cannot remotely log into a Google account that consists of a Username and Password to send mail. This fails at the logging in stage so it is not related to the mail content at this point. All browsers can log into Google and most other devices. The exception is likes of older IOS (under 6) and some other bits I cant remember. So I think the first thing to do is find out exactly what would be needed to upgrade the login security, is it just in need of a higher encrytion SSL certificate or am I missing things.

Im just trying to understand exactly what it is Google wants to be accepted then work out if it is viable in any capacity for SMF.
Title: Re: smf mail
Post by: Hj Ahmad Rasyid Hj Ismail on September 04, 2014, 01:10:57 AM
It wont be asking an authorization code all the time. You need to activate it just once. The remaining email sending will depends on what type of gmail accont you are using. Free one will definitely have limitations.
Title: Re: smf mail
Post by: 青山 素子 on September 05, 2014, 01:21:55 AM
Doing some research, it looks like Google is actively trying to stop use of older SMTP Authentication protocols to send mail. Their new "security" is now requiring OAuth 2.0 (http://googleonlinesecurity.blogspot.co.uk/2014/04/new-security-measures-will-affect-older.html) for logins. This is a major change and affects any applications wanting to specifically use Google's mail or other services. OAuth is a multi-step process that requires redirecting to Google's website and then back. When you follow the steps to "lower security", you're just re-enabling standard SMTP Authentication.

Not that OAuth isn't a bad idea, but it's very new technology, and Google's move to start causing disruptions is going to affect a lot of applications. Heck, they even note that Microsoft Outlook (not Express, not Windows Mail) is affected as well as Apple devices running iOS 6 and older (like my 4th gen iPod Touch, stuck on 6).

I personally highly advise that the server on which you are hosted is what you use to send the mail. It's pretty easy for the server admins to get mail sending working, and if you must use SPF, it's not hard to add in the webserver as an allowed server for mail sending. SMF has always tried to leave things that aren't part of the forum experience to those components that do it best. SMF isn't a mail client, so it allows you to use your server to handle that part, or another server if you need.

If this becomes a big issue, I could see an official modification (if it's even possible) for supporting OAuth with Google for SMF 2.1. There likely won't be anything done for SMF 2.0. Right now, I don't know of any other services requiring this type of setup.

Note that if you really want to secure your account, enable two-factor authentication and then generate an app-specific password for just SMF to use. I personally use two-factor on all my Google accounts because I'm paranoid like that.
Title: Re: smf mail
Post by: Hj Ahmad Rasyid Hj Ismail on September 05, 2014, 02:25:42 AM
I think that is with regards to forum registration options, right?
Title: Re: smf mail
Post by: 青山 素子 on September 05, 2014, 03:19:17 AM
This is the SMF mail setting, so it would be anything that generates an e-mail to send.
Title: Re: smf mail
Post by: Dragooon on September 05, 2014, 09:09:45 AM
Quote from: 青山 素子 on September 05, 2014, 01:21:55 AM
Doing some research, it looks like Google is actively trying to stop use of older SMTP Authentication protocols to send mail. Their new "security" is now requiring OAuth 2.0 (http://googleonlinesecurity.blogspot.co.uk/2014/04/new-security-measures-will-affect-older.html) for logins. This is a major change and affects any applications wanting to specifically use Google's mail or other services. OAuth is a multi-step process that requires redirecting to Google's website and then back. When you follow the steps to "lower security", you're just re-enabling standard SMTP Authentication.

Not that OAuth isn't a bad idea, but it's very new technology, and Google's move to start causing disruptions is going to affect a lot of applications. Heck, they even note that Microsoft Outlook (not Express, not Windows Mail) is affected as well as Apple devices running iOS 6 and older (like my 4th gen iPod Touch, stuck on 6).

I personally highly advise that the server on which you are hosted is what you use to send the mail. It's pretty easy for the server admins to get mail sending working, and if you must use SPF, it's not hard to add in the webserver as an allowed server for mail sending. SMF has always tried to leave things that aren't part of the forum experience to those components that do it best. SMF isn't a mail client, so it allows you to use your server to handle that part, or another server if you need.

If this becomes a big issue, I could see an official modification (if it's even possible) for supporting OAuth with Google for SMF 2.1. There likely won't be anything done for SMF 2.0. Right now, I don't know of any other services requiring this type of setup.

Note that if you really want to secure your account, enable two-factor authentication and then generate an app-specific password for just SMF to use. I personally use two-factor on all my Google accounts because I'm paranoid like that.
Damn, in that case official support is pretty low priority as of now. You can use a third party service like http://mandrill.com (allows up to 12k mails/mo for free).
Title: Re: smf mail
Post by: Night09 on September 05, 2014, 10:59:07 AM
Quote from: Dragooon on September 05, 2014, 09:09:45 AM
Quote from: 青山 素子 on September 05, 2014, 01:21:55 AM
Doing some research, it looks like Google is actively trying to stop use of older SMTP Authentication protocols to send mail. Their new "security" is now requiring OAuth 2.0 (http://googleonlinesecurity.blogspot.co.uk/2014/04/new-security-measures-will-affect-older.html) for logins. This is a major change and affects any applications wanting to specifically use Google's mail or other services. OAuth is a multi-step process that requires redirecting to Google's website and then back. When you follow the steps to "lower security", you're just re-enabling standard SMTP Authentication.

Not that OAuth isn't a bad idea, but it's very new technology, and Google's move to start causing disruptions is going to affect a lot of applications. Heck, they even note that Microsoft Outlook (not Express, not Windows Mail) is affected as well as Apple devices running iOS 6 and older (like my 4th gen iPod Touch, stuck on 6).

I personally highly advise that the server on which you are hosted is what you use to send the mail. It's pretty easy for the server admins to get mail sending working, and if you must use SPF, it's not hard to add in the webserver as an allowed server for mail sending. SMF has always tried to leave things that aren't part of the forum experience to those components that do it best. SMF isn't a mail client, so it allows you to use your server to handle that part, or another server if you need.

If this becomes a big issue, I could see an official modification (if it's even possible) for supporting OAuth with Google for SMF 2.1. There likely won't be anything done for SMF 2.0. Right now, I don't know of any other services requiring this type of setup.

Note that if you really want to secure your account, enable two-factor authentication and then generate an app-specific password for just SMF to use. I personally use two-factor on all my Google accounts because I'm paranoid like that.
Damn, in that case official support is pretty low priority as of now. You can use a third party service like http://mandrill.com (allows up to 12k mails/mo for free).

Its low at the moment yes but for those who maybe paid into google for enterprise accounts it may not be so easy to change. I understand its not a big concern at the moment but at least now its known knowledge it will allow people to at least begin thinking of alternatives or to upgrade the existing system to accomodate.  I think its just a matter of time before other mail providers copy googles lead as thats what tends to happen as people know so even if its two years down the line when it does become serious then hopefully its been resolved in some way. No one will be able to recieve mail registrations if all providers follw suit.
Title: Re: smf mail
Post by: ApplianceJunk on September 05, 2014, 11:10:35 AM
This has bee working great for me for just over a month now.
http://www.simplemachines.org/community/index.php?topic=504772.0

I like being able to see what mail is being sent from my forum through gmail, something I could not do before.
When i would send out a newsletter I would just assume it worked. Now I can check my gmail sent folder and see that the newsletter went out. Can also see all other email being sent though my forum, including PM's.

Just nice to be able to verify so easily that emails are working.
Title: Re: smf mail
Post by: 青山 素子 on September 05, 2014, 11:49:13 AM
Quote from: ApplianceJunk on September 05, 2014, 11:10:35 AM
This has bee working great for me for just over a month now.
http://www.simplemachines.org/community/index.php?topic=504772.0

I like being able to see what mail is being sent from my forum through gmail, something I could not do before.

Yeah, accounts using two-factor aren't currently affected as you're using application-specific passwords anyway.


Quote from: Dragooon on September 05, 2014, 09:09:45 AM
You can use a third party service like http://mandrill.com (allows up to 12k mails/mo for free).

Looks nice. I personally like Mailgun (http://www.mailgun.com/), which allows 10k for free, and additional at very low prices. Their mail logging capability is also really good. It's super easy to configure with SMF too since it's standard SMTP.

Disclaimer: Mailgun is owned by Rackspace and the place where I work is a Rackspace partner. To the best of my knowledge, I don't have any financial interest in the product nor does the company where I am employed.
Title: Re: smf mail
Post by: Hj Ahmad Rasyid Hj Ismail on September 05, 2014, 12:08:49 PM
Quote from: 青山 素子 on September 05, 2014, 11:49:13 AM
Quote from: ApplianceJunk on September 05, 2014, 11:10:35 AM
This has bee working great for me for just over a month now.
http://www.simplemachines.org/community/index.php?topic=504772.0

I like being able to see what mail is being sent from my forum through gmail, something I could not do before.

Yeah, accounts using two-factor aren't currently affected as you're using application-specific passwords anyway.

That what I was referring to earlier in post #7 (http://www.simplemachines.org/community/index.php?topic=527044.msg3738550#msg3738550) and was trying to explain in post #12 (http://www.simplemachines.org/community/index.php?topic=527044.msg3738895#msg3738895). ;)
Title: Re: smf mail
Post by: ApplianceJunk on September 05, 2014, 12:27:37 PM
Mailgun looks great too! Think I will give them a try. Think gmail limit is 3,000 a day. At some point I would like to send a email newsletter out to all 29,000 of our members and I'm not so sure that would work with gmail. Would be interesting to see some of the stats gunmail talks about too.
Title: Re: smf mail
Post by: margarett on September 05, 2014, 12:48:45 PM
It shouldn't cause you any issues if you set up your mail queue right ;)
3000 a day are 125/h, roughly 2/min.

This means that your newsletter would take 10days to be dispatched...
Title: Re: smf mail
Post by: ApplianceJunk on September 05, 2014, 08:10:17 PM
10 days is to long if I'm a letting them know about a giveaway promotion we are doing for the week, lol...
Title: Re: smf mail
Post by: Hj Ahmad Rasyid Hj Ismail on September 05, 2014, 11:21:25 PM
Naaah... Most users will not use email to check their favorite forum for latest promotions, if any. They'll frequently come to and see directly in the forum.
Title: Re: smf mail
Post by: ApplianceJunk on September 06, 2014, 07:09:04 AM
Yes, most will not. But why upset the few who do by sending them a email about it after it's to late.
10 days still just seems way to long. So what happens if during them ten days well I'm trying to get my emails out the server goes down or is reset or what ever. Then I have to start over?
Title: Re: smf mail
Post by: Hj Ahmad Rasyid Hj Ismail on September 06, 2014, 07:37:17 AM
Hahaha... Why should we be sending email at the first place? IMO there is no need to bother the users via email after email after email... There are already a lot of emails coming with promotions and everything and they might not even want to see them. Most of the time email has been abused by many including forum owners knowingly or unknowingly. I myself have a lot of email junks received every day. :)

Anyway, it is up to forum owners to decide as whether email is always the best way or otherwise in doing their promotions.
Title: Re: smf mail
Post by: ApplianceJunk on September 06, 2014, 07:42:02 AM
Yea, um... thanks!?
Title: Re: smf mail
Post by: Hj Ahmad Rasyid Hj Ismail on September 06, 2014, 07:53:05 AM
Since you got 29,000 members that read emails from your forum, yeah why not. Send them emails, lots of them. And get a better email service provider that can do that immediately, for free or otherwise. May be that's the best way for you and your members.

Don't worry about what I'm saying, because most of the time, I don't know what I'm saying anyway. Good luck!? ;)
Title: Re: smf mail
Post by: Night09 on September 06, 2014, 06:14:30 PM
Email is the most effective marketing medium on the planet. For every person here who may say they just delete mail fair enough but for each there is thousands who dont and follow a lot of emailed information and offers to the respective sites.