Post from Email

[NoFeaR / NFSMW]

Please?? I love this feature. It would be gr8 with a mailing list

[lisam9]

I've just used Package Manager to install this into my 1.1.11 board, but nothing seems to have happened - Package Manager says the package is there, but there's no new option in Features and Options, and no change whatsoever to e-mail notifications...

[madman71]

Im pretty sure it's not   1.1.11 compatible
1.1 is not the same as 1.1.11

[lisam9]

I was afraid of that...but since someone else mentioned getting it to work on 1.1.10, I was hoping there might be a fix.

[madman71]

ahh ok,  then you may be in luck 

[dakotaroar]

I'm in the process of trying to change this mod to work with 2.0 RC3.  Most of the difficulty for me is figuring out how the new database functions work (the replacements for db_query()), so it's going to take some time.  If anyone has tried this already and figure out that it doesn't work or needs a complete overhaul, please let me know and I'll save myself the trouble.  It would be quite useful on my particular board because some people hate communicating by anything other than email or text message.

[NoFeaR / NFSMW]

I would appreciate it if you can get it to work. Some of my users feel the same. Since they get email by phone using the website is a drag to some.

[dakotaroar]

Okay.  This is neither perfect nor complete, but seems to work for my test installation on SMF 2.0 RC2.  Sorry about that, I will need to upgrade and see if it still works.

If you would like to try this out on our test board, I ask that you restrict your test posts to the Test Topic at http://campusdakota.com/smf2/index.php?topic=4920.0 so we can all work on this together.  You will need to create an account and set up email notifications for yourself.

If you would like to try it out on your own board, you'll need to make a few secondary changes to Subs-Post.php (after making the changes according to the manual instructions) and replace the emailpost.php file (in your forum root) with the attached emailpost_changed.php file.  I cannot guarantee this will work for you.  Also attached is an example Subs-Post.php file with changes already made (made from a fresh copy of Subs-Post.php).

Now, to the coding issues.  I could use someone else looking at the emailpost.php file to see if they can solve some of the problems I was having (I simply commented out some parts).  I'm not familiar with some of the variables so some of the functionality is currently limited, especially the permissions-checking.  The code WILL check to see if the email address that is responding matches the address and timestamp of the notification email that was sent out, but if someone wanted to spoof that at this point, they probably could.

As always, use at your own risk and I can't guarantee any type of support whatsoever (but I will reciprocate further work if others contribute).  Comments on how I can express the changes better are welcome; at present I thought showing everything side-by-side was the most helpful.  If you're using a Mac, I overzealously suggest TextWrangler as a PHP editor.

[Kindred]

I think this mod can be viewed as "abandoned" so please submit your updates to the mod site so that they can be checked by the smf staff and validated as complying with the coding standards and safety standards.

[dakotaroar]

I think this mod can be viewed as "abandoned" so please submit your updates to the mod site so that they can be checked by the smf staff and validated as complying with the coding standards and safety standards.

I was looking for a little help before I went through all that trouble, but I will keep this in mind for when I get something more stable working.

[KensonPlays]

I would deff. use it if it was even just ported to smf 2.0rc3, or made for it, i would LOVE this feature!

[pftq]

Still running 1.1.x here, but I was stripping down the mod a bit to see if I could get it to work (removing PM support, etc).  Basically I had it down to where it just retrieved the topic it was replying to based on the subject, which was all I really wanted.

Just incase anyone else gets this as well, I found out that single quotes in messages were causing the script to fail.  The message body was also unreadable because it was encoded as a quote-printable string; I'm not sure if it was just because of my email client or not.  I fixed this on my end by inserting:

Code Select Expand

$text=mysql_real_escape_string(quoted_printable_decode ($text));
below

Code Select Expand

$text = trim($text);

It seems to be working so far for me (the post shows up, no errors on SMF).  I realize this mod is no longer supported, but just incase anyone else decides to tweak it to work on their forum.

The only other thing that is bothering me right now is that the original message shows up in the post as well, but I'm not sure if it's something I broke on my end or not so I will check up on that.

[dakotaroar]

What I was experiencing (before I fixed it in development for 2.0) was that HTML wasn't being stripped, it was just being inserted, and it would insert any quoted text as well (if the notification message was quoted, it would insert that into the post as HTML).

I ran into the quoted-printable issue as well; it seems to be an email client issue rather than an issue with the mod (although I'm adding support in, obviously).  Email from Gmail doesn't cause the problems but email from my school's webmail does.

I'll check to see if single quotes cause failure in my 2.0 development version.  Good catch.

[pftq]

Here's a modified version of emailpost.php I worked on the last few days.  It works for SMF 1.1.11 as well as older boards like 1.1.4.  I think the only thing that was breaking it was the missing mysql_real_escape_string (the single quote) and then maybe the quoted_printable_decode not being applied.

Anyways, I'm quite satisfied with it so far,  but as I stated above, I stripped out a lot of the extra features I didn't want (PMs, no consecutive replies, etc).  The only thing it does now basically is take an email with a matching subject and posts it to the board.  This means you can reply to both announcements and notifications.

The interesting part is that you only need the emailpost.php file now (no modifications of other files), so you can just place it in the main forum directory and have the email piping + 755 permissions set up for it.  Same instructions as before except there is no plugin installation needed.

Main changes I did are:

- Replies get isolated properly - I'm using regex as an extra check to look for "From.. To... Subject" blocks in the original message.  The original plugin didn't seem to differentiate between plain or html messages, so I have the script isolating just the plain text reply.  I am not sure how I would deal with HTML converting back down to bbcode, so I left that out for now.  I also found that '=A0' caused quoted_printable_decode to truncate messages, so I have a check to remove those as well.  Obviously, I can't test for every possible email client, so if you see the original message getting quoted, please let me know.  I'll want to fix it on my end as well.

- Topic reply is based on the email subject, instead of sent notifications.  It allows for replies from any email with the correct subject (announcements + notifications), but I think the original author was concerned about spam as a result.  The script still checks that the email matches the correct member + permissions, so it's not as if any email address can reply to the board.  The email can be spoofed, but it was the same case before.  It's just a risk I'm willing to take on my part.

- Smilies enabled for posts via email.  It was overlooked in the original.

- Notifications get sent out if someone replies by email.  It was also overlooked in the original probably.
^^ What I like about this being enabled is that you can now basically carry out an entire thread without once visiting the board - it functions almost like a mailing list.  I actually have my forum modified so that announcements/notifications are formatted to look like mailing lists.  See below...

Few points:
- Since the topic selection is based on email subject, I have to strip out prefixes like "Topic reply: ", etc.  Not quite sure how to grab that information (especially if everyone has a different language), so I have the text hardcoded in for now.  If anyone knows how to deal with this issue, please let me know.  Otherwise, right now, I'm basically just stripping out everything before ":" in the email subject to be safe; just don't use ":" in the subject of any forum topic.

On a side note, I'll say upfront that I'm not highly experienced in PHP, so I can't guarentee the code is fully secure.  If anyone wants to take a look and give suggestions, I'm more than open to them.  I personally don't think I made any changes that would make it any less secure than the original, but who knows.

Attached separately is also a Mailing List plugin I wrote which also formats the emails to:
- Send "From" the person posting the reply. Emailing back still goes to the forum though, no worries.
- Allow HTML (smilies/bbc) in notifications/announcements.
- Changes "Topic reply" to "Re: " and removes "New announcement: " in email subjects.
- And a few text-rewordings so it looks less like topic notifications and more like an email thread.

It's up to you if you want to install that as well (just put into plugins folder after extracting).  Like I said before, the emailpost.php works perfectly fine as a stand-alone file.

Anyways I hope someone finds this useful.  Just really happy on my end that I got it working for my board.

[Kindred]

please submit this as an update to the mod - t the mod site so that it can be checked by the mod team and validated.

[pftq]

Oh okay.  I wasn't sure how it worked if we were making changes to someone else's mod.

I'm still testing this right now though (adding cases to check for every time I see the original message not stripped from the reply).  It's posted if anyone else wants to go ahead and use it or help make improvements on it, since it does work so far as I can tell.

[dakotaroar]

pftq, that's great work!  I'll try to find time to look through your code this weekend to see if there is anything I can help with for that version.  I'm glad you got to fixing the lack of notification to emailed topics.

Can you clarify a bit more why you chose to match according to subject rather than sent notifications?  Would it be possible to match only on the sent time rather than the sent time and address in order to duplicate what you've put together, or would this break on a busy board with lots of notifications being sent out every second?

[pftq]

I am not sure what you mean by matching sent time or address - can you clarify that part?

The reason I matched by subject is so that announcements can be replied to as well as notifications.  I also didn't mind members replying multiple times to the same topic - some of my members do that actually.

One thing I noticed was that the post count was sometimes updating but not always for people who reply this way.  I see that the code for updating post count is definitely there (few lines above createPost) but not sure why it wouldn't work.  I will look into it later when I get a chance.

[pftq]

Updated the attachment with a later version.  I improved the email parsing a bit, so it's becoming pretty consistent for stripping replies from the emails (as far as I can tell).

I also fixed this line here:

Code Select Expand

'update_post_count' => $topic_info['countPosts'],
to

Code Select Expand

'update_post_count' => 1,

Previously, it wasn't updating the post count for the member.  It might have been a change in SMF version that caused the original line not to work.

In addition, I've added the following customization:
$txt['notifyXAnn3_event'] = 'View Original Event: ';

You can change this in your own theme so that it has different text for calendar events being announced.  This might be useful if you have the Event Registration mod for example, where you'd want to tell people people "Sign Up" through the link and not just read the email.  I left it as View Original Event by default for people who might not have the other mod.

[pftq]

Hi all - This is something I spent the last couple days on but finally got working fully.   Basically the mod now almost completely mimicks Googlegroups or other listserves/mailing lists (what's the correct term?).

I have the entire mod set up properly as a package with settings under Features/Options, so this should be much improved over my last attachment.

Here's a description from the readme for all the new functions I inserted:

This modification sets up your board to basically behave as a list serve or mailing list.  The best example to think of is "Google Groups", in that the discussion board still exists while any emails sent to the site are automatically posted or added as replies to existing topics.
      
      Credits for the original mod "Post By Email" go to Matthew Wolf (a.k.a Grudge).  The mod I've created based off of the original differs in that it:
      - Allows replies to any email (announcements and notifications) by the board.
      - Sending emails with new subjects also creates a new topic to a default board of your choosing (under Features/Options).
      - New topics started by email are  automatically announced (emailed) to members who can see that board, just as a real mailing list would do.  If the number of members able to see the board is large, this might cause a lot of server load by announcing to everyone. Then you might want to either pick a different board to start topics (with fewer members able to access) or disable this entirely (again, under Features/Options).
      - For all announcements (posted by email or board), members are automatically subscribed so they receive replies through email and can reply back through email.
      - Notifications of replies are formatted to look like email replies (with the last 10 posts quoted and indented).
      - All posts created through email show the original poster as the sender, rather than the board.


      INSTALLING ----
      To install this mod, you will need to configure your server to "pipe" email replies to this script.
      In cPanel go into Email -> Forwarding and then make the forum's email address forward to "|/home/path/to/SMF/emailpost.php" without quotes.
      Where the path is the path to the script (Note that path - not the URL!) If you are not using cPanel you'll
      need to set this up manually.

      If you have PHP installed in a path other than /usr/bin/php you will need to change the top line of emailpost.php to point to the correct path. Please note that you *may* also need to chmod the script "emailpost.php" to ensure it is executable (usually 755 or 777).
      
      If you have your own theme, make sure to copy these lines into the appropriate language files:
      
      For yourtheme/languages/Post.english.php
         // pftq / Mailing List: Shorter, more concise text for emails.
         $txt['mailingList_announce'] = "Reply to This Email or View Original Post: ";
         $txt['mailingList_announce2'] = "Unsubscribe from emails for this topic: ";
         $txt['mailingList_event'] = "Reply to This Email or View Original Event: ";
         $txt['notification_reply'] = "Reply to This Email or View Original Topic: ";
         $txt['mailingList_disable'] = 'To disable all future announcements, check the notification settings in your profile page.';
         
      For yourtheme/languages/Modifications.english.php
         // pftq / Mailing List: Settings
         $txt['mailingList_defaultBoard_help'] = "Board for Topics Started by Email";
         $txt['mailingList_autoAnnounce_help'] = "Announce Topics Started by Email";

Obviously, this has strayed a lot from the author's original intent as just to receive emails for notifications and PMs, so I am not sure whether this should still be considered an upgrade on the original or a different mod altogether (besides a few pieces in the emailpost.php, everything else was rewritten).

I'm still testing this and trying to see where I  may have overdid the email features a bit (as in made it too spammy), but so far it's been working great.  I'm currently using it on 3 sites for smaller groups, so load isn't too much of an issue.  I'm not sure about larger sites, but I imagine if you can announce to that many members, this shouldn't be too much different.  You can always disable the sending out of emails anyways (it's an option).

If you want to help test this (or just go ahead and start using it ), just let me know through email or PM; I'll be more than happy to send you a link.  Unfortunately, since it's still in Beta, SMF's site policy does not allow me to post a public link yet.  As usual, suggestions and anyone else helping test this would be great.  There were some areas I wasn't too sure on how to best code, so feedback is always welcome.