How to get the SMF 2.0 mail queue to be more responsive, can a CRON job help?

[imrich]

It appears that the SMF 2.0 mail queue is working well, at least to keep too many emails from going out per minute.

This is more important as many shared hosting servers limit outgoing emails to a certain number per hour (the actual number varies per host). The shared hosts that limit the number of outgoing emails per hour are doing so to help to keep the amount of spam generated by users to a minimum and also to not to place too heavy of a load on the shared server resources.

Unfortunately, the SMF 2 mail queue seems to work by sending out X emails per page load. This works well for a busy forum, but for a small forum that is not very busy, it means that sending out emails can take hours (or a day or two) if you have several hundred members, but not a lot of hourly activity on the forum.

Every once in a while, I need to use the SMF 'newsletter' feature to email all members about a special event.  This tends to generate one email for each member (about 500 or 600 emails). This can take up to 24 to 48 hours to process the queue, not because of my queue settings, but because often my forum is not accessed (it just doesn't have a lot of activity).

So the question is: Is it possible to use a CRON JOB to run a small task that will keep the outgoing mail queue flowing smoothly, without placing too much of a burden on a shared server? If so, can someone please provide instructions or details on how this could be configured?

I've searched for this information and most questions about this topic simply say to use the SMF 2.0 mail queue feature. This is not working well for me.

Thanks for any help!

[MrPhil]

One possibility would be to use PHPlist instead, to send out large scale broadcasts. It doesn't depend on "poor man's cron" that SMF appears to be using.  Assuming that SMF is not tunable (to use real cron if available), is there any possibility of using cron to "poke" your forum every minute or two? Say, run ..../forum/index.php from the cron command line, or maybe something else that will awaken SMF enough to get it to cough up another batch of emails? I don't know specifically what SMF loads would be enough to trigger the PMC without actually doing some operation, or worse, hanging up the system. Maybe you'll have to write a stripped-down index.php that does nothing but run long enough to fire off emails and then quit (name it pokeemail.php).

You're on a Linux server (which has cron), and not a Windows server (which doesn't)?

[imrich]

Yes I'm on a Linux server (so I have cron).

If i had to use PHPlist, then I have to create yet another database of users, and I was hoping not to do this. It will also not get user notifications out as fast too.

I also don't really want to invoke the whole forum every minute, as that could put a load on the shared server. I would prefer to have something that will process the email queue only.

Thanks for the ideas, but it sounds like there is no way to do this within SMF 2 today?

[SlammedDime]

You could possibly just call ssi.php via a wget call with cron to run the mail queue.  I'm not sure if SSI will run it though, I honestly haven't looked at the code too much.

[imrich]

Now there's a good hint, thanks!!!

Is there a way to test (or to check) if invoking SSI will cause the mail queue to get tickled?

[SlammedDime]

Browse the source code...

[imrich]

Of course, the source Luke, the source!!

Actually, my question was to wonder if there was some sort of log entry, or other indication of when and how the mail queue was last invoked.

[imrich]

Doing some digging into the source, I don't see that SSI would cause scheduled tasks to function, but it does appear that I can fire off a CRON that executes a curl (or wget) of (for example) www.simplemachines.org/community/index.php?scheduled=mailq to tickle the mail queue into getting processed.

Does this make sense or am I making some bad assumptions?

[imrich]

I'm currently testing using CRON as follows:

curl http://www.mydomain.tld/smfforum/index.php?scheduled=mailq

The output of curl shows:

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0    43    0    43    0     0      8      0 --:--:--  0:00:05 --:--:--     8
  0    43    0    43    0     0      8      0 --:--:--  0:00:05 --:--:--     0
GIF89a
�
����������!�
����,����
�
��D
�;

Which indicates to me that it's working. Note: When the scheduled mail task finished it outputs the a transparent GIF file (43 bytes), that's why you see the 'GIF89a' and some binary (unprintable) bytes in the output.

To not get this emailed to you every time you fire your CRON, use this command line instead:

curl http://www.mydomain.tld/smfforum/index.php?scheduled=mailq >/dev/null 2>&1

Right now, I'm testing by having this cron fire every 5 minutes:
*/5 * * * * curl http://www.mydomain.tld/smfforum/index.php?scheduled=mailq >/dev/null 2>&1

If you don't want the wget progress meter output, then use this command line:
*/5 * * * * curl -s http://www.mydomain.tld/smfforum/index.php?scheduled=mailq

or the same but also do not send an email to you when this exexcutes:
*/5 * * * * curl -s http://www.mydomain.tld/smfforum/index.php?scheduled=mailq >/dev/null 2>&1


I hope it will get the mail queues delivered in a more regular pattern instead of taking hours.

[MrPhil]

"Real" cron will email you with anything sent to the stdout (regular output) or stderr (error output). You normally >/dev/null to deep-six the routine stuff you don't want to see (after development is done). You can also do 2>&1 to also can the error messages, but it may be a good idea to let them through (errors should not be routine). That particular statement sends stderr to stdout's device, which happens to be the trash can. It sounds like the "poor man's cron" may be working the same way.

[imrich]

MrPhil,
Thanks for your information. It helps to make this thread more complete.

So far this has been working on my system. I need more time to test it to be sure there are no problems and that it does indeed cure my original problem of email sitting in the mail queue for a long time when the forum is not accessed often.

I'm also using phpList for some other functions. I added an easy method to import email addresses as needed, so we may be switching to using phpList for our newsletter needs, but I'm still glad to have SMF 2.0's email queue running more smoothly.

It might be nice if the developers added a method to support tickling the mail queue via CRON for those forums that are not accessed very often. This way a user could use the built in scheduler, or use it enhanced vai CRON.