Notification email questions

[motechman]

I manage a forum for a very small community (61 members total) and I'm getting reports some of the members receive notifications sporadically. I noticed that the mail queue was not enabled, so I turned it on and set the number of emails per minute to a low number (5). I presume the system will keep filling the queue and sending emails until all notifications have been sent. I initially set the emails per page to be 4, but I have upped that to 100. I am using the default php mailer, not the smtp mailer. Here are my questions:

• Can the schedule be changed for sending notifications that is listed under Maintenance / Scheduled Tasks?
• Are board and topic notifications sent at the same time, and when does that occur?
• What is the "emails per page" setting used for, and is that in any way related to notifications?

Lastly, I have made some adjustments to the database so that all members are notified by email whenever anything gets posted to any board on the forum. Since the number of members is low I don't believe this is a problem. It may have been before I enabled the email queue, but with it now enabled it should not be. Perhaps I'll need to tweak the parameters, but 61 members in 38 boards results in a maximum of 2318 rows in the  smf_log_notify table. If we had 10 posts on the forum in any single day that would be unusual. The volume of traffic is extremely low, and although 2318 may sound big to some people it is actually quite trivial, especially considering only a very small subset of that number would have notifications sent.

[motechman]

Bump. BTW - I don't recall posting this here. I am using a mysql DB not PostGress or SQLite. But then I don't see any topics specific to MySql, so perhaps this is the best place, since it is a question concerning the database.

Seriously, nobody has replied on this? The DB update seems to work, at least partially, but a handful of members complain they aren't getting notified of new posts, yet when I check the notifications under their profile all of the boards are listed, they haven't opted out.

Surely one of the devs or mod makers could address this. The SQL I use to "opt in" all members into the board notifications is:

Code Select Expand


insert into smf_log_notify (`id_member`, `id_topic`, `id_board`, `sent`)
select distinct m.id_member, 0, b.id_board, 0 from smf_boards b join smf_members m where m.id_member not in (select distinct id_member from smf_log_notify)


I can see a 1 in the sent column for various boards where messages have been posted, the question is why don't ALL members get notified?

I'm suspecting an issue in the email subsystem, since the sent flag looks like it is getting updated correctly as new posts are created in the boards. Activating the mail queue doesn't seem to have had an impact.

[Kindred]

Actually no...   SMF's maintenance tasks are only run the next time after the schedule that a user opens a page...

So, if the system has 40 emails to send...  But can only send 5 at a time, then it will send. 5 on the first page load, then 5 more after the nfirst st page load after the  time period has incremented...

In other words, with a low queue rate and a slow traffic board, it could take a long time to send all the queued messages

[motechman]

Actually no...   SMF's maintenance tasks are only run the next time after the schedule that a user opens a page...

So, if the system has 40 emails to send...  But can only send 5 at a time, then it will send. 5 on the first page load, then 5 more after the nfirst st page load after the  time period has incremented...

In other words, with a low queue rate and a slow traffic board, it could take a long time to send all the queued messages

Thank you kindred for your reply. I'm still a bit foggy of the operation. First, I'm not sure I understand the use of  "page load" in this email context. Can you describe operation in terms of trigger events (one of which may be a page load), such as a maintenance schedule or user post to a board?

Are you saying the email queue is only processed when posts are made? i.e. if the queue depth is lower than the number of parties to be notified (as I currently have it configured), the queue will not be processed again until another post is made to a board? (which adds even more items to the queue)

I thought the intent of the queue size was to control or limit the number of messages being fed to the SMTP email layer. I also thought that processing the email queue was taken care of asynchronously as a background task, such as with a scheduler or cron job. If it is only processed synchronously to posts or page load events I will say definitively that is not a good design approach. I would be surprised if that is how the code is implemented, as there is 2 settings, a queue depth AND a rate of message transmission, which implies a timer or some type of schedule management to oversee the emptying of the email queue. I can see such a scheme working on high traffic forums, at least until the traffic flow fell off to a low rate, which it almost always will at some point.

How would you recommend I change my settings to ensure all messages in the queue are processed in a timely fashion, and without overloading the email subsystem or triggering "spam" filters due to high rate of dumping messages into the SMTP layers?

I currently have my rate set to 5 messages per minute, and the queue length set to 100, which is more than the number of users we currently have. If 1 user posts to 1 board, that is roughly 50 notifications, which would take 10 minutes to send @5 / minute. However, if that queue processing takes place ONLY upon a page load trigger event, and is not based on a cron job or similar asynchronous process, emptying the queue is non-deterministic and depends on user traffic.

If I understand you correctly (and I'm not sure I do), the queue size needs to be set to a multiple of the number of users. i.e., if I have 50 members the minimum queue length would be 50, which covers 1 post to a single board to be sent as a notice to all 50 members. If several users are posting or posts are made in multiple boards that just increases the need for a bigger queue depth. Do I have that right?

[motechman]

Bumping this question again. I can't find much information about how notifications are processed and I could really use some help in understanding this.

I have a low traffic forum with a low number of users. How should the system best be configured to notify all users of new items posted anywhere on the forum?

[motechman]

I posted my question under support back in February but still haven't gotten the answers I need.

I am trying to configure the system to notify all users whenever anything is posted on any board. The changes I made seem to work for some users but not for all.

I really would like to understand how notification emails are processed and how I can troubleshoot why they're not going out to everyone. I also need to understand the mail queue configuration and the role maintenance & "page loads"  have on this. I haven't found any documentation about how that works.

More details can be found here.

Thx for any suggestions you may have.

[nend]

Each user account has notification specific settings. I am guessing some may have email notifications disabled. 

[motechman]

Thx for the reply nend, but that isn't the case. I have verified those who have reported the issue.

[motechman]

I believe I found an explanation in this thread:

Are there users online in the time where no mails were sent? Because SMF has no "background daemon" that keeps sending emails at fixed intervals. SMF sends emails on page load, with calculated numbers. Which means that, if there are no users loading pages, no mails are sent.

That is crucial information that should be added to the wiki. This is not an issue for higher volume forums but it sure is for those with low traffic! If "page loads" can be any page load by anybody (no logged in user), this can be s[t]imulated by external means such as a cron job or monitor service like monit.

Now that I know this part of the puzzle I will run some experiments to see if the queue is processed on each reload of the main forum page. Another question that arises is: are items added to the email queue upon page load or are they removed and sent out on page load? It's also possible that inputs AND outputs from the queue are all done on page loads.

Although this "page load" design approach does not require setting up cron or some other asynchronous process, it has a major drawback of adding overhead to incoming requests. 

I still believe there is another missing piece of the puzzle however. If notifications were working correctly as I want them to be each new post should generate 61 emails (the number of registered users) in the queue, and that's not happening. That isn't precisely what should happen, but after a long period of no posts (like several days) the first post should generate an email for every registered user. Several posts could be made by a user in a single session, but they should not generate additional emails.

[Illori]

if you did not have the mail queue enabled and still had this issue then i dont think it is an issue with the pages being load to cause the emails to be sent.

[motechman]

Yes, I agree, but I do have it enabled.