Editing marks posts "unread"

[djr33]

My question should be simple (but implementing it might not be).

I want a post to be marked "unread" when it is edited. [I'm using SMF 2.0.6.]


There are two reasons:
1. Edit spam is a major problem and there is no good solution to directly deal with it. This modification would solve it.
2. Although this is unusual behavior for a forum, I think it's a good idea in general. If someone changes their post, in general (unless it's a typo or something) we want to know about it-- a conversation on a forum isn't just linear when editing can occur. If editing marks posts/topics unread, it will also help to discourage double-posting to bump a thread-- the edit will do that in itself.

Certainly this isn't for everyone. Some forums would prefer to keep the current system. That's fine. Personally I'd suggest this as a default optional feature of SMF because it helps to prevent spam. It would be useful for at least moderators to have this option, possibly not to always bother regular members with it.


I'm experienced with PHP and I've spent quite a bit of time working with the code behind SMF (although more for SMF1 than SMF2, which I've recently started using).

To start, I'd simply like to make this modification manually ("destructively" if necessary). Looking into making it a mod later is also an option, even a default feature in the future if there's interest.


So my question is simple: what help can you give me to point me in the right direction to do this more or less in the "right" way rather than cutting corners too much?

I haven't yet looked through the code to figure out how the "read" feature works. Is it on a per-user basis or is it based on post time? I could imagine changing this either by manually removing all having-been-marked-read instances in the database, or by updating the post time but that would have unwanted effects.




By the way, to really do this right, I think it should be tied into the distinction between a marked edit and an unmarked edit-- I think that's called "courtesy edit time" or something. If you make a quick change in the first minute or so after your post, it doesn't say "edited by X at 00:00". So likewise, I'd think that it should only mark the topic unread if that window has not yet passed. This might make it more complicated to implement.



Thanks!

[Arantor]

It does flag as unread on the message index, but not in the main unread page. Changing this behaviour would be... interesting from a performance perspective.

[djr33]

It does flag as unread on the message index, but not in the main unread page.

What do you mean exactly? Is there a way of intentionally utilizing that feature?
Let's say that a spammer edits a link into a post from 6 months ago, or just a week, after the thread has stopped being bumped by new posts.
Is there some way to know that the thread has new information?

Changing this behaviour would be... interesting from a performance perspective.

It would be different, yes. But why the vague answer? Would it be bad in your opinion?

[Arantor]

If you edit a post, it will be shown on the regular board listing of topics as unread. There will be the 'new' icon next to that topic. But it won't show up in unread (though it should show up in unreadreplies which is handled a different way)

The reason for the vague answer was because I was on my iPad at the time and didn't want to get into a lengthy treatise about the way unread is calculated, seeing how it's stored across multiple tables and forcing adjustments the way you have in mind is likely to have some interesting side effects, of which the only real way to be sure is to cut one of the performance saving systems.

As far as edit spam goes, there is always: admin > forum > post settings > maximum time after posting to allow edit.

[djr33]

If you edit a post, it will be shown on the regular board listing of topics as unread. There will be the 'new' icon next to that topic.

I don't understand. As far as I can tell, this is simply false. I edited a post with my main account then I looked at the page again from a second testing account. Nothing was marked "unread" anywhere.
Is there some detail I'm missing?
Does this depend on the default theme (which I'm not using)?

But it won't show up in unread (though it should show up in unreadreplies which is handled a different way)

Personally what I'd like would be either:
1. The "there are new posts" icon and unread thread formatting on the forum, just like with a new post.
2. A specific page that lists all recent edits that were after the thread was last read by the user. (This would be a moderator's page probably, not for regular members.)

But in general all I require is some way to check (I hope conveniently) for new edits. The details beyond that don't matter much.

The reason for the vague answer was because I was on my iPad at the time

I wasn't in a hurry, but I would like an answer. I didn't know you were planning to return; that's fine, let's discuss it then, when you have time. Thanks

...a lengthy treatise about the way unread is calculated, seeing how it's stored across multiple tables and forcing adjustments the way you have in mind is likely to have some interesting side effects, of which the only real way to be sure is to cut one of the performance saving systems.

No offense, but that seems like a design flaw in SMF if the system is really so complicated that performance would be messed up by switching posting time for editing time!
For best performance, this should be as simple as possible, and a simple system like that shouldn't cause massive performance issues if changed.
(Now, I still haven't actually looked through all of this. I realize it might be complicated, and certainly that it would be interacting with core functions, so I was asking first here to see if there were any pointers. Maybe once I do look at the code it will be easy and cause no problems. But from what you said, it isn't.)
Roughly, I'd imagine it would look something like this:
1. Whether a topic/post is unread is based on whether the new information (new posts or newest edits) is newer than the user's "last read time" for that thread or their global last read time.
2. Last read time is stored on a per topic (or per message/page) basis when we read a topic/post with a per-user database entry. "Mark all topics as read" removes all saved "last read times" and adds a global last read time to the database, a starting point.
[The only modification is in italics. Fairly simple.]
It's possible the system works nothing like that, I suppose. But I'd hope so-- that seems by far the most efficient to me.

As far as edit spam goes, there is always: admin > forum > post settings > maximum time after posting to allow edit.

That's not a direct solution, and that's why I said "or just one week" in my post above.
Setting maximum edit time is a balance:
1. We want to stop spammers from editing late.
2. We want to allow users to edit their posts.
There's no logical way to solve that to be certain that the thread will always receive new replies after a post is edited. That really isn't a solution, and that's not what a maximum edit time is designed for.
A maximum edit time is designed to stop users from changing content after a certain time has expired-- a week, a year, etc. For example, the original poster might go in and remove all of the information, making the thread useless for future readers. If an admin decides to disallow that (which is reasonable), that's why, not because it helps to prevent spam.

If editing is allowed for one day or two weeks or three years, spammers can still edit the spam until the end of that window and there is no way to know-- so, yes, by setting the maximum edit time to one week, it would be better than one year, because I'd know to be constantly re-reading every thread younger than one week to check for spam. Obviously, that's not a solution.

The only solution is setting the maximum edit time to 5 minutes or something, so that it's really implausible for spammers to edit after you've read the post. But that's a huge inconvenience for users, unacceptable to me.

I have thought this through. I appreciate the reply! But please don't try to talk me out of this. I know exactly what I want. I'm just wondering if you have any advice on accomplishing it-- so far you've hinted at some useful information I didn't know about editing marking posts unread (somewhere). I hope that leads to a solution for this.

[Arantor]

For best performance, this should be as simple as possible, and a simple system like that shouldn't cause massive performance issues if changed.

Sorry but that's wrong. We've been studying the largest SMF forums... consider trying to store the unread status of millions of topics, for hundreds of thousands of users. Worst case scenario becomes number of topics x number of users - except the system SMF has had in place for 10 years makes it drastically smaller (and thus faster) by not storing all those rows.

*shrug* Best of luck to you.

[Kindred]

why don't you just stop the spammers before they even get into your forum?

I will confirm what Arantor has said -- if you edit a post, the (NEW) icon appears next to that thread for other users.
(it may only do it if another user has posted SINCE the original post and the edit was made AFTER a repsonse... not sure. I have nto looked at the specific logic)
I see it regularly on this forum when k@ edits his older posts.

As for what you are asking...    as Arantor says, it would likely be non-trivial in terms of performance impact.

The way thigns are calculated is not nearfly a simplistic as you seem to think they should be (for some good reasons, and for some not so good reasons - but the fact is what it is)

[djr33]

Sorry but that's wrong. We've been studying the largest SMF forums... consider trying to store the unread status of millions of topics, for hundreds of thousands of users. Worst case scenario becomes number of topics x number of users - except the system SMF has had in place for 10 years makes it drastically smaller (and thus faster) by not storing all those rows.

Is there some documentation for this?
Is it layered, such as marking a post read, a page read, a topic read, a subforum read, a forum read, and then the whole site? That would be better for performance. But it still (I would hope!) shouldn't make it impossible to compare editing time rather than posting time.

A solution that seems to work just fine for many websites is a cutoff, such as one session. This can be slightly annoying if you don't get through everything in one session, but it overall seems to be fine and certainly helps performance. Track read-ness for a whole session, but beyond that everything is marked unread, by assuming that you read everything you wanted to in that session. This is, I believe, implemented (at least) on earlier versions of vBulletin, possibly other systems (and I'm not sure about current vB versions).
While that's not ideal for all, it certainly seems like a great (optional) solution for a massive site like you describe




Hi Kindred,

why don't you just stop the spammers before they even get into your forum?

That's a myth. Popular websites are attacked by human spammers who know how to blend in.
A significant portion of spam can be stopped by automated filters and manual filtering of the original posts, but some still slip through. As a moderator on a very successful web design forum, edit spam is a pet peeve of mine. Admittedly it's probably not a very large percentage of the spam! I'm not attempting to exaggerate anything here.
Once I started thinking about it this way I wondered why no forum software (to my knowledge) allows this feature-- in my opinion it's good for normal usage anyway, because it's good to know when someone updated information. Editing makes forums non-linear, so the notification system should also be non-linear.

I will confirm what Arantor has said -- if you edit a post, the (NEW) icon appears next to that thread for other users.
(it may only do it if another user has posted SINCE the original post and the edit was made AFTER a repsonse... not sure. I have nto looked at the specific logic)
I see it regularly on this forum when k@ edits his older posts.

Hmm.... sounds useful, but I haven't been able to replicate it. Any ideas?

As for what you are asking...    as Arantor says, it would likely be non-trivial in terms of performance impact.

The way thigns are calculated is not nearfly a simplistic as you seem to think they should be (for some good reasons, and for some not so good reasons - but the fact is what it is)

That's the problem. What would the solution be?
Is there documentation somewhere?

In general, for the record, I'm having a hard time with the switch to SMF2 because I taught myself how SMF1 works so I generally knew the ins and outs. With SMF2 I'm having to learn again (which is fine, and my own problem of course) but I can't find any documentation on how the actual code works-- am I just missing that? Or does everyone here who knows about it just know from experience and/or being an original developer? If I'm missing info I should have, just let me know.

[Kindred]

well, we have the wiki for information on using and integrating SMF and the function database for listing the functions in SMF and what they do... but most people learn the code of SMF by looking at the code....

And no...   a read by session would not ever be considered a good idea. There are far too many times that users log in, skim one or two and want to come back for others later...   we're not going to shut down the normal way of using a forum for some perceived desire (which, since it's never actually been asked before, can't be something that most people consider important). The assumption that someone has read everything they want in a single session is one of the worst asusmptions that I have heard...   if anything, the opposite should be assumed.

And also... redoing the way that SMF handles unread posts is another non-trivial task... I'd go so far as to suggest it is a MAJOR re-write.

[djr33]

well, we have the wiki for information on using and integrating SMF and the function database for listing the functions in SMF and what they do... but most people learn the code of SMF by looking at the code....

Ok. Thanks. I just sometimes have trouble finding where something is located. For example, it will take me a long time to read through all of the pages to track down exactly where the 'read' values are created, stored and modified. But I can do it.

And no...   a read by session would not ever be considered a good idea. There are far too many times that users log in, skim one or two and want to come back for others later...   we're not going to shut down the normal way of using a forum for some perceived desire (which, since it's never actually been asked before, can't be something that most people consider important). The assumption that someone has read everything they want in a single session is one of the worst asusmptions that I have heard...   if anything, the opposite should be assumed.

Optional features are a good thing. I would never suggest this should replace the current system.
You're right that not many people request this. I think it's because not many people think of it. But I can imagine that some (I don't know how many) would like it.
I'm genuinely not asking you to develop this for me. I can do it on my own. But discussing it here before I start is helpful.
Also "perceived desire" is a bit harsh (and a tautology). It is most certainly my desire; I don't claim it for anyone else.

And also... redoing the way that SMF handles unread posts is another non-trivial task... I'd go so far as to suggest it is a MAJOR re-write.

Certainly. But:
1. I'm at least interested in exploring how this would be done.
2. That *might* be the fault of a bad design, given that, at least in theory, using editing time rather than posting time (both of which are already stored in the database) isn't all that complicated.

I'm still not convinced this is really going to be so hard. Generally speaking when I modify SMF, the hardest part is finding where operations are performed. The rest is trivial in most cases.




Anyway, I promise that I'm being reasonable here-- help me to the extent that you have the time (and interest). I understand that may be limited. And thanks

[emanuele]

But it still (I would hope!) shouldn't make it impossible to compare editing time rather than posting time.

It does simply because SMF doesn't compare read/unread against time, but against message ids.

Once I started thinking about it this way I wondered why no forum software (to my knowledge) allows this feature-- in my opinion it's good for normal usage anyway, because it's good to know when someone updated information. Editing makes forums non-linear, so the notification system should also be non-linear.

Well... TBH it would be annoying to me...
I mean: I read a 100 pages topic. After a while someone decides he wants to edit a message on page 2, then someone a message on page 15. The topic is marked (again) as unread, and when I click on it I'm pushed to page 2 (of 100) instead of the one I expected (since I already read all of the pages). My first reaction would be... WTF! Close the page and go.

Is there documentation somewhere?

In the code, very well hidden. (j/k... to a certain extent )

In general, for the record, I'm having a hard time with the switch to SMF2 because I taught myself how SMF1 works so I generally knew the ins and outs. With SMF2 I'm having to learn again (which is fine, and my own problem of course) but I can't find any documentation on how the actual code works-- am I just missing that? Or does everyone here who knows about it just know from experience and/or being an original developer? If I'm missing info I should have, just let me know.

As far as I remember there are no big differences in the code from SMF 1.x to 2.0, the biggest change is the use of $smcFunc and few other things (like "menus" or "areas"), but it's more things moved around than anything else.

Back on the topic, I could see potentially interesting a log for moderators of edited messages, sort of reported posts) that can be investigated and closed.

[djr33]

It does simply because SMF doesn't compare read/unread against time, but against message ids.

Time is never involved? Odd.
But in that case, all database entries for a message id being read can be wiped from the database when someone edits a post. Also a relatively simple solution.

Well... TBH it would be annoying to me...
I mean: I read a 100 pages topic. After a while someone decides he wants to edit a message on page 2, then someone a message on page 15. The topic is marked (again) as unread, and when I click on it I'm pushed to page 2 (of 100) instead of the one I expected (since I already read all of the pages). My first reaction would be... WTF! Close the page and go.

That's a valid point!
In the most basic version, that would be confusing, especially for long threads. Additional features could add to this to help, such as including an "unread messages" page (integrated with the new version) that would specifically bring up exactly which posts were edited. There are also other options, such as not allowing edits to any post more than 2 pages old, which isn't a crazy idea anyway (but could require some work to set up). As an idea, this is workable. In practical ways, there may be some problems.
One simple option would be to make this available on a per-user basis. Personally I'd suggest it for at least one moderator. I'd personally be willing to do that. Remember, edits don't happen all that often... maybe 10% as often as posts. And within a 100 page thread would be very rare in that, so maybe a handful of times per year. Fine by me.
But, as with anything, it's a tradeoff. I am thinking about how to work out some of the bugs.

In the code, very well hidden. (j/k... to a certain extent )

Oh. I do understand the code while I'm reading it. I just often have trouble finding where to read.
If someone feels like a project, putting together an index of where things happen (file names and approximate line numbers) would be incredibly useful. I realize that's a huge task and would vary by what we're individually looking for. As-is, I must track through all of the code to find what dependencies exist and eventually arrive at where I need to edit; that works, it's just a bit slow, especially while I'm adjusting to the new setup. It's getting faster.

As far as I remember there are no big differences in the code from SMF 1.x to 2.0, the biggest change is the use of $smcFunc and few other things (like "menus" or "areas")

It doesn't feel completely alien, no.

, but it's more things moved around than anything else.

LMAO. That's the problem. I knew where things were in SMF1. Now I need to find them all over again in SMF2. I'm not complaining at you, but it is an adjustment. This is an inherent problem working with someone else's code.

Back on the topic, I could see potentially interesting a log for moderators of edited messages, sort of reported posts) that can be investigated and closed.

Yes, absolutely. My idea for making editing cause a post to be unread is a preference/idea, not a requirement. Certainly an "edited log" would be great for the purpose of moderation. In fact, I suppose that could be easier for me to write possibly, just making a not-so-integrated page and reading the database directly. "Edits in the last 2 months" with links or something. That wouldn't be polished (inconvenient) but it's a start. And a good version of that would be possible with more work, certainly.
There could also be a cutoff-- only edits made after X days would go into the log-- that would cut down on a huge percentage of them.

[djr33]

New/secondary idea:

Someone at my forum had a great observation:

The "a new reply has been posted" warning while replying to a topic is very helpful!

But that's where this would actually be most relevant. Most edits happen soon after posting, and if you're in the middle of an active conversation (a debate for example) then being notified of edits would be a huge help here.

What do you think? Would this be a desired improvement for many/most users?


And how would that work? Any pointers?



I have, for the record, started to look into the code and work a few things out. You're certainly right, though, that this is somewhat complicated.


What I've worked out so far is the following:
mysql_query('REPLACE INTO pfx_log_topics(`id_msg`, `id_member`, `id_topic`) VALUES (246, 1, 61)');
mysql_query('REPLACE INTO pfx_smf_log_boards(`id_msg`, `id_member`, `id_board`) VALUES (246, 1, 8)');
I was testing with the my admin ('1') account. I'll generalize later.
I was also testing with message 246, which is in thread 61, which is in board 8.

Those are flat queries, so they're not yet re-integrated into the more proper coding style, but they do the job, at least part of it.

The first line marks the topic unread from board view. The second line marks the board unread from the index. (I also tested this, and it works properly for later being marked read when you do view the message.)

It doesn't, however, integrate into the 'unread posts' page.

The next steps are to work out a few details and then generalize this:
The query should be conditional so that anyone who has read posts in the thread is bumped back to not having read them, but not that other people have. I'll work out the details later.



FYI, the markRead() function (Subs-Boards.php, line 247) is very messy and confusing. It has a bizarre long middle section that is an exception for marking things unread based on the $_REQUEST info, which seems completely backwards compared to how most of the board relies on first parsing the actions then calling a function with parameters. I know some corners need to be cut to actually finish writing software, but if there's time in the future this would be a good thing to look into cleaning up. It's semantically and procedurally anomalous.

[margarett]

The "a new reply has been posted" warning while replying to a topic is very helpful!

This was discussed not so long ago and, IIRC, there is a tip or even a MOD for it (not entirely sure now and I can't find that topic...)

[Illori]

It is a setting in your profile under look and layout

[margarett]

I don't think it is that. He's talking about a message being displayed while the user is replying, not just when posting.

[Illori]

that would take a lot of server load and i dont know if any mod that exists that does that.

[margarett]

Yup, this was the main discussion point...

[Arantor]

Notification of *new* posts could be done easily enough as part of the autosaving drafts in 2.1 (and not particularly unpleasantly), but doing edited posts would be expensive.

[djr33]

I'm confused by the last 6 replies. As far as I know, this is simply the default behavior in SMF2....

Admin > Configuration > Themes and Layout > Member Options
Then change the settings for a particular theme (for all members, new members, etc.)
Or individual members can access this in their profiles.

The particular option is:
[_] Don't warn on new replies made while posting.

Clearly the default is to warn when a new reply has been made. It can be disabled by default by the admin and changed by individual members.



I hope I'm not missing anything-- this doesn't come from a mod I installed as far as I can tell.
In fact, it is an option here within my "Look and Layout" settings in my profile!





So...
1. This is already part of the forum.
2. It warns about new posts, not new edits.




Sorry, but I just don't understand the last posts. This is already a solve problem, except that it only deals with new replies not new edits.

Notification of *new* posts could be done easily enough as part of the autosaving drafts in 2.1 (and not particularly unpleasantly), but doing edited posts would be expensive.

It's already implemented (though I don't know if it was easy or not), but why would responding to edits be all that difficult? "expensive" is not relative to utility of the feature. It's just about as hard as having the current check to see if any `poster_time` entries are later than the beginning of the new post-- just add the same check but for `edit_time`. It's fine if you don't personally like the suggestion, but I think it would be useful and am sure there's a reasonable way to do this.




Now, to be clear, I'm talking about already submitted/saved/posted edits/posts. I'm not looking for anything about drafts or any notifications of "in progress" editing/posting.

Simply, if someone has changed a post while you were writing the reply, it wouldn't be a bad idea to be notified. And this would be VERY useful for active debates and other kinds of fast threads.


I hope I've clarified it.