MyStatus Update Mod (beta) - a social network extension for SMF 2.0

[sangwe11]

I really dont see the point in deniying them access to this board through all these places.

Much easier to just create a new table.

IMHO, you need to rethink how this mod works, and again IMHO the design of the layouts.

[SoLoGHoST]

SSI.php is supposed to be used to show any board.  May I ask, why don't you just create Tables.  For example status_message and status_topic tables can be created.  Or perhaps you really only need 1 table:  status_message.  In any case, I think the short version of what Arantor was saying in his post, and I could be wrong, is that your mod will not get approved as long as you are using boards for this, since there are too many concerns as SMF ties directly and indirectly into topics/posts more so than one could possibly comprehend... STATS, Post Count, Topic Count, etc. etc. etc..  And getting rid of all of these in your mod poses a huge security risk as well as risking the functions of SMF in itself.  So, in short, just create a table or 2, to handle this as it's really the only way to get this functioning properly without interfering with SMF's posts and topics.

[ElectricSquid]

For all of you who think that hiding a board is a security risk, and that it is too big a job to do, keep in mind that the Recycle Bin is hidden exactly the same way I am hiding the MyStatus board. In fact, it's been easy to hide the mystatus board from normal view because I'm simply excluding it everywhere the recycle bin is excluded already, and the same way the recycle bin is excluded. Directly in the WHERE.

Why do I want to use the SMF board and topics tables?
That one is easy, because to allow for ALL the functions of SMF to happen for my mod, and do it with new tables, I would have to pretty much rewrite SMF.

Instead, I am simply using 1 board, hiding it like the recycle bin, and allowing access ONLY through my mods pages.

Say a member has a Status Update that they would like to port over to the forum as a real topic. How do you suggest they do that the way you are saying to code this mod? They can't without extensive coding.
With the way this mod is coded right now, that is possible. All a admin needs to do is go to the board directly via URL, or just have the user hit the spam button, and that will bring them to the Report to Moderator page, where they can ask to have the Status Update moved to the forum. In the moderator page, the link there will bring the mod to the normal forum view for the MyStatus board, where ALL moderator functions exist, including MOVE, SPLIT, LOCK, etc.

That is why I am using the existing SFM board and topic tables, and coding, just showing it differently.

As far as skewing the Stats page, I just fixed that with version 1.2, which will be available as an attachment in the first post in about 2 minutes.

[Arantor]

Let me clarify that statement.

It is NOT hidden the same way at all. If it were you wouldn't have to modify all those files, because those files do not investigate the recycled board variables, and that is where my issue is. Might want to investigate what {query_see_board} and {query_wanna_see_board} do.

Converting a status to a full blown topic? Create an action/subaction and push it via createPost. Really not a lot of code there.

How often would you seriously need to split/merge status updates?


The one issue I will come back to, however, is performance. This will have a non trivial effect on the performance of SMF as a whole. Don't forget, when you modify the tables you are, you're also modifying multiple OTHER tables at the same time and creating a situation where you will pretty seriously impact the log_mark_read table as well. It will grow fairly huge, in fact, if you're determined to use the board in the regular way.

While I see where you're going with this, there are some real concerns about doing this. We have a standing policy with respect to mod approvals, one of the terms is no significant performance hits. This WILL cause performance issues, and if anything your purging ideas on the tables will actually make it worse, not better, because you will have significant fragmentation of two of the biggest-hit tables in SMF.

This would be resolved through the Optimisation scheduled task, but then that blocks performance of the entire forum (while that operation runs, NOTHING else can).

Earlier I said we had concerns. The longer I look at this, the more I'm concerned this will have performance issues on any forum with more than 1k regular posts + arbitrary number of status updates, or 98% or more of the forums that would use this mod.

[ElectricSquid]

In all of the files I'm modifying to block the Mystatus board view, I am finding this in the WHERE...

Code Select Expand


<?php

AND b.id_board != {int:recycle_board_id}

?>


All I'm doing is adding...

Code Select Expand


<?php

AND b.id_board != {int:mystatus_board}

?>


So I'm not sure why that would be there to exclude the recycle board if it wasn't needed

[ElectricSquid]

If I'm not mistaken, I addressed the read/non-read issue in ver 1.0
I'll have to go back and look at that. I believe I addressed it by making it where ALL posts are automatically marked as always read, so there wouldn't be this huge unread impact.

[Arantor]

If posts are unread, there's no performance concern because you're not filling the table. If you are marking them read for all users, you're filling up log_mark_read possibly unnecessarily.

* Arantor hasn't examined the code on that part too deeply though.

[ElectricSquid]

If posts are unread, there's no performance concern because you're not filling the table. If you are marking them read for all users, you're filling up log_mark_read possibly unnecessarily.

* Arantor hasn't examined the code on that part too deeply though.

I might be mistaken on that. I took care of Notify, but after looking into the read/unread, I need to address that.
For this mod, there is no need for read/unread.

So, you say if posts are unread, that's fine?
Basically I would need to keep them from being log_mark_read, as far as you say.
That sounds easy.

Hmmm... how do I keep them from being marked read when they read them..... looking now.....



Oh, I'm on ver 1.3 now.

[Arantor]

Then you'd have to modify Display.php to not update that table, and also the markasread (think that's its name) action also to minimise folks marking the entire board as read.

And also should the admin ever change the board id, all this changes too.

[ElectricSquid]

OK, it's only marking them read when a user posts.
You can look at the posts all day, and it's not marked as read.

Views aren't logged either, only the number of replies, which I believe is just a count of replies.

So basically all I need to do is adress the marked read for MyStatus Board on post issue and I have the log_mark_read issue licked.

That's in Post.php, I'll be right back.....

Then you'd have to modify Display.php to not update that table, and also the markasread (think that's its name) action also to minimise folks marking the entire board as read.

And also should the admin ever change the board id, all this changes too.

I need to make make it where the board is created during install, that would limit the need to ever change the board number. That'll be tomorrows work

As far as Display.php, it's not used for this mod. I'm calling the data straight from the database tables in MyStatus.php, which is how I've slimmed down the amount that is called upon. Remember, I I needed was the topic_id, and the messages in it, nothing else (well, almost).

[ElectricSquid]

OK, got it done. Now even on post or reply, the topics are not marked as read.
Kiss that performance issue goodbye!!

Thanks Arantor, for the heads up on that one.
What else ya got?


Oh, I'll post the new package with this change in a few minutes once I package it.
There have been a lot of changes since ver. 1.0, so I hope the team looking this over catches that I've uploaded the newer versions (to the mod site) since they downloaded the first one.

Version 1.25 will be up in about 20 minutes

[Arantor]

You still haven't addressed my even greater concern: table fragmentation of the {db_prefix}messages table, and the fact that come optimisation, the entire forum will grind to a halt while the entire table is rebuilt.


Oh, by the way, you kind of missed my point with {query_see_board}. If you had modified where that was defined, you'd nail pretty much every place in one go.

[ElectricSquid]

You still haven't addressed my even greater concern: table fragmentation of the {db_prefix}messages table, and the fact that come optimisation, the entire forum will grind to a halt while the entire table is rebuilt.

I'm not sure what you mean by this. Why/how does a table get fragmented?

Oh, by the way, you kind of missed my point with {query_see_board}. If you had modified where that was defined, you'd nail pretty much every place in one go.

I have to search that one. I don't know where that is defined.



Sorry for the delay with the newest version,
I had an issue with the install package I needed to fix.
It's uploaded now.

[Arantor]

Same way a disk gets fragmented, and for the same reasons.

Imagine a table with 1000 rows in, the messages table. Let's say for the sake of argument that everything's the same size, 1000 characters.

Now let's say we delete item 1. That gives us a space at the start of the table of 1000 characters.

Next time we want to create a new record of 2000 characters. We have two choices how we do this, we can either split the row in half, and put half in that first space and half at the end. OR, we can put the entire thing at the end, and hope that some other record comes along to claim that 1000 character space.

Now, imagine you're doing this all the time with regular purging and regular new records. Over time this gets worse and worse in performance terms.

So what happens is, periodically, we optimise the table, restructuring it to be in a single cohesive list. BUT while we're doing that, the entire table is locked, meaning it can neither be read nor written to.

Now, I know that the average user will more likely make status updates than posts, as Facebook and Twitter have proven. Meaning this whole process is going to go on and on.

So when it comes for the process of optimising, you have many many small rows clogging up the process, meaning that during that time while it's mulching through that, NO user can use the forum (not even the admin).

On any sufficiently large forum this is likely to be a real performance killer over time.

[ElectricSquid]

Here's where we get into the need to limit the amount of messages that can be in this table.

Personally, I don't think anyone who would be posting to this mod would give two $#!+$ about a Status Update 3 weeks old unless it was their own and they didn't visit the site much.

In fact, if an average power user goes back more than a day or two on a busy site, I would be shocked.

So if we limit the life span or amount of Updates a user can have, that should limit this issue down to a serviceable amount.

The big question is, where do we set this limit?
Is there an existing process that can be used?

[Arantor]

No, there isn't an existing process for limiting user messages, other than preventing them being posted too quickly.


You're still missing where I'm getting at. You're still injecting smaller-than-average messages into a table not designed for it, meaning you'll be inserting these in between regular messages.

Over time this WILL cause degradation of performance. By limiting user capacity to post status updates (other than to 1, and storing it in the member's table), you only mitigate rather than resolve.

You also realise, of course, the ENTIRETY of the forum is being slowed down by adding status updates to the primary messages/topics table, and by adding extra data unnecessarily.


I'm sorry. I don't believe, and have never believed, this is the road you should go down. I personally believe you should have moved this data to a separate table, and even granted separate permissions for management, which would avoid all these other problems.

I am not going to continue arguing on this matter. I personally will not approve this mod, however I will not prevent anyone else on the Customization Team doing the same if they have no such concerns over it. The internal discussion has flagged up this topic and is being taken into account during the approval process.

[sangwe11]

IMHO, scrap this post idea and use your own tables !!!

Why find hundreds of solutions to make it work when there is a better alternative!!!

As of now, I wont be using this for the post reason, and for the *hideous* (IMHO) design.

[ElectricSquid]

You're still missing where I'm getting at. You're still injecting smaller-than-average messages into a table not designed for it, meaning you'll be inserting these in between regular messages.

The reason I'm missing this point is because every forum I'm a member of has a ton of these small messages all over the forum posted by daily post whores (no offense meant)
So what you are telling me here is that those forums are having this issue already?

Or if I understand correctly, those forums are fine because most of the posts are small anyway.
In that case, I don't see how this mod doing the same thing that is already being done is such an issue.

I am not going to continue arguing on this matter.

and I'm not arguing with you about this as you implied
I've taken most of what you said and did something about it.
I just don't get the fragmentation thing because of what I just posted above.

The internal discussion has flagged up this topic and is being taken into account during the approval process.

Good, I was hoping they would

[Arantor]

Are you regularly deleting those messages?

It wouldn't be quite so bad if you weren't.

But if they are posting them, you'll get double benefit keeping them OUT of the main message tables, because you won't get the same fragmentation across the primary tables (which means everything's kept faster), AND you can prune them at your leisure without hurting the main tables. Plus you can run your own prune/optimise schedule without having it affect the entire post stack, plus you won't get the additional pain of the search index being extended.

[ElectricSquid]

Ok, now I think I fully "get it".
LOL, I wish I would have fully gotten it before I got this far But that's water under the bridge.

So, on to creating the MyStatus table in the database.....