Topic: Ignore Topics/Mark as Read all the time/Unsubscribe topic

[Adish - (F.L.A.M.E.R)]

Well, I suppose something officially for big boards can be done rather than using the hack every now and then ?

btw, I asked in to write in their input from one of their request. It is just something demanding and so I thought of letting user's put in their vote. Not a fault of the user.

[Arantor]

The problem is performance.

There are two basic ways you can do it, one is the way vbgamer/Karl's mod does it, and one is the way I do it. The way vbgamer/Karl's mod does it is clean, but due to the way it all ends up in SQL, woefully painful with large numbers of topics being ignored, and equally painful on large boards.

My way is by my own admission, inelegant and ugly, but has a much lower performance penalty.

I'd point out that this has been requested by the team for a year and hasn't happened yet, I don't see any official nod prior to 2.1, and even then I don't see this being implemented for a bit because there are no really good ways of doing it.

[Adish - (F.L.A.M.E.R)]

I get your point dude, even if it in waiting, it is still worth having the user's post in their vote. This can gain some level of importance if it is in quite alot of demand. While it is already in discussion privately, maybe some other user might be able to kick in their ways to make it work better without any performance penalties. The more the interest of the users, the more it will be a priority and maybe someone (not necessarily the team) can make up a quite decent fix up which is not ugly and inelegant..

[Arantor]

*sigh* There are only two basic ways of doing it, elegant and performance hurting, or inelegant and fast. These are not some random rules, they are certainties within the environment of a relational database like MySQL.

A user vote is somewhat pointless because we all KNOW it should be implemented but the two methods proposed have significant drawbacks and no way around them.

[kat]

The performance hit. Just how bad would it be, in reality?

[Arantor]

Colossal, actually, and scales up at *best* directly in proportion to the size of the forum, worst case exponentially to the size of the forum. Let me explain the problem.

SQL works on relationships, joining tables together. As such, you build indexes, a list of what items are in a given table. And when you join, you look up against that table.

Which means, when you start from the topics table and link to the messages table, you've got a map as to where things are.

Now, this is where the butthurt begins. You build a list of the items you have, not a list of items you don't have.

So you build a list of topics and have to compare each one to the ignore list, looking for it not to exist. And since you don't have a list of what doesn't exist, every single topic has to be matched to the ignore list, one by one. IIRC the original version of Karl's mod limited you to 10 (yes, 10 TOTAL) topics to be ignored, simply because that limits how far the ignore list gets, which limits how much time is spent looking for non-matches.

If it wouldn't eat your disk insanely, it'd probably be quicker to build a list of every topic and every user and match against that.

Thing is, we already sort of have that list - in the form of the unread topics list, which my hack - well - hijacks in a not so good way.

[kat]

Would this affect everyone who uses the forum, Pete, or just those who use the feature, in some way?

[Arantor]

Hmmm, interesting question. If you write the queries carefully enough you could I suppose limit it slightly differently... you know, if you do it carefully enough you limit the scope to whoever you give permission to.

Here, I expect it would be given to those who've been here long enough to warrant it (so, 1k post count groups and up), which would greatly cut it back, but it would still be a significant hit on the rest of folks who do have it enabled.

Hmm, I do have another idea, not sure it'll work out, and I need to benchmark it to prove or disprove whether its performance would be better or not, it's still basically the same methodology as vbgamer/Karl's mod though but with a twist that, if I'm right, will cut a lot of the performance hit - but may require MySQL 5 and disabling a security feature (ooo-er!)

[kat]

Without wishing to come-over as elitist, or anything, what about if it was restricted to, say, SMF team members?

[Arantor]

The more you limit the scope of it, the better it gets in the scheme of things - the fewer people that have permission, the fewer queries you're doing where there's butt-hurt.

That said, remember that team members are likely to be among the more active members of the community and so it's more likely to be loading the server.

[kat]

I would guess that it's us that would use the "All unread posts" feature.


That's where all the alien stuff muddies the water and we miss stuff.

[Arantor]

*nods* That's where it would *really* grind, bogging down one of the already heaviest processes with something moreso :S

The hack I published does the job, hackishly, but without any of the aforementioned performance penalties because it's just using a mechanism that's already there.

[kat]

Just out of pure curiosity, how difficult would it be to make team use of "Show unread posts since last visit" totally, completely, separate from other member's?


Impossible, I'd assume, as it'd all be using the same database...

[Arantor]

Pretty much impossible. The only alternative is to sync all the data to another database, which would likely be as punishing as not doing it in the first place.

[kat]

Poo.

* Kat wonders how much money Pete could make, if he invented a way of doing this, without the perofrmance hit, or hacking...

[Arantor]

If I could solve this problem, I'd be turning conventional relational databases on their head, because ultimately that's where the problem lies - databases are designed to structure data logically, and map the logical joins between them, i.e. topics -> messages. They're not designed for mapping joins where the join doesn't exist.

As I said I've got an idea that might *limit* the pain, cleanly, but it won't be as efficient as the hackish method I already proposed. I don't see how you can possibly do it any more efficiently, given that it doesn't add any tables, any rows or change any queries that are already present and optimised.

[kat]

If I could solve this problem, I'd be turning conventional relational databases on their head

Off you go, then.


I, hereby, volunteer to be a beta tester.

[Arantor]

I'm good, but I'm not good enough to turn 40+ years of engineering and logic on their head. Not this year, anyway

I'm gonna unbreak SimpleDesk after all the changes I made today, then gonna go play with doing something about this, I have an idea, will experiment and see if hurts any less than the current plan...

[kat]

Be done by this evening, then!

[Adish - (F.L.A.M.E.R)]

I love the way this topic sticks some new ideas in the head..

* (F.L.A.M.E.R) smells lots of sarcasm in the air!