News:

Bored?  Looking to kill some time?  Want to chat with other SMF users?  Join us in IRC chat or Discord

Main Menu

Rethinking hooks - thoughts from other places

Started by Arantor, August 18, 2019, 02:12:03 PM

Previous topic - Next topic

Arantor

I'll be honest, I have never been *entirely* comfortable with SMF's idea of hooks, even though I've been championing them since basically 2.0 RC4 when they got upgraded and expanded. (And the implementation was nicer than my first draft inside SimpleDesk.)

And since I'm at a point where StoryBB has established its direction in my head about things I want to do and some of the stuff I want to be able to manage going forward, it's let me to rethink the position of hooks in the universe and what I plan on doing - the reason I'm posting here is so whatever SMF 2.2 or 3.0 or whatever does, it can at least have the results of what I did both past and present and what I learned from it. All of what follows stems from what SMF did and I want to open the discussion with it. None of this should be taken as 'SMF should do this' but at most it should be 'these are things future SMF could do if it made sense'.

First up, Wedge hooks. Wedge hooks were mostly the same as SMF 2.0 hooks but with some additional functionality:

1. Hook types
Firstly there were two kinds of hooks - functions and language. (Technically there were 3 but we'll come back to that.)

This meant that you could hook a function onto whatever was going on, like in SMF 2.0/2.1, but you could also just merely say 'load a language file at this point'. The main use I remember for it was the Help, if you wanted to expand something for the popup help you needed to load a new file. Now, you *can* do this in 2.1 (not 100% sure about 2.0) but you have to add a regular hook whose contents were basically a loadLanguage call. Wedge bypassed this to just be able to load the language directly.

2. Priorities
Hooks in SMF occur in a somewhat arbitrary order, namely whichever order they had been installed, which normally would be fine but sometimes there were hooks that had to go out of their way to remain last (the old SimpleSEF mod comes to mind, for legitimate technical reasons). Wedge hooks got a priority when installed, from 1 to 100 (lower = earlier), defaulting to 50, so plugins could set up their hooks in any setup they wanted including knowing about other plugins and making sure that they reordered themselves appropriately.

3. Safety
In SMF, when hooks are registered by a plugin, this is done forcibly in the database at install time. While performant and convenient, this has some issues if a mod gets removed especially if cleanup wasn't properly carried out for whatever reason or if the plugin is buggy in some fashion. What Wedge did was to defer attaching hooks until just after the list was loaded from the database when settings were loaded - it would then look up which plugins were marked as installed, looked up the hooks they declared and attached them for that one run.

While this is slower, the fact that it is done this way means you can rename a plugin folder and the plugin will effectively become disabled - you don't need to add any additional files, or change the DB, it's handled automatically. Rename the folder, delete the folder, plugin hooks all disengage. (Or at least did when I was still a dev, the latter changes that pseudocached file edits too... not so much)

4. Hook registration
Before installing a plugin, the list of hooks that a plugin uses are checked against a list of known hooks before trying to install - limiting the amount of need to do explicit version checking if you can do feature detection. But additionally, a plugin could declare its own hooks and these get added to the list of hooks the system knows about before trying to install a plugin. It was a neat idea but the manual list of hooks was a maintenance trouble since every time a new hook was made, we had to remember to update the master list of hooks.


The above mostly made a lot of sense and was generally fairly serviceable - the worst mistake I made was writing the plugin was continuing to use XML (only compounding it with SimpleXML). The other hurdle is that there's no convenient way to hack up the templates, Nao 'fixed' that by allowing plugins to make editing again, but that's something I'd never encourage.

So what the hell did I do in StoryBB that was different? And why? What benefits? Well, I took the principles behind what I'd done in Wedge - priorities were always a good idea, and so is the safety stuff. I haven't rethought what I'm doing with the language problem yet.

But what I did differently this time was build hooks as classes. Every hook is a class, and it either extends a class called Observable or a class called Mutatable - partly so it's super clear whether the hook is read-only or read-write, partly so that it's trivial to detect if a hook is present (class_exists on the hook's name and you're done) and if you wire up something like phpDocumentor, it's even documented automatically as to what parameters it accepts and the class comments can tell you where it's used.

The only difference really is that for most cases what used to be something like:

call_integration_hook('integrate_after_create_post', array($msgOptions, $topicOptions, $posterOptions, $message_columns, $message_parameters));

now becomes something like:
(new \StoryBB\Hook\Observable\Post\Created($msgOptions, $topicOptions, $posterOptions, $message_columns, $message_parameters))->execute();

(I like namespaces. In the current code Observable is an alias so in reality the call becomes new Observable\Post\Created. Either way, you pass all the parameters into the object which is an instance of Observable or Mutatable, which both extend a base class called Hookable, which handles some other stuff. The hook object then deals with giving you access to the parameters either way.)

The called functions are always any callable, though in practice this is static methods in autoloadable classes - I see 2.1 has added the # parameter to call_integration_hook which means you get an instance of the class but this mostly seems to me to be about a different type of syntactic sugar rather than anything externally changeable, you just end up with using static:: or self:: instead of $this-> which is a change I don't really get and I've never really sat down and looked at whether call_helper would end up preserving the one instance of the class or making new ones, which is potentially expensive depending on what's going on.

But when I started looking at the various hooks in place, I began to realise that there are really 3 broad types used throughout SMF, and I'm thinking about how I want to fix that because I find it inelegant and I'm at a place where I'm free to tear it all apart.

Specifically, as I see it in SMF, you have hooks that deal with routing (integrate_actions, integrate_XMLhttpMain_subActions for example), hooks that deal with what amounts to sideloading API-able functionality (integrate_autosuggest, things like loadCacheAPIs and loadPaymentGateways are contenders for upgrades to this style of thinking) and hooks that change behaviour either in local or global states (everything else).

The first one essentially is a question about gutting the entire routing system - right now it's two tiered, between top level actions and everything else, where some actions need no additional routing (e.g. the pseudo-action that is board index), some need very minimal and light routing (the aforementioned XMLhttpMain) and some need hugely complex routing (profile, admin).

Exactly what I think could/should happen is a little bit murky but ultimately it revolves around having a routing table that delineates all the known overarching routes, specifying parameters that make up the route itself - e.g. /board/myboard.1/?sortby=author has the board/myboard.1 part as the route itself with the sortby as an additional option that isn't really part of the route if that makes sense. (This also encourages pretty URLs by default, not because it boosts SEO but because it simplifies routing in practice)

Part of me wants to go to what XenForo does, where a route specifies a class and subroutes specify methods in that class, which can be overloaded by way of their class proxy system, which is very neat and generally not *that* hard to get your head around for building extensions to existing things. It also pretty exclusively means you don't have to have hooks everywhere just to bolster routing, you can let the routing component do that and just have plugins say 'I want this route to go to this class/method'.

Then we have the hooks that exist to offer functionality choices for things - this is actually really nice to solve, you can do things like make the autosuggest handler look for any class that implements a specific interface, and make those the known autosuggest options. The only time you'd then want a hook is if you wanted to completely replace a given autosuggest handler for any reason (which is, interestingly enough, surprisingly rare). Ditto for payment gateways, or components for the CAPTCHA system, make it all essentially pluggable and give it the power to look stuff up in the code for classes that implement the right interfaces. The goal here is to have as little time and energy spent on writing connectors between things when there's no reason the system can't do that for you. It also forces a more elegant separation of concerns especially for something like the CAPTCHA system.

Lastly the 'everything else' hooks, these all pretty much have to stay as-is, but that's OK, they're either being event emitters (Observables) or allowing you to modify state in-flight (Mutatables), and these are perfectly understandable, just it would be nice to get away a bit from using hooks as all-purpose tools when they really shouldn't be.

Phew. And please before any angry replies come in, this isn't meant to be a lecture on what SMF should do or could do. This is merely a collection of thoughts based on what I've seen and done and where my started-out-as-SMF setup is currently looking to go. If nothing else, the hope is merely to inspire the question of 'is there a better way' based on what I'm experimenting with, and to generate some discussion about how we can do things going forward in the ecosystem.

Aleksi "Lex" Kilpinen

I'm not really familiar with the code behind hooks, and not really all too experienced in even using them, but I see a lot of good food for thought in this. So, let me be the first to say thank you for sharing.
Slava
Ukraini!


"Before you allow people access to your forum, especially in an administrative position, you must be aware that that person can seriously damage your forum. Therefore, you should only allow people that you trust, implicitly, to have such access." -Douglas

How you can help SMF

lurkalot

Quote from: Aleksi "Lex" Kilpinen on August 18, 2019, 11:47:52 PM
I'm not really familiar with the code behind hooks, and not really all too experienced in even using them, but I see a lot of good food for thought in this. So, let me be the first to say thank you for sharing.

Me neither.  I can't get my head around them at all, I have tried, but fear I'm too old to take it in.  ???

I would however like to know if there's a list of Hooks 2.1 vs 2.0.15 for comparison anywhere.  I have a mod that uses just hooks in 2.1 and would like to do the same for 2.0.15

Arantor

There is no such list, bu doing a search on "call_integration_hook" in the two would be a useful starting point. 2.1 has many many more hooks but you can usually improvise with a bit of creativity.

SychO

https://gist.github.com/SychO9/e534681410c546a34c2daba16dff3831

Thanks for sharing Arantor,

Do you mean to create a class for every hook though? Wouldn't that be way too much ?
Checkout My Themes:
-

Potato  •  Ackerman  •  SunRise  •  NightBreeze

Arantor

Firstly, no, not really because autoloading is a thing in my world (as is opcache, and especially the new stuff in PHP 7.4)

Secondly, a number of those hooks will be going away because of better ways of doing it, e.g. integrate_actions will be replaced by a global routing tables (which also fixes several of the subactions that need routing too)

Thirdly, a number of those hooks will also go away by being replaced by API patterns, where plugins don't need to explicitly deal with hooks because the plugin manager deals with the classes that need to be connected in other ways, such as auto suggest (when it sees a plugin, it looks at the classes it provides, and looks at whether any of those match interfaces it knows to look for, e.g. PaymentProvider or the cache API, and connects them up removing the hooks)

d3vcho

This is indeed a very interesting topic Arantor, and before going further, I would like to comment this:

Quote from: Arantor on August 18, 2019, 02:12:03 PM
Phew. And please before any angry replies come in, this isn't meant to be a lecture on what SMF should do or could do. This is merely a collection of thoughts based on what I've seen and done and where my started-out-as-SMF setup is currently looking to go. If nothing else, the hope is merely to inspire the question of 'is there a better way' based on what I'm experimenting with, and to generate some discussion about how we can do things going forward in the ecosystem.

Your opinions and thoughts are very valued among SMF, you know that already. Angry replies should not come basically because we're not perfect and as time passes by, new ways of doing things appear. I mean it when I saw it's really fascinant because in college I'm being taught rather old stuff that's indeed the base for everything else, but we're always looking for the new ways in what we can solve problems.

I do know I'm still a newbie of programming, in general, and even more if we're talking about SMF's codebase. I've never understood hooks itself. I know what do they do and why they're good, but never completely understood the logic behind them. I've been playing with SMF 2.1 recently, and with that huge amount of hooks I was like: "Woah, almost no code edits needed. This is really good for the core itself". But then someone with high knowledge of programming and SMF's codebase comes and tells you that everything is good so far, but we can make it even better using more modern technologies and solutions.

Now, a few questions from my side. First of all, I don't understand the priorities thing. Can you put an example on why is it important to implement? I just can't visualize it in my mind.

Another thing. Why it would be better to have an instance for every hook created than a mere hook as we know them? What about the classes itself? Will the codebase become simpler if we make a class for every hook and an instance for every hook we need?
"Greeting Death as an old friend, they departed this life as equals"

Arantor

Ok, priorities. You have SimpleColorizer and SimpleSEF installed. Both of these use integrate_buffer but if the hook for SinpleSEF runs first, SimpleColorizer can't find the links it needs because SimpleSEF already rewrote them.

If you happened to install SimpleSEF first then SimpleColorizer, the latter wouldn't work without some kind of help. SimpleSEF itself knew about this and went to some effort to make sure that it was always the last hook called for integrate_buffer specifically so it wouldn't break anything else.

Having a priority at setup time explicitly enables you to solve that problem by just letting a given plugin operate last without having to manually check and reset it regularly.

The use of classes does, for me, make it simpler. Partially it means that you can have a very clear and distinctive way of documenting what the hooks are, what parameters they accept and what hooks exist, just by having them be explicit classes. And if you're set up to autogenerate docs with phpDocumentor, this is even more clear.

As a bonus you can then use IDE typehinting in all the relevant places because you're passing around an actual class in all the cases.

SychO

Sweet, once you start making use of the OOP concepts and what Php7 has to offer, much and more can be taken to a different level.

Too bad I can't offer more to the discussion, I am still learning about all of that.
Checkout My Themes:
-

Potato  •  Ackerman  •  SunRise  •  NightBreeze

d3vcho

And how would you deal with different instances of the same classes for the particular case of hooks?
"Greeting Death as an old friend, they departed this life as equals"

Arantor

Quote from: SychO on August 19, 2019, 08:34:24 AM
Sweet, once you start making use of the OOP concepts and what Php7 has to offer, much and more can be taken to a different level.

Too bad I can't offer more to the discussion, I am still learning about all of that.

That was what I was hoping to do, stimulate discussion on the subject :)

Quote from: d3vcho(); on August 19, 2019, 08:35:15 AM
And how would you deal with different instances of the same classes for the particular case of hooks?

You instance the class once and pass that object to each hook in turn, giving you much the same functionality you have today (since the values are passed to each in turn, modified in place if they want to modify it).

d3vcho

Sounds great actually. So you're planning to implement this in StoryBB right? No drafts yet?
"Greeting Death as an old friend, they departed this life as equals"

Arantor

Some of this is already done (the actual hook implementation is done), I'm just in the process of retooling the plugin manager from Wedge to use the new hook stuff.  Check the plugin-core branch if you're curious about the hook implementation, there's a couple of examples there.

albertlast

Could you not convert the existing system without change anything on hookside and on addon side?
So only in the middle part between them.

Arantor

Quote from: albertlast on August 19, 2019, 11:15:07 AM
Could you not convert the existing system without change anything on hookside and on addon side?
So only in the middle part between them.

I could but that would mean I get the worst of both worlds. Remember, the first half of this post talks about Wedge where I was not nearly so radical and I decided I didn't like the way it worked, plus it didn't do anything nice for IDEs.

rocknroller

Doesn't should be? As base concept., backend - API -frontend.

Maybe:
PHP MYSQL / API / so FE could be at least available for more programming languages.

Good luck.

SpacePhoenix

Quote from: SychO on August 19, 2019, 08:34:24 AM
Sweet, once you start making use of the OOP concepts and what Php7 has to offer, much and more can be taken to a different level.

Too bad I can't offer more to the discussion, I am still learning about all of that.

I just used Notepad++ to do a quick search for "function __construct(" and it seems like the only OOP code is 3rd party code. Why hasn't OOP been used for 2.1?

vbgamer45

I am glad SMF 2.1 is not OOP. One of the reasons why I stick with it. A mixture is fine like what MyBB does. But when I look at code like Xenforo it makes it unworkable for me. I dislike having to trace each and every class to figure out what is going on. And it raises the barrier of entry of users who are are looking to modify the platform.
Community Suite for SMF - Take your forum to the next level built for SMF, Gallery,Store,Classifieds,Downloads,more!

SMFHacks.com -  Paid Modifications for SMF

Mods:
EzPortal - Portal System for SMF
SMF Gallery Pro
SMF Store SMF Classifieds Ad Seller Pro

SpacePhoenix

Roughly what % of the backend was changed between 2.0.15 and the current RC of 2.1?

Arantor

Not nearly as much as it might seem, actually.

It's hard to give a figure but I can't imagine it would top 25%.

Advertisement: