[WIP Docs] Integration Hooks and You [2.0 RC3]

[Arantor]

People hear the phrase "integration hook" and figure it's something SMF can do to bridge to third party code, and they're right.

Sadly it's one of the more subtle and underused features, because actually it's kinda cool. I warn you though, this is one of the more powerful features of SMF and I don't expect this to be a particularly simple discussion - because it's not a simple thing it's doing.

THIS TOPIC IS NOT FOR SUPPORT. IF YOU NEED HELP, OPEN A NEW TOPIC IN CODING DISCUSSION.


Most modders will be familiar with the idea of $modSettings, a store of various variables used by SMF. Well, the integration hooks as far as the modder/developer is concerned are just more entries in $modSettings, meaning you can reference them happily in the settings table, without touching the core code.

Why is that so important? Well, let me give you a specific example of the benefits. SlammedDime's SimpleSEF mod makes several edits to SMF's code - but if you look at the parsing instructions, they are limited to Admin.php and ManageSettings.php (and language files).

Take another look at the parse instructions for the mod... it's only adding admin options, so how does the rest of it work?!

That's where integration hooks come in. SlammedDime calls the integration hooks to load the main SimpleSEF file and to make it work - and so it doesn't have to modify the core code, which if used carefully simplifies a lot of things for larger or more intricate mods! It's certainly a useful tool in the arsenal of modding and definitely if you plan to integrate with another system.


So, what are the hooks and how do I use them? Well, I'll document each of the hooks first, what it does, what it expects from you, then I'll cover how broadly to use them specifically for you. I'm using the names below, e.g. integrate_pre_include. This would be referenced in $modSettings as $modSettings['integrate_pre_include'], and managed in the settings table under that name too.

Under each int hook I'm detailing in the following manner:
* Called from: - lists which file it's called from, and at what point the hook will be called on your behalf.

* Purpose: - what the hook is designed for and what it can allow you to do.

* Accepts: - what type of input the hook is expecting. In almost every case this will simply be the name of a function to call, though can be a static method of a class. It should be obvious that the function will need to already be loaded and thus available to SMF already - if not, you may need to ensure integrate_pre_include loads it, the main SMF files contain it, or you otherwise ensure it is loaded.

* Sends: - the list of parameters in order that the hook will be sent from SMF. May be no variables, may be many. If the hook sends variables that you may want to modify before your function ends, make sure your function states it wants the variables by reference.

For example if a hook states that it sends $value1, $value2 and you want to be able to modify those in your function myfunc(), the specification would be:

Code Select Expand

function myfunc(&$value1, &$value2)


General hooks
integrate_output_error
Called from: Errors.php, just after the error itself has been logged but before it has been output to the user.

Purpose: To allow reformatting of the error message, prior to notification to user, or perhaps to log or process in an outside system (e.g. sending an email to administrators on certain types of error occurring)

Accepts: 1 function name.

Sends: $message, $error_type, $error_level, $file, $line

$message is the message text itself.
$error_type is the SMF category of an error, e.g. General (general), undefined variables/indexes (undefined_vars), user errors like forgotten passwords (user), database/query errors (database).
$error_level is the PHP error level ident. See the PHP manual for the values this is.
$file is the filename this occurred in (full server path).
$line is the line number of the error.


integrate_pre_include
Called from: Load.php, just after $modSettings has been loaded, which is very, very early on in SMF processing.

Purpose: Allows you to load a single file of PHP source, of your own code. Can be used for other integration functions, as it will be loaded on every single page load.

Accepts: 1 filename. The string should contain a path to the file specified with $boarddir, e.g. a file called MyFile.php in Sources would normally be $boarddir/Sources/MyFile.php. Note that the function checks the file exists before it attempts to load it - silently aborting if not found.

Sends: No parameters, just looks for the file and if found, loads it.


integrate_pre_load
Called from: Load.php, just after $modSettings has been loaded, which is very, very early on in SMF processing - immediately after integrate_pre_include is checked.

Purpose: Allows you to call one or more functions before general SMF processing begins. These can be any defined functions, though it is entirely possible that they are defined in the file loaded by integrate_pre_include.

Accepts: 1 function name.

Sends: No variables, just calls the function - after verifying the function exists.


integrate_whos_online
Called from: Who.php, if index.php was the loaded page of the user but it's because something else with integration is happening.

Purpose: To be able to add listed actions to Who's Online from integrated apps.

Accepts: 1 function name.

Sends: $actions array from Who.php which will basically be the components of the URL, e.g. index.php?action=page;sa=something results in $actions being array('action' => 'page', 'sa' => 'something') which the integrated function can use to identify what the action really is.


integrate_egg_nog
Called from: ViewQuery.php, before the query log (full page) is built.

Purpose: To be able to add any pre-processing to the query log that may be necessary prior to outputting. Will be of limited use.

Accepts: 1 function name.

Sends: No variables, just calls the function(s) - after verifying the function exists.


integrate_load_theme
Called from: Load.php, at the end of loadTheme(), to load any information that potentially affects top level loading of the theme itself.

Purpose: Designed to modify the layers to be loaded during page generation, for example to avoid calling the 'html' layer if the page is part of a CMS output (where there will already be a similar layer). Orstio expands on it a little more, covering off my initial theory that it was for exporting data to a CMS, rather than its actual use of pulling from.

Accepts: 1 function name.

Sends: No variables, just calls the function - after verifying the function exists.



Log in/log out
integrate_verify_user
Called from: Load.php, loadUserSettings(), as the first option.

Purpose: To attempt to log in from external app first, by reusing login credentials; this is taken in preference to either cookie or session details.

Accepts: 1 function name.

Sends: No variables, just calls the function - after verifying the function exists.

Unlike other functions, this hook explicitly requires a return value - few other hook functions handle return values. The return value is cast to an integer to represent user id; 0 = not logged in and to refer to cookie then session, otherwise it represents the user id.


integrate_verify_password
Called from: multiple places
Profile.php, ModifyProfile(), to ensure password is valid before modifying profile details that require password confirmation.
Security.php, validateSession() twice - once, to ensure password is valid during a regular check, and once to ensure adminstrator's password is correct (i.e. even though we were logged in, we need to reauthenticate e.g. admin panel)

Purpose: To run any further tests to validate the user supplied password.

Accepts: 1 function name.

Sends: two variables, representing user name and current unhashed password.

Unlike other functions, this hook explicitly requires a return value - not all other hook functions handle return values. The return value is either exactly identical to true if acceptable, or anything else is a fail.


integrate_validate_login
Called from: LogInOut.php, Login2(), just before actually calling for the user's record in the database.

Purpose: To validate credentials in an external data source as well as with SMF.

Accepts: 1 function name.

Sends: three variables: string of user's username, string of password (hashed) or null if not supplied, length of cookie lifetime in minutes

Unlike other functions, this hook explicitly requires a return value - not all other hook functions handle return values. The return value is a string - either "retry" if there was an issue and user needs to try again, or anything else if the hooked function does not have an issue with the supplied details.


integrate_login
Called from:
LogInOut.php, DoLogIn(), almost first instruction in the function - used during user actively logged in (session exists, more validating that than anything else)
Register.php, Register2(), once registration is completed, just before setting up user cookie - so registration will log them in to both SMF and integrated app

Purpose: To log user in on both integrated code and in SMF at the same time.

Accepts: 1 function name.

Sends: three variables: string of user's username, string of password (hashed) or null if not supplied, length of cookie lifetime in minutes


integrate_logout
Called from: LogInOut.php, LogOut(), if the user is not a guest, after unsetting some session variables, but before clearing their entry in {db_prefix}log_online. They are, technically, still listed as online when the hook is called.

Purpose: To initiate logout in integrated code.

Accepts: 1 function name.

Sends: string representing user's name


integrate_reset_pass
Called from: Lots of places
Profile-Modify.php, authentication(), just before a new password is hashed and about to be inserted into the database (user modifying own password, I think)
Profile.php, ModifyProfile(), for admin modifying another user's password (I think)
Reminder.php, setPassword2(), for when a user resets their password from reminder email
Reminder.php, SecretAnswer2(), for when a user successfully posts their secret answer
Subs-Auth.php, resetPassword(), for when user has forgotten their password and a new one is emailed to them

Purpose: For notifying external code when a user's password is modified outside of registration.

Accepts: 1 function name.

Sends: $old_user, $user, $newPassword

$old_user - frequently the same as $user, in some cases it can be an unsanitised version of the username
$user - the user name
$newPassword - the newly set password, from whatever source



Registration, and other user hooks
integrate_register
Called from: Subs-Members.php, just before a user is physically added to the members table. (Same route runs whether it is a conventional user signup or from the admin panel) Validation on details has been carried out by this time.

Purpose: Could be used to do further validation of a user, e.g. an external lookup service or other system that the user accounts are linked to. Alternatively could be used to export user details to an external app and sign them up at the same time.

Accepts: 1 function name.

Sends: $regOptions, $theme_vars

$regOptions is a complex array variable. While the full list is in Subs-Members, the most pertinent for most cases are:
$regOptions['username'] = username used to sign up
$regOptions['password'] = password supplied (unhashed!)
$regOptions['email'] = email address supplied on signup
$regOptions['require'] = whether the account is immediate registration ('normal'), email activation ('activation'), COPPA form ('coppa') or admin approval (anything else, notionally 'admin')
$regOptions['memberGroup'] = primary membergroup id to assign (not validated)


integrate_activate
Called from: Multiple places.
ManageMembers.php - during admin approval of members, immediately after approving a member
Profile-Actions.php - activating an account through the profile area
Register.php - activating an account from member activation, immediately after sending the message and immediately before actioning it.

Purpose: To ensure a member is activated in a separate system when activated in SMF itself.

Accepts: 1 function name.

Sends: string representing the member's name (since that is guaranteed to be consistent, member id is not)


integrate_delete_member
Called from: Subs-Members.php, deleteMembers(), immediately before logging that the action has been done (which is before the SMF tables are all updated)

Purpose: To ensure that the external app is updated when a user's account is removed.

Accepts: 1 function name.

Sends: Number representing user id being deleted from SMF tables.


integrate_change_member_data
Called from: Subs.php, updateMemberData(), immediately before processing to add to SMF's own tables.

Purpose: To ensure data is migrated to an external system as well as updating SMF's own tables.

Accepts: 1 function name.

Sends: $member_names, $var, $data[$var]

$member_names - an array of the names of members
$var - name of the variable being updated
$data[$var] - the new value

NOTE: Not all values are exported through this hook. Only the following will be exported - any other values updated will not.
'member_name' - user name
'real_name' - display name
'email_address' - email address
'id_group' - primary user group (only)
'gender' - male/female
'birthdate' - date of birth
'website_title' - name of website specified in profile
'website_url' - address of website specified in profile
'location' - the location stated in profile
'hide_email' - whether the user's email address should be considered private (admin visible only)
'time_format' - the user's time format string for their own time format
'time_offset' - the user's own time offset (from profile)
'avatar' - gallery or URL specified avatar
'lngfile' - user's language



Posting and Personal Messages
integrate_outgoing_email
Called from: Subs-Post.php, once an email has been assembled, just before it is inserted into the master email queue.

Purpose: Allows you to process or otherwise do something with any email before it is actually sent out.

Accepts: 1 function name.

Sends: $subject, $message, $headers - listing the email's subject, its main message and the headers (which includes who it's to, who it's from, date, return path and other things)

integrate_personal_message
Called from: Subs-Post.php, sendpm() once the post is sanitised.

Purpose: To allow you to send or otherwise preprocess personal messages, you could export them to a third party CMS or similar.

Accepts: 1 function name.

Sends: $recipients, $from['username'], $subject, $message

$recipients - an array containing potentially two elements, 'to' and 'bcc', each of which will be an array of membernames that the message is going to.
$from['username'] - the username sending the message
$subject - PM's subject
$message - message body


integrate_create_topic
Called from: Subs-Post.php, if a post creates a new topic, once all other SMF processing has been done (like stats updates)

Purpose: Allows you to add topics to a CMS once they are posted. It would likely be better for something such as the Twitter mod to use this as a point to call, rather than modifying the code itself.

Accepts: 1 function name.

Sends: $msgOptions, $topicOptions, $posterOptions - the same three variables used to actually create a topic (see the Function Database for createPost for more), once the changes have been applied, so $topicOptions['id'] is the topic's id, $msgOptions['id'] is the id of its message, for example.



Internal functions
integrate_fix_url
Called from: News.php, during fix_possible_url().

Purpose: To allow the RSS feeds to have adjustments made for queryless style URLs - and any other URL changes that may be required.

Accepts: 1 function name.

Sends: $val, the original URL

Unlike other functions, this hook explicitly requires a return value - almost no other hook function handles return values. The return value is the modified URL.


integrate_redirect
Called from: Subs.php, during redirectexit(), immediately prior to physically issuing redirect headers.

Purpose: To allow code to modify the destination or duration before redirect, potentially of any redirect issued in the forum.

Accepts: 1 function name.

Sends: $setLocation, $refresh

$setLocation - the final URL it should redirect to, as a full URL.
$refresh - boolean, whether to send a Refresh header, or more preferably a Location header.


integrate_buffer
Called from: Subs.php, during obExit, used to add functions to be run on the content prior to it being sent to the user, in the spirit of last-minute widespread content changes.

Purpose: To allow you to modify the content globally before sending to the user, for example SimpleSEF uses this hook to actually replace links.

Accepts: 1+ function names. This function expects a single item or comma separated list, e.g. 'function' or 'function1,function2', the functions will be run in the order specified.

Sends: $buffer, a string representing the page as a whole.

Unlike other functions, this hook explicitly requires a return value - not all other hook functions handle return values. The return value is the buffer contents.


integrate_exit
Called from: Subs.php, during obExit, used when finally leaving the theme system either in the event of normal execution or fatal errors (i.e. any time a full page is generated that isn't a redirect/attachment)

Purpose: To allow you to do final shutdown on external apps, close connections, free resources. Can also be used with portals or other output handling functions.

Accepts: 1 function name.

Sends: boolean value whether the footer should be output (i.e. footer should be done and not in wireless template)


integrate_magic_quotes
This one isn't actually a function, it's a setting that's relevant. PHP's idea of magic quoting is one of the most irritating ideas it has ever had, to attempt to automatically be able to quote values that are incoming from userland. Unfortunately its behaviour is somewhat unpredictable and therefore unreliable.

In the process of sanitising $_GET, SMF does its own clean-up of magic quotes, handling the various types that may be incoming, but other systems do not do this and rely on default PHP behaviour. This setting overrides SMF's behaviour and returns it to the default.





Right, so that's the hooks. How do you get values into the settings table? How do you fill in the details for the above?

Actually, that's fairly easy. If it's a one-off thing, use phpMyAdmin and just add a new row to the smf_settings (or {db_prefix}settings) table. I'm not going to get into the mechanics of that because generally if you're at a level where you understand how these can be used, you probably understand the mechanics of that part too.

If it's for a mod, you would generally do this in an installation script that runs during mod install - like SimpleSEF does. Just be careful not to overwrite any settings already there.

If you need help on any of the above, please do create a new topic rather than replying here; this isn't meant to be a support thread.

Happy integrating!


ONCE AGAIN, THIS TOPIC IS NOT FOR SUPPORT. IF YOU NEED HELP, OPEN A NEW TOPIC IN CODING DISCUSSION.


EDIT: Added notes from Orstio.
EDIT 2: Added PM function.
EDIT 3: Made corrections as identified by SlammedDime.

[Nick Whetstone]

Awesome job. Thanks, Arantor. It'll definitely help me out a lot, the details of which I told you elsewhere, heh.

[KensonPlays]

Thanks! I don't have time to read it now, I will over time tho!

[~DS~]

Thanks! I don't have time to read it now, I will over time tho!

Wow, that's rude, if you don't have time to read, DON'T post.

[KensonPlays]

Didn't mean to be so, it's just 10:30PM my time.

[Orstio]

You've got a few errors there.

For example:

integrate_egg_nog is nothing but an Easter Egg (thus the quirky name, and lack of any other documentation). 

integrate_load_theme can't load anything useful from outside of SMF because of its location in the workflow.  It optionally ignores SMF's algorithm for determining which layers to add to the array, and adds only those layers determined in the integration function.  For example, in a default SMF page load, there are two layers:  html and body.  If you integrate your forum into your website, you don't want the html layer, otherwise you end up with double <head> tags and broken javascript.  integrate_load_theme was implemented to control the layers on all themes automatically without having to modify the theme.  This eliminates the reason for GooseMoose's tag cleanup mod, for example.

integrate_pre_load is completely unrelated to integrate_pre_include.  Let's say you wanted to turn off magic_quotes, for example.  You would do it in this function.  Anything you need to do before SMF starts working should go in here.  You can have a function for integrate_pre_include without using integrate_pre_load, and you can have a file for integrate_pre_include without an integrate_pre_include function.

I find the addition of integrate_buffer amusing, as integrate_exit was designed for, and can do, the same thing.

[Arantor]

Orstio: Thanks for the feedback, I really appreciate you looking over it because it's one of the more useful parts of SMF that's never really been documented (and I was sorta trying to guess the idea) and I know you know it better than I do!


integrate_egg_nog is an easter egg by name but it's callable, and called if defined, and thus it could be usable for modifying the query log before being sent to the user because it can pull details in from $_SESSION.

integrate_pre_load is unrelated, but the fact it's directly after integrate_pre_include seemed suggestive to me. Sure, they're unrelated but you would be mad to have them the other way around normally (because then you may have to modify code to load a file instead of using the hooks). But given lack of any other documentation, the fact they were placed directly one after the other, and have the same prefix, I took the cue that they were related.

integrate_load_theme was one I didn't quite see the logic of before, thanks for the heads-up.

integrate_exit can do everything that integrate_buffer could, granted. Not sure why that was included; again I'm going on inference from the source, nothing more.

[Norv]

This subject is also extensively documented in Bridges, Portals and Integrations board: A guide to the SMF integration hooks.

[Arantor]

And I missed one, yay for 7 hours wastes of time.

[Orstio]

Orstio: Thanks for the feedback, I really appreciate you looking over it because it's one of the more useful parts of SMF that's never really been documented (and I was sorta trying to guess the idea) and I know you know it better than I do!

NP.  I was trying to be helpful, as not a lot of people know these hooks are even in the code, let alone their purpose and use.

integrate_egg_nog is an easter egg by name but it's callable, and called if defined, and thus it could be usable for modifying the query log before being sent to the user because it can pull details in from $_SESSION.

Yes.  It's kind of a "write your own Easter Egg" function, really.  I had written an Easter Egg function for it into one of the bridges at one point, but I'm not sure which one or if those changes ever took place in anything distributable.

integrate_pre_load is unrelated, but the fact it's directly after integrate_pre_include seemed suggestive to me. Sure, they're unrelated but you would be mad to have them the other way around normally (because then you may have to modify code to load a file instead of using the hooks). But given lack of any other documentation, the fact they were placed directly one after the other, and have the same prefix, I took the cue that they were related.

The reason for their proximity in the code is that integrate_pre_include needs to happen before any other integration hook, in case the integration developer wants to do the smart thing and put all the integration functions into the file called by integrate_pre_include.  It needs to happen before all the other hooks, so it's logically right before the very first hook in the workflow, which happens to be integrate_pre_load.

integrate_load_theme was one I didn't quite see the logic of before, thanks for the heads-up.

NP.  It was one added based on discussion over integrations and the need to remove portions of the SMF output all the time, and problems with changing themes.  You know, it could probably be expanded in the future, if SMF had multiple theme layers for different things.  Like, say, the user panel next to each post in the message index.  If it were its own layer, this hook could be used to replace the default layer with a custom one.  That custom layer would work, without modifying any themes, on every installed theme.  If SMF themes were divided into more layers, the power of this hook would grow.

integrate_exit can do everything that integrate_buffer could, granted. Not sure why that was included; again I'm going on inference from the source, nothing more.

Yeah, not blaming you for the redundancy, just pointing out the same functionality is in there twice.  I think there may have been confusion about the name, and possibly the use within PHP 5 of throwing exceptions within the exit hook. 

[SlammedDime]

_exit and _buffer *can* do the same thing, but operate in two different methods... _buffer can be a comma delimited string of output buffer functions, each of which are called via ob_start('buffer') in succession.  _exit is simply called via call_user_func() and you can only have one _exit function or class::method call.

Accepts: - what type of input the hook is expecting. Mostly this will be '1+ function names', for which it should just be a string listing the name of a function, without any brackets, e.g. simply 'functionname' if you wish it to call function functionname(). If it accepts multiple function names to call, they must be separated with ::, e.g. 'myfunction::myotherfunction'. They will be called in the order listed, and all with the same parameters. It should be obvious that the function will need to already be loaded and thus available to SMF already - if not, you may need to ensure integrate_pre_include loads it, the main SMF files contain it, or you otherwise ensure it is loaded.

This is incorrect. 

Code Select Expand

call_user_func(strpos($modSettings['integrate_exit'], '::') === false ? $modSettings['integrate_exit'] : explode('::', $modSettings['integrate_exit']), $do_footer && !WIRELESS);

if :: exists in the function string, it is meant to indicate class::method, not function1::function2.  As you can see in the 'explode' call, the string is exploded by ::, returning an array of two parts.  when call_user_func receives an array, it is treated as a class::method call.  From the call_user_func documentation:

function - The function to be called. Class methods may also be invoked statically using this function by passing array($classname, $methodname) to this parameter. Additionally class methods of an object instance may be called by passing array($objectinstance, $methodname) to this parameter.

[Arantor]

* Arantor is glad he labelled it as a WIP.

Never realised it was used that way to name static methods. Though that does somewhat limit the scope of functions if you can only call a single function and a single file.

* Arantor will edit the above in just a moment, thanks SlammedDime

[Orstio]

if :: exists in the function string, it is meant to indicate class::method, not function1::function2.  As you can see in the 'explode' call, the string is exploded by ::, returning an array of two parts.  when call_user_func receives an array, it is treated as a class::method call.  From the call_user_func documentation:

Can I ask why you bother to explode it into an array when it can work just fine as a string without the performance hit of looking for it and then treating it differently?

$f='class::function';
call_user_func($f);

Has exactly the same result as:

$f='class::function';
call_user_func(strpos($f,'::')!==0 ? explode('::',$f) : $f);

If you don't have to strpos and explode it, there really is no reason for the performance hit.

[SlammedDime]

I have no idea why it was done that way, I wasn't the one who made the change (and I don't particularly like the change either)

[Orstio]

It just seems like a fair bit of bloat on something that is already going to be strained for resources.  I never did like the use of call_user_func either, but that's minor.

This is the current logout hook, for example:

Code Select Expand

if (isset($modSettings['integrate_logout']) && is_callable($modSettings['integrate_logout']))
call_user_func(strpos($modSettings['integrate_logout'], '::') === false ? $modSettings['integrate_logout'] : explode('::', $modSettings['integrate_logout']), $user_settings['member_name']);


And really all that is needed is this:

Code Select Expand

if (isset($modSettings['integrate_logout']) && is_callable($modSettings['integrate_logout']))
$modSettings['integrate_logout']($user_settings['member_name']);

[SlammedDime]

I'll take your word for it that it works (i'm at work and am unable to test).  One of the reasons I dislike the call_user_func is that it removes the ability to pass variables by reference, as all variables are copied to new, and then sent to the called function.

[Orstio]

I'll take your word for it that it works (i'm at work and am unable to test).  One of the reasons I dislike the call_user_func is that it removes the ability to pass variables by reference, as all variables are copied to new, and then sent to the called function.

True, which is bad, because it breaks the integrate_outgoing_email functionality.