Topic: More flow to SMF.

[Benchtech]

Sorry for the influx of suggestions but once I get thinking then ideas keep coming.

I think SMF should flow more, it's hard to convey what I mean but currently I think SMF has to many page loads. It would be good to get instant feedback and autoupdating of threads. For example on forms, instant feedback with regards to passwords not matching etc. Also threads updating automatically without page loads and options in the moderation centre not being links but flowing more.

Sorry if this makes no sense, really hard to get my point accross.

[Arantor]

While I appreciate where you're coming from, the fact is that there is no good way to implement that on PHP. You'd have to have requests made to the server on very regular intervals to check for new content. Even if 10 users are online, checking every 5 seconds, that's still 2 requests per second on average over and above what normally happens.

On this site, for example, there are typically over 1000 users online at any one time. Having each of them poll once every five seconds means that you'd have to be able to cope with 200 requests every single second, which adds literally thousands of database queries per second.

To achieve that you need to have a really serious server to cope with it, far beyond what most shared hosting solutions will provide.

Before we get into the suggestion of 'long polling' or 'Comet style' or even websockets, none of those work in a PHP based environment (except maybe if you're doing it through nginx/PHP-FPM and even then, experience of actually trying to make it work suggests it just doesn't)... because you implicitly have to hold the connection open on the server side to make it work.

Doing so on Apache (which is still the most popular webserver) means holding a PHP instance open for the entire time. Since that means an overhead of 8MB or so per PHP instance for the interpreter and the script being run, it doesn't take much before you get into needing a serious server to make that work - just for the PHP instances, 1000 users (like on this site) would require a machine with 8GB of RAM - just being kept in use, often idly, waiting for an update.

Even a much more modest setup, 10 concurrent users, say, that's still 80MB RAM just idling around, also far more than the average shared host is going to allow you to set up.


Unfortunately this is one of those 'nice in theory, hard to make work in practice' deals.

[Kindred]

Additionally.... doing all that ajax-type stuff actually makes it nearly impossible for handicapped readers (like blind screen displays) to properly function.

[Angelina Belle]

It would be possible to do partial-page updates following certain actions.  Everyone is thinking about social-networking type actions, right?
Frequent whole-page auto-updating could be very bad for large/active sites. But user-event-driven partial-page updates could actually save load on the server, if implemented properly.

I think it would be possible to switch between "classic" and "AJAX" to make access friendly for the handicapped.

[Arantor]

Unless the partial page update is initiated by the user, it would be feasible - e.g. if the quick reply AJAXively updates the page. But, if it's a partial update that figures out newly updated posts, that's not going to save you anything and frequently would make it worse.

[Angelina Belle]

Exactly.

If, for example, you get an AJAX update when you click a "Karma" type or "Rate this post" type button.

Benchtech suggested

For example on forms, instant feedback with regards to passwords not matching etc.

This sort of thing would certainly be possible -- instead of a page load, some Javascript gives you a pop-up and/or changes the page content, to allow you to un-set the caps-lock key and try again.

[Arantor]

There is already certain instances of that in SMF, like the username check when registering to make sure the username is valid and isn't already in use. Passwords not matching is also already instant feedback.

It's a bit different in the profile but I see no reason why that couldn't be done.

Though, really, it was more important to me to deal with the meat of the proposal:

Also threads updating automatically without page loads and options in the moderation centre not being links but flowing more.

Which requires a lot more grunt than is currently practical to do.

[Benchtech]

There is already certain instances of that in SMF, like the username check when registering to make sure the username is valid and isn't already in use. Passwords not matching is also already instant feedback.

It's a bit different in the profile but I see no reason why that couldn't be done.

Though, really, it was more important to me to deal with the meat of the proposal:

Also threads updating automatically without page loads and options in the moderation centre not being links but flowing more.

Which requires a lot more grunt than is currently practical to do.

Perhaps just a notification that it has been updated, then you can click to load it. I just used that as an example, I mean more the general flow such as not loading a new page to go between options on the settings page and that. If you look at a demo of IP.Board I like the flow of that, I don't know how intensive it is but they have it mastered pretty well.

[Arantor]

It doesn't matter whether it's a notification or the full content... you still have to figure out that something has been updated, and you still have to do that every few seconds. It adds up and gets expensive.

If it were just a simple case of using notifications, we'd all be using the Enotify mod. Then we'd find our servers couldn't cope, like all the other people who already use that mod to do just that have found.

As for going between options on the settings page, honestly, it's not worth the effort of implementation, because you'd still have to provide a secondary option for partially-sighted users (in which case, you're creating a whole lot of extra work for very little benefit), and there are security matters to contend with too, that don't have the same problems when using regular page loads.

[Fustrate]

If a server runs both PHP and something like Python or Node.js, we could actually do this quite easily with websockets. While PHP websockets are a pain, asynchronous languages have a much easier time maintaining and using the connections.

It's a thought for the future, one that I'm particularly interested in

[Angelina Belle]

But not "automatic check and notify every few seconds" of course.  Well, you could, but any large forum would regret it when they saw the overage bill.

Asynchronous. Seriously? Python?

[Fustrate]

With websockets, you don't need to check every few seconds. You open the connection, and then when the server has something to tell you, it sends the data along. No more empty requests/responses!

Yep, seriously. It's not impossible to bridge two languages, or even just run a script when needed. Of course it would be a plugin, not built into the core, since we're a PHP project first and foremost. Just adds some extra "oomph" for some people

[Angelina Belle]

So the "additional query" required to give the server "something to tell you" could be triggered by the database commit.

That's too much like having your cake and eating it too.

[Arantor]

First up, that would typically necessitate either the Python server (running as a sockets type daemon) or Node.JS app running on a port other than port 80, or it means you have to make that able to serve PHP files.

The other thing is that you'd have to write the authentication separately in Python or Node.JS because you can't allow PHP to do it. I couldn't find a good way whereby you could have the connection come in, allow PHP to authenticate and then leave the connection to idle. It might be doable if you're going down the road of Python/Node.JS just handling the connection and having some short PHP script purely to do the authentication but either way, you're still putting the core of it in Python/Node.JS.

Bridging the two languages is more than doable (I have stuff integrated between Node.JS and PHP) but it's a pain to maintain at times.

Also, this still isn't going to happen on anything less than a VPS, no shared host is going to let you play with long lived scripts of any nature.

(Technically, WebSockets is still a joke, there are multiple versions that conflict with each other, support between browsers is fractious at best and Firefox has more than once removed WS support due to security concerns. Long polling is better in that regard.)

[Fustrate]

Oh yes, WS is still immature, but it's shaping up into a good solution. >=VPS would be a requirement, as would some technical knowledge, but that doesn't mean we can't dream

[Arantor]

Oh, dreaming is great, but you just know that if you make it generally available, you're going to have even more support nightmares than you do now.

[Akyhne]

In 10-15 years, this discussion won't even be here, because servers got way fast enough. It's also a bit funny, because until maybe a year ago, we talked about optimazion for the user, due to slow internet connections. Now We hardly care anymore, as internet connections improved a lot.

The question is wether there really is any bennefit to a feature like this or not and wether it could be done differently. Let's take the 5 seconds update Arantor is talking about. 5 seconds would be total overkill in most situations, as you'd have to be at the bottom of the topic to benefit from it.
So how about every minute? If it's a new topic to you and there are several posts, you can easily use several minutes to read the topic. So every minute would be fine. And what do you really miss, if you leave the topic right before a new post is made?! If it's really such a big deal to you not to miss anything, even 5 seconds between updates would not be good enough.
And if you're hooked on the topic, you have probably already enabled notifications on the topic. So you are noted "instantly" by email. That's how I do it anyway.
There are other ways it could be done. Like if you hover over the last post, a request is made. And/or if you write in the quick reply, like I do now, a request is made every 30 seconds or so.
And there is already a notification on new replies to a post, if enabled by the user (which is overkill as one would always like to be notified. Remove the option to disable it in the next SMF!).

So what I'm trying to say is that if you are not reading the entire topic, and you are not on the last page of a topic and in the bottom of it where you can see the last post, there's no reason to update it.
I can see the benefit of instant notifications when we talk about a build-in chat, but people generally have a lot more to say in a a forum.

I'm personally hooked on Ajax requests, but I mainly see them as benefits when I click or do an action, like changing to the next page of a topic or posting in the quick reply. I don't see them as a benefit when I update my forum profile, that's a task I maybe do every 3 month in average.

[Angelina Belle]

until maybe a year ago, we talked about optimazion for the user, due to slow internet connections. Now We hardly care anymore, as internet connections improved a lot.[/quote]
This is true in many places in the world, especially in the cities and suburbs of first-world countries.  But not in rural areas, and some emerging internet markets. In many areas, mobile users have limited cellular bandwidth when away from "home". Even if they can find a 3G or better cell.  In some locations, wifi-only users are limited to "internet cafes" and other semi-public venues (stores, RV parks, marinas, hospitals) with slow wifi or just too many people trying to use the wifi bandwidth at once. Think of the kind of bandwidth that was available to "Arab Spring" users attempting to create internet hotspots using tools designed just for that purpose.  And think of people trying to use a forum and also trying to avoid over-running their cellular data plan at the end of the month.

Many forum owners still need to consider the load this kind of thing places on the server as well as on the user's internet connection.
If AJAX can be used to put up needed new information without imposing an entire page re-load on the server, user, and internet connection -- so much the better.

Where AJAX winds up being used to make "more cool stuff", like shoutboxes that poll every second from every open window open to the forum everywhere in the world, it will be a problem for some forums and users.

It is still a matter of what it is used for, and how it is used.

That said, some of what the user suggests is doable just in JS, without AJAX at all.
Form validation can be "interactive" not requiring the user to hit the submit button (or even disabling the submit button until the user fixes the data). That would improve "flow".  "enter the password again" comparison can be done as soon as the user fills the second field.

In some cases, using multi-tabbed forms for admin settings MIGHT make sense, for some forms, avoiding extra page loads. It might also lead to users switching between forms and not saving changes, unless it were implemented very carefully.

It could also mean re-arranging the forms, so that things that logically belong together will be found together, instead of (or in addition to) on the forms they have historically be on for the last 7 years. SMF 2.0 has some more complicated options than 1.x has, and that means a little re-thinking might be in order as we move on into future SMF versions. This is more about design focussed on usability, and may have nothing to do with page loads per se.