[3.0] Integrated mods/upgrades for package manager?

[Kindred]

I guess I just don't get what you are asking...  because, to me, it seems as if ALL of that is alreayd do-able with the currently package system.

The Package manager already has package servers
The mod install already has the capability to do "upgrade" (most mod authors do not use it for a variety of reasons)
I prefer my mods installations to actually ask me (like I set my windows... only do what I tell you, don't ever assume anything)

However, these are third party add-ons that we are talking about. We can set all the "standards" that we want. What will happen is that fewer people will submit to the official mod site and fewer people will write mods at all.

[SlammedDime]

Personally, I would rather alienate some mod authors in the interests of a more strict packaging and coding standards system.  All of this mickey mouse ****** on the mod site could be avoided, or more easily maintained, if there were higher standards.  It would also better serve the community to be able to easily upgrade modifications and keep up to date on their software.   Wordpress, Joomla, Drupal et al. all got this one right, SMF falls flat on it's face in this area.  The package manager is novel, and maybe it was someone's attempt to make something innovative back in the day, but with as many modifications as there are available now a days, it's simply not the right solution.

[texasman1979]

Almost all the functionality for what i propose is there in one form or another. What i propose is the reinvestment of innovarion to make the tools currently available, more robust and user friendly for both the coder and the user. With all due respect, the mod site and the package manager as well as integration hooks are in its infancy. I bid thee to sprinkle some miracle grow on it and see what it can do for smf 3.0. While your fixing bugs for the 2.0 gold release is when to begin looking forward at the future. If it isnt taken hold of, someone else will.

[Norv]

While I was reading the topic, I meant to start answering by quoting texas' posts, but until I got to the end, I agree with pretty much everything he said, so I won't bother quoting.

Thank you for the suggestions, and everyone for the discussion. Funnily enough, if I may say so, almost all concerns are correct, IMHO. For different stages of development we're talking about.

Yes, functionality seems to exist in some shape, but it's cumbersome and "extra-work" in a sense. Yes, it needs all refactored/rewritten such that the behaviors described in the OP are possible easily. There are some little improvements that may or not belong to the 2.x line, but mostly, they're for the line further down the road. We are already planning exactly that for smCore/SMF 3.0 package manager.
Okay, I give in and quote:

Everyone knows what windows update is as well as apt-get in linux. Id like to see in 3.0 a package manager linked to its own mods site. The package manager would display all installed mods, but also with a ping, display current versions with the ability to upgrade the mod if it is an old version. Also sub-version upgrades can be installed through it as well, ie *.1 or *.2. With this package manager, there would be a button to view/search for available mods and the ability to install then directly from the mod site. Download and install in a single click rather than downloading a zip and then uploading to the server then refreshing the package manager page, then installing the mod, etc. This idea would make simple machines mods that much more simple.

I agree, and we're looking at exactly that for the future.
Now, people seem to think of the current mod site, and by that people think of the current problems, which is exactly the issue here: the current problems (with extra work to make an <upgrade> package for example, and "enforcing standards"), should be addressed by exactly the easiness to do it instead, by a standard that doesn't really need to be enforced (or not in such strong sense in which it is understood today, as "approve mods only if" and "otherwise people will never do it"), but instead, as I've said elsewhere, provide people the right tools for the job in core, and prepare to be surprised.
Oh wait, you already said that too.
Thank you for the discussion, and please feel free to add ideas as to how it would better behave.

I don't think this has anything to do with premium services, or anyway not in the sense of putting them down in any way. On the contrary. I'm not aware of problems of Ubuntu having a commercial community because of apt-get.

Nothing prevents the user, or a plugin, to add yet another site to the list package manager would automatically "ping" packages from, and nothing prevents it today either. Exactly like apt-get sources, or Eclipse's updates sites. If plugins are not on a site (and on the contrary, nothing says the current mod site should be the only site offering modules or plugins, even today it's not), then they may be on another. If they aren't on any, then they need installed separately, and their updates won't be available through package manager unless they add an update site. Their choice.

[Arantor]

I'm not aware of problems of Ubuntu having a commercial community because of apt-get.

Not a particularly great comparison actually.

Ubuntu is, slowly, growing a commercial/premium ecosystem through the likes of Ubuntu One. Except that, the vast, vast majority of stuff users will be installing will come from the main Ubuntu repo first and foremost, then through Ubuntu One, then after that it's a wilderness. (And bear in mind, I'm using Ubuntu 11.04 *right this moment* Unity sucks, btw.)

You want anything outside the mainstream, you will be managing it yourself, that means including dealing with updates.

The exact same thing will apply here - the minute you want something outside the mainstream, you will have to manage it manually, that is to say the likes of things on BlocWeb, DzinerStudio et al. That is, unless someone plans to create a unified storefront and package server combination that allows automatic management of everything from a single place - *including* premium services, which is why Apple is so effective with its App Store.


You want to talk about premium services... the current offerings for managing premium services are nil. The current package server system provides no facilities for authentication (and even until 2.0, it didn't even advise what version is in use), which means that for premium purposes, it's useless. Last thing I want to do is provide a package server for premium customers without any kind of authentication. Chalk another requirement up, then. (And I think you'll find the same is true of Ubuntu/apt-get.)


You still miss the point though: the update platform is one thing. Actually having packages that can upgrade themselves is another. Under the current environment where so much has to be code edits, the only thing you can do is the current <upgrade> block system in the package manager, which basically instructs on how to mod a mod. I won't get into how flawed this approach is, and how fragile it is.

So not only will you need to figure out a mechanism for making mods more robust, you will also need to devise a whole new system for getting mods to be updated, and preferably one that doesn't require mod authors figuring out what changes need to be made between versions so that they don't have to maintain two separate code branches, one of the trunk and one of changes between versions.


And please, don't tell me that parable of 'if you build it, they will come'. It isn't happening: SMF has very, very long since had the ability to do a great many things without touching the themes, yet mod authors have been touching the themes for a very long time. Even current mods by current team members are doing this very thing when they do not need to be.

As you've said: the majority of the tools are already there... why not start by looking at why no-one's using them (because it's structurally broken, impractical and far too unwieldy to use for any mod of actual size), and start figuring out how to fix that, instead of looking outwards for a solution?

[texasman1979]

this is by far the intended point of this discussion.

you have to have an actual mod interface in order to have a smf apt-get.

and as far as the upgrades of future existing mods, not that hard to code an uninstall of the old and install the new.

what im pushing is a whole new way of looking at modding and the act of installing and using those mods.

and this in itself will mold the rest of the software to be better, totally cause of the modular design necessary to achieve it.

if you build an environment that is, within reason, easily changed, the quantity and quality of the mods will blow our minds.

big mods, little mods, they would all have a place as long as the receptacle of those ideas is designed to receive them.

have just a basic software sitting there. then a built in method to to modify it, in any way. then plugin what you want. its designed for it.

then you have a great many sites out there that are all based off the same core, but can do virtually anything.

take out the "core" features, and make them mods as well.

make it where people in as simple of a way as possible, build the site how they want it to be.

if this is made the goal, i think with a reasonable amount of care and effort, it can be made to happen.

an array is a list of variables, a class is an array of functions and arrays, and a collection is an array of classes.

you cant tell me that built in that way, just about everything couldn't be changed dynamically to suit the user.

i hope someone can see the vision i have. i would kill to see it in reality.

[Arantor]

you have to have an actual mod interface in order to have a smf apt-get.

Which there is and has been for years.

and as far as the upgrades of future existing mods, not that hard to code an uninstall of the old and install the new.

This has been possible for years. Consider why so few mods provide this option when it has been around for years.

what im pushing is a whole new way of looking at modding and the act of installing and using those mods.

What I'm pushing is that no matter how many different ways you look at it, there are certain fundamental truths that will trump it, that mod authors are fundamentally hobbyists who release something that they wrote, or they're premium authors who want to lock their content behind some kind of paywall. What's in the middle as far as mod writing goes, is minimal. (I speak from experience; I've been all three groups of modder, my first few mods were because it was something I wanted and just felt like releasing, my last few mods were essentially premium mods, and the ones in the middle, well I was about the only person at the time just writing mods on request)

Certain things just will not change.

and this in itself will mold the rest of the software to be better, totally cause of the modular design necessary to achieve it.

*shrug* You're not talking about a new way of looking at modding, you're talking about a new distribution method. Maybe by forcing a better distribution method it'll encourage the core software to be better built but I doubt it. In any case you still need to build that foundation first. Consider that Apple had iOS before it had the App Store.

if you build an environment that is, within reason, easily changed, the quantity and quality of the mods will blow our minds.

You have to build that environment first. Then you have to find people capable of writing to that level. Go take a look at the mods on the current mod site; they are not that level of coding - see before my comment about hobbyist coders.

big mods, little mods, they would all have a place as long as the receptacle of those ideas is designed to receive them.

Except they don't. Right now there is the SMF mod site. It doesn't cater to those who want to provide premium services, and I doubt it ever will. Nor does it provide any facilities other than vague hosting and a single thread for discussion, which is why so many of the bigger mods operate their own modding sites. Just off the top of my head that list is: every single portal, SimpleDesk, Project Tools, SMF Wiki, SMF Arcade, plus all of the premium theme sites. The reason is because the SMF mod site does not provide the facilities that those sites need. And I doubt it ever will (bear in mind: I used to be on the Customization Team, I'm aware of the plans that have been had over the last 2 years for the mod site. Even then, I suspect it won't meet all the needs for all those who would use it.)

have just a basic software sitting there. then a built in method to to modify it, in any way. then plugin what you want. its designed for it.

Which is what we have right now. And it's a shambles, that you can literally modify any line of code - because so many mods trip over each other. It takes real skill to avoid that, given the limitations of the current design. What's needed isn't an overhaul, it's a total from-fresh rewrite. Which is years away.

then you have a great many sites out there that are all based off the same core, but can do virtually anything.

You have a great many sites that all use the same core, and often don't do a great deal more. The size and complexity of most of the installed mods is not that large in practice.

take out the "core" features, and make them mods as well.

Where do you draw that line? Do you draw it at say the calendar? Or something more fundamental like post moderation?

make it where people in as simple of a way as possible, build the site how they want it to be.

Which massively increases the support load.

if this is made the goal, i think with a reasonable amount of care and effort, it can be made to happen.

Good luck with that. Come back to me in 5 years and tell me if it was achieved.

an array is a list of variables, a class is an array of functions and arrays, and a collection is an array of classes.

You can quote programming theory, well done. It's also utterly irrelevant here.

you cant tell me that built in that way, just about everything couldn't be changed dynamically to suit the user.

Security, customisability, speed, pick any two.

i hope someone can see the vision i have. i would kill to see it in reality.

Prepare to become a mass murderer.

[青山 素子]

I'm not going to go into the depths that some of the others have in this topic, but I would like to say a bit.

First of all, it's not a bad idea. A quite decent one at that, really.

The problem is that you have to balance complexity with ease-of-entry. If you make the system too complex, you'll shut out folks that have good ideas and can do some basic work, but have neither the time nor skills to master the necessary steps to develop. You'll also only encourage higher-level coders, who will want some kind of compensation for their expertise. Basically, you push the modification community into a commercial pay-for-everything direction (much like, say, Magento where everything above the basics will cost you). While it might raise the quality considerably, you've also shrunk the community and also alienated many people. Remember, this isn't a Linux distribution, it's a forum software.

However, it's not wrong to find ways to get better quality code without raising the skillset needed too high. Making things like updates easier for folks would encourage them to use that capability. As has been stated, people in general and coders especially are lazy. They'll take the easiest option. Doing things like modifying code directly is usually the easiest option. Handling upgrade differences (patches upon patches) is hard, so it won't be done.

It's possible to make some of the existing things that can be done easier to do to encourage use. For other things you suggest, you would need a complete re-write in a lot of areas. Heck, you could go fully object-oriented, but is that really the best structure for your goals? It would also raise the necessary skillset considerably and even make debugging much more difficult (at least in PHP, it can get painful to trace code execution paths when using objects).

In summary, some of your ideas are quite good. However, they probably aren't practical for this project at this stage.

[texasman1979]

With all due respect, is smf 3.0 going to only include a version number change? I wish i could sit around and write code all day, but my current situation does not allow it. In objects you have an easier time creating over loaded functions and complete replacement of functions. Without it your almost certainly going to have to change core code to get things done. What in proposing is a method that would almost eliminate that. Some things would have to be inline and file includes, but most can be strictly dynamic, and it essentially have the same performance, simply cause an alternate path is expected. We aint talking about the current mod site and the current software, we are talking about how it can be done in the next major release. If it is not the goal of the developers and users alike that troll this forum, to grow and learn, and extend our horizons, please tell me what the point is to write and care for and love this creation known as smf. The mainstream of the internet and computers and software as well is only 20 years old. Are yall saying that we have reached the extent of what we are capable of doing? The whole point is to push the envelope and drag lazy people with us. Is it not?

[Arantor]

-sigh-

Well done, yet again you misinterpreted everything I said. Even if a magic wand were to be waved and all the current issues were to be fixed, everything Motoko says (which echoes what I said) still stands: you alienate people and reduce the group of people who will contribute code. That's it. You will make it easier to update at the cost of reducing the amount of things being written.

Are yall saying that we have reached the extent of what we are capable of doing? The whole point is to push the envelope and drag lazy people with us. Is it not?

Yet again, no that's not what we are saying. We are saying that while things could change, you can push the envelope as much as you like but the more you push it, the more will fall out of it. You won't drag lazy people with you, you'll leave them behind.

[texasman1979]

If you dive deep into what im saying, youll see that im saying that even people that dont even know how to code, could create mods. The concept of integration hooks, and a modular environment designed to be changed, would make it possible for the novice and professional to comingle. Like i said, all the tools are there. They are just not in the form they need to be. The general population will no doubt follow the lead of the pioneer. They have no choice cause they are the builders of the road. It similar, in concept, to dreamweaver or visual studio. The tools where made and the entire world is different and better because of it. Smf is forum software, but there us no reason it cant be a wiki site, or blog, or even a shopping cart. What im suggesting is build a core that is totally dynamic, and designed to be manipulated. Then in conjunction to that, build in all the tools necessary for both novice and professional programmers to modify how they wish. Then provide an area where the end user can readily and quickly setup the site how ever they wish. think of an exe with a bunch of dlls. When upgrades take place, the dlls are replaced, not modified. Design the exe to work with a db table that instructs it what parts it has and the pathing to achieve the requested action. If you really put some thought to what im saying, even a 8 year old could write a mod, following a good guideance for the mod interface. And you mentioned above about template edits. Well with this type of setup, the templates are just an extension as well, to be manipulated. My practical expertise limits my ability to explain my thought as good as id like to, but all of you main developers if either smf or its future forks, coders of mods both novice and pro, really look at what im saying. Your seeing the forest, not the road that actually there, you just need to move the trees.

[青山 素子]

The general population will no doubt follow the lead of the pioneer.

Not all, and not for some time. It took decades for the Western US to be settled to a level where the general populace would travel over. Very few pioneers actually made the journey. Most of the population stayed in the very comfortable and civilized East.

It similar, in concept, to dreamweaver or visual studio. The tools where made and the entire world is different and better because of it.

I wouldn't say that... I've seen a lot of badly-done Dreamweaver sites; more than good sites. Likewise, how many average people do you know using Visual Studio to write up a DLL for fun? I know a lot who wouldn't even know where to start, but can make small edits to SMF if they wanted.

What im suggesting is build a core that is totally dynamic, and designed to be manipulated.

Okay, but know that you'll sacrifice considerable speed in doing so. The more dynamic you make it, the more you'll have to abstract things and the more layers you'll have. Those layers and abstractions come at the price of speed.

Maybe you want to look at a framework? There are many for PHP even: CakePHP, Codeigniter, Sapphire, symfony...

Then in conjunction to that, build in all the tools necessary for both novice and professional programmers to modify how they wish. Then provide an area where the end user can readily and quickly setup the site how ever they wish.

Who would be doing this? How many people will you be bringing to do this? Building a whole ecosystem takes many tens or people at the very least. Also, explain how you'll get novice programmers to look at all these objects and complex abstractions and keep them from running away in fear.

think of an exe with a bunch of dlls. When upgrades take place, the dlls are replaced, not modified.

The EXE usually is as well, unless you're not stripping the debug symbols. Usually then as well. If it's not replaced you'll get crashes since it's almost certain the entry point locations have changed.

Well with this type of setup, the templates are just an extension as well, to be manipulated.

What happens when three different modifications want to edit the same templates?

My practical expertise limits my ability to explain my thought as good as id like to, but all of you main developers if either smf or its future forks, coders of mods both novice and pro, really look at what im saying. Your seeing the forest, not the road that actually there, you just need to move the trees.

Sometimes you need to look at the big picture, you know. Sure, the road looks nice, but if turning SMF into a complex, do-anything, framework will require tripling the number of developers (not currently feasible) or cutting the audience who can make modifications to 2% of current, you need to decide if that's something you want to do.

[texasman1979]

There was at a point in time when when communication over 50 miles was a hard days ride. Today, i could call you and have a real time conversation with, anywhere on earth. There was once a horseless carriage that could maintain 20 mph for several hours. Today, there is production cars nearly 300 mph. There was once a mayan that peered at the moon. Today, there is human garbage on the moon. Looking at computers, there was once a computer that could fit in a single room. Today, the iphone has a million+ times the computing power of the space craft that left the garbage on the moon. Visions and dreams dont get realized by stating what is or was, they are realized by stating what can be. Instead of putting so much effort into how my idea wouldnt work, lets talk about how it could. Open your minds ladies and gentlemen. Lets invest in what can be.

[Arantor]

As I have told you before, having grand ideas are wonderful, so start explaining how you will make this glorious dream a reality (so you can leave the lazy, and hobbyist, people behind)

Having dreams is wonderful, but somewhere along the line someone has to make it happen. You have had several different programmers explaining to you why these ideas are not entirely workable, so do us all a favour and either start accepting that reality or start telling us how we are all so wrong and that our combined years of experience are all so inferior to your magnificent wisdom, by showing us how you plan to make it happen. Don't tell, show.

[~DS~]

Someone's obsessed with ideology.

[texasman1979]

No, arantor is right. Why try to come up with somerhing new when you can hide behind years of experience. Its like a tenured professor, doesnt have to compete, so they dont. Then you get good ideas flushed cause they swear up and down its bad. Arantor, if only i had the time you have to sit around and code something. What forum siftware can you show me disproving my hypothesis. Lets sit down and chat with the original coders of unix, and ask them what they think about ubuntu 11. The hard part is the core. The easy part is what is above it in the form of modules and dynamics. If you cant code it yourself, dont bash the guys idea and say produce some code, knowing i spend my time raising my son instead if coding. I drive a truck all day, and ponder things cause i dont have the time to produce it. Im beginning to see where the idea of lazy programmers comes from. Its too much work rewriting the core, so its a stupid idea. Lol dont bother giving me an opinion on my active thread in coding discussion. Just tell me again how stupid i am. Lol

[Kindred]

texasman...

you are missing the point. the core does not need to be seriously redesigned... most of what you seem to be discussing is already present. So no... the hard part is not the core.
Yup, it could (and probably will) be made a little cleaner and more obvious...   HOWEVER... you have either missed or purposefully ignored our comments based on years of experience in dealing with modders in the open source environment.

Yes... it would be great if we made upgrade capabilities to be required for every mod
-- we'd lose at least 60% of the modders there.
Yes... it would be great if every single mod used the integration hooks that are there, and those that will be added in future versions.
-- we'd lose at least 20% of the modders there.
your other examples are nice too...
-- we'd lose more modders with every one of them.

So, now we're left with 10% or less of the original modding community.


You have just said that you're not a coder. You obviously have no experience dealing with coding or coders. "Pie in the Sky" is nice... but it's just not realistic. You've presented some ideas. Thanks! We'll take (some of) them under consideration. At this point, you are beating a dead horse and every time you post further, you are sounding less like a user with some ideas to present and more like one of those annoying televangelists. ("Convert to my way of thinking because I'm right, you're wrong and you'll all burn")

[Norv]

Fear of change should not stop people from proposing, nor will it stop us from making it reality. Regardless of everything else, it seems to me we continue dealing with here with a deeper and deeper confusion between what belongs to the 2.x line (and current habits, and current mod site, and current core behaviors and their consequences), and what belongs to the 3.0 line and the future. That affects the way people read the proposals made by texas.

Allow me to state quickly for the record: as strange as his wording may come out at times, I believe texas is right on many points. Seeing them from the 2.x paradigm bias makes them seem unworkable, but his thinking experiment is welcome, for the future SMF, not for today nor tomorrow.

I respect your opinions, on the issues raised here (quite far off the original request). Thank you for suggesting what you see as problems, and what you see as solutions. Please consider keeping it friendly though.

Allow me to come back to the OP: the behavior of the package manager in smCore/3.0 will follow pretty much the request. (not only that much of the behavior is sort of there already, but we also are already planning it exactly that way). Therefore I marked it as accepted. This was the actual feature request here. Well, initially, anyway.

It may be good to split this topic, as it really got far off.

[texasman1979]

Innovation, by definition, is the search for and found destinations of new truths, from both new ideas, as well as the revisitaion of the old. Did we get our modern technology from roswell, nm, or did it in fact come from the depths of our own intelect, challenged. A childs eye, i recently prayed for, and a faith that would see me through. To what end, will a man wander, to quench his thirst for knowledge. I butt my way into a room of honed cut shiny gems. Will they see the glimmer within the new, seemingly discarded stone?