my self-inflicted hell (a cautionary tale - also, HELP!)

[permutations]

10 years ago, in 2005, I took a copy of SMF 1.1 Beta 3 and drastically modified it to add a bunch of features that I needed for my business site:

(1) Paid access only - by either a one-time fee (lifetime membership) or a monthly subscription.
(2) Ability to moderate the posts of individual members - queue them for approval.
(3) Ability to receive, as site owner, copies of every message posted in email.
(4) Enhanced notification options for forum members.
(5) More - I can't even remember everything I added.

Then PHP was changed so drastically with v5.2 then v5.3 that the forum started to break down in serious ways. For example, suddenly my settings for board order are ignored. Sections moved around on their own and I can't move them back. I would like to reorganize the forum boards, but I'm scared to make any substantive changes for fear of breaking it entirely.

I can't just start anew because these are paying customers. I can't just erase their accounts. I need to keep my member database, and I also need my modifications to tell me which members should have access and which should not.

The thought of recreating what amounts to a fork using a new version of SMF is overwhelming. On the plus side, I kept a record of every SMF file I modified or added. On the downside, this is the record: I created 15 new PHP files, added 20 new images, and modified 47 [sic] original SMF files. (((sob)))

I started off trying to make this an installable mod (especially because many people asked to use my modifications), but the changes became so extensive that it was easier to just make the changes directly. I wasn't thinking about what I'd face 10 years later if I had to upgrade my version of SMF. (It's all PHP's fault.)

What I'd like to do - if at all possible - is fix the SMF version I'm currently using, SMF 1.1 Beta 3, to work properly with PHP 5.3. I realize that's nontrivial, but the alternative seems like it might be worse.

Does anyone have any suggestions?

[margarett]

That's not really doable. 1.1B3 has had over 20(!!!) updates, mostly security fixes. You are beyond lucky you weren't hacked already and loose everything in the process.

Your best bet is really to upgrade. Really...
Your 1, 2 and 3 are doable with default features in 2.0.x (although 3 is really a bad idea, but that's your call )
Others can for sure be tweaked

To setup a test copy, upgrade and play with it takes literally minutes. An hour or 2, tops (of course, depending on your database size )

Even if someone would fix it for you, it would be just a temporary fix until other thing breaks whenever a new php version comes along...
Read this also --> http://www.simplemachines.org/community/index.php?topic=534915.0

[Colin]

Hi there,

That certainly is a sticky situation. I think for the amount of work making 1.1 Beta 3 work with 5.3 it is better to just put that effort into rewriting your modifications to properly for 2.1 as an installable package. It is going to be a lot of work, but I don't see any reason to keep digging the hole and deeper and trying to glue one more thing together until it all falls down. :\

[Colin]

Margarett beat me .

[permutations]

Thanks so much for the input!!

I realize, somewhere deep inside, that installing the latest version and implementing my changes as proper mods is the only way to go. I don't want it to be true, but it's clearly true.

Question: Should I use 2.0 or 2.1? What's new in 2.1?

I found the paid subscription and moderation features in 2.0. I need a way to test this without destroying my forum. I guess I'll see if I can clone the database, copy all the forum files to a subdomain, and upgrade the copy. Once I know how to make that work, I'll do it on the live site.

[permutations]

One other question... I need to keep my member database. The upgrade version says it will remove all mods and give a clean installation. Will it get confused by my heavily modified files and additional files? Also, importantly, what does it NOT wipe out? Will it keep my membership table? the forum message archive? Will it not (please no!) delete the custom tables I've created for tracking payments and product information (stuff I sell on the site)?

[permutations]

Read this also --> http://www.simplemachines.org/community/index.php?topic=534915.0

SMF 1.1 final was released in late 2006, which makes it a 8+ years old product.

While the foundations of the codebase remain very fast, stable and secure even by today's standards, the evolution of the technologies on which it is based (especially PHP and MySQL) has reached a point where it is no longer viable to fix incompatibilities.

Starting from PHP 5.5 and MySQL 5.6, your existing SMF 1.1.x setup will no longer function properly. Immediate symptoms are:
* Warnings (thousands in a short amount of time) in your error log, about a PHP deprecated function

Code Select Expand

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in ...................
* Wrong boards order in your Board Index

My Web host is running PHP 5.3.29 and MySQL 5.6.23. I've got the board order problem, and my error log is ginormous. Ugh!!!!

I created mods for SMF 1.1 back in 2005, but that was 10 years ago now, and I have no recollection of what I did or how I did it. I suppose it will come back to me.

[Kindred]

the upgrade will not delete tables....

it makes changes to core SMF tables -- so, hopefully you have not changed the core tables and columns too much.

Files don't matter, since none of the 1.1.x files will work with 2.0.x -- so all 1.1 files should be removed completely to your backup location and 2.0 should be installed clean.

As for the features...   please have a look through the wiki/manual link to the top right of your screen.

[permutations]

Lots of nice features in 2.1. Is it fairly stable?

It looks like the paid access features in SMF2 can duplicate what I wrote myself. If so, it makes more sense to use that than to recreate the wheel. But then I will lose all my prior payment information. The easiest solution would be to give permanent free access to all current members. They'd like that. Or possibly I could write a program that would read my original table, and assign members to different user groups based on their payment status.

The downside of not using my own system is that I will no longer have all payments by a customer in one place. I sell more on the site than forum memberships. I like being able to look up a customer and see everything they've purchased. Also, my forum payment system hooks into my accounting system - how I know how much money I'm making from everything I sell on the site (forum memberships plus other stuff). I wish there was some way to integrate the SMF2 payment system into my own so I wouldn't lose the advantages of my custom setup. But I can't see a way to do that.

Anyone have thoughts or suggestions on this? Maybe the payment system has integration hooks of some sort??

[permutations]

I suspect the reason my forum has never been hacked is that it's off the radar. I used a robots.txt file to make it invisible to search engines. It's a support group and I did this to protect the privacy of members as best I can, given that it's the internet.

[permutations]

I have a clone site set up in a subdomain, but I'm unclear how I go about applying the upgrade. If I delete everything in the forum directory before uploading the new files, a lot of stuff will be lost (pictures uploaded by members, etc). But if I don't first delete everything, will I have a mess? I don't want to mix old code files with new ones.

Also, my entire site is built around an SMF API feature that lets you add sidebars and really customize what the site looks like. I can't remember exactly where or how I implemented this. But if it's not in place, my site will break.

Given this, how do I go about upgrading the site??

[Illori]

2.1 is beta you should not be upgrading to 2.1 at this time. focus on 2.0.

http://wiki.simplemachines.org/smf/Upgrading

but by moving things to a new folder you are just making it harder on you.

did you setup a new database as well? if so you need to run What is repair_settings.php? then do the upgrade.

without knowing how you coded something we cant tell you if it may break or not.

[Colin]

I would suggest backing everything up then creating another instance of your forum and then upgrading that second instance. It is always better safe than sorry.

As Kindred stated, it will not wipe out your members or posts or anything like that.

[margarett]

Question: Should I use 2.0 or 2.1? What's new in 2.1?

2.0 for sure
2.1 is in beta stage, will suffer significant changes and, when the time comes to make it "final" you will still to have to perform the so-called "large upgrade" which will overwrite all your files, thus destroying your changes again. 2.0.x will be supported for a long time
The big advantage for 2.1 is the responsive design and some other goodies. But for now it is *really* not advised to run it in a production site.

I found the paid subscription and moderation features in 2.0. I need a way to test this without destroying my forum. I guess I'll see if I can clone the database, copy all the forum files to a subdomain, and upgrade the copy. Once I know how to make that work, I'll do it on the live site.

Yes, that's the proper way to do it. In order to clone and upgrade your site, you just need to:
* Duplicate your database
* Duplicate your forum files to another folder (subdomain or not, doesn't really matter )
* Load and run repair_settings (in the duplicate folder of course) to fix the database info to the "duplicate", then your files paths. Make sure you set everyone to use the default theme. Delete the tool after use.
* Delete your Sources and Themes folders (and some other "own" files you might have, eg, on the root of your duplicate folder). If you use custom images for, eg, membergroups, they might get lost (so you'll need to restore them later, which is why a full backup is always a good idea )
* Load the large upgrade package for 2.0.10, overwrite everything in the way
* Run the upgrade.

There you have it, a complete and upgraded forum. Of course, all your members, posts, etc are kept. And so are your custom database fields/tables. Which means that no information is lost

About the custom "things" you have around SMF... You really need to try, no other way to put it.

Also, my entire site is built around an SMF API feature that lets you add sidebars and really customize what the site looks like. I can't remember exactly where or how I implemented this. But if it's not in place, my site will break.

This API (whatever it is ) can surely be updated to work with SMF 2.0. The amount of work it requires is... not possible to know without looking at it and what it does. But if you put sidebars and stuff, it's most likely putting SMF inside an iframe (which is something we don't really advise and 2.0.6+ has even a "block" for it - which can easily be disabled).

The downside of not using my own system is that I will no longer have all payments by a customer in one place

Again, all your previous content will be kept. And you *CAN* still use your payment system. Allow me to elaborate:
In SMF, 1.0/2.0/2.1/whatever, access is controlled by membergroups. So when your client "pays", you most likely grant him a certain membergroup which, in its turn, grants the user some rights.
If that's the case, it should be dead easy to make it work in 2.0 or even 2.1 as this mechanism works exactly the same.

Yes, you are in for some work, no doubt. But it's much, much better to take it with (some) ease, test and implement, than the alternatives... Because:
* Alternative 1: you do nothing and any further PHP update might cause your site to drop dead, making it *much* worse to recover from.
* Alternative 2: you tweak SMF to work with PHP 5.3, then one of these days you will need to do it for PHP-something again. And you will still remain with a *severely* outdated forum, with lots of security holes, waiting for a hack to occur... Not a good idea

[permutations]

First, I just have to say how grateful I am for the wonderful help you all are giving me. SMF is great software, and the support is fantastic.

* Alternative 1: you do nothing and any further PHP update might cause your site to drop dead, making it *much* worse to recover from.

I learned this the hard way on another of my sites. Not a chance I'm going to wait until the site breaks completely.

{RANT}
I have another site that uses Drupal. I was warned by my Web host that PHP 4 was going away and I knew the version of Drupal I was running was incompatible with PHP 5, but I procrastinated because upgrading Drupal is such a nightmare, and then I forgot about it. When PHP 4 went away, the site became completely crippled, and now I have to write a script using Drupal's Migrate API (which is horrendously complicated) to move the data from my old database into a new installation of Drupal. Plus the support on Drupal.org is terrible. People are snotty and clique-ish. Most of my questions were completely ignored, but maybe I was lucky. The people who got answered were mocked and ridiculed for being ignorant. When I have Drupal problems, I have to spend hours and hours searching the archives - and lots of problems crop up. Every update is completely incompatible with the previous version so all the mods break. Also, I'm the maintainer of a mod so I have to deal with their API (not to mention their butt-ugly code). They do things like changing the parameters in hook functions without changing the function name, and do no error checking on what's passed to the function. So I'll have a function that worked perfectly in a previous Drupal version, I get no error messages, but it just doesn't work in the new version. After 5 hours of searching, I'll discover that they changed the input parameters of an obscure function - silently. SMF is a breath of fresh air after that.
{/RANT}

2.0 for sure
2.1 is in beta stage, will suffer significant changes and, when the time comes to make it "final" you will still to have to perform the so-called "large upgrade" which will overwrite all your files, thus destroying your changes again. 2.0.x will be supported for a long time
The big advantage for 2.1 is the responsive design and some other goodies. But for now it is *really* not advised to run it in a production site.

Good to know! That is what I will do.

* Duplicate your database
* Duplicate your forum files to another folder (subdomain or not, doesn't really matter )
* Load and run repair_settings (in the duplicate folder of course) to fix the database info to the "duplicate", then your files paths. Make sure you set everyone to use the default theme. Delete the tool after use.

I did all this, but without touching the theme. I had a weird thing happen with repair_settings.php. I correctly entered all the database information (host, username, password, db name) - and this stuff was already correct in Settings.php. But repair_settings.php insists it does not have access to my database. Yet the clone site works fine - I certainly do have access. I don't know what's up with that. Seems like a bug in repair_settings.php. It was late last night so I didn't look (yet).

Also, my entire site is built around an SMF API feature that lets you add sidebars and really customize what the site looks like. I can't remember exactly where or how I implemented this. But if it's not in place, my site will break.

This API (whatever it is ) can surely be updated to work with SMF 2.0. The amount of work it requires is... not possible to know without looking at it and what it does. But if you put sidebars and stuff, it's most likely putting SMF inside an iframe (which is something we don't really advise and 2.0.6+ has even a "block" for it - which can easily be disabled).

No, it's definitely not inside a frame. I thought I used an SMF API, but maybe I didn't. I recall playing around with an SMF API that let you customize your layout, but it seems I didn't end up using it. Now that I look at the code, it appears I did it all manually with CSS. It's coming back to me... I originally had the SMF forum running inside a table, then at some point my CSS skills improved, and I got rid of that. I think it's a vanilla theme, but I manually customized it, tweaking the colors and changing the bitmaps used.

I use FileZilla, so I'm glad I found the warning about how they changed their default settings.

Again, all your previous content will be kept. And you *CAN* still use your payment system. Allow me to elaborate:
In SMF, 1.0/2.0/2.1/whatever, access is controlled by membergroups. So when your client "pays", you most likely grant him a certain membergroup which, in its turn, grants the user some rights.
If that's the case, it should be dead easy to make it work in 2.0 or even 2.1 as this mechanism works exactly the same.

Yes, I use membergroups for this, so that part will be easy. The hard part is maintaining my original customer database without hooks into your payment feature. I have APN turned on in PayPal, so maybe I will still receive the notifications that I use to update my database. I need to know when people pay so my accounting system doesn't break. There's a warning that SMF is going to generate errors if I turn on APN, but I really need to do this. I'm selling other things on my site besides forum access. Everything else seems like it will work. I have a page I direct people do after paying and SMF allows me to specify that. I can't know for sure until I try it, but it seems like it will be fairly straightforward.

Question: My paid-access code checks whether the person paid by check (and thus there is a 2-3 day wait). I don't give access until the check clears. It's automated in my PayPal response script. Does SMF do that?

There is one feature I have in my paid-access code that SMF doesn't (and at least I don't see it). I could give members a free trial for 7 days or whatever. I'm not currently using this, just mentioning it as a feature suggestion.

[Kindred]

SMF *CAN* do a "free trial" sort of thing, it just won't handle "autoconverting" them to the full paid....

SMF payments all go through paypal - but can be overridden locally by admin assignemtn of the user to a membergroup with an expiration date.


regarding the blocks and columns...  you migt want to take a look at some of the portal mods. They do exactly that and integrate directly into the smf admin interface.

[permutations]

SMF *CAN* do a "free trial" sort of thing, it just won't handle "autoconverting" them to the full paid....

SMF payments all go through paypal - but can be overridden locally by admin assignemtn of the user to a membergroup with an expiration date.

Sounds good! I need to actually use it to fully understand how it works. My main concern is the handling of PayPal notifications (the APN notifications). I really need to receive and handle these within my own code because I sell more on the site than forum access. If SMF requires APN notifications to go to a script within SMF, I have a problem. That is hairy coding. My PayPal script is very complicated, and debugging it is hard. I really don't want to touch that.

regarding the blocks and columns...  you migt want to take a look at some of the portal mods. They do exactly that and integrate directly into the smf admin interface.

Nothing "broke" here to fix. My handcoded CSS-based wrapper works perfectly and I use it throughout the site. There is much more on the site than a forum. I'm glad I didn't use an SMF API for this. My upgrade process would be much more painful if I had.

My main concern right now is why repair_settings.php thinks I haven't specified how to access the database when I clearly have. I do not want to accidentally update the wrong database. I want to make sure everything on the cloned site is pointing to the cloned database. I look at the code and see why it thinks that.

[permutations]

I looked at repair_settings.php and I see why it wasn't finding my database settings. There are a bunch of database variables in Settings.php that SMF names db_* (db_server, etc.). In my customized version of SMF, I changed this to $smf_db* ($smf_dbserver, etc.), because I use two different databases and I didn't want ambiguity in the code about which I was using. I had to create my own custom version of repair_settings.php with different variables names for it to find everything.

I wish SMF used the naming scheme $smf_db* to name these variables rather than $db_*, to make coding clearer in the case of multiple databases.

[permutations]

There's something wrong in the guess_attachments_directories() function in repair_settings.php. This is the code starting at line 742:

Code Select Expand


if (empty($userdDirs))
{
$usedDirs = array();
$request = $smcFunc['db_query'](true, '
SELECT {raw:select_tables}, file_hash
FROM {db_prefix}attachments
{raw:smf1_limit}',
array(
'select_tables' => $context['is_legacy'] ? ($id . ' as id_folder, ID_ATTACH as id_attach') : 'DISTINCT(id_folder), id_attach',
// If it's SMF 1.x then check only the last attachment loaded
'smf1_limit' => $context['is_legacy'] ? ' ORDER BY ID_ATTACH DESC LIMIT 1' : '',
)
);

if ($smcFunc['db_num_rows']($request) > 0)
{
while ($row = $smcFunc['db_fetch_assoc']($request))
$usedDirs[$row['id_folder']] = $row;
$smcFunc['db_free_result']($request);
}
}


For starters, there appears to be a typo in the first line - shouldn't $UserdDir be $UsedDir (no extra r)?. But changing this doesn't fix it. That just means the if-condition always succeeds and the array is created there.

I get an error message on Line 756 (the nested if-statement) because $request does not contain a resource - i.e., the query is failing. But I can't tell why because I'm not familiar with the database API. I don't think this is due to my changing some variable names because everything else in the script is working fine. Do others get this error?

[permutations]

What's that weird 0-length file Settings.php1 that keeps getting created?