my self-inflicted hell (a cautionary tale - also, HELP!)

[Kindred]

it took me 4 days to run an upgrade once (old server, LARGE forum)

[permutations]

Check this topic for a possible fix
http://www.simplemachines.org/community/index.php?topic=533314.msg3790994#msg3790994

I changed $timeLimitThreshold to 6, but how do I handle the fact that it got halfway through? I need to reload upgrade.php to apply this change. Is the script smart enough to continue where it left off (I hope??). I don't want to have to wipe the files and database, re-clone, and start again.

Also, when I ran upgrade.php, I selected extended error reporting. Where do I find this report?

--------------

Edit: I reloaded the script, and it gave me the option to continue where it left off. Yay!! It's running again - hopefully it will finish this time!!

[permutations]

I wiped everything so I could start again, but not everything is wiped. When I tried to run upgrade.php, it complains that it can't find what upgrade step I'm on. AUGH!! Where is it storing this? I want to clear everything and start again. I deleted and recreated the database and I deleted and recopied all the files. So how does it know? Cookies?

Also, when I ran upgrade.php, I selected extended error reporting. Where do I find this report?

[permutations]

I deleted all the cookies, and I'm still getting a lot of error messages on the upgrade.php script - all related to step number.

Notice: Undefined index: upgrade_progress in /home/normaleating/www/smf2/forum/upgrade.php on line 3490

Notice: Undefined index: upgrade_step in /home/normaleating/www/smf2/forum/upgrade.php on line 3495
1: Login

Notice: Undefined index: upgrade_step in /home/normaleating/www/smf2/forum/upgrade.php on line 3495
2: Upgrade Options

Notice: Undefined index: upgrade_step in /home/normaleating/www/smf2/forum/upgrade.php on line 3495
3: Backup

Notice: Undefined index: upgrade_step in /home/normaleating/www/smf2/forum/upgrade.php on line 3495
4: Database Changes

Notice: Undefined index: upgrade_step in /home/normaleating/www/smf2/forum/upgrade.php on line 3495
5: Delete Upgrade
0%


Notice: Undefined index: upgrade_overall_progress in /home/normaleating/www/smf2/forum/upgrade.php on line 3504

Notice: Undefined index: upgrade_time_elapsed in /home/normaleating/www/smf2/forum/upgrade.php on line 3529
: 0
Notice: Undefined index: upgrade_time_mins in /home/normaleating/www/smf2/forum/upgrade.php on line 3530
, 1
Notice: Undefined index: upgrade_time_secs in /home/normaleating/www/smf2/forum/upgrade.php on line 3530

I'm sure this is related to my having partially run the script before. How does it know this? How do I clear this and start fresh??

[permutations]

I tried truncating the sessions table - no joy.

NONE of these variables are in the $upgrade.php script. Where is this coming from?

[permutations]

I just found the same problem described here:

http://www.simplemachines.org/community/index.php?topic=527583.0

I'll reset everything and try again!

[margarett]

Take it easy. You're doing too many things at once

I'll reset everything and try again!

What do you mean by resetting? Remember all the things you had to change and your issues with repair_settings

Undefined index: upgrade_step in /home/normaleating/www/smf2/forum/upgrade.php on line 3495

Which upgrader are you running? In my upgrade.php for 2.0.10 that string is on line 3493...

Undefined errors for language strings (it is not related to which step you are on) are always related to wrong paths that came from your previous 1.1
Make sure that you move your 1.1 installation to a new folder + db then run repair_settings (which you had to modify). In repair_settings make sure that you set everyone to the default theme. Also in the duplicate 1.1 database, truncate the log_search* tables, as well as sessions.
Then, again, replace all files with the large upgrade package. Change again the $timeLimitThreshold to a bigger value. Take my word for it: set it to 500
Finally, run upgrade.php. Because you set a big time limt, leave it to run on his own and go to bed

[permutations]

The errors in Take 2 were because I didn't run repair_settings.php again. I didn't realize it was making changes in the database.

This time, before I ran upgrade.php, I went into the administration panel and emptied all unimportant logs, checked for errors (there was one, and it was fixed), and optimized all the tables. But I still have the exact same problem I described in detail here:

http://www.simplemachines.org/community/index.php?topic=536615.msg3813598#msg3813598

Though I increased the timeout to 60 seconds, it times out here:

Executing database changes
Please be patient - this may take some time on large forums. The time elapsed increments from the server to show progress is being made!
Executing upgrade script 1 of 2.
Executing: "Updating search cache..." (14 of 19 - of this script)

I even tried 5 minutes, and that wasn't enough. The problem isn't just that it needs more time. It's something else, and I don't have the faintest idea what it is. I have followed all instructions to the letter. How do I fix this?? Please help!!

[margarett]

Please, do that. Set the time limit to a huuuuuuge value, leave it to run and go to sleep. Should be ready in the morning (I hope )

Did you truncate the log_search tables, as well as the sessions?

[permutations]

Set the time limit to a huuuuuuge value, leave it to run and go to sleep. Should be ready in the morning (I hope )

This will not help. When the time has expired, I will get the same error. The guy quoted below raised the timeout to 60 seconds and the update ran:

My board has over 2 millions posts in it and 20,000+ users.

My forum is far smaller than this. I have about 115,000 posts and 3000 users. If that guy can upgrade with a 60 second time-out, I surely should be able to.

Also, it's hanging on a table that was empty because I cleared unnecessary logs before running the upgrade (log_search). This table does not even exist anymore. It's apparently deleted by the upgrade script after the new log_search* tables are created.

Did you truncate the log_search tables, as well as the sessions?

Yes, I did. That is not the problem.

It's stopping at such a trivial point that I wonder if the limitation is from my Web host. I asked them about that, and was told that the process timeout on my account is 60 seconds. They don't believe I'm hitting a limit, though - they asked to run the script themselves. I said to go ahead. They'll see the "timed out at 300 seconds" message, then maybe they'll believe me. They also have more ability to monitor what's happening on the server as it runs.

If Web host limitations are not the problem, then I don't know what it is. Has anyone else had the script hang at Step 14 and nothing could get it past that point? Could there be a bug in upgrade.php?

[permutations]

You seem to have great control over your payment process. My honest opinion is that you should keep it. As I told you, I'm not very into SMF's Paid Subscriptions but I am pretty sure that it will not replace your current system without *major* overhaul...

I suspect this may be true. I made a huge number of changes to the SMF code - adding numerous features and tweaking how things worked. But I don't absolutely need all of that. All I really need is the payment system. This was where I started with the changes, and I originally planned to make it a mod, so much of it is well-documented and self-contained. It may not be as horrendous to move over as I fear. I'll have to do a lot of file comparisons to find every piece of it, but that's not so bad. I'm starting to think it will be a lot easier than trying to adapt the SMF payment system, which is much less full-featured.

What I still want to know is: how do you grant forum users the "special access" after they complete payment? And how do you withdraw it if they don't renew the payment?
Where exactly do you interact with SMF's tables in the payment process?

I still may not be completely clear about what you are asking, but I'll try again. Basically, I rely on PayPal to implement everything, and my script just responds to PayPal's notifications. From the descriptions of the SMF payment system that you and others have posted, it sounds like SMF is reinventing the wheel to some extent. For example, you wrote:

There is one feature I have in my paid-access code that SMF doesn't (and at least I don't see it). I could give members a free trial for 7 days or whatever. I'm not currently using this, just mentioning it as a feature suggestion.

SMF *CAN* do a "free trial" sort of thing, it just won't handle "autoconverting" them to the full paid....

Actually it can. I guess that the OP would be interested in the "how"?
It's fairly easy actually. You just need a non-standard feature but which can easily implemented, together with whatever system you use for the paid subscriptions.
a)when a member subscribes (by whichever means you implement ) you change them to a membergroup (eg: "Full Member" or whatever). This means that a non-subscribed member will have no primary membergroup
b) You implement a scheduled task which runs once a day and gets users without primary membergroup, registered over 7 days ago. If found, shazam, wipe them!
Problem: this doesn't remove the already available payment at Paypal. If someone pays them after being auto-shazam'd ( ) you have to manually fix the mess (but I guess that also happens now, right?)

This is a lot of unnecessary work. The cleaner and better way to implement this is to use PayPal's free trial feature. When the customer clicks the payment button, I send PayPal the parameters for a free trial of whatever length I want. If the customer cancels before the end of the trial, PayPal sends a notification to my script to inform of that, and my script handles changing the customer's status. If the user doesn't cancel during the free trial, PayPal automatically initiates payment and sends my script notification that the customer has paid, which my script then handles. My script is event-oriented, rather than procedural.

Here's another example of how SMF is doing more work than necessary (from Kindred):

Smf can handle the forum payments very similarly...   A user subscribes to a group which you have defined for a time period and a price. They get access when PayPal responds with a paid notification.

I don't have different member groups for different time periods. It's not necessary because I don't manage that part of the process. I rely on PayPal to tell me when someone's access terminates. Nor do I associate price paid with a member group. I set up a relational customer database for this, as described in my previous message. (Actually, it's a set of tables within the same database that SMF uses, but with a different prefix.)

My customer database tracks details of whether and how they got access (no access, fee waived, free trial, contribution, one-time subscription, recurring subscription, one-time fee) and what they paid. When I look up a particular customer (I have two scripts that access the customer database - a sales tracking system and a data entry system), I can see everything that customer purchased on my site. Sometimes people who bought access to my forum for a month 5 years ago will come back, and I can see the history. I'll know if they also bought my book or anything else on the site. I'm able to see if my clients are also forum members. Etc.

SMF doesn't need to know any of this - it just needs to know if the person has access or doesn't - the payment status. Thus I use only four SMF member groups (the additionalGroups field): Current, End Near, Expired, Never Paid. I have a near-end member group so I can send automated renewal instructions. I distinguished between Expired and Never Paid so my sales tracking script can analyze whether free trials lead to conversions (payments).

Length of access isn't stored in SMF member groups because granting and removing access is handled by the script that receives the PayPal notifications, and the different types of sales (1 months, 3 months, etc.) are stored in my customer database (for sales analysis - not to control access). Again, my script does nothing but respond to PayPal notifications, and update member groups and tables accordingly.

Oh - and I have one more script that checks the access status of everyone who's logged in, and if their access has expired, it logs them out. It's run from the main Paid Subscriptions script, if 12 hours have passed. The check access script also clears out any pending registrations that weren't completed if the specified wait time has passed (i.e., the person abandoned the sale after being transferfed to PayPal).

Did I answer your question this time?

[permutations]

Assuming I'm ever able to update my site (which is seeming somewhat hopeless at the moment), I'd like to implement my payment system for SMF 2.0 as a mod, so it's not hell-on-earth to upgrade the next time it's necessary. I once wrote an SMF mod, but it was 10 years ago and I have no recollection whatsoever of the format. I've been searching the SMF site for documentation on the structure of mods, but I can't find anything. I want something concise and clear - e.g. mods have to contain a, b, and c. SMF programming standards are not relevant for me because I'm not going to upload it. As I said previously, it lacks an administrative interface, and I have no interest in writing one. Also, my implementation includes a theme because of the very tight integration with the SMF UI. Even if it had an administrative interface, if I were to upload it with a single unchangeable theme, I'd be deluged with complaints. So the mod is just for me - not because I don't want to share, but because I don't want to do the additional work to make it appropriate for public consumption.

But I do, very much, want to implement it as a mod that I can turn on and off - for my own sanity. Where do I find this information?

Thanks.

[permutations]

The tech support guy at my Web host is really driving me crazy. He ignores what I say to him, and just wants to run everything himself. I finally said fine, go ahead and run it. His brilliant conclusion is that the upgrade script times out after 5 minutes at step 14. DUH!!!

Another tech support guy told me that I have a 60 second process limit on my account. That's clearly the problem because the upgrade process may need to run for hours because my database is large. They need to raise my process limit, if only temporarily. Usually the tech support people there are very helpful. I've never had to deal with someone like this guy before. I wish he'd give the ticket to someone else.

[margarett]

I'm really sorry that you are having so many problems with the upgrade. It's really unfortunate.
I just (just in, 5 minutes ago ) upgraded a forum to 2.0.10

113252 Posts in 12967 Topics by 4799 Members

in no more than 5 minutes. So no, I don't believe that there is a bug in upgrade.php (actually I think there is , but not related to the problems you are having)
I would suggest that it might be a server limitation. Are you running on a shared server?
You can also export your database + files (again...) and import them in your local machine (eg, with XAMPP), perform the upgrade locally, then reimport the upgraded database + files to your server again.
This is something that I had to do more then once, when upgrading some user's forums...

Out of curiosity, who is your host?

[margarett]

SMF doesn't need to know any of this - it just needs to know if the person has access or doesn't - the payment status. Thus I use only four SMF member groups (the additionalGroups field): Current, End Near, Expired, Never Paid. I have a near-end member group so I can send automated renewal instructions. I distinguished between Expired and Never Paid so my sales tracking script can analyze whether free trials lead to conversions (payments).

This. This is what I was trying to understand. And, as I suspected, you don't need to change almost *ANYTHING* for this to work. The upgrader will keep your membergroups and its permissions so, as long as your script continues to update additional groups, everything works.
You just need to tweak your script to read/modify the column "additional_groups" instead of "additionalGroups"

[permutations]

My ticket was transferred to another tech support guy, thank goodness. It doesn't seem to be a server limit. The script is choking. If I were running into a system limit, I'd be getting a 500 server error. The tech support guy says there are no 500 errors in the log - the script is simply not running. It stops dead, like it's waiting for something that never happens.

Remember I'm running an older version of SMF than the upgrade script was designed to handle. SMF 1.1 RC1 is the oldest version it recognizes, and I'm running SMF 1.1 Beta 3. There could have been changes in the database between those versions that upgrade.php doesn't know what to do with.

The idea of downloading everything to a local server and doing the upgrade that way sounds dreadful. Still, I'd do it if I thought it would help, but I don't think it will. I think the problem is in the script, not the server.

I don't know how I will ever find and fix this. The upgrade script is complicated. I'd need whoever wrote it to help me find the problem - or someone who really understands what it's doing. Can you help me with this, margarett?

[permutations]

This. This is what I was trying to understand. And, as I suspected, you don't need to change almost *ANYTHING* for this to work. The upgrader will keep your membergroups and its permissions so, as long as your script continues to update additional groups, everything works.
You just need to tweak your script to read/modify the column "additional_groups" instead of "additionalGroups"

"Everything" will work only if I add "everything" back in - i.e., reimplement my payment system in SMF 2.0. I want to do it as a mod this time, but I can't remember how to write mods.

But first I have to get past this upgrade problem. BIG problem. I really need help from someone who understands that script. I need to know where it's hanging to know why.

[permutations]

I found a report from someone who had the exact same problem as me - hangs at Step 14. And he had a small forum. Now I really think it's the script.

http://www.simplemachines.org/community/index.php?topic=535795.0

He solved it by upgrading to a later version of SMF 1.1, and then upgrading to SMF 2.0. Maybe I'll try that.

[permutations]

I found two more reports of the exact same problem I'm having (hanging at Step 14). That's three reports so far, with a quick search:

http://www.simplemachines.org/community/index.php?topic=528587.0
http://www.simplemachines.org/community/index.php?topic=455979.0
http://www.simplemachines.org/community/index.php?topic=535795.0

Here's a report of it choking at Step 16. The problem turned out to be a typo in the script:

http://www.simplemachines.org/community/index.php?topic=530917.0

There is definitely a problem with the upgrade script. Maybe the log file can help me find it. The problem is, my live site is filling up the error log file very fast, so it's hard to find anything.

[margarett]

The upgrade script handles everything from SMF 1.0 so it surely handles your 1.1 beta

You still didn't tell me, who is your host?

You can't really reach who developed it, none of the people who worked on those versions is still around (maybe Oldiesmann). Anyway, I believe that there is nothing wrong with the upgrader... But your recent research makes me wonder...

I can look at it for you if you provide me with access to your files and database