Upgrade Stuck at Making SMF MySQL strict compatible...

[Arantor]

It's been a long time since I looked at the 1.1 upgrade steps :-x

[Biology Forums]

Did I stumble upon a bug here? That upgrade.php cannot do what it advertises (e.g. going from 1.x to 2.1)?

I will try 2 things before updating you:

a) Try the code suggested by albert
b) Convert to 2.0 then 2.1, see if that helps

[shawnb61]

I also suggest using the code from github.

I'm  not sure if it a bug or a data issue yet...

[Biology Forums]

No dice. I tried (a) and I couldn't find a copy of upgrade.php on GITHUB

I was thinking, would having removed a column in members have anything to do with this? I removed ICQ, MSN columns a long time ago.

[shawnb61]

Yes, removing standard columns breaks your forum.  Never remove any standard columns...  Upgrades will definitely not work.  (E.g., in the 2.1 upgrade step, it converts those to user-defined user profile fields.)

Never download only one file from the latest github, as nothing is in just one file...  Most changes & fixes span many files.  Getting them piecemeal breaks things.  You should always get a complete set of files. 

Option 1 is to download the new full set from github here:
https://github.com/SimpleMachines/SMF2.1

If you download a full set, you will need to copy all 8 upgrade* files from the /other folder to your root SMF folder (where settings.php is) before upgrading. 

Option 2 is to use the nightly build from one of our contributors here:
https://www.simplemachines.org/community/index.php?topic=558451.0

[Sir Osis of Liver]

Did you try upgrading to 2.0 using 2.0.15 upgrade package?  As mentioned last night, upgraders often don't work the way they're supposed to, you have to dick around with them a bit to get them to go.

[albertlast]

Did I stumble upon a bug here? That upgrade.php cannot do what it advertises (e.g. going from 1.x to 2.1)?

Well from my point of view is your database to slow for the amount of data.

[shawnb61]

That upgrader step is, in fact, attempting to update the ICQ and MSN columns, which have been deleted.  When the upgrader experiences a true syntax error, it dies. 

I suggest re-adding those columns and trying again.  As noted above, I'd recommend using the current github version.

(Look in upgrade_1-1.sql around lines 2226-2237 - which is in the step that failed.)

[Biology Forums]

This upgrade process has been one of the worst SMF experiences I've had. I've spent over 400 minutes tweaking the sql and upgrade.php file to get it to work properly. I think I'm experiencing seizures, honestly. The JSON step was even worse than the previous, namely the log_actions part -- it reached 450,000 records and would suddenly timeout, and unlike in the previous steps, if you refreshed the page, it'd start back at 0.

I never, ever want to convert my forum to 2.1 if this is the experience that follows, it's seriously unpleasant having to use this upgrade process. I only did this to ready my forum and myself for a potential switch in the near future. By the looks of it, my forum will have to be down for 24 hours before a working version is online, because technically I'll have to do this all over again in a few months!

The upgrade.php file should show recommendations on how to tweak your server before starting the process. For example, stipulate that php memory is too low, execution time is too low, etc., so that the end user knows what to change server side, especially since we're dealing with important data. There should also be some sort of estimate of how long to anticipate based on the number of rows, for example, in a table.

[Sir Osis of Liver]

Did you try upgrading to 2.0 using 2.0.15 upgrade package?

I'm not recommending 2.1 to any of the forums I support, at very least until it goes final, and probably not even after that.  2.0 has been the best SMF branch to date, and at the glacial pace things have been moving here, will undoubtedly be supported for a long time to come.  However, I expect you'll have the same problem upgrading to 2.0.

[albertlast]

This upgrade process has been one of the worst SMF experiences I've had. I've spent over 400 minutes tweaking the sql and upgrade.php file to get it to work properly. I think I'm experiencing seizures, honestly. The JSON step was even worse than the previous, namely the log_actions part -- it reached 450,000 records and would suddenly timeout, and unlike in the previous steps, if you refreshed the page, it'd start back at 0.

I never, ever want to convert my forum to 2.1 if this is the experience that follows, it's seriously unpleasant having to use this upgrade process. I only did this to ready my forum and myself for a potential switch in the near future. By the looks of it, my forum will have to be down for 24 hours before a working version is online, because technically I'll have to do this all over again in a few months!

The upgrade.php file should show recommendations on how to tweak your server before starting the process. For example, stipulate that php memory is too low, execution time is too low, etc., so that the end user knows what to change server side, especially since we're dealing with important data. There should also be some sort of estimate of how long to anticipate based on the number of rows, for example, in a table.

Maybe i'm blind,
but i see not that you add you "tweaks" here.

you run a forum(smf 1.1) which is release 2006 (windows vista time),
would you get support for sutch old os?

Your hardware, like mention befor, look like not right sized for the size of your forum.

To tweak a forum is part of the forum admin not task of the upgrade process,
also everyone should had in mind which size the forum got,
big one like yours are less possible to test by the dev team(we didn't had sutch test env) and
the amound of setups are low of this size.

When you want that this process work more smoothly,
you want to pm a dev like me and provide a test env where we could look into how to process can be improved.

[shawnb61]

Study Force -

First of all, thank you for your patience & persistence testing your upgrade.  I think we all learned a few lessons here, thanks for documenting this.  If you were here in San Francisco, I'd buy you a pint.

Second, I agree with SOoL, that you should upgrade to 2.0 asap.  I know you've heard this before, but it really is time.  For many reasons - security, functionality - but also because baby steps are always easier than huge jumps. 

It sounds like you are on the right track practicing doing this in a test environment.  Good move. 

Finally, I believe there is a place for SMF support for very large forums.  If interested, let me know & I'll chase that down for you. 

Shawn

[Sir Osis of Liver]

There's a large forums board somewhere, you may be able to see it on team, I can't.  IIRC requires admin approval for access.

[Antechinus]

Yep, the big forums board is here: https://www.simplemachines.org/community/index.php?board=123.0
Offhand I'm not sure why it's not visible to everyone, but it has always been hidden from mere mortals. I suppose the reason is that they don't want n00bs on GoDaddy grumbling about their "big forum" not working when they get to 1,000 posts.

[drewactual]

what is the entry for the 'large forum'?  I'd like to see what tweaks exist for my own which may or may not meet that criteria...

also.....

i tell y'all what would be slick....  a hosted 'service' for upgrades.  it would conceivably be a place where a person logs in their credentials and a script live converts to the selected version, and the user can then download it.  it would allow not only users to simplify things, but also allow the team to see potential bugs or issues?  the service wouldn't even require write access, just read access... it would basically just copy over, in the proper form, a new db based on data from the old, and plant them in an updated release of SMF.   it would surely be easier for new users by punching a button "convert my DB to 2.1", and it would assist developers finding these bugs with the slight deviation in db's due to customizing...

if managed in this manner, a user could have their forum up and running all the way to flipping the switch to the new forum, and by simply loading the new forum engine in a different directory and 'testing' it for their purpose.  when they're satisfied? put the original forum in maintenance mode, copy over the new posts/users,ect, to the new db, and flip the switch.  it would be nice for making the move as all the user would have to do is point to the same images/files used by the prior to make their forum their own. 

conceivable?

and what is the post minimum for entry into the big leagues?

thanks!

[Antechinus]

AFAIK there's no actual post minimum, or not that I recall. Traditionally you need an invitation from the SMF team head honchos, which is based on their impression of your bigness. Size does matter, they reckon. Offhand I'd say that if you have a million or so posts up, it might be worth asking (politely).

[Illori]

you can apply to join via your profile. as the description of the group states your forum has to have 500k posts or higher to qualify. at this time we are no longer sending invites to join the group.

[drewactual]

i'm a year or so at my current trajectory from that mark.  no biggie one way or another as my site is working fine as can be.  (knocking on wood).

[shawnb61]

conceivable?

IMO, the short answer is no...   

Taking this thread as an example - the root cause of this specific issue is that certain columns were removed from the database. 

The further complication here is that we're attempting an upgrade across many versions, of a large, highly customized forum.  In many ways, a worst-case scenario.  (And again, thank you Study Force for documenting this.  There will be more...) 

Between every variant of OS/MySQL/PG/php/apache, and different folks' configurations for each, the simple truth is that admins must know their environments and configs, do their analysis, and test...  A lot... 

Shawn

[albertlast]

For me how do the performance optimization is
100k - 500k mid size (my test forum got 500k)
large is when you reached 2m.

Like mention from the other post always a question of env,
when you use a calculator as web/db server then already some hundred post could be large.
But in this case would be to change the env a more valid solution than to try change smf.