How do I make my SMF2.0 RC3 Upgrade 100% mod & edit free - database & files?

[ElectricSquid]

Hi all, I'm trying to upgrade to smf 2.0 rc3 from smf 1.1.9
Due to the amount of custom edits, mods, and customization of the theme, I'm not able to upgrade past 1.1.9 without errors.

What I'm looking to do is start fresh, as in clean install of rc3 kind of fresh, without any of the leftover coding and/or database leftovers from the past few years. BUT, I need to keep all the members, topics, and messages currently in the database.

During a normal upgrade to 2.0 rc3, the members, topics, and messages are converted over, but with it comes all the leftovers I'm trying to leave behind. One of which was TinyPortal 0.83, which took me almost a day to get around due to it's left over coding and database tables.

Anyway, is there a way to do a clean install of smf2.0 rc3 then import the members, topics/posts, messages, and any other needed database tables?

[Kindred]

it would not be easy...  As a matter of fact, it would be rather manually intensive

What kind of errors do you get, if you run the 2.0RC3 upgrade.php on your 1.1.9 database?

[ElectricSquid]

1.1.9 upgrades to 2.0 RC3 without issue, as long as I disable TinyPortal first.
There is an issue if TinyPortal is still active (via mod setting), the template system crashes immediately on pageload. Error was something about not being able to load 'main_above' or something like that.

The reason for wanting to do what I mentioned was to basically be able to start fresh (database and code) but retain the members, posts, and topics of a 3-4 year old forum. There are edits still in my code from back before I knew what the hell I was doing. There are also many old mod edits that are custom, so uninstalling doesn't work.

[Kindred]

but wait....    upgrading to 2.0, you will replace ALL of your forum files. None of the files for 1.1.x will work with 2.0, so how would any custom edits interfere?

switch to default theme
Copy all 2.0 files, overwriting your current files.
run upgrade.php

[ElectricSquid]

but wait....    upgrading to 2.0, you will replace ALL of your forum files. None of the files for 1.1.x will work with 2.0, so how would any custom edits interfere?

switch to default theme
Copy all 2.0 files, overwriting your current files.
run upgrade.php

Replaced, or re-written?
I tried doing exactly what you said, and some of the leftovers were still there. In fact, the Core theme was not Curve after the upgrade. It was a barely working leftover of the 1.1.9 core theme (which I had edited back when it was 1.1.4). Weird.

After the upgrade, I had to replace the default theme files with a set I grabbed from a fresh RC3 install.

Anyway, if what you say is correct, then I could/should be able to do the upgrade, then take all the files and replace them with a fresh install set of files, correct? This would insure all code edits are gone, along with any extra files that some mods add to the webroot folder. Unfortunately that didn't work when I tried it.

Still leaves the database leftovers.
I need to figure out what belongs to SMF compared to what belongs to an old mod. Then attempt to remove the latter. That's why I figured salvaging the members and topics would be easier.

You're saying there's no easy way to import the already upgraded tables over to a fresh install?

[Kindred]

no... there are relationships between tables...

it sounds like some of your file permissions are stopping the overwrite of the new 2.0 files.

So, what I suggest

Delete all files in the Sources directory
Delete all files and subdirectories in the Themes/default subdirectory
delete index.php in the root.

NOW upload the files in the large upgrade archive.


leftover mods in the database should have no effect until the are re-installed

[ElectricSquid]

Oh, by the way, this work is being done on a devbox test environment with database and forum file backups from the live server. So I can install, uninstall, totally screw it up, and start over without issue. ... and the server is right here in the room with me, not remote hosted. I'm also using MySQL Administrator and Query Browser to manipulate the database if needed.

no... there are relationships between tables...

it sounds like some of your file permissions are stopping the overwrite of the new 2.0 files.

So, what I suggest

Delete all files in the Sources directory
Delete all files and subdirectories in the Themes/default subdirectory
delete index.php in the root.

NOW upload the files in the large upgrade archive.


leftover mods in the database should have no effect until the are re-installed

Oh, so I need to delete first, then ...
    ... what about uploading the upgrade archive into a fresh new empty webroot folder, then doing a quick rename of the existing and new webroot folders to complete the swap. That way the old files were never there in that folder to begin with.

That would eliminate a lot of my worries about my less educated coding days haunting me.
Still leaves the database stuff.

[Oya]

that would work

the upgrade will change the structure from 1.1 style tables to 2.0 style tables and anything left over from mods will just be sat there, dormant

[ElectricSquid]

that would work

the upgrade will change the structure from 1.1 style tables to 2.0 style tables and anything left over from mods will just be sat there, dormant

LOL, dormant, confusing me

The files being all new is the big part.
Though I would like to trim the old fat outta the database some day soon.

[Kindred]

extraneous data in the database will be ignored.

The only reason I told you delete first is because you seem to have permission problems that are preventing you from doing the normal overwrite. Normally, you would have have to do that....


the only problem with the way you suggested is that things like avatars and attachments would be lost. The way I suggested, only the files in Sources and Themes/default are replaced.

[ElectricSquid]

I got it.
Used the updated (from 1.1.9 to RC3) database and Settings.php
Renamed the existing web root folder
Created new webroot folder (with proper name to be active)
Uploaded RC3 upgrade archive
Uploaded Settings.php (from the 1.1.9 > RC3 upgrade)
Uploaded attachments folder, avatars folder, and a few other folders containing images and such

Pointed my browser to the devbox site and it loaded right up

As far as the database, TinyPortal is the bulk of the leftover stuff. Thankfully, the tinyportal coders were cool enough to have a "tp" prefix on most, if not all their database edits. So it'll be easy to find it.

[ElectricSquid]

Though my issues are near fixed, how does one import members, topics/posts, & messages to a clean install of SMF2.0 RC3?

[Kindred]

see... this is what we have been telling you... one does not.

one runs the upgrade on the existing database...  why are you so opposed to running upgrade.php on your database?

[Oya]

you would have to import probably half the tables which will need many columns renamed... so the upgrader is by far the quickest and cleanest solution

[ElectricSquid]

...  why are you so opposed to running upgrade.php on your database?

I'm not.
I think I see the confusion here.
I have run upgrade.php on the database that I want to import this data from.
The import I want to do would be from an already upgraded database > fresh install database.

I would like to create a clean install w/ database, then import only our old messages, old topics, old members, and whatever other table data is needed to get that data to run properly, but leave the extraneous data behind. Both databases are already updated to 2.0 RC3. One is through the Upgrade Package, the other is through the Install Package, it's just a matter of moving some of the upgrades data to the installs database. I could do an add or a replace with the data I want to import, whatever is easier and will work. I just need to know what data is able to be moved like I intend, without some weird issue popping up later on down the road. I know some tables have connecting data, that's where I need your help, figuring out what data needs to be moved with the data I want to move.

There is a second reason for wanting to do this. As I create the theme for the upgraded RC3 site, the live forum still runs 1.1.9 and will gain more members and topics than are on the Devbox test site. There will come a time where I will need to add these new posts and members to the test sites database, or I will loose them in the "live" upgrade. So no matter how I look at it, there will come the need to add the data I want to move to the database (after running it through the upgrade script first).

(note: whenever I do a large scale upgrade like this, I use two machines. The devbox ends up being the new live server and vice-versa. This makes swapping live sites almost seamless, no downtime.)

.... the upgrader is by far the quickest and cleanest solution

I agree with that, only a crazy person would want to manually upgrade the database to 2.0

[Kindred]

moving tables like that gets complicated and could indeed lead to odd things happening.

what I would do (and have done)
take a backup of the db
run the upgrade script on your production machine.
done...


the way you are suggesting has potential for missing data (as you already note) and messed up data with associated tables.

Again, I fail to see how your desired method has any real benefit or simplicity.

[ElectricSquid]

Here's another example where needing to move table data would be needed.
Lets say two of our members get in a argument. Member 1 gets mad, deletes some of his topics, then deletes his account. Then comes back 2 weeks later after he's not angry anymore (member 2 was right), and would like to put his info back because he admittedly was an idiot when he deleted it.

Now I have his info on a database backup, but I can't just run a restore, because 2 weeks of posts and new members have happened since his incident.

So what are the choices?
1) I could tell him to shove off, but I'm not that kind of admin.
2) I could run the restore on the devbox, and then import his data (somehow) over to the live sites database.

This is almost the same as what I want to do here with the upgrade>install.



note: and if that's not easy to do, then it should be looked into making that easy to do. It's a good feature to have available.

[Kindred]

you could also activate the recycle bin option, so that deleted posts go there... 

Honestly, I can say that I have never seen anyone do it the way you are asking....   it is complicated because of the associated tables.

Every time I have had to "restore" a member, I have just done a SQL command to reset posts with his old username to the new usernumber...

and as I said... I do not see any simple way to do what you are asking...

[ElectricSquid]

OK, what if we did it the other way around?
Upgrade the database, compare tables to that of a fresh install, remove extra tables, then import from a fresh install the few known places mods might have added data, like modsettings (for example).



Oh, and I do use the recycle bin, and to delete an account, you need admin approval on our site. I was using that as a last resort example.

[Kindred]

you could try it...   again, I have never seen it done the way you seem to want to do it...

[ElectricSquid]

Does the upgrade package clear the all the old data from modsettings?

[Kindred]

no.

[ElectricSquid]

Do you know if modsettings is connected in some way to other data (in a way were I couldn't simply replace it with the install version)?
The mods are gone, so the data should go too.

[Hj Ahmad Rasyid Hj Ismail]

Hmmm... reading all these posts I just want to note one thing. Why don't you upgraded to 1.1.11 first instead of directly trying it from 1.1.9. I would not recommend this form of upgrade even if it works (partially). Any version of SMF 1.1.x other then 1.1.11 should be upgraded to 1.1.11 first. When that is done, check whether the forum works (with or without mods - mod is not important since you gonna upgrade it to 2.0 then). Only if all is working fine, then upgrade it to 2.0 RC3. This is how I will do it. Long way but for me it's the safest.

[ElectricSquid]

Why not 1.1.11?
Because the forum errors out if I try to upgrade past 1.1.9, probably due to some old edits I did years ago before I had a clue what I was doing.
Removing mods doesn't work, there are too many custom edits that interfere, again, from years ago before I had a clue what I was doing

This is what I'm trying to get away from by starting fresh, but I don't want to loose all the members and posts.

[Hj Ahmad Rasyid Hj Ismail]

If I were you, this is what I'll do. I may not be able to remove all mods or any of them. So I will create a dummy test site with the same (copied) files and database. I will overwrite my current forum with the 1.1.9 upgrade package first. I will see whether that is working first before I upgrade it to 1.1.11. Care to test this method?

[ElectricSquid]

Let me get this correct, on the devbox* you suggest I overwrite the existing (copied from the live site) 1.1.9 with a 1.1.9 upgrade package, then upgrade to 1.1.11, then 2.0 RC3?

Will the 1.1.9 upgrade package wipe out the moded Source and Theme files?






*devbox = development box - this is the test server.

[ElectricSquid]

... and don't forget this question about 2.0 RC3...

Do you know if modsettings is connected in some way to other data (in a way were I couldn't simply replace it with the install version)?
The mods are gone, so the data should go too.

[Hj Ahmad Rasyid Hj Ismail]

Yes. And I know that this is SMF 2.0 support board. I am just sharing my idea on how the upgrade to SMF 2.0 can be worked around. Feel free to use any other methods.

[ElectricSquid]

I welcome your 1.1.9 suggestion, and find it to be relevant to the subject at hand.
Thanks.

I'll try your idea later on tonight.

[Oya]

theres really no need to go 1.1.9 to 1.1.11 to 2.0

there are no db changes after 1.1.9

and the modsettings are all in the database in smf_settings and will in almost all cases just be reused

[Kindred]

thank you oya.... exactly.   My instructions stand as the best upgrade route.

[ElectricSquid]

theres really no need to go 1.1.9 to 1.1.11 to 2.0
there are no db changes after 1.1.9

Cool, I didn't want to go through all that again anyway.

and the modsettings are all in the database in smf_settings and will in almost all cases just be reused

On my upgraded version of SMF2.0 RC3, can I swap smf_settings with a fresh install version of this database info?
Or is the default vale something like null? or nothing?
(I haven't seen the default value in so long, I don't know what it looks like anymore )

thank you oya.... exactly.   My instructions stand as the best upgrade route.

As long as I can clean up the files and database to a fresh install value, I agree with you.
Here's where I'm at with this...

Files - Solved
Assuring the SMF files are new has already been taken care of by the Upgrade Package. Just to be 100% sure the files were new, I was able to manually copy fresh new files from the smf upgrade archive to a fresh new webroot folder, so I KNOW there are no leftover mods & edits there anywhere.
This might have been overkill, but I had to be sure. 100% sure

Database - Almost Solved
The 1.1.9 database has been updated correctly by the Upgrade Package.
The mod & edit leftovers still reside in modsettings, in anything prefixed 'tp_' (tinyportal), and possibly elsewhere.

I need to:

• change the modsettings data to a default value
• delete anything prefixed 'tp_'
• find any other leftovers and delete them

Please give me suggestions on how to do task 1 and 3.
Thanks

[Oya]

except you actually can't use totally fresh modsettings values

some of the values relate to the current state of the forum, like the current number of posts total, the last registered member, and so on

you're better off just leaving it

[ElectricSquid]

except you actually can't use totally fresh modsettings values

some of the values relate to the current state of the forum, like the current number of posts total, the last registered member, and so on

you're better off just leaving it

Thank you, it's good to know ahead of time that there are, in fact, a whole slue of values in there that need to be set correctly. Not something a delete or swap would cure.

I must disagree with "just leaving it". This is what I've done for years now, and that causes problems years later when something clashes and crashes and no-one knows why. It's this kind of thing that prevents me from upgrading past 1.1.9 to 1.1.10 or 11.
Right now, during this upgrade to 2.0, I have a once in a decade chance to get rid of ALL the old problems all at once and be 100% sure I did. I can't just get lazy and let this opportunity pass. I know that in a year, I'll kick myself right in the @$$ for just leaving it. So I can sleep well at night, I need a 100% fresh clean slate to work from, and I can't stop at 98% because it's too hard to do the rest.

So help me out here, there has to be some way to do this last 2% without having to kill ourselves at it.
Take a day, think about it.

[Kindred]

see... I just do not get it. leaving the db will have NO effect.... and there are so many settings for individial users in that table that it would be a nightmare to find the mod type stuff ... and why... I have been running a forum on upgrades for 8 years now with no problems... since 1.0

[vbgamer45]

The only real way which I don't advise is to convert to another forum software then convert that software back.

[ElectricSquid]

see... I just do not get it. leaving the db will have NO effect.... and there are so many settings for individial users in that table that it would be a nightmare to find the mod type stuff ... and why... I have been running a forum on upgrades for 8 years now with no problems... since 1.0

... because I've been running this forum for about 4-5 years now, and some of the edits and mods done were from back in a day where I had no clue what I was doing. I want to be sure I'm working with a clean slate. Maybe you haven't had problems because you're a good coder, that's great for you. But I have had problems because at one time, I was a terrible coder. I would like to rid myself of ALL those edits from those days.

The only real way which I don't advise is to convert to another forum software then convert that software back.

LOL, that's an interesting idea.
A bit "outside the box" thinking, I like that
Why is it that you don't advise it?

[Kindred]

because, you almost always "lose" stuff in any conversion.... and a double conversion probably will have more problems than just leaving the fields in there.

and Electric...  it has very little to do with coding.
See, this is where I think we are having a disconnect.
Having extra database tables or database rows will have little, if ANY effect on your site, unless you have something installed which ACCESSES those tables/rows.

So, leaving a database row in place, for a mod which is no longer installed will have NO effect (ok, it might have a effect on the database performance if you had an extra 20 million rows... but you're talking only a few dozen, maybe a few hundred extra)

Clean coding isn't an issue...   coding is done on the FILE side of things. The database just contains DATA

[ElectricSquid]

I'm in total agreement on the file side of things.

On the database side, I realize that the performance hit is probably minimal.
As far as errors, that would/could only happen if on a one in a billion chance that smf decides to use the same name as what is already in the database (as a leftover), or errors could happen after I install a mod (though that will be limited at this point since smf integrated all the mods I would like to add).

I'm just trying to get a 100% clean new install with my old posts intact.
I know it's hard, but I am willing to see it through (correctly).

[Hj Ahmad Rasyid Hj Ismail]

Have you seen it through (correctly) yet?