News:

Bored?  Looking to kill some time?  Want to chat with other SMF users?  Join us in IRC chat or Discord

Main Menu

Copy a Category and all it's posts from one SMF to another SMF

Started by jjsmith, June 05, 2013, 04:14:19 PM

Previous topic - Next topic

jjsmith

Hi,

I would like to copy all the posts from a single Category (with three boards) in one SMF installation to another SMF installation. 

Both are ver 2.0.4. 

The destination boards have many categories/boards - but the names of the Category and boards to be copied are not in the destination.

Is there a "standard" utility/mod/process/other that will do this?

I realize that one major issue with this would be how to identify the post owner in the new board.  What I would do is add all those members to the destination board manually before copying the posts - there's only about 10 users.

Background - I just have about 100 posts that were added into a "temp" SMF installation, and I'd just like to add those to an existing SMF installation.

Thank you!

Josh


Burke ♞ Knight

If you have phpMyAdmin access, you can export the topics and messages then import them into the other site.

jjsmith

Thanks.  Yep I do have phpmyadmin and am comfortable with sql.

But - won't there be conflicts with all the id's - eg the id's for message, topic, board and category? 


Burke ♞ Knight

The id's will be in it. However, the member might not. It might re-attribute the posts to someone who is, if that member number is used. Also, there is a mod to re-attribute topics and posts to other members. ;)

Colin

Quote from: jjsmith on June 05, 2013, 05:01:55 PM
Thanks.  Yep I do have phpmyadmin and am comfortable with sql.

But - won't there be conflicts with all the id's - eg the id's for message, topic, board and category? 



Goodness, yes, please don't do that. You would need to modify all of the ids for the topics and posts to make sure they are unique in the new database as well. Also the reference to the authors in the database would be messy. The entire ordeal would be large and messy. Unfortunately, I advise against it.
"If everybody is thinking alike, then somebody is not thinking." - Gen. George S. Patton Jr.

Colin

Colin

Quote from: BurkeKnight on June 05, 2013, 05:22:57 PM
The id's will be in it. However, the member might not. It might re-attribute the posts to someone who is, if that member number is used. Also, there is a mod to re-attribute topics and posts to other members. ;)
This is not correct, at all. Sure there will be id's in the database, but they will overlap with the new database.
"If everybody is thinking alike, then somebody is not thinking." - Gen. George S. Patton Jr.

Colin

Burke ♞ Knight

Only if there are existing topics.
I was under the assuption that it was being moved to new clean forum.

Kindred

ther eis no good way to do this, unfortunately...

You could TRY:
export the contents of the messages table to a text file
write a script which calls create_post and adds the message to the new forum

but you'd have to write the script... I don't believe that anyone has done so successfully.
Слaва
Украинi

Please do not PM, IM or Email me with support questions.  You will get better and faster responses in the support boards.  Thank you.

"Loki is not evil, although he is certainly not a force for good. Loki is... complicated."

jjsmith


@Kindred - ooh ooh - that sounds promising ...

See ... this process is a trial run for my next task - migrate *lots* of content from an existing Fusetalk boards to the existing SMF boards.

So what I'm thinking is ...  Use the registerMember() and the createPost() functions to get it done - as follows ....

- Extract the following from my source forum (smf, fusetalk).  Each extract is one temp db table:
  - Members - the extract will include email address and userid
  - Posts - each row to include category name, board name, subject, userid, post body

- Use the Members extract and the registerMember() function to register all my members.  The return value of this function is the newly created smf user id, so for each registered member, create a new temp table with: email addr, source userid, and newly created userid

- Use the Posts extract and createPost() to add all the posts.  I would be able to get the new userid from the table output from the previous step.

I suspect I'm missing some (lots of?) things.

Thoughts????

Thank you.


Here's some sql that would get me an extract of posts from my source SMF

SELECT a.name AS cat_name, b.name AS board_name, c.subject, c.poster_name, c.body,
   a.id_cat, b.id_board, c.id_topic, c.id_msg, c.poster_time
FROM smf_categories a
INNER JOIN smf_boards b
ON b.id_cat = a.id_cat
INNER JOIN smf_messages c
ON b.id_board = c.id_board
ORDER BY cat_name, board_name, c.id_topic, c.poster_time
--------------
registerMember() - http://support.simplemachines.org/function_db/index.php?action=view_function;id=594 -
createPost() - http://support.simplemachines.org/function_db/index.php?action=view_function;id=323


Biology Forums


margarett

This looks like a *VERY* interesting topic. Really interesting, indeed!

You should look for how the actual converters work, because it's pretty much what you want to do.

Good luck!
Se forem conduzir, não bebam. Se forem beber... CHAMEM-ME!!!! :D

QuoteOver 90% of all computer problems can be traced back to the interface between the keyboard and the chair

jjsmith


Margaret, Liam - Thank you for taking an interest!

Ok - after a bit more research, this is gonna be a few more steps than I thought ... But I've also increased my scope a little ...

I'm compiling my analysis and will post it here - but that will take a little bit .... 

In summary, I think my tasks are ...

A) Resolve duplicates
B) Extract source data: members, categories, boards, posts, attachments
C) Migrate the data to destination:
- Add members
- Add categories
- Add boards
- Add posts
- Add attachments

I've got most of this analysis hammered out, but duty to other tasks calls .... Will get back to this later in the day and post my analysis ...

Josh

jjsmith

Ok - here's my high level analysis ...

Would appreciate any insight :-)

(Sorry for this post length - Don't read this if you're sleepy!  :-)



Summary ...

A) Resolve duplicates
B) Extract source data: members, categories, boards, posts, attachments
C) Insert the data to destination:
- Add members
- Add categories
- Add boards
- Add posts
- Add attachments (still thinking on this one - might just not import attachments)

NOTE - This is only going to migrate the above data - not an entire forum.


Open questions ......

- In createPost() - How to identify that the post is the first for that topic, or a reply?  I'll assume it's just the timestamp.

- Which to insert first - attachment or post?  Each create...() function requires the id of the other.

- Questions about resolving duplicate attachment filenames - is my process under "5) Attachments ..." correct?

.. I need to understand more how SMF handles attachments.  Are all attachments stored as files in the /boards/attachments directory and then referenced via smf_attachments.filename?  If so, how does SMF resolve duplicate filenames normally when attachments are added? 


........... Detail ............


A) Resolve Duplicates ........

- Members - see task under "1) Members ...." below.

- Categories, Boards - There are only about 10 categories and about 30 boards that I'm migrating from Fusetalk, so I'll just manually verify that there are no categories or boards with the same name in the destination.  If there's a duplicate, I'll just manually rename either the source or destination.

Note - Duplicate Categories and Boards *are* allowed, I just don't want to have any dupes.

- Posts - Not applicable.

- Attachments - Ughh - Unresolved as yet - see questions above. Also see task under "5) Attachments ..." below.  Baring any weird things, I think the steps in this task will work fine.



B) Get source data ...

Extract from my source as temp mysql tables:
- All members
- All Categories I want to migrate
- All Boards from those categories
- All Posts from those boards
- All Attachments for those posts

Posts...

- Make sure the Posts extract is in the following sorted order: Category, Board, topic id, dt/time.
- In createPost(), if the topic id already exists in smf_topics, SMF will not insert a new topic (this is as expected).  However, since probably all our exported topic id's will already exist in the destination, this will cause a problem, because *all* our topics are new.  So, I'll just manually get the current max topic id in destination, add 1000 to it for good measure, and use sql to add that to every topic id.
So....
select max(id_topic) from smf_topics;  -- currently this is 67719
update posts_temp set dest_topic_id = src_topic_id + 69000;

Attachments...

- Remember that Posts<>Attachments is 1<>many.
- This means that my attachments extract will have one or more rows for each post.  This will be used the create the array of attachment-ids that are used in createPost().



C) Migrate the data ....


1) Members ....

- Adjudicate the duplicate member id's between source and destination. 

I will do this based on email address.  Look up source email address in destination (table smf_members) and if there, retrieve the member id. 

Add this new member id column to the source posts table.

- registerMember()

Register all members from the source that aren't in the destination.  This returns the newly created id.

Again, add this new member id to the source posts table.



2) Categories ....

- createCategory()

This does *not* return the newly created category id.  So ...

- Use my temp source table to retrieve the id of all the newly created categories and add that column to my source posts table.



3) Boards ....

- createBoard()

This does return the newly created board id - so add this column to my source posts table as well.



4) Posts ....

- createPost()

My source posts table will now have all my info for the insert:
- new id's for member, category, board, topic
- source body, subject

NOTE - This function returns a bool for whether the function was successful - it doesn't return the id of the newly created post.  HOWEVER - that id *is* inserted into the '$msgOptions' array, which is a parameter to the function.  So I can get that id for the createAttachment() function.




5) Attachments ...

- Add column "destination_filename" to my Attachments temp table.  Using the source_filename in the Attachments temp table, see if that filename exists in 'smf_attachments.filename'.  If so, insert the destination_filename with dt/tm appended to it as destination_filename.  Otherwise just insert the source_filename as the destination_filename

- Using this updated Attachments temp table, copy all those files from /boards/attachments to /boards/attachments-temp with the destination_filename.


Have a gin and tonic :)


Koppo

That topic is really old, but I am currently struggling with the same question.
So... Did you manage to do this?
My idea was to completely clone my current forum to a new domain, exchange the links in it if necessary, then remove everything until only the category is left that I need.
That would fix the issue with members etc...

jjsmith

Alas, I never did do this ... not enough hours in the day/week/month/... I ended up just not migrating that data at all - the board admins wanted to start using the board right away and so they were ok with that.

For your solution ... if you want the new board to have only that one category, then you idea should work.

My requirement was to copy posts to an existing board - so I couldn't just export/import the data using sql because id's would be duplicated, so I would have had to write a php program that used the SMF functions to insert the posts into the new board.


Irisado

Thanks for answering this.  Given the age of the topic, it can now be closed.  Koppo, for future reference, it's usually better to start a new topic, rather than to revive a really old one.
Soñando con una playa donde brilla el sol, un arco iris ilumina el cielo, y el mar espejea iridescentemente

Advertisement: