Multiple Boards (again)

Started by drhamad, December 10, 2004, 03:07:59 AM

Previous topic - Next topic

[Unknown]

As someone else pointed out, the only major problem is that you cannot move topics between one forum and the other.  This can be resolved by modifying MoveTopic.php a bit to ignore the ID_FORUM restriction.

-[Unknown]

drhamad

Quote from: [Unknown] on March 25, 2005, 04:42:47 AM
As someone else pointed out, the only major problem is that you cannot move topics between one forum and the other.  This can be resolved by modifying MoveTopic.php a bit to ignore the ID_FORUM restriction.

-[Unknown]

Yeah... but then, can't do that with our current multi-forum setup either.

I still need to test it out on multiple domains, also, since right now it's just different folders on the same domain.

The big problem is, of course, how to move our current boards to this.

This is what I'm thinking so far:
-Flatten the heirarchy on all boards (phpBB->SMF converter can't handle subforums (child boards)) which is a pain but doable.
-Convert the 4 boards to SMF seperately
-Combine them into the same db w/different prefixes
-Delete the excess tables that need to end up the same (ex: there will be board1_members, board2_members, etc, but they're really all teh same, so delete board2_members)

The hard part is combining the categories, boards, posts.
The cat's and boards can be made to use different id's (mostly they already do, to avoid some phpbb conflicts, but some of the newer ones aren't).  Make sure posts stay associated with the right boards/etc, of course.  Then, post id's will need to be changed.  Luckily, 3 of our 4 boards are quite small, but still, at a couple thousand posts each, definitely needs to be a recurring mysql function that changes them ;)  So I'm thinking, leave the big board's posts alone (240,000 of them).  Then, go to the small boards, and change their post id's to +240,000 for the first, then +242xxx (whatever the next # is) for the 2nd, and so on.
Then, move all the posts into one table.

And of course, all the boards/cats would need to be moved into one table as well.

Does that sound like it would work?  I'm no mysql expert, but it sounds reasonable to me?
FMVperformance:  3.51m posts, 63k members, 11 boards, 1 database

Mazda3Forums - SmallVolvos - MazdaSpeeders Mazda Club - FordFusionClubMazda CX-7 Club - MyMazda6
Now introducing: MKSdrivers.com - FocusDrivers - TaurusDrivers

drhamad

#62
Quote from: [Unknown] on December 22, 2004, 05:21:08 AM
ALTER TABLE smf_members
ADD posts1 smallint(5) unsigned NOT NULL default 0,
ADD posts2 smallint(5) unsigned NOT NULL default 0;

Now, you have to be able to access said post count... so, in Load.php, find this:
IFNULL(lo.logTime, 0) AS isOnline, IFNULL(a.ID_ATTACH, 0) AS ID_ATTACH, a.filename, mem.signature,

Replace it (twice) with:
IFNULL(lo.logTime, 0) AS isOnline, IFNULL(a.ID_ATTACH, 0) AS ID_ATTACH, a.filename, mem.signature, mem.posts1, mem.posts2,

Again, add more if you want more forums.  Now, find this:
'location' => &$profile['location'],

Add after:
'posts1' => &$profile['posts1'],

Rinse and repeat, once again.... then you have to edit Display.template.php, using $message['member']['posts1'], etc.

-[Unknown]


What exactly is this section doing.  Is it keeping the post counts for each forum seperately?
FMVperformance:  3.51m posts, 63k members, 11 boards, 1 database

Mazda3Forums - SmallVolvos - MazdaSpeeders Mazda Club - FordFusionClubMazda CX-7 Club - MyMazda6
Now introducing: MKSdrivers.com - FocusDrivers - TaurusDrivers

[Unknown]

Quote from: drhamad on March 25, 2005, 10:29:56 AM
This is what I'm thinking so far:
-Flatten the heirarchy on all boards (phpBB->SMF converter can't handle subforums (child boards)) which is a pain but doable.

I may be able to change this.  How are they stored in the phpBB database, even?

Quote
-Convert the 4 boards to SMF seperately

Eeek.

QuoteThe hard part is combining the categories, boards, posts.
The cat's and boards can be made to use different id's (mostly they already do, to avoid some phpbb conflicts, but some of the newer ones aren't).  Make sure posts stay associated with the right boards/etc, of course.  Then, post id's will need to be changed.  Luckily, 3 of our 4 boards are quite small, but still, at a couple thousand posts each, definitely needs to be a recurring mysql function that changes them ;)  So I'm thinking, leave the big board's posts alone (240,000 of them).  Then, go to the small boards, and change their post id's to +240,000 for the first, then +242xxx (whatever the next # is) for the 2nd, and so on.
Then, move all the posts into one table.

Theoretically, that should work.  There are a bunch of ID columns though - you'll need to do ID_TOPIC, ID_BOARD, ID_LAST_MSG, and ID_FIRST_MSG on topics; ID_MSG, and ID_BOARD, ID_TOPIC on messages; ID_LAST_MSG, ID_CAT, and ID_BOARD on boards; and ID_CAT on categories.  That's a lot of columns.

Quote from: drhamad on March 25, 2005, 11:33:35 AM
What exactly is this section doing.  Is it keeping the post counts for each forum seperately?

Yes, exactly.

-[Unknown]

drhamad

Quote from: [Unknown] on March 27, 2005, 02:08:50 AM
Quote from: drhamad on March 25, 2005, 10:29:56 AM
This is what I'm thinking so far:
-Flatten the heirarchy on all boards (phpBB->SMF converter can't handle subforums (child boards)) which is a pain but doable.

I may be able to change this.  How are they stored in the phpBB database, even?

Well, officially phpBB does not support child boards.  Which is why I assume the SMF converter doesn't.  Unofficially, I don't think any major phpBB board runs without this:
http://www.phpbb.com/phpBB/viewtopic.php?t=265040&highlight=categories+heirarchy


Quote
Quote
-Convert the 4 boards to SMF seperately

Eeek.

QuoteThe hard part is combining the categories, boards, posts.
The cat's and boards can be made to use different id's (mostly they already do, to avoid some phpbb conflicts, but some of the newer ones aren't).  Make sure posts stay associated with the right boards/etc, of course.  Then, post id's will need to be changed.  Luckily, 3 of our 4 boards are quite small, but still, at a couple thousand posts each, definitely needs to be a recurring mysql function that changes them ;)  So I'm thinking, leave the big board's posts alone (240,000 of them).  Then, go to the small boards, and change their post id's to +240,000 for the first, then +242xxx (whatever the next # is) for the 2nd, and so on.
Then, move all the posts into one table.

Theoretically, that should work.  There are a bunch of ID columns though - you'll need to do ID_TOPIC, ID_BOARD, ID_LAST_MSG, and ID_FIRST_MSG on topics; ID_MSG, and ID_BOARD, ID_TOPIC on messages; ID_LAST_MSG, ID_CAT, and ID_BOARD on boards; and ID_CAT on categories.  That's a lot of columns.

Actually in talking to my admins I've reversed this - do all the manual work on this on the phpbb side, before conversion.  That way only converting once.  Still, a ton of manual work, and a bunch of columns.  We've done it before for Forum and Category ID's, but never for posts.  A recurring function should make it relatively "easy", just gotta make sure we do everything correctly.

Quote
Quote from: drhamad on March 25, 2005, 11:33:35 AM
What exactly is this section doing.  Is it keeping the post counts for each forum seperately?

Yes, exactly.

-[Unknown]

If a board with an ID_FORUM of 0 (shown across all boards) is posted in, it adds post count to the site's your on, I'd assume?  Or to all?

By the way, this is being tested at:
http://www.mazdaowners.com/testforum/
http://development.fmvperformance.com/multiforumtest/
http://developing.fmvperformance.com/
FMVperformance:  3.51m posts, 63k members, 11 boards, 1 database

Mazda3Forums - SmallVolvos - MazdaSpeeders Mazda Club - FordFusionClubMazda CX-7 Club - MyMazda6
Now introducing: MKSdrivers.com - FocusDrivers - TaurusDrivers

[Unknown]

Quote from: drhamad on March 27, 2005, 02:56:35 PM
If a board with an ID_FORUM of 0 (shown across all boards) is posted in, it adds post count to the site's your on, I'd assume?  Or to all?

Ah, sorry, missed that.  Thanks.

updateMemberData($ID_MEMBER, array('posts' => 'posts + 1'));

Replace:
if ($GLOBALS['ID_FORUM'] != 0)
   updateMemberData($ID_MEMBER, array('posts' => 'posts + 1', 'posts' . $GLOBALS['ID_FORUM'] => 'posts' . $GLOBALS['ID_FORUM'] . ' + 1'));
else
   updateMemberData($ID_MEMBER, array('posts' => 'posts + 1', 'posts1' => 'posts1 + 1', 'posts2' => 'posts2 + 1'));


And similar...

I'll look into the category thing.  I took a brief look, and I couldn't see exactly how child boards were stored differently.

-[Unknown]

drhamad

OK so looking over this code more, every time you post:
posts=posts+1
AND
posts1=posts1+1

Correct?

So next, I need to write code to make posts1 visible.  No big deal, I'm just confirming with you that as of right now, nothing actually SHOWS posts1?
FMVperformance:  3.51m posts, 63k members, 11 boards, 1 database

Mazda3Forums - SmallVolvos - MazdaSpeeders Mazda Club - FordFusionClubMazda CX-7 Club - MyMazda6
Now introducing: MKSdrivers.com - FocusDrivers - TaurusDrivers

drhamad

Oooh just came across one problem:

At the moment, the board is showing me as online as both a member and a guest, even though in the 2 boards I'm browsing, I'm logged in on both.
FMVperformance:  3.51m posts, 63k members, 11 boards, 1 database

Mazda3Forums - SmallVolvos - MazdaSpeeders Mazda Club - FordFusionClubMazda CX-7 Club - MyMazda6
Now introducing: MKSdrivers.com - FocusDrivers - TaurusDrivers

[Unknown]

Quote from: drhamad on March 28, 2005, 01:00:31 PM
OK so looking over this code more, every time you post:
posts=posts+1
AND
posts1=posts1+1

Correct?

So next, I need to write code to make posts1 visible.  No big deal, I'm just confirming with you that as of right now, nothing actually SHOWS posts1?

Yes, but it's available after those changes as $message['member']['posts1'] and similar.  You could do $message['member']['posts' . ID_FORUM]...

Quote from: drhamad on March 28, 2005, 01:05:55 PM
Oooh just came across one problem:

At the moment, the board is showing me as online as both a member and a guest, even though in the 2 boards I'm browsing, I'm logged in on both.

Interesting.  That shouldn't happen; I bet they have different session codes.  Hmm... that could get strange.  I wonder if it's theoretically possible to share the session...

-[Unknown]

drhamad

#69
As database driven sessions, wouldn't it be theoretically possible?

Under phpBB it does not let you log in once / be logged in everywhere, however it will not double count you.  The only problem with the phpBB method is that when you switch between forums, it totally loses what you've read / not read.

Oh btw, I'm currently using the default cookie name on all forums - not sure if that matters.

On further testing, it seems like once I log in, the 2nd (guest) copy dissappears... that didn't happen last time.  Not sure what the difference is.


Edit:
It did the 2 user thing again.  Seems like it keeps the Guest on for a little bit at the beginning, then it dissappears.
FMVperformance:  3.51m posts, 63k members, 11 boards, 1 database

Mazda3Forums - SmallVolvos - MazdaSpeeders Mazda Club - FordFusionClubMazda CX-7 Club - MyMazda6
Now introducing: MKSdrivers.com - FocusDrivers - TaurusDrivers

Goosemoose

Quote from: goosemoose on February 18, 2005, 02:21:10 AM
Okay, right now I have three rfc_boards cfc_boards and dfc_boards, I know I need to merge them all into rfc_boards to move things from rfc boards to the other two, but should I leave the existing boards in place so that other functions work properly, or will everything work okay directed to rfc? Meaning post and other stats. Right now most of the other tables are shared and everything counts perfectly.

Any ideas on this? I want to switch to [Unknown]'s method of integrating forums. This is becoming too much of a headache the current way when I have to have 3 seperate upgrades, 3 seperate mod installs and everything else.

drhamad

#71
Quote from: goosemoose on March 30, 2005, 04:09:43 PM
Quote from: goosemoose on February 18, 2005, 02:21:10 AM
Okay, right now I have three rfc_boards cfc_boards and dfc_boards, I know I need to merge them all into rfc_boards to move things from rfc boards to the other two, but should I leave the existing boards in place so that other functions work properly, or will everything work okay directed to rfc? Meaning post and other stats. Right now most of the other tables are shared and everything counts perfectly.

Any ideas on this? I want to switch to [Unknown]'s method of integrating forums. This is becoming too much of a headache the current way when I have to have 3 seperate upgrades, 3 seperate mod installs and everything else.


Well, this method will not really save you from 3 upgrades/etc.  You're still going to be running 3 seperate copies of the software.  The only thing that would carry over between the forums is database changes.  Generally, updates don't make db changes though.  In addition, it means you have to be careful what mods you install, because mods won't know you have 3 forums - you have to make them aware of it, if it's something that affects the board structure, or if it's something that you want each board to act on independantly - you'd have to mod the mod. 

In other words, this would not simplify your process - it would make it more complex.  What this does do is allow you to use the same userdb, banlist, templates, and a few other things among all 3 boards.  It also allows you to setup forums in which posts can be viewed across all 3 boards.

Late edit:  I should mention it also lets you run the boards off different domains, which is the primary reason I do it instead of just having one huge board.
FMVperformance:  3.51m posts, 63k members, 11 boards, 1 database

Mazda3Forums - SmallVolvos - MazdaSpeeders Mazda Club - FordFusionClubMazda CX-7 Club - MyMazda6
Now introducing: MKSdrivers.com - FocusDrivers - TaurusDrivers

drhamad

This is getting to be a ridiculous amount of work :(  Not your fault at all, [unknown]... it's the conversion process that is screwing everything up.

I found a script that merges phpbb db's, so I run that, then I run the phpbb to smf conversion.  Somewhere along the line things are getting screwed up though - posts aren't where they should be /etc.  I haven't figured out if that is happening before or after the conversion though
FMVperformance:  3.51m posts, 63k members, 11 boards, 1 database

Mazda3Forums - SmallVolvos - MazdaSpeeders Mazda Club - FordFusionClubMazda CX-7 Club - MyMazda6
Now introducing: MKSdrivers.com - FocusDrivers - TaurusDrivers

drhamad

Well, SMF better be worth it, because I think I've finally succeeded ;)

I've managed to get all my db's merged, I think (test copies obviously) and I'm about to try the SMF conversion.
FMVperformance:  3.51m posts, 63k members, 11 boards, 1 database

Mazda3Forums - SmallVolvos - MazdaSpeeders Mazda Club - FordFusionClubMazda CX-7 Club - MyMazda6
Now introducing: MKSdrivers.com - FocusDrivers - TaurusDrivers

drhamad

Holy crap... I think it's working.
http://development.fmvperformance.com/smf/
http://development.fmvperformance.com/smf2/
http://www.mazdaowners.com/smf3/

Test copy of the real db, 3 of the four boards that needed to be merged (The last is a bit more complex and didn't feel like dealing with it just now)... everything seems to be working... some messy stuff, mainly from 2 sources - forums that were just links, which SMF doesn't recognize (turns them into regular forums), and some sub-categories that the converter doesn't recognize.  Not big deals.
FMVperformance:  3.51m posts, 63k members, 11 boards, 1 database

Mazda3Forums - SmallVolvos - MazdaSpeeders Mazda Club - FordFusionClubMazda CX-7 Club - MyMazda6
Now introducing: MKSdrivers.com - FocusDrivers - TaurusDrivers

drhamad

So, one problem I've noticed that needs to be rectified is that by default the search function wants to search all boards.  I'm sure there's a way to make it default to searching only id_forum 0 and (whatever board it's accessed from).
FMVperformance:  3.51m posts, 63k members, 11 boards, 1 database

Mazda3Forums - SmallVolvos - MazdaSpeeders Mazda Club - FordFusionClubMazda CX-7 Club - MyMazda6
Now introducing: MKSdrivers.com - FocusDrivers - TaurusDrivers

drhamad

One other problem, this one major:
When I try to move sections and categories, they aren't going where they're told.
FMVperformance:  3.51m posts, 63k members, 11 boards, 1 database

Mazda3Forums - SmallVolvos - MazdaSpeeders Mazda Club - FordFusionClubMazda CX-7 Club - MyMazda6
Now introducing: MKSdrivers.com - FocusDrivers - TaurusDrivers

drhamad

Just wanted to know those couple problems have been solved.  Gettin closer! :)
FMVperformance:  3.51m posts, 63k members, 11 boards, 1 database

Mazda3Forums - SmallVolvos - MazdaSpeeders Mazda Club - FordFusionClubMazda CX-7 Club - MyMazda6
Now introducing: MKSdrivers.com - FocusDrivers - TaurusDrivers

drhamad

If I wanted to make it so ID_FORUM = 5 is accessible from 2 of the 4 boards, could I do:

$user_info['query_see_board'] .= ' AND b.ID_FORUM IN (0, 5, ' . $GLOBALS['ID_FORUM'] . ')';


And use that code on the 2 boards that should see 5?  Any better way to do it?

Anything else I'd have to do?
FMVperformance:  3.51m posts, 63k members, 11 boards, 1 database

Mazda3Forums - SmallVolvos - MazdaSpeeders Mazda Club - FordFusionClubMazda CX-7 Club - MyMazda6
Now introducing: MKSdrivers.com - FocusDrivers - TaurusDrivers

[Unknown]

You could do that, yes.  Another way would be to replace the ID_FORUM column with a comma separated list of columns, and use:

$user_info['query_see_board'] .= ' AND (FIND_IN_SET(' . $GLOBALS['ID_FORUM'] . ', b.showInForums) OR FIND_IN_SET(0, b.showInForums))';

But, it'd probably be simpler to stick with what you already have.

-[Unknown]

Advertisement: