If I install 100 insances of SMF, how will this impact

Started by DenDen60, October 23, 2014, 12:57:52 PM

Previous topic - Next topic

DenDen60

the performance of my server?

Each instance will have:

       
  • 4 to 7 categories
  • 5 to 20 boards, per category
  • 5 to 10 sub-boards by boards, and
  • 5 to 10 sub-sub boards by sub-boards, and
  • maybe a few sub-sub-sub boards by sub-sub boards
I have printed the Twenty-four things you can do to make SMF go faster (Updated June 16th, 2010) post, so I would do this for each forum.

Now, even if I do this, could I have all instances running on only one server or should I have more than one?

If I have all of them on one server, how would this affect the performance of SMF. Would it impact the user. I was told that the number of user does not have an impact on SMF, but the number of boards yes.

I want only one member database. Can this impact the performance of the server?

Thank you for giving me some specific information. It is appreciated.

Arantor

Of *course* it's going to have an impact. It's always going to have an impact. The more work it's doing, the more it's going to have an impact.

If you have say 10 boards per category, 5 sub-boards per board, 5 sub-sub-boards per board, that's 2500 boards before you start. Aside from being insane to manage, you're going to be hitting some quite serious internal performance issues. And doing this 100 times, you're effectively talking 250000 boards. Even 1 post per board is 250000 posts which is immediately into VPS territory and likely into dedicated server territory.

And if you have only one actual member database, you're going to have many, many more problems because that's not the way to go about this, like I already told you multiple times now. You NEED to be sharding this stuff across servers if there's *any* load whatsoever.

I know you're looking for people to do this, but the number of people that can actually handle this is already low enough and you're trying to make it even more complicated without listening to the people who understand the consequences of this.

DenDen60

Quote from: Arantor on October 23, 2014, 01:03:02 PM
I know you're looking for people to do this, but the number of people that can actually handle this is already low enough and you're trying to make it even more complicated without listening to the people who understand the consequences of this.

Arantor, I am listening. This is why I am asking more questions and trying to receive more specific answers

QuoteYou NEED to be sharding this stuff across servers if there's *any* load whatsoever.

Ok, so how do I do this? How many servers would I need? How do we make one registration process work across servers? How does LinkedIn, Facebook and Kijiji do it?


Arantor

No, that's my point, you're not. I have been saying these things for *months* to you.

How many servers? How long is a piece of string? That's the thing: you cannot just randomly predict this stuff in advance. I could pull numbers out of my butt for all the good it would do - because that's about all I *can* do at this point.

100 forums with the described load would be fine on even terrible shared hosting if no-one ever goes there.
100 forums with hundreds of thousands of posts each would be in the server farm territory (i.e. 1 server per every couple of forums, or even one server per forum if they're big enough)

And this presumes you're going to be busy. Every instinct, every thing I have ever seen suggests that you will not be nearly as busy as you think you will be.

In addition, I have never at any point been convinced that SMF was ever the right tool for what you're trying to do but you never gave me enough information to actually figure out if that was so. I have never at any point believed you're trying to run a discussion forum network, but something that's almost better handled by a content management system where the scaling criteria are vastly different. Or even a wiki, where the rules again are different.

vbgamer45

You really need only a vps/server. I am able to host tens of thousands of forums off one webserver and one db server. I had done many different performance tweaks though.
Community Suite for SMF - Take your forum to the next level built for SMF, Gallery,Store,Classifieds,Downloads,more!

SMFHacks.com -  Paid Modifications for SMF

Mods:
EzPortal - Portal System for SMF
SMF Gallery Pro
SMF Store SMF Classifieds Ad Seller Pro

Arantor

If the load is what is described, you're talking the millions of posts category, and I'm not sure your hosting actually has to contend with that sort of load.

But as you seem to have this one covered and clearly know more than I do, best of luck to you.

DenDen60

Quote from: Arantor on October 23, 2014, 01:23:49 PM
No, that's my point, you're not. I have been saying these things for *months* to you.

Maybe it is the other way around Arantor? ;)

QuoteHow many servers? How long is a piece of string? That's the thing: you cannot just randomly predict this stuff in advance. I could pull numbers out of my butt for all the good it would do - because that's about all I *can* do at this point.

100 forums with the described load would be fine on even terrible shared hosting if no-one ever goes there.
100 forums with hundreds of thousands of posts each would be in the server farm territory (i.e. 1 server per every couple of forums, or even one server per forum if they're big enough)

Then problem is I am having different answers to this. The last comment by vbgamer shows this. So where is the answer?  :-\

QuoteAnd this presumes you're going to be busy. Every instinct, every thing I have ever seen suggests that you will not be nearly as busy as you think you will be.
Well maybe, but I do have the potential of being busy and I need to plan for this at the beginning.

QuoteIn addition, I have never at any point been convinced that SMF was ever the right tool for what you're trying to do but you never gave me enough information to actually figure out if that was so. I have never at any point believed you're trying to run a discussion forum network, but something that's almost better handled by a content management system where the scaling criteria are vastly different. Or even a wiki, where the rules again are different.

My dear Arantor, I have been using the Internet since the end of the 80's. I have a MBA in e-Business and I have been an e-Business consultant. I understand the difference between a discussion forum, a content management system and a wiki.

When I started this project 9 years ago, I have checked and tried the available software including Joomla, Drupal, Xoops, and Typo. I also look at SMF competitors including phpBB, Vbulletin and Invision as well as a few others on the Windows environment. I have selected SMF because it meets my needs. Not only does it meet my needs but the proof of concept that I launched back in 2007 proved that having several forums connected with one database and using a small dashboard was possible.

Now the question is can we upgrade the MODs that were done back then and make them work with SMF 2.09? Furthermore, how should I plan for the increase load?


Colin

I haven't been following all of the dialog. Is there any reason you need 100 instances of SMF instead of just using different categories?
"If everybody is thinking alike, then somebody is not thinking." - Gen. George S. Patton Jr.

Colin

margarett

There are other topics of interest, Colin ;)
In short: the user wants to have several (several as in 100s) of SMF instances, one for each community, using a shared central database of users.
Se forem conduzir, não bebam. Se forem beber... CHAMEM-ME!!!! :D

QuoteOver 90% of all computer problems can be traced back to the interface between the keyboard and the chair

DenDen60

Quote from: vbgamer45 on October 23, 2014, 01:25:44 PM
You really need only a vps/server. I am able to host tens of thousands of forums off one webserver and one db server. I had done many different performance tweaks though.

Do you need two machines or just one? At one time do you think I would need two machines? Theoretically of course.

DenDen60

Quote from: Colin on October 23, 2014, 02:10:07 PM
I haven't been following all of the dialog. Is there any reason you need 100 instances of SMF instead of just using different categories?

Hi Collin, thanks for your question. Yes there is a reason. Each community needs to have it's own theme. Furthermore, the user will usually participate in 3 to 5 forums and I want the system to facilitate his movement from one forum to another.

DenDen60

Quote from: Arantor on October 23, 2014, 01:43:03 PM
If the load is what is described, you're talking the millions of posts category, and I'm not sure your hosting actually has to contend with that sort of load.

But as you seem to have this one covered and clearly know more than I do, best of luck to you.

Arantor, don't you think this is possible?

margarett

What I think Arantor is trying to tell you is that is pretty much impossible to predict that... It depends on the amount of users online at the same time and what they might be doing...
The biggest SMF forum (that we know of) is http://warriorcatsrpg.com/
Millions of posts (71+), a lot of users online at once, a real post frenzy and I believe it runs in a single dedicated server. Its specs? No idea :P

If things go according your plan (and we hope so ;) ), your stats might grow this big. Your requirements might vary but this would be a nice comparison for you.
Although, the structure you're defining might cause a bigger stress when compared to this. But again, that depends on users activity...

edit: Denis, I PM'd you yesterday, maybe you missed it?
Se forem conduzir, não bebam. Se forem beber... CHAMEM-ME!!!! :D

QuoteOver 90% of all computer problems can be traced back to the interface between the keyboard and the chair

vbgamer45

Start with a normal server/vps if you get more traffic you can always upgrade.
Community Suite for SMF - Take your forum to the next level built for SMF, Gallery,Store,Classifieds,Downloads,more!

SMFHacks.com -  Paid Modifications for SMF

Mods:
EzPortal - Portal System for SMF
SMF Gallery Pro
SMF Store SMF Classifieds Ad Seller Pro

LiroyvH

Quote from: margarett on October 23, 2014, 02:31:00 PM
The biggest SMF forum (that we know of) is http://warriorcatsrpg.com/
Millions of posts (71+), a lot of users online at once, a real post frenzy and I believe it runs in a single dedicated server. Its specs? No idea :P

Nope, it runs on two with quite the nice specifications.
((U + C + I)x(10 − S)) / 20xAx1 / (1 − sin(F / 10))
President/CEO of Simple Machines - Server Manager
Please do not PM for support - anything else is usually OK.

DenDen60


LiroyvH

You're gonna have to ask the owner. It's been posted here, but in a private area.
Not up to anyone here to decide for her whether or not it'll be posted in public. :)
((U + C + I)x(10 − S)) / 20xAx1 / (1 − sin(F / 10))
President/CEO of Simple Machines - Server Manager
Please do not PM for support - anything else is usually OK.

DenDen60

Maybe the best way to do this is to test it. 8)

Now I can first test  with dougiefresh split forum mod. Then I could do it with separate instances.

JBlaze

Denis, you're better off posting this job at a site like freelancer.com or the like. I highly doubt anyone here is willing to take on a task of this size and difficulty. It's something that will require too much modification of SMF's internals to the point that it would require more than one person working on it.

Basically, you're wanting to write your own OAuth implementation for a very limited use. Something that requires multiple servers to accomplish (at least two, one for your members and authentication, the other for the multiple forum instances).

You are not going to get any more information than has already been provided to you.
Jason Clemons
Former Team Member 2009 - 2012

DenDen60

#19
Please view added text at the end.

Zilladotexe, I will if I need to. :( However, I would prefer to encourage people who volunteer on SMF  8) than to go to freelancer.com. How many people on freelancer.com have an understanding of how SMF works? I am sure that there are programmers here who can do the job.

If developers here can develop this great software, or develop a portal, ad management system and other MODs,  than they have the ability of creating this small but trickcy MOD. Please remember that what I am asking for as already been developed for SMF 1.1 version. Now I understand SMF2 is different then SMF1, but still, it has been done before and by one person.   

Finally, I do need to specify that at the beginning there will be a maximum of 25 forums created.  However, as the need grows, I will need to expand.

Added text

Furthermore, we could have a mix of solutions. I could use Dougiefresh split forum MOD to create board clusters with 10 or 12 instances of SMF linked together for the Members database.

Al this is to say that I am open to suggestion.  O:)

Advertisement: