Filed under: Uncategorized | Tags: Scaling, Social Networking, Web Development
Okay, so i’m involved in a project where i have to scale wordpress MU to 50,000+ users. I’m new to this whole scaling game, but i do get the bottom line of it and i have enough skills to make it happen. So i did a bunch of reading, and here are the options which i found out of googling for an hour. PLEASE, we NEED FEEDBACK 🙂
1: The most simple solution was presented in this MU support thread, As showen in the diagram on the side. His main stratagy is to install “out of the box” MU on a whole bunch of machine, and to find a way to distrubte the traffic between all of them using apache rewriting rules. To achive that, he is using a local DNS which would be a MUCH better idea than using IPes. This idea is simple to implement and it requires NO messing around with any code.
2: The second way (dont have a picture for it yet) which i’m thinking of doing is a bit more involved but it does make more sense to me. So, you will have a load balancer in the front, distributing the load among X amount of web servers. The webservers will have nothing but apache and MU on them. They will store no information other than sessions. Mounted to these servers, we’re going to have file servers which have LOTS of space on them, this is where all the user files will be kept. And finally the DB’s, this is where most of the work is going to happen. THe idea is to create LOTS of databases, and to determine what user should be using what DB based on thier user name. Best explained by Matt.
It would also be possible to have some sort of domain based key with random distribution, for example you could take an MD5 hash of the incoming username, then use the first character of that hash to go to one of 16 clusters or databases. If you grow beyond that start using the first two characters, which increases your number of machines by a factor. Then when you grow more use the first 3 characters… 🙂
The second way requires fiddling with the source code. Which i dont mind doing, but i just want to make SURE that i HAVE to. 🙂
2 Comments so far
Leave a comment
Virus In Training 
This is ridiculously complex, for 50k blogs you can run on 2-3 servers no problem. (We did.)
Comment by Matt February 7, 2007 @ 7:35 pmPlease join for discussions on topics about The Virtual Reality.
Comment by Gnoletcom April 13, 2007 @ 9:22 pmTechnology development of The Virtual Reality and its perspectives.
Gnolet.com