|04-08-2007, 01:44 AM||#1|
Join Date: Jan 2003
Glad to see TMS still going strong, it has been awhile since I last stopped in.
About a month and a half or so ago I returned to semi-active duty at SoI to address some serious lag issues that the game and website were suffering. At the time, we were hosted on a dedicated AMD box - plenty of CPU power, RAM, and hard disk space, or so I thought.
The lag issues were created by a CPU bottleneck - we had been running our website, email, game server and mySQL database backend for all three items on the same box.
Obviously, we needed an upgrade - but as far as hardware goes you can really only upgrade a single monolithic server so far before you're paying $400+ a month in hosting fees for your dual- or quad-core Xeon, heh.
So I decided to take something of the opposite approach, i.e. cobbling together a bunch of cheap, so-called "commodity hardware" into a cluster of servers. Theoretically, with segregated functionality performance should increase, even though each individual server is a lot less powerful than the old dedicated machine.
I spent some time drawing up a workable model, and intiially settled upon a three-server setup. One server would run our two game ports, one server would run our website and related functions such as email, and the third server would house the mySQL server that provides data to the other two over the data center's LAN.
I won't ramble on too much about it; if you're interested in nitty gritty details you can see the original announcement I made about it here.
After migrating to that new setup, most of our lag issues were successfully alleviated; the game server itself was much more responsive and "snappy" with user input.
However, every now and again, we would occasionally be hit with seemingly random spikes of lag. Sometimes they'd last as long as a minute or two at a time, and completely hang the game server.
Performance inbetween the spikes was quite smooth and lag-free; but the spikes themselves were almost more disruptive than the previous all-around consistent lag we had been suffering.
After a couple weeks of headscratching I figured out that the culprit was our website - primarily our forums. With over 175.000 posts and getting close to hitting the 7.000 user mark, the sheer number of SELECT results to format any given page or function on the forum would be pretty brutal on most any database server.
The lag was being created when queries from the game server were backed up due to a surge in queries from the website or the forum.
So, I went back to the drawing board. Again. Heh.
What I came up with is in my opinion a much more technically interesting solution that basically involves adding a fourth server to act a second dedicated mySQL machine. I have our two dedicated mySQL servers set up to replicate one another in a circular replication schema, so that when a change is made to either one it propogates almost instantaneously to the other.
This differs from standard mySQL replication, in which a single master server replicates itself to a slave server; the data cannot be changed on the slave, only read.
The game server then directs all of its queries to one database server, and the website directs all of its queries at the other. Because each is only querying its own dedicated server, we don't hit the same issue with query bottlenecking that we did previously.
Again, the nitty-gritty details can be found in my announcement on our website here.
And sure enough, after I worked out all the server-side kinks in getting the replication set up and working properly today (mySQL is NOT the biggest fan of circular replication!, the lag pretty much disappeared.
I haven't seen the game running this smoothly in a long, long time - funnily enough, I logged into a copy of the live port that's still running on our original monolithic machine, and it wasn't performing nearly as well with just one user online as our live port is under the new setup with over 45 people connected!
Anyway, enough rambling. I decided to post this here in hopes that some other community members might have their own unique hosting architectures they could share, or perhaps have feedback/suggestions/critiques that might assist me in refining our new setup and shortcircuiting any problems that might arise down the road.
Thanks in advance. Happy mudding.
|04-08-2007, 05:07 PM||#2|
I'm running my sql and mud code on a different server then my website and forums. So I think that helps immensely.
Am also using Argila 3.0 but we've altered the code so much it's taking on it's own codebase.
|04-09-2007, 04:33 PM||#3|
If cost is an issue, you might want to look at using virtualization as well. You could have the base O/S running the game environment then utilize a virtual module environment to handle the database and website.
I utilized VMware Server (which is now free) and setup SuSe 10.2 for the base, installed VMware Server upon it and then created a SuSe 10.2 VM solely for running the hosting website and databases. Game accounts screamed and the website/database VM worked just as fast.
One nice thing about VMware is if you have more up-to-date AMD or Intel chipsets, you can actually take advantage of the virtualization modules the newer chipsets have built into them.
|Distributed hosting architectures? - Similar Threads|
|Thread||Thread Starter||Forum||Replies||Last Post|
|MU* Hosting Available||Murpe||MUD Announcements||0||09-12-2004 10:06 PM|
|In need of MUD Hosting||7roc7||Advertising for Staff||31||08-21-2004 01:01 AM|
|MUD Hosting||macdaddy7||MUD Administration||4||01-25-2004 07:25 AM|
|Hosting||re'tak||MUD and RPG Webmasters||0||04-28-2003 02:15 PM|
|areas built then distributed..||Cerise||MUD Builders and Areas||14||05-12-2002 10:44 AM|