Just recently I was asked what it would take to make a web service massively scalable. My answer was lengthy and maybe it is also for other people interesting. So I share it with you here in my blog and split it into parts to make it easier to read. New parts are released on a regular basis. Have fun and your comments are always welcomed!
Part 1 - Clones
Public servers of a scalable web service are hidden behind a load balancer. This load balancer evenly distributes load (requests from your users) onto your group/cluster of application servers. That means that if, for example, user Steve interacts with your service, he may be served at his first request by server 2, then with his second request by server 9 and then maybe again by server 2 on his third request.
Steve should always get the same results of his request back, independent what server he is “landing on”. That leads to the first golden rule for scalability: every server contains the exactly same codebase and does not store any user-related data, like sessions or profile pictures, on local disc or memory.
Sessions need to be stored in a centralized data storage which is accessible to all your application servers. It can be an external database or an external persistent cache, like Redis, whereby you should prefer the last one for performance reasons. With external I mean: not residing on the application servers, but somewhere in or near the data center of your application servers.
But what about deployment? How can you make sure that a code change is sent to all your servers without having one server still serving old code? A tricky problem which luckily already got solved by the great tool Capistrano. It requires some learning, especially when you are not into Ruby on Rails, but it is definitely the effort worth.
After “outsourcing” your sessions and having all your servers serving the same codebase, you can now create an image file from one of these servers (AWS calls this AMI - Amazon Machine Image). Use this AMI as “super-clone” where all your new instances are based upon. Whenever you start a new instance/clone, just do an initial deployment of your latest code and ready you are!