Scaling to millions on AWS
Take a service from one box to millions of users, one bottleneck at a time — the whole primer applied in order.
- Start with a single server
- Traffic grows steadily
- Keep costs proportional to load
- Minimize downtime during each step
- Measure before each change
Web server, application and database on a single machine, with DNS pointing at its IP. This works longer than people expect, and every next step should be justified by a measurement rather than a prediction.
Gather requirements and scope the problem. Ask questions to clarify use cases and constraints. Discuss assumptions.
Without an interviewer to address clarifying questions, we'll define some use cases and constraints.
Use cases
Solving this problem takes an iterative approach of: 1) Benchmark/Load Test, 2) Profile for bottlenecks 3) address bottlenecks while evaluating alternatives and trade-offs, and 4) repeat, which is good pattern for evolving basic designs to scalable designs.
Unless you have a background in AWS or are applying for a position that requires AWS knowledge, AWS-specific details are not a requirement. However, much of the principles discussed in this exercise can apply more generally outside of the AWS ecosystem.
We'll scope the problem to handle only the following use cases
- User makes a read or write request
- Service does processing, stores user data, then returns the results
- Service needs to evolve from serving a small amount of users to millions of users
- Discuss general scaling patterns as we evolve an architecture to handle a large number of users and requests
- Service has high availability
Constraints and assumptions
State assumptions
- Traffic is not evenly distributed
- Need for relational data
- Scale from 1 user to tens of millions of users
- Denote increase of users as:
- Users+
- Users++
- Users+++
- ...
- 10 million users
- 1 billion writes per month
- 100 billion reads per month
- 100:1 read to write ratio
- 1 KB content per write
- Denote increase of users as:
Calculate usage
Clarify with your interviewer if you should run back-of-the-envelope usage calculations.
- 1 TB of new content per month
- 1 KB per write * 1 billion writes per month
- 36 TB of new content in 3 years
- Assume most writes are from new content instead of updates to existing ones
- 400 writes per second on average
- 40,000 reads per second on average
Handy conversion guide:
- 2.5 million seconds per month
- 1 request per second = 2.5 million requests per month
- 40 requests per second = 100 million requests per month
- 400 requests per second = 1 billion requests per month