Q&A Platform System Design
The prompt sounds like a weekend project: "Design a question-and-answer platform like Stack Overflow or Quora. People ask questions, others answer, anyone can comment, and votes push the best answers to the top. Then we'll talk about what changes when comments nest like Reddit's."
Everyone can draw the first version: a questions table, an answers table, a votes table, and a web server in front. That version is correct, and it is about a tenth of the answer. It renders every page from the database, even though four out of five readers are anonymous visitors from a search engine who all want exactly the same page. It counts votes with score = score + 1, so a retried request or a second browser tab counts twice. It loads a 50,000-comment thread with a recursive query on every view. And it lets people post the same question for the hundredth time, because nothing showed them the ninety-nine answers already written.
The Trap Built Into a Familiar Question
Two failures show up again and again:
- The CRUD App: draws the three tables, adds read replicas "for scale", and stops. Nobody asked what happens when one question is linked from a newsletter and gets 280 views a second, why a vote is
PUTand notPOST, or how a page can be cached for everyone when every signed-in user sees their own vote arrows. Signal: knows the entities, hasn't thought about where the load lands. - The Planet-Scale Everything: shards the votes into Cassandra, counts them in Redis, streams every page view through Kafka and puts a graph database behind the comments. Each piece sounds senior. Together they lose votes on failover, make "one vote per person" impossible to enforce, and spend a fortune on a write load that peaks at about 400 writes a second. Signal: reaches for machinery before doing the arithmetic.
The passing performance starts from two numbers. Three hundred million page views a day is about 10,500 page views a second at peak, but almost all of them can be the same cached HTML, so with an 85% edge hit ratio the origin renders only about 1,600 pages a second and the database sees a few thousand cheap queries. Meanwhile all writes together peak at around 400 a second, and 86% of those are votes. So this is a caching problem first and a correctness problem second, and the design follows: one page for everyone with a tiny private "viewer state" call beside it, a relational store sharded by question so a page and a vote each touch one shard, a vote table that is the truth with the score as a cache of it, and an event stream that feeds search, reputation and ranking without slowing anyone's click.
Sign in to continue reading
The rest of this lesson is available with a free account. Signing in with Google or Microsoft is free.
Sign in to read the full lesson