System design practice

Flash Sale

hard Based on a system published by Shopify rate-limiting caching consistency spiky-traffic real-world

Shopify's flash sales: cache at the edge, throttle checkout, never oversell.

Solve it in your browser Read the lesson first

When a celebrity launches a product on Shopify, a crowd that is waiting for the moment hits one shop at once: tens of thousands of requests a second, most of them for the same page, then a rush on checkout for a stock that is gone in minutes. In early sales, checkout writes took down not just the shop but whole segments of the platform with it.

Shopify's answer was to shape the load before it reaches the database. The load balancers serve cached pages, and a throttle at the edge lets only as many buyers into checkout as the database can serve; the rest wait on a queue page. Checkout itself must never sell a unit twice, and an order must exist before the buyer is told it was placed.

Design the sale for one shop.

Functional requirements

Use these use case and scenario names exactly: the traffic, requirements and tests in problem.proschi refer to them.

Scale

Constraints

The simulation sees the hot inventory rows only as the lower write capacity the problem sets for the pod; it does not model lock waits. Because the problem sets the pod's capacity, it cannot be split into shards here (capacity { pod shards N } is a duplicate): a shop's data lives on one pod, and a flash sale is no time to move it.

What is given

problem.proschi declares the buyer and pod, the shop's MySQL shard with its write limit, and holds the traffic, requirements and tests. Add the edge (load balancers with a page cache and the throttle), the app servers, the connections and the two use cases.

Unlike Ticket Booking, buyers do not pick a particular item and hold it: the problem is a crowd twenty times bigger than checkout can take.

Based on

How your design is checked

You write the design as text in Proschi. Tests run in your browser: a simulation of the traffic above checks latency, availability, cost and what happens when a machine fails. How the simulation works.

Start designing

Lesson · 12 min read

Learn it: Flash Sale #

Flash sale: shaping a crowd before it reaches the database #

What you'll learn #

The problem, explained #

A celebrity launches a lip kit on Shopify. Hundreds of thousands of fans arrive at once: tens of thousands of product-page views a second, then a wall of checkout attempts for a stock gone in minutes. In early flash sales the checkout writes knocked over not just the shop but other shops sharing its database.

There are two use cases:

ScenarioWhat happensAnswer
"Queued"Checkout is full: a queue page polls again in a few seconds; once admitted, a signed cookie skips the queue429
"Order placed"A unit is taken from the stock and the order stored201
"Sold out"Admitted, but too late409

The non-functional requirements: never sell a unit twice or record a sale without its order, and make the order durable before 201. Buyers who wait never reach the app or the database. p99 is under 50 ms for the page and 150 ms for a placed order. Pages are available 99.99% of the time, checkout 99.95%. The sale survives losing any machine and costs at most $2,000 a month, database included.

The given fixes the pod: the shop's MySQL shard, a primary and a replica. Its write capacity is lowered to 2k writes a second: every checkout hammers the same few inventory rows, and a real database spends its time waiting on row locks, which the simulation expresses as lower capacity. Nor can you add shards: a shop lives on one pod, and you do not move it mid-sale. The only way out is to send less load.

The tests check:

This is the sibling of Ticket Booking with one big difference: buyers do not pick a seat. Everybody wants the same thing, which turns a set of rows into one hot counter and the crowd into the main enemy.

Back-of-the-envelope #

arrive at the edge
90k rps
pod writes once admitted, ≈ 43% of 2k
850 / s
pod writes with no throttle
≈ 17k / s
reach the app
4.5k rps
of the $2,000 is the pod
$800
FlowRateWhere it ends
Page views, cache hit (95%)76k rpsedge only
Page views, cache miss (5%)4k rpsapp + 4k pod reads
Checkout, queued (95%)9.5k rpsedge only
Checkout, order placed (3.5%)350 rpsapp + 2 pod writes each = 700 writes
Checkout, sold out (1.5%)150 rpsapp + 1 pod write each = 150 writes

The pod. Admitted checkouts produce 700 + 150 = 850 writes a second against 2k: about 43% utilisation. With no throttle, all 10k checkout attempts reach the app, and with the same split the pod would see on the order of 17k writes a second, eight or nine times its capacity. No number of app servers fixes that; with an unshardable pod, the only lever is admitting fewer buyers.

The edge. Everything arrives here: 80k + 10k = 90k rps. A simulated load balancer takes 100k rps per replica, so even a small fleet sits around 30%: answering here is nearly free.

The app sees only page misses plus admitted checkouts: 4k + 500 = 4.5k rps. At about 2k rps per replica that is 2.25 replicas at 100%; with a ~70% target and room to lose one, a handful. Rendering every page in the app takes 80k ÷ 2k = 40 replicas at 100%, well over 50 with headroom, at $100 each: that alone breaks the $2,000 budget.

Where the time goes. A page-cache hit is one edge hop of about 2 ms, a few milliseconds at p99. A placed order is edge → app → two pod writes → back: roughly 2 + 10 + 2 × ~8 ms of mean latency. The model's p99 for such a path is a few times its mean, so it fits under 150 ms only if neither the app nor the pod is near saturation.

Cost. The pod's two MySQL replicas are $800 of the $2,000; load balancers are $50 each, app replicas $100. "Cache the pages and throttle checkout at the edge" fits comfortably; "scale the app" does not.

Concepts #

Admission control and virtual waiting rooms #

Admission control means deciding at the door how much work you accept, so the accepted work finishes quickly. Accepting everything lets every request slow down together until timeouts start, retries pile on and the system collapses. Under overload, serving 500 buyers a second and politely telling 9,500 to wait beats trying to serve 10,000 and serving nobody.

A virtual waiting room is admission control with a user interface: a page that says "you are in line", polls, and lets you through when there is room. The admitted buyer gets a signed token (Shopify used a signed cookie) so the next step does not throttle them again.

Deep dive Leaky bucket or token bucket

The throttle itself is usually a leaky bucket: requests fill a bucket that drains at a fixed rate (the rate checkout can handle); when it is full, new arrivals are queued or rejected. A token bucket is the mirror image (tokens refill at a fixed rate and each request spends one) and allows short bursts.

When not to use it: when demand never exceeds capacity by much, a waiting room is friction for nothing; autoscaling or a per-client rate limiter is enough. Nor does it replace correctness: the throttle limits load, the database still prevents overselling.

Deep dive The throttle in Proschi
title "Throttle at the edge"
user "User" [Actor]
edge "Edge" [nginx] x2
app  "App"  [REST API] x2
user -> edge
edge -> app

usecase "Enter" {
  user -> edge : POST /enter
  alt "Wait" when "the bucket is full" {
    edge --> user : 429 waiting page
  } alt "Admitted" when "the bucket has room" {
    edge  -> app  : POST /enter
    app  --> edge : 200
    edge --> user : 200
  }
}

Caching at the edge #

When thousands of people a second request the same page, the cheapest place to answer is the first box they hit. A load balancer with a page cache (nginx can do this, and Shopify did) or a CDN keeps rendered pages briefly and answers repeats without the app.

The trade-off is staleness: a cached page can show "in stock" for a few seconds after the last unit sold. That is fine because the decision is made at checkout, not on the page. Keep personalised fragments (cart count, login state) out of the cached page, or load them separately.

When not to use it: for pages that differ per user, or where stale data is itself harmful (say, prices that legally must be exact at display time).

The stock decision: one store, one transaction #

To avoid overselling, decrement the stock with a conditional write in the database that also stores the order:

UPDATE inventory SET available = available - 1
WHERE variant_id = 42 AND available > 0;
-- 1 row: the unit is yours, INSERT the order and COMMIT
-- 0 rows: sold out

The condition available > 0 is checked and applied atomically, so two buyers cannot both take the last unit. With the decrement and the order insert in the same transaction, a crash never leaves a unit taken without its order, or an order without its unit.

The price of keeping it in the database is the hot row: every checkout locks the same inventory row, so the effective write rate drops far below the usual capacity. Hence the pod's 2k writes a second, and why admission control is what makes the database approach viable.

Designing it step by step #

1. Scope. One shop, one product drop, one unit per checkout, no seat selection. Confirm the guarantees (no overselling, no unit without an order, order durable before 201) and the numbers: 80k page views and 10k checkout attempts a second, checkout capacity about 500 a second, the pod's 2k writes.

2. High level. Buyers hit an edge tier (load balancers), which forwards to the app servers (the Rails monolith), which talk to the pod. Draw both flows end to end without optimisation, then ask of each arrow "how many requests a second cross this?" The answers (90k to the app, about 17k writes to the pod) show the naive design fails by an order of magnitude.

3. Deep dive. Take the load away tier by tier.

Pages. Most of the 80k views are one page. Cache it in the edge, where a hit calls nothing; only misses reach the app, which reads the pod and returns the page for the edge to keep.

Checkout admission. Add a leaky bucket in the edge sized to checkout's capacity. Buyers over the limit get a 429 with a queue page (also cached in the edge) that polls again; admitted buyers get a signed pass. Explain why it must be the edge, not the app.

The stock. For admitted buyers, the app runs the conditional UPDATE, then inserts the order, in one transaction on the pod. One row changed → order placed; zero rows → sold out. No cache or Redis on this path.

Sizing and failure. Size the edge for 90k rps with one replica lost and the app for misses plus admitted checkouts with headroom; check the pod stays well below 2k writes, and check the bill.

4. Wrap up. Mention fairness (Shopify's queue compared the timestamp of a buyer's first attempt so early arrivals were not starved), bot protection, isolating the shop's pod so a sale spares other shops, and monitoring the admission rate to tune the bucket mid-sale.

Common mistakes #

No throttle (wrong/no-throttle). Everyone enters checkout, and the design doubles the app servers to cope. The app can take it; the pod cannot: thousands of writes a second land on hot rows, lock waits pile up, and in production the shard falls over, taking neighbouring shops with it. Caught by "The throttle keeps the crowd out of checkout" (there is no "Queued" scenario) and p99 of Checkout scenario Order placed < 150 ms, because the saturated pod fails every latency requirement that sends it load.

Throttle in the app (wrong/throttle-in-the-app). Right idea, wrong place: every rejected buyer still costs an app request, so 9.5k rejections a second swamp the app tier, and page misses queue behind them. Caught by "The throttle keeps the crowd out of checkout" (the queued scenario calls a service) and p99 of View product < 50 ms.

Uncached storefront (wrong/uncached-storefront). Render every page in the app, maybe with an in-app fragment cache. It works with dozens of app servers: around 56 replicas to stay out of saturation. Caught by "Product pages are served from the edge cache" and cost ≤ $2,000/month.

Stock in Redis (wrong/stock-in-redis). DECR in Redis, order in MySQL: fast, until a crash or a failover makes them disagree, with units sold without orders or more orders than units. Caught by "The pod's database decides the stock and records the order" (checkout calls an eventual store).

Classic mistakes beyond the tests:

In the interview #

Likely follow-ups:

Further reading #

Now design it

Review cards for this problem

Short questions on the ideas it needs, from the review cards. Review them daily so they stick.

More system design problems