Numbers to know

A one-page cheat sheet of the round numbers a system design interview runs on: how long things take, how much one machine does, how big things are and what they cost. None of them is exact, and none needs to be. What matters is the order of magnitude: whether a number is a thousand or a million decides whether one machine is enough or you need fifty. Every number below is rounded, so read each one as "about".

Where Proschi's simulation has a default of its own, the section says how it relates; How the simulation works lists every default. To see the numbers used in a whole interview, read How to approach a system design interview, and to drill them, the estimate cards in daily review.

Latency #

How long one operation takes. These are typical figures for current server hardware and cloud networks; the classic table they come from is Latency numbers every programmer should know.

How to use it:

• Memory is about 1,000 times faster than an SSD read, and an SSD read is about 100 times faster than an HDD seek. That is why hot data lives in a cache and why random reads from spinning disks are avoided.
• Light in fiber covers about 200 km per millisecond, so each 100 km of distance adds about 1 ms to a round trip. No hardware fixes that: a request that crosses an ocean pays 100 ms or more however fast the servers are.
• A new connection costs round trips before the first byte. TCP plus TLS 1.3 plus the request is 3 round trips: 450 ms for a user 150 ms away. Keep connections open and terminate TLS at an edge near the user.

Proschi's defaults sit in the same range: 1 ms for a cache, 5 ms for a database, 10 ms for a service, 30 ms for object storage and 200 ms for a third-party API, per call when the node is idle; queueing adds more as it gets busy.

Throughput and capacity #

How much one machine or one partition does. These are order-of-magnitude rules of thumb, not benchmarks: real numbers vary several-fold with the hardware, the request size, the query and the configuration. Use them to decide between one, ten and a hundred machines, then measure.

Servers and data stores #

Divide the peak load by what one node does, keep each node well below full (50–70% busy, because latency climbs steeply near 100%), and add a node or two so that losing one still leaves enough. Reads scale by adding replicas; writes to a single-primary database do not, which is when you shard.

Proschi's per-replica defaults are inside these ranges: a service 2,000 requests/s, a cache 100,000, a relational database 20,000 reads and 5,000 writes per shard, a queue 50,000 messages/s (all defaults).

Disks and networks #

Networks are measured in bits per second and files in bytes: divide Gbit/s by 8 to get GB/s. A cloud VM's network is usually "up to" a figure, from about 1 to 25 Gbit/s by instance size, and a cloud volume's speed is what you provision, not what the disk underneath could do.

Proschi gives each replica a bandwidth: 200 MB/s for a service (about 1.6 Gbit/s), 1,000 MB/s for load balancers, gateways and CDNs, and 100 MB/s for everything else.

Connections #

A connection held open costs memory, not CPU, so long-lived connections (chat, push, WebSockets) are sized by memory: plan on tens of thousands per server. A database connection is expensive (in PostgreSQL, a process each), so services share a small pool, and a pooler such as PgBouncer sits in front when there are many services. By Little's law, the connections in use = requests per second × seconds each holds one.

Time and conversions #

Seconds in a day, month and year #

Powers of two and ten #

KB, MB, GB, TB and PB are powers of ten (a GB is 10⁹ bytes); KiB, MiB, GiB, TiB and PiB are powers of two. The gap grows with the size: a GiB is 7% more than a GB, a TiB 10% more than a TB, a PiB 13% more than a PB. For estimates, treat them as equal; for a disk you are buying or a bill, check which one is meant (operating systems and some cloud prices use GiB). A byte is 8 bits, and 2³² is about 4.3 billion, the number of 32-bit values.

Per day to per second #

Divide a daily count by 100,000 for a quick answer (about 14% low, which rounding forgives), and a monthly count by 2.5 million. Then size for the peak, not the average: peaks are usually 2–5 times the average, and say which multiple you chose.

Availability nines #

Each extra nine divides the downtime by ten. Components a request needs in a chain multiply their availabilities (three at 99.9% give about 99.7%); independent replicas multiply their downtime (two at 99% give 99.99%). Proschi computes a use case's availability the same way, from each node's per-replica availability.

Sizes #

A minute of anything streamed is its bitrate × 60 ÷ 8: Mbit/s × 7.5 = MB per minute. Storage then multiplies out: items per day × bytes per item × days kept × replicas. A billion items of 1 KB is a terabyte; video services store several resolutions of every video.

Cloud costs #

Approximate public list prices of the big cloud providers in US regions, rounded; they change, and they differ by provider, region, volume and contract. Check the provider's pricing page before you rely on one. They are here for orders of magnitude: which part of a design dominates the bill.

A month has about 730 hours, so a VM at $0.10 an hour is about $70 a month. Two patterns show up in almost every design: sending data costs more than storing it (a GB sent out once costs about as much as keeping it in object storage for a few months), and cold tiers are an order of magnitude cheaper for data that is rarely read, at the price of slower and pay-per-read retrieval.

Proschi charges a flat monthly price per replica (a service $100, a cache $150, a relational database $400, object storage $50) plus egress at $0.09/GB from a node you run and $0.02/GB from a CDN. Those are teaching values that keep designs comparable, not a quote.

How to estimate #

A back-of-the-envelope estimate goes the same way every time. Say each assumption out loud and round at every step.

1. Load: users × actions per user per day ÷ 86,400 = average requests per second, separately for reads and writes.
2. Peak: multiply the average by 2–5.
3. Storage: new items per day × bytes per item × days kept × replicas.
4. Bandwidth: requests per second × bytes per response, × 8 for bits.
5. Servers: peak ÷ (what one node does × the share you will run it at), plus one or two spares; repeat for each tier.

Worked example: a photo-sharing app #

Assume 10 million daily active users, each viewing 20 photos a day; 1 in 10 users uploads one photo a day; a served photo is 100 KB, and the original plus its resized copies take 2 MB in storage.

• Load. Views: 10M × 20 = 200M a day ÷ 86,400 ≈ 2,300 reads/s. Uploads: 1M a day ÷ 86,400 ≈ 12 writes/s. Reads outnumber writes 200 to 1.
• Peak. At 3× the average: about 7,000 reads/s and 35 writes/s.
• Storage. Photos: 1M × 2 MB = 2 TB a day, about 730 TB a year, in object storage (which replicates for you). At $0.02 per GB-month, a year's photos cost about $15,000 a month to keep. Metadata: 1M rows × 1 KB = 1 GB a day, under 0.5 TB a year: one database primary holds it.
• Bandwidth. Average: 2,300 × 100 KB = 230 MB/s, about 1.8 Gbit/s; peak 700 MB/s, about 5.6 Gbit/s. A month is 230 MB/s × 2.6 million s ≈ 600 TB sent: about $12,000 through a CDN at $0.02/GB, several times that as plain egress. Serve photos from a CDN.
• Servers. API: 7,000 ÷ (2,000 × 60%) ≈ 6 servers, 7 or 8 with a spare. Cache: 7,000 reads/s is a small fraction of one Redis node; run 2 for availability. Database: with a 90% cache hit rate, 700 reads/s and 35 writes/s, well within one primary with a replica.

The conclusion is the useful part: the bill and the bottleneck are photo bytes (storage and delivery), not requests, so the design's effort goes into object storage, a CDN and cheaper storage tiers for old photos. The request path is small enough for a handful of servers.

To check estimates like these in a design, set traffic on a Proschi use case and let the simulation divide it over the components; to practise them, try the estimate cards (how they are written).