Streaming · Estimate

A topic must take 300 MB/s, and one consumer instance can process about 10 MB/s. How many partitions does the topic need so consumers keep up?

easy Estimation

Answer

About 30 partitions. In review, anything from 20 to 45 counts as right.

The worked estimate

You need 300 ÷ 10 = 30 consumers, and within a consumer group each partition is read by at most one consumer, so the topic needs at least 30 partitions. Add headroom, because adding partitions later moves keys to other partitions.

Numbers: Numbers to know.

Review this in your daily deck All cards in Streaming