withsoon
withsoon/System Design/Netflix Data EngineeringPRODUCTION

Capacity / Cost

Derive event rates, partitions, retention, storage, compute costs, and map the stack to AWS plus open-source services.

View Backend Track

Chapter

14 / 17

On this page

Scale math

PRODUCTIONChapter 147 min

Make scale math, tool choices, and cost controls explicit and defensible.

I derive capacity from event math, then show how topic retention, compaction, cluster sizing, storage layout, and query engine choices control cost.

PRODUCTION7 minMake scale math, tool choices, and cost controls explicit and defensible.

Quick jumps

Click to jump

Monthly users

200M-250M

Docs use both 200M+ and 250M assumptions

Daily active users

80M

Used for heartbeat and Kafka math

Peak concurrency

15M

Useful for QoE and live ops workloads

Scale math

Derive capacity from event math, not guesses

Show the interviewer you can work from DAU and heartbeat frequency to partition count and storage.

Interactive scale calculator

Turn interview assumptions into throughput, storage, and partition math.

Heartbeats / user / day

240

watch_seconds_per_day / heartbeat_interval

Heartbeat events / day

19.20B

80M users x 240

Raw TB / day

34.6 TB

Only compressed events, before replication and long-term lifecycle

Avg events / sec

400.0K

daily_events / 86,400

Peak events / sec

2.00M

avg events/sec x peak multiplier

Kafka partitions

260

ceil(peak / per_partition x headroom)

Bronze hot storage

3.11 PB

Assumes 90 hot days of raw history

Silver retention

12.61 PB

Assumes Silver compresses to 50% and retains 2 years

Heartbeats per user/day

watch_seconds_per_day / heartbeat_interval_sec

7,200 / 30 = 240

Peak events/sec

daily_events / 86,400 x peak_multiplier

400.0K x 5 = 2.00M

Kafka partitions

ceil(peak_events_per_sec / safe_events_per_partition x headroom)

ceil(2.00M / 10000 x 1.3) = 260

Cost levers

Show where cost changes as the platform scales

Retention, partitions, cluster sizing, and serving engine choices all move spend.

Trade-off matrix

Kafka vs Kinesis

Option A

Kafka / MSK

Option B

Kinesis

Recommendation

Kafka when you want rich ecosystem, strong replay patterns, and open-source portability.

Why

Netflix-like scale, topic design, consumer ecosystem, and replay tooling often fit Kafka well.

When to change

Choose Kinesis when managed AWS integration and ops simplicity beat ecosystem flexibility.

Required trade-offs

Kafka = more flexibility

Kinesis = simpler managed path

Both need careful partition/shard math

AWS + OSS map

Map the design to concrete services

Make the stack feel deployable, not abstract.

Event backbone

AWS

MSK

Open source

Kafka + Schema Registry

Why here

Durable fan-out, replay, and independent consumers.

Streaming compute

AWS

Kinesis Data Analytics / EMR on EKS

Open source

Flink

Why here

Stateful stream processing, watermarks, and session logic.

Batch compute

AWS

EMR / Glue

Open source

Spark + dbt

Why here

Large joins, reconciliation, and official daily truth.

Lakehouse format

AWS

S3 + Glue Catalog

Open source

Iceberg

Why here

Schema evolution, snapshots, time travel, and controlled backfills.

Serving

AWS

Redshift / Athena

Open source

Trino / Pinot

Why here

Separate BI, ad hoc, and real-time OLAP workloads.
Say this

Use a scale-and-cost line with real numbers

Good answers derive capacity instead of hand-waving it.

Say This In Interview

I derive capacity from event math, then show how topic retention, compaction, cluster sizing, storage layout, and query engine choices control cost.

Last reviewed June 2026 Β· By Prasoon Parashar

Numbers are interview assumptions, not real Netflix internal figures.

Was this tab useful?