Data EngineeringNetflixProgress
Chapter 1 / 10
On this page
Platform mission
Start with the platform shape.
Start here
Design the shared data platform
Keep the opening light: what data comes in, what the platform does, and who it serves.
Producers
Producer
Playback
Playback is the stream of actions produced during viewing. This is where the platform receives watch behavior directly from the session.
Examples: play_started, pause_clicked, seek_to_timestamp, heartbeat_every_30s, playback_completed.
Playback is the strongest behavioral signal for watch-time truth, completion analysis, QoE correlation, and recommendation feedback.
Producer
Browse
Browse covers how members move through rows, title cards, and detail pages while deciding what to watch.
Examples: homepage_row_seen, title_impression, row_scrolled, detail_page_opened, click_from_top10.
Browse explains discovery quality, homepage ranking performance, and why users did or did not convert into playback.
Producer
Search
Search captures what the member asked for, what results came back, and whether the session ended in success or frustration.
Examples: query_submitted, autocomplete_clicked, result_opened, zero_results_shown.
Search is one of the clearest signals of intent, so it matters for analytics, content demand, and search-quality improvement.
Producer
QoE
QoE means quality-of-experience telemetry. It tracks whether the stream felt smooth or degraded on the device and network.
Examples: startup_delay_ms, rebuffer_started, bitrate_drop, device_error_code, dropped_frames.
QoE streams let teams detect and explain bad viewing experiences quickly, often on sub-minute freshness targets.
Producer
CDC
CDC stands for change data capture. It brings operational database updates into the platform so behavior can be analyzed with business context.
Examples: subscription_plan_changed, profile_created, title_metadata_updated, entitlement_changed.
Without CDC, downstream consumers only see behavior and miss the changing business state around that behavior.
Shared data platform
Accept event streams and CDC once, then shape them into both fast and historical data products.
Platform step
Ingest
Ingestion is the front door. This is where events and CDC changes are accepted, checked, and safely entered into the shared data platform.
Examples: producer authentication, schema checks, timestamp normalization, durable handoff into the event backbone.
If ingestion is weak, everything downstream becomes late, lossy, or untrustworthy.
Platform step
Stream
Streaming turns raw input into live counters, alerts, trends, and online-serving updates for fast consumers.
Examples: QoE alert metrics, trending counters, fresh personalization features.
Streaming is what serves consumers that cannot wait for hourly or daily processing.
Platform step
Store
Storage is the long-term system of record. Raw and curated history live here in a form that can be queried, replayed, and evolved over time.
Examples: raw append-only history, curated analytical tables, long-retention partitions.
Durable history is what makes analytics, training, audits, and backfills possible.
Platform step
Query
Query is where humans and systems read answers out of the platform, from dashboards to analytical SQL.
Examples: ad hoc SQL, experiment reads, funnel analysis, consumer-facing aggregates.
A platform matters only if teams can ask useful questions of it reliably.
Platform step
Recover
Recovery is how the platform corrects late data, broken jobs, or bad releases without giving up trust in the results.
Examples: replay from the log, rerun a correction job, rebuild a curated dataset after a bad release.
Data systems must do more than survive incidents; they must restore correctness afterward.
Consumers
Consumer
QoE alerts
QoE alerting consumers watch fresh windows for degraded viewing experience and operational spikes.
Examples: sudden rebuffer spikes, startup-delay regressions, device-specific playback failures.
These consumers justify the fastest freshness targets in the opening conversation.
Consumer
Trending
Trending consumers want to know what is suddenly becoming popular so the product or dashboards can reflect that quickly.
Examples: top titles right now, fastest-rising content, region-specific popularity changes.
Trending shows why the platform needs a fast path that is fresher than traditional BI.
Consumer
Online recs
Online recommendation consumers use recent behavior to power personalization in user-facing product flows.
Examples: recent watch intent, fresh affinities, latest engagement counters used by row ranking or feature reads.
This is the best example of why freshness and serving latency are separate requirements.
Consumer
Analytics
Analytics consumers want trustworthy historical data for understanding product behavior rather than instant reads.
Examples: retention, funnels, content performance, A/B test reads, watch-hour analysis.
Analytics proves the platform cannot be only a streaming system; it also needs durable history and rich queryability.
Consumer
Training
Training consumers build or refresh offline datasets for recommendation and other ML workflows.
Examples: recommendation training snapshots, feature backfills, historical label generation.
Training is the clean reason to state hourly or daily freshness without pretending everything is live.
I would design the member-interaction data platform: events and CDC flow into one shared platform, then serve fast monitoring, personalization, analytics, training, and reporting consumers with different freshness needs.
Requirements
Keep the requirements light
Use these as a quick checklist. Hover for the exact meaning and examples.
Ingest events + CDC
Behavior plus business context together.
Requirement
Ingest events + CDC
The platform must ingest both member actions and operational changes so downstream teams can combine behavior with business context.
Examples: playback events, browse/search actions, QoE telemetry, subscription changes, content metadata updates.
This keeps the design from splitting into disconnected event-only and database-only systems.
Support fast + batch consumers
One platform, different time horizons.
Requirement
Support fast + batch consumers
The same platform must serve near-real-time consumers and slower historical consumers without becoming two unrelated designs.
Examples: sub-minute QoE alerting and daily training outputs from the same platform story.
This is the main reason the interview problem is a platform problem, not just a streaming pipeline.
Keep durable history
Cheap, queryable, evolvable retention.
Requirement
Keep durable history
Raw and curated history should remain affordable to retain, easy to query, and able to evolve as schemas and consumers change.
Examples: long-retention raw events, curated tables, replayable historical partitions.
Without durable history, the platform cannot support analytics, training, audits, or recovery.
Protect accepted data
Loss hurts more than small delay.
Requirement
Protect accepted data
Once the platform accepts an event, losing it is usually worse than being a little late. Durability leads the opening tradeoff discussion.
Examples: accepted events survive worker failure, broker failure, and common recovery scenarios.
This is a senior data-platform judgment call: small latency slips are often cheaper than data loss.
Stay cost-aware
Shape storage by workload.
Requirement
Stay cost-aware
A platform at this scale cannot assume every layer is premium low-latency infrastructure. Cost-aware storage and shared tooling are part of the design from the start.
Examples: object storage for history, specialized fast stores only where necessary, shared platform services.
Cost is not just a later optimization; it changes what architecture is feasible.
Govern schemas
Trust through strong contracts.
Requirement
Govern schemas
Events need strong contracts so downstream consumers do not silently break when producers change meanings, fields, or timestamps.
Examples: schema registration, compatibility checks, blocking breaking changes before production.
At platform scale, trust collapses quickly if schema discipline is weak.
Scope
Go deep on one clean slice
Member interaction data is the main design. The rest stays extension-only.
In scope
playback, browse, search, impression, click, QoE telemetry
real-time monitoring and personalization signals
historical analytics and recommendation training data
CDC from operational stores
Mention only
video encoding
Open Connect internals
payment processing internals
recommendation model design
Freshness
Real-time needs numbers
Different consumers need very different freshness targets.
Next
Short handoff
One line is enough before estimation.
“We have the platform boundary, the main consumers, the requirement shape, and the freshness targets. Next I would size the scale before drawing detailed architecture.”
Last reviewed June 2026 · By Prasoon Parashar
Numbers are interview assumptions, not real Netflix internal figures.