withsoon
withsoon/System Design/Netflix Data EngineeringPRODUCTION

Feature Store / Experimentation

Separate online and offline feature paths, point-in-time correctness, entity keys, and experimentation support.

View Backend Track

Chapter

11 / 17

On this page

Online vs offline

PRODUCTIONChapter 118 min

Show how DE supports online features, offline training, and experiments.

DE owns the freshness and correctness of feature data by separating online and offline paths while preserving point-in-time joins and experiment exposure lineage.

PRODUCTION8 minShow how DE supports online features, offline training, and experiments.

Quick jumps

Click to jump

Monthly users

200M-250M

Docs use both 200M+ and 250M assumptions

Daily active users

80M

Used for heartbeat and Kafka math

Peak concurrency

15M

Useful for QoE and live ops workloads

Online vs offline

Separate low-latency features from training truth

This avoids the page feeling like an ML Engineer mode while still covering the DE responsibility well.

Offline Feature Store

S3 + Iceberg feature tables used for reproducible training and backtesting.

Stores historical training features with point-in-time aligned snapshots

Built from Silver and Gold history using Spark or dbt feature pipelines

Feeds training datasets, feature evaluation, and experimentation analysis

Online Feature Store

Redis or DynamoDB for low-latency serving features.

Stores recently watched, session context, device-time context, and hot popularity counters

Updated from streaming jobs for low-latency recommender calls

Needs explicit freshness SLAs and entity-key consistency with offline features

Feature flow

Playback / search / impression events→
Flink real-time features→
Redis / DynamoDB online features→
Silver and Gold history→
Spark feature generation→
Offline feature store→
Training dataset

Point-in-time correctness

Correct point-in-time logic means feature_time must be less than or equal to label_event_time. Do not join a Monday training row to a Friday-computed user feature.

Experimentation

Show how experimentation data fits the pipeline

Assignments, exposure events, and outcome joins should feel like first-class consumers of the platform.

1

experiment.assigned event

2

recommendation.served / title.impression exposure

3

playback / click / retention outcome events

4

Silver user timeline

5

Gold experiment analysis mart

Say this

Describe how DE supports features and experiments

Keep the focus on data correctness and freshness.

Say This In Interview

DE owns the freshness and correctness of feature data by separating online and offline paths while preserving point-in-time joins and experiment exposure lineage.

Last reviewed June 2026 Β· By Prasoon Parashar

Numbers are interview assumptions, not real Netflix internal figures.

Was this tab useful?