Real-Time Streaming
Cover Flink jobs, watch-time logic, sessionization, watermarks, late data handling, and exactly-once style guarantees.
On this page
Streaming jobs
Explain how raw events become trusted real-time metrics and features.
The streaming layer turns raw events into trusted near-real-time facts using keyed state, watermarking, sessionization, and clearly defined metric logic like heartbeat-based watch time.
Quick jumps
Click to jumpMonthly users
200M-250M
Docs use both 200M+ and 250M assumptions
Daily active users
80M
Used for heartbeat and Kafka math
Peak concurrency
15M
Useful for QoE and live ops workloads
Visualize the near-real-time processing layer
This is where topics become trusted live facts, alerts, and features.
Streaming job detail
Playback Sessionizer
Input topics
playback.heartbeat.raw, video.buffer.*, video.pause, video.seek
KeyBy
user_id + profile_id + content_id + device_id
State
active session, seen event_ids, watched_segments, pause_count, buffering_seconds
Window
Session windows with inactivity timeout
Watermark
10 minutes
Output
silver.fact_watch_session updates
Failure behavior
Event animation
Make metric logic visual instead of prose-heavy
Heartbeat truth and coverage logic should be obvious from the timeline.
Interactive timeline
Heartbeats count. Pause does not. Buffering is tracked separately.
Counted watch seconds
120
Paused seconds
Not counted
Buffering seconds
Tracked separately
Toggle definitions
Engagement Watch Time
Total Watch Time
0–10 min + 5–15 min = 20 minutes
Unique Content Coverage
0–15 min = 15 minutes of unique coverage
Count heartbeat_interval_sec only when is_playing = true, is_paused = false, and the event is valid and non-duplicate.
Track total_watch_seconds, buffering_seconds, session_seconds, and unique_content_seconds_watched as separate metrics.
Use unique_content_seconds_watched / content_duration_seconds for completion_pct, not total_watch_seconds.
If buffering should count for a UX metric, store it separately instead of contaminating engagement watch time.
Turn noisy raw events into trusted sessions
The user should see exactly where pause logic, duplicates, and device changes are handled.
Session output
Normal playback
One playback session with clean start/end, watch seconds, and QoE counters.
Raw events → session
Flink session state
Long pause logic
30 min inactivity timeout → close active session
4 hour pause continuation → still same logical journey possible
24 hour journey threshold → link related sessions into one viewing journey
Streaming speed still needs correction paths
Watermarks and late updates belong right next to the real-time pipeline story.
Late event simulator
Drag arrival delay to see how the event should be handled.
On-time event
Process normally with streaming state and live outputs.
Late but allowed
Update previous windows or sessions if within allowed lateness.
Very late event
Route to late_events and correction flow.
Process normally in the stream path.
Replay flow
Controlled correction, not manual copy-paste.
DLQ / quarantine catches malformed, schema-invalid, too-late, or business-rule-failed records
Engineer fixes mapping, schema, or transformation logic
Replay service marks selected records replayable and republishes to replay.playback.events
Streaming and batch pipelines consume replay idempotently
Silver and Gold tables are corrected using MERGE / partition overwrite
Audit records capture what was replayed, why, by whom, and what changed
DLQ is not data loss storage. It needs owner, SLA, dashboard, replay tooling, and audit.
Use a compact streaming explanation
This line should separate live facts from official batch truth.
Say This In Interview
The streaming layer turns raw events into trusted near-real-time facts using keyed state, watermarking, sessionization, and clearly defined metric logic like heartbeat-based watch time.
30-second answer
FastThe streaming layer turns raw events into trusted near-real-time facts using keyed state, watermarking, sessionization, and clearly defined metric logic like heartbeat-based watch time.
2-minute answer
BalancedCover Flink jobs, watch-time logic, sessionization, watermarks, late data handling, and exactly-once style guarantees. Focus on PIPELINES, 10 min, Explain how raw events become trusted real-time metrics and features..
Deep answer
DeepThe streaming layer turns raw events into trusted near-real-time facts using keyed state, watermarking, sessionization, and clearly defined metric logic like heartbeat-based watch time. Then go layer by layer through PIPELINES, 10 min, Explain how raw events become trusted real-time metrics and features., including trade-offs, failure handling, and why this shape is right for the consumer SLAs.
Last reviewed June 2026 · By Prasoon Parashar
Numbers are interview assumptions, not real Netflix internal figures.