YouTube Data Engineering — Event Sources

Progress

Chapter 3 / 9

01Start Here 02Requirements 03Event Sources 04Architecture 05Ingestion / Kafka 06Real-Time Streaming 07Data Modeling 08Batch + Lakehouse 09Governance / Quality

→

Producers

→

Event Families

FirehoseContinuous or exposure-driven telemetry

InteractionViewer intent, quality, and money

ReferenceLower-volume authoritative state

Event Contract

playback.heartbeat · v4JSON

{  "event_id": "uuid",  "event_name": "playback.heartbeat",  "schema_version": 4,  "metric_semantics_version": 7,  "event_time": "2026-09-06T12:00:15Z",  "client_send_time": "2026-09-06T12:00:17Z",  "ingest_time": "2026-09-06T12:00:18Z",  "user_or_anon_id": "privacy-scoped-id",  "device_id": "rotatable-id",  "session_id": "viewer-session",  "playback_attempt_id": "attempt-42",  "video_id": "video-123",  "channel_id": "channel-9",  "position_ms": 42000,  "played_delta_ms": 15000,  "buffered_delta_ms": 0,  "playback_state": "PLAYING",  "is_live": false,  "surface": "HOME",  "experiment_assignments": ["exp123:treatment"],  "consent": {"analytics": true, "personalization": true},  "client": {"platform": "TV", "app_version": "x.y"},  "network": {"type": "WIFI"},  "trace_id": "trace-abc"}

Playback fields

position_msplayed_delta_msbuffered_delta_msplayback_state

played_delta_ms is watch-time evidence. YouTube validates it against position movement, state transitions, buffering, video duration, retry patterns, and traffic-quality signals.

Playback Evidence

What it means

Playback evidence is the ordered set of player events that shows how one attempt to watch a video actually progressed.

playback_attempt_id groupsevent_id dedupesevent_time ordersplayed_delta_ms measures

“Use stable event IDs, group by playback attempt, process in event time, sum only validated played deltas, apply a versioned qualification policy, then certify with complete Bronze history.”

Workloads