Amazon · Big Data Engineering · The commerce platform in one screen

Start Here

Separate Amazon-like transactional systems from the analytical data platform, then trace customer behavior, orders, inventory, and payments into live signals and certified business truth.

Platform mission

Move from commerce facts to decisions

Hover, focus, or tap any component. The map keeps high-volume behavioral data, strict transactional lifecycles, fast operational signals, and reconciled business truth in one explainable design.

Commerce producers

Shared backbone

Two-speed data plane

Decision consumers

FAST · component explanation

Real-time path

Flink validates, deduplicates, enriches, windows, and updates operational views and online features.

Amazon example

A five-minute conversion window and seller order counter update within seconds.

Big-data responsibility

Flink reads versioned Kafka events, validates schemas, deduplicates stable event IDs, enriches from broadcast reference data, and computes event-time windows. Checkpoints protect operator state, while idempotent sinks publish live counters, alerts, seller views, and online features within seconds.

Interview point

The streaming result is low latency and operationally useful, but explicitly provisional until complete history is reconciled through the truth path.

System boundary

Keep checkout online. Make analytics replayable.

Follow a single order through the boundary. The transaction plane fulfills the customer promise; the data plane observes, explains, forecasts, and reconciles it.

STEP 1

Transaction plane

Place the order

Checkout validates price, promotion, payment authorization, inventory reservation, and delivery promise against operational services.

Customer outcome · order confirmed

STEP 2

Durable handoff

Publish the facts

Outbox and CDC prevent dual-write loss. Domain events carry stable IDs, source sequence, event time, money values, and schema version.

Evidence · replayable lifecycle

STEP 3

Data plane

Produce two truths

Streaming serves provisional operations now; the lakehouse rebuilds complete history and publishes certified finance and BI later.

Decisions · live + certified

Fast path · seconds to minutes

Alerts, seller views, personalization, operational decisions

Bounded dedupe and watermarks produce useful values quickly. They are explicitly provisional.

Truth path · minutes to T+1

Finance, BI, certified metrics, historical ML

Complete Bronze plus CDC, settlement, late data, deterministic replay, and reconciliation publish official results.

Interview scope

Cover the commerce product, then narrow to the data platform

An Amazon-like marketplace is broad. Name the surrounding product domains once, then spend interview time on the data responsibilities that determine the architecture.

Product context · acknowledge briefly

Show product awareness by mapping each commerce surface to the data it creates.

Customer + discovery

identity, sessions, search, clicks, experiments

Catalog + pricing

products, SKUs, seller offers, price history

Cart + orders

cart mutations and order-item lifecycles

Payments + refunds

money movements, fees, settlement, disputes

Inventory + fulfillment

reservations, movements, shipments, returns

Marketplace + support

seller performance, service, trust signals

Implementing checkout, search ranking, payment authorization, inventory reservation, or warehouse execution is outside this big-data round.

Design deeply · interview focus

01event production

02CDC + outbox

03stream ingestion

04real-time processing

05lakehouse

06batch + backfill

07data modeling

08quality + contracts

09financial reconciliation

10privacy + access

11multi-region recovery

12serving + ML

Interview boundary

Design how commerce facts become timely, replayable, reconciled, governed data products. Keep transactional decisions in the services that own them.

Data products

Give every product a fast path and a truth path

Select a product to inspect its lifecycle, exact grain, serving contract, consumers, and the mistake an interviewer expects you to avoid.

SOURCE LIFECYCLE

Order lifecycle

created → confirmed → allocated → shipped → delivered / returned

Product grain

Silver keeps one row per order item and an immutable row per status transition. Gold summarizes item outcomes by seller, market, and promised date.

Source events

Checkout outbox events, order CDC, seller confirmations, allocation decisions, shipment scans, delivery outcomes, cancellations, and returns.

Big-data build

Order events by source sequence, reject invalid transitions, preserve the full history, derive the latest item state, and reconcile items to order headers and shipments.

Serving path

Streaming serves live seller and operations views; Iceberg and the warehouse serve certified fulfillment, returns, and revenue-eligibility reporting.

Amazon example

One order contains three items from two sellers. Each item can ship, arrive, cancel, or return independently, so analytics never collapses them into one order-level state.

Correctness rule

Never join order, order-item, shipment, and payment facts without respecting their different grains.

Say this

Open with the correctness boundary, then follow one order

60-second answer

“I separate the transaction plane that owns checkout, payments, inventory reservations, and fulfillment from a replayable data plane. Domain events, outbox records, CDC, clickstream, and partner feeds enter Kafka through versioned contracts. Flink creates low-latency operational views and online features, while immutable Bronze evidence lands in S3. Spark and Iceberg build deduplicated Silver entities and reconciled Gold products for finance, BI, sellers, and ML. The fast path is provisional; the truth path includes late data, settlements, refunds, deterministic replay, quality gates, and atomic publication. I design every product around an explicit grain, owner, freshness and completeness SLO, privacy class, and recovery plan.”

1Start with boundaries2Define grains3Separate live vs certified4Explain late data5Show reconciliation6End with recovery

Preparing chapter

Loading your data design lesson

The curriculum shell is ready while the requested chapter is being prepared. You can wait here or return to the design library.

Open design library

Continue learning

NetflixStreaming analytics platform

UberMarketplace data platform

YouTubeVideo analytics platform