withsoon

04 Β· Architecture

One event backbone, two processing paths

Explore the complete Uber data flow from marketplace producers to live decisions and trusted historical products.

SOURCESKAFKAPROCESSINGSERVINGPRODUCTSsecondsreplay + batchPRODUCERDriver appGPS + availabilityPRODUCERRider appDemand + trip requestsPRODUCERDispatchMatch + trip statePRODUCERPaymentsPayment + settlementPRODUCERMapsRoutes + trafficDURABLE BACKBONEKafkaDurable event backboneSECONDS PATHFlink streamingLive marketplace updatesTRUSTED-HISTORY PATHSpark + AirflowReconcile + backfillONLINE SERVINGRedis / CassandraLatest keyed stateDURABLE STORAGELakehouseBronze to Silver to GoldOPERATIONAL CONSUMERLive marketplaceSurge + ETA + dispatchSERVING LAYERWarehouse + featuresSQL + model featuresHISTORICAL CONSUMERSBI, Ops, ML + FinanceReporting + Ops + ML

Architecture decision

Why the pipeline splits after Kafka

Same durable Kafka topics
One event record, two freshness guarantees, no competing source of truth.