Connected Vehicle Telemetry Platform
Built an event-driven telemetry platform handling 1.8 billion data points daily from 120,000+ connected vehicles with sub-5s p99 latency.
The Challenge
An automotive OEM with 120,000+ connected vehicles on the road was collecting telemetry data (GPS, speed, engine diagnostics, battery state, tyre pressure) through a legacy batch pipeline that processed data in 6-hour windows. This meant that safety-critical alerts (airbag deployment, collision detection, breakdown assistance) had an average latency of 3 hours — unacceptable for any real-time safety use case.
The existing pipeline also could not scale beyond 80,000 vehicles. As the connected fleet grew, data processing fell further behind, and the 6-hour batch window was stretching to 10+ hours.
Our Approach
We built a real-time telemetry platform using an event-driven architecture designed to scale to 1M+ vehicles.
Ingestion: Vehicles transmit telemetry via MQTT to an IoT gateway (AWS IoT Core) that authenticates each vehicle using X.509 certificates provisioned at the factory. The gateway routes messages to Amazon Kinesis Data Streams, partitioned by vehicle ID to preserve per-vehicle event ordering. Peak ingestion rate is 1.8 billion events per day (21,000 events per second).
Stream Processing: Apache Flink consumers process the event stream in real-time. Safety-critical events (airbag deployment, collision g-force threshold, SOS button press) are detected within 500ms and routed to the emergency response system via a dedicated low-latency path. Non-critical events are enriched with vehicle metadata and written to a Delta Lake on S3.
Lakehouse Analytics: The Delta Lake stores the full telemetry history (currently 2.8 PB) with time-travel capability. Data engineers query the lakehouse using Spark SQL for fleet-wide analytics: predictive maintenance models, driving behaviour scoring, battery degradation curves, and warranty claim validation. Dashboards are served through Apache Superset.
Driver Safety Scoring: A real-time scoring model evaluates driving behaviour based on hard braking events, rapid acceleration, cornering g-forces, speeding incidents, and phone usage detection. Scores update in real-time and are exposed to the driver through the companion mobile app and to fleet managers through a web dashboard.
Results
| Metric | Before | After |
|---|---|---|
| Safety alert latency | 3 hours (batch) | < 500ms (real-time) |
| End-to-end data latency (p99) | 6–10 hours | < 5 seconds |
| Scalable vehicle capacity | 80,000 | 1M+ (tested) |
| Data processing backlog | Growing | Zero |
| Predictive maintenance accuracy | N/A | 89% (30-day RUL) |