Real-Time EV Telemetry Analytics for Clean Energy Management

Overview

A leading clean energy management platform needed to process massive volumes of real-time EV telemetry data from multiple manufacturers and charging stations to generate critical utility reports. JashDS implemented a scalable streaming architecture using Apache Kafka and Databricks Delta Live Tables. Reduced report generation time and enabled near real-time delivery of both granular usage analytics and peak-demand summaries through automated medallion architecture pipelines.

About the Client

A premier sustainability platform serving North America's major utilities with data-driven insights that optimize energy consumption and promote sustainable EV charging behaviors. The company transforms telemetry data from leading providers like ChargePoint, EVgo, and Wallbox into actionable intelligence for utility grid management and infrastructure planning.

The Challenge

The client faced critical challenges in processing massive volumes of real-time EV telemetry data from diverse sources.

Data Volume and Velocity: Processing continuous streams of telemetry data from multiple EV manufacturers (Tesla, BMW, Mercedes-Benz, Audi, Porsche, Nissan, Honda, Chevrolet, Ford, Jaguar, Hyundai, KIA, Subaru, Volvo, Chrysler) and charging station providers (Wallbox, Chargepoint, RER) in real-time
Complex Data Transformation Requirements: Converting raw telemetry data into multiple distinct reports, BI dashboards and ML ready data for downstream analytics and applications.
Scalability and Performance: Traditional batch processing approaches could not handle the high-frequency data ingestion and transformation requirements while maintaining near real-time reporting capabilities
Data Integration Complexity: Harmonizing telemetry data from disparate OEM/EVSE sources with varying data schemas and transmission protocols into standardized reporting formats
Data Cleanup and Reporting Missing Data: The EV data platform quality is only as good as the data it has ingested from various sources. Hence it is critical to remove duplicates, cleanup the data and also report the missing data pro-actively.

Key Results

Reduced data ingestion time by 30%, enabling near real-time delivery of critical utility insights
Processed 100s of GBs of telemetry data daily with 99.9% uptime and reliability
Achieved great improvement in data processing efficiency through automated streaming pipelines

Our Solution

Real-Time EV Telemetry Analytics for Clean Energy Management — solution architecture diagram

The solution implemented a modern data architecture leveraging streaming technologies and cloud-native platforms to handle real-time EV telemetry processing.

Real-Time Data Ingestion: Deployed Apache Kafka as the primary event streaming platform to capture and buffer telemetry data from OEM/EVSE sources, ensuring reliable data capture with configurable retention policies and automatic checkpointing
Medallion Architecture Implementation: Built a comprehensive data transformation pipeline in Databricks using Delta Live Tables (DLT) for stateful streaming processing, implementing Bronze (raw data), Silver (cleansed and transformed), and Gold (business-ready aggregations) layers
Automated Report Generation: Developed parallel processing workflows to generate both SAP reports (daily on-peak/off-peak kWh summaries) and USAGE reports (granular 15-minute interval charging data with power, current, and energy metrics)
Cloud Storage Integration: Implemented automated delivery of processed CSV reports to S3 storage with standardized naming conventions for seamless consumption by utility systems

Technologies Used

Apache KafkaDatabricks Delta Live TablesPySparkAmazon S3Delta LakeJSON/CSV Processing

Related Case Studies

← Back to All Case Studies

Data Engineering

Scalable Serverless AWS Architecture for Dash Cam Data Processing

JashDS developed a scalable serverless AWS architecture for a dash cam technology company, replacing their legacy FTP system with a comprehensive solution featuring ESE Fargate, Lambda functions, S3 storage, SQS and Kinesis queuing to handle video footage and telemetry data processing. The implementation supported a projected tenfold increase in data volume while providing automated scaling, reduced latency, and improved reliability for their new dash cam product line through modern cloud-native data processing pipelines utilizing Aurora RDS, ECS Fargate, and site-to-site VPN connectivity.

Data Engineering

Automated Outbound IVR System for Lead Management

JashDS developed an automated outbound IVR system for a home care services provider that integrated Salesforce with Amazon Connect, reducing manual calling efforts by 80% and processing 4,000-4,800 automated calls per campaign. The event-driven solution leveraged AWS Step Functions, Lambda, and EventBridge to intelligently schedule and route calls while maintaining comprehensive call recordings and automated CRM updates.

Data Engineering

Transforming QuickBase Data into Conversational AI Analytics using Amazon Q and Quicksight

We transformed Quickbase data into an AI-powered conversational analytics platform using Amazon Q Business, enabling business users to gain instant insights through natural language queries and reducing time-to-insight by 85%. We also maintained data accuracy through an automated ETL pipeline with daily synchronized updates.

Have a similar challenge?

Connect with us