Role & responsibilities :

- Design, develop, and maintain scalable batch and streaming pipelines that ingest data from vehicles, sensors, APIs, files, databases, and cloud services.

- Build reusable ETL/ELT frameworks for transforming raw data into curated, analytics-ready and ML-ready datasets.

- Design data models and storage patterns across data lakes, lakehouses, warehouses, and SQL and NoSQL systems.

- Develop APIs and data services that enable secure, efficient access to raw and processed data.

- Implement automated data validation, reconciliation, schema checks, lineage, metadata, and quality monitoring across the data lifecycle.

- Improve pipeline reliability through observability, alerting, retry and recovery patterns, testing, and systematic root-cause analysis.

- Optimise distributed processing, storage, and query performance while balancing scalability, availability, and cloud cost.

- Apply security, privacy, access-control, retention, and governance requirements to data products and platforms.

- Build and maintain automated CI/CD pipelines, infrastructure-as-code, containerised workloads, and deployment practices for data services across AWS and/or Azure.

- Partner with AI/ML and MLOps teams to deliver high-quality datasets, features, and interfaces for model development, validation, and production workflows.

- Collaborate with global product, platform, embedded, validation, security, and domain teams to translate requirements into maintainable data solutions.

- Contribute to architecture reviews, coding standards, technical documentation, peer reviews, and the continuous improvement of engineering practices.

Preferred candidate profile

- Bachelors or masters degree in Computer Science, Information Technology, Data Science, Engineering, or a related discipline, or equivalent practical experience.

- 510 years of professional experience in data engineering, software engineering, or a closely related field.

- Strong programming skills in Python and advanced working knowledge of SQL.





- Hands-on experience designing and operating production ETL/ELT pipelines for large-scale datasets;

experience at petabyte scale is preferred.

- Experience with data modelling, relational databases, NoSQL systems, and cloud object storage.

- Experience with distributed data processing technologies such as Apache Spark or an equivalent managed platform.

- Practical experience with at least one public cloud platform (AWS or Azure) and its data, compute, storage, security, and monitoring services.

- Experience with workflow orchestration and scheduling tools such as Apache Airflow, Azure Data Factory, AWS Step Functions, or equivalent.

- Proficiency with Git, automated testing, code review, and CI/CD practices.

- Good understanding of data quality, schema evolution, idempotency, partitioning, performance tuning, observability, and failure recovery.

- Ability to troubleshoot complex data and platform issues using a structured, evidence-driven approach.

- Strong communication and collaboration skills, with experience working across functions and geographies.

Valuable to Have:

- Experience with streaming and messaging platforms such as Apache Kafka, Kinesis, or Event Hubs.

- Experience with Docker, Kubernetes, serverless architectures, and infrastructure-as-code tools such as Terraform or CloudFormation.

- Exposure to data lakehouse, transformation, or table-format technologies such as Databricks, dbt, Delta Lake, Apache Iceberg, or equivalent.

- Experience with monitoring and visualisation tools such as Grafana, Kibana, OpenSearch/ELK, Qlik Sense, or Power BI.

- Familiarity with MLOps platforms, feature or dataset management, and data workflows for model training and validation.

- Experience with ADAS, autonomous driving, automotive, IoT, telemetry, sensor, image, video, or other high-volume engineering datasets.

- Working knowledge of Java, Scala, C, or C++ where integration with platform or embedded systems is required.

- Experience mentoring engineers or leading the technical delivery of a workstream.

📌 Data Engineer - MLOPS (Chennai)
🏢 Aptiv
📍 Chennai