Description - ExternalJob Description: Senior Data Engineer (6–9 Years Experience)
Location: Bengaluru
Employment Type: Full time

Role Overview
We are seeking a Senior Data Engineer with 6–9 years of experience to design, build, and optimize scalable data platforms and pipelines using modern cloud-based data technologies. The ideal candidate will have strong expertise in Azure Databricks, PySpark, SQL, Unity Catalog, and workflow orchestration tools, along with experience in data governance, security, and performance optimization.

This role involves leading data engineering initiatives, enabling analytics and data science use cases, implementing governance frameworks, and ensuring high-performance, reliable data systems across enterprise environments.

Key Responsibilities

1. Data Architecture & Platform Engineering
- Design and implement scalable data architectures using Azure Databricks and modern lakehouse principles.
- Architect batch and streaming data pipelines for large-scale structured and unstructured datasets.




- Implement Databricks Delta Lake patterns including Medallion Architecture (Bronze, Silver, Gold layers).
- Define data modeling strategies for analytics and reporting workloads.
- Collaborate with architects and stakeholders to align platform design with business objectives.

2. Advanced Pipeline Engineering
- Design and manage Delta Live Tables (DLT) pipelines and Structured Streaming solutions for real-time analytics.
- Develop complex ETL/ELT pipelines using PySpark and SQL.
- Lead development of high-performance data workflows including advanced Spark tuning and optimization.
- Implement reusable frameworks and engineering standards for pipeline development.
- Optimize data processing workflows for performance, scalability, and cost efficiency.

3. Performance Engineering & Optimization
- Analyze system performance and identify bottlenecks across data pipelines and infrastructure layers.
- Optimize cluster configuration, resource utilization

📌 Fullstack Lead- ITO Transition (Bengaluru)
🏢 Mercedes Benz Research and Development India
📍 Bengaluru