About The Position
We are looking for a Senior/Lead Big Data Engineer to drive the design, architecture, and implementation of scalable data platforms and ETL pipelines. This role requires deep technical expertise in Python, PySpark, and distributed data systems, along with strong leadership in guiding teams, defining best practices, and delivering high-quality data solutions. Supply Chain domain experience is highly desirable.

Key Responsibilities
• Architect and design scalable, high-performance ETL/data pipeline frameworks using Python and PySpark.
• Lead end-to-end data engineering solutions on Databricks (batch + streaming)
• Define and enforce data engineering standards, best practices, and governance
• Drive design and implementation of real-time data pipelines using Kafka
• Oversee integration of heterogeneous data sources (MongoDB, Oracle, APIs, external systems)
• Collaborate with business stakeholders, product owners, and analytics teams to translate requirements into scalable solutions




• Enable data-driven decision-making by supporting advanced reporting and dashboards (Tableau)
• Lead performance tuning, cost optimization, and reliability improvements of data pipelines
• Mentor and guide junior engineers; conduct code reviews and design reviews
• Ensure production stability with solid monitoring, alerting, and incident management practices
• Contribute to CI/CD setup, automation, and DevOps best practices for data pipelines

Required Skills & Qualifications
• At least 4+ years in Big Data/Data Engineering
• Strong expertise in Python and PySpark (hands-on coding + optimization)
• Advanced SQL skills (complex transformations, performance tuning)
• Proven experience in Databricks (Delta Lake, notebooks, job orchestration)
• Hands-on experience with Kafka for event-driven architecture
• Strong experience with MongoDB and Oracle databases
• Experience working with Tableau or similar BI/reporting tools
• Deep understanding of ETL frameworks, data modelling

📌 Data Engineer (Bengaluru)
🏢 Tata Elxsi
📍 Bengaluru