Role description Job Title: Senior / Lead Big Data Engineer

Experience:
9 – 15 Years

Location:
Bangalore (Hybrid) Mode of interview: 2 Virtual+1 in-person # of roles: 2 Notice period: Immediate to 30 Days joiners only

About the Role
We are looking for a seasoned Senior/Lead Data Engineer with 9 to 15 years of experience to architect, build, and optimize large-scale data platforms. In this role, you will bridge the gap between complex big data infrastructure, functional business domains, and enterprise AI initiatives. Beyond taking end-to-end ownership of batch data pipelines that process massive customer and behavioral datasets, you will play a key leadership role in building, mentoring, and scaling a high-performing data engineering team.

Key Responsibilities
Platform Optimization & Governance Performance Tuning: Proactively troubleshoot and fine-tune complex Spark and Hive/HQL jobs to reduce latency, optimize memory usage, and manage cluster resource allocation (YARN). Data Quality & Reliability: Establish automated data validation frameworks, lineage tracking, and strict SLA monitoring to guarantee data accuracy and platform reliability. Platform Operations: Oversee on-prem big data platform stability, capacity planning, and workflow orchestration using Airflow. Team Building & Technical Leadership Build & Scale the Team: Partner with leadership to hire, onboard, and build a strong data engineering team from the ground up. Mentorship & Guidance: Provide hands-on technical leadership, conduct architecture and code reviews, and mentor junior and mid-level engineers. Engineering Standards: Establish and enforce best practices around Git version control, code quality, documentation, and Agile delivery methodologies. Architecture & Pipeline Development End-to-End ETL Ownership: Architect, develop, and maintain scalable, fault-tolerant batch ETL/ELT pipelines across on-premises Hadoop-based data platforms. Large-Scale Data Processing:



Design high-throughput data ingestion and transformation workflows for massive customer and behavioral datasets. Data Modeling: Lead dimensional and relational data modeling efforts to support enterprise analytics, reporting, and downstream AI solutions. Functional Alignment & Cross-Team Collaboration Business & Functional Translation: Translate complex functional business requirements into scalable technical data architectures, ensuring data assets directly solve business challenges. Cross-Functional Partnership: Collaborate closely with analytics, product, and AI teams to understand data needs and deliver reliable, analytics-ready datasets. Qualifications & Skills Educational Qualification B.
E. or B.
Tech in Computer Science, Information Technology, or a related engineering discipline. Mandatory Qualifications

Experience:
9 to 15 years of progressive experience in Data Engineering, with deep hands-on expertise in Hadoop-based big data ecosystems (HDFS, YARN). Leadership: Proven experience leading engineering initiatives and building or mentoring technical teams. Big Data Stack: Expert-level proficiency in Apache Spark (Core, Spark SQL, PySpark) and Apache Hive / HQL. Programming & Querying: Strong command of Python and advanced SQL, paired with solid experience in enterprise data modeling. Pipeline Engineering: Proven track record of designing and maintaining production-grade batch ETL pipelines. AI & Functional Acumen: Practical experience working on data initiatives supporting AI applications, combined with solid functional business understanding. Version Control: Proficient in Git for collaborative software development. Preferred Qualifications Domain

Experience:
Prior experience working with e-commerce platforms and processing high-volume customer/transactional datasets is an added advantage. Workflow Orchestration: Hands-on experience managing complex DAGs and schedules using Apache Airflow or Oozie. Infrastructure & Tuning: Deep understanding of on-premise big data platform operations and advanced performance tuning for Hive and Spark workloads. Methodology: Familiarity with Agile/Scrum development methodologies.

📌 Senior/Lead Big Data Engineer with Hadoop,Hive,Spark,SQL-Bangalore (Bengaluru)
🏢 Tredence
📍 Bengaluru