Job Title: Data Engineering Hadoop SME

Experience: 1015 years

Role: Hadoop / Big Data Subject Matter Expert

Job Summary

We are looking for an experienced Hadoop/Data Engineering SME with strong hands-on expertise in Hadoop, Spark, Hive, Python/Scala, SQL, and Airflow. The role will be responsible for designing scalable data solutions, resolving complex technical challenges, optimizing large-scale data workloads, and providing technical guidance to development teams.

Key Responsibilities

- Act as the SME for Hadoop and Big Data engineering solutions.
- Design and develop highly scalable ETL/data pipelines for large-volume data processing.
- Provide technical design and solution guidance using Hadoop, HDFS, Spark, Hive, and related technologies.
- Develop and optimize complex Spark/PySpark/Scala processing jobs.
- Build and manage workflow orchestration using Airflow.
- Write and optimize complex SQL/HiveQL queries.
- Drive performance tuning and troubleshoot complex production issues involving Spark, Hive, and distributed workloads.
- Design ingestion solutions for structured and unstructured data using Kafka, APIs, files, and other sources.
- Provide expertise in data modelling, data quality,



lineage, and governance.
- Conduct technical reviews, code reviews, and enforce engineering best practices.
- Mentor developers and provide technical guidance to engineering teams.
- Collaborate with architects, business stakeholders, and upstream/downstream teams on solution design.
- Create and maintain technical design and architecture documentation.

Mandatory Skills

- 10+ years of Data Engineering/Big Data experience, with strong hands-on development expertise.
- Expert-level knowledge of Hadoop ecosystem and HDFS.
- Robust hands-on experience with Apache Spark / PySpark.
- Strong knowledge of Hive / HiveQL.
- Strong programming skills in Python and/or Scala.
- Advanced SQL and strong data modelling fundamentals.
- Hands-on experience with Airflow or similar orchestration platforms.
- Strong experience in performance tuning and troubleshooting Spark/Hadoop workloads.
- Good understanding of distributed computing and large-scale data processing.
- Linux and Shell scripting knowledge.
- Experience with Git and Agile development practices.

📌 Hiring Bigdata SME - Chennai/ Bangalore (Bengaluru)
🏢 Tech Mahindra
📍 Bengaluru