Job Title: Data Engineering Hadoop SME

Experience: 1015 years
Role: Hadoop / Big Data Subject Matter Expert

Job Summary

We are looking for an experienced Hadoop/Data Engineering SME with strong hands-on expertise in Hadoop, Spark, Hive, Python/Scala, SQL, and Airflow. The role will be responsible for designing scalable data solutions, resolving complex technical challenges, optimizing large-scale data workloads, and providing technical guidance to development teams.

Key Responsibilities
Act as the SME for Hadoop and Big Data engineering solutions.
Design and develop highly scalable ETL/data pipelines for large-volume data processing.
Provide technical design and solution guidance using Hadoop, HDFS, Spark, Hive, and related technologies.
Develop and optimize complex Spark/PySpark/Scala processing jobs.
Build and manage workflow orchestration using Airflow.
Write and optimize complex SQL/HiveQL queries.
Drive performance tuning and troubleshoot complex production issues involving Spark, Hive, and distributed workloads.
Design ingestion solutions for structured and unstructured data using Kafka, APIs, files, and other sources.
Provide expertise in data modelling, data quality,



lineage, and governance.
Conduct technical reviews, code reviews, and enforce engineering best practices.
Mentor developers and provide technical guidance to engineering teams.
Collaborate with architects, business stakeholders, and upstream/downstream teams on solution design.
Create and maintain technical design and architecture documentation.

Mandatory Skills
10+ years of Data Engineering/Big Data experience, with strong hands-on development expertise.
Expert-level knowledge of Hadoop ecosystem and HDFS.
Strong hands-on experience with Apache Spark / PySpark.
Robust knowledge of Hive / HiveQL.
Solid programming skills in Python and/or Scala.
Advanced SQL and robust data modelling fundamentals.
Hands-on experience with Airflow or similar orchestration platforms.
Strong experience in performance tuning and troubleshooting Spark/Hadoop workloads.
Good understanding of distributed computing and large-scale data processing.
Linux and Shell scripting knowledge.
Experience with Git and Agile development practices.

📌 Hiring Bigdata Sme Chennai/ Bangalore
🏢 Tech Mahindra
📍 Chennai