Senior Data Engineer Hadoop + PySpark

J Location: Chennai, Hyderabad Grade: P4 (8–12 years)

Job Summary

We are seeking a Senior Data Engineer with 8–12 years of experience in Hadoop and PySpark to design, develop, and optimize scalable big data solutions. The role involves building highperformance data pipelines, ensuring data quality, and collaborating with crossfunctional teams to deliver enterprisegrade analytics solutions.

Key Responsibilities

- Design, develop, and deploy scalable big data solutions using Hadoop and Spark ecosystems
- Implement and optimize PySpark and Spark SQL jobs for performance and resource utilization
- Develop Python scripts to automate data processing and integration within Hadoop ecosystem
- Integrate data ingestion and transformation tools such as HDFS, Hive, Sqoop, Oozie, and Kafka
- Monitor and maintain Hadoop clusters for stability, capacity planning, and performance tuning
- Collaborate with data engineers,



analysts, and stakeholders to deliver actionable insights
- Conduct code reviews, provide technical guidance, and mentor junior team members
- Document technical specifications, processes, and best practices
- Ensure compliance with security and governance standards in big data environments

Mandatory Skills
- Hadoop ecosystem (HDFS, MapReduce, Hive, Sqoop, Oozie, Kafka)
- PySpark and Spark SQL
- Robust Python scripting for data workflows
- Data pipeline design, optimization, and troubleshooting

Candidate Profile
- 8–12 years of relevant experience in big data engineering
- Proven expertise in Hadoop + PySpark with largescale data processing
- Strong problemsolving and performance tuning skills
- Ability to mentor and lead technical teams

📌 Walk-in || Senior Data Engineer(Hadoop + PySpark) (Chennai)
🏢 Alike Thoughts
📍 Chennai