Lead Data Engineer (Hadoop)
Location: Hyderabad
Experience: 6 to 10 Years
Employment Type: Full-Time
*Immediate joiners*
Key Responsibilities
Design, build, and maintain scalable ETL/ELT data pipelines using Hadoop, Hive, HDFS, and Spark/Py Spark Develop and optimize complex SQL queries and Hive scripts for large-scale data processing Write clean, efficient, and reusable Python code for data transformation and automation Work with structured and unstructured data across distributed storage systems (HDFS) Optimize Spark/Py Spark jobs for performance, scalability, and resource efficiency Collaborate with data analysts, data scientists, and business stakeholders to understand data requirements Ensure data quality, consistency, and integrity across pipelines Troubleshoot and resolve issues related to data pipeline failures, performance bottlenecks,



and cluster resource management Participate in code reviews and follow best practices for data engineering and version control Document technical designs, data flows, and pipeline architecture Required Skills & Experience
6 to 10 years of hands-on experience in Data Engineering Strong working knowledge of Hadoop ecosystem (HDFS, YARN, Map Reduce concepts) Proficiency in Hive for data warehousing and query optimization Solid experience with Spark/Py Spark for distributed data processing Solid programming skills in Python Advanced SQL skills — query optimization, joins, window functions, performance tuning Good to Have
Experience with No SQL databases (HBase, Cassandra) Familiarity with CI/CD pipelines for data engineering workflows Educational Qualification
Bachelor's or Master's degree in Computer Science, Information Technology, or a related field Regards,
Manvendra Singh
manvendra.singh1@

📌 Lead data engineer (Hyderabad)
🏢 Incedo
📍 Hyderabad