Hadoop Data Engineer
Location: Hyderabad
Experience: 3–6 Years
Employment Type: Full-Time

Key Responsibilities
Design, build, and maintain scalable ETL/ELT data pipelines using Hadoop, Hive, HDFS, and Spark/PySpark
Develop and optimize complex SQL queries and Hive scripts for large-scale data processing
Write clean, effective, and reusable Python code for data transformation and automation
Work with structured and unstructured data across distributed storage systems (HDFS)
Optimize Spark/PySpark jobs for performance, scalability, and resource efficiency
Collaborate with data analysts, data scientists, and business stakeholders to understand data requirements
Ensure data quality, consistency, and integrity across pipelines
Troubleshoot and resolve issues related to data pipeline failures, performance bottlenecks, and cluster resource management
Participate in code reviews and follow best practices for data engineering and version control
Document technical designs,



data flows, and pipeline architecture

Required Skills & Experience
3–6 years of hands-on experience in Data Engineering
Solid working knowledge of Hadoop ecosystem (HDFS, YARN, MapReduce concepts)
Proficiency in Hive for data warehousing and query optimization
Solid experience with Spark/PySpark for distributed data processing
Strong programming skills in Python
Advanced SQL skills — query optimization, joins, window functions, performance tuning

Good to Have
Experience with NoSQL databases (HBase, Cassandra)
Familiarity with CI/CD pipelines for data engineering workflows

Educational Qualification
Bachelor's or Master's degree in Computer Science, Information Technology, or a related field

Regards,
Manvendra Singh
[email protected]

📌 Hadoop Data Engineer (Hyderabad)
🏢 Incedo
📍 Hyderabad