Good to have skills: Spark SQL, YARN, HDFS, Oozie, Airflow

Key Responsibilities:

• Lead the design and development of scalable Big Data solutions using Hadoop and PySpark for batch and large-scale processing.
• Architect and implement end-to-end data pipelines, ensuring reliability, performance tuning, and efficient resource utilization on Hadoop clusters.
• Develop and optimize Hive data models, queries, and partitioning strategies to support analytics and downstream consumption.
• Drive technical planning, estimation, and delivery for data engineering initiatives, ensuring timelines and quality standards are met.
• Establish coding standards, review code, and enforce best practices for maintainability, testing, and production readiness.
• Troubleshoot production issues, perform root-cause analysis, and implement preventive measures to improve stability and throughput.
• Collaborate with product, analytics, and platform teams to translate requirements into scalable technical solutions.
• Mentor team members, guide technical decisions,



and support skill development across Hadoop, PySpark, Big Data, and Hive. Minimum Qualifications:
• Education: BTECH, MTECH, MCA, MSC (or equivalent).
• 5–9 years of overall experience with robust hands-on expertise in Hadoop and PySpark for large-scale data processing.
• Proven experience building and maintaining Big Data pipelines and working with Hive for querying and data modeling.
• Strong understanding of distributed processing concepts, performance optimization, and data reliability practices.
• Experience leading technical execution through code reviews, design discussions, and delivery ownership. Preferred Qualifications:
• Experience designing reusable frameworks and standardized pipeline patterns to improve team productivity and consistency.
• Strong expertise in optimizing Spark jobs (partitioning, caching, shuffles) and Hive performance (file formats, partitions, bucketing).
• Experience implementing data quality checks

📌 Hadoop / PySpark (Bengaluru)
🏢 Infosys
📍 Bengaluru