Job DescriptionStep into a high-impact consulting role where youll lead the design and delivery of scalable big data solutions that turn complex datasets into actionable insights. As a Lead Consultant, youll collaborate closely with data engineers, architects, analysts, and business stakeholders to shape modern data platforms built on Hadoop ecosystems and PySpark-based processing. Youll guide teams through best practices, performance tuning, and reliable deliverybalancing hands-on problem solving with technical leadership.

This role is ideal for someone who enjoys mentoring, driving technical decisions, and building data pipelines that are resilient, efficient, and production-ready. If youre motivated by solving large-scale data challenges and enabling teams to deliver measurable outcomes, youll find a collaborative environment here that values ownership, clarity, and continuous improvement.

Roles Responsibilities- Lead end-to-end delivery of big data solutions using Hadoop and PySpark, from requirements to production rollout.

- Design and implement scalable batch/ETL pipelines for large datasets with strong focus on reliability and performance.
- Drive technical architecture discussions, define standards, and ensure best practices for distributed data processing.
- Optimize Spark jobs through partitioning strategies, caching, shuffle tuning, and efficient file formats where applicable.
- Collaborate with stakeholders to translate business needs into technical specifications, delivery plans, and milestones.
- Establish data quality checks, validation frameworks, and operational monitoring for production pipelines.
- Perform root-cause analysis for pipeline failures and performance bottlenecks; implement preventive fixes.




- Mentor engineers through code reviews, design reviews, and knowledge sharing to uplift team capability.
- Ensure documentation, runbooks, and handover artifacts are created and maintained for support readiness.

Technical Requirement

- Primary skills:Technology->Big Data - Data Processing->PySpark,Technology->Big Data - Hadoop->Hadoop

Additional Responsibility

- Bachelors or Masters degree (or equivalent) in Engineering/Technology/Computer Applications/Science (BE/BTech/MTech/MCA/MSc or equivalent).
- 911 years of overall experience in data engineering and large-scale data processing environments.
- Solid hands-on experience with Hadoop ecosystem components and distributed data processing concepts.
- Strong hands-on experience building data pipelines using PySpark.
- Proven ability to lead technical delivery, guide teams, and manage stakeholder expectations in consulting engagements.

Preferred Qualifications

- Experience designing and implementing Spark-based data processing patterns (batch and incremental loads).
- Strong understanding of data modeling and storage patterns for big data platforms (partitioning, compaction, schema evolution).
- Experience with workflow orchestration and scheduling for data pipelines and dependency management.
- Demonstrated expertise in production hardening: monitoring, alerting, SLAs, and incident management for data jobs.
- Strong consulting mindset with ability to present solutions, document designs clearly, and influence technical decisions across teams. Educational RequirementMCA,MSc,MTech,Bachelor of Engineering,BTech

Preferred SkillsTechnology->Big Data - Data Processing->PySpark,Technology->Big Data - Hadoop->Hadoop

Service LineData Analytics Unit

📌 Hadoop / PySpark (Bengaluru)
🏢 Infosys
📍 Bengaluru