Job Description

Senior PySpark Developer (Cloudera, Iceberg, Airflow & AWS)

Job Description

We are seeking a highly skilled Senior PySpark Developer to support a large-scale cloud transformation initiative focused on building up-to-date data platforms on Cloudera, Apache Iceberg, Airflow, and AWS. The role will be responsible for designing, developing, and optimizing scalable data pipelines that ingest, transform, and process large volumes of enterprise data.

The successful candidate will work closely with data architects, business analysts, cloud engineers, and application teams to build reliable, high-performance data solutions supporting analytics, reporting, and operational workloads. Responsibilities include developing PySpark applications, implementing ETL/ELT processes, designing Iceberg-based data models, orchestrating workflows using Airflow, and leveraging AWS services to support cloud-native data processing.

This requires strong expertise in distributed data processing, performance tuning, data quality, and cloud-based architectures.



The candidate will participate in end-to-end solution delivery, from requirements analysis and solution design through development, testing, deployment, and production support. They will also contribute to establishing best practices, reusable frameworks, CI/CD processes, and data governance standards across the platform.

The ideal candidate is a hands-on developer with deep technical expertise in modern data engineering technologies and experience working in large-scale enterprise cloud migration and modernization programs.

Required Qualifications

Bachelor's degree in Computer Science, Information Systems, Engineering, or related field.
10+ years of IT experience with 6+ years in Data Engineering and Big Data development.
Robust hands-on experience with PySpark and Spark-based data processing.
Experience developing data pipelines on Cloudera Data Platform (CDP).
Solid knowledge of Apache Iceberg table architecture and d

📌 Senior Technical Lead (Bengaluru)
🏢 Infinite Computer Solutions
📍 Bengaluru