Job highlights

- 8+ years IT experience with expertise in PySpark, Apache Spark, Spark SQL, ETL development, Data Warehousing, and Cloud platforms (AWS/Azure/GCP)

- Design and develop scalable data pipelines, optimize ETL/ELT frameworks, implement data quality and governance, support production deployments, and mentor junior developers

Job Summary

We are looking for an experienced Senior PySpark Developer with 8+ years of experience in Data Engineering and Big Data technologies. The ideal candidate should possess solid expertise in PySpark, Python, Spark SQL, ETL development, Data Warehousing, and Cloud platforms (AWS/Azure/GCP). The candidate will be responsible for designing, developing, and optimizing large-scale data processing solutions.

Responsibilities

- Design and develop scalable data pipelines using PySpark and Apache Spark.

- Build and optimize ETL/ELT frameworks for processing large volumes of structured and unstructured data.

- Develop high-performance Spark applications using Spark SQL, DataFrames, and RDDs.





- Work closely with data architects and business teams to define data requirements.

- Implement data quality, validation, and governance processes.

- Optimize Spark jobs for performance, memory management, and scalability.

- Integrate data from various sources including databases, APIs, cloud storage, and streaming platforms.

- Support production deployments and resolve performance bottlenecks.

- Mentor junior developers and participate in code reviews.

Technical Requirements Mandatory Skills

- 8+ years of IT experience.

- Strong hands-on experience in PySpark and Python.

- Expertise in Apache Spark, Spark SQL, DataFrames, and ETL Development.

- Strong SQL and database concepts.

- Experience with Hadoop Ecosystem (Hive, HDFS, YARN).

- Experience in Data Warehousing and Data Modeling.

- Hands-on experience with Unix/Linux.

Educational Requirement

- MTech

- Bachelor of Engineering

- BTech

Preferred Skills

- Big Data - Data Processing - PySpark

Service Line

- Data Analytics Unit

📌 Senior PySpark Developer (Bengaluru)
🏢 Infosys
📍 Bengaluru