Job Description:

- We are looking for an experienced Senior PySpark Developer with 8 years of experience in Data Engineering and Big Data technologies
- The ideal candidate should possess strong expertise in PySpark Python Spark SQL ETL development Data Warehousing and Cloud platforms AWS Azure GCP
- The candidate will be responsible for designing developing and optimizing large scale data processing solutions

Key Responsibilities:

- Key Responsibilities
- Design and develop scalable data pipelines using PySpark and Apache Spark
- Build and optimize ETL ELT frameworks for processing large volumes of structured and unstructured data
- Develop high performance Spark applications using Spark SQL Data Frames and RDDs
- Work closely with data architects and business teams to define data requirements
- Implement data quality validation and governance processes




- Optimize Spark jobs for performance memory management and scalability
- Integrate data from various sources including databases APIs cloud storage and streaming platforms
- Support production deployments and resolve performance bottlenecks
- Mentor junior developers and participate in code reviews

Technical Requirements:

- Mandatory Skills
- 8 years of IT experience
- Strong hands on experience in PySpark and Python
- Expertise in Apache Spark Spark SQL Data Frames and ETL Development
- Solid SQL and database concepts
- Experience with Hadoop Ecosystem Hive HDFS YARN
- Experience in Data Warehousing and Data Modeling
- Hands on experience with Unix Linux

Preferred Skills: Technology->Big Data - Data Processing->PySpark

📌 Senior PySpark Developer (India)
🏢 Infosys
📍 India