Position

Location

Qualification

Age Limit

Manager & Deputy Manager – Pyspark Developer

Mumbai

Bachelor's or Master's degree in Computer Science, Data Science, or a related field.

Below 35

Experience & Role:
1. Having experience 4-5 years 2. Having experience 2-3 years - Strong expertise in Big Data ecosystem like Spark, Hive, Sqoop, HDFS, Map Reduce, Oozie, Yarn, HBase, Nifi.

- Design, develop, and maintain PySpark data processing pipelines to process large volumes of structured and unstructured data.

- Collaborate with data engineers and data scientists to understand data requirements and design productive data models and transformations.

- Optimize and tune PySpark jobs for performance, scalability, and reliability.

- Implement data quality checks, error handling, and monitoring mechanisms to ensure data accuracy and pipeline robustness.

- Develop and maintain documentation for PySpark code, data pipelines, and data workflows.

- Developed production ready Spark applications using Spark RDD apis, Data frames, Datasets,



Spark SQL and Spark Streaming.

- Strong experience of HIVE Bucketing and Partitioning.

- Well versed in writing complex hive queries using analytical functions.

- Knowledge in writing custom UDF's in Hive to support custom business requirements.

Job Description :

- We are looking for a skilled PySpark Developer having 4-5 or 2-3 years of experience to join our team.

- As a PySpark Developer, you will be responsible for developing and

- maintaining data processing pipelines using PySpark, Apache Spark's Python API.

- You will work closely with data engineers, data scientists, and other stakeholders to design and implement scalable and efficient data processing solutions.

Current CTC and Expected CTC to be mentioned clearly.
Please email your resume, mentioning the position applied for in the subject column at :[email protected]

📌 Manager & Deputy Manager – Pyspark Developer (India)
🏢 CDSL
📍 India