Job Summary

We are looking for a skilled PySpark Developer with around 4 years of experience in designing, developing, and optimizing large-scale data processing solutions using Apache Spark and Python. The candidate should have strong expertise in ETL development, big data technologies, data warehousing concepts, and cloud platforms.

Key Responsibilities

- Design, develop, and maintain scalable data pipelines using PySpark.
- Process and analyze large datasets from multiple data sources.
- Develop ETL/ELT workflows for data ingestion, transformation, and loading.
- Optimize Spark jobs for performance, scalability, and reliability.
- Work with structured and unstructured data in distributed environments.
- Collaborate with data engineers, data analysts, and business stakeholders to understand requirements.
- Implement data quality checks and ensure data integrity.
- Troubleshoot and resolve performance bottlenecks in Spark applications.
- Develop and maintain data models, workflows, and documentation.
- Participate in code reviews and follow best coding practices.





Required Skills

- 4+ years of experience in Python and PySpark Development.
- Robust knowledge of Apache Spark Core, Spark SQL, Data Frames, and RDDs.
- Experience with ETL/Data Engineering projects.
- Hands-on experience with Hive, HDFS, SQL, and Data Warehousing concepts.
- Strong SQL querying and performance tuning skills.
- Experience with workflow orchestration tools such as Airflow, Oozie, or Control-M.
- Knowledge of version control tools like Git.
- Familiarity with Linux/Unix environments.
- Experience working in Agile/Scrum methodology.

Preferred Skills

- Experience with cloud platforms such as AWS, Azure, or GCP.
- Knowledge of Databricks.
- Experience with Delta Lake, Parquet, Avro, and ORC file formats.
- Understanding of Kafka or other streaming technologies.
- Knowledge of CI/CD pipelines and Dev Ops practices.
- Exposure to machine learning pipelines is an added advantage

📌 PySpark (India)
🏢 Objectwin Technology India
📍 India