Roles and Responsibilities :
- Design, develop, test, deploy and maintain large-scale data processing pipelines using PySpark on AWS EMR.
- Collaborate with cross-functional teams to gather requirements and deliver high-quality solutions that meet business needs.
- Troubleshoot complex issues related to data processing, Spark jobs, and ETL processes.
- Develop technical documentation for the developed solutions.
Job Requirements :
- 5-8 years of experience in software development with a focus on big data technologies such as Hadoop, Spark, etc.
- Solid proficiency in Python programming language and its libraries like NumPy, Pandas, Matplotlib.
- Experience working with cloud-based platforms like AWS EMR (Elastic Map Reduce) is required.
- Proficiency in Data Bricks is an added advantage.
📌 Databricks Engineer (Pune)
🏢 Crisil
📍 Pune