We are looking for a skilled Senior Pyspark Developer with 4 to 9 years of experience to join our team at Luxoft India LLP. The ideal candidate will have expertise in developing scalable data processing pipelines using Pyspark.
Roles and Responsibility
Design, develop, and deploy large-scale data processing pipelines using Pyspark.
Collaborate with cross-functional teams to identify business requirements and design solutions.
Develop and maintain complex SQL queries for data extraction and transformation.
Troubleshoot and optimize performance issues in existing data pipelines.
Implement automated testing and deployment scripts using tools like Jenkins or GitLab CI/CD.
Participate in code reviews to ensure high-quality code standards.
Job Requirements
Robust proficiency in Python programming language and its ecosystem.
Experience with big data technologies such as Hadoop, Spark, and NoSQL databases.
Knowledge of cloud-based platforms such as AWS or GCP.
Familiarity with agile development methodologies and version control systems like Git.
Excellent problem-solving skills and attention to detail.
Ability to work collaboratively in a team setting and communicate effectively with stakeholders.