We are looking for a skilled PySpark Data Engineer with 2-5 years of experience in building scalable data processing solutions and data pipelines. The ideal candidate should have solid expertise in PySpark, Python, SQL, and big data technologies.
Key Responsibilities
- Design, develop, and maintain scalable data pipelines using PySpark.
- Process and transform large volumes of structured and unstructured data.
- Develop ETL workflows and data integration solutions.
- Optimize Spark jobs for performance and scalability.
- Collaborate with cross-functional teams to deliver data engineering solutions.
- Ensure data quality, reliability, and operational excellence.
Required Skills
- Strong hands-on experience with PySpark and Apache Spark.
- Proficiency in Python programming.
- Strong knowledge of SQL.
- Experience in Data Engineering and ETL development.
- Understanding of big data concepts and distributed data processing.