We are looking for a Senior AWS Data Engineer with strong expertise in building scalable data pipelines and designing productive data architectures.
The ideal candidate should have a solid background in data engineering, strong programming skills, and deep experience working with AWS cloud services.
Key Responsibilities
- Design, develop, and maintain data pipelines and data assets using modern data engineering techniques.
- Work extensively with PySpark, Python, and Boto3 to implement robust data solutions.
- Optimize and refactor existing codebases to improve performance, readability, and maintainability.
- Write and maintain unit tests following TDD principles to ensure code quality and functionality.
- Debug and fix complex issues related to performance, concurrency, and logic.
- Manage code versioning and collaboration using Git repositories and JFrog Artifactory.
- Collaborate with cross-functional teams to design and implement AWS-based architectures.
Required Skills & Experience
- 7+ years of hands-on experience in data engineering.
- Robust programming skills in Python, including frameworks and libraries aligned with Python best practices.
- Proven experience with PySpark and Spark SQL for large-scale data processing and code optimization.
- Proficient in AWS services such as S3, EC2, Lambda, Redshift, CloudFormation, etc., and able to explain their benefits and use cases.
- Solid understanding of Git version control and artifact management tools (e.g., JFrog Artifactory).
- Experience in code refactoring, unit testing, and debugging complex systems.
Preferred Qualifications
- Experience in designing data lake or data warehouse architectures on AWS.
- Exposure to DevOps and CI/CD pipelines for data workflows.
- Strong analytical and problem-solving skills with a focus on performance tuning.