- Design, develop, and optimize scalable data pipelines using Python.
- Build and maintain ETL/ELT solutions for large-scale data processing.
- Develop cloud-native data solutions leveraging AWS services.
- Process and transform structured and unstructured data from multiple sources.
- Implement data quality, validation, and monitoring frameworks.
- Configure and manage data workflows and orchestration processes.
- Collaborate with business, analytics, and engineering teams to gather requirements and deliver data solutions.
- Support production deployments, troubleshooting, and performance tuning.
- Adhere to Agile development and DevOps best practices. [JD_Vanguar...n with AWS | Word], [SCALA WITH AWS | Word]
Mandatory Skills
- 5+ years of experience in Data Engineering.
- Strong programming expertise in Python.
- Hands-on experience with AWS cloud services including:
- AWS S3
- AWS Glue
- AWS Lambda
- AWS Step Functions
- AWS EventBridge
- ECS / EKS
- Experience building ETL/Data Pipelines.
- Solid SQL and database skills.
- Experience with data transformation and data integration frameworks.
- Knowledge of REST APIs and microservices.
- Experience with Git and version control.
- Understanding of CI/CD practices and cloud deployment processes. [Experian-Python | Word], [SCALA WITH AWS | Word]
Preferred Skills
- PySpark or Spark Framework.
- Terraform or CloudFormation.
- Docker and Kubernetes.
- Jenkins CI/CD.
- Data Lake and Data Warehousing concepts.
- FastAPI or Flask.
- Exposure to SonarQube and code quality tools.
- AWS Certifications. [SCALA WITH AWS | Word], [Experian-Python | Word]