- Design, develop, and maintain scalable data pipelines and systems.
- Implement data integration, transformation, and validation processes.
- Optimize data storage and retrieval for performance and cost-efficiency.
- Monitor and troubleshoot data pipeline issues.
Role Responsibilities:
- Collaborate with cross-functional teams to understand data requirements and ensure data quality and integrity.
- Work with data pipeline orchestration tools such as Dagster, Apache Airflow.
- Develop and maintain APIs, supporting microservices and containerization efforts.
- Implement security and compliance measures for data handling.