- Design, develop, and maintain scalable data pipelines and systems.
- Implement data integration, transformation, and validation processes.
- Optimize data storage and retrieval for performance and cost-efficiency.
- Monitor and troubleshoot data pipeline issues.
Role Responsibilities:
- Collaborate with cross-functional teams to understand data requirements and ensure data quality and integrity.
- Work with data pipeline orchestration tools such as Dagster or Apache Airflow.
- Implement security and compliance measures for data handling.
- Develop APIs and support microservices and application containerization efforts.