Description
Data Engineering & Development
- Design, develop, and maintain data pipelines on GCP.
- Build scalable ETL/ELT solutions for ingesting, transforming, and loading data from multiple source systems.
- Develop batch and real-time data processing solutions.
- Optimize data workflows for performance, reliability, and scalability.
GCP Platform Development
- Work with GCP services including:
- BigQuery
- Dataflow
- Dataproc
- Pub/Sub
- Cloud Storage
- Cloud Composer (Airflow)
- Cloud Functions
- Implement cloud-native data architectures and best practices.
Data Warehousing & Analytics
- Design and maintain data warehouse solutions in BigQuery.
- Develop data models to support reporting and analytics requirements.
- Ensure data integrity, quality, and consistency across platforms.
Production Support & Monitoring
- Support production environments and troubleshoot data pipeline issues.
- Monitor scheduled jobs and resolve failures in a timely manner.
- Perform root cause analysis and implement permanent fixes.
- Ensure adherence to SLA and operational requirements.
Collaboration
- Work closely with Data Architects, Business Analysts, Product Owners, and Reporting teams.
- Participate in Agile ceremonies including sprint planning, daily stand-ups, and retrospectives.
- Contribute to technical design discussions and code reviews.