As a key member of the Data Team at Equinix, we are seeking a skilled GCP Data Engineer who will be responsible for end-to-end development of Data Engineering use cases, Equinix Data Lake platform and tools. You will design, build, and maintain scalable data infrastructure and analytics solutions on Google Cloud Platform. The ideal candidate will have strong expertise in cloud-native data technologies and a passion for building robust, effective data pipelines that drive business insights.
Responsibilities
- Design, develop, and maintain scalable ETL/ELT pipelines using Cloud Dataflow, Cloud Composer (Apache Airflow), Dataform/dbt and Cloud Functions
- Build real-time streaming data pipelines using Cloud Pub/Sub, Kafka and Dataflow
- Implement automated data quality checks and monitoring across all data workflows
- Optimize pipeline performance and cost efficiency through proper resource allocation and scheduling
- Architect and implement data lake and data warehouse solutions using Dataproc, BigQuery, Cloud Storage, and Cloud SQL
- Design optimal data models,
partitioning strategies, and clustering for analytical workloads
- Manage data lifecycle policies and implement automated archival and retention strategies
- Ensure data security, encryption, and access control across all storage layers
- Build and optimize BigQuery datasets for analytics and reporting use cases
- Create and maintain dimensional models and fact tables for business intelligence
- Implement data marts and aggregation layers for improved query performance
- Support self-service analytics through proper data cataloging and documentation
- Having good knowledge of Dataplex and Analytics hub
- Integrate data from various sources including databases, APIs, SaaS applications, and file systems
- Implement change data capture (CDC) solutions for real-time data synchronization
- Work with third-party data providers and external data feeds
- Implement comprehensive monitoring and alerting us