Job Summary

We are seeking a skilled GCP Data Engineer with 5+ years of experience in designing, developing, and maintaining scalable data pipelines and cloud-based data solutions on Google Cloud Platform (GCP). The ideal candidate should have strong expertise in BigQuery, GCS, Pub/Sub, Cloud Composer (Airflow), DBT, data quality frameworks, and monitoring solutions.

Key Responsibilities

- Design, develop, and maintain scalable ETL/ELT data pipelines on GCP.
- Build and optimize data ingestion frameworks using Pub/Sub, GCS, and other GCP services.
- Develop and manage workflows using Cloud Composer (Apache Airflow).
- Create and optimize data models and transformations using DBT.
- Implement robust data quality checks, validation rules, and data quality scans.
- Design and maintain enterprise-scale data warehouses in BigQuery.
- Monitor production pipelines and implement proactive monitoring and alerting mechanisms.
- Troubleshoot and resolve data pipeline, performance, and quality issues.
- Collaborate with data architects, analysts, and business stakeholders to deliver data solutions.
- Ensure adherence to security, governance, and best practices across data platforms.
- Optimize query performance and cloud infrastructure costs.

Mandatory Skills





Google Cloud Platform (GCP)

- Google Cloud Storage (GCS)
- BigQuery
- Pub/Sub
- Cloud Composer (Apache Airflow)
- Monitoring & Alerting

Data Engineering
- ETL/ELT Development
- Data Pipeline Design and Development
- Data Warehousing Concepts
- Advanced SQL

Data Quality & Governance
- Data Quality Scans
- Data Validation Frameworks
- Data Monitoring and Observability

Data Transformation
- DBT (Data Build Tool)
- Data Modeling
- Performance Optimization

Programming
- Python
- SQL

Required Experience
- 5+ years of experience in Data Engineering.
- Minimum 3+ years of hands-on experience in GCP Data Engineering solutions.
- Robust experience with BigQuery, GCS, Pub/Sub, Cloud Composer (Airflow), and DBT.
- Experience in implementing monitoring, alerting, and data quality frameworks.
- Hands-on experience in designing and supporting production-grade data pipelines.

Preferred Skills
- Dataflow
- Terraform
- CI/CD Pipelines
- GitHub/GitLab
- Spark/PySpark
- Kubernetes

Educational Qualification
- Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.

📌 GCP Data Engineer (Tamil Nadu)
🏢 Tata Consultancy Services
📍 Tamil Nadu