03 Nov
|
Prodapt
|
Chennai
Overview We are looking for a skilled and detail-oriented GCP Data Engineer to join our growing data engineering team. The ideal candidate will have hands-on experience designing, building, and optimizing scalable data pipelines on Google Cloud Platform, along with strong problem-solving and communication skills to collaborate effectively with cross-functional and global stakeholders. Responsibilities
Key Responsibilities
• Design, develop, and maintain robust, scalable, and efficient data pipelines on Google Cloud Platform (GCP).
• Work extensively with BigQuery (BQ) for data warehousing, query optimization, and large-scale data analysis.
• Build and manage big data processing workflows using Dataproc (Hadoop/Spark ecosystem on GCP).
• Write, optimize, and troubleshoot complex SQL queries for data transformation, validation, and reporting.
• Design, schedule, and monitor data workflows using Cloud Composer (Apache Airflow).
• Perform root-cause analysis and troubleshooting of pipeline failures, data quality issues, and performance bottlenecks in production environments.
• Collaborate with data analysts, data scientists, and business stakeholders to understand requirements and deliver reliable data solutions.
• Ensure data governance, security, and compliance standards are followed across all GCP data engineering solutions.
• Participate in code reviews, design discussions, and knowledge-sharing sessions within the team.
• Document technical designs, data flows, and operational runbooks for supportability.
Requirements Required Technical Skills SkillProficiency Expected Google Cloud Platform (GCP)Strong hands-on experience BigQuery (BQ)Query optimization, partitioning, clustering, cost management DataprocSpark/Hadoop job design and management SQLAdvanced query writing, performance tuning Cloud Composer / Apache AirflowDAG design, scheduling, monitoring TroubleshootingStrong analytical and debugging skills for pipeline/data issues Good to Have (Additional Skills)
• Ab Initio – ETL development and pipeline migration experience.
• DevOps – CI/CD pipelines, version control (Git), containerization (Docker/Kubernetes), infrastructure as code (Terraform) is a plus.
• Prior Data/ETL Migration Experience – Hands-on experience migrating data pipelines, ETL workloads, or on-premise data warehouses to cloud platforms (e.g., on-prem to GCP, or cross-cloud migrations) is a strong value add. Soft Skills
• Excellent communication skills – ability to clearly articulate technical concepts to both technical and non-technical stakeholders.
• Strong ownership mindset and ability to work independently as well as within a team.
• Good stakeholder management skills, especially in a global/MNC delivery environment.
• Ability to work under pressure and manage multiple priorities in a rapid-paced environment. Qualifications
• Bachelor's/Master's degree in Computer Science, Information Technology, Engineering, or a related field.
• GCP certification (e.g., Professional Data Engineer) is a plus.
📌 DT - Technical Architect (Chennai)
🏢 Prodapt
📍 Chennai