Requirement ID: 10373065
Job Title: Engineer(Job Title: Data Engineer – GCP Framework Implementation)
Work Location: Pune, Hyderabad, Chennai, Bangalore, Delhi
Duration: 6 months
Skills Required: Digital: Kubernetes, Digital: Google Data Engineering
Experience Range in Required Skills: 5+ Years
Job Description: GCP (E2), Big Query Cloud Composer, GKE
Role Overview:
We are looking for a Data Engineer with hands-on experience in Google Cloud Platform (GCP) services to work on implementing and managing data ingestion using our in- house Origin Data Product (ODP) framework. The role primarily involves configuring pipelines, loading data, debugging issues, and ensuring smooth operation of the data ingestion process. Should be capable of handling parameter changes, data issues, and basic fixes within the scope of ingestion
jobs. If you understand the framework well, you are encouraged to suggest improvements and may contribute to enhancements in collaboration with the core
development team.
Key Responsibilities:
• Configure and execute data ingestion pipelines using our reusable GCP-based framework.
• Work with services such as GCS, BigQuery, Composer, Data Fusion, DataProc for ETL/ELT operations.
• Manage parameters, job configurations, and metadata for ingestion.
• Debug and resolve issues related to data, parameters, and job execution.
• Escalate framework-related bugs to the core development team when required.
• Monitor daily job runs, troubleshoot failures, and ensure SLAs are met.
• Collaborate with cross-functional teams for smooth delivery.
• Follow version control best practices using Git.
• Maintain deployment scripts and infrastructure configurations via Terraform.
• (Nice-to-have) Work with TWSd job scheduling/monitoring tools.
• Suggest improvements to the framework or processes based on usage experience.
• Contribute to small enhancements in the framework, where applicable.
Required Skills & Experience:
• 5+ years of experience as a
📌 Data Engineer – GCP Framework Implementation (India)
🏢 Clifyx
📍 India