02 Aug
|
GainInsights
|
India
02 Aug
GainInsights
India
Location: Chennai / Bangalore
Experience: 3 – 5 years
Employment Type: Full-time
About the role
We are seeking a highly skilled Data Engineer with hands-on experience in Google Cloud Platform (GCP) to design, build, and optimize data pipelines, data models, and analytics infrastructure. You will collaborate closely with data analysts, data scientists, and software engineers to ensure scalable, reliable, and effective data solutions that power business insights and decision-making.
Key responsibilities
Design, develop, and maintain ETL/ELT pipelines using tools such as Dataflow, Dataproc(Apache Beam), Cloud Composer (Airflow) and DBT.
Build and optimize data warehouses and data lakes on BigQuery and Cloud Storage (GCS).
Work with structured and semi-structured data from multiple sources and ensure data quality, lineage, and governance.
Exposure to CI/CD pipelines for data workflows using Cloud Build, Terraform & helm charts.
Implement & workw ith data transformation frameworks (DBT, SQL, Spark, or Python-based frameworks).
Optimize query performance and storage costs within BigQuery and GCS.
Collaborate with cross-functional teams to define data models, schemas, and partitioning strategies.
Integrate with third-party APIs and streaming platforms to process real-time data.
Ensure security, compliance,
and best practices in data architecture and access management.
Monitor, debug, and maintain existing pipelines to ensure data freshness and reliability.
Required qualifications
Bachelor’s or Master’s degree in computer science, information systems, or a related field.
3-5 years of experience as a Data Engineer or in a similar data-focused role.
Strong experience with Google Cloud Platform (GCP) — particularly BigQuery, Cloud Storage, Dataflow, Composer, AlloyDB.
Proficient in SQL, Python, Airflow, DBT.
Experience with data modelling, schema design, and partitioning strategies.
Working knowledge of version control (Git) and CI/CD workflows.
Familiarity with infrastructure-as-code tools such as Terraform or Terragrunt.
Strong analytical and problem-solving skills with attention to detail.
Preferred qualifications
Experience with streaming data pipelines using Pub/Sub, KMS and Cloud Spanner.
Exposure to machine learning pipelines (Vertex AI, AI Platform, or MLflow).
Knowledge of containerization and Kubernetes (GKE).
Familiarity with IAM, data governance, metadata management, and security best practices on GCP.
Experience in building data products for analytics or AI/ML use cases.
📌 Data Engineer – Google Cloud Platform (GCP) (India)
🏢 GainInsights
📍 India