Job Summary

Design and deliver end‑to‑end data pipelines on cloud platforms (GCP preferred).Build scalable data ingestion, transformation, and processing workflows using distributed technologies such as Spark, Flink, Storm, or similar.Develop robust ELT/ETL processing and migration pipelines, including support for legacy Datastage decommissioning and modernisation.Work with a variety of database technologies including relational, NoSQL, MPP and columnar stores (BigQuery, Redshift, Azure SQLDW, HBase, MongoDB).Implement streaming and messaging‑based pipelines using Kafka, Pulsar or Pub/Sub.Build optimised, scalable data models to support diverse consumption patterns, applying partitioning, sharding, bucketing and aggregation strategies.Apply performance tuning and optimisation across storage, compute and query layers.

Key Responsibilities

Design and deliver end‑to‑end data pipelines on cloud platforms (GCP preferred).Build scalable data ingestion, transformation,



and processing workflows using distributed technologies such as Spark, Flink, Storm, or similar.Develop robust ELT/ETL processing and migration pipelines, including support for legacy Datastage decommissioning and modernisation.Work with a variety of database technologies including relational, NoSQL, MPP and columnar stores (BigQuery, Redshift, Azure SQLDW, HBase, MongoDB).Implement streaming and messaging‑based pipelines using Kafka, Pulsar or Pub/Sub.Build optimised, scalable data models to support diverse consumption patterns, applying partitioning, sharding, bucketing and aggregation strategies.Apply performance tuning and optimisation across storage, compute and query layers.

Skill Requirements

Must Have Skill

Databricks,Google Dataproc,MS SQL,Python

Valuable to Have Skill

Other Requirements

📌 Senior Technical Architect (India)
🏢 HCLTech
📍 India