Role : Senior Data Engineer

Experience Level: 4 to 7 Years

Work location : Bangalore (2 days office)

Shift

- Rotational shift including night

Technical Architecture & Delivery

- Lead the end-to-end design and delivery of complex data engineering solutions on GCP — from architecture through production deployment
- Architect scalable, cost-effective data platforms using BigQuery, Dataflow, Cloud Composer, Pub/Sub, Dataplex, and Cloud Storage
- Design robust data models using Dimensional (Kimball), 3NF, and Data Vault methodologies — selecting the right approach for each use case
- Implement SCD strategies and historical data management patterns for long-lived datasets
- Lead the migration of legacy data structures to GCP, defining parallel testing and data parity validation strategies
- Provision and govern cloud infrastructure using Terraform; champion IaC as a non-negotiable standard
- Design and implement CI/CD pipelines for all data solutions — with automated testing, linting, and deployment gates.

AI-Era Responsibilities

- AI-ready architecture: Design every data platform component to be downstream-AI compatible — appropriate partitioning, feature store integration, and schema design for ML consumption
- GenAI data infrastructure: Architect data pipelines for LLM-based applications, including embedding generation pipelines, vector store population, and RAG data retrieval layers
- Feature store engineering: Build and maintain centralised feature stores on Vertex AI, ensuring reproducibility and low-latency serving for ML models
- AI-assisted development leadership: Champion GitHub Copilot, Gemini Code Assist, and Cursor as engineering productivity tools — set standards for how the team uses them responsibly
- AI-powered data quality: Design ML-based anomaly detection into pipeline monitoring — moving beyond threshold alerts to intelligent pattern recognition
- LLMOps data layer: Build the data infrastructure that underpins model evaluation,



fine-tuning dataset curation, and prompt tracking pipelines

Leadership & Collaboration

- Lead code reviews; hold the bar for quality, testability, and maintainability
- Define and document reusable engineering patterns — pipeline templates, transformation standards, naming conventions
- Actively mentor junior engineers through pairing, structured feedback, and technical design sessions
- Work closely with global Data Engineering counterparts to align on platform standards
- Engage directly with senior business stakeholders to translate complex requirements into technical solutions
- Contribute to hiring: review take-home tasks, conduct technical interviews, calibrate assessments
- Define and execute testing strategies for regulated workloads, including parallel-run validation against legacy systems

Operational Excellence

- Own pipeline reliability: define SLAs, implement alerting, lead incident resolution
- Drive DataOps practices: automated testing, data contracts, observability-first design
- Monitor and optimise GCP costs; propose and implement efficiency improvements
- Ensure compliance with data security, encryption, and governance standards in all solutions built

What We're Looking For

Essential Technical

- 5+ years of data engineering experience in production, cloud-native environments
- 5+ years of advanced SQL: BigQuery specifics, query profiling, partitioning/clustering optimisation, complex analytical queries
- 3+ years of GCP production experience: architecture design and delivery at scale
- Deep, hands-on expertise across: BigQuery, Dataflow, Cloud Composer (Airflow), Pub/Sub, Dataplex, Cloud Storage, Terraform, Cloud Build
- Mastery of data modelling methodologies: Dimensional/Kimball, 3NF, Data Vault — with real world application of each
- Production-level Python: OOP design patterns, async processing, unit/integration testing, GCP SDK usage
- Demonstrated experience designing CI/CD pipelines for data products
- Track record of leading legacy-to-cloud migrations

📌 Senior Data Engineer (GCP) (Bengaluru)
🏢 Quantiphi
📍 Bengaluru