Senior Data Engineer – Data Pipelines & Cloud Databases Location: Pune | Experience: 4+ years | Type: Full time

Role Overview

Design, build, manage enterprise data pipelines on Azure and Databricks. Own schema design, API development, and data infrastructure for analytics and intelligence products.

Key Responsibilities

Data Pipeline Development

Implement robust, scalable data pipelines using Microsoft Azure and Databricks stack

Build reusable data pipeline components and frameworks

Design and optimize data workflows for performance and reliability

Monitor pipeline health and implement automated alerting

Database Architecture

Design relational and non-relational database schemas

MongoDB: schema design, aggregation pipelines, indexing, sharding, replica sets, performance tuning

Azure Cosmos DB: multi-model access, partitioning, consistency levels, throughput management (RU/s), global distribution

Azure Cosmos DB Gremlin API: graph data modeling, traversals, vertex/edge design, relationship analytics

API Development





Build RESTful APIs using FastAPI with async endpoint design

Implement Pydantic models, middleware, dependency injection, background tasks

Generate and maintain auto-generated OpenAPI/Swagger documentation

Configure Azure API Management (APIM) for API gateway and security

Project & Stakeholder Management

Collect progress updates from squads regularly

Consolidate updates into weekly project status reports

Develop L3-level project plans with task details, milestones, dependencies

Participate in early-stage design and feature definition

Communicate complex data insights to non-technical stakeholders

Collaboration & Integration

Work across multiple engineering teams on prototype integration

Support integration of proven prototypes into core intelligence products

Strong team collaboration and cross-functional communication

Knowledge sharing and documentation

Required Experience:

Data Engineering & Databases

~4+ years data engineeri

📌 IT Data Architect (Pune)
🏢 InfoVision
📍 Pune