Senior Data Engineer – Data Pipelines & Cloud Databases /n Location: Pune | Experience: 4+ years | Type: Full time /n Role Overview /n Design, build, manage enterprise data pipelines on Azure and Databricks. Own schema design, API development, and data infrastructure for analytics and intelligence products. /n Key Responsibilities /n Data Pipeline Development /n /n

- Implement robust, scalable data pipelines using Microsoft Azure and Databricks stack

/n

- Build reusable data pipeline components and frameworks

/n

- Design and optimize data workflows for performance and reliability

/n

- Monitor pipeline health and implement automated alerting

/n /n Database Architecture /n /n

- Design relational and non-relational database schemas

/n

- MongoDB: schema design, aggregation pipelines, indexing, sharding, replica sets, performance tuning

/n

- Azure Cosmos DB: multi-model access, partitioning, consistency levels, throughput management (RU/s), global distribution

/n

- Azure Cosmos DB Gremlin API: graph data modeling, traversals, vertex/edge design, relationship analytics

/n /n API Development /n /n

- Build RESTful APIs using FastAPI with async endpoint design

/n

- Implement Pydantic models, middleware, dependency injection, background tasks

/n

- Generate and maintain auto-generated OpenAPI/Swagger documentation

/n

- Configure Azure API Management (APIM) for API gateway and security

/n /n Project & Stakeholder Management /n /n

- Collect progress updates from squads regularly

/n

- Consolidate updates into weekly project status reports

/n

- Develop L3-level project plans with task details, milestones, dependencies

/n

- Participate in early-stage design and feature definition

/n

- Communicate complex data insights to non-technical stakeholders





/n /n Collaboration & Integration /n /n

- Work across multiple engineering teams on prototype integration

/n

- Support integration of proven prototypes into core intelligence products

/n

- Strong team collaboration and cross-functional communication

/n

- Knowledge sharing and documentation

/n /n Required Experience: /n Data Engineering & Databases /n /n

- 4+ years data engineering or data platform experience

/n

- Relational and non-relational database expertise

/n

- MongoDB: advanced schema design, aggregation, indexing, sharding, performance optimization

/n

- Azure Cosmos DB: multi-model APIs, partitioning, consistency, throughput management

/n

- Graph databases: Cosmos DB Gremlin API, graph modeling, traversals, relationship analytics

/n /n API & Backend Development /n /n

- FastAPI proficiency: async endpoints, Pydantic models, middleware, dependency injection

/n

- Background tasks and job scheduling

/n

- RESTful API design and best practices

/n

- OpenAPI/Swagger documentation

/n /n Cloud Platform /n /n

- Azure fundamentals and hands-on experience

/n

- Azure Data Factory or Databricks for ETL/ELT

/n

- Azure Cosmos DB multi-region setup

/n

- Azure API Management (APIM) configuration

/n /n Data Pipeline Skills /n /n

- ETL/ELT pipeline design and development

/n

- Data quality validation and monitoring

/n

- Schema design for analytics and reporting

/n

- Performance optimization and scalability

/n /n Preferred Experience /n /n

- Databricks Delta Lake experience

/n

- Azure Synapse Analytics

/n

- Python for data engineering

/n

- Spark SQL optimization

/n

- Real-time data streaming

/n

- Data governance and metadata management

/n

- Agile/Scrum development model

/n

- CI/CD pipeline experience

/n

📌 Data Engineer (Pune)
🏢 InfoVision
📍 Pune