Senior Data Engineer

Location:
Bangalore (Hybrid - 3 days from client office)

Experience:
8-10 Years

Job Summary:
We are seeking a highly skilled Senior Data Engineer with deep expertise in Azure Databricks and Cosmos DB. The ideal candidate will be responsible for designing, building, and optimizing scalable, cloud-native data platforms and data pipelines. This role requires a hands-on approach to developing high-performance data solutions that support analytics, AI/ML initiatives, real-time processing, and enterprise reporting requirements while ensuring data quality, security, and operational excellence.

Responsibilities:1. Data Engineering & Pipeline Development
- Design, develop, and maintain scalable batch and real-time data pipelines using Databricks and Apache Spark.
- Build robust ETL/ELT processes to ingest, transform, and curate large volumes of structured and unstructured data.
- Develop reusable data engineering frameworks and accelerators.
- Optimize data processing jobs for performance, scalability, and cost efficiency.

2. Databricks Platform Development




- Build and maintain data solutions using Azure Databricks.
- Develop notebooks and workflows using Apache Spark RDD, Apache Spark SQL, and Python.
- Implement Delta Lake architectures for reliable and effective data management.
- Configure and optimize Databricks clusters and workloads.
- Support CI/CD implementation for Databricks deployments.

3. Cosmos DB Development & Optimization
- Design and implement scalable solutions using Azure Cosmos DB (NoSQL).
- Model data effectively for NoSQL workloads.
- Optimize partition strategies, indexing policies, and query performance.
- Manage data replication, consistency models, and throughput provisioning.
- Integrate Cosmos DB with analytics and downstream systems.

4. Azure Data Platform Integration
- Develop and integrate end-to-end solutions using Azure services such as Azure Data Factory, Azure Functions, and Azure Logic Apps.

5. Real-Time & Stre

📌 TECHNICAL LEAD (Bengaluru)
🏢 Happiest Minds Technologies
📍 Bengaluru