Role Overview
We are looking for an experienced Databricks Data Engineer to support, maintain, and enhance existing Databricks-based data applications and pipelines. The role focuses on ensuring reliability, performance, and scalability of production Databricks workloads rather than building net-new platforms from scratch. You will work closely with data, analytics, and engineering teams to keep critical data applications stable, optimized, and aligned with business needs.
Key Responsibilities
- Support and maintain existing Databricks applications, notebooks, jobs, and Delta Lake pipelines in production.
- Monitor, troubleshoot, and resolve issues related to job failures, performance degradation, data quality, and cluster utilization.
- Optimize existing Spark jobs, SQL queries, and Delta tables for cost, performance, and reliability.
- Manage and improve Databricks workspace configurations, including clusters, job scheduling, access controls, and Unity Catalog (where applicable).
- Implement and maintain data quality checks, logging, alerting,



and basic observability for Databricks workloads.
- Collaborate with stakeholders to understand requirements for enhancements or bug fixes on existing applications.
- Perform incremental improvements, refactoring, and technical debt reduction on current Databricks solutions.
- Ensure adherence to best practices around security, governance, and cost management within the Databricks environment.
- Document existing pipelines, dependencies, and operational runbooks.
- Participate in on-call or support rotations as needed to maintain production stability

Required Qualifications
- 6–9 years of overall experience in data engineering, with strong hands-on experience in Databricks.
- Solid proficiency in Apache Spark (PySpark and/or Scala) and SQL.
- Proven experience supporting and optimizing production Databricks workloads (jobs, notebooks, Delta Lake, workflows).
- Robust understanding of Delta Lake concepts (ACID transactions,

📌 Databricks Data Engineer (Bengaluru)
🏢 Cogniify
📍 Bengaluru