Expereince: 8 to 12 Years

Role Summary:

The primary responsibility of this role is to implement and manage comprehensive observability and monitoring for Databricks and Apache Spark workloads. The objective is to ensure high platform reliability, optimal performance, and transparent cost tracking.

Key Responsibilities:-

Continuously monitor Spark jobs, clusters, SQL warehouses, and data pipelines to identify performance issues, failures, or breaches of service level agreements (SLAs).

- Develop intuitive dashboards and automated alerts by leveraging Databricks system tables, including those for jobs, clusters, billing, audits, and query history.
- Proactively address operational challenges through troubleshooting, root cause analysis, and performance tuning of Spark workloads.
- Facilitate cost, usage, and capacity transparency across Databricks environments, empowering teams to optimise resource allocation.
- Seamlessly integrate Databricks monitoring capabilities with enterprise-grade tools such as Azure Monitor, Log Analytics, or similar platforms.

Required Skills:-

Demonstrated, hands-on expertise in working with Databricks and Apache Spark for both batch and streaming processes.- In-depth understanding of Databricks system tables, job orchestration, cluster management,



and workflow automation.- Experience with data and platform observability frameworks, as well as advanced alerting and monitoring solutions.- Proficiency in SQL, PySpark, and dashboard creation to deliver actionable operational insights.

Reinvent your world. We are building a up-to-date Wipro. We are an end-to-end digital transformation partner with the boldest ambitions. To realize them, we need people inspired by reinvention. Of yourself, your career, and your skills. We want to see the constant evolution of our business and our industry. It has always been in our DNA - as the world around us changes, so do we. Join a business powered by purpose and a place that empowers you to design your own reinvention.

Profile description

Demonstrated, hands-on expertise in working with Databricks and Apache Spark for both batch and streaming processes.- In-depth understanding of Databricks system tables, job orchestration, cluster management, and workflow automation.- Experience with data and platform observability frameworks, as well as advanced alerting and monitoring solutions.- Proficiency in SQL, PySpark, and dashboard creation to deliver actionable operational insights.

📌 AWS/Databricks Lead (Mumbai)
🏢 Wipro
📍 Mumbai