Job Summary

- Administer and operate enterprise Databricks workspaces across development, QA, production, and disaster recovery environments.
- Manage Databricks workspace configuration including clusters, jobs, pools, policies, secrets, and init scripts.
- Design, configure, and enforce cluster policies to ensure security, stability, and cost control.
- Manage Databricks compute including autoscaling, instance pools, job clusters, and all‑purpose clusters.
- Administer Databricks Jobs, Workflows, and scheduling for batch and streaming workloads.
- Configure and manage Databricks security including workspace access controls, IAM integration, permissions, and secrets management.
- Implement and manage Databricks Unity Catalog for data governance, access control, and lineage.
- Configure and support integration with cloud storage (Azure Data Lake, S3, GCS) and external data sources.
- Administer Delta Lake features including table management, versioning, optimization, and retention policies.
- Monitor platform health, performance, and availability using logs, metrics, and alerts.
- Troubleshoot Spark workloads including job failures, performance degradation, memory issues, and scaling problems.
- Support ML and AI workloads using Databricks ML Runtime, Model Registry, Feature Store, and notebooks.
- Implement CI/CD processes for Databricks notebooks, jobs,



and configurations using Git-based workflows.
- Manage Databricks REST APIs, CLI tools, and automation scripts for platform operations.
- Perform capacity planning, usage analysis, and cost optimisation across Databricks environments.
- Apply platform patches, runtime upgrades, and configuration changes following change management best practices.
- Act as the escalation point for Databricks production incidents and lead root cause analysis.
- Maintain operational documentation, standards, and platform runbooks.
- Collaborate with data engineers, data scientists, MLOps, cloud, and security teams.
- Provide on‑call production support through a rotation schedule.

Key Responsibilities

Skill Requirements

- Databricks Administrator or Data Engineer certification.
- Experience supporting ML and AI workloads on Databricks.
- Experience with streaming technologies such as Spark Structured Streaming or Kafka.
- Familiarity with Infrastructure as Code tools such as Terraform for Databricks.
- Experience working in regulated or security‑sensitive environments.
- Familiarity with ITIL‑based enterprise service management processes.

Must Have Skill

Azure Data Lake,Azure Databricks,CI/CD,Python,AI Platform Engineering

Positive to Have Skill

Other Requirements

📌 Senior Technical Lead (India)
🏢 HCLTech
📍 India