04 Aug
|
TRIGENT SOFTWARE PRIVATE
|
Bengaluru
04 Aug
TRIGENT SOFTWARE PRIVATE
Bengaluru
We are seeking an experienced Databricks Administrator with 5+ years of IT experience and a minimum of 4+ years of hands-on platform administration experience on Databricks.
In this role, you will be responsible for the end-to-end administration, security, maintenance, and operational support of our enterprise Databricks platform across cloud environments (AWS / Azure / GCP). You will work closely with Data Engineers, Data Scientists, and Cloud Security teams to ensure high availability, strong governance, and optimized compute usage.
Key Responsibilities 1. Platform Administration & Workspace Governance
- Provision, configure, and maintain multi-workspace Databricks environments across Dev, QA, and Production.
- Configure and manage Unity Catalog, metastores, storage credentials, external locations, and object-level permissions.
- Manage user onboarding, group sync via SCIM, SSO/SAML integration, and Service Principals.
- Enforce platform standardization across all workspaces using Cluster Policies, global Init Scripts, and Databricks Asset Bundles (DABs).
2. Security, Compliance & Access Control
- Implement and enforce Role-Based Access Control (RBAC) for tables, schemas, views, and compute resources.
- Configure secure networking including VNet/VPC Peering, Private Link/Private Endpoints, and IP access lists.
- Monitor audit logs, query execution history, and system tables to ensure data governance and regulatory compliance.
3. Compute Management, Optimization & Cost Control
- Configure, monitor, and optimize Spark cluster settings, instance types, autoscaling, and driver/worker nodes.
- Track DBU (Databricks Unit) usage and implement cost-saving strategies like auto-termination, spot instances, and compute sizing limits.
- Resolve day-to-day platform incidents including cluster boot failures,
Out of Memory (OOM) errors, library conflicts, and failed jobs.
4. Automation & Dev Ops Support
- Automate workspace setup, policies, and permissions using Terraform (Databricks Provider) and Databricks REST APIs / CLI.
- Support CI/CD pipelines (Git Hub Actions, Azure Dev Ops, Git Lab) for code deployment and automated data job execution.
- Approve, install, and manage Python/Scala/R library deployments across compute clusters.
5. Technical Support & Stakeholder Coordination
- Serve as the primary administrator handling tier-2/3 platform support tickets from data teams.
- Coordinate with internal Cloud Infrastructure, Networking, and Information Security teams.
- Manage technical escalations and ticket resolutions directly with Databricks Vendor Support.
Required Skills & Qualifications
- Dedicated Experience: 4+ years of direct, hands-on Databricks Platform Administration experience (within 5+ years of total IT experience).
- Databricks Deep-Dive: Mastery of Unity Catalog, Delta Lake, Cluster Policies, Workspaces, REST APIs, and Databricks CLI.
- Cloud Platform Administration: Robust working knowledge of cloud primitives on AWS (S3, IAM, VPC), Azure (ADLS Gen2, Entra ID, VNets), or GCP.
- Infrastructure as Code (IaC): Hands-on experience writing and deploying Terraform for Databricks resources.
- Scripting: Proficiency in Python and Shell/Bash scripting for operational task automation.
- Troubleshooting: Deep ability to analyze Spark logs, cluster metrics, and system tables to resolve operational issues quickly.
Preferred Qualifications
- Databricks Certified Platform Administrator / Associate / Professional.
- Cloud Administration Certification (AWS Sys Ops / Azure Administrator).
- Experience with Delta Live Tables (DLT), MLflow workspace administration, or Serverless Compute configurations.
📌 Databricks Administrator __PAN India__Hybrid (Bengaluru)
🏢 TRIGENT SOFTWARE PRIVATE
📍 Bengaluru