Description
Databricks Platform Manager / Engineer (AWS)
Experience
5–10 years of overall IT experience, including 3–5+ years of hands-on experience administering and managing Databricks platforms in production environments on AWS.
Job Summary
We are looking for a Databricks Platform Manager/Engineer responsible for designing, administering, securing, and optimizing the enterprise Databricks platform on AWS. The ideal candidate will own the platform lifecycle, ensuring scalability, security, governance, cost optimization, and operational excellence while enabling data engineering, analytics, and AI teams to work efficiently.
This role focuses on platform engineering rather than data engineering or analytics development.
Responsibilities
Platform Administration
Design, deploy, and manage Databricks workspaces across Development, Test, UAT, and Production environments.
Establish workspace standards, networking, and setting isolation.
Manage platform upgrades, including Databricks Runtime (DBR) version planning, testing, and rollout.
Maintain platform availability, reliability,
and performance.
Compute & Cluster Management
Create and manage cluster policies to enforce organizational standards.
Configure instance pools and job clusters.
Prevent oversized or non-compliant clusters.
Standardize cluster configurations across teams.
Optimize cluster startup times and resource utilization.
Security & Identity Management
Integrate Databricks with AWS IAM.
Configure and manage authentication and authorization.
Manage service principals, personal access tokens (PATs), and secret scopes.
Implement least-privilege access controls.
Support enterprise identity management and SSO.
Governance & Data Access
Administer Unity Catalog.
Manage catalogs, schemas, tables, and permissions.
Implement role-based access control (RBAC).
Enforce data governance and access policies.
Support audit and compliance requirements.
DevOps & CI/CD
Configure Git integratio
📌 Lead Assistant Manager (India)
🏢 EXL
📍 India