16 Aug
|
Navlakha Management Services
|
Pune
16 Aug
Navlakha Management Services
Pune
We are hiring a Databricks Platform Engineer
Description
Are you passionate about building enterprisegrade data platforms that power both advanced analytics and nextgeneration Agentic AI solutions?
Join us as a Databricks Platform Engineer, where you will play a key role in engineering, scaling, and operating our enterprise Databricks platform across a highly regulated, multicloud environment. You will enable data engineers, AI/ML teams, and downstream platform consumers by building a secure,
automated, and wellgoverned data foundation.
In this role, you will work at the intersection of data engineering, platform engineering, and AI enablement.
You will design and automate cloud-native platform components, implement governance frameworks,
optimize performance, and ensure compliance with strict security and dataprivacy requirements.
What you do
Platform Engineering & Operations
- Engineer, deploy, and operate Databricks workspaces, clusters, jobs, and ML runtimes across Azure, AWS, and multi-cloud architectures.
- Build and manage automated CI/CD pipelines, infrastructure-as-code (IaC), and platform lifecycle processes (provisioning, scaling, upgrades).
- Ensure platform reliability, observability, and cost efficiency through proactive monitoring, tuning,
and optimization.
Data Platform & AI Enablement
- Enable scalable data ingestion, transformation, and processing pipelines using Delta Lake, Unity
Catalog, and Databricks Jobs/Workflows.
- Collaborate with AI/ML teams to support feature stores, model training pipelines, and data services powering Agentic AI use cases.
- Implement and maintain enterprise-wide metadata, lineage, and data quality controls.
Governance, Security & Compliance
- Ensure strong security baselines, including Identity & Access Management (IAM), network isolation, data encryption, secret management,
and secure cluster configurations.
- Implement governance guardrails aligned with regulatory, compliance, and risk frameworks (e.g.,
GDPR, financial regulations).
- Enforce platform standards, policies, cluster controls, and workspace configurations across
Operating Entities.
Automation & Tooling
- Build reusable automation components, templates, and frameworks for platform onboarding and daytoday operations.
- Support the development of monitoring dashboards, alerting systems, and platform health indicators.
Collaboration & Support
- Work with data engineers, data scientists, AI architects, cloud teams, and security teams to ensure seamless integration of Databricks with enterprise data and AI ecosystems.
- Troubleshoot platform issues and support stakeholders in building robust, scalable workloads.
- Contribute to platform documentation, best practices, and engineering playbooks.
What you bring
- Hands-on engineering experience with Databricks in enterprise environments (Azure or AWS;
multi-cloud preferred).
- Strong understanding of Delta Lake, Unity Catalog, cluster architectures, Databricks
Jobs/Workflows, and ML runtimes.
- Solid expertise in IaC tools such as Terraform, Pulumi, or ARM/Bicep, and CI/CD platforms such as GitHub Actions, Azure DevOps, or GitLab.
- Deep knowledge of data platform architecture, including ingestion pipelines, ETL/ELT patterns,
data governance, lineage, and data quality frameworks.
- Experience operating platforms in regulated, security-sensitive industries, ensuring compliance and strong platform governance.
- Familiarity with enabling AI/ML workloads, data products, feature engineering, or MLOps pipelines.
- Solid grounding in cloud-native engineering, distributed systems, and event-driven architectures.
- Ability to work collaboratively across cross-functional engineering, cloud, security, and AI teams.
📌 Databricks Platform Engineer (Pune)
🏢 Navlakha Management Services
📍 Pune