05 Sep
|
Navlakha Management Services
|
Pune
05 Sep
Navlakha Management Services
Pune
We are hiring a Databricks Platform Engineer
Description
Are you passionate about building enterprisegrade data platforms that power both advanced analytics
and nextgeneration Agentic AI solutions?
Join us as a Databricks Platform Engineer, where you will play a key role in engineering, scaling, and
operating our enterprise Databricks platform across a highly regulated, multicloud environment. You will
enable data engineers, AI/ML teams, and downstream platform consumers by building a secure,
automated, and wellgoverned data foundation.
In this role, you will work at the intersection of data engineering, platform engineering, and AI
enablement.
You will design and automate cloud-native platform components, implement governance frameworks,
optimize performance, and ensure compliance with strict security and dataprivacy requirements.
What you do
Platform Engineering & Operations
• Engineer, deploy, and operate Databricks workspaces, clusters, jobs, and ML runtimes
across Azure, AWS, and multi-cloud architectures.
• Build and manage automated CI/CD pipelines, infrastructure-as-code (IaC), and platform lifecycle
processes (provisioning, scaling, upgrades).
• Ensure platform reliability, observability, and cost efficiency through proactive monitoring, tuning,
and optimization.
Data Platform & AI Enablement
• Enable scalable data ingestion, transformation, and processing pipelines using Delta Lake, Unity
Catalog, and Databricks Jobs/Workflows.
• Collaborate with AI/ML teams to support feature stores, model training pipelines, and data
services powering Agentic AI use cases.
• Implement and maintain enterprise-wide metadata, lineage, and data quality controls.
Governance, Security & Compliance
• Ensure strong security baselines, including Identity & Access Management (IAM), network
isolation, data encryption, secret management,
and secure cluster configurations.
• Implement governance guardrails aligned with regulatory, compliance, and risk frameworks (e.g.,
GDPR, financial regulations).
• Enforce platform standards, policies, cluster controls, and workspace configurations across
Operating Entities.
Automation & Tooling
• Build reusable automation components, templates, and frameworks for platform onboarding and
daytoday operations.
• Support the development of monitoring dashboards, alerting systems, and platform health
indicators.
Collaboration & Support
• Work with data engineers, data scientists, AI architects, cloud teams, and security teams to
ensure seamless integration of Databricks with enterprise data and AI ecosystems.
• Troubleshoot platform issues and support stakeholders in building robust, scalable workloads.
• Contribute to platform documentation, best practices, and engineering playbooks.
What you bring
• Hands-on engineering experience with Databricks in enterprise environments (Azure or AWS;
multi-cloud preferred).
• Strong understanding of Delta Lake, Unity Catalog, cluster architectures, Databricks
Jobs/Workflows, and ML runtimes.
• Solid expertise in IaC tools such as Terraform, Pulumi, or ARM/Bicep, and CI/CD platforms such
as GitHub Actions, Azure DevOps, or GitLab.
• Deep knowledge of data platform architecture, including ingestion pipelines, ETL/ELT patterns,
data governance, lineage, and data quality frameworks.
• Experience operating platforms in regulated, security-sensitive industries, ensuring
compliance and robust platform governance.
• Familiarity with enabling AI/ML workloads, data products, feature engineering, or MLOps
pipelines.
• Strong grounding in cloud-native engineering, distributed systems, and event-driven
architectures.
• Ability to work collaboratively across cross-functional engineering, cloud, security, and AI teams.
📌 Databricks Platform Engineer (Pune)
🏢 Navlakha Management Services
📍 Pune