Role Overview

Seasoned Data-bricks Senior Platform Engineer who can implement
scalable, end-to-end data solutions on Data-bricks over AWS. This role is
pivotal in building the foundational infrastructure for data products, ensuring
data quality, and enabling machine learning capabilities

Key
Responsibilities
Design and implement end-to-end
ETL pipelines in Databricks, sourcing data from Amazon S3 and Amazon RDS
and other sources.
Design and deploy Data-bricks
workspaces and clusters optimized for performance, scalability, and
cost-efficiency.
Set up data quality frameworks
and anomaly detection mechanisms to ensure reliability and trust in data
products.
Enable machine learning
capabilities within Data-bricks by configuring necessary services,
libraries, and integrations.
Collaborate with data
scientists, engineers, and DevOps teams to ensure seamless platform
operations.
Automate deployment and
monitoring using CI/CD pipelines, GitHub
Document architecture,
workflows, and operational procedures for platform governance.

Requirements

Core Platform & Cloud
5+ years of experience in
platform engineering roles




Robust expertise in Databricks
on AWS
Proficient with Amazon S3,
Amazon RDS, and IAM roles/policies
Experience with Delta Lake,
Structured Streaming, and Data-bricks Workflows

ETL & Data
Quality
Hands-on experience building ETL
pipelines using Spark
Familiarity with data
validation, profiling, and anomaly detection frameworks
Knowledge of tools like Outstanding Expectations
or custom validation scripts

ML Platform Enablement
Experience
setting up ML infrastructure including:
MLflow for experiment tracking
and model registry
Feature Store for reusable
feature pipelines
Data-bricks Runtime for ML
Integration with AWS SageMaker
or custom ML libraries
Understanding of model lifecycle
management, though model development is out of scope

DevOps & Automation
Proficiency with GitHub, CI/CD
pipelines, and Terraform or CloudFormation

Preferred
Qualifications
Experience working with
healthcare data
Exposure to data mesh or data
product architecture principles

📌 Databricks Platform Engineer Bengaluru (India)
🏢 Important Group
📍 India