Role Overview

Seasoned Data-bricks Senior Platform Engineer who can implement
scalable, end-to-end data solutions on Data-bricks over AWS. This role is
pivotal in building the foundational infrastructure for data products, ensuring
data quality, and enabling machine learning capabilities

Key
Responsibilities

- Design and implement end-to-end
ETL pipelines in Databricks, sourcing data from Amazon S3 and Amazon RDS
and other sources.

- Design and deploy Data-bricks
workspaces and clusters optimized for performance, scalability, and
cost-efficiency.

- Set up data quality frameworks
and anomaly detection mechanisms to ensure reliability and trust in data
products.

- Enable machine learning
capabilities within Data-bricks by configuring necessary services,
libraries, and integrations.

- Collaborate with data
scientists, engineers, and DevOps teams to ensure seamless platform
operations.

- Automate deployment and
monitoring using CI/CD pipelines, GitHub

- Document architecture,
workflows, and operational procedures for platform governance.

Requirements

Core Platform & Cloud

- 5+ years of experience in
platform engineering roles





- Strong expertise in Databricks
on AWS

- Proficient with Amazon S3,
Amazon RDS, and IAM roles/policies

- Experience with Delta Lake,
Structured Streaming, and Data-bricks Workflows

ETL & Data
Quality

- Hands-on experience building ETL
pipelines using Spark

- Familiarity with data
validation, profiling, and anomaly detection frameworks

- Knowledge of tools like Outstanding Expectations
or custom validation scripts

ML Platform Enablement

- Experience
setting up ML infrastructure including:

- MLflow for experiment tracking
and model registry

- Feature Store for reusable
feature pipelines

- Data-bricks Runtime for ML

- Integration with AWS SageMaker
or custom ML libraries

- Understanding of model lifecycle
management, though model development is out of scope

DevOps & Automation

- Proficiency with GitHub, CI/CD
pipelines, and Terraform or CloudFormation

Preferred
Qualifications

- Experience working with
healthcare data

- Exposure to data mesh or data
product architecture principles

📌 Databricks Platform Engineer (Bengaluru)
🏢 Important Group
📍 Bengaluru