We are looking for a Lead Data Engineer / Technical Anchor with solid expertise in Databricks on AWS to lead the architecture, engineering, and delivery of enterprise-scale Data Lakehouse solutions. This is a hands-on technical leadership role involving architecture, development, mentoring, and stakeholder collaboration.

Required Skills

Databricks

AWS (S3, EMR, Athena, Glue, CloudWatch)

PySpark / Apache Spark

Python

Advanced SQL

Delta Lake & Delta Live Tables (DLT)

Unity Catalog

Medallion Architecture (Bronze, Silver, Gold)

Data Engineering

Performance Tuning & Spark Optimization

Key Responsibilities

Design and implement scalable Data Lakehouse architectures using Databricks on AWS.

Develop robust batch and streaming data pipelines using PySpark, SQL, Delta Lake, and Delta Live Tables.

Configure and manage Unity Catalog across multiple environments.





Implement secure AWS data platform architectures using IAM, KMS, VPC, and PrivateLink.

Optimize Spark performance, cluster utilization, and compute costs.

Lead architecture discussions, conduct code reviews, and mentor Data Engineers.

Collaborate with product owners, architects, and client stakeholders to deliver enterprise data solutions.

Preferred Qualifications

8+ years of Data Engineering experience.

Minimum 4+ years of hands-on experience with Databricks on AWS.

Strong expertise in PySpark, Python, SQL, and AWS.

Experience with Data Lakehouse architecture and enterprise-scale data platforms.

Excellent communication and leadership skills.

📌 Lead Data Engineer – Databricks on AWS | PySpark | Delta Lake | Unity Catalog (Bengaluru)
🏢 Obrimo Technologies
📍 Bengaluru