Position Summary

We are seeking a skilled Databricks Developer to design, develop, and optimize data pipelines and analytics solutions on the Databricks Lakehouse Platform. The ideal candidate will have 8 years of total experience in data engineering and analytics, with proven expertise in building robust data pipelines, implementing advanced analytics, and enabling machine learning workflows. You will collaborate with data engineers, analysts, and stakeholders to transform raw data into actionable insights, ensuring performance, scalability, and reliability.

Key Responsibilities
Design, build, and maintain scalable ETL/ELT pipelines using Databricks, Apache

Spark, and Delta Lake.
Develop and optimize PySpark and Spark SQL jobs for performance, cost, and reliability on large-scale datasets.
Implement data ingestion from various sources including REST APIs, streaming platforms, and cloud storage into the Lakehouse architecture.
Build and manage Unity Catalog for data governance, lineage, security, and access control.
Collaborate with data scientists and analysts to enable ML workflows, feature stores, and real-time analytics using Databricks.




Automate workflows using Databricks Workflows, Jobs API, and CI/CD pipelines with Azure DevOps or GitHub Actions.
Monitor cluster performance, troubleshoot bottlenecks, and implement best practices for cost optimization.
Ensure data quality, validation, and testing frameworks are implemented across all pipelines.

Required Qualifications

Technical Skills
5+ years of hands-on experience with Databricks and Apache Spark
Solid proficiency in Python and/or Scala
Expert-level Spark SQL and Delta Lake experience
Experience with cloud platforms: Azure, AWS, or GCP
Data warehousing concepts and dimensional modeling

Experience & Education
Bachelors degree in Computer Science, Engineering, or related field
8+years overall in data engineering or related roles
Experience with Git, CI/CD, and Agile methodologies
Solid problem-solving and communication skills

Preferred Qualifications
Databricks Certified Data Engineer Skilled
Unity Catalog
Delta Live Tables dbt Kafka / Event Hubs
Azure Data Factory
Power BI / Tableau

📌 Azure Data Engineer Hyderabad (India)
🏢 Texplorers
📍 India