Position Summary

We are seeking a skilled Databricks Developer to design, develop, and optimize data pipelines and analytics solutions on the Databricks Lakehouse Platform. The ideal candidate will have 8 years of total experience in data engineering and analytics, with proven expertise in building robust data pipelines, implementing advanced analytics, and enabling machine learning workflows. You will collaborate with data engineers, analysts, and stakeholders to transform raw data into actionable insights, ensuring performance, scalability, and reliability.

Key Responsibilities

- Design, build, and maintain scalable ETL/ELT pipelines using Databricks, Apache

Spark, and Delta Lake.
- Develop and optimize PySpark and Spark SQL jobs for performance, cost, and reliability on large-scale datasets.
- Implement data ingestion from various sources including REST APIs, streaming platforms, and cloud storage into the Lakehouse architecture.
- Build and manage Unity Catalog for data governance, lineage, security, and access control.
- Collaborate with data scientists and analysts to enable ML workflows, feature stores, and real-time analytics using Databricks.




- Automate workflows using Databricks Workflows, Jobs API, and CI/CD pipelines with Azure DevOps or GitHub Actions.
- Monitor cluster performance, troubleshoot bottlenecks, and implement best practices for cost optimization.
- Ensure data quality, validation, and testing frameworks are implemented across all pipelines.

Required Qualifications

Technical Skills

- 5+ years of hands-on experience with Databricks and Apache Spark
- Strong proficiency in Python and/or Scala
- Expert-level Spark SQL and Delta Lake experience
- Experience with cloud platforms: Azure, AWS, or GCP
- Data warehousing concepts and dimensional modeling

Experience & Education
- Bachelors degree in Computer Science, Engineering, or related field
- 8+years overall in data engineering or related roles
- Experience with Git, CI/CD, and Agile methodologies
- Solid problem-solving and communication skills

Preferred Qualifications

- Databricks Certified Data Engineer Professional
- Unity Catalog
- Delta Live Tables dbt Kafka / Event Hubs
- Azure Data Factory
- Power BI / Tableau

📌 Azure Data Engineer (Hyderabad)
🏢 Texplorers
📍 Hyderabad