Databricks Engineer / Architect

Experience: 5–10 Years

We are seeking an experienced Databricks Data Engineer to design, develop, and optimize scalable data engineering solutions using the Databricks Lakehouse Platform. The ideal candidate will have strong expertise in Apache Spark, PySpark, SQL, Delta Lake, data pipelines, cloud platforms, and enterprise data architecture.

Key Responsibilities

- Design and develop scalable ETL/ELT data pipelines using Databricks, Apache Spark and PySpark.
- Develop and maintain Lakehouse architecture using Delta Lake and Bronze/Silver/Gold data layers.
- Build batch and real-time data processing and streaming solutions.
- Develop and manage Databricks Jobs, Workflows and Delta Live Tables (DLT).
- Implement data governance, security and access controls using Unity Catalog.
- Optimize Spark jobs, SQL queries and data pipelines for performance and cost efficiency.
- Integrate Databricks with enterprise data sources and cloud services.
- Implement data quality, validation, monitoring and error-handling frameworks.
- Work with Git, CI/CD and DevOps practices for automated deployments.
- Troubleshoot production issues and ensure reliability and availability of data platforms.




- Collaborate with Data Architects, Developers, Analysts and business stakeholders to deliver data solutions.

Required Technical Skills

- 5–10 years of experience in Data Engineering, with strong hands-on Databricks experience.
- Advanced knowledge of Databricks, Apache Spark, PySpark and SQL.
- Strong expertise in Delta Lake and Lakehouse architecture.
- Hands-on experience with Unity Catalog, Databricks Workflows and DLT.
- Experience with batch and streaming data processing.
- Strong understanding of data modelling, ETL/ELT and data warehousing concepts.
- Experience with at least one major cloud platform: Azure, AWS or GCP.
- Valuable understanding of CI/CD, Git and DevOps.
- Strong troubleshooting, analytical and problem-solving skills.

Certifications

- Databricks Certified Data Engineer – Associate — Mandatory
- Databricks Certified Data Engineer – Professional — Preferred
- Relevant Azure/AWS/GCP Data Engineering certification — Preferred

Education

Bachelor’s degree in computer science, Information Technology, Data Engineering, or a related discipline.

📌 Databricks Architect (India)
🏢 VaporVM
📍 India