Job DescriptionExperience: 8–10 Years Location: Remote Employment Type: Full time Job Summary We are seeking a highly experienced Senior Databricks Engineer to design, build, and optimize large-scale data engineering solutions on the Databricks Lakehouse Platform. The ideal candidate will have deep expertise in big data processing, cloud architecture, and end-to-end data pipeline development, along with strong leadership skills to guide technical teams. Key Responsibilities
- Design, develop, and optimize scalable data pipelines and ETL/ELT workflows using Databricks (PySpark, Spark SQL, Delta Lake)
- Architect and implement Lakehouse solutions integrating data lakes and data warehouses
- Lead the migration of legacy data platforms to Databricks on Azure/AWS/GCP
- Optimize Spark jobs for performance, cost, and scalability
- Implement data governance, security, and access control using Unity Catalog
- Collaborate with data scientists, analysts, and business stakeholders to understand data requirements
- Design and manage CI/CD pipelines for Databricks notebooks and jobs
- Mentor junior engineers and provide technical leadership on best practices
- Troubleshoot performance bottlenecks and ensure high availability of data pipelines
- Work with structured, semi-structured,



and unstructured data at scale Required Skills & Qualifications
- 8–10 years of overall experience in data engineering/big data, with strong hands-on expertise in Databricks
- Proficiency in PySpark, Spark SQL, and Delta Lake
- Strong experience with at least one major cloud platform (Azure Databricks, AWS Databricks, or GCP)
- Solid understanding of data warehousing concepts, data modeling, and ETL/ELT design patterns
- Experience with Unity Catalog, Delta Live Tables, and Databricks Workflows
- Proficiency in Python and SQL
- Knowledge of Azure Data Factory (ADF) and Medallion Architecture
- Experience with orchestration tools (Airflow, ADF, or similar)
- Familiarity with CI/CD tools (Git, Azure DevOps, Jenkins)
- Strong understanding of data governance, security, and compliance frameworks
- Excellent problem-solving, communication, and stakeholder management skills Good to Have
- Databricks Certified Data Engineer Professional/Associate certification
- Experience with MLflow and machine learning pipelines
- Knowledge of Kafka or other streaming technologies
- Experience working in the US healthcare industry is a plus Education
- Bachelor's/Master's degree in Computer Science, Information Technology, or a related field

📌 Senior Databricks Engineer (Maharashtra)
🏢 Quantorus
📍 Maharashtra