Job Type: Full time

Experience: 8–10 YearsLocation: RemoteEmployment Type: Full-TimeJob SummaryWe are seeking a highly experienced Senior Databricks Engineer to design, build, and optimize large-scale data engineering solutions on the Databricks Lakehouse Platform. The ideal candidate will have deep expertise in big data processing, cloud architecture, and end-to-end data pipeline development, along with strong leadership skills to guide technical teams.Key ResponsibilitiesDesign, develop, and optimize scalable data pipelines and ETL/ELT workflows using Databricks (PySpark, Spark SQL, Delta Lake)Architect and implement Lakehouse solutions integrating data lakes and data warehousesLead the migration of legacy data platforms to Databricks on Azure/AWS/GCPOptimize Spark jobs for performance, cost, and scalabilityImplement data governance, security, and access control using Unity CatalogCollaborate with data scientists, analysts, and business stakeholders to understand data requirementsDesign and manage CI/CD pipelines for Databricks notebooks and jobsMentor junior engineers and provide technical leadership on best practicesTroubleshoot performance bottlenecks and ensure high availability of data pipelinesWork with structured, semi-structured,



and unstructured data at scaleRequired Skills & Qualifications8–10 years of overall experience in data engineering/big data, with strong hands-on expertise in DatabricksProficiency in PySpark, Spark SQL, and Delta LakeStrong experience with at least one major cloud platform (Azure Databricks, AWS Databricks, or GCP)Solid understanding of data warehousing concepts, data modeling, and ETL/ELT design patternsExperience with Unity Catalog, Delta Live Tables, and Databricks WorkflowsProficiency in Python and SQLKnowledge of Azure Data Factory (ADF) and Medallion ArchitectureExperience with orchestration tools (Airflow, ADF, or similar)Familiarity with CI/CD tools (Git, Azure DevOps, Jenkins)Strong understanding of data governance, security, and compliance frameworksExcellent problem-solving, communication, and stakeholder management skillsGood to HaveDatabricks Certified Data Engineer Qualified/Associate certificationExperience with MLflow and machine learning pipelinesKnowledge of Kafka or other streaming technologiesExperience working in the US healthcare industry is a plusEducation• Bachelor's/Master's degree in Computer Science, Information Technology, or a related field

📌 Senior Databricks Engineer (Madhya Pradesh)
🏢 Quantorus
📍 Madhya Pradesh