Job Summary

We are seeking an experienced Senior Databricks Developer with 5+ years of experience in designing and implementing scalable data engineering solutions. The ideal candidate will have strong expertise in big data processing, data pipeline development, and cloud-based data platforms, with hands-on experience using Databricks and distributed data processing frameworks like Apache Spark.

The role involves building robust data pipelines, optimizing large-scale data processing workloads, and collaborating with data engineers, analysts, and business stakeholders to enable data-driven decision-making.

Key Responsibilities
- Design, develop, and maintain scalable data pipelines using Databricks.
- Build and optimize large-scale data processing solutions using Apache Spark.
- Develop and manage ETL/ELT workflows for ingesting, transforming, and processing structured and unstructured data.
- Write efficient data processing scripts using Python, Scala, and SQL.
- Implement and manage data lake and data warehouse architectures.




- Optimize performance of data pipelines and Spark jobs.
- Integrate Databricks with cloud data services and storage platforms.
- Collaborate with data scientists, analysts, and engineering teams to deliver reliable data solutions.
- Implement best practices for data governance, data quality, and security.
- Troubleshoot data pipeline issues and ensure system reliability and scalability.
- Document data architecture, workflows, and operational processes.

Required Skills & Qualifications
- 5+ years of experience in data engineering or big data development.
- Solid hands-on experience with Databricks.
- Extensive experience with distributed data processing using Apache Spark.
- Proficiency in Python, Scala, and SQL.
- Strong experience building scalable ETL/ELT data pipelines.
- Experience working with large datasets in data lake or data warehouse environments.
- Experience with cloud platforms such as Microsoft Azure, Amazon Web Services,

📌 Databricks Developer (Chennai)
🏢 Kumaran Systems
📍 Chennai