We are looking for a Senior Databricks Engineer with 7 years of experience in data engineering and strong hands-on expertise in Databricks and Apache Spark . The ideal candidate will design and build scalable data pipelines optimize large-scale data processing and work with cloud-based data platforms.

Key Responsibilities

- Design develop and maintain scalable data pipelines using Databricks and Apache Spark .

- Develop and optimize PySpark/Spark jobs for large-scale data processing.

- Build reliable ETL/ELT pipelines using Databricks workflows Delta Lake and SQL .

- Implement data transformations data quality checks and performance optimization.

- Work with cloud data platforms such as AWS Azure or GCP .

- Optimize Spark jobs cluster configurations partitioning joins and data processing performance.

- Collaborate with data engineers data scientists analysts and business teams to deliver data solutions.

- Implement best practices for CI/CD version control monitoring and production support .

- Troubleshoot data pipeline and performance issues in production environments.





Required Skills

- 7 years of experience in Data Engineering .

- Robust hands-on experience with Databricks .

- Strong expertise in Apache Spark / PySpark .

- Experience with Delta Lake Databricks Workflows and Spark SQL .

- Strong proficiency in Python and SQL .

- Experience building and optimizing large-scale ETL/ELT pipelines.

- Experience with at least one cloud platform: AWS Azure or GCP .

- Good understanding of data warehousing data modeling and distributed data processing.

- Experience with Git and CI/CD practices.

Preferred Skills

- Experience with Unity Catalog and Databricks governance.

- Experience with Azure Data Factory AWS Glue or similar orchestration tools .

- Knowledge of streaming technologies such as Kafka or Spark Structured Streaming .

- Experience with Infrastructure as Code or cloud automation.

- Databricks certifications are a plus.

Required Skills:

Databricks Engineer

📌 Senior Databricks Engineer (Chennai)
🏢 Numentica
📍 Chennai