Job Title: Data Engineer Databricks
Role: Data Engineer

Job Summary

We are looking for an experienced Data Engineer with strong expertise in Databricks, PySpark, Python, and SQL to design, develop, and maintain scalable data engineering solutions. The ideal candidate will have experience building ETL/ELT pipelines, working with large datasets, and implementing data processing solutions using Apache Spark and Delta Lake.

Key Responsibilities

- Design, develop, and maintain scalable data pipelines using Databricks and PySpark.
- Build and optimize ETL/ELT workflows for structured and unstructured data.
- Develop data transformation logic using Python, PySpark, and SQL.
- Work with Delta Lake, Delta Tables, and Lakehouse architecture.
- Implement and manage workflows using Databricks Workflows.
- Integrate data from multiple sources, including databases, APIs, and cloud storage.
- Optimize Spark jobs, cluster performance, and SQL queries.




- Implement data quality checks, error handling, and monitoring.
- Collaborate with data architects, analysts, and business stakeholders.
- Follow data security, governance, and access-control best practices.
- Troubleshoot pipeline failures and ensure reliable data delivery.

Required Skills

- Strong hands-on experience with Databricks.
- Proficiency in PySpark, Python, and SQL.
- Experience with Apache Spark and distributed data processing.
- Robust knowledge of ETL/ELT development and data warehousing concepts.
- Experience with Delta Lake and Delta Tables.
- Knowledge of data modeling and performance optimization.
- Experience with cloud platforms such as Azure, AWS, or GCP.
- Understanding of Git, CI/CD, and version control.

📌 Walk-in || Data Engineer (Chennai)
🏢 Orcapod Consulting Services
📍 Chennai