We are looking for a Databricks Developer with 2--5 years of experience in building scalable data engineering solutions and processing large volumes of data. The ideal candidate should have robust expertise in Databricks, PySpark, Spark SQL, and cloud-based data platforms.
Role & Responsibilities
- Design, develop, and maintain data pipelines using Databricks.
- Build and support ETL/ELT solutions for large-scale data processing.
- Develop data transformation workflows using PySpark and Spark SQL.
- Work with Delta Lake and modern data lake architectures.
- Integrate data from multiple source systems into enterprise platforms.
- Develop and optimize data processing jobs for performance and scalability.
- Collaborate with business and technical stakeholders to understand requirements.
- Ensure data quality, reliability, and governance standards.
- Troubleshoot and resolve issues related to data pipelines and processing.
- Support deployment and maintenance of data engineering solutions.
Preferred Candidate Profile:
- 2-5 years of experience in Data Engineering or Big Data technologies.
- Strong hands-on experience with Databricks and PySpark.
- Good knowledge of Spark SQL and SQL.
- Experience in ETL/ELT development and data integration.
- Understanding of Data Warehousing concepts.
- Experience with Azure Data Factory or similar cloud integration tools.
- Exposure to Delta Lake and cloud-based data platforms.
- Strong analytical and problem-solving skills.
- Good communication and stakeholder management abilities.