We are looking for a highly skilled
Senior Databricks Engineer
with 7+ years of experience in Data Engineering and Big Data technologies. The ideal candidate should have strong expertise in
Databricks, Python, and PySpark
, along with experience in designing, developing, and optimizing scalable data pipelines and data processing solutions in cloud environments.
Key Responsibilities
- Design, develop, and maintain scalable data pipelines using
Databricks, PySpark, and Python
.
- Build and optimize ETL/ELT processes for large-scale structured and unstructured datasets.
- Develop data ingestion frameworks and transformation workflows.
- Work closely with business stakeholders, data architects, and analytics teams to understand data requirements.
- Optimize Spark jobs for performance, scalability, and cost efficiency.
- Implement data quality checks, monitoring, and troubleshooting mechanisms.
- Develop reusable frameworks and best practices for data engineering solutions.
- Collaborate with DevOps and Cloud teams for deployment and automation.
- Ensure compliance with data governance, security, and operational standards.
- Support production issues, root cause analysis, and performance tuning activities.
Required Skills
- 7+ years of experience in Data Engineering.
- Strong hands-on experience with
Databricks
.
- Expertise in
Python
and
PySpark
.
- Experience in designing and implementing ETL/ELT frameworks.
- Robust knowledge of Spark architecture and distributed data processing.
- Experience working with relational databases and SQL.
- Understanding of Data Warehousing concepts and dimensional modeling.
- Experience with version control systems such as Git.
- Strong problem-solving and analytical skills.
- Excellent communication and stakeholder management skills.