09 Sep
|
Persistent Systems
|
Pune
09 Sep
Persistent Systems
Pune
About Position:
We are looking for an experienced Databricks Developer / Lead who can build, optimize, and maintain scalable data platforms and enterprise-grade data pipelines using Databricks and Spark technologies. The ideal candidate will have robust expertise in distributed data processing, performance tuning, Delta Lake architecture, and cloud-native data engineering solutions.
- Role: Databricks Developer
- Location: Pune
- Experience: 7 to 12 Years
- Job Type: Full Time Employment
What You'll Do:
- Develop and maintain scalable ETL/ELT pipelines using Databricks, PySpark, SQL, and Notebooks.
- Design and implement end-to-end data processing solutions using Delta Lake architecture.
- Build and manage Bronze, Silver, and Gold layer data pipelines for enterprise data platforms.
- Develop batch and real-time streaming solutions using Spark Structured Streaming.
- Perform Spark performance tuning, workload optimization, and resource utilization improvements.
- Optimize Databricks clusters, jobs, and infrastructure to improve performance and control operational costs.
- Collaborate with data architects, business analysts, data scientists, and stakeholders to deliver scalable data solutions.
- Ensure data quality, validation, reconciliation, and compliance with enterprise governance standards.
- Integrate Databricks solutions with Azure, AWS, or GCP cloud-native services.
- Implement and maintain CI/CD pipelines for automated deployment and release management.
- Utilize Git and version control best practices for collaborative development.
- Monitor data jobs, troubleshoot failures, and resolve cluster and environment-related issues.
- Support data lake modernization, cloud migration, and enterprise analytics initiatives.
- Participate in architecture reviews, design discussions, and technical mentoring activities.
- Drive best practices around scalability, reliability, observability, and operational excellence.
Expertise You'll Bring:
- 7 to 12 years of experienc
📌 Databricks Developer (Pune)
🏢 Persistent Systems
📍 Pune