09 Sep
|
Persistent Systems
|
Pune
09 Sep
Persistent Systems
Pune
Job DescriptionAbout Position: We are looking for an experienced Databricks Developer / Lead who can build, optimize, and maintain scalable data platforms and enterprise-grade data pipelines using Databricks and Spark technologies. The ideal candidate will have strong expertise in distributed data processing, performance tuning, Delta Lake architecture, and cloud-native data engineering solutions. - Role: Databricks Developer - Location: Pune - Experience: 7 to 12 Years - Job Type: Full Time Employment What You'll Do: - Develop and maintain scalable ETL/ELT pipelines using Databricks, PySpark, SQL, and Notebooks. - Design and implement end-to-end data processing solutions using Delta Lake architecture. - Build and manage Bronze, Silver, and Gold layer data pipelines for enterprise data platforms. - Develop batch and real-time streaming solutions using Spark Structured Streaming. - Perform Spark performance tuning, workload optimization, and resource utilization improvements. - Optimize Databricks clusters, jobs,
and infrastructure to improve performance and control operational costs. - Collaborate with data architects, business analysts, data scientists, and stakeholders to deliver scalable data solutions. - Ensure data quality, validation, reconciliation, and compliance with enterprise governance standards. - Integrate Databricks solutions with Azure, AWS, or GCP cloud-native services. - Implement and maintain CI/CD pipelines for automated deployment and release management. - Utilize Git and version control best practices for collaborative development. - Monitor data jobs, troubleshoot failures, and resolve cluster and setting-related issues. - Support data lake modernization, cloud migration, and enterprise analytics initiatives. - Participate in architecture reviews, design discussions, and technical mentoring activities. - Drive best practices around scalability, reliability, observability, and operational excellence. Expertise You'll Bring: - 7 to 12 years
📌 Databricks Developer (Pune)
🏢 Persistent Systems
📍 Pune