About Position:

We are seeking a highly experienced Lead Databricks Data Engineer with strong expertise in Databricks, PySpark, Python, SQL, and AI-powered analytics solutions. The ideal candidate will lead the design, development, and optimization of enterprise-scale data platforms, contemporary Lakehouse architectures, and AI-driven analytics solutions. This role requires hands-on experience in building scalable data pipelines, implementing cloud-native data architectures, leveraging Databricks AI capabilities such as Genie, and driving enterprise data modernization initiatives. The successful candidate will work closely with Data Architects, Data Scientists, Product Owners, and business stakeholders to deliver creative data solutions that enable intelligent decision-making and business growth.

• Role: Lead Databricks Data Engineer
• Location: All Persistent Locations
• Experience: 12 to 15 Years
• Job Type: Full Time Employment

What You'll Do:

• Design, develop, and maintain scalable enterprise-grade data pipelines using Databricks, PySpark, Python, and SQL.




• Build and optimize batch and real-time data processing frameworks for high-volume data workloads.
• Develop robust ETL/ELT solutions for structured, semi-structured, and unstructured data.
• Implement data quality, validation, reconciliation, and monitoring frameworks.
• Establish reusable components, standards, and best practices for enterprise data engineering.
• Lead initiatives focused on scalability, reliability, and performance optimization of data platforms.
• Design and implement contemporary Databricks Lakehouse architectures.
• Work extensively with Delta Lake, Unity Catalog, Databricks Workflows, and Databricks SQL.
• Tune Spark jobs and optimize cluster configurations for large-scale processing workloads.
• Implement enterprise-grade security, governance, lineage, and access management controls.
• Define architectural standards and best practices for Databricks implementations.
• Develop AI-driven ana

📌 Lead Databricks Data Engineer Pune (India)
🏢 Persistent Systems
📍 India