About Position:

We are seeking an experienced Senior Data Engineer with robust expertise in Azure Databricks, Python, Spark, and cloud-based data engineering solutions. The ideal candidate will be responsible for designing, building, and maintaining scalable data pipelines and distributed data processing frameworks. You will play a key role in supporting enterprise analytics initiatives by developing secure, reliable, and high-performance data solutions on Azure cloud platforms.

• Role: Senior Data Engineer - Azure Databricks & Spark
• Location: All Persistent Locations
• Experience: 6 to 9 Years
• Job Type: Full-Time Employment

What You'll Do:

• Design, develop, and maintain distributed data pipelines using Python and Spark.
• Build and manage large-scale data processing workloads on Azure Databricks.
• Develop scalable ETL/ELT frameworks for processing structured and unstructured data.
• Manage data storage and cloud deployments using Azure Storage, Azure Container Registry (ACR), and Azure Kubernetes Service (AKS).




• Orchestrate workflows and automate pipeline execution using Apache Airflow and internal scheduling frameworks.
• Design and implement high-performance Spark applications using PySpark.
• Maintain source code repositories using GitHub and document technical solutions and processes in Confluence.
• Implement secure data engineering practices including encryption, access controls, credential management, and sensitive data handling.
• Support vulnerability assessment and remediation activities using tools such as CodeQL, Cycode, and related security scanning solutions.
• Optimize data pipelines and Spark workloads for performance, scalability, and reliability.
• Collaborate with architects, developers, analysts, and business stakeholders to deliver enterprise data solutions.
• Participate in Agile ceremonies including sprint planning, daily stand-ups, code reviews, and release activities.
• Monitor production environments, troubleshoot issues, and pe

📌 Senior Data Engineer (Pune)
🏢 Persistent Systems
📍 Pune