11 Sep
|
Persistent Systems
|
Pune
11 Sep
Persistent Systems
Pune
About Position:
We are looking for an experienced Databricks Developer / Lead with 7+ years of experience in Data Engineering, Analytics, and Cloud Data Platforms. The ideal candidate will possess deep expertise in Databricks, Apache Spark, distributed data processing, and cloud-native data architectures. This role involves designing, developing, and optimizing large-scale data pipelines while ensuring high performance, scalability, reliability, and operational excellence across enterprise data platforms.
Role: Databricks Developer
Location: All Persistent Locations
Experience: 7 to 12 years
Job Type: Full time Employment
What You'll Do:
Design, build, and optimize distributed data pipelines using Python and Apache Spark.
Develop, manage, and maintain large-scale data processing workloads on Azure Databricks.
Build scalable ETL and ELT solutions for data ingestion, transformation, and processing.
Manage data storage and cloud resources using Azure Storage, Azure Container Registry (ACR), and Azure Kubernetes Service (AKS).
Develop container-based deployment solutions for data engineering workloads.
Orchestrate data workflows using Apache Airflow and enterprise scheduling platforms.
Implement robust data security controls including access management, encryption, and secure credential handling.
Support vulnerability remediation activities across data platforms using tools such as CodeQL, Cycode, and related security frameworks.
Monitor data pipelines and troubleshoot performance, reliability, and operational issues.
Collaborate with data architects, business analysts, and application teams to deliver business-driven data solutions.
Maintain source code repositories using GitHub and ensure proper documentation in Confluence.
Participate in code reviews and enforce data engineering best practices and development standards.
Support CI/CD processes and deployment automation for data platform components.
Contribute to Agile ceremonies including sprint planning, develop
📌 Databricks Developer (Pune)
🏢 Persistent Systems
📍 Pune