06 Sep
|
Persistent Systems
|
Pune
06 Sep
Persistent Systems
Pune
About Position:
We are looking for an Azure Data Engineer with 4 to 8 years of experience and strong expertise in Azure Databricks, Python, and Generative AI. The candidate will design and build scalable data platforms and AI-enabled data solutions on Azure for analytics and business use cases. The ideal candidate will collaborate with cross-functional teams to develop reliable data pipelines, integrate Generative AI capabilities, and ensure high standards of performance, data quality, security, and scalability across the Azure ecosystem.
Role: Azure Data Engineer
Location: Bangalore, Pune & Chennai
Experience: 4 to 8 Years
Job Type: Full Time Employment
What You'll Do:
Design, develop, and optimize scalable data pipelines using Azure Databricks and Python
Build and maintain robust ETL/ELT workflows for batch and streaming data processing
Develop reusable Python frameworks and Databricks notebooks for data ingestion, transformation, and validation
Process and transform large-scale datasets using Apache Spark and PySpark
Design and implement Delta Lake and medallion architecture for reliable data processing
Integrate data solutions with Azure services such as Data Factory (ADF), Data Lake Storage (ADLS), and Synapse
Implement data quality, governance, security, monitoring, and performance-tuning mechanisms
Collaborate with business, analytics, and engineering teams to translate requirements into production-ready solutions
Troubleshoot and resolve data pipeline, Databricks, Python, and production issues
Expertise You'll Bring:
4 to 8 years of experience in Data Engineering
Robust hands-on experience with Azure Databricks, Apache Spark, PySpark, and Delta Lake
Advanced proficiency in Python, including data processing, APIs, reusable modules, and automation
Strong proficiency in SQL and experience with query optimization
Experience with Azure services such as Data Factory (ADF), Data Lake Storage (ADLS), and Synapse
Strong understanding of data warehousing, ETL/EL
📌 Azure Data Engineer (Pune)
🏢 Persistent Systems
📍 Pune