07 Sep
|
Persistent Systems
|
Nellore
07 Sep
Persistent Systems
Nellore
About Position:
We are looking for an Azure Data Engineer with 4 to 8 years of experience and robust expertise in Azure Databricks and Python. The candidate will be responsible for designing and building scalable, high-performance data solutions on Azure to support analytics and business use cases. The ideal candidate will collaborate with cross-functional teams to develop reliable data pipelines and ensure high standards of performance, data quality, security, and scalability across the Azure ecosystem.
Role: Azure Data Engineer
Location: Bangalore, Pune & Chennai
Experience: 4 to 8 Years
Job Type: Full Time Employment
What You'll Do:
Design, develop, and optimize scalable data pipelines using Azure Databricks and Python
Build and maintain robust ETL/ELT workflows for batch and streaming data processing
Develop reusable Python frameworks and Databricks notebooks for data ingestion, transformation, and validation
Process and transform large-scale datasets using Apache Spark and PySpark
Design and implement Delta Lake and medallion architecture for reliable data processing
Integrate data solutions with Azure services such as Data Factory (ADF), Data Lake Storage (ADLS), and Synapse
Implement data quality, governance, security, monitoring, and performance-tuning mechanisms
Collaborate with business, analytics, and engineering teams to translate requirements into production-ready solutions
Troubleshoot and resolve data pipeline, Databricks, Python, and production issues
Expertise You'll Bring:
4 to 8 years of experience in Data Engineering
Strong hands-on experience with Azure Databricks, Apache Spark, PySpark, and Delta Lake
Advanced proficiency in Python, including data processing, APIs, reusable modules, and automation
Solid proficiency in SQL and experience with query optimization
Experience with Azure services such as Data Factory (ADF), Data Lake Storage (ADLS), and Synapse
Robust understanding of data warehousing, ETL/EL
📌 Azure Data Engineer Nellore
🏢 Persistent Systems
📍 Nellore