10 Aug
|
Persistent Systems
|
Pune
10 Aug
Persistent Systems
Pune
About Position:
We are seeking a highly skilled Data Engineer with 6 to 10 years of experience in building scalable, cloud-based data solutions using Azure, Databricks, and PySpark. The ideal candidate will be responsible for designing, developing, and optimizing enterprise-grade data pipelines, enabling data-driven decision-making across the organization. You will work closely with business stakeholders, data scientists, and analytics teams to deliver reliable, secure, and high-performance data solutions.
Role: Data Engineer
Location: All Persistent Location
Experience: 6 to 10 Years
Job Type: Full-Time Employment
What You'll Do:
As a Data Engineer, you will be responsible for designing, developing, and maintaining scalable data solutions on the Azure cloud platform using Databricks and PySpark. You will work closely with business stakeholders, data scientists, and analytics teams to build robust data pipelines and support enterprise data initiatives.
Design, develop, and optimize scalable ETL/ELT pipelines using PySpark and Azure Databricks.
Build and maintain data ingestion frameworks for structured and unstructured data.
Develop data solutions leveraging Azure services such as Azure Data Factory (ADF), Azure Data Lake Storage (ADLS), Azure Synapse, and Azure SQL.
Implement batch and real-time data processing solutions.
Optimize Spark workloads for performance, scalability, and cost efficiency.
Collaborate with business and technical teams to understand data requirements and deliver high-quality solutions.
Ensure data quality, governance, security, and compliance standards are met.
Monitor, troubleshoot, and resolve issues related to data pipelines and processing jobs.
Support CI/CD implementation and deployment of data engineering solutions.
Contribute to data architecture discussions and best practices.
Expertise You'll Bring:
We are seeking a highly skilled Data Engineer with robust expertise in Databricks, PySpark, and Azure cloud technologies, capable
📌 Data Engineer (Pune)
🏢 Persistent Systems
📍 Pune