Role & responsibilities
Key Responsibilities: •
Develop and maintain scalable and efficient data pipelines to ensure seamless data flow across various systems adddress any issues or bottlenecks in existing pipelines. • Implement robust data checks to ensure the accuracy and integrity of data. Summarize and validate large datasets to ensure they meet quality standards. • Monitor data jobs for successful completion. Troubleshoot and resolve any issues that arise to minimize downtime and ensure continuity of data processes. • Regularly review and audit data processes and pipelines to ensure compliance with internal standards and regulatory requirements • Familiar with working on Agile methodologies - scrum, sprint planning, backlog refinement etc. Candidate Profile: •
4-9 years experience on Data Engineering role working with Databricks & Azure Cloud technologies. • Bachelors degree in computer science, Information Technology, or related field. • Strong proficiency in PySpark, Python, SQL. • Robust experience in data modeling, ETL/ELT pipeline development, and automation • Hands-on experience with performance tuning of data pipelines and workflows • Proficient in working on Azure cloud components Azure Data Factory, Azure DataBricks, Azure Data Lake etc. • Experience with data modeling, ETL processes, Delta Lake and data warehousing.
Preferred candidate profile
📌 Databricks Data Engineer (Pune)
🏢 EXL
📍 Pune