Roles and Responsibilities :
- Design, develop, test, deploy and maintain large-scale data pipelines using Azure Data Factory (ADF) to extract, transform, load data from various sources into Azure Databricks.
- Collaborate with cross-functional teams to gather requirements for data processing needs and design solutions that meet business needs.
- Develop complex SQL queries to optimize database performance and troubleshoot issues in PySpark applications.
- Ensure high availability of the system by monitoring its performance, identifying bottlenecks, and implementing improvements.

Job Requirements :
- 4-7 years of experience in a similar role as a Data Engineer or related field.
- Robust expertise in Azure Data Factory (ADF), Azure Databricks, Python programming language, SQL querying languages such as T-SQL.
- Experience working with big data technologies like Hadoop ecosystem components including HDFS, MapReduce, Hive etc.

📌 Data Engineer (Pune)
🏢 Infovision labs
📍 Pune