Position Summary:

We are seeking a talented Data Engineer with a strong background in data engineering to join our team. You will play a key role in designing, building, and maintaining data pipelines using a variety of technologies, with a focus on the Microsoft Azure cloud platform.

Responsibilities:

- Design, develop, and implement data pipelines using Azure Data Factory (ADF) or other orchestration tools.
- Write efficient SQL queries to extract, transform, and load (ETL) data from various sources into Azure Synapse Analytics.
- Utilize PySpark and Python for complex data processing tasks on large datasets within Azure Databricks.
- Collaborate with data analysts to understand data requirements and ensure data quality.
- Implement data governance practices to ensure data security and compliance.
- Monitor and maintain data pipelines for optimal performance and troubleshoot any issues.




- Develop and maintain unit tests for data pipeline code.
- Work collaboratively with other engineers and data professionals in an Agile development workplace.

Must Have tech skills

Databricks

- Fabric
- Sql
- Pyspark
- Python

Must Have competencies

- Pipeline - Data Ingestion and data quality , transformation – Databricks, ADF
- Consumption of rest api
- Optimization of pyspark job , sql procedure , views

Nice to have ( These are design and admin skills)

- Warehouse and lake house creation
- Modeling
- Semantic model design
- Refresh management
- Logging and monitoring
- Roles and permissions

New Vision is an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees

📌 Sr. Data Engineer (Pune)
🏢 NewVison
📍 Pune