- Design, develop, and maintain scalable ETL/ELT data pipelines using Azure Data Factory and Databricks.
- Develop data transformation and processing logic using PySpark, Python, and Spark SQL.
- Build and maintain Databricks notebooks, workflows, and Spark-based data processing jobs.
- Develop and manage Azure Data Factory pipelines, including activities, parameters, triggers, and dependencies.
- Perform data cleansing, transformation, validation, and integration from multiple data sources.
- Optimize PySpark and Spark SQL jobs for performance, scalability, and productive resource utilization.
- Troubleshoot data pipeline failures and production issues and implement appropriate fixes.
- Work with large datasets and ensure data quality, accuracy, and reliability.




- Use Azure DevOps for source-code management, branching, pull requests, and CI/CD processes.
- Follow coding standards, best practices, and development processes.
- Collaborate with business analysts, data engineers, developers, and other stakeholders to understand requirements and deliver data solutions.
- Prepare technical documentation for pipelines, transformations, and data processes.
- Support deployment and production releases across different environments.

Qualifications

- Bachelor's or Master's degree in Engineering or Technology• 3-5 years of experience

📌 Hiring For Azure Data Engineer (Databricks & ADF) (Gurugram)
🏢 WNS Holdings
📍 Gurugram