Role and Responsibility:

- Build and maintain scalable pipelines with Azure Databricks, Python, and PySpark.
- Orchestrate data movement and workflows with ADF pipelines, triggers, and integration runtimes.
- Ingest and transform large datasets from sources such as SQL Server and Oracle.
- Manage data governance and access control with Unity Catalog.
- Implement cloud-native data solutions following best practices, with experience in the Azure or AWS cloud.
- Monitor, troubleshoot, and tune pipelines for performance and reliability; take part in the on-call rotation.
- Turn business requirements into robust data solutions and support production releases.
- Maintain documentation and runbooks.

Qualifications:

Graduate

- Minimum 5 years of experience in Data Engineering, Data Warehousing, or related areas.
- Bachelors degree in Computer Science, Information Technology, or a related discipline or equivalent practical experience.




- Solid hands-on experience with Azure Databricks (jobs/workflows, Delta Lake, Unity Catalog) in enterprise-scale environments.
- Advanced Python skills and experience developing distributed data processing solutions using PySpark.
- Hands-on experience with Azure Data Factory pipelines and ETL/ELT design, development, and optimization.
- Strong understanding of Azure cloud services, networking and storage, data modeling, and SQL (T-SQL preferred).
- Working knowledge of Windows and Linux environments.
- Experience with CI/CD and DevOps practices for automated deployment of data solutions (GitHub Actions preferred).
- Proven ability to troubleshoot complex data processing and pipeline issues in production environments.

📌 Azure Databricks Data Engineer (Kolkata)
🏢 WTW
📍 Kolkata