Job Summary

Location: Pune, Bengaluru, NCR, Kolkata, Hyderabad

Exp: 6-12 years

We are seeking a skilled Data Engineer with expertise in Azure Services, Python, PySpark, and Databricks to design, develop, and maintain scalable data transformation pipelines. The ideal candidate will have experience managing large datasets, SQL/NoSQL databases, and implementing complex data processing and calculation engines while collaborating closely with software engineering teams.

Key Responsibilities

- Design, develop, and maintain data transformation pipelines using Azure Services (Azure Functions, Azure Data Factory, Logic Apps, etc.) and Python.
- Build and optimize data processing solutions using PySpark and Databricks.
- Develop and maintain complex data calculation and transformation frameworks.
- Process and manage large-scale structured and unstructured datasets efficiently.
- Design, implement, and maintain databases such as PostgreSQL and MongoDB.
- Collaborate with Software Engineers, Product Teams, and stakeholders to deliver scalable data solutions.
- Participate in code reviews, architecture discussions, and technical planning.




- Contribute to CI/CD, automation, testing, and deployment processes.
- Monitor and optimize data pipelines for performance, reliability, and scalability.

Required Skills

Must-Have

- Robust proficiency in Python
- Hands-on experience with Azure Cloud Services
- Experience with PySpark and Databricks
- SQL expertise with PostgreSQL
- Knowledge of MongoDB or other NoSQL databases
- Experience handling and optimizing large datasets
- Understanding of data transformation, ETL/ELT pipelines, and data modeling
- Familiarity with Git, code reviews, and software development best practices

Nice-to-Have

- Azure Certifications (AZ-900, DP-203, etc.)
- Experience with CI/CD and DevOps practices
- Knowledge of data quality, monitoring, and testing frameworks

Qualifications

- Bachelor's degree in Computer Science, Information Technology, Engineering, or equivalent experience.
- Adaptable educational requirements for candidates with relevant hands-on experience.

📌 Azure Data Engineer (West Bengal)
🏢 Tata Consultancy Services
📍 West Bengal