Key Responsibilities

- Design, develop, and optimize end-to-end data pipelines and data integration solutions on Azure.
- Build scalable data ingestion, transformation, and processing frameworks using Azure-native services.
- Develop and maintain ETL/ELT processes for structured and unstructured data.
- Design and implement modern data lake, data warehouse, and lakehouse architectures.
- Collaborate with business analysts, data scientists, architects, and application teams to understand data requirements.
- Ensure data quality, governance, security, and compliance standards are maintained.
- Optimize data processing performance, scalability, and cost efficiency.
- Implement data modeling strategies including dimensional and relational models.
- Develop solutions for batch, near real-time, and streaming data processing.
- Automate deployment and monitoring processes using DevOps and Infrastructure as Code principles.
- Troubleshoot and resolve data pipeline and platform issues.
- Create and maintain technical documentation, standards, and best practices.
- Mentor junior team members and contribute to data engineering excellence initiatives.

Required Skills & Qualifications





- Bachelor's degree in Computer Science, Information Technology, Engineering, Mathematics, or related field.
- 8+ years of experience in Data Engineering, Data Warehousing, or Big Data solutions.
- 5+ years of hands-on experience with Microsoft Azure Data Services.
- Robust experience in designing and developing enterprise-scale data pipelines.
- Expertise in SQL and database performance tuning.
- Strong knowledge of data warehousing concepts and dimensional modeling.
- Hands-on experience with Azure data services including:
- Azure Data Factory (ADF)
- Azure Synapse Analytics
- Azure Data Lake Storage (ADLS)
- Azure Databricks
- Azure SQL Database
- Azure Event Hub
- Azure Stream Analytics
- Azure Functions
- Azure Key Vault
- Microsoft Fabric (preferred)

- Experience with big data processing frameworks and distributed computing.
- Strong programming skills in Python, PySpark, Scala, or SQL.
- Experience working with structured, semi-structured, and unstructured data.
- Familiarity with CI/CD pipelines and DevOps practices.
- Strong troubleshooting and analytical skills.

📌 Azure Senior Data Engineer - S (Kolkata)
🏢 Tata Consultancy Services
📍 Kolkata