Key Responsibilities
Design, develop, and optimize end-to-end data pipelines and data integration solutions on Azure.
Build scalable data ingestion, transformation, and processing frameworks using Azure-native services.
Develop and maintain ETL/ELT processes for structured and unstructured data.
Design and implement up-to-date data lake, data warehouse, and lakehouse architectures.
Collaborate with business analysts, data scientists, architects, and application teams to understand data requirements.
Ensure data quality, governance, security, and compliance standards are maintained.
Optimize data processing performance, scalability, and cost efficiency.
Implement data modeling strategies including dimensional and relational models.
Develop solutions for batch, near real-time, and streaming data processing.
Automate deployment and monitoring processes using DevOps and Infrastructure as Code principles.
Troubleshoot and resolve data pipeline and platform issues.
Create and maintain technical documentation, standards, and best practices.
Mentor junior team members and contribute to data engineering excellence initiatives.

Required Skills & Qualifications




Bachelor's degree in Computer Science, Information Technology, Engineering, Mathematics, or related field.
8+ years of experience in Data Engineering, Data Warehousing, or Big Data solutions.
5+ years of hands-on experience with Microsoft Azure Data Services.
Robust experience in designing and developing enterprise-scale data pipelines.
Expertise in SQL and database performance tuning.
Strong knowledge of data warehousing concepts and dimensional modeling.
Hands-on experience with Azure data services including:
Azure Data Factory (ADF)
Azure Synapse Analytics
Azure Data Lake Storage (ADLS)
Azure Databricks
Azure SQL Database
Azure Event Hub
Azure Stream Analytics
Azure Functions
Azure Key Vault
Microsoft Fabric (preferred)
Experience with big data processing frameworks and distributed computing.
Strong programming skills in Python, PySpark, Scala, or SQL.
Experience working with structured, semi-structured, and unstructured data.
Familiarity with CI/CD pipelines and DevOps practices.
Robust troubleshooting and analytical skills.

📌 Azure Senior Data Engineer S Kolkata (India)
🏢 Tata Consultancy Services
📍 India