Senior Data Engineer – Azure & Microsoft Fabric

Position Overview

We are looking for a Senior Data Engineer with 6–8 years of experience in designing and developing scalable data engineering solutions using Azure Data Factory (ADF), Microsoft Fabric, PySpark, SQL, and modern cloud data platforms .

The ideal candidate should have strong hands-on experience in ETL/ELT development, data modeling, SQL Stored Procedures, Azure Data Lake Storage, and data warehouse concepts . Experience with Microsoft Fabric, Fabric Notebooks, PySpark, and AI-assisted development tools such as Claude/Claude Code or GitHub Copilot will be an added advantage.

Experience

6–10 Years

Primary Skills

- Azure Data Factory (ADF)
- Microsoft Fabric / Fabric Data Factory
- Fabric Notebooks
- PySpark
- SQL / T-SQL
- SQL Stored Procedures
- Data Modeling
- Azure Data Lake Storage Gen2 (ADLS Gen2)
- Data Warehouse / Lakehouse
- Azure DevOps / Git
- AI-assisted development tools – Claude, Claude Code, GitHub Copilot

Key Responsibilities Data Engineering & Platform Development
- Design, develop, and maintain scalable ETL/ELT pipelines using Azure Data Factory and Microsoft Fabric Data Factory .
- Develop data engineering solutions using Fabric Notebooks, PySpark, and SQL .
- Analyze, transform, integrate, and process data from multiple sources.
- Develop and maintain Lakehouse, Data Warehouse, and Medallion Architecture implementations.
- Develop and optimize complex SQL queries, T-SQL, and Stored Procedures .
- Design and implement effective data models , including Fact/Dimension structures and Star Schema.
- Work with Azure Data Lake Storage Gen2 (ADLS Gen2) and other Azure data services.
- Ensure data quality, consistency, governance, security, validation, reconciliation, and exception handling.




- Contribute to scalable and reusable data engineering frameworks and solutions.

Azure & Cloud Data Engineering
- Work extensively with Azure Cloud architecture and data services .
- Collaborate with Solution Architects and technical stakeholders to design scalable and reliable cloud data solutions.
- Troubleshoot data pipelines, integration issues, and production incidents.
- Participate in technical design discussions, code reviews, and production support activities.

AI-Assisted Engineering
- Leverage AI-assisted development tools such as Claude, Claude Code, GitHub Copilot, and Microsoft Fabric AI capabilities to improve development productivity.
- Apply AI-assisted techniques for coding, debugging, documentation, unit testing, optimization, and troubleshooting .
- Identify opportunities to use Generative AI to automate repetitive data engineering and development activities.
- Follow organizational guidelines and best practices when using AI-assisted development tools.

Collaboration & Delivery
- Work closely with Product Owners, Business SMEs, Data Analysts, Data Scientists, BI Developers, QA teams, and Architects to understand requirements and deliver data solutions.
- Participate in Agile ceremonies, technical discussions, design reviews, and code reviews.
- Support data consumers and stakeholders with technical troubleshooting and data-related requirements.
- Contribute to coding standards, engineering best practices,



documentation, and reusable components.

DevOps & Automation
- Implement CI/CD pipelines using Azure DevOps and/or GitHub .
- Follow Git-based version control and branching best practices.
- Automate deployment, testing, monitoring, and operational processes.
- Follow modern DataOps and engineering practices for continuous and reliable delivery.

Required Skills
- 6–8 years of hands-on experience in Data Engineering .
- Strong hands-on experience with Azure Data Factory (ADF) .
- Strong knowledge of Azure SQL Database / SQL Server, T-SQL, and Stored Procedures .
- Strong understanding of data modeling and data warehouse concepts .
- Experience with Azure Data Lake Storage Gen2 (ADLS Gen2) .
- Hands-on experience with ETL/ELT pipeline development and data integration .
- Good knowledge of Python .
- Working knowledge of PySpark and Fabric Notebooks.
- Positive understanding of Microsoft Fabric , including Lakehouse, Warehouse, and Data Factory.
- Experience with Azure DevOps and Git .
- Strong analytical, problem-solving, troubleshooting, and communication skills.

Good to Have
- Microsoft Fabric – Data Factory, Lakehouse, Warehouse, and Notebooks.
- Advanced PySpark experience.
- Azure Logic Apps.
- Power BI Semantic Models.
- Experience with Medallion Architecture.
- Experience with DataOps practices.
- Experience using Claude / Claude Code or other AI-assisted development tools.
- Knowledge of Generative AI applications in data engineering.

Key Technology Stack Azure Data Factory | Microsoft Fabric | Fabric Notebooks | PySpark | SQL | T-SQL | Stored Procedures | ADLS Gen2 | Data Modeling | Lakehouse | Data Warehouse | Azure DevOps | Git | Python | Claude | Claude Code | GitHub Copilot

📌 Senior Data Engineer – Azure & Microsoft Fabric (Bengaluru)
🏢 Sonata Software
📍 Bengaluru