Role Summary

We are seeking an experienced Data Engineer Microsoft Fabric with strong expertise in designing and developing enterprise-scale data solutions using Microsoft Fabric. The ideal candidate should have extensive hands-on experience with PySpark, SQL, Medallion Architecture, and Lakehouse design principles. This role involves building scalable data ingestion frameworks, developing production-grade ETL/ELT pipelines, and delivering high-quality data solutions for analytics and reporting.

Key Responsibilities

- Perform data analysis to understand source systems and business requirements.
- Translate business requirements into scalable data models and pipeline designs.
- Design, build, and maintain data ingestion pipelines from multiple data sources using batch and incremental (delta) loading techniques.
- Develop ingestion scripts for databases, flat files, APIs, and other enterprise data sources.
- Ingest and manage raw data in the Bronze layer while ensuring reliability, scalability, and traceability.
- Transform raw data into standardized Silver datasets through validation, deduplication, and conformance.
- Design and develop Gold layer models, including fact tables, dimension tables, and aggregated datasets for analytics.
- Build end-to-end ETL/ELT pipelines across Bronze, Silver, and Gold layers.
- Develop semantic models and reporting-ready datasets.
- Implement data quality rules and validation checks to ensure data completeness, accuracy, freshness, and reconciliation.
- Design robust error-handling mechanisms, including retry logic, logging, exception handling, and alerting.
- Develop self-healing pipelines using checkpointing, idempotent processing, auto-retry, and recovery mechanisms.
- Optimize pipeline performance for production-grade workloads.
- Monitor pipeline execution and proactively resolve failures and performance bottlenecks.
- Implement CI/CD pipelines and support deployments across Development, Test, and Production environments.
- Develop, debug, and optimize PySpark and SQL scripts.




- Follow software engineering standards and coding best practices.
- Collaborate with business, BI, architecture, and engineering teams to deliver end-to-end data solutions.

Required Skills Experience Microsoft Fabric

- Extensive hands-on experience with Microsoft Fabric
- Experience designing and developing reusable ingestion frameworks
- Strong understanding of Microsoft Fabric Lakehouse architecture
- Experience building reusable and metadata-driven ingestion pipelines
- Expertise in enterprise-scale Microsoft Fabric implementations

Data Engineering

- Strong experience building scalable ETL/ELT pipelines
- Expertise in Medallion Architecture (Bronze, Silver, Gold)
- Strong understanding of Lakehouse design principles
- Experience with batch and incremental (delta) data processing
- Data modeling and semantic model development

Programming

- Strong hands-on experience with:
- PySpark
- SQL

Cloud DevOps

- Azure DevOps
- CI/CD pipeline implementation
- Environment deployment (Development, Test, Production)
- Azure Key Vault integration and secrets management

Performance Optimization

- Pipeline performance tuning
- Cost optimization
- Monitoring and troubleshooting
- Enterprise-scale data platform best practices

Preferred Skills

- Experience with metadata-driven and configurable ingestion frameworks
- Strong understanding of Microsoft Fabric deployment strategies
- Experience implementing scalable Lakehouse solutions
- Knowledge of workspace management and governance within Microsoft Fabric
- Robust analytical and problem-solving skills
- Excellent communication and stakeholder collaboration skills

Additional Expectations

- Strong PySpark fundamentals
- Deep expertise in Microsoft Fabric beyond Azure Data Factory (ADF) and SQL
- Hands-on experience with Fabric workspace management and deployment using Azure DevOps
- Experience with Lakehouse maintenance and governance
- Knowledge of Azure Key Vault integration for secure secret management
- Strong understanding of Microsoft Fabric best practices for enterprise implementations

📌 Data Engineer Microsoft Fabric (India)
🏢 Aliqan Services
📍 India