Role Summary

We are hiring a Senior Data Engineer to drive the development of scalable, high-performance data pipelines for a migration project to a modern Azure Native architecture. This role will be instrumental in building an AI-ready data platform using Microsoft Fabric (Fabric IQ) and enabling seamless integration with AI/ML workflows (Foundry IQ).

Key Responsibilities

- Design and build robust, scalable data pipelines (batch + near real-time) using Microsoft Fabric (Data Pipelines, Lakehouse)
- Lead migration of existing pipelines (AKS/custom ETL) to Fabric-native architecture
- Implement Medallion Architecture (Bronze Silver Gold) for structured and unstructured healthcare data
- Develop and optimize data models and transformations for analytics and AI use cases

- Build ingestion frameworks for
- APIs (FHIR, EHR systems)
- Files (JSON, HL7, CSV)
- Ensure data quality, validation (FHIR/claims), and observability across pipelines
- Enable AI/ML workflows by preparing datasets for:

- RAG pipelines
- Feature engineering and model consumption
- Optimize pipeline performance, cost, and scalability
- Collaborate with Architects, AI/ML engineers, and Product teams to translate requirements into data solutions
- Mentor junior engineers and drive engineering best practices

Required Experience

- 4-6 years of experience (3+ years in data engineering)
- Strong hands-on experience with Azure Data Platform
- Proven experience in building and optimizing large-scale data pipelines




- Experience in data platform modernization or migration projects

Technical Skills

- Languages: Python, PySpark, SQL (advanced)

- Azure / Fabric:
- Microsoft Fabric (preferred) OR Azure Data Factory, Synapse, Data Lake

- Data Architecture:
- Lakehouse, Medallion Architecture
- Batch + streaming data processing

- Data Handling:
- JSON, Parquet, CSV, API integrations

- Performance Optimization:
- Partitioning, indexing, query tuning

Nice to Have

- Experience with healthcare data (FHIR, claims, EHR)
- Exposure to RAG pipelines / vector databases / AI data prep
- Familiarity with event-driven architectures
- Understanding of data security & governance in regulated environments

Success Metrics

- High-quality, scalable pipelines delivered on time
- Seamless migration with minimal business disruption
- Improved data performance, reliability, and cost efficiency
- Robust enablement of AI/ML and analytics use cases

What Were Looking For

- Strong hands-on engineer with ownership mindset
- Ability to operate in fast-paced, evolving architecture environment
- Problem-solver with focus on performance, scalability, and data quality
- Mentor and team player

Why This Role Matters This role is critical to building the foundation of AI-first platform, enabling:

- Scalable data processing via Fabric IQ
- AI-driven intelligence via Foundry IQ
- Future-ready healthcare data platform

📌 Senior Data Engineer- Azure (Ahmedabad)
🏢 RAF Solutions
📍 Ahmedabad