Data Engineer: Microsoft Fabric, Batch and Streaming Pipelines

Experience: 5+ years (adjust to the client's level) | Hours: Full-time, 40 hrs/week, aligned to client's team time zone

Role Overview

Join a product-focused team to build and run the data pipelines behind customer-facing applications and analytics. You'll own your pipelines from first design through production support, with a focus on Microsoft Fabric.

Key Responsibilities

- Develop robust batch and streaming pipelines that pull data from APIs, databases, event streams, and external vendor systems.
- Create reusable patterns for ingesting, transforming, and storing data, and model it into consistent, trusted datasets.
- Build and troubleshoot workflows in Microsoft Fabric Notebooks and move them into production.
- Put automated testing, version control, and CI/CD around every pipeline you ship.
- Set up data quality checks, alerting, and monitoring so issues surface before users notice them.
- Tune queries, pipelines, and storage to keep runtime and cloud spend under control.
- Maintain clear documentation of data flows, lineage, and metadata.




- Work within agile ceremonies (planning, standups, retros) alongside product managers, developers, and analysts.
- Follow data privacy and security standards, and help mentor junior engineers.

Mandatory Skills

- Programming: Python, Java, or Scala for pipeline development
- Data Frameworks: Hands-on with at least two of Spark, Kafka, Flink, dbt
- Databases: Robust SQL plus working knowledge of NoSQL; solid data modeling fundamentals
- Platform: Microsoft Fabric Notebooks, used in real project work
- Containers & Orchestration: Docker and Kubernetes
- Monitoring: Experience with observability tooling; Datadog strongly preferred
- Performance: Proven SQL and pipeline optimization for speed and cost
- Good to have: Fabric Lakehouse, Warehouse, Data Pipelines, Dataflows Gen2, Semantic Models; Azure or AWS data services; CI/CD tools (GitHub Actions, Jenkins, Azure DevOps); use of generative or agentic AI tools to speed up engineering work

📌 Data Engineer (Hyderabad)
🏢 Virtues R u0026 D
📍 Hyderabad