Data Engineer (Streaming & Integration)

Position: Data Engineer (Streaming &

Integration)

Location: offshore/India

Experience: Minimum 8+ years

Role Overview

We are hiring one Data Engineer to lead the design, development, and operation of streaming and enterprise data pipelines.

This role owns technical direction, makes architectural trade-offs, and works directly with onsite clients in the US.

This is a builder + owner role. You will write code, review designs, debug production issues,

and set engineering standards.

Responsibilities

- Architect, build, and operate real-time and near-real-time data pipelines
- Lead development using Confluent

Kafka and Apache Flink o Apache Spark Structured

Streaming is acceptable instead of

Flink

Required Skills & Experience:

- 8+ years of hands-on data engineering experience
- Exceptional SQL skills (complex analytics, performance tuning)
- Solid expertise in Python, Java, or .NET

(C#)

(at least one OOP language is mandatory)

- Deep experience with Confluent

Kafka (architecture, partitioning, delivery semantics)
- Hands-on streaming experience with:

o Apache Flink (preferred) or o Apache Spark Structured

Streaming

- Strong understanding of:





o ETL / ELT patterns o MPP processing concepts o Data modeling and orchestration
- Experience with SQL Server
- Good understanding of data governance,

secure PII handling within streaming data pipelines.
- Experience with event serialization formats like Protobuf / JSON Schema

Nice to Have

- SSIS experience
- Docker / Kubernetes
- Design and maintain batch and hybrid

ETL pipelines using SQL Server and

SSIS

- Own data quality, observability, and reliability across pipelines
- Define and enforce best practices for:

o Streaming semantics o Error handling o Backpressure and performance tuning
- Work directly with onsite clients:

o Translate requirements into technical solutions o Defend design decisions o Troubleshoot complex production issues
- Mentor engineers and raise overall engineering bar
- Collaborate with DevOps and platform teams in containerized, distributed environments
- Manage event schema registries and backward compatibility standards across streaming pipelines.
- Partner with downstream and upstream system teams to define robust data contracts and payloads.

📌 Data Engineer (India)
🏢 Awign Expert
📍 India