Job Title: PySpark Data Engineer

Experience: 4+ Years

Location: Hyderabad

Job Summary:

We are seeking a Senior Spark Engineer to design and implement high-performance Spark execution patterns inside xFlows, supporting batch and streaming pipelines with built-in data quality, observability, and governance.

Requirements

Key Responsibilities
Design and implement Spark-based execution frameworks for xFlows pipelines.
Build reusable Spark components for:
Readers (JDBC, Files, Kafka, CDC)
Transformers (Join, Filter, Aggregate, Window, Union)
Writers (Iceberg, Delta, Parquet, Snowflake)
Optimize Spark performance (partitioning, caching, shuffles, memory).
Implement Data Quality & Reconciliation execution patterns.
Handle schema evolution, CDC, watermarking, and checkpoints.
Integrate Spark jobs with EMR Serverless / Data bricks / Kubernetes.
Publish execution metrics, logs, and lineage for observability.
Work closely with platform & UI teams to support no-code execution.

Required Skills
4+ years of experience with Apache Spark




Strong expertise in PySpark (preferred) or Spark Scala
Deep understanding of Spark internals (DAGs, stages, shuffles, caching)
Experience with Iceberg / Delta Lake
Robust SQL skills
Experience with batch and streaming pipeline

Benefits
Comprehensive Medical Coverage:

Health insurance of INR 7.0 Lakhs for you and your family (up to 6 members), ensuring complete peace of mind.
Robust Protection Plans:

Group Personal Accident Insurance and Group Term Life Insurance to safeguard you and your loved ones.
Retirement Advantages:

PF and Gratuity provided as per standard government regulations.
Versatile Work Options:

Enjoy hybrid work arrangements & flexible working hours
Generous Leave Policy:

21 days of annual leave, in addition to 10 company-declared holidays.
Employee Well-being Spaces:

Access to a dedicated break-out area with round-the-clock refreshments for relaxation and rejuvenation.

📌 Sr Pyspark Data Engineer Hyderabad
🏢 DataEconomy
📍 Hyderabad