Unlock Premium Benefits

Get Immediate Access to 3,932,000 premium jobs
Direct Access to Top Employers

Gain full access to exclusive job listings from leading companies worldwide.
Verified, High-Quality Jobs Only

No ads, scams, or junk-just genuine opportunities.
Focus on Real Opportunities

Explore thousands of open positions tailored to your lifestyle, including versatile remote jobs.
Exclusive Resume Review

Receive expert feedback with personalized suggestions to enhance your resume.

Continue →

Senior Data Engineer - Responsibilities (7+ Years Exp)
Build scalable data ingestion pipelines for relational systems and unstructured/semi-structured sources.
Design and optimize lakehouse workloads using Apache Iceberg.
Optimize table layouts, partitioning strategies, compaction, and schema evolution for Iceberg-based datasets.
Implement best practices for versioning, time travel, incremental processing, and ACID guarantees.
Develop and tune Apache Spark (batch & streaming) for performant, scalable data processing.
Work with AWS services (Glue, EMR, Lambda, Step Functions, S3)



focusing on cost optimization and performance tuning.
Implement and manage Kafka and Kafka streaming pipelines for large-scale real-time processing.
Build and orchestrate data workflows using DBT and Airflow.
Leverage AI/LLM automation to accelerate ETL/ELT development, validation, and debugging.
Implement observability—monitoring, logging, and alerting—to ensure platform stability and reliability.
Participate in code reviews to ensure code quality and share best practices with the team.

Required Skills
7 Years of relevant experience
Robust SQL, Python, and Spark experience
Deep experience ingesting and processing structured & unstructured data
Solid knowledge of Apache Iceberg and lakehouse architectures
Hands-on AWS experience with tuning and cost optimization
Expertise with Kafka, Kafka Streaming, DBT, and Airflow
Solid understanding of distributed systems and data engineering best practices

📌 Senior Data Engineer Hyderabad
🏢 Milestone Technologies
📍 Hyderabad