Overview

We are looking for a skilled Data Engineer II to join our Data Platform team. You will

play a key role in building and optimizing our next-generation data infrastructure.

Operating at the scale of Flipkart (Petabytes of data), you will design, develop, and

maintain high-throughput distributed systems, bridging traditional big data

engineering with modern cloud-native and AI-driven workflows.

Key Responsibilities

Data Pipeline Development & Optimization

- Build Scalable Pipelines: Design, develop, and maintain robust ETL/ELT

pipelines using Scala and Apache Spark/Flink (Core, SQL, Streaming) to process

massive datasets with low latency.

- Performance Tuning: optimize Spark jobs and SQL queries for efficiency,

resource utilization, and speed.

- Lakehouse Implementation: Implement and manage data tables using modern

Lakehouse formats like Apache Iceberg, Hudi, or Delta Lake, ensuring efficient

storage and retrieval.

Data Management & Quality

- Data Modeling: Apply Medallion Architecture principles (Bronze/Silver/Gold) to

structure data effectively for downstream analytics and ML use cases.

- Data Quality: Implement data validation checks and automated testing using

frameworks (e.g., Deequ, Great Expectations) to ensure data accuracy and

reliability.

- Observability: Integrate pipelines with observability tools to monitor data health,

freshness, and lineage.

Cloud Native Engineering

- Cloud Infrastructure: Deploy and manage workloads on GCP DataProc and

Kubernetes (K8s), leveraging containerization for scalable processing.

- Infrastructure as Code: Contribute to infrastructure automation and deployment

scripts.

Collaboration & Innovation

- GenAI Integration: Explore and implement GenAI and Agentic workflows to





automate data discovery and optimize engineering processes.

- Agile Delivery: Work closely with architects and product teams in an

Agile/Scrum environment to deliver features iteratively.

- Code Reviews: Participate in code reviews to maintain code quality, standards,

and best practices.

Required Qualifications

- Experience: 3-5 years of hands-on experience in Data Engineering.
- Primary Tech Stack:

Solid proficiency in Scala and Apache Spark (Batch & Streaming).

Solid understanding of SQL and distributed computing concepts.

Experience with GCP (DataProc, GCS, BigQuery) or equivalent cloud

platforms (AWS/Azure).

Hands-on experience with Kubernetes and Docker.

- Architecture & Storage:

Experience with Lakehouse table formats (Iceberg, Hudi, or Delta).

Understanding of data warehousing and modeling concepts (Star schema,

Snowflake schema).

- Soft Skills:

Strong problem-solving skills and ability to work independently.

Good communication skills to collaborate with cross-functional teams.

Education Qualification

- Bachelors or Master’s degree in Computer Science, Information Technology,

Engineering, or a related quantitative field.

Preferred Qualifications

- Machine Learning Background: Familiarity with ML concepts, feature

engineering, or experience building data pipelines for ML models is highly

preferred.

- Experience with workflow orchestration tools (Airflow, Azkaban, etc.).
- Familiarity with real-time analytics databases (Druid, ClickHouse, HBase).
- Experience with CI/CD pipelines for data applications.

Why Join Us?

- Work on petabyte-scale challenges that define the industry standard.
- Collaborate with top-tier engineers in a high-growth environment.
- Opportunity to work with cutting-edge technologies like Iceberg, K8s, and GenAI.

📌 Data Engineer (Bengaluru)
🏢 Smart IMS
📍 Bengaluru