30 Jul
|
Snapmint
|
India
Job Description
:
Snapmint is looking for a skilled Data Engineer with 3–5 years of experience to design, build, and manage real-time data pipelines using technologies like Kafka, Flink, and Spark Streaming. The role involves optimizing scalable, fault-tolerant pipelines, performing real-time transformations, and collaborating with data scientists for feature development. The ideal candidate will have strong programming skills in Python, Java, or Scala, along with solid SQL expertise and a good understanding of data modeling, data warehousing, and OLTP vs. OLAP systems. Experience with CDC tools, data lakes/lakehouse architectures (Databricks), open table formats (Delta Lake, Iceberg, Hudi), and orchestration tools like Airflow is essential.
Skills Required
:
SQL,PYTHON,Jav,ETL
Role
:
Key Responsibilities:
- Design, build, and manage real-time data pipelines using tools like Apache Kafka, Apache Flink, Apache Spark Streaming.
- Optimize data pipelines for performance, scalability, and fault-tolerance.
- Perform real-time transformations, aggregations, and joins on streaming data.
- Collaborate with data scientists to onboard new features and ensure they're discoverable, documented, and versioned.
- Optimize feature retrieval latency for real-time inference use cases.
- Ensure strong data governance: lineage, auditing, schema evolution, and quality checks using tools such as dbt, and Open Lineage.
Requirements:
- Bachelor's degree in Engineering from a premier institute (IIT/NIT/ BIT)
- 3-5 years of experience in an Indian startup/ tech company
- Solid programming skills in Python, Java, or Scala and proficient in SQL.
- Solid understanding of data modeling, data warehousing concepts, and the differences between OLTP and OLAP workloads.
- Experience ingesting and processing various data formats, including semi-structured (JSON, Avro), unstructured, and document-based data from sources like NoSQL databases (e.g., MongoDB), APIs, and event tracking platforms (e.g., PostHog).
- Hands-on experience with Change Data Capture (CDC) tools such as Debezium or AWS DMS for replicating data from transactional databases.
- Proven experience designing and building scalable data lakes or lakehouse architectures on platforms like Databricks.
- Hands-on experience with up-to-date open table formats such as Delta Lake, Apache Iceberg, or Apache Hudi.
- Hands-on experience with real-time streaming technologies like Kafka, Flink, and Spark Streaming.
- Proficiency with data pipeline orchestration tools like Apache Airflow.
- Exposure to event-driven microservices architecture.
- Strong written and verbal communication skills.
Good to have:
- Familiarity with cloud data warehouse systems like BigQuery or Snowflake.
- Experience with real-time analytical databases like ClickHouse.
- Familiarity with designing, building, and maintaining feature store infrastructure to support machine learning use cases.
Location
:
Bengaluru, Karnataka, India
Desirable Skills
:
sql,python,java
Years Of Exp
:
3 to 5 years
Posted On
:
30-Mar-2026
📌 Data Engineer (India)
🏢 Snapmint
📍 India