: Data Engineer – PySpark

Company: ZAGENO

Role: Data Engineer – PySpark / Data Platform

Experience: 3+ years

Employment Type: Full-time

Primary Skills: Python, SQL, PySpark, Databricks, Delta Lake, Apache Airflow, Change Data Capture (CDC)

Location: Bangalore

Notice Period Preference: Candidates who can join within 30 days are preferred.

About ZAGENO

ZAGENO is a digital platform focused on simplifying the procurement of life-sciences research products. Reliable, accurate, and well-structured product data is essential to helping researchers and life-sciences organizations discover and procure the products they need.

We are looking for a hands-on Data Engineer with strong PySpark expertise and proven production pipeline ownership to strengthen the reliability, scalability, and correctness of our data platform.

In this role, you will work on the data pipelines and systems that power our life-sciences product catalog. You will address existing pipeline failures and technical debt, build efficient data synchronisation mechanisms, improve product data quality, and develop maintainable data products for business and technical stakeholders.

This is an engineering role for someone who enjoys solving real production problems, taking ownership from design through deployment, and building reliable systems that scale.

Role Overview

As a Data Engineer, you will own the design, development, testing, deployment, and operational reliability of production-grade data pipelines.

You will work with large and complex datasets using PySpark, Python, SQL, Databricks, Delta Lake, and Apache Airflow. You will also build and maintain change data capture (CDC) workflows,



low-latency APIs, and data synchronisation processes.

A key part of the role involves transforming inconsistent product and supplier data into standardised, accurate, and reusable datasets. You will collaborate with Catalog Operations, Analytics, Data Science, and other business stakeholders to translate evolving requirements into robust engineering solutions.

We are looking for an engineer who can independently investigate failures, identify causes, implement lasting fixes, and continuously improve data quality and system performance.

Key Responsibilities

- Production Data Pipeline Engineering

- Design, develop, maintain, and optimize scalable data pipelines using PySpark, Python, and SQL.
- Take primary ownership of production pipelines, from data ingestion and transformation through validation, deployment, and ongoing support.
- Troubleshoot unreliable pipelines, resolve recurring failures, and eliminate fragile implementations and manual workarounds.
- Improve pipeline performance, execution efficiency, reliability, and maintainability.
- Implement incremental processing, productive transformations, and appropriate data partitioning strategies.
- Develop reusable components and engineering patterns that reduce duplication and technical debt.

2. Databricks and Delta Lake
- Build and maintain data processing workflows on Data bricks using PySpark.
- Work with Delta Lake tables and implement reliable data storage, transformation, and update patterns.
- Apply appropriate incremental processing and table optimization techniques.
- Maintain data consistency and correctness across pipeline runs

📌 Data Engineer (Pyspark) (Bengaluru)
🏢 MantriAI
📍 Bengaluru