:

Role Overview

We are looking for an experienced Data Engineer with 4–5 years of hands-on experience in building scalable data pipelines, managing large datasets, and designing efficient data architectures.

Key Responsibilities

- Design, develop, and maintain scalable data pipelines for batch and real-time processing.

- Build and optimize ETL/ELT workflows to integrate data from multiple sources.

- Develop and optimize complex SQL queries, database schemas, and data models.

- Work with analytical databases and data warehouses to support business intelligence and reporting.

- Implement data ingestion pipelines using APIs, databases, event streams, and files.

- Ensure data quality, consistency, reliability, and governance.

- Optimize pipeline performance, query execution, storage, and infrastructure costs.

- Collaborate with backend engineers, product teams, and business stakeholders.

- Implement monitoring, logging, error handling, and automated recovery for data pipelines.

- Maintain technical documentation and follow engineering best practices.

Required Skills

- Experience: 4–5 years in data engineering or a similar role.

- Programming: Solid proficiency in Python.

- Databases: Advanced SQL and hands-on experience with MySQL or PostgreSQL.

- Data Processing: Experience building ETL/ELT pipelines and data transformation workflows.

- Analytical Databases: Hands-on experience with ClickHouse.

- Cloud: Practical experience with AWS services such as S3, Glue, Lambda, Athena, or EMR.

- Big Data:



Working knowledge of Apache Spark or PySpark.

- Streaming: Understanding of Apache Kafka or other event-streaming technologies.

- Orchestration: Experience with Apache Airflow or similar workflow orchestration tools.

- DevOps: Familiarity with Docker, Git, CI/CD, and monitoring tools.

- Strong understanding of data modelling, partitioning, indexing, and query optimization.

Preferred Qualifications

- Experience handling high-volume transactional and analytical datasets.

- Experience with CDC (Change Data Capture) and incremental data loading.

- Familiarity with lakehouse architecture and data formats such as Parquet.

- Experience building near-real-time analytics and reporting platforms.

- Knowledge of data security, access control, and data governance.

What We Expect

- Strong problem-solving and analytical skills.

- Ability to independently own data pipelines from design to production.

- Focus on reliability, scalability, performance, and maintainability.

- Ability to work effectively with cross-functional engineering teams.

- A proactive approach to identifying and resolving data quality and performance issues.

Success Metrics

- Reliability and freshness of data pipelines.

- Reduction in data processing time and infrastructure costs.

- Accuracy and consistency of analytical datasets.

- Performance and scalability of data infrastructure.

- Timely delivery of business-critical data requirements.

📌 Data Engineer for Kolkata
🏢 RECEX
📍 Kolkata