Experience

10–15 Years

Job Overview

We are looking for a Data Engineering Manager with 10–15 years of experience to lead data engineering initiatives, build scalable data pipelines, manage data lake architecture, and enable real-time analytics and AI use cases.

Key Responsibilities

- Own the data engineering roadmap from streaming ingestion to analytics-ready data layers.
- Lead real-time pipelines using Kafka/MSK, Debezium CDC, and Apache Pinot.
- Design and maintain Java/Python-based frameworks for ingestion, validation, transformation, and replication.
- Design and manage data lakes using Apache Iceberg, AWS Glue, Athena, and S3.
- Define data modeling standards across Pinot, Redshift, and warehouse layers.
- Lead orchestration and workflow platforms using Apache Airflow and Superset.
- Drive cloud-native deployments using ECS/EKS, Docker, Terraform/CDK, and CI/CD.
- Support ML, analytics, product, and AI initiatives with reliable data solutions.
- Ensure data quality, observability, lineage, governance, and security across the data stack.




- Mentor data engineers and promote modern architecture and engineering best practices.

Required Skills

- Real-Time Data: Kafka/MSK, Debezium, Event-Driven Architecture, CDC
- Data Platforms: Apache Pinot, Redshift, RocksDB/NoSQL, Apache Iceberg, Athena, AWS Glue
- Programming: Java, Python
- Orchestration: Apache Airflow, Superset
- AWS: S3, IAM, Lambda, Glue, ECS/EKS, CloudWatch
- Data Architecture: Data Lakes, Streaming & Batch Processing, Data Modeling, Parquet, Avro
- DevOps: Docker, Terraform/CDK, CI/CD, GitHub Actions

Good to Have

- dbt, Trino/Presto, ClickHouse, or Druid.
- Data security, encryption at rest/in transit, GDPR/PCI compliance.
- CloudFormation and advanced DevOps practices.
- Experience with scalable cloud-native data architectures.

Pay: ₹500,000.00 - ₹1,000,000.00 per year

Advantages

- Health insurance
- Life insurance
- Provident Fund

Work Location: In person

📌 Data Engineering manager (Noida)
🏢 AVISOFT
📍 Noida