Job Overview:

We are looking for a Data Engineering Manager with 10 - 15 years of experience to lead data engineering initiatives, build scalable data pipelines, manage data lake architecture, and enable real-time analytics and AI use cases. The role will work closely with application, ML, analytics, and business teams while mentoring data engineers and driving modern engineering practices.

Key Responsibilities:

- Lead real-time pipelines using Kafka/MSK, Debezium CDC, and Apache Pinot.
- Design and maintain Java/Python-based frameworks for ingestion, validation, transformation, and replication.
- Design and manage data lakes using Apache Iceberg, AWS Glue, Athena, and S3.
- Define data modeling standards across Pinot, Redshift, and warehouse layers.
- Lead orchestration and workflow platforms using Apache Airflow and Superset.
- Drive cloud-native deployments using ECS/EKS, Docker, Terraform/CDK, and CI/CD.
- Support ML, analytics, product, and AI initiatives with reliable data solutions.
- Ensure data quality, observability,



lineage, governance, and security across the data stack.
- Mentor data engineers and promote contemporary architecture and engineering best practices.

Required Skills:

- Real-Time Data: Kafka/MSK, Debezium, Event-Driven Architecture, CDC
- Data Platforms: Apache Pinot, Redshift, RocksDB/NoSQL, Apache Iceberg, Athena, AWS Glue
- Programming: Java, Python
- Orchestration: Apache Airflow, Superset
- AWS: S3, IAM, Lambda, Glue, ECS/EKS, CloudWatch
- Data Architecture: Data Lakes, Streaming & Batch Processing, Data Modeling, Parquet, Avro
- DevOps: Docker, Terraform/CDK, CI/CD, GitHub Actions

Good to Have:

- dbt, Trino/Presto, ClickHouse, or Druid.
- Data security, encryption at rest/in transit, GDPR/PCI compliance.
- CloudFormation and advanced DevOps practices.
- Experience with scalable cloud-native data architectures.

📌 Data Engineering Manager - Data Platforms (India)
🏢 AVISOFT
📍 India