Job Description

About Halcer Halcer is a leading IT services, consulting, and staffing organization committed to delivering creative technology solutions and top-tier technical talent to global enterprises.

/n

/n

Job Overview We are seeking an experienced Data Engineer (6+ Years of Experience) to build and maintain scalable, high-performance data pipelines and infrastructure for our next-generation data platform. The platform ingests and processes real-time and historical data from diverse industrial sources such as airport systems, sensors, cameras, and APIs. You will work closely with AI/ML engineers, data scientists, and DevOps teams to enable reliable analytics, forecasting, and anomaly detection use cases.

/n

/n

Key Responsibilities

/n

/n
Design and implement real-time (Kafka, Spark/Flink) and batch (Airflow, Spark) pipelines for high-throughput data ingestion, processing, and transformation.
/n
Develop data models and manage data lakes and warehouses (Delta Lake, Iceberg, etc.) to support both analytical and ML workloads.
/n




Integrate data from diverse sources: IoT sensors, databases (SQL/NoSQL), REST APIs, and flat files.
/n
Ensure pipeline scalability, observability, and data quality through monitoring, alerting, validation, and lineage tracking.
/n
Collaborate with AI/ML teams to provision clean and ML-ready datasets for training and inference.
/n
Deploy, optimize, and manage pipelines and data infrastructure across on-premise and hybrid settings.
/n
Participate in architectural decisions to ensure resilient, cost-effective, and secure data flows.
/n
Contribute to infrastructure-as-code and automation for data deployment using Terraform, Ansible, or similar tools.
/n

/n

/n

Qualifications & Experience Requirements

/n

/n
Total Experience: 6+ years of total experience in core data engineering roles.
/n
Streaming & Real-Time Experience: 2+ years of direct experience building and maintaining real-time or streaming pipeline

📌 Data Engineer Bengaluru
🏢 Halcer
📍 Bengaluru