Job DescriptionAbout Halcer Halcer is a leading IT services, consulting, and staffing organization committed to delivering innovative technology solutions and top-tier technical talent to global enterprises. NnJobOverview We are seeking an experienced Data Engineer (6+ Years of Experience) to build and maintain scalable, high-performance data pipelines and infrastructure for our next-generation data platform. The platform ingests and processes real-time and historical data from diverse industrial sources such as airport systems, sensors, cameras, and APIs. You will work closely with AI/ML engineers, data scientists, and DevOps teams to enable reliable analytics, forecasting, and anomaly detection use cases. NnKeyResponsibilities Nn Design and implement real-time (Kafka, Spark/Flink) and batch (Airflow, Spark) pipelines for high-throughput data ingestion, processing, and transformation. N Develop data models andmanage data lakes and warehouses (Delta Lake, Iceberg, etc.) to support both analytical and ML workloads. N Integrate data from diverse sources: IoT sensors, databases (SQL/NoSQL), REST APIs, and flat files. N Ensure pipeline scalability, observability, and data quality through monitoring, alerting, validation, and lineage tracking. N Collaborate with AI/ML teams to provision clean and ML-ready datasets for training and inference. N Deploy, optimize, and manage pipelines and data infrastructure across on-premise and hybrid environments. N Participate in architectural decisions to ensure resilient, cost-effective, and secure data flows.



N Contribute to infrastructure-as-code and automation for data deployment using Terraform, Ansible, or similar tools. NnnQualifications & Experience Requirements Nn Total Experience: 6+ years of total experience in core data engineering roles. N Streaming & Real-Time Experience: 2+ years of direct experience building and maintaining real-time or streaming pipelines. N Education: Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field. NnnRequired Skills Nn Strong programming proficiency in Python or Java, alongside expert-level SQL. N Hands-on experience with Apache Kafka, Apache Spark, or Apache Flink for real-time and batch processing. N Proficiency with workflow orchestration tools like Airflow, dbt, or similar technologies. N Deep familiarity with data modeling (OLAP/OLTP), schema evolution, and file formats (Parquet, Avro, ORC). N Hands-on experience with hybrid/on-premise and cloud platform deployments (AWS, GCP, or Azure). N Proven track record working with data lakes and modern data warehouses (Snowflake, BigQuery, Redshift, or Delta Lake). N Strong working knowledge of DevOps practices, Docker, Kubernetes, and IaC tools like Terraform or Ansible. N Knowledge of data observability, data cataloging, and quality frameworks (e.G., Excellent Expectations, OpenMetadata). NnnGood-to-Have Skills Nn Experience with time-series databases (e.G., InfluxDB, TimescaleDB) and processing high-frequency sensor data. N Prior domain experiencein aviation, manufacturing, or logistics. N

📌 Data Engineer (Vasanthanagar)
🏢 Halcer
📍 Vasanthanagar