Key Responsibilities

- Design, develop, and maintain scalable data pipelines on Azure to support batch and real-time processing.

- Implement and manage real-time data streaming solutions using Kafka, IoT, and Azure Event Hub.

- Develop robust, production-grade solutions using Databricks and PySpark to handle large-scale data sets.

- Ensure seamless data ingestion and processing by leveraging Azure Data Factory and Azure Data Lake Storage.

- Collaborate with cross-functional teams to deliver comprehensive data-driven solutions and insights.

- Optimize and maintain data processing workflows, ensuring high availability and performance across platforms.

- Provide technical leadership and guidance in the adoption of current data technologies and methodologies.

- Monitor and troubleshoot data pipelines to prevent and resolve production issues swiftly.

Overview

- The Realtime Streaming Azure Engineer role based in Bangalore is a pivotal position within our consulting sector, focusing on designing and maintaining data pipelines and streaming solutions.

- The candidate will leverage their expertise in Azure services, big data technologies, and real-time data streaming to deliver high-quality, production-ready data solutions.

- This position requires collaboration with cross-functional teams to implement scalable solutions using tools like Databricks, PySpark, and Kafka.

- The ideal candidate will bring a minimum of 6-9 years of relevant experience, demonstrating strong communication skills and the ability to work in a fast-paced, data-driven environment.

- This role emphasizes the development of both batch and real-time data processing solutions, ensuring efficient data ingestion and processing.





- The engineer will be responsible for implementing production-grade streaming solutions, requiring hands-on experience with technologies like Azure Data Factory, Azure Data Lake Storage, and Kafka.

- The position offers the opportunity to work on cutting-edge data technologies and contribute to the delivery of innovative data-driven solutions.

- The role supports the organization in achieving its data strategy by ensuring robust data pipelines and real-time streaming capabilities.

- Candidates will benefit from exposure to a hybrid work model, fostering a balance of remote and on-site work in Bangalore.

- The position is critical for maintaining seamless data operations and supporting the organization's goals of providing top-tier consulting services.

- With a strong focus on production implementations, the Azure Engineer will handle end-to-end pipeline development, contributing to the organization's ability to manage large-scale data environments effectively.

- This role offers a challenging yet rewarding opportunity for individuals passionate about data engineering and real-time analytics.

Requirements

- 6-9 years of experience in data engineering with a focus on real-time streaming and big data technologies.

- Proficiency in Azure services, including Azure Data Factory, Azure Data Lake Storage, and Azure Databricks.

- Strong programming skills in Python and PySpark, with demonstrated experience in production solutions.

- Hands-on experience with Kafka-based streaming projects and expertise in implementing IoT solutions.

- Working knowledge of SQL for data querying and transformation tasks.

- Experience with Delta Live Tables and Azure Event Hub is a plus.

- Excellent communication skills with the ability to explain complex technical concepts to non-technical stakeholders.

📌 Realtime Streaming Engineer (Bengaluru)
🏢 Net Connect
📍 Bengaluru