:

We are looking for an experienced Big Data Developer with 7-11 years of experience to design, develop, and optimize large-scale data processing solutions. The ideal candidate should have strong expertise in Apache Spark, Python, Hadoop, distributed computing, streaming technologies, and cloud-based data platforms, along with the ability to provide technical guidance to development teams.

Key Responsibilities :

- Design, develop, and maintain scalable Big Data solutions for high-volume and complex data workloads.
- Develop distributed data processing applications using Apache Spark and Python.
- Design and implement batch and real-time data processing pipelines.
- Work with Hadoop ecosystem technologies including HDFS, MapReduce, Sqoop, Hive, and Impala.
- Develop stream-processing solutions using Spark Streaming, Apache Storm, or equivalent technologies.
- Design and integrate messaging and event-streaming solutions using Apache Kafka, RabbitMQ, or similar platforms.
- Build data pipelines integrating data from multiple sources including RDBMS, ERP systems, APIs, and files.
- Develop and optimize SQL queries, joins, stored procedures, and relational data models.
- Work with NoSQL databases such as HBase, Cassandra, or MongoDB based on application requirements.
- Implement ETL processes and data integration frameworks for large-scale data environments.
- Perform Spark job optimization and performance tuning to improve processing efficiency and resource utilization.




- Design and implement cloud-based Big Data solutions using Azure Databricks and related Azure data services.
- Troubleshoot production issues and ensure reliability, scalability, and performance of data platforms.
- Collaborate with data engineers, architects, analysts, and business stakeholders to translate requirements into technical solutions.
- Provide technical guidance and mentor team members while contributing to design and architecture decisions.

Required Skills :

- 7+ years of experience in Big Data and data engineering technologies.
- Solid understanding of distributed computing principles and architectures.
- Expert-level hands-on experience with Apache Spark.
- Strong programming experience in Python.
- Proficiency in Hadoop v2, HDFS, MapReduce, and Sqoop.
- Experience with Spark Streaming, Apache Storm, or equivalent stream-processing technologies.
- Hands-on experience with Kafka, RabbitMQ, or similar messaging platforms.
- Strong knowledge of Hive, Impala, and Big Data querying technologies.
- Strong SQL skills, including complex joins, stored procedures, and relational schemas.
- Experience working with NoSQL databases such as HBase, Cassandra, or MongoDB.
- Strong understanding of ETL techniques and data integration frameworks.
- Experience in Spark performance tuning and optimization.
- Hands-on experience with Azure Databricks and cloud-based data services.
- Strong problem-solving, analytical, and communication skills.
- Experience leading or technically guiding a team is required.
- Ability to design and implement scalable, reliable, and high-performance data solutions.

📌 Big Data Developer (India)
🏢 Agile Ventures
📍 India