Big Data Developer

Experience

48 Years

Employment Type

Full-Time

Job Summary

We are seeking a highly skilled Big Data Developer to design, develop, and maintain large-scale data processing systems and data pipelines. The ideal candidate will have experience working with distributed computing frameworks, cloud platforms, and modern big data technologies to support data analytics, reporting, and machine learning initiatives.

The candidate will collaborate with data engineers, data scientists, business analysts, and technology teams to build scalable, reliable, and high-performance data solutions.

Key Responsibilities

- Design, develop, and maintain scalable big data solutions and data pipelines.
- Build and optimize ETL/ELT processes for processing structured and unstructured data.
- Develop data ingestion frameworks from multiple data sources including databases, APIs, streaming systems, and file-based systems.
- Work with distributed data processing technologies such as Spark, Hadoop, and Kafka.
- Develop batch and real-time data processing applications.
- Ensure data quality, integrity, security, and governance across data platforms.
- Monitor and troubleshoot production data workflows and resolve performance issues.
- Collaborate with Data Architects and Business Stakeholders to understand data requirements.
- Implement best practices for data modeling, code quality, testing, and deployment.




- Participate in code reviews and contribute to continuous improvement initiatives.
- Support cloud-based data solutions and migration projects.

Required Technical Skills

Big Data Technologies

- Apache Spark (PySpark/Scala Spark)
- Hadoop Ecosystem (HDFS, Hive, Yarn)
- Apache Kafka
- Apache Airflow
- Databricks

Programming Languages

- Python
- Scala
- Java
- SQL

Database Technologies

- SQL Server
- Oracle
- PostgreSQL
- MySQL
- NoSQL Databases (MongoDB, Cassandra, HBase)

Cloud Platforms

- Microsoft Azure (ADLS, Azure Data Factory, Synapse Analytics)
- AWS (S3, EMR, Glue, Redshift)
- Google Cloud Platform (BigQuery, DataProc) – Preferred

DevOps & Version Control

- Git
- Azure DevOps
- Jenkins
- CI/CD Pipelines

Required Qualifications

- Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.
- 3+ years of experience in Big Data development and data engineering projects.
- Solid experience with Spark, Hadoop, Kafka, and distributed computing environments.
- Proficiency in Python, Scala, or Java.
- Strong SQL and data modeling skills.
- Experience working with cloud-based data platforms (Azure/AWS/GCP).
- Understanding of data warehousing concepts and ETL design patterns.
- Experience in Agile/Scrum development methodologies.

📌 Big Data Developer (Bengaluru)
🏢 StatusNeo Technology Consulting
📍 Bengaluru