: Big Data Developer (Spark Scala)

Position: Big Data Developer

Experience: 5–7 years (relevant)

About the Role

We are looking for an experienced Big Data Developer with strong expertise in Apache Spark and Scala to design, build, and optimize large-scale data processing pipelines. The ideal candidate will have a solid background in distributed computing, big data ecosystems, and a team-oriented mindset to work closely with cross-functional teams.

Key Responsibilities

- Design, develop, and maintain scalable data processing pipelines using Apache Spark with Scala
- Optimize Spark jobs for performance, including partitioning, caching, and resource tuning
- Work with large-scale structured and unstructured datasets across distributed systems
- Collaborate with data architects, analysts, and business stakeholders to understand requirements and translate them into technical solutions
- Write clean, efficient, and well-documented code following best engineering practices
- Debug and resolve performance bottlenecks in big data jobs
- Participate in code reviews, design discussions, and sprint planning
- Ensure data quality, consistency, and integrity across pipelines
- Support production deployments and troubleshoot issues in live environments

Required Skills & Experience

- 5–7 years of overall IT experience, with substantial hands-on experience in Big Data development




- Strong proficiency in Apache Spark and Scala (mandatory)
- Solid understanding of distributed computing concepts (RDDs, DataFrames, Datasets, partitioning, shuffling)
- Experience with Hadoop ecosystem components (HDFS, Hive, YARN, etc.)
- Proficiency in SQL and working with large relational/non-relational datasets
- Experience with performance tuning and optimization of Spark applications
- Familiarity with version control systems (Git) and CI/CD pipelines
- Strong problem-solving and analytical skills
- Excellent verbal and written communication skills — must be able to clearly articulate technical concepts to both technical and non-technical stakeholders

Good to Have

- Experience with PySpark
- Hands-on experience with Databricks (notebooks, clusters, Delta Lake, job orchestration)
- Exposure to cloud platforms (AWS/Azure/GCP)
- Knowledge of workflow orchestration tools (Airflow, Oozie)
- Experience with Kafka or other streaming/messaging systems
- Familiarity with Agile/Scrum methodologies

Education

- Bachelor's or Master's degree in Computer Science, Engineering, or a related field

Soft Skills

- Strong team player with the ability to work independently
- Ability to manage multiple priorities and meet deadlines
- Client-facing communication experience is a plus

📌 Spark Scala Developer (Hyderabad)
🏢 Quess Corp
📍 Hyderabad