Role & responsibilities

: Spark/Scala Developer

Experience: 68 Years

Required Technical Skills: Apache Spark, Scala, SQL

Positive-to-Have Skills: AWS, Python, AI

Job Summary

We are looking for an experienced Spark/Scala Developer with strong expertise in Apache Spark, Scala programming, SQL, and big data application development. The ideal candidate should have hands-on experience in developing complex Directed Acyclic Graphs (DAGs), troubleshooting failures, analyzing Spark UI metrics, and optimizing Spark applications for performance and scalability.

Key Responsibilities

- Participate in the complete Software Development Life Cycle (SDLC), including requirement analysis, technical design, coding, testing, deployment, and application maintenance.
- Develop and maintain scalable big data applications using Apache Spark and Scala.
- Design, develop, and troubleshoot complex DAGs and resolve job execution failures.
- Analyze Spark UI parameters, execution plans, stages, tasks, and performance metrics to identify bottlenecks.
- Implement Spark performance optimization techniques, including partitioning, caching, and resource tuning.
- Write efficient Scala code on top of the Spark architecture.
- Gather business and technical requirements and develop prototypes as required.
- Prepare functional specifications, technical design documents, and test cases.
- Perform debugging, root cause analysis, unit testing,



and application support.
- Collaborate with Integration Leads, Technical Leads, Project Managers, and geographically distributed teams.
- Support project deliveries and provide technical guidance to team members when required.

Mandatory Skills
- Strong hands-on experience with Apache Spark.
- Excellent Scala programming skills.
- Strong SQL skills and experience working with large datasets.
- In-depth understanding of Spark architecture and execution flow.
- Experience developing and troubleshooting complex DAGs.
- Strong knowledge of Spark UI, metrics, stages, tasks, and performance analysis.
- Experience in Spark performance tuning and optimization.
- Good understanding of SDLC, software development practices, testing, and documentation.
- Strong analytical, debugging, communication, and problem-solving skills.

Good-to-Have Skills
- Experience with AWS cloud services and big data workloads on AWS.
- Knowledge of Python programming.
- Exposure to AI/ML capabilities and related technologies.

Required Qualifications
- 6–8 years of relevant experience in Spark/Scala development.
- Hands-on experience building, optimizing, and maintaining Spark-based applications.
- Ability to work independently and collaboratively in a team environment.
- Strong written and verbal communication skills.

📌 Spark/Scala Developer (Bengaluru)
🏢 Cerebra
📍 Bengaluru