Role & responsibilities

Job Description
Spark/Scala Developer

Experience:
68 Years Required Technical

Skills:
Apache Spark, Scala, SQL Good-to-Have

Skills:
AWS, Python, AI Job Summary We are looking for an experienced Spark/Scala Developer with strong expertise in Apache Spark, Scala programming, SQL, and big data application development. The ideal candidate should have hands-on experience in developing complex Directed Acyclic Graphs (DAGs), troubleshooting failures, analyzing Spark UI metrics, and optimizing Spark applications for performance and scalability.

Key Responsibilities
- Participate in the complete Software Development Life Cycle (SDLC), including requirement analysis, technical design, coding, testing, deployment, and application maintenance.
- Develop and maintain scalable big data applications using Apache Spark and Scala.
- Design, develop, and troubleshoot complex DAGs and resolve job execution failures.
- Analyze Spark UI parameters, execution plans, stages, tasks, and performance metrics to identify bottlenecks.
- Implement Spark performance optimization techniques, including partitioning, caching, and resource tuning.
- Write efficient Scala code on top of the Spark architecture.
- Gather business and technical requirements and develop prototypes as required.
- Prepare functional specifications, technical design documents, and test cases.
- Perform debugging, root cause analysis,



unit testing, and application support.
- Collaborate with Integration Leads, Technical Leads, Project Managers, and geographically distributed teams.
- Support project deliveries and provide technical guidance to team members when required. Mandatory Skills
- Strong hands-on experience with Apache Spark.
- Excellent Scala programming skills.
- Strong SQL skills and experience working with large datasets.
- In-depth understanding of Spark architecture and execution flow.
- Experience developing and troubleshooting complex DAGs.
- Strong knowledge of Spark UI, metrics, stages, tasks, and performance analysis.
- Experience in Spark performance tuning and optimization.
- Good understanding of SDLC, software development practices, testing, and documentation.
- Strong analytical, debugging, communication, and problem-solving skills. Positive-to-Have Skills
- Experience with AWS cloud services and big data workloads on AWS.
- Knowledge of Python programming.
- Exposure to AI/ML capabilities and related technologies. Required Qualifications
- 6–8 years of relevant experience in Spark/Scala development.
- Hands-on experience building, optimizing, and maintaining Spark-based applications.
- Ability to work independently and collaboratively in a team environment.
- Strong written and verbal communication skills.

📌 Spark/Scala Developer (Bengaluru)
🏢 Cerebra
📍 Bengaluru