Job Description

Project Role: Custom Software Engineer

Project Role Description: Develop custom software solutions to design, code, and enhance components across systems or applications. Use modern frameworks and agile practices to deliver scalable, high-performing solutions tailored to specific business needs.

Must have skills: Scala

Good to have skills : NA

Minimum 5 Year(s) Of Experience Is Required

Educational Qualification: 15 years full time education

Summary:

We are seeking a hands-on Technical Lead to design, develop, and manage real-time data pipelines that power core banking data products. In this role, you will build and operate Apache Spark Structured Streaming applications to process CDC events from Confluent Kafka, apply complex business transformations, and store data in MongoDB Atlas, all running on Amazon EMR in AWS.

You will collaborate closely with mainframe, CDC, and platform teams to deliver scalable, reliable, and high-performance data solutions while providing technical leadership and driving engineering best practices..

Roles & Responsibilities:

Lead end-to-end development of Spark Structured Streaming pipelines in Scala, from ingestion to persistence

Own technical decisions around data modeling, processing logic and MongoDB write strategies

Coordinate with IBM CDC team, Confluent Kafka team and MongoDB/GFT teams for cross-system integrations





Review and approve code contributions from team members via Bitbucket pull requests

Ensure code quality through Sonar Qube quality gates, JaCoCo coverage and Checkstyle compliance

Manage CI/CD pipelines via Jenkins for INT, PRE and PRO setting deployments on Amazon EMR

Translate business requirements into technical specifications and low-level design documents

Mentor junior and mid-level engineers on Spark internals, Scala best practices and streaming concepts

Drive incident resolution and root cause analysis for production issues

Maintain technical documentation including Technical Documents, Confluence pages and READMEs

Technical Skills:

Must have:

Scala — strong proficiency in functional and object-oriented programming

Apache Spark Structured Streaming — foreach Batch, micro-batch processing, offset management, checkpointing

Kafka — consumer configuration, offset management, SASL/SSL authentication, Confluent Cloud

MongoDB — bulk write operations, upsert strategies, index design, Atlas connectivity

AWS — Amazon EMR, Amazon S3, Cloud Watch log analysis

Good to have:

IBM CDC / Change Data Capture concepts

Delta Lake on S3

Jenkins CI/CD pipelines

Sonar Qube, JaCoCo, Checkstyle

Bitbucket / Git branching strategies (Git Flow)

Java (for mixed Scala/Java codebases)

Experience: Minimum 7 years of experience in Scala and distributed data processing. Prior experience leading a team of 3+ engineers preferred.

📌 Custom Software Engineer (Bengaluru)
🏢 Accenture
📍 Bengaluru