Project Role : Custom Software Engineer
Project Role Description : Develop custom software solutions to design, code, and enhance components across systems or applications. Use up-to-date frameworks and agile practices to deliver scalable, high-performing solutions tailored to specific business needs.
Must have skills : Scala
Good to have skills : NA
Minimum 5 year(s) of experience is required
Educational Qualification : 15 years full time education

Summary:
We are seeking a hands-on Technical Lead to design, develop, and manage real-time data pipelines that power core banking data products. In this role, you will build and operate Apache Spark Structured Streaming applications to process CDC events from Confluent Kafka, apply complex business transformations, and store data in MongoDB Atlas, all running on Amazon EMR in AWS. You will collaborate closely with mainframe, CDC, and platform teams to deliver scalable, reliable, and high-performance data solutions while providing technical leadership and driving engineering best practices..
Roles & Responsibilities:
Lead end-to-end development of Spark Structured Streaming pipelines in Scala, from ingestion to persistence
Own technical decisions around data modeling, processing logic and MongoDB write strategies
Coordinate with IBM CDC team, Confluent Kafka team and MongoDB/GFT teams for cross-system integrations
Review and approve code contributions from team members via Bitbucket pull requests




Ensure code quality through SonarQube quality gates, JaCoCo coverage and Checkstyle compliance
Manage CI/CD pipelines via Jenkins for INT, PRE and PRO environment deployments on Amazon EMR
Translate business requirements into technical specifications and low-level design documents
Mentor junior and mid-level engineers on Spark internals, Scala best practices and streaming concepts
Drive incident resolution and root cause analysis for production issues
Maintain technical documentation including Technical Documents, Confluence pages and READMEs
Technical Skills:
Must have:
Scala — strong proficiency in functional and object-oriented programming
Apache Spark Structured Streaming — foreachBatch, micro-batch processing, offset management, checkpointing
Kafka — consumer configuration, offset management, SASL/SSL authentication, Confluent Cloud
MongoDB — bulk write operations, upsert strategies, index design, Atlas connectivity
AWS — Amazon EMR, Amazon S3, CloudWatch log analysis
Good to have:
IBM CDC / Change Data Capture concepts
Delta Lake on S3
Jenkins CI/CD pipelines
SonarQube, JaCoCo, Checkstyle
Bitbucket / Git branching strategies (GitFlow)
Java (for mixed Scala/Java codebases)
Experience: Minimum 7 years of experience in Scala and distributed data processing. Prior experience leading a team of 3+ engineers preferred.

15 years full time education

📌 Custom Software Engineer (India)
🏢 Accenture
📍 India