123891

Senior PySpark Developer

Locations: Bengaluru and Chennai, India
Experience: 8+ Years
Domain: Trade Surveillance Investment Banking

Project Description

We are looking for a Senior PySpark Developer to work with a leading investment bank client in the Trade Surveillance Team. The role requires strong experience in data-intensive PySpark development and core Python development, along with hands-on expertise in data processing, data pipelines, concurrency, RESTful services, application development, testing, and debugging.

Skills Required

- 8+ years of experience in data-intensive PySpark development.
- Strong experience as a Core Python Developer.
- Basic experience in Generative AI / Agentic AI.
- Solid understanding of Classes, OOPS, exception handling, and parallel processing.
- Strong knowledge of database connectivity, data loading, data transformation, and calculations.
- Extensive experience with Pandas/NumPy DataFrames, including slicing, data wrangling, and aggregations.
- Hands-on experience with Lambda Functions and Decorators.
- Experience with vector operations on Pandas DataFrames/Series.
- Experience using applymap, apply, and map functions.
- Experience with concurrency and error handling for data pipeline batches ranging from 110 GB.
- Ability to understand business requirements and translate them into technical requirements.
- Ability to design the architecture of data pipelines for concurrent data processing.
- Familiarity with creating/designing RESTful services and APIs.
- Familiarity with application unit testing.
- Experience working with Git source control.
- Understanding of service-oriented architecture,



including the ability to consider integrations with other applications and services.
- Strong application debugging skills.

Nice to Have

- Knowledge of web backend technologies such as Django, Python, and PostgreSQL.
- Experience with Apache Airflow.
- Experience with Atlassian Jira.
- Understanding of Financial Markets Asset Classes, including:
- FX
- FI
- Equities
- Rates
- Commodities
- Credit

- Understanding of various trade types, including:

- OTC
- Exchange Traded
- Spot
- Forward
- Swap
- Options

- Knowledge of Trade Surveillance domain, regulations such as MAR, MIFID, CAT, and Dodd Frank, and related systems is a plus.

Responsibilities

- Develop software applications based on business requirements.
- Maintain software applications and make enhancements according to project specifications.
- Participate in requirement analysis, design, development, testing, and implementation activities.
- Propose new techniques and technologies for software development.
- Perform unit testing and user acceptance testing to evaluate application functionality.
- Ensure assigned development tasks are completed within the defined deadlines.
- Work in compliance with coding standards and best practices.
- Provide assistance to Junior Developers when required.
- Perform code reviews and recommend improvements.
- Review business requirements and recommend changes to develop reliable applications.
- Develop coding documentation and technical specifications for assigned projects.
- Act as the primary contact for development queries and concerns.
- Analyze and resolve development issues accurately.

📌 Senior PySpark Developer (Chennai)
🏢 BaseBiz
📍 Chennai