: Lead Data Engineer (Banking &
- Financial Services)
- Position:
Lead Data Engineer
- Total Experience:
8+ years
- Domain:
Banking, Financial Services, and Insurance (BFSI)
- Location:
Chennai -Hybrid
- Employment Type:
Full-time
Key Responsibilities
- Pipeline Architecture &
- Strategy:
Architect, build, and optimize high-throughput, fault-tolerant batch and streaming data pipelines capable of handling mission-critical financial transactions and accounting workflows.
- Technical Leadership &
- Delivery:
Lead a pod of data engineers, conduct rigorous code reviews, enforce design best practices, and ensure timely delivery aligned with banking release cycles.
- Data Quality &
- Reconciliation:
Implement automated data validation, integrity checks, and ledger reconciliation frameworks to adhere to strict banking accuracy standards.
- ETL Frameworks &
- Optimization:
Build reusable, modular ETL/ELT frameworks; tune distributed cluster compute, query execution plans, and storage layout to meet aggressive SLAs.
- Stakeholder Management:
Work closely with delivery heads, enterprise architects, product owners,
and risk/compliance officers to translate banking business requirements into scalable technical implementations.
Mandatory Skills &
- Qualifications
- Experience:
Minimum
8+ years of hands-on data engineering experience, including at least 2 years in a technical leadership or module lead capacity.
- Databricks:
Deep expertise in Databricks Lakehouse architecture, Delta Lake features (Time Travel, ACID transactions, vacuuming, compaction), and Unity Catalog governance.
- PySpark &
- Distributed Computing:
Advanced proficiency with Apache Spark / PySpark (dataframes, RDDs, partitioning strategies, memory management, shuffle tuning).
- Python &
- SQL:
Strong software engineering fundamentals in
Python
(OOP, error handling, unit testing) and expert-level
SQL
(window functions, query tuning, complex aggregations).
- ETL &
- Data Integration:
Proven track record designing enterprise-scale ETL pipelines integrating diverse legacy systems, core banking databases, and contemporary data warehouses.
- BFSI Domain Knowledge:
Solid familiarity with banking data models (e.g., core transactions, cards, loans, AML, or risk reporting).
📌 Data Team Lead (Chennai)
🏢 Recro
📍 Chennai