Lead Data Engineer – Databricks, Python, PySpark & AWS

Job Summary

We are seeking a Senior Data Engineer / SME to join the SSIM Business Architecture vertical in a hands-on Individual Contributor (IC) role.

The role requires strong expertise in Databricks, Python, PySpark, AWS, and SQL to design, build, and support scalable, production-grade data pipelines and cloud data solutions. Snowflake and enterprise data integration experience are desired.

The successful candidate will demonstrate strong technical ownership, problem-solving skills, and engineering best practices across the data ecosystem.

Role location : Bangalore

• Business Architecture: The role sits within the SSIM Business Architecture vertical and requires close collaboration with Architecture, Security, engineering, platform, and business technology stakeholders.
• Hybrid working: The role requires 4 days per week working from the office, in line with the applicable workplace expectations.
• Working hours: The primary working window will be 12:00 PM to 9:00 PM India time,



providing alignment with US stakeholders and teams.
• Flexibility: Given the enterprise and production nature of the role, the individual must be versatile to work outside standard hours and, where required, over weekends to support critical deliveries, production issues, releases,DR or other business needs.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT, data ingestion, and transformation pipelines.
• Build and optimize data processing solutions using Python and PySpark.
• Develop and support production-grade workloads and pipelines on Databricks.
• Design and implement cloud-native data engineering solutions on AWS.
• Work with large-scale structured and unstructured datasets.
• Optimize Spark and Databricks workloads for performance, scalability, reliability, and cost efficiency.
• Develop reusable data engineering components, frameworks, and engineering patterns.
• Implement data quality, validation, r

📌 Lead Data Engineer (Bengaluru)
🏢 State Street
📍 Bengaluru