Responsibilities

Specialist Software Engineer – Data Engineer (Databricks-pySaprk)
Responsibilities
- Understand business and technical requirements and translate them into scalable, maintainable data engineering solutions.
- Own the design and development of robust, high‑performance data pipelines using Databricks(pySpark) .
- Build and optimize data flows within the Hadoop ecosystem (Hive, HDFS, Starburst, Oozie).
- Ensure performance tuning, resource optimization, and efficient cluster usage on Databricks platforms.
- Contribute to data architecture decisions related to storage, processing, security, and exposure.
- Apply industrialization best practices including CI/CD, unit testing, integration testing, and pipeline validation.
- Model data based on business needs and maintain shared data components, dictionaries, and governance rules.
- Perform functional and technical analysis, document specifications, and validate implemented solutions.
- Participate in Agile ceremonies and collaborate with Product Owners, architecture, business, and Ops teams.




- Support production processes, troubleshoot issues, update runbooks, and maintain system stability.
- Ensure compliance with data quality, traceability, and security standards across the data lifecycle.
- Engage in continuous improvement activities, proposing upgrades, refactoring, and automation opportunities.
- Work closely with global teams and support cross‑location coordination when required.
- Demonstrate ownership during key delivery phases, including planned on‑call or deployment activities.
Profile required

Profile Required

Experienced Data Engineer with robust expertise in Databricks , Hadoop ecosystem components, and large‑scale data processing environments.

Minimum Qualifications
- Bachelor of Engineering in Computer Science, Information Technology and equivalent streams.
- Strong hands‑on experience in pySpark for distributed processing and performance optimization.
- Solid understanding of Hadoop ec

📌 Specialist Software Engineer (Bengaluru)
🏢 Société Générale
📍 Bengaluru