Specialist Software Engineer – Data Engineer (Databricks-py Spark)Responsibilities

- Understand business and technical requirements and translate them into scalable, maintainable data engineering solutions.
- Own the design and development of robust, high-performance data pipelines using Databricks(py Spark).
- Build and optimize data flows within the Hadoop ecosystem (Hive, HDFS, Starburst, Oozie).
- Ensure performance tuning, resource optimization, and productive cluster usage on Databricks platform.
- Contribute to data architecture decisions related to storage, processing, security, and exposure.
- Apply industrialization best practices including CI/CD, unit testing, integration testing, and pipeline validation.
- Model data based on business needs and maintain shared data components, dictionaries, and governance rules.
- Perform functional and technical analysis,



document specifications, and validate implemented solutions.
- Participate in Agile ceremonies and collaborate with Product Owners, architecture, business, and Ops teams.
- Support production processes, troubleshoot issues, update runbooks, and maintain system stability.
- Ensure compliance with data quality, traceability, and security standards across the data lifecycle.
- Engage in continuous improvement activities, proposing upgrades, refactoring, and automation opportunities.
- Work closely with global teams and support cross-location coordination when required.
- Demonstrate ownership during key delivery phases, including planned on-call or deployment activities.

📌 Specialist Software Engineer - Data Engineer Databricks pySpark (India)
🏢 Société Générale
📍 India