Immediate joiners or candidates with 30-day notice will be preferred

Senior Data Platform Engineer (Python/Spark Platform Engineering)

We are looking for a highly skilled and self-driven Senior Data Platform Engineer with solid expertise in Python, Spark, Big Data Platforms, and Cloud Data Engineering. The ideal candidate should be capable of independently understanding existing data frameworks, enhancing them with new capabilities, and leading modernization initiatives involving cloud-native technologies.

Key Responsibilities :
Design, develop, and maintain scalable data ingestion and transformation frameworks using Python, Spark, and Shell Scripting.
Independently analyze existing frameworks and implement new features and enhancements.
Drive platform modernization through cloud-native technologies, automation, and engineering best practices.
Develop optimized integrations with object storage platforms, databases, and enterprise data systems.
Build and support high-performance data pipelines on large-scale big data platforms.

Required Skills :





10 + Years of experience of hands on experience
Strong hands-on expertise in Python and Apache Spark development and framework design.
Experience building reusable and scalable data ingestion and transformation frameworks.
Deep understanding of Hive, Impala, HDFS, and the Hadoop ecosystem, preferably on Cloudera-based platforms.
Strong knowledge of HDFS internals, data storage architecture, compaction strategies, partition management, resource utilization, and cluster performance optimization.
Proven ability to troubleshoot complex platform, storage, compute, and cluster-level issues, perform root cause analysis, and drive performance improvements.
Experience working with diverse database technologies and object storage platforms, with a focus on optimized connectivity and data access patterns.
Solid understanding of distributed data processing, Spark optimization, and platform engineering best practices.

Preferred Sk

📌 Senior Data Platform Engineer (Bengaluru)
🏢 Genpact
📍 Bengaluru