We are looking for a Lead Software Engineer – Data to design, build, and optimize large-scale data solutions while guiding a team through the full lifecycle of Big Data implementation. This role requires deep technical expertise in distributed computing and up-to-date data processing frameworks, along with strong leadership capabilities to drive projects in an Agile environment.
Responsibilities
- Lead the design and implementation of Big Data solutions across the organization
- Develop and optimize distributed data processing pipelines using Apache Spark
- Write efficient, scalable code in Python to support data engineering initiatives
- Build stream-processing systems leveraging technologies such as Apache Storm or Spark-Streaming
- Integrate data from multiple sources, including RDBMS, ERP systems, and file-based inputs
- Apply ETL techniques and frameworks to ensure reliable data transformation and delivery
- Tune and optimize Spark job performance to meet business and technical requirements
- Guide the team in adopting native cloud data services on Azure or AWS with Databricks




- Manage and mentor a team of engineers to deliver high-quality data solutions efficiently
- Drive Agile practices throughout the software development lifecycle

Requirements
- 8-14 years of experience in Big Data and related data technologies
- Expertise in distributed computing principles and Apache Spark
- Proficiency in Hadoop v2, MapReduce, HDFS, and Sqoop
- Experience with messaging systems such as Kafka or RabbitMQ
- Familiarity with Big Data querying tools such as Hive and Impala
- Understanding of SQL queries, joins, stored procedures, and relational schemas
- Experience with NoSQL databases such as HBase, Cassandra, or MongoDB
- Knowledge of ETL techniques and frameworks
- Experience with native cloud data services on Azure or AWS with Databricks
- Capability to lead a team efficiently
- Background in designing and implementing Big Data solutions
- Practitioner of AGILE methodology

📌 Lead Software Engineer (Bengaluru)
🏢 EPAM Systems
📍 Bengaluru