We are looking for a Lead Software Engineer – Data to design, build, and optimize large-scale data solutions while guiding a team through the full lifecycle of Big Data implementation. This role requires deep technical expertise in distributed computing and up-to-date data processing frameworks, along with strong leadership capabilities to drive projects in an Agile environment.
Responsibilities
Lead the design and implementation of Big Data solutions across the organization
Develop and optimize distributed data processing pipelines using Apache Spark
Write efficient, scalable code in Python to support data engineering initiatives
Build stream-processing systems leveraging technologies such as Apache Storm or Spark-Streaming
Integrate data from multiple sources, including RDBMS, ERP systems, and file-based inputs
Apply ETL techniques and frameworks to ensure reliable data transformation and delivery
Tune and optimize Spark job performance to meet business and technical requirements




Guide the team in adopting native cloud data services on Azure or AWS with Databricks
Manage and mentor a team of engineers to deliver high-quality data solutions efficiently
Drive Agile practices throughout the software development lifecycle
Requirements
8-14 years of experience in Big Data and related data technologies
Expertise in distributed computing principles and Apache Spark
Proficiency in Hadoop v2, MapReduce, HDFS, and Sqoop
Experience with messaging systems such as Kafka or RabbitMQ
Familiarity with Big Data querying tools such as Hive and Impala
Understanding of SQL queries, joins, stored procedures, and relational schemas
Experience with NoSQL databases such as HBase, Cassandra, or MongoDB
Knowledge of ETL techniques and frameworks
Experience with native cloud data services on Azure or AWS with Databricks
Capability to lead a team efficiently
Background in designing and implementing Big Data solutions
Practitioner of AGILE methodology

📌 Lead Software Engineer (Bengaluru)
🏢 EPAM Systems
📍 Bengaluru