Data Software Engineer – Spark, Python, Databricks (L2 / L4)


Experience:

- L2: 4–5 Years

- L4: 8–12 Years

Mode: FTE (Full -Time Employment)

Job Location: Bangalore

Work Mode: Hybrid

Notice Period:

- L2: Immediate to 15 days

- L4: Immediate to 15 days

Drive Type: F2F

Drive Location: Bangalore

CTC Band:

- L2: Up to 21 LPA

- L4: Up to 42 LPA

Role Overview


We are hiring Data Software Engineers with strong expertise in Apache Spark, Python, and AWS/Azure Databricks. The ideal candidates will have deep Big Data engineering experience, strong distributed systems knowledge, and the ability to work on complex end -to -end data platforms at scale.

Key Responsibilities


Big Data Engineering

- Design and build distributed data processing systems using Spark and Hadoop.

- Develop and optimize Spark applications, ensuring performance and scalability.

- Create and manage ETL/ELT pipelines for large -scale data ingestion and transformation.

Streaming & Event Processing

- Build and manage real -time streaming systems using Spark Streaming or Storm.

- Work with Kafka / RabbitMQ for event -driven ingestion and messaging patterns.

Cloud & Databricks Engineering

- Develop & optimize workloads on AWS Databricks or Azure Databricks.

- Perform cluster management, job scheduling, performance tuning, and automation.

Data Integration & Storage

- Integrate data from diverse sources: RDBMS (Oracle, SQL Server), ERP, file systems.

- Work with query engines like Hive and Impala.

- Experience with NoSQL stores: HBase, Cassandra, MongoDB.

Programming & Scripting

- Solid hands -on coding in Python for data transformations and automations.

- Strong SQL skills for data validation, tuning, and complex queries

Team Leadership (L4)

- Provide technical leadership and mentoring to junior engineers.

- Drive solution design for Big Data platforms end -to -end

Ways of Working

- Work in Agile teams, participate in sprint ceremonies and planning.

- Collaborate with engineering, data science, and product teams.

Required Skills & Expertise (Both L2 & L4)

- Apache Spark – Expert level (core, SQL, streaming)

- Python – Strong hands -on

- Distributed computing fundamentals

- Hadoop ecosystem: Hadoop v2, MapReduce, HDFS, Sqoop

- Streaming systems: Spark Streaming / Storm

- Messaging: Kafka or RabbitMQ

- SQL – Advanced (joins, stored procedures, query optimization)

- NoSQL: HBase, Cassandra, MongoDB

- ETL frameworks & data pipeline design

- Hive / Impala querying

- Performance tuning of Spark jobs

- AWS or Azure Databricks

- Experience working in Agile

Experience & Level Mapping