Senior Software Engineer (Bengaluru)

Senior Software Engineer (Bengaluru)

15 Nov
|
Starhome Mach
|
Bengaluru

15 Nov

Starhome Mach

Bengaluru

## Job highlights
- Shift Timings: Regular. Work Mode: Hybrid. Industry Type: Telecom. Travel Required: NA
- Any additional job-related instructions or tasks that are requested by the manager must be completed by the employee. . Disclaimer: This job posting has been aggregated from external source
- Experience,that will be an advantage ### Job Summary We are seeking a highly skilled and analytical Senior Data Engineer with deep expertise in designing, building, and maintaining end-to-end distributed data pipelines within the Apache Hadoop ecosystem. In this role,
- You will lead the architecture and implementation of scalable batch and real-time streaming pipelines.
- You will be responsible for ingest, storage, large-scale processing, and serving layers, transforming massive volumes of unstructured, semi-structured, and structured data into high-performance datasets ready for downstream analytics and machine learning applications. ### Job Details
- Job Title: Senior Data Engineer
- Department: RD
- Position Type: Full time
- Job

Location:
Bangalore
- Shift Timings: Regular
- Work Mode: Hybrid
- Industry Type: Telecom
- Travel Required: NA ### Responsibilities
- Pipeline Architecture: Design, build, and optimize scalable end-to-end data pipelines (ETL/ELT) using Hadoop ecosystem components.
- Data Ingestion: Implement high-throughput ingestion pipelines utilizing Apache NiFi or Apache Kafka or Flume to capture streaming event data and transactional batch files.
- Distributed Processing: Develop robust data transformation workflows using Apache Spark (Core, SparkSQL, Structured Streaming) and MapReduce.
- Data Storage Modeling: Optimize data layout in HDFS utilizing columnar formats (Parquet, ORC) and manage NoSQL persistence layers using HBase or Cassandra.
- Query Performance Optimization: Author and tune highly complex distributed queries in Apache Hive to meet tight SLAs.
- Orchestration Workflow Management: Automate and monitor multi-stage pipeline DAGs using orchestration tools like Apache Airflow or Oozie.



### Cluster Performance
- Cluster Performance Tuning: Partner with DevOps/Infrastructure teams to debug issues related to data skew, resource allocation (YARN), cluster memory management, and Spark join strategies. ### Requirements
- Relevant bachelor s degree or master s degree
- 6-8 years of relevant experience in IT industry (Telecom preferred but not mandatory).
- 3 to 7+ years of dedicated data engineering experience, with at least 3 years actively managing end-to-end production pipelines on Hadoop clusters
- Proficiency working in Linux/Unix operating systems, shell scripting, and using code versioning tools like Git. ### Experience, that will be an advantage
- Core Big Data Frameworks: Hands-on mastery of HDFS, MapReduce, YARN, Hive.
- Distributed Compute: Advanced proficiency in Apache Spark using Python (PySpark) or Scala.
- Streaming Ingestion: Deep operational knowledge of Apache Nifi or Kafka (brokers, producers, consumers, and schema registry).
- Programming Languages: Strong foundations in Python, SQL, and Scala or Java.
- NoSQL Database Systems: Direct experience handling unstructured/semi-structured data architectures in HBase, MongoDB, or Cassandra.
- Data Warehouse Modelling: Strong grasp of dimensional data modelling, star/snowflake schemas, and data governance practices. ### Disclaimer The aforementioned

job description
is aimed to provide an overview of the overall type and level of work being performed for this role; it shouldnt be interpreted as a complete list of all obligations, responsibilities, and qualifications needed for the job. In no way does this

job description
imply or say that these are the exclusive responsibilities of the employee holding this position. Any additional job-related instructions or tasks that are requested by the manager must be completed by the employee. Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.

📌 Senior Software Engineer (Bengaluru)
🏢 Starhome Mach
📍 Bengaluru

Reply to this offer

Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.

Subscribe to this job alert:

Get the latest job offers by email for: senior software engineer (bengaluru) / bengaluru