Bigdata and Hadoop Developer (Bengaluru)

Bigdata and Hadoop Developer (Bengaluru)

17 Nov
|
Talent21
|
Bengaluru

17 Nov

Talent21

Bengaluru

– Big Data SME / Architect

Position: Big Data SME / Architect

Experience: 4–10 Years

Location: Chennai (Sholinganallur) / Bangalore (RMZ Ecoworld)

Work Mode: Hybrid

Budget: Up to 40 LPA

Employment Type: Full-Time

Job Overview

Tech Mahindra is looking for an experienced Big Data SME / Architect with 4–10 years of overall experience and strong hands-on expertise in Big Data Engineering.

The ideal candidate should have extensive experience in designing, developing, and optimising scalable data pipelines using Hadoop ecosystem technologies, including Apache Hive, Spark, PySpark, and Airflow. The role involves handling large-scale data processing, ETL development, workflow orchestration, performance optimisation, and providing technical leadership.

Key Responsibilities

1. Design and develop scalable ETL processes and data pipelines using Hadoop ecosystem technologies.
2. Build, optimise, and troubleshoot large-scale data processing workloads using Apache Spark, PySpark, and Hive.
3. Design, develop, and manage workflow orchestration using Apache Airflow.
4. Write complex SQL and HiveQL queries for data transformation and processing.
5. Handle structured and unstructured data ingestion from Kafka, Sqoop, APIs, and other sources.
6. Ensure data quality, data governance, lineage, and pipeline reliability.
7. Perform performance tuning and optimisation of Spark, Hive, and Hadoop workloads.
8. Troubleshoot complex issues in distributed data-processing environments.
9.



Develop technical documentation and maintain engineering best practices.
10. Provide technical guidance and collaborate with architecture, development, and business teams.

Mandatory Skills
- Hadoop Ecosystem – HDFS, Hive
- Apache Spark and PySpark
- Apache Airflow
- Python and/or Scala
- Advanced SQL / HiveQL
- ETL and Data Pipeline Development
- Data Modelling
- Linux / Unix and Shell Scripting
- Distributed Computing
- Performance Tuning and Optimisation

Good-to-Have Skills
- Apache Kafka
- Sqoop and API-based Data Ingestion
- Git / Version Control
- Agile Methodology
- Data Governance and Data Lineage
- Production Support and Troubleshooting
- Large-scale Data Processing

Experience & Qualification
- 4–10 years of overall IT experience.
- Strong hands-on experience in Big Data Engineering.
- Expertise in Hive, Spark, PySpark, and Airflow.
- Experience designing and optimising enterprise-level data pipelines.
- Strong problem-solving and technical leadership capabilities.

Preferred Candidate Profile

Candidates should have robust hands-on technical expertise in Big Data technologies along with solution design and architecture capabilities. The ideal candidate should be capable of handling complex data engineering challenges, optimising distributed workloads, and guiding technical teams.

Note: Candidates with strong hands-on experience in Hive, Spark, PySpark, and Airflow will be preferred.

📌 Bigdata and Hadoop Developer (Bengaluru)
🏢 Talent21
📍 Bengaluru

Reply to this offer

Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.

Subscribe to this job alert:

Get the latest job offers by email for: bigdata and hadoop developer (bengaluru) / bengaluru

Subscribe to this job alert:

Get the latest job offers by email for: bigdata and hadoop developer (bengaluru) / bengaluru