Big Data Engineer (Chennai)

Big Data Engineer (Chennai)

19 Nov
|
Tech Mahindra
|
Chennai

19 Nov

Tech Mahindra

Chennai

Job highlights

- 3-8 years Hadoop/big data development experience with expertise in Apache Iceberg, Python, Pyspark, SQL, and Airflow
- Design and develop ETL/data pipelines, optimize Spark/Hive jobs, ingest data from multiple sources, ensure data quality and governance, monitor and tune performance, create technical documentation

Job Title: Data Engineering Hadoop Developer (3-8 Years Experience) Core Python, Pyspark, SQL. Mandatory background/Hands on exp on Apache Iceberg

Job Summary

Seeking a Hadoop Developer with 3-8 years of experience in big data engineering to design, build, and optimize scalable data pipelines on Hadoop ecosystem technologies. The role involves batch/stream processing, data integration, performance tuning, and close collaboration with business teams.

Key Responsibilities

1. Design and develop robust ETL/data pipelines using Hadoop ecosystem tools.
2. Build and optimize large-scale data processing jobs using Spark and/or Hive.
3. Develop workflows using Airflow.
4.



Write complex SQL/HiveQL for data transformation and reporting needs.
5. Ingest structured and unstructured data from multiple sources (Kafka, Sqoop, APIs, files).
6. Ensure data quality, lineage, and governance standards are followed.
7. Monitor, troubleshoot, and tune jobs for performance and scalability.
8. Create technical documentation and follow coding best practices.

Required Skills

1. 3-8 years of hands-on experience in Hadoop/big data development.
2. Solid knowledge of HDFS, Hive, Spark,

3. String knowledge in Python/Scala
4. Strong SQL and data modelling fundamentals.
5. Experience with workflow orchestration tools (Airflow).
6. Good understanding of Linux, shell scripting, and distributed systems.
7. Familiarity with version control (Git) and Agile delivery practices.

📌 Big Data Engineer (Chennai)
🏢 Tech Mahindra
📍 Chennai

Reply to this offer

Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.

Subscribe to this job alert:

Get the latest job offers by email for: big data engineer (chennai) / chennai

Subscribe to this job alert:

Get the latest job offers by email for: big data engineer (chennai) / chennai