06 Aug
|
Airtel
|
Gurugram
Bharti Airtel Ltd.
Job Title: Lead - Big Data Architect & Analytics
Location: Gurgaon,
Department: Network
Key Responsibilities:
- Must be capable of handling existing or recent Apache HDFS cluster having name node, data node & edge node commissioning & decommissioning.
- Work closely with data architects and analysts to design technical solutions.
- Integrate and ingest data from multiple source systems into big data environments.
- Develop end-to-end data transformations and workflows, ensuring logging and recovery mechanisms.
- Must able to troubleshoot spark job failures.
- Design and implement batch, real-time, and near-real-time data pipelines.
- Optimize Big Data transformations using Apache Spark, Hive, and Tez
- Work with Data Science teams to enhance actionable insights.
- Ensure seamless data integration and transformation across multiple systems.
- 4+ years of mandatory experience with big data.
- 4+ years mandatory experience in Apache Spark.
- Proficiency in Apache Spark, Hive on Tez,
and Hadoop ecosystem components.
- Strong coding skills in Python & Pyspark.
- Experience building reusable components or frameworks using Spark
Expertise data ingestion from multiple sources using APIs, HDFS, and NiFi.
- Solid experience working with structured, unstructured, and semi-structured data formats (Text, JSON, Avro, Parquet, ORC, etc.).
- Experience with UNIX Bash scripting and databases like Postgres, MySQL and Oracle.
- Ability to design, develop, and evolve fault-tolerant distributed systems.
- Strong SQL skills, with expertise in Hive, Impala, Mongo and NoSQL databases.
- Hands-on with Git and CI/CD tools
- Experience with streaming data technologies (Kafka, Spark Streaming, Apache Flink, etc.).
- Proficient with HDFS, or similar data lake technologies
- Excellent problem-solving skills — you will be evaluated through coding rounds
📌 Hadoop Administrator (Gurugram)
🏢 Airtel
📍 Gurugram