Role Summary:

We are seeking an experienced Senior Hadoop Administrator to own the health, performance, scalability, and security of our enterprise-scale big data platform built on Apache Hadoop. The ideal candidate has hands-on expertise administering multi-node Apache Hadoop clusters in production, along with deep knowledge of the surrounding ecosystem - HDFS, YARN, Kafka, HBase, Solr, Spark, Airflow, and ZooKeeper. You will be the go-to person for cluster reliability, capacity planning, troubleshooting, and continuous platform improvement, working closely with data engineers, data scientists, and application teams.

Role & responsibilities

Key Responsibilities:

- Manage and maintain enterprise Hadoop clusters, including installation, configuration, upgrades, performance tuning, capacity planning, high availability, and cluster scalability.
- Administer core Hadoop components such as HDFS and YARN, ensuring optimal resource allocation, storage management, scheduler configuration, and system reliability.
- Support and optimize the Hadoop ecosystem, including Apache Kafka, HBase, Spark, Solr, Airflow, and ZooKeeper to ensure seamless data processing, messaging, and workflow orchestration.
- Plan and execute platform upgrades, patch deployments, and configuration changes with minimal service disruption through rolling upgrades and controlled release management.
- Implement and manage security frameworks including Kerberos, LDAP/Active Directory integration, Apache Ranger, encryption, access controls, auditing, and compliance standards.
- Establish monitoring, alerting, and performance management practices using tools such as Prometheus, Grafana, ELK, Nagios, and Zabbix to ensure platform availability and health.
- Drive automation initiatives using Shell scripting, Python, Ansible, Terraform,



and Hadoop APIs to streamline administration, provisioning, and operational processes.
- Ensure operational excellence and service reliability through SLA/SLO management, incident response, root cause analysis, disaster recovery planning, backup strategies, documentation, and on-call support.

Preferred candidate profile

1) 6+ years of hands-on experience administering production Apache Hadoop clusters (no-vendor / pure open-source distributions).

2) Expert-level knowledge of HDFS and YARN internals, including HA setup, troubleshooting, and performance tuning.

3) Strong hands-on experience with Kafka, HBase, Solr, Spark, Airflow, and ZooKeeper in production.

4) Proficiency in Linux/Unix administration (RHEL / CentOS / Ubuntu) -networking, storage, system, performance tuning.

5) Strong scripting skills in Shell and Python; solid experience with at least one config-management tool (Ansible preferred) for automating cluster operations.

6) Hands-on experience implementing Kerberos, Apache Ranger, LDAP/AD integration, and SSL/TLS across Hadoop services.

7) Solid understanding of JVM tuning, garbage collection, and log analysis for distributed services.

8) Experience setting up open-source monitoring stacks (Prometheus, Grafana, ELK) for Hadoop and its ecosystem components.

9) Proven experience in capacity planning, cluster sizing, and performance benchmarking.

Preferred / Good-to-Have Skills:

1) Exposure to up-to-date Data Platform concepts -data lakehouse architectures (Delta Lake, Iceberg, Hudi), data mesh, or data fabric.

2) Exposure to CI/CD pipelines (Jenkins / GitLab CI) for platform and DAG deployments.

3) Experience with Hive, Tez, Presto/Trino, or Druid.

4) Understanding of cost and resource optimization for large-scale on-prem data platforms.

📌 Hadoop and Big Data Administrator (Madhya Pradesh)
🏢 ClearTrail Technologies
📍 Madhya Pradesh