We are looking for a Hadoop Platform Administrator to operate and support a large-scale, production Cloudera Data Platform (CDP) workplace for a North American enterprise client. You will be responsible for keeping a highly available, Kerberos-secured big data platform stable, patched, and performant.
Platform overview
- Cloudera Data Platform (CDP)
- 150+ physical servers running RHEL 7.9 on Dell hardware
- Single-namespace HDFS filesystem of ~3 PB with a ~24 TB memory resource pool
- Kerberos (Active Directory) as core authentication; data permissions primarily controlled via AD groups
- High availability / load balancing across nearly all Hadoop services
Key responsibilities
- Support regular Linux patching cycles, including controlled Hadoop shutdown and startup (approx. one hour each)
- Maintain storage balance and service redundancy during routine hardware maintenance (mainly disk replacements)
- Triage system-generated platform alerts: act on genuine issues, and identify and dismiss false alarms or self-recovering conditions
- Support the data governance team with Hadoop data permission requests and access issues
- Perform annual system vulnerability remediation
- Participate in a weekly rotational 24x7 on-call for overnight (North American Eastern Time) data ingestion issues at the platform level only
- Handle miscellaneous short tasks via ServiceNow changes/requests, such as data decommissioning commands, data fix commands, and user login failures