**About the Role** The **Senior HPC Engineer** is the **"Foreman" of the "AI Factory"** . While the Domain Architect defines the architectural vision, you are responsible for the hands-on build and deployment. You act as the **Technical Squad Lead** for our offshore engineering teams, bridging the gap between the onshore architectural vision and the hands-on execution. As a System Integrator, we thrive on velocity and precision. You will not just "maintain" clusters; you will lead the **automated deployment** of NVIDIA SuperPOD and BasePOD infrastructure for global enterprise clients. You are the "Lieutenant" to the Domain Architect, translating High-Level Designs (HLDs) into executable Ansible playbooks and ensuring your squad of HPC Engineers delivers defect-free infrastructure. In this role, you are **100% Delivery-Focused** , split between **Technical Leadership (40%)** and **Hands-on Engineering (60%)** . You are the escalation point for complex kernel panics, the guardian of our Infrastructure-as-Code (IaC) repository, and the mentor who unblocks junior engineers when a Slurm job fails to schedule. **Key Responsibilities** 1. **Hands-on Engineering & Automation (60%)** + **Cluster Provisioning Factory:** + Lead the deployment of **NVIDIA Base Command Manager (BCM)** (formerly Bright Cluster Manager) to provision bare-metal DGX/HGX nodes at scale. + Develop and maintain the **Ansible / Terraform** library used to configure OS settings, user authentication (LDAP/AD), and storage mounts across hundreds of nodes. + Execute **HPL (High-Performance Linpack)** and **NCCL-tests** to validate cluster performance, tuning BIOS and OS parameters to hit "Gold Standard" benchmarks. + **Scheduler & Workload Orchestration:** + Configure complex **Slurm Workload Manager** policies, including Fair Share, Preemption, and GPU Partitioning (MIG). + Integrate Kubernetes-based orchestrators (e.g., **NVIDIA Base Command** , **Run:AI** , or **Red Hat OpenShift** ) with the underlying HPC hardware. + **Deep-Dive Troubleshooting:** + Debug "Silent Data Corruption" and "Xid Errors" on GPUs,



analysing nvidia-smi logs and kernel message buffers (dmesg). + Diagnose fabric-related performance drops (e.g., Identifying a specific flapping link causing global slowdowns) in collaboration with the Network Squad. 1. **Squad Leadership & Quality Assurance (40%)** + **Technical Direction (The "Foreman"):** + Translate the **Low-Level Design (LLD)** provided by the Domain Architect into granular Jira tasks for your squad of HPC Engineers. + Conduct daily stand-ups to unblock engineers, clarifying requirements and making technical decisions on the fly (e.g., "Use Ansible roles, not shell scripts for this"). + **Code Quality & Governance:** + Act as the **Primary Gatekeeper** for the code repository. Perform mandatory Code Reviews on all Pull Requests (PRs) to ensure idempotency and error handling. + Enforce "Config-as-Code" discipline, ensuring no manual changes are made to production clusters without a committed playbook. + **Mentorship:** + Guide mid-level and junior engineers on best practices for Linux Systems Administration and HPC environments. **Essential Skills** + **HPC & AI Infrastructure:** + Expert-level knowledge of **NVIDIA Base Command Manager (BCM)** or Metal-as-a-Service (MaaS) provisioning tools. + Deep understanding of **Slurm** configuration (cgroups, plugin development, accounting). + Proficiency with **NVIDIA DGX/HGX** hardware architecture and the associated software stack (Drivers, CUDA, DCGM). + **Linux & Automation (DevOps for Hardware):** + Mastery of **Red Hat Enterprise Linux (RHEL) / Ubuntu** internals (Systemd, Kernel Tuning, Hugepages). + Advanced proficiency in **Ansible** (writing custom modules/roles) and **Python** (automating admin tasks). + Experience with **Git** workflows (Branching, PRs, CI/CD).



+ **Containerisation:** + Hands-on experience with **Docker** , **Singularity/Apptainer** , and **Kubernetes** (specifically NVIDIA GPU Operator and NVIDIA Network Operator). **Desirable Experience** + **Network Awareness:** Ability to troubleshoot basic InfiniBand/RoCEv2 issues (ibstat, perf query) to distinguish between a "Node Issue" and a "Network Issue." + **Storage Integration:** Experience mounting high-performance parallel file systems (VAST/Lustre/WEKA/GPFS) and tuning client-side performance. + **Certifications:** + NVIDIA Certified Associate - AI in the Data Center. + Red Hat Certified Engineer (RHCE). + CKA (Certified Kubernetes Administrator). **Success Metrics (KPIs)** 1. **Deployment Velocity:** Reduction in "Time-to-Hello-World" (time from power-on to running the first successful GPU job) for recent clusters. 2. **Code Quality:** >95% of Pull Requests pass automated linting and require fewer than 2 review cycles before merge. 3. **Stability:** Zero "Configuration Drift" incidents in production (e.g., manual changes breaking the cluster) due to strict IaC enforcement. #LI-RB1 WWT will consider for employment, without regard to disability, a disabled applicant who satisfies the requisite skill, experience, education, and other job-related requirements of the job and is capable of performing the essential requirements of the job with or without reasonable accommodation. World Wide Technology is an Equal Opportunity Employer. Employment decisions are made without regard to race, color, religion, sex (including pregnancy), sexual orientation, gender identity, national origin, age, disability, veteran status, genetic information, or other characteristics protected by law. We are committed to working with and providing reasonable accommodations to individuals with disabilities. If you have a disability and you believe you need a reasonable accommodation in order to search for a job opening or to submit an online application, please call 1-(phone hidden) and ask for Human Resources.

📌 HPC Lead (India)
🏢 World Wide Technology
📍 India