Senior Infrastructure MLOps Engineer

Remote Role

Responsibilities:

● Manage the core infrastructure with a focus on supporting ML and LLM workloads. This

infrastructure is relied on by a large subscriber base product as well as internal

stakeholders across the company.

● Design, build, and maintain cloud infrastructure components (AWS and Kubernetes)

using a combination of in-house technology and open source software.

● Build and scale model serving and inference infrastructure for both real-time

recommendation models and LLM workloads including model servers (e.G. NVIDIA

Triton), LLM inference engines (e.G. vLLM, TGI, TensorRT-LLM), autoscaling, and

latency/throughput optimization.

● Manage GPU and accelerator infrastructure on Kubernetes scheduling, capacity and

quota management, and cost optimization across on-demand and spot capacity.

● Design, build, and maintain monitoring and observability systems spanning both system

health and model quality (drift, evaluation, and online performance) that enable engineers





and the support team to gain insight and to discover and debug issues.

● Build and operate model evaluation infrastructure: offline and online eval pipelines,

golden/regression datasets, LLM-as-judge harnesses, A/B testing and experimentation,

and feedback/ground-truth loops

● Design, build, and maintain CI/CD systems enabling engineers to create pipelines to test

and deploy their code.

● Establish tools, methods and best practices for other engineers interfacing with the ML

infrastructure. Ensure reliability, security, and scalability of the platform. Promote

Infrastructure as Code.

● Work closely with other engineers to deploy and instrument software systems.

● Drive evaluation, selection, and integration of third-party vendor systems and work

closely with vendors to configure and manage them.

Qualifications:

● 8+ years of software development and infrastructure management experience.

● 4+ years of experience developing and

📌 Hiring: Senior Infrastructure Mlops Engineer (India)
🏢 Prelect Networks
📍 India