The AI DevOps Engineer will partner with AI data scientists, architects, AI solutions engineers, and platform engineering teams to deliver production-grade AI services while ensuring security, scalability, governance, observability, and operational excellence. The role also contributes to AI solution development, so that operational practices are shaped by real delivery experience. This role reports to the Senior AI Solutions Architect.

Key Responsibilities

LLMOps Generative AI DevOps

- Design and operate LLMOps pipelines for prompt management, evaluation, versioning, release, and rollback of LLM-based applications.

- Run and scale LLM gateways and model-serving layers across providers such as GPT, Claude, Gemini, Llama, and Mistral, including routing, rate limiting, caching, and failover.

- Operate Retrieval-Augmented Generation (RAG) platforms, including vector database management, embedding refresh, and index lifecycle.

- Build automated evaluation frameworks to measure quality, groundedness, hallucination rates, safety, latency, and cost per request.

- Support deployment and monitoring of AI agents and agentic workflows, including MCP and A2A integrations.

- Track token usage and spend, and optimize model selection, context size, and inference configuration for cost and performance.

MLOps Model Lifecycle Management

- Design and implement end-to-end MLOps pipelines for model training, validation, registration, deployment, monitoring, and retirement.

- Automate model training, testing, and promotion across development, staging, and production environments.

- Manage experiment tracking, model registries, feature pipelines, and reproducible environments.

- Monitor model performance, data drift, concept drift, and operational metrics, and trigger retraining or remediation.

DevOps, Cloud Platform Engineering





- Build and maintain CI/CD pipelines for AI applications, models, and infrastructure using Git-based workflows.

- Provision and manage cloud infrastructure on AWS (preferred), Azure, or Google Cloud using Infrastructure as Code such as Terraform, CloudFormation, or Bicep.

- Containerize and orchestrate AI workloads with Docker and Kubernetes, including GPU scheduling, autoscaling, and serverless options.

- Implement observability for AI platforms: metrics, logs, traces, dashboards, alerting, and on-call runbooks.

- Drive reliability practices, including incident response, root cause analysis, capacity planning, and disaster recovery.

- Manage AI platform cost through tagging, budgets, right-sizing, and FinOps reporting.

AI Solutions Engineering

- Collaborate with data scientists and solution teams to productionize machine learning, Generative AI, and RAG solutions.

- Develop Python-based APIs, microservices, and integrations that connect AI services with enterprise systems and data platforms.

- Contribute to prompt engineering, fine-tuning, and model customization efforts where operational input improves outcomes.

- Provide reusable templates, SDKs, and reference architectures that help teams adopt AI platform standards quickly.

AI Governance, Security Compliance

- Ensure compliance with Responsible AI, security, privacy, and regulatory requirements across the AI lifecycle.

- Implement secure design and operation: secrets management, identity and access controls, network isolation, and data protection.

- Put guardrails in place, such as content filtering, prompt-injection defenses, PII handling, and audit logging.

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.

📌 AI DevOps Engineer (Bengaluru)
🏢 Replicon
📍 Bengaluru