IND Staff Engineer, Reliability - GCC070 We're determined to make a difference and are proud to be an insurance company that goes well beyond coverages and policies. Working here means having every opportunity to achieve your goals - and to help others accomplish theirs, too. Join our team as we help shape the future. **Position Summary** **We are seeking a highly skilled T7 AI Operations & Site Reliability Engineer to join our engineering team in Hyderabad, India. This role is laser-focused on the availability, reliability, and performance of our production AI systems. You will own the operational health of AI-powered products - ensuring LLM-based services, agentic workflows, RAG pipelines, and ML inference platforms maintain enterprise-grade uptime while scaling to meet demand. You will build the observability, automation, and incident response capabilities that keep our AI products running 24/7.** **Level: T7 (Senior Engineer)** **Location: Hyderabad, India** **Employment Type: Full time** **Key Responsibilities** **AI Platform Reliability & Uptime** + **Own the end-to-end reliability of production AI systems including LLM services, RAG pipelines, agentic workflows, and inference endpoints** + **Define and maintain SLOs/SLIs/SLAs for AI products - latency, availability, error rates, token throughput, and response quality** + **Design and implement high-availability architectures for AI workloads: multi-region failover, load balancing, auto-scaling, and graceful degradation** + **Build circuit breakers, retry logic, fallback models, and rate-limiting strategies to ensure AI services remain available under stress** + **Drive availability targets of 99.9%+ for critical AI-powered products** + **Establish disaster recovery procedures and regularly test backup/restore for AI data stores and model artifacts** **Observability & Monitoring** + **Build and maintain comprehensive observability stacks for AI systems - metrics, logs, traces, and AI-specific signals (hallucination rates, model drift, token costs)** + **Implement real-time dashboards and alerting for AI service health, model performance,



and infrastructure utilization** + **Design anomaly detection and proactive alerting to identify degradation before users are impacted** + **Monitor LLM provider dependencies (GCP Vertex AI, OpenAI, etc.) and implement automated failover when external services degrade** + **Track and optimize cost-per-inference, token utilization, and resource efficiency across AI workloads** **Incident Management & Response** + **Lead incident response for AI system outages and degradations - triage, mitigate, resolve, and communicate** + **Build and maintain runbooks for common AI failure modes: model timeouts, context window overflows, embedding pipeline failures, vector DB issues** + **Establish on-call rotations and escalation procedures tailored to AI system failure patterns** + **Automate incident detection and remediation where possible - self-healing pipelines and auto-rollback** **AI Infrastructure & Platform Operations** + **Operate and scale cloud-native AI infrastructure (GCP, AWS) including model serving platforms, Kubernetes containers, and vector databases** + **Implement and maintain Infrastructure-as-Code (Terraform) for AI platform environments** + **Automate deployment pipelines for model updates, configuration changes, and infrastructure scaling** **Collaboration & Documentation** + **Partner closely with AI Engineers to ensure new features are built with operability, observability, and reliability in mind** + **Define production-readiness criteria for AI services - ensuring all systems meet reliability standards before launch** + **Maintain comprehensive operational documentation: architecture diagrams, runbooks, playbooks, and SOPs** + **Contribute to architecture reviews with a reliability lens - identifying single points of failure, blast radius, and operational risk** + **Participate in on-call rotations and drive continuous improvement of operational practices** **Required Qualifications** + **Experience:



8+ years of professional experience in software engineering, DevOps, or site reliability engineering, with 1+ year operating AI/ML systems in production** + **Education: Bachelor's degree in Computer Science, Software Engineering, or related field (or equivalent experience)** + **SRE Fundamentals:** + **Deep understanding of SRE principles: SLOs, error budgets, toil reduction, incident management, and capacity planning** + **Proven track record maintaining high availability (99.9%+) for production systems at scale** + **Experience integrating into observability platforms (Prometheus, Grafana, Datadog, Splunk, or equivalent)** + **Strong incident response skills with experience leading war rooms and post-incident reviews** + **AI/ML Operations:** + **Understanding of AI-specific failure modes: model drift, hallucination spikes, token limit errors, embedding pipeline failures, and provider outages** + **Familiarity with LLM providers and platforms (GCP Vertex AI, OpenAI, AWS Bedrock) from an operational perspective** + **Cloud & Infrastructure: Advanced-level experience with cloud platforms (GCP, AWS), Kubernetes, containerization, and Infrastructure-as-Code (Terraform)** + **Programming: Strong proficiency in Python and at least one systems language; comfortable writing automation scripts, custom exporters, and operational tooling** + **Networking & Security: Solid understanding of networking, load balancing, DNS, TLS, and security best practices for cloud-native systems** + **CI/CD: Experience building and maintaining deployment pipelines (Jenkins, GitHub Actions, ArgoCD) with automated rollback capabilities** + **Communication: Excellent communication skills for incident coordination, stakeholder updates, and cross-team collaboration** **Preferred Qualifications** + **Knowledge of AI cost optimization strategies - model routing, caching, batching, and tiered inference** + **Experience in regulated industries (insurance, finance, healthcare) with compliance and audit requirements** + **Cloud certifications (GCP Professional Cloud Architect, AWS Solutions Architect, CKA/CKAD)** + **Experience with AIOps - using AI/ML to improve operational intelligence and automated remediation**

📌 IND Staff Engineer, Reliability (Hyderabad)
🏢 The Hartford
📍 Hyderabad