19 Nov
|
EY
|
Thiruvananthapuram
19 Nov
EY
Thiruvananthapuram
TechOps - CloudOps - AI/ML Platform Operations Engineer: Senior
Job Summary
We are looking for a seasoned and proactive Senior AI/ML Platform Operations Engineer to operate, support, and continuously improve enterprise machine learning and generative AI platforms. The role will focus on MLOps and LLMOps across model deployment, monitoring, API and endpoint operations, model pipelines, prompt and model configuration, observability, and incident resolution. The engineer will act as a technical escalation point for production AI services and work closely with data scientists, ML engineers, application teams, cloud platform teams, security, DevOps, and business stakeholders to deliver reliable, secure, scalable, and cost-effective AI capabilities.
Responsibilities
- Provide day-to-day production and operational support for machine learning, generative AI, and large language model platforms across development, test, staging, and production environments.
- Deploy, configure, validate, and support machine learning models, foundation models, embedding models, rerankers, and associated inference services using controlled MLOps and LLMOps practices.
- Monitor real-time and batch inference workloads, APIs, endpoints, model pipelines, queues, scheduled jobs, and dependent services, and restore failed or degraded services within agreed SLAs.
- Troubleshoot endpoint errors, latency, throttling, token limits, authentication failures, networking issues, dependency conflicts, model-loading errors, memory constraints, and scaling problems.
- Operate model training, evaluation, registration, approval, deployment, rollback, and retirement pipelines with traceability across model and setting versions.
- Support prompt templates, system instructions, model parameters, routing rules, guardrails, content filters, retrieval configuration, tool connections,
and environment-specific configuration.
- Implement and maintain observability for model quality, response latency, throughput, error rate, availability, token consumption, cost, drift, data quality, hallucination risk, safety events, and user feedback.
- Investigate production incidents using application, model, prompt, API gateway, orchestration, cloud service, infrastructure, and security logs, and coordinate service recovery.
- Perform root cause analysis for recurring incidents and implement corrective actions, preventive alerts, automated recovery, runbook improvements, and platform engineering enhancements.
- Develop and maintain Python utilities, automation scripts, health checks, deployment tooling, test harnesses, evaluation workflows, and operational dashboards.
- Support enterprise integrations with OpenAI services, Azure OpenAI where applicable, AWS Bedrock, model gateways, vector stores, API gateways, identity providers, secrets management, and monitoring platforms.
- Ensure secure handling of model endpoints, API keys, service identities, secrets, private networking, customer data, prompts, responses, and model artifacts.
- Support capacity planning, autoscaling, quota management, rate-limit management, GPU or accelerator utilization, token forecasting, and cost optimization for AI workloads.
- Coordinate controlled releases using Git and CI/CD, including testing evidence, model and prompt versioning, approvals, deployment validation, rollback planning,
and post-release monitoring.
- Manage complex incidents, service requests, problems, and changes, and mentor junior engineers in troubleshooting, platform operations, and operational best practices.
- Create and maintain runbooks, SOPs, architecture and dependency records, known-error articles, monitoring standards, escalation paths, and support knowledge bases.
- Responsible for decision-making, optimizing processes, resource management, and overseeing team management as needed for task execution.
- Accountable for allocating personnel, supervising team members, assigning tasks, ensuring that the team has the necessary tools and support to succeed in their roles and optimizing and evaluating their performance to meet organizational goals.
Skills and attributes for success
- Strong hands-on knowledge of MLOps practices covering experiment tracking, model packaging, model registry, validation, promotion, deployment, monitoring, rollback, and lifecycle governance.
- Strong understanding of LLMOps, including foundation model access, prompt and configuration management, evaluation, observability, retrieval-augmented generation, guardrails, endpoint operations, and cost controls.
- Experience operating model APIs and endpoints, including authentication, authorization, throttling, load management, autoscaling, latency troubleshooting, retries, timeouts, and high availability.
- Strong Python skills for platform automation, API integration, log analysis, operational tooling, testing, and incident diagnosis.
Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.
📌 TechOps-CloudOps-AI-ML Platform Operations Sr Engineer (Thiruvananthapuram)
🏢 EY
📍 Thiruvananthapuram