21 Nov
|
TALPRO INDIA PRIVATE
|
Mumbai
21 Nov
TALPRO INDIA PRIVATE
Mumbai
Application & AI Operations Engineer
Experience: 3–6 years, including at least 1 year integrating LLM APIs in production
Work Mode: Remote / Hybrid
Role Type: Contract
Contract Duration: Initially 6 months, extendable based on project requirements.
Budget: ₹8–10 LPA, annualised
Only Immediate Joiners
Role Overview
We are seeking an Application & AI Operations Engineer to maintain production web applications, AI-enabled learning products, and self-hosted AI platforms.
You will combine application maintenance and release management with AI quality evaluation, security reviews, provider cost control, and user support, working closely with platform engineers to keep services reliable, secure, and current.
Key Responsibilities
- Maintain applications built with Node.js, TypeScript, React/Next.js, Python, and PostgreSQL. Fix defects, deliver small enhancements, update dependencies and runtimes, and accommodate provider API and model changes.
- Manage releases through testing, type checks, backups, migrations, deployment, smoke testing, change records, and rollback procedures.
- Support workspaces, model routing, user administration, and upgrades on self-hosted AI platforms.
- Complete weekly application health checks, monthly dependency updates, and quarterly security scans. Maintain staging environments and rehearse rollback procedures.
- Maintain application-specific model-provider credentials with spending caps, named owners, rotation dates, and alerts. Review spend weekly, investigate unusual increases, and respond to exposed credentials.
- Review AI interactions monthly for answer quality, hallucinations, prompt injection, abuse, errors, and token usage. Improve prompts, guardrails, and routing, and report cost per feature.
- Maintain evaluation datasets, test prompt and model changes before release, and check for production quality drift. Track model retirements and validate replacements in staging.
- Document provider data-retention, training opt-out, and residency settings. Maintain moderation, personal-data protection, retention controls, and least-privilege tool access.
- Apply secure delivery checks covering secrets, dependencies, input validation, authentication, sessions, and file uploads. Require human review of AI-assisted changes before production.
- Verify weekly database and object-storage backups, maintain structured logs, and troubleshoot database and application performance.
- Provide first-line user support and maintain FAQs and runbooks. Participate in the alternate-week application on-call rota, respond to Severity 1 incidents within 15 minutes while on call, and document incident reviews.
Required Skills and Experience
- 3–6 years building and supporting web applications,
including at least one year integrating LLM APIs in production.
- Solid Node.js/TypeScript and Python skills, including Express or similar frameworks, React or Next.js, and FastAPI or Flask.
- Production model-provider API experience covering streaming, function calling, rate limits, token accounting, cost controls, prompt-injection defences, and evaluations.
- Experience with platform-as-a-service deployments and production secrets management. Replit is preferred; Vercel, Render, or Railway experience is acceptable with willingness to learn Replit.
- PostgreSQL migrations, backups, and query troubleshooting; experience with Redis and background workers such as Celery or BullMQ.
- Knowledge of application security, dependency auditing, secret scanning, authentication, and session handling.
- Experience with Git/GitHub, pull requests, branch protection, CI checks, and Docker for reproducing issues.
- Clear communication with non-technical users, patient support, and ownership of production issues.
Preferred Skills
- pgvector, retrieval pipelines, and administration of platforms such as Dify or LibreChat.
- Voice and avatar integrations such as LiveKit or HeyGen, and video rendering with Remotion.
- AWS services such as S3 and Lambda.
- Awareness of data privacy requirements for learner and candidate information.
Selection process: Shortlisted candidates complete a two-hour practical exercise on a sample environment before interview.
📌 Application & AI Operations Engineer (Mumbai)
🏢 TALPRO INDIA PRIVATE
📍 Mumbai