Experience

- 4+Years Key Responsibilities
- Redesign LLM Integration : Resolve LLMs rate-limiting issues by implementing request queuing (SQS) , SemaphoreSlim concurrency caps, and Polly-based retry patterns with circuit breakers.
- Evaluate Open-Source LLMs : To bypass third-party API limits, you will evaluate and deploy open-source models like Llama 3, Qwen AI or Mistral using vLLM on AWS GPU instances , targeting sub-800ms latency.
- Optimize Aurora MySQL : Address database saturation by implementing RDS Proxy , enforcing reader-endpoint routing for SELECT queries, and utilizing SQL validation middleware for row estimation and write-operation rejection.
- Architect High-Throughput Caching : Implement a Redis-based system (e.g., for session management or balance reservation) using atomic operations to eliminate unnecessary database roundtrips.
- Tune Vector DB Performance : Maximize the hit rate of the Qdrant cache by adjusting similarity thresholds and seeding the cache from MongoDB history to reduce expensive AI calls.
- Advanced Observability : Set up distributed tracing (OpenTelemetry/AWS X-Ray) and conduct rigorous load tests with k6 at 10,000 virtual users .

Required Technical Experience

- .NET Core Expertise : 5+ years experience with high-concurrency patterns, including async/await , IHostedService background workers, and SignalR .
- LLM Scaling : 2/4+ years of integrating LLMs at scale, focusing on token optimization , rate limit handling, and prompt schema pruning.
- 2/4+ yrs Experience in LLM Inference Engineering , specifically scaling open-source models (Llama 3, Mistral) using vLLM, Text Generation Inference (TGI),



or NVIDIA Triton on AWS GPU instances
- AWS Infrastructure : Deep knowledge of Aurora MySQL Serverless v2 ACU tuning , ElastiCache Redis, SQS, and ECS/Fargate.
- Database Mastery : Expert-level skills in MySQL/Aurora EXPLAIN plan analysis , index design, and connection pool tuning.
- Vector & NoSQL Databases : Qualified experience with Qdrant (embedding design, HNSW index configuration) and MongoDB async write patterns

Required Troubleshooting & Architecture Skills

- Concurrency Diagnosis: Proven ability to diagnose threadpool exhaustion, connection pool saturation, and lock contention under high loads
- Agentic Systems :Familiarity with advanced AI workflows, including tool calling and RAG (Retrieval-Augmented Generation)

Preferred Qualifications

- Experience with evaluation frameworks for prompt testing and offline benchmarks
- Mentor the team on best practices in distributed systems, AI orchestration, and infra scaling.

Nice‑to‑Have

- Hands-on deployment and benchmarking of open-source LLMs (vLLM, Ollama, llama.cpp) on AWS GPU instances (g5 / g6 family).
- Experience fine-tuning a model for Text-to-SQL on a proprietary schema.
- Experience with open‑source LLMs (LLaMA, Qwen ai, etc) and self‑hosting readiness.
- Knowledge of JavaScript/TypeScript/Python for API/service development.
- Exposure to evaluation frameworks for prompt testing, offline benchmarks, and release guardrails.
- React and SignalR experience for streaming query-result UIs.
- AWS Bedrock or Azure OpenAI as fallback LLM endpoints

Pay: ₹1,000,000.00 - ₹1,500,000.00 per year

Work Location: In person

📌 Junior / Senior Ai Engineer (Delhi)
🏢 Delphic
📍 Delhi