Experience
- 4+Years
Key Responsibilities
- Redesign LLM Integration: Resolve LLMs rate-limiting issues by implementing request queuing (SQS), SemaphoreSlim concurrency caps, and Polly-based retry patterns with circuit breakers.
- Evaluate Open-Source LLMs: To bypass third-party API limits, you will evaluate and deploy open-source models like Llama 3, Qwen AI or Mistral using vLLM on AWS GPU instances, targeting sub-800ms latency.
- Optimize Aurora MySQL: Address database saturation by implementing RDS Proxy, enforcing reader-endpoint routing for SELECT queries, and utilizing SQL validation middleware for row estimation and write-operation rejection.
- Architect High-Throughput Caching: Implement a Redis-based system (e.g., for session management or balance reservation) using atomic operations to eliminate unnecessary database roundtrips.
- Tune Vector DB Performance: Maximize the hit rate of the Qdrant cache by adjusting similarity thresholds and seeding the cache from MongoDB history to reduce expensive AI calls.
- Advanced Observability:
Set up distributed tracing (OpenTelemetry/AWS X-Ray) and conduct rigorous load tests with k6 at 10,000 virtual users.
Required Technical Experience
- .NET Core Expertise: 5+ years experience with high-concurrency patterns, including async/await, IHostedService background workers, and SignalR.
- LLM Scaling: 2/4+ years of integrating LLMs at scale, focusing on token optimization, rate limit handling, and prompt schema pruning.
- 2/4+ yrs Experience in LLM Inference Engineering, specifically scaling open-source models (Llama 3, Mistral) using vLLM, Text Generation Inference (TGI), or NVIDIA Triton on AWS GPU instances
- AWS Infrastructure: Deep knowledge of Aurora MySQL Serverless v2 ACU tuning, ElastiCache Redis, SQS, and ECS/Fargate.
- Database Mastery: Expert-level skills in MySQL/Aurora EXPLAIN plan analysis, index design, and connection pool tuning.
- Vector & NoSQL Databases: Qualified experience with Qdrant (embedding de
📌 Junior / Senior Ai Engineer (Delhi)
🏢 Delphic
📍 Delhi