16 Nov
|
Covalense Digital
|
Bengaluru
16 Nov
Covalense Digital
Bengaluru
Job highlights
Experience with LLM inference frameworks like vLLM, model quantization, LoRA/QLoRA fine-tuning, and benchmarking
Set up and optimize LLM inference infrastructure, develop fine-tuning pipelines, benchmark and optimize models, and build evaluation frameworks
Key Responsibilities
Set up and optimize LLM inference infrastructure, including migrating from Ollama to vLLM
Benchmark and optimize open-weight models such as Gemma, Qwen, Llama, Mistral, etc.
Optimize GPU utilization, latency, throughput, memory usage, and inference cost
Develop and experiment with model quantization and calibration techniques to minimize quality degradation
Build fine-tuning pipelines using techniques such as LoRA/QLoRA for specific use cases
Establish automated LLM and RAG evaluation frameworks
Benchmark models across quality, latency, memory, and throughput
Profile and optimize workloads on NVIDIA RTX 4090 / RTX 5090 and future GPU infrastructure
Evaluate emerging open-source models, inference frameworks, and optimization techniques
Build reusable foundations for model serving, evaluation, experimentation, and deployment What We're Looking For
Solid Python and Linux engineering skills
Hands-on experience with LLMs and Hugging Face Transformers
Experience with vLLM or other LLM inference frameworks
Solid understanding of LLM inference, GPU memory, attention, KV cache, batching, etc.
Experience with quantization, model optimization, or compression
Experience with LoRA/QLoRA/fine-tuning
Experience designing LLM evaluation and benchmarking systems
Strong debugging, profiling, and performance optimization mindset Valuable to Have
CUDA / Triton / FlashAttention
GPTQ / AWQ / FP8 / INT8 / INT4
SGLang / TensorRT-LLM / llama.cpp
Distributed or multi-GPU inference
RAG evaluation and retrieval optimization
Experience working with production LLM infrastructure
📌 Ai / Llm Systems Engineer Consultant/freelance Bengaluru
🏢 Covalense Digital
📍 Bengaluru