AI / LLM Systems Engineer (Consultant/Freelance) (Bengaluru)

AI / LLM Systems Engineer (Consultant/Freelance) (Bengaluru)

15 Nov
|
Covalense Digital
|
Bengaluru

15 Nov

Covalense Digital

Bengaluru

## Job highlights
- Experience with LLM inference frameworks like vLLM, model quantization, LoRA/QLoRA fine-tuning, and benchmarking
- Set up and optimize LLM inference infrastructure, develop fine-tuning pipelines, benchmark and optimize models, and build evaluation frameworks

Key Responsibilities

- Set up and optimize LLM inference infrastructure, including migrating from Ollama to vLLM
- Benchmark and optimize open-weight models such as Gemma, Qwen, Llama, Mistral, etc.
- Optimize GPU utilization, latency, throughput, memory usage, and inference cost
- Develop and experiment with model quantization and calibration techniques to minimize quality degradation
- Build fine-tuning pipelines using techniques such as LoRA/QLoRA for specific use cases
- Establish automated LLM and RAG evaluation frameworks
- Benchmark models across quality, latency, memory, and throughput
- Profile and optimize workloads on NVIDIA RTX 4090 / RTX 5090 and future GPU infrastructure
- Evaluate emerging open-source models, inference frameworks,



and optimization techniques
- Build reusable foundations for model serving, evaluation, experimentation, and deployment What We're Looking For
- Strong Python and Linux engineering skills
- Hands-on experience with LLMs and Hugging Face Transformers
- Experience with vLLM or other LLM inference frameworks
- Solid understanding of LLM inference, GPU memory, attention, KV cache, batching, etc.
- Experience with quantization, model optimization, or compression
- Experience with LoRA/QLoRA/fine-tuning
- Experience designing LLM evaluation and benchmarking systems
- Strong debugging, profiling, and performance optimization mindset Good to Have
- CUDA / Triton / FlashAttention
- GPTQ / AWQ / FP8 / INT8 / INT4
- SGLang / TensorRT-LLM / llama.cpp
- Distributed or multi-GPU inference
- RAG evaluation and retrieval optimization
- Experience working with production LLM infrastructure

📌 AI / LLM Systems Engineer (Consultant/Freelance) (Bengaluru)
🏢 Covalense Digital
📍 Bengaluru

Reply to this offer

Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.

Subscribe to this job alert:

Get the latest job offers by email for: ai / llm systems engineer (consultant/freelance) (bengaluru) / bengaluru

Subscribe to this job alert:

Get the latest job offers by email for: ai / llm systems engineer (consultant/freelance) (bengaluru) / bengaluru