LLM Engineer - Manager (Pune)

LLM Engineer - Manager (Pune)

21 Nov
|
PwC
|
Pune

21 Nov

PwC

Pune

Role & responsibilities

Mandatory Skills

1. Advanced Prompt Engineering.

- Experience with structured outputs (JSON mode, function/tool calling) and system prompt design.
- Ability to systematically test, version, and optimize prompts for quality and token efficiency.
- Knowledge of prompt injection risks and mitigation techniques.

2. LLM Architecture Design

- Solid understanding of transformer architecture, attention mechanisms, tokenization, and context windows.
- Ability to design scalable LLM system architectures, including agents, memory, tool use, and orchestration layers.
- Experience making trade-off decisions on model selection, latency, cost, and accuracy.

3. RAG System Implementation

- Hands-on experience building production RAG systems end-to-end.
- Proficiency with vector databases such as Pinecone, Weaviate, FAISS, Chroma, Milvus, Qdrant, or pgvector.
- Knowledge of embedding models, chunking strategies, hybrid search (semantic + keyword), re-ranking, and query transformation.
- Experience evaluating RAG pipelines using tools like RAGAS, TruLens, or custom metrics.

4. Fine-tuning Strategy

- Ability to determine when fine-tuning is the right approach compared with prompting or RAG.
- Hands-on experience with PEFT techniques (LoRA, QLoRA), instruction tuning, and alignment methods (RLHF, DPO).
- Experience with dataset curation, cleaning, and preparation for fine-tuning.




- Familiarity with Hugging Face Transformers, PEFT, TRL, Axolotl, or Unsloth.

5. LangChain / LlamaIndex (Advanced)

- Deep, hands-on experience building complex chains, agents, and multi-step workflows.
- Proficiency with LangGraph, LCEL, custom tools, retrievers, memory modules, and callbacks.
- Experience with LlamaIndex indexing strategies, query engines, and data connectors.
- Ability to extend frameworks with custom components when needed.

6. Multi-model Integration

- Experience integrating and orchestrating multiple LLM providers and open-source models.
- Ability to implement model routing, fallback mechanisms, load balancing, and cost optimization.
- Exposure to multimodal models (text, image, audio, vision-language).
- Familiarity with gateways and abstraction tools such as LiteLLM, OpenRouter, or similar.

7. Production Deployment

- Proven experience deploying LLM applications to production at scale.
- Proficiency with model serving frameworks such as vLLM, TGI, Ollama, Triton, or TensorRT-LLM.
- Strong knowledge of Docker, Kubernetes, CI/CD, and cloud platforms (AWS, Azure, or GCP), including services like Bedrock, Azure OpenAI, or Vertex AI.
- Experience with LLMOps and observability tools such as LangSmith, MLflow, Weights & Biases, Langfuse, or Arize.
- Understanding of caching, batching, quantization, rate limiting, and latency optimization.

📌 LLM Engineer - Manager (Pune)
🏢 PwC
📍 Pune

Reply to this offer

Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.

Subscribe to this job alert:

Get the latest job offers by email for: llm engineer - manager (pune) / pune