Role & Responsibilities
- Design, implement, and experiment with advanced Foundation Model and LLM architectures, including Transformers, Mixture-of-Experts (MoE), State-Space Models, and diffusion-based models.
- Develop and scale large-scale pre-training and distributed training pipelines across multi-GPU/multi-node environments.
- Work with PyTorch, JAX, DeepSpeed, Megatron-LM, FSDP, or equivalent frameworks.
- Curate, clean, deduplicate, tokenize, and manage large-scale pre-training datasets.
- Monitor training runs and troubleshoot loss spikes, numerical instability, GPU failures, and performance bottlenecks.
- Implement post-training techniques such as SFT, RLHF/RLAIF, DPO, PPO, and GRPO.
- Develop model evaluation and benchmarking frameworks covering reasoning, accuracy, hallucination, safety, and domain adaptability.
- Optimize models for compute efficiency, memory usage, inference latency, and scalability.
- Collaborate with AI infrastructure teams to improve GPU utilization and distributed training performance.
- Apply quantization, compression,



and other optimization techniques for production deployment.
- Conduct root-cause analysis of model failures and continuously improve model quality.

Preferred Candidate Profile
- 4+ years of hands-on experience in AI/ML, Deep Learning, LLM, or Foundation Model development.
- Solid proficiency in PyTorch, JAX, or similar deep-learning frameworks.
- Proven experience training or fine-tuning large-scale LLMs/Foundation Models, preferably 10B+ parameter models.
- Strong understanding of Transformer architecture, attention mechanisms, tokenization, and positional encoding.
- Hands-on experience with distributed multi-GPU/multi-node training and frameworks such as Megatron-LM, DeepSpeed, or PyTorch FSDP.
- Strong Python programming skills with good systems engineering fundamentals.
- Experience with SFT, RLHF/RLAIF, DPO, PPO, GRPO, or preference optimization.
- Experience with large-scale dataset preparation and pre-tr

📌 Foundation Model Builder (Bengaluru)
🏢 Sunovaa Tech
📍 Bengaluru