We are seeking a versatile engineer to bridge the gap between creative prompt design and robust system
orchestration. You will own the design, evaluation, and production operation of our foundation model-based
systems. As an LLM Ops Specialist, you will be responsible for building the infrastructure that makes these systems
scalable, optimized and observable.
Core Responsibilities
System Architecture amp; Orchestration:
Design end-to-end compound AI systems integrating multiple models, retrieval systems, and agentic
workflows.
Implement Retrieval-Augmented Generation (RAG) pipelines with optimized chunking, vector
embeddings, and re-ranking strategies.
Prompt Engineering amp; Versioning:
Craft structured, few-shot, and chain-of-thought prompts to reduce hallucinations and ensure predictable
outputs.
Develop and manage a Prompt Registry to track versioning and A/B test results.
LLM Operations (LLM Ops)
Observability: Instrument real-time monitoring for LLM-specific metrics, such as:
Time to First Token (TTFT)
Token throughput
Drift detection
Cost Control: Implement semantic caching and intelligent model routing to reduce API spend.
CI/CD for AI: Build evaluation suites and methodologies to test for regressions on every deployment.
Core Tech Skills
Python Proficiency
Hands-on with Programmatic Prompting Frameworks
Experience with Retrieval-Augmented Generation (RAG) Pipelines
Experience with Structured Data Control and Integrations into other Software Systems
Mandatory Soft Skills
Excellent Written and Spoken Communication Skills, to the Degree of Clinical Precision
Excellent Tech-to-Business Translations
Analytical Thinking pertaining to Problem Decomposition and Adversarial Thinking
Cooperative Agility - Must have ability and willingness for Cross-Team Functionality and Faster Feedback
Integration Loop
Constant Experimentation Mindset and High Degree of Learning Agility