- Deep practical experience integrating and deploying LLMs in production RAG retrieval embeddings pipelines . Hands-on experience with agent orchestration frameworks LangGraph LangChain or custom agent runtimes and stateful workflow design .
- Proven track record building observability cost controls and policy enforcement for production services Preferred differentiators . .
Role summary Youll design and own core components that enable enterprise customers to run production agentic AI safely and efficiently on TrueFoundry This includes building robust orchestration for multi-step agents graphstateful workflows modelrouting logic observability and policy enforcement cost data residency rate limiting and integrating upstream tooling like LangGraph LangChain vector stores and specialized LLM runtimes What youll do
****- Architect and implement scalable agent orchestration patterns graph-based executors state management multi-agent coordination for production workloads
- Own critical integrations: model adapters LLM gateway hooks vector DBs tools &; external APIs and the platforms LLMops flows
- Build and improve tracing benchmarking and observability for LLMs and agents tokencost accounting latency p95 throughput and correctness checks
- Drive design for safetyguardrails: moderation hooks human-in-the-loop checkpoints replayable audit trails and policy enforcement
- Mentor junior engineers run design reviews and improve engineering practices testing CICD chaos testing for agents
- Work directly with strategic customers to prototype complex agentic solutions and translate them into product features Must-have **- 4 10 years of software engineering with substantial experience building distributed systems infra or ML platforms
- Deep practical experience integrating and deploying LLMs in production RAG retrieval embeddings pipelines
- Hands-on experience with agent orchestration frameworks LangGraph LangChain or custom agent runtimes and stateful workflow design
- Proven track record building observability cost controls and policy enforcement for production services Preferred differentiators **- Experience building or contributing to open-source LLM orchestration tools LangGraph LangChain or similar
- Familiarity with enterprise constraints: on-premcloud hybrid deployments data residency compliance requirements
- Background in security privacy or model governance for LLMs
- Demonstrated leadership in cross-functional projects and direct customer engagement Qualifications &; signals we like **- Open-source contributions architecture blogs or public talks on agentic LLMs or LLMops
- Examples of productizing research or shipping complex infra features**