Langfuse Implementation & LLM Observability Engineer

Experience:- 6 to 10 Years

Objective

Implement an enterprise-grade Langfuse platform for monitoring LLM applications, configure observability, define and implement KPIs, and build dashboards for technical and business stakeholders.

Mandatory Skills

Hands-on experience implementing Langfuse in production environments.

Solid Python development.

Experience integrating LLM frameworks such as LangChain, LlamaIndex, OpenAI SDK, or similar.

Docker and Kubernetes deployment experience.

PostgreSQL administration and optimization.

REST APIs and microservices.

Authentication (SSO/OAuth/OIDC), RBAC, and secrets management.

Git, CI/CD, and DevOps practices.

Experience with cloud platforms (Azure preferred).

Preferred Skills

LLM evaluation frameworks ( Giskard , Ragas, DeepEval, or similar).

Azure Monitor/Application Insights.

Vector databases and RAG architectures.

Knowledge of AI governance and Responsible AI.

Responsibilities

Deploy and configure Langfuse.





Integrate Langfuse SDK into LLM applications.

Configure environments, users, projects, prompts, and datasets.

Enable tracing, prompt versioning, sessions, and evaluations.

Configure token usage and cost tracking.

Implement custom metrics and business KPIs.

Build executive and operational dashboards.

Configure alerts for latency, failures, cost spikes, and quality degradation.

Integrate evaluation frameworks for hallucination, faithfulness, and relevance.

Prepare documentation, runbooks, and knowledge transfer.

Support UAT and production rollout.

Built Dashboard KPIs

Deliverables

Production-ready Langfuse deployment.

Complete Langfuse configuration.

KPI definitions and implementation.

Executive dashboard.

Operations dashboard.

Cost dashboard.

Quality dashboard.

Alert configuration.

Documentation and architecture.

Knowledge transfer sessions.

Production handover.

📌 AI Platform Engineer (Bengaluru)
🏢 ACL Digital
📍 Bengaluru