19 Nov
|
WOW Softech
|
Gurugram
19 Nov
WOW Softech
Gurugram
Duties & Accountabilities
- Agent engineering: design and build the initial BA, Dev, and QA agents: prompt architecture, context assembly, tool use, and schema-validated JSON outputs, with prompts and skills packs versioned in git and treated as core IP.
- Retrieval & context: build the codebase retrieval layer (syntax-aware chunking with tree-sitter, embeddings in pgvector) and the context-pack system: project glossary, org conventions, constraint register, and exemplar retrieval.
- Evaluation harness: own the golden dataset (2530 expert-verified items) and the harness that runs nightly in CI: per-agent scores against published acceptance bars, trend monitoring, and regression alerts that block release.
- Failure modes: design for the ways agents fail: schema-repair loops, reject-loop re-invocation with reviewer comments injected, retrieval-verified symbols to guard against hallucinated APIs, and honest gap declaration over coverage inflation.
- Confidence & quality: implement and tune the confidence-scoring model that routes low-confidence outputs to senior review, and use every human modification (deltas captured at the gates) as improvement signal.
- Token economics: own cost per story: per-invocation token and cost telemetry, model-mix tuning (fast tier for enrichment and test generation, top tier for code), and budget-aware prompt design.
- AI-native practice: work daily with agentic coding tools; the friction you hit is product input.
Education & Experience
- 5+ years of software engineering experience, including 2.5+ years building LLM-powered applications in real production use, not prototypes or side projects.
- Demonstrable experience with retrieval-augmented generation and context engineering in a production system.
- Experience building or operating systematic evaluation for LLM outputs (golden datasets, regression harnesses, rubric scoring).
- Experience in a startup or small-team environment is a strong plus.
- Bachelor's degree in Computer Science, Engineering, or a related field, or equivalent demonstrable experience.
Technical Skills Required
- Strong Python and production experience with LLM provider APIs (Anthropic Claude family experience valued); structured outputs and JSON-schema validation.
- Prompt and context engineering with measurable iteration, exemplar selection, context budgeting, instruction design for consistency.
- Embeddings and vector search in production (pgvector or similar); code-aware chunking and indexing is a strong plus.
- Evaluation engineering: building harnesses, defining acceptance bars, wiring them into CI, and reading the trends.
- Token and cost optimisation: metering, model-tier routing, and cost-per-outcome thinking.
- Agent orchestration frameworks (LangGraph or similar) are a strong plus, as is SQL fluency for telemetry analysis.
- Practical, hands-on fluency with contemporary agentic coding tools (e.g., Claude Code or equivalent) is required.
📌 AI Engineer: Agents & Evaluation (Gurugram)
🏢 WOW Softech
📍 Gurugram