We are looking for a Senior AI Data Engineer / Data Scientist who can turn messy enterprise data into AI-ready, high-quality knowledge assets. You will lead the cleanup, preparation, and enrichment of unstructured content (SharePoint/document repositories) and structured/semi-structured data (data lakes, databases) so our agents, copilots, and RAG systems are accurate, trustworthy, and scalable.

This is a senior, hands-on role. You will own data quality outcomes end-to-end: discovery - cleanup - enrichment - ingestion - refresh cycles - governance. We value AI-native generalists who can remove bottlenecks by working directly with AI Engineers, Architects, and business stakeholders to decide what data is worth using and how to structure it for retrieval and reasoning.

Our standardized stack includes (and this role actively uses it): ingestion/ETL foundations, Postgres + pgvector as default RAG store, Redis caching, LLM gateway patterns, Langfuse observability, DeepEval/RAGAS evaluation, and Presidio for PII detection/masking when required.

KEY RESPONSIBILITIES

- Enterprise data discovery & cleanup
- Lead data triage for AI use cases: identify authoritative sources, duplicates, outdated content, and low-quality documents.
- Clean, normalize, deduplicate, and standardize enterprise content at scale (documents, PDFs, Word/Excel, wiki pages, etc.).
- Define what data should be excluded from AI systems (stale, contradictory, low-trust, or sensitive content).
- Unstructured ingestion (SharePoint + document repositories)
- Build robust ingestion pipelines for SharePoint and file repositories: parsing,



text extraction, structure recovery, and metadata capture.
- Implement document normalization strategies (naming, taxonomy, metadata standards, canonical IDs).
- RAG-ready data engineering (context engineering, not just ETL)
- Design chunking strategies, metadata enrichment, and document structuring optimized for retrieval performance and cost.
- Improve retrieval quality through practical techniques such as filtered retrieval and post-retrieval optimization where appropriate (e.g., reranking), collaborating with AI Engineers on the retrieval interface.
- Prepare and maintain AI-ready knowledge sets that can be embedded and served via Postgres + pgvector (default).
- Data quality, evaluation, and feedback loops (non-negotiable)
- Define and implement data quality gates (freshness, completeness, relevance, dedupe rate, metadata coverage).
- Partner with AI Engineers to evaluate retrieval and RAG performance using frameworks like RAGAS (answer correctness, context recall/precision) and to monitor trust metrics over time.
- Establish human feedback loops where needed (review queues, sampling, targeted audits) to continuously improve data usefulness and user trust.
- Governance, privacy, and auditability




- Ensure traceability: lineage from source processed artifact retrieval corpus production usage.
- Apply privacy and enterprise constraints; where required, implement PII detection/masking using Presidio patterns.
- Reuse
- Package reusable data cleanup + RAG readiness recipes: ingestion templates, metadata schemas, chunking playbooks, dedupe strategies.
- Build a repeatable data foundation that accelerates future use cases (not a one-off cleanup project).

REQUIRED EXPERIENCE (MUST-HAVE)

- 5+ years in data engineering / applied data science / analytics engineering with ownership of production pipelines.
- Strong Python for data processing (parsing, cleaning, normalization, pipelines).
- Proven experience working with unstructured enterprise data (documents, PDFs, Office files, wikis, knowledge bases).
- Proven experience building data pipelines for retrieval/NLP/RAG-like use cases (embedding prep, metadata, corpus management).
- Solid understanding of data quality engineering: validation, monitoring, lineage, refresh cycles.
- Solid stakeholder skill: can work with business to define what data matters and what “good” looks like.

NICE-TO-HAVE

- Experience with Postgres + pgvector (or similar vector stores), retrieval optimization, and hybrid search concepts.
- Familiarity with observability practices for AI pipelines and the use of RAG evaluation metrics (RAGAS-style).
- Experience with governance tooling and privacy controls for enterprise AI (e.g., PII workflows).

📌 Senior Data Engineer (Pune)
🏢 Billennium
📍 Pune