This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Data Engineer - Data Platform based in India.

This is a hands-on data engineering role focused on building the pipelines and infrastructure behind a large-scale B2B data platform. You will transform billions of raw records into clean, reliable, and modelled datasets that power products, analytics, and AI systems. A key part of the role is integrating LLMs directly into production data pipelines for extraction, enrichment, entity resolution, and semantic validation. You will help make probabilistic AI components reliable through measurable evaluations, regression testing, observability, cost controls, and robust fallback mechanisms. Working with modern data technologies such as Python, SQL, dbt, Airflow, Snowflake, and AWS, you will own systems end to end. This is an AI-native, rapid-moving environment with small senior teams, significant autonomy, and a strong emphasis on shipping high-quality engineering work.

Accountabilities:




- LLM-Powered Data Pipelines: Build and operate production systems that use LLMs for extraction, enrichment, entity resolution, and semantic validation across millions of records, with structured outputs, retries, and human-review fallbacks.
- Evaluation & Quality: Develop labelled evaluation datasets, scoring systems, and regression suites that measure LLM performance and gate prompt or model changes using metrics such as precision and recall.
- Rules vs. AI: Determine when deterministic approaches such as SQL, regex, dbt tests, and data contracts are more appropriate than LLM-based decisions, applying sound engineering judgment to balance reliability and flexibility.
- Cost & Model Management: Establish token budgets, route workloads between models based on complexity and cost, and monitor model behavior for drift following vendor or model updates.
- Observability: Ensure production LLM calls and data pipelines

📌 Data Engineer (India)
🏢 Jobgether
📍 India