Sr. Engineer – AI Data Platform | ML Infrastructure & Evaluation

Location: Viman Nagar, Pune (Hybrid 3 days)

Experience: 7+ years

We are looking for a Senior Engineer to build and operate ML infrastructure focused on model evaluation, regression detection, observability, and production AI monitoring.

Role Overview

You will work on the engineering systems that evaluate AI models, identify model and pipeline regressions before production, monitor model behavior after deployment, and provide measurable signals around accuracy, reliability, drift, and inference cost.

Key Responsibilities

- Build and maintain a three-tier ML evaluation framework covering component-level benchmarking, end-to-end pipeline evaluation, and longitudinal performance analysis.
- Manage versioned test sets and implement regression detection for model and pipeline changes.
- Maintain and improve multimodal AI evaluation and debugging infrastructure.
- Build and operate production model observability using monitoring, tracing, alerting, and performance signals.
- Instrument training and inference pipelines to make model behavior measurable end to end.
- Build and maintain drift detection, false-positive management, confidence scoring, and model-quality monitoring pipelines.
- Implement regression testing that identifies differences between household-specific performance and aggregate benchmarks.
- Build cost monitoring for ML workloads and identify factors affecting inference cost, latency, and model performance.




- Investigate model-quality issues and contribute to root-cause analysis and platform health reporting.
- Work with Computer Vision, AI Platform, and Agentic AI teams to define evaluation and observability requirements for new model generations.
- Contribute to code reviews, automated testing, engineering standards, and ML infrastructure design practices.

What We’re Looking For

- 5+ years of software engineering experience with strong production-quality Python skills.
- Hands-on experience building or operating ML infrastructure, MLOps, model evaluation, monitoring, or observability systems.
- Experience building or operating model evaluation or regression-detection systems in production.
- Strong understanding of the ML lifecycle: training → evaluation → deployment → monitoring.
- Experience with production ML platforms and real operational constraints.
- Hands-on experience with tools such as MLflow, Weights & Biases/Weave, Datadog, or equivalent platforms.
- Robust understanding of automated testing, system design, code reviews, and production software engineering.

Good to Have

- Computer Vision, Video AI, VLM, LLM evaluation, or multimodal AI experience.
- Experience with video or event-based AI pipelines.
- Experience with model monitoring, drift detection, model serving, or inference optimization.
- Experience with IoT, smart-home, connected devices, robotics, or consumer technology products.
- AWS ML experience or AWS Machine Learning certification.

📌 Senior ML Platform Engineer (Pune)
🏢 Peoplefy
📍 Pune