About Us

Deccan AI Experts is a pioneering AI company founded by IIT Bombay and IIM Ahmedabad alumni, with a robust founding team from IITs, NITs, and BITS. We specialize in high-quality human-curated data, AI-first operations, and advanced AI evaluation systems. About the Role

We are seeking a Sampling Analyst (Freelancer) to support advanced AI evaluation initiatives focused on quality assurance, audit sampling, reviewer performance analysis, error classification, and continuous quality improvement.

In this role, you will evaluate AI-generated outputs and human review workflows by designing sampling strategies, identifying quality trends, analyzing reviewer errors, and recommending corrective actions. Your expertise will help improve AI evaluation systems by ensuring consistent quality, accurate reporting, and scalable quality assurance processes.

This position is ideal for professionals with experience in quality assurance, audit operations, AI data operations, process excellence, analytics, or quality management. Responsibilities

Create deliverables addressing real-world quality assurance and audit sampling scenarios.

Annotate and evaluate AI-generated quality reports, audit findings, reviewer performance metrics, and quality dashboards.

Assess AI outputs for accuracy, consistency, completeness, and compliance with evaluation guidelines.

Design and evaluate audit sampling methodologies to ensure representative and statistically sound quality reviews.

Develop and maintain error taxonomies by categorizing reviewer errors, annotation issues, policy violations, and quality defects.

Perform root cause analysis (RCA) to identify recurring quality issues, workflow bottlenecks, reviewer inconsistencies, and process gaps.

Track corrective and preventive actions (CAPA),



monitor implementation progress, and evaluate the effectiveness of quality improvement initiatives.

Prepare and review accuracy reports, quality dashboards, reviewer scorecards, and trend analyses to support operational decision-making.

Identify quality risks, sampling biases, recurring error patterns, and reporting inconsistencies in AI-generated outputs.

Provide structured feedback to improve AI performance, reviewer quality, and operational workflows.

Review peer-developed deliverables to maintain quality and consistency standards. Requirements

Bachelor's degree in Business Administration, Statistics, Data Analytics, Computer Science, Operations Management, Engineering, or a related field.

3+ years of hands-on experience in quality assurance, audit operations, AI data operations, business analytics, process excellence, or quality management.

Designing and executing audit sampling methodologies for quality evaluation and operational reviews.

Developing and maintaining error taxonomies to classify defects, reviewer errors, policy violations, and quality issues.

Conducting root cause analysis (RCA) to identify underlying causes of quality failures and operational inefficiencies.

Tracking corrective actions, monitoring CAPA implementation, and validating the effectiveness of quality improvements.

Preparing accuracy reports, quality scorecards, trend analyses,



and operational dashboards using quantitative metrics.

Analyzing reviewer performance, quality trends, and operational KPIs to recommend process improvements.

Proficiency in Microsoft Excel, Google Sheets, SQL, Power BI, Tableau, or similar reporting and analytics tools is preferred.

Strong analytical thinking, problem-solving, and attention to detail.

Excellent written English communication skills.

Ability to critically evaluate AI-generated reports, quality metrics, and operational analyses.

Ability to work independently in a remote, fast-paced environment.

Preferred

Qualifications

Experience working in AI data operations, annotation quality, BPO quality assurance, audit teams, trust & safety, or business process management.

Familiarity with quality frameworks such as Six Sigma, Lean, ISO 9001, CAPA, or operational excellence methodologies.

Experience designing reviewer quality programs, audit dashboards, or performance reporting systems.

Knowledge of statistical quality control, sampling techniques, and quality measurement frameworks.

Experience using AI-assisted quality monitoring, analytics, or workflow management tools.

Professional certifications in Quality Management, Lean Six Sigma, Data Analytics, or Process Excellence are an advantage.

Why Join

Us

Competitive hourly pay: ₹1,500/hour

Fully remote with flexible working hours.

Prospect to contribute to cutting-edge AI evaluation and quality assurance initiatives.

Exposure to advanced AI systems focused on quality intelligence, reviewer performance, and operational excellence.

Flexible project-based opportunities with global teams.

Work on next-generation AI solutions supporting scalable, high-quality human evaluation.

📌 Sampling Analyst (Freelancer) (India)
🏢 Deccan AI Experts
📍 India