Mercor is seeking a Bengali Audio Generalist Evaluator Expert to contribute to a high-impact audio AI research project with a leading research lab. In this role, you will work on transcription, annotation, and evaluation tasks that help train and benchmark advanced language models. This is a short-term, structured engagement ideal for candidates with strong academic or analytical backgrounds who are fluent in Bengali and English and enjoy translating complex audio and visual information into precise, well-structured text. Job Responsibilities Transcribe and Optimize Audio & Video Listen to, analyze, and transcribe audio and video content in Bengali , following detailed constraints and instructions. Produce high-quality written outputs in Bengali, with supporting work in English when required. Ensure clarity, accuracy, and strict adherence to formatting and stylistic guidelines. Capture nuances such as tone, intent, and regional language variations where relevant.



Define and Document Evaluation Standards Establish explicit expectations for correct and high-quality responses in general consumer audio contexts. Develop detailed evaluation rubrics and grading guidelines in Bengali and English . Document standards to ensure consistency across reviewers and model evaluations. Identify edge cases, ambiguities, and linguistic complexities specific to Bengali. Conduct Model Testing and Grading Run prompts through language models and assess generated outputs. Evaluate responses against predefined criteria for accuracy, completeness, fluency, and instructional clarity. Provide structured feedback to improve model performance in Bengali audio tasks. Support Benchmarking and Quality Assurance Participate in QA and review cycles to ensure tasks, rubrics, and outputs meet Mercor’s quality bar. Maintain consistency and reliability before datasets are integrated into official benchmarks. Collaborate with project leads to resolve ambiguities and improve task design. Minimum Qualificatio

📌 Bengali Generalist Evaluator Expert (Kolkata)
🏢 Mercor
📍 Kolkata