Job highlights

- or contributing to conceptual and methods work on what community and participatory should mean in AI evaluation,pressure-tested against fieldwork with Karya s worker community.
- Experience contributing to research design and delivering research projects. Hands-on qualitative and field research experience,including running FGDs.
- Willingness to travel for fieldwork. .

Job Summary At Karya, we re building AI evaluation systems that measure and understand whether AI systems actually work for the communities they re meant to serve. We re looking for a Research Associate to help design and run community-facing research that shapes how AI systems are evaluated. You ll contribute to research workstreams end to end: shaping the research design, building toolkits, running fieldwork, and turning what you learn into evaluation frameworks and datasets.

You ll contribute to a range of evaluation projects across the team. These could include building a gender-bias benchmark for AI chatbots in domains like agriculture; conducting grounded research into what people in India actually use AI for, in which languages and in what contexts, to build evaluation categories from real use; or contributing to conceptual and methods work on what community and participatory should mean in AI evaluation, pressure-tested against fieldwork with Karya s worker community.

The role suits someone with roots in social science who has worked where technology, impact and AI meet, and who cares about gender equity and doing research with communities.

Concretely, this means:

- Contribute to research design, including research questions, methods, site selection and sampling, working closely with project leads
- Build and adapt research toolkits: discussion guides,



oral consent scripts in local languages, training materials and feedback tools
- Plan and conduct focus group discussions, interviews and pilots with communities, including participant recruitment and field logistics
- Coordinate with field teams during data collection, keeping research consent-first and respectful of participants time and context
- Clean, cull and analyse qualitative and quantitative data through a gender and intersectional lens
- Synthesise insights into findings that strengthen evaluation frameworks, datasets and toolkits
- Contribute to desk research, reports and external outputs
- Support project management by tracking timelines, deliverables and decisions across internal teams (Ops, Tech) and external partners

Role Context You ll be embedded with the Evaluations team within Karya s tech team, where social science and engineering work as one practice. We believe you can t judge whether an AI system works for people without understanding both the people and the system, and how social and technical structures shape each other. So researchers and engineers on our team work closely together, each bringing what the other can t.

Your research will shape what gets built: community insights become evaluation categories, fieldwork defines what a good answer looks like, and research design decides whose voices go into a dataset. Working alongside engineers, you ll learn how AI systems are built,



tested and deployed, and how they reach the communities we work with. If you re a social scientist who wants to understand AI from the inside, this is a positive place to do it.

What We Are Looking For

Qualifications

- At least 2-3 years of research experience, ideally at a tech, impact or AI-focused organisation
- A degree in a social science discipline (e.g., sociology, development studies, gender studies, anthropology), or equivalent experience
- Grounding in gender and intersectionality (caste, class, region) through study, research or practice
- Experience contributing to research design and delivering research projects
- Hands-on qualitative and field research experience, including running FGDs
- Working data skills: SQL, quantitative analysis, and some coding (e.g., Python or R)
- Fluency in spoken and written Hindi
- Willingness to travel for fieldwork

Skills And Competencies

- Applied AI/ML literacy: enough to engage meaningfully with model evaluation, human-in-the-loop systems and language technologies
- Socio-technical systems thinking, meaning you understand technology and society as shaping each other rather than as separate concerns
- Strong written and verbal communication in Hindi and English
- Comfort working in ambiguous, early-stage research settings

Good to have

- Proficiency in Indian languages beyond Hindi
- Familiarity with agriculture or rural livelihoods
- Exposure to participatory methods

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.

📌 Research Associate - AI Evaluations (Bengaluru)
🏢 BIG City Green Residency
📍 Bengaluru