A Guest Column - Building AI for Researchers: Notes from the Field

After several years of building an AI platform for researchers, some things have become clearer, while others remain uncertain. Four questions stand out: i) what pain points should the field focus on now; ii) what design philosophy leads to better tools; iii) how AI changes the 'say-do gap' in research, and iv) how the researcher’s role is evolving.

Early on, many assumed the biggest value of AI in research would be cost reduction. It was an easy story to tell, and the demos looked turned out to be something deeper: leverage. AI gave researchers their time back and amplified what they could do with it, while giving clients access to depth and scale that previous budgets could not buy. The lesson was clear — what looks most marketable is not always what matters most.

1. The Pain Points Have Moved

Three years ago, the central concern was whether AI was accurate enough for client work. Researchers were skeptical, and rightly so. The safe response was to keep human oversight in the loop - treating AI output as a draft rather than a deliverable. That was appropriate for the technology of the time.

Today, things look different. AI systems have improved significantly, and practitioners who tested them rigorously are now comfortable using them in production. AI-driven translation across major Indian languages produces results comparable to professional human translators for analytical purposes. Content analysis of very large studies - once requiring weeks of analyst time - can now surface themes similar to those identified by senior coders in a fraction of the time. Multilingual qualitative analysis is in routine production use across surveys, diary studies, and in-depth interview programmes: no longer a research project, but repeatable methodology.

What still requires human craft is the harder, less compressible work. Emotionally complex topics, group dynamics, and nuanced cultural inference still demand human expertise. Accuracy gains have been real but uneven, and judgment remains essential.

What enabled progress was not just better models but the supporting systems around them - evidence ladders linking claims to source quotes, query-able playgrounds for direct interrogation of data, and side-by-side benchmarking that lets researchers compare AI-generated themes against their own analysis.

The pain points now are structural, not technical. Senior researchers must rethink how to design their week when mechanical tasks are automated. The next generation must be trained differently, since the apprenticeship of coding thousands of transcripts is shrinking. And most importantly, the time AI saves must be redirected into deeper insight, stronger interpretation, and more meaningful work.

2. Designing for Cognition, Not Automation

The design philosophy behind AI tools has not fully caught up with these changes. Early tools were built with an automation mindset: “The researcher does X; let the machine do X.” This approach flattened the work, making tasks faster but not fundamentally changing what researchers could achieve.

A better frame is cognition. Instead of asking “what can we automate?” the more useful question is “which cognitive moves, when augmented by a machine, become newly possible?” Automation tools narrow options to whatever the machine produced. Cognition tools widen them, giving researchers new ways to engage with data.

Consider multilingual analysis. Automation tools translate everything into English, stripping away idioms and cultural texture. Cognition tools preserve the original language alongside translation, allowing analysts to move fluidly between respondent voices and analytical frames. In synthesis, automation tools produce one-shot summaries. Cognition tools create query-able corpora that researchers can interrogate directly, shifting the relationship from “tell me what the data says” to “let me ask the data what I want to know.”

A missing pattern, an unexpected silence, or a segment behaving differently - these are the moves that change a study, and they require human judgment. Tools that support this engagement extend the researcher’s work in ways summarization alone never can.

In India, cognition framing is especially powerful. Researchers already operate in cognitive bilingualism: English for reporting, regional languages for respondents, and methodological theory often translated from Western frameworks. Cognition tools respect this bilingualism, extending a capability the field already has. Automation tools flatten it, discarding richness in the name of efficiency. The choice is not trivial - the same tool can either diminish or amplify the researcher’s ability, depending on its philosophy.

3. The Say-Do Gap, After AI

The “say-do gap” - the difference between what respondents say they do and what they actually do - remains the oldest problem in qualitative research. AI does not close it, but it changes its dimensions.

Traditionally, respondents reconstruct rather than recall. They tell stories that feel coherent, even if not entirely true. In India, this gap is often larger because of aspirational overlays - what people claim to buy versus what they actually buy can differ widely.

AI reshapes the toolkit in two key ways. First, it enables integration of stated and observed data. Researchers can now analyze what respondents say and what they actually do together in a single corpus, asking cross-cutting questions: where do customers say one thing and do another, and what does that divergence reveal? Historically, stated and behavioral findings lived in separate environments, often owned by different teams. AI makes it possible to bring them together.

Second, AI improves methods. Diary studies, where respondents record behaviour in the moment, are now viable at scale thanks to AI-assisted analysis. What was once boutique and expensive has become practical. Similarly, AI-driven probing in surveys and interviews ensures more specific answers. Instead of vague statements like “the product was useful” respondents are asked to describe a specific time they used it, narrowing the gap between aspiration and behaviour. AI platforms are now actively exploring voice-based interviewing as the next step, where the same probing intelligence runs in spoken conversation rather than text — though emotionally complex topics will continue to demand human moderators.

By making classical methodological moves, triangulating across methods, layering stated against observed, and capturing behaviour close to the moment - cheaper and faster, AI has shifted the landscape. The gap will never fully close, but it can now be worked around with far more rigor than was previously affordable. Researchers who adopt these habits early will produce qualitative insights that are more trustworthy and nuanced than what the field could deliver even a few years ago.

4. The Evolving Role of the Researcher

The most important question is not what AI tools can do, but how they reshape the profession. Mechanical layers of research - scheduling, transcription, translation, first-pass coding - are being absorbed by AI. This changes entry-level roles most directly. The traditional apprenticeship of coding thousands of transcripts is shrinking, and the new apprenticeship has not yet been invented. It must be, because the strategic layers - study design, interpretation, framing arguments, and leading client conversations - are precisely the areas AI cannot handle well. These skills must be deliberately cultivated.

At the same time, AI is enabling entirely new methodologies. Multi-market diary studies can now be conducted across many Indian languages, with continuous AI-supported probing. Longitudinal panels, revisiting the same respondents over years, have become affordable for consumer brands. Mixed-mode programs integrating qualitative, quantitative, and behavioral data into one environment are moving from rare custom builds to repeatable practice. Research itself is becoming a persistent, query-able asset, where every study is digitized and searchable. Even synthetic respondents, used carefully for hypothesis generation or pilot testing, are finding a legitimate place in the toolkit. These are not simply faster versions of old studies - they are new studies, asking questions the field could not previously afford to ask.

There is also a quieter shift the industry has not fully absorbed: enterprise clients now bring privacy and compliance expectations around ISO 27001, SOC 2, DPDPA, and GDPR into the contract from day one, and the senior researcher of 2030 will need to be conversant with these frameworks in a way the predecessor was not.

For India, the implications are profound. Researchers here have always worked across more languages, segments, and economic realities than their Western peers. In the past, much of their visible labour was manual coordination. Now, with the mechanical layer absorbed, what remains visible is judgment. And judgment, honed in the complexity of the Indian market, is abundant here. This makes Indian researchers uniquely well positioned to thrive.

Commercially, this means the value of senior interpretive judgment is rising. A senior researcher who can run a hundred-interview multi-market study with rigor, interrogate the corpus live in a client room, defend findings with traceable evidence, and design hybrid studies that combine human and AI moderators, multimodal capture, and mixed methods in a single coherent programme, is doing work with no historical comparison. That work should be priced accordingly.

Research volume is about to expand at a scale the field has not seen before. The researchers who bring the right technology toolkit into 2030 will be operating at a depth and breadth that simply was not available to anyone five years earlier.

The big question is who captures the surplus AI frees up - clients through lower costs, or agencies through richer work. In reality, both benefit. But researchers that reinvest the surplus into research that was previously impossible will build a stronger profession. And the Indian research community, with its multilingual fluency and long experience of complexity, is well placed to lead this transformation.