If you have started using AI in your research or evaluation work, you have probably felt the siren song: output so clean and fast that stopping to question it feels like a burden. The answer isn't to resist AI, but to understand how it reaches its confident conclusions and then to apply human judgement and guidance where it matters most.
Over the past two years, Resonance has been integrating AI tools across our analytical and applied research. Our trials surfaced a pattern. Hallucinations, those confidently stated wrong facts, are relatively easy to catch once you know how to look. The harder problem was that AI tended to lead with conclusions and surface convenient data points that reflected how a question was framed rather than the full weight of the evidence.
Practitioners across the sector are bumping into the same thing. Writing in the Stanford Social Innovation Review in September 2025, Nicholas Andreou, Philipp Essl, and Jeremy Rogers described testing AI against their own analysts across 45 portfolio companies. The AI-generated scores matched their final fully-reviewed assessments 81 percent of the time, versus 64 percent for the human analysts. This reflected genuine efficiency gains, yet the authors noted a critical flaw. The risk, they wrote, is “that the team trusts AI over their own judgment (AI can be very compelling, even when wrong).”
The heart of the problem is not that AI gets things wrong—human researchers do too. The danger of AI is that it gets things wrong so persuasively. Dangerously, it is most persuasive precisely when a reviewer is least likely to push back: after the conclusion has been written down in clean, confident prose. The task in front of every research team right now is figuring out exactly where and when not to trust AI.
Resonance built ARC, our framework for AI Research Collaboration, to require human input at precisely the right junctures. Rather than asking whether an analyst reviews the work, ARC asks where the technical judgment of an experienced researcher is needed. ARC does not wait for a finished product before bringing in human review. It requires engagement throughout the analytical process at the moments where the evidence is still open, and where an expert team can best shape the direction of inquiry. ARC reflects how we've always worked: applying the rigor a qualitative researcher brings to evidence, paired with the analytical frameworks consultants use to turn findings into something actionable for clients. AI gets our team there faster and organizes the data to sharpen analysis.
In the first pass, AI works through every interview transcript and desk review document to sort responses into descriptive themes. That gives analysts a structured evidence base to determine what the data means efficiently. From there, the team can build an analytical framework to map the evidence, align it to the report structure, and answer the client’s questions. This is where human judgment sets direction. AI may organize the evidence, but people decide what it means.
Once the framework is set, AI codes transcripts and documents, organizing preliminary findings so analysts can see exactly how each one connects back to the evidence. With this well-structured evidence base, analysts spend their time discussing and stress-testing conclusions. People make the judgment calls, ultimately deciding what the data is really saying, and how to present it most clearly and honestly. While this sounds obvious, it's surprisingly rare in practice..
ARC helps us deliver real efficiencies and quality improvements, making AI invaluable, while also protecting rigor and quality. For one client, our team cut average preparation time for detailed due diligence reports by more than 40 percent, while reducing the error rate significantly. Speed and accuracy are usually a trade-off, but ARC turns it into a differentiator.
Beyond efficiency, ARC improves the quality of our thinking. When evidence is clean and traceable, our team can engage with the data together in focused, learning-oriented conversations that produce strong insights. At a moment when sustainability strategy increasingly hinges on getting the evidence right, that can translate into stronger, more evidence-driven decisions for our partners.
The organizations that get the most out of AI in research will be the ones that figure out where to leverage their own judgment. That is the question every research team should be asking right now.
Do you have an upcoming research project or evaluation, or are you looking to integrate AI into how your team works?
Get in touch with our team.