AI search agents have been marketed as if they can do something close to live investigative work: browse, compare, verify, and synthesize current facts on demand. A new study covered by The Decoder suggests that, for a meaningful slice of today’s performance gains, that story is too generous. The agents often appear to search the web, but they are frequently confirming answers already embedded in their parameters rather than genuinely discovering them online.
The researchers from the Harbin Institute of Technology and Xiaohongshu call the pattern intrinsic knowledge dependence (IKD). In plain terms, the model leans on what it already learned during training, then uses browsing more as a light check than as the main source of truth. That distinction matters because it explains a growing mismatch between benchmark scores and real-world usefulness. On static tasks, a model can look increasingly capable as the same kinds of facts recur across generations and move from “external” to memorized knowledge. On live, time-sensitive questions, that advantage can disappear quickly.
What changed now: IKD dominates and true web research remains elusive
The study’s central claim is not that web-enabled agents never browse. It is that browsing is often not doing the heavy lifting vendors imply. On static benchmarks, especially ones where the target information is stable, the model can succeed by retrieving a memory trace from training and then assembling an answer with only limited web interaction. Over time, those benchmarks can become easier in a self-reinforcing way: the knowledge required to answer them migrates into the model’s parameters and stops functioning as a real test of live search.
That dynamic becomes much clearer in LiveBrowseComp, the study’s more time-sensitive evaluation. LiveBrowseComp uses time-bound questions, forcing models to work with fresher facts and to gather them under tighter constraints. That setup exposes fragility that static suites can hide. The Decoder’s coverage of the study emphasized this split: strong scores on benchmark browsing do not necessarily imply robust research behavior when the question depends on current information.
For product teams, that is the first uncomfortable implication. A model may look as if it can verify a fact because it can produce a fluent answer with a few supporting links. But if the answer is primarily drawn from intrinsic knowledge, the system is not really doing verification in the operational sense most buyers expect.
How IKD works in practice inside contemporary models
IKD is not mystical. It is a byproduct of how large language models are trained and evaluated.
During pretraining, models absorb a broad factual substrate from text. Some of that substrate becomes robust enough that the model can answer certain questions without needing to consult the live web at all. As generations of models improve, benchmark performance can rise partly because the benchmark’s knowledge base is no longer external to the model. It has been internalized.
That helps explain the divergence between static benchmarks and live tasks. A benchmark such as BrowseComp can reward multi-step browsing, but if the underlying facts are already likely to live in the model’s weights, the agent’s browsing behavior may become opportunistic rather than necessary. The system can surface a plausible path through search without relying on search as the decisive source of truth.
In practical terms, that means product claims about “web research” can blur several different capabilities together:
- recalling memorized facts,
- fetching a source that supports a memorized fact,
- and actually discovering a new fact from current web materials.
Those are not equivalent. IKD is the reason they can look equivalent in demos.
Implications for product deployment and risk
The risk is not just that a model answers incorrectly. It is that it can answer with enough confidence, and enough apparent citation support, to pass as verified when it is not.
That creates several operational issues:
- Trust leakage in decision workflows. If teams assume a web agent is doing live verification, they may use it for market monitoring, policy tracking, incident triage, or compliance support when the system is really averaging over stale or prelearned information.
- Regulatory and audit exposure. A workflow that appears to rely on current sources can become hard to defend if the system cannot show provenance for how it arrived at a claim.
- Benchmark mismatch. Purchasing decisions based on static browsing scores may overestimate performance in domains where the answer changes quickly.
The Decoder’s coverage of the Harbin Institute of Technology and Xiaohongshu work is useful precisely because it reframes the issue from a philosophical complaint into a deployment problem. The question is not whether AI search is “good” in the abstract. It is whether a given product can be trusted when the cost of stale facts is real.
A practical testing and mitigation playbook
The paper’s implications point toward a fairly concrete engineering response.
1. Test on live, time-bound tasks
Do not rely on static question sets alone. Build internal evaluations that require:
- current facts,
- multiple source types,
- and completion within a defined time window.
That is the closest operational proxy for whether a system is actually researching the web rather than summarizing its memory.
2. Track provenance explicitly
If an answer depends on browsing, the product should preserve source lineage. That means logging which URLs were visited, which passages informed the final output, and whether the answer changed after retrieval. Citation provenance should not be decorative; it should be auditable.
3. Use retrieval-augmented generation as a mitigation, not a guarantee
Retrieval-augmented generation (RAG) can help reduce IKD-driven failures by forcing the model to ground responses in retrieved documents instead of internal memory alone. But RAG only helps if retrieval is strong, fresh, and checked. Weak retrieval can still leave the model defaulting back to its priors.
4. Add contradiction checks
When a model makes a high-stakes claim, run a second pass that asks for supporting evidence from at least two independent sources. If the model cannot reconcile discrepancies, it should surface uncertainty rather than compressing disagreement into a single fluent answer.
5. Measure refusal and uncertainty behavior
A system that always answers is not necessarily more useful than one that says the evidence is insufficient. For time-sensitive domains, uncertainty is a feature. Track how often the model declines to overclaim when retrieval is thin or sources conflict.
Market positioning: what vendors and teams should do next
The commercial message around web-enabled agents needs to become narrower and more precise. Vendors should distinguish between three different claims:
- the model can browse the web,
- the model can cite the web,
- the model can independently verify current facts on the web.
IKD shows why those promises should not be merged into one.
That does not make web agents worthless. It makes them more specific. They can be strong assistants for exploration, summarization, and source gathering. But if teams need true freshness and verifiable accuracy, they should demand evidence that the system was tested on live tasks, not just legacy browsing benchmarks.
The broader strategic shift is clear: model cards, sales materials, and evaluation suites need to reflect that memory and research are different modes of behavior. If the benchmark rewards intrinsic knowledge dependence, then product roadmaps will keep optimizing for the wrong thing.
For technical buyers, the practical stance is simple. Treat current-generation search agents as tools that may help with research, not as systems that automatically perform research. Then test them like that assumption matters — because with IKD, it does.
