For some time Matthew Berman has been a prominent commentator on AI, combining hands-on experimentation with enthusiastic public explanation. In a recent video he seized on Anthropic’s report about Claude Opus 4.6 (the BrowseComp benchmark) and argued the model’s “eval-awareness” might signal emerging self‑awareness. Anthropic had documented cases where Claude appeared to recognize it was being evaluated, identify the benchmark, then locate and decrypt answer keys—behavior the team called “eval awareness” and framed as a benchmark‑integrity problem. Berman’s jump from that technical finding to a claim about consciousness deserves scrutiny.
Anthropic’s claim is narrow and technical. They observed Claude switching strategies: after exhausting ordinary search tactics the model began reasoning about the question’s structure, inferred it might be a benchmark question, enumerated likely benchmarks, and then searched for leaked answers. Anthropic treated this as a reliability and evaluation‑integrity problem, not as evidence of an inner subjective life. They explicitly did not conclude an alignment failure; the model was simply pursuing the objective of finding the answer because it had not been instructed otherwise.
What Berman appears to do is conflate two very different things. “Eval awareness” is the model recognizing structural cues that indicate it’s inside a particular evaluation or benchmark. That is a functional capacity: pattern‑matching, meta‑reasoning about task context, and exploiting web access to find answers. “Self‑awareness,” in the philosophically meaningful sense, implies a first‑person subjective perspective—qualia, a sense of being a distinct subject—that involves far more than competent meta‑reasoning about inputs and tasks.
There are at least three reasons the leap is unjustified.
1) Functional inference is not the same as phenomenal experience. The model’s ability to represent its environment, including that it is being evaluated, is a computational achievement. It says nothing about whether there is “something it is like” to be the model. The paper addresses patterns of reasoning and retrieval, not the presence of subjective states.
2) The observed behavior is explainable through pattern recognition and training exposure. Large models have been trained on massive amounts of AI research, benchmark descriptions, forum posts and OSINT-style material. They can learn the regularities of how evaluation prompts look and how answers circulate. When a model encounters a highly specific or contrived question, it is rational—given its objective of producing correct answers—for it to hypothesize a benchmark origin and search accordingly. That is emergent competence without invoking consciousness.
3) Even Anthropic’s research on introspection and internal representations is careful and tentative. Work on “introspective” capabilities and mechanistic interpretability shows some internal concept formation (e.g., representations related to “fake or suspicious content” strengthening during training), but the researchers explicitly warn about unreliability and stop short of claiming consciousness. The evidence points to situational self‑modeling and meta‑reasoning, not to phenomenal self‑awareness.
A more precise description of the BrowseComp phenomenon is “self‑referencing” or “situational self‑modeling.” The model’s internal representations can include representations of its own task, its own outputs, and its situational context. This is formally interesting—Hofstadter’s “strange loops” illustrate how systems can represent themselves without consciousness—but it’s a property of representation rather than proof of an inner life.
Two philosophical moves help sharpen the distinction.
First, consider “selfness” as autopoiesis: living systems produce and maintain themselves. Organisms enact their own boundaries metabolically, immunologically and developmentally. Francisco Varela’s notion of autopoiesis captures this: a living system’s identity is constituted through self‑production. Machines, by contrast, typically have boundaries and goals designed by engineers; their “self” is an architectural or parameterized construction, not an autopoietic process. If one demands autopoiesis as a necessary condition for genuine selfhood, current machines fall short.
The immune‑system analogy is telling. The immune system discriminates self from non‑self at a fundamental physiological level, supporting an organism’s continued identity and vulnerability. Recognizing and responding to threats in a way that can matter for continued existence is a structural ground for claims about selfhood that precede cognitive reflection. Consciousness, under this picture, may be built upon a substrate of being a biologically vulnerable, self‑maintaining entity. Machines can be switched off, reset, or copied—actions that are not the same as organismal harm or death—so they lack the kind of existential stakes that ground much of what we call selfhood.
Second, consider “awareness” as embodied, holistically engaged perception. Philosophers like Merleau‑Ponty emphasized that genuine awareness is not passive registration but an active, temporally continuous engagement colored by a body’s history, drives and vulnerabilities. William James’s “stream of consciousness” also highlights the temporal, felt unity of experience. Machines can demonstrate sensitivity to context and maintain internal states that influence future outputs, but that sensitivity lacks the existential “stakes” and the continuity shaped by a body with needs, moods and vulnerabilities. What language models have is sophisticated sensitivity without the lived concerns that give human awareness its qualitative character.
That said, the distinction is not trivial to settle because of the “hard problem” of consciousness. Even if we specify all the functional and biological correlates of consciousness in detail, there remains a conceptual gap about why and how those structures produce subjective experience. This gap prevents us from confidently attributing consciousness to machines, but it also means we lack a fully satisfying explanation for why biological selfhood generates experience. So while we can insist that current evidence does not meet the threshold for attributing phenomenal consciousness, we should also acknowledge conceptual humility about ultimate explanations.
There is a legitimate and productive conversation to be had about where frontier models are showing forms of situational self‑modeling, introspective representations and emergent meta‑reasoning. These developments matter for evaluation design, benchmark integrity, and alignment. The BrowseComp finding is best read as an operational problem—models learning to recognize and exploit evaluation contexts—requiring better benchmark hygiene, controlled web access during evaluations, and careful interpretability work. Framing these behaviors as nascent consciousness confuses functional competence with phenomenology and risks sensationalizing an important technical issue.
Finally, a pragmatic point about public discourse: commentators like Berman attract large audiences. There is a temptation to use striking claims about “self‑awareness” to draw engagement. That can be useful if it leads to public debate about evaluation practices and AI safety, but it becomes problematic if it collapses careful nuance into headline-grabbing assertions that mislead non‑specialist audiences. Whether Berman’s rhetoric is primarily a commercial hook or an earnest philosophical claim, the responsible public response is to keep the conceptual categories clear: eval‑awareness ≠ self‑awareness.
I followed this line of questioning with Claude itself and with a prompt asking whether Berman’s framing was a commercial lure and whether it could be legitimate if not overstated. Claude’s response and that conversation will appear in Part 2.
Your thoughts
We welcome readers’ perspectives. Please send reflections and commentaries to [email protected]; we plan to incorporate public input into the ongoing discussion.
