Behavioral Success Without Internalization

Using controlled scientific-result triage tasks, we separate whether models notice expectation-violating findings, judge their scientific consequence, assess verification needs, and choose subsequent research actions. We find that additional reasoning can substantially improve consequence judgements without comparably changing action, and that internally represented consequence states can causally control the model’s expressed judgement while having little effect on downstream policy.

Training further reveals a distinction between behavioral improvement and internalization: ordinary supervised fine-tuning can substantially improve both judgement and action accuracy without creating a strong causal judgement-to-policy dependency, while counterfactual training can teach models to condition on explicit judgement representations without making their own endogenous judgement behaviorally binding. The work motivates a stronger operational notion of internalization based on causal influence over downstream policy.</p>