Calibration-Aware Reinforcement Learning for Large Language Models: A Survey of Objectives, Optimization, and Decision-Making

Between the stated objective and the implemented update, group standardization, the treatment of parameter-dependent rewards, and finite-ensemble scoring can change what training optimizes. Between ranking and decision value, a score can keep its ordering while losing the numerical meaning that a cost-derived threshold requires. Exact constructions make each gap concrete, and a method landscape traces representative protocols to primary sources.

We then give a comparator guide; an evaluation contract that separates reporting gains from changes in the answer policy, effects of the implemented update, and operating-point selection; and open questions. The practical conclusion is a comparator rule: post-hoc fitting is the comparator to beat when predictions can remain fixed, and direct supervision when the desired report has a tractable differentiable loss; RL is a distinct intervention only when reasoning, retrieval, answering, abstention, or information acquisition must adapt through outcome feedback.</p>