Beyond the Assistant Turn: User Turn Generation as a Probe of LLM Interaction Awareness

arXiv·medium signal

Standard benchmarks only evaluate the assistant turn — this paper proposes letting models generate under the user role as a probe of whether LLM weights encode awareness of conversational dynamics beyond next-token prediction. If a model's weights encode interaction awareness, generated user turns should reflect coherent follow-up questions and topic shifts. This introduces a new evaluation axis orthogonal to existing benchmarks, testing whether models understand conversation structure rather than just producing correct responses.