Ezra Edelman, Benjamin Edelman, Surbhi Goel and Eran Malach submitted Understanding and Mitigating Role Spoofing in LLMs to arXiv on 29 October 2026. The authors report that malicious text in tool outputs, such as a webpage accessed by an agent, can impersonate a user by exploiting a model’s confusion about who supplied the text.[1]
The abstract describes role confusion as attributing text to a speaker based on its style rather than its labeled role. According to the authors, large language models (LLMs) track speakers using both content cues, including semantics and writing style, and structural cues, including control tokens and formatting. They report high susceptibility to attacks combining fake structure with deceptive content.[1]
In a toy setting, the authors say adversarial training against content-based injections reduced those attacks while leaving susceptibility to attacks based on structure. They also propose linear probes trained to predict conversation roles and report effective detection across different settings. The saved abstract supplies no numerical results, named model versions or operational detection thresholds; its findings do not establish a deployed mitigation.[1]
[1]: Understanding and Mitigating Role Spoofing in LLMs. arXiv, 29 October 2026. https://arxiv.org/abs/2610.40396
Lead photo: Ezra Edelman, Benjamin Edelman, Surbhi Goel, Eran Malach · CC BY 4.0.
Sources
Check or correct this page
Verification record
The digest below shows whether the text you are reading matches the published source. It does not show that the reporting is correct. The build computes it from this page's Markdown source and lists it in the site manifest.
374dfd440f5681159d3b3e6494164704a89c3a68b625efd4ae2392c7bcb7c338
