Researchers Shuaidong Li and colleagues submitted a preprint on 3 November 2026 proposing Extraction with Hierarchical Optimization (ExtraHerO), a method for extracting system prompts from large language model (LLM) services. The authors report improvements on chatbots and agents, including settings where attackers observe only post-processed outputs, such as agent actions.[1]

The abstract describes two obstacles: longer, structured prompts containing tool schemas, execution logic and safety guardrails; and restricted visibility into raw model responses. ExtraHerO reconstructs prompts hierarchically using a predefined structural template. It updates adversarial queries using differences between feedback and that template, and monitors information gain to stop or reset optimization.[1]

The authors also argue that prompt-fidelity metrics can mislead when they ignore structure. Their approach combines global textual similarity with emphasis on structural keywords, and introduces “adversarial utility” to measure whether extracted prompts help downstream attacks. The abstract reports average improvements in fidelity, functionality and adversarial utility of 12.5%, 2.3% and 6.0% on chatbots, respectively, and 12.2%, 34.3% and 30.5% on agents. These are author-reported experimental results; the saved abstract does not specify the comparison baselines or whether the percentages represent relative changes.[1]

[1]: https://arxiv.org/abs/2611.04803

Sources

  1. Extracting System Prompt of LLM Services in Complex (Agentic) Settings: Hierarchical Optimization and Reliable Evaluation MetricsarXiv · 3 Nov 2026

Check or correct this page

Verification record

The digest below shows whether the text you are reading matches the published source. It does not show that the reporting is correct. The build computes it from this page's Markdown source and lists it in the site manifest.

68fabd90a8192aca7608cc47184eb2ba90ee6e5e6bcedc08ca0df6a7969ef4e1