Lenny Aharon, Neta Glazer and Lior Aharon submitted Audit Before You Merge: Provenance, Probing, and Continual LoRA Composition to arXiv on 7 November 2026. The authors propose auditing public low-rank adaptation (LoRA) adapters before studying how merging them affects a model’s existing capabilities. Their abstract reports silent base-version mismatches and near-zero adapter update norms, whose inclusion inflates measured interference by roughly 10 percentage points.[1]

The study treats each adapter merge as a sequential learning task, with the instruction-tuned base model’s capabilities as Task 0. Its proposed audit checks that the adapter matches the base model and that its parameter update has a non-trivial norm. The authors also propose a probing protocol they describe as leakage-free.[1]

On an audited set of four adapters for Llama-3.1-8B-Instruct, the authors report that naive merging reduces GSM8K performance by 10.5 percentage points and HumanEval performance by 18.3 points. Task Arithmetic with a coefficient of 0.5 falls within one standard error of the unmerged baseline. A coordinate search selects a coefficient of −0.5 for the math adapter as probe-optimal. On Mistral-7B-Instruct-v0.3, two of four candidate adapters fail the audit; the audited merge yields the same ordering of merge methods on GSM8K.[1]

Sources

  1. Audit Before You Merge: Provenance, Probing, and Continual LoRA CompositionarXiv · 7 Nov 2026

Check or correct this page

Verification record

The digest below shows whether the text you are reading matches the published source. It does not show that the reporting is correct. The build computes it from this page's Markdown source and lists it in the site manifest.

92ba1fbadfbab08f9f17ee555d2c1ea48238687d0b04e18fa37d536f4446a4b8