Can an LLM improve another model by editing its weights?

On November 2, 2026, MIT CSAIL researcher Rikiya Takehi and colleagues reported that GPT-6 Astra had edited Qwen3.5-0.8B's weights and raised its MATH-500 score from 7.9% to 48.1%. Their weight-space programming experiments asked whether a general-purpose LLM could interpret and change the numerical parameters inside other neural networks.

Yes, the reported experiments show an LLM repairing another model through weight edits. Astra used Python and practice feedback to reduce repetitive reasoning loops. But the headline gain starts from a configuration where Qwen rarely finishes an answer. In a separate test, Carlos Toledo reports that unedited Qwen reached 44.0% with thinking disabled and recommended text settings, close to the published edit's 48.1% under a different protocol.

The independent weightspace project also reports finding a different repair with Claude Opus 5.5. Together, the results support automated diagnosis and parameter repair. These reports leave the edited checkpoints untested under Qwen's recommended non-thinking text settings.

What does editing a model's weights mean?

A neural network's weights are numerical parameters used to compute its outputs. Weight-space programming asks an LLM to inspect and change those parameters directly, bypassing the gradient-based updates used in conventional training.

The released Qwen checkpoint describes a tool-assisted process. Astra wrote Python to inspect and modify weight tensors, the arrays storing the parameters. It had practice problems from the GSM8K and MATH training sets and a budget of 50 forward passes. The batches averaged 64 generated sequences and could contain up to 512 problems. Gradient descent and training loops were prohibited. The resulting edit persists in the saved model used for subsequent prompts.

Why did the original framing draw criticism?

The research combines experiments with very different access conditions.

In the small-network game-identification task, Astra received the weights as text, plus descriptions of the network's inputs and actions. It had no tools to execute code. The authors scored all four identifications as correct, although the Freeway policy received a Frogger answer that their stated scoring rule accepted. In the Qwen experiment, Astra had Python and could observe the model's behavior on practice problems.

Conceptual comparison of the game-identification and Qwen experiments. The lower panel shows the 50-pass Qwen run. Held-out benchmark results stay outside the editing process. Source: the authors' experimental setups.

In his critique, spaCy creator Matthew Honnibal argues that the summary makes these separate results sound like a single ability to read and repair a large model directly from its raw numbers. His experiment-by-experiment comparison shows that the language-model experiments use Python, and the Qwen repair also uses feedback from running the model.

Python access lets an agent calculate statistics and test hypotheses that would be unavailable in a raw-text-only task. The original post discloses the tool use. We agree with Honnibal that presenting these results together overstates what the raw-weight experiments establish about larger models.

Sean Goedecke makes a related point in his analysis of the experiments. He describes the Qwen intervention as familiar model-steering work that a competent human researcher could perform. There is no measured human comparison establishing superhuman research ability.

Simon Willison reports running the game-identification prompts with Qwen 3.8 27B on his laptop. It identified Breakout, Pong and Pac-Man, but mistook Freeway for a Pac-Man-like game. That provides independent evidence for interpreting tiny networks, a separate task from repairing Qwen.

What did Astra repair in Qwen?

Qwen3.5-0.8B could spend its output budget repeating reasoning and never deliver an answer. Its own model card warns about thinking loops and states that this model runs in non-thinking mode by default.

The researchers enabled thinking and allowed up to 32,768 generated tokens per answer. Astra traced looping to a direction in the outputs of the feed-forward networks in layers 15 and 19, then edited selected columns of their mlp.down_proj.weight tensors. Toledo's audit describes the operation more precisely: subtracting six times each column's projection onto that direction reverses the component and amplifies it fivefold.

The authors report changing 131,072 weights. Toledo counts 130,188 entries that actually differ in the saved checkpoint, because 884 of the targeted positions round back to their original values.

The edit-and-test process used practice data. Toledo's check of the released logs found 3,680 generated practice answers before the edit strength was fixed, with no MATH-500 problem IDs or texts in those requests.

The published results show fewer unfinished outputs alongside higher scores:

The checkpoint documentation also reports a rerun of the edited model scoring 51.0% on MATH-500 under the post's protocol. This was a re-evaluation of the saved intervention. The separate Opus experiment described below searched for its own edit.

IFEval tests whether a model follows instructions. The edited score of 62.3% also exceeds the unedited model's 44.0% thinking and 52.1% non-thinking scores in Qwen's own card. The score suggests transfer beyond math. Because the harnesses differ, the gain over those model-card baselines still needs a matched evaluation.

On Gemma 3 270M, which rarely looped, Astra found arithmetic and algebra mistakes without one shared failure it could remove. Its edits produced no established math improvement. The authors explicitly distinguish making existing reasoning finish from giving a model an ability it lacked.

How much does the baseline change the result?

The original comparison holds sampling settings fixed between unedited and edited Qwen. It answers whether the intervention helps under that thinking-enabled configuration. A deployment decision also needs a comparison against the unedited model's other operating modes.

Toledo's independent audit uses the post's own prompt and chat template. His control run with the post's settings scored 7.2%, close to the reported 7.9%. Changing to Qwen's recommended thinking-mode text settings, including temperature 1.0 and an 81,920-token cap, raised unedited Qwen to 22.3%. With recommended non-thinking text settings, it reached 44.0%.

Those are his all-sample scores with answers his grader could not resolve counted as wrong. His grader differs from the original evaluation, but the control reproduces its low starting score. His verdict on the headline multiplier is "mostly a settings effect."

Reported results, grouped by evaluation. Compare bars within each study and configuration; prompts, sampling, token budgets and grading differ across groups. Toledo's group tests only the unedited model.

A separate weightspace replication report includes all three checkpoints in each mode:

The project reports that Opus 5.5 found its own intervention using seven forward passes. It amplified four feed-forward neurons, changing 4,096 weights. The thinking-enabled score rose from 8.4% to 40.0%, supporting the premise that another LLM can find a loop repair. The same edit scored 41.8% with thinking disabled, below the unedited model's 46.4% in that test.

The project reports one sample per problem. Its non-thinking run also changes sampling parameters, so comparisons belong within each row. We inspected its published report and evaluation code; we did not rerun the benchmarks. The percentages carry sampling uncertainty and do not establish a general regression from editing.

The replication's non-thinking run uses temperature 0.7, top-p 0.8 and presence penalty 1.5, matching Qwen's recommendation for vision-language tasks. For text, the card recommends temperature 1.0, top-p 1.0 and presence penalty 2.0. Toledo already tested the unedited checkpoint with those text settings. The remaining gap in these reports is an edited-checkpoint run under the same settings, prompt, grading and token budget.

What would count as broader model improvement?

The Astra result and the reported Opus intervention support a specific finding: an LLM agent can diagnose this looping failure and produce a persistent parameter change that improves measured performance in the affected configuration. The instruction-following scores also warrant a matched test of whether the benefit extends beyond recovering math answers.

A broader result would need gains against strong, consistently configured baselines, across tasks and models. It would also need checks for damage outside the targeted behavior. The fraction of parameters touched cannot answer those questions.

Both Astra and Opus edited a different, smaller model. The authors discuss recursive self-improvement as a possible future direction, with repeated self-improvement left untested.

For Qwen3.5-0.8B, start by testing the unedited model with thinking disabled and the card's text settings. The weightspace project provides a pip-installable experimental framework and editing/evaluation scripts for investigating a repair. Configure the intended serving settings explicitly, retain the original checkpoint, and compare edited and unedited models on both the targeted failure and unrelated tasks. A measured advantage there would justify considering the edit for deployment.