Hacker News
Show HN: Duplicate 3 Layers in Devstral-24B — Logical Deduction .22→.76, No Training
Developer replicated David Ng's RYS (Run Your Subgraph) method and found that duplicating layers 12-14 in Devstral-Small-2-24B routes hidden states through the reasoning circuit twice, boosting BBH Logical Deduction from 0.22 to 0.76 (+245%) and GSM8K from 0.48 to 0.64 — with zero training and no weight changes. The same approach applied to Qwen2.5-Coder-32B (layers 7-9) improved reasoning probes by 23%. Discovered on two AMD consumer GPUs in one evening; includes a toolkit for finding circuit boundaries in any transformer model.
Source
↳ Follow the thread