TinyLoRA – Learning to Reason in 13 Parameters (arxiv.org) AI

The paper introduces TinyLoRA, a parameter-efficient adapter method that scales reasoning performance using extremely small low-rank updates (as few as 13 trained parameters). The authors report that training an 8B Qwen2.5 model with TinyLoRA reaches about 91% accuracy on GSM8K and recovers roughly 90% of performance gains on harder reasoning benchmarks while using 1,000× fewer parameters than typical approaches. They also find the strong results depend on reinforcement learning, with supervised fine-tuning requiring much larger updates to match performance.