In a thread posted to X, artificial intelligence (AI) popularizer Matthew Berman has commented on a new arXiv paper about language models that think. Berman, who seems enthusiastic about this paper, has also posted a video to YouTube about it.
Berman explains the central concept of latent reasoning, which means processing information in a hidden space before writing. This is different from chain of thought methods, where models generate words as they think. Latent reasoning helps models work better even without special training data. They can handle tasks with smaller windows of context, and understand ideas that are hard to express in words.
The researchers made a model with 3.5 billion parameters. This model has a part that turns input into a hidden thought space, another part that does the thinking, and a final part that turns thoughts back into words. The model decides how much thinking it needs based on the task's difficulty, similar to how humans think more about hard problems.
This approach doesn't need special data to train, works with less context, and can capture complex reasoning. The model shows patterns in its thinking process, like orbits and sliders, which are ways it organizes thoughts in the hidden space. Performance improves with more thinking iterations, matching larger models. It also allows for tricks like zero-shot adaptive compute, where the model adjusts its thinking without training, and sharing of key-value caches for efficiency. Continuous chain-of-thought means the model keeps a flow of reasoning.
Toward language models that truly reason?
This research could help language models truly reason, addressing critiques like those from Yann LeCun about their inability to reason deeply. While still experimental, it suggests a new path for language models, combining internal reasoning with traditional word generation.
Of course, readers should study the paper carefully before jumping to enthusiastic conclusions. Another thread posted to X by Jonas Geiping, one of the authors of the arXiv paper, can assist.