Scientists face a big challenge in staying current with research because millions of papers come out each year. Artificial intelligence (AI) helps by summarizing information quickly, but many AI models invent facts, a problem called hallucination. For example, some models create fake citations, and they often miss papers published after their training data ends.
Researchers at the University of Washington and The Allen Institute for AI developed OpenScholar, an open-source AI model made just for combining and citing recent scientific research. They also built ScholarQABench, the first large test set across multiple fields to check how well AI handles scientific questions. In tests, OpenScholar matched human experts in citing sources accurately, and 16 scientists liked its answers better than expert-written ones about half the time.
How OpenScholar works
To create OpenScholar, researchers trained it on 45 million scientific papers to base answers on real research. They added retrieval-augmented generation, a method that lets the model find and include new sources after training, with proper citations. Early tries with general search tools were not good, as they picked irrelevant papers or non-scientific sources. By focusing on scientific papers and making the system flexible for new research, OpenScholar became more reliable.
For testing, ScholarQABench used 3,000 questions and 250 detailed answers from experts in fields like computer science and physics. OpenScholar beat other top AI models, such as those from OpenAI and Meta, in accuracy, quality, and relevance. When combined with a larger model, it outperformed human answers 70 percent of the time. Scientists are already using it, and follow-up work aims to make responses even more complete by gathering information in steps. This tool shows promise for trustworthy AI in science, where correct facts are crucial.
The researchers have described the methods and results of this study in a paper published in Nature.