Researchers have created a new technique that makes large language models (LLMs) better at specific tasks without needing more computing power to adjust them. Large language models are artificial intelligence systems (AI) trained on vast amounts of text data. However, since this training covers many topics, the models often need improvement for focused tasks like solving math problems or writing computer code.
The process of improving a LLM for specific tasks is called fine-tuning. Tianfu Wu, a professor at North Carolina State University, explains that these models are so big that retraining them entirely is not practical. Instead, the goal is to make the smallest changes to boost performance. The new technique, named WeGeFT, marks a big step forward in this area. It builds on an earlier method called LoRA, introduced in 2022, which uses math tools to find a small set of key parameters to adjust for better results.
Better results with less effort
Previous attempts to improve LoRA either needed more computing power or didn’t enhance performance with the same resources. WeGeFT improves on this by adding math tools that identify which parameters the model already knows and which it needs to learn. By focusing more on new parameters, WeGeFT boosts performance over LoRA without adding significant computing demands. Tests showed it works as well as or better than LoRA and its variations across tasks like understanding common sense, doing math, following instructions, generating code, and recognizing images.
Wu believes this is a helpful advance and hopes to use WeGeFT to find parts of the model that cause harmful outputs. The aim is to make AI safer and more aligned with user needs, with more research expected soon. The study, titled “WeGeFT: Weight-Generative Fine-Tuning for Multi-Faceted Efficient Adaptation of Large Models,” will be presented at the International Conference on Machine Learning in Vancouver, in July.. The work was supported by grants from the National Science Foundation and the Army Research Office.