AUS
NZ
UK
CA
USA
FRA
GER
ESP
ITA
NLD
PRT
SWE
GRC
IND
POL
BRA
ZAF
RUS
CHN
UAE
Traditionally, incorporating new information into AI systems relying on neural networks has posed a significant challenge: training on new tasks often leads to a phenomenon known as catastrophic forgetting, where the model loses knowledge of previously learned tasks. To avoid this, models typically must be retrained from scratch – a time-consuming and usually large, resource-demanding process.
Neurotechnology conducted a large-scale continual learning experiment in which researchers integrated additional linguistic fluency in the Lithuanian language into the full parameter set of the Gemma2 large language model. The experiment simultaneously preserved the model's existing domain knowledge by inhibiting updates of its parameters that are related to the Massive Multitask Language Understanding (MMLU) task.
The results of this experiment reveal that it allowed balancing of the retention of the existing models' capabilities with the acquisition of new ones, in terms of both fluency and domain knowledge. Post-training evaluations demonstrated improved generation quality in Lithuanian while maintaining strong performance across the language understanding benchmarks.
"Continual learning methods have the potential to drive more adaptive and sustainable AI development," said Artūras Nakvosas, Research and Development Team Lead at Neurotechnology. "Our findings demonstrate that it’s possible to mitigate catastrophic forgetting and optimize learning efficiency across multilingual and task-specific domains.”
The benefits of continual learning are wide-ranging. With this approach, large language model developers can potentially expand models' performance, support additional customer languages or adapt to new domains while preserving existing capacities.
The full technical report detailing Neurotechnology’s findings is available on arXiv at https://www.arxiv.org/abs/2505.05946, and the code used in the research is available on GitHub at https://github.com/Neurotechnology/LLM_EWC.
About Neurotechnology
Neurotechnology is a developer of high-precision algorithms and software based on deep neural networks and other AI-related technologies. The company was launched in 1990 in Vilnius, Lithuania, with the key idea of leveraging neural network capabilities for various applications, such as biometric person identification, natural language processing (NLP), computer vision, and artificial intelligence. The company focuses heavily on continuous research and development to advance the capabilities of artificial intelligence technologies. With a team of scientists and engineers, Neurotechnology consistently explores new methods in deep learning, computer vision, natural language processing, and neural network optimization. This ongoing innovation allows the company to deliver cutting-edge solutions across a wide range of applications.
Jennifer Allen Newton
Bluehouse Consulting Group, Inc.
+1 503-805-7540
email us here
Visit us on social media:
LinkedIn
Facebook
X
Legal Disclaimer:
EIN Presswire provides this news content "as is" without warranty of any kind. We do not accept any responsibility or liability for the accuracy, content, images, videos, licenses, completeness, legality, or reliability of the information contained in this article. If you have any complaints or copyright issues related to this article, kindly contact the author above.
