주요메뉴 바로가기 본문 바로가기

주메뉴

IBS Conferences

AI's memory-forming mechanism found to be strikingly similar to that of the brain

An interdisciplinary team consisting of researchers from the Center for Cognition and Sociality and the Data Science Group within the Institute for Basic Science (IBS) revealed a striking similarity between the memory processing of artificial intelligence (AI) models and the hippocampus of the human brain. This new finding provides a novel perspective on memory consolidation, which is a process that transforms short-term memories into long-term ones, in AI systems.

In the race towards developing Artificial General Intelligence (AGI), with influential entities like OpenAI and Google DeepMind leading the way, understanding and replicating human-like intelligence has become an important research interest. Central to these technological advancements is the Transformer model [Figure 1], whose fundamental principles are now being explored in new depth.

The key to powerful AI systems is grasping how they learn and remember information. The team applied principles of human brain learning, specifically concentrating on memory consolidation through the NMDA receptor in the hippocampus, to AI models.

The NMDA receptor is like a smart door in your brain that facilitates learning and memory formation. When a brain chemical called glutamate is present, the nerve cell undergoes excitation. On the other hand, a magnesium ion acts as a small gatekeeper blocking the door. Only when this ionic gatekeeper steps aside, substances are allowed to flow into the cell. This is the process that allows the brain to create and keep memories, and the gatekeeper's (the magnesium ion) role in the whole process is quite specific.

The team made a fascinating discovery: the Transformer model seems to use a gatekeeping process similar to the brain's NMDA receptor [see Figure 1]. This revelation led the researchers to investigate if the Transformer's memory consolidation can be controlled by a mechanism similar to the NMDA receptor's gating process.

In the animal brain, a low magnesium level is known to weaken memory function. The researchers found that long-term memory in Transformer can be improved by mimicking the NMDA receptor. Just like in the brain, where changing magnesium levels affect memory strength, tweaking the Transformer's parameters to reflect the gating action of the NMDA receptor led to enhanced memory in the AI model. This breakthrough finding suggests that how AI models learn can be explained with established knowledge in neuroscience.

C. Justin LEE, who is a neuroscientist director at the institute, said, “This research makes a crucial step in advancing AI and neuroscience. It allows us to delve deeper into the brain's operating principles and develop more advanced AI systems based on these insights.”

CHA Meeyoung, who is a data scientist in the team and at KAIST, notes, “The human brain is remarkable in how it operates with minimal energy, unlike the large AI models that need immense resources. Our work opens up new possibilities for low-cost, high-performance AI systems that learn and remember information like humans.”

What sets this study apart is its initiative to incorporate brain-inspired nonlinearity into an AI construct, signifying a significant advancement in simulating human-like memory consolidation. The convergence of human cognitive mechanisms and AI design not only holds promise for creating low-cost, high-performance AI systems but also provides valuable insights into the workings of the brain through AI models.



Fig. 1: 
        (a) Diagram illustrating the ion channel activity in post-synaptic neurons. AMPA receptors are involved in the activation of post-synaptic neurons, while NMDA receptors are blocked by magnesium ions (Mg²⁺) but induce synaptic plasticity through the influx of calcium ions (Ca²⁺) when the post-synaptic neuron is sufficiently activated. (b) Flow diagram representing the computational process within the Transformer AI model. Information is processed sequentially through stages such as feed-forward layers, layer normalization, and self-attention layers. The graph depicting the current-voltage relationship of the NMDA receptors is very similar to the nonlinearity of the feed-forward layer. The input-output graph, based on the concentration of magnesium (α), shows the changes in the nonlinearity of the NMDA receptors.
Fig. 1:
(a) Diagram illustrating the ion channel activity in post-synaptic neurons. AMPA receptors are involved in the activation of post-synaptic neurons, while NMDA receptors are blocked by magnesium ions (Mg²⁺) but induce synaptic plasticity through the influx of calcium ions (Ca²⁺) when the post-synaptic neuron is sufficiently activated. (b) Flow diagram representing the computational process within the Transformer AI model. Information is processed sequentially through stages such as feed-forward layers, layer normalization, and self-attention layers. The graph depicting the current-voltage relationship of the NMDA receptors is very similar to the nonlinearity of the feed-forward layer. The input-output graph, based on the concentration of magnesium (α), shows the changes in the nonlinearity of the NMDA receptors.



Notes for editors

- References
Transformer as a hippocampal memory consolidation model based on NMDAR-inspired nonlinearity, Dong-Kyum Kim, Jea Kwon, Meeyoung Cha, C. Justin Lee (2023): 37th Conference on Neural Information Processing Systems, DOI: TBA


- Media Contact
For further information or to request media assistance, please contact William I. Suh at the IBS Public Relations Team (willisuh@ibs.re.kr).


- About the Institute for Basic Science (IBS)
IBS was founded in 2011 by the government of the Republic of Korea with the sole purpose of promoting the development of basic science in South Korea. IBS has 33 research centers in a variety of disciplines.


- About Center for Cognition and Sociality & Data Science Group
The IBS Center for Cognition and Sociality is at the forefront of understanding brain function, from exploring how our brains support thinking and memory to how we process social interactions and empathy. It also pioneers cutting-edge technologies to map and manipulate brain activity at the molecular level, potentially leading to new treatments for a range of brain disorders. See more on https://www.ibs.re.kr/glia/.

The Data Science Group at the IBS Center for Mathematical and Computational Sciences is dedicated to leveraging AI and data science to solve societal issues. The team focuses on computational social science, developing algorithms to detect fake news, analyze human behavior, and predict socio-economic trends through advanced data analysis. See more on https://ds.ibs.re.kr/.


Research

Are you satisfied with the information on this page?

Content Manager
Public Relations Team : Yim Ji Yeob   042-878-8173
Last Update 2023-11-28 14:20