• Forgetting reduces the influence of outdated information
• Forgetting enables us to make a bet in spite of specific past events
• Forgetting is inherent for creative thinking and imagination
• And, forgetting is a key strategy for adaptation as well

Sometimes, we forget naturally, and other times we make a strategic decision to forget (and yes, this is a request to those of you who have not pardoned my past mistakes yet). They both count.  However, most AI techniques today focus on learning and not on forgetting. 

It is a remarkable oversight that the field of AI has actually never taken forgetting seriously.

It is a remarkable oversight that the field of AI has actually never taken forgetting seriously. Most AI techniques do not address:

• Reducing the influence of outdated information (that was used for building the model)
• Strategically adapting prediction based on the new information
• A creative solution that is not learnt from data

Before we discuss how we can enable AI forget, let’s take a look at forgetting happening inside our brain.

How do People Forget?

Ever since the discovery that the brain has a vast network of billions of nerve cells that communicate with each other via junctions called ‘synapses’, the role that synapses play in learning and memory has been a matter of extensive research.   We continually build, lose, alter, and rebuild these synaptic connections.  A memory is nothing but a group of synaptic connections and the physical representation of the memory like this is referred to as engrams. Building or altering the engrams is the foundation of learning and memory. 

Forgetting happens when these engrams break down or become harder to access:

• New synaptic connections overriding old connections
• Decay of the synaptic connections between neurons
• Random death of some neurons

Most past theories about forgetting have emphasized on forgetting happening as a passive process, however, new research in the past decade, suggests that the loss of memories is not a passive process. Rather, the brain is built to forget.  Forgetting is an active mechanism that is constantly at work in the brain.

Forgetting is an active mechanism that is constantly at work in the brain.

• The brain actively works to forget information of lower value (i.e not useful, valuable, conflicting or relevant) 
• There is a group of neurons in the brain that are typically active when we are sleeping in R.E.M. stage are responsible for active forgetting in the brain.

There is a lot that we still don’t know about how the forgetting process works and perhaps thats the reason that AI is significantly less intelligent than humans.

How to make AI Forget?

Well the “traditional” BIG data AI learning is all about learning.  There is no concept of forgetting.  In most cases the technique to forget is by learning again with the whole new data set.

For example, let’s assume you have built a model to learn handwritten digits from 0 to 9.  You can do it with any handwritten digit data set.  Let’s assume this dataset had all 7s written without a line through it.  And, now you want to update this model to relearn how 7s are written.  You want to make it forget that 7 is written without a line through it and only remember 7 with a line through it.  You have a new data set of 7s with a line through them available but you can’t access the old dataset.

We need the ability to incrementally forget or acquire new knowledge.  This makes it interesting.  You want to change the weights of your model but do it in such a way that your model can predict new knowledge, forget selective old knowledge, and be stable for existing knowledge. 

To have the ability to learn in continuous environments, we need to see beyond the current AI techniques.  How you will do weight regularization, define acceptable forgetting levels, current model protection, or even rehearsal to verify the impact of new learning in absence of old data to learn.  Current AI techniques result in models that are “stable” (i.e. perform well based on the data they are built upon) but not “plastic” (i.e. able to learn/forget incremental without disrupting what is learnt previously).  We need to find a technique that provides the right balance between stability and plasticity.

Future of AI running on less data or sequential data is not far. If you are struggling with lack of incremental learning in your AI solutions, or would like to brainstorm on how SECOMIND does this, please feel free to reach out to me.

The brain is the control centre of the body and mind, comprising billions of neurons that allow us to function and thrive the way that we do. The stronger the memory recall, the stronger the neural connection, and vice versa.

So, why do we forget?

Earlier research suggested a passive process involving the cleansing of old memories that were unused for a period of time. This passive process was known as decay. However, in recent years, a greater understanding of the brain has been achieved.

The discovery? The brain happens to be coded to forget, intentionally and purposefully. This active process is what comprises the latest body of evidence regarding the brain, its memory, and its forgetfulness.

Why Does the Brain Forget?

In short, research suggests that actively forgetting memories is what allows us to retain and remember new ones. In fact, the more we understand the true functionality of the brain, the more we understand that forgetfulness is, though contradicting, what allows our memory to function properly.

In other words, without the ability to forget, our brain would have no memory at all.

The fact of the matter is that while the brain and its billions of neurons have grand capacities, its ability to remember and recall is limited. This is, in part, due to several factors.

Below is a summary of why the brain forgets and what’s to blame for this lapse in functionality:

Decay

The Decay Theory suggests that if a particular memory is not regularly recalled and rehearsed, it decays and disappears from our memory over time.

Interference

Obviously named, this phenomenon suggests that when two memories are similar in nature, they tend to interfere with one another. This ultimately results in a confusion of memory and an inability to remember an old or new memory.

Lack of Storage

The failure to store a memory in long-term care. If a memory lacks specificity, it likely fails to get stored for long-term recall.

Intentional Forgetfulness

Sometimes we as humans are simply motivated to forget. This may be due to a negative experience, traumatic event, or unpleasant memory of sorts. As such, suppression of memories occurs, ultimately resulting in forgetting it over time.

Lifestyle Factors (i.e. substance consumption, mental illness, stress, poor sleep, etc.)

Several lifestyle factors are associated with forgetfulness as well. While the majority of them are controllable, if not controlled, a lapse in memory will be the result.

How Does the Brain Forget?

So, what’s to blame for how and why we forget? In short, the hippocampus…

The hippocampus relies on several storage mechanisms when it comes to memories and recall. Depending on how a particular memory is stored and categorized will largely determine whether or not the brain forgets it in time.

Another reason for memory loss and forgetfulness is a neurological condition known as neurogenesis. While neurogenesis is largely a positive outcome for the brain, it’s been shown in research to lack old memory recall.

In other words, while the birth of new neurons allows the brain to store new memories, among other functions, it seems to eliminate the ability to continue to retain and recall old memories.

Although it’s still unsure why this occurs, it’s likely related to the decay and interference mechanisms of memory storage.

No matter the cause of memory or lack thereof, forgetfulness is caused by the degeneration and regeneration of neural cells, the remodelling of neural pathways and thought patterns, and a shift in the hippocampus circuitry.

Final Thoughts

While the brain’s ability to forget has previously been thought of as a negative consequence of several factors, new research is suggesting the contrary, proving its purpose is good.

Forgetting is a function of how memories are stored. In other words, memories that we regularly recall are likely to stick around, while memories that we ignore are likely to decay over time.

There is a lot we still don’t understand when it comes to the brain and its functionality, memory and forgetfulness included. While more research certainly needs to be done, there’s no doubt that modern science has shown, at the very least, the value of forgetting when it comes to the function of memory at large.