AI Learns to Forget

We sentient beings do not spend our lives collecting, storing, and regurgitating every experience that has passed through our sensory organs. If we did, we’d know every leaf our eyes had ever set on, every whisper and every roar that had ever reached our ears, every fact or face we’d ever come across. We’d remember every detail equally and use up our finite memory banks.

Thankfully, not everything sticks, and that which is not repeated disappears from our memory, or is stored in more remote vaults. Though often overlooked, the ability to forget, and to forget wisely, is crucial to our intelligence and our survival. This is one area where flesh and blood still towers over machines. Artificial intelligence systems do not know how to efficiently forget what is unimportant.

For now, that is. Researchers at Purdue University have learned how to use a quantum material, samarium nickelate, to create a device that will know what to forget. It emulates the learning and forgetting habits of animals. The researchers have used what they’ve learned of this material to create algorithms that can be applied to efficient, practical “lifelong learning,” as they call it, for the world of artificial intelligence.

Currently, the only forgetting neural networks do is the disastrous kind.

“For most of the algorithms that are available, if they have already learned a data set and you try to show them a new data set, they incur catastrophic forgetting,” says Kaushik Roy, a Professor of Electrical and Computer Engineering and head of Purdue’s Nanoelectronics Research Laboratory. “From a neural computing point of view, we may not have enough memory to remember everything. In an environment where we do incremental learning, where we keep on learning new things, it is very, very important to be able to forget.”

The key to forgetting—and forgetting the right things—is samarium nickelate’s ability tobreathe. Professor Shriram Ramanathan’s team at Purdue showed that when it is bombarded with hydrogen protons its conductivity increases, and when it is subsequently exposed to air its conductivity goes down. But the lattice structure of the material—and its overall conductivity—changes every time it’s exposed to hydrogen protons.

“This breathing sort of phenomenon gives rise to this conductivity change,” says Priyadarshini Panda, a researcher at the university’s school of electrical and computer engineering, who worked on the project.“That conductivity change is what we are able to utilize in our computing model to do this learning to forget.” It’s because of this adaptive, life-like respiratory behavior, or habituation, that Panda and her fellow researchers have decided to call the devices organismoids.

After creating their first organismoid with the samarium nickelate, they emulated the material’s unique ability to create both algorithms and an electronic spin-based device that can be used with conventional hardware. The resulting device uses current, instead of hydrogen protons, to change conductivity. Though the algorithms could be used with any computer, greater forgetting efficiency can be achieved with the hardware device.

Though Roy and Panda were inspired by our own excellent capacity for memory loss, they hope their algorithm will help artificial intelligence systems forget things better than humans can, making them smarter than we could ever imagine.

“When we learned to fly, we studied how a bird flaps its wing. But now we have a 747 which goes half way around the world and carries a payload of 400 passengers plus baggage,” Roy says. “We want to do better than the brain, go faster than the brain—however, it may not do exactly what the brain does.”

Michael Abrams is an independent writer.

For Further Discussion