Despite all of the comparisons to the human brain, AI still doesn’t look much like us. Maybe that’s okay. In the animal kingdom, brains come in all shapes and sizes. So in a new approach to machine learning, engineers have done away with the human brain and all of its wonderful complexities – instead turning to the brain of a simple worm for inspiration.
It turns out that simplicity has its advantages. The resulting neural network is efficient, transparent and here’s the kicker: it’s a lifelong learner.
Where most machine learning algorithms fail to improve their skills after an initial training period, the researchers say the new approach, called a fluid neural network, has a kind of built-in “ neuroplasticity. ” That is, while doing its job – for example, perhaps driving a car or driving a robot in the future – it can learn from experience and adjust its connections instantly.
In a world that is noisy and chaotic, such adaptability is essential.
Driver with worm brains
The algorithm’s architecture is inspired by just 302 neurons that make up the nervous system C. elegans, a small nematode (or worm).
In work published last year, the group, which includes researchers from MIT and the Austrian Institute of Science and Technology, said that despite its simplicity, C. elegans is capable of surprisingly interesting and varied behavior. So they developed equations to mathematically model the worm’s neurons and then built them into a neural network.
Their worm-brain algorithm was much simpler than other advanced machine learning algorithms, and yet it was able to perform similar tasks, such as keeping a car in its lane.
“Today, deep learning models with many millions of parameters are often used for learning complex tasks such as autonomous driving,” said Mathias Lechner, a PhD student at the Austrian Institute of Science and Technology and author of the study. “Our new approach, however, allows us to reduce the size of the networks by two orders of magnitude. Our systems use only 75,000 trainable parameters. “
Now, in a new article, the group continues their worm-inspired system by adding an all-new capability.
Old worm, new tricks
The output of a neural network – for example, turn the steering wheel to the right – depends on a number of weighted connections between the ‘neurons’ of the network.
It’s the same in our brains. Every brain cell is connected to many other cells. Whether a particular cell fires depends on the sum of the signals it receives. Above a certain threshold – or weight – the cell fires a signal to its own network of downstream links.
In a neural network these weights are called parameters. As the system feeds data over the network, its parameters converge in the configuration to provide the best results.
Typically, the parameters of a neural network are locked in place after the workout and the algorithm is put to work. But in the real world, this could mean it’s a little brittle – show an algorithm something that’s too different from its training, and it will break. Not an ideal result.
In contrast, in a fluid neural network, the parameters are allowed to continue to change over time and with experience. The AI learns on the job.
This adaptability means that the algorithm is less likely to break down as the world throws new or noisy information in its path, such as when rain obscures an autonomous car’s camera. Unlike larger algorithms, whose inner workings are largely unfathomable, the algorithm’s simple architecture allows researchers to look inside and control decision-making.
Neither his newfound ability nor his still-diminished stature seemed to stop the AI. The algorithm performed just as well or better than other advanced time series algorithms in predicting next steps in a sequence of events.
“Everyone is talking about expanding their network,” said Ramin Hasani, the lead author of the study. “We want to downsize, to have fewer but richer nodes.”
An adaptable algorithm that consumes relatively little computing power would make an ideal robotic brain. Hasani believes the approach could be useful in other applications that require real-time analysis of new data, such as video processing or financial analysis.
He plans to continue dialing in the approach to make it practical.
“We have a demonstrably more expressive neural network that is inspired by nature. But this is just the beginning of the process, ”said Hasani. “The obvious question is: how do you extend this? We think these types of networks could be a key element of future intelligence systems. “
Is bigger better?
At a time when big players like OpenAI and Google are regularly making headlines with gigantic machine learning algorithms, it’s a fascinating example of an alternative approach going in the opposite direction.
OpenAI’s GPT-3 algorithm collectively dropped jaws last year, both for its size – a record 175 billion parameters at the time – and its capabilities. A recent Google algorithm topped the charts with over a trillion parameters.
Critics, however, are concerned that the pursuit of an ever-larger AI is wasteful and expensive, and the research is consolidating in the hands of a few companies with money to fund large-scale models. Furthermore, these enormous models are “black boxes,” whose actions are largely impenetrable. This can be especially problematic when models are trained unsupervised on the unfiltered Internet. There is no telling (or perhaps controlling) what bad habits they will pick up.
Scientific researchers are increasingly trying to tackle some of these problems. While companies like OpenAI, Google, and Microsoft try to prove the bigger-is-better hypothesis, it’s possible that serious AI efficiency innovations are emerging elsewhere – not despite a lack of resources, but because of it. As they say, necessity is the mother of invention.
Image credit: benjamin henon / Unsplash