Title: Building Performant and Brain-Like Recurrent Models from Neurons and Astrocytes
Abstract:
The brain’s ability to perform challenging tasks is facilitated by its many inductive biases— hardwired biological features that predispose it to process information in certain ways over others. These features include anatomically distinct brain areas, as well as specialized cell types such as neurons and glia. Inductive biases grant the brain computational powers that currently surpass artificial intelligence systems in many domains. In this seminar, I will cover two recent avenues of research that leverage the brain’s inductive biases to build highly performant, recurrent artificial networks.
In the first half of my talk, I will discuss how and why the brain maintains a balance between flexibility and stability through “dynamic attractors”, which are reproducible patterns of neural activity in response to (potentially time-varying) stimuli. This work reveals an unexpected and useful theoretical link between dynamic attractors and modularity. Specifically, recurrent neural networks with dynamic attractors can be combined into large “networks of networks”, reminiscent of the brain’s macroscopic organization, in ways that provably preserve stability. These higher-order, stable networks can then be optimized for state-of-the-art performance on benchmark sequential processing tasks, demonstrating that dynamic stability is a useful inductive bias for building brain-like performant recurrent models.
In the second half of my talk, I will cover recent progress in understanding the computational role of different cell types. I will focus in particular on neuron-glial interactions. An intriguing fact is that most human brain cells are not neurons, but rather glia. There is mounting experimental evidence suggesting that astrocytes, a specialized type of glial cell, play a significant role in learning, memory, and behavior. However, their precise computational function is not well understood. I will discuss recent work that aims to bridge this gap by relating dynamical, energy-based neuron-astrocyte networks to powerful AI models such as Modern Hopfield Networks and transformers.