Modeling sensorimotor circuits with machine learning: hypotheses, inductive biases, latent noise and curricula
Hierarchical sensorimotor processing, modularity and experience are all essential for adaptive motor control. Recent efficient musculoskeletal simulators and machine learning algorithms provide new computational approaches to gain insights into those concepts for biological motor control.
Firstly, I will present a hypothesis-driven modeling framework to quantitatively assess the computations underlying proprioception. We trained thousands of models to transform muscle spindle inputs according to 16 different hypotheses from the literature. For all those hypotheses, we found that hierarchical models that better satisfy those hypotheses, also explain neural recordings in the brain stem and cortex better. We furthermore find that models trained to estimate the state of the body are best at explaining neural data.
Secondly, I will discuss methods to close the gap between reinforcement learning algorithms and biological motor control. I will highlight several ingredients for learning controllers for high-dimensional musculoskeletal systems. I will discuss how biologically-inspired, attention-based policy networks beat state of the art systems-identification algorithms, how implicit, latent-driven exploration beats state of the art reinforcement learning algorithms such as SAC and PPO and provide evidence that curriculum-based learning is still required to be competitive.
Taken together, these results highlight the importance of inductive biases, and experience for biological motor control.