In the latest episode of OIST’s podcast featuring researchers from the OIST Global Bioconvergence Center of Innovation, a center dedicated to promoting mental, physical, and environmental health based on the “One World, One Health” approach, OIST Podcaster in Residence, DJ Nick Luscombe spoke with Prof. Kenji Doya, leader of the Neural Computation Unit. Prof. Doya and his team work to understand how a healthy mind works by combining artificial intelligence and biological models.
“You may think there is a large gap between a robot and the human mind, but we can learn a lot by trying to let a robot learn,” says Prof. Doya. For his undergraduate project, he developed a robot that learned to walk through random exploration and reward feedback, now known as the principle of reinforcement learning. More recently at OIST, by mimicking biological life, his group created robots that strive to survive by finding battery packs to recharge and to reproduce by exchanging “artificial genes” via infrared. The aim was to see whether robots can find their own reward functions for reinforcement learning and indeed the robots did.
The motivation to engage in a behavior depends heavily on the time elapsing between action and reward. If the reward is not offered in close timely proximity to the behavior, the subject might give up – a concept also referred to as temporal discounting. The importance of this became clear when a robot’s setting was focused solely on the near future. The robot, who needed to reach a battery pack, would not move even when a pack was in sight. Only once the researchers moved the battery very close, would the robot attempt to get there.
“The robot looked like being in a depressed state, which showed us that proper temporal discounting is important for healthy behavior,” says Prof. Doya. This surprising finding was the starting point for new research on what is necessary to keep motivated to reach a future goal, a hallmark of a healthy mind. Common medications used to treat depression in humans act on the serotonin system. Based on the researcher’s work with robots, the lab developed the hypothesis that the neurotransmitter serotonin could be in control of temporal discounting in the brain.
“Serotonin probably helps you to decide whether you should focus on the immediate outcome or consider the far future. If a situation is urgent, you should focus on the now, but if you have time and resources, you can shift that focus to the future” explains Prof. Doya. A series of experiments using rats and mice supported their hypothesis that serotonin plays a key role in balancing this focus. “We found that when the animal is waiting for a delayed reward, the release of serotonin is high and serotonin neurons keep firing until the reward is delivered or the animal quits waiting,” says Prof. Doya.
Recently, with the use of optogenetic mice, where the researchers can control serotonin neurons directly through shining blue light on them, their hypothesis was confirmed. With activation of the serotonin neurons in the brain, the mice waited more patiently for a reward. "Usually, mice quit waiting after about 10 seconds but with the stimulation, they waited 15 seconds or more for a food reward. Quite a strong effect!” emphasizes Prof. Doya.
