FY2011 Annual Report

Neurobiology Research Unit

Professor Jeff Wickens

Abstract

The long-range goal of the Neurobiology Research Unit is to understand the cellular mechanisms and neural circuitry underlying learning and adaptive behavior in the mammalian brain. Our collaborative, interdisciplinary program of research is focused on the striatum of the basal ganglia and the neuromodulators, dopamine and acetylcholine, which play a central role in the mechanisms of reinforcement learning. Our main achievements have been: to characterize synaptic plasticity in the striatum and its modulation by dopamine; to measure dopamine signaling during learning and its role in the therapeutic mechanisms of methylphenidate; and, to elucidate the dynamics of neural assemblies in the striatum and their significance for behavior. These findings are of broad, general significance for the neuroscience of learning and motivation, and of fundamental importance for clinical understanding of major neuropsychiatric disorders. We have also initiated studies on a nanoparticle based drug delivery system for translating our basic science insights into new treatment approaches. We use a powerful and unique combination of approaches extending from cellular to behavioral levels of biological organization, including 2-photon microscopy, electrophysiology, fast-scan cyclic voltammetry, behavior, optogenetics and computational modeling, taking advantage of the opportunities for cross-disciplinary research at OIST. Our research has the forward goal of developing better treatments for attention-deficit hyperactivity disorder and Parkinson’s disease, which are debilitating neurological disorders of great importance to children and adults.

1. Staff

• Dr Tomomi Shindou, Researcher
• Dr Mayumi Ochi-Shindou, Researcher
• Dr Adam Ponzi, Researcher
• Dr Luca Aquili, Researcher
• Dr Takashi Nakano, Researcher
• Dr Saori Miura, Technician
• Mr. Andy Liu, Senior Technician
• Mr. Kavinda Liyanagama, Technician
• Mr Mayank Aggarwal, Research Assistant
• Ms Yukako Suzuki, Research Administrator

2. Collaborations

• Theme: Cellular and Behavioural Mechanisms of Hyperactivity and Movement Disorders
  • Type of collaboration: Joint research
  • Researchers:
  • Associate Professor Brian Hyland, University of Otago
  • Dr Brent Alsop, University of Otago
  • Dr Alison Mercer, University of Otago
  • Professor Gail Tripp, Okinawa Institute of Science and Technology
• Theme: Research on neuroplasticity using wireless dopamine sensing and microPET
  • Type of collaboration: Research collaboration
  • Researchers: 
  • Professor Jia-Jin J. Chen, National Cheng Kung University
  • PhD Student Mr Yu-Ting Li, National Cheng Kung University

3. Activities and Findings

Research activity has focused on experimental studies of synaptic plasticity in the corticostriatal pathway, and theoretical studies of striatal network dynamics and reinforcement mechanisms important in learning. Our main achievements have been: to characterize synaptic plasticity in the striatum and its modulation by dopamine; to measure dopamine signaling during learning and its role in the therapeutic mechanisms of methylphenidate; and, to elucidate the dynamics of neural assemblies in the striatum and their significance for behavior. Activity has been conducted in the following main areas:

3.1 Temporal requirements for dopamine modulation and the eligibility trace

OIST Researchers: Dr Mayumi Shindou, Dr Tomomi Shindou

Our working hypothesis is that synaptic efficacy is controlled by a conjunction of presynaptic and postsynaptic activity, together with phasic release of dopamine. Since delay of reinforcement is inevitable in real learning situations, a crucial issue is to understand how phasic dopamine release in response to delayed reinforcement is able to act on the decision-making substrates that were active at an earlier time. One solution proposed is the concept of an eligibility trace, localized to synapses that were active prior to the actions that led to the reward. Subsequent release of dopamine may then act on eligible synapses, thereby assigning credit to neural activity that preceded the decision. Eligibility traces have been widely assumed in computational models of learning but with little direct evidence. Therefore, an important aim of our research is to undertake a direct test of the eligibility trace hypothesis. To induce a dopamine pulse with the physical characteristics of the dopamine signal that occurs in vivo, we used photolytic uncaging of a caged dopamine compound to approximate the time course and concentration of dopamine release that occurs due to an unexpected reward in vivo. Using this method we applied a dopamine pulse at different time points before and after stimulation with different STDP protocols. In contrast to the predictions of the synaptic trace hypothesis, dopamine was most effective when uncaged immediately (30 ms) before the conjunction of presynaptic and postsynaptic spikes, and had no effect when uncaged after a delay (Fig. 1). This result is consistent with our computational models of the cellular mechanisms underlying the timing requirements of plasticity and the effects of dopamine that we have developed in collaboration with the Doya unit. It is a fundamentally important result with strong implications for theories such as TD learning and underlying neural mechanisms, which we plan to investigate vigorously in our future work.

3.2 The nature and timing of the dopamine signal in the striatum

OIST Researchers:     Mr Yu-Ting Li (PhD student), Mr Mayank Aggarwal (Technician), Mr Kavinda Liyanagama (Technician)

One of our main aims is to understand the actions of the dopamine signal in the striatum. Several lines of evidence indicate that phasic firing of dopamine neurons plays a crucial role in reinforcement learning. Because of the low rate of dopamine cell firing activity, changes indicating learning cannot be detected in unit recordings without averaging many trials. To obtain trial-by-trial measures of dopamine responses we developed our expertise in fast-scan cyclic voltammetry (FSCV). We wrote in-house software for data acquisition and analysis by principal components regression, designed new head-mounted amplifiers, and developed new methods for fabricating carbon-fiber electrodes with higher sensitivity, lower noise and smaller size. With these methods we are able to measure dopamine concentration in freely moving rats with nanomolar sensitivity and sub second resolution. Fig. 2 shows dopamine release in response to electrical stimulation of dopamine neurons, and to a conditioned stimulus after pairing with electrical stimulation.

Using this preparation we investigated the mechanism of action of methylphenidate (Ritalin). Methylphenidate is widely used in treatment of attention-deficit hyperactivity disorder (ADHD). We found that dopamine release in response to a classically conditioned cue that precedes reward is deficient in an ADHD rat model. We also found that methylphenidate specifically rescued the deficit in anticipatory dopamine release, at doses commonly used therapeutically (data not shown). This specific action of methylphenidate led us to propose a novel basis for its therapeutic action: by facilitating anticipatory release of dopamine, methylphenidate helps to bridge delays between actions and rewarding outcomes, enhancing the ability to stay on task when reward is delayed, and improving the performance of children with ADHD in classroom situations.

3.3 Optogenetic inhibition of nucleus accumbens neurons

OIST Researchers: Dr Luca Aquili, Dr Mayumi Shindou, Dr Tomomi Shindou, Mr Andrew Liu (Technician)

The ability to learn from errors is critical for survival. Reversal learning provides an exacting test of this ability, by requiring an animal to flexibly alter behavior in response to the unpredictable switching of stimulus-reward contingencies. Several studies have shown that the nucleus accumbens is important in modulating various forms of behavioral flexibility, including reversal learning. However, until the recent introduction of optogenetic methods, we have been unable to experimentally perturb neural systems during specific timeframes in relation to task planning and behavioral execution. One of our main technical goals has been to develop expertise in using light activated halorhodopsin (eNpHR) to inhibit neural activity in specific timeframes. Using this method we were able to tease out the contribution of nucleus accumbens neurons during the planning, performance (choosing a lever) and waiting for feedback (reward delivery or omission because of an incorrect choice). We found that optogenetic inhibition of accumbal activity in the period preceding bar presses for food rewards had no effect on behavior in a reversal learning task. However, inhibition occurring in the period between bar presses and rewards or non-rewards had the somewhat surprising effect of reducing errors during reversal. Our results demonstrate a critical time window during which accumbens neurons modulate learning.

3.4 Activity dynamics of the striatal network

OIST Researcher: Dr Adam Ponzi

The principal neurons of the striatum are projection neurons with local inhibitory collaterals. We have suggested that they form a lateral inhibition type of neural network with sparse excitatory input from the cerebral cortex. Using computational models we simulated realistic networks and investigated the effect of lateral inhibition on network dynamics. We found that sparse lateral inhibition, which is the reality in the striatum, played an important role in dynamics, and was optimal for spontaneous generation of assemblies that fire in sequence in response to unstructured input. This is a in important new insight into the functional role of sparse lateral inhibition in the striatum, which may be relevant to neural activity sequences encoding behavior. Further work has investigated the effects of more structured inputs.

3.5 Implications for attention deficit hyperactivity disorder (ADHD)

Collaborators: Professor Gail Tripp (Human Developmental Neurobiology Unit, OIST) and members of D'Or Institute for Research and Education (IDOR), Rio de Janeiro, Brazil

Our theoretical and experimental studies have also contributed to understanding the neurobiology of ADHD. In collaboration with Gail Tripp of the Human Developmental Neurobiology Unit (OIST), we developed an influential theory to explain altered processing of reward in children with ADHD, in terms of a dopamine-transfer deficit (DTD). The DTD theory proposes that deficient transfer of dopamine signaling to cues that predict reward underlies key symptoms of ADHD. To test this theory we first developed an animal version of tasks used in children with ADHD and then applied these to the validation of animal models for the disorder. Based on the results of our animal studies we collaborated with Professor Tripp (OIST) and colleagues in Brazil (IDOR) in the design of a human study using functional magnetic resonance imaging (fMRI) to measure brain activation in response to classically conditioned cues predicting reward. We found increased activation in the caudate, nucleus accumbens and ventral putamen during reward anticipation in controls but not in ADHD. In contrast, increased activation was observed during reward delivery in ADHD but not in controls (Furukawa et al., submitted). These findings support our hypothesis that impaired transfer of phasic dopamine release from reward to cues predicting reward may underlie altered reinforcement sensitivity in ADHD.

4. Publications

4.1 Journals

1. Tripp, G., and Wickens, J.R. (2012) Reinforcement, dopamine and animal models in drug development for ADHD. [Invited review] Neurotherapeutics, in press.
2. Aggarwal, M., Hyland, B.I. and Wickens, J.R. (2012) Neural control of dopamine release: implications for reinforcement learning. European Journal of Neuroscience 35: 1115-1123.
3. Ponzi, A., and Wickens, J.R. (2012) Input dependent cell assembly dynamics in a model of the striatal medium spiny neuron network. Frontiers in Systems Neuroscience 6: 1-14.
4. Aggarwal, M. and Wickens, J.R. (2011) A role for phasic dopamine neuron firing in habit learning. [Invited Preview] Neuron, 72, 892-894 [Impact factor 14.027]
5. Shindou, T., Ochi-Shindou, M. and Wickens, J.R. (2011) A Ca²⁺ threshold for induction of spike-timing dependent depression in the mouse striatum. Journal of Neuroscience 31: 13015-22. [Impact factor 7.271] (Featured in This Week in the Journal)
6. Schulz, J.M., Pitcher, T.L., Savanthrapadian, S. Wickens, J.R., Oswald, M.J. and Reynolds, J.R. (2011) In vivo membrane potential and spike dynamics in projection neurons and interneurons of the striatum show distinct frequency-dependence. Journal of Physiology 589: 4365-81. [Impact factor 5.139]
7. Dejean, C., Arbuthnott, G., Wickens, J. R., Le Moine, C., Boraud, T. and Hyland, B. I. (2011) Power fluctuations in beta and gamma frequencies in rat globus pallidus: association with specific phases of slow oscillations and differential modulation by dopamine D1 and D2 receptors. Journal of Neuroscience 31: 6098-6107. [Impact factor 7.271] (Awarded “best paper” prize for 2011, University of Otago, School of Medical Sciences).
8. Wickens, J.R., Hyland, B.I. and Tripp, G. (2011) Animal models to guide clinical drug development in ADHD: Lost in translation? British Journal of Pharmacology 164: 1107–1128. [Impact factor 5.204]

4.2 Books and other one-time publications

1. Ponzi, A., and Wickens, J.R. (2012) Input dependent variability in a model of the striatal medium spiny neuron network. Proceedings of The 3^rd International Conference on Cognitive Neurodynamics (ICCN), Advances in Cognitive Neurodynamics (III), Yamaguchi, Y. (Ed) Springer, 600p. [ISBN 978-94-007-4791-3]

4.3 Oral Presentations

1. Wickens, J.R. A cellular mechanism for reinforcement learning: dopamine-dependent plasticity in the corticostriatal pathway. Plenary lecture for the 2012 Korean Society for Brain and Neural Science annual meeting (ISN-Wiley-Blackwell-JNC-International Lecturer). Seoul National University, Seoul, September 25, 2012.
2. Tripp, G., Wickens, J.R., Furukawa, E. Dopamine, Reinforcement and ADHD. D’Or Institute for Research and Education. Rio de Janeiro, Brazil, August 10, 2012.
3. Wickens, J.R. Basal ganglia circuitry. RIKEN BSI Summer Program, July 5, 2012.
4. Wickens, J.R. Synaptic Plasticity and Learning. RIKEN BSI Summer Program, July 6, 2012.
5. Wickens, J.R. Methylphenidate-mediated rescue of deficient dopamine reward prediction in an ADHD model. SHIONOGI & OIST Collaborative Seminar, Shionogi R&D Center, Toyonaka-city, Osaka, April 26, 2012.
6. Wickens, J.R., Plasticity and neural dynamics in the corticostriatal network, The Japan-France Joint Symposium on Neural Dynamics and Plasticity: from Synapse to Network. Kyoto, Japan. January 13, 2012
7. Wickens, J.R. Fun and challenges in combining theoretical and experimental neurosciences. Japan Neural Network meeting satellite symposium. Okinawa Institute of Science and Technology, Japan. December 15, 2011
8. Ponzi, A. Input dependent cell assembly dynamics in a spiking model of the striatal MSN network, OIST Junior Researcher Retreat, Okinawa, Japan. October 28, 2011
9. Ponzi, A. Input dependent cell assembly dynamics in a model of the striatal MSN network, Young Computational Neuroscientist Workshop. Daejeon, South Korea. December 5, 2011
10. Shindou, T., Ochi-Shindou, M., Wickens, J.R., A Ca²⁺ threshold for induction of spike-timing dependent depression in the mouse striatum, 12th RIES-Hokudai International Symposium, Sapporo, Japan. November 22, 2011
11. Tripp, G., Wickens, J.R. Dopamine, reinforcement and ADHD, The 22nd European Network for Hyperactivity Disorders meeting: The next 10 years, Budapest, Hungary. October 1, 2011
12. Wickens, J.R., Ponzi, A. Cell assemblies in the neostriatal network, The 3rd International Conference on Cognitive Neurodynamics, Dynamic Brain Forum, Niseko, Hokkaido, Japan. June 12, 2011
13. Ponzi, A. Input dependent variability in a model of the striatal medium spiny neuron network.  The 3rd International Conference on Cognitive Neurodynamics (ICCN 2011), Dynamic Brain Forum, Niseko, Hokkaido, Japan. (June 12, 2011)
14. Wickens, J. R. Cell assemblies in the neostriatal network, The Japan-Germany Workshop, Okinawa, Japan, March 5, 2011

4.4 Poster Presentations

1. Furukawa, E., Bado, P., Tripp, G., Mattos, P., Wickens, J., Bramati, I E., Alsop, B., Lima, D., Tovar-Moll, F., Sergeant, J. A., Moll, J. Reward sensitivity in ADHD: Exploring distinctive effects of reward cues and outcomes using functional MRI. International Neuroethics Society, 2011 Annual Meeting, Washingon, USA, November 2011.
2. Ponzi, A., Wickens J.R., Interaction of signal and noise in a computational model of the striatal medium spiny neuron network. The 41st Annual Meeting of the Society for Neuroscience, San Diego, USA. November, 2011
3. Aquili, L., Wickens, J.R., NAcc neurons inhibition during reversal learning: a pharmacological and an optogenetic approach, The 41st Annual Meeting of the Society for Neuroscience. Washington DC, USA. November 2011
4. Parr-Brownlie, L.C. Wickens, J.R., Hutchison, M., Hyland, B.I. Levodopa selectively increases high gamma power in the globus pallidus during functional but not dyskinetic movements in the hemiparkinsonian rat. The 41st Annual Meeting of the Society for Neuroscience. Washington DC, USA. November, 2011
5. Parr-Brownlie, L.C. Wickens, J.R., Hutchison, M., Hyland, B.I. Globus pallidus neuronal activity does not underlie L-DOPA induced dyskinesias in parkinsonian rats. Australasian Winter Conference on Brain Research, Queenstown, New Zealand, August 29, 2011.
6. Ochi-Shindou, M., Shindou, T., Wickens, J.R., State-dependent expression of spike-timing dependent plasticity in dopamine D2 receptor expressing spiny neurons in the neostriatum of adult mice, 34th Annual Meeting of the Japan Neuroscience Society. Yokohama City, Japan. September 15, 2011
7. Ponzi, A, Wickens J.R., Signal and noise in a biologically realistic spiking neural network model of the striatum, 34th Annual Meeting of the Japan Neuroscience Society, Yokohama City, Japan. September 17, 2011
8. Aquili, L., Wickens J.R., VTA and NAcc neurons inhibition during reversal learning: a pharmacological and an optogenetic approach, 34th Annual Meeting of the Japan Neuroscience Society. Yokohama City, Japan. September 17, 2011
9. Nakano, T., Otsuka, M., Spiking Neural Network Model of Free-Energy-based Reinforcement Learning, 20^th Computational Neuroscience Meeting (OCNS). Stockholm, Sweden. July 25, 2011
10. Ponzi, A. Input dependent cell assembly dynamics in an inhibitory spiking network model, Japan-Germany Joint Workshop on Computational Neuroscience, Okinawa, Japan, Mar 4, 2011
11. Aquili, L. & Wickens, J. R. VTA and NAcc neurons' inhibition during reversal learning: a pharmacological and an optogenetic approach, Australian Neuroscience Society, Auckland, New Zealand, Jan 31-Feb 3 2011

5. Intellectual Property Rights and Other Specific Achievements

Nothing to report

6. Meetings and Events

6.1 Seminar

• Date:March 5, 2012
• Venue: OIST Campus Lab1
• Speaker: Professor Min Han Jun, Neuroscience Laboratory, Institute for Medical Sciences, Ajou University, Korea

7. Other

Nothing to report.