publication . Article . Other literature type . Preprint . 2020

Introducing neuromodulation in deep neural networks to learn adaptive behaviours.

Vecoven, Nicolas; Ernst, Damien; Wehenkel, Antoine; Drion, Guillaume;
Open Access
  • Published: 27 Jan 2020 Journal: PLOS ONE, volume 15, page e0227922 (eissn: 1932-6203, Copyright policy)
  • Publisher: Public Library of Science (PLoS)
  • Country: Belgium
Abstract
Animals excel at adapting their intentions, attention, and actions to the environment, making them remarkably efficient at interacting with a rich, unpredictable and ever-changing external world, a property that intelligent machines currently lack. Such an adaptation property relies heavily on cellular neuromodulation, the biological mechanism that dynamically controls intrinsic properties of neurons and their response to external stimuli in a context-dependent manner. In this paper, we take inspiration from cellular neuromodulation to construct a new deep neural network architecture that is specifically designed to learn adaptive behaviours. The network adaptat...
Subjects
free text keywords: reinforcement learning, neural nets, neuromodulation, deep learning, : Computer science [Engineering, computing & technology], : Sciences informatiques [Ingénierie, informatique & technologie], General Biochemistry, Genetics and Molecular Biology, General Agricultural and Biological Sciences, General Medicine, Medicine, R, Science, Q, Computer Science - Machine Learning, Computer Science - Neural and Evolutionary Computing, Statistics - Machine Learning, Research Article, Computer and Information Sciences, Neural Networks, Biology and Life Sciences, Neuroscience, Cell Biology, Cellular Types, Animal Cells, Neurons, Cellular Neuroscience, Biochemistry, Neurochemistry, Evolutionary Biology, Evolutionary Processes, Convergent Evolution, Physical Sciences, Mathematics, Applied Mathematics, Algorithms, Machine Learning Algorithms, Research and Analysis Methods, Simulation and Modeling, Artificial Intelligence, Machine Learning, Neuronal Tuning, Evolutionary Adaptation, Cognitive Science, Cognitive Psychology, Learning, Psychology, Social Sciences, Learning and Memory
Related Organizations
Download fromView all 7 versions
PLoS ONE
Article . 2020
Provider: Crossref
PLoS ONE
Article
Provider: UnpayWall
PLoS ONE
Article . 2020

The algorithm works as follows. First, we set 00 to k, X0 to ;, m to 0, vCritic0 to 0 and zCritic0 to 0. Then, we repeat the following procedure CE number of times.

[1] Hk : The set of trajectories of length L.

[1] L( ) : A loss function which is dependendent on a function approximate v( ) ; .

Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy P. Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. Asynchronous methods for deep reinforcement learning. CoRR, abs/1602.01783, 2016. URL http://arxiv.org/ abs/1602.01783.

Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, and Timothy Lillicrap. Meta-learning with memory-augmented neural networks. In International conference on machine learning, pages 1842{1850, 2016.

John Schulman, Sergey Levine, Pieter Abbeel, Michael Jordan, and Philipp Moritz. Trust region policy optimization. In International Conference on Machine Learning, pages 1889{1897, 2015a.

John Schulman, Philipp Moritz, Sergey Levine, Michael I. Jordan, and Pieter Abbeel. High-dimensional continuous control using generalized advantage estimation. CoRR, abs/1506.02438, 2015b. URL http://arxiv.org/abs/1506.02438.

John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. CoRR, abs/1707.06347, 2017. URL http://arxiv.org/ abs/1707.06347.

David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al. Mastering the game of go with deep neural networks and tree search. nature, 529(7587):484, 2016.

Jane X. Wang, Zeb Kurth-Nelson, Dhruva Tirumala, Hubert Soyer, Joel Z. Leibo, Remi Munos, Charles Blundell, Dharshan Kumaran, and Matthew Botvinick. Learning to reinforcement learn. CoRR, abs/1611.05763, 2016. URL http://arxiv.org/abs/1611. 05763.

Abstract
Animals excel at adapting their intentions, attention, and actions to the environment, making them remarkably efficient at interacting with a rich, unpredictable and ever-changing external world, a property that intelligent machines currently lack. Such an adaptation property relies heavily on cellular neuromodulation, the biological mechanism that dynamically controls intrinsic properties of neurons and their response to external stimuli in a context-dependent manner. In this paper, we take inspiration from cellular neuromodulation to construct a new deep neural network architecture that is specifically designed to learn adaptive behaviours. The network adaptat...
Subjects
free text keywords: reinforcement learning, neural nets, neuromodulation, deep learning, : Computer science [Engineering, computing & technology], : Sciences informatiques [Ingénierie, informatique & technologie], General Biochemistry, Genetics and Molecular Biology, General Agricultural and Biological Sciences, General Medicine, Medicine, R, Science, Q, Computer Science - Machine Learning, Computer Science - Neural and Evolutionary Computing, Statistics - Machine Learning, Research Article, Computer and Information Sciences, Neural Networks, Biology and Life Sciences, Neuroscience, Cell Biology, Cellular Types, Animal Cells, Neurons, Cellular Neuroscience, Biochemistry, Neurochemistry, Evolutionary Biology, Evolutionary Processes, Convergent Evolution, Physical Sciences, Mathematics, Applied Mathematics, Algorithms, Machine Learning Algorithms, Research and Analysis Methods, Simulation and Modeling, Artificial Intelligence, Machine Learning, Neuronal Tuning, Evolutionary Adaptation, Cognitive Science, Cognitive Psychology, Learning, Psychology, Social Sciences, Learning and Memory
Related Organizations
Download fromView all 7 versions
PLoS ONE
Article . 2020
Provider: Crossref
PLoS ONE
Article
Provider: UnpayWall
PLoS ONE
Article . 2020

The algorithm works as follows. First, we set 00 to k, X0 to ;, m to 0, vCritic0 to 0 and zCritic0 to 0. Then, we repeat the following procedure CE number of times.

[1] Hk : The set of trajectories of length L.

[1] L( ) : A loss function which is dependendent on a function approximate v( ) ; .

Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy P. Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. Asynchronous methods for deep reinforcement learning. CoRR, abs/1602.01783, 2016. URL http://arxiv.org/ abs/1602.01783.

Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, and Timothy Lillicrap. Meta-learning with memory-augmented neural networks. In International conference on machine learning, pages 1842{1850, 2016.

John Schulman, Sergey Levine, Pieter Abbeel, Michael Jordan, and Philipp Moritz. Trust region policy optimization. In International Conference on Machine Learning, pages 1889{1897, 2015a.

John Schulman, Philipp Moritz, Sergey Levine, Michael I. Jordan, and Pieter Abbeel. High-dimensional continuous control using generalized advantage estimation. CoRR, abs/1506.02438, 2015b. URL http://arxiv.org/abs/1506.02438.

John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. CoRR, abs/1707.06347, 2017. URL http://arxiv.org/ abs/1707.06347.

David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al. Mastering the game of go with deep neural networks and tree search. nature, 529(7587):484, 2016.

Jane X. Wang, Zeb Kurth-Nelson, Dhruva Tirumala, Hubert Soyer, Joel Z. Leibo, Remi Munos, Charles Blundell, Dharshan Kumaran, and Matthew Botvinick. Learning to reinforcement learn. CoRR, abs/1611.05763, 2016. URL http://arxiv.org/abs/1611. 05763.

Powered by OpenAIRE Research Graph
Any information missing or wrong?Report an Issue